Geodesic source code for Silverlight 4.0

Geodesics

The shortest path between two points on an arbitrary surface is called a Geodesic, and on a sphere, it is a Great Circle. Modelling the surface of the earth as a perfect sphere, the shortest distance between any two locations on the surface is then described by a section of a Great Circle, ie an arc that lies on the plane that is described by the vectors between its start and end points and the Earth’s centre ( see figure 1 ).

With this information, one way ( and the way I have adopted ) to plot such a curve is as follows:

1. Generate the points of the curve in two dimensions using the parametric equation of a circle.
2. Transform the plane of the 2d curve into 3D space such that it intersects the end points on the sphere, and the sphere’s centre.
3. Project the transformed points back into 2D space using the Mercator projection equations.

[silverlight: Geodesic.xap,520,384,false]

A Parametric Representation of a Great Circle

The Mercator projection gives the 2D rectilinear coordinates (x,y) as a function of the latitude and longitude of a point on a sphere. However, it is easier to draw the point using a typical drawing API, if we have a representation that gives each point of the curve in terms of a single parameter. To derive such a function, we observe that the parametric equation for a circle is given by

Since a great circle is a rigid transformation of a circle in 3D space, it can also be represented as a function of a single parameter

where is a 2×3 matrix that transforms the plane circle to a location on a sphere. Using spherical coordinates, the Mercator projection of the curve specified above is then

Where and are the longitude and latitude of the point to be projected, respectively. Writing the equation out in full gives,

Now that we have a suitable parametric equation, we can draw the geodesic with a series of connected line segments by varying the parameter, t.

Mathematical note – the parameterization given by this expression is highly non-uniform, meaning that there are many more points generated in some parts of the curve than in others. The mathematics of generating uniform ( or natural ) parameterizations belongs to the field of differential curve geometry and is beyond the scope of this article ( and my brain ).

Implementation notes

Inevitably, the mathematics alone is not sufficient to produce an implementation of a reuseable class for the Bing Silverlight control. There are two main issues to resolve; firstly, it is not immediately obvious how to derive from the provided MapShapeBase class to create new shape overlays and secondly, how to handle drawing the curves when they ‘wrap’ beyond the map’s viewable area ( this is easier to illustrate than to describe – see the figure opposite ).

Inheriting from MapShapeBase

I must confess that I cheated slightly in implementing the MapGeodesicPath class, in that I used Reflector to peer into the implementation of the base class. The existing derivations of this class simply defer to MapShapeBase for most of the work, which they can do since for a MapPolygon and MapPolyline there is a one-to-one relationship between Locations, latitude and longitude points on the map and Points, the actual 2D cartesian coordinates used to draw the shape. For the Geodesic, this is not the case, because we only want to specify the start and ending points of the curve, not every point in between. A solution is to delegate the point generation code to a secondary class, one that can be independently tested.

Splitting the curves at the map boundary

When the curves wrap around the map projection, they need to be split at the boundary. This is done by finding the parameter t for the longitude value of the boundary. Where the longitude value is +/- 180, this is straightforward as the equation above reduces to,

For other longitude values, we simply offset the longitude values by the required amount, and calculate the value of t for the new matrix.

Last week myself and a group of friends took a winter camping trip up to the Lake District. An odd decision, it could be argued, seen as the region has recently seen both some of its worst flooding and coldest winters in recent years.  Nonethless, equipped with ice axe, crampons and a generous supply of Kendal Mint Cake we set out to climb Scafell.

Some of the things I learned

1. Elephants, though clearly signposted, are not common in the lake district. I didn’t see a single one all weekend. Sheep, however, are plentiful.
2. It may be effective as a deterrent against the Ravenous Bugblatter Beast of Traal, but a towel is no replacement for an insulating mattress.
3. Pop up tents, though useful on the beach, are less effective in torrential rain and gale force winds.
4. Retracing your steps back through the snow sounds easy, and it probably is for other people.
5. The Wasdale Head Inn serves an excellent selection of whiskeys, of which Lagavulin is the best. I should know, I tried all of them.
6. Swimming in Wastwater in January is almost exactly as cold as you’d expect.

The Route

The map below, which requires Microsoft Silverlight, was exported from my GPS wristwatch and shows the route we took up Scafell before visibility made further progress a bit risky, and so we retraced our steps back through the snow. Or at least tried to before our steps mysteriously disappeared. There is also a Photosynth from part of our route available.

[silverlight: osmap.xap,520,350,show]

Swimming in Wastwater

Although as an amateur triathlete I’ve done many open water swims, I’ve never done one in winter and never without a wetsuit. So, in an attempt to kill both birds with one stone I decided to go for a swim in Wastwater in January. I lasted about two minutes. I’m no Lewis Gordon Pugh. It was cold, but it really wasn’t as bad as you might think.  The water was beautifully clear but unfortunately sharp rocks do not a beach make.

Gallery

[Show as slideshow]

Microsoft have recently released a beta of Silverlight 4, which has limited support for native interoperation using COM. Potentially, this example could be applied to any number of native interop scenarios, however for this example I have chosen to use Nvidia’s CUDA technology.

Disclaimer : This is an example of what can be done, not necessarily, and in all likelihood, an example of how it should be done.

About CUDA

Up until around 2001 PC graphics cards, though powerful, implemented a fixed function pipeline that limited use to whatever was exposed by the APIs, usually Direct3D or OpenGL. The addition of a programmable pixel pipeline led to the use of graphics cards for more general computation tasks; at first using shaders directly, followed by higher level GPU specific programming languages, such as Brook, SH, and later NVidia’s CUDA. Most of this work was, and is, documented by the GPGPU group. NVIDIA’s website shows CUDA being used in a wide variety of applications but in practice it is best employed in so called “embarassingly parallel” problems.

The demonstration application

The demo below shows a Silverlight 4 beta application, which implements a recursive gaussian filter. Note that this is not the same algorithm provided by the sample in the CUDA SDK, but a more efficient method, which is described in detail in [1] for those interested. The main advantage of a filter implemented in this way is that the computation time is independent of the width of the filter.
To enable CUDA interop, you’ll need a CUDA compatible graphics card. Then do the following,

1. Install the MFC COM application (link below). The installer should register the application with COM automatically.
2. Right click on the Silverlight App and install it for running outside of the browser. The CUDA option should now be available from the Combo box.

Source code : SilverlightCudaInteropDemo.zip
Install MFC COM Application (5.5 Mb)
[silverlight: cuda_interop/SlCudaInteropDemo.xap,520,580,false]

The native component

The native component takes the form of a COM Automation server, implemented as a client side MFC application.

Note: Make sure you run Visual Studio with Administrator privileges, otherwise registering the automation server with COM will fail.

MFC and Automation are beyond the scope of this article, but the basic process I followed was thus

1. Create an MFC Dialog application using the Wizard. Make sure to enable Automation support
2. Add a method to the autmation interface using the add Method wizard from the Class View
3. Add a dual interface using this Technical Note from MSDN.
4. If you get link errors, make sure to include the output of MIDL in the application class ( the one that contains OnInitInstance). I couldn’t find any reference to this step, but it’s how the samples work.
5. Make sure that the run time library options passed to nvcc and msvc match, ie they should all use a DLL or Static linking, not a mixture of both
6. If you get stuck, take a look at the MFC Samples, particularly acdual.

when you pass a native array through COM Automation, it is converted to a SAFEARRAY on the native side. Note that I couldn’t find any documentation on this, I discovered it through experience. The code snippets below show sending and receiving array data between Silverlight and the MFC application.

// note that ComAutomationFactory has become AutomationFactory
// in Silverlight 4 RC
dynamic cuda = AutomationFactory.CreateObject("CudaServer.Application");
float[] data = new [] {1.0f, 3.14f };
dynamic retData = cuda.Process( data );
// retData is a managed float array

VARIANT CCudaServer::Process(VARIANT &data)
{
  SAFEARRAY *pSrcData =  data.parray;

  // this will copy the safe array into the variant
  CComVariant var(pSrcData);

  // when we return the VARIANT containing the SAFEARRAY
  // it will be marshaled to Silverlight as a managed array
  VARIANT retVal;
  VariantInit( &retVal );
  var.Detach( &retVal );
  retVal.vt = VT_ARRAY | VT_R4;
  return retVal;
}

Using MEF to implement the application

The Managed Extensibility Framework is an extensible plugin framework for .NET applications and Silverlight. I have used it to dynamically discover implementations of IProcessorProvider based on the permissions available to the Silverlight application. The figure below shows the component structure of the application.

Performance notes

Silverlight

Unlike the Reaction Diffusion simulation, for this application I have chosen to use Silverlight’s WriteableBitmap, introduced in Silverlight 3, rather than dynamic PNG encoding. This revealed an interesting performance issue when using a typical double loop to iterate over the pixels. Initial timings revealed that the vast majority of the time was spent in updating the WriteableBitmap rather than actually performing the image processing. The initial update loop used the PixelWidth and PixelHeight properties to bound the loop counters, taking about 200ms to iterate over the loop.

for (int j = 0; j < bmp.PixelHeight; ++j)
{
  for (int i = 0; i < bmp.PixelWidth; ++i)
  {
     // update pixels
   }
}

By caching the bitmap properties in local variables, the timing was reduced to ~5ms. Needless to say I was shocked by how much of a difference such a seemingly trivial change made.

int pxWidth =  bmp.PixelWidth;
int pxHeight = bmp.PixelHeight;
for (int j = 0; j < pxHeight; ++j)
{
  for (int i = 0; i < pxWidth; ++i)
  {
     // update pixels
   }
}

OLE Automation

The guidelines for building performant automation code is much the same as that for other unmanaged interop scenarios in .NET : avoid chatty interfaces. Note that this is exactly what I have not done here. In fact, the time it takes CUDA to perform the image processing is dwarfed by the time it takes to marshal the data between Silverlight and COM. This can be mitigated somewhat by splitting the blur call into two operations, one to load the image, which is called only upon initialization, and one to perform the blur.

CUDA

CUDA operations are extremely sensitive to data alignment and the order in which threads access data. Kernels should be written in such a way that threads access adjacent data elements, meaning that the row major access pattern familiar to C and C# developers would produce suboptimal performance ( sometimes by as much as an order of magnitude ). Instead, array accesses should be performed in a manner more reminiscent of FORTRAN. In addition, 2D arrays should be padded out so that threads access data elements that are correctly aligned ( see the CUDA documentation for the correct alignment values ). A full exposition of performance optimization for CUDA is really beyond the scope of this article, there are many examples in the NVIDIA documentation although the terminology can be somewhat opaque. One of the clearest explanations I have found is this presentation from Mark Harris at Supercomputing 2007.

__global__ void kernel( float *destData, float *srcData, int stride, int height )
{
  // suboptimal access. Each thread accesses elements
 // in a striding pattern
  int rowStart = (blockDim.x*blockIdx.x+threadIdx.x)*stride;
  for ( int i = rowStart; i < rowStart+stride; ++i ) {
    destData[i] = srcData[i];
  }
}

__global__ void kernel( float *destData, float *srcData, int stride, int height )
{
  // optimal access pattern, each thread accesses adjacent elements
  int colStart = blockDim.x*blockIdx.x+threadIdx.x;
  // this case, 16*sizeof(float)= 64 bytes
  for ( int i = colStart ; i < colStart+(stride*height); i+=stride ) {
    destData[i] = srcData[i];
  }
}

References

1. Young, I.T. & van Vliet,L.J, 1995. Recursive implementation of the Gaussian filter. Signal Processing, 44, pp.139-151.

Silverlight toolkit themes do not support merged dictionaries

Unfortunately, merged dictionaries are not supported by the Silverlight Toolkit’s Themes feature – at least not as it stands. For example, the following XAML and accompanying class causes a XamlParseException if you try to use it.

public class MergedTheme : Theme {
 public MergedTheme() :
  base(typeof(MergedTheme).Assembly, "MergedTheme.Theme.xaml") {
  DefaultStyleKey = typeof(MergedTheme);
 }
}

All is not lost however, since full source code is provided for the Silverlight Toolkit, so it becomes a straightforward matter to track down the problem and supply a patch.

Writing a test to reproduce the problem

Since the toolkit comes with an accompanying test suite, we proceed in textbook TDD fashion by first writing a test that exercises the issue. We can reuse some of the existing code to do this, adding a resource dictionary containing a merged dictionary reference and adding the following test to the ImplicitStyleManagerTest class

///

/// Test that styles contained in a  merged dictionary can be successfully applied
/// 

[TestMethod]
[Asynchronous]
[Description("Test that a dictionary contining merged dictionaries can be successfully loaded.")]
public void TestResourceDictionaryWithMergedDictionaries() {
 Uri uri = new Uri("System.Windows.Controls.Testing.Theming;
  component/ImplicitStyleManager/MergedResourceDictionary.xaml", UriKind.Relative);
 TestAuto( (stackPanel) => {
  SetResourceDictionaryUri(stackPanel, uri);
  ImplicitStyleManager.SetApplyMode(stackPanel, ImplicitStylesApplyMode.Auto);
  }, Colors.Blue);
}

The test fails as we would expect.

Fix the code to make the test pass

The Theme class works by transforming the ResourceDictionary xaml into a ContentControl. A quick glance at the xaml generated by this process in the Visual Studio debugger shows why an exception is being raised.

The ResourceDictionary.MergedDictionaries element is missing its parent ResourceDictionary element, so we add a couple of lines of code to ResourceParser.ParseElement to detect the element and wrap it in a ResourceDictionary, using LINQ to XML.

private static void ParseElement(XmlReader reader, XmlWriter writer, bool checkTypes) {
 if (reader.Depth == 0) {...}
 else {
  // handle merged dictionaries by wrapping them in a ResourceDictionary element
  if (reader.Name == "ResourceDictionary.MergedDictionaries") {
   XElement parent = new XElement("ResourceDictionary");
   parent.Add(XElement.ReadFrom(reader));
   writer.WriteRaw(parent.ToString());
  }
  else { ... }
 }
}

Having made the change, we rerun the test.

As we would hope, our custom theme now works as expected and we also have a unit test that ensures the bug does not reoccur. Note the Xaml code above, in that the Resource Dictionary is referenced using Pack Uri syntax. This is because Button.xaml is compiled as a Resource, rather than Content, since the latter doesn’t appear to work reliably and can result in obfuscated error messages such as XamlParseException: attribute “Button.xaml” value out of range.

I have submitted the patch for this fix to the toolkit site on Codeplex.

Rendering

The first obstacle to implementing the RD simulation is that Silverlight 2.0 does not by default provide a means of generating dynamic images. WPF has a WriteableBitmap, but no equivalent exists in Silverlight. However, it does support PNG streams so we can dynamically update a bitmap by encoding it to PNG on the fly. For this I have used Joe Stegman’s PNGEncoder class, which I have modified slightly to deal with RGB data and to reduce memory usage.

Performance notes

Performance of the simulator is dominated by the time it takes to update the arrays containing the concentration values of the chemicals. Actual rendering time is negligible, even though the image has to be dynamically encoded to a PNG file in order for Silverlight to display it. There are two ways of implementing 2D arrays in Silverlight with C#, rectangular arrays or C style jagged arrays. If this were a desktop application, the most performant method would be to use rectangular arrays and unsafe access, however that option is not available to us in Silverlight so it turns out that, for safe access, jagged arrays are faster.

For example, originally the simulation step looked like this

for (int j = 0; j < H; j++)
{
 int jm = j != 0 ? j - 1 : H - 1;
 int jp = j != H - 1 ? j + 1 : 0;
 for (int i = 0; i < W; i++)
 {
  int im = i != 0 ? i - 1 : W - 1;
  int ip = i != W - 1 ? i + 1 : 0;
  double d2u = U[j,im]+U[j,ip]+U[jm,i]+U[jp,i]-4*U[j,i];
  double d2v = V[j,im]+V[j,ip]+V[jm,i]+V[jp,i]-4*V[j,i];
  double uv2 = U[j,i]*V[j,i]*V[j,i];
  Uu[j,i] = du*d2u-uv2+F+U[j,i]*one_minus_F;
  Vv[j,i] = dv*d2v+uv2+V[j,i]*one_minus_F_minus_k;
 }
}

On my machine this took on the order of 600ms to perform 100 iterations. Using jagged arrays instead, we have the following

for (int j = 0; j < H; j++)
{
 int jm = j != 0 ? j - 1 : H - 1;
 int jp = j != H - 1 ? j + 1 : 0;
 for (int i = 0; i < W; i++)
 {
  int im = i != 0 ? i - 1 : W - 1;
  int ip = i != W - 1 ? i + 1 : 0;
  double d2u = U[j][im]+U[j][ip]+U[jm][i]+U[jp][i]-4*U[j][i];
  double d2v = V[j][im]+V[j][ip]+V[jm][i]+V[jp][i]-4*V[j][i];
  double uv2 = U[j][i]*V[j][i]*V[j][i];
  Uu[j][i] = du*d2u-uv2+ F+U[j][i]*one_minus_F;
  Vv[j][i] = dv*d2v+uv2+V[j][i]*one_minus_F_minus_k;
 }
}

Just replacing the array type and making no other changes this now takes about 500ms, an improvement of about 13%. However, now that we are using jagged arrays we can do better and move the row accesses out of the inner loop.

for (int j = 0; j < H; j++)
{
 int jm = j != 0 ? j - 1 : H - 1;
 int jp = j != H - 1 ? j + 1 : 0;
 var u_m = U[jm];
 var u = U[j];
 var u_p = U[jp];
 var v_m = V[jm];
 var v = V[j];
 var v_p = V[jp];
 var uu = Uu[j];
 var vv = Vv[j];
 for (int i = 0; i < W; i++)
 {
  int im = i != 0 ? i - 1 : W - 1;
  int ip = i != W - 1 ? i + 1 : 0;
  double d2u = u[im]+u[ip]+u_m[i]+u_p[i]-4*u[i];
  double d2v = v[im]+v[ip]+v_m[i]+v_p[i]-4*v[i];
  double uv2 = u[i]*v[i] *v[i];
  uu[i] = du*d2u-uv2+F+u[i]*one_minus_F;
  vv[i] = dv*d2v+uv2+v[i]*one_minus_F_minus_k;
 }
}

This brings the count down to about 330ms, an improvement of about 40% over the original implementation.

We’ve taken the sequential code about as far as we can go. The next step is to try to take advantage of muliple cores that may be available on the user’s machine.

One of Alan Turing‘s many contributions to mathematics and science during the 20th century was his 1952 paper on “The Chemical Basis of Morphogenesis” in which he suggested that a simple model of coupled differential equations could account for pattern formation in natural processes such as those found on animal coats. Such models are known as Reaction-Diffusion models, and take the following general form

The equation describes how the concentration of each chemical components in q evolves as a function of the other components. I have chosen to illustrate the Gray-Scott model; the physical derivation of the reactant term is described in detail in [2], the addition of the diffusion term and the resultant behaviour in [3]. The model has two chemical components, U and V and is described as follows.

The model is what is knows as an activator-inhibitor model, in which one chemical acts to inhibit the growth of the other.

Simulation

The simulation is implemented with Silverlight 2.0. For full details of the implementation see the separate post. For guidance in parameter selection, see [3], or just select a preset and modify it. Note that the Png image generation feature currently requires the Firefox browser.

Gallery

[Show as slideshow]

References

1. Turing,A.,1952. The chemical basis of morphogenesis [pdf]. Philosophical Transactions of the Royal Society, 237 pp.37-72
2. Gray,P. and Scott,S.K. 1985. Sustained Oscillations and Other Exotic Patterns of Behavior in Isothermal Reactions. Journal of Physical Chemistry, 89 pp.22-32
3. Pearson,J.,1993. Complex patterns in a simple system. Science, 261(5118) pp.189-192

It also happens to be the model used by the Open University’s course on Mathematical Logic and Number Theory, and is the reason for this article since I studied the course in 2007.

Emulator

The implementation details of the emulator are quite interesting, using a mixture of Microsoft technologies including F# and Silverlight, however I will save that for another article for those interested in such things. To use the Emulator just hit Reset, enter the program and initial registers and click Step to trace through the program.
[silverlight: Urm.App.xap,400,300,false]