Geodesic source code for Silverlight 4.0

Geodesics

The shortest path between two points on an arbitrary surface is called a Geodesic, and on a sphere, it is a Great Circle. Modelling the surface of the earth as a perfect sphere, the shortest distance between any two locations on the surface is then described by a section of a Great Circle, ie an arc that lies on the plane that is described by the vectors between its start and end points and the Earth’s centre ( see figure 1 ).

With this information, one way ( and the way I have adopted ) to plot such a curve is as follows:

1. Generate the points of the curve in two dimensions using the parametric equation of a circle.
2. Transform the plane of the 2d curve into 3D space such that it intersects the end points on the sphere, and the sphere’s centre.
3. Project the transformed points back into 2D space using the Mercator projection equations.

A Parametric Representation of a Great Circle

The Mercator projection gives the 2D rectilinear coordinates (x,y) as a function of the latitude and longitude of a point on a sphere. However, it is easier to draw the point using a typical drawing API, if we have a representation that gives each point of the curve in terms of a single parameter. To derive such a function, we observe that the parametric equation for a circle is given by

Since a great circle is a rigid transformation of a circle in 3D space, it can also be represented as a function of a single parameter

where is a 2×3 matrix that transforms the plane circle to a location on a sphere. Using spherical coordinates, the Mercator projection of the curve specified above is then

Now that we have a suitable parametric equation, we can draw the geodesic with a series of connected line segments by varying the parameter, t.

Mathematical note – the parameterization given by this expression is highly non-uniform, meaning that there are many more points generated in some parts of the curve than in others. The mathematics of generating uniform ( or natural ) parameterizations belongs to the field of differential curve geometry and is beyond the scope of this article ( and my brain ).

Implementation notes

Inevitably, the mathematics alone is not sufficient to produce an implementation of a reuseable class for the Bing Silverlight control. There are two main issues to resolve; firstly, it is not immediately obvious how to derive from the provided MapShapeBase class to create new shape overlays and secondly, how to handle drawing the curves when they ‘wrap’ beyond the map’s viewable area ( this is easier to illustrate than to describe – see the figure opposite ).

Inheriting from MapShapeBase

I must confess that I cheated slightly in implementing the MapGeodesicPath class, in that I used Reflector to peer into the implementation of the base class. The existing derivations of this class simply defer to MapShapeBase for most of the work, which they can do since for a MapPolygon and MapPolyline there is a one-to-one relationship between Locations, latitude and longitude points on the map and Points, the actual 2D cartesian coordinates used to draw the shape. For the Geodesic, this is not the case, because we only want to specify the start and ending points of the curve, not every point in between. A solution is to delegate the point generation code to a secondary class, one that can be independently tested.

Splitting the curves at the map boundary

When the curves wrap around the map projection, they need to be split at the boundary. This is done by finding the parameter t for the longitude value of the boundary. Where the longitude value is +/- 180, this is straightforward as the equation above reduces to,

For the first time, the main Prime Ministerial candidates for the 2010 UK General Elections, will take part in three live debates. Since the BBC have kindly made the full transcripts available, I decided to have a go at analyzing the data and creating a visual representation in the form of word clouds. I am currently working on my own visualization software, but in the meantime these have been done using Wordle.

Preparing the data

Download the raw text of the first debate.

Having done that, a command line tool such as awk can be used to split the data by speaker. For example, the following command outputs Clegg’s comments into a separate file:

awk 'BEGIN {RS=""; FS="[@]"} $1=="NC" { print $2 }' debate.txt > clegg.txt

Parsing the data

Python’s Natural Language Toolkit provides all the functions needed to analyze the text data, such as tokenizing the text by word and even categorizing each word by type, such as proper nouns and prepositions. For example, having extracted Nick Clegg’s speech as above and read the file as a string using Python, the following commands parse the input for sentences, and then tokenize each word procucing a complete word list.

from __future__ import division
import nltk, re, pprint

sentences = nltk.sent_tokenize(text)
tokens=[]
for s = sentences:
 tokens.extend(nltk.word_tokenize(s))
words=[t.lower() for t in tokens]

We can then categorize each word with a POS tag and extract a list appropriately, for example, using the word tokens above the following extracts all the nouns

# this operation takes some time to execute
taggedwords=nltk.pos_tag(words)
nouns=[word for (word,tag) in words if t == 'NN']

Such a list is enough to use with Wordle, however it’s straightforward to create a word frequency list for use with other software.

# this operation takes some time to execute
nounfrequencies = nltk.FreqDist(nouns)

Going further

Word frequency analyses are fairly straightforward, however NLTK is a powerful library and allows for much more detailed and informative analysis based on grammar and sentence structure. It would be interesting to see the results of a more sophisticated approach.

Gallery of word clouds from the first debate

As 2010 is the 100th anniversary of the first published infra red photograph, I thought I’d try my hand using my own digital camera and some easily acquired accessories. If you want, you can skip the theory and go straight to the description of the method and a script for Photoshop.

A quick overview of IR photography

The CCD that is responsible for recording the images photographed by most digital cameras, is already sensitive to the near infra red part of the spectrum. That is, the part of the spectrum outside of the range visible to the human eye, but not so far as that used for, for example, thermal imaging. Since most photographers are not interested in light that they can’t see, this light is usually filtered out by an infra red cutoff filter placed inside the camera body, directly in front of the CCD. However, such filters are imperfect, so with some camera models by combining a long exposure with an infra red transmitting filter placed in front of the lens, some of that IR light can be recovered. The figure shows the basic principle, though I should add that the graphs are just a sketch to illustrate the principle and don’t represent an actual CCD response curve.

Pseudocolour

The LAB Colour Model

The most common method of representing a colour image is by a combination of Red, Green and Blue ‘colour primaries’. The reason for this is simply because it then becomes relatively trivial to display an image using a video display, which also constructs an image using a combination of red, green and blue LEDs or phosphors. This is why such a model is known as a ‘device dependent’ model. However, RGB is not the only colour model, and if you are at all familiar with Photoshop you may be aware of the LAB model.

LAB, or CIELAB to give it its full name, is a device independent model, which like RGB is composed of three components ( or channels ) but in this case only the ‘A’ and ‘B’ components contain colour information, whereas the ‘L’ component contains pure luminosity information. Thus, by separating an image into LAB components it becomes possible to manipulate the luminosity and colour components of an image independently.

Method and Implementation in Photoshop

Equipment

1. Canon G10 Digital Compact Camera
2. LA-DC58K lens adapter for same
3. Hoya R72 58mm Infra Red filter
4. Tripod
5. Photoshop ( or GIMP )

The images in this post have been created from two source images, one a long exposure taken using the IR filter and the other a conventional colour image. Both images are converted into the LAB colour space, and a single image is then produced, using the Luminosity channel from the IR image and the ‘A’ and ‘B’ channels from the colour image. The photoshop script below will automate the process when applied to two source images, you can then manipulate the final image using the adjustment tools. You can also do much the same thing in GIMP using the Decompose/Compose features from the Colour menu, but it does not include Photoshop’s image registration or LAB colour adjustment features.

Download ir_composite.jsx.zip

To use the script, open the IR and corresponding colour image, and ensure that the IR image is selected. Then apply the script.

Practicalities

There are a number of limitations to multiple exposure photography methods such as this, especially ones which require long exposures and changes of camera equipment. It is best suited to landscape photography in good conditions where the multiple exposures can be easily aligned. If either image contains rapidly moving subjects these can ( and will ) show up as artefacts in the resulting image.

Gallery

Microsoft have recently released a beta of Silverlight 4, which has limited support for native interoperation using COM. Potentially, this example could be applied to any number of native interop scenarios, however for this example I have chosen to use Nvidia’s CUDA technology.

Disclaimer : This is an example of what can be done, not necessarily, and in all likelihood, an example of how it should be done.

About CUDA

Up until around 2001 PC graphics cards, though powerful, implemented a fixed function pipeline that limited use to whatever was exposed by the APIs, usually Direct3D or OpenGL. The addition of a programmable pixel pipeline led to the use of graphics cards for more general computation tasks; at first using shaders directly, followed by higher level GPU specific programming languages, such as Brook, SH, and later NVidia’s CUDA. Most of this work was, and is, documented by the GPGPU group. NVIDIA’s website shows CUDA being used in a wide variety of applications but in practice it is best employed in so called “embarassingly parallel” problems.

The demonstration application

The demo below shows a Silverlight 4 beta application, which implements a recursive gaussian filter. Note that this is not the same algorithm provided by the sample in the CUDA SDK, but a more efficient method, which is described in detail in [1] for those interested. The main advantage of a filter implemented in this way is that the computation time is independent of the width of the filter.
To enable CUDA interop, you’ll need a CUDA compatible graphics card. Then do the following,

1. Install the MFC COM application (link below). The installer should register the application with COM automatically.
2. Right click on the Silverlight App and install it for running outside of the browser. The CUDA option should now be available from the Combo box.

Source code : SilverlightCudaInteropDemo.zip
Install MFC COM Application (5.5 Mb)

The native component

The native component takes the form of a COM Automation server, implemented as a client side MFC application.

Note: Make sure you run Visual Studio with Administrator privileges, otherwise registering the automation server with COM will fail.

MFC and Automation are beyond the scope of this article, but the basic process I followed was thus

1. Create an MFC Dialog application using the Wizard. Make sure to enable Automation support
2. Add a method to the autmation interface using the add Method wizard from the Class View
3. Add a dual interface using this Technical Note from MSDN.
4. If you get link errors, make sure to include the output of MIDL in the application class ( the one that contains OnInitInstance). I couldn’t find any reference to this step, but it’s how the samples work.
5. Make sure that the run time library options passed to nvcc and msvc match, ie they should all use a DLL or Static linking, not a mixture of both
6. If you get stuck, take a look at the MFC Samples, particularly acdual.

when you pass a native array through COM Automation, it is converted to a SAFEARRAY on the native side. Note that I couldn’t find any documentation on this, I discovered it through experience. The code snippets below show sending and receiving array data between Silverlight and the MFC application.

Using MEF to implement the application

The Managed Extensibility Framework is an extensible plugin framework for .NET applications and Silverlight. I have used it to dynamically discover implementations of IProcessorProvider based on the permissions available to the Silverlight application. The figure below shows the component structure of the application.

Performance notes

Silverlight

Unlike the Reaction Diffusion simulation, for this application I have chosen to use Silverlight’s WriteableBitmap, introduced in Silverlight 3, rather than dynamic PNG encoding. This revealed an interesting performance issue when using a typical double loop to iterate over the pixels. Initial timings revealed that the vast majority of the time was spent in updating the WriteableBitmap rather than actually performing the image processing. The initial update loop used the PixelWidth and PixelHeight properties to bound the loop counters, taking about 200ms to iterate over the loop.

By caching the bitmap properties in local variables, the timing was reduced to ~5ms. Needless to say I was shocked by how much of a difference such a seemingly trivial change made.

OLE Automation

The guidelines for building performant automation code is much the same as that for other unmanaged interop scenarios in .NET : avoid chatty interfaces. Note that this is exactly what I have not done here. In fact, the time it takes CUDA to perform the image processing is dwarfed by the time it takes to marshal the data between Silverlight and COM. This can be mitigated somewhat by splitting the blur call into two operations, one to load the image, which is called only upon initialization, and one to perform the blur.

CUDA

CUDA operations are extremely sensitive to data alignment and the order in which threads access data. Kernels should be written in such a way that threads access adjacent data elements, meaning that the row major access pattern familiar to C and C# developers would produce suboptimal performance ( sometimes by as much as an order of magnitude ). Instead, array accesses should be performed in a manner more reminiscent of FORTRAN. In addition, 2D arrays should be padded out so that threads access data elements that are correctly aligned ( see the CUDA documentation for the correct alignment values ). A full exposition of performance optimization for CUDA is really beyond the scope of this article, there are many examples in the NVIDIA documentation although the terminology can be somewhat opaque. One of the clearest explanations I have found is this presentation from Mark Harris at Supercomputing 2007.

References

1. Young, I.T. & van Vliet,L.J, 1995. Recursive implementation of the Gaussian filter. Signal Processing, 44, pp.139-151.

Silverlight toolkit themes do not support merged dictionaries

Unfortunately, merged dictionaries are not supported by the Silverlight Toolkit’s Themes feature – at least not as it stands. For example, the following XAML and accompanying class causes a XamlParseException if you try to use it.

All is not lost however, since full source code is provided for the Silverlight Toolkit, so it becomes a straightforward matter to track down the problem and supply a patch.

Writing a test to reproduce the problem

Since the toolkit comes with an accompanying test suite, we proceed in textbook TDD fashion by first writing a test that exercises the issue. We can reuse some of the existing code to do this, adding a resource dictionary containing a merged dictionary reference and adding the following test to the ImplicitStyleManagerTest class

The test fails as we would expect.

Fix the code to make the test pass

The Theme class works by transforming the ResourceDictionary xaml into a ContentControl. A quick glance at the xaml generated by this process in the Visual Studio debugger shows why an exception is being raised.

The ResourceDictionary.MergedDictionaries element is missing its parent ResourceDictionary element, so we add a couple of lines of code to ResourceParser.ParseElement to detect the element and wrap it in a ResourceDictionary, using LINQ to XML.

Having made the change, we rerun the test.

As we would hope, our custom theme now works as expected and we also have a unit test that ensures the bug does not reoccur. Note the Xaml code above, in that the Resource Dictionary is referenced using Pack Uri syntax. This is because Button.xaml is compiled as a Resource, rather than Content, since the latter doesn’t appear to work reliably and can result in obfuscated error messages such as XamlParseException: attribute “Button.xaml” value out of range.

I have submitted the patch for this fix to the toolkit site on Codeplex.

Rendering

The first obstacle to implementing the RD simulation is that Silverlight 2.0 does not by default provide a means of generating dynamic images. WPF has a WriteableBitmap, but no equivalent exists in Silverlight. However, it does support PNG streams so we can dynamically update a bitmap by encoding it to PNG on the fly. For this I have used Joe Stegman’s PNGEncoder class, which I have modified slightly to deal with RGB data and to reduce memory usage.

Performance notes

Performance of the simulator is dominated by the time it takes to update the arrays containing the concentration values of the chemicals. Actual rendering time is negligible, even though the image has to be dynamically encoded to a PNG file in order for Silverlight to display it. There are two ways of implementing 2D arrays in Silverlight with C#, rectangular arrays or C style jagged arrays. If this were a desktop application, the most performant method would be to use rectangular arrays and unsafe access, however that option is not available to us in Silverlight so it turns out that, for safe access, jagged arrays are faster.

For example, originally the simulation step looked like this

On my machine this took on the order of 600ms to perform 100 iterations. Using jagged arrays instead, we have the following

Just replacing the array type and making no other changes this now takes about 500ms, an improvement of about 13%. However, now that we are using jagged arrays we can do better and move the row accesses out of the inner loop.

This brings the count down to about 330ms, an improvement of about 40% over the original implementation.

We’ve taken the sequential code about as far as we can go. The next step is to try to take advantage of muliple cores that may be available on the user’s machine.

One of Alan Turing’s many contributions to mathematics and science during the 20th century was his 1952 paper on “The Chemical Basis of Morphogenesis” in which he suggested that a simple model of coupled differential equations could account for pattern formation in natural processes such as those found on animal coats. Such models are known as Reaction-Diffusion models, and take the following general form

The equation describes how the concentration of each chemical components in q evolves as a function of the other components. I have chosen to illustrate the Gray-Scott model; the physical derivation of the reactant term is described in detail in [2], the addition of the diffusion term and the resultant behaviour in [3]. The model has two chemical components, U and V and is described as follows.

The model is what is knows as an activator-inhibitor model, in which one chemical acts to inhibit the growth of the other.

Simulation

The simulation is implemented with Silverlight 2.0. For full details of the implementation see the separate post. For guidance in parameter selection, see [3], or just select a preset and modify it. Note that the Png image generation feature currently requires the Firefox browser.

Gallery

References

1. Turing,A.,1952. The chemical basis of morphogenesis [pdf]. Philosophical Transactions of the Royal Society, 237 pp.37-72
2. Gray,P. and Scott,S.K. 1985. Sustained Oscillations and Other Exotic Patterns of Behavior in Isothermal Reactions. Journal of Physical Chemistry, 89 pp.22-32
3. Pearson,J.,1993. Complex patterns in a simple system. Science, 261(5118) pp.189-192

It also happens to be the model used by the Open University’s course on Mathematical Logic and Number Theory, and is the reason for this article since I studied the course in 2007.

Emulator

The implementation details of the emulator are quite interesting, using a mixture of Microsoft technologies including F# and Silverlight, however I will save that for another article for those interested in such things. To use the Emulator just hit Reset, enter the program and initial registers and click Step to trace through the program.