MathsJam is a monthly meeting for people interested in recreational mathematics, and drinking of beer. At the Edinburgh meeting in August, we were presented with this Listener Crossword, from The Times Crossword Club. At the time of writing, there are plenty of solutions on the web from those who tackled the puzzle in the traditional manner of brain, pen and paper. I opted to cheat.

The description of the puzzle itself is quite complex, but in a nutshell the task is to substitute each of the given letters for one of the first ten prime numbers ( with no duplication ). The expression for each clue will then give a numerical solution. Substitute each digit in the solution with the first letter of its name and write the result in the crossword. There are two ways of substituting digits for letters, either in English or in German, but a solution to a given clue must be consistently one language or the other.

An Algorithmic Solution

I have decided to use Mathematica due to its built in symbolic evaluation capabilities, but in principle any sufficiently powerful programming language would do. If you wish you can download a copy of the mathematica notebook.

The goal of the algorithm will be to find two sets of rules that map the first 10 primes to the given letters, which give valid solutions. One straightforward way to achieve this is to enumerate every single possibility until we find two such sets. So how many possibilities are there?

There are possible sets of rules to examine. However, for each of the 27 clues, the solution generated by each rule set can be one of two options: English or German. That means that the total number of candidate solutions is

This is what’s called the ‘solution space’, and at a rate of 1000 candidates per second, it would take over 15,000 years to examine them all. Clearly this kind of brute force enumeration is impractical.

However, most of these solutions will be invalid in some way. In fact, if we trust the crossword compilers, all but two of them will be invalid. Some rules will produce solutions of the wrong size, others may conflict with other solutions in the crossword, and some may be invalid from a mathematical point of view ( such as a division that does not produce a whole number ). If instead of trying to solve the puzzle in one go, we construct ‘partial’ solutions in some systematic fashion, we could avoid exploring parts of the solution space that cannot possibly lead to a correct solution. This is the principle behind the backtracking search, one of the cornerstones of artificial intelligence.

Implementation of a Backtracking Search

The notion of a partial solution comes quite easily when considering a crossword puzzle, as it is simply a puzzle that is incomplete – ie, the solutions to at least one clue has been filled in. As an example, consider clue 19 Down : .

There are ten ways of mapping the single letter in 19 down to each prime

In[1]  := rules = {{a -> 2}, {a -> 3}, {a -> 5}, {a -> 7}, {a -> 11}, {a -> 13}, {a ->
   17}, {a -> 19}, {a -> 23}, {a -> 29}}

In[2]  := a^2/.rules
Out[2] = {4, 9, 25, 49, 121, 169, 289, 361, 529, 841}

Since we know 19 Down must be three letters long, immediately we can discard the first four candidates. The remainder are all possibilities, since at this point we have no more information. Lets take the first matching possibility, given by the rule .

In English, this gives

In[3]  := IntegerDigits[121] /. {1 -> "O", 2 -> "T"}
Out[3] = {"O", "T", "O"}

If we now look at 21 Across – – There are 2 symbols to map to the remaining 9 primes ( “A” has already been mapped to 11 by 19 Down ). The number of ordered k-subsets of n is given by,

So we have new sets of rules to examine

In[4] := rules = {a -> 11, b -> #[[1]], i -> #[[2]]} & /@
 Permutations[{2, 3, 5, 7, 13, 17, 19, 23, 29}, {2}]
Out[4] = {{a -> 11, b -> 2, i -> 3}, {a -> 11, b -> 2, i -> 5},...

In[5] := a b i/. rules
Out[5] = {66, 110, 154, 286, 374, 418, 506, 638, 66,...

Again, many of the partial solutions produced by these rules are invalid. Only 36 are the required 4 digits long, and we also now have information about the form of this solution. We know that the first letter is “T”, given to us by the puzzle setter, and also that the last letter is “T”, because of our solution to 19 Down : “OTO”.

In[6] := candidates = Select[a b i /. rules, Length[IntegerDigits[#]] == 4 &]
Out[6] = {1045, 1265, 1595, 1001, 1309, 1463, 1771, 2233,...

In[7] := englishRules = {0 -> "Z", 1 -> "O", 2 -> "T", 3 -> "T",...
In[8] := germanRules = {0 -> "N", 1 -> "E", 2 -> "Z", 3 -> "D",...
In[9] := candidates = StringJoin @@@ ((IntegerDigits /@ candidates)
   /. Join[english, german])
Out[9] = {"OZFF", "OTSF", "OFNF", "OZZO", "OTZN", "OFST",...

In[10] := Select[candidates, StringMatchQ[#, RegularExpression["T..T"]] &]
Out[10] = {"TTTT", "TFFT", "TTTT", "TFFT"}

Note the duplicates. This is because , but we cannot eliminate them because the rules used to produce them are different, and so will lead to different partial solutions. By applying the constraints systematically, we have reduced the number of potential solutions for 21 Across from 90 to just 4.

We then repeat this process in a recursive manner, using one of the valid solutions to 21 across, and the rules that were used to generate it, to examine the next clue. We continue in this fashion until either the crossword is filled, or we reach a dead end ( which is more likely ). If we reach a dead end, we backtrack ( hence the name ) to one of the possibilities we haven’t tried yet.

Why is this approach any more efficient than the brute force enumeration described above? The key is the generation of partial solutions in a tree like fashion, and the ability to eliminate them when we know that they could not possibly form part of a full solution. When we eliminate such a partial solution because it didn’t satisfy the constraints, we eliminate all the possible solutions leading from that branch. For example, when we eliminated the rule because it produced a solution to 19 Down that was too short, it eliminates not just one solution but all solutions incorporating that rule. Not bad for a single calculation.

The Solution

In abbreviated form, the algorithm looks something like this.

ExamineNextCandidate[{solved_, unsolved_},
  rules_] := {solved, rules} /; Length[unsolved] == 0

ExamineNextCandidate[{solved_, unsolved_}, rules_] := Module[{...},
   {clueIdx, cluePattern} = First[unsolved];
  nextRuleSet =
   GenerateRules[GetSymbolsInClue[ClueExpression[clueIdx]], rules];
  solution = Null;
  While[Length[nextRuleSet] > 0 && solution == Null,
   nextRule = First[nextRuleSet];
   nextRuleSet = Rest[nextRuleSet];
   candidates =
    StringJoin @@@ (IntegerDigits[
        ClueExpression[clueIdx] /. nextRule] /. {english, german});
   candidates = FilterCandidatesByConstraints[candidates];
   If[Length[candidates] > 0,
     nextPatterns =
      GenerateNewPatterns[unsolved, {#, clueIdx}] & /@ candidates;
     nextArgs = {Append[solved, First[#]], Rest[#]} & /@ nextPatterns;
     While[solution == Null && Length[nextArgs] > 0,
      solution = ExamineNextCandidate[First[nextArgs], nextRule];
      nextArgs = Rest[nextArgs]
      ];
     ]
    ];
  solution

Note that the function ExamineNextCandidate calls itself until the pattern on the list of unsolved solutions is met, ie the list is empty. We call this function using an initially empty rules list, and the patterns provided by the puzzle setter.

In[1]:=ExamineNextCandidate[{{}, {...,{20, ".."}, {21, "T..."}, {23, "...."},...}, {}]
Out[1]={...,{20, "FE"}, {21, "TTTT"}, {23, "FZFO"},...}

On my dual core netbook ( A 2010 Alienware M11xR2 ), this took just over 2 seconds to find a solution.

For the second puzzle, we alter the pattern to specify that the first letter of 21 Across ( and last of 9 Down ) can be any letter except ‘T’, using the regular expression "[^T]...". In the code snippets, I have abbreviated the output somewhat. See the attached Mathematica notebook file for full details.

In[2]:=ExamineNextCandidate[{{}, {...,{20, ".."}, {21, "[^T]..."}, {23, "...."},...}, {}]
Out[2]={...,{20, "EE"}, {21, "FFTT"}, {23, "FZFO"},...}

Finally, the last part of the puzzle is to complete the following task:

“Solvers must identify the English transcriptions of the answers entered using German in one of the grids, which, in clue order, form two words of equal length that are to be entered below the grids, one on each side of the colon; this will relate to a claim made above.”

To identify the answers entered in German, we apply the rules returned by the solver to the original expressions and use the German digit replacement rules. We then compare these to the completed grid.

In[1]:={solution,rules}=ExamineNextCandidate[{{}, {...,{20, ".."}, {21, "T..."}, {23, "...."},...}, {}]
Out[1]={...,{a -> 11, b -> 17, d -> 19, e -> 7,...}}

In[2]:=Intersection[(clues /. rules1) /. {a_, b_} :> {a,
    IntegerDigits[b] /. german}, solution][[;;,1]]
Out[2]={10, 11, 13, 16, 25, 29, 30}

In[3]:=(Select[clues, MemberQ[%2, First[#]] &] /. rules) /. {a_, b_} :> {a,
    StringJoin @@ (IntegerDigits[b] /. english)}
Out[3]={{10, "SF"}, {11, "NF"}, {13, "FOO"}, {16, "TNOTE"}, {25, "NON"}, {29,
   "SENSE"}, {30, "SSF"}}

Having identified those clues, we reevaluate the expressions this time using the English rules. There are 4 clues that are the same in both German and English, but it is pretty clear that the required words are “Footnote” and “Nonsense”

Download

Mathematica Notebook (111K)

Geodesic source code for Silverlight 4.0

Geodesics

The shortest path between two points on an arbitrary surface is called a Geodesic, and on a sphere, it is a Great Circle. Modelling the surface of the earth as a perfect sphere, the shortest distance between any two locations on the surface is then described by a section of a Great Circle, ie an arc that lies on the plane that is described by the vectors between its start and end points and the Earth’s centre ( see figure 1 ).

With this information, one way ( and the way I have adopted ) to plot such a curve is as follows:

1. Generate the points of the curve in two dimensions using the parametric equation of a circle.
2. Transform the plane of the 2d curve into 3D space such that it intersects the end points on the sphere, and the sphere’s centre.
3. Project the transformed points back into 2D space using the Mercator projection equations.

[silverlight: Geodesic.xap,520,384,false]

A Parametric Representation of a Great Circle

The Mercator projection gives the 2D rectilinear coordinates (x,y) as a function of the latitude and longitude of a point on a sphere. However, it is easier to draw the point using a typical drawing API, if we have a representation that gives each point of the curve in terms of a single parameter. To derive such a function, we observe that the parametric equation for a circle is given by

$$!begin{pmatrix} x \ y end{pmatrix} = begin{pmatrix} r cos t \ r sin t end{pmatrix} $$

Since a great circle is a rigid transformation of a circle in 3D space, it can also be represented as a function of a single parameter

$$!begin{pmatrix} x \ y \ z end{pmatrix} = mathbf{R}cdotbegin{pmatrix} r cos t \ r sin t end{pmatrix} $$

where $$mathbf{R}$$ is a 2×3 matrix that transforms the plane circle to a location on a sphere. Using spherical coordinates, the Mercator projection of the curve specified above is then
$$!mathbf{C}(t) =begin{pmatrix} lambda \ tanh^{-1} left(sin phi right)end{pmatrix} = begin{pmatrix} tan^{-1}left(y/xright)\ tanh^{-1}z end{pmatrix}$$
Where $$lambda$$ and $$phi$$ are the longitude and latitude of the point to be projected, respectively. Writing the equation out in full gives,

$$!mathbf{C}(t) =begin{pmatrix}

tan^{-1}left(frac{R_{2,1}cos t + R_{2,2}sin t}{R_{1,1}cos t + R_{1,2}sin t}right)\
tanh^{-1}(R_{3,1}cos t +R_{3,2}sin t)
end{pmatrix}
$$

Now that we have a suitable parametric equation, we can draw the geodesic with a series of connected line segments by varying the parameter, t.

Mathematical note – the parameterization given by this expression is highly non-uniform, meaning that there are many more points generated in some parts of the curve than in others. The mathematics of generating uniform ( or natural ) parameterizations belongs to the field of differential curve geometry and is beyond the scope of this article ( and my brain ).

Implementation notes

Inevitably, the mathematics alone is not sufficient to produce an implementation of a reuseable class for the Bing Silverlight control. There are two main issues to resolve; firstly, it is not immediately obvious how to derive from the provided MapShapeBase class to create new shape overlays and secondly, how to handle drawing the curves when they ‘wrap’ beyond the map’s viewable area ( this is easier to illustrate than to describe – see the figure opposite ).

Inheriting from MapShapeBase

I must confess that I cheated slightly in implementing the MapGeodesicPath class, in that I used Reflector to peer into the implementation of the base class. The existing derivations of this class simply defer to MapShapeBase for most of the work, which they can do since for a MapPolygon and MapPolyline there is a one-to-one relationship between Locations, latitude and longitude points on the map and Points, the actual 2D cartesian coordinates used to draw the shape. For the Geodesic, this is not the case, because we only want to specify the start and ending points of the curve, not every point in between. A solution is to delegate the point generation code to a secondary class, one that can be independently tested.

Splitting the curves at the map boundary

When the curves wrap around the map projection, they need to be split at the boundary. This is done by finding the parameter t for the longitude value of the boundary. Where the longitude value is +/- 180, this is straightforward as the equation above reduces to,
$$!t = -tan^{-1}frac{R_{1,2}}{R_{2,2}} $$
For other longitude values, we simply offset the longitude values by the required amount, and calculate the value of t for the new matrix.

For the first time, the main Prime Ministerial candidates for the 2010 UK General Elections, will take part in three live debates. Since the BBC have kindly made the full transcripts available, I decided to have a go at analyzing the data and creating a visual representation in the form of word clouds. I am currently working on my own visualization software, but in the meantime these have been done using Wordle.

Preparing the data

The BBC only provides the data in PDF form – to analyze it we need it in text form. Although this is easily done with Acrobat Reader’s "Save as Text" function, the output it produces is not really suitable for automatic processing, so some work has to be done by hand. This basically involves making sure each speaker’s comments are headed by their name and some kind of special character to split each record ( here I have used ‘@’ ), which took about 15 minutes or so.

Download the raw text of the first debate.

Having done that, a command line tool such as awk can be used to split the data by speaker. For example, the following command outputs Clegg’s comments into a separate file:

awk 'BEGIN {RS=""; FS="[@]"} $1=="NC" { print $2 }' debate.txt > clegg.txt

Parsing the data

Python‘s Natural Language Toolkit provides all the functions needed to analyze the text data, such as tokenizing the text by word and even categorizing each word by type, such as proper nouns and prepositions. For example, having extracted Nick Clegg’s speech as above and read the file as a string using Python, the following commands parse the input for sentences, and then tokenize each word procucing a complete word list.

from __future__ import division
import nltk, re, pprint

sentences = nltk.sent_tokenize(text)
tokens=[]
for s = sentences:
 tokens.extend(nltk.word_tokenize(s))
words=[t.lower() for t in tokens]

We can then categorize each word with a POS tag and extract a list appropriately, for example, using the word tokens above the following extracts all the nouns

# this operation takes some time to execute
taggedwords=nltk.pos_tag(words)
nouns=[word for (word,tag) in words if t == 'NN']

Such a list is enough to use with Wordle, however it’s straightforward to create a word frequency list for use with other software.

nounfrequencies = nltk.FreqDist(nouns)

Going further

Word frequency analyses are fairly straightforward, however NLTK is a powerful library and allows for much more detailed and informative analysis based on grammar and sentence structure. It would be interesting to see the results of a more sophisticated approach.

Gallery of word clouds from the first debate

[Show as slideshow]

As 2010 is the 100th anniversary of the first published infra red photograph, I thought I’d try my hand using my own digital camera and some easily acquired accessories. If you want, you can skip the theory and go straight to the description of the method and a script for Photoshop.

A quick overview of IR photography

The CCD that is responsible for recording the images photographed by most digital cameras, is already sensitive to the near infra red part of the spectrum. That is, the part of the spectrum outside of the range visible to the human eye, but not so far as that used for, for example, thermal imaging. Since most photographers are not interested in light that they can’t see, this light is usually filtered out by an infra red cutoff filter placed inside the camera body, directly in front of the CCD. However, such filters are imperfect, so with some camera models by combining a long exposure with an infra red transmitting filter placed in front of the lens, some of that IR light can be recovered. The figure shows the basic principle, though I should add that the graphs are just a sketch to illustrate the principle and don’t represent an actual CCD response curve.

Pseudocolour

Since IR light is invisible, some method is needed to represent it in terms of the colours that we can see. Typically, this is often done by simply mapping the intensities of the IR image to a greyscale, with the result being a black and white photo with a slightly surreal look. For my photographs, however, I wanted to try an alternative method.

The LAB Colour Model

The most common method of representing a colour image is by a combination of Red, Green and Blue ‘colour primaries’. The reason for this is simply because it then becomes relatively trivial to display an image using a video display, which also constructs an image using a combination of red, green and blue LEDs or phosphors. This is why such a model is known as a ‘device dependent’ model. However, RGB is not the only colour model, and if you are at all familiar with Photoshop you may be aware of the LAB model.

LAB, or CIELAB to give it its full name, is a device independent model, which like RGB is composed of three components ( or channels ) but in this case only the ‘A’ and ‘B’ components contain colour information, whereas the ‘L’ component contains pure luminosity information. Thus, by separating an image into LAB components it becomes possible to manipulate the luminosity and colour components of an image independently.

Method and Implementation in Photoshop

Equipment

1. Canon G10 Digital Compact Camera
2. LA-DC58K lens adapter for same
3. Hoya R72 58mm Infra Red filter
4. Tripod
5. Photoshop ( or GIMP )

The images in this post have been created from two source images, one a long exposure taken using the IR filter and the other a conventional colour image. Both images are converted into the LAB colour space, and a single image is then produced, using the Luminosity channel from the IR image and the ‘A’ and ‘B’ channels from the colour image. The photoshop script below will automate the process when applied to two source images, you can then manipulate the final image using the adjustment tools. You can also do much the same thing in GIMP using the Decompose/Compose features from the Colour menu, but it does not include Photoshop’s image registration or LAB colour adjustment features.

Download ir_composite.jsx.zip

To use the script, open the IR and corresponding colour image, and ensure that the IR image is selected. Then apply the script.

Practicalities

There are a number of limitations to multiple exposure photography methods such as this, especially ones which require long exposures and changes of camera equipment. It is best suited to landscape photography in good conditions where the multiple exposures can be easily aligned. If either image contains rapidly moving subjects these can ( and will ) show up as artefacts in the resulting image.

Gallery

[Show as slideshow]

Microsoft have recently released a beta of Silverlight 4, which has limited support for native interoperation using COM. Potentially, this example could be applied to any number of native interop scenarios, however for this example I have chosen to use Nvidia’s CUDA technology.

Disclaimer : This is an example of what can be done, not necessarily, and in all likelihood, an example of how it should be done.

About CUDA

Up until around 2001 PC graphics cards, though powerful, implemented a fixed function pipeline that limited use to whatever was exposed by the APIs, usually Direct3D or OpenGL. The addition of a programmable pixel pipeline led to the use of graphics cards for more general computation tasks; at first using shaders directly, followed by higher level GPU specific programming languages, such as Brook, SH, and later NVidia’s CUDA. Most of this work was, and is, documented by the GPGPU group. NVIDIA’s website shows CUDA being used in a wide variety of applications but in practice it is best employed in so called “embarassingly parallel” problems.

The demonstration application

The demo below shows a Silverlight 4 beta application, which implements a recursive gaussian filter. Note that this is not the same algorithm provided by the sample in the CUDA SDK, but a more efficient method, which is described in detail in [1] for those interested. The main advantage of a filter implemented in this way is that the computation time is independent of the width of the filter.
To enable CUDA interop, you’ll need a CUDA compatible graphics card. Then do the following,

1. Install the MFC COM application (link below). The installer should register the application with COM automatically.
2. Right click on the Silverlight App and install it for running outside of the browser. The CUDA option should now be available from the Combo box.

Source code : SilverlightCudaInteropDemo.zip
Install MFC COM Application (5.5 Mb)
[silverlight: cuda_interop/SlCudaInteropDemo.xap,520,580,false]

The native component

The native component takes the form of a COM Automation server, implemented as a client side MFC application.

Note: Make sure you run Visual Studio with Administrator privileges, otherwise registering the automation server with COM will fail.

MFC and Automation are beyond the scope of this article, but the basic process I followed was thus

1. Create an MFC Dialog application using the Wizard. Make sure to enable Automation support
2. Add a method to the autmation interface using the add Method wizard from the Class View
3. Add a dual interface using this Technical Note from MSDN.
4. If you get link errors, make sure to include the output of MIDL in the application class ( the one that contains OnInitInstance). I couldn’t find any reference to this step, but it’s how the samples work.
5. Make sure that the run time library options passed to nvcc and msvc match, ie they should all use a DLL or Static linking, not a mixture of both
6. If you get stuck, take a look at the MFC Samples, particularly acdual.

when you pass a native array through COM Automation, it is converted to a SAFEARRAY on the native side. Note that I couldn’t find any documentation on this, I discovered it through experience. The code snippets below show sending and receiving array data between Silverlight and the MFC application.

// note that ComAutomationFactory has become AutomationFactory
// in Silverlight 4 RC
dynamic cuda = AutomationFactory.CreateObject("CudaServer.Application");
float[] data = new [] {1.0f, 3.14f };
dynamic retData = cuda.Process( data );
// retData is a managed float array

VARIANT CCudaServer::Process(VARIANT &data)
{
  SAFEARRAY *pSrcData =  data.parray;

  // this will copy the safe array into the variant
  CComVariant var(pSrcData);

  // when we return the VARIANT containing the SAFEARRAY
  // it will be marshaled to Silverlight as a managed array
  VARIANT retVal;
  VariantInit( &retVal );
  var.Detach( &retVal );
  retVal.vt = VT_ARRAY | VT_R4;
  return retVal;
}

Using MEF to implement the application

The Managed Extensibility Framework is an extensible plugin framework for .NET applications and Silverlight. I have used it to dynamically discover implementations of IProcessorProvider based on the permissions available to the Silverlight application. The figure below shows the component structure of the application.

Performance notes

Silverlight

Unlike the Reaction Diffusion simulation, for this application I have chosen to use Silverlight’s WriteableBitmap, introduced in Silverlight 3, rather than dynamic PNG encoding. This revealed an interesting performance issue when using a typical double loop to iterate over the pixels. Initial timings revealed that the vast majority of the time was spent in updating the WriteableBitmap rather than actually performing the image processing. The initial update loop used the PixelWidth and PixelHeight properties to bound the loop counters, taking about 200ms to iterate over the loop.

for (int j = 0; j < bmp.PixelHeight; ++j)
{
  for (int i = 0; i < bmp.PixelWidth; ++i)
  {
     // update pixels
   }
}

By caching the bitmap properties in local variables, the timing was reduced to ~5ms. Needless to say I was shocked by how much of a difference such a seemingly trivial change made.

int pxWidth =  bmp.PixelWidth;
int pxHeight = bmp.PixelHeight;
for (int j = 0; j < pxHeight; ++j)
{
  for (int i = 0; i < pxWidth; ++i)
  {
     // update pixels
   }
}

OLE Automation

The guidelines for building performant automation code is much the same as that for other unmanaged interop scenarios in .NET : avoid chatty interfaces. Note that this is exactly what I have not done here. In fact, the time it takes CUDA to perform the image processing is dwarfed by the time it takes to marshal the data between Silverlight and COM. This can be mitigated somewhat by splitting the blur call into two operations, one to load the image, which is called only upon initialization, and one to perform the blur.

CUDA

CUDA operations are extremely sensitive to data alignment and the order in which threads access data. Kernels should be written in such a way that threads access adjacent data elements, meaning that the row major access pattern familiar to C and C# developers would produce suboptimal performance ( sometimes by as much as an order of magnitude ). Instead, array accesses should be performed in a manner more reminiscent of FORTRAN. In addition, 2D arrays should be padded out so that threads access data elements that are correctly aligned ( see the CUDA documentation for the correct alignment values ). A full exposition of performance optimization for CUDA is really beyond the scope of this article, there are many examples in the NVIDIA documentation although the terminology can be somewhat opaque. One of the clearest explanations I have found is this presentation from Mark Harris at Supercomputing 2007.

__global__ void kernel( float *destData, float *srcData, int stride, int height )
{
  // suboptimal access. Each thread accesses elements
 // in a striding pattern
  int rowStart = (blockDim.x*blockIdx.x+threadIdx.x)*stride;
  for ( int i = rowStart; i < rowStart+stride; ++i ) {
    destData[i] = srcData[i];
  }
}

__global__ void kernel( float *destData, float *srcData, int stride, int height )
{
  // optimal access pattern, each thread accesses adjacent elements
  int colStart = blockDim.x*blockIdx.x+threadIdx.x;
  // this case, 16*sizeof(float)= 64 bytes
  for ( int i = colStart ; i < colStart+(stride*height); i+=stride ) {
    destData[i] = srcData[i];
  }
}

References

1. Young, I.T. & van Vliet,L.J, 1995. Recursive implementation of the Gaussian filter. Signal Processing, 44, pp.139-151.

Silverlight toolkit themes do not support merged dictionaries

Unfortunately, merged dictionaries are not supported by the Silverlight Toolkit’s Themes feature – at least not as it stands. For example, the following XAML and accompanying class causes a XamlParseException if you try to use it.

public class MergedTheme : Theme {
 public MergedTheme() :
  base(typeof(MergedTheme).Assembly, "MergedTheme.Theme.xaml") {
  DefaultStyleKey = typeof(MergedTheme);
 }
}

All is not lost however, since full source code is provided for the Silverlight Toolkit, so it becomes a straightforward matter to track down the problem and supply a patch.

Writing a test to reproduce the problem

Since the toolkit comes with an accompanying test suite, we proceed in textbook TDD fashion by first writing a test that exercises the issue. We can reuse some of the existing code to do this, adding a resource dictionary containing a merged dictionary reference and adding the following test to the ImplicitStyleManagerTest class

///

/// Test that styles contained in a  merged dictionary can be successfully applied
/// 

[TestMethod]
[Asynchronous]
[Description("Test that a dictionary contining merged dictionaries can be successfully loaded.")]
public void TestResourceDictionaryWithMergedDictionaries() {
 Uri uri = new Uri("System.Windows.Controls.Testing.Theming;
  component/ImplicitStyleManager/MergedResourceDictionary.xaml", UriKind.Relative);
 TestAuto( (stackPanel) => {
  SetResourceDictionaryUri(stackPanel, uri);
  ImplicitStyleManager.SetApplyMode(stackPanel, ImplicitStylesApplyMode.Auto);
  }, Colors.Blue);
}

The test fails as we would expect.

Fix the code to make the test pass

The Theme class works by transforming the ResourceDictionary xaml into a ContentControl. A quick glance at the xaml generated by this process in the Visual Studio debugger shows why an exception is being raised.

The ResourceDictionary.MergedDictionaries element is missing its parent ResourceDictionary element, so we add a couple of lines of code to ResourceParser.ParseElement to detect the element and wrap it in a ResourceDictionary, using LINQ to XML.

private static void ParseElement(XmlReader reader, XmlWriter writer, bool checkTypes) {
 if (reader.Depth == 0) {...}
 else {
  // handle merged dictionaries by wrapping them in a ResourceDictionary element
  if (reader.Name == "ResourceDictionary.MergedDictionaries") {
   XElement parent = new XElement("ResourceDictionary");
   parent.Add(XElement.ReadFrom(reader));
   writer.WriteRaw(parent.ToString());
  }
  else { ... }
 }
}

Having made the change, we rerun the test.

As we would hope, our custom theme now works as expected and we also have a unit test that ensures the bug does not reoccur. Note the Xaml code above, in that the Resource Dictionary is referenced using Pack Uri syntax. This is because Button.xaml is compiled as a Resource, rather than Content, since the latter doesn’t appear to work reliably and can result in obfuscated error messages such as XamlParseException: attribute “Button.xaml” value out of range.

I have submitted the patch for this fix to the toolkit site on Codeplex.

Rendering

The first obstacle to implementing the RD simulation is that Silverlight 2.0 does not by default provide a means of generating dynamic images. WPF has a WriteableBitmap, but no equivalent exists in Silverlight. However, it does support PNG streams so we can dynamically update a bitmap by encoding it to PNG on the fly. For this I have used Joe Stegman’s PNGEncoder class, which I have modified slightly to deal with RGB data and to reduce memory usage.

Performance notes

Performance of the simulator is dominated by the time it takes to update the arrays containing the concentration values of the chemicals. Actual rendering time is negligible, even though the image has to be dynamically encoded to a PNG file in order for Silverlight to display it. There are two ways of implementing 2D arrays in Silverlight with C#, rectangular arrays or C style jagged arrays. If this were a desktop application, the most performant method would be to use rectangular arrays and unsafe access, however that option is not available to us in Silverlight so it turns out that, for safe access, jagged arrays are faster.

For example, originally the simulation step looked like this

for (int j = 0; j < H; j++)
{
 int jm = j != 0 ? j - 1 : H - 1;
 int jp = j != H - 1 ? j + 1 : 0;
 for (int i = 0; i < W; i++)
 {
  int im = i != 0 ? i - 1 : W - 1;
  int ip = i != W - 1 ? i + 1 : 0;
  double d2u = U[j,im]+U[j,ip]+U[jm,i]+U[jp,i]-4*U[j,i];
  double d2v = V[j,im]+V[j,ip]+V[jm,i]+V[jp,i]-4*V[j,i];
  double uv2 = U[j,i]*V[j,i]*V[j,i];
  Uu[j,i] = du*d2u-uv2+F+U[j,i]*one_minus_F;
  Vv[j,i] = dv*d2v+uv2+V[j,i]*one_minus_F_minus_k;
 }
}

On my machine this took on the order of 600ms to perform 100 iterations. Using jagged arrays instead, we have the following

for (int j = 0; j < H; j++)
{
 int jm = j != 0 ? j - 1 : H - 1;
 int jp = j != H - 1 ? j + 1 : 0;
 for (int i = 0; i < W; i++)
 {
  int im = i != 0 ? i - 1 : W - 1;
  int ip = i != W - 1 ? i + 1 : 0;
  double d2u = U[j][im]+U[j][ip]+U[jm][i]+U[jp][i]-4*U[j][i];
  double d2v = V[j][im]+V[j][ip]+V[jm][i]+V[jp][i]-4*V[j][i];
  double uv2 = U[j][i]*V[j][i]*V[j][i];
  Uu[j][i] = du*d2u-uv2+ F+U[j][i]*one_minus_F;
  Vv[j][i] = dv*d2v+uv2+V[j][i]*one_minus_F_minus_k;
 }
}

Just replacing the array type and making no other changes this now takes about 500ms, an improvement of about 13%. However, now that we are using jagged arrays we can do better and move the row accesses out of the inner loop.

for (int j = 0; j < H; j++)
{
 int jm = j != 0 ? j - 1 : H - 1;
 int jp = j != H - 1 ? j + 1 : 0;
 var u_m = U[jm];
 var u = U[j];
 var u_p = U[jp];
 var v_m = V[jm];
 var v = V[j];
 var v_p = V[jp];
 var uu = Uu[j];
 var vv = Vv[j];
 for (int i = 0; i < W; i++)
 {
  int im = i != 0 ? i - 1 : W - 1;
  int ip = i != W - 1 ? i + 1 : 0;
  double d2u = u[im]+u[ip]+u_m[i]+u_p[i]-4*u[i];
  double d2v = v[im]+v[ip]+v_m[i]+v_p[i]-4*v[i];
  double uv2 = u[i]*v[i] *v[i];
  uu[i] = du*d2u-uv2+F+u[i]*one_minus_F;
  vv[i] = dv*d2v+uv2+v[i]*one_minus_F_minus_k;
 }
}

This brings the count down to about 330ms, an improvement of about 40% over the original implementation.

We’ve taken the sequential code about as far as we can go. The next step is to try to take advantage of muliple cores that may be available on the user’s machine.

One of Alan Turing‘s many contributions to mathematics and science during the 20th century was his 1952 paper on “The Chemical Basis of Morphogenesis” in which he suggested that a simple model of coupled differential equations could account for pattern formation in natural processes such as those found on animal coats. Such models are known as Reaction-Diffusion models, and take the following general form

The equation describes how the concentration of each chemical components in q evolves as a function of the other components. I have chosen to illustrate the Gray-Scott model; the physical derivation of the reactant term is described in detail in [2], the addition of the diffusion term and the resultant behaviour in [3]. The model has two chemical components, U and V and is described as follows.

The model is what is knows as an activator-inhibitor model, in which one chemical acts to inhibit the growth of the other.

Simulation

The simulation is implemented with Silverlight 2.0. For full details of the implementation see the separate post. For guidance in parameter selection, see [3], or just select a preset and modify it. Note that the Png image generation feature currently requires the Firefox browser.

[silverlight: ReactionDiffusion.xap, 450, 300]

Gallery

[Show as slideshow]

References

1. Turing,A.,1952. The chemical basis of morphogenesis [pdf]. Philosophical Transactions of the Royal Society, 237 pp.37-72
2. Gray,P. and Scott,S.K. 1985. Sustained Oscillations and Other Exotic Patterns of Behavior in Isothermal Reactions. Journal of Physical Chemistry, 89 pp.22-32
3. Pearson,J.,1993. Complex patterns in a simple system. Science, 261(5118) pp.189-192

It also happens to be the model used by the Open University’s course on Mathematical Logic and Number Theory, and is the reason for this article since I studied the course in 2007.

Emulator

The implementation details of the emulator are quite interesting, using a mixture of Microsoft technologies including F# and Silverlight, however I will save that for another article for those interested in such things. To use the Emulator just hit Reset, enter the program and initial registers and click Step to trace through the program.
[silverlight: Urm.App.xap,400,300,false]