Tim Bray is hosting an interesting experiment in parallel computing: he chose a simple, mundane task, and he's throwing a 32-thread Sun Fire T2000 server at it. Anyone can submit programs in any programming language; the goal is to see how easily can a parallel task be implemented using today's tools. Take a look at the wiki for more information about the proposed task.

I intend to have a whack at the problem. My language of choice is Python - it's reasonably fast, very programmer-friendly, and there seems to be a nice selection of parallel computing libraries to choose from. If they're not up to the task, I have an idea on how to roll my own (and I kinda hope that I'll need to do that - should be fun). I intend to keep my implementation as short and straightforward as possible. In particular, I won't optimize much the parts that can be parallelized; rather, i'll focus my effort on removing multiprocessing bottlenecks.

So far I've written a simple, single-threaded implementation (simple.py) that's similar to Tim's reference implementation, to use as a starting point. It doesn't exactly match the reference output because, in its current form, it's flawed: sort order for lines with the same count is not specified. Apart from that, the output of simple.py is correct.

Now, let's see if we can make this baby dance...