python-csp

Europython was excellent this year and while the range of talks was very wide, the main themes seemed to be web scripting and web applications, virtual machines and concurrency. Russel Winder gave a keynote on the future of multicore computers and ... [More] their implications, and a longer talk about Python and the CSP style of concurrency, Michael Sparks gave a lightening talk about his work on Kamaelia, Andrew Francis talked about implementing ALTing in Stackless Python and Tony Ibbs talked about KBus, which is like DBus but sensible, and I gave an introductory talk on our python-csp project. Following Europython, Guido van Rossum has a nice post on his experience of the conference, which mentions CSP related work.   python-csp sprint report Post-talks we had two days of Sprints and, having never been to a Sprint before I'm really amazed at how much we managed to get done in the python-csp sprint over such a short amount of time. I concentrated on the online tutorial which was well overdue for an overhaul, and the below is just a brief account of all our other activities -- apologies if I've missed anyone out or mis-attributed anything: Fabian Kreutz fixed and merged Russel's Python3 port into the default repository, closed a bunch of ancient branches, tidied up our strategy for importing the code library, fixed a whole load of bugs and hosted a Mercurial clone of the main repository on BitBucket for us. Russel started a MacOS port, found a whole world of issues with that, and fixed a few bugs.  Richard Taylor joined us on the Friday and helped configure Linux to solve the "too many open files on disk" OS error that is thrown when too many channels or processes are created. Tony Heskett told us how to fix the same problem on Mac OS. Richard also gave us some really great ideas about how to use the POSIX standard to finish off Sam Wilson's work on a C implementation of channels. Joe Metcalfe, Wout Tankink, Urban Skudnik and others went through the tutorials in some detail, fixing bugs and posting a bunch of bug reports. Fabian tried to eliminate some of the dependencies from the examples and make them a bit easier to understand. Urban also wrote a channel poisoning example and the tutorial page that goes with it. Michael Sparks had to go home, but joined us over IRC and gave us a better understanding of the differences between Kamaelia and other asynchronous forms of message passing and the CSP style of programming. Later Michael produced a really interesting OCCAM model of one form of Kamaelia synchronisation primitive, which has been a good prompt to sort out guarded ALTing for python-csp. Stefan Schwarzer went through our testing strategy and started formalising a really neat way to unit test python-csp code and wrote up some unit tests for many of the built-in processes we have in the library. Stefan also got us thinking about licensing issues for python-csp, which probably deserves a wider discussion elsewhere.  Andrew Francis dropped in briefly on his way to the PyPy sprint, and sent over a deadlock detection algorithm and some other ideas. Thanks to everyone who came along to the Sprint, it was great fun and hopefully we can keep up some momentum. During the Sprint we added six new project members with commit privileges and set up a public mailing list. If you were at the Sprint or already had commit access to the repository you should already have received an invite to the list, but if not it is public so feel free to join. Permalink | Leave a comment  » [Less]

Much of the recent activity on the python-csp code base has to do with finding useful ways to debug networks of CSP processes. The dynamic analysis of CSP programs has a long history, dating back to Hoare's original work. I'll post more about that ... [More] another time, but this post details a small change to the core library that enables debugging to run effectively.  A common idiom in CSP-style code is to have many processes in a process network (connected via channels, or other guards) which all loop forever. If the program needs to be terminated, that is usually done via channel poisoning. I call these processes server processes, because they continually serve values to their output channels (if they have any). In fact, all of the processes in the csp.builtins library are server processes, and python-csp has quite a few of these. It has most of the processes from the JCSP "plugNplay" package and a distinct process for every built-in unary or binary operator in Python. So, for example, the csp.builtins.Plus processes reads two pieces of data from its input channels, and sends their addition to an output channel. One possible implementation of csp.builtins.Plus would be this: @process def Plus(inchan1, inchan2, outchan):     while True:         outchan.write(inchan1.read() + inchan2.read() In reality, this isn't quite how csp.builtins.Plus is implemented, as there are quite a few unary and binary operators in Python, so we make use of metaprogramming to generate most of these builtin processes. The problem with server processes is that if you want your debugger to deduce useful information about your program then at some point your code needs to terminate so that the debugger can compute a call graph, or some other structure, from which other useful information can be gathered. Ideally, it would also be nice if the debugger could deduce that your server processes are intending to run indefinitely, even though it actually terminates when the debugger executes them. To achieve this effect, python-csp now has a special class for creating server processes called CSPServer, and a new decorator, called @forever. These work exactly like the usual CSPProcess and @process types that have been included in python-csp since its inception. Apart from using the new constructors, the only change you need to make in your code to use @forever or CSPServer is that a server process should be a generator and, usually, it should yield at the end of each iteration of its internal loop. As an example, if we were to reimplement csp.builtins.Plus with the new constructors, we can do this in two ways, firstly with the decorator (which is the recommended method) and secondly using the CSPServer class directly. In the code below I am using two other processes from csp.builtins: Generate which sends an infinite supply of integers down its output channel and Printer which prints anything it reads from its input channel to STDOUT (by default, you can customise this to use any other file type). So now, Plus could be implemented thus: @forever def Plus(inchan1, inchan2, outchan):     while True:         outchan.write(inchan1.read() + inchan2.read())         yield ... # Silly way to print even numbers in1, in2, out = Channel(), Channel(), Channel() Generate(in1) & Generate(in2) & Plus(in1, in2, out) & Printer(out) If we decide not to use decorators, then the code is essentially the same, we just need to remember to construct any Plus processes with the CSPServer class: @forever def Plus(inchan1, inchan2, outchan):     while True:         outchan.write(inchan1.read() + inchan2.read())         yield ... # Silly way to print even numbers in1, in2, out = Channel(), Channel(), Channel() Generate(in1) & Generate(in2) & CSPServer(Plus, in1, in2, out) & Printer(out) Permalink | Leave a comment  » [Less]

Pygame is a popular and excellent library for writing 2d arcade games and animations. It takes an approach that has become very popular in the Python world, it wraps the low-level SDL library giving programmers the best of both worlds -- ... [More] efficiency from the underlying C++ code and a simple, productive scripting environment from the high-level Python wrappers. Pygame is so simple, it is used in a number of introductory text books, including our own book, Python for Rookies. One reason for the simplicity of Pygame is it's approach to events: events exist, but Pygame is not event driven. The programmer has to write his / her own event loop and process events which are relevant to the application explicitly. This makes for a model of interaction that is very easy for beginners to understand and very easy for experts to get right. For nearly a year now I have been working on python-csp which adds Hoare's Communicating Sequential Processes (CSP) toPython and is nearing a full release. You can read more about CSP on the WoTUG site, but briefly it is a neat way of implementing "message-passing" concurrency to construct concurrent and parallel programs. CSP eliminates several classes of well-known bugs in this sort of software, including race conditions, and makes it much easier to avoid deadlocks. In a CSP program, there are no "locks" which cuts out a lot of difficult boilerplate code. Instead, a CSP program typically consists of a number of CSP processes (which may be reified as threads, processes, coroutines, or anything else) running in parallel. These communicate by sending data along synchronous "channels", which you can think of as being similar to UNIX pipes. Wherever you might use shared data or fire an event in another style of concurrency, in CSP you would send and receive data down a channel. Because the communication between processes is synchronous, the flow of data between processes can only happen in the order in which it appears statically in your code (a big advantage compared to event-driven systems).  The details of python-csp will keep for another post, but I wanted to document here a pattern for fixing a very irritating problem that occurs when writing python-csp code which uses Pygame: it's very difficult to kill the application in the way you normally would, by pressing the "Close" button on the application window, pressing Alt+F4, or whatever. In CSP programs, the usual way of terminating running processes is to "poison" the channels which they use, which causes all processes which read / write to those channels to propagate the poisoning on any channels they know about and terminate themselves. Neil Brown has a very nice post on poisoning here if you want more details and nice graphics. In Pygame, to quit the application there's a handy pygame.quit() function. Mixing these two requires a bit of alchemy, so it's worth knowing a pattern that works. For simple programs, the following is enough: just place all Pygame related code in a single process which draws to the application window and have one or more channels to pass information from the rest of the running program to the drawing process. When a pygame.QUIT event is received drop out of the main animation loop and then (only then) poison any channels and quit the graphical application. Just like this: @process def Drawme(channel, _process=None): import pygame # Constants width, height = 512, 256 # Open window pygame.init() screen = pygame.display.set_mode((width, height), 0) quit = False while not quit: data = channel.read() # Drawing code goes here... for event in pygame.event.get(): if event.type == pygame.QUIT: quit = True # Process other events here ... channel.poison() pygame.quit() return   If you try this out, make sure to have a separate terminal open to watch the number of Python processes running in your OS and check that quitting really does work. On UNIX systems you can use the 'watch' utility for this: watch -n 0.5 'ps h -C python -o pid' Here's an example Pygame / python-csp application, a simple demonstration of Reynold's "flocking" algorithm, which simulates a flock of birds or other wildlife moving in unison:   Permalink | Leave a comment  » [Less]