Metaphor, Complexity and Postmodern Programming

A recent interview with architect Victoria Livschitz of Sun entitled Envisioning a New Language provoked some comment on the Postmodern Programming discussion group recently. The article describes "Metaphors", a new conceptual vision for a programming language and environment that will, in one stroke, eliminate all the tricky aspects of programming, letting developers concentrate purely on conceptual modelling and algorithms without worrying about hardware, networks, memory, existing systems and other pointless, technical stuff. From the interview:

I believe it's time to step back and consider the first 50 years of computing as one gigantic prototype. If we had the courage to start over, we could build a new software creation and execution model to address the software engineering challenges of the 21st century, such as distributed processing, autonomic computing, and software evolution -- natively, within one conceptual universe [my emphasis].

There's one fatal flaw with this plan: it's only a matter of time before some poor soul would have to integrate systems in Sun's "one conceptual universe" with those in Microsoft's "one conceptual universe" or the multiple, competing "one conceptual universes" that the open source and free software communities would create.

To quote Keith Braithwaite, the great thing about single conceptual universes, as with standards, is that there are so many to choose from.

For some time I bought in to Fred Brooks' notion that programming problems can be divided into "essential" — the inherent complexities of the application domain   and "accidental" — complexities produced as a by product of using the technologies themselves. I now think that's wrong; accidental complexity is the essential complexity of programming itself, of writing programs that must fit into an existing context of people, organisations, their needs and wants and the legacy of their history.

I think trying to get rid of accidental complexity is like harking back to a classical golden age that never really existed. It can be source of inspiration at best (think of St Pauls' neo-classical architecture) but dangerous at worst (think of Mussolini's attempt to recreate the Roman empire in Abyssinia). The attempt to create a single conceptual universe to rid the world of accidental complexity is doomed to failure. Wren could not convince Londoners to live within his grand vision of what London should be. Today St Pauls is situated within the old medieval street plan rebuilt after the fire of London and is within sight of many examples of later architectural movements, equally grand in conceptual scope yet totally incompatible with Wren's vision. That mix of styles is one reason why London is such an interesting place to be. The challenge of building to meet people's conflicting needs and wants while preserving the legacy of the past is what makes architecture so fascinating. I think it's the same for software.

Thanks to Keith Braithwaite, Steve Freeman and Martin Fowler for inspiration, bon mots and encouraging me to get this off my chest.

Before Iteration Zero

Steve Freeman and I ran a workshop at XP Day 2005 entitled "What to do Before Iteration Zero". Simon Baker has written a report of the session with photos. Some photos I took are below.

Inspired by Dave Snowdon's Cynefin methods we used story-telling to facilitate the initial, brainstorming phase of the workshop. Steve and I kicked off by each telling a story of something that went wrong on a project that could have been avoided by a bit more forethought before development started. We then asked the participants, who were seated cafe-style, in small groups, to share stories about things that worked well on a project because the right support had been put in place, or things that didn't go well but could have been avoided with some earlier preparation. While they told stories, each table recorded good and bad issues on green and red index cards for later analysis and discussion and presentation in poster form.

My story had a technical bent while Steve concentrated on the business side of things. I told of when I was called in to help with an emergency fix to a production system only to find out that they didn't know what version of the system was running in production, couldn't run the system in a test environment without corrupting production data, didn't know that the system in the test environment had been corrupting production data, couldn't build the system correctly without laborious, error prone manual patching, and that they didn't even know where all the code of the system was. The code wasn't all in the source repository because, they explained, "there's no point in checking in code until you know it runs in production". If an automated build and deployment process had been in place before work started on feature development debugging would have been much easier or, I suspect, never been necessary.

The story telling seemed to work well. Discussions started quickly; there was no awkward muttering as people worked out what they were meant to do. From what I could tell as I wandered between the tables, everyone stayed focused on the topic and each table produced a lot of cards. A lot of cards: I had to pick them all up after they had been spatially sorted on the floor. Next time I run the workshop I'll get the participants to do that job.

Sorting the cards	Discussion
Drawing a poster	The posters on display