For XP Day Benelux 2005 we are working on a more agile session selection process. So far, we had a semi-traditional process, where we put a lot of thought in the text of the call for sessions, made sure the candidate sessions received adequate feedback on the description as well as the supposed working of the session. The selection itself was BDUF (Big Design Up Front).

You could send in a session (I'm looking for a different word than 'submit' which suggests submission to the conference organisers - we are slowly and surely coming down from our mountain ;-) ), and then you'd wait for a month or two and hear wether your session was accepted or not (with an indication of why and why not). After that, we shepherded the accepted sessions.

Now we are trying to come up with a more SDUF-like process. The S can stand for Small as well as Strategic. We want to start shepherding long before the session deadline, and we want to open up the review and shepherding process, so session organisers don't feel they are judged by an invisible elite. Last year we invited quite a number of people from outside the program committee. This year, we're taking this one step further, by inviting everyone who sends in a session to be a reviewer as well. So we have four roles per session , three of which can be assumed by anyone involved: one or more organisers, a shepherd and three reviewers.

There is but one restriction: anyone can fulfil only one of these roles per session at the same time. Shepherds will announce their availability, interests and skills on a wiki somewhere (location to be defined yet). Session organisers are free to ask any shepherd they want - as long as the shepherd agrees. Reviewers can choose any sessions they wish to review - as long as there is no conflict of interest.

The fourth role is that of program committee member. Since the determination of quality has already been made in the review process prior to session selection, the program committee can focus on creating a balanced program. The program committee is semi-open. We've accepted a few new members this year after they asked to participate.

In a way, we are open sourcing the workshop creation pipeline we've been having with the people from Agile Systems. For a few years we've been collaborating on each others' sessions. It is happening this weekend again, as the deadline for XP2005 workshops and tutorials is approaching. I can hardly explain how great it is to work with like-minded people on workshops, e.g. recently with Rob Westgeest on Rightsizing your unittests and Marc Evers on a series of Systems thinking workshops. This weekend I'm co-working on a session on congruent communication and interactions with Nynke Fokma and Marc Evers, as well as a session on value stream mapping with Marc. I may tag along on some other sessions as well. I hope we can scale this up through Agile Open and the new XP Days session generation process, so more people can experience it, and we get an even greater diversity of workshop and training material.

What we envision for the process itself is to start ten weeks before the final acceptance deadline. In these ten weeks, we offer session organisers the possibility to organise three iterations around their sessions. We are going to recommend organisers take a period of three weeks, and do all the iterations in this period. Such a short timespan enables the organisers to focus on their session, and not have to swap thoughts about the session too much. Three iterations is in my opinion a good amount. If it doesn't work after three iterations it is probably best to leave it and try out another idea - not all ideas translate well into sessions. If the idea does work, after three iterations the organiser, shepherd, reviewers and program committee can have a sufficient idea of how it will work. Fleshing out the session further can be done between acceptance and running the actual session.

At the end of the first three weeks, we'll already have some sessions in the conference pipeline, and the session organisers already have some idea if their session stands a chance or not, since it is already reviewed. At the end of ten weeks, we hope to have more than enough sessions for the programme. We'll find other venues for the high quality sessions we can't fit in the programme (e.g. we moved a session to an Agile Seminar last year). That is the end of the pipeline - finished, high quality sessions come out, and only scheduling remains.

I'm curious to see how this will develop. Worse comes to worst, we can always revert to old big up-front session acceptance. Something tells me we're not going to have to - as the new process involves everyone much more.

Registration

FOSDeM has no formal registration and name tags, which appeared to be a good thing, as there were many more visitors this year (around 1400 on the first day alone) than last year (around 300). With 1400 people, registration processing might take forever.

Open Protocols and Data Formats

Due to getting lost earlier, I dropped in late on Tim O Reilly’s talk. "The Internet is open source’s biggest success […] open protocols and data formats are even more important than open source".

I felt right at home, as my colleagues at CQ2 are completing the implementation of the OA-x (Open Archive extended) protocol and data format.

Contemplating on this, I see that open protocols and data formats are more accessible than open source - since you can choose whichever programming language and operating system you want to make use of the protocol and the data (TCP-IP and e-mail being succesful examples of this). Even if one part of the protocol is complex (i.e. The ‘google server’) you can write a simple client (google access with soap) re-using the implementation and installation complexity someone else has resolved for you.

For I-tor this might mean the amount of people working on the i-tor server might remain small - since it contains a lot of advanced features not always needed, while other people might more easily make clients for editing specific kinds of OA-x documents, or make simple servers publishing and storing documents in OA-x format. The i-tor server acts as an intermediary for combining collections of oa-x documents, publishing them through a web-interface and allowing them to be queried interactively by end-users.

Ruby

I went to Richard Kilmer’s talk on Ruby , to see how someone who uses it everyday presents it, and to pick up some stuff I might have missed. New stuff I picked up from this talk:

• YAML (included in ruby 1.8) - a non-XML human readable/editable serialization format for ruby and perl. The library uses ruby’s class extension facilities effectively - after loading YAML all your objects have a ‘to_yaml’ method. YAML is made for readability and ease of use. Existing serialisation methods are more efficient.
• ROMP - "(the Ruby Object Message Proxy) is a set of classes for providing distributed object support to a Ruby program."

ROMP does the same as DRB (Distributed Ruby), but is much more efficient according to Rich Kilmer. (I couldn’t get more than 50 messages a second out of DRB, but apparently, there now is a faster version. ROMP handles up to 40000 messages per second). It is also very easy to use - with a few lines of ruby, you can make an object available on a tcp port. Using ROMP seams a decent way of splitting presentation from storage of objects. The server stores objects and deals with concurrency, and several presentation server applications and threads can access the objects.

Ruby-GNOME

This talk was on the integration of Ruby with GTK libraries. The design of GNOME is interesting, since it is highly plugin-oriented, with a pipes-and-filters like architecture for multimedia applications. The ruby-bindings convinced me less, since creating a GUI is doable, but requires double steps - the GUI ends up in a separate configuration file from the code, and the names of events have to be specified twice. For small applications, such as the media player that was demonstrated, this is doable. For larger applications I foresee it will be difficult to keep an overview on everything defined.

ReiserFS - the fast pluggable filesystem

Hans Reisers’ talk on ReiserFS once again proves that good design and high performance software are not opposites, but go hand in hand. ReiserFS version 4, to be released this week, features plugins for various aspects of the filesystem, from allocating blocks, to storing individual files. The reason for the plugin architecture was, as mr. Reiser said, that optimizing a filesystem is an empirical science. You need to try something at least three times, fail two times and then find a better solution. The plugins allow many people to try out their hypotheses. The ReiserFS distribution can then include the most succesful plugins.

One of the plugins planned for the near future is compressed files (like e.g. Atacker in the old days, only more reliable), since disk speed has not kept up with cpu speed increases, this will allow for faster disk (read) access. To me it seems a good solution for storing XML files… Another thing which is easy to do with plugins is encryption.

The vision behind ReiserFS is also interesting - the file system should take care as much as possible for things like atomicity, so applications built on top of it can be as simple as possible. ReiserFS is intended to replace databases as a means of storing many small objects - it does not allocate a lot of space for individual files anymore, so it is possible to efficiently store each object in its own file, even if it is just a few fields of data. ReiserFS takes care of efficiently storing all the objects. The explanation of ‘dancing trees’ (a variation on B-Trees) was interesting. A file system is basically a huge map, with file-id’s as keys, and file data as values.

Concerns influencing the design of ReiserFS, such as plugins and granularity of security are familiar to me, since we face them in designing I-Tor as well, so maybe a close mapping on ReiserFS could enhance performance (small objects optimized by the filesystem) and facilitate development (atomicity, security).

Frequent releases

Agile Software development things like frequent releases come through. I was talking to the GNOME packager for MandrakeSoft at dinner, and he said they recently switched to weakly developer releases, regardless of content (just release frequently), and every 6 months there is an end-user release, where stable features are chosen and released. API’s can change every 6 months, old methods are deprecated and kept for one more release, ensuring a stable application development environment, as well as evolution of the API’s.

Summary

FOSDEM is an interesting conference for free and open source software developers. You can quickly get an idea what is going on in major open source projects. Also, many groups take the opportunity to actually develop software together in the developers’ rooms for specific projects.

Lotus Notes for instance, was released five years after development, and it took another six years before the user base started to really grow. As an example from my own experience, we're using Linux on our laptops and servers, an open source variant of Unix. Unix exists since at least the 70's. In the first twenty years of its' existence, high-priced unix workstations and servers were mainly to be found in universities and corporate research and development environments. Now, after over thirty years, through Linux (and to a lesser extent, FreeBSD and its' derivative, MacOS-X) it is spreading to a much larger user-base.

Starting last December, I am programming on the e-laborateproject. Not exactly a long-running project. It is, however, based on i-tor, open-source software that has developed through several projects starting January 2002. So, I-tor is now in its third year. Having been involved as a coach and programmer, I can recognize several of the pitfalls Joel Spolsky managed in the project. Having been there, I can say, that it is possible to survive those pitfalls, even if you've stepped right in them.

Yesterday, someone described us at CQ2 as "Software Efficiency Experts" . Probably, because we try to eliminate waste in software development, and we continually search for, and find practices that work well for us and our clients.

But what about Effectiveness? It is nice to reduce costs, but if you develop the wrong things, you are getting nowhere. You are getting there more cheaply than otherwise, maybe faster as well, it is all well tested, but it still is the wrong software!

The best way, in my opinion, to increase effectiveness is to eliminate waste in requirements:

• Don’t automate things that don’t have to be automated. If you leave it up to some technical people (I know I’ve been one), you would have everything automated. Instead, automate work that costs much more to do by hand than with software, or automate something so you can deliver much more value to your customer than otherwise possible.
• Program your software ‘just in time’. Delay implementation until you really need the software. It is more effective to invest in software you need now, than in software you don’t need right now, and might never need at all.
• Make sure all the features developed so far provide value. Measure usage of features, remove them if they aren’t used anymore. Unused features cost money for time spent maintaining software, and possibly planning things based on these features. Sometimes it takes time to explore software by using it, to find out which features are really essential and which aren’t.

Now that we are developing the right thing, we can focus on developing the thing right. What works well for us in this respect, is Test Driven Development (which makes reliable software cheaper to build and maintain than unreliable software), Refactoring and some other techniques from Extreme Programming in combination with increasing the personal effectiveness of development team members and better and more enjoyable teamwork through e.g. Retrospectives.

Cost-effective is a nice word to describe this, because it focuses on cost (efficiency) as well as on value (effectiveness).

The metaphor Erik and I use for this these days, is that you have an engine which starts running very efficiently and is very powerful, but the car around it hasn’t changed, so the transmission can’t cope…

Then you can decide to optimize the car as well, but you discover there are holes in the road. Then you can start to fix the holes in the road and so on…

The xp-team is the engine, the organisation around it the car, the organisation around the organisation the road etc.

At XP-Day London, Mary Poppendiecks lecture on Lean Development reminded me of some of the interesting things in Lean Manufacturing and the theory of constraints.

Value Stream Mapping

At least there are tools to identify why your xp-team has diffculty interoperating with other teams within the organisation, and maybe with your customer as well, especially Value Stream Mapping seems interesting. This is where an organisation or team identifies which activities, at what point in time produce value, and other times when there is just waiting for e.g. supplies from outside. An example of elements in a value stream are a finished piece of software waiting for acceptance (not providing, value, just waiting) or customer and programmers discussing requirements (providing value, because the vision for the system is growing). The idea is to look end-to-end end shorten the time it takes a requirement to be turned into deployed software, so that an organisation can respond faster to changing demands.

Kaizen Events

There are also things from Lean Manufacturing with which you can get a whole organisation (the car) to work on their process collaboratively. Kaizen Events look particularly interesting. This is where you take a day with a group, identify things that can improve, and actions to take to do the improvement. I guess the Retrospectives we run are similar, but Kaizen events seem to work with very large groups as well.

Jim Highsmith also hinted at this in his workshop at AYE on team decision making. He gave the example of toyota, where, when developing a car, several car bodies, transmissions engines etcetera are developed. Measurements etc. are taken and discussed with tool-makers, assembly workers, so that when it is time to produce the actual car, there is always a choice of components. When there is a problem with a certain component, it can easily be exchanged.

Interestingly enough, while checking out Dave Thomas’s blog today, I found a related note, "mechanism versus policy" . One size does not fit all, he says when discussing user interface implementations. For expert users, NakedObjects (where users can manipulate objects in the application domain more or less directly through a very thin GUI) might be an excellent solution, since they are free to manipulate everything, while for novices or casual users, a wizard-driven interface is more suitable. Why then, not wrap your NakedObjects in a wizard, and offer the choice which to use to the user?

I have the same feeling about editing web-sites through browsers. As a power-user, I prefer to work with files and be able to do batch modifications with scripts and tools like secure shell. For casual users, the previous sentence might contain 4 strange words (‘batch’, ‘scripts’, ‘tools’, ‘secure shell’), so they’d think, ‘what the …’ is he talking about? The other way around, I feel constrained when having to upload 100 jpeg files one-by-one through an upload dialog - an experience I had when trying to get some digital photos printed recently.

I’ve been walking around with this idea for some time, at least with the frustration of working on one solution (e.g. in this rant), but expressing it to others is much easier now I have a term (Set Based Development) to describe the alternative.

References:

Toyota’s principles of set-based concurrent engineering (text only link )from Sloan Management Review, Winter, 1999, by Durward K. Sobek, II, Allen C. Ward, Jeffrey K. Liker