Registration

FOSDeM has no formal registration and name tags, which appeared to be a good thing, as there were many more visitors this year (around 1400 on the first day alone) than last year (around 300). With 1400 people, registration processing might take forever.

Open Protocols and Data Formats

Due to getting lost earlier, I dropped in late on Tim O Reilly’s talk. "The Internet is open source’s biggest success […] open protocols and data formats are even more important than open source".

I felt right at home, as my colleagues at CQ2 are completing the implementation of the OA-x (Open Archive extended) protocol and data format.

Contemplating on this, I see that open protocols and data formats are more accessible than open source - since you can choose whichever programming language and operating system you want to make use of the protocol and the data (TCP-IP and e-mail being succesful examples of this). Even if one part of the protocol is complex (i.e. The ‘google server’) you can write a simple client (google access with soap) re-using the implementation and installation complexity someone else has resolved for you.

For I-tor this might mean the amount of people working on the i-tor server might remain small - since it contains a lot of advanced features not always needed, while other people might more easily make clients for editing specific kinds of OA-x documents, or make simple servers publishing and storing documents in OA-x format. The i-tor server acts as an intermediary for combining collections of oa-x documents, publishing them through a web-interface and allowing them to be queried interactively by end-users.

Ruby

I went to Richard Kilmer’s talk on Ruby , to see how someone who uses it everyday presents it, and to pick up some stuff I might have missed. New stuff I picked up from this talk:

• YAML (included in ruby 1.8) - a non-XML human readable/editable serialization format for ruby and perl. The library uses ruby’s class extension facilities effectively - after loading YAML all your objects have a ‘to_yaml’ method. YAML is made for readability and ease of use. Existing serialisation methods are more efficient.
• ROMP - "(the Ruby Object Message Proxy) is a set of classes for providing distributed object support to a Ruby program."

ROMP does the same as DRB (Distributed Ruby), but is much more efficient according to Rich Kilmer. (I couldn’t get more than 50 messages a second out of DRB, but apparently, there now is a faster version. ROMP handles up to 40000 messages per second). It is also very easy to use - with a few lines of ruby, you can make an object available on a tcp port. Using ROMP seams a decent way of splitting presentation from storage of objects. The server stores objects and deals with concurrency, and several presentation server applications and threads can access the objects.

Ruby-GNOME

This talk was on the integration of Ruby with GTK libraries. The design of GNOME is interesting, since it is highly plugin-oriented, with a pipes-and-filters like architecture for multimedia applications. The ruby-bindings convinced me less, since creating a GUI is doable, but requires double steps - the GUI ends up in a separate configuration file from the code, and the names of events have to be specified twice. For small applications, such as the media player that was demonstrated, this is doable. For larger applications I foresee it will be difficult to keep an overview on everything defined.

ReiserFS - the fast pluggable filesystem

Hans Reisers’ talk on ReiserFS once again proves that good design and high performance software are not opposites, but go hand in hand. ReiserFS version 4, to be released this week, features plugins for various aspects of the filesystem, from allocating blocks, to storing individual files. The reason for the plugin architecture was, as mr. Reiser said, that optimizing a filesystem is an empirical science. You need to try something at least three times, fail two times and then find a better solution. The plugins allow many people to try out their hypotheses. The ReiserFS distribution can then include the most succesful plugins.

One of the plugins planned for the near future is compressed files (like e.g. Atacker in the old days, only more reliable), since disk speed has not kept up with cpu speed increases, this will allow for faster disk (read) access. To me it seems a good solution for storing XML files… Another thing which is easy to do with plugins is encryption.

The vision behind ReiserFS is also interesting - the file system should take care as much as possible for things like atomicity, so applications built on top of it can be as simple as possible. ReiserFS is intended to replace databases as a means of storing many small objects - it does not allocate a lot of space for individual files anymore, so it is possible to efficiently store each object in its own file, even if it is just a few fields of data. ReiserFS takes care of efficiently storing all the objects. The explanation of ‘dancing trees’ (a variation on B-Trees) was interesting. A file system is basically a huge map, with file-id’s as keys, and file data as values.

Concerns influencing the design of ReiserFS, such as plugins and granularity of security are familiar to me, since we face them in designing I-Tor as well, so maybe a close mapping on ReiserFS could enhance performance (small objects optimized by the filesystem) and facilitate development (atomicity, security).

Frequent releases

Agile Software development things like frequent releases come through. I was talking to the GNOME packager for MandrakeSoft at dinner, and he said they recently switched to weakly developer releases, regardless of content (just release frequently), and every 6 months there is an end-user release, where stable features are chosen and released. API’s can change every 6 months, old methods are deprecated and kept for one more release, ensuring a stable application development environment, as well as evolution of the API’s.

Summary

FOSDEM is an interesting conference for free and open source software developers. You can quickly get an idea what is going on in major open source projects. Also, many groups take the opportunity to actually develop software together in the developers’ rooms for specific projects.

Lotus Notes for instance, was released five years after development, and it took another six years before the user base started to really grow. As an example from my own experience, we're using Linux on our laptops and servers, an open source variant of Unix. Unix exists since at least the 70's. In the first twenty years of its' existence, high-priced unix workstations and servers were mainly to be found in universities and corporate research and development environments. Now, after over thirty years, through Linux (and to a lesser extent, FreeBSD and its' derivative, MacOS-X) it is spreading to a much larger user-base.

Starting last December, I am programming on the e-laborateproject. Not exactly a long-running project. It is, however, based on i-tor, open-source software that has developed through several projects starting January 2002. So, I-tor is now in its third year. Having been involved as a coach and programmer, I can recognize several of the pitfalls Joel Spolsky managed in the project. Having been there, I can say, that it is possible to survive those pitfalls, even if you've stepped right in them.

Yesterday, someone described us at CQ2 as "Software Efficiency Experts" . Probably, because we try to eliminate waste in software development, and we continually search for, and find practices that work well for us and our clients.

But what about Effectiveness? It is nice to reduce costs, but if you develop the wrong things, you are getting nowhere. You are getting there more cheaply than otherwise, maybe faster as well, it is all well tested, but it still is the wrong software!

The best way, in my opinion, to increase effectiveness is to eliminate waste in requirements:

• Don’t automate things that don’t have to be automated. If you leave it up to some technical people (I know I’ve been one), you would have everything automated. Instead, automate work that costs much more to do by hand than with software, or automate something so you can deliver much more value to your customer than otherwise possible.
• Program your software ‘just in time’. Delay implementation until you really need the software. It is more effective to invest in software you need now, than in software you don’t need right now, and might never need at all.
• Make sure all the features developed so far provide value. Measure usage of features, remove them if they aren’t used anymore. Unused features cost money for time spent maintaining software, and possibly planning things based on these features. Sometimes it takes time to explore software by using it, to find out which features are really essential and which aren’t.

Now that we are developing the right thing, we can focus on developing the thing right. What works well for us in this respect, is Test Driven Development (which makes reliable software cheaper to build and maintain than unreliable software), Refactoring and some other techniques from Extreme Programming in combination with increasing the personal effectiveness of development team members and better and more enjoyable teamwork through e.g. Retrospectives.

Cost-effective is a nice word to describe this, because it focuses on cost (efficiency) as well as on value (effectiveness).

The metaphor Erik and I use for this these days, is that you have an engine which starts running very efficiently and is very powerful, but the car around it hasn’t changed, so the transmission can’t cope…

Then you can decide to optimize the car as well, but you discover there are holes in the road. Then you can start to fix the holes in the road and so on…

The xp-team is the engine, the organisation around it the car, the organisation around the organisation the road etc.

At XP-Day London, Mary Poppendiecks lecture on Lean Development reminded me of some of the interesting things in Lean Manufacturing and the theory of constraints.

Value Stream Mapping

At least there are tools to identify why your xp-team has diffculty interoperating with other teams within the organisation, and maybe with your customer as well, especially Value Stream Mapping seems interesting. This is where an organisation or team identifies which activities, at what point in time produce value, and other times when there is just waiting for e.g. supplies from outside. An example of elements in a value stream are a finished piece of software waiting for acceptance (not providing, value, just waiting) or customer and programmers discussing requirements (providing value, because the vision for the system is growing). The idea is to look end-to-end end shorten the time it takes a requirement to be turned into deployed software, so that an organisation can respond faster to changing demands.

Kaizen Events

There are also things from Lean Manufacturing with which you can get a whole organisation (the car) to work on their process collaboratively. Kaizen Events look particularly interesting. This is where you take a day with a group, identify things that can improve, and actions to take to do the improvement. I guess the Retrospectives we run are similar, but Kaizen events seem to work with very large groups as well.

Jim Highsmith also hinted at this in his workshop at AYE on team decision making. He gave the example of toyota, where, when developing a car, several car bodies, transmissions engines etcetera are developed. Measurements etc. are taken and discussed with tool-makers, assembly workers, so that when it is time to produce the actual car, there is always a choice of components. When there is a problem with a certain component, it can easily be exchanged.

Interestingly enough, while checking out Dave Thomas’s blog today, I found a related note, "mechanism versus policy" . One size does not fit all, he says when discussing user interface implementations. For expert users, NakedObjects (where users can manipulate objects in the application domain more or less directly through a very thin GUI) might be an excellent solution, since they are free to manipulate everything, while for novices or casual users, a wizard-driven interface is more suitable. Why then, not wrap your NakedObjects in a wizard, and offer the choice which to use to the user?

I have the same feeling about editing web-sites through browsers. As a power-user, I prefer to work with files and be able to do batch modifications with scripts and tools like secure shell. For casual users, the previous sentence might contain 4 strange words (‘batch’, ‘scripts’, ‘tools’, ‘secure shell’), so they’d think, ‘what the …’ is he talking about? The other way around, I feel constrained when having to upload 100 jpeg files one-by-one through an upload dialog - an experience I had when trying to get some digital photos printed recently.

I’ve been walking around with this idea for some time, at least with the frustration of working on one solution (e.g. in this rant), but expressing it to others is much easier now I have a term (Set Based Development) to describe the alternative.

References:

Toyota’s principles of set-based concurrent engineering (text only link )from Sloan Management Review, Winter, 1999, by Durward K. Sobek, II, Allen C. Ward, Jeffrey K. Liker

A new camera sure is fun, but it has one major drawback: I make much more pictures than usual :-). The three categories below contain over 550 images in all. Since I promised several people to put them online, there was no way around but to make myself some scripts to scale the images and generate index files. The program ImageMagick makes this relatively painless. If you want my scripts, just drop me a mail.

Just click one of the images below to see all of the images in the category. At AYE I took so many pictures (about 350) that that one is divided into subcategories...

Visiting the AYE conference and the Grand Canyon in Arizona, USA	XP Day Benelux 2003	A photoshoot with Charles Vermeulen around the Grevelingen lake in Zeeland