So far I find the chaos domain the most puzzling. I'm not sure thinking about software is appropriate in that domain. In chaos brainstorming and deciding on a choreography to one of the other spaces (preferably complex) might be more appropriate. I can see how XP could fit (and misfit) in known, knowable and complex - although in each context it has a different effect, and you can use it for a different purpose. I hope to work on the mindmap a bit more and then write a longer post about it.

For discussion and event information about cynefin and agile, there is now the cynefin-agile mailinglist.

We came to the conclusion, that the number of decisions involved in creating any computer program is too large to record meaningfully. Another conclusion we arrived at, was that many decisions are made at lightning speed inside our head, and some decisions we are unable to articulate. I think that is one of the underlying reasons attempts at documenting software in detail often fail (besides the fact that very few software people enjoy writing prose).

One thing I usually do when working with a team, is to regularly (every hour at least) check in code to the version control system, and add a one-line comment when checking in. These comments reflect the closest practical thing I found to keep a trace of design decisions. I noticed after some time, that we rarely look back on the comments - I think it is because team members find it more effective to ask each other 'why is this part built the way it is', re-create the assumptions underlying the design by asking 'why', and either working from those assumptions, or re-defining the assumptions and start refactoring.

The best way I found to get design decisions out in the open is to do Pair Programming, so two programmers are forced to articulate a larger part of their design decisions. Even then, it is recommended practice to code out versions of an idea as a Spike Solution, if a discussion takes to long, and continue the discussion after two versions of the idea are complete: sometimes an idea is more easily expressed in source code than it is in natural language...

I'm asking myself: Why would I want to record design decisions?. I haven't felt the desire to do that since leaving university. If the design is simple, and we keep requirements as automated acceptance tests, we are free to change the design whenever we want to. The acceptance tests provide the context we need to start finding appliccable (design) patterns. Perhaps the desire to record design decisions is an indicator we are building overly complicated software?

Rather than making an effort to record more design decisions, we can get leverage from simplifying our design and doing more automated acceptance testing.

It seems, that as more and more blogs appear, it becomes harder to follow blogs, or even distill trends from blogs, since, as Ton Zijlstra writes, a lot of the information in blogs depends on their context. I personally haven't made much time for reading all blogs I subscribed to recently, since I can't make space for all the ideas generated by them. Don't get me wrong, I'm grateful for all the excellent blogs around, but I can easily 'lose' a day by surfing the blogs.

Marc Evers and I discussed this idea with several people during the xp days and afterwards (Marc also blogged about this ). The idea is, that we'd like more time to explore topics that come up during xp days and regular conferences in depth, and have a place where people from around Europe (and possibly elsewhere) can experiment with new sessions and show them to various xp-days organisers. Most XP days (even though they're completely independentently organised) now have a session submission and review process. This works pretty well, although something kept nagging me. Duncan Pierce spelled it out clearly to me after XP-days London:

We're reviewing session descriptions, instead of sessions.

Even though we're trying to be independent and objective, it is much easier to trust someone you know with organising a session than someone you don't know. I talked to some peope who thought about organising a session for this year, but didn't get around to it. The threshold to start is quite high if you've never done it and are unsure about your facilitation skills. I'm starting to get more comfortable with it myself, based on the good feedback I've gotten over the workshops I've been doing. Even with experience though, getting a session accepted at a conference remains a somewhat mysterious process.

At XP Day Benelux we nevertheless had many new presenters this year. I hope next year we'll have more new presenters, helped by the Agile Open - we're thinking of an open space conference, where the program is decided on by the particpants at the start of the conference. We are looking to grow an overview of supply and demand with respect to workshops on Agile. With a stimulus to present session ideas before Agile Open, as well as questions for sessions from people with only a question or problem and no idea on how to organise a session around it.

One thing we're already looking for from the demand side is more technical and entry-level sessions for XP Day Benelux.

Another reason to organise Agile Open is a recurring theme in many ongoing discussions: how to sustain a viable agile consultancy business, software house or corporate department. These questions are currently mostly debated at the bar or local user groups (and Mary Poppendiecks' XP2004 keynote). I believe it is worthwile to go in-depth on this theme with a larger crowd.

We now seem to have a critical mass of people who want to co-organise, as well as participate so we're going ahead with it. I'm going to send out a mail with a cost-estimate and details so far tonight to those who reacted until now. If you'd like to participate or co-organise or know someone else who might be interested, please contact me or Marc directly. We're looking forward to make this event a reality and expand the ideas we're having so far!

I just tried a 5-minute freewriting exercise on a question Marc Evers asked me today: What does product management mean? The exercise didn't come out with a specific answer, it did turn up some hints about what I think it should be: about a long-term vision translated in a concrete fashion to day-to-day actions, where software is developed in small increments without taking a mortgage on the future. I also started writing about some situations where I succeeded or failed to collectively do effective product-management with clients and developers on one hand, and consulting clients of mine whom I am helping to deal more effectively with outsourcing on the other hand.

As I was writing this, I was thinking I haven't done much freewriting before. In fact I have, but I wasn't aware of it. I have three or four notebooks filled with notes and diagrams, written at moments when I was contemplating my work. Amongst others, it is promoted in Becoming a technical leader by Gerald Weinberg, to set a few minutes apart every day to write about what you experienced that day. I did that for a few weeks in a notebook, but found it difficult to keep it up. On the other hand, I am writing about my experiences every day in e-mail messages to my friends and sometimes a blog-entry such as this one.

One of the reasons it may be hard to write regularly, is that writing is one of those things that is always important, never urgent. Freewriting may help to get started, as it takes only five minutes to begin.

Getting good at anything requires lots and lots of practice, and sometimes letting go of the critic inside you, so you can enjoy small successes and failures.

Even though JRuby (at the time of writing) has not had a release since October 2002, it wasn’t so hard to get it to work. I first start with running jruby from the command-line with:

 java -jar ~/install/jruby/jruby-0.5.3/lib/jruby.jar

after that, you can add your own file, I tried a ‘hello world’ file hello.rb, containing just ‘puts hello jruby world’ and it worked straight away. Running the tag-cleaner proved a bit more difficult. First, it complained it was missing files. The documentation of JRuby doesn’t state much about the use of paths, but apparently, it supports command-line options ruby supports as well. I found the -I option, for including library directories worked.

So now my command-line (with jruby invocation in a home-made shell script) looks like this:

 jruby -I/usr/lib/ruby/site_ruby/1.6 -I/usr/lib/ruby/1.6 tag-cleaner.rb aDocument.html

Now it complains it hasn’t opened the file. So first, I try the unit-tests in tagcleanertest.rb to see if the SGML parser will run at all, and it does. The unit tests, however, do not test opening a file (they work by feeding the tag-cleaner with html in strings). Apparently, the function IO.readlines doesn’t succeed in opening the file (the message is a bit cryptic). After rewriting the code to ‘File.new(inputFile).gets ’ it appears jruby can’t find the file. If I append the full path of ‘aDocumen.html’ it works :-) With bash, we can replace the current path with $PWD like so:

 java -jar ~/install/jruby/jruby-0.5.3/lib/jruby.jar -I/usr/lib/ruby/1.6
  -I/usr/lib/ruby/site_ruby/1.6/ tag-cleaner.rb $PWD/aDocument.html

unfortunately, jruby does not support the -C command line option (to change to a directory) like ruby does. IO.readlines (retrying after I found the path problem) doesn’t seem to work either. File.gets does. So, after modifying two lines, the tag-cleaner now also works in Java.

only now I find out, the JRuby distribution includes a ready-made shell-script in the bin directory… (DOH!) that script sets the path correctly, so the $PWD hack is obsolete. The -I’s are still necessary.

Integrating TagCleaner with Java.

In order to integrate TagCleaner in Itor, it seems best to rewrite the unit-tests in tagcleanertest.rb in Java, and then integrate the implementation with JRuby. That way, we can modify the tests to suit integration into Itor if necessary, and do the integration in small, repeatable steps. JRuby works according to something called the BSF, which is short for the BeanScriptingFramework, a Jakarta project. The BSF is a standard way to call other languages from within Java.

Before I start with the unit tests, I try to get an example provided by JRuby to run from within Eclipse. This example is, how appropriately, called BSFExample. This example requires jruby.jar and bsf.jar (from JRuby’s lib directory) to be on the classpath. This example works straight away, showing a small dialog, with the dialogs’ components accessible from a ruby command line. I couldn’t resist trying this bit of reflection:

 puts $frame.methods

And it works! showing me all methods of the Java frame, mixed in with the methods of Ruby’s Object. Now, to get my own thing running, I dive into the BSF’s documentation and use the example provided on the JRuby homepage. After having evaluated a simple expression and checking the result, it is time to run the Ruby version of the unit tests, to see if the ruby libraries are included correctly. This turns out a bit difficult, since there is no method on the BSFManager to set the library path. Eventually, it works by accessing the ruby library path from the ruby side, by adding elements to the variable $:, which turns out to be empty by default, so JRuby only looks in the current working directory.

This line of ruby does the trick:

 $: << '/usr/lib/ruby/1.6' << '/usr/lib/ruby/site_ruby/1.6'

In order to have my ruby files in the org.cq2.spike package, I need to add that as well:

 $: << '/usr/lib/ruby/1.6' << '/usr/lib/ruby/site_ruby/1.6' << 'src/org/cq2/spike'

the ’<<’ is a shorthand notation for adding elements to an array. For integration into the Itor package it is probably better to include the necessary libraries in the Eclipse project, but at least we know now how to tell JRuby where to look for files.

I don’t like the ‘eval’ and ‘exec’ functions of BSF very much. Why would I want to specify a line number every time? I probably missed some convenience functions, but with Eclipse it’s easy to add my own.

Here’s the full sourcecode of running the tagcleanertest. I tricked the BSF into loading (and executing) the tagcleaner script, by asking it to ‘require’ (include) the script. Probably missed something there as well, but I couldn’t find it in the documentation.

The printouts show how to retrieve a String back from JRuby and print it. Strangely, $: is not returned as an array.

        public static void main(String[] args) throws BSFException {
                BSFManager.registerScriptingEngine(
                        "ruby",
                        "org.jruby.javasupport.bsf.JRubyEngine",
                        new String[] { "rb" });

                BSFManager manager = new BSFManager();

                rubyEval(manager, $: << '/usr/lib/ruby/1.6' << '/usr/lib/ruby/site_ruby/1.6'
                  << 'src/org/cq2/spike');
                Object obj = rubyEval(manager, "$:");
                System.out.println(obj.toString());
                System.out.println(obj.getClass());

                String expression = "require 'tagcleanertest.rb'";
                exec(manager, expression);
        }

        private static void exec(BSFManager manager, String expression) throws BSFException {
                manager.exec("ruby",
                "(java)",
                1,
                1,expression);
        }

        private static Object rubyEval(BSFManager manager, String rubyExpression)
          throws BSFException {
                return manager.eval(
                        "ruby",
                        "(java)",
                        1,
                        1,
                        rubyExpression);
        }

After running this, the output shows the library path, its’ class and four running tests:

        /usr/lib/ruby/1.6/usr/lib/ruby/site_ruby/1.6src/org/cq2/spike
        class java.lang.String

        CleanFontTagsTest#testNoStripping .
        CleanFontTagsTest#testSpecialCharacters .
        CleanFontTagsTest#testStripTagsOnly .
        CleanFontTagsTest#testTwoAttributesPreserved .
        Time: 0.803
        OK (4/4 tests  4 asserts)

So, I’m fairly happy with this result. Now I can continue and integrate the main tagcleaner with my Java code. The tests in JRuby do run almost 80 times slower than the tests in ‘native’ ruby, that one only takes 0.012543 seconds. But at least I get to recycle my code instead of rewriting it and profit from within Java from well-written parser libraries etc. in Ruby.

After some nice entries on management, hiring, firing, nice photographs etc., it is time to get down to technichal detail again (yes, I have been programming today as well, maybe more about that later). So, if you don't use linux or are not in to digital photography with large RAW images, this post is not for you... See you next time ;-)

From my trip to Arizona I returned with a Canon Digital Rebel digital camera, which features exchangable lenses, SLR (the viewfinder looks through the lens with a mirror) and 6.3 mega pixels :-), and everything can be automatic or adjusted by hand. One thing I like about it especially, is that it is a photo camera: no diary, not games, no "boot up splash image", no movies. A very nice camera, even though I had to return it after two days because of a mechanical failure.

Since the images are large, I have a 512 MB compact flash card with it - I prefer to buy equipment that uses simple, cheap, industry standard components (as opposed to SD cards, Sony specific memory etc.). The camera is apparently supported under linux by gpoto2, but not by the version I have installed currently (SuSe 8.2) not wishing to install extra software (which would prove to be slow, given that the camera and my laptop support usb 1.0 only) I decided to get a compact flash reader for my laptop - which proved to be industry-standard cheap: 7 Euro and 50 cents at my local computer shop.

If you read the manual (pcmcia howto) carefully, it is easy. Of course I started reading after encountering problems. lsmod found pcmcia-core and ide-cs installed, which means pcmcia is installed, and there is an ide-hard drive attached to it. The compact flash card acts as a harddisk, which is nice. /var/log/messages.

cat /var/lib/pcmcia/stab
Socket 0: ATA/IDE Fixed Disk 
0       ide     ide-cs  0       hde     33      0

tells me the flash card is assigned to /dev/hde. Now I could mount it, if I had noted this line in the pcmcia howto:
A common error when using IDE drives is try to mount the wrong device file. Generally, you want to mount a partition of the device, not the entire device (i.e., /dev/hde1, not /dev/hde
instead, when doing

mount -t vfat /dev/hde  pcflash

mount -t vfat /dev/hde1 pcflash

eat it RAW

Charles Vermeulen asked me innocently 'can your camera do RAW images', sure it can... RAW images are, unlike JPEG images, compressed without information loss (so-called 'lossless compression'), kind of like ZIP, but with meta-data about exposure, white-balance etc. The digital rebel spits them out, together with a small thumbnail (in JPEG) so you can see what the picture is about in your browser. Since there is no standard for RAW images, each manufacturer makes them differently.

Luckily, there is dcraw by Dave Coffin, which is reportedly faster and more accurate than the software provided by Canon (I am going to check that out, but the windows software for my Digital Ixus / Elph was so buggy it was hardly usable, so I find it believable for now). Dave's code is also used in commercial packages such as Photoshop.

Compiling dcraw is easy, just follow the instruction on the homepage, and one .c file leads to one executable. Dcraw transforms the raw image into PPM, which is universally read under linux.

Having GIMP (the GNU Image Manipulation Program, the open source equivalent of Photoshop) read RAW files was a bit more work. Again, basic instructions on how to use gimptool to create a plugin are on Dave's page. But, you must have installed the gimp development toolkit. SuSe provides packages for that through YAST, and after that, it is simple, although you have to run gimptool in a user's directory where you have used the GIMP before, otherwise gimptool doesn't know where to put the freshly created plugin.

And then, it just works, you can choose to use 4 colour RGB (whatever that is), use the camera's white balance or not (apparently just metadata) and some other options I do not yet understand.