Rants and Ruminations 47 to 53 of 149 articles InfoSyndicate: full/short
Testdriving a mailserver   26 Apr 05
[print link all ]
To make detection of the various problems I had yesterday easier, and to build for the future (I would like better spam and virus protection on my mailserver) I created automated tests for my mailserver using RubyUnit. I started out by copying the various addressing tests I did with exim -bt, like so:
def testUser
	result = `exim4 -bt user@domain.com`
	assert(result.include?('user@domain.com -> /home/user/Maildir/'))
end

Please note I changed the username actually used to a fake value, to not attract even more spam. The test above verifies the correct routing by checkng the final delivery to the users mail directory. I've got similar tests for wildcards and a mailing list, because these required adjustments to the configuration in order to work. I didn't add tests for spam yet, if anyone has some feel free to contact me :-).

These tests are not fully end to end, as they test only the routing on exim itself. Especially for the mailinglist software, these tests only check that exim delivers mail to the mailinglist, not what the mailinglist does after that. Therefore, I added two other tests, that use the smtp module provided by ruby , one for an ordindary user, and one for a mailinglist.

For the mailing list test, I set up a special test list, with only one subscriber on it. Letting the mailing list test fail for the first time was interesting, as it fell into the mailinglists error handling. I try to specify as little information as possible in the test mail, so the first one was rejected because it contained an implicit destination - in e-mail, the To: field is optional, so I had left it out for the plain e-mail test I adapted the mailinglist test from. So now I am sure asserting that the body of the message is present is the right thing to test for - the error message from mailman does not contain the message body, only some of the headers.

The mailinglist test (names have been changed, to protect the innocent) looks like this:

def testMailinglist
	`rm /home/user/Maildir/new/*`
	require 'net/smtp'
	msg = ["Subject: Test\n", "To: some_list@domain.com", "\n", "Now is the time for a mailinglist\n"]
	Net::SMTP.start('domain.com') do | smtp |
		smtp.sendmail(msg, 'user@domain.com', ['some_list@domain.com'])
	end
	
	sleep 5 #wait for delivery
	
	dir = Dir["/home/user/Maildir/new/*"]
	assert_equal(1, dir.size)
	filename = dir.first
	contents = IO.readlines(filename)
	assert(contents.include?("Now is the time for a mailinglist\n"), contents)
 end

Note the sleep in the middle - the test has to wait a few seconds for smtp, exim and mailman to do their job and deliver the mail to the directory of the test user.

I'm quite happy with the automated tests - the testscript also auto-generates the exim configuration file for me, so it functions as a sort of automated build for the mailserver configuration. There is nothing as reassuring as

7 tests, 9 assertions, 0 failures, 0 errors

If you'd like to have the full source code for the testcases, contact me directly, as I don't feel like search-and-replacing the e-mail addresses etc. in the test. Feedback on the way I set this up is also very welcome.

Mailserver upgrade surprises   26 Apr 05
[print link all ]
So instead of the hour I hoped to spend, I spent most of my afternoon yesterday getting mailinglists back up and running. Or so I believed. As an old detergent add used to say the detergent is right, but the temperature is wrong. Apparantly, I'm better of working testdriven for systems administration tasks as well. I had e-mail up and running , and according to all the logfiles I could find (and there are quite some in exim and mailman combined) mailinglists were working, only I as a subcriber wasn't receiving any mail from them.

After taking a long break (tip: take a break when stuff like this doesn't work. It's usually more effective than labouring on). My mail cilent kmail is picky in what mail it downloads, depending on which button I press... So, for the next round I'll take the stuff I did on the command line and copy it into a test script, so I can see if the mail is still working (ordinary users, wildcards, mailinglists). Then I can use that as a basis to improve spam- and virusfiltering as well (spam is apparently still sent out over the mailinglists).

Configuring a mailserver is bewilderingly complex. With only an afternoon spent to upgrade about every package on my web- and mailserver a few years into the future, I may consider myself lucky - on other platforms than debian linux this would take quite a bit more work. Much of the old configuration was taken into consideration by upgrade scripts.

Nevertheless, I'm surprised that what I consider to be a fairly standard setup of mailserver, anti-spam and virus protection and mailinglists takes so much manual configuration. Most configuration I do is copy-paste from websites and mailinglists.

For my own memory and possibly your configuration pleasure, I include some of the details below:

Stepping through why the mailinglists didn't work yet, I found out that the *: willem alias wasn't working anymore. Exim configuration is much different from version 3 to version 4... Surprisingly, handling e-mail with a 'catch all' address does'nt work out of the box (anymore). See this message in the debian mailinglist how to create a catch all address in exim4.

Mailserver configuration seems so different, it is best to start over with the mailinglists and antispam configuration.

This installation log shows some useful defaults (including some nifty spam-filtering and anti-virus addons I haven't used yet) and commands to check the configuration.

exim4 -bV
checks the configuration file, and shows which one is used. Take note that debian generates the config file from one or more files in /etc/exim4 depending on if you choose multiple small files or one large file.

Upgrading mailing lists   25 Apr 05
[print link all ]
The xp day benelux and agile open mailinglists are temporarily unavailable. I upgraded my webserver from Debian Stable to Debian Testing. Apparently, mailserver configuration (Exim) is completely different. I hope to have the lists back up in a couple of hours.

Agile Alliance election - candidate statement   21 Apr 05
[print link all ]
I worked on my candidate statement for the agile alliance board elections today. Yesterday, at the agile seminar in Nieuwegein, Nynke Fokma, Marc Evers and Peter Schrier spontaneously self-organized into a playful campaign team (we're not going to take this too seriously, mind you). During drinks they asked around what people expect from the agile alliance.

One of the concerns that came up, was with agile becoming mainstream, there will be a surge in 'true believers' who take a bunch of practices from a book (e.g. extreme programming explained or the scrum book) and take it as 'the thing' rather than as a gate to further understanding - turning off sceptics through their zeal.

If you want to read more about this, check out the xp mailinglist archives a few years back, or read about Shu Ha Ri on the c2 wiki or in Alistair Cockburn's book on agile software development. I believe it is good to start out with practices, try them out a hundred percent and then reflect and make them your own.

By drawing in more beginners we, as a community, can support our industry in making a smooth transition towards more effective and enjoyable work. I interviewed a bunch of people recently for the agile2005 experience reports and yesterday at the agile seminar, and one thing that most of them had in common was how much difference they experienced in their working environment after starting with simple practices like stand-up meetings and iteration planning. An older engineering manager told me, that now he saw his engineers have so much fun he would almost consider going back into engineering. Now is a great time to be working in software development.

Ok, so here is my candidate statement, within acceptance tests provided by Diana Larsen : max 150 words, has to contain something about non-profit and community experience and what I would like to do for the agile alliance.

Hi, my name is Willem van den Ende. I'd like to help build agile alliance branding strength by fostering thriving local communities, international conferences, a website and other expressions that model our values.

I work as software development coach, currently based in Eindhoven, the Netherlands. I enjoy growing communities, so I co-founded xpnl, xp day benelux, agile open, agile systems and systemsthinking.net. Growing up in a tiny country that cultivated its golden age by accepting diversity when that was outlawed elsewhere, makes me realize how big this world is - we can prosper only by learning from each other.

As agile becomes mainstream, I believe we need to emphasize values over practices and individual methodologies. I see the agile alliance as rallying point for pragmatists seeking like minds advancing the "State of The Practice" in management and software development. We'll be the change we wish to create.

Agile Alliance board elections   20 Apr 05
[print link all ]
I've been invited to run for Agile Alliance board member elections. For me, this is the first time I participated in this way in an election. So, in the spirit of communication, feedback and respect, I would like to know how I can best serve other Agile Alliance members in this way. if you're a member, and tell me what you would like me to do for you, if I were elected.

I'm articulating the ideas I've already got so far in a 150 word blurb for a mailing to the members. That is due tomorrow, so If you reach me before then, I'll be able to weave it in there as well.

XP Explained second edition summaries   20 Apr 05
[print link all ]
I recently read the second edition of Extreme Programming Explained - Embrace Change by Kent Beck. As the dust settles, the community is embracing the changes in this second edition (or as I say, version two).

For your and my linking pleasure, here are summaries by Michele Marchesi, Bill Wake and Keith Ray.

I used to recommend the first edition as the quickest way to get started with extreme programming. Currently, as 'getting started hands on guide' for beginners I'd recomment the Scrum book (for iterative planning) together with Test Driven Development by Kent Beck for programming.

I'd recommend the second edition for those who are interested in the pragmatics behind extreme programming. It now feels less action oriented, but compensates that with a much better explanation of the reasoning behind the practices.

Worthwile reading if you've been doing (some of) the practices for a while, and start wondering why they work. Over the past years, I've found it worthwile systems thinking about this with some colleagues. XP Explained second edition can help you get there a bit faster.

Commoditization of IT   24 Mar 05
[print link all ]
I just came across this, from the Portland Roundtable on software development, Commodization of IT
One idea that comes to mind borrows from the experience of the computer hardware industry, and relates to aggregation concepts. If one can combine commodity components to create a new product that is both useful and affordable (i.e. high-value) then commoditization becomes an enabling force, an opportunity, if you will, to provide something that was not feasible to provide before. Using this analogy, how might this concept of commodity aggregation apply to IT services?
Last year I was thinking with some colleagues on commoditization from a perspective of fixed price bidding. The roundtable report provides some fresh insight. I am thinking of integrated (yet loosely coupled) products recently, but had not yet linked it to commoditization.

What I get from the roundtable is: If you're competing on price alone, it is because you are not differentiated enough.

Copyright © 2008 Willem van den Ende