Monthly Archives: October 2008

Eras of Continuous Integration

November 3, 2008: updated to reflect some very good comments from some fine commentators.

Continuous Integration is a practice. You can do CI without any automation at all – the original Fowler paper says as much. James Shore wrote a good guide on this.

This post is about the eras automated Continuous Integration. The title wouldn’t seem so snappy if I put all that in there.

The First Era of Continuous Integration (1999 – 2006) is of course characterised by the dominant CruiseControl product and it’s variants. Although CruiseControl wasn’t the first automated CI server (that honour seems to go to Tinderbox), it became very popular. Many projects were won for ThoughtWorks as companies would attempt to install CruiseControl, fail, and then get a friendly ThoughtWorker in to do the install. Who would get two friends in. And so on. Also notable in this era were, DamageControl, Gump, Continuum, and a few others.

I like to think of this era as the equivalent of the original .com boom. Everybody knew it was going to be interesting. Exactly how things were going to work out was another thing. Hence it made sense to have a framework for CI, rather than a fixed product.

Also, XML and XSL were briefly cool during this time, and these technologies left their mark.

The present era belongs to no single company or group. For me it began the day Team City was introduced. That product changed the game by bringing a very functional user interface and some clever build engineering and integration together. Hudson deserves a smaller mention for things like the weather metaphor. I really hate the butler; perhaps someone will come up with a bartender patch.

Ultimately I think this era will give us more usable tools and a shift towards systems integration and deployment: Ant Hill Pro has had a story about that for a while, and the Cruise team seem to be sashayingin that direction as well.

One thing that begins to matter in an enterprise context is authorisation. Can anybody drop in and change configuration? Once you do that, how do you maintain the list of users to map rights to? Active Directory/LDAP integration seems to be the trend here. Step forward Team City, Hudson, Parabuild, and Pulse and others.

(image taken from debaird’s photostream)


Six Tips for Automated Releases

Eric Lefevre: yes

(image taken from the JetBrains Team City Photostream)

Today’s guest post is from Paulo Schneider at YouDevise. YouDevise is a financial markets information company based in the City of London, and they are hiring! You can also check out their developer blog.

A common goal of agile methodologies is releasing often. With shorter
feedback loops you can be more in tune with your users’ needs, and adapt
the development tasks accordingly. However the release process can be a
hurdle if a lot of manual work is required. Here are some tips for
automating your release process:

1- Releasing to test environments should be the same as releasing to
production: during the development cycle we release quite often to our
internal test environments. Thus, it is quite sensible that the release
process should be as straightforward as releasing to production and it
keeps us from having to deal with special cases.

2- Deploy successful builds from continuous integration: it is easy to
automate our release process to grab and build the code from source
control. However, how do we guarantee the commited code is not broken?
If we use a continuous integration tool (such as CruiseControl or
Hudson), it should provide us with completed, deployable build artifacts.
Why not use these instead of source control to get the code for deployment?
At least we know for sure that they have passed the unit and functional tests.

3- Keep per server configuration separate from application: application
configuration is hard. To avoid complicating the automated release process,
separate the configuration of the application from the application
itself. This way we can release the same application to multiple
servers and leave the hard work of configuration to the humans. For instance,
if your application reads a Java properties file on startup, put that file
in a known location on each server, say $MYAPP_CONFIG/config.props, rather than
deploying it as part of a release.

4- Use shell scripting instead of more general scripting languages: the
process of deployment involves a lot of low-level commands and monitoring
system processes, which are suited for shell scripting. If we used a more
general scripting language such as Perl and Python, we would be forever
calling “exec” or “system” and parsing result codes.

5- Use rsync for file transfers: nothing brings down the excitement of a
release than a slow file transfer to production machines. But in this modern
day and age there is a magic pill called rsync
to bring the excitement back. Rsync provides fast file transfers by only transmitting
the bits of the new application that differ from the old version on the
production machine.

6- Use the SSH “authentication agent” to forward credentials: running commands
remotely is a very common task for releasing, and if we run a server farm, we’ll
need to access several machines during the process. For security, we should use
SSH to connect to the servers – but it would hardly be an automated
process if we have to enter a password for connection to every server. Using the SSH
agent-forwarding option, you can forward the security credentials and sidestep
the whole login process for all but the first login. Just type “ssh -A myserver”
and you’ll be on your way.

The dust settles

It’s been a busy couple of weeks with CITCON, my Scrum Master training, moving the blog and trying to have a life.

This blog is now migrated away from Blogger. It now runs WordPress hosted on a Debian Linux VPS, humming away in a datacentre somewhere. I have all the articles moved over. The Feedburner RSS feed now serves out the new versions of the pages. Some of the internal links need updating. Where I find 404 errors in the Apache logs, I am making redirects so the right content can be found. This will take some time to fix. Normal service should resume on Tuesday with a guest post.