Monthly Archives: December 2007

Continuous Integration just got interesting

I used to describe CI as “cron with style”. I was trying to emphasise the value of simplicity in your CI implementation (no fancy stuff, just something that works) and pushing logic into the build, where you can hopefully handle it better. Time to kill that name anyway. Sorry Buzz.

For a long time, CruiseControl has been the daddy of CI systems. Competitors have come and gone in the past few years that I have been doing CI, but the biggest bump on the CI landscape has been Cruise. Why?

  • It’s established.
  • It supports the most SCM systems.
  • It supports the most build systems.
  • The code has had a workout through some tough projects.

Now there’s some other shapes on the landscape: Team City, and Hudson, and AntHill Pro.

Team City has features that you shouldn’t need, if you’re doing the right thing. But if you’ve gone astray, they might help you a lot. For example, you can stop a running build and change the execution order of queued builds. You can also do distributed builds. The feature that the developers will like is the delayed commit. With the tight IDE integration you can let TeamCity run the build and checkin only if the build passes. You can also do a test run on the server (called a Private Build) for your own perusal and find out if the build will pass. Very thoughtful features.

Hudson is a new tool, but it could become a very compelling tool at the entry level of Continuous Integration. It’s very simple and is designed to have a great out of the box experience. That wouldn’t be so newsworthy if it didn’t also have distributed build features and a simple point and click configuration interface.

No discussion of the CI landscape would be complete without discussing the very established product AntHill Pro. The guys at urbancode have also moved towards added convenience for the team by making it easy to use any kind of build tool, and adding a lot of useful features. The one I wanted to mention here is promoting builds to different environments from the CI server. Again, thoughtful.

The CruiseControl Enterprise team aren’t resting on their laurels, either. I’ll update this if I can get a comment.

Irradiating the whole team

You want to check in? Is the CruiseControl build green? What you do mean, you don’t know? Everybody in the project room (and ideally, those people the project room) should know what is going on with the build. Broken Continuous Integration builds mean that the project can’t release working software.

So to keep the team in the loop I set up a Extreme Feedback device recently. On our present engagement the developers have been using CCtray with some custom JSP’s to hook into CruiseControl (CCtray is now supported in CruiseControl 2.7.1) which worked well for keeping developers and technical people informed. Well, if they are running Windows.

What we had failed to do was involve everyone – BA’s QA’s, Project Managers and Business people. They all have a very real stake in the delivery of the project; and as the “heartbeat” of the peoject is the activity around the Continuous Integration system, I thought it was time to share. I wrote a Ruby application that would report on the state of the build in simple, traffic-light terms, ran on it on disused workstation with a browser. That was a few months ago.

Since then I managed to convince our Programme Manager to fund a 32-inch display to run the application on. By the time the display had been sourced and installed on the wall, the scope had crept for the application; now it is a Ruby on Rails app that displays build status on the left hand side of the screen, and cycles through several screens to display project information: Message of the Day, bug counts, burnup chart, etc.

My favourite part is the build duration chart. I used a CruiseControl publisher to invoke wget, which will perform an HTTP post to a specific URL. This pushes the timing information of each completed build to a Rails application. Then it’s easy to draw a chart of the cumulative timings with Gruff.

Making information visible to an entire room is natural behaviour for systems administrators and NORAD staff. We’re just borrowing a great idea.

Continuous Integration anti-patterns

Author, CI Expert and great guy Paul Duval has written up some Continuous Integration Anti-Patterns at IBM developerWorks. Link (via the Disco Blog).

On confusing Build and Deploy

We’ve been having a conversation at work about the different goals of build and deploy. I thought I’d share some of my response to a question “does it matter if you mix up build and deploy? Why don’t we use build tools like Ant to deploy with?”:

To me, build is the place where you compile and test your code, package code and other artifacts up for later deployment. “build” is a broad and overloaded term; we spend much more time testing than we do building. I’ve also seen people use the term build as a superset of everything build and deploy.

So “deploy” is taking the artifacts that we made and then actually making them run on a production server(or similar).

I guess because deploy is often a subset of build (you need to deploy an application to a container to test it for example), it’s easy to think of it being the same. But deployment should be incredibly simple, as production deployments can be tense affairs. It’s my belief that you should be rehearsing your production deployments hundreds or thousands of times by making the CI server exercise the process. Once you have:

  • a simple repeatable deployment process
  • tests to make the CI build fail if dependencies aren’t met (e.g. if the deployment scripts aren’t valid),
  • and it’s run a lot,

then you remove a lot of risk and tension at deployment time.

So I guess my answer is that there shouldn’t be much need for a complicated deployment process. You might argue that we might still use a build tool on the production system, and only use some small percentage of the features. But you haven’t met the systems administrators yet.

The systems administrators have (or should have) incentives to keep systems secure and stable. They won’t like having build tools on the box because (especially if they are Unix sysadmins):

  • they probably have other tools (like, most of the Unix userland)
  • it’s long been considered a security hole to leave that stuff lying around production
  • it might be perceived as the thin end of the wedge with respect to circumventing the release process (“No problem. We’ll rebuild the code with that fix on the master server”)

The key is to find a happy medium between making your code easily deployable without compromising the release process. For all stakeholders. I appreciate that some of this may not hold true for .NET projects. Sorry.