Monthly Archives: February 2008

Refactor your configuration file

Part 5 of CruiseControl Best Practices

Tell me that you think duplication in computer systems is a good thing. I dare you. Drop me a comment and we’ll talk about it. I’m someone who spent the best part of ten years doing systems administration and running other people’s code. So you might guess that I’m a card carrying member of the “Duplication is evil” party. If you’re not a member too (I must work on the secret handshake!), this might not be a post for you.

Duplication doesn’t just creep into our business code. It also seeps into the build and CI systems when you’re not looking. There’s probably someone on your team who is copy-pasting something into the build right now. Don’t worry about stopping them. Read on to find out how to put the CruiseControl configuration right.

Here’s an example of duplication. You can see that there’s a load of repetition in the file. I didn’t exactly pull if from a project (I’d love to work on a project called Groucho), but it’s not far away from a real life scenario.

<?xml version="1.0"?>
<cruisecontrol>
 <project name="chico" buildafterfailed="false">
 <listeners>
   <currentbuildstatuslistener file="/var/tmp/cruise/logs/chico/status.txt"/>
 </listeners>
  <bootstrappers>
   <svnbootstrapper localWorkingCopy="/var/tmp/cruise/projects/chico/"/>  </bootstrappers>
 <log dir="/var/tmp/cruise/logs">
 <merge dir="/var/tmp/cruise/projects/chico/build/test/log"/>
 </log>
  <schedule interval="60">
   <ant antWorkingDir="/var/tmp/cruise/projects/chico"           antscript="/var/tmp/cruise/ant/bin/ant"
uselogger="true"/>
  </schedule>
 </project>

  <project name="groucho" buildafterfailed="false">  <listeners>
  <currentbuildstatuslistener file="/var/tmp/cruise/logs/groucho/status.txt"/>
 </listeners>
  <bootstrappers>
<svnbootstrapper localWorkingCopy="/var/tmp/cruise/projects/groucho/"/>  </bootstrappers>
 <log dir="/var/tmp/cruise/logs">
  <merge dir="/var/tmp/cruise/projects/groucho/build/test/log"/>
  </log>
  <schedule interval="60">
  <ant antWorkingDir="/var/tmp/cruise/projects/groucho"           antscript="/var/tmp/cruise/ant/bin/ant"
 uselogger="true"/>
   </schedule>
</project>
</cruisecontrol>

Fortunately, there is a way to keep this in check in CruiseControl.

CruiseControl has had the feature of properties in the config.xml file since 2005. They are modeled on Apache Ant style properties, with a dollar sign and curly braces: ${cruise.home}. These are immediately useful in replacing constant style values in your config.xml file, like the location of your Ant script or the log directory.

This is a huge boost, but it’s not enough. What about the things that are defined in the config file itself, like the name of the project? The author was thinking ahead on this one: there is a magic property called ${project.name} for you to use. It’s scoped to the enclosing project so that you can’t refer to chico’s project harpo, or vice versa.

Take a look at the config file now that properties have been introduced:

<?xml version="1.0"?>
<cruisecontrol>
 <property name="cruise.dir" value="/var/tmp/cruise" />
<property name="log.dir" value="${cruise.dir}/logs/" />
<property name="projects.dir" value="${cruise.dir}/projects" />
<property name="ant.script" value="${cruise.dir}/ant/bin/ant" />
<project name="chico" buildafterfailed="false">
 <listeners>
  <currentbuildstatuslistener file="${log.dir}/${project.name}/status.txt"/>  </listeners>
<bootstrappers>
<svnbootstrapper localWorkingCopy="${projects.dir}/${project.name}/"/>  </bootstrappers>
 <log dir="${log.dir}">
  <merge dir="${projects.dir}/${project.name}/build/test/log"/>  </log>  <schedule interval="60">
       <ant
antWorkingDir="${projects.dir}/${project.name}"
     antscript="${ant.script}"
   uselogger="true"/>
   </schedule>
 </project>
 <project name="groucho" buildafterfailed="false">
 <listeners>
  <currentbuildstatuslistener file="${log.dir}/${project.name}/status.txt"/>  </listeners>
 <bootstrappers>
  <svnbootstrapper localWorkingCopy="${projects.dir}/${project.name}/"/>  </bootstrappers>
 <log dir="${log.dir}">
  <merge dir="${projects.dir}/${project.name}/build/test/log"/>
 </log>
  <schedule interval="60">
      <ant
antWorkingDir="${projects.dir}/${project.name}"           antscript="${ant.script}"
    uselogger="true"/>
  </schedule>
 </project>
</cruisecontrol>

Things look much better. If you wanted to change the name of the project, or one of the paths that it references, you could change it once. But there’s still some patterns that repeat themselves again and again – we can probably do better.

There’s a bit more magic to the properties. The ${project.name} property dynamically changes depending on the enclosing project. Not only can you use it in the middle of configuration values, but you can declare it as a part of the value of another property. What this means for your long-suffering configuration file is that you can declare one property for the location of the project, and have that property lazily evaluate to to fit the project. This might confuse your colleagues when they look at the configuration file for the the first time – “How does the same property work in so many different cases?”.

<?xml version="1.0"?><cruisecontrol>
<property name="cruise.dir" value="/var/tmp/cruise" />
 <property name="log.dir" value="${cruise.dir}/logs/" />
<property name="project.dir" value="${cruise.dir}/projects/${project.name}" />
<property name="ant.script" value="${cruise.dir}/ant/bin/ant" />
<property name="status.file" value="${log.dir}/${project.name}/status.txt" />  <project name="chico" buildafterfailed="false">
  <listeners>
  <currentbuildstatuslistener file="${status.file}"/>
  </listeners>
<bootstrappers>
 <svnbootstrapper localWorkingCopy="${project.dir}"/>
</bootstrappers>
<log dir="${log.dir}">
<merge dir="${project.dir}/build/test/log"/>
  </log>  <schedule interval="60">
       <ant antWorkingDir="${project.dir}"
 antscript="${ant.script}"
uselogger="true"/>
</schedule>
</project>
<project name="groucho" buildafterfailed="false">
<listeners>
<currentbuildstatuslistener file="${status.file}"/>
 </listeners>
 <bootstrappers>
 <svnbootstrapper localWorkingCopy="${project.dir}"/>
  </bootstrappers>
  <log dir="${log.dir}">
<merge dir="${project.dir}/build/test/log"/>
  </log>  <schedule interval="60">
   <ant antWorkingDir="${project.dir}"
antscript="${ant.script}"
uselogger="true"/>
</schedule>
</project>
</cruisecontrol>

One thing that happens on bigger projects is that the configuration file becomes huge. While you can do things like use XSLT to generate a configuration file (I’ve made myself some headaches that way), you can just break your configuration file into smaller chunks. There’s a great feature called <include.projects> in the CruiseControl configuration file – it lets you refer to another configuration file. This works just like the <import> feature in Ant. You end up maintaining one config.xml file that contains variable definitions (they work over the includes as well), and a host of smaller configuration files that contain one project’s definition. On my current project this features has made it a lot simpler to add or remove projects. Even if someone forgets to delete the include.projects element from the config.xml when they delete the file, it doesn’t matter – that project won’t be imported as the file doesn’t exist. Tracking the changes to the CruiseControl configuration becomes a lot simpler as well.

Here’s the config.xml now:

<?xml version="1.0"?>
<cruisecontrol> <
property name="cruise.dir" value="/var/tmp/cruise" />
<property name="log.dir" value="${cruise.dir}/logs/" />
<property name="project.dir" value="${cruise.dir}/projects/${project.name}" /> <property name="ant.script" value="${cruise.dir}/ant/bin/ant" />
 <property name="status.file" value="${log.dir}/${project.name}/status.txt" />  <include.projects file="projects/chico.xml" />
<include.projects file="projects/groucho.xml" />
 </cruisecontrol>

Here’s one of the projects:

<?xml version="1.0"?><cruisecontrol>
 <project name="chico" buildafterfailed="false">
 <listeners>
 <currentbuildstatuslistener file="${status.file}"/>
</listeners>
  <bootstrappers>
 <svnbootstrapper localWorkingCopy="${project.dir}"/>
 </bootstrappers>
 <log dir="${log.dir}">
<merge dir="${project.dir}/build/test/log"/>
</log>  <schedule interval="60">
  <ant antWorkingDir="${project.dir}"
  antscript="${ant.script}"
uselogger="true"/>
</schedule>
</project>
</cruisecontrol>

Hopefully this will help make your CruiseControl configuration file easier to maintain. I believe that Continuous Integration should be easy, even if that puts me out of a job!

Tagged

it really does work on my computer

A long time ago my erstwhile colleague and friend Ben once said: “We wouldn’t be writing software on Solaris machines if this were a .NET project – why do they make us write software on Windows machines when we’ll be deploying to Solaris? Ben did have a point. It wasn’t just the fact that software projects the world over force people to use Windows – that’s their lookout. The problem is that by writing software on the wrong operating system*, your people don’t get exposure to the right OS, and you delay vital feedback to the developers.

These issues manifest themselves in several ways. You might observe that the developers and the systems people don’t get along famously. You might also notice the testers being grumpy as they will be the first people to find out that the application won’t actually deploy. And when it comes to diagnosing production faults, the developers might start staring at their
shoes.

Anyhow. I just started working for a company that does .NET projects. Today’s interesting fact: The developers run Windows Server 2003 on their desktops. Just like production. Same IIS version, same patch level. I can’t fault the logic.

* The wrong operating system is defined as “the operating system that you aren’t running in production”.

Ant Best Practices: Put the build file at the root of your project tree

In December 2003, Eric Burke wrote a set of best practices for Ant. Given that I’ve seen some horrors in the last 4 and-a-bit years, I think it’s time to review some of them. Seeing that Ant’s cousin Nant didn’t seem to have so much market share back then, I’ll comment on that where applicable, as well. Eric’s article can still be found at O’Reilly.

Today’s advice:

Put build.xml in the Project Root Directory

If I see anther [N]Ant build with the basedir attribute set to “..”, I think I might scream. Your build file sits well at root of your project. It just works that way. If your build refers to paths, this step makes them resolvable from the build files.

“So what” you might think. “it works, doesn’t it?”. Yes, it might work. But this is a post about best practices. Personally, I get fed up with the doublethink required to deal with paths that are magically rooted elsewhere. Your mileage might vary.

Here’s my favourite reason for doing this: Your IDE should be able to parse your build files and have a stab at telling you if there are errors. IDEA is particularly good at this. Got no budget? Use Eclipse. On a .NET gig? Use ReSharper. Beg your boss to buy you a license after the evaluation expires. The great thing is that when you have errors in your build files (and you will), a decent IDE will try and tell you what’s going on. If you pay attention to the warnings in the build that your IDE generates, you might be able to give your build the TLC that it deserves. And your colleagues might start maintaining the build more.

For the love of God don’t make a seperate project for the build either. Try and devolve the build to the projects that use them. My verdict on this point: Eric was bang on.

Tagged

Maven and Ivy appreciation from Ivan Moore

I did comment on this blog post, but due to excessive spam, the comment system is disabled. Oh well.

Hi Ivan,

Ivy does transitive dependency management, but none of the other stuff. I did a project once where they retro-fitted ivy to get rid of hundreds of binary dependencies checked into the source repository. It worked pretty well. The developers did have to change their working practices a little to accommodate it.

Best

Julian.

Build Status Radiator for CruiseControl.rb

radiator_success
Jason Miller of ThoughtWorks just announced a Build Status Radiator for Beta Brite LCD signs. Works with CruiseControl.rb but looks like it could be easily extended to work with the rest of the CruiseControls.

Link

CI Factory hits Version 1

Nice guy Jay Flowers maintains a wrapper around CruiseControl.NET called CI Factory. He just announced version 1 on Sunday. New features include integration with Perforce, FxCop, and automated branch creation.

Link