Refactor your configuration file

Part 5 of CruiseControl Best Practices

Tell me that you think duplication in computer systems is a good thing. I dare you. Drop me a comment and we’ll talk about it. I’m someone who spent the best part of ten years doing systems administration and running other people’s code. So you might guess that I’m a card carrying member of the “Duplication is evil” party. If you’re not a member too (I must work on the secret handshake!), this might not be a post for you.

Duplication doesn’t just creep into our business code. It also seeps into the build and CI systems when you’re not looking. There’s probably someone on your team who is copy-pasting something into the build right now. Don’t worry about stopping them. Read on to find out how to put the CruiseControl configuration right.

Here’s an example of duplication. You can see that there’s a load of repetition in the file. I didn’t exactly pull if from a project (I’d love to work on a project called Groucho), but it’s not far away from a real life scenario.

<?xml version="1.0"?>
<cruisecontrol>
 <project name="chico" buildafterfailed="false">
 <listeners>
   <currentbuildstatuslistener file="/var/tmp/cruise/logs/chico/status.txt"/>
 </listeners>
  <bootstrappers>
   <svnbootstrapper localWorkingCopy="/var/tmp/cruise/projects/chico/"/>  </bootstrappers>
 <log dir="/var/tmp/cruise/logs">
 <merge dir="/var/tmp/cruise/projects/chico/build/test/log"/>
 </log>
  <schedule interval="60">
   <ant antWorkingDir="/var/tmp/cruise/projects/chico"           antscript="/var/tmp/cruise/ant/bin/ant"
uselogger="true"/>
  </schedule>
 </project>

  <project name="groucho" buildafterfailed="false">  <listeners>
  <currentbuildstatuslistener file="/var/tmp/cruise/logs/groucho/status.txt"/>
 </listeners>
  <bootstrappers>
<svnbootstrapper localWorkingCopy="/var/tmp/cruise/projects/groucho/"/>  </bootstrappers>
 <log dir="/var/tmp/cruise/logs">
  <merge dir="/var/tmp/cruise/projects/groucho/build/test/log"/>
  </log>
  <schedule interval="60">
  <ant antWorkingDir="/var/tmp/cruise/projects/groucho"           antscript="/var/tmp/cruise/ant/bin/ant"
 uselogger="true"/>
   </schedule>
</project>
</cruisecontrol>

Fortunately, there is a way to keep this in check in CruiseControl.

CruiseControl has had the feature of properties in the config.xml file since 2005. They are modeled on Apache Ant style properties, with a dollar sign and curly braces: ${cruise.home}. These are immediately useful in replacing constant style values in your config.xml file, like the location of your Ant script or the log directory.

This is a huge boost, but it’s not enough. What about the things that are defined in the config file itself, like the name of the project? The author was thinking ahead on this one: there is a magic property called ${project.name} for you to use. It’s scoped to the enclosing project so that you can’t refer to chico’s project harpo, or vice versa.

Take a look at the config file now that properties have been introduced:

<?xml version="1.0"?>
<cruisecontrol>
 <property name="cruise.dir" value="/var/tmp/cruise" />
<property name="log.dir" value="${cruise.dir}/logs/" />
<property name="projects.dir" value="${cruise.dir}/projects" />
<property name="ant.script" value="${cruise.dir}/ant/bin/ant" />
<project name="chico" buildafterfailed="false">
 <listeners>
  <currentbuildstatuslistener file="${log.dir}/${project.name}/status.txt"/>  </listeners>
<bootstrappers>
<svnbootstrapper localWorkingCopy="${projects.dir}/${project.name}/"/>  </bootstrappers>
 <log dir="${log.dir}">
  <merge dir="${projects.dir}/${project.name}/build/test/log"/>  </log>  <schedule interval="60">
       <ant
antWorkingDir="${projects.dir}/${project.name}"
     antscript="${ant.script}"
   uselogger="true"/>
   </schedule>
 </project>
 <project name="groucho" buildafterfailed="false">
 <listeners>
  <currentbuildstatuslistener file="${log.dir}/${project.name}/status.txt"/>  </listeners>
 <bootstrappers>
  <svnbootstrapper localWorkingCopy="${projects.dir}/${project.name}/"/>  </bootstrappers>
 <log dir="${log.dir}">
  <merge dir="${projects.dir}/${project.name}/build/test/log"/>
 </log>
  <schedule interval="60">
      <ant
antWorkingDir="${projects.dir}/${project.name}"           antscript="${ant.script}"
    uselogger="true"/>
  </schedule>
 </project>
</cruisecontrol>

Things look much better. If you wanted to change the name of the project, or one of the paths that it references, you could change it once. But there’s still some patterns that repeat themselves again and again – we can probably do better.

There’s a bit more magic to the properties. The ${project.name} property dynamically changes depending on the enclosing project. Not only can you use it in the middle of configuration values, but you can declare it as a part of the value of another property. What this means for your long-suffering configuration file is that you can declare one property for the location of the project, and have that property lazily evaluate to to fit the project. This might confuse your colleagues when they look at the configuration file for the the first time – “How does the same property work in so many different cases?”.

<?xml version="1.0"?><cruisecontrol>
<property name="cruise.dir" value="/var/tmp/cruise" />
 <property name="log.dir" value="${cruise.dir}/logs/" />
<property name="project.dir" value="${cruise.dir}/projects/${project.name}" />
<property name="ant.script" value="${cruise.dir}/ant/bin/ant" />
<property name="status.file" value="${log.dir}/${project.name}/status.txt" />  <project name="chico" buildafterfailed="false">
  <listeners>
  <currentbuildstatuslistener file="${status.file}"/>
  </listeners>
<bootstrappers>
 <svnbootstrapper localWorkingCopy="${project.dir}"/>
</bootstrappers>
<log dir="${log.dir}">
<merge dir="${project.dir}/build/test/log"/>
  </log>  <schedule interval="60">
       <ant antWorkingDir="${project.dir}"
 antscript="${ant.script}"
uselogger="true"/>
</schedule>
</project>
<project name="groucho" buildafterfailed="false">
<listeners>
<currentbuildstatuslistener file="${status.file}"/>
 </listeners>
 <bootstrappers>
 <svnbootstrapper localWorkingCopy="${project.dir}"/>
  </bootstrappers>
  <log dir="${log.dir}">
<merge dir="${project.dir}/build/test/log"/>
  </log>  <schedule interval="60">
   <ant antWorkingDir="${project.dir}"
antscript="${ant.script}"
uselogger="true"/>
</schedule>
</project>
</cruisecontrol>

One thing that happens on bigger projects is that the configuration file becomes huge. While you can do things like use XSLT to generate a configuration file (I’ve made myself some headaches that way), you can just break your configuration file into smaller chunks. There’s a great feature called <include.projects> in the CruiseControl configuration file – it lets you refer to another configuration file. This works just like the <import> feature in Ant. You end up maintaining one config.xml file that contains variable definitions (they work over the includes as well), and a host of smaller configuration files that contain one project’s definition. On my current project this features has made it a lot simpler to add or remove projects. Even if someone forgets to delete the include.projects element from the config.xml when they delete the file, it doesn’t matter – that project won’t be imported as the file doesn’t exist. Tracking the changes to the CruiseControl configuration becomes a lot simpler as well.

Here’s the config.xml now:

<?xml version="1.0"?>
<cruisecontrol> <
property name="cruise.dir" value="/var/tmp/cruise" />
<property name="log.dir" value="${cruise.dir}/logs/" />
<property name="project.dir" value="${cruise.dir}/projects/${project.name}" /> <property name="ant.script" value="${cruise.dir}/ant/bin/ant" />
 <property name="status.file" value="${log.dir}/${project.name}/status.txt" />  <include.projects file="projects/chico.xml" />
<include.projects file="projects/groucho.xml" />
 </cruisecontrol>

Here’s one of the projects:

<?xml version="1.0"?><cruisecontrol>
 <project name="chico" buildafterfailed="false">
 <listeners>
 <currentbuildstatuslistener file="${status.file}"/>
</listeners>
  <bootstrappers>
 <svnbootstrapper localWorkingCopy="${project.dir}"/>
 </bootstrappers>
 <log dir="${log.dir}">
<merge dir="${project.dir}/build/test/log"/>
</log>  <schedule interval="60">
  <ant antWorkingDir="${project.dir}"
  antscript="${ant.script}"
uselogger="true"/>
</schedule>
</project>
</cruisecontrol>

Hopefully this will help make your CruiseControl configuration file easier to maintain. I believe that Continuous Integration should be easy, even if that puts me out of a job!

Tagged

11 thoughts on “Refactor your configuration file

  1. Ben says:

    I have a couple of questions:1. Is the include.project really an include? We have defined “plugins” as project templates, so if these are defined in the parent file and the projects listed in the imported files, then presumably it will be read in OK?2. Does the CruiseControl Validator you posted about a little while ago work with included files?

  2. Julian says:

    Hi Ben,You can define objects in the parent file and have them visible to projects that are included from that parent file. I don’t have a working copy of CruiseControl on me at the moment to test that, but I *think* it ought to work.The validator that I wrote just calls CruiseControl’s own config system to read the file, so it will just read the included files. It shouldn’t fail the build if a project file is missing. I quite like that because it means that even if you can’t convince somebody to edit the XML file when they remove a project, most people will be okay to delete the file.Slightly annoying at validation time, however :)BestJulian

  3. Ben says:

    Is there a way to force it to fail if it cannot read an included file? Otherwise someone could delete an included file and some projects will stop being monitored/built.

  4. Sean M. says:

    No mention of plugin preconfiguration? Don’t build without ’em.

  5. Julian says:

    Sean,You’re right. I have used them to great effect with things like the email publishers. I’ll update the post some time soon.Thank you!BestJulian.

  6. Anonymous says:

    I notice that both chico and groucho project configs are identical barring the project name…This matches my configuration where I regularly add/remove live branches to the configuration and have to copy/paste previous project configs.What would be really cool is if I could pass in the project name like this -<include.projects file="/project-config1.xml" name="trunk"><include.projects file="/project-config1.xml" name="bugfix_branch1">Then I would only have to maintain one project config.

  7. Julian says:

    Anon,You can template out projects more: if every configuration detail is derived from the project name, there’s a way to template out the whole shebang. The ${project.name} value is lazily loaded, so it becomes dynamic. Not sure if I have the example any more – let me mull it over.

  8. John says:

    But this doesn’t help much with ant scripting.

  9. simpsonjulian says:

    John,

    What help do you need?

    Julian.

  10. […] Refactor your configuration file This one really seemed to strike a […]

  11. Nigel says:

    One of the few examples of using include.projects, which is good!
    However, the tagging in the project file is incorrect, either missing a tag at the start or a redundant tag at the end.

Comments are closed.

%d bloggers like this: