CruiseControl Best Practices: Configuration the CruiseControl way

This is the third article in the CruiseControl practices series

You just started using CruiseControl. You use a Version Control System to manage your code. You installed CruiseControl on a spare computer in the office; now it is giving you immediate feedback on the changes that occur in that codebase. Life is good. Then the disk on that spare computer fails, and your build server resumes its previous role as a doorstop.

“No problem”. you think: “All the code changes are in the VCS. We can regenerate any of the artifacts that we need to. In fact, all we need is the config file … “. Yes. That config file. The config file on the hard disk that doesn’t work anymore. This post will outline how to manage your configuration for CruiseControl without fear of losing it. Like many tools, CruiseControl becomes quite useless without configuration.

In projects that I did for ThoughtWorks, we always needed to allow someone to have access to the build server to make configuration changes for CruiseControl. Once projects grow past a few developers, it becomes hard to have everyone be familiar with the installation. Typically we end up with one person (sometimes your humble narrator) becoming the dedicated CruiseControl administrator for the project. This change creates a bottleneck in the team because all the changes to CruiseControl then become funnelled through that one person.

The first step in mending this situation is getting CruiseControl to apply its own configuration. Let’s get started. In addition to the projects that build and test your code, you will need a new project to apply the configuration to the server. We have been doing this to put the configuration file in the right place, using CruiseControl’s <bootstrapper> plug-in to update the configuration files when they change:


<?xml version="1.0"?>
<cruisecontrol>
<project name="config">
<labelincrementer defaultLabel="${project.name}-1" separator="-"/>
<listeners>
<currentbuildstatuslistener file="/var/spool/cruisecontrol/logs/${project.name}/currentbuildstatus.txt"/>
</listeners>
<bootstrappers>
<svnbootstrapper localWorkingCopy="/etc/cruisecontrol"/>
</bootstrappers>
<modificationset quietperiod="30">
<svn LocalWorkingCopy="/etc/cruisecontrol"/>
</modificationset>
<schedule interval="60">
<ant antWorkingDir="/etc/cruisecontrol"antscript="/var/spool/cruisecontrol/tools/apache-ant-1.6.5/bin/ant"  uselogger="true"/>
</schedule>
<publishers>
<artifactspublisher  file="${project.name}/build.log"  dest="logs/${project.name}" />
</publishers>
</project>
</cruisecontrol>

This will robotically update the configuration until the end of time. It’s simple but surprisingly effective. There’s no longer a dependency on the person who can make changes to CruiseControl. Suddenly they don’t need to make the trivial changes on behalf of the rest of the team because anybody can safely change the configuration. If someone does check in a broken configuration, it’s all under version control. Once you revert the change you can find the person who changed it and try and understand what they wanted to do.This is a big step forward. But if you check in a broken configuration it will still be applied to CruiseControl. Fortunately CruiseControl has the good sense not to apply broken configuration; but you’re missing a vital piece of feedback, and for that you need to write a simple validator like this one:

package org.juliansimpson;

import java.io.File;
import net.sourceforge.cruisecontrol.CruiseControlException;
import net.sourceforge.cruisecontrol.config.XMLConfigManager;
import org.apache.tools.ant.BuildException;
import org.apache.tools.ant.Task;

public class ConfigValidator extends Task {
    public String configFile;

    public void execute() throws BuildException {
	try {
	    File file = new File(configFile);
	    new XMLConfigManager(file);
	} catch (CruiseControlException e) {
	    throw new BuildException("Invalid CruiseControl Config");
	}
    }

    public void setConfigFile(String config) {
	configFile = config;
    }
}

The validator uses internal classes of CruiseControl itself to validate the configuration. Ideally we would have an external interface to do this – perhaps a command line option or an “official” Ant task. This approach does mean that you need to set the classpath so that the validator can find your CruiseControl install, but this way you find out with certainty that the configuration is valid for your version of CruiseControl. I like to run these as an Ant task. It’s very simple and easy for everyone to see what it does. Here’s how I included it in a simple Ant build:


<project name="cruisevalidator" default="publish" >
 <import file="build-library.xml"/>
 <target name="validated-config"depends="cruise-validator.jar">
<taskdef name="validate" classname="org.juliansimpson.ConfigValidator" classpathref="main"/>     <echo message="validating ${config}" />
<validate configFile="${config}" />
</target>
<target name="publish" depends="validated-config">
<echo level="info" message="copying CruiseControl config to server" />
<copy file="${config}" todir="${cruisecontrol.dir}" failonerror="true" description="Copy configuration to CruiseControl server" />
<echo level="info" message="forcing a reload of config on server" />
<get src="http://localhost:8000/invoke?operation=reloadConfigFile&amp;objectname=CruiseControl+Manager%3Aid%3Dunique"      dest="${build.dir}/reload.html" />
</target>
</project>

It all works together like this: The CruiseControl BootStrapper fetches us the latest CruiseControl configuration, but in isolation from CruiseControl install – you still don’t know if it is a valid configuration file yet. The “validated-config” target calls the ConfigValidator Ant task. This invokes enough of CruiseControl to make sure that the configuration is legal, and that some of the directories referred to in the configuration exist. If that passes, the “publish” target copies the configuration to the CruiseControl server itself. Finally the same target forces a reload of the CruiseControl configuration using a simple HTTP request to the JMX interface. This ensures that the configuration is reloaded immediately, so that the team knows the configuration is valid. Thanks to my erstwhile colleague Tim Brown for this great idea.

Summary: I have to admit being careless sometimes with XML configuration files. This approach works particularly well for me because I have the safety net of the validation. I do a similar thing with my email and web server installation as well, which I hope to write about soon. The validator code and build files are available here.

Update 2008-12-04: Fixed the link. Thanks to Tom Howard for pointing this out!

Tagged

8 thoughts on “CruiseControl Best Practices: Configuration the CruiseControl way

  1. Anonymous says:

    What would go inside the <import file="build-library.xml"/> file?What are you trying to include?

  2. Julian says:

    Anon: The included buildfile builds the validator itself.

  3. David says:

    Great Blog! I just stumbled accross it just the other day. I am attempting to put this post into practice, but have a few questions: * It looks like you are assuming a different directory layout here than that illustrated in the “Keep Your Dependencies to yourself” post, is that correct? I think it is assuming a /project directory directly under the cc installation dir? * Will this rebuild the cruise-validator.jar again and again or just once? (I think I'm getting thrown off because the cruise-validator is just another project, rather than a .jar file manually added to cruise control) * When developers want to modify the cruise control config file, do they modify the one in the cruise-validator project (set via the "config" property in build-library)? Or is that config.xml file merely meant to be <include.project>ed into my main config file?Sorry if I've missed something, I'm attempting to follow what you've done here and map it to the crazy layout and config files I've inherited at the same time.

  4. Julian says:

    David,Directory layout: yes. I think I assumed the default layout from the distribution (but on my old linux laptop, I think it was rooted somewhere in /var/tmp/cruisecontrol-2.7 or sommat)Rebuilds? I should have used the <uptodate> task so that the validator didn't get rebuilt every time.The config file? You can pass any file you like to the config attribute of the validate element. That is the file that will be validated.It should be pretty straightforward to map to your own config. I believe that you'll need to tell the validator (at runtime) where the cruisecontrol.jar file is, and where the config file under test is.

  5. […] He talked about deployment scripts and how these should be kept under version control.  He gave the example of a bash script used to deploy a system which be put under version control and ‘tested’ but simply running it and checking the exit value.  This actually picked up an error introduced into the script by mistake.  It is also a good idea to have the CI server grab its own configuration from version control. […]

  6. Alexandru Pintilie says:

    Thank you for this excelent idea!

    There’s one thing that I can’t figure out from your config.xml. If the bootstrapper updates the local copy of a project every time, then the modificationset will never find a change, thus never trigger the build. It happened to me.

    Since requireModification defaults to “true”, CruiseControl will never build that project because it’s always up to date 🙂 Btw. I use CVS, and the solution was just to remove the bootstrapper.


    Alex

  7. simpsonjulian says:

    Alexandru,

    Ouch. I think it’s the CVS ModificationSet in CruiseControl. It has no such entity as the repository revision number in SVN, so it uses the working copy. Sorry to put you crook.

  8. Jeff Hall says:

    Julian,

    Thanks for this great trick; it has worked well for us with version 2.7.3 of CruiseControl, but I’m having difficulties with version 2.8.2. Have you tried this with 2.8.2? I’m getting a “java.lang.VerifyError” being thrown somewhere inside the constructor to XmlConfigManager. We’re using Java 6 on linux.

Comments are closed.

%d bloggers like this: