Tag Archives: dependencies

Dependency Integration: Promiscuous vs. Reluctant

When you’re really doing Continuous Integration, what strategy should you choose for dealing with dependencies? Should you aggressively pull in changes to that library that the guys in the ivory tower make you use? Or should you wait until there’s a compelling need?

Your hand may be forced by your tooling. If you’re checking in your binary dependencies into your codebase, it may be too painful to get updates frequently. If you’re using a build system built around a dependency manager, you might opt to get the latest, shiniest libraries.

You and your team should make a conscious, strategic choice however: The promiscuous strategy will lead to more integration work, in batches. On the flip side, you’ll never be left wading through updates when there is a compelling need to upgrade through many versions of a library. Proponents of this approach might argue that the batch size on this type of change should be kept small, to minimise risk.

If you’re reluctant to take changes, you’ll experience less integration pain on the project. It’ll be your choice when you update. Proponents of the reluctant approach would argue that they are delivering value, and not wasting development capacity on work that doesn’t deliver business benefits.

Whichever approach you choose: make sure it’s an informed choice.


A way to cool dependency Hell?

How to break a deploy: Take one codebase. Sieve in a new class. Mix in the dry ingredients and a new runtime dependency. Place another dependency on a pre-warmed Hudson, bake for 10 minutes (on a medium heat) and then deploy. Oh dear. It didn’t deploy.

We’re a bit crap about managing the external dependencies of our code. I’m not talking about libraries but more basic dependencies that your application might have, like native code libraries, or commands. There’s two ways you can do this:

  • You can make people responsible for the care and feeding of your testing and production environments. This is easy to implement, but stupid. I think it would only work in an environment with exceptional communication.
  • Or you can insist that any application must declare what it depends on.

Keeping environments up to date keeps lots of people in a job. It’s a really dumb job. At my day job we’ve taken the latter route, using Puppet.

Puppet is a tool for systems administration. But you can use it even if you don’t know fsck from fmt. The way we’re using it is to be an executable specification of the dependencies that your application needs. For example: A test just failed on a new server – with this output:

Validation failed: Avatar /tmp/stream20091208-22414-y3anvf-0 is not recognized by the 'identify' command.

I realised that it needed thelibmagic1 package and possibly libmagic-dev. I could have installed them then and there onto the machine. I’d have forgotten about it in the excitement. So I added them to a file on that project called dependencies.rb. This file is run by Puppet before we deploy. It gives our developers enough control over the target operating systems so that they can make small changes to the deployment environments. We’ve been running this via a Capistrano task on our project; we typically run it as the deployment user; that way we can easily make changes to crontabs. Puppet won’t exit cleanly if it can’t install all the dependencies, so it’s a good way to test.

Here’s an abridged version of our dependencies file:

class dependencies { 

  include $operatingsystem

  class ubuntu {
    package {
       'libcurl3':             ensure => present;
       'libcurl3-gnutls':      ensure => present;
       'libcurl4-openssl-dev': ensure => present;
       'g++':                  ensure => present;
       'build-essential':      ensure => present;
       'libmysqlclient15-dev': ensure => present;
       'libxml2-dev':          ensure => present;
       'libxslt1-dev':         ensure => present;
       'libmagic1':            ensure => present;
       'libmagic-dev':         ensure => present;


  class gentoo {
    cron {
      'some cron job':
        command   => '/engineyard/bin/command blah blah',
        user      => 'admin',
        hour      => ['*/2'],
        minute    => ['12'],
        ensure    => 'present';

  class darwin {
    notice("Nothing to do in this OS.")
node default {
  include dependencies

In this file we define a class (dependencies), which doesn’t do much but look for a corresponding inner class to match your operating system. Right now we have a very simple arrangement: The dependencies::gentoo class contains crontabs and the like for EngineYard. The dependencies::ubuntu class names all the native dependencies of our rubygems. We have an empty class for Darwin to stop the Mac OS X machines from complaining. That’s it. Here’s the Capistrano task:

  desc "Run Puppet to make sure that dependencies are met"
  task :dependencies, roles => :app, :except => {:no_release => true} do
    run "cd #{release_path} && rake dependencies"

Image courtesy of eflon


Build tool philosophy

Holy dependencies, Batman! Make this man do the build on your project …



CruiseControl Best Practices: Keep your dependencies to yourself

This is the second of ten practices for CruiseControl

The average Java project has many dependencies – open source tools and frameworks, third party libraries, libraries that come from your project or organization – the list is endless. When I wrote this article, my current project had 84 jar files that it depended on (or could have depended on!). Much pain can come from the way you manage these dependencies. Sometimes it’s harder than you think to make software build or execute if it’s not clear what the dependencies are. A lot of the pain of joining a new project and taking a day to make the code build comes from unclear code or environmental dependencies.

It’s a good idea to keep your CruiseControl installation and its dependencies clearly separated from your projects and their dependencies. So for the most basic example, lay out your CruiseControl install something like this:

|– cruise
| |– 2.7.0
| `– 2.7.1
|– logs
| |– blowfish
| `– scorpion
`– projects
|– blowfish
`– scorpion

Upgrading CruiseControl should be the work of minutes.If you need to add libraries to CruiseControl itself, this should be a warning sign that your project has dependencies it can’t itself satisfy. The only exception to this rule that I can think of is custom CruiseControl bootstrappers or publishers and the like.

There are two things in particular to take note of here: the logs and the ‘.ser’ files. The logfiles represent the history of your project: this is why I always try to keep them in a different directory hierarchy from the CruiseControl installs. CruiseControl will also by default persist the state of its projects in files with the extension ‘.ser’. Make sure to keep those when you upgrade. If you do this, and your project has no dependencies on CruiseControl, it should be simple to upgrade.

Next, think about the dependencies you have baked into your build tools. There’s a common pattern of putting dependencies into your Apache Ant’s ‘lib’ directory. For things that are hard to wire in like Junit, fine. But if your build depends on something that is actually resident in your build tool, then you have a problem. It will work today, but not necessarily tomorrow. If you upgrade libraries that your code depends on in the build tool, you can’t build yesterday’s code – which makes things interesting when you’re trying to resolve a production bug!

The good news is, it’s easy to fix: make the project contain its dependencies. For example, if your Ant build depends on Oracle (say, to support rebuild the database with the ‘sql’ task, and your standard project Ant build contains the Oracle drivers, your build may look like this:

			<target name="drop_all_tables">

			<sql driver="oracle.jdbc.driver.OracleDriver"
			userid="user" password="donttell"

			<transaction src="${sql.dir}/drop_all_tables.sql"/>



However in this case, the dependency on the ‘in the
default classpath’ Oracle driver isn’t stated; Ant will just look for the ‘oracle.jdbc.driver.OracleDriver’
class in the default classpath. The first thing to do is put the driver jar file
into the project and set a classpath:

`– ojdbc14.jar

			<path id="buildtime">
			<pathelement location="${lib.dir}/oracle.jar"/>

			<target name="drop_all_tables">
			<sql driver="oracle.jdbc.driver.OracleDriver" userid="user" password="donttell"
			delimiter=";" classpathref="buildtime">
			<transaction src="${sql.dir}/drop_all_tables.sql"/>


Note the addition of the ‘classpathref’ attribute in the ‘sql’
element. This useful little attribute allows you to refer to paths elsewhere
within the build, reducing duplication. When you have moved a dependency into a
project, take to ceremonially deleting the dependency the moment that you can.
On a project where I recently undertook this process, I was fortunate to have a
regular release working in my favour. I fixed the dependencies in the trunk and
revisited a few weeks later, once the release branches that depended on things
outside the project weren’t used.

So in summary, think about your project’s dependencies, including the ones that
are satisfied because they just happen to be satisfied. If you make those
dependencies explicit, I promise you that you’ll be asked less questions from
your colleagues about why they can’t compile! If you have many projects with the
same dependencies, that’s something that I hope to address in a future post.

Tagged ,

Ant Best Practices: Define Proper Target Dependencies

Ant Best Practices: Define Proper Target Dependencies

(Image taken from Nick Sieger’s Photostream)

Wondering what this post is about? Have a look here.

Last time I wrote, it was about reusing paths. Tonight, it’s about the dependency graph. My [N]Antcall is evil post goes into some detail about dependency graphs. Let’s just agree here that Ant targets tend to accumulate dependencies. The point that Eric makes is that the dependency graph (otherwise known as the dependency tree) grows over time as you add targets. Being a thing that accumulates over time, the graph can get crufty. That target that was just a short term thing while Bob worked on a refactoring exercise (oh, the irony) is now central to your build. So from time to time you need to clean the dependencies out, like that burned cheese that melts off of pizzas and sticks to the bottom of the oven.

Some projects seem to run into trouble and throw dependencies out completely. Umm, good luck with that. Eric suggests a half-way house approach: a few well known targets that contain dependencies on lesser known but functional targets for the more experienced to use at their own risk. I can go along with that.

My last thought is this: your default build should never depend on a clean target. Leave it to the compiler to have a go at working out what it should do. If you use Continuous Integration, you can make a little target that depends on clean just for that case. Your fellow developers will thank you.

Tagged ,

Ant Best Practices: Manage Dependencies Using Ant

Ant Best Practices: Manage Dependencies Using Ant

(image taken from cotes’ photostream)

In the last post, we discussed the importance of a clean target. Is it your first time? Have a look at the index page.

Today, I want to talk about managing dependencies with Ant. What does that mean? In this case, it means managing the dependencies on code within your project: dependencies in code, third party libraries, etc. I want to cover dependencies between other projects in another post.

The advice is to start from the very bottom up. You have a bunch of common utility files? Compile them up in one target. And make sure that target outputs one thing and one thing only: a jar file called common.jar. Now, that file isn’t going to be your enterprise application on it’s own, so do the same to the code that references the classes inside common.jar. Make sure that each one of those outputs a single artifact, and that each one of them depends on the target for common.jar. Now, you probably have a web application or two that depends on a jar file of business code, which depends on a bunch of common files that nobody can decide where to put – common.jar.

What if there’s a separate project for the front-end project, vs. the back-end? You still want to package up the code as jar files for consumption by the other project. But for the love of God try to resist doing that. Most projects really only have one product that they make. To split them up into different, smaller projects adds a whole world of complexity.

What we just did was stop anybody from breaking the build by making the back-end code depend on the front-end code. Which is quite easy to do when your IDE allows you to do that in a few keystrokes.

This technique will create some good karma for your project, as long as you follow the sane practice of making every developer run the Ant build before they check in. For me, this is critical. Nice one, Eric.

Tagged ,