Monthly Archives: March 2010

Links for 2010-03-31

Here’s Wednesday’s links from My Twitter Stream.

Is Maven agile?

Is Maven agile?Is Maven agile? Here’s my half-assed reasons why:

  • agile methodologies can uncover issues in your organisation. I’ve got a pet theory that Maven will do the same for your code.  Some build tools allow you to paper over the cracks of your code and tests.  Tests too slow?  Then parallelize them.  Code won’t compile cleanly?  That’s okay, you can frig the build file until it does.  Maven doesn’t help you do this kind of thing.  Maven makes you do the right thing.  I’m sure there’s a plugin that spits at your feet for even thinking about accommodating software quality problems in your build tool.
  • agile engineering techniques ensure that we can continue to deliver at an appropriate pace.  I was arguing about Maven with Steve Freeman recently; he pointed out that an Ant build should be a trivial thing to write.  What I didn’t say is that once you’ve got your maven build sorted, the plugins that you can employ come at a very low cost: Cargo and Sonar can save you literally hours of fiddling with XML.  So don’t compare time spent implementing the two tools so you can compile your code; look at what you have to do over the whole lifecycle.

Disclaimer: Your Milage May Vary.  Maven may not suit your needs.  Ditto with Ant. The value of your technical debt may go up as well as down.

(Image via http://www.flickr.com/photos/zenera/)

Links for 2010-03-26

Here’s Friday’s links from My Twitter Stream.

Supporting Multiple Environments – Part 2

Supporting Multiple Environments(see part one of this series here)
In this installment, I’m going to cover the four cheap-fast ways I’ve seen discussed to generate configuration for your application.

Managing Configuration

There are a few ways I’ve found or seen discussed over the years for managing the configuration portion of a deployment (local or otherwise).  I don’t think one way is the best way for everyone, each company is unique.  The decision should be based on what suits your company best, not how many plugins you can configure or how can you change the way business operates (within reason).  There are other, more complex ways; like Spring based configuration.  This discussion is limited to the cheapest (simplest) way to manage configuration.  Below are four very common approaches.

– Spin a version of your deployable unit (preconfigured) for each deployment option (one artifact)

This is the most expensive (with regard to time and space) option of the four.  This option assumes that you have very few deployable units and very few stacks to deploy to.  This option can be configured via profiles in the POM and leverage values either stored in profiles or in a settings.xml (or a host of other inconvenient ways). After what I’ve seen during my recent stint interviewing and what I’ve seen come across the Maven mailing lists, I think this is no longer an option for most.  This becomes painfully obvious when the artifact starts getting beyond 50 MB. This concept fails to scale with the addition of machines or deployment environments.

– Spin a single version of your deployable unit and build multiple configuration artifacts (two artifacts)

At a previous employer, this was the initial choice for a first foray into configuration for our deployable units.  This regularly took about three hours to finish and each additional machine was painful to manage (all stored in the build user’s settings.xml checked into Perforce), let alone adding additional stacks.  To add insult to injury, having to sit through another three-hour build because ONE machine in ONE stack had ONE property tweaked was both painful and embarrassing.  This option also fails to scale with the size or complexity of the environments you’re trying to support.  Each additional stack could add upwards of 10 minutes each.

– Spin a single version of your deployable unit and have it include templates and configuration for those templates (one artifact)

This is a step in the right direction.  But the configuration is still bound to the artifact you’re attempting to deploy.  It seems like a waste to generate a new deployable unit if the configuration for a particular application in a given deployment environment has changed.  To a degree, this approach also fails to scale.  You’ll end up including more and more configuration.

– Spin a single version of your deployable unit and have it include templates with separate configuration (two independent cycling artifacts)

This is the final evolution I chose.  Breaking out the configuration from the actual deployable unit and making the configuration its own artifact (ie pushed to our repository manager) allowed the same version of an application to be instantly redeployed (with the option of just reapplying configuration to an already deployed application) with a configuration change that took merely seconds to build.

Continued…

In the next installment, I’m going to cover the configuration storage mechanism for this separate configuration jar approach.

UKUUG Spring Conference 2010

As I write this post I’m sat on the train back from the UKUUG Spring Conference Manchester, getting a fascinating insight into the world of asset finance sales from the passenger sitting opposite me.

Highlights of the conference were:

  • Spending time with some of the DevOps crew – R.I. Pienaar, Kris Buytaert, and Patrick Debois.
  • Co-presenting ‘Hudson hit my Puppet with a Cucumber’ with Patrick. (above) It was hard to rehearse (what with living in difference countries and all) but I enjoyed it once the adrenalin kicked in and the nerves relaxed. Dialogue is interesting. With an audience, and with a co-presenter. I’d like to explore more ways to interact with an audience than to drone at them.
  • Manchester.
  • Kris giving me an ‘Everything is a FSCKing DNS problem’ Tshirt – thanks Kris!
  • Hearing about the process of creating an Ubuntu LTS releases from Thierry Carrez of Canonical. We discussed the pain of building Java and Ruby software for a distrinution afterwards. I hope people realise how much work distro maintainers go to! I met a one-time commitrer for the JPackage project – they confirmed what a PITA it was back then before Sun allowed people to distribute Java.
  • Kris Buytaert and Andrew Stribblehill‘s MySQL scaling talks.
  • The Ubuntu Enterprise Cloud presentation was interesting. Not sure I’d want to use it in production, but for in-house test environment use it might be great.

UKUUG attracts some seriously talented people. It’s a small conference (north of 100 people), but that’s the way I like them.

Links for 2010-03-17

Here’s Wednesday’s links from My Twitter Stream.

Supporting Multiple Environments

Supporting Multiple EnvironmentsOne of the longest running debates I’ve watched in the Maven community over the years, is how to best support different configurations for different deployment environments (where “deployment environment” means a QA environment or staging environment – anything other than a local developer build). Some people argue simplicity over the complexity of some of the other options mentioned below.  Others don’t have spare disk or time to burn building up complete and unique deployment artifacts.

In this first installment, I’m going to cover deploying my favorite deployment tool – Ant.

Packaging and Deployment

I tend to prefer using Ant as a deployment mechanism, forgoing perl or various shells because of the simplicity of the language.  Anyone can read and understand a well-written Ant script (Ant is inherently self-documenting).  One of the features I love is the “expandproperties” filterchain used in combination with a copy command.  If your resources are in directories that mirror where they will live in their final locations, then it’s really easy to manage this with assembly descriptors (where the filtering can happen) and in the deploy script (if you’ll play along and use Ant).  There are all kinds of little perks in using Ant, like being able to add a “-Dname=value” on the command line to override a known configuration setting, without having to rebuild your configuration.  Here’s a very simple example of how I’ve configured applications in the past in a deploy script:

<copy todir="somelocation">
<fileset dir="templates" />
<filterchain>
<expandproperties/>
</filterchain>
</copy>

It is (or can be) as simple as that.

Assembly descriptors can be a bit trickier.  Initially, we used the stanzas to grab lists of wild-carded files, but this (obviously) won’t fail if the descriptor can’t find a file.  So we migrated (as much as we reasonably could) to using the individual file listings via the stanzas.  We also have three types of assembly descriptors: “dev” for local builds which has filtering set to true; “prod” for deployable artifacts which plucks files directly from the source tree and has filtering turned off; and a “common” component descriptor for grabbing everything that is uniform between dev and prod and requires no processing.

This approach has some pros and cons – obviously one major con is having to list out each file individually – but the giant upside is, if a file is missing because of relocation, the build will fail and you’ll instantly be alerted.  Another downside is the fact that the assembly descriptors are essentially the same, with a difference in “outputDirectory” and filtering the only real differences.  One of the benefits is that you don’t have to memorize where a template file lives in its final resting place.  A cursory glance at the resources directory gives you the exact target location.

Another implied concept here is deploying an exploded WAR/EAR (we don’t need to digitally sign the artifacts as they are consumed internally).  While there may be some benefits to startup or load times of the archived file, but being able to slightly alter configuration and redeploy without repackaging is a greater benefit – why build a archive if you don’t have to?

In the next installment, I’m going to cover the four cheap-fast ways I’ve seen discussed to generate configuration for your application.

(photo thanks to tacomabibleot)

Tagged

Links for 2010-03-16

Here’s Tuesday’s links from My Twitter Stream.

  • 08:20: Have an interesting #devops story to tell? Why don’t you submit yours for devopsdays 2010 US http://bit.ly/cfVCAH (via @patrickdebois)
  • 06:57: sin.tigris.org http://bit.ly/aHK96u [Yet another CI server – based on SOAP?]
  • 06:52: Accidental Simplicity: The Checklist Manifesto as Agile Primer http://bit.ly/bOXhmV [I’ve used checklists. It’s pulling out the big guns]

Links for 2010-03-15

Here’s Monday’s links from My Twitter Stream.

  • 07:39: Before DevOps, Don’t You Need OpsOps? | Web Admin Blog http://bit.ly/bglZZu [@gdb_ has hit the nail on the head!]

ThoughtWorks UK are hiring Build Managers

Want to work with some of the best programmers that one can hire? TWUK are looking for people like me, but with a better accent! It’s all in the ad below.

This is the gig that got me 2 very happy months’ work in San Francisco. And they don’t have a project in Hemel Hempstead any more.

Link