Monthly Archives: April 2010

Links for 2010-04-29

Here’s Thursday’s links from My Twitter Stream.

Links for 2010-04-27

Here’s Tuesday’s links from My Twitter Stream.

Links for 2010-04-23

Here’s Friday’s links from My Twitter Stream.

Separation of concerns in Ant

Separation of concerns in AntThere’s nothing wrong with Ant. No, really! True, there’s some nasty Ant files out there. Perhaps that’s because we often treat our build as a second class citizen. How do you keep your build files from becoming bloated and hard to maintain? Break ’em up!

I’m going to use a classic problem to illustrate this: deployment. Have you seen an Ant build that seemed to know far too much about the innards of the container? First you end up with a lot of properties. Then you need to maintain loads of odd targets to try and make it all work together. We can do better.

Step 1: Break it out. You want to have a totally separate Ant buildfile that you can use for deployment. Acceptance criteria: You can kick off a deploy by calling the separate file with a few properties (like where the deployable is, etc.)

Step 2: Import it. Use Ant’s import task in the body of your buildfile. Never inside a task!

Step 3: Prefix. A colon is a legal character in an Ant property or task name. So make the prefix match the name of the project. Each distinct buildfile should have the name attribute of the project element set with a meaningful name. Use that.

Step 4: Maintain discipline. It doesn’t matter how you do this. Cold showers, if you like. Just make sure that you keep the properties in the right place with the right name.

Here’s an example:

<project name="base" default="deploy">
	<property name="container" value="tomcat" description="The Java container that we use" />
    <import file="${container}.xml" />
</project>

Note that there’s no deploy target in the file. That resides elsewhere. Running the default target will kick off a deploy to Tomcat from …

<project name="tomcat">
	
  	<property name="container:hostname" value="some.great.hostname" />
	<property name="container:remote.user" value="deploy" />
	<property name="tomcat:admin_url" value="http://${container:hostname}/admin" />
	
	<target name="tomcat:deploy" description="This throws a war file at tomcat">
		<echo message="gory details of tomcat deploy go here"/>
	</target>
	<target name="deploy" depends="tomcat:deploy" />
	

</project>


.. here. Note that there are properties with a nice generic prefix. Keep those generic because …

<project name="jboss">
	
  	<property name="container:hostname" value="some.great.other.hostname" />
	<property name="container:remote.user" value="fleury" />
	<property name="jboss:some.jboss.property" value="Paula is brilliant" />
	
	<target name="jboss:deploy" description="This throws a war file at jboss">
		<echo message="jboss deploy goodness here"/>
	</target>
	<target name="deploy" depends="jboss:deploy" />
	

</project>

… all you need to do is pass a different container property to have it deploy elsewhere. What I love about this is that the two implementations cannot exist side-by-side. Only one can be imported, and the property namespace isn’t polluted.

(image thanks to Hansol)

Links for 2010-04-21

Here’s Wednesday’s links from My Twitter Stream.

Welcome to our new Sponsor – AntHillPro

Welcome to our new Sponsor - AntHillProAnd there’s a sponsor. AntHillPro. Sponsorship gets you an ad on the web site and a message on the RSS feed. It’s not too different to things I’ve had before but there’s a crucial difference: 99% of the ads that I got via ad networks were crap.

So now we have the same thing but with the branding of one sponsor, rather than a rash of ads. I’ve agreed a three-month trial with UrbanCode (who publish AntHillPro).

What will I do with the money?
I’m going to pay for:

  • all the hosting and service costs
  • sending T-shirts for supporters and guest authors,
  • buying drinks for people I interview,
  • getting help to keep the blog maintained and functioning

Why this sponsor?

  • They have a compelling DevOps offering
  • they’ve been selling a tool with good lifecycle features for years
  • and they’re a bunch of great guys who really get Continuous Integration

Will you pimp their product at every chance you get? Nope. I’ll still wear my Clover t-shirt, use Hudson, and write about competing products. I might write more about their product but that’s because I have pretty good communications. If more vendors kept in touch, I’d write about their products, too.*

* unless you want me to do a huge review. I don’t often have the time to do those. Sorry.

(image from Jessica)

Supporting Multiple Environments – Part 4

Supporting Multiple Environments - Part 4In the final installment, I’m going to talk about how to share configuration between developer level environments on through to clustered or “stack” type environments.

Recycling Configuration

Ok – so now your configuration is its own standalone Maven module with its own series of branches and build process.  But what if there are shared items between the generic configuration for both development boxes and your deployment environments?  This is where my little Maven plugin comes in.  With the configuration stored in property files and built and treated as a very simply library and deployed to your repository manager, you can then put a dependency on this configuration jar.

My simple plugin reads in the development configuration as ordered and outlined above and jams the resulting property set into the Maven project’s property set.  I bound my configuration plugin to the validate phase and this allowed the “process-resources” phase to leverage these settings.

One thing I haven’t had a chance to implement is the actual recycling of the code that does the layering.  Ideally, the plugin that does the layering would live close to (or be part of) the actual configuration project so when a new layer is added (or one removed), you could address this in a single location versus updating N number of deploy scripts.

Caveats

There are a few problems this overall concept brings with it.  Here are a few of the main ones I’m aware of (that haven’t already been addressed):

– Confusion on overlay procedure – Of course you’ll have to explain how various layers of configuration get compressed into a single flat configuration.  The  plugin as written allows the user to print out what the final (local) configuration will look like.  The deploy process similarly generates a flattened property file that is processed in isolation and gives people who deploy an idea of the flattened landscape.  A well annotated set of source files as well as detailed “mvn site” output is also strongly suggested (you are publishing site documentation, aren’t you?  Hudson makes it really easy, just do it!).

– Who’s allowed to touch which layer  – I mentioned this above, but pushing each deployment specific bundle of property files into a single directory within the configuration project allows (in Perforce at least) your SCM admins to restrict access by group to various bundles.  We block QA from touching any configuration outside of their own stacks but let them have read only access to everything else for example.

– Preventing duplication and unnecessary overrides – A few times during testing, we noticed a property set at a low level (in the general bucket) and at an another deeper level.  Same property, same value.  Ideally, we’d have some set of unit testing to go through various combinations of properties and verify that this doesn’t happen.  Also, in a perfect world, there would be a report tied to generating the config jar that would say how many times a particular value comes up across various stacks.  This would highlight things such as a port number that was initially introduced in testing on a particular deployment environment and rather than pushed higher in the config tree, it was simply replicated.

– Migrating from settings.xml or profiles to property based configuration – This was one of the larger challenges, how to take everything that has been established and move it to an entirely new way of thinking and construction?  The solution for us was a one-off Maven plugin that could look at profiles (and a combination of profiles) and generate the various files listed above.

If you’ve followed me this far, you’ll see very little configuration the further you get out in the tree (if you find you need all the layers at all).  You can also imagine breaking the configuration into smaller more specific chunks.

This may also this give you a chance to also revisit what things deserve a property and what doesn’t (there’s a tendency to make everything under the sun configurable).

Hopefully this gets your gears turning and helps you find an efficient solution for managing the configuration for your product.

Links for 2010-04-19

Here’s Monday’s links from My Twitter Stream.

Links for 2010-04-13

Here’s Tuesday’s links from My Twitter Stream.

  • 07:13: apache.org incident report for 04/09/2010 : Apache Infrastructure Team http://bit.ly/ao5eyt [ JIRA attack on apache.org]

Links for 2010-04-12

Here’s Monday’s links from My Twitter Stream.

  • 21:52: InfoQ: Chris Wanstrath on GitHub http://bit.ly/aY6dfy
  • 21:43: InfoQ: Key Takeaway Points and Lessons Learned from QCon London 2010 http://bit.ly/9Oc7L8 [including our devops track]
  • 21:06: Mike Nash’s Two Cents Worth » Blog Archive » Continuous Releasability with Maven and TeamCity http://bit.ly/aTIZ5e
  • 15:11: a little madness » Blog Archive » Zero To Continuous Integration in Three Minutes http://bit.ly/bm2Vdl