Monthly Archives: March 2012

A Neo Chapter

I was too quick to choose my post-ThoughtWorks job. I interviewed with an ‘Agile’ development shop that worked off of detailed requirements, and an insurance company who prided themselves on their agile record. They were both misguided, and so was I for electing to work at one of them.

So I jumped ship. To a startup, that wanted to change the world of media, but changed investor money into the exact opposite of money. Freelance seemed like the only safe option.

Running The Build Doctor Limited has been a great ride. But I miss camaraderie. Which I’ve found, with the Neo Technology crew. On Monday, I join Neo as a full-time employee. We’re going to deliver a fabulous graph database, and you’re going to love it.

XFD: open sourced

As promised, I’ve open sourced the core of XFD, under an AGPL license:

https://github.com/builddoctor/xfd-core

Thanks to the efforts of Kush, you now have:

  • a plain vanilla XFD, that you can style to fit your organisation
  • all the code to talk to different CI servers
  • with rspec support, and a Travis build:

http://travis-ci.org/#!/builddoctor/xfd-core

The next release of XFD will use this project. Dogfood will be eaten.

Happy hacking!

 

Bamboo 4.0 – interview with James Dumay

Bamboo 4.0 is out today. I had a brief chat with James Dumay (@i386) from Atlassian, who is the Product Manager for Bamboo.

What’s new with Bamboo?

The Bamboo team has been hard at work for the last few months rethinking and rebuilding the way developers use and interact with CI when a team uses branches. Many thought leaders have come to the conclusion that feature branching and continuous integration are ultimately incompatible because all branches cannot be tested easily. Bamboo 4 is all about proving this wrong.

For example, when a developer commits to a new branch, the developer must go to the CI server to clone and update the build configuration to take into account the new branch. This is needed to ensure commits to the branch are tested correctly by running the full test suite. This is a big problem because a developer has to remember to manually create a new build for their branch each time, as otherwise code does not get tested for large spans of time. And let’s face it, we are a lazy bunch when it comes to processes.

Keeping the configuration between the main build and the branches synchronised is a lot of work and we end up with a maintenance headache.

Bamboo 4.0 solves all these problems by sharing build configuration between the main build and the branches. Working with branches and CI has never been faster and easier.

If you are using Git or Mercurial, Bamboo can automate even more for you. Bamboo will automatically setup a new build when a developer pushes a new branch to the repository and will also automatically clean up any branch build configurations that have been unused. This means that your CI server doesn’t remain cluttered with dead branches (of course all of this is configurable and we never delete branches in your repository).

The other problem is that when we create the branch and manually setup a build for it, we are not sure whether the changes on the branch will integrate with the master branch until it’s time to merge the branch into master (or another branch for that matter). If you don’t run the full test suite before pushing your changes back to master, then you could be potentially slowing the whole team down by introducing bugs and failing tests.

For Git and Mercurial repositories, Bamboo will attempt to merge changes from (or to) the master branch then executes a build against the merged changes. If the build passes, the changes are optionally committed and pushed to either the branch or the master. That’s fantastic for situations where you need to keep your branch updated with the latest changes from master or want to automatically push changes on the bug fix branch back onto master when the build passes.

We think we have really nailed it and we can’t wait for developers to start working with branches and CI together the right way.

We’re in an age where CI servers are all very similar, what makes Bamboo stand out?

Ecosystem

Firstly and foremostly, we truly believe Bamboo is the best CI server out there and it only gets better if your team is using JIRA, Fisheye and Bitbucket. Mention a JIRA issue in a commit message and Bamboo will automatically provide detailed information about that issue on the UI wherever the commit is displayed. If your repository is indexed in Fishseye, then looking at the commit details is only a single click away in Bamboo. As Atlassian builds and acquires more products, Bamboo will add integrations with them (so stay tuned!). We’ve already got a plugin that allows you to broadcast build notifications to the entire team using our cool new communication tool, HipChat.

Bottling best practices

One thing our team and I believe in strongly is that Bamboo should be flexible enough to work your way, not the way that ivory-tower droids think development should strictly work.

Our approach is to marry best practices with flexible features wherever possible to allow you to setup your CI server the way you want it. For example, Continuous Delivery influenced the design of the Stages feature heavily. Stages are the flux capacitor of Continuous Delivery – they not only make it possible, but make configuring it pretty painless. Stages give developers control over their workflow by allowing the steps in the build and deploy pipeline to be split into groups. A Stage can be automatically triggered when the build runs or be launched manually by pushing a button – whatever works better for the team.

The kicker is that Stages let you run a few steps sequentially, then run a few steps in parallel, then go back to sequential execution. For example, you can run compile, package and deploy steps in sequence, then run a several batches of tests in parallel, then switch back to running one step at a time as you deploy the package to a staging environment (only if, and only when , all your batches of tests finish successfully!). You could then have another stage that deploys into a production environment. Going back and forth like that is a huge pain to configrure with other CI servers as the configuration tends to be rather brittle. We have tried to make configuring deployment pipelines like this as easy as possible in Bamboo.

Extensibility and stability

Having a stable API is a must for anyone who wants to rely on plugins. In Bamboo 3.1, we introduced the concept of Tasks which represent user configurable steps of the build process. The Task API been stable for over four major releases, from Bamboo 3.1 all the way to Bamboo 4. Right now we are working with a host of commercial software tool vendors to integrate their tools as Tasks. They simply wouldn’t have had the confidence to try to do this without the stability or the simplicity that the Task API has to offer. That fact has really shined through with the amazing growth we have seen in the number of plugins available for Bamboo since Tasks were introduced.

Design

We believe that not only should tools make it easier todo your job but they should look, feel and behave great if you are going to use them day in and day out. No CI server is as easy to understand and work with as Bamboo. Every day, we think about how to make Bamboo’s UI as clear and simple as possible so teams can find the information they need faster, and get back to the business of making awesome software.

24/7 Support

Bamboo support is 24/7 which is a rarity in our space. Our support team kick ass – plain n’ simple. They go out of their way to help resolve issues when things go wrong and feed info directly back to the Bamboo dev team. If you contact support at Atlassian, you are going to get a fast response.

Analysis: Atlassian are raising the bar; taking things of clear value (auto merging of loony tunes features branches, etc.) and making the accessible. Not a bad strategy, and shows you that James has done his share of developer time.

Sponsor Danger: Atlassian sponsor this blog, and I am disclosing this interest.


Tagged

The Godmother of Continuous Integation: An interview with Tara Hernandez

Continuous Integration existed before CruiseControl. Before Martin Fowler wrote about it. Of course, it always was a primarily a practice, but some don’t know that there was a Continuous Integration server that existed before CruiseContol: Mozilla Tinderbox. Tara Hernandez was instrumental in developing the first version of Tinderbox, and I asked her for a little more info:

Q: Where did you start out?
A: Borland was a great place to start out (first real job out of college), and I learned a lot. However, the CEO was insane and there was a strong personality cult around him that made it hard to reign in his bad ideas. For example, Borland was really really well regarded with regards to its compilers and if had stuck to that we might have been okay. But Phillipe Kahn (CEO) had this idea he could beat Microsoft across all technical verticals and so started acquiring other technologies like databases and stuff. We had so many products that we were competing against ourselves in some instances.

Microsoft finally noticed and squished us like a bug. The team at Borland I was on (Languages) got totally pillaged. Microsoft lured away almost all the top talent, including our VP. They probably spent 10 million dollars on signing bonuses. It was a mess.

Anyway, I got hired at Netscape after about 3 years of Borland. Somebody I had previously known at Borland pinged me and told me of an opportunity at this new startup Netscape. I was the first build/release engineer hired, and I only had 3 years of experience at a very sedate well-established company under my belt. Best analogy I can think of was that it was sort of like jumping out of an 747 without a parachute. There wasn’t a consistent build platform, even within the same general operating systems group.

For example, there were two distinct make systems in play in the unix world. The revision control system (CVS) only worked on unix at the time, so for Windows and Mac platforms we had to run this crazy software to export the local hard drives as mounts readable on the unix network, and then people had to run xterm sessions to do any SCM transactions. It sucked.

Q: What was the problem that caused you to need Continuous Integration?

A: It was the fact that we had 2 version of MacOS (68k and PPC), two versions of Windows (16 and 32 bit) and (initially) 8 versions of Unix. At any given time our ability to successfully generate a set of builds was close to nil. Things would work great on one platform, but not on 6 others.

The most famous of these was when Brenden Eich took out the world by checking in the use of a NaN in the early javascript code (not called that then of course, but that’s what LiveScript turned into). Turns out NaN was totally unsupported by 60% of our platforms. Sadness. On the other hand, to show his contriteness he bought me and my team a whole lot of expensive booze, so that worked out okay in the end.

Tinderbox really evolved out of desperate self-defence, because it was almost impossible for us to do our jobs — mainly, get builds out the door in a timely fashion, and let developers know as soon as possible that they broke something somewhere and to please go figure out how to get it fixed.

About Tara:

Tara Hernandez is a specialist in software engineering infrastructure having developed procedures and tools at assorted Silicon Valley companies. Her most notable stint was at Netscape Communications Corporation where she initially helped develop new tools such as Tinderbox, Bugzilla, and Bonsai, and later helped adapt those tools to a public environment after the creation of mozilla.org and the release of the Netscape web browser as open source. Now at Pixar Animation Studios, Tara and her team are reinventing development infrastructure to handle the challenges of writing software in a film production environment.

 

Vendor news, 12 March

  • ThoughtWorks recently released Go 12.1. It’s not version inflation, they’e adopted the Perforce model of doing regular releases in the year. The biggest feature: TFS support. Hello Steve! [link]
  • The Prags just released ‘Deploying with JRuby’, which looks fascinating. [link]
  • UrbanCode [warning: sponsor danger] are doing a webinar on March 22: ‘Using AntHillPro with uDeploy’ – their play on the pipeline [link]
  • Amazon have released an m1.medium instance type, that is available in 32 and 64-bit flavours. Perfect for CI nodes.
  • Do double check your GitHub SSH keys [link]