Tag Archives: atlassian

Vote for us in the Ultimate Wallboard Challenge

We’ve been beavering away to get XFD up to scratch for this competition and we’re almost ready.

XFD has a whole host of UI improvements (see the credits for the superstars who helped), and we’re working on supporting more CI servers, very soon.

Would you take 2 minutes to give us a vote? Thank you!

Atlassian are also looking for local representatives, by the way.

Tagged

Tool News, February 2010

There’s a new Hudson out. It’s got 7 bug fixes and 3 enhancements, including some profiling and optimisation work to speed up the UI.

Atlassian have embarked on a Build Engineer hiring offensive. They seem to be based in Sydney. Atlassian would appear to be a great place to work.

Electric Cloud have released new versions of ElectricAccelerator (now at version 5) and ElectricCommander (at version 3). The former works with make, MSBuild, scons and the like to parallelize the build. There’s a new feature in Electrify, a private cloud that allows to to parallelize more software production. I think they’re on the money with the private cloud. Cloud services may be the meme du jour, but cloud technologies used internally would be the way to for many companies to start.

Scott Castle:

Electrify is interesting because it’s a seamless way to tap into distributed computing, and it’s not tied to using a specific build tool. If you’ve looked at the various efforts to build Ant tasks in parallel, or to upgrade Maven for parallel execution, they all focus on local-box parallelism, leaving out the tricky task of distribution across hosts. Electrify bridges the gap, converting any parallel task into a distributed task, and it has all the network, caching, and management ‘nice-to-haves’ that homebuilt systems generally lack.

ElectricCommander looks like it aims to be the tool to make build and release managers redundant. And if that means not doing deployments for other people, I’m not bothered. I’m surprised that the tool hasn’t had more coverage as it seems squarely in the same arena as Cruise and Ant Hill Pro. Today’s release brings a lot of customisation: here’s Dax Fahang, (who came to Citcon Paris, some readers may recall):

Developers are often resistant to the introduction of new tools or processes, so we find that customizability is critical to adoption. With ElectricCommander 3.5, users truly have a build-test-deploy system that works and looks exactly the way they want it to without all the traditional effort and costs associated with customization of a homegrown system. Version 3.5 allows customers to customize the UI without any application language or upgrade constraints – thus enabling Commander to easily look like a legacy tool, integrate data from multiple applications, and provide role-based user experiences.

Tagged , ,

Story: Problem? What problem?

Andy has a consulting horror story …

We were doing a 2 day ‘architectural overview’ for a potential client – they were suffering performance issues as their production site started scaling, and they wanted a hand investigating it. Whilst digging around, my colleague noticed that one of their ASPs didn’t compile. He suggested a potential fix, and asked how they would test and promote the change. “Ah”, said client dev that he was working with – who promptly logged on to the production box, found the non-compiling production code, edited the file in place, and bounced the server …

Tagged

Story: M-ANT-ven

Thanks to Graeme Gillies for this funny anecdote from the Atlassian Giveaway. I love the fact that he persisted in nailing this.

When I started at a company (which shall remain nameless) being the
new (junior) guy I was made the “build guy” of one of their biggest
projects to date. I quickly learnt that the code had cyclic
dependencies amongst other things, which mean that if you did a fresh
checkout into a fresh workspace (no builds before) the code would not
compile because it relied on “cruft” and left over artifacts of
previous builds to build correctly.

They were using Apache Ant at the time (this was a J2EE project) and
then the decision was eventually made to migrate to Maven 1. This
would seem like a great thing to all, and finally a way of reviewing
and getting the build process back under order. Unfortunately what
happened instead was that when you tried to compile the build with
Maven it would break half way through complaining about dependencies,
after which I was instructed to then run the ant build which would run
and break half way through as well, when that happened, run maven
again, it should get further this time, when it breaks, run ant, then
maven, then ant, etc. Eventually what would happen is each build
script would break, but get further and further each time until
eventually one of them somehow gave you a completed build.

Took me the better half of 6 months but eventually I pulled it apart
and got it sorted.

Tagged

Story: I call it “The Wheel”

This anonymous user gets top points for offending the most people in this Atlassian Giveaway entry.

I was tasked to evaluate continuous integration systems for my workplace so I tried CruiseControl, Hudson and Bamboo. CruiseControl was a pain in the ass to configure, and I couldn’t get it to work – so stuff that. Hudson was really cool, and good looking, but it failed the Maven sample projects I was using – so stuff that.

Bamboo trial worked flawlessly, and was really the only one I could recommend.
However, our “architect” had CruiseControl on his brain (and used it at home on his .Net projects) and insisted that was good enough, but not good enough for him to lend a hand in configuring it!

He then got another developer to have a look at CruiseControl, who also failed to set it up. Said developer then, without telling anyone else, goes and writes his own continuous integration system with a PHP web interface, making system calls to Ant and Maven, using shell scripts to fetch from Subversion. We used that for months, but eventually sanity prevailed and we got Hudson, because it was free and had matured enough.

(Image thanks to Fimb)


Tagged

Story: Rolling your own

Thanks to Daniel Spiewak for this great story from the Atlassian Giveaway.

It was a dark and stormy night. No, actually it was a pleasant summer day, but daylight lacks a certain dramatic flare which is so necessary for a good story, especially a story about build systems.

I was working as the semi-lead developer for a mid-sized project run out of London, UK. My job was primarily to work on the Java clone of the Cocoa client application. Through a very clever and dangerous bit of hackery, the Cocoa and Java clients shared a single, Java-based backend which communicated with the server via xml-rpc. Because of the project’s architecture, there were a number of inter-dependent sub-projects. As I was working on a clone of the Cocoa client, it was often necessary for me to build a new copy of the client after each new change. However, this was, in and of itself, a non-trivial undertaking. Once you added the building of the other sub-projects both individually and as dependencies, and my days started to look more and more like the “dark and stormy” variety.

Now, each project (with the exception of the Cocoa frontend) had an Ant build script which I had carefully crafted. These build scripts would invoke each other as need be, meaning that I could build a copy of my Java clone by simply invoking its build and allowing Ant to handle the rest. This solved a lot of my dependency headaches, but building every single project was still a tedious undertaking. Thus, I build another layer of abstraction above Ant, consisting primarily of Ruby scripts hacked together on top of Ant. The idea was that I could invoke my master build script, passing a list of project descriptors, and this build script would determine the optimal way to build each project and its dependencies. I was even able to rig this script with cron so that it automatically built a new version of each sub-project as necessary.

Unfortunately, this build script worked a little too well. My boss got wind of it and decided that it should be put onto the main development servers as a sort of continuous integration solution. This sounded like a good idea at the time, but it ultimately led to far more trouble than it was worth. I got sucked into the position of permanent build system maintainer; and, given the hacky nature of the system’s architecture, it ended up being quite the position. As more sub-projects were added and more flexibility was needed, I actually had to rewrite the entire system from scratch more than once. Looking back, I’m actually astonished by the sheer number of hours I spent cursing those scripts into behaving.

I was probably on my third or fourth rewrite before I realized the idiocy of what I was doing. I had literally created a full continuous integration build tool from scratch (complete with web admin system and XML-RPC API) without ever considering the alternatives. It only took me a few minutes of research to realize that Hudson, Cruise Control, Bamboo, and really any CI system would solve exactly the same problems without any need for hacky scripts or unnecessary effort. It took my boss a little while longer to come around, but eventually he too saw the wisdom of relying on a pre-existing solution rather than rolling our own convoluted hack job.

The really amazing part of this story is how I didn’t even see what I was doing until very late in the process. It started out as just an innocent collection of scripts to aid my own development process. Each step I took toward hacky maintenance hell was so gradual, so subtle in form that I completely failed to see where I was headed until I was already there. An while my build system didn’t actually require a defunct Pentium series processor to run, it does certainly certainly qualify as a bizarre polyglot, home-grown build system which should never have been allowed to fester.

Image by Mark Cummins

Tagged

Story: Wolf++ fixes the Deployment Process

The last of the top three stories for the Giveaway was this one from Wolf++. Enjoy!

My last gig I was hired on to be the build guy. On my first day I sat
shotgun to their deployment process. The manual process was as
follows.

1. Logon to the ‘build box’
2. Get latest
3. Open visual studio and compile the application
4. FTP the resulting app to a staging area on our production webserver
5. Put the website offline
6. Run any new SQL files against the production database (Hopefully
you guessed the execution order correctly)
7. Copy the app into place
8. Put website back online
9. Hope nothing had broken.

I had quite a task ahead of me. It turned out not only was I the buy
guy, I also had to manage the development and QA servers, help the QA
staff with even the smallest technical hurdles (including
demonstrating many times how to use FTP, etc), help developers with
source control, teach developers it works on my box is not an excuse,
etc. We had one big shared database with a number of different
applications that had no deployment schedule other than ‘yesterday’
and as soon as it’s ready. Quick fixes went straight to production!

The first thing I tackled was how to divine which SQL scripts were
needed with which application. I created a file in each project that
developers would list source control locations of the SQL DDL files
that needed to be executed. It was a simple run first to last
ordering. Along with this I created a tool to read and package the
files along with a script to execute the package against each
environment. This became a part of the automated build process.
Later I even created an editor with syntax highlighting, source
control integration, and validation to easily construct this file.

Since all the applications shared a single database (a flaw they were
going to fix someday), I finally convinced them to deploy all the
applications together. So any changes to the database would be tested
in all the apps at the same time. Builds would first be tested on the
QA server and only fully vetted builds went into production.

I brought in FinalBuilder which I used to construct build scripts and
implement continuous builds. Quick build break feedback is a must.

Prior to this job I never had to work on live websites. I can say for
certain now working on live websites ratchets up the pressure. The
desire to improve the build and deployment process correlates directly
to the desire to not break services a large global customer base use.

Tagged

Story: Daniel’s Continuous Integration System

Daniel Spiewak gave us this great story for the Atlassian Giveaway. He’s earned a t-shirt. Have a good weekend, where-ever you are.

It was a dark and stormy night. No, actually it was a plesant summer day, but daylight lacks a certain dramatic flare which is so necessary for a good story, especially a story about build systems.

I was working as the semi-lead developer for a mid-sized project run out of London, UK. My job was primarily to work on the Java clone of the Cocoa client application. Through a very clever and dangerous bit of hackery, the Cocoa and Java clients shared a single, Java-based backend which communicated with the server via xml-rpc. Because of the project’s architecture, there were a number of inter-dependent sub-projects. As I was working on a clone of the Cocoa client, it was often necessary for me to build a new copy of the client after each new change. However, this was, in and of itself, a non-trivial undertaking. Once you added the building of the other sub-projects both individually and as dependencies, and my days started to look more and more like the “dark and stormy” variety.

Now, each project (with the exception of the Cocoa frontend) had an Ant build script which I had carefully crafted. These build scripts would invoke each other as need be, meaning that I could build a copy of my Java clone by simply invoking its build and allowing Ant to handle the rest. This solved a lot of my dependency headaches, but building every single project was still a tedious undertaking. Thus, I build another layer of abstraction above Ant, consisting primarily of Ruby scripts hacked together on top of Ant. The idea was that I could invoke my master build script, passing a list of project descriptors, and this build script would determine the optimal way to build each project and its dependencies. I was even able to rig this script with cron so that it automatically built a new version of each sub-project as necessary.

Unfortunately, this build script worked a little too well. My boss got wind of it and decided that it should be put onto the main development servers as a sort of continuous integration solution. This sounded like a good idea at the time, but it ultimately led to far more trouble than it was worth. I got sucked into the position of permanent build system maintainer; and, given the hacky nature of the system’s architecture, it ended up being quite the position. As more sub-projects were added and more flexibility was needed, I actually had to rewrite the entire system from scratch more than once. Looking back, I’m actually astonished by the sheer number of hours I spent cursing those scripts into behaving.

I was probably on my third or fourth rewrite before I realized the idiocy of what I was doing. I had literally created a full continuous integration build tool from scratch (complete with web admin system and XML-RPC API) without ever considering the alternatives. It only took me a few minutes of research to realize that Hudson, Cruise Control, Bamboo, and really any CI system would solve exactly the same problems without any need for hacky scripts or unnecessary effort. It took my boss a little while longer to come around, but eventually he too saw the wisdom of relying on a pre-existing solution rather than rolling our own convoluted hack job.

The really amazing part of this story is how I didn’t even see what I was doing until very late in the process. It started out as just an innocent collection of scripts to aid my own development process. Each step I took toward hacky maintenance hell was so gradual, so subtle in form that I completely failed to see where I was headed until I was already there. An while my build system didn’t actually require a defunct Pentium series processor to run, it does certainly certainly qualify as a bizarre polyglot, home-grown build system which should never have been allowed to fester.

Tagged

FishEye and Crucible get social

It’s 23:30 at Build Doctor HQ, so this is a quick post: Atlassian announced a couple of new releases today. FishEye 2 and Crucible 2. Both come with social features, which is interesting. From the press releases:

Since code review is typically an iterative process — you code, someone else comments, you code again, more comments.. etc. — Crucible 2 allows you to easily incorporate code revisions into the review process. Simple navigation let’s you step through code changes and/or comments, tracking what has been read and reviewed, letting you easily compare any number of revisions along the way. Crucible’s goal is to make the code review process as easy as email; ideally, something you do for about 10-15 minutes everyday.

… and …

FishEye 2 has a completely new user interface that brings GUI efficiency to web based source code browsing. Every source artifact has a unique URL, everything is queryable, and there are cool charts and reports. FishEye 2 is now faster to navigate with improved usability to make it easier to understand your context. It also has better integration with Atlassian tools. For example, FishEye pulls in activity from related JIRA issues and Crucible code reviews and lets you organize projects across multiple repositories.

Atlassian have really been busting ass to release loads of new developer tools recently. They might do a demo for me, which I’ll post.

Right now the developer tool that we need at the day job is an espresso equipped Piaggio Ape. I’d like to see one on our fourth floor office.


Tagged

Atlassian Giveaway Winners!

Thanks to everyone who wrote a story for the Atlassian Giveaway. The entries are in, the judging is complete, and here’s the results!

First prize goes to John Martin, who wrote a story of despair and hope around Continuous Integration. John’s an Atlassian user already, with two other products, so he’s happy to make the hat trick with a Bamboo license.

I wish I had two more licenses to give away to Wolfplusplus and Daniel Spiewak. They both had great stories. Martin Scholl had an amazing tale of folly from a telco firm who should have known better.

I wish I had a special prize for Patrick Debois, who came up with this inspired entry.

Tshirts go to the guys above and: Graeme Gillies, Chris Melikian, Andy Yates, Neil Tingley, and two readers who had to remain anonymous. I’ll be asking for your addresses to hand on for fulfilment. Huge thanks to the Atlassian crew for sponsoring this giveaway. I know they’d be grateful if you took a few minutes to see what they are doing with Bamboo and the rest of the developer tools that they sell.

Here’s John’s winning entry:

I think my best experience with continuous integration had nothing to do with integration _testing_. When I joined the project as the cm/build guy, they were in a cycle of work real hard for two weeks then give the stuff to the build guy to build on Thursday evening. It had to be Thursday because all day Friday and sometimes the weekend would be spent trying to get the thing to actually compile. Typical stuff: developers checked in ‘simple’ changes without compiling and testing, they didn’t always update to get the others’ stuff, they forgot to add new files to source control, they worked in windows and our server was linux, maven dependencies hadn’t been added to our local site, etc. Tracking these down and running around from developer to developer was time consuming. And then the deployment to test could take another half day because something would be wrong with the way the EAR was generated and it wouldn’t load into test’s weblogic server. And, of course, to be honest it wasn’t always the developers’ fault — sometimes we didn’t make a necessary change to the server or upgrade java or whatever.

We introduced cruisecontrol simply to run a compile on every checkin, and within a month, official test builds were smoother because compile problems appeared immediately upon check in _and_ we knew who to talk to without all the hassle of layering everybody’s changes in at 5pm on Thursday. By the time we did the official build, we hardly ever saw a compile problem again. Then we added automatic weblogic deployment (locally), just to prove the EAR loaded, and we could build and deliver to test almost immediately upon code freeze.

That particular project never got to the point where it was using the continuous build for any real test coverage. Two years later, the new build guy asked me why they should still bother doing the cruisecontrol thing if it wasn’t testing. He hadn’t seen an error in a year. I told him that testing should really be added, but even so, if he took away his continuous build, he was going to have to learn a little more about reading stack traces.


Tagged , ,