Category Archives: Ruby

Giternal Debian package

Giternal Debian packageGiternal is a handy tool for managing git submodules. Give it a YAML file of git repositories, and it’ll ensure that the repos are all checked out. I’d use it to bootstrap all of the Puppet modules that I need, except for one fact:

It’s a rubygem. Happy to use them, but I don’t want dependencies on Rubygems at the bottom of my toolchain. I also have a boostrapping problem because Rubygems get installed from a Puppet module, which I fetch using Giternal.

What to do? Make a Debian package! So I did. You can download it here. It depends on ruby. I can make a Debian repository if anybody likes that idea.

Here’s a tiny example of the yaml file.

apache:
  repo: git@github.com:simpsonjulian/puppet-apache-ubuntu.git
  path: modules
common:
  repo: git@github.com:simpsonjulian/puppet-common.git
  path: modules

Here’s the giternal Debian package install.

jsimpson@fox:~/Documents/workspace/giternal$ sudo dpkg -i giternal.deb
(Reading database ... 145351 files and directories currently installed.)
Preparing to replace giternal 0.1.0-debian1 (using giternal.deb) ...
Unpacking replacement giternal ...
Setting up giternal (0.1.0-debian1) ...

Here’s giternal doing its thing.

Socks:puppet jsimpson$ giternal update
Updating common
 ..updated
Updating wordpress
 ..updated
Updating sudo
 ..updated

Image via (Git book author) tswicegood

Tagged

Rubygems on Ubuntu (with Puppet if you like)

Rubygems on Ubuntu (with Puppet if you like)

Debian packages and Rubygems: they get on like two angry cats in a sack. This post explains how you get Rubygems and Dpkg to play nicely on Ubuntu Hardy Heron.

What’s the issue? The Debian Packaging System (DPKG) is pretty good as packaging systems go. It’s had dependency support baked in for years, which means you can install one package, and any package that it needs, and so on. Rubygems is good in other ways. It’s a very convenient way to use and distribute other people’s Ruby code. Rubygems is optimised for developer convenience. The Debian system is optimised for stability. What happens when you try and make them work together? Bloodshed.

How do we cope? Most people get around it by installing the stock Rubygems into the Debian or Ubuntu system. This overwrites ‘/usr/bin/gem’ with a version of Rubygems that will install gems into the ‘/usr/’ directory on your system. What happens if you upgrade your Ubuntu distribution? The new version of Rubygems will overwrite the stock one. Because the Debian maintainers changed rubygems to install gems into a safer place (/var/lib/gems), you’ll probably notice installed gems disappear.

But what’s the downside? When the new Ubuntu LTS (Lucid Lynx) is released in April, you’ll probably want to upgrade. I’m finding Hardy harder to support as new tools arrive (couchdb is a good example). Is it time to get your systems sorted out now, so that you don’t break your apps on the upgrade?

How to do this? You can install the rubygems packages from Karmic Koala on Hardy Heron. Have a look at these instructions below. Note: this might make your Gems disappear. Do have some gem list –local output so you can put them back!

# HINT: run the sudo first or it will swallow any other text
sudo -s 
# rubygems and this scripts's dependencies
# if they are already installed, it won't reinstall
aptitude install wget ruby1.8 rdoc1.8  

# get the new gems on the system
cd /tmp
/usr/bin/wget   http://mirrors.kernel.org/ubuntu/pool/universe/libg/libgems-ruby/rubygems_1.3.5-1ubuntu2_all.deb
/usr/bin/wget http://mirrors.kernel.org/ubuntu/pool/universe/libg/libgems-ruby/rubygems1.8_1.3.5-1ubuntu2_all.deb
dpkg -i *.deb


# tell your system about them
cat <<HERE > /etc/profile.d/rubygems.sh
  export GEM_HOME=/var/lib/gems/1.8
  export GEM_PATH=/var/lib/gems/1.8
  PATH=${PATH}:${GEM_HOME}/bin
HERE

You’ll need to log out and back in again to see the changes. I also have this handy Puppet manifest to do all this en masse. Comment if you want me to publish.

Image thanks to Jijy

Update: I also want to help try and get rubygems working out of the box in either Debian or Ubuntu. Watch this space. Hopefully I’ll look at this over Xmas.

Update: Thanks to @auxesis for the comment. I’ve made a couple of changes to this post.

Hosted Continuous Integration: Run Code Run

Hosted Continuous Integration is here. Run Code Run have announced support for private projects. What does this mean?

They already build open source projects from GitHub for free – for an example see my own Build Rosetta Stone. Now, you can build a closed source project – which isn’t visible to the world via GitHub.

There’s a strong Ruby flavour to Run Code Run. They appear to support a lot of RubyGems natively so that you don’t have to. Being so Ruby-centric can’t hurt as the Ruby community seem to have a strong preference for Continuous Integration.


They might want to do something about all the broken builds on the open source projects however. About a third of the projects are broken and many of those have ben broken for months. Perhaps I’m obsessive [okay, I am obsessive] , but those broken builds imply that people are wanting Continuous Integration in name but not deed.

Anyhow, those aren’t the paying customers. I’d give my eyeteeth to see some stats on build success and failure once they have a few hundred projects to mine. Well done, guys.

(Image taken by Flrnt)

A solution to broken Gems?

RubyGems have been exceptionally successful as a way for Ruby developers to share code. We generally think that sharing code is a good thing. Certainly, the Rails community can write projects exceptionally quickly; thanks, in part to RubyGems.

My beef has been that RubyGems doesn’t play nice with operating systems. We have standards for where things should live in any operating system. In the world I inhabit most (Unix systems), there’s a well defined place for everything. This is more than just being tidy. It helps keep things consistent and stable: when you install new software, dependencies can be easily found, for example.

Gems don’t really work like that. You can have many gem installations on your system, and they can all provide code to an application. It’s for this reason that the Debian team took the RubyGems code, modified it, and put it in a Debian package. This helped bridge the worlds of Debian packages (a packaging system that does dependency resolution to work out which packages you need in order to run the packages you want) and RubyGems (which mostly does the same thing).

Unfortunately this often lags behind the rapid development of RubyGems, so we’re forced to do things like install the newer version by hand over the top of the Debian version. This gets the job done, in doing so breaks the packaging system, which is a key reason that people use Debian or Ubuntu. Think it’s just me complaining about this? Some other people who know a lot about operating systems aren’t happy either:

At least for Debian or Ubuntu users, there’s one solution: DebGem. DebGem (from the guys who brought you Phusion Passenger and Ruby Enterprise Edition) bundles RubyGems inside Debian packages. If each package has its dependencies declared right, you’ll be able to install any gem and it’s native dependencies, in one go.

I hope it works.

The issue here is that it takes lots of time to maintain packages. Especially when you’re trying to track the packages that come from an extremely prolific community that knows how to use distributed version control systems (git is now the VCS du jour for Rubyists). Let’s hope that Phusion can charge enough to make it worth their while to continue; once you’re hooked on this, it would be hard to stop.

Not sure that this really solves the problem. The core issue here is that the developers are elegantly solving the problem of how to reuse code as developers. Systems administrators are trying to make systems that are stable and easy to maintain. The two might never meet.

Final thoughts: they might do well pursuing a freelance model to get gems packaged. Also, a commercial distribution might buy it to gain market share. My fee for this idea is a snip at 10%.

( image from Ed Yourdon )


Every Continuous Integration server that supports Ruby + Git

As I posted, I’m on a quest for the right continuous integration server for my day job. I had some helpful suggestions in the comments (thanks guys). I thought I’d maintain the whole list here (in no particular order):

If I’ve missed any other continuous integration tools, please let me know. Thanks.

Update: Added Bamboo. Thanks Ken.
Update(August ’09): Added Ci Joe.

 

 

Tagged , ,

The quest for a decent Ruby Continuous Integration tool

Git has become very popular in the Ruby community. Github in particular has become a focal point for Ruby innovation. So which Continuous Integration tools support Git and Rake (the Ruby community’s build tool of choice)? I’ve been trying a few out for the 1Click2Fame build process.

What I am looking for is a Continuous Integration server that:

  • supports Rake
  • supports Git, via Github
  • can run multiple agents (I want to test our website against many operating systems)
  • can stop a running build (sometimes, you know it’s not going to work out)
  • has a configuration gui

First out the gate is the daddy of Continuous Integration tools, CruiseControl. It supports both git and rake out of the box. Good.

Pros:

  • It works
  • I know it very well

Cons:

  • It works on a single node
  • You can’t stop a build
  • No GUI (unless you go visit Al Wick)
  • The new Java GUI doesn’t render Ruby output very well. The XSL approach may have been as old as the hills, but people did tinker with it.

Next out is CruiseControl.rb. This project was sponsored by ThoughtWorks as a way to make it easier for Ruby projects to use Continuous Integration. I kind of hate it. The original CruiseControl ended up releasing Rake support before the Ruby port happened.

Anyway, it’s done. There used to be this meme that you had to have a CI server written in the language that your engineers wrote every day. I don’t think that’s the case now. These days, I’d suggest that you could do something cool with JRuby to not write all that code yet again (ThoughtWorks were forced to port CruiseControl to .NET by a customer who actually banned Java code from their site – not at all insecure then).

It supports both git and rake, but it’s always felt a little fiddly to me. You have to add a project from the shell on the machine, not from the GUI. The Git support seems like an aftermarket accessory rather than a factory option. Here’s how it stacks up:

Pros:

  • It works
  • Ruby hackers can hack it
  • It does render rake output nicely, until you reach a limit, and then it truncates the output. You can see the full raw output.

Cons:

  • Single node
  • Can’t stop a build
  • No Config GUI

While CC.rb works, it’s not rocking our world. The quest for Continuous Integration greatness continues. Next episode: Cruise.


CruiseControl Build Radiator

Sudhindra Rao from ThoughtWorks has released a Build Radiator (Big Visible thing) for all the CruiseControls that support CCTray.

I installed it tonight. Needed to install the rack and activerecord gems. It would be nice to see it wrap around if you have more than a few projects. Most teams would probably do fine with this on a big display, I suspect.

This is pretty much what Michael and I did last year, except Sudhindra has made his nicely generic. Oh, and open source and available to the public. That helps a lot.

Nice work, Sudhindra.

Link


Tagged