Hosted Docker-Enabled CI Tooling - Show down!

The goal

At Container Solutions we’re currently building a Mesos framework and I was tasked with getting CI up and running. The framework uses Gradle for builds and Docker to build images. We use the Gradle wrapper which only requires a JDK, in our case version 1.8. I wanted to be up and running as soon as possible which meant the following:

1. Low barrier to entry: I wanted something that either worked out of the box or would tell me quickly what I needed to do.
2. Depth first search: The hunt would stop as soon as I found something that worked which means that this is not an exhaustive comparison.
3. Hosted: Although the ubiquitous Jenkins can ‘do anything’ I’ve always found it to be way more work than people expect so I wanted to avoid running our own instance entirely.

I opened up Google and entered: “Hosted CI solution docker gradle” and after some searching I narrowed the search down to the following tools: 

Travis CI - https://travis-ci.org/

drone.io - https://drone.io/

CircleCI - https://circleci.com/

DockerHub - https://hub.docker.com/

Shippable - http://www.shippable.com/

Snap-CI - https://snap-ci.com/

Initial Confusion

After some reading around I hit the first hurdle: the difference between supporting Docker builds and being Docker based. Drone for example runs all of its builds inside Docker containers so it was all over my Google search, but it took a bit of digging around before I realised that Drone builds do not support Docker commands. For me this was not initially clear and cost me a couple of hours. Admittedly though, I could have gleaned this from the documentation more quickly if I had been paying more attention.

After banging my head on Travis-CI for a while I found this post explaining that again indeed, the builds can run in a container but cannot themselves call Docker commands. I then moved on to Drone.io and had the same experience. From my point of view I don’t really care if my build ends up running on a VM or in a container as long as I can do the following:

• Set a specific java version (1.8)
• ./gradlew build
• docker build …
• docker push ...

So I was very pleased when I hit CircleCI.

A new hope

CircleCI supports all of the above and getting it set up is very simple. You log-in using your Github account and then select the repositories that you would like to build. CircleCI will try to guess your build configuration which it did incorrectly for my repository so I had to edit the build configuration myself.

As with many of these hosted CI tools this is achieved by including a configuration file in the root of your repository (in this case circle.yml.) The configuration file allows to you tune various environment settings (java versions, docker daemon etc) and also to define the specific build commands that should be run.

After an hour of figuring out the syntax the build was working and my job was done and to my surprise even the free version of CircleCI was very fast. My only criticisms are that 1) the UI, although beautiful, felt a little unintuitive at first and 2) the Docker version seems to be fixed at a custom build of 1.4.1 which makes me wonder about compatibility issues when the gap between CircleCI and latest (at time of writing 1.6) becomes too large.

The best of the rest

Above I said that my investigation would stop when I found something that worked, but I was too curious so I did a little more digging and arrived at the following rough comparisons. I didn’t dive into the rest in any great detail, so if you’re a developer on one of the projects below and I have misrepresented your product somehow, please let me know and I’ll be happy to update the post.

Conclusion

There’s a lot of action in the hosted CI space right now and I hope that some of you find my quick delvings into the area useful somehow. I would imagine that the market will converge to a standard set of features relatively soon and that those features will include Docker functionality.

What I find more interesting though is the other side of the build/deploy wall. Now that an increasing number of companies are using containers as their vehicle of software deployment I expect to see growth in tools that start from the container registry and deal with deployment challenges, which are more interesting by definition because they have to deal with time as an axis in the problem space.