Drone is a continuous integration (CI) tool, a tool whose aim is to automate the building, testing and deploying of software. There are many CI tools available, with a lot of feature overlap. A comparison will not be provided here; an entire post could be dedicated to a comparison.
Apache Mesos is a general purpose scheduling framework that abstracts the idea of processing power. An application would simply request that it requires a certain amount of resources and Mesos would run these applications on a series of slaves.
This project is a work in progress and this is the first in a series of posts documenting our development. In this part the goal is to gather all the dependencies and produce a simple working example of Drone working on a Mesos cluster.
The goal is to build install and run Drone on Mesos. In this part Drone is simply an application running on Mesos, not a framework. This will come in part 2. The tasks for this part are:
- 1. Create a mesos cluster
- 2. Build Drone
- 3. Setup repository
- 4. Setup Drone
- 5. Run Drone standalone
- 6. Run Drone in Mesos
1. Creating a mesos cluster
The first task is to create a mesos cluster. We found that the getting started guide from Mesosphere was the most comprehensive tutorial. Please see here.
Depending on where you run your cluster (e.g. on AWS? Locally?) the key was setting the hostname of each server. If using the Mesosphere method of installtion, you can simply add the public hostname of each server to /etc/mesos-slave/hostname for the slaves and /etc/mesos-master/hostname for the master. This will allow you to inspect the logs from the Mesos gui.
2. Building drone
Drone is currently under heavy development. The 0.4.0 branch has the most recent code but is unstable. An older version, where the bulk of the development was performed in April 2014, is available on the master branch. The aim of version 0.4 is to allow plugins for the backend database, building, queuing and deploying. The prospect of this amount of flexibility makes it a perfect choice for Mesos.
Before you build, make sure you have the most recent version of the 0.4.0 branch. Older versions would not build. To build, you will need Go 1.4.2 and the usual build-essentials-like package for your distribution. I would recommend building on Ubuntu. I would not recommend trying to build on a Mac.
To build, ensure that the drone repository is in the correct GOPATH structure. I.e. go/src/github.com/drone/drone. Then run:make deps
make bindata
make build
This will produces two binaries in the bin folder. The drone binary is the server and the drone-agent is the optional remote builder (more about this in part 2).
Finally, you will need to install Docker in order to run builds.
3. Setup repository
I would recommend that you use github.com for testing. Drone should work with many providers, but we have only tested with github.
Follow these instructions to allow Drone to access your github account. You will require a client ID and a secret which you will enter into the Drone settings file (drone.toml).
4. Setup Drone
The Drone settings are provided by a toml config file. This should be passed into the Drone server binary as a command line argument. Our config file looks like this:
[server]
addr=":80"
cert = ""
key = ""
[session]
secret = ""
expires = 0
[database]
driver="sqlite3"
path = "/etc/drone/drone.db"
[docker]
cert = ""
key = ""
nodes = [
"unix:///var/run/docker.sock"
]
[service]
name = "github"
base = "https://github.com"
orgs = []
open = false
private_mode = false
skip_verify = true
[service.oauth]
client = "YOURCLIENTID"
secret = "YOURSECRET"
authorize = "https://github.com/login/oauth/authorize"
access_token = "https://github.com/login/oauth/access_token"
request_token = ""
[agents]
secret = ""
Please add your github credentials to the file. Note that this config is different to their current documentation. The changes include:
- Adding: driver="sqlite3"
- Changing expires to an int: expires = 0
- Only using one docker host
5. Run Drone standalone
First, I would run the drone server binary as a standalone application. It is easier to debug issues when standalone. To run:
/path/to/bin/drone --config=/path/to/config.toml
6. Run Drone in Mesos
To run in Mesos, we submitted our job through Marathon. You could simply run the command, although we decided to make life difficult by running Drone inside a Docker VM.
For this method, you must create a Dockerfile and a supervisord.conf file. Supervisord was required so that the Docker VM would start the Docker daemon, inside the Docker VM.
drone/drone/Dockerfile:
FROM ubuntu
ENV DRONE_SERVER_PORT :80
RUN apt-get update && apt-get install -y wget build-essential
RUN wget https://storage.googleapis.com/golang/go1.4.2.linux-amd64.tar.gz
RUN tar -C /usr/local -xzf go1.4.2.linux-amd64.tar.gz
ENV PATH=$PATH:/usr/local/go/bin
ADD . /go/src/github.com/drone/drone/
WORKDIR /go/src/github.com/drone/drone
ENV GOPATH=/go
RUN apt-get install -y libsqlite3-dev sqlite3 gcc git
&& make deps bindata
&& make build
RUN wget -qO- https://get.docker.com/ | sh
RUN apt-get -y install supervisor &&
mkdir -p /var/log/supervisor &&
mkdir -p /etc/supervisor/conf.d
ADD ./supervisord.conf /etc/supervisor/conf.d/supervisord.conf
EXPOSE 80
ENTRYPOINT ["/usr/bin/supervisord"]
drone/drone/Supervisord.conf:
[supervisord]
nodaemon=true
[program:docker]
user=root
autostart=true
autorestart=true
command=/usr/bin/docker -d
redirect_stderr=true
stdout_logfile=/var/log/%(program_name)s.log
stdout_logfile_maxbytes=10MB
stdout_logfile_backups=10
[program:drone]
command=/go/src/github.com/drone/drone/bin/drone --config=/go/src/github.com/drone/drone/config.toml
Then build the Dockerfile, from the drone/drone directory, with:sudo docker build -t drone .
Finally, submit the job through the Marathon GUI using the following command (CMD):sudo docker run --privileged --name drone --publish 80:80 --rm drone
Priviledged mode is required so that Docker can instantiate docker.
Hopefully, you should have an app in marathon:
An active task in mesos:
And the Drone UI!
What's next?
After that rather painful experience, you might wonder why we bothered. We learnt a lot about Drone and about why a Mesos framework is necessary. Using a framework with pre-built binaries will make this process far simpler for the end user.
In the next part, we will develop a simple Mesos framework for Drone.
PS. Notes
Note that a lot of the Drone UI is broken.
Also note that you need to add a .drone.yml file to your repository and manually add the repository to drone. See https://github.com/arjang/drone-test for a quick example or the documentation from Drone.
This work is sponsored by Cisco building Microservices infrastructure.
Previous article
