MesosCon Europe in Dublin is in full swing as we announced our Mesos framework testing tool, MiniMesos. We launched a new website at minimesos.org and an online Twitter identity at @MiniMesos. This blog introduces MiniMesos, explains the background of the project, what is currently implemented and what's up ahead. MiniMesos is a fast moving project so in the coming days and weeks there will be several blogs that cover different parts of MiniMesos as it is being developed further. Topics will range from showing features to how we implemented certain things to the philosophy behind the project.
Using MiniMesos
MiniMesos is a Java-based tool to spin up Mesos clusters running in Docker. See the following asciinema
The story behind MiniMesos
The story begins when we were starting to work on the Mesos Elasticsearch framework. Developing a Mesos framework is not a trivial task and we found that the feedback loop was very long. The15 minutes it took to build, package, upload and run the code, only to see the dreaded TASK_LOST, made development a bit frustrating. Initially we used a Docker Compose setup to spin up a Mesos cluster. This worked to a degree but it was it lacking one feature: it was not programmable. What we really wanted was a programmable Mesos cluster that we could spin up and destroy at will. This not only speeded things up but allowed us to do TDD and automated systems testing.
Thus MiniMesos was born. MiniMesos is written in Java and uses Docker to run Mesos. The 0.2.10 version currently used by Mesos Elasticsearch runs Docker-in-Docker. Mesos Elasticsearch itself runs in Docker although Jar support is underway. Because of this we had to build in features to transport the executor images inside MiniMesos. This is done via a registry container. We are currently working on adding Jar support to Elasticsearch and MiniMesos so that jars will run inside MiniMesos as they are fetched from the scheduler. During these changes we encountered some interesting challenges which we will blog about soon.
System testing with MiniMesos
As said before, Mesos Elasticsearch uses MiniMesos extensively and we have written system tests for every major feature. A system test performs end-to-end checks on a given feature. For instance, we have a discovery system test which spins up MiniMesos, deploys Mesos Elasticsearch, checks if all nodes can find each other via Zookeeper and then destroys the cluster. The same is done to test many of the config options. The major benefit of this approach is that we can catch hard to find bugs that occur due to changes in the code that are hard to detect with unit tests.
What's next?
The goals of MiniMesos is to make framework testing and development easier. The major features we are working on is moving to a flat container structure together with Jar support. To do this we had to make a few changes as well as contributions to the underlying Docker-Java project. Alex will talk about this in an upcoming blog on MiniMesos. In the future we like to make it easier to install frameworks, improve the cli and add a remote API so MiniMesos can be used from languages other than Java.
We like to hear from you!
Try out MiniMesos, give it a spin and if you have any comments get in touch through @MiniMesos or @ContainerSoluti.
