In the last few days my colleagues and I at Container Solutions have made major releases for several projects. We’ve been busy developing a range of products that are evolving into a suite of applications that can significantly improve the efficiency of writing frameworks for Apache Mesos. In this post I will show you why and how you can use:

• a custom framework using Mesos-Starter
• a generic framework using MesosFramework
• an Apache Mesos framework for Kibana, and
• testing with minimesos.

Mesos-Starter

Mesos-Starter is a Spring module intended to remove as much Mesos boilerplate as possible. We’ve written about Mesos-Starter before. But this release marks the first time that the project contains enough features to be useful in the real world. Below is a review of some new features.

• An exposed bean to control the instance count (to provide horizontal scaling capabilities).
• Managed ports. Users are now able to request static or Mesos specified ports and use them as environmental variables.
• Configurable docker network types
• Mesos authorisation
• Users can specify URI’s for downloading binaries and assets to the task’s sandbox

These new features now allow frameworks ultimate flexibility in the deployment of tasks. For example, when using the Docker Containerizer on minimesos, it is important to run in Docker bridge mode to prevent port clashes on the host (all agents use the same docker daemon). But in real life, we want the containers to run in host mode, so that they take the ip address of the host. These configuration options make these distinct deployment tasks possible.

MesosFramework

MesosFramework is an distributable Spring-based binary or Docker container based upon Mesos-Starter. It enables users to instantly create a framework for Apache Mesos with a single configuration file. Users simply tell MesosFramework what it should run as its task. It is generic enough so that users are in full control over the deployment.

We’ve talked about why you might want to write a Mesos framework in the past and I would hope that now you would consider using Mesos-Starter. And many simple applications can be started via an orchestration tool, like Marathon. But there are some use cases that lie somewhere in the middle of these two extremes. For example:

• You want to take advantage of the amazing features that are contained within Mesos-Starter, but you don’t want the burden maintaining any code. (e.g. authorisation, a particular orchestration strategy, etc.)
• You want to package a specific framework configuration, so that your users don’t need to worry about configuration (again with no code).
• You want to quickly deploy a POC for a framework.

For us, we have a number of frameworks that lie within this grey area. The key benefit is to reduce the amount of code we have to write and therefore reduce the amount of maintenance we have to do to to improve or fix these frameworks.

MesosFramework also adds a few opinionated features on top of the standard Mesos-Starter. For example, REST endpoints for task health and horizontal scaling.

Kibana framework

The new Kibana framework is based upon MesosFramework and Mesos-Starter. It inherits all of the features of the upstream projects. This means that any new features or bug fixes will be automatically included in the downstream projects like Kibana. This must be heaven, new features with no new code!

But the most remarkable part about the framework is indeed that; there is no code. This shocked a few colleagues when I merged a PR that deleted 2500 lines of their code. A similar refactoring is also going on for the Logstash framework. But the point of doing this is an important one. Mesos frameworks contain an inordinate amount of boilerplate code. Every framework on the market adds the feature “Mesos Authorisation” in exactly the same way. Engineers all over the world are needlessly duplicating code. And as we all know, more code == more bugs.

By removing Kibana specific code, we’re mitigating against potential bugs, adding new features for free and reducing the amount of maintenance we have to perform.

Minimesos

I would also like to acknowledge the leaps forward that minimesos has made recently. We’ve just written a blog post announcing 0.7.1. These new features are paving the way towards something really exciting; the prospect of making minimesos not so mini. But in the short term, the ability to provision the cluster via a configuration file has significantly increased its usability. Previously, I used to use a live Mesos cluster for all my demos. Now I only use minimesos. For example, the other day I wanted to test whether the Elasticsearch framework worked on 0.27.1 (because it is compiled against 0.25). In the past I would have had to write scripts to form a live 0.27 cluster, spin it up and test remotely. Now I can change one line in the minimesos configuration file and test it in exactly the same way as I perform all my tests. Thanks minimesos!

Finally

We want to hear from you! If you like or use any of these projects, then please provide feedback in whatever medium you prefer.

And as always, we’re here to help. If any of the subjects in this post highlight one of your needs, then please get in touch via the usual routes.