Programming Languages for Microservices (and Containers)

I'm not a programming language elitist. I'm not going to tell you that you're doing things wrong by using Java or Kotlin or Fortran or Eiffel or $insert programming language here$. It's highly unlikely that the choice of programming language will make or break your business. Most of the time you should be using the language you and your team are most comfortable with.

But some languages are more effective in certain contexts. Using a more appropriate language won't fix a bad business plan, but it will give you and your team an edge. In recent years I've been working with containers and microservices, and I'd like to share my thoughts on what makes a more effective language in this context.

Speaking the Language of Microservices

First, what do I mean by microservices? I'm talking about an architectural style where a system is composed of multiple fine-grained distributed co-operating components (for a better definition take a look at Martin Fowler and James Lewis' article). Less accurately, but probably more usefully, I'm talking about containers (typically Docker) running in a cluster (typically orchestrated by Kubernetes).

In practice, the most important characteristics of a running microservice architecture are ephemerality and scalability. By ephemerality, I mean that individual instances of microservices (containers) will come and go over time. This can be for various reasons (scaling, updates, node failure etc), but the end effect is the same — your containers will die. Scalability (more accurately horizontal scalability) means that if you need to service more traffic, your microservice can easily be scaled up simply by adding more instances.

In order to make the most of these characteristics, microservices should:

• Start up quickly. Microservices can scale much quicker if they can boot new instances in a few microseconds.
• Shut down cleanly. They should listen for a shutdown signal from the cluster orchestrator (Kubernetes) and respond fully and appropriately. This means handling any open connections, flushing caches and logging out events before exiting.
• Be resilient. Individual microservices should cope with transient errors, possibly through graceful degradation or back-offs rather than crashing. Health checks should be used to deal with unresponsive instances automatically. At the larger system level, techniques such as timeouts, throttling, bulkheads and circuit-breakers will be required to avoid cascading failures and outages.
• Have a small image size. Containers that have a smaller footprint in terms of on-disk size can be distributed to nodes much quicker. This can significantly reduce start-up time.
• Have a small memory & CPU footprint. The smaller the footprint, the more containers will fit on a node, making it cheaper to run and easier to scale.
• Be internally stateless. Any data and state should be stored in an external service (such as a database). This means that individual instances can be fungible; there are no differences between separate instances of a service and definitely no need for such horrors as sticky-sessions.

Better Language Choices

Most of these do not come for free; whatever programming language you use, it will take some thought and design to achieve them in practice. That being said, I have found that some languages work better than others. My advice:

• Avoid VM based languages. By this I primarily mean the JVM, but this is also relevant to the BEAM. So no Java, Clojure, Erlang or Elixir. VMs have a lot of drawbacks in a microservice-based world. They are slow to start up and use a lot of memory. Also, they are designed to handle large (monolithic) applications, and come with lots of features for handling memory and CPU and threads. These features have a certain tension with the cluster orchestrator itself, which has an overlapping set of responsibilities.
• Use a language that can create static binaries. As well as reducing image size, this reduces reliance on large base images that need constant updating lest they pose a security risk.

Also, a little more controversially (or at least more dependent on context):

• Use something fast. Life gets easier when a single instance can handle a lot of traffic and responds snappily to requests. There will be less scaling and more throughput. Python is one of my favourite languages and is great for a lot of tasks, but if you need to write something that's going to handle a lot of traffic and scale, I'd pick something else.
• Use a language that makes it easy to create correct code. The harder it is to make mistakes, the more reliable your service will be. Yes, I'm saying avoid C unless you really have a good reason to use it.

Rust Never Sleeps

From this, you can probably see why a lot of people use Go for microservices. It's reasonably fast, can produce small, static binaries and is garbage collected. But I'd also like to suggest an alternative that has been gaining a lot of traction recently: Rust.

Rust beats Go for speed (hands down, Go isn't really that fast), can make static binaries easily and has generics (forgive me, I couldn't resist that one). The biggest difference with Rust is how it manages memory. It's not garbage collected, instead it has an "ownership" model that ensures all memory is accounted for and prevents various memory faults (full details can be found in the Rust documentation). This makes for a different programming style when using Rust, but one that emphasises safety and immutability. Which naturally pushes programmers towards creating stateless and reliable code that naturally works well in a microservice architecture.

Having said this, I want to reiterate my opening point about language choice. I'm not saying you're doing it wrong by writing microservices in Java or Ruby. If you have a team of Java programmers that have never touched Rust, please don't retrain them all and start a big rewrite tomorrow. What I am saying is that it is worth considering using other languages where appropriate. This can be done in a piecemeal fashion when using a microservice architecture, a technique highlighted in the Deliveroo blog Moving from Ruby to Rust.

Disagree? Got a better suggestion for a microservices programming language? Let us know!

Further reading:

• Heroku created the 12-factor app manifesto around 2011. Most of the principles are still very applicable to microservices.
• Kelsey Hightower applies 12-factor to containers in 12-fractured apps
• Some Microservice Performance Patterns for services operating at scale.