Accidental Rewrite of a Microservice

This is a guest post by Robert Hensing.

In this blog post, I describe my experience adapting a microservice for use with AWS' DynamoDB.

Background

The Sock Shop is Weave Works' reference microservice implementation. For their customers, they want to demonstrate that their cloud solutions for deployment, monitoring and more work well on Amazon Web Services.

Therefore the task is to adapt one of the services, the orders service, to use Amazon's DynamoDB instead of MongoDB.

Spring Boot

The original orders service was created with Spring Boot. Spring Boot is built on the Spring framework. Whereas Spring provides you with a Swiss Army knife for calling your setters for you, AOP wrappers, et cetera, the Spring Boot library focuses on ‘configuring’ your application automatically.

That's easy when you're setting up a new application or microservice. How I wish it made things simple.

My task was to introduce a new persistence back end, which I started out with happily, knowing that using dependency injection makes this possible. The service was already using Spring after all.

As you must have guessed from the tone and the title, things did not go so well. I ran into a series of problems, most of which could be worked around.

Spring Boot's autoconfiguration, which is triggered by the presence of classes on the classpath, had to be configured to ignore some ‘autoconfigurations’ depending on the runtime configuration. That turned out ugly, with a ‘missing bean factory exception’. Apparently the bean factory, basically a big bag of objects like database connection pools and such, was not quite there yet.

Build with Docker

Docker has some potential for making projects build a bit more reliably. That did not quite work out for this project though. A build script was in place that compiles the project inside a temporary docker container, but it reused the host user's Maven repository (package cache), which is bad for accurate reproducibility, and, worse, it writes root-owned files there, locking out all tools that run on the host. No-one noticed that before, since I was apparently the first one to work from a Linux host. I have fixed it for this project.

The Problem

After a couple of days of trial and error, I hit upon a big problem. Apparently the Spring Data component for DynamoDB, which is just a community module, does not have special support that is required for Spring HATEOAS. This, combined with the ugliness that I had to introduce into the configuration, and the risk that the annotations on the domain classes will be inappropriate for one of the persistence back ends, made me decide to propose a rewrite of the service.

The Rewrite

For the rewrite, I made the following decisions:

• No Spring Boot, because while it is easy to set up, it is harder to maintain
• Keep the Spring Framework: future maintainers are likely familiar with Spring and the conceptual model of the dependency injection part of Spring is simple enough
• Use seperate representations for persistence and presentation
• Use Scala as a ‘better Java’: although Scala can be used in mind-bending ways, simple use of Scala is very similar to Java, which many are familiar with, and promotes good practices like immutability. Having a short syntax for declaring data relieves most of the pain of having more than one representation for similar concepts. Using ‘advanced’ functional programming techniques is unnecessary and possibly harmful for this service, considering that neither Container Solutions nor Weave Works are functional programming shops.
• Stick with Jersey for the API, to keep it accessible to Java programmers. Scala has solutions of its own, but they can get complicated.
• Use an embedded web server (a web server library)

Choice of web server

There is a wide range of HTTP server implementations for the JVM. I have considered the following:

• finagle-http a high performance asynchronous web server. Although it looks very promising, I decided that it is probably too complex a solution for this simple microservice.
• undertow an asynchronous web server backed by Red Hat. Its async interface did not add any value, but did slow me down setting up Jersey on top of it.
• Jetty: it may be old but does what it does without surprises: the final choice.

As it turns out, Jersey, the library for providing the API comes with its own dependency injector. That was one of those scary moments, but luckily you can also construct the object graph of endpoints and mappers and such by hand.

Testing

For testing I have re-used the Dredd-based test suite of the original service. Sadly I did not have the time to look into the full details of the framework. It seems to be like doctest, running the examples from a piece of documentation, but that did not seem to be quite the way it was used in the original orders service, which relied a lot on hooks.

I have decided to skip unit testing for this service, because the bulk of the code is about integration and there is basically no risk to be covered in this demo app.

Swagger codegen

I have tried to use the swagger specification to generate API scaffolding or to derive an interface for the code to implement, but the swagger code generator failed on the spec when generating Java code, with a NullPointerException, probably because that is what Jackson returns when something is missing. I would have fixed the API spec, but not if that means debugging maven first. The Scala code generation did succeed, but was not very usable with my Java-esque approach. I was able to reuse the generated Scala classes for scaffolding, which was nice.

Still missing

At this point the API is not entirely compatible with the old version, but it is sufficient for the front end.

Support for asynchronicity and parallellism would be nice to have for performance. Async programming is pretty nice in Scala, but the integration with the Java libraries is probably not as clean. It is a bit sad that the JVM needs asynchronous programming for performance.

Conclusion

Looking back at the effort, I can conclude

• An abstraction like Spring Boot is hard to tear down once it does not suffice any more.
• It is preferable to use a system with clear and consistent semantics,
  • Example: a programming language
  • Counterexample: heap of inconsistent mutable components
  • But often there is no choice
• Autoconfiguration is more convenient than scaffolding, but scaffolding is easier to change

Besides the technical stuff, I would like to conclude it was nice to work for Container Solutions. I got in contact with them through their meetups, got talking and discovered that I could do some interesting stuff for them alongside the other things I do. It is quite a young company with a positive, balanced atmosphere.