I attended the London Microservice User Group (LMSUG) July meetup last week, which focused on deployment infrastructure for microservice-based systems, and there was some great presentations. A lot of what was talked about relates directly to the technology we are using here at Container Solutions, and so I’m keen to share the event highlights.
Modern applications, microservices and Pivotal CF - Sufyaan Kazi, Pivotal
Sufyaan started his talk with an introduction to the much loved (and often abused) monolithic application stack, and then contrasted this with the current approach to building microservices. Although the benefits of microservices are numerous - modularity based on component services, change cycles decoupled, efficient scaling and the elimination of commitment to long-term a technical stack etc. - there are increased operational burdens. Sufyaan commented that the current trend of using container technology, such as Docker, has only shifted the burden of environment configuration from operators to developers.
Sufyaan suggested that some teams are not ready to react to this operational change, and instead may benefit from leveraging a platform-as-a-service (PaaS) like Pivotal CF. Based on the principles provided by the 12factor app, Pivotal CF allows applications to be packaged and run within Warden containers (similar in concept to Docker).
It was an interesting talk, and it looks like the IaaS/PaaS battle may continue for some time. We chat about CloudFoundry quite regularly with our sister company CloudCredo, and it’s an interesting platform. CF is sometimes viewed as a more niche offering in comparison with the other infrastructure fabrics in this space, such as OpenStack, OpenShift and offerings provided by AWS, Google Cloud Platform and the like etc. We’re also seeing organisations like Capgemini, Cisco and Yelp! developing PaaS-like application fabric using the Apache Mesos cluster manager as a foundation, which is interesting to us as we work closely with the very innovative team at Mesosphere who produce the Mesos-powered Datacenter Operating System.
Creating distributed applications with Weave and the Docker platform - Ilya Dmitrichenko, WeaveWorks
Illya provided an overview of Weaveworks ‘Weave’ virtual network that connects Docker containers without using container linking. Weave is an overlay network with built-in DNS for service discovery, and it allows containers to be transparently moved around a cluster without changing IP address. As Weave operates at Layer 2, it also supports transport level security and access controls etc.
Here at Container Solutions we are working alongside Weave, and they are doing some fantastic work in the container networking and monitoring space. I wrote an InfoQ piece about their new ‘Weave Scope’ monitoring tool, which is a very interesting piece of kit and has a lot of potential - as Adrian Cockcroft often comments, monitoring within microservices is not yet a solved problem. Weave are looking to provide the solution.
Shape your Infrastructure with Terraform - Robert Firek, Codurance
Robert gave a superb overview of HashiCorp’s Terraform, including details of what problem the tool is attempting to solve, and how to use Terraform effectively. Terraform provides a common configuration platform to launch infrastructure across datacenter/cloud vendors, and it also allows change ‘plans’ to be created that allow infrastructure ‘diffs’ to be generated, compared and shared. For example, you can configure the creation of an AWS EC2 instance and a DigitalOcean Droplet, and wire them together using DynDNS. All of these resources are specified using a JSON-friendly HashiCorp Configuration Language (HCL).
Robert discussed how the requirements of a recent Codurance client project required the creation of an environment within AWS. This was a relatively standard environment configuration, with a DMZ, several virtual private clouds (VPCs), and database etc, and Robert commented that creating this via the AWS console was quite labour intensive. CloudFormation goes some way to automate this process by allowing the definition of an AWS environment through XML (‘infrastructure as code’), but this proprietary format is slightly verbose. A colleague recommended Terraform to Robert, and he began experimenting successfully with this. Robert provided a great live coding demo that showed both the power and simplicity of Terraform using AWS, and he commented that although there are still a few rough edges around the current beta release this is now his preferred AWS infrastructure definition tool of choice.
We are big supporters of HashiCorp at Container Solutions, and we’re using many of their products within client work that is deployed to production. I’ve also chatted to Mitchell Hashimoto about deploying containers at scale, and written about his thoughts on ‘automating the modern datacenter with Terraform and Consul’. HashiCorp are already creating some amazing developer and operator tools, and I’m tipping them for (infrastructure) world domination over the next twelve months!
Orchestrating storage for a Docker microservices stack - Kai Davenport, ClusterHQ
The final talk of the night was provided by Kai Davenport from ClusterHQ, and Kai provided some great information on the recent v1 release of Flocker, a tool to allow easy data volume management and state migration within Docker containers and clusters. Kai began the talk by setting some context with a discussion and contrast of the monolithic and microservice approaches to software design and deployment. Kai suggested, that although he could see the benefits of both architectural styles, he believes that care must be taken when developing a monolith, as it is all too easy to add lots of features into a single application:
“...the idea of pouring loads of different features [into a monolith] - it’s like a Black and Decker workbench, and what you really want is a hammer and a chisel and some nails, and you can combine those as you want…”
Kai suggested that if you are using a microservice-based architecture, then container technology is very appealing for packaging and deployment. Kai then discussed that many solutions to container ‘orchestration’ are emerging, including Docker Swarm, Kubernetes, Mesos/Marathon and CloudFoundry. However, the handling of state within containers, and the mobility of storage volume data across the associated host machines within a cluster is often a forgotten problem. This is where the ‘Flocker’ container data volume manager for Docker can provide a solution.
In addition to providing mobility of container state across a cluster, Flocker can also act as an abstraction layer for Docker volumes. Flocker currently supports multiple block-based shared storage implementations, such as Amazon Web Service's EBS, OpenStack Cinder, EMC ScaleIO and XtremIO, and local storage using an experimental ZFS storage backend (the Flocker documentation contains a detailed storage comparison matrix). Kai suggested that using Flocker data volumes will reduce the need for additional work if a developer or operator decides to swap to a different storage device, or change cloud platform vendor.
Kai bravely demonstrated the use of Flocker with Docker Swarm live at the meetup, which was very interesting to watch (instructions to replicate the demonstration can be found on the ClusterHQ Github account). Kai mentioned that Flocker works with Mesosphere’s Marathon Mesos scheduler ‘out of the box’, and Kubernetes requires a patch, which will be made available soon. The prototype Flocker volumes GUI was also demonstrated, and this showed the location and status of the container volume before and after the host migration. You can find more detail of Kai’s talk within the summary I’ve written for InfoQ.
Wrapping Up
This was a great meetup in London, and we were fortunate to learn about CloudFoundry, Terraform, Weave and Flocker, all of which are currently hot topics within the microservice space. In particular, I'm tipping Flocker and Terraform for greatness over the next twelve months. Here at Container Solutions we’re using and experimenting with all of these platforms and tools, and so please feel free to reach out to us if you have any questions, or want to share notes on the technology.
Want to know more about monitoring in the cloud native era? Download our whitepaper below.
Previous article
