The introduction of third-generation programming languages (3GL) into mainstream software development during the 1950’s was arguably the point of inflexion for the programming industry, and languages such as COBOL, C and Java have since paved the ways for the development of incredible technologies and platforms. Are we now seeing a similar trend with operational ‘programmable infrastructure’ languages that are being used to automate the modern datacenter? And will this allow us to form cross-functional teams with access to more specialism in the operational space (core operators vs application operators)?
I was watching for the second time Mitchell Hashimoto’s excellent CraftConf talk “Automating the Modern Datacenter” as I was writing a summary of the presentation for InfoQ, and I happened to notice what could almost be interpreted as a throwaway comment by Mitchell, but on second thought, could be very profound. To set the scene, Mitchell (founder of Hashicorp, and creator of Vagrant and Packer) was talking about the varying complexities of tasks that are necessary for automating a modern datacenter and deploying applications:
"This is where I make the distinction between core operators and application operators, in every company there are the operators who understand how to make a highly-available database cluster, and there as those who don't, but want a highly available database cluster. You can teach them to do this, or give them the abstraction [that Terraform provides...]
If we don't start moving up the stack, then we won't be able to do more interesting things.
They aren't that many developers today that know how to write assembly [...] We've gotten past the point where you can write code and be confident that it is going to compile to the right thing. That's where we eventually need to get to with operations"
(emphasis mine) Mitchell Hashimoto, CraftConf 2015 http://www.ustream.tv/recorded/61446237
Mitchell was introducing the Hashicorp tool Terraform, which enables the building, combining and launching of infrastructure efficiently across datacenters and disparate vendors. Like many of the other provisioning tools currently out there, Terraform allows the declarative specification of infrastructure, but at a higher-level of abstraction and also provides cross-vendor support. For example, you declare that an AWS EC2 instance be spun up along with a DigitalOcean Droplet, and Dyn DNS be used to provide access to both.
Now I’m paraphrasing Mitchell here, but I believe he also suggested that there is still a place for tooling such as Puppet and Chef, however, it might be further ‘down the stack’, perhaps complementing instance-level bash shell scripts and SDKs. However, the makeup and flexibility of the modern datacenter requires that we as operators and DevOps engineers focus on the big picture - the organisation’s infrastructural estate - which is now at the top of the stack.
“You can have as much Chef and Puppet automation as you want to set up your application, but if you don’t have an automated way to set up all of the services it needs as well, then what’s the point? Your applications won’t work anyway…”
Mitchell Hashimoto, CraftConf 2015 http://www.ustream.tv/recorded/61446237
This is important, and as much as the software development industry is relatively young, the infrastructure-as-code industry is even younger. Accordingly, we should be clear about how we build, maintain and lead the development of programmable infrastructure. In my opinion, there will always be a need for ‘full-stack’ engineers or T-shaped specialists, particularly at the leadership level, but dividing responsibility across more focused and specialist individuals (working as part of a bigger cross-functional team) can be highly beneficial, both in terms of focus and also accountability.
Perhaps operations tooling and infrastructure languages can be divided like this?:
I guess the main point I’m trying to make here, is that when we started developing with third-generation programming languages, good things happened. We, as 3GL developers, were freed (to some degree) from knowing about the lower-level system details, and this enabled us to think more about other more ‘interesting’ things. Now I’m not saying the someone who codes in a 3GL is ‘better’ than someone who codes in a 2GL - each generation’s developers are simply working at (and are responsible for) a different level of abstraction. Often when responsibility can be divided locally, more can be achieved at a global level.
I believe the same is true for operations, and we should start to embrace these concepts more thoroughly. I definitely believe that languages like Terraform are going to provide many opportunities for developers and operations to focus on different things, but there is still clearly a need for scripting and ‘traditional’ provisioning approaches. The key questions are how do we utilise each level of tooling correctly? Do we build things ourselves, or buy infrastructure modules off-the-shelf? And how do we create effective teams around these skills?
If you want to know more, or get advice on which infrastructure language(s) you should be using on your latest projects then please get in touch with us at Container Solutions. We use most of these tools on a daily basis, and we have been using Terraform recently to good effect. You can read more about Terraform, Mesos and Google Cloud or the creation of a Mesos Terraform module on our blog.