The Cloud Native Computing Foundation (CNCF) has announced that External Secrets Operator (ESO), an open-source solution for synchronising secrets from external APIs into Kubernetes, can begin incubating as an early stage project in the CNCF Sandbox.
Created in November of 2020, ESO is a Kubernetes operator written in Go and licensed under Apache version 2. It integrates external secret management systems including AWS Secrets Manager, Azure Key Vault, Google Secrets Manager, HashiCorp Vault, and many more, reading information from external APIs and injecting the values into a Kubernetes secret.
ESO has garnered considerable interest from developers and organisations and has a healthy, rapidly growing community. Over 130 people have contributed to the project, including engineers from Container Solutions, EA, GoDaddy, Digital Ocean and Form3. The operator has also been adopted by Form3, Amadeus, Mixpanel, Pento and a number of other companies.
This level of interest isn’t surprising. Containers require access to sensitive information, such as database passwords, to perform basic operations. Kubernetes secrets provide a mechanism to store this kind of information in a central repository called etcd. However whilst this is more secure than using the pod definition or a container image, secrets are by default stored unencrypted, and Kubernetes doesn’t have the capabilities needed to manage their lifecycle. In view of this, it is very common for organisations to use more robust third-party secrets management tools.
Given the challenges involved in managing secrets in Kubernetes it is perhaps unsurprising that there were a number of forerunners to the ESO project. The most popular was the now deprecated Kubernetes External Secrets from GoDaddy. According to Moritz Johner, a Senior Software Engineer from Form3 who took the lead getting ESO off the ground, GoDaddy's project unfortunately hit something of a dead end. “One issue was that it was written in Node.js, which isn't the Kubernetes native language per se, and so the client SDK you had to use just wasn’t up to the standard of the Go one.” Because of this, a decision was reached to create a brand new codebase, building on ideas from the GoDaddy project and others.
By joining the CNCF, the ESO contributors will gain additional contacts into the Cloud Native space, and it should help more potential users to discover the project. The CNCF also provides access and lends credibility in the regulated sector. “A lot of my background is in financial services,” Johner told me, “and I always see the problem of how to implement secrets management in a safe way. The most important factor is how you can trust a project or tool that handles your crown jewels. The CNCF helps with providing trust, but more importantly can help with the expertise behind it as they have more access to banks and other financial institutions. I’d very much like to see these highly-regulated companies feeling able to recommend our product,” Johner said.
Alongside clean-ups to make the ESO custom resource definition easier to use, the next major feature on the roadmap is to have the ability to push secrets to other clusters and other secret providers, something being worked on by engineers from Container Solutions and EngineerBetter. In addition, “We’re aiming to get the APIs stable and release a GA version by the end of the year,” Gustavo Carvalho, a Cloud Native Engineer at Container Solutions and major contributor to ESO, told me.
You can find more information about ESO from the official project website. Container Solutions has released a free eBook (registration required) which includes tutorials for the major providers, and the project source code is available via GitHub.
Previous article
