Goodness, have I read and written a lot about platform engineering so far this year. Platform engineering is the sociotechnical discipline of crafting, building and combining all the common tools needed to help most or all your DevOps teams move faster in delivering business value. And there’s a lot out there in every medium about what platform engineering is, and how to build an internal developer platform (IDP). But, as always with trends folks rush to tick off, mistakes have been made. Today we dive into where you are (likely) missing the mark in creating an IDP that your developers actually want to use.
What is an internal developer platform?
Before we can start talking about what isn’t part of an internal developer platform (IDP), it’s best we have a baseline definition. That’s obvious, you’d assume, it’s a platform. Well, that’s not entirely true. It could be a single platform, an amalgamation of tools, best practices and standards, an open source framework, or a mixture of some or all of these things. Kind of like Netflix’s "Guardrails not Gates" applied to a specific part of your stack, your IDP is whatever is needed to create a "Golden Path" so you help the most developers do the least amount of repetitive work, usually around infrastructure, security, observability, compliance and operations.
I like to analogize it to Dorothy and her friends—the Yellow Brick Road is the recommended way to go, but, if your team decides to venture into that poppy field of experimental tech, they are responsible for maintaining it. Now, if you discovered a particularly effective kind of poppy tech, and this becomes a best practice adopted by your whole Oz organisation, then the platform team will likely, eventually build it into the IDP structure and take over responsibility for maintaining it.
In truth though, the platform is really the least interesting part of an IDP. The internal developer part is the crucial bit. The discipline of platform engineering is focused on your internal developer customer. That doesn’t make it an easier audience to sell to—but it should make feedback loops shorter, if done right. The internal developer platform is often serviced by the platform engineering team. Being a nascent discipline, there’s not just one definition, but we can learn from the perspectives of many.
"Platform engineering is the discipline of designing and building toolchains and workflows that enable self-service capabilities for software engineering organisations in the Cloud Native era. Platform engineers provide an integrated product most often referred to as an ‘Internal Developer Platform’ covering the operational necessities of the entire lifecycle of an application," explained Humanitec’s Luca Galante.
Gartner’s definition is that “Platform engineering is an emerging technology approach that can accelerate the delivery of applications and the pace at which they produce business value.”
Our own Cari Liebenberg calls the developer platform another pattern to speed up delivery, whilst her colleague Charl Klein says “It’s the tooling and processes that you put in place to allow developers to deliver their code to an environment that equals business value at the end of the day.” But he doesn’t actually refer to it as platform engineering so much as platform enablement. Both are engineering leads at Container Solutions.
Basically, the internal developer platform is the most tangible outcome of platform engineering.
So now, what is not an IDP?
The IDP is not the death of DevOps
But it could be the greater fulfilment of it. Admittedly there’s this rumour going around that DevOps is dead, long-live platform engineering, but we’re inclined to think of it more as natural progression than death knell.
This isn’t a little trend either—2023 is emerging as the year of platform engineering, as Independent Technology Sam Newman said on our recent webinar, and Puppet has dedicated its entire annual State of DevOps report to the topic. Again, that doesn’t mean platform engineering is replacing DevOps, it’s just building on it.
“Platform development has arisen, because, with the expression of DevOps, there was this phrase that everyone was using 'You build it, you run it.' And then development teams who had a single core skill like needing to learn this coding language and maybe some framework and then become an expert in that suddenly had to learn a bunch of new things,” Liebenberg remarked.
Suddenly DevOps teams had to learn a full stack of skills, from Terraform scripts to YAML files to the whole Kubernetes orchestrator. Yes, everyone should keep security awareness in mind, but should every DevOps team be in charge of enforcing it? This responsibility became another cause of the burnout and cognitive overload that’s endemic to our industry.
Syntasso’s Abigail Bangser, who will be speaking at our WTF is SRE conference in May, calls this "non-differentiating but not unimportant work”, that’s typically repeated across an organisation. Therefore, to her, a platform team "produces and operationalizes tools to help other teams produce and operationalize their software applications to their end users". It makes sure these DevOps teams stay focused on delivering value to end users, not all that stack complexity.
The core values of DevOps—focus on flow, shortening feedback loops, and continuous experimentation and improvement—are still highly applicable, and especially should be applied when building an IDP.
An IDP alone won’t work
An IDP is not an "If you build it, they will come” situation. Like all sales and customer success, the most common point of failure is not working with your customer early and often. The customers are your internal development teams. You cannot just build a platform and then bequeath it to them.
“A platform is not a bunch of Terraform scripts in a folder without documentation. It's not a pipeline where nobody has any idea what it actually does. That almost assembly-line style thinking where the developer is expected to be able to produce and then just leave a piece, but they're not actually enabled to deliver they pretty much become a cog in a machine not essentially being able to expand that to be more than just that cog is not what we want”, Klein warns. An IDP is about enabling the engineer as a creative worker, not reinforcing code monkey stereotypes.
A tool without users is an utter waste of money. An important remit of a platform engineering team is developer advocacy within an organisation. Not only to gain essential feedback in building the product, but in compelling your users to actually use it.
For a platform engineering programme to be successful, it has to offer transparency to the developer so they understand why they are doing something and how it connects to business value. And, of course, do it in a way that business stakeholders can see the value driven from it, too.
It also has to aim for self-service with the right IDP strategy. Like all good developer experience, that means good, clear, and well-maintained documentation is crucial, including clarifying when something is a multiple-choice decision—like deciding between three approved programming languages—versus what is mandatory—like updating security patches on your open source software. These docs should probably be longer than your external ones because they outline technical and access-based decision making and should clarify things like if the platform team runs all the security checks or if they just provide the tooling and the application teams are meant to use them. It should include a self-service catalogue.
An IDP is not one and done
You don’t want to end up with another monolith, but rather you want to continuously iterate, releasing small additions to your platform.
“When we internalise that building a platform is no different than any other product, the mistakes become more obvious. For example, attempting to build a complete and refined product before releasing it for feedback from customers leads to poor adoption. Feature development must be managed alongside bug fixes and investments in reliability and security,” Bangser told WTF is Cloud Native. “While Platform-as-a-Product is gaining popularity, the fact that both the creators and the users are engineers can cloud judgement and lead to the product team making dangerous assumptions.”
This really ties to the previous step. A platform is a living, breathing artefact because it changes along with your internal developer customers’ needs. This is why she warns against automating everything, as your developers may want or need more flexibility or only want to use part of your platform the first time.
"A platform codifies business practices. Automation codifies understanding,’ Bangser continued. "While an end state goal may be to automate a platform, it is much easier to focus on identifying, refining, and codifying a business process without the assumption that it is ready to be fully automated. A great platform can naturally allow for offerings to evolve from early prototypes through to mature automation."
That’s why she recommends to not gear up to inflict a mega-platform upon unsuspecting developers, but to instead kick off your platform engineering journey with what Team Topologies refers to as the Thinnest Viable Platform, like creating a Wiki page, and then continue to aim to keep it the thinnest viable.
Liebenberg calls this aiming to build a platform based on empathy: "Being able to own your Software-as-a-Product team becomes a lot more viable when you have a well-running, empathetic platform."
An IDP does not automate developers out of a job
Yes one of the goals of an internal developer platform is to automate the hum and drum out of developer lives. So, in these increasingly troubling economic times — or at least troubling tech headlines — that could be interpreted to mean do more with less. Which could be easily stretched to mean, automate development teams out of jobs. Don’t do that.
Instead, in times of staff cuts, platform engineering serves as a great way to support remaining staff, by decreasing cognitive load.
"I think reducing that cognitive load on the development teams that [have] already been pushed to extreme hours can only be good for the developers and the development team, and, in general, company health, and quality of life and mental health," Klein remarked. And helping them focus on their differentiators, not the cloud and Kubernetes which are far outside their job description, help increase their confidence in their ability to do their jobs well.
Again, this is where empathetic platform enablement comes in. For example, imagine "We’ve got 10 teams, solving 20 things each. But if we have some common elements, and take 10 things off their plates, and it’s actually collectively 10 things off everyone’s plate”. Leibenberg emphasises that platform enablement should focus on common pain points among DevOps teams. Not making up excuses to fire people.
What else is the IDP mistaken for?
Let’s just do a quick countdown of the top things an internal developer platform should not be:
10. An IDP is not Kubernetes. Nor Kubernetes control planes. Yes, the complexity of Kubernetes has driven an urgency for platform engineering, but the platform should do and serve more.
9. The platform isn’t "Ops for Developer code’. The platform team is responsible for the availability and user experience of the platform, but not for the developers’ deliverables. Plus that team should help the full DevOps team do its job, not just the operations folks.
8. An IDP is not a purely technical solution. Note the word "sociotechnical” at the start. An IDP, like most things really, involves as much people and processes as it does tech.
7. An IDP is not a single vendor game. Of course, we’ve seen a lot of platform products on the market over the past year. But, Bangser warns, "control of the platform’s API and user experience needs to live with the organisation and not the vendor”. She’s not saying vendors don’t have a role to play, but warned that "lock-in comes from pushing out the vendors as the experience”.
6. An IDP is not about the shiny-shiny. Engineers generally love to experiment with new things. The platform isn’t about that, yet it’s built by engineers. Be sure you’re focusing on proven best practices, not experimenting with new tech nobody uses in your platform.
5. An IDP is not just visualisation. IBM Product Manager Lee Ditiangkin recently put it best with: "Put a pane of glass on a pile of sh*t and all your developers can see is a pile of sh*t.” Things have to be restructured for self-service, giving teams more than just transparency.
4. An IDP is not serving a single kind of developer. You can’t have an Ideal Customer Profile when your job is to serve the entire engineering department. You need to work with the different teams to do your research up front and find common pain points, but enable developers to access your IDP via an API so they can decide how they’ll use it.
3. An IDP doesn’t have to do it all. Especially at the start—and it probably never will be a finished product as demands and best practices evolve. Start by looking for your most common issues across different teams or ask them their bottlenecks. Find patterns to identify your own company’s own low-hanging fruit—it won’t be the same for all teams.
2. An IDP isn’t about building a fancy UI. Founder and CEO of Humanitec Kaspar Von Grunberg recently drove this point home that, while they have a role to play, organisations often get distracted focusing on developer portals and service catalogues, which leads to developer animosity, providing few tangible benefits, and adding to instead of decreasing complexity. Instead, he writes, focus on building a Platform-as-a-Product.
1. An IDP should not be built in a silo. Your platform won’t be adopted unless you work hand-in-hand with your developer customers. This could be borrowing application developers to help build the platform or having platform engineers embed on the app teams or just frequent demos and feedback sessions.
Still uncertain of where you want to get started building an actually used and useful internal developer platform? Go on, bribe your developer colleagues with pizza and start asking them what they need!
Previous article
