The tech industry is littered with acronyms and abbreviations which can sometimes trigger strong reactions. The letters RfP always send me ricocheting between glum resignation and extreme frustration.
Why? Because the request for proposal process highlights how the way we procure and deliver software and infrastructure is essentially broken.
On the face of it, the traditional request for proposal process makes perfect sense. A customer explains its business, details a specific problem it needs solving, and a set of requirements for a proposed solution, and says how it will grade the proposals. Suppliers come back with their proposals for a solution, as well as listing their technical capabilities and financial bona fides.
Ideally, the RFP helps both customers and suppliers reduce risk, by clearly scoping out both the problem and the proposed solutions. And it ensures fair competition, particularly important when it comes to really big projects, both in the public and private world. (Let’s just pretend things can’t be swayed by the occasional fancy lunch or exotic conference venue.)
As the tech world works its way through the paradigm shift that comes with Cloud Native, negotiating the additional risk associated with upsetting traditional ways of doing things, this is all a good thing, right?
Well, it should be, but it’s really not. The shift to Cloud Native simply amplifies the problems of traditional procurement and how they contribute to failed projects.
Just under a decade ago, research from McKinsey found software projects worth $15m or more generally overshot their budgets by 66 per cent and overran their predicted schedule by a third. The only area they didn’t overshoot was in the benefits the projects delivered - there were on average 17 per cent below expectations.
But we should be doing better than that now, shouldn’t we? Well, no it seems. Gartner recently found that 60 percent of infrastructure and ops leaders find themselves hit with public cloud cost overruns big enough to undermine their on-prem budgets.
Budget blackholes and pavement sinkholes
Some might say this just shows that IT and software engineers clearly aren’t engineers at all. I’d say let’s not go there, then I’d point out that two of my favourite project disaster stories are from outside the tech world and focus on traditional civil engineering.
In Holland, the extension of Amsterdam’s Metro Line 52 ran seven years late and 40 per cent over budget, with the city government having to contribute €3.1bn rather than the €317m planned. This has blown a massive hole in the city’s finances, just as the city is facing up to a plague of sinkholes, crumbling canals, and working out how to repair 1,700 bridges. Yes, the trains keep on coming, but so do the city’s other problems.
Over the border in Germany, Berlin’s Brandenburg Airport was 30 years in the making, with seven missed opening dates, 106,000 miles of mis-installed, dangerous cabling, and still has minimal ongoing transport connections. Tellingly, the airport’s plans – presumably arrived at after an extensive RfP process - disappeared after the initial opening was postponed, only to turn up in a dumpster.
This sounds like two cases of really bad luck, but it’s worse than that. Back in 2011 the University of Oxford’s Bent Flyvbjerg detailed how “great planning disasters” such as the Channel Tunnel represented business as usual, with cost overruns on 90 percent of major products, while demand and benefit predictions were typically out by 20 to 70 per cent. How did this keep happening? Traditional explanations based around simple bad luck or error were not enough, he said, and the real reasons were optimism bias and strategic misrepresentation.
So, if traditional RFP processes ever really worked, it’s fair to say they’ve now degenerated into long winded theatre to create the illusion of both competition and the controlling and elimination of risk.
The above examples were public infrastructure projects, so no one is going to go out of business. And much of the scandal centres on the cover ups and denials that followed the original screwups. But once budgets are blown, they can’t be spent on other projects, like fixing bridges or recruiting more DevOps engineers.
If you’re a commercial organisation and you screw up, you might not have time to cover-up, because you’ll be too busy going out of business.
And that’s the problem. In technology today, trying to eliminate risk is the riskiest path you can take.
Traditional approaches can end up encouraging you to focus on a tediously detailed goal, which results in you solving a series of small problems, probably over a long period of time, without running any particular risk – because the RFP process seeks to eliminate the risks it can quantify.
Life is risky. Here’s how to deal with it
This limited focus can steer you away from tackling bigger problems where you’ll never be able to completely eliminate risk, but where the rewards are potentially much greater. And it leaves you completely unprepared for the utterly unexpected, such as a pandemic.
For example, putting CI/CD to work doesn’t eliminate risk – sometimes releases will fail and you’ll have to roll back. But that’s the point. It also allows you to develop rapidly and run multiple experiments to see what works, repeatedly. If you really wanted to eliminate risk, you wouldn’t change anything at all.
And experimentation and adaptation is the whole point of Cloud Native. It has to be, because the world is changing fast, and is inherently risky. Historic product cycles no longer apply, and while some in your organisation have to care about predictability and stability, others have to spot the new opportunities that will help the business continue to grow. All at the same time. And yes, this means tackling cultural change as much as technological change, as Holly Cunnins has suggsted elsewhere on WTF.
In other words, when you say “We want to go Cloud Native – can you fill in this 100 page proposal doc”, you have already started down the wrong path.
And if someone plays along, and even promises they can take you to cloud native AND eliminate risk, they are wasting your time and money at best. At worst they are lying and putting you at risk of extinction. Because you’ll get the same old thing with added Kubernetes. But you won’t get the help with reworking your culture and attitude to experimentation that is at least as important as adopting any given new technology.
There has to be a better way to plan and buy for the Cloud Native era.
So, what’s my prescription? First, start experimenting with Cloud Native. That’s the whole point of it after all. But you don’t have to do this on your own. You can engage with a partner – yes, like Container Solutions – who can help you explore the terrain and map it out. You can work out what you think your requirements are – but expect these to be much more open and fluid than when speccing RFPs in the past.
The next step is to begin some more formal engagements with suppliers, around smaller projects. I’m not talking about a single supplier. Work with a few across a range of different, smaller projects. You’ll learn more about each potential partner, as well as more about what it is you’re really looking for. When it works out, you can start building a longer term relationship.
Then you’re in a position to actually start working on an MVP, and really start to do some Cloud Native work. But this doesn’t mean the hard work is over anytime soon. As I’ve said before the aim is to produce something that is built for continuous improvement. Something that will enable change and growth in the future, rather than leave you at risk of having your progress stymied by systems that made sense in an RFP doc compiled five years ago.
Is this approach going to eliminate risk? No, because you can never eliminate risk. But you can recognise that the world is risky and chaotic, then look for ways to take advantage of that to achieve your goals. And Cloud Native can be one of the tools you use to do this. When you’ve grasped that world view, you’re in a much better position to deal with whatever the future throws at you.
And if someone says they can help you develop an RfP, I suggest you arrange a meeting with them, perhaps at Berlin airport.
Previous article
