Having the privilege of working in software in the 2020s, I hear variations on the following ideas expressed frequently:
- ‘There must be some direct relationship between your work and customer value!’
- ‘The results of your actions must be measurable!’
These ideas manifest in statements like this, which sound very sensible and plausible:
- ‘This does not benefit the customer. This is not a feature to the customer. So we should not do it.’
- ‘We are not in the business of doing X, so should not focus on it. We are in the business of serving the customer’
- ‘This does not improve any of the key metrics we identified’
I want to challenge these ideas. In fact, I want to turn them on their head:
- Many peoples’ work generates value by focussing on things that appear to have no measurable or apparently justifiable customer benefit.
- Moreover, judgements on these matters are what people are (and should be) paid to exercise.
Alex Ferguson and Canteen Design
To encapsulate these ideas I want to use an anecdote from the sporting world, that unforgiving laboratory of success and failure. In that field, the record of Alex Ferguson, manager of Manchester United (a UK football, or soccer team) in one of their ‘golden eras’ from 1986 to 2013, is unimpeachable. During those 27 years, he took them from second-from-bottom in the UK premier league table in 1986 to treble trophy winners in Europe in 1998-1999.
Fortunately, he’s recorded his recollections and lessons in various books, and these books provide a great insight into how such a leader thinks, and what they’re paid to do.
Now, to outsiders, the ‘business value’ he should be working towards is obvious. Some kind of variation of ‘make a profit’, or ‘win trophies’, or ‘score more goals than you concede in every match’ is the formulation most of us would come up with. Obviously, these goals break down to sub-goals like:
- Buy players cheaply and extract more value from them than you paid for
- Optimise your tactics for your opponents
- Make players work hard to maintain fitness and skills
Again, we mortals could guess these. What’s really fascinating about Ferguson’s memoirs is the other things he focuses on, which are less obvious to those of us that are not experts in elite-level soccer.
Sometimes if I saw a young player, a lad in the academy, eating by himself, I would go and sit beside him. You have to make everyone feel at home. That doesn’t mean you’re going to be soft on them–but you want them to feel that they belong. I’d been influenced by what I had learned from Marks & Spencer, which, decades ago in harder times, had given their staff free lunches because so many of them were skipping lunch so they could save every penny to help their families. It probably seems a strange thing for a manager to be getting involved in–the layout of a canteen at a new training ground–but when I think about the tone it set within the club and the way it encouraged the staff and players to interact, I can’t overstate the importance of this tiny change.
Alex Ferguson, Leading
Now, I invite you to imagine a product owner, or scrum master for Manchester United going over this ‘update’ with him:
- How does spending your time with junior players help us score more goals on Saturday?
- Are we in the business of canteen architecture or soccer matches?
- How do you measure the benefit of these peripheral activities?
- Why are you micromanaging building design when we have paid professionals hired in to do that?
- How many story points per sprint should we allocate to your junior 1-1s and architectural oversight?
It is easy to imagine how that conversation would have gone, especially given Ferguson’s reputation for robust plain speaking. (The linked article is also a mini-goldmine of elite talent management subtleties hiding behind a seemingly brutish exterior.)
Software and Decision Horizons
It might seem like managing a soccer team and working in software engineering are worlds apart, but there’s significant commonality.
Firstly, let’s look at the difference of horizon between our imagined sporting scrum master and Alex Ferguson.
The scrum master is thinking in:
- Very short time periods (weeks or months)
- Specific and measurable goals (score more goals!)
Alex Ferguson, by contrast, is thinking in decades-long horizons, and (practically) unmeasurable goals:
- If I talk to this player briefly now, they may be motivated to work for us for the rest of their career
- I may encourage others to help their peers by being seen to inculcate a culture of mutual support
I can think of a specific example of such a clash of horizons that resulted in a questionable decision in a software business.
Twenty years ago I worked for a company that had an ‘internal wiki’ – a new thing then. Many readers of this piece will know of the phenomenon of ‘wiki-entropy’ (I just made that word up, but I’m going to use it all the time now) whereby an internal documentation system gradually degrades to useless whatever the value of some of the content on there due to it getting overwhelmed by un-maintained information.
Well, twenty years ago we didn’t have that problem. We decided to hire a young graduate with academic tendencies to maintain our wiki. He assiduously ranged across the company, asking owners of pages whether the contents were still up to date, whether information was duplicated, complete, no longer needed, and so on.
The result of this was a wiki that was extremely useful, up to date, where content was easily found and minimal time was wasted getting information. The engineers loved it, and went out of their way to praise his efforts to save them from their own bad habits.
Of course, the wiki curator was first to be let go when the next opportunity arose. While everyone on the ground knew of the high value of this in saving lost time and energy chasing around bad information across hundreds of engineers, the impact was difficult or never measured, and in any case, shouldn’t the engineers be doing that themselves?
For years afterwards, whenever we engineers were frustrated with the wiki, we always cursed whoever it was that made the short-sighted decision to let his position go.
So-called ‘business people’, such as shareholders, executives, project managers, and product owners are strongly incentivised to deliver short term, which most often means prioritise short-term goals (‘mission accomplished’) over longer-term value. Those that don’t think short-term often have a strong background in engineering and have succeeded in retaining their position despite this handicap.
What To Do? Plan A – The Scrum Courtroom
So your superiors don’t often think long term about the work you are assigned, but you take pride in what you do, and want the value of your work to be felt over a longer time than just a sprint or a project increment. And you don’t want people cursing your name as they suffer from your short-term self-serving engineering choices.
Fortunately, a solution has arisen that should handle this difference of horizon: scrum. This methodology (in theory, but that’s a whole other story) strictly defines project work to be done within a regular cadence (eg two weeks). At the start of this cadence (the sprint), the team decides together what items should go in it.
At the beginning of each cadence, therefore, you get a chance to argue the case for the improvement, or investment you want to make in the system you are working on being included in the work cadence.
The problem with this is that these arguments mostly fail because the cards are still stacked against you, in the following ways:
- The cadence limit
- Uncertainty of benefit
- Uncertainty of completion
- Uncertainty of value
Plan A Mitigators – The Cadence Limit
First, the short-term nature of the scrum cadence has an in-built prejudice against larger-scale and more speculative/innovative ideas. If you can’t get your work done within the cadence, then it’s more easily seen as impractical or of little value.
The usual counter to this is that the work should be ‘broken down’ in advance to smaller chunks that can be completed within the sprint. This often has the effect of making the work seem either profoundly insignificant (‘talk to a young player in the canteen’), and of losing sight of the overall picture of the work being proposed (‘change/maintain the culture of the organisation’).
Plan A Mitigators – Uncertainty of Benefit
The scrum approach tries to increment ‘business value’ in each sprint. Since larger-scale and speculative/innovative work is generally riskier, it’s much harder to ‘prove’ the benefit for the work you do in advance, especially within the sprint cadence.
The result is that such riskier work is less likely to be sanctioned by the scrum ‘court’.
Plan A Mitigators – Uncertainty of Completion
Similarly, there is an uncertainty as to whether the work will get completed within the sprint cadence. Again, this makes the chances of success arguing your case less likely.
Plan A Mitigators – Uncertainty of Value
‘Business Value’ is a very slippery concept the closer you look at it. Mark Schwartz wrote a book I tell everyone to read deconstructing the very term, and showing how no-one really knows what it means. Or, at the very least, it means very different things to different people.
The fact is that almost anything can be justified in terms of business value:
- Spending a week on an AWS course
- As an architect, I need to ensure I don’t make bad decisions that will reduce the flow of features for the product
- Spending a week optimising my dotfiles
- As a developer, I need to ensure I spend as much time coding efficiently as possible so I can produce more features for the product
- Tidying up the office
- As a developer, I want the office to be tidier so I can focus more effectively on writing features for the product
- Hiring a Michelin starred chef to make lunch
- As a developer, I need my attention and nutrition to be optimised so I can write more features for the product without being distracted by having to get lunch
The problem with all these things is that they are effectively impossible to measure.
There’s generally no objective way to prove customer value (even if we can be sure what it is). Some arguments just sound rhetorically better to some ears than others.
If you try and justify them within some of business framework (such as improving a defined and pre-approved metric), you get bogged down in discussions that you effectively can’t win.
- How long will this take you?
- “I don’t know, I’ve never done this before”
- What is the metric?
- “Um, culture points? Can we measure how long we spend scouring the wiki and then chasing up information gleaned from it? [No, it’s too expensive to do that]”
‘Plan A’ Mitigators Do Have Value
All this is not to say that these mitigators should be removed, or have no purpose. Engineers, as any engineer knows, can have a tendency to underestimate how hard something will be to build, how much value it will bring, and even do ‘CV-driven development’ rather than serve the needs of the business.
The same could be said of soccer managers. But we still let soccer managers decide how to spend their time, and more so the more the more experienced they are, and the more success they have demonstrated.
But…
In any case, I have been involved in discussions like this at numerous organisations that end up taking longer than actually doing the work, or at least doing the work necessary to prove the value in proof of concept form.
So I mostly move to Plan B…
What To Do? Plan B – Skunkworks It
Plan B is to skip court and just do the work necessary to be able to convince others that yours is the way to go without telling anyone else. This is broadly known as ‘skunkworks‘.
The first obvious objection to this approach is something like this:
‘How can this be done? Surely the the time taken for work in your sprint has been tightly defined and estimated, and you therefore have no spare time?’
Fortunately this is easily solved. The thing about leaders who don’t have strong domain knowledge is that their ignorance is easily manipulated by those they lead. So the workers simply bloat their estimates, telling them that the easy official tasks they have will take longer than they actually will take, leaving time left over for them to work on the things they think are really important or valuable to the business.
Yes, that’s right: engineers actually spend time they could be spending doing nothing trying to improve things for their business in secret, simply because they want to do the right thing in the best way. Just like Alex Ferguson spent time chatting to juniors, and micromanaging the design of a canteen when he could have enjoyed a longer lunch alone, or with a friend.
It’s Not A Secret
Good leaders know this happens, even encourage it explicitly. A C-level leader (himself a former engineer) once said to me “I love that you hide things from me. I’m not forced to justify to my peers why you’re spending time on improvements if I don’t know about them and just get presented with a solution for free.”
The Argument
When you get paid to make decisions, you are being paid to exercise your judgement exactly in the ways that can’t be justified within easily measurable and well-defined metrics of value.
If your judgement could be quantified and systematised, then there would be no need for you to be there to make those judgements. You’d automate it.
This is true whether you are managing a soccer team, or doing software engineering.
Making software is all about making classes of decision that are the same in shape as Alex Ferguson’s. Should I:
- Fix or delete the test?
- Restructure the pipeline because its foundations are wobbly, or just patch it for now?
- Mentor a junior to complete this work over a few days, or finish the job myself in a couple of hours?
- Rewrite this bash script in Python, or just add more to it?
- Spend the time to containerise the application and persuade everyone else to start using Docker, or just muddle along with hand-curated environments as we’ve always done?
- Spend time getting to know the new joiner on the team, or focus on getting more tickets in the sprint done?
Each of these decisions has many consequences which are unclear and unpredictable. In the end, someone needs to make a decision where to spend the time based on experience as the standard metrics can’t tell you whether they’re a good idea.
Conclusion
At the heart of this problem in software is what I call the ‘bricklayer fallacy’. Many view software engineering as a series of tasks analogous to laying bricks: once you are set up, you can say roughly how long it will take to do something, because laying one brick takes a predictable amount of time.
This fallacy results in the treatment of software engineering as readily convertible to what business leaders want: a predictable graph of delivery over time. Attempts to maintain this illusion for business leaders results in the fairy stories of story points, velocity, and burn-down charts. All of these can drive the real value work underground.
If you want evidence of this not working, look here. Scrum is conspicuously absent as a software methodology at the biggest tech companies. They don’t think of their engineers as bricklayers.
Soccer managers don’t suffer as much from this fallacy because we intuitively understand that building a great soccer team is not like building a brick wall.
But software engineering is also a mysterious and varied art. It’s so full of craft and subtle choices that the satisfaction of doing the job well exceeds the raw compensation for attendance and following the rules. Frequently, I’ve observed that ‘working to rule’ gets the same pay and rewards as ‘pushing to do the right thing for the long term’, but results in real human misery. At a base level, your efforts and their consequences are often not even noticed by anyone.
If you remove this judgement from people, you remove their agency.
This is a strange novelty of knowledge work that didn’t exist in the ‘bricklayer’, or age of piece-work and Taylorism. In the knowledge work era, the engineers who like to actually deliver the work of true long term value get dissatisfied and quit.And paying them more to put with it doesn’t necessarily help, as the ones that stay are the ones that have learned to optimise for getting more money rather than better work. These are exactly the people you don’t want doing the work.
If you want to keep the best and most innovative staff – the ones that will come up with 10x improvements to your workflows that result in significant efficiencies, improvements, and savings – you need to figure out who the Alex Fergusons are, and give them the right level autonomy to deliver for you. That’s your management challenge.
Previous article
