Jamie Dobson

Jamie Dobson

Jamie Dobson is co-founder and chief executive officer at Container Solutions.

Don't Be A Deadbeat: Cloud Native Transformation Design Fails

January 16, 2020 by Jamie Dobson

In his 1979 book, The Timeless Way of Building, Christopher Alexander tells a wonderful story of two barns. The first was extremely long, over 240 feet, and instead of having doors at the end (where you usually find them) they were in the middle on both of the long sides. The second barn was nestled into the side of a hill, had three stories instead of two, and five doors. The barns were unique, completely different, and yet instantly recognisable as barns.

The Financial Times and SkyScanner both completed successful Cloud Native transformations. These transformations were unique; for example, one company used Infrastructure as a Service, the other used a hybrid cloud. And yet both are instantly recognisable as Cloud Native transformations.

How is this possible?

Even though these instances of buildings or systems may, on the surface, appear very different, we nonetheless still immediately recognise they are related. We do this because their characteristics, however diverse, were selected from a common set—a library, if you will—of ideas, or elements. These individual elements are patterns.

PSbook_cover_web

A pattern, whether for a barn, a Cloud Native system, or any other thing we may set out to build, is a context-specific design decision. A pattern language is a collection of domain-specific patterns. The patterns are finite but the permutations in which they can be arranged are nearly infinite. This is exactly why anything designed from a specific language can be utterly unique, and yet instantly recognisable.

Our pattern language is a set of context-specific design decisions, too, in the domain of Cloud Native transformation. It explains why diverse transformations, such as the Financial Times and SkyScanner, are completely different yet instantly recognisable as Cloud Native transformations.

Using a Pattern Language to Design Systems

The power of patterns comes from their combination. To be able to do this effectively, it is essential that the patterns are easy to grasp. This is why they have a specific, easy to parse, five-part structure:

  • Definition
  • Context (situations where this pattern applies)
  • Therefore (actions to take)
  • Consequently (the positive and negative things that will change or happen once the pattern has been implemented)
  • Related patterns, which frequently represent one (or more) logical next steps.

To show how practical this is, jump over to the patterns website CNpatterns.org and take a look at Business Case. You can see just how easy this pattern is to parse and after reading it you will no doubt have a good idea where you can use it in your current efforts. (For more on patterns, check out the new O’Reilly book I co-wrote with Pini Reznik and Michelle Gienow, Cloud Native Transformation: Practical Patterns for Innovation. Get a free excerpt here.)

Dead Designs

If you check out our Financial Times and SkyScanner case studies it is clear that they both began their CN transformations at the same point. They both had a business case because they both faced existential threats. However, after this common beginning their implementations quickly diverged. They chose different architectures, namely Infrastructure as a Service for one of them and Hybrid Cloud for the other.

But both companies drew from the Cloud Native pattern language. Both combined patterns into a design in order to create their own Cloud Native systems. And these designs worked wonderfully for them because they were good.

SkyScanner and the FT got this right. Many others get this wrong. How? They use patterns in the wrong context or they use the wrong pattern language. By doing this, they create dead designs.

To understand this, let’s take a look at another example from Alexander. In his book he shares the language for his high school, which he described as ‘an ugly place, terrible, dark and dead’. Here are some of the patterns that were used to design the school:

  • Long and narrow
  • Daylight only at one end
  • Fluorescent light
  • Flat concrete wall
  • Unpainted concrete ceiling surface
  • Steel window

Each of these patterns represent a design decision. None of them are inherently ‘wrong.’ In a different context, such as an industrial warehouse, they are in fact perfectly fine. However, in the context of a school, they are patently terrible. In fact, the combination of the above would create something like this, a building that many English people my age would indeed recognise as a school:

Bad_School

If we draw from the Cloud Native pattern language without understanding the context— and we tend to do this after watching an inspirational conference talk—we will likely create dead designs.

To understand this, let’s look at two very common ‘dead on arrival’ Cloud Native scenarios, which are worth examining as they represent the industry norm rather than the industry standard.

Scenario 1: Lift and Shift

Here’s the ‘Lift and Shift’ high-level pattern design:


Here is the setup this design is likely to produce, mapped on Container Solutions’ Cloud Native Maturity Matrix:

Lift_and_Shift

In this design, the original system is left alone, aside from being packed up and placed on the cloud. The real design decision here is a non-decision. The culture does not become collaborative—and even if it did, the architecture does not support it. In this design we may be consuming public cloud infrastructure . . . meaning we will be paying for expensive tools but won’t be able to put them to proper use, unable to evolve our products in line with user behaviour. This design is as dead as dog shit.

Scenario 2: Cloud Native as an Extension of Agile

Here is the design:


Here’s what that looks like on our Cloud Native Maturity Matrix:

CN_as_Agile_Extension

Here Cloud Native is misdiagnosed as a technical problem. However, Cloud Native is a sociotechnical system and therefore requires changes in technology and management. In other words, it is a genuine paradigm shift.

In this scenario, Cloud Native is considered a technical add-on to a management system that uses the Agile software methods to develop its products. This design can come from a ‘platform iteration’, where you install Kubernetes and two sprints later declare victory.

But without the organisational and cultural aspects of Cloud Native, you can’t make the most out of the technologies. This is only a little bit better than the Lift and Shift scenario. Here, at least, you have made significant changes to the architecture and technology: good. But you are unable to evolve your products because your user behaviour is working the old way on your new system: bad.

So How Do You Design a Cloud Native Transformation?

To answer this question, let’s think about how a city is designed. The broad brushstrokes of a city are often laid down by nature in the form of natural constraints like rivers. The people who live there draw on a limited set of design patterns to build their homes. These patterns always represent the current state of knowledge and so the houses from a certain time period are often similar. Here in London, where I live, you can pick out distinctively Georgian or Edwardian areas, for example.

As more patterns are discovered—often in conjunction with changes in technology—the city evolves. Through a remarkably simple process, thousands of ‘designers’ create the city by iteratively drawing from the same set of design patterns. And in fact this is the only way a complex artefact can be designed.

It’s the same for Cloud Native systems. Natural constraints, such as regulations, lay down the broad brushstrokes of the system. Then, through an iterative process, patterns are applied and reapplied and the Cloud Native system emerges from this never-ending, iterative, and distributed design process.

What might this process look like?

What we have noticed, after five years of studying succeeding and failing Cloud Native transformations, and of course only with the power of hindsight, is that there is a meta-pattern for some successful transformations. It goes something like this:

Firstly, there is a Business Case. Secondly, there is always a Think period that leads into the iterative Design process that I described above, followed by a Build phase where the design is actually implemented into a new Cloud Native platform. Finally, as the transformation completes, we enter the Run phase. (Run emphasises ongoing proficient delivery of business product or services; since now we are focused not on creating but rather efficiently operating the new Cloud Native system, Run draws from a different set of patterns).

If a company gets this far, they have successfully used Cloud Native to remake the way they work. They do this by tying user behaviour to rapid product development and in doing that, they have created a system of innovation for themselves. (The end state of Cloud Native really is to create a system of sustainable ongoing innovation, which is why so many CEOs are interested in it.)

Let’s visualise this meta-pattern as a transformation design.

 

Transformation design

Moving from left to right, we see companies first tackling the strategy and management side of things by applying the Business Case, Executive Commitment, and Transformation Champion. We call this the ‘think’ phase as it’s very much the moment where companies envision Cloud Native as a system of innovation and work out what their first steps should be.

As we move towards the design phase, patterns like Vision First and Objective Setting are used to create both coherence and cascading objectives, without which actions cannot be taken. These lead directly to strategic execution patterns in phase three, build. In this phase, Exploratory Experiments and Proof of Concept (PoC) move work forward in a way that lets us learn as we go.

We also see, in this example, a divergence between how we Manage for Proficiency and Manage for Creativity. This is a very important explicit split that makes sure our creative work doesn’t get buried in our business as usual, but that the bottom line still receives the necessary attention.

Finally, we see that Gradual Onboarding and Lift and Shift at the End are used to smoothly (and slowly, so we can learn as we go) help the whole organisation migrate to the cloud platform where they (might) be taken care of by an SRE Team (which itself is a bridge to the Run pattern set).

This is not easy.

The above example has emerged in many organisations we have observed or worked with. However, it’s very important to note that in each case it emerged due to great care and attention, and with thanks to leaders and engineers who had the courage to test, fail, and learn.

This transformation design does not emerge spontaneously. Patterns are not hacks; they are not one time, out-of-the-box solutions to obvious problems. Their power lies in how they are combined to solve complex problems.

Only the most serious and determined people will succeed in combining them in a way that works for them. And only those comfortable with design, i.e. comfortable with failure, will be able to do this. This is why success with Cloud Native is the exception and not the rule.

Conclusion

Christopher Alexander did us a great service by introducing the world to the idea of patterns and the iterative process that is essential for designing complex systems. Through our pattern language, we hope to help people design Cloud Native systems that are ‘right’: fit for purpose, beautiful, easy to change, and a source of joy for those using them. These are the same qualities, abstract yet essential, that Alexander strove to bring into the buildings we work in and inhabit.

cloud native transformation book excerpt

Add a comment