Some days, everything seems like it’s hard. And getting harder.
Credentials, configurations, and checklists.
And audits, and provisioning, and versions, and tracing, and where is that bug?
Is the cloud helping, or hurting?
Does the cloud make our work more enjoyable? Is that even the right question?
Should our work even be enjoyable, or should we resign ourselves to ever-longer yaml files?
In this precursor to WTF is SRE?, Holly will share some organisational and technical anti-patterns that turn the cloud from the good place into the bad place, how to get back to the good place and why happiness is not only important, but valuable. The cloud can be an amazingly fun place - but only if we do it right.
Holly Cummins is a Senior Principal Software Engineer on the Red Hat Quarkus team. Before joining Red Hat, Holly was a long time IBMer. In her time at IBM, Holly has been a full-stack javascript developer, a WebSphere Liberty build architect, a client-facing consultant in the IBM Garage, a JVM performance engineer, and an innovation leader. Holly led the developer community in the IBM Garage for several years and became a bit of a methods geek.
During her time in the IBM Garage, Holly led projects for enormous banks, tiny startups, and everything in between. Holly has used the power of cloud to understand climate risks, count fish, help a blind athlete run ultra-marathons in the desert solo, and invent stories (although not at all the same time). Holly is also an Oracle Java Champion, IBM Q Ambassador, and JavaOne Rock Star. Before joining the IBM Garage, she was Delivery Lead for the WebSphere Liberty Profile (now Open Liberty). Holly co-authored Manning’s Enterprise OSGi in Action and is a regular keynote speaker. She has spoken at KubeCon (keynote), GOTO, JavaOne, Devoxx, Sonar+D, JavaZone, JFokus, The ServerSide Java Symposium, GOTO, JAX London, QCon, GeeCon, and the Great Indian Developer Summit, as well as a number of user groups.
(Skip the gossip and jump straight to Holly’s talk).
Charles Humble:
So hello, everybody, and welcome to our final WTFinar of 2022, unbelievably. I do have a little bit of housekeeping to go through, so I'm going to do that first. The very first thing to say is that we have a code of conduct. I think Tereza will put the link to it in the chat, but the short version of this is that our events are dedicated to providing a harassment-free experience for everyone, and that's regardless of gender, gender identity and expression, age, sexual orientation, disability, physical appearance, body size, race, religion, or lack thereof. We don't tolerate harassment of participants in any form. If you violate these rules, you may be sanctioned, you may be expelled from the event at the discretion of the organisers, and if necessary, you may be banned from participating in future CS events. Our code of conduct is, as I say, it's something we enforce. It's also Creative Commons, so if you're an event person and need a code of conduct, feel free to steal ours. We'd be delighted. We like to propagate these things out as much as we can.
You've probably guessed if you haven't been on one of these before from the fact it's called a what-the-fuck-inar that we do swear a certain amount at Container Solutions. If that bothers you, I can only apologise. It's part of the branding. If it really bothers you, then this is probably not the event for you. Sorry.
The session is being recorded and will be posted on the Container Solutions YouTube channel. I think Tereza's going to share the link for that as well. If you are concerned about your name or your face appearing on the video, if you let me or Tereza or one of the other CS people know, we can pixelate you out or blank you out before we upload the video. So there's no need to worry about that. We can take care of that for you.
We do like to make these sessions interactive, so we do encourage questions. You can drop those in the chat at any time, and when Holly feels it's appropriate, she can try and answer a few of those towards the end of the session. And if you want to grab the microphone, if you're feeling brave and want to actually be on camera and speak, we can do that as well.
This webinar is part of a larger range of content that we produce, known as WTF is Cloud Native. I am massively biassed because I'm the chief editor for this, so one of my jobs is to put this together every month, so obviously I think it's brilliant, but it is genuinely, I think, honestly, a really good piece of work. We publish around about seven blogs a month. Some of these are from CS people, both our engineers and other people who work at Container Solutions, but we also get some fantastic external writers involved, with recent contributors having included people like Matt Turner of Istio fame, Jennifer Riggins, Joe Fay, and indeed today's webinar guest, Holly. We also have a podcast and we've had some tremendous people on that, including Randy Shoup, Christian Idiodi, Matt LeMay, lots of really, really good people.
If you subscribe to the newsletter and you do not want to get any other spam from us, I can promise you, you won't. So you can just subscribe and get the newsletter. We are not a spammy company. And if you do that, you'll hear about upcoming webinars, you'll get notifications of the blogs, the podcasts, and all the rest of it. Thinking of our events, our big conference, WTF is SRE, is back for I believe the third year, and it's going to be held in person for the first time, which I'm really excited about. It's going to be in London, May 4th to 5th 2023. We've got four tracks covering observability, DevSecOps, reliability, and DevEx.
The call for papers is now open, so if you fancy speaking, and particularly if you are a first time speaker, maybe from an underrepresented group in tech, we'd love to hear from you. The url, which I hope Tereza will also share, is www.cloud-native-sre.wtf/cfp, but hopefully that'll be put in the chat, or just google WTF is SRE and you'll find it easily enough. Really, really looking forward to that, and if you put a call for papers, submit your talk in, then I look forward to reviewing those with the rest of the content committee in the new year. Really looking forward to that one.
CS is also hiring. So we are looking for delivery managers, we're looking for cloud native engineers. There are lots of other roles as well, so have a look at our careers page. Speaking personally, you can probably tell even on my rubbish webcam, I'm not in my first flush of youth. I've been around the block a bit, and I can honestly genuinely say, Container Solutions is the best company I've worked at, just in terms of the people, in terms of the culture, very, very hot on psychological safety, very hot on the whole bringing your whole self to work thing, really good in terms of being a learning organisation, in terms of allowing people to try things, and even when those things don't work, just a really great place to work. So if any of these opportunities appeal, if you are looking for somewhere new, perhaps a new job in the new year or something, we are actively expanding, and as I say, it's a great company, so honestly can't recommend it highly enough. So do apply if you are looking for an opportunity.
One of our examples of an experiment is the gossip section that we have in these. So we did this as a one-off, and people kept saying they liked it, and so it is now a regular feature of our webinars. So I believe Julien is our gossiper today, so I'm going to turn it over to him, and he can chat a bit about whatever it is he wishes to chat about. Julien, over to you.
Julien Sudan:
Thank you, Charles. So hi, I'm Julien. I'm going to be indeed talking about the goss today, which will be about ChatGPT. What does it mean for cloud native? I went for a pretty standard title instead of everything you will see on the internet, because you might have heard about it. It was put out in the wild as a test phase by OpenAI a few days ago, at the beginning of December. In a few words, ChatGPT is a large language model that has been trained by OpenAI. It is designed to generate natural language text and can be used for a variety of tasks, such as generating code, writing documentation, or creating code snippets on the fly. ChatGPT is a powerful tool that can save time and effort for developers and consultants, but it is not perfect and may produce errors or mistakes in the generated text. Using ChatGPT can save time and effort compared to manually writing code or documentation. This can be especially useful for consultants who are working on tight deadlines or who need to produce a large amount of content quickly.
So as you might have noticed, I read this intro because I created it using ChatGPT. I asked it to create an intro about itself. It took quite some time, though. I thought about doing the full piece like this, but it turned out to be utterly impossible, because it was trying to sell itself way too much, and tried to say that we should use it and implement it on client side, which I'm not sure we are even able to do so right now as it's still in the test phase.
So it came out in December. It hit the one million user mark after five days, which is quite astounding compared to other technologies that came out or websites that came out in the past years. You might have read a ton of things about it on the internet. So the two main things that I took out were the fact that when it's stupid, which I would say when we ask stuff to it, it can produce wrong answers. But I don't know if anyone has tried asking people things, it also might produce wrong answers at some point. The other thing that I've read quite a lot is that it will take away jobs. While I think it might be true for some type of jobs that are maybe more repetitive or more simple, as I said before, producing text, producing complex content or complex code, is still an impossible task for it. It can produce things, but it cannot produce, for example, an entire program or product and maintain it over years.
So it does change everything, like cameras before, or typewriters, or even the internet, when suddenly we had access to information everywhere. This type of technology also will change everything, but it will do so... We have the choice to use it, to be in line with the topic of the what-the-fuck-inar. We can actually use it as a tool, as a fun tool, to keep the fun in our daily job. For example, we tested it recently in Container Solutions, and it really helps save time for documentation, and also for searching things like we would always do, when you go on Google to look for your different information on the product, or the things you don't remember by heart. This really helps to do so, but in a much, much quicker way.
It has a learning curve, because you still have to ask the right questions, which I even saw some specific terms on this, and this is something that we all will have to learn I think if we want to keep the pace with people actually using it like Copilot before from GitHub.
So that was my piece about ChatGPT. If you haven't tried it yet, feel free to do so. It's free for now, it will change in the future. And if you want more information about it, we will have a link to a very nice video of a senior software developer from Netflix giving his take on it, which is pretty interesting and neutral, and otherwise you can always find information either on ChatGPT or Google if you still use that.
Charles Humble:
That's brilliant, Julien. Thank you. I do think it's really interesting, actually. I think we're seeing a bit of a an S-curve on machine learning stuff. We've had stable diffusion doing a similar sort of thing for images to what ChatGPT is doing for text. And I just think it's really, really fascinating. It still feels a bit like playing at the moment, but I'm sure that use cases for these things are coming, and I think it's just a really, really interesting time. It is interesting as well, it does produce very plausible sounding nonsense a lot of the time. Obviously it's just basically giving you back patterns that it finds from the body of text that it's been given, so it's basically regurgitating stuff like a typical school child doing their GCSE exams or something, but I do think it's really interesting. I think it's going to be a fascinating time. So thank you very much for that.
Julien Sudan:
You're welcome.
Charles Humble:
And with that it's my absolute pleasure to introduce Holly. Holly Cummins is a senior principal software engineer on the Red Hat Quarkus team. Before joining Red Hat, she was a longtime IBM-er. She led the developer community in the IBM Garage for several years. She's used the power of the cloud to understand everything from climate risk to counting fish to helping a blind athlete run ultramarathons in the desert. She's also an Oracle Java Champion, an IBM Q ambassador, and a JavaOne Rock Star, and just an all round wonderful human being. So I'm delighted to have her doing our final webinar of the year. I am going to stop sharing, Holly, and then you can hopefully share your slide deck and take over.
Holly Cummins:
Yeah. This is always the tricky bit.
Charles Humble:
The awkward bit. Yes.
Holly Cummins:
Yeah. And normally you say, okay, I'll just make small talk while I do it, but then I can't actually talk and try and share my screen at the same time. It's too challenging. But I think I've succeeded, so hopefully we're okay.
Charles Humble:
We're all good.
Holly Cummins:
Yeah. I do have a few bits of small talk before we start though, just because I was listening to what Julien was saying about ChatGPT, and then what you were saying. And I've got another talk on fun, which isn't this one, it's a little bit more focused on process and the psychology of fun, and a bit more technology agnostic. And one of the things I talk about in that talk is some of the effects of gamification. And I think we all know that gamification can make these amazing effects and really motivate people to do things, but then it can really motivate people to do stupid and wrong things as well.
And there's a citizen science project to do with mapping and labelling craters on the Moon, and so you get a picture and then you just get all the craters, and I saw a talk from the scientist who'd started it, and she was saying that they looked into gamification so that people would get scores for the labelling, and then they stopped it because they realised that what it was doing is it was driving people to label things really as quickly as they could, and they were making errors. And the interesting and stupid thing about it is, why would you go to a citizen science project to label craters on the Moon? There is nothing in it for you except the good feeling that you're advancing citizen science, and if you do it wrong then your score is going up but you're worsening citizen science. So there's no financial incentive, there's nothing, and yet the effect of the gamification is enough to make people do the wrong thing.
And I saw a similar insight about ChatGPT, because of course there was a little kerfuffle where Stack Overflow had to stop people doing ChatGPT answers. And it's a similar thing of, what do you get by posting an answer Stack Overflow that... There is no money in it. Maybe you could put it on your CV, maybe, but I've never seen it on a CV. And so the only reason to do it is that perk of feeling your score go up. And so why would you do it with ChatGPT? It's just interesting of how we get motivated to do things that are actually counterproductive, and sometimes our tools allow us to do counterproductive things with more speed.
I liked as well what you were saying, Charles, about how it feels like play at the moment. And I think you're exactly right, and it's something I come back to in this talk a bit, but the fact that so many things, when we start doing them, they don't seem useful, and then at some point something clicks, or the environment changes, and then all of the stuff that was just a toy all of a sudden is what we're building our business on.
Charles Humble:
Yeah. I think it's a really interesting part of how innovation works, actually, that I'm sure you know far more about than I do. But I do think it's a really interesting... I just think it's a really interesting phenomenon. It's the classic point where everyone goes, "It's just a toy, and you don't need to take it seriously." and you're like, "yeah, there might be something here." So it's really fun times, I think.
Holly Cummins:
Mm-hmm. And now I'm going completely off piste, but it's sort of, I think, perhaps a positive aspect of our current VC ecosystem, where you end up with these weird things where VCs will pour tons of money into companies that have really quite a small chance of return, because we get these distorted economics and this distorted perception of value. But I think that does allow things like OpenAI that are producing really interesting toys to go for much longer than they might in a ecosystem that had a different economic model.
Charles Humble:
Yeah. Nice.
Holly Cummins:
But it also leads me on quite neatly to my first slide. So as Charles said, I work for Red Hat, and I'm a software engineer on the team that builds Quarkus, and I've got a bit of a duck theme going here, and it's very hard to find actual photographs of ducks wearing red hats. But if you go to DALL-E it is very easy to make a photograph of a duck wearing a red hat. So we're starting to see some really actually... I was going to say this is useful, and probably there is no way that you could describe making a photo of a duck in a red hat actually useful, but it's being used, which is kind of the same.
Yeah. So I work for Red Hat. I work on Quarkus. A lot of the stories in this talk are also from when I worked at IBM, and I was a consultant, and my job really was helping businesses take advantage of the cloud and get to the cloud. And so I saw lots of things that I thought, "That doesn't seem like quite the best practice." And so every time something bad happened, I harvested it away and saved it, because I knew it would be a good story sometime.
And I really like doing the what-the-fuck-inars, because there's just so much potential to play with the title, and to get all sorts of salty language into my presentations that I couldn't normally. And you can see, this is another DALL-E picture. Because I think sometimes when you say to someone that the cloud should be fun, the first response is just, "What? What does this mean? I like cloud, I like fun. I would not consider those two to go together." But I think actually there is a really strong and interesting alignment between cloud and fun. But before I go into that in particular, I want to talk a little bit more about just what I mean by fun, and why it might be really a strange idea to talk about the cloud being fun, or why it might be quite a correct idea.
So what is this fun? Often I think when we talk about fun we think of something like partying. So there's going to be alcohol, there's going to be silly hats, there's going to be... There is actually literally a rubber chicken in that photo. And that's the kind of thing. And that obviously doesn't seem... I mean, this was a work event, but it doesn't seem particularly conducive to work or to business. Or we might think of something like memes, and just sharing animals doing silly things and jokes. And again, all of this, it doesn't seem particularly professional.
As an aside on the jokes, going back to why are there ducks in this slide, a researcher did a set of controlled experiments where they did the world's largest science experiment, and I think they had maybe 350,000 participants, and they had a site, it was basically like ratemyjoke.com, but they were able to then see what people found funny and see national differences between it. And they were also able to do controlled experiments like, you take the same joke and you change the animal, and then you see what gets rated funniest. And it turns out that, scientifically proven, that ducks do make jokes funnier. So if you have a joke and it doesn't depend what animal's in it choose a duck.
So that's one side of fun. But then I think also when we think about fun, there's a deeper thing as well of, particularly at this time of year, but hopefully all around the year, this joy. Or maybe joy is too much. Maybe we settle for just happiness, or even contentment, and that's enough. So I think those are all things that I'm including in my fun, but we can be more specific about fun as well, and we can think about... When we think about play, there's quite distinct kinds of play, and all of those I think count as fun.
So the first is exploration. So exploration is focused investigation. So for example, when you're a baby and you know nothing, almost everything that you do to interact with the world, you're trying to figure out, what does this button do? What does this thing here... If I hit the bell, what happens? And with engineering we definitely have that. I think three quarters of our job sometimes is that. So, often when we're talking about a new technology, we'll say, "Our engineers had to play with this," and then depending who the audience is, we might then correct it to say, "Our engineers investigated this," or, "Our engineers explored this," but really our engineers were totally playing with it, and we just tried it out and we saw what the buttons did it.
Another form of play is the puzzle. And this I think has a really obvious analogy to our work. So a puzzle, unlike exploration, a puzzle has rules, and a puzzle has a correct answer, and a puzzle also has lots of incorrect answers. And this is something that we've all seen pretty much every day. Why is this not working? What am I doing wrong? Let me try and debug this. Why am I getting this stack trace? And the downside is, it can be quite frustrating, until we get the right answer, and then it can be incredibly satisfying.
Another form of play is games. Like puzzles, games have rules, but games also have a winner. So when you make your program work, it's not like you won against your colleagues, but a lot of things that we do at work do get a little bit more into that gamification territory, where maybe you are winning against your colleagues. Who among us hasn't tried to gamify the velocity chart, and choose the things that we know are going to be... They maybe got sized a bit too big and they're actually a little bit easier, or we know that we've got the right skills and we can just knock it off, and it feels so good when we're on the top of the velocity leaderboard. And I mentioned at the beginning Stack Overflow. Stack Overflow is another really great example of gamified work, and it took support and documentation and gamified it really successfully.
Finally, the final form of play is just literal play. Play is flexible, there's no rules. And unlike exploration, it's not necessarily something new to us, it's maybe something that we understand but we're still going to play with it. So Easter eggs are a really great example of this, where engineers will, just for the sheer joy of it, just for the sheer fun of it, build Easter eggs into their product. I don't think I have the screenshot in here, but there was a lovely one in Google at one point where, if you went into Google Maps and you said, "I want to get from the Shire to Mordor," then it would give you the standard walking disclaimer to say, "Walking instructions are in beta, caution," and then it says, "One does not simply walk from the Shire to Mordor." And you know that there was a person behind that, and it's not there any more, so you also know that at some point the person's boss went in and said, "I'm not sure we want this in the product."
And play, like we were talking about at the beginning, play seems like it's not at all useful, and some things like the Shire to Mordor maybe are genuinely useless, but a lot of them are actually quite valuable. They're valuable for us, and they're also valuable for teaching. This is actually a screenshot of a webinar from a colleague of mine. And the reason it's work is he's using this as an analogy to explain Kubernetes concepts. And so each one of the animals is a different type of resource. And so then you can see visually how many resources are spawned by a typical Kubernetes installation. And he was really surprised how many there were.
And this is also, kind of by accident, a really good way of learning about resiliency as well. So he had this video, but then while he was doing the webinar, what happened, which he wasn't expecting at all, is of course Minecraft is not a... Well, I was going to say it's not a closed world. It is a closed world, but it's not an environment that you fully control. So while he's doing this talk and his chickens are the services and that kind of thing, and then a fox came along, and that was not his fox, that was just a random Minecraft fox, the fox came along and ate one of the chickens, and the service went down. And of course that happens in the real world too, not because of foxes, but things affect your services. So we were able to see how the resiliency worked, and we were also able to see that, just like in Minecraft there's foxes that we might forget about, in the real world there's the distributed computing fallacies that we might forget about.
And with all of these, I've talked about a whole bunch of different things of play and exploration, and there's a scientific term to describe all of these. Because what all of these have in common is positive affect. And so this picture, I love this picture, because this is Katie Bouman when she discovered that her algorithm for imaging the black holes worked. So this was from a few years ago, but it sparks joy when you look at it, because you can just see that expression on her face. And that's I think what we all hope for in our work. And we don't get it all the time obviously, but we do get it some of the time.
And the thing is, when we do get it, it's a good thing. And we've known this for a really, really long time, and we have to keep saying it, because it's easy to forget. But Aristotle said, "Pleasure in the job puts perfection in the work." So it's the idea that, if you're enjoying what you're doing, you're going to do a better job than if you're doing something that makes you miserable and you wish you were somewhere else. And of course that was 3,000 years ago, and that was just one person, but we can now demonstrate this in a modern workplace context.
So if you look at the DORA research, what they found is pretty much exactly the same thing, that job satisfaction is the number one predictor of organisational performance. And you could look at this and you could say, "Okay, but I think you've got the correlation and the causation backwards here. It's not like if you're happy, your company does well. It's like, if your company does well, you're happy." And I think it's probably a bit of both, but part of it is, it doesn't actually matter, because we have this idea I think that came from the Puritans, that if we're happy at work we're doing something wrong, and it's terrible if we're happy at work. And no, it's not. If we're happy at work, it means things are going well.
And so a while ago I was working on a project, and most of the team were in South Africa, and I was here, and they had really terrible phone lines. And you know how it is when you're on a conference call, where most of the people are in person and you're remote, and they have one phone in the middle of the room, and you can't hear, and it's just a nightmare. So I was on this call because I was supposedly, if not leading, then at least supervising this project, and I had no idea what was going on. And so then my manager said to me afterwards, "So what happened on the call?" And I had to say, "You know what? I have no idea. I couldn't hear a thing."
But the one thing that I could hear was that they were laughing. And that made me pretty confident that actually the project was okay, because when a project is going wrong and nothing's working and they're missing the deadlines, the team usually aren't happy. The team are usually then starting to do things like, "It's the fault of you, Bob, and why didn't Alice get me the thing I needed, and why hasn't Fred done the spec properly?" And you start to get these tensions in the team. But in this case, because the team were laughing, I knew that meant that they wanted to work together, that they were happy working as a team, that they were pulling in the same direction. And so it meant I felt pretty confident that it was going to be okay. And actually it was okay.
And I think particularly when we're a consultant, I think there's this idea that we have to not have fun at the job, and we have to be really serious, because otherwise the person who's paying us is going to be annoying that they're paying us to have fun. But actually... And I think I would probably do that too. I need to get my bathroom redone, and I think if I saw the people redoing my bathroom were just hanging around all day and drinking tea all day and laughing and watching YouTube videos all day, I'd be like, wait a minute, I'm not paying you to have fun, I'm paying you to redo my bathroom. But there's a balance. But if there is a little bit of happiness certainly among the contractors, what that shows is that they're in their zone of competence. They know what they're doing, and that's good news for you. Whereas if the people redoing your bathroom are swearing at each other all the time, then you know that something terrible is happening to your bathroom, and that it's probably going to leak.
And again, not in the context of bathrooms but more generally, we can back this up with evidence. So studies show that, if employees are having fun, then they take less sick leave, so that's direct profit. They work harder, which is direct profit. And overall they're more productive. So the business results are better if employees are happier. And we see this, this just gets confirmed over and over again. There was another study and it said that your brain at positive, so in a positive mindset, was 31% more productive than your brain at negative, neutral, or stressed. And you think, if I could go and I could get a 31% more productive workforce, I would totally do that. I would not ask questions, I would just run directly to whatever it is. But of course getting your workforce in a positive state of mind, it's easy to say, it sounds really glib, but it's not as easy as just watching cat videos on the internet or something.
Except actually it is literally as easy as watching cat videos on the internet. So there was another bit of research, and what they did was they had people doing a task, and some they gave a comedy video to beforehand and some they didn't. And the people who watched the video had 12% greater productivity than those who didn't, which is a huge thing. And so again, if you would go to your boss and you'd say, "Hey boss, I know how to make us 12% more productive," they'd be like, "Okay, do it. Here's my credit card." And it is something that we can do.
And again, there's the balance. If we spent 100% of our work time watching cat videos, it's just like the bathroom contractors who are spending all their time drinking tea and laughing, then you don't get the benefit of the productivity because you're not doing any work. But certainly those moments of levity, those moments of laughter, those breaks can be really beneficial. And it's just like how the best debugging tool is a good night's sleep. The second best debugging tool is a good step away and a break.
And there's a lot of benefits to laughter. One is, if we're working in a team and there's a tense situation, as a group we can diffuse that tense situation by making a joke, by having a laugh. And then a secondary effect of that is, it creates a team cohesion. So it means that, if we laugh together as a group, we start to become more like an us. If we become an us, the downside of that is there is usually a them, but that's a different topic. And then just physically, exercise is good, getting circulation going is good, and laughter is actually really good exercise.
So how do you get the fun? You could just buy a joke book and sit there in your standup going through the jokes. There's probably better ways. But I think when we think about how to achieve fun, actually that's probably what most of us start with is, maybe if we bring some rubber chickens into work and have silly hats, we'll all have more fun. Well, maybe. But before you can do that, you need to get rid of the things that are destroying the fun. So you need to find the un-fun things and get rid of them.
And then actually, even before you can do that, because I've been talking about fun for a while, I've started to hear some really interesting and terrible stories from people. And one of the... What I've heard is, in some workplaces, it's not just that they have an atmosphere that doesn't encourage fun, fun is actually prohibited. So I heard a story that someone was... They were coding away, and they were smiling because it was going well, and a project manager crept up behind them as project managers do and said, "Why are you smiling? Work isn't a place to have fun." Which again just seems like such a counterproductive attitude. You don't want people to be scowling at the code, you want people to be smiling at the code because the code is cooperating. So step one, don't prohibit fun. Step two, get rid of those un-fun things. And then once you've done all that, you can go on to the joke book and the silly hats.
And with the getting rid of un-fun, the reason that feels so good is because flow feels good. And so if we're in a state of flow at work, we will have that joy, we will have that happiness, and we will also have that productivity. Whereas if everything that you need to do at work, you need to fill in the form, and then you need to go talk to the person, and then you need to go find the documentation, but the documentation isn't where it's supposed to be, and then you need to get a special sign-on to go find the documentation, and then you need to go talk to accounting to get the thing, and then you need to go fill in the other form, and the spreadsheet, and it's just... It destroys energy. It's so miserable.
And the reason it makes us miserable, I think, is because those un-fun processes... The reason we hate them isn't because we're lazy. The reason we hate them is because we know that actually they're wasting our time. They're things that aren't adding value, and we're being made to do that. And people really, really hate that. You want to be contributing, you want to be valuable, you don't want to be just a cog in a pointless machine.
So then that brings us to cloud, and is cloud fun? And again, before we can answer that, we need to think about, what is this cloud thing? And the cloud, obviously in a strict level, it's just a computer in someone else's data centre. But there's a whole bunch of reasons why we do it, and there's a whole bunch of emergent behaviours that have come out from that.
So one is, the cloud can potentially save a lot of money. I think that's good. I don't think anybody would say, "Yes, this is a laugh a minute, I'm saving money." It's just a good thing. Again, the cloud gives you lots of elasticity. That's probably good. I don't think we can say, "The elasticity of my cloud gives me so much joy." It's just... There's other benefits, financial benefits. Another thing that we see on the cloud is exotic hardware. So I used to work for IBM, and IBM is now putting quantum computers on the cloud, which is kind of amazing. Or with things like the GPUs, they're kind of expensive to get a big array of GPUs just for yourself, but you can share GPUs in the cloud, and then you can do cool things with the GPUs. So I think that exotic cloud hardware probably does have to count as fun.
But the most fun part of the cloud, I think, is the emphasis on low friction and fast speed to market. And that is definitely fun. And we're seeing a whole bunch of new ways of working that have come from the cloud. So if we look at something like DevOps, now DevOps is sometimes just used to mean the infrastructure team or the build team, and it's just a job title. But if we ignore that and if we think about the original pure meaning of DevOps, I really like a lot of the ideas around this. So Gene Kim who wrote The Phoenix Project, he said, "DevOps helps make our lives humane and win in the marketplace." And so it's this idea that our work is better and we are happier because of these practices, and by the way we also win. So again, it's back to that idea that fun has a business benefit, fun improves productivity.
And we see this now more recently with SRE, which again is another practice that's really come up concurrent with the cloud. And I love the ideals of SRE, because the founding principle of SRE is that you should eliminate toil from ops. And toil means those repetitive actions that aren't really using your time effectively, and they are not valuable. They're not fun, and they're not valuable. And so it's this duality that we can get rid of the crap part of our job and do better.
So all that, I think, is the dream. That's the promise of cloud, is this world where we have so much less friction, where everything that can be is automated, and where we don't focus on the infrastructure, where instead we're able to think about the higher level things. I think the reality is sometimes not quite matching up to that. And so if you look, what a lot of people are seeing is, cloud computing, it's hard, it's complicated. These infrastructure concerns that we're supposed to be not worrying about are now actually the majority of our job, and instead of focusing on the code, which is what we were supposed to be doing when we went to the cloud, instead now we're focusing on configuring our ingress when we don't even know anything about what an ingress is supposed to do, and it's different in every cloud provider, and it's just a little bit tedious.
When I was looking for this, it's one of those things probably everybody has this experience, where you see something go by, and then you don't grab it, and then you try and find it later and you cannot find it. So I'd seen a lot of complaints quite well worded about how the cloud was so hard, and I went to go back and look for them, and I couldn't find... I found some of them, but I couldn't find all of them.
But one of the things that I did find was this article about how clouds are hard, which is great. And so you read a little bit more closely, and then if you look quite closely you can see that what they're actually talking about here, they're not talking about someone else's data center, they're talking about a fluffy thing in the sky. And when they say it's hard, they don't mean it's actually physically hard, because it's fluffy. What they mean is that, if you were to try and model a cloud, it's apparently a really problematic challenge, and they can model lots of elements of the atmospheric system, which is important for weather and climate change and all that kind of thing, but they're not very good at modelling clouds. So there you go, no matter what domain you're in, clouds are hard.
But I think one of the things that makes software clouds hard is the tools. We end up stuck with these tools that just are not good enough. And often the reason for that is that the buying decisions for these tools aren't driven from the bottom up from the developers, they're driven from the top down, and someone takes the CEO to play golf, and they buy this tool, and it wasn't fit for purpose, but it's really quite difficult to get rid of it. This is, I think, one of the things that's really nice about open source, is it has driven a bit of a shift. So we still see this, but we do see as well a lot more bottom-up buying behaviour, because OpenShift means that anybody can try before they buy. If it's adopted within a business and people are happy with it, that's when maybe the business can make a decision to get an enterprise support or whatever. But actually that decision comes from usage, rather than usage coming from the buying decision. So I think it's quite positive.
But that definitely hasn't eliminated all bad tools from the marketplace. And so for example, here's someone. "I feel like every cloud platform UI is extremely hard to use. I thought system configuration was supposed to get easier, but I'd rather muddle my way through config files and a terminal than config files and a web UI." So here we have something like a web UI that's supposed to be the easy on-ramp, and actually it's so badly designed that it's worse than the hard option of terminals.
And of course the other problem that we see with modern cloud is massive, massive fragmentation. So every cloud provider has different ways of doing things, and our stacks are potentially so complex that every team has different variations of the stack. So then we just end up fighting the tools, and we just don't seem to be able to get a consistent set of standards, which is a good thing, because we don't want to all be forced into the one way of doing things, but it does add this friction, because if we want to work across multiple clouds, we need to hold a lot of knowledge in our head. And even if we want to work on one cloud, we need to hold a lot of knowledge in our head. So if you look at the catalogue for any cloud provider, it just goes on and on and on, and it's like, how do I figure out what I'm supposed to do?
And this of course isn't unique to the cloud providers. If you look at the CNCF landscape, this is now... Yeah. They know this is complex, but it is incredibly complex, at this resolution probably none of you can actually even see what these tools are, much less hold a mental model in your head of how you might choose between them and which ones you need to do your job. And the fun thing about the CNCF landscape is, because it is so overwhelmingly, bewilderingly complex, there's now a whole set of memes about the CNCF landscape and how totally complicated it is. So I liked this one, the 1,000 piece puzzle that you might get someone for Christmas.
There's an interesting aside about this too, because I think probably this tweet, if I showed it to any of you, you would all laugh, or at the very least you know would smile, and if you were in a room probably together, you probably would laugh. If I showed it to some of my friends who do not work in tech, they would stare at it completely blankly, and they'd be like, "I don't even understand why this is supposed to be funny." And that goes back a little bit to the psychology of laughter and the role of laughter in creating team cohesion. So when we laugh at that CNCF puzzle, it is genuine. We're laughing because it makes us laugh, but we're also laughing to show that we get it and to show that we're the sort of person who knows enough about CNCF and cloud computing to know that it's really quite complex and that we're laughing at it. And the person who's never heard of CNCF is there going, "What? Why is this funny?" And all the rest of us are going, "Well, we're a bit cooler than you, aren't we?"
And you see this definitely in rooms. So people will laugh if they're together, because there's that social aspect to it, and people will laugh more if there's a HiPPO phenomenon in laughter. So if the highest paid person in the room is laughing, the others are more likely to laugh as well. So what seems like this really spontaneous thing is actually complex and social.
Back to complexity, though. What ruins cloud for a lot of us isn't the actual technology, even though the technology's hard, it's what we do with it, which is that we take the cloud, this beautiful frictionless environment, and we put it in a cage. And there's really good reasons to do that. There's cost management, there's security, there's all of these good reasons, but it still means that some of the fun comes out of the experience.
And I was talking to a colleague a while ago, and he told me a story, because he was at a client and he was talking to the developers, and he eventually worked out that the developers didn't like OpenShift. And this was quite depressing for him, of course, because he works for Red Hat. And one of the things Red Hat has been doing with OpenShift is trying to take Kubernetes, which is really hard, and make it easier and more integrated and more delightful and more frictionless and all of these good things, and really focused on the developer experience and trying to get this good developer experience. So then he was like, "This is so depressing. We're doing all of this work to give a really good developer experience for OpenShift over raw Kubernetes, and developers don't like it. Why?"
And then he worked out what was going on was that, in that organisation, OpenShift was where the controls were. So OpenShift was the natural layer at which the business could put in all of the governance, and that meant that the developers had... Their flow was impeded, they couldn't do things, and it wasn't anything to do with the developer experience that we'd built for OpenShift, it was just that the business had these needs and they were opposed to the needs of the developers, and OpenShift was the thing that was acting out that little conflict.
And I heard a similar story many, many years ago from an IBM colleague, and this was way back in the prototype beginning of the cloud, when before then it was really hard to provision something. And we'd written something that would work with virtual machines, and it would allow them to provision a virtual machine in 10 minutes, which now is just like, duh, but at the time was amazing. And so we sold this thing that would allow them to provision virtual machines within 10 minutes. And then they came back to us and said, "IBM, nice try, but this provisioning software you sold us is broken." And we thought "that's odd. I'm pretty sure we tested it. What's going on?"
So we investigated, and they were getting... It was taking them three months to provision something. And we're like, that's definitely not what we wrote, what's going on? And then we found that they'd put an approval process in front, and then they'd put an approval process in front of the approval process, and then they'd put an approval process in front of the approval process, and then some other team had its approval process, and by the time you added in all of the approval processes, it was an 84-step approval process, which is kind of terrible. So no wonder they hated it.
I spoke to another client, and they were telling me that they were going to change their cloud provider. And when we asked, "Why are you changing cloud provider?" It was because they had so much procurement wrapped around the cloud provider that they couldn't actually provision things, and they hoped if they changed to a new provider it would be easier to provision things. Which would be true, but probably only for about 10 minutes until the procurement department caught up with it. So it didn't really seem like the right solution.
And I spoke to another business, and they were a bank, and they were really interested in how we set up our developer laptops. And eventually after going on this call for a while with them, I realized that the problem they had was that their developer laptops, they were experimenting with the cloud, and they had so much governance and firewall and everything on their laptops that they had a complete separation between the internal systems and the cloud systems. And so it meant that, as an employee, you had one laptop and you had to choose whether that laptop was able to access Jira and things like internal company meetings, or access the cloud servers. And you couldn't do both.
And so of course they just fundamentally could not do their job because of the way this was set up, and every single process that they did was dragged down into this friction of this setup that made their lives miserable. And there was really big business consequences to this, because they were losing developers. I think their team had lost something like seven developers that quarter, because people go, "I'm not doing this. I could go get a job where I can actually talk to both Jira and my cloud server."
So how do you fix it? Because you can't make the need for governance go away, but it does seem like something has to be done. Well, automation is a really big part of the solution, I think, because all of this crap, we should be giving it to the computers, because so far computers don't expect to have fun, and so we should let them do the tedious stuff. And the bonus is that automating stuff is fun, so then we get this double win. The small risk is that at some point computers may decide that actually they deserve to have the fun, and that they've noticed that we've been giving them all of the most tedious jobs and then they come seeking revenge, but assuming we can leave that and not worry about that, definitely the more we can automate the better.
Another thing that I think really helps is trying to make these infrastructure concerns and these integration concerns someone else's problem. So wherever you can, go platform as a service, rather than just raw bag of bits. This is from IBM Code Engine, which was an implementation of Knative, but I've got it in here because I just really, really love the slogan of "Enjoy your cloud again." And that's what we want the providers to be doing for us, is really focusing on that. And I think Knative in general, and serverless, has quite a lot of potential to abstract away some of these infrastructure concerns. So it used to be when we did serverless it would be functions. That is cool, but it's limited. But if we go to being able to deploy containers using that same model, it can actually be quite easy and fun. But the problem with it is what we call it.
I saw an article and it described them as funtainers, and what it meant by that was a serverless container. And the reason for the name funtainer was sort of function container, because it's deployed like a function, but it's written like a container. This name has completely not taken off. And I can tell you why it hasn't taken off, because if you try to google for funtainer, you'll find lots of pictures of Thermoses. Thermos, for some reason they have a product for children, which is called a Funtainer. So if you try and be more specific, because you know how to do search engines, and you type "funtainer cloud," then what you get is a picture of a Thermos with a picture of a cloud on it. So we still don't have a good name for these things, but I think they are potentially quite a good way of deploying software that gets rid of some of that toil and some of those low level concerns. So there's the original article that talks about them as funtainers, the one article.
But also go managed. I mentioned go PaaS, I tried to illustrate go PaaS. It turns out this is really, really hard for DALL-E. Trying to get it to draw a platform is hard, you only end up with trains. So then I tried to do a cake stand, and I could not get a cake stand without also getting a cake, because no-one on the internet photographs a cake stand without a cake, because why would you photograph a cake stand without a cake? So you have here a computer on a platform, because it's platform as a service, but you also have a bonus cake, which is actually an accident, but I think this is good, because this tries to, actually more accurately than I had realised, captures the full thing of you have that delight, you have that joy, you have the happiness that comes from cakes, and you have a platform and you have a computer. So the computer knew best after all.
And I think it's really good that now we're seeing increasing focus from a lot of vendors on developer experience, which they shorten to DevEx, or if they're feeling really cool they shorten it to DX. And certainly this is something that we see in my day job. So one of the things that I really liked about Quarkus that drove me to join the team was, Quarkus is a cloud native run time, so it's really designed to work well on the cloud, and it's got lots of things that make it really easy to deploy to Kubernetes, but it's also more generally just optimised for developer joy, and developer joy is one of the things that we talk about with Quarkus, which I think every product should be doing.
So I talked a little bit about PaaS and containers and that kind of thing, but I don't want to talk too much about the tools, because going back to some of those firewall things and some of those governance things, those were nothing to do with the technology. The developers weren't having a problem with OpenShift because OpenShift was designed badly. The developers were having a problem with OpenShift because of how it was being used for governance. And friction... Occasionally you get the bad tools, but mostly friction isn't about the product. It's not about the tools. It's about the process. And so we need to try and persuade the business, and also for us, do it so that the easiest thing to do is also the right thing to do, to try and eliminate some of that friction and also improve the results.
So sometimes say, if you care about it, automate it. And it should be the same thing that, if you care about it, make it easy to do, make it easy for your employees to do. DevOps has gone a long way to doing some of this. It kind of missed out security. So again, I think it's really positive that we're seeing the DevSecOps, where we're trying to get that easiness and that flow and that friction for security considerations as well. Because what we really want is... There's this idea from sociology where, if you have two sides with a core conflict, so for example often we have this conflict between innovation and efficiency, or between security and productivity, if we can find this third economy where we find a middle ground, where there's the double win and where actually everybody's better off, we have less friction and we're more secure. I think that's an idea that's at the heart of some of the cloud native development methodologies, it's at the heart of Kubernetes, and I think it's potentially really, really powerful.
So flow and feedback make things fun, so optimise at the tooling level, optimise at the process level for flow, optimise for feedback. Because just like that child with a bingly toy where when they bing something and something happens, that's what's gratifying. If you bing the bell and the bell makes a noise two minutes later, that's miserable. If you bing the bell and the bell makes a noise three months later because it got stuck in a deployment process, that's truly terrible.
So finally, I know I've told a lot of stories about things going wrong and the magic world where everything is frictionless. I do recognize that that can be a bit judgy, and the title is "You're doing it wrong," but of course I think fundamentally probably all of us get it wrong sometime, and it is okay. If our work isn't all fun all the time, we still do have to do the dishes. And fundamentally some platforms are hard to... And changing process is really hard, and it can be really frustrating. And people are hard, and that's just how it is. But hopefully we can have a little bit more of that fun. I told Dan North about this talk, and he suggested an alternate title of "Send in the Clouds," instead of "Send In the Clowns." So hopefully we get a little bit more of that and a little bit less of the tedious and the drag and the horrible and the process.
And with that, I think I've talked quite enough, and we've got some time for questions and some discussion.
Charles Humble:
That was wonderful, Holly. Thank you very much.
Holly Cummins:
Thank you.
Charles Humble:
So does anyone want to put a question in the chat? While we're waiting for people to get going, I'm tempted to ask you about your duck joke survey, because that's awesome, but maybe a more useful question would be... The business of tooling and the idea that the tools we use are often quite difficult and quite frustrating, do you have any thoughts on why that is, and do you think it's changing at all?
Holly Cummins:
I think some of it is a measurement problem. Some of these things that are hard only surface... So from the vendors, the things that are hard sometimes only surface after a month or two of usage. So sometimes you see a demo and it looks really slick, and then it's only when you realise you want to go off that really narrow golden path, it becomes really hard. And in fact that's particularly a problem with some of the tools that have the most slick and delightful golden path, is that the cliff around the golden path is highest. No, that's not a cliff. That's a cliff. In any case, I think we all know the terrible analogy that I'm trying to make. You have this really deep channel for the golden path, and then it's harder to get out of it. So that's on the tooling side.
But I think in the process side as well, it's a bit like technical debt. Technical debt slows us down, and yet sometimes it's really hard to make the business case for fixing it, and it's because new features are easy to measure. Developer unhappiness is hard to measure, and it's hard to quantify the impact of. And so then it just means that, because it's hard to measure, it ends up at the bottom of the priority list.
Charles Humble:
Yes. Yeah. I think that's probably spot on. I think getting metrics for those kind of things is really difficult.
Holly Cummins:
Because you can start to do things like the chief fun officer, but you just know... I talk about that as a joke, because you know immediately when I say that you end up with this dystopian image, don't you, of the chief fun officer who goes around and measures morale, and anybody who's not happy is going to be punished, and everybody has to have six laughs a day and take at least two dips in the corporate ball pit or they're going to be fired. And you just know that's horrible, because it doesn't seem like the sort of thing that can or should be quantified in that way.
Charles Humble:
Yes. I don't know if it's the Britishness in me or something, but I have a complete aversion to enforced jollity of any kind. It's just such a turnoff. I run a mile from companies that do that kind of thing.
Holly Cummins:
Yeah. I have a statistic, it's something like 37% of people dislike team building activities, and that's an American statistic, so I imagine for us in Britain it's probably more like 106% or something.
Charles Humble:
Yes. Excellent. Do we have any other questions from anybody? Holly, are there any things you've... In terms of improving flow, improving productivity, that kind of area, are there any things that you've particularly seen work? Are there any particular approaches that you've seen work? You said that that kind of corporate change is really difficult. I'm just wondering what successes you've seen in it.
Holly Cummins:
I think the starting point is that you need it from bottom up and top down both. So you need that buy-in at the executive level, but they probably shouldn't be the ones deciding what's frictionful and what needs to get fixed. That needs to be bottom-up. And I think there's two things that you can do. One thing that I've seen work really well actually, I've seen it work for innovation, but I think it can work for these process improvements too potentially, is when you have the equivalent of a hackathon, but perhaps ideally during working hours rather than on the weekend, so it's a bit more sustainable. So you say, "For this sprint, we're going to throw away the project plan, we're just going to have everybody work on their passion projects." And often the things that come out of those passion projects are the fixes for the paper cuts. They're the things where it's like, this was really, really hurting me and I couldn't justify fixing it, but now I can.
That works for things where it's a technological thing, of course it doesn't work for people. You can't have a week of meetings in your off-plan sprint in order to persuade everybody to change a process. So for those, I think again often it's... Sometimes I think it's about talking and it's about trying to... Instead of getting frustrated by the process, it's what is this person trying to achieve? What problem do they have? What are they really trying to achieve?
Because they're not really trying to achieve me filling in a form, they're trying to achieve something else. Sometimes actually they're trying to achieve plausible deniability. So they make me fill in the form so that they're excused if something goes wrong. Can I find a way of achieving that same tick-y box exercise in a way that's less friction for me? Or can we actually find a way of overachieving and getting something that actually reduces the likelihood of bad things happening while reducing the effort for me? And that has to be a collaboration across the thing. So talking to security, what are they trying to achieve? What are they afraid of as well? What bad things are they trying to protect against that I could help them protect against in a different way?
Charles Humble:
Yes. Yes, absolutely. There's a related question here from someone called Quentin in the chat saying that security is obviously a huge headache, particularly in the finance sector, so how can high security companies have more fun?
Holly Cummins:
I always recommend everybody for this to Lean Enterprise. So in the final section of Lean Enterprise, there's a section on regulated industries, and they talk a lot about how they manage to find ways of working with flow and without waste in those companies. And that ends up translating quite a lot to finance. And there's a comment in the chat as well that it can be lots of fun to do the things right, especially in security. And I think what we should be going for, and I think what we're getting a bit, is things like... Even something as simple as Dependabot. I would never exactly say Dependabot is fun, it's not like a laugh a minute, but Dependabot has just made something that was kind of tedious completely go away as a concern.
And we're seeing that as well, again, with things like more of the automated tools, so we scan the images for vulnerabilities. Whereas on one of my first cloud projects, what we did was they had an architecture board, and every library that we wanted to use, we had to submit it to the architecture board, and then three weeks later they would come back with an answer. And that's just the opposite of flow, so it was no fun and it was completely counterproductive, and it meant that we ended up doing things like... We couldn't get permission for the version of Jackson that we wanted to use, so we ended up re-implementing Jackson. It was such a phenomenal waste of time, and it was so counterproductive. And so if we can get to where it's automated, to where it's easy, then there's the productivity benefit as well.
Charles Humble:
Brilliant. I'm just... Anyone else got a question? I think at that point we should wrap it up. So maybe I could ask everyone to, I don't know, turn your microphones on or your cameras on and give Holly a virtual round of applause, which always feels a bit weird, but slightly less weird than stopping in total silence. Holly, thank you so much for doing that. It's been a really good session, so thank you very much indeed.
Holly Cummins:
Thank you.
Charles Humble:
Really appreciate it.
Holly Cummins:
My pleasure.
Charles Humble:
And thanks everybody for coming along. We will be back with another of these either in January or February in the New Year, but in the meantime, have a lovely Christmas holiday, whatever you're doing over the next couple of weeks. And Holly, thank you again very much indeed for joining us. Tereza, have I missed anything?
Tereza Astrop:
No, it was A+.
Charles Humble:
Awesome. Jolly good.
Julien Sudan:
Thank you.
Charles Humble:
Fantastic.
Tereza Astrop:
Awesome.
Charles Humble:
Thanks everyone so much. Bye-bye.
Tereza Astrop:
Take care. Bye.