Solving 2 Common Deployment Dilemmas in Kubernetes With Istio

If you’re an engineer working with containers, and especially with Kubernetes, you’re going to hear about Istio. For the uninitiated, Istio is the service mesh for Kubernetes. A service mesh is a networking layer that allows you to dynamically manage service traffic, and do so in a safe and well-defined way.

Getting the most out of Istio is definitely beyond the scope of any single blog post. But here, I’ll introduce some of its features and, more importantly, some ways you can leverage them to automate elegant solutions to some real world problems.

Istio allows you to manage network traffic with a collection of custom Kubernetes resources, and can help you secure and encrypt network traffic between services as well as in and out of your cluster(s). Its comprehensive integration with the Kubernetes API means that your Istio settings can be defined and managed in the exact same way as the rest of your Kubernetes configurations.

Caveat Emptor

If you want to started with Istio, you should first ask yourself why. Istio offers some very valuable features, but you don’t get to use them without adding some complexity. It will also require a reasonable time investment to gain the necessary expertise. That said, if your use case fits, you can (and should) adopt Istio’s features in your own cluster(s), carefully and incrementally.

If you’re building a new environment from scratch, and have weighed the pros and cons and decided to go ahead with Istio, then by all means, set it up from the start with strict mutual TLS, embrace its power, and don’t look back. Here’s how to do that.

Enter containersol/k8s-deployment-strategies

In order for this all to make sense, we need to think about Istio in the context of a real-world application, but doing so without a quick disclaimer would be irresponsible. If you only need to manage a small number of services, living on a single cluster, Istio will probably introduce far more complexity than it’s worth.

With that caveat out of the way, here’s a small single-cluster application that probably doesn’t warrant using Istio—but sometimes hitting a nail with a sledgehammer is fun and educational. The application is about as simple as it can be: it serves a web page that gives you a little information about the pod it’s running on, and prints the version number, so you know which version you are being served.

The code examples in this article won’t get you all the way there, but all the code you need, along with detailed instructions on how to use it are available on GitLab. What follows are two common problems you are likely to encounter in your Cloud Native journey and how you can deploy Istio to handle them.

Problem 1: ‘I don’t trust my tests’

If you can write production code that covers all cases then you can write tests that cover all cases.

— Uncle Bob Martin (@unclebobmartin) 3 June 2019

If you introduce changes to an application with less than complete test coverage, you might be moving fast, but you also might be breaking things.

In an ideal world, features would never get added to a software project without making sure that every code path is thoroughly tested. But we don’t live in an ideal world. Deadlines get set, features get prioritized, and tests don’t get written or updated.

The Solution: Slow your roll(out)

So how can I make changes and deploy new features, while ensuring that (the vast majority of) my users are unaffected by any unforeseen bugs lurking in my code? The answer is to minimise the blast-radius of those little buggers by first deploying your new version to a minimal number of users.

Once you are satisfied that the changes work as expected, you can slowly increase the percentage of users that get served by the new version. If the support calls start rolling in, or your graphs start to plummet, it’s easy to roll back your changes, and try again.

Can you run canary deployments on Kubernetes without Istio? You sure can, but if want to automate the process (and you do want to automate the process, right?), you’ll be up to your neck in jq, web server code, and custom automation scripts. This is not the kind of complexity you should be adding to your environment.

Istio has some really elegant solutions for traffic distribution that we can use to serve the right clients with the right version at the right time, and we only need to worry about adjusting one or two parameters.

In order to make it happen, you’ll need to set up an ingress gateway, a virtual service ,and a destination rule. This will sit on top of your usual deployment and service, and take care of traffic distribution for you.

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: http-gateway
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hos
ts:
    - "*"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: my-app
spec:
  hosts:
    - "*"
  gateways:
    - http-gateway
  http:
  - match:
    - uri:
        prefix: "/my-app"
    rewrite:
      uri: "/"
    route:
      - destination:
          host: my-app
          subset: v1
          port:
            number: 80
        weight: 90
      - destination:
          host: my-app
          subset: v2
          port:
            number: 80
        weight: 10
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: my-app
spec:
  host: my-app
  subsets:
  - name: v1
    labels:
      version: v1.0.0
  - name: v2
    labels:
      version: v2.0.0
    

As you can see from the weight field of the virtual service, Istio will divide traffic between both versions of your application according to the specified values. These values must add up to 100%, otherwise, the API will refuse to apply the definition.

Then you (or, ideally, one or more manual steps in your Continuous Integration/Continuous Delivery pipeline), will adjust the weights to roll out your new version to more of your users, until all requests are served by the new version, and the previous version can be taken out of production.

It is also possible to integrate Istio into your integration-testing strategy, by using its fault-injection features to simulate network interruptions and performance degradation on real traffic.

If the idea of testing in production puts a bitter taste in the back of your throat, then you definitely aren’t doing enough of it. For example, try adding the following snippet to your VirtualService spec to add a little turmoil into the mix, and look out for another article about chaos engineering with Istio in the near future.

spec:
  hosts:
  - my-app
  http:
  - fault:
      delay:
        fixedDelay: 7s
        percent: 100
   route:
    - destination:
        host: ratings
        subset: v2

    

Problem #2: Marketing can’t make up its mind

Often there’s a business need to test multiple versions of your application against your actual users. Sometimes it’s unclear which marketing strategy will lead to the best conversion rates, or what design choices will lead to the best customer retention.

With vanilla Kubernetes, you can split your traffic between two versions, but getting any valuable insight from the exercise will again require a whole bunch of custom code to capture the relevant information and process it in a way that your non-technical colleagues can digest.

The Solution: Practical A/B testing with Istio

Istio’s traffic distribution rules can once again come to the rescue, and its tight integration with Prometheus and Grafana can help you present the results of your A/B tests in a compelling and meaningful way. There are a nearly unlimited number of ways to decide who gets which version of your application, usually based on some part of the incoming packet’s content.

In this example, we’ll use the User-Agent field to serve different versions to different browsers.

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: my-app
spec:
  hosts:
    - "*"
  gateways:
    - http-gateway
  http:
  - match:
    - headers:
        user-agent:
          regex: ".*Chrome.*"
      uri:
        prefix: "/my-app"
    rewrite:
      uri: "/"
    route:
      - destination:
          host: my-app
          subset: v1
          port:
            number: 80
  - match:
    - headers:
        user-agent:
          regex: ".*Mozilla.*"
      uri:
        prefix: "/my-app"
    rewrite:
      uri: "/"
    route:
      - destination:
          host: my-app
          subset: v2
          port:
            number: 80

    

As you can see from the above code, people using Firefox will get version 1 of the application, and Chrome users will get version 2. If the browser’s User-Agent field doesn’t contain “mozilla” or “chrome”, then they will get neither.

In order to serve any other clients, you need to add a default route—which I will leave as an exercise for the reader.

If you don’t want to install different browsers just to try this out, you can use curl with the header flag to masquerade as whatever browser you want. For example:

curl /my-app -H "User-Agent: Chrome"
    

By changing the value of the user-agent, you can test all of your different routes from the command line.

More to Come

These two scenarios barely scratch the surface of what you can do with Istio, but hopefully they’ve given you a taste of its power. Without Istio, you could still do canary deployments and A/B testing, but you’d have to implement the traffic distribution yourself, and that kind of stuff doesn’t really belong in your application code anyway.

I hope this has given you a good sense of some of the things you can do with Istio, and motivates you to try it out for yourself. If you’re interested in learning more, there are some great resources available on the Istio website, and don’t forget to check back here for more tutorials in the future.

We have recently launched Istio training workshop! In this one day workshop you will learn how to get started with Istio and how it helps to upgrade your microservices infrastructure. Click below to learn more: