Skip to main content

Traffic Management with Istio on OVHcloud Managed Kubernetes

Michael M.
Updated

Istio is an open source service mesh and platform to reduce the complexity of deploying, securing, controlling and observing distributed services. As the Istio site explains, Istio helps you to:

  • Control the flow of traffic between services
  • Secure the services and manage the authentication, authorization and encryption of inter-service communications
  • Apply and enforce policies on distributes services
  • Monitor the services gathering metrics, logs and traces

In this tutorial we are showing some of the traffic management features of Istio, and how you can use them on your OVHcloud Managed Kubernetes cluster.

Before you begin

This tutorial presupposes that you already have a working OVHcloud Managed Kubernetes cluster, and some basic knowledge of how to operate it. If you want to know more on those topics, please look at the deploying a Hello World application documentation.

It also supposes you have a basic knowledge of Istio, and that you have installed it in your Kubernetes cluster. If it isn’t the case, please follow Istio's tutorial. We are going to use the Bookinfo example application, as we did in that tutorial, so if you haven’t installed it, please do it now.

Deploying an application

To verify that Istio is truly working in the cluster, you are going to deploy a test application. We have choosen the Bookinfo application, as it’s a multi-technology multi-instance microservices-based application that let’s you verify if Istio works as intended.

installing-istio-bookinfo.png

Installing Bookinfo

The Istio-Sidecar-injector, that you installed with Istio, will automatically inject Envoy containers into your application pods. The injector assumes the application pods are running in namespaces labeled with istio-injection=enabled. Let’s create and label a istio-apps namespace:

# kubectl create namespace istio-apps

And now, deploy the bookinfo manifest into the namespace:

# kubectl create -n istio-apps -f <(./bin/istioctl kube-inject -f samples/bookinfo/platform/kube/bookinfo.yaml)

The above command installs and launches all four microservices as illustrated in the above diagram: details, productpage, ratings and the three versions of reviews:

# kubectl create -n istio-apps -f <(./bin/istioctl kube-inject -f samples/bookinfo/platform/kube/bookinfo.yaml) 
service/details created
deployment.extensions/details-v1 created
service/ratings created
deployment.extensions/ratings-v1 created
service/reviews created
deployment.extensions/reviews-v1 created
deployment.extensions/reviews-v2 created
deployment.extensions/reviews-v3 created
service/productpage created
deployment.extensions/productpage-v1 created

Now you can verify that all services and pods are correctly defined and running:

  1. Use kubectl -n istio-apps get services to verify that the details, productpage, ratings and reviews services are up un running:

     

    # kubectl -n istio-apps get services
NAME          TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)    AGE
details       ClusterIP   10.3.203.54    <none>        9080/TCP   2m
productpage   ClusterIP   10.3.238.61    <none>        9080/TCP   2m
ratings       ClusterIP   10.3.184.226   <none>        9080/TCP   2m
reviews       ClusterIP   10.3.230.177   <none>        9080/TCP   2m

     

  2. Use kubectl -n istio-apps get pods to verify that the details-v1-*, productpage-v1-*, ratings-v1-*, reviews-v1-*, reviews-v2-* and reviews-v3-* are up and running:

    In the case of my example cluster:

     

    # kubectl -n istio-apps get pods
NAME                             READY   STATUS    RESTARTS   AGE
details-v1-6764bbc7f7-tcgph      2/2     Running   0          4m
productpage-v1-54b8b9f55-jvdzm   2/2     Running   0          4m
ratings-v1-7bc85949-tmb8q        2/2     Running   0          4m
reviews-v1-fdbf674bb-gkzdq       2/2     Running   0          4m
reviews-v2-5bdc5877d6-k6x6q      2/2     Running   0          4m
reviews-v3-dd846cc78-lcwrm       2/2     Running   0          4m

     

Determining the ingress IP and port

Now that the Bookinfo services are up and running, you need to make the application accessible from outside of your Kubernetes cluster, e.g., from a browser. An Istio Gateway is used for this purpose.

  1. Define the ingress gateway for the application:

    # kubectl -n istio-apps apply -f samples/bookinfo/networking/bookinfo-gateway.yaml

  2. Confirm the gateway has been created:

    # kubectl -n istio-apps get gateway

  3. Set GATEWAY_URL, the URL of the istio-gateway service.

    You can get it with kubectl -n istio-system get service istio-ingressgateway, under the EXTERNAL_IP fiel or directly using a jsonpath in the request:

    # export GATEWAY_URL=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')

In the case of my example cluster:

# kubectl -n istio-apps apply -f samples/bookinfo/networking/bookinfo-gateway.yaml
gateway.networking.istio.io/bookinfo-gateway created
virtualservice.networking.istio.io/bookinfo created
# kubectl -n istio-apps get gateway
NAME               AGE
bookinfo-gateway   28s
# kubectl -n istio-system get service istio-ingressgateway
NAME                   TYPE           CLUSTER-IP     EXTERNAL-IP                        PORT(S)  
istio-ingressgateway   LoadBalancer   10.3.209.149   xxxrs44urc.lb.c1.gra.k8s.ovh.net   15020:31621/TCP,80:31380/TCP,443:31390/TCP,31400:31400/TCP...

# export GATEWAY_URL=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')

# echo $GATEWAY_URL
203.0.113.1

Confirm the app is running

To confirm that the Bookinfo application is running, run the following curl command:

# curl -o /dev/null -s -w "%{http_code}\n" http://${GATEWAY_URL}/productpage

You should get an HTTP status code 200 indicating that your productpage is OK.

# curl -o /dev/null -s -w "%{http_code}\n" http://${GATEWAY_URL}/productpage
200

You can also point your browser to http://<YOUR_GATEWAY_URL>/productpage (in my example to http://clusterId.nodes.c1.gra.k8s.ovh.net:31380/productpage) to view the Bookinfo web page. If you refresh the page several times, you should see different versions of reviews shown in productpage, presented in a round robin style (red stars, black stars, no stars), since we haven’t yet used Istio to control the version routing.

installing-istio-02.jpeg

What’s next?

Now you have a working Bookinfo app deployed on Istio, you can follow the suggestions of the Bookinfo sample app page and use this sample to experiment with Istio’s features for traffic routing, fault injection, rate limiting, etc. To proceed, refer to one or more of the Istio Examples, depending on your interest. Intelligent Routing is a good place to start for beginners.

Cleanup

To uninstall the Bookinfo app, the easiest way is to use the provided cleanup.sh script:

# ./samples/bookinfo/platform/kube/cleanup.sh
namespace ? istio-apps
using NAMESPACE=istio-apps
Application cleanup may take up to one minute
service "details" deleted
deployment.extensions "details-v1" deleted
service "ratings" deleted
deployment.extensions "ratings-v1" deleted
service "reviews" deleted
deployment.extensions "reviews-v1" deleted
deployment.extensions "reviews-v2" deleted
deployment.extensions "reviews-v3" deleted
service "productpage" deleted
deployment.extensions "productpage-v1" deleted
Application cleanup successful

To confirm the shutdown you can list the virtual services, destination rules, gateway and pods in the istio-apps namespace:

# kubectl -n istio-apps get virtualservices   #-- there should be no virtual services
kubectl -n istio-apps get destinationrules  #-- there should be no destination rules
kubectl -n istio-apps get gateway           #-- there should be no gateway
kubectl -n istio-apps get pods              #-- there should be no pod

In my example cluster:

# kubectl -n istio-apps get virtualservices   #-- there should be no virtual services
No resources found.
# kubectl -n istio-apps get destinationrules  #-- there should be no destination rules
No resources found.
# kubectl -n istio-apps get gateway           #-- there should be no gateway
No resources found.
# kubectl -n istio-apps get pods              #-- there should be no pod
No resources found.

Now you can uninstall Istio using again the istio-demo-auth manifest:

# kubectl delete -f install/kubernetes/istio-demo.yaml

Example on my cluster:

# kubectl delete -f install/kubernetes/istio-demo.yaml
configmap "istio-galley-configuration" deleted
configmap "istio-statsd-prom-bridge" deleted
configmap "prometheus" deleted
configmap "istio-security-custom-resources" deleted
configmap "istio" deleted
configmap "istio-sidecar-injector" deleted
serviceaccount "istio-galley-service-account" deleted
serviceaccount "istio-egressgateway-service-account" deleted
serviceaccount "istio-ingressgateway-service-account" deleted
serviceaccount "istio-mixer-service-account" deleted
serviceaccount "istio-pilot-service-account" deleted
serviceaccount "prometheus" deleted
serviceaccount "istio-cleanup-secrets-service-account" deleted
clusterrole.rbac.authorization.k8s.io "istio-cleanup-secrets-istio-system" deleted
...

Preparing the Bookinfo app

Before you can use Istio to control the Bookinfo version routing, you need to define the available versions, called subsets, in destination rules.

Go to your Istio install folder and apply the DestinationRules for Bookinfo:

# kubectl -n istio-apps apply -f samples/bookinfo/networking/destination-rule-all-mtls.yaml

Then wait a few moments to for the destination rules to propagate and verify them:

# kubectl -n istio-apps get destinationrules -o yaml

In the case of my example cluster:

# kubectl -n istio-apps apply -f samples/bookinfo/networking/destination-rule-all-mtls.yaml
destinationrule.networking.istio.io/productpage created
destinationrule.networking.istio.io/reviews created
destinationrule.networking.istio.io/ratings created
destinationrule.networking.istio.io/details created

# kubectl -n istio-apps get destinationrules -o yaml
apiVersion: v1
items:
- apiVersion: networking.istio.io/v1alpha3
  kind: DestinationRule
  metadata:
    annotations:
      kubectl.kubernetes.io/last-applied-configuration: |
        {"apiVersion":"networking.istio.io/v1alpha3","kind":"DestinationRule","metadata":{"annotations":{},"name":"details","namespace":"istio-apps"},"spec":{"host":"details","subsets":[{"labels":{"version":"v1"},"name":"v1"},{"labels":{"version":"v2"},"name":"v2"}],"trafficPolicy":{"tls":{"mode":"ISTIO_MUTUAL"}}}}
    creationTimestamp: 2019-07-02T23:23:02Z
    generation: 1
    name: details
    namespace: istio-apps
    resourceVersion: "13136422929"
    selfLink: /apis/networking.istio.io/v1alpha3/namespaces/istio-apps/destinationrules/details
    uid: 56437acb-9d20-11e9-8f94-f2cd6677333a
  spec:
    host: details
    subsets:
    - labels:
        version: v1
      name: v1
    - labels:
        version: v2
      name: v2
    trafficPolicy:
      tls:
        mode: ISTIO_MUTUAL
[...]

A/B Testing with Istio

A/B Testing is used when we want to try two different versions of an application and compare user interaction and engagement to choose the best one. It requires to be able to push the two versions into production at the same time, split traffic between the two versions, and collect metrics to be able to do an informed choice.

A/B Testing used to be a difficult problem with traditional deployment methods, and it’s very hard to do it directly in Kubernetes since there is no notion of versions, but Istio make it rather simple.

Let’s use the Bookinfo application to show how easily you can do A/B Testing on Kubernetes with Istio. The Bookinfo application composed of four separate microservices:

  • productpage: it calls the reviews and details service and build the page
  • reviews: it contains with book reviews, and it calls the rating service
  • details: it contains the book information
  • ratings: it contains the book rating information

To setup A/B Testing on Bookinfo we are using the reviews microservice, as it has three versions:

  • v1: it doesn’t call the ratings service
  • v2: it calls the ratings service and displays the rating as black stars
  • v3: it calls the ratings service and displays the rating as red stars

bookinfo-architecture.png

By default, the Bookinfo install deploys the three versions without explicit routing definitions. Istio then routes the requests to all available versions of reviews in a round robin fashion, so sometimes the book review output contains star ratings and other times it does not.

Let’s say we want to send 50% of traffic to v2 and get the black stars, and the other 50% to v3 and its red stars. We can create a VirtualService to define this behavior:

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: reviews
spec:
  hosts:
    - reviews
  http:
  - route:
    - destination:
        host: reviews
        subset: v2
      weight: 50
    - destination:
        host: reviews
        subset: v3
      weight: 50

Write down the VirtualService into a reviews-50-v2-50-v3.yaml file and apply it:

# kubectl apply -n istio-apps -f reviews-50-v2-50-v3.yaml

and confirm that the rule is created:

# kubectl -n istio-apps get virtualservice reviews -o yaml

In the case of my example cluster:

# kubectl apply -n istio-apps -f ./reviews-50-v2-50-v3.yaml 
virtualservice.networking.istio.io/reviews created

# kubectl -n istio-apps get virtualservice reviews -o yaml
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
[...]
spec:
  hosts:
  - reviews
  http:
  - route:
    - destination:
        host: reviews
        subset: v2
      weight: 50
    - destination:
        host: reviews
        subset: v3
      weight: 50

Now, on the /productpage of the Bookinfo app, at every refresh you will see that stars change alternatively between back (v2) and red (v3).

ab-testing-001.png

Canary Testing with Istio

Similar to A/B Testing, Canary Testing is pushing a new version of a service to a small group of users. The idea is to test the new release in a reduced number of real-world users, to quickly discovering potential bugs while minimizing the number of users impacted.

The strategy is called Canary Testing because canaries were once used in coal mining to alert miners when toxic gases reached dangerous levels. Like the canary in a coal mine, the end user who is selected to receive the new release is unaware he is being used to provide an early warning.

Doing Canary Testing on Kubernetes with Istio is similar to A/B Testing.

Let’s begin by redirecting all the reviews traffic to v1, it will be our stable release:

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: reviews
spec:
  hosts:
    - reviews
  http:
  - route:
    - destination:
        host: reviews
        subset: v1
      weight: 100

Write down the VirtualService into a review-all-v1.yaml file and apply it:

# kubectl apply -f review-all-v1.yaml

and confirm that the rule is created:

# kubectl -n istio-apps get virtualservice reviews -o yaml

At this moment all the traffic goes to the v1 version of reviews, without ratings:

canary-testing-001.png

Now let’s say we want to send the v2 to 5% of the traffic. We simply define the new behavior in a YAML file:

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: reviews
spec:
  hosts:
    - reviews
  http:
  - route:
    - destination:
        host: reviews
        subset: v1
      weight: 90
    - destination:
        host: reviews
        subset: v2
      weight: 10

Write down the VirtualService into a reviews-90-v1-10-v2.yaml file and apply it:

# kubectl -n istio-apps apply -f review-90-v1-10-v2.yaml

and confirm that the rule is created:

# kubectl -n istio-apps get virtualservice reviews -o yaml

Now, on the /productpage of the Bookinfo app, 9 times out of 10 you will get the v1, without ratings, and 1 out of 10 times you will get the v2, with black stars.

canary-testing-002.png

Rolling Deployments and Blue/Green Deployments

Both Rolling Deployments and Blue/Green Deployments are deployment strategies to ensure that new releases can be shipped without any downtime.

rolling-deployment-001.png

Doing a Rolling Deployment on Istio is rather simple, you can take as base te examples of Canary Testing and A/B Testing.

Let’s apply again the reviews-all-v1.yaml file to redirect all the reviews traffic to the v1:

# kubectl -n istio-apps apply -f reviews-all-v1.yaml

The v1 is then our initial release. Let’s say we want to ship v2 as new release, but using Rolling Deployment to be sure there is no downtime. The easiest wa would be pass by an intermediate state where traffic is split 50%-50% between v1 and v2:

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: reviews
spec:
  hosts:
    - reviews
  http:
  - route:
    - destination:
        host: reviews
        subset: v1
      weight: 50
    - destination:
        host: reviews
        subset: v2
      weight: 50

Write down the VirtualService into a reviews-50-v1-50-v2.yaml file and apply it:

# kubectl -n istio-apps -f reviews-50-v1-50-v2.yaml

At this moment traffic will be equally split between the two releases of reviews.

rolling-deployment-002.png

If everything is correct, we can safely route all the traffic to v2:

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: reviews
spec:
  hosts:
    - reviews
  http:
  - route:
    - destination:
        host: reviews
        subset: v2
      weight: 100

Write down the VirtualService into a reviews-all-v2.yaml file and apply it:

# kubectl -n istio-apps apply -f reviews-all-v2.yaml

And now the v2 of reviews receives all the traffic, and our Rolling Deployment is completed.

rolling-deployment-003.png

What’s next?

Now you have seem some of the traffic management capabilities of Istio, you can explore other examples of Istio traffic management: fault injection, circuit breaking, mirroring…

Related articles