istio.io/content/en/docs/tasks/traffic-management/mirroring/index.md

title	description	weight	keywords	owner	test
Mirroring	This task demonstrates the traffic mirroring/shadowing capabilities of Istio.	60	traffic-management mirroring	istio/wg-networking-maintainers	yes

This task demonstrates the traffic mirroring capabilities of Istio.

Traffic mirroring, also called shadowing, is a powerful concept that allows feature teams to bring changes to production with as little risk as possible. Mirroring sends a copy of live traffic to a mirrored service. The mirrored traffic happens out of band of the critical request path for the primary service.

In this task, you will first force all traffic to v1 of a test service. Then, you will apply a rule to mirror a portion of traffic to v2.

{{< boilerplate gateway-api-support >}}

Before you begin

1. Set up Istio by following the Installation guide.

2. Start by deploying two versions of the [httpbin]({{< github_tree >}}/samples/httpbin) service that have access logging enabled:

   1. Deploy httpbin-v1:

      {{< text bash >}} $ kubectl create -f - <<EOF apiVersion: apps/v1 kind: Deployment metadata: name: httpbin-v1 spec: replicas: 1 selector: matchLabels: app: httpbin version: v1 template: metadata: labels: app: httpbin version: v1 spec: containers: - image: docker.io/kennethreitz/httpbin imagePullPolicy: IfNotPresent name: httpbin command: ["gunicorn", "--access-logfile", "-", "-b", "0.0.0.0:80", "httpbin:app"] ports: - containerPort: 80 EOF {{< /text >}}

   2. Deploy httpbin-v2:

      {{< text bash >}} $ kubectl create -f - <<EOF apiVersion: apps/v1 kind: Deployment metadata: name: httpbin-v2 spec: replicas: 1 selector: matchLabels: app: httpbin version: v2 template: metadata: labels: app: httpbin version: v2 spec: containers: - image: docker.io/kennethreitz/httpbin imagePullPolicy: IfNotPresent name: httpbin command: ["gunicorn", "--access-logfile", "-", "-b", "0.0.0.0:80", "httpbin:app"] ports: - containerPort: 80 EOF {{< /text >}}

   3. Deploy the httpbin Kubernetes service:

      {{< text bash >}} $ kubectl create -f - <<EOF apiVersion: v1 kind: Service metadata: name: httpbin labels: app: httpbin spec: ports:

      • name: http port: 8000 targetPort: 80 selector: app: httpbin EOF {{< /text >}}

3. Deploy the sleep workload you'll use to send requests to the httpbin service:

   {{< text bash >}} $ cat <<EOF | kubectl create -f - apiVersion: apps/v1 kind: Deployment metadata: name: sleep spec: replicas: 1 selector: matchLabels: app: sleep template: metadata: labels: app: sleep spec: containers: - name: sleep image: curlimages/curl command: ["/bin/sleep","3650d"] imagePullPolicy: IfNotPresent EOF {{< /text >}}

Creating a default routing policy

By default Kubernetes load balances across both versions of the httpbin service. In this step, you will change that behavior so that all traffic goes to v1.

1. Create a default route rule to route all traffic to v1 of the service:

   {{< tabset category-name="config-api" >}}

   {{< tab name="Istio APIs" category-value="istio-apis" >}}

   {{< text bash >}} $ kubectl apply -f - <<EOF apiVersion: networking.istio.io/v1 kind: VirtualService metadata: name: httpbin spec: hosts: - httpbin http:

   • route:
     • destination: host: httpbin subset: v1 weight: 100

   ---

   apiVersion: networking.istio.io/v1 kind: DestinationRule metadata: name: httpbin spec: host: httpbin subsets:

   • name: v1 labels: version: v1
   • name: v2 labels: version: v2 EOF {{< /text >}}

   {{< /tab >}}

   {{< tab name="Gateway API" category-value="gateway-api" >}}

   {{< text bash >}} $ kubectl apply -f - <<EOF apiVersion: v1 kind: Service metadata: name: httpbin-v1 spec: ports:

   • port: 80 name: http selector: app: httpbin version: v1

   ---

   apiVersion: v1 kind: Service metadata: name: httpbin-v2 spec: ports:

   • port: 80 name: http selector: app: httpbin version: v2

   ---

   apiVersion: gateway.networking.k8s.io/v1 kind: HTTPRoute metadata: name: httpbin spec: parentRefs:

   • group: "" kind: Service name: httpbin port: 8000 rules:
   • backendRefs:
     • name: httpbin-v1 port: 80 EOF {{< /text >}}

   {{< /tab >}}

   {{< /tabset >}}

2. Now, with all traffic directed to httpbin:v1, send a request to the service:

   {{< text bash json >}} $ kubectl exec deploy/sleep -c sleep -- curl -sS http://httpbin:8000/headers { "headers": { "Accept": "/", "Content-Length": "0", "Host": "httpbin:8000", "User-Agent": "curl/7.35.0", "X-B3-Parentspanid": "57784f8bff90ae0b", "X-B3-Sampled": "1", "X-B3-Spanid": "3289ae7257c3f159", "X-B3-Traceid": "b56eebd279a76f0b57784f8bff90ae0b", "X-Envoy-Attempt-Count": "1", "X-Forwarded-Client-Cert": "By=spiffe://cluster.local/ns/default/sa/default;Hash=20afebed6da091c850264cc751b8c9306abac02993f80bdb76282237422bd098;Subject="";URI=spiffe://cluster.local/ns/default/sa/default" } } {{< /text >}}

3. Check the logs from httpbin-v1 and httpbin-v2 pods. You should see access log entries for v1 and none for v2:

   {{< text bash >}} $ kubectl logs deploy/httpbin-v1 -c httpbin 127.0.0.1 - - [07/Mar/2018:19:02:43 +0000] "GET /headers HTTP/1.1" 200 321 "-" "curl/7.35.0" {{< /text >}}

   {{< text bash >}} $ kubectl logs deploy/httpbin-v2 -c httpbin {{< /text >}}

Mirroring traffic to httpbin-v2

1. Change the route rule to mirror traffic to httpbin-v2:

   {{< tabset category-name="config-api" >}}

   {{< tab name="Istio APIs" category-value="istio-apis" >}}

   {{< text bash >}} $ kubectl apply -f - <<EOF apiVersion: networking.istio.io/v1 kind: VirtualService metadata: name: httpbin spec: hosts: - httpbin http:

   • route:
     • destination: host: httpbin subset: v1 weight: 100 mirror: host: httpbin subset: v2 mirrorPercentage: value: 100.0 EOF {{< /text >}}

   This route rule sends 100% of the traffic to v1. The last stanza specifies that you want to mirror (i.e., also send) 100% of the same traffic to the httpbin:v2 service. When traffic gets mirrored, the requests are sent to the mirrored service with their Host/Authority headers appended with -shadow. For example, cluster-1 becomes cluster-1-shadow.

   Also, it is important to note that these requests are mirrored as "fire and forget", which means that the responses are discarded.

   You can use the value field under the mirrorPercentage field to mirror a fraction of the traffic, instead of mirroring all requests. If this field is absent, all traffic will be mirrored.

   {{< /tab >}}

   {{< tab name="Gateway API" category-value="gateway-api" >}}

   {{< text bash >}} $ kubectl apply -f - <<EOF apiVersion: gateway.networking.k8s.io/v1 kind: HTTPRoute metadata: name: httpbin spec: parentRefs:

   • group: "" kind: Service name: httpbin port: 8000 rules:
   • filters:
     • type: RequestMirror requestMirror: backendRef: name: httpbin-v2 port: 80 backendRefs:
     • name: httpbin-v1 port: 80 EOF {{< /text >}}

   This route rule sends 100% of the traffic to v1. The RequestMirror filter specifies that you want to mirror (i.e., also send) 100% of the same traffic to the httpbin:v2 service. When traffic gets mirrored, the requests are sent to the mirrored service with their Host/Authority headers appended with -shadow. For example, cluster-1 becomes cluster-1-shadow.

   Also, it is important to note that these requests are mirrored as "fire and forget", which means that the responses are discarded.

   {{< /tab >}}

   {{< /tabset >}}

2. Send the traffic:

   {{< text bash >}} $ kubectl exec deploy/sleep -c sleep -- curl -sS http://httpbin:8000/headers {{< /text >}}

   Now, you should see access logging for both v1 and v2. The access logs created in v2 are the mirrored requests that are actually going to v1.

   {{< text bash >}} $ kubectl logs deploy/httpbin-v1 -c httpbin 127.0.0.1 - - [07/Mar/2018:19:02:43 +0000] "GET /headers HTTP/1.1" 200 321 "-" "curl/7.35.0" 127.0.0.1 - - [07/Mar/2018:19:26:44 +0000] "GET /headers HTTP/1.1" 200 321 "-" "curl/7.35.0" {{< /text >}}

   {{< text bash >}} $ kubectl logs deploy/httpbin-v2 -c httpbin 127.0.0.1 - - [07/Mar/2018:19:26:44 +0000] "GET /headers HTTP/1.1" 200 361 "-" "curl/7.35.0" {{< /text >}}

Cleaning up

1. Remove the rules:

   {{< tabset category-name="config-api" >}}

   {{< tab name="Istio APIs" category-value="istio-apis" >}}

   {{< text bash >}} $ kubectl delete virtualservice httpbin $ kubectl delete destinationrule httpbin {{< /text >}}

   {{< /tab >}}

   {{< tab name="Gateway API" category-value="gateway-api" >}}

   {{< text bash >}} $ kubectl delete httproute httpbin $ kubectl delete svc httpbin-v1 httpbin-v2 {{< /text >}}

   {{< /tab >}}

   {{< /tabset >}}

2. Delete httpbin and sleep deployments and httpbin service:

   {{< text bash >}} $ kubectl delete deploy httpbin-v1 httpbin-v2 sleep $ kubectl delete svc httpbin {{< /text >}}