litmus-docs/website/versioned_docs/version-1.7.0/pod-network-duplication.md

id	title	sidebar_label	original_id
pod-network-duplication	Pod Network Duplication Experiment Details	Pod Network Duplication	pod-network-duplication

---

Experiment Metadata

Type	Description	Tested K8s Platform
Generic	Inject Packet Duplication Into Application Pod	GKE, Packet(Kubeadm), EKS, Minikube > v1.6.0, AKS

Prerequisites

• Ensure that the Litmus Chaos Operator is running by executing kubectl get pods in operator namespace (typically, litmus). If not, install from here
• Ensure that the pod-network-duplication experiment resource is available in the cluster by executing kubectl get chaosexperiments in the desired namespace. If not, install from here
  NOTE: Experiment is supported only on Docker Runtime. Support for containerd/CRIO runtimes will be added in subsequent releases.

Entry Criteria

• Application pods are healthy before chaos injection

Exit Criteria

• Application pods are healthy post chaos injection

Details

• pod-network-duplication injects chaos to disrupt network connectivity to kubernetes pods.
• The application pod should be healthy once chaos is stopped. Service-requests should be served despite chaos.
• Causes Injection of network duplication on the specified container by starting a traffic control (tc) process with netem rules to add egress delays. It Can test the application's resilience to duplicate network.

Steps to Execute the Chaos Experiment

• This Chaos Experiment can be triggered by creating a ChaosEngine resource on the cluster. To understand the values to provide in a ChaosEngine specification, refer Getting Started

• Follow the steps in the sections below to create the chaosServiceAccount, prepare the ChaosEngine & execute the experiment.

Prepare chaosServiceAccount

• Use this sample RBAC manifest to create a chaosServiceAccount in the desired (app) namespace. This example consists of the minimum necessary role permissions to execute the experiment.

Sample Rbac Manifest

apiVersion: v1
kind: ServiceAccount
metadata:
  name: pod-network-duplication-sa
  namespace: default
  labels:
    name: pod-network-duplication-sa
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
  name: pod-network-duplication-sa
  namespace: default
  labels:
    name: pod-network-duplication-sa
rules:
  - apiGroups: ["", "litmuschaos.io", "batch"]
    resources:
      [
        "pods",
        "jobs",
        "events",
        "pods/log",
        "chaosengines",
        "chaosexperiments",
        "chaosresults",
      ]
    verbs: ["create", "list", "get", "patch", "update", "delete"]
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: pod-network-duplication-sa
  namespace: default
  labels:
    name: pod-network-duplication-sa
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: pod-network-duplication-sa
subjects:
  - kind: ServiceAccount
    name: pod-network-duplication-sa
    namespace: default

Prepare ChaosEngine

• Provide the application info in spec.appinfo
• Override the experiment tunables if desired in experiments.spec.components.env
• To understand the values to provided in a ChaosEngine specification, refer ChaosEngine Concepts

Supported Experiment Tunables

Variables	Description	Specify In ChaosEngine	Notes
NETWORK_INTERFACE	Name of ethernet interface considered for shaping traffic	Mandatory
TARGET_CONTAINER	Name of container which is subjected to network latency	Optional	By default it will take the first container of the target application pod
NETWORK_PACKET_DUPLICATION_PERCENTAGE	The packet duplication in percentage	Optional	Default to 100 percentage
TOTAL_CHAOS_DURATION	The time duration for chaos insertion (seconds)	Optional	Default (60s)
LIB	The chaos lib used to inject the chaos	Optional	only `pumba` supported currently
TC_IMAGE	Image used for traffic control in linux	Optional	default value is `gaiadocker/iproute2`
LIB_IMAGE	Image used to run the netem command	Optional	Defaults to `litmuschaos/litmus-go:latest`
RAMP_TIME	Period to wait before and after injection of chaos in sec	Optional
INSTANCE_ID	A user-defined string that holds metadata/info about current run/instance of chaos. Ex: 04-05-2020-9-00. This string is appended as suffix in the chaosresult CR name.	Optional	Ensure that the overall length of the chaosresult CR is still < 64 characters

Sample ChaosEngine Manifest

# chaosengine.yaml
apiVersion: litmuschaos.io/v1alpha1
kind: ChaosEngine
metadata:
  name: nginx-network-chaos
  namespace: default
spec:
  # It can be delete/retain
  jobCleanUpPolicy: "delete"
  # It can be true/false
  annotationCheck: "true"
  # It can be active/stop
  engineState: "active"
  #ex. values: ns1:name=percona,ns2:run=nginx
  auxiliaryAppInfo: ""
  monitoring: false
  appinfo:
    appns: "default"
    # FYI, To see app label, apply kubectl get pods --show-labels
    applabel: "app=nginx"
    appkind: "deployment"
  chaosServiceAccount: pod-network-duplication-sa
  experiments:
    - name: pod-network-duplication
      spec:
        components:
          env:
            - name: TOTAL_CHAOS_DURATION
              value: "60" # in seconds

            - name: LIB_IMAGE
              value: "litmuschaos/go-runner:latest"

            #Network interface inside target container
            - name: NETWORK_INTERFACE
              value: "eth0"

            - name: NETWORK_PACKET_DUPLICATION_PERCENTAGE
              value: "100"

            #If not provided it will take the first container of the target pod
            - name: TARGET_CONTAINER
              value: ""

Create the ChaosEngine Resource

• Create the ChaosEngine manifest prepared in the previous step to trigger the Chaos.

  kubectl apply -f chaosengine.yml

• If the chaos experiment is not executed, refer to the troubleshooting section to identify the root cause and fix the issues.

Watch Chaos progress

• View network latency by setting up a ping on the affected pod from the cluster nodes

  ping <pod_ip_address>

Check Chaos Experiment Result

• Check whether the application is resilient to the Pod Network Duplication, once the experiment (job) is completed. The ChaosResult resource name is derived like this: <ChaosEngine-Name>-<ChaosExperiment-Name>.

  kubectl describe chaosresult <ChaosEngine-Name>-<ChaosExperiment-Name> -n <application-namespace>

Application Pod Network Duplication Demo

• A sample recording of this experiment execution will be provided very soon.