litmuschaos
/
litmus
mirror of https://github.com/litmuschaos/litmus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
litmus/mkdocs/docs/experiments/categories/pods/container-kill.md

12 KiB
Raw Permalink Blame History

Introduction

  • It Causes container failure of specific/random replicas of an application resources.
  • It tests deployment sanity (replica availability & uninterrupted service) and recovery workflow of the application
  • Good for testing recovery of pods having side-car containers

!!! tip "Scenario: Kill target container"
Container-Kill

Uses

??? info "View the uses of the experiment" coming soon

Prerequisites

??? info "Verify the prerequisites" - Ensure that Kubernetes Version > 1.16 - Ensure that the Litmus Chaos Operator is running by executing kubectl get pods in operator namespace (typically, litmus).If not, install from here - Ensure that the container-kill experiment resource is available in the cluster by executing kubectl get chaosexperiments in the desired namespace. If not, install from here

Default Validations

??? info "View the default validations" The application pods should be in running state before and after chaos injection.

Minimal RBAC configuration example (optional)

!!! tip "NOTE"
If you are using this experiment as part of a litmus workflow scheduled constructed & executed from chaos-center, then you may be making use of the litmus-admin RBAC, which is pre installed in the cluster as part of the agent setup.

??? note "View the Minimal RBAC permissions"

    [embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/generic/container-kill/rbac.yaml yaml)
    ```yaml
    ---
    apiVersion: v1
    kind: ServiceAccount
    metadata:
      name: container-kill-sa
      namespace: default
      labels:
        name: container-kill-sa
        app.kubernetes.io/part-of: litmus
    ---
    apiVersion: rbac.authorization.k8s.io/v1
    kind: Role
    metadata:
      name: container-kill-sa
      namespace: default
      labels:
        name: container-kill-sa
        app.kubernetes.io/part-of: litmus
    rules:
    - apiGroups: [""]
      resources: ["pods","events"]
      verbs: ["create","list","get","patch","update","delete","deletecollection"]
    - apiGroups: [""]
      resources: ["pods/exec","pods/log","replicationcontrollers"]
      verbs: ["list","get","create"]
    - apiGroups: ["batch"]
      resources: ["jobs"]
      verbs: ["create","list","get","delete","deletecollection"]
    - apiGroups: ["apps"]
      resources: ["deployments","statefulsets","daemonsets","replicasets"]
      verbs: ["list","get"]
    - apiGroups: ["apps.openshift.io"]
      resources: ["deploymentconfigs"]
      verbs: ["list","get"]
    - apiGroups: ["argoproj.io"]
      resources: ["rollouts"]
      verbs: ["list","get"]
    - apiGroups: ["litmuschaos.io"]
      resources: ["chaosengines","chaosexperiments","chaosresults"]
      verbs: ["create","list","get","patch","update"]
    ---
    apiVersion: rbac.authorization.k8s.io/v1
    kind: RoleBinding
    metadata:
      name: container-kill-sa
      namespace: default
      labels:
        name: container-kill-sa
        app.kubernetes.io/part-of: litmus
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: Role
      name: container-kill-sa
    subjects:
    - kind: ServiceAccount
      name: container-kill-sa
      namespace: default
    ```
    Use this sample RBAC manifest to create a chaosServiceAccount in the desired (app) namespace. This example consists of the minimum necessary role permissions to execute the experiment.

Experiment tunables

??? info "check the experiment tunables"

Optional Fields

<table>
  <tr>
    <th> Variables </th>
    <th> Description  </th>
    <th> Notes </th>
  </tr>
  <tr>
    <td> TARGET_CONTAINER  </td>
    <td> The name of container to be killed inside the pod </td>
    <td> If the TARGET_CONTAINER is not provided it will delete the first container </td>
  </tr>
  <tr>
    <td> CHAOS_INTERVAL  </td>
    <td> Time interval b/w two successive container kill (in sec) </td>
    <td> If the CHAOS_INTERVAL is not provided it will take the default value of 10s </td>
  </tr>
  <tr>
    <td> TOTAL_CHAOS_DURATION  </td>
    <td> The time duration for chaos injection (seconds) </td>
    <td> Defaults to 20s </td>
  </tr>
  <tr>
    <td> PODS_AFFECTED_PERC </td>
    <td> The Percentage of total pods to target </td>
    <td> Defaults to 0 (corresponds to 1 replica), provide numeric value only </td>
  </tr> 
  <tr>
    <td> TARGET_PODS </td>
    <td> Comma separated list of application pod name subjected to container kill chaos</td>
    <td> If not provided, it will select target pods randomly based on provided appLabels</td>
  </tr>
  <tr>
    <td> LIB_IMAGE  </td>
    <td> LIB Image used to kill the container </td>
    <td> Defaults to <code>litmuschaos/go-runner:latest</code></td>
  </tr>
  <tr>
    <td> LIB  </td>
    <td> The category of lib use to inject chaos </td>
    <td> Default value: litmus, supported values: pumba and litmus </td>
  </tr>
  <tr>
    <td> RAMP_TIME </td>
    <td> Period to wait before injection of chaos in sec </td>
    <td> </td>
  </tr>
  <tr>
    <td> SEQUENCE </td>
    <td> It defines sequence of chaos execution for multiple target pods </td>
    <td> Default value: parallel. Supported: serial, parallel </td>
  </tr>
  <tr>
    <td> SIGNAL </td>
    <td> It contains termination signal used for container kill </td>
    <td> Default value: SIGKILL </td>
  </tr>
  <tr>
    <td> SOCKET_PATH </td>
    <td> Path of the containerd/crio/docker socket file </td>
    <td> Defaults to `/var/run/docker.sock` </td>
  </tr>
  <tr>
    <td> CONTAINER_RUNTIME  </td>
    <td> container runtime interface for the cluster</td>
    <td>  Defaults to docker, supported values: docker, containerd and crio for litmus and only docker for pumba LIB </td>
  </tr>
</table>

Experiment Examples

Common and Pod specific tunables

Refer the common attributes and Pod specific tunable to tune the common tunables for all experiments and pod specific tunables.

Kill Specific Container

It defines the name of the targeted container subjected to chaos. It can be tuned via TARGET_CONTAINER ENV. If TARGET_CONTAINER is provided as empty then it will use the first container of the targeted pod.

# kill the specific target container
apiVersion: litmuschaos.io/v1alpha1
kind: ChaosEngine
metadata:
  name: engine-nginx
spec:
  engineState: "active"
  annotationCheck: "false"
  appinfo:
    appns: "default"
    applabel: "app=nginx"
    appkind: "deployment"
  chaosServiceAccount: container-kill-sa
  experiments:
  - name: container-kill
    spec:
      components:
        env:
        # name of the target container
        - name: TARGET_CONTAINER
          value: 'nginx'
        - name: TOTAL_CHAOS_DURATION
          VALUE: '60'

Multiple Iterations Of Chaos

The multiple iterations of chaos can be tuned via setting CHAOS_INTERVAL ENV. Which defines the delay between each iteration of chaos.

# defines delay between each successive iteration of the chaos
apiVersion: litmuschaos.io/v1alpha1
kind: ChaosEngine
metadata:
  name: engine-nginx
spec:
  engineState: "active"
  annotationCheck: "false"
  appinfo:
    appns: "default"
    applabel: "app=nginx"
    appkind: "deployment"
  chaosServiceAccount: container-kill-sa
  experiments:
  - name: container-kill
    spec:
      components:
        env:
        # delay between each iteration of chaos
        - name: CHAOS_INTERVAL
          value: '15'
        # time duration for the chaos execution
        - name: TOTAL_CHAOS_DURATION
          VALUE: '60'

Container Runtime Socket Path

It defines the CONTAINER_RUNTIME and SOCKET_PATH ENV to set the container runtime and socket file path:

  • CONTAINER_RUNTIME: It supports docker, containerd, and crio runtimes. The default value is docker.
  • SOCKET_PATH: It contains path of docker socket file by default(/var/run/docker.sock). For other runtimes provide the appropriate path.
## provide the container runtime and socket file path
apiVersion: litmuschaos.io/v1alpha1
kind: ChaosEngine
metadata:
  name: engine-nginx
spec:
  engineState: "active"
  annotationCheck: "false"
  appinfo:
    appns: "default"
    applabel: "app=nginx"
    appkind: "deployment"
  chaosServiceAccount: container-kill-sa
  experiments:
  - name: container-kill
    spec:
      components:
        env:
        # runtime for the container
        # supports docker, containerd, crio
        - name: CONTAINER_RUNTIME
          value: 'docker'
        # path of the socket file
        - name: SOCKET_PATH
          value: '/var/run/docker.sock'
        - name: TOTAL_CHAOS_DURATION
          VALUE: '60'

Signal For Kill

It defines the Linux signal passed while killing the container. It can be tuned via SIGNAL ENV. It defaults to the SIGTERM.

# specific linux signal passed while kiiling container
apiVersion: litmuschaos.io/v1alpha1
kind: ChaosEngine
metadata:
  name: engine-nginx
spec:
  engineState: "active"
  annotationCheck: "false"
  appinfo:
    appns: "default"
    applabel: "app=nginx"
    appkind: "deployment"
  chaosServiceAccount: container-kill-sa
  experiments:
  - name: container-kill
    spec:
      components:
        env:
        # signal passed while killing container
        # defaults to SIGTERM
        - name: SIGNAL
          value: 'SIGKILL'
        - name: TOTAL_CHAOS_DURATION
          VALUE: '60'

Pumba Chaos Library

It specifies the Pumba chaos library for the chaos injection. It can be tuned via LIB ENV. The defaults chaos library is litmus.

# pumba chaoslib used to kill the container
apiVersion: litmuschaos.io/v1alpha1
kind: ChaosEngine
metadata:
  name: engine-nginx
spec:
  engineState: "active"
  annotationCheck: "false"
  appinfo:
    appns: "default"
    applabel: "app=nginx"
    appkind: "deployment"
  chaosServiceAccount: container-kill-sa
  experiments:
  - name: container-kill
    spec:
      components:
        env:
        # name of the lib
        # supoorts pumba and litmus
        - name: LIB
          value: 'pumba'
        - name: TOTAL_CHAOS_DURATION
          VALUE: '60'