Migrated all versions. (#3)
Signed-off-by: Vedant Shrotria <vedant.shrotria@mayadata.io>
This commit is contained in:
VEDANT SHROTRIA
GitHub
parent
e67110a933
commit
cb04a13e48
|
|
@ -1,80 +0,0 @@
|
|||
---
|
||||
id: version-1.0.0-chaoshub
|
||||
title: Using and contributing to ChaosHub
|
||||
sidebar_label: ChaosHub
|
||||
original_id: chaoshub
|
||||
---
|
||||
------
|
||||
|
||||
|
||||
**Important links**
|
||||
|
||||
Chaos Hub is maintained at https://hub.litmuschaos.io
|
||||
|
||||
To contribute new chaos charts visit: https://github.com/litmuschaos/chaos-charts
|
||||
|
||||
**Introduction**
|
||||
|
||||
Litmus chaos hub is a place where the chaos engineering community members publish their chaos experiments. A set of related chaos experiments are bundled into a `Chaos Chart`. Chaos Charts are classified into the following categories.
|
||||
|
||||
- [Generic Chaos](#generic-chaos)
|
||||
- [Application Chaos](#application-chaos)
|
||||
- [Platform Chaos](#platform-chaos)
|
||||
|
||||
|
||||
|
||||
### Generic Chaos
|
||||
|
||||
Chaos actions that apply to generic Kubernetes resources are classified into this category. Following chaos experiments are supported under Generic Chaos Chart
|
||||
|
||||
| Experiment name | Description | User guide link |
|
||||
| ----------- | ----------------------------------------- | --------------------------------------------------------- |
|
||||
| Container Kill | Kill one container in the application pod | [container-kill](container-kill.md)|
|
||||
| Pod Delete | Fail the application pod | [pod-delete](pod-delete.md) |
|
||||
| Pod Network Latency | Experiment to inject network latency to the POD | [pod-network-latency](pod-network-latency.md) |
|
||||
| Pod Network Loss | Experiment to inject network loss to the POD | [pod-network-loss](pod-network-loss.md) |
|
||||
| CPU Hog | Exhaust CPU resources on the Kubernetes Node | [cpu-hog](cpu-hog.md) |
|
||||
| Disk Fill | Fillup Ephemeral Storage of a Resource | [disk-fill](disk-fill.md) |
|
||||
| Disk Loss | External disk loss from the node | [disk-loss](disk-loss.md)|
|
||||
| Node Drain| Drain the node where application pod is scheduled | [node-drain](node-drain.md) |
|
||||
| Pod CPU Hog | Consume CPU resources on the application container | [pod-cpu-hog](pod-cpu-hog.md) |
|
||||
| Pod Network Corruption | Inject Network Packet Corruption Into Application Pod |[pod-network-corruption](pod-network-corruption.md) |
|
||||
|
||||
### Application Chaos
|
||||
|
||||
While Chaos Experiments under the Generic category offer the ability to induce chaos into Kubernetes resources, it is difficult to analyze and conclude if the chaos induced found a weakness in a given application. The application specific chaos experiments are built with some checks on *pre-conditions* and some expected outcomes after the chaos injection. The result of the chaos experiment is determined by matching the outcome with the expected outcome.
|
||||
|
||||
<div class="danger">
|
||||
<strong>NOTE:</strong> If the result of the chaos experiment is `pass`, it means that the application is resilient to that chaos.
|
||||
</div>
|
||||
|
||||
|
||||
#### Benefits of contributing an application chaos experiment
|
||||
|
||||
Application developers write negative tests in their CI pipelines to test the resiliency of the applications. These negative can be converted into Litmus Chaos Experiments and contributed to ChaosHub, so that the users of the application can use them in staging/pre-production/production environments to check the resilience. Application environments vary considerably from where they are tested (CI pipelines) to where they are deployed (Production). Hence, running the same chaos tests in the user's environment will help determine the weaknesses of the deployment and fixing such weaknesses leads to increased resilience.
|
||||
|
||||
|
||||
|
||||
Following Application Chaos experiments are available on ChaosHub
|
||||
|
||||
|
||||
|
||||
| Application | Description | Chaos Experiments |
|
||||
| ----------- | ----------------------------------------- | --------------------------------------------------------- |
|
||||
| OpenEBS | Container Attached Storage for Kubernetes | [openebs-pool-pod-failure](openebs-pool-pod-failure.md)<br>[openebs-pool-container-failure](openebs-pool-container-failure.md)<br>[openebs-target-pod-failure](openebs-target-pod-failure.md)<br>[openebs-target-container-failure](openebs-target-container-failure.md)<br>[openebs-target-network-delay](openebs-target-network-delay.md)<br>[openebs-target-network-loss](openebs-target-network-loss.md) |
|
||||
| Kafka | Open-source stream processing software | [kafka-broker-pod-failure](kafka-broker-pod-failure.md)<br>[kafka-broker-disk-failure](kafka-broker-disk-failure.md)<br> |
|
||||
| CoreDns | CoreDNS is a fast and flexible DNS server that chains plugins | [coredns-pod-delete](coredns-pod-delete.md)|
|
||||
|
||||
### Platform Chaos
|
||||
|
||||
Chaos experiments that inject chaos into the platform resources of Kubernetes are classified into this category. Management of platform resources vary significantly from each other, Chaos Charts may be maintained separately for each platform (For example, AWS, GCP, Azure, etc)
|
||||
|
||||
Following Platform Chaos experiments are available on ChaosHub
|
||||
|
||||
|
||||
|
||||
| Platform | Description | Chaos Experiments |
|
||||
| -------- | ------------------------------------------- | ----------------- |
|
||||
| AWS | Amazon Web Services platform. Includes EKS. | None |
|
||||
| GCP | Google Cloud Platform. Includes GKE. | None |
|
||||
| Azure | Microsoft Azure platform. Includes AKS. | None |
|
||||
|
|
@ -1,42 +0,0 @@
|
|||
---
|
||||
id: version-1.0.0-plugins
|
||||
title: Using other chaos libraries as plugins
|
||||
sidebar_label: Plugins
|
||||
original_id: plugins
|
||||
---
|
||||
------
|
||||
|
||||
Litmus provides a way to use any chaos library or a tool to inject chaos. The chaos tool to be compatible with Litmus should satisfy the following requirements:
|
||||
|
||||
- Should be available as a Docker Image
|
||||
- Should take configuration through a `config-map`
|
||||
|
||||
The `plugins` or `chaos-libraries` host the core logic to inject chaos.
|
||||
|
||||
These plugins are hosted at https://github.com/litmuschaos/litmus-ansible/tree/master/chaoslib
|
||||
|
||||
Litmus project has integration into the following chaos-libraries.
|
||||
|
||||
| Chaos Library | Logo | Experiments covered |
|
||||
| ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ |
|
||||
| <a href="https://github.com/litmuschaos/litmus-ansible" target="_blank">Litmus</a> | <img src="https://camo.githubusercontent.com/953211f24c1c246f7017703f67b9779e4589bf76/68747470733a2f2f6c616e6473636170652e636e63662e696f2f6c6f676f732f6c69746d75732e737667" width="50"> | Litmus native chaos libraries that encompasses the chaos capabilities for `pod-kill`, `container-kill`, `cpu-hog` |
|
||||
| <a href="https://github.com/alexei-led/pumba" target="_blank">Pumba</a> | <img src="https://github.com/alexei-led/pumba/raw/master/docs/img/pumba_logo.png" width="50"> | Pumba provides chaos capabilities for `network-delay` |
|
||||
| <a href="https://github.com/bloomberg/powerfulseal" target="_blank">PowerfulSeal</a> | <img src="https://github.com/bloomberg/powerfulseal/raw/master/media/powerful-seal.png" width="50"> | PowerfulSeal provides chaos capabilities for `pod-kill` |
|
||||
| | | |
|
||||
|
||||
|
||||
|
||||
Usage of plugins is a configuration parameter inside the chaos experiment.
|
||||
|
||||
> Add an example snippet here.
|
||||
|
||||
<br>
|
||||
|
||||
<br>
|
||||
|
||||
<hr>
|
||||
|
||||
<br>
|
||||
|
||||
<br>
|
||||
|
||||
|
|
@ -1,80 +0,0 @@
|
|||
---
|
||||
id: version-1.1.0-chaoshub
|
||||
title: Using and contributing to ChaosHub
|
||||
sidebar_label: ChaosHub
|
||||
original_id: chaoshub
|
||||
---
|
||||
------
|
||||
|
||||
|
||||
**Important links**
|
||||
|
||||
Chaos Hub is maintained at https://hub.litmuschaos.io
|
||||
|
||||
To contribute new chaos charts visit: https://github.com/litmuschaos/chaos-charts
|
||||
|
||||
**Introduction**
|
||||
|
||||
Litmus chaos hub is a place where the chaos engineering community members publish their chaos experiments. A set of related chaos experiments are bundled into a `Chaos Chart`. Chaos Charts are classified into the following categories.
|
||||
|
||||
- [Generic Chaos](#generic-chaos)
|
||||
- [Application Chaos](#application-chaos)
|
||||
- [Platform Chaos](#platform-chaos)
|
||||
|
||||
|
||||
|
||||
### Generic Chaos
|
||||
|
||||
Chaos actions that apply to generic Kubernetes resources are classified into this category. Following chaos experiments are supported under Generic Chaos Chart
|
||||
|
||||
| Experiment name | Description | User guide link |
|
||||
| ----------- | ----------------------------------------- | --------------------------------------------------------- |
|
||||
| Container Kill | Kill one container in the application pod | [container-kill](container-kill.md)|
|
||||
| Pod Delete | Fail the application pod | [pod-delete](pod-delete.md) |
|
||||
| Pod Network Latency | Experiment to inject network latency to the POD | [pod-network-latency](pod-network-latency.md) |
|
||||
| Pod Network Loss | Experiment to inject network loss to the POD | [pod-network-loss](pod-network-loss.md) |
|
||||
| Node CPU Hog | Exhaust CPU resources on the Kubernetes Node | [node-cpu-hog](node-cpu-hog.md) |
|
||||
| Disk Fill | Fillup Ephemeral Storage of a Resource | [disk-fill](disk-fill.md) |
|
||||
| Disk Loss | External disk loss from the node | [disk-loss](disk-loss.md)|
|
||||
| Node Drain| Drain the node where application pod is scheduled | [node-drain](node-drain.md) |
|
||||
| Pod CPU Hog | Consume CPU resources on the application container | [pod-cpu-hog](pod-cpu-hog.md) |
|
||||
| Pod Network Corruption | Inject Network Packet Corruption Into Application Pod |[pod-network-corruption](pod-network-corruption.md) |
|
||||
|
||||
### Application Chaos
|
||||
|
||||
While Chaos Experiments under the Generic category offer the ability to induce chaos into Kubernetes resources, it is difficult to analyze and conclude if the chaos induced found a weakness in a given application. The application specific chaos experiments are built with some checks on *pre-conditions* and some expected outcomes after the chaos injection. The result of the chaos experiment is determined by matching the outcome with the expected outcome.
|
||||
|
||||
<div class="danger">
|
||||
<strong>NOTE:</strong> If the result of the chaos experiment is `pass`, it means that the application is resilient to that chaos.
|
||||
</div>
|
||||
|
||||
|
||||
#### Benefits of contributing an application chaos experiment
|
||||
|
||||
Application developers write negative tests in their CI pipelines to test the resiliency of the applications. These negative can be converted into Litmus Chaos Experiments and contributed to ChaosHub, so that the users of the application can use them in staging/pre-production/production environments to check the resilience. Application environments vary considerably from where they are tested (CI pipelines) to where they are deployed (Production). Hence, running the same chaos tests in the user's environment will help determine the weaknesses of the deployment and fixing such weaknesses leads to increased resilience.
|
||||
|
||||
|
||||
|
||||
Following Application Chaos experiments are available on ChaosHub
|
||||
|
||||
|
||||
|
||||
| Application | Description | Chaos Experiments |
|
||||
| ----------- | ----------------------------------------- | --------------------------------------------------------- |
|
||||
| OpenEBS | Container Attached Storage for Kubernetes | [openebs-pool-pod-failure](openebs-pool-pod-failure.md)<br>[openebs-pool-container-failure](openebs-pool-container-failure.md)<br>[openebs-target-pod-failure](openebs-target-pod-failure.md)<br>[openebs-target-container-failure](openebs-target-container-failure.md)<br>[openebs-target-network-delay](openebs-target-network-delay.md)<br>[openebs-target-network-loss](openebs-target-network-loss.md) |
|
||||
| Kafka | Open-source stream processing software | [kafka-broker-pod-failure](kafka-broker-pod-failure.md)<br>[kafka-broker-disk-failure](kafka-broker-disk-failure.md)<br> |
|
||||
| CoreDns | CoreDNS is a fast and flexible DNS server that chains plugins | [coredns-pod-delete](coredns-pod-delete.md)|
|
||||
|
||||
### Platform Chaos
|
||||
|
||||
Chaos experiments that inject chaos into the platform resources of Kubernetes are classified into this category. Management of platform resources vary significantly from each other, Chaos Charts may be maintained separately for each platform (For example, AWS, GCP, Azure, etc)
|
||||
|
||||
Following Platform Chaos experiments are available on ChaosHub
|
||||
|
||||
|
||||
|
||||
| Platform | Description | Chaos Experiments |
|
||||
| -------- | ------------------------------------------- | ----------------- |
|
||||
| AWS | Amazon Web Services platform. Includes EKS. | None |
|
||||
| GCP | Google Cloud Platform. Includes GKE. | None |
|
||||
| Azure | Microsoft Azure platform. Includes AKS. | None |
|
||||
|
|
@ -1,131 +0,0 @@
|
|||
---
|
||||
id: version-1.2.0-admin-mode
|
||||
title: Administrator Mode
|
||||
sidebar_label: Administrator Mode
|
||||
original_id: admin-mode
|
||||
---
|
||||
------
|
||||
|
||||
### What is Adminstator Mode?
|
||||
|
||||
Admin mode is one of the ways the chaos orchestration is set up in Litmus, wherein all chaos resources (i.e., install time resources like the operator, chaosexperiment CRs, chaosServiceAccount/rbac and runtime resources like chaosengine, chaos-runner, experiment jobs & chaosresults) are setup in a single admin namespace (typically, litmus). In other words, centralized administration of chaos.
|
||||
This feature is aimed at making the SRE/Cluster Admins life easier by doing away with setting up chaos pre-requisites on a per namespace basis (which may be more relevant in an autonomous/self-service cluster sharing model in dev environments).
|
||||
This mode typically needs a "wider" & "stronger" ClusterRole, albeit one that is still just a superset of the individual experiment permissions. In this mode, the applications in their respective namespaces are subjected to chaos while the chaos job runs elsewhere, i.e., admin namespace.
|
||||
|
||||
### How to use Adminstator Mode?
|
||||
|
||||
In order to use Admin Mode, you just have to create a ServiceAccount in the *admin* or so called *chaos* namespace (`litmus` itself can be used), which is tied to a ClusterRole that has the permissions to perform operations on Kubernetes resources involved in the selected experiments across namespaces.
|
||||
Provide this ServiceAccount in ChaosEngine's .spec.chaosServiceAccount.
|
||||
|
||||
### Example
|
||||
|
||||
#### Prepare RBAC Manifest
|
||||
|
||||
Here is an RBAC definition, which in essence is a superset of individual experiments RBAC that has the permissions to run all chaos experiments across different namespaces.
|
||||
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/pages/master/docs/litmus-admin-rbac.yaml)
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: litmus-admin
|
||||
namespace: litmus
|
||||
labels:
|
||||
name: litmus-admin
|
||||
---
|
||||
# Source: openebs/templates/clusterrole.yaml
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
kind: ClusterRole
|
||||
metadata:
|
||||
name: litmus-admin
|
||||
labels:
|
||||
name: litmus-admin
|
||||
rules:
|
||||
- apiGroups: ["","apps","batch","extensions","litmuschaos.io","openebs.io","storage.k8s.io"]
|
||||
resources: ["chaosengines","chaosexperiments","chaosresults","cstorpools","cstorvolumereplicas","configmaps","secrets","pods","pods/exec","pods/log","pods/eviction","jobs","replicasets","deployments","daemonsets","statefulsets","persistentvolumeclaims","persistentvolumes","storageclasses","services","events"]
|
||||
verbs: ["create","delete","get","list","patch","update"]
|
||||
- apiGroups: [""]
|
||||
resources: ["nodes"]
|
||||
verbs: ["get","list","patch"]
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
kind: ClusterRoleBinding
|
||||
metadata:
|
||||
name: litmus-admin
|
||||
labels:
|
||||
name: litmus-admin
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: ClusterRole
|
||||
name: litmus-admin
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: litmus-admin
|
||||
namespace: litmus
|
||||
```
|
||||
|
||||
|
||||
#### Prepare ChaosEngine
|
||||
|
||||
```yaml
|
||||
apiVersion: litmuschaos.io/v1alpha1
|
||||
kind: ChaosEngine
|
||||
metadata:
|
||||
name: nginx-chaos
|
||||
namespace: litmus #Chaos Resources Namespace
|
||||
spec:
|
||||
appinfo:
|
||||
appns: 'default' #Application Namespace
|
||||
applabel: 'app=nginx'
|
||||
appkind: 'deployment'
|
||||
# It can be true/false
|
||||
annotationCheck: 'true'
|
||||
# It can be active/stop
|
||||
engineState: 'active'
|
||||
#ex. values: ns1:name=percona,ns2:run=nginx
|
||||
auxiliaryAppInfo: ''
|
||||
chaosServiceAccount: litmus-admin
|
||||
monitoring: false
|
||||
# It can be delete/retain
|
||||
jobCleanUpPolicy: 'delete'
|
||||
experiments:
|
||||
- name: pod-delete
|
||||
spec:
|
||||
components:
|
||||
env:
|
||||
# set chaos duration (in sec) as desired
|
||||
- name: TOTAL_CHAOS_DURATION
|
||||
value: '30'
|
||||
|
||||
# set chaos interval (in sec) as desired
|
||||
- name: CHAOS_INTERVAL
|
||||
value: '10'
|
||||
|
||||
# pod failures without '--force' & default terminationGracePeriodSeconds
|
||||
- name: FORCE
|
||||
value: 'false'
|
||||
```
|
||||
|
||||
### Create the ChaosEngine Resource
|
||||
|
||||
- Create the ChaosEngine manifest prepared in the previous step to trigger the Chaos.
|
||||
|
||||
`kubectl apply -f chaosengine.yml`
|
||||
|
||||
### Watch Chaos Engine
|
||||
|
||||
- Describe Chaos Engine for chaos steps.
|
||||
|
||||
`kubectl describe chaosengine nginx-chaos -n litmus`
|
||||
|
||||
### Watch Chaos progress
|
||||
|
||||
- View pod terminations & recovery by setting up a watch on the pods in the application namespace
|
||||
|
||||
`watch -n 1 kubectl get pods -n default`
|
||||
|
||||
### Check Chaos Experiment Result
|
||||
|
||||
- Check whether the application is resilient to the pod failure, once the experiment (job) is completed. The ChaosResult resource name is derived like this: `<ChaosEngine-Name>-<ChaosExperiment-Name>`.
|
||||
|
||||
`kubectl describe chaosresult nginx-chaos-pod-delete -n litmus`
|
||||
|
|
@ -1,42 +0,0 @@
|
|||
---
|
||||
id: version-1.6.0-plugins
|
||||
title: Using other chaos libraries as plugins
|
||||
sidebar_label: Plugins
|
||||
original_id: plugins
|
||||
---
|
||||
------
|
||||
|
||||
Litmus provides a way to use any chaos library or a tool to inject chaos. The chaos tool to be compatible with Litmus should satisfy the following requirements:
|
||||
|
||||
- Should be available as a Docker Image
|
||||
- Should take configuration through a `config-map`
|
||||
|
||||
The `plugins` or `chaos-libraries` host the core logic to inject chaos.
|
||||
|
||||
These plugins are hosted at https://github.com/litmuschaos/litmus-ansible/tree/master/chaoslib
|
||||
|
||||
Litmus project has integration into the following chaos-libraries.
|
||||
|
||||
| Chaos Library | Logo | Experiments covered |
|
||||
| ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ |
|
||||
| <a href="https://github.com/litmuschaos/litmus" target="_blank">Litmus</a> | <img src="https://camo.githubusercontent.com/953211f24c1c246f7017703f67b9779e4589bf76/68747470733a2f2f6c616e6473636170652e636e63662e696f2f6c6f676f732f6c69746d75732e737667" width="50"> | Litmus native chaos libraries that encompasses the chaos capabilities for `pod-kill`, `container-kill`, `cpu-hog` |
|
||||
| <a href="https://github.com/alexei-led/pumba" target="_blank">Pumba</a> | <img src="https://github.com/alexei-led/pumba/raw/master/docs/img/pumba_logo.png" width="50"> | Pumba provides chaos capabilities for `network-delay` |
|
||||
| <a href="https://github.com/bloomberg/powerfulseal" target="_blank">PowerfulSeal</a> | <img src="https://github.com/bloomberg/powerfulseal/raw/master/media/powerful-seal.png" width="50"> | PowerfulSeal provides chaos capabilities for `pod-kill` |
|
||||
| | | |
|
||||
|
||||
|
||||
|
||||
Usage of plugins is a configuration parameter inside the chaos experiment.
|
||||
|
||||
> Add an example snippet here.
|
||||
|
||||
<br>
|
||||
|
||||
<br>
|
||||
|
||||
<hr>
|
||||
|
||||
<br>
|
||||
|
||||
<br>
|
||||
|
||||
|
|
@ -1,92 +0,0 @@
|
|||
---
|
||||
id: version-1.8.0-chaoshub
|
||||
title: Using and contributing to ChaosHub
|
||||
sidebar_label: ChaosHub
|
||||
original_id: chaoshub
|
||||
---
|
||||
------
|
||||
|
||||
|
||||
**Important links**
|
||||
|
||||
Chaos Hub is maintained at https://hub.litmuschaos.io
|
||||
|
||||
To contribute new chaos charts visit: https://github.com/litmuschaos/chaos-charts
|
||||
|
||||
**Introduction**
|
||||
|
||||
Litmus chaos hub is a place where the chaos engineering community members publish their chaos experiments. A set of related chaos experiments are bundled into a `Chaos Chart`. Chaos Charts are classified into the following categories.
|
||||
|
||||
- [Generic Chaos](#generic-chaos)
|
||||
- [Application Chaos](#application-chaos)
|
||||
- [Platform Chaos](#platform-chaos)
|
||||
|
||||
|
||||
|
||||
### Generic Chaos
|
||||
|
||||
Chaos actions that apply to generic Kubernetes resources are classified into this category. Following chaos experiments are supported under Generic Chaos Chart
|
||||
|
||||
| Experiment name | Description | User guide link |
|
||||
| ----------- | ----------------------------------------- | --------------------------------------------------------- |
|
||||
| Container Kill | Kills the container in the application pod | [container-kill](container-kill.md)|
|
||||
| Pod Delete | Deletes the application pod | [pod-delete](pod-delete.md) |
|
||||
| Pod Network Latency | Injects network latency into the pod | [pod-network-latency](pod-network-latency.md) |
|
||||
| Pod Network Loss | Injects network loss into the pod | [pod-network-loss](pod-network-loss.md) |
|
||||
| Node CPU Hog | Exhaust CPU resources on the Kubernetes Node | [node-cpu-hog](node-cpu-hog.md) |
|
||||
| Node Memory Hog | Exhaust Memory resources on the Kubernetes Node | [node-memory-hog](node-memory-hog.md) |
|
||||
| Disk Fill | Fillup Ephemeral Storage of a Resource | [disk-fill](disk-fill.md) |
|
||||
| Disk Loss | External disk loss from the node | [disk-loss](disk-loss.md)|
|
||||
| Node Drain| Drains the node where application pod is scheduled | [node-drain](node-drain.md) |
|
||||
| Pod CPU Hog | Consumes CPU resources on the application container | [pod-cpu-hog](pod-cpu-hog.md) |
|
||||
| Pod Memory Hog | Consumes Memory resources on the application container | [pod-memory-hog](pod-memory-hog.md) |
|
||||
| Pod Network Corruption | Injects Network Packet Corruption into Application Pod |[pod-network-corruption](pod-network-corruption.md) |
|
||||
| Kubelet Service Kill | Kills the kubelet service on the application node |[kubelet-service-kill](kubelet-service-kill.md) |
|
||||
| Docker Service Kill | Kills the docker service on the application node |[docker-service-kill](docker-service-kill.md) |
|
||||
| Node Taint| Taints the node where application pod is scheduled | [node-taint](node-taint.md) |
|
||||
| Pod Autoscaler| Scales the application replicas and test the node autoscaling on cluster | [pod-autoscaler](pod-autoscaler.md) |
|
||||
| Pod Network Duplication | Injects Network Packet Duplication into Application Pod |[pod-network-duplication](pod-network-duplication.md) |
|
||||
| Pod IO Stress | Injects IO stress resources on the application container | [pod-io-stress](pod-io-stress.md) |
|
||||
| Node IO stress| Injects IO stress resources on the application node |[node-io-stress](node-io-stress.md) |
|
||||
|
||||
### Application Chaos
|
||||
|
||||
While Chaos Experiments under the Generic category offer the ability to induce chaos into Kubernetes resources, it is difficult to analyze and conclude if the chaos induced found a weakness in a given application. The application specific chaos experiments are built with some checks on *pre-conditions* and some expected outcomes after the chaos injection. The result of the chaos experiment is determined by matching the outcome with the expected outcome.
|
||||
|
||||
<div class="danger">
|
||||
<strong>NOTE:</strong> If the result of the chaos experiment is `pass`, it means that the application is resilient to that chaos.
|
||||
</div>
|
||||
|
||||
|
||||
#### Benefits of contributing an application chaos experiment
|
||||
|
||||
Application developers write negative tests in their CI pipelines to test the resiliency of the applications. These negative can be converted into Litmus Chaos Experiments and contributed to ChaosHub, so that the users of the application can use them in staging/pre-production/production environments to check the resilience. Application environments vary considerably from where they are tested (CI pipelines) to where they are deployed (Production). Hence, running the same chaos tests in the user's environment will help determine the weaknesses of the deployment and fixing such weaknesses leads to increased resilience.
|
||||
|
||||
|
||||
|
||||
Following Application Chaos experiments are available on ChaosHub
|
||||
|
||||
|
||||
|
||||
| Application | Description | Chaos Experiments |
|
||||
| ----------- | ----------------------------------------- | --------------------------------------------------------- |
|
||||
| OpenEBS | Container Attached Storage for Kubernetes | [openebs-pool-pod-failure](openebs-pool-pod-failure.md)<br>[openebs-pool-container-failure](openebs-pool-container-failure.md)<br>[openebs-target-pod-failure](openebs-target-pod-failure.md)<br>[openebs-target-container-failure](openebs-target-container-failure.md)<br>[openebs-target-network-delay](openebs-target-network-delay.md)<br>[openebs-target-network-loss](openebs-target-network-loss.md) <br>[openebs-control-plane-chaos](openebs-control-plane-chaos.md) <br>[openebs-nfs-provisioner-kill](openebs-nfs-provisioner-kill.md) <br>[openebs-target-network-loss](openebs-target-network-loss.md) <br>[openebs-pool-disk-loss](openebs-pool-disk-loss.md) <br>[openebs-pool-network-loss](openebs-pool-network-loss.md) <br>[openebs-pool-network-delay](openebs-pool-network-delay.md)|
|
||||
| Kafka | Open-source stream processing software | [kafka-broker-pod-failure](kafka-broker-pod-failure.md)<br>[kafka-broker-disk-failure](kafka-broker-disk-failure.md)<br> |
|
||||
| CoreDns | CoreDNS is a fast and flexible DNS server that chains plugins | [coredns-pod-delete](coredns-pod-delete.md)|
|
||||
| Cassandra | Cassandra is an opensource distributed database | [cassandra-pod-delete](cassandra-pod-delete.md)|
|
||||
|
||||
### Platform Chaos
|
||||
|
||||
Chaos experiments that inject chaos into the platform resources of Kubernetes are classified into this category. Management of platform resources vary significantly from each other, Chaos Charts may be maintained separately for each platform (For example, AWS, GCP, Azure, etc)
|
||||
|
||||
Following Platform Chaos experiments are available on ChaosHub
|
||||
|
||||
|
||||
|
||||
| Platform | Description | Chaos Experiments |
|
||||
| -------- | ------------------------------------------- | ----------------- |
|
||||
| AWS | Amazon Web Services platform. Includes EKS. | None |
|
||||
| GCP | Google Cloud Platform. Includes GKE. | None |
|
||||
| Azure | Microsoft Azure platform. Includes AKS. | None |
|
||||
|
||||
|
||||
|
|
@ -1,42 +0,0 @@
|
|||
---
|
||||
id: version-1.8.0-plugins
|
||||
title: Using other chaos libraries as plugins
|
||||
sidebar_label: Plugins
|
||||
original_id: plugins
|
||||
---
|
||||
------
|
||||
|
||||
Litmus provides a way to use any chaos library or a tool to inject chaos. The chaos tool to be compatible with Litmus should satisfy the following requirements:
|
||||
|
||||
- Should be available as a Docker Image
|
||||
- Should take configuration through a `config-map`
|
||||
|
||||
The `plugins` or `chaos-libraries` host the core logic to inject chaos.
|
||||
|
||||
These plugins are hosted at https://github.com/litmuschaos/litmus-ansible/tree/master/chaoslib
|
||||
|
||||
Litmus project has integration into the following chaos-libraries.
|
||||
|
||||
| Chaos Library | Logo | Experiments covered |
|
||||
| ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ |
|
||||
| <a href="https://github.com/litmuschaos/litmus" target="_blank">Litmus</a> | <img src="https://camo.githubusercontent.com/953211f24c1c246f7017703f67b9779e4589bf76/68747470733a2f2f6c616e6473636170652e636e63662e696f2f6c6f676f732f6c69746d75732e737667" width="50"> | Litmus native chaos libraries that encompasses the chaos capabilities for `pod-kill`, `container-kill`, `cpu-hog`, `network-chaos`, `disk-chaos`, `memory-hog`|
|
||||
| <a href="https://github.com/alexei-led/pumba" target="_blank">Pumba</a> | <img src="https://github.com/alexei-led/pumba/raw/master/docs/img/pumba_logo.png" width="50"> | Pumba provides chaos capabilities for `network-delay` |
|
||||
| <a href="https://github.com/bloomberg/powerfulseal" target="_blank">PowerfulSeal</a> | <img src="https://github.com/bloomberg/powerfulseal/raw/master/media/powerful-seal.png" width="50"> | PowerfulSeal provides chaos capabilities for `pod-kill` |
|
||||
| | | |
|
||||
|
||||
|
||||
|
||||
Usage of plugins is a configuration parameter inside the chaos experiment.
|
||||
|
||||
> Add an example snippet here.
|
||||
|
||||
<br>
|
||||
|
||||
<br>
|
||||
|
||||
<hr>
|
||||
|
||||
<br>
|
||||
|
||||
<br>
|
||||
|
||||
|
|
@ -1,92 +0,0 @@
|
|||
---
|
||||
id: version-1.9.0-chaoshub
|
||||
title: Using and contributing to ChaosHub
|
||||
sidebar_label: ChaosHub
|
||||
original_id: chaoshub
|
||||
---
|
||||
------
|
||||
|
||||
|
||||
**Important links**
|
||||
|
||||
Chaos Hub is maintained at https://hub.litmuschaos.io
|
||||
|
||||
To contribute new ChaosCharts visit: https://github.com/litmuschaos/chaos-charts
|
||||
|
||||
**Introduction**
|
||||
|
||||
Litmus chaos hub is a place where the Chaos Engineering community members publish their chaos experiments. A set of related chaos experiments are bundled into a `Chaos Chart`. Chaos Charts are classified into the following categories.
|
||||
|
||||
- [Generic Chaos](#generic-chaos)
|
||||
- [Application Chaos](#application-chaos)
|
||||
- [Platform Chaos](#platform-chaos)
|
||||
|
||||
|
||||
|
||||
### Generic Chaos
|
||||
|
||||
Chaos actions that apply to generic Kubernetes resources are classified into this category. Following chaos experiments are supported under Generic Chaos Chart
|
||||
|
||||
| Experiment name | Description | User guide link |
|
||||
| ----------- | ----------------------------------------- | --------------------------------------------------------- |
|
||||
| Container Kill | Kills the container in the application pod | [container-kill](container-kill.md)|
|
||||
| Pod Delete | Deletes the application pod | [pod-delete](pod-delete.md) |
|
||||
| Pod Network Latency | Injects network latency into the pod | [pod-network-latency](pod-network-latency.md) |
|
||||
| Pod Network Loss | Injects network loss into the pod | [pod-network-loss](pod-network-loss.md) |
|
||||
| Node CPU Hog | Exhaust CPU resources on the Kubernetes Node | [node-cpu-hog](node-cpu-hog.md) |
|
||||
| Node Memory Hog | Exhaust Memory resources on the Kubernetes Node | [node-memory-hog](node-memory-hog.md) |
|
||||
| Disk Fill | Fillup Ephemeral Storage of a Resource | [disk-fill](disk-fill.md) |
|
||||
| Disk Loss | External disk loss from the node | [disk-loss](disk-loss.md)|
|
||||
| Node Drain| Drains the node where application pod is scheduled | [node-drain](node-drain.md) |
|
||||
| Pod CPU Hog | Consumes CPU resources on the application container | [pod-cpu-hog](pod-cpu-hog.md) |
|
||||
| Pod Memory Hog | Consumes Memory resources on the application container | [pod-memory-hog](pod-memory-hog.md) |
|
||||
| Pod Network Corruption | Injects Network Packet Corruption into Application Pod |[pod-network-corruption](pod-network-corruption.md) |
|
||||
| Kubelet Service Kill | Kills the kubelet service on the application node |[kubelet-service-kill](kubelet-service-kill.md) |
|
||||
| Docker Service Kill | Kills the docker service on the application node |[docker-service-kill](docker-service-kill.md) |
|
||||
| Node Taint| Taints the node where application pod is scheduled | [node-taint](node-taint.md) |
|
||||
| Pod Autoscaler| Scales the application replicas and test the node autoscaling on cluster | [pod-autoscaler](pod-autoscaler.md) |
|
||||
| Pod Network Duplication | Injects Network Packet Duplication into Application Pod |[pod-network-duplication](pod-network-duplication.md) |
|
||||
| Pod IO Stress | Injects IO stress resources on the application container | [pod-io-stress](pod-io-stress.md) |
|
||||
| Node IO stress| Injects IO stress resources on the application node |[node-io-stress](node-io-stress.md) |
|
||||
|
||||
### Application Chaos
|
||||
|
||||
While Chaos Experiments under the Generic category offer the ability to induce chaos into Kubernetes resources, it is difficult to analyze and conclude if the chaos induced found a weakness in a given application. The application specific chaos experiments are built with some checks on *pre-conditions* and some expected outcomes after the chaos injection. The result of the chaos experiment is determined by matching the outcome with the expected outcome.
|
||||
|
||||
<div class="danger">
|
||||
<strong>NOTE:</strong> If the result of the chaos experiment is `pass`, it means that the application is resilient to that chaos.
|
||||
</div>
|
||||
|
||||
|
||||
#### Benefits of contributing an application chaos experiment
|
||||
|
||||
Application developers write negative tests in their CI pipelines to test the resiliency of the applications. These negative can be converted into Litmus Chaos Experiments and contributed to ChaosHub, so that the users of the application can use them in staging/pre-production/production environments to check the resilience. Application environments vary considerably from where they are tested (CI pipelines) to where they are deployed (Production). Hence, running the same chaos tests in the user's environment will help determine the weaknesses of the deployment and fixing such weaknesses leads to increased resilience.
|
||||
|
||||
|
||||
|
||||
Following Application Chaos experiments are available on ChaosHub
|
||||
|
||||
|
||||
|
||||
| Application | Description | Chaos Experiments |
|
||||
| ----------- | ----------------------------------------- | --------------------------------------------------------- |
|
||||
| OpenEBS | Container Attached Storage for Kubernetes | [openebs-pool-pod-failure](openebs-pool-pod-failure.md)<br>[openebs-pool-container-failure](openebs-pool-container-failure.md)<br>[openebs-target-pod-failure](openebs-target-pod-failure.md)<br>[openebs-target-container-failure](openebs-target-container-failure.md)<br>[openebs-target-network-delay](openebs-target-network-delay.md)<br>[openebs-target-network-loss](openebs-target-network-loss.md) <br>[openebs-control-plane-chaos](openebs-control-plane-chaos.md) <br>[openebs-nfs-provisioner-kill](openebs-nfs-provisioner-kill.md) <br>[openebs-target-network-loss](openebs-target-network-loss.md) <br>[openebs-pool-disk-loss](openebs-pool-disk-loss.md) <br>[openebs-pool-network-loss](openebs-pool-network-loss.md) <br>[openebs-pool-network-delay](openebs-pool-network-delay.md)|
|
||||
| Kafka | Open-source stream processing software | [kafka-broker-pod-failure](kafka-broker-pod-failure.md)<br>[kafka-broker-disk-failure](kafka-broker-disk-failure.md)<br> |
|
||||
| CoreDns | CoreDNS is a fast and flexible DNS server that chains plugins | [coredns-pod-delete](coredns-pod-delete.md)|
|
||||
| Cassandra | Cassandra is an opensource distributed database | [cassandra-pod-delete](cassandra-pod-delete.md)|
|
||||
|
||||
### Platform Chaos
|
||||
|
||||
Chaos experiments that inject chaos into the platform resources of Kubernetes are classified into this category. Management of platform resources vary significantly from each other, Chaos Charts may be maintained separately for each platform (For example, AWS, GCP, Azure, etc)
|
||||
|
||||
Following Platform Chaos experiments are available on ChaosHub
|
||||
|
||||
|
||||
|
||||
| Platform | Description | Chaos Experiments |
|
||||
| -------- | ------------------------------------------- | ----------------- |
|
||||
| AWS | Amazon Web Services platform. Includes EKS. | None |
|
||||
| GCP | Google Cloud Platform. Includes GKE. | None |
|
||||
| Azure | Microsoft Azure platform. Includes AKS. | None |
|
||||
|
||||
|
||||
|
|
@ -1,59 +0,0 @@
|
|||
{
|
||||
"version-1.0.0-docs": {
|
||||
"Getting Started": [
|
||||
"version-1.0.0-getstarted",
|
||||
"version-1.0.0-chaoshub",
|
||||
"version-1.0.0-plugins",
|
||||
"version-1.0.0-architecture",
|
||||
"version-1.0.0-resources",
|
||||
"version-1.0.0-community",
|
||||
"version-1.0.0-devguide"
|
||||
],
|
||||
"Experiments": [
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "Generic",
|
||||
"ids": [
|
||||
"version-1.0.0-pod-delete",
|
||||
"version-1.0.0-container-kill",
|
||||
"version-1.0.0-pod-network-latency",
|
||||
"version-1.0.0-pod-network-loss",
|
||||
"version-1.0.0-pod-network-corruption",
|
||||
"version-1.0.0-pod-cpu-hog",
|
||||
"version-1.0.0-disk-fill",
|
||||
"version-1.0.0-disk-loss",
|
||||
"version-1.0.0-cpu-hog",
|
||||
"version-1.0.0-node-drain"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "OpenEBS",
|
||||
"ids": [
|
||||
"version-1.0.0-openebs-target-container-failure",
|
||||
"version-1.0.0-openebs-target-network-delay",
|
||||
"version-1.0.0-openebs-target-network-loss",
|
||||
"version-1.0.0-openebs-target-pod-failure",
|
||||
"version-1.0.0-openebs-pool-pod-failure",
|
||||
"version-1.0.0-openebs-pool-container-failure",
|
||||
"version-1.0.0-openebs-target-network-delay"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "Kafka",
|
||||
"ids": [
|
||||
"version-1.0.0-kafka-broker-pod-failure",
|
||||
"version-1.0.0-kafka-broker-disk-failure"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "CoreDns",
|
||||
"ids": [
|
||||
"version-1.0.0-coredns-pod-delete"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
|
|
@ -1,66 +0,0 @@
|
|||
{
|
||||
"version-1.1.0-docs": {
|
||||
"Getting Started": [
|
||||
"version-1.1.0-getstarted",
|
||||
"version-1.1.0-chaoshub",
|
||||
"version-1.1.0-plugins",
|
||||
"version-1.1.0-architecture",
|
||||
"version-1.1.0-resources",
|
||||
"version-1.1.0-community",
|
||||
"version-1.1.0-devguide"
|
||||
],
|
||||
"Experiments": [
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "Generic",
|
||||
"ids": [
|
||||
"version-1.1.0-pod-delete",
|
||||
"version-1.1.0-container-kill",
|
||||
"version-1.1.0-pod-network-latency",
|
||||
"version-1.1.0-pod-network-loss",
|
||||
"version-1.1.0-pod-network-corruption",
|
||||
"version-1.1.0-pod-cpu-hog",
|
||||
"version-1.1.0-disk-fill",
|
||||
"version-1.1.0-disk-loss",
|
||||
"version-1.1.0-node-cpu-hog",
|
||||
"version-1.1.0-node-drain"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "OpenEBS",
|
||||
"ids": [
|
||||
"version-1.1.0-openebs-target-container-failure",
|
||||
"version-1.1.0-openebs-target-network-delay",
|
||||
"version-1.1.0-openebs-target-network-loss",
|
||||
"version-1.1.0-openebs-target-pod-failure",
|
||||
"version-1.1.0-openebs-pool-pod-failure",
|
||||
"version-1.1.0-openebs-pool-container-failure",
|
||||
"version-1.1.0-openebs-pool-network-delay",
|
||||
"version-1.1.0-openebs-pool-network-loss",
|
||||
"version-1.1.0-openebs-control-plane-chaos",
|
||||
"version-1.1.0-cStor-pool-chaos"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "Kafka",
|
||||
"ids": [
|
||||
"version-1.1.0-kafka-broker-pod-failure",
|
||||
"version-1.1.0-kafka-broker-disk-failure"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "CoreDns",
|
||||
"ids": [
|
||||
"version-1.1.0-coredns-pod-delete"
|
||||
]
|
||||
}
|
||||
],
|
||||
"Litmus FAQs": [
|
||||
"version-1.1.0-faq-general",
|
||||
"version-1.1.0-faq-troubleshooting"
|
||||
]
|
||||
}
|
||||
}
|
||||
|
|
@ -1,74 +0,0 @@
|
|||
{
|
||||
"version-1.2.0-docs": {
|
||||
"Getting Started": [
|
||||
"version-1.2.0-getstarted",
|
||||
"version-1.2.0-chaoshub",
|
||||
"version-1.2.0-plugins",
|
||||
"version-1.2.0-architecture",
|
||||
"version-1.2.0-resources",
|
||||
"version-1.2.0-community",
|
||||
"version-1.2.0-devguide"
|
||||
],
|
||||
"Concepts": [
|
||||
"version-1.2.0-chaosengine"
|
||||
],
|
||||
"Experiments": [
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "Generic",
|
||||
"ids": [
|
||||
"version-1.2.0-pod-delete",
|
||||
"version-1.2.0-container-kill",
|
||||
"version-1.2.0-pod-network-latency",
|
||||
"version-1.2.0-pod-network-loss",
|
||||
"version-1.2.0-pod-network-corruption",
|
||||
"version-1.2.0-pod-cpu-hog",
|
||||
"version-1.2.0-disk-fill",
|
||||
"version-1.2.0-disk-loss",
|
||||
"version-1.2.0-node-cpu-hog",
|
||||
"version-1.2.0-node-memory-hog",
|
||||
"version-1.2.0-node-drain"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "OpenEBS",
|
||||
"ids": [
|
||||
"version-1.2.0-openebs-target-container-failure",
|
||||
"version-1.2.0-openebs-target-network-delay",
|
||||
"version-1.2.0-openebs-target-network-loss",
|
||||
"version-1.2.0-openebs-target-pod-failure",
|
||||
"version-1.2.0-openebs-pool-pod-failure",
|
||||
"version-1.2.0-openebs-pool-container-failure",
|
||||
"version-1.2.0-openebs-pool-network-delay",
|
||||
"version-1.2.0-openebs-pool-network-loss",
|
||||
"version-1.2.0-openebs-control-plane-chaos",
|
||||
"version-1.2.0-cStor-pool-chaos",
|
||||
"version-1.2.0-openebs-pool-disk-loss"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "Kafka",
|
||||
"ids": [
|
||||
"version-1.2.0-kafka-broker-pod-failure",
|
||||
"version-1.2.0-kafka-broker-disk-failure"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "CoreDns",
|
||||
"ids": [
|
||||
"version-1.2.0-coredns-pod-delete"
|
||||
]
|
||||
}
|
||||
],
|
||||
"Litmus FAQs": [
|
||||
"version-1.2.0-faq-general",
|
||||
"version-1.2.0-faq-troubleshooting"
|
||||
],
|
||||
"Advanced": [
|
||||
"version-1.2.0-admin-mode"
|
||||
]
|
||||
}
|
||||
}
|
||||
|
|
@ -1,83 +0,0 @@
|
|||
{
|
||||
"version-1.3.0-docs": {
|
||||
"Getting Started": [
|
||||
"version-1.3.0-getstarted",
|
||||
"version-1.3.0-chaoshub",
|
||||
"version-1.3.0-plugins",
|
||||
"version-1.3.0-architecture",
|
||||
"version-1.3.0-resources",
|
||||
"version-1.3.0-community",
|
||||
"version-1.3.0-devguide"
|
||||
],
|
||||
"Concepts": [
|
||||
"version-1.3.0-chaosengine"
|
||||
],
|
||||
"Experiments": [
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "Generic",
|
||||
"ids": [
|
||||
"version-1.3.0-pod-delete",
|
||||
"version-1.3.0-container-kill",
|
||||
"version-1.3.0-pod-network-latency",
|
||||
"version-1.3.0-pod-network-loss",
|
||||
"version-1.3.0-pod-network-corruption",
|
||||
"version-1.3.0-pod-cpu-hog",
|
||||
"version-1.3.0-pod-memory-hog",
|
||||
"version-1.3.0-disk-fill",
|
||||
"version-1.3.0-disk-loss",
|
||||
"version-1.3.0-node-cpu-hog",
|
||||
"version-1.3.0-node-memory-hog",
|
||||
"version-1.3.0-node-drain"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "OpenEBS",
|
||||
"ids": [
|
||||
"version-1.3.0-openebs-target-container-failure",
|
||||
"version-1.3.0-openebs-target-network-delay",
|
||||
"version-1.3.0-openebs-target-network-loss",
|
||||
"version-1.3.0-openebs-target-pod-failure",
|
||||
"version-1.3.0-openebs-pool-pod-failure",
|
||||
"version-1.3.0-openebs-pool-container-failure",
|
||||
"version-1.3.0-openebs-pool-network-delay",
|
||||
"version-1.3.0-openebs-pool-network-loss",
|
||||
"version-1.3.0-openebs-control-plane-chaos",
|
||||
"version-1.3.0-cStor-pool-chaos",
|
||||
"version-1.3.0-openebs-pool-disk-loss",
|
||||
"version-1.3.0-openebs-nfs-provisioner-kill"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "Kafka",
|
||||
"ids": [
|
||||
"version-1.3.0-kafka-broker-pod-failure",
|
||||
"version-1.3.0-kafka-broker-disk-failure"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "CoreDns",
|
||||
"ids": [
|
||||
"version-1.3.0-coredns-pod-delete"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "Cassandra",
|
||||
"ids": [
|
||||
"version-1.3.0-cassandra-pod-delete"
|
||||
]
|
||||
}
|
||||
],
|
||||
"Litmus FAQs": [
|
||||
"version-1.3.0-faq-general",
|
||||
"version-1.3.0-faq-troubleshooting"
|
||||
],
|
||||
"Advanced": [
|
||||
"version-1.3.0-admin-mode"
|
||||
]
|
||||
}
|
||||
}
|
||||
|
|
@ -1,96 +0,0 @@
|
|||
{
|
||||
"version-1.4.0-docs": {
|
||||
"Getting Started": [
|
||||
"version-1.4.0-getstarted",
|
||||
"version-1.4.0-chaoshub",
|
||||
"version-1.4.0-plugins",
|
||||
"version-1.4.0-architecture",
|
||||
"version-1.4.0-resources",
|
||||
"version-1.4.0-community",
|
||||
"version-1.4.0-devguide"
|
||||
],
|
||||
"Concepts": [
|
||||
"version-1.4.0-chaosengine",
|
||||
"version-1.4.0-chaosschedule"
|
||||
],
|
||||
"Platforms": [
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "OpenShift",
|
||||
"ids": [
|
||||
"version-1.4.0-openshift-litmus"
|
||||
]
|
||||
}
|
||||
],
|
||||
"Experiments": [
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "Generic",
|
||||
"ids": [
|
||||
"version-1.4.0-pod-delete",
|
||||
"version-1.4.0-container-kill",
|
||||
"version-1.4.0-pod-network-latency",
|
||||
"version-1.4.0-pod-network-loss",
|
||||
"version-1.4.0-pod-network-corruption",
|
||||
"version-1.4.0-pod-cpu-hog",
|
||||
"version-1.4.0-pod-memory-hog",
|
||||
"version-1.4.0-disk-fill",
|
||||
"version-1.4.0-disk-loss",
|
||||
"version-1.4.0-node-cpu-hog",
|
||||
"version-1.4.0-node-memory-hog",
|
||||
"version-1.4.0-node-drain"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "OpenEBS",
|
||||
"ids": [
|
||||
"version-1.4.0-openebs-target-container-failure",
|
||||
"version-1.4.0-openebs-target-network-delay",
|
||||
"version-1.4.0-openebs-target-network-loss",
|
||||
"version-1.4.0-openebs-target-pod-failure",
|
||||
"version-1.4.0-openebs-pool-pod-failure",
|
||||
"version-1.4.0-openebs-pool-container-failure",
|
||||
"version-1.4.0-openebs-pool-network-delay",
|
||||
"version-1.4.0-openebs-pool-network-loss",
|
||||
"version-1.4.0-openebs-control-plane-chaos",
|
||||
"version-1.4.0-cStor-pool-chaos",
|
||||
"version-1.4.0-openebs-pool-disk-loss",
|
||||
"version-1.4.0-openebs-nfs-provisioner-kill"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "Kafka",
|
||||
"ids": [
|
||||
"version-1.4.0-kafka-broker-pod-failure",
|
||||
"version-1.4.0-kafka-broker-disk-failure"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "CoreDns",
|
||||
"ids": [
|
||||
"version-1.4.0-coredns-pod-delete"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "Cassandra",
|
||||
"ids": [
|
||||
"version-1.4.0-cassandra-pod-delete"
|
||||
]
|
||||
}
|
||||
],
|
||||
"Scheduler (alpha)": [
|
||||
"version-1.4.0-scheduling"
|
||||
],
|
||||
"Litmus FAQs": [
|
||||
"version-1.4.0-faq-general",
|
||||
"version-1.4.0-faq-troubleshooting"
|
||||
],
|
||||
"Advanced": [
|
||||
"version-1.4.0-admin-mode"
|
||||
]
|
||||
}
|
||||
}
|
||||
|
|
@ -1,97 +0,0 @@
|
|||
{
|
||||
"version-1.5.0-docs": {
|
||||
"Getting Started": [
|
||||
"version-1.5.0-getstarted",
|
||||
"version-1.5.0-chaoshub",
|
||||
"version-1.5.0-plugins",
|
||||
"version-1.5.0-architecture",
|
||||
"version-1.5.0-resources",
|
||||
"version-1.5.0-community",
|
||||
"version-1.5.0-devguide"
|
||||
],
|
||||
"Concepts": [
|
||||
"version-1.5.0-chaosengine",
|
||||
"version-1.5.0-chaosschedule"
|
||||
],
|
||||
"Platforms": [
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "OpenShift",
|
||||
"ids": [
|
||||
"version-1.5.0-openshift-litmus"
|
||||
]
|
||||
}
|
||||
],
|
||||
"Experiments": [
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "Generic",
|
||||
"ids": [
|
||||
"version-1.5.0-pod-delete",
|
||||
"version-1.5.0-container-kill",
|
||||
"version-1.5.0-pod-network-latency",
|
||||
"version-1.5.0-pod-network-loss",
|
||||
"version-1.5.0-pod-network-corruption",
|
||||
"version-1.5.0-pod-cpu-hog",
|
||||
"version-1.5.0-pod-memory-hog",
|
||||
"version-1.5.0-disk-fill",
|
||||
"version-1.5.0-disk-loss",
|
||||
"version-1.5.0-node-cpu-hog",
|
||||
"version-1.5.0-node-memory-hog",
|
||||
"version-1.5.0-node-drain",
|
||||
"version-1.5.0-kubelet-service-kill"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "OpenEBS",
|
||||
"ids": [
|
||||
"version-1.5.0-openebs-target-container-failure",
|
||||
"version-1.5.0-openebs-target-network-delay",
|
||||
"version-1.5.0-openebs-target-network-loss",
|
||||
"version-1.5.0-openebs-target-pod-failure",
|
||||
"version-1.5.0-openebs-pool-pod-failure",
|
||||
"version-1.5.0-openebs-pool-container-failure",
|
||||
"version-1.5.0-openebs-pool-network-delay",
|
||||
"version-1.5.0-openebs-pool-network-loss",
|
||||
"version-1.5.0-openebs-control-plane-chaos",
|
||||
"version-1.5.0-cStor-pool-chaos",
|
||||
"version-1.5.0-openebs-pool-disk-loss",
|
||||
"version-1.5.0-openebs-nfs-provisioner-kill"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "Kafka",
|
||||
"ids": [
|
||||
"version-1.5.0-kafka-broker-pod-failure",
|
||||
"version-1.5.0-kafka-broker-disk-failure"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "CoreDns",
|
||||
"ids": [
|
||||
"version-1.5.0-coredns-pod-delete"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "Cassandra",
|
||||
"ids": [
|
||||
"version-1.5.0-cassandra-pod-delete"
|
||||
]
|
||||
}
|
||||
],
|
||||
"Scheduler (alpha)": [
|
||||
"version-1.5.0-scheduling"
|
||||
],
|
||||
"Litmus FAQs": [
|
||||
"version-1.5.0-faq-general",
|
||||
"version-1.5.0-faq-troubleshooting"
|
||||
],
|
||||
"Advanced": [
|
||||
"version-1.5.0-admin-mode"
|
||||
]
|
||||
}
|
||||
}
|
||||
|
|
@ -1,113 +0,0 @@
|
|||
{
|
||||
"version-1.6.0-docs": {
|
||||
"Getting Started": [
|
||||
"version-1.6.0-getstarted",
|
||||
"version-1.6.0-chaoshub",
|
||||
"version-1.6.0-plugins",
|
||||
"version-1.6.0-architecture",
|
||||
"version-1.6.0-resources",
|
||||
"version-1.6.0-community",
|
||||
"version-1.6.0-devguide"
|
||||
],
|
||||
"Litmus Demo": [
|
||||
"version-1.6.0-litmus-demo"
|
||||
],
|
||||
"Concepts": [
|
||||
"version-1.6.0-chaosengine",
|
||||
"version-1.6.0-chaosschedule"
|
||||
],
|
||||
"Platforms": [
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "OpenShift",
|
||||
"ids": [
|
||||
"version-1.6.0-openshift-litmus"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "Rancher",
|
||||
"ids": [
|
||||
"version-1.6.0-rancher-litmus"
|
||||
]
|
||||
}
|
||||
],
|
||||
"Experiments": [
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "Generic",
|
||||
"ids": [
|
||||
"version-1.6.0-pod-delete",
|
||||
"version-1.6.0-container-kill",
|
||||
"version-1.6.0-pod-network-latency",
|
||||
"version-1.6.0-pod-network-loss",
|
||||
"version-1.6.0-pod-network-corruption",
|
||||
"version-1.6.0-pod-cpu-hog",
|
||||
"version-1.6.0-pod-memory-hog",
|
||||
"version-1.6.0-disk-fill",
|
||||
"version-1.6.0-disk-loss",
|
||||
"version-1.6.0-node-cpu-hog",
|
||||
"version-1.6.0-node-memory-hog",
|
||||
"version-1.6.0-node-drain",
|
||||
"version-1.6.0-kubelet-service-kill",
|
||||
"version-1.6.0-pod-network-duplication",
|
||||
"version-1.6.0-node-taint",
|
||||
"version-1.6.0-docker-service-kill"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "OpenEBS",
|
||||
"ids": [
|
||||
"version-1.6.0-openebs-target-container-failure",
|
||||
"version-1.6.0-openebs-target-network-delay",
|
||||
"version-1.6.0-openebs-target-network-loss",
|
||||
"version-1.6.0-openebs-target-pod-failure",
|
||||
"version-1.6.0-openebs-pool-pod-failure",
|
||||
"version-1.6.0-openebs-pool-container-failure",
|
||||
"version-1.6.0-openebs-pool-network-delay",
|
||||
"version-1.6.0-openebs-pool-network-loss",
|
||||
"version-1.6.0-openebs-control-plane-chaos",
|
||||
"version-1.6.0-cStor-pool-chaos",
|
||||
"version-1.6.0-openebs-pool-disk-loss",
|
||||
"version-1.6.0-openebs-nfs-provisioner-kill"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "Kafka",
|
||||
"ids": [
|
||||
"version-1.6.0-kafka-broker-pod-failure",
|
||||
"version-1.6.0-kafka-broker-disk-failure"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "CoreDns",
|
||||
"ids": [
|
||||
"version-1.6.0-coredns-pod-delete"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "Cassandra",
|
||||
"ids": [
|
||||
"version-1.6.0-cassandra-pod-delete"
|
||||
]
|
||||
}
|
||||
],
|
||||
"Scheduler": [
|
||||
"version-1.6.0-scheduling"
|
||||
],
|
||||
"Chaos Workflow": [
|
||||
"version-1.6.0-chaos-workflows"
|
||||
],
|
||||
"Litmus FAQs": [
|
||||
"version-1.6.0-faq-general",
|
||||
"version-1.6.0-faq-troubleshooting"
|
||||
],
|
||||
"Advanced": [
|
||||
"version-1.6.0-admin-mode"
|
||||
]
|
||||
}
|
||||
}
|
||||
|
|
@ -1,126 +0,0 @@
|
|||
{
|
||||
"version-1.7.0-docs": {
|
||||
"Getting Started": [
|
||||
"version-1.7.0-getstarted",
|
||||
"version-1.7.0-chaoshub",
|
||||
"version-1.7.0-plugins",
|
||||
"version-1.7.0-architecture",
|
||||
"version-1.7.0-resources",
|
||||
"version-1.7.0-community",
|
||||
"version-1.7.0-devguide"
|
||||
],
|
||||
"Litmus Demo": [
|
||||
"version-1.7.0-litmus-demo"
|
||||
],
|
||||
"Concepts": [
|
||||
"version-1.7.0-chaosengine",
|
||||
"version-1.7.0-chaosschedule",
|
||||
"version-1.7.0-litmus-probe"
|
||||
],
|
||||
"Platforms": [
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "OpenShift",
|
||||
"ids": [
|
||||
"version-1.7.0-openshift-litmus"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "Rancher",
|
||||
"ids": [
|
||||
"version-1.7.0-rancher-litmus"
|
||||
]
|
||||
}
|
||||
],
|
||||
"Experiments": [
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "Generic",
|
||||
"ids": [
|
||||
"version-1.7.0-pod-delete",
|
||||
"version-1.7.0-container-kill",
|
||||
"version-1.7.0-pod-network-latency",
|
||||
"version-1.7.0-pod-network-loss",
|
||||
"version-1.7.0-pod-network-corruption",
|
||||
"version-1.7.0-pod-cpu-hog",
|
||||
"version-1.7.0-pod-memory-hog",
|
||||
"version-1.7.0-disk-fill",
|
||||
"version-1.7.0-disk-loss",
|
||||
"version-1.7.0-node-cpu-hog",
|
||||
"version-1.7.0-node-memory-hog",
|
||||
"version-1.7.0-node-drain",
|
||||
"version-1.7.0-kubelet-service-kill",
|
||||
"version-1.7.0-pod-network-duplication",
|
||||
"version-1.7.0-node-taint",
|
||||
"version-1.7.0-docker-service-kill",
|
||||
"version-1.7.0-pod-autoscaler"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "Kubernetes",
|
||||
"ids": [
|
||||
"version-1.7.0-Kubernetes-Chaostoolkit-Application",
|
||||
"version-1.7.0-Kubernetes-Chaostoolkit-Service",
|
||||
"version-1.7.0-Kubernetes-Chaostoolkit-Cluster-Kiam",
|
||||
"version-1.7.0-Kubernetes-Chaostoolkit-AWS"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "OpenEBS",
|
||||
"ids": [
|
||||
"version-1.7.0-openebs-target-container-failure",
|
||||
"version-1.7.0-openebs-target-network-delay",
|
||||
"version-1.7.0-openebs-target-network-loss",
|
||||
"version-1.7.0-openebs-target-pod-failure",
|
||||
"version-1.7.0-openebs-pool-pod-failure",
|
||||
"version-1.7.0-openebs-pool-container-failure",
|
||||
"version-1.7.0-openebs-pool-network-delay",
|
||||
"version-1.7.0-openebs-pool-network-loss",
|
||||
"version-1.7.0-openebs-control-plane-chaos",
|
||||
"version-1.7.0-cStor-pool-chaos",
|
||||
"version-1.7.0-openebs-pool-disk-loss",
|
||||
"version-1.7.0-openebs-nfs-provisioner-kill"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "Kafka",
|
||||
"ids": [
|
||||
"version-1.7.0-kafka-broker-pod-failure",
|
||||
"version-1.7.0-kafka-broker-disk-failure"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "CoreDns",
|
||||
"ids": [
|
||||
"version-1.7.0-coredns-pod-delete"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "Cassandra",
|
||||
"ids": [
|
||||
"version-1.7.0-cassandra-pod-delete"
|
||||
]
|
||||
}
|
||||
],
|
||||
"Scheduler": [
|
||||
"version-1.7.0-scheduling"
|
||||
],
|
||||
"Chaos Workflow": [
|
||||
"version-1.7.0-chaos-workflows"
|
||||
],
|
||||
"Litmus FAQs": [
|
||||
"version-1.7.0-faq-general",
|
||||
"version-1.7.0-faq-troubleshooting"
|
||||
],
|
||||
"Advanced": [
|
||||
"version-1.7.0-admin-mode",
|
||||
"version-1.7.0-namespaced-mode"
|
||||
]
|
||||
}
|
||||
}
|
||||
|
|
@ -1,128 +0,0 @@
|
|||
{
|
||||
"version-1.8.0-docs": {
|
||||
"Getting Started": [
|
||||
"version-1.8.0-getstarted",
|
||||
"version-1.8.0-chaoshub",
|
||||
"version-1.8.0-plugins",
|
||||
"version-1.8.0-architecture",
|
||||
"version-1.8.0-resources",
|
||||
"version-1.8.0-community",
|
||||
"version-1.8.0-devguide"
|
||||
],
|
||||
"Litmus Demo": [
|
||||
"version-1.8.0-litmus-demo"
|
||||
],
|
||||
"Concepts": [
|
||||
"version-1.8.0-chaosengine",
|
||||
"version-1.8.0-chaosschedule",
|
||||
"version-1.8.0-litmus-probe"
|
||||
],
|
||||
"Platforms": [
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "OpenShift",
|
||||
"ids": [
|
||||
"version-1.8.0-openshift-litmus"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "Rancher",
|
||||
"ids": [
|
||||
"version-1.8.0-rancher-litmus"
|
||||
]
|
||||
}
|
||||
],
|
||||
"Experiments": [
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "Generic",
|
||||
"ids": [
|
||||
"version-1.8.0-pod-delete",
|
||||
"version-1.8.0-container-kill",
|
||||
"version-1.8.0-pod-network-latency",
|
||||
"version-1.8.0-pod-network-loss",
|
||||
"version-1.8.0-pod-network-corruption",
|
||||
"version-1.8.0-pod-cpu-hog",
|
||||
"version-1.8.0-pod-memory-hog",
|
||||
"version-1.8.0-disk-fill",
|
||||
"version-1.8.0-disk-loss",
|
||||
"version-1.8.0-node-cpu-hog",
|
||||
"version-1.8.0-node-memory-hog",
|
||||
"version-1.8.0-node-drain",
|
||||
"version-1.8.0-kubelet-service-kill",
|
||||
"version-1.8.0-pod-network-duplication",
|
||||
"version-1.8.0-node-taint",
|
||||
"version-1.8.0-docker-service-kill",
|
||||
"version-1.8.0-pod-autoscaler",
|
||||
"version-1.8.0-Kubernetes-Chaostoolkit-Application",
|
||||
"version-1.8.0-Kubernetes-Chaostoolkit-Service",
|
||||
"version-1.8.0-Kubernetes-Chaostoolkit-Cluster-Kiam",
|
||||
"version-1.8.0-pod-io-stress",
|
||||
"version-1.8.0-node-io-stress"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "Kube-AWS",
|
||||
"ids": [
|
||||
"version-1.8.0-Kubernetes-Chaostoolkit-AWS"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "OpenEBS",
|
||||
"ids": [
|
||||
"version-1.8.0-openebs-target-container-failure",
|
||||
"version-1.8.0-openebs-target-network-delay",
|
||||
"version-1.8.0-openebs-target-network-loss",
|
||||
"version-1.8.0-openebs-target-pod-failure",
|
||||
"version-1.8.0-openebs-pool-pod-failure",
|
||||
"version-1.8.0-openebs-pool-container-failure",
|
||||
"version-1.8.0-openebs-pool-network-delay",
|
||||
"version-1.8.0-openebs-pool-network-loss",
|
||||
"version-1.8.0-openebs-control-plane-chaos",
|
||||
"version-1.8.0-cStor-pool-chaos",
|
||||
"version-1.8.0-openebs-pool-disk-loss",
|
||||
"version-1.8.0-openebs-nfs-provisioner-kill"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "Kafka",
|
||||
"ids": [
|
||||
"version-1.8.0-kafka-broker-pod-failure",
|
||||
"version-1.8.0-kafka-broker-disk-failure"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "CoreDns",
|
||||
"ids": [
|
||||
"version-1.8.0-coredns-pod-delete"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "Cassandra",
|
||||
"ids": [
|
||||
"version-1.8.0-cassandra-pod-delete"
|
||||
]
|
||||
}
|
||||
],
|
||||
"Scheduler": [
|
||||
"version-1.8.0-scheduling"
|
||||
],
|
||||
"Chaos Workflow": [
|
||||
"version-1.8.0-chaos-workflows"
|
||||
],
|
||||
"Litmus FAQs": [
|
||||
"version-1.8.0-faq-general",
|
||||
"version-1.8.0-faq-troubleshooting"
|
||||
],
|
||||
"Advanced": [
|
||||
"version-1.8.0-admin-mode",
|
||||
"version-1.8.0-namespaced-mode"
|
||||
]
|
||||
}
|
||||
}
|
||||
|
|
@ -1,130 +0,0 @@
|
|||
{
|
||||
"version-1.9.0-docs": {
|
||||
"Getting Started": [
|
||||
"version-1.9.0-getstarted",
|
||||
"version-1.9.0-chaoshub",
|
||||
"version-1.9.0-plugins",
|
||||
"version-1.9.0-architecture",
|
||||
"version-1.9.0-resources",
|
||||
"version-1.9.0-community",
|
||||
"version-1.9.0-devguide"
|
||||
],
|
||||
"Litmus Demo": [
|
||||
"version-1.9.0-litmus-demo"
|
||||
],
|
||||
"Concepts": [
|
||||
"version-1.9.0-chaosengine",
|
||||
"version-1.9.0-chaosexperiment",
|
||||
"version-1.9.0-chaosschedule",
|
||||
"version-1.9.0-chaosresult",
|
||||
"version-1.9.0-litmus-probe"
|
||||
],
|
||||
"Platforms": [
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "OpenShift",
|
||||
"ids": [
|
||||
"version-1.9.0-openshift-litmus"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "Rancher",
|
||||
"ids": [
|
||||
"version-1.9.0-rancher-litmus"
|
||||
]
|
||||
}
|
||||
],
|
||||
"Experiments": [
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "Generic",
|
||||
"ids": [
|
||||
"version-1.9.0-pod-delete",
|
||||
"version-1.9.0-container-kill",
|
||||
"version-1.9.0-pod-network-latency",
|
||||
"version-1.9.0-pod-network-loss",
|
||||
"version-1.9.0-pod-network-corruption",
|
||||
"version-1.9.0-pod-cpu-hog",
|
||||
"version-1.9.0-pod-memory-hog",
|
||||
"version-1.9.0-disk-fill",
|
||||
"version-1.9.0-disk-loss",
|
||||
"version-1.9.0-node-cpu-hog",
|
||||
"version-1.9.0-node-memory-hog",
|
||||
"version-1.9.0-node-drain",
|
||||
"version-1.9.0-kubelet-service-kill",
|
||||
"version-1.9.0-pod-network-duplication",
|
||||
"version-1.9.0-node-taint",
|
||||
"version-1.9.0-docker-service-kill",
|
||||
"version-1.9.0-pod-autoscaler",
|
||||
"version-1.9.0-Kubernetes-Chaostoolkit-Application",
|
||||
"version-1.9.0-Kubernetes-Chaostoolkit-Service",
|
||||
"version-1.9.0-Kubernetes-Chaostoolkit-Cluster-Kiam",
|
||||
"version-1.9.0-pod-io-stress",
|
||||
"version-1.9.0-node-io-stress"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "Kube-AWS",
|
||||
"ids": [
|
||||
"version-1.9.0-Kubernetes-Chaostoolkit-AWS"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "OpenEBS",
|
||||
"ids": [
|
||||
"version-1.9.0-openebs-target-container-failure",
|
||||
"version-1.9.0-openebs-target-network-delay",
|
||||
"version-1.9.0-openebs-target-network-loss",
|
||||
"version-1.9.0-openebs-target-pod-failure",
|
||||
"version-1.9.0-openebs-pool-pod-failure",
|
||||
"version-1.9.0-openebs-pool-container-failure",
|
||||
"version-1.9.0-openebs-pool-network-delay",
|
||||
"version-1.9.0-openebs-pool-network-loss",
|
||||
"version-1.9.0-openebs-control-plane-chaos",
|
||||
"version-1.9.0-cStor-pool-chaos",
|
||||
"version-1.9.0-openebs-pool-disk-loss",
|
||||
"version-1.9.0-openebs-nfs-provisioner-kill"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "Kafka",
|
||||
"ids": [
|
||||
"version-1.9.0-kafka-broker-pod-failure",
|
||||
"version-1.9.0-kafka-broker-disk-failure"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "CoreDns",
|
||||
"ids": [
|
||||
"version-1.9.0-coredns-pod-delete"
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "subcategory",
|
||||
"label": "Cassandra",
|
||||
"ids": [
|
||||
"version-1.9.0-cassandra-pod-delete"
|
||||
]
|
||||
}
|
||||
],
|
||||
"Scheduler": [
|
||||
"version-1.9.0-scheduling"
|
||||
],
|
||||
"Chaos Workflow": [
|
||||
"version-1.9.0-chaos-workflows"
|
||||
],
|
||||
"Litmus FAQs": [
|
||||
"version-1.9.0-faq-general",
|
||||
"version-1.9.0-faq-troubleshooting"
|
||||
],
|
||||
"Advanced": [
|
||||
"version-1.9.0-admin-mode",
|
||||
"version-1.9.0-namespaced-mode"
|
||||
]
|
||||
}
|
||||
}
|
||||
|
|
@ -1,6 +1,6 @@
|
|||
---
|
||||
id: faq-troubleshooting
|
||||
title: Troubleshooting Litmus
|
||||
title: "Troubleshooting Litmus"
|
||||
sidebar_label: Troubleshooting
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -1,7 +1,6 @@
|
|||
---
|
||||
id: getstarted
|
||||
title: Getting Started with Litmus
|
||||
slug: "/"
|
||||
sidebar_label: Introduction
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -42,7 +42,7 @@ module.exports = {
|
|||
tagline: "A website for testing",
|
||||
url: "https://docs.litmuschaos.io",
|
||||
baseUrl: "/",
|
||||
onBrokenLinks: "ignore",
|
||||
onBrokenLinks: "throw",
|
||||
favicon: "img/favicon.ico",
|
||||
organizationName: "litmuschaos",
|
||||
projectName: "litmus",
|
||||
|
|
@ -72,11 +72,11 @@ module.exports = {
|
|||
},
|
||||
...versions.slice(1).map((version) => ({
|
||||
label: version,
|
||||
to: `docs/${version}/`,
|
||||
to: `docs/${version}/getstarted`,
|
||||
})),
|
||||
{
|
||||
label: "Master/Unreleased",
|
||||
to: "docs/next/",
|
||||
to: "docs/next/getstarted",
|
||||
},
|
||||
],
|
||||
},
|
||||
|
|
@ -121,9 +121,9 @@ module.exports = {
|
|||
"@docusaurus/preset-classic",
|
||||
{
|
||||
docs: {
|
||||
routeBasePath: "docs",
|
||||
sidebarPath: require.resolve("./sidebars.js"),
|
||||
editUrl: "https://github.com/litmuschaos/litmus-docs-beta/edit/staging/",
|
||||
editUrl:
|
||||
"https://github.com/litmuschaos/litmus-docs-beta/edit/staging/",
|
||||
showLastUpdateTime: true,
|
||||
},
|
||||
theme: {
|
||||
|
|
|
|||
|
|
@ -0,0 +1,7 @@
|
|||
import React from "react";
|
||||
import { Redirect } from "@docusaurus/router";
|
||||
function Docs() {
|
||||
return <Redirect to="/docs/getstarted" />;
|
||||
}
|
||||
|
||||
export default Docs;
|
||||
|
|
@ -1,13 +1,13 @@
|
|||
---
|
||||
id: version-1.0.0-architecture
|
||||
id: architecture
|
||||
title: Litmus Architecture
|
||||
sidebar_label: Architecture
|
||||
original_id: architecture
|
||||
---
|
||||
|
||||
<hr>
|
||||
<hr />
|
||||
|
||||
<img src="/docs/assets/architecture.png" width="800">
|
||||
<img src={require('./assets/architecture.png').default} width="800" />
|
||||
|
||||
**Chaos-Operator**
|
||||
|
||||
|
|
@ -23,32 +23,24 @@ During installation, the following three CRDs are installed on the Kubernetes cl
|
|||
|
||||
`chaosresults.litmuschaos.io`
|
||||
|
||||
|
||||
|
||||
**Chaos-Experiments**
|
||||
|
||||
Chaos Experiment is a CR and are available as YAML files on <a href=" https://hub.litmuschaos.io" target="_blank">Chaos Hub</a>. For more details visit Chaos Hub [documentation](chaoshub.md).
|
||||
|
||||
|
||||
Chaos Experiment is a CR and are available as YAML files on <a href="https://hub.litmuschaos.io" target="_blank">Chaos Hub</a>. For more details visit Chaos Hub [documentation](chaoshub.md).
|
||||
|
||||
**Chaos-Engine**
|
||||
|
||||
ChaosEngine CR links application to experiments. User has to create ChaosEngine YAML by specifying the application label and experiments and create the CR. The CR is watched by Chaos-Operator and chaos-experiments are executed on a given application.
|
||||
|
||||
|
||||
|
||||
**Chaos-Exporter**
|
||||
|
||||
Optionally metrics can be exported to a Prometheus database. Chaos-Exporter implements the Prometheus metrics endpoint.
|
||||
Optionally metrics can be exported to a Prometheus database. Chaos-Exporter implements the Prometheus metrics endpoint.
|
||||
|
||||
<br />
|
||||
|
||||
<br />
|
||||
|
||||
<br>
|
||||
<hr />
|
||||
|
||||
<br>
|
||||
<br />
|
||||
|
||||
<hr>
|
||||
|
||||
<br>
|
||||
|
||||
<br>
|
||||
<br />
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 33 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 15 KiB |
|
|
@ -0,0 +1,429 @@
|
|||
<?xml version="1.0"?>
|
||||
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="1349.33333328" height="862.4" font-family="Consolas, Menlo, 'Bitstream Vera Sans Mono', monospace, 'Powerline Symbols'" font-size="14px">
|
||||
<style>
|
||||
<!-- asciinema theme -->
|
||||
|
||||
.default-text-fill {fill: #cccccc}
|
||||
.default-bg-fill {fill: #121314}
|
||||
|
||||
.c-0 {fill: #000000}
|
||||
.c-1 {fill: #dd3c69}
|
||||
.c-2 {fill: #4ebf22}
|
||||
.c-3 {fill: #ddaf3c}
|
||||
.c-4 {fill: #26b0d7}
|
||||
.c-5 {fill: #b954e1}
|
||||
.c-6 {fill: #54e1b9}
|
||||
.c-7 {fill: #d9d9d9}
|
||||
.c-8 {fill: #4d4d4d}
|
||||
.c-9 {fill: #dd3c69}
|
||||
.c-10 {fill: #4ebf22}
|
||||
.c-11 {fill: #ddaf3c}
|
||||
.c-12 {fill: #26b0d7}
|
||||
.c-13 {fill: #b954e1}
|
||||
.c-14 {fill: #54e1b9}
|
||||
.c-15 {fill: #ffffff}
|
||||
.c-8, .c-9, .c-10, .c-11, .c-12, .c-13, .c-14, .c-15 {font-weight: bold}
|
||||
<!-- 256 colors -->
|
||||
|
||||
.c-16 {fill: #000000}
|
||||
.c-17 {fill: #00005f}
|
||||
.c-18 {fill: #000087}
|
||||
.c-19 {fill: #0000af}
|
||||
.c-20 {fill: #0000d7}
|
||||
.c-21 {fill: #0000ff}
|
||||
.c-22 {fill: #005f00}
|
||||
.c-23 {fill: #005f5f}
|
||||
.c-24 {fill: #005f87}
|
||||
.c-25 {fill: #005faf}
|
||||
.c-26 {fill: #005fd7}
|
||||
.c-27 {fill: #005fff}
|
||||
.c-28 {fill: #008700}
|
||||
.c-29 {fill: #00875f}
|
||||
.c-30 {fill: #008787}
|
||||
.c-31 {fill: #0087af}
|
||||
.c-32 {fill: #0087d7}
|
||||
.c-33 {fill: #0087ff}
|
||||
.c-34 {fill: #00af00}
|
||||
.c-35 {fill: #00af5f}
|
||||
.c-36 {fill: #00af87}
|
||||
.c-37 {fill: #00afaf}
|
||||
.c-38 {fill: #00afd7}
|
||||
.c-39 {fill: #00afff}
|
||||
.c-40 {fill: #00d700}
|
||||
.c-41 {fill: #00d75f}
|
||||
.c-42 {fill: #00d787}
|
||||
.c-43 {fill: #00d7af}
|
||||
.c-44 {fill: #00d7d7}
|
||||
.c-45 {fill: #00d7ff}
|
||||
.c-46 {fill: #00ff00}
|
||||
.c-47 {fill: #00ff5f}
|
||||
.c-48 {fill: #00ff87}
|
||||
.c-49 {fill: #00ffaf}
|
||||
.c-50 {fill: #00ffd7}
|
||||
.c-51 {fill: #00ffff}
|
||||
.c-52 {fill: #5f0000}
|
||||
.c-53 {fill: #5f005f}
|
||||
.c-54 {fill: #5f0087}
|
||||
.c-55 {fill: #5f00af}
|
||||
.c-56 {fill: #5f00d7}
|
||||
.c-57 {fill: #5f00ff}
|
||||
.c-58 {fill: #5f5f00}
|
||||
.c-59 {fill: #5f5f5f}
|
||||
.c-60 {fill: #5f5f87}
|
||||
.c-61 {fill: #5f5faf}
|
||||
.c-62 {fill: #5f5fd7}
|
||||
.c-63 {fill: #5f5fff}
|
||||
.c-64 {fill: #5f8700}
|
||||
.c-65 {fill: #5f875f}
|
||||
.c-66 {fill: #5f8787}
|
||||
.c-67 {fill: #5f87af}
|
||||
.c-68 {fill: #5f87d7}
|
||||
.c-69 {fill: #5f87ff}
|
||||
.c-70 {fill: #5faf00}
|
||||
.c-71 {fill: #5faf5f}
|
||||
.c-72 {fill: #5faf87}
|
||||
.c-73 {fill: #5fafaf}
|
||||
.c-74 {fill: #5fafd7}
|
||||
.c-75 {fill: #5fafff}
|
||||
.c-76 {fill: #5fd700}
|
||||
.c-77 {fill: #5fd75f}
|
||||
.c-78 {fill: #5fd787}
|
||||
.c-79 {fill: #5fd7af}
|
||||
.c-80 {fill: #5fd7d7}
|
||||
.c-81 {fill: #5fd7ff}
|
||||
.c-82 {fill: #5fff00}
|
||||
.c-83 {fill: #5fff5f}
|
||||
.c-84 {fill: #5fff87}
|
||||
.c-85 {fill: #5fffaf}
|
||||
.c-86 {fill: #5fffd7}
|
||||
.c-87 {fill: #5fffff}
|
||||
.c-88 {fill: #870000}
|
||||
.c-89 {fill: #87005f}
|
||||
.c-90 {fill: #870087}
|
||||
.c-91 {fill: #8700af}
|
||||
.c-92 {fill: #8700d7}
|
||||
.c-93 {fill: #8700ff}
|
||||
.c-94 {fill: #875f00}
|
||||
.c-95 {fill: #875f5f}
|
||||
.c-96 {fill: #875f87}
|
||||
.c-97 {fill: #875faf}
|
||||
.c-98 {fill: #875fd7}
|
||||
.c-99 {fill: #875fff}
|
||||
.c-100 {fill: #878700}
|
||||
.c-101 {fill: #87875f}
|
||||
.c-102 {fill: #878787}
|
||||
.c-103 {fill: #8787af}
|
||||
.c-104 {fill: #8787d7}
|
||||
.c-105 {fill: #8787ff}
|
||||
.c-106 {fill: #87af00}
|
||||
.c-107 {fill: #87af5f}
|
||||
.c-108 {fill: #87af87}
|
||||
.c-109 {fill: #87afaf}
|
||||
.c-110 {fill: #87afd7}
|
||||
.c-111 {fill: #87afff}
|
||||
.c-112 {fill: #87d700}
|
||||
.c-113 {fill: #87d75f}
|
||||
.c-114 {fill: #87d787}
|
||||
.c-115 {fill: #87d7af}
|
||||
.c-116 {fill: #87d7d7}
|
||||
.c-117 {fill: #87d7ff}
|
||||
.c-118 {fill: #87ff00}
|
||||
.c-119 {fill: #87ff5f}
|
||||
.c-120 {fill: #87ff87}
|
||||
.c-121 {fill: #87ffaf}
|
||||
.c-122 {fill: #87ffd7}
|
||||
.c-123 {fill: #87ffff}
|
||||
.c-124 {fill: #af0000}
|
||||
.c-125 {fill: #af005f}
|
||||
.c-126 {fill: #af0087}
|
||||
.c-127 {fill: #af00af}
|
||||
.c-128 {fill: #af00d7}
|
||||
.c-129 {fill: #af00ff}
|
||||
.c-130 {fill: #af5f00}
|
||||
.c-131 {fill: #af5f5f}
|
||||
.c-132 {fill: #af5f87}
|
||||
.c-133 {fill: #af5faf}
|
||||
.c-134 {fill: #af5fd7}
|
||||
.c-135 {fill: #af5fff}
|
||||
.c-136 {fill: #af8700}
|
||||
.c-137 {fill: #af875f}
|
||||
.c-138 {fill: #af8787}
|
||||
.c-139 {fill: #af87af}
|
||||
.c-140 {fill: #af87d7}
|
||||
.c-141 {fill: #af87ff}
|
||||
.c-142 {fill: #afaf00}
|
||||
.c-143 {fill: #afaf5f}
|
||||
.c-144 {fill: #afaf87}
|
||||
.c-145 {fill: #afafaf}
|
||||
.c-146 {fill: #afafd7}
|
||||
.c-147 {fill: #afafff}
|
||||
.c-148 {fill: #afd700}
|
||||
.c-149 {fill: #afd75f}
|
||||
.c-150 {fill: #afd787}
|
||||
.c-151 {fill: #afd7af}
|
||||
.c-152 {fill: #afd7d7}
|
||||
.c-153 {fill: #afd7ff}
|
||||
.c-154 {fill: #afff00}
|
||||
.c-155 {fill: #afff5f}
|
||||
.c-156 {fill: #afff87}
|
||||
.c-157 {fill: #afffaf}
|
||||
.c-158 {fill: #afffd7}
|
||||
.c-159 {fill: #afffff}
|
||||
.c-160 {fill: #d70000}
|
||||
.c-161 {fill: #d7005f}
|
||||
.c-162 {fill: #d70087}
|
||||
.c-163 {fill: #d700af}
|
||||
.c-164 {fill: #d700d7}
|
||||
.c-165 {fill: #d700ff}
|
||||
.c-166 {fill: #d75f00}
|
||||
.c-167 {fill: #d75f5f}
|
||||
.c-168 {fill: #d75f87}
|
||||
.c-169 {fill: #d75faf}
|
||||
.c-170 {fill: #d75fd7}
|
||||
.c-171 {fill: #d75fff}
|
||||
.c-172 {fill: #d78700}
|
||||
.c-173 {fill: #d7875f}
|
||||
.c-174 {fill: #d78787}
|
||||
.c-175 {fill: #d787af}
|
||||
.c-176 {fill: #d787d7}
|
||||
.c-177 {fill: #d787ff}
|
||||
.c-178 {fill: #d7af00}
|
||||
.c-179 {fill: #d7af5f}
|
||||
.c-180 {fill: #d7af87}
|
||||
.c-181 {fill: #d7afaf}
|
||||
.c-182 {fill: #d7afd7}
|
||||
.c-183 {fill: #d7afff}
|
||||
.c-184 {fill: #d7d700}
|
||||
.c-185 {fill: #d7d75f}
|
||||
.c-186 {fill: #d7d787}
|
||||
.c-187 {fill: #d7d7af}
|
||||
.c-188 {fill: #d7d7d7}
|
||||
.c-189 {fill: #d7d7ff}
|
||||
.c-190 {fill: #d7ff00}
|
||||
.c-191 {fill: #d7ff5f}
|
||||
.c-192 {fill: #d7ff87}
|
||||
.c-193 {fill: #d7ffaf}
|
||||
.c-194 {fill: #d7ffd7}
|
||||
.c-195 {fill: #d7ffff}
|
||||
.c-196 {fill: #ff0000}
|
||||
.c-197 {fill: #ff005f}
|
||||
.c-198 {fill: #ff0087}
|
||||
.c-199 {fill: #ff00af}
|
||||
.c-200 {fill: #ff00d7}
|
||||
.c-201 {fill: #ff00ff}
|
||||
.c-202 {fill: #ff5f00}
|
||||
.c-203 {fill: #ff5f5f}
|
||||
.c-204 {fill: #ff5f87}
|
||||
.c-205 {fill: #ff5faf}
|
||||
.c-206 {fill: #ff5fd7}
|
||||
.c-207 {fill: #ff5fff}
|
||||
.c-208 {fill: #ff8700}
|
||||
.c-209 {fill: #ff875f}
|
||||
.c-210 {fill: #ff8787}
|
||||
.c-211 {fill: #ff87af}
|
||||
.c-212 {fill: #ff87d7}
|
||||
.c-213 {fill: #ff87ff}
|
||||
.c-214 {fill: #ffaf00}
|
||||
.c-215 {fill: #ffaf5f}
|
||||
.c-216 {fill: #ffaf87}
|
||||
.c-217 {fill: #ffafaf}
|
||||
.c-218 {fill: #ffafd7}
|
||||
.c-219 {fill: #ffafff}
|
||||
.c-220 {fill: #ffd700}
|
||||
.c-221 {fill: #ffd75f}
|
||||
.c-222 {fill: #ffd787}
|
||||
.c-223 {fill: #ffd7af}
|
||||
.c-224 {fill: #ffd7d7}
|
||||
.c-225 {fill: #ffd7ff}
|
||||
.c-226 {fill: #ffff00}
|
||||
.c-227 {fill: #ffff5f}
|
||||
.c-228 {fill: #ffff87}
|
||||
.c-229 {fill: #ffffaf}
|
||||
.c-230 {fill: #ffffd7}
|
||||
.c-231 {fill: #ffffff}
|
||||
.c-232 {fill: #080808}
|
||||
.c-233 {fill: #121212}
|
||||
.c-234 {fill: #1c1c1c}
|
||||
.c-235 {fill: #262626}
|
||||
.c-236 {fill: #303030}
|
||||
.c-237 {fill: #3a3a3a}
|
||||
.c-238 {fill: #444444}
|
||||
.c-239 {fill: #4e4e4e}
|
||||
.c-240 {fill: #585858}
|
||||
.c-241 {fill: #626262}
|
||||
.c-242 {fill: #6c6c6c}
|
||||
.c-243 {fill: #767676}
|
||||
.c-244 {fill: #808080}
|
||||
.c-245 {fill: #8a8a8a}
|
||||
.c-246 {fill: #949494}
|
||||
.c-247 {fill: #9e9e9e}
|
||||
.c-248 {fill: #a8a8a8}
|
||||
.c-249 {fill: #b2b2b2}
|
||||
.c-250 {fill: #bcbcbc}
|
||||
.c-251 {fill: #c6c6c6}
|
||||
.c-252 {fill: #d0d0d0}
|
||||
.c-253 {fill: #dadada}
|
||||
.c-254 {fill: #e4e4e4}
|
||||
.c-255 {fill: #eeeeee}
|
||||
|
||||
.br { font-weight: bold }
|
||||
.it { font-style: italic }
|
||||
.un { text-decoration: underline }
|
||||
</style>
|
||||
|
||||
<rect width="100%" height="100%" class="default-bg-fill" />
|
||||
|
||||
<svg x="0.625%" y="1.136%" class="default-text-fill">
|
||||
<g style="shape-rendering: optimizeSpeed">
|
||||
<rect x="5.625%" y="6.818%" width="0.625%" height="19.7" class="c-7" />
|
||||
<rect x="0.000%" y="95.455%" width="98.750%" height="19.7" class="c-2" />
|
||||
</g>
|
||||
|
||||
<text class="default-text-fill">
|
||||
<tspan y="0.000%">
|
||||
<tspan dy="1em" x="0.000%">c</tspan><tspan x="0.625%">h</tspan><tspan x="1.250%">a</tspan><tspan x="1.875%">o</tspan><tspan x="2.500%">s</tspan><tspan x="3.125%">:</tspan><tspan x="3.750%">~</tspan><tspan x="4.375%">$</tspan><tspan x="5.625%">#</tspan><tspan x="6.875%">B</tspan><tspan x="7.500%">u</tspan><tspan x="8.125%">i</tspan><tspan x="8.750%">l</tspan><tspan x="9.375%">d</tspan><tspan x="10.625%">a</tspan><tspan x="11.250%">n</tspan><tspan x="11.875%">d</tspan><tspan x="13.125%">a</tspan><tspan x="13.750%">p</tspan><tspan x="14.375%">p</tspan><tspan x="15.000%">l</tspan><tspan x="15.625%">y</tspan><tspan x="16.875%">C</tspan><tspan x="17.500%">h</tspan><tspan x="18.125%">a</tspan><tspan x="18.750%">o</tspan><tspan x="19.375%">s</tspan><tspan x="20.000%">E</tspan><tspan x="20.625%">n</tspan><tspan x="21.250%">g</tspan><tspan x="21.875%">i</tspan><tspan x="22.500%">n</tspan><tspan x="23.125%">e</tspan><tspan x="24.375%">C</tspan><tspan x="25.000%">R</tspan><tspan x="26.250%">t</tspan><tspan x="26.875%">o</tspan><tspan x="28.125%">u</tspan><tspan x="28.750%">n</tspan><tspan x="29.375%">l</tspan><tspan x="30.000%">e</tspan><tspan x="30.625%">a</tspan><tspan x="31.250%">s</tspan><tspan x="31.875%">h</tspan><tspan x="33.125%">C</tspan><tspan x="33.750%">h</tspan><tspan x="34.375%">a</tspan><tspan x="35.000%">o</tspan><tspan x="35.625%">s</tspan><tspan x="49.375%" class="c-2">│</tspan><tspan x="50.000%">E</tspan><tspan x="50.625%">v</tspan><tspan x="51.250%">e</tspan><tspan x="51.875%">r</tspan><tspan x="52.500%">y</tspan><tspan x="53.750%">1</tspan><tspan x="54.375%">.</tspan><tspan x="55.000%">0</tspan><tspan x="55.625%">s</tspan><tspan x="56.250%">:</tspan><tspan x="57.500%">k</tspan><tspan x="58.125%">u</tspan><tspan x="58.750%">b</tspan><tspan x="59.375%">e</tspan><tspan x="60.000%">c</tspan><tspan x="60.625%">t</tspan><tspan x="61.250%">l</tspan><tspan x="62.500%">g</tspan><tspan x="63.125%">e</tspan><tspan x="63.750%">t</tspan><tspan x="65.000%">p</tspan><tspan x="65.625%">o</tspan><tspan x="83.750%">F</tspan><tspan x="84.375%">r</tspan><tspan x="85.000%">i</tspan><tspan x="86.250%">O</tspan><tspan x="86.875%">c</tspan><tspan x="87.500%">t</tspan><tspan x="89.375%">4</tspan><tspan x="90.625%">1</tspan><tspan x="91.250%">9</tspan><tspan x="91.875%">:</tspan><tspan x="92.500%">3</tspan><tspan x="93.125%">2</tspan><tspan x="93.750%">:</tspan><tspan x="94.375%">3</tspan><tspan x="95.000%">5</tspan><tspan x="96.250%">2</tspan><tspan x="96.875%">0</tspan><tspan x="97.500%">1</tspan><tspan x="98.125%">9</tspan>
|
||||
</tspan>
|
||||
<tspan y="2.273%">
|
||||
<tspan dy="1em" x="0.000%">c</tspan><tspan x="0.625%">h</tspan><tspan x="1.250%">a</tspan><tspan x="1.875%">o</tspan><tspan x="2.500%">s</tspan><tspan x="3.125%">:</tspan><tspan x="3.750%">~</tspan><tspan x="4.375%">$</tspan><tspan x="49.375%" class="c-2">│</tspan>
|
||||
</tspan>
|
||||
<tspan y="4.545%">
|
||||
<tspan dy="1em" x="0.000%">c</tspan><tspan x="0.625%">h</tspan><tspan x="1.250%">a</tspan><tspan x="1.875%">o</tspan><tspan x="2.500%">s</tspan><tspan x="3.125%">:</tspan><tspan x="3.750%">~</tspan><tspan x="4.375%">$</tspan><tspan x="5.625%">v</tspan><tspan x="6.250%">i</tspan><tspan x="7.500%">c</tspan><tspan x="8.125%">h</tspan><tspan x="8.750%">a</tspan><tspan x="9.375%">o</tspan><tspan x="10.000%">s</tspan><tspan x="10.625%">e</tspan><tspan x="11.250%">n</tspan><tspan x="11.875%">g</tspan><tspan x="12.500%">i</tspan><tspan x="13.125%">n</tspan><tspan x="13.750%">e</tspan><tspan x="14.375%">.</tspan><tspan x="15.000%">y</tspan><tspan x="15.625%">a</tspan><tspan x="16.250%">m</tspan><tspan x="16.875%">l</tspan><tspan x="49.375%" class="c-2">│</tspan><tspan x="50.000%">N</tspan><tspan x="50.625%">A</tspan><tspan x="51.250%">M</tspan><tspan x="51.875%">E</tspan><tspan x="69.375%">R</tspan><tspan x="70.000%">E</tspan><tspan x="70.625%">A</tspan><tspan x="71.250%">D</tspan><tspan x="71.875%">Y</tspan><tspan x="74.375%">S</tspan><tspan x="75.000%">T</tspan><tspan x="75.625%">A</tspan><tspan x="76.250%">T</tspan><tspan x="76.875%">U</tspan><tspan x="77.500%">S</tspan><tspan x="80.625%">R</tspan><tspan x="81.250%">E</tspan><tspan x="81.875%">S</tspan><tspan x="82.500%">T</tspan><tspan x="83.125%">A</tspan><tspan x="83.750%">R</tspan><tspan x="84.375%">T</tspan><tspan x="85.000%">S</tspan><tspan x="87.500%">A</tspan><tspan x="88.125%">G</tspan><tspan x="88.750%">E</tspan>
|
||||
</tspan>
|
||||
<tspan y="6.818%">
|
||||
<tspan dy="1em" x="0.000%">c</tspan><tspan x="0.625%">h</tspan><tspan x="1.250%">a</tspan><tspan x="1.875%">o</tspan><tspan x="2.500%">s</tspan><tspan x="3.125%">:</tspan><tspan x="3.750%">~</tspan><tspan x="4.375%">$</tspan><tspan x="49.375%" class="c-2">│</tspan><tspan x="50.000%">h</tspan><tspan x="50.625%">e</tspan><tspan x="51.250%">l</tspan><tspan x="51.875%">l</tspan><tspan x="52.500%">o</tspan><tspan x="53.125%">-</tspan><tspan x="53.750%">d</tspan><tspan x="54.375%">e</tspan><tspan x="55.000%">p</tspan><tspan x="55.625%">l</tspan><tspan x="56.250%">o</tspan><tspan x="56.875%">y</tspan><tspan x="57.500%">-</tspan><tspan x="58.125%">d</tspan><tspan x="58.750%">d</tspan><tspan x="59.375%">5</tspan><tspan x="60.000%">9</tspan><tspan x="60.625%">b</tspan><tspan x="61.250%">8</tspan><tspan x="61.875%">9</tspan><tspan x="62.500%">5</tspan><tspan x="63.125%">6</tspan><tspan x="63.750%">-</tspan><tspan x="64.375%">h</tspan><tspan x="65.000%">x</tspan><tspan x="65.625%">c</tspan><tspan x="66.250%">j</tspan><tspan x="66.875%">v</tspan><tspan x="69.375%">1</tspan><tspan x="70.000%">/</tspan><tspan x="70.625%">1</tspan><tspan x="74.375%">R</tspan><tspan x="75.000%">u</tspan><tspan x="75.625%">n</tspan><tspan x="76.250%">n</tspan><tspan x="76.875%">i</tspan><tspan x="77.500%">n</tspan><tspan x="78.125%">g</tspan><tspan x="80.625%">0</tspan><tspan x="87.500%">1</tspan><tspan x="88.125%">9</tspan><tspan x="88.750%">m</tspan>
|
||||
</tspan>
|
||||
<tspan y="9.091%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan>
|
||||
</tspan>
|
||||
<tspan y="11.364%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan>
|
||||
</tspan>
|
||||
<tspan y="13.636%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan>
|
||||
</tspan>
|
||||
<tspan y="15.909%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan>
|
||||
</tspan>
|
||||
<tspan y="18.182%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan>
|
||||
</tspan>
|
||||
<tspan y="20.455%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan>
|
||||
</tspan>
|
||||
<tspan y="22.727%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan>
|
||||
</tspan>
|
||||
<tspan y="25.000%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan>
|
||||
</tspan>
|
||||
<tspan y="27.273%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan>
|
||||
</tspan>
|
||||
<tspan y="29.545%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan>
|
||||
</tspan>
|
||||
<tspan y="31.818%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan>
|
||||
</tspan>
|
||||
<tspan y="34.091%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan>
|
||||
</tspan>
|
||||
<tspan y="36.364%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan>
|
||||
</tspan>
|
||||
<tspan y="38.636%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan>
|
||||
</tspan>
|
||||
<tspan y="40.909%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan>
|
||||
</tspan>
|
||||
<tspan y="43.182%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan>
|
||||
</tspan>
|
||||
<tspan y="45.455%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan>
|
||||
</tspan>
|
||||
<tspan y="47.727%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">├</tspan><tspan x="50.000%">─</tspan><tspan x="50.625%">─</tspan><tspan x="51.250%">─</tspan><tspan x="51.875%">─</tspan><tspan x="52.500%">─</tspan><tspan x="53.125%">─</tspan><tspan x="53.750%">─</tspan><tspan x="54.375%">─</tspan><tspan x="55.000%">─</tspan><tspan x="55.625%">─</tspan><tspan x="56.250%">─</tspan><tspan x="56.875%">─</tspan><tspan x="57.500%">─</tspan><tspan x="58.125%">─</tspan><tspan x="58.750%">─</tspan><tspan x="59.375%">─</tspan><tspan x="60.000%">─</tspan><tspan x="60.625%">─</tspan><tspan x="61.250%">─</tspan><tspan x="61.875%">─</tspan><tspan x="62.500%">─</tspan><tspan x="63.125%">─</tspan><tspan x="63.750%">─</tspan><tspan x="64.375%">─</tspan><tspan x="65.000%">─</tspan><tspan x="65.625%">─</tspan><tspan x="66.250%">─</tspan><tspan x="66.875%">─</tspan><tspan x="67.500%">─</tspan><tspan x="68.125%">─</tspan><tspan x="68.750%">─</tspan><tspan x="69.375%">─</tspan><tspan x="70.000%">─</tspan><tspan x="70.625%">─</tspan><tspan x="71.250%">─</tspan><tspan x="71.875%">─</tspan><tspan x="72.500%">─</tspan><tspan x="73.125%">─</tspan><tspan x="73.750%">─</tspan><tspan x="74.375%">─</tspan><tspan x="75.000%">─</tspan><tspan x="75.625%">─</tspan><tspan x="76.250%">─</tspan><tspan x="76.875%">─</tspan><tspan x="77.500%">─</tspan><tspan x="78.125%">─</tspan><tspan x="78.750%">─</tspan><tspan x="79.375%">─</tspan><tspan x="80.000%">─</tspan><tspan x="80.625%">─</tspan><tspan x="81.250%">─</tspan><tspan x="81.875%">─</tspan><tspan x="82.500%">─</tspan><tspan x="83.125%">─</tspan><tspan x="83.750%">─</tspan><tspan x="84.375%">─</tspan><tspan x="85.000%">─</tspan><tspan x="85.625%">─</tspan><tspan x="86.250%">─</tspan><tspan x="86.875%">─</tspan><tspan x="87.500%">─</tspan><tspan x="88.125%">─</tspan><tspan x="88.750%">─</tspan><tspan x="89.375%">─</tspan><tspan x="90.000%">─</tspan><tspan x="90.625%">─</tspan><tspan x="91.250%">─</tspan><tspan x="91.875%">─</tspan><tspan x="92.500%">─</tspan><tspan x="93.125%">─</tspan><tspan x="93.750%">─</tspan><tspan x="94.375%">─</tspan><tspan x="95.000%">─</tspan><tspan x="95.625%">─</tspan><tspan x="96.250%">─</tspan><tspan x="96.875%">─</tspan><tspan x="97.500%">─</tspan><tspan x="98.125%">─</tspan>
|
||||
</tspan>
|
||||
<tspan y="50.000%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan><tspan x="50.000%" class="br c-10">H</tspan><tspan x="50.625%" class="br c-10">e</tspan><tspan x="51.250%" class="br c-10">l</tspan><tspan x="51.875%" class="br c-10">l</tspan><tspan x="52.500%" class="br c-10">o</tspan><tspan x="53.750%" class="br c-10">W</tspan><tspan x="54.375%" class="br c-10">o</tspan><tspan x="55.000%" class="br c-10">r</tspan><tspan x="55.625%" class="br c-10">l</tspan><tspan x="56.250%" class="br c-10">d</tspan><tspan x="57.500%" class="br c-10">i</tspan><tspan x="58.125%" class="br c-10">s</tspan><tspan x="59.375%" class="br c-10">o</tspan><tspan x="60.000%" class="br c-10">n</tspan><tspan x="60.625%" class="br c-10">l</tspan><tspan x="61.250%" class="br c-10">i</tspan><tspan x="61.875%" class="br c-10">n</tspan><tspan x="62.500%" class="br c-10">e</tspan><tspan x="63.750%">H</tspan><tspan x="64.375%">T</tspan><tspan x="65.000%">T</tspan><tspan x="65.625%">P</tspan><tspan x="66.250%">/</tspan><tspan x="66.875%">2</tspan><tspan x="68.125%">2</tspan><tspan x="68.750%">0</tspan><tspan x="69.375%">0</tspan><tspan x="70.625%">O</tspan><tspan x="71.250%">K</tspan>
|
||||
</tspan>
|
||||
<tspan y="52.273%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan><tspan x="50.000%" class="br c-10">H</tspan><tspan x="50.625%" class="br c-10">e</tspan><tspan x="51.250%" class="br c-10">l</tspan><tspan x="51.875%" class="br c-10">l</tspan><tspan x="52.500%" class="br c-10">o</tspan><tspan x="53.750%" class="br c-10">W</tspan><tspan x="54.375%" class="br c-10">o</tspan><tspan x="55.000%" class="br c-10">r</tspan><tspan x="55.625%" class="br c-10">l</tspan><tspan x="56.250%" class="br c-10">d</tspan><tspan x="57.500%" class="br c-10">i</tspan><tspan x="58.125%" class="br c-10">s</tspan><tspan x="59.375%" class="br c-10">o</tspan><tspan x="60.000%" class="br c-10">n</tspan><tspan x="60.625%" class="br c-10">l</tspan><tspan x="61.250%" class="br c-10">i</tspan><tspan x="61.875%" class="br c-10">n</tspan><tspan x="62.500%" class="br c-10">e</tspan><tspan x="63.750%">H</tspan><tspan x="64.375%">T</tspan><tspan x="65.000%">T</tspan><tspan x="65.625%">P</tspan><tspan x="66.250%">/</tspan><tspan x="66.875%">2</tspan><tspan x="68.125%">2</tspan><tspan x="68.750%">0</tspan><tspan x="69.375%">0</tspan><tspan x="70.625%">O</tspan><tspan x="71.250%">K</tspan>
|
||||
</tspan>
|
||||
<tspan y="54.545%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan><tspan x="50.000%" class="br c-10">H</tspan><tspan x="50.625%" class="br c-10">e</tspan><tspan x="51.250%" class="br c-10">l</tspan><tspan x="51.875%" class="br c-10">l</tspan><tspan x="52.500%" class="br c-10">o</tspan><tspan x="53.750%" class="br c-10">W</tspan><tspan x="54.375%" class="br c-10">o</tspan><tspan x="55.000%" class="br c-10">r</tspan><tspan x="55.625%" class="br c-10">l</tspan><tspan x="56.250%" class="br c-10">d</tspan><tspan x="57.500%" class="br c-10">i</tspan><tspan x="58.125%" class="br c-10">s</tspan><tspan x="59.375%" class="br c-10">o</tspan><tspan x="60.000%" class="br c-10">n</tspan><tspan x="60.625%" class="br c-10">l</tspan><tspan x="61.250%" class="br c-10">i</tspan><tspan x="61.875%" class="br c-10">n</tspan><tspan x="62.500%" class="br c-10">e</tspan><tspan x="63.750%">H</tspan><tspan x="64.375%">T</tspan><tspan x="65.000%">T</tspan><tspan x="65.625%">P</tspan><tspan x="66.250%">/</tspan><tspan x="66.875%">2</tspan><tspan x="68.125%">2</tspan><tspan x="68.750%">0</tspan><tspan x="69.375%">0</tspan><tspan x="70.625%">O</tspan><tspan x="71.250%">K</tspan>
|
||||
</tspan>
|
||||
<tspan y="56.818%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan><tspan x="50.000%" class="br c-10">H</tspan><tspan x="50.625%" class="br c-10">e</tspan><tspan x="51.250%" class="br c-10">l</tspan><tspan x="51.875%" class="br c-10">l</tspan><tspan x="52.500%" class="br c-10">o</tspan><tspan x="53.750%" class="br c-10">W</tspan><tspan x="54.375%" class="br c-10">o</tspan><tspan x="55.000%" class="br c-10">r</tspan><tspan x="55.625%" class="br c-10">l</tspan><tspan x="56.250%" class="br c-10">d</tspan><tspan x="57.500%" class="br c-10">i</tspan><tspan x="58.125%" class="br c-10">s</tspan><tspan x="59.375%" class="br c-10">o</tspan><tspan x="60.000%" class="br c-10">n</tspan><tspan x="60.625%" class="br c-10">l</tspan><tspan x="61.250%" class="br c-10">i</tspan><tspan x="61.875%" class="br c-10">n</tspan><tspan x="62.500%" class="br c-10">e</tspan><tspan x="63.750%">H</tspan><tspan x="64.375%">T</tspan><tspan x="65.000%">T</tspan><tspan x="65.625%">P</tspan><tspan x="66.250%">/</tspan><tspan x="66.875%">2</tspan><tspan x="68.125%">2</tspan><tspan x="68.750%">0</tspan><tspan x="69.375%">0</tspan><tspan x="70.625%">O</tspan><tspan x="71.250%">K</tspan>
|
||||
</tspan>
|
||||
<tspan y="59.091%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan><tspan x="50.000%" class="br c-10">H</tspan><tspan x="50.625%" class="br c-10">e</tspan><tspan x="51.250%" class="br c-10">l</tspan><tspan x="51.875%" class="br c-10">l</tspan><tspan x="52.500%" class="br c-10">o</tspan><tspan x="53.750%" class="br c-10">W</tspan><tspan x="54.375%" class="br c-10">o</tspan><tspan x="55.000%" class="br c-10">r</tspan><tspan x="55.625%" class="br c-10">l</tspan><tspan x="56.250%" class="br c-10">d</tspan><tspan x="57.500%" class="br c-10">i</tspan><tspan x="58.125%" class="br c-10">s</tspan><tspan x="59.375%" class="br c-10">o</tspan><tspan x="60.000%" class="br c-10">n</tspan><tspan x="60.625%" class="br c-10">l</tspan><tspan x="61.250%" class="br c-10">i</tspan><tspan x="61.875%" class="br c-10">n</tspan><tspan x="62.500%" class="br c-10">e</tspan><tspan x="63.750%">H</tspan><tspan x="64.375%">T</tspan><tspan x="65.000%">T</tspan><tspan x="65.625%">P</tspan><tspan x="66.250%">/</tspan><tspan x="66.875%">2</tspan><tspan x="68.125%">2</tspan><tspan x="68.750%">0</tspan><tspan x="69.375%">0</tspan><tspan x="70.625%">O</tspan><tspan x="71.250%">K</tspan>
|
||||
</tspan>
|
||||
<tspan y="61.364%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan><tspan x="50.000%" class="br c-10">H</tspan><tspan x="50.625%" class="br c-10">e</tspan><tspan x="51.250%" class="br c-10">l</tspan><tspan x="51.875%" class="br c-10">l</tspan><tspan x="52.500%" class="br c-10">o</tspan><tspan x="53.750%" class="br c-10">W</tspan><tspan x="54.375%" class="br c-10">o</tspan><tspan x="55.000%" class="br c-10">r</tspan><tspan x="55.625%" class="br c-10">l</tspan><tspan x="56.250%" class="br c-10">d</tspan><tspan x="57.500%" class="br c-10">i</tspan><tspan x="58.125%" class="br c-10">s</tspan><tspan x="59.375%" class="br c-10">o</tspan><tspan x="60.000%" class="br c-10">n</tspan><tspan x="60.625%" class="br c-10">l</tspan><tspan x="61.250%" class="br c-10">i</tspan><tspan x="61.875%" class="br c-10">n</tspan><tspan x="62.500%" class="br c-10">e</tspan><tspan x="63.750%">H</tspan><tspan x="64.375%">T</tspan><tspan x="65.000%">T</tspan><tspan x="65.625%">P</tspan><tspan x="66.250%">/</tspan><tspan x="66.875%">2</tspan><tspan x="68.125%">2</tspan><tspan x="68.750%">0</tspan><tspan x="69.375%">0</tspan><tspan x="70.625%">O</tspan><tspan x="71.250%">K</tspan>
|
||||
</tspan>
|
||||
<tspan y="63.636%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan><tspan x="50.000%" class="br c-10">H</tspan><tspan x="50.625%" class="br c-10">e</tspan><tspan x="51.250%" class="br c-10">l</tspan><tspan x="51.875%" class="br c-10">l</tspan><tspan x="52.500%" class="br c-10">o</tspan><tspan x="53.750%" class="br c-10">W</tspan><tspan x="54.375%" class="br c-10">o</tspan><tspan x="55.000%" class="br c-10">r</tspan><tspan x="55.625%" class="br c-10">l</tspan><tspan x="56.250%" class="br c-10">d</tspan><tspan x="57.500%" class="br c-10">i</tspan><tspan x="58.125%" class="br c-10">s</tspan><tspan x="59.375%" class="br c-10">o</tspan><tspan x="60.000%" class="br c-10">n</tspan><tspan x="60.625%" class="br c-10">l</tspan><tspan x="61.250%" class="br c-10">i</tspan><tspan x="61.875%" class="br c-10">n</tspan><tspan x="62.500%" class="br c-10">e</tspan><tspan x="63.750%">H</tspan><tspan x="64.375%">T</tspan><tspan x="65.000%">T</tspan><tspan x="65.625%">P</tspan><tspan x="66.250%">/</tspan><tspan x="66.875%">2</tspan><tspan x="68.125%">2</tspan><tspan x="68.750%">0</tspan><tspan x="69.375%">0</tspan><tspan x="70.625%">O</tspan><tspan x="71.250%">K</tspan>
|
||||
</tspan>
|
||||
<tspan y="65.909%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan><tspan x="50.000%" class="br c-10">H</tspan><tspan x="50.625%" class="br c-10">e</tspan><tspan x="51.250%" class="br c-10">l</tspan><tspan x="51.875%" class="br c-10">l</tspan><tspan x="52.500%" class="br c-10">o</tspan><tspan x="53.750%" class="br c-10">W</tspan><tspan x="54.375%" class="br c-10">o</tspan><tspan x="55.000%" class="br c-10">r</tspan><tspan x="55.625%" class="br c-10">l</tspan><tspan x="56.250%" class="br c-10">d</tspan><tspan x="57.500%" class="br c-10">i</tspan><tspan x="58.125%" class="br c-10">s</tspan><tspan x="59.375%" class="br c-10">o</tspan><tspan x="60.000%" class="br c-10">n</tspan><tspan x="60.625%" class="br c-10">l</tspan><tspan x="61.250%" class="br c-10">i</tspan><tspan x="61.875%" class="br c-10">n</tspan><tspan x="62.500%" class="br c-10">e</tspan><tspan x="63.750%">H</tspan><tspan x="64.375%">T</tspan><tspan x="65.000%">T</tspan><tspan x="65.625%">P</tspan><tspan x="66.250%">/</tspan><tspan x="66.875%">2</tspan><tspan x="68.125%">2</tspan><tspan x="68.750%">0</tspan><tspan x="69.375%">0</tspan><tspan x="70.625%">O</tspan><tspan x="71.250%">K</tspan>
|
||||
</tspan>
|
||||
<tspan y="68.182%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan><tspan x="50.000%" class="br c-10">H</tspan><tspan x="50.625%" class="br c-10">e</tspan><tspan x="51.250%" class="br c-10">l</tspan><tspan x="51.875%" class="br c-10">l</tspan><tspan x="52.500%" class="br c-10">o</tspan><tspan x="53.750%" class="br c-10">W</tspan><tspan x="54.375%" class="br c-10">o</tspan><tspan x="55.000%" class="br c-10">r</tspan><tspan x="55.625%" class="br c-10">l</tspan><tspan x="56.250%" class="br c-10">d</tspan><tspan x="57.500%" class="br c-10">i</tspan><tspan x="58.125%" class="br c-10">s</tspan><tspan x="59.375%" class="br c-10">o</tspan><tspan x="60.000%" class="br c-10">n</tspan><tspan x="60.625%" class="br c-10">l</tspan><tspan x="61.250%" class="br c-10">i</tspan><tspan x="61.875%" class="br c-10">n</tspan><tspan x="62.500%" class="br c-10">e</tspan><tspan x="63.750%">H</tspan><tspan x="64.375%">T</tspan><tspan x="65.000%">T</tspan><tspan x="65.625%">P</tspan><tspan x="66.250%">/</tspan><tspan x="66.875%">2</tspan><tspan x="68.125%">2</tspan><tspan x="68.750%">0</tspan><tspan x="69.375%">0</tspan><tspan x="70.625%">O</tspan><tspan x="71.250%">K</tspan>
|
||||
</tspan>
|
||||
<tspan y="70.455%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan><tspan x="50.000%" class="br c-10">H</tspan><tspan x="50.625%" class="br c-10">e</tspan><tspan x="51.250%" class="br c-10">l</tspan><tspan x="51.875%" class="br c-10">l</tspan><tspan x="52.500%" class="br c-10">o</tspan><tspan x="53.750%" class="br c-10">W</tspan><tspan x="54.375%" class="br c-10">o</tspan><tspan x="55.000%" class="br c-10">r</tspan><tspan x="55.625%" class="br c-10">l</tspan><tspan x="56.250%" class="br c-10">d</tspan><tspan x="57.500%" class="br c-10">i</tspan><tspan x="58.125%" class="br c-10">s</tspan><tspan x="59.375%" class="br c-10">o</tspan><tspan x="60.000%" class="br c-10">n</tspan><tspan x="60.625%" class="br c-10">l</tspan><tspan x="61.250%" class="br c-10">i</tspan><tspan x="61.875%" class="br c-10">n</tspan><tspan x="62.500%" class="br c-10">e</tspan><tspan x="63.750%">H</tspan><tspan x="64.375%">T</tspan><tspan x="65.000%">T</tspan><tspan x="65.625%">P</tspan><tspan x="66.250%">/</tspan><tspan x="66.875%">2</tspan><tspan x="68.125%">2</tspan><tspan x="68.750%">0</tspan><tspan x="69.375%">0</tspan><tspan x="70.625%">O</tspan><tspan x="71.250%">K</tspan>
|
||||
</tspan>
|
||||
<tspan y="72.727%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan><tspan x="50.000%" class="br c-10">H</tspan><tspan x="50.625%" class="br c-10">e</tspan><tspan x="51.250%" class="br c-10">l</tspan><tspan x="51.875%" class="br c-10">l</tspan><tspan x="52.500%" class="br c-10">o</tspan><tspan x="53.750%" class="br c-10">W</tspan><tspan x="54.375%" class="br c-10">o</tspan><tspan x="55.000%" class="br c-10">r</tspan><tspan x="55.625%" class="br c-10">l</tspan><tspan x="56.250%" class="br c-10">d</tspan><tspan x="57.500%" class="br c-10">i</tspan><tspan x="58.125%" class="br c-10">s</tspan><tspan x="59.375%" class="br c-10">o</tspan><tspan x="60.000%" class="br c-10">n</tspan><tspan x="60.625%" class="br c-10">l</tspan><tspan x="61.250%" class="br c-10">i</tspan><tspan x="61.875%" class="br c-10">n</tspan><tspan x="62.500%" class="br c-10">e</tspan><tspan x="63.750%">H</tspan><tspan x="64.375%">T</tspan><tspan x="65.000%">T</tspan><tspan x="65.625%">P</tspan><tspan x="66.250%">/</tspan><tspan x="66.875%">2</tspan><tspan x="68.125%">2</tspan><tspan x="68.750%">0</tspan><tspan x="69.375%">0</tspan><tspan x="70.625%">O</tspan><tspan x="71.250%">K</tspan>
|
||||
</tspan>
|
||||
<tspan y="75.000%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan><tspan x="50.000%" class="br c-10">H</tspan><tspan x="50.625%" class="br c-10">e</tspan><tspan x="51.250%" class="br c-10">l</tspan><tspan x="51.875%" class="br c-10">l</tspan><tspan x="52.500%" class="br c-10">o</tspan><tspan x="53.750%" class="br c-10">W</tspan><tspan x="54.375%" class="br c-10">o</tspan><tspan x="55.000%" class="br c-10">r</tspan><tspan x="55.625%" class="br c-10">l</tspan><tspan x="56.250%" class="br c-10">d</tspan><tspan x="57.500%" class="br c-10">i</tspan><tspan x="58.125%" class="br c-10">s</tspan><tspan x="59.375%" class="br c-10">o</tspan><tspan x="60.000%" class="br c-10">n</tspan><tspan x="60.625%" class="br c-10">l</tspan><tspan x="61.250%" class="br c-10">i</tspan><tspan x="61.875%" class="br c-10">n</tspan><tspan x="62.500%" class="br c-10">e</tspan><tspan x="63.750%">H</tspan><tspan x="64.375%">T</tspan><tspan x="65.000%">T</tspan><tspan x="65.625%">P</tspan><tspan x="66.250%">/</tspan><tspan x="66.875%">2</tspan><tspan x="68.125%">2</tspan><tspan x="68.750%">0</tspan><tspan x="69.375%">0</tspan><tspan x="70.625%">O</tspan><tspan x="71.250%">K</tspan>
|
||||
</tspan>
|
||||
<tspan y="77.273%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan><tspan x="50.000%" class="br c-10">H</tspan><tspan x="50.625%" class="br c-10">e</tspan><tspan x="51.250%" class="br c-10">l</tspan><tspan x="51.875%" class="br c-10">l</tspan><tspan x="52.500%" class="br c-10">o</tspan><tspan x="53.750%" class="br c-10">W</tspan><tspan x="54.375%" class="br c-10">o</tspan><tspan x="55.000%" class="br c-10">r</tspan><tspan x="55.625%" class="br c-10">l</tspan><tspan x="56.250%" class="br c-10">d</tspan><tspan x="57.500%" class="br c-10">i</tspan><tspan x="58.125%" class="br c-10">s</tspan><tspan x="59.375%" class="br c-10">o</tspan><tspan x="60.000%" class="br c-10">n</tspan><tspan x="60.625%" class="br c-10">l</tspan><tspan x="61.250%" class="br c-10">i</tspan><tspan x="61.875%" class="br c-10">n</tspan><tspan x="62.500%" class="br c-10">e</tspan><tspan x="63.750%">H</tspan><tspan x="64.375%">T</tspan><tspan x="65.000%">T</tspan><tspan x="65.625%">P</tspan><tspan x="66.250%">/</tspan><tspan x="66.875%">2</tspan><tspan x="68.125%">2</tspan><tspan x="68.750%">0</tspan><tspan x="69.375%">0</tspan><tspan x="70.625%">O</tspan><tspan x="71.250%">K</tspan>
|
||||
</tspan>
|
||||
<tspan y="79.545%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan><tspan x="50.000%" class="br c-10">H</tspan><tspan x="50.625%" class="br c-10">e</tspan><tspan x="51.250%" class="br c-10">l</tspan><tspan x="51.875%" class="br c-10">l</tspan><tspan x="52.500%" class="br c-10">o</tspan><tspan x="53.750%" class="br c-10">W</tspan><tspan x="54.375%" class="br c-10">o</tspan><tspan x="55.000%" class="br c-10">r</tspan><tspan x="55.625%" class="br c-10">l</tspan><tspan x="56.250%" class="br c-10">d</tspan><tspan x="57.500%" class="br c-10">i</tspan><tspan x="58.125%" class="br c-10">s</tspan><tspan x="59.375%" class="br c-10">o</tspan><tspan x="60.000%" class="br c-10">n</tspan><tspan x="60.625%" class="br c-10">l</tspan><tspan x="61.250%" class="br c-10">i</tspan><tspan x="61.875%" class="br c-10">n</tspan><tspan x="62.500%" class="br c-10">e</tspan><tspan x="63.750%">H</tspan><tspan x="64.375%">T</tspan><tspan x="65.000%">T</tspan><tspan x="65.625%">P</tspan><tspan x="66.250%">/</tspan><tspan x="66.875%">2</tspan><tspan x="68.125%">2</tspan><tspan x="68.750%">0</tspan><tspan x="69.375%">0</tspan><tspan x="70.625%">O</tspan><tspan x="71.250%">K</tspan>
|
||||
</tspan>
|
||||
<tspan y="81.818%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan><tspan x="50.000%" class="br c-10">H</tspan><tspan x="50.625%" class="br c-10">e</tspan><tspan x="51.250%" class="br c-10">l</tspan><tspan x="51.875%" class="br c-10">l</tspan><tspan x="52.500%" class="br c-10">o</tspan><tspan x="53.750%" class="br c-10">W</tspan><tspan x="54.375%" class="br c-10">o</tspan><tspan x="55.000%" class="br c-10">r</tspan><tspan x="55.625%" class="br c-10">l</tspan><tspan x="56.250%" class="br c-10">d</tspan><tspan x="57.500%" class="br c-10">i</tspan><tspan x="58.125%" class="br c-10">s</tspan><tspan x="59.375%" class="br c-10">o</tspan><tspan x="60.000%" class="br c-10">n</tspan><tspan x="60.625%" class="br c-10">l</tspan><tspan x="61.250%" class="br c-10">i</tspan><tspan x="61.875%" class="br c-10">n</tspan><tspan x="62.500%" class="br c-10">e</tspan><tspan x="63.750%">H</tspan><tspan x="64.375%">T</tspan><tspan x="65.000%">T</tspan><tspan x="65.625%">P</tspan><tspan x="66.250%">/</tspan><tspan x="66.875%">2</tspan><tspan x="68.125%">2</tspan><tspan x="68.750%">0</tspan><tspan x="69.375%">0</tspan><tspan x="70.625%">O</tspan><tspan x="71.250%">K</tspan>
|
||||
</tspan>
|
||||
<tspan y="84.091%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan><tspan x="50.000%" class="br c-10">H</tspan><tspan x="50.625%" class="br c-10">e</tspan><tspan x="51.250%" class="br c-10">l</tspan><tspan x="51.875%" class="br c-10">l</tspan><tspan x="52.500%" class="br c-10">o</tspan><tspan x="53.750%" class="br c-10">W</tspan><tspan x="54.375%" class="br c-10">o</tspan><tspan x="55.000%" class="br c-10">r</tspan><tspan x="55.625%" class="br c-10">l</tspan><tspan x="56.250%" class="br c-10">d</tspan><tspan x="57.500%" class="br c-10">i</tspan><tspan x="58.125%" class="br c-10">s</tspan><tspan x="59.375%" class="br c-10">o</tspan><tspan x="60.000%" class="br c-10">n</tspan><tspan x="60.625%" class="br c-10">l</tspan><tspan x="61.250%" class="br c-10">i</tspan><tspan x="61.875%" class="br c-10">n</tspan><tspan x="62.500%" class="br c-10">e</tspan><tspan x="63.750%">H</tspan><tspan x="64.375%">T</tspan><tspan x="65.000%">T</tspan><tspan x="65.625%">P</tspan><tspan x="66.250%">/</tspan><tspan x="66.875%">2</tspan><tspan x="68.125%">2</tspan><tspan x="68.750%">0</tspan><tspan x="69.375%">0</tspan><tspan x="70.625%">O</tspan><tspan x="71.250%">K</tspan>
|
||||
</tspan>
|
||||
<tspan y="86.364%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan><tspan x="50.000%" class="br c-10">H</tspan><tspan x="50.625%" class="br c-10">e</tspan><tspan x="51.250%" class="br c-10">l</tspan><tspan x="51.875%" class="br c-10">l</tspan><tspan x="52.500%" class="br c-10">o</tspan><tspan x="53.750%" class="br c-10">W</tspan><tspan x="54.375%" class="br c-10">o</tspan><tspan x="55.000%" class="br c-10">r</tspan><tspan x="55.625%" class="br c-10">l</tspan><tspan x="56.250%" class="br c-10">d</tspan><tspan x="57.500%" class="br c-10">i</tspan><tspan x="58.125%" class="br c-10">s</tspan><tspan x="59.375%" class="br c-10">o</tspan><tspan x="60.000%" class="br c-10">n</tspan><tspan x="60.625%" class="br c-10">l</tspan><tspan x="61.250%" class="br c-10">i</tspan><tspan x="61.875%" class="br c-10">n</tspan><tspan x="62.500%" class="br c-10">e</tspan><tspan x="63.750%">H</tspan><tspan x="64.375%">T</tspan><tspan x="65.000%">T</tspan><tspan x="65.625%">P</tspan><tspan x="66.250%">/</tspan><tspan x="66.875%">2</tspan><tspan x="68.125%">2</tspan><tspan x="68.750%">0</tspan><tspan x="69.375%">0</tspan><tspan x="70.625%">O</tspan><tspan x="71.250%">K</tspan>
|
||||
</tspan>
|
||||
<tspan y="88.636%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan><tspan x="50.000%" class="br c-10">H</tspan><tspan x="50.625%" class="br c-10">e</tspan><tspan x="51.250%" class="br c-10">l</tspan><tspan x="51.875%" class="br c-10">l</tspan><tspan x="52.500%" class="br c-10">o</tspan><tspan x="53.750%" class="br c-10">W</tspan><tspan x="54.375%" class="br c-10">o</tspan><tspan x="55.000%" class="br c-10">r</tspan><tspan x="55.625%" class="br c-10">l</tspan><tspan x="56.250%" class="br c-10">d</tspan><tspan x="57.500%" class="br c-10">i</tspan><tspan x="58.125%" class="br c-10">s</tspan><tspan x="59.375%" class="br c-10">o</tspan><tspan x="60.000%" class="br c-10">n</tspan><tspan x="60.625%" class="br c-10">l</tspan><tspan x="61.250%" class="br c-10">i</tspan><tspan x="61.875%" class="br c-10">n</tspan><tspan x="62.500%" class="br c-10">e</tspan><tspan x="63.750%">H</tspan><tspan x="64.375%">T</tspan><tspan x="65.000%">T</tspan><tspan x="65.625%">P</tspan><tspan x="66.250%">/</tspan><tspan x="66.875%">2</tspan><tspan x="68.125%">2</tspan><tspan x="68.750%">0</tspan><tspan x="69.375%">0</tspan><tspan x="70.625%">O</tspan><tspan x="71.250%">K</tspan>
|
||||
</tspan>
|
||||
<tspan y="90.909%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan><tspan x="50.000%" class="br c-10">H</tspan><tspan x="50.625%" class="br c-10">e</tspan><tspan x="51.250%" class="br c-10">l</tspan><tspan x="51.875%" class="br c-10">l</tspan><tspan x="52.500%" class="br c-10">o</tspan><tspan x="53.750%" class="br c-10">W</tspan><tspan x="54.375%" class="br c-10">o</tspan><tspan x="55.000%" class="br c-10">r</tspan><tspan x="55.625%" class="br c-10">l</tspan><tspan x="56.250%" class="br c-10">d</tspan><tspan x="57.500%" class="br c-10">i</tspan><tspan x="58.125%" class="br c-10">s</tspan><tspan x="59.375%" class="br c-10">o</tspan><tspan x="60.000%" class="br c-10">n</tspan><tspan x="60.625%" class="br c-10">l</tspan><tspan x="61.250%" class="br c-10">i</tspan><tspan x="61.875%" class="br c-10">n</tspan><tspan x="62.500%" class="br c-10">e</tspan><tspan x="63.750%">H</tspan><tspan x="64.375%">T</tspan><tspan x="65.000%">T</tspan><tspan x="65.625%">P</tspan><tspan x="66.250%">/</tspan><tspan x="66.875%">2</tspan><tspan x="68.125%">2</tspan><tspan x="68.750%">0</tspan><tspan x="69.375%">0</tspan><tspan x="70.625%">O</tspan><tspan x="71.250%">K</tspan>
|
||||
</tspan>
|
||||
<tspan y="93.182%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan>
|
||||
</tspan>
|
||||
<tspan y="95.455%">
|
||||
<tspan dy="1em" x="0.000%" class="c-0">[</tspan><tspan x="0.625%" class="c-0">d</tspan><tspan x="1.250%" class="c-0">e</tspan><tspan x="1.875%" class="c-0">m</tspan><tspan x="2.500%" class="c-0">o</tspan><tspan x="3.125%" class="c-0">]</tspan><tspan x="4.375%" class="c-0">0</tspan><tspan x="5.000%" class="c-0">:</tspan><tspan x="5.625%" class="c-0">s</tspan><tspan x="6.250%" class="c-0">s</tspan><tspan x="6.875%" class="c-0">h</tspan><tspan x="7.500%" class="c-0">*</tspan><tspan x="75.625%" class="c-0">"</tspan><tspan x="76.250%" class="c-0">r</tspan><tspan x="76.875%" class="c-0">a</tspan><tspan x="77.500%" class="c-0">h</tspan><tspan x="78.125%" class="c-0">u</tspan><tspan x="78.750%" class="c-0">l</tspan><tspan x="79.375%" class="c-0">-</tspan><tspan x="80.000%" class="c-0">T</tspan><tspan x="80.625%" class="c-0">h</tspan><tspan x="81.250%" class="c-0">i</tspan><tspan x="81.875%" class="c-0">n</tspan><tspan x="82.500%" class="c-0">k</tspan><tspan x="83.125%" class="c-0">P</tspan><tspan x="83.750%" class="c-0">a</tspan><tspan x="84.375%" class="c-0">d</tspan><tspan x="85.000%" class="c-0">-</tspan><tspan x="85.625%" class="c-0">E</tspan><tspan x="86.250%" class="c-0">4</tspan><tspan x="86.875%" class="c-0">9</tspan><tspan x="87.500%" class="c-0">0</tspan><tspan x="88.125%" class="c-0">"</tspan><tspan x="89.375%" class="c-0">0</tspan><tspan x="90.000%" class="c-0">1</tspan><tspan x="90.625%" class="c-0">:</tspan><tspan x="91.250%" class="c-0">0</tspan><tspan x="91.875%" class="c-0">2</tspan><tspan x="93.125%" class="c-0">0</tspan><tspan x="93.750%" class="c-0">5</tspan><tspan x="94.375%" class="c-0">-</tspan><tspan x="95.000%" class="c-0">O</tspan><tspan x="95.625%" class="c-0">c</tspan><tspan x="96.250%" class="c-0">t</tspan><tspan x="96.875%" class="c-0">-</tspan><tspan x="97.500%" class="c-0">1</tspan><tspan x="98.125%" class="c-0">9</tspan>
|
||||
</tspan>
|
||||
</text>
|
||||
|
||||
<g transform="translate(-50 -50)">
|
||||
<svg x="50%" y="50%" width="100" height="100">
|
||||
<svg version="1.1" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 866.0254037844387 866.0254037844387">
|
||||
<defs>
|
||||
<mask id="small-triangle-mask">
|
||||
<rect width="100%" height="100%" fill="white"/>
|
||||
<polygon points="508.01270189221935 433.01270189221935, 208.0127018922194 259.8076211353316, 208.01270189221927 606.217782649107" fill="black"></polygon>
|
||||
</mask>
|
||||
</defs>
|
||||
<polygon points="808.0127018922194 433.01270189221935, 58.01270189221947 -1.1368683772161603e-13, 58.01270189221913 866.0254037844386" mask="url(#small-triangle-mask)" fill="white"></polygon>
|
||||
<polyline points="481.2177826491071 333.0127018922194, 134.80762113533166 533.0127018922194" stroke="white" stroke-width="90"></polyline>
|
||||
</svg>
|
||||
</svg>
|
||||
</g>
|
||||
</svg>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 41 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 11 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 354 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 105 KiB |
|
|
@ -0,0 +1,71 @@
|
|||
---
|
||||
id: chaoshub
|
||||
title: Using and contributing to ChaosHub
|
||||
sidebar_label: ChaosHub
|
||||
original_id: chaoshub
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
**Important links**
|
||||
|
||||
Chaos Hub is maintained at https://hub.litmuschaos.io
|
||||
|
||||
To contribute new chaos charts visit: https://github.com/litmuschaos/chaos-charts
|
||||
|
||||
**Introduction**
|
||||
|
||||
Litmus chaos hub is a place where the chaos engineering community members publish their chaos experiments. A set of related chaos experiments are bundled into a `Chaos Chart`. Chaos Charts are classified into the following categories.
|
||||
|
||||
- [Generic Chaos](#generic-chaos)
|
||||
- [Application Chaos](#application-chaos)
|
||||
- [Platform Chaos](#platform-chaos)
|
||||
|
||||
### Generic Chaos
|
||||
|
||||
Chaos actions that apply to generic Kubernetes resources are classified into this category. Following chaos experiments are supported under Generic Chaos Chart
|
||||
|
||||
| Experiment name | Description | User guide link |
|
||||
| ---------------------- | ----------------------------------------------------- | --------------------------------------------------- |
|
||||
| Container Kill | Kill one container in the application pod | [container-kill](container-kill.md) |
|
||||
| Pod Delete | Fail the application pod | [pod-delete](pod-delete.md) |
|
||||
| Pod Network Latency | Experiment to inject network latency to the POD | [pod-network-latency](pod-network-latency.md) |
|
||||
| Pod Network Loss | Experiment to inject network loss to the POD | [pod-network-loss](pod-network-loss.md) |
|
||||
| CPU Hog | Exhaust CPU resources on the Kubernetes Node | [cpu-hog](cpu-hog.md) |
|
||||
| Disk Fill | Fillup Ephemeral Storage of a Resource | [disk-fill](disk-fill.md) |
|
||||
| Disk Loss | External disk loss from the node | [disk-loss](disk-loss.md) |
|
||||
| Node Drain | Drain the node where application pod is scheduled | [node-drain](node-drain.md) |
|
||||
| Pod CPU Hog | Consume CPU resources on the application container | [pod-cpu-hog](pod-cpu-hog.md) |
|
||||
| Pod Network Corruption | Inject Network Packet Corruption Into Application Pod | [pod-network-corruption](pod-network-corruption.md) |
|
||||
|
||||
### Application Chaos
|
||||
|
||||
While Chaos Experiments under the Generic category offer the ability to induce chaos into Kubernetes resources, it is difficult to analyze and conclude if the chaos induced found a weakness in a given application. The application specific chaos experiments are built with some checks on _pre-conditions_ and some expected outcomes after the chaos injection. The result of the chaos experiment is determined by matching the outcome with the expected outcome.
|
||||
|
||||
<div class="danger">
|
||||
<strong>NOTE:</strong> If the result of the chaos experiment is `pass`, it means that the application is resilient to that chaos.
|
||||
</div>
|
||||
|
||||
#### Benefits of contributing an application chaos experiment
|
||||
|
||||
Application developers write negative tests in their CI pipelines to test the resiliency of the applications. These negative can be converted into Litmus Chaos Experiments and contributed to ChaosHub, so that the users of the application can use them in staging/pre-production/production environments to check the resilience. Application environments vary considerably from where they are tested (CI pipelines) to where they are deployed (Production). Hence, running the same chaos tests in the user's environment will help determine the weaknesses of the deployment and fixing such weaknesses leads to increased resilience.
|
||||
|
||||
Following Application Chaos experiments are available on ChaosHub
|
||||
|
||||
| Application | Description | Chaos Experiments |
|
||||
| ----------- | ------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| OpenEBS | Container Attached Storage for Kubernetes | [openebs-pool-pod-failure](openebs-pool-pod-failure.md)<br/>[openebs-pool-container-failure](openebs-pool-container-failure.md)<br/>[openebs-target-pod-failure](openebs-target-pod-failure.md)<br/>[openebs-target-container-failure](openebs-target-container-failure.md)<br/>[openebs-target-network-delay](openebs-target-network-delay.md)<br/>[openebs-target-network-loss](openebs-target-network-loss.md) |
|
||||
| Kafka | Open-source stream processing software | [kafka-broker-pod-failure](kafka-broker-pod-failure.md)<br/>[kafka-broker-disk-failure](kafka-broker-disk-failure.md)<br/> |
|
||||
| CoreDns | CoreDNS is a fast and flexible DNS server that chains plugins | [coredns-pod-delete](coredns-pod-delete.md) |
|
||||
|
||||
### Platform Chaos
|
||||
|
||||
Chaos experiments that inject chaos into the platform resources of Kubernetes are classified into this category. Management of platform resources vary significantly from each other, Chaos Charts may be maintained separately for each platform (For example, AWS, GCP, Azure, etc)
|
||||
|
||||
Following Platform Chaos experiments are available on ChaosHub
|
||||
|
||||
| Platform | Description | Chaos Experiments |
|
||||
| -------- | ------------------------------------------- | ----------------- |
|
||||
| AWS | Amazon Web Services platform. Includes EKS. | None |
|
||||
| GCP | Google Cloud Platform. Includes GKE. | None |
|
||||
| Azure | Microsoft Azure platform. Includes AKS. | None |
|
||||
|
|
@ -0,0 +1,24 @@
|
|||
---
|
||||
id: community
|
||||
title: Join Litmus Community
|
||||
sidebar_label: Community
|
||||
original_id: community
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
Litmus community is a subset of the larger Kubernetes community. Have a question? Want to stay in touch with the happenings on Chaos Engineering on Kubernetes? Join `#litmus` channel on Kubernetes Slack.
|
||||
|
||||
<br/><br/>
|
||||
|
||||
<a href="https://kubernetes.slack.com/messages/CNXNB0ZTN" target="_blank"><img src={require("./assets/join-community.png").default} width="400"/></a>
|
||||
|
||||
<br/>
|
||||
|
||||
<br/>
|
||||
|
||||
<hr/>
|
||||
|
||||
<br/>
|
||||
|
||||
<br/>
|
||||
|
|
@ -1,5 +1,5 @@
|
|||
---
|
||||
id: version-1.0.0-container-kill
|
||||
id: container-kill
|
||||
title: Container Kill Experiment Details
|
||||
sidebar_label: Container Kill
|
||||
original_id: container-kill
|
||||
|
|
@ -1,5 +1,5 @@
|
|||
---
|
||||
id: version-1.0.0-coredns-pod-delete
|
||||
id: coredns-pod-delete
|
||||
title: CoreDNS Pod Delete Experiment Details
|
||||
sidebar_label: CoreDNS Pod Delete
|
||||
original_id: coredns-pod-delete
|
||||
|
|
@ -1,5 +1,5 @@
|
|||
---
|
||||
id: version-1.0.0-cpu-hog
|
||||
id: cpu-hog
|
||||
title: CPU Hog Experiment Details
|
||||
sidebar_label: CPU Hog
|
||||
original_id: cpu-hog
|
||||
|
|
@ -1,11 +1,11 @@
|
|||
---
|
||||
id: version-1.0.0-devguide
|
||||
id: devguide
|
||||
title: Developer Guide for Chaos Charts
|
||||
sidebar_label: Developer Guide
|
||||
original_id: devguide
|
||||
---
|
||||
|
||||
------
|
||||
---
|
||||
|
||||
This page serves as a guide to develop either a new Chaos Chart or a new experiment in a Chaos Chart which are published at <a href="https://hub.litmuschaos.io" target="_blank">ChaosHub</a>.
|
||||
|
||||
|
|
@ -21,7 +21,7 @@ The experiments & chaos libraries are typically written in Ansible, though not m
|
|||
the experiments can be executed in a container & can read/update the litmuschaos custom resources. For example,
|
||||
if you are writing an experiment in Go, use this [clientset](https://github.com/litmuschaos/chaos-operator/tree/master/pkg/client)
|
||||
|
||||
<hr>
|
||||
<hr/>
|
||||
|
||||
## Glossary
|
||||
|
||||
|
|
@ -33,7 +33,6 @@ A chaos chart also consists of a `package` manifest that is an index of availabl
|
|||
|
||||
Here is an example of the [ChartServiceVersion](https://github.com/litmuschaos/chaos-charts/blob/master/charts/generic/generic.chartserviceversion.yaml) & [package](https://github.com/litmuschaos/chaos-charts/blob/master/charts/generic/generic.package.yaml) manifests of the generic chaos chart.
|
||||
|
||||
|
||||
### Chaos Experiment
|
||||
|
||||
ChaosExperiment is a CRD that specifies the nature of a Chaos Experiment. The YAML file that constitutes a Chaos Experiment CR
|
||||
|
|
@ -53,17 +52,16 @@ Here is an example of the litmus book for the [pod-delete](https://github.com/li
|
|||
|
||||
The `ansible` business logic inside Litmus books can make use of readily available chaos functions. The chaos functions are available as `task-files` which are wrapped in one of the chaos libraries. See [plugins](plugins.md) for more details.
|
||||
|
||||
<hr>
|
||||
<hr/>
|
||||
|
||||
## Developing a Chaos Experiment
|
||||
|
||||
A detailed how-to guide on developing chaos experiments is available [here](https://github.com/litmuschaos/litmus-ansible/tree/master/contribute/developer_guide)
|
||||
|
||||
<br>
|
||||
<br/>
|
||||
|
||||
<hr>
|
||||
<hr/>
|
||||
|
||||
<br>
|
||||
|
||||
<br>
|
||||
<br/>
|
||||
|
||||
<br/>
|
||||
|
|
@ -1,10 +1,11 @@
|
|||
---
|
||||
id: version-1.0.0-disk-fill
|
||||
id: disk-fill
|
||||
title: Disk Fill Experiment Details
|
||||
sidebar_label: Disk Fill
|
||||
original_id: disk-fill
|
||||
---
|
||||
------
|
||||
|
||||
---
|
||||
|
||||
## Experiment Metadata
|
||||
|
||||
|
|
@ -54,28 +55,28 @@ spec:
|
|||
ephemeral-storage: "2Gi"
|
||||
limits:
|
||||
ephemeral-storage: "4Gi"
|
||||
```
|
||||
```
|
||||
|
||||
## Entry-Criteria
|
||||
|
||||
- Application pods are healthy before chaos injection.
|
||||
- Application pods are healthy before chaos injection.
|
||||
|
||||
## Exit-Criteria
|
||||
|
||||
- Application pods are healthy post chaos injection.
|
||||
- Application pods are healthy post chaos injection.
|
||||
|
||||
## Details
|
||||
|
||||
- Causes Disk Stress by filling up the ephemeral storage of the pod (in the /var/lib/docker/container/{{container_id}}) on any given node.
|
||||
- Causes the application pod to get evicted if the capacity filled exceeds the pod's ephemeral storage limit.
|
||||
- Tests the Ephemeral Storage Limits, to ensure those parameters are sufficient.
|
||||
- Tests the application's resiliency to disk stress/replica evictions.
|
||||
- Causes Disk Stress by filling up the ephemeral storage of the pod (in the /var/lib/docker/container/{{container_id}}) on any given node.
|
||||
- Causes the application pod to get evicted if the capacity filled exceeds the pod's ephemeral storage limit.
|
||||
- Tests the Ephemeral Storage Limits, to ensure those parameters are sufficient.
|
||||
- Tests the application's resiliency to disk stress/replica evictions.
|
||||
|
||||
## Integrations
|
||||
|
||||
- Disk Fill can be effected using the chaos library: `litmus`, which makes use of `dd` to create a file of
|
||||
specified capacity on the node.
|
||||
- The desired chaoslib can be selected by setting the above options as value for the env variable `LIB`
|
||||
- Disk Fill can be effected using the chaos library: `litmus`, which makes use of `dd` to create a file of
|
||||
specified capacity on the node.
|
||||
- The desired chaoslib can be selected by setting the above options as value for the env variable `LIB`
|
||||
|
||||
## Steps to Execute the Chaos Experiment
|
||||
|
||||
|
|
@ -106,9 +107,18 @@ metadata:
|
|||
labels:
|
||||
name: nginx-sa
|
||||
rules:
|
||||
- apiGroups: ["","apps","litmuschaos.io","batch"]
|
||||
resources: ["pods","jobs","pods/exec","daemonsets","chaosengines","chaosexperiments","chaosresults"]
|
||||
verbs: ["create","list","get","patch","update","delete"]
|
||||
- apiGroups: ["", "apps", "litmuschaos.io", "batch"]
|
||||
resources:
|
||||
[
|
||||
"pods",
|
||||
"jobs",
|
||||
"pods/exec",
|
||||
"daemonsets",
|
||||
"chaosengines",
|
||||
"chaosexperiments",
|
||||
"chaosresults",
|
||||
]
|
||||
verbs: ["create", "list", "get", "patch", "update", "delete"]
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
kind: ClusterRoleBinding
|
||||
|
|
@ -121,10 +131,9 @@ roleRef:
|
|||
kind: ClusterRole
|
||||
name: nginx-sa
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: nginx-sa
|
||||
namespace: default
|
||||
|
||||
- kind: ServiceAccount
|
||||
name: nginx-sa
|
||||
namespace: default
|
||||
```
|
||||
|
||||
### Prepare ChaosEngine
|
||||
|
|
@ -184,12 +193,12 @@ metadata:
|
|||
namespace: default
|
||||
spec:
|
||||
# It can be app/infra
|
||||
chaosType: 'infra'
|
||||
chaosType: "infra"
|
||||
#ex. values: ns1:name=percona,ns2:run=nginx
|
||||
auxiliaryAppInfo: ""
|
||||
appinfo:
|
||||
appns: default
|
||||
applabel: 'app=nginx'
|
||||
applabel: "app=nginx"
|
||||
appkind: deployment
|
||||
chaosServiceAccount: nginx-sa
|
||||
monitoring: false
|
||||
|
|
@ -203,7 +212,7 @@ spec:
|
|||
- name: disk-fill
|
||||
spec:
|
||||
components:
|
||||
# specify the fill percentage according to the disk pressure required
|
||||
# specify the fill percentage according to the disk pressure required
|
||||
- name: FILL_PERCENTAGE
|
||||
value: "80"
|
||||
- name: TARGET_CONTAINER
|
||||
|
|
@ -1,10 +1,12 @@
|
|||
---
|
||||
id: version-1.0.0-disk-loss
|
||||
id: disk-loss
|
||||
title: Disk Loss Experiment Details
|
||||
sidebar_label: Disk Loss
|
||||
original_id: disk-loss
|
||||
---
|
||||
------
|
||||
|
||||
---
|
||||
|
||||
## Experiment Metadata
|
||||
|
||||
<table>
|
||||
|
|
@ -21,10 +23,11 @@ original_id: disk-loss
|
|||
</table>
|
||||
|
||||
## Prerequisites
|
||||
- Ensure that the Litmus Chaos Operator is running by executing `kubectl get pods` in operator namespace (typically, `litmus`). If not, install from [here](https://raw.githubusercontent.com/litmuschaos/pages/master/docs/litmus-operator-latest.yaml)
|
||||
- Ensure that the `disk-loss` experiment resource is available in the cluster by `kubectl get chaosexperiments` in the desired namespace. If not, install from <a href="https://hub.litmuschaos.io/charts/generic/experiments/disk-loss" target="_blank">here</a>
|
||||
- Ensure to create a Kubernetes secret having the gcloud/aws access configuration(key) in the namespace of `CHAOS_NAMESPACE`.
|
||||
- There should be administrative access to the platform on which the cluster is hosted, as the recovery of the affected node could be manual. Example gcloud access to the project
|
||||
|
||||
- Ensure that the Litmus Chaos Operator is running by executing `kubectl get pods` in operator namespace (typically, `litmus`). If not, install from [here](https://raw.githubusercontent.com/litmuschaos/pages/master/docs/litmus-operator-latest.yaml)
|
||||
- Ensure that the `disk-loss` experiment resource is available in the cluster by `kubectl get chaosexperiments` in the desired namespace. If not, install from <a href="https://hub.litmuschaos.io/charts/generic/experiments/disk-loss" target="_blank">here</a>
|
||||
- Ensure to create a Kubernetes secret having the gcloud/aws access configuration(key) in the namespace of `CHAOS_NAMESPACE`.
|
||||
- There should be administrative access to the platform on which the cluster is hosted, as the recovery of the affected node could be manual. Example gcloud access to the project
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
|
|
@ -39,30 +42,30 @@ stringData:
|
|||
|
||||
## Entry-Criteria
|
||||
|
||||
- The disk is healthy before chaos injection
|
||||
- The disk is healthy before chaos injection
|
||||
|
||||
## Exit-Criteria
|
||||
|
||||
- The disk is healthy post chaos injection
|
||||
- If `APP_CHECK` is true, the application pod health is checked post chaos injection
|
||||
- The disk is healthy post chaos injection
|
||||
- If `APP_CHECK` is true, the application pod health is checked post chaos injection
|
||||
|
||||
## Details
|
||||
|
||||
- In this experiment, the external disk is detached from the node for a period equal to the `TOTAL_CHAOS_DURATION`.
|
||||
- This chaos experiment is supported on GKE and AWS platforms.
|
||||
- If the disk is created as part of dynamic persistent volume, it is expected to re-attach automatically. The experiment re-attaches the disk if it is not already attached.
|
||||
- In this experiment, the external disk is detached from the node for a period equal to the `TOTAL_CHAOS_DURATION`.
|
||||
- This chaos experiment is supported on GKE and AWS platforms.
|
||||
- If the disk is created as part of dynamic persistent volume, it is expected to re-attach automatically. The experiment re-attaches the disk if it is not already attached.
|
||||
|
||||
<b>Note:</b> Especially with mounted disk. The remount of disk is a manual step that the user has to perform.
|
||||
|
||||
## Integrations
|
||||
|
||||
- Disk loss is effected using the litmus chaoslib that internally makes use of the aws/gcloud commands
|
||||
- Disk loss is effected using the litmus chaoslib that internally makes use of the aws/gcloud commands
|
||||
|
||||
## Steps to Execute the Chaos Experiment
|
||||
|
||||
- This Chaos Experiment can be triggered by creating a ChaosEngine resource on the cluster. To understand the values to provide in a ChaosEngine specification, refer [Getting Started](getstarted.md/#prepare-chaosengine)
|
||||
- This Chaos Experiment can be triggered by creating a ChaosEngine resource on the cluster. To understand the values to provide in a ChaosEngine specification, refer [Getting Started](getstarted.md/#prepare-chaosengine)
|
||||
|
||||
- Follow the steps in the sections below to create the chaosServiceAccount, prepare the ChaosEngine & execute the experiment.
|
||||
- Follow the steps in the sections below to create the chaosServiceAccount, prepare the ChaosEngine & execute the experiment.
|
||||
|
||||
### Prepare chaosServiceAccount
|
||||
|
||||
|
|
@ -87,9 +90,17 @@ metadata:
|
|||
labels:
|
||||
name: nginx-sa
|
||||
rules:
|
||||
- apiGroups: ["","litmuschaos.io","batch"]
|
||||
resources: ["pods","jobs","secrets","chaosengines","chaosexperiments","chaosresults"]
|
||||
verbs: ["create","list","get","patch","update","delete"]
|
||||
- apiGroups: ["", "litmuschaos.io", "batch"]
|
||||
resources:
|
||||
[
|
||||
"pods",
|
||||
"jobs",
|
||||
"secrets",
|
||||
"chaosengines",
|
||||
"chaosexperiments",
|
||||
"chaosresults",
|
||||
]
|
||||
verbs: ["create", "list", "get", "patch", "update", "delete"]
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
kind: ClusterRoleBinding
|
||||
|
|
@ -102,17 +113,16 @@ roleRef:
|
|||
kind: ClusterRole
|
||||
name: nginx-sa
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: nginx-sa
|
||||
namespace: default
|
||||
|
||||
- kind: ServiceAccount
|
||||
name: nginx-sa
|
||||
namespace: default
|
||||
```
|
||||
|
||||
### Prepare ChaosEngine
|
||||
|
||||
- Provide the application info in `spec.appinfo`
|
||||
- Provide the auxiliary applications info (ns & labels) in `spec.auxiliaryAppInfo`
|
||||
- Override the experiment tunables if desired
|
||||
- Provide the application info in `spec.appinfo`
|
||||
- Provide the auxiliary applications info (ns & labels) in `spec.auxiliaryAppInfo`
|
||||
- Override the experiment tunables if desired
|
||||
|
||||
### Supported Experiment Tunables for application
|
||||
|
||||
|
|
@ -267,17 +277,21 @@ spec:
|
|||
- name: CHAOS_SERVICE_ACCOUNT
|
||||
value: ''
|
||||
```
|
||||
|
||||
## Create the ChaosEngine Resource
|
||||
- Create the ChaosEngine manifest prepared in the previous step to trigger the Chaos.
|
||||
|
||||
- Create the ChaosEngine manifest prepared in the previous step to trigger the Chaos.
|
||||
|
||||
`kubectl apply -f chaosengine.yml`
|
||||
|
||||
## Watch Chaos progress
|
||||
- Setting up a watch of the app which is using the disk in the Kubernetes Cluster
|
||||
|
||||
- Setting up a watch of the app which is using the disk in the Kubernetes Cluster
|
||||
|
||||
`watch -n 1 kubectl get pods`
|
||||
|
||||
## Check Chaos Experiment Result
|
||||
- Check whether the application is resilient to the disk loss, once the experiment (job) is completed. The ChaosResult resource name is derived like this: <ChaosEngine-Name>-<ChaosExperiment-Name>.
|
||||
|
||||
- Check whether the application is resilient to the disk loss, once the experiment (job) is completed. The ChaosResult resource name is derived like this: `{"<ChaosEngine-Name>-<ChaosExperiment-Name>"}`.
|
||||
|
||||
`kubectl describe chaosresult nginx-chaos-disk-loss -n <CHAOS_NAMESPACE>`
|
||||
|
|
@ -1,10 +1,11 @@
|
|||
---
|
||||
id: version-1.0.0-getstarted
|
||||
id: getstarted
|
||||
title: Getting Started with Litmus
|
||||
sidebar_label: Introduction
|
||||
original_id: getstarted
|
||||
---
|
||||
------
|
||||
|
||||
---
|
||||
|
||||
## Pre-requisites
|
||||
|
||||
|
|
@ -28,11 +29,9 @@ Running chaos on your application involves the following steps:
|
|||
|
||||
[Observe chaos results](#observe-chaos-results)
|
||||
|
||||
<hr>
|
||||
<hr/>
|
||||
|
||||
|
||||
|
||||
### Install Litmus
|
||||
### Install Litmus
|
||||
|
||||
```
|
||||
kubectl apply -f https://litmuschaos.github.io/pages/litmus-operator-v1.0.0.yaml
|
||||
|
|
@ -48,14 +47,9 @@ The above command install all the CRDs, required service account configuration,
|
|||
kubectl get pods -n litmus
|
||||
```
|
||||
|
||||
Expected output:
|
||||
|
||||
|
||||
|
||||
|
||||
>chaos-operator-ce-554d6c8f9f-slc8k 1/1 Running 0 6m41s
|
||||
|
||||
Expected output:
|
||||
|
||||
> chaos-operator-ce-554d6c8f9f-slc8k 1/1 Running 0 6m41s
|
||||
|
||||
- Verify if chaos CRDs are installed
|
||||
|
||||
|
|
@ -65,15 +59,15 @@ kubectl get crds | grep chaos
|
|||
|
||||
Expected output:
|
||||
|
||||
> chaosengines.litmuschaos.io 2019-10-02T08:45:25Z
|
||||
> chaosengines.litmuschaos.io 2019-10-02T08:45:25Z
|
||||
>
|
||||
> chaosexperiments.litmuschaos.io 2019-10-02T08:45:26Z
|
||||
> chaosexperiments.litmuschaos.io 2019-10-02T08:45:26Z
|
||||
>
|
||||
> chaosresults.litmuschaos.io 2019-10-02T08:45:26Z
|
||||
> chaosresults.litmuschaos.io 2019-10-02T08:45:26Z
|
||||
|
||||
- Verify if the chaos api resources are successfully created in the desired (application) namespace.
|
||||
|
||||
*Note*: Sometimes, it can take a few seconds for the resources to be available post the CRD installation
|
||||
_Note_: Sometimes, it can take a few seconds for the resources to be available post the CRD installation
|
||||
|
||||
```
|
||||
kubectl api-resources | grep chaos
|
||||
|
|
@ -81,13 +75,11 @@ kubectl api-resources | grep chaos
|
|||
|
||||
Expected output:
|
||||
|
||||
> chaosengines litmuschaos.io true ChaosEngine
|
||||
> chaosengines litmuschaos.io true ChaosEngine
|
||||
>
|
||||
> chaosexperiments litmuschaos.io true ChaosExperiment
|
||||
> chaosexperiments litmuschaos.io true ChaosExperiment
|
||||
>
|
||||
> chaosresults litmuschaos.io true ChaosResult
|
||||
|
||||
|
||||
> chaosresults litmuschaos.io true ChaosResult
|
||||
|
||||
<div class="danger">
|
||||
<strong>NOTE</strong>:
|
||||
|
|
@ -97,9 +89,9 @@ deployed in the default namespace.
|
|||
|
||||
### Install Chaos Experiments
|
||||
|
||||
Chaos experiments contain the actual chaos details. These experiments are installed on your cluster as Kubernetes CRs (Custom Resources). The Chaos Experiments are grouped as Chaos Charts and are published on <a href=" https://hub.litmuschaos.io" target="_blank">Chaos Hub</a>.
|
||||
Chaos experiments contain the actual chaos details. These experiments are installed on your cluster as Kubernetes CRs (Custom Resources). The Chaos Experiments are grouped as Chaos Charts and are published on <a href="https://hub.litmuschaos.io" target="_blank">Chaos Hub</a>.
|
||||
|
||||
The generic chaos experiments such as `pod-kill`, `container-kill`,` network-delay` are available under Generic Chaos Chart. This is the first chart you install. You can later install application specific chaos charts for running application oriented chaos.
|
||||
The generic chaos experiments such as `pod-kill`, `container-kill`,` network-delay` are available under Generic Chaos Chart. This is the first chart you install. You can later install application specific chaos charts for running application oriented chaos.
|
||||
|
||||
```
|
||||
kubectl apply -f https://hub.litmuschaos.io/api/chaos?file=charts/generic/experiments.yaml
|
||||
|
|
@ -132,9 +124,18 @@ metadata:
|
|||
labels:
|
||||
name: nginx-sa
|
||||
rules:
|
||||
- apiGroups: ["","litmuschaos.io","batch","apps"]
|
||||
resources: ["pods","jobs","daemonsets","pods/exec","chaosengines","chaosexperiments","chaosresults"]
|
||||
verbs: ["create","list","get","patch","update","delete"]
|
||||
- apiGroups: ["", "litmuschaos.io", "batch", "apps"]
|
||||
resources:
|
||||
[
|
||||
"pods",
|
||||
"jobs",
|
||||
"daemonsets",
|
||||
"pods/exec",
|
||||
"chaosengines",
|
||||
"chaosexperiments",
|
||||
"chaosresults",
|
||||
]
|
||||
verbs: ["create", "list", "get", "patch", "update", "delete"]
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
kind: RoleBinding
|
||||
|
|
@ -148,9 +149,9 @@ roleRef:
|
|||
kind: Role
|
||||
name: nginx-sa
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: nginx-sa
|
||||
namespace: default
|
||||
- kind: ServiceAccount
|
||||
name: nginx-sa
|
||||
namespace: default
|
||||
```
|
||||
|
||||
### Annotate your application
|
||||
|
|
@ -174,28 +175,28 @@ metadata:
|
|||
namespace: default
|
||||
spec:
|
||||
# It can be app/infra
|
||||
chaosType: 'app'
|
||||
chaosType: "app"
|
||||
#ex. values: ns1:name=percona,ns2:run=nginx
|
||||
auxiliaryAppInfo: ''
|
||||
auxiliaryAppInfo: ""
|
||||
components:
|
||||
runner:
|
||||
image: 'litmuschaos/chaos-executor:1.0.0'
|
||||
type: 'go'
|
||||
image: "litmuschaos/chaos-executor:1.0.0"
|
||||
type: "go"
|
||||
# It can be delete/retain
|
||||
jobCleanUpPolicy: 'delete'
|
||||
jobCleanUpPolicy: "delete"
|
||||
monitoring: false
|
||||
appinfo:
|
||||
appns: 'default'
|
||||
appns: "default"
|
||||
# FYI, To see app label, apply kubectl get pods --show-labels
|
||||
applabel: 'app=nginx'
|
||||
appkind: 'deployment'
|
||||
applabel: "app=nginx"
|
||||
appkind: "deployment"
|
||||
chaosServiceAccount: nginx-sa
|
||||
experiments:
|
||||
- name: container-kill
|
||||
spec:
|
||||
components:
|
||||
- name: TARGET_CONTAINER
|
||||
value: 'nginx'
|
||||
value: "nginx"
|
||||
```
|
||||
|
||||
### Override Default Chaos Experiments Variables
|
||||
|
|
@ -212,11 +213,8 @@ experiments:
|
|||
value: nginx
|
||||
```
|
||||
|
||||
|
||||
|
||||
### Run Chaos
|
||||
|
||||
|
||||
```console
|
||||
kubectl apply -f chaosengine.yaml
|
||||
```
|
||||
|
|
@ -227,9 +225,9 @@ kubectl apply -f chaosengine.yaml
|
|||
|
||||
### Observe Chaos results
|
||||
|
||||
Describe the ChaosResult CR to know the status of each experiment. The ```spec.verdict``` is set to Running when the experiment is in progress, eventually changing to either pass or fail.
|
||||
Describe the ChaosResult CR to know the status of each experiment. The `spec.verdict` is set to Running when the experiment is in progress, eventually changing to either pass or fail.
|
||||
|
||||
<strong> NOTE:</strong> ChaosResult CR name will be `<chaos-engine-name>-<chaos-experiment-name>`
|
||||
<strong> NOTE:</strong> ChaosResult CR name will be `{"<chaos-engine-name>-<chaos-experiment-name>"}`
|
||||
|
||||
```console
|
||||
kubectl describe chaosresult engine-nginx-container-kill
|
||||
|
|
@ -1,5 +1,5 @@
|
|||
---
|
||||
id: version-1.0.0-kafka-broker-disk-failure
|
||||
id: kafka-broker-disk-failure
|
||||
title: Kafka Broker Disk Failure Experiment Details
|
||||
sidebar_label: Broker Disk Failure
|
||||
original_id: kafka-broker-disk-failure
|
||||
|
|
@ -7,9 +7,9 @@ original_id: kafka-broker-disk-failure
|
|||
|
||||
## Experiment Metadata
|
||||
|
||||
| Type | Description | Kafka Distribution | Tested K8s Platform
|
||||
| ----- | -------------------------------|----------------------|---------------------
|
||||
| Kafka | Fail kafka broker disk/storage | Confluent, Kudo-Kafka| GKE
|
||||
| Type | Description | Kafka Distribution | Tested K8s Platform |
|
||||
| ----- | ------------------------------ | --------------------- | ------------------- |
|
||||
| Kafka | Fail kafka broker disk/storage | Confluent, Kudo-Kafka | GKE |
|
||||
|
||||
## Prerequisites
|
||||
|
||||
|
|
@ -78,9 +78,19 @@ metadata:
|
|||
labels:
|
||||
name: kafka-sa
|
||||
rules:
|
||||
- apiGroups: ["","litmuschaos.io","batch","apps"]
|
||||
resources: ["pods","jobs","pod/exec","statefulsets","secrets","chaosengines","chaosexperiments","chaosresults"]
|
||||
verbs: ["create","list","get","patch","delete"]
|
||||
- apiGroups: ["", "litmuschaos.io", "batch", "apps"]
|
||||
resources:
|
||||
[
|
||||
"pods",
|
||||
"jobs",
|
||||
"pod/exec",
|
||||
"statefulsets",
|
||||
"secrets",
|
||||
"chaosengines",
|
||||
"chaosexperiments",
|
||||
"chaosresults",
|
||||
]
|
||||
verbs: ["create", "list", "get", "patch", "delete"]
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
kind: ClusterRoleBinding
|
||||
|
|
@ -93,10 +103,9 @@ roleRef:
|
|||
kind: ClusterRole
|
||||
name: kafka-role
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: kafka-sa
|
||||
namespace: default
|
||||
|
||||
- kind: ServiceAccount
|
||||
name: kafka-sa
|
||||
namespace: default
|
||||
```
|
||||
|
||||
### Prepare ChaosEngine
|
||||
|
|
@ -106,28 +115,28 @@ subjects:
|
|||
|
||||
#### Supported Experiment Tunables
|
||||
|
||||
| Variables | Description |Type | Notes |
|
||||
| ----------------------| ------------------------------------------------------------ |-----------|---------------------------------------------------------|
|
||||
| KAFKA_NAMESPACE | Namespace of Kafka Brokers | Mandatory | May be same as value for `spec.appinfo.appns` |
|
||||
| KAFKA_LABEL | Unique label of Kafka Brokers | Mandatory | May be same as value for `spec.appinfo.applabel` |
|
||||
| KAFKA_SERVICE | Headless service of the Kafka Statefulset | Mandatory | |
|
||||
| KAFKA_PORT | Port of the Kafka ClusterIP service | Mandatory | |
|
||||
| ZOOKEEPER_NAMESPACE | Namespace of Zookeeper Cluster | Mandatory | May be same as value for KAFKA_NAMESPACE or other |
|
||||
| ZOOKEEPER_LABEL | Unique label of Zokeeper statefulset | Mandatory | |
|
||||
| ZOOKEEPER_SERVICE | Headless service of the Zookeeper Statefulset | Mandatory | |
|
||||
| ZOOKEEPER_PORT | Port of the Zookeeper ClusterIP service | Mandatory | |
|
||||
| CLOUD_PLATFORM | Cloud platform type on which to inject disk loss | Mandatory | Supported platforms: GCP |
|
||||
| PROJECT_ID | GCP Project ID in which the Cluster is created | Mandatory | |
|
||||
| DISK_NAME | GCloud Disk attached to the Cluster Node where specified broker is scheduled | Mandatory | |
|
||||
| ZONE_NAME | Zone in which the Disks/Cluster are created | Mandatory | |
|
||||
| KAFKA_BROKER | Kafka broker pod which is using the specified disk | Mandatory | Experiment verifies this by mapping node details |
|
||||
| KAFKA_KIND | Kafka deployment type | Optional | Same as `spec.appinfo.appkind`. Supported: `statefulset`|
|
||||
| KAFKA_LIVENESS_STREAM | Kafka liveness message stream | Optional | Supported: `enabled`, `disabled` |
|
||||
| KAFKA_LIVENESS_IMAGE | Image used for liveness message stream | Optional | Image as `<registry_url>/<repository>/<image>:<tag>` |
|
||||
| KAFKA_REPLICATION_FACTOR| Number of partition replicas for liveness topic partition | Optional | Necessary if KAFKA_LIVENESS_STREAM is `enabled` |
|
||||
| KAFKA_INSTANCE_NAME | Name of the Kafka chroot path on zookeeper | Optional | Necessary if installation involves use of such path |
|
||||
| KAFKA_CONSUMER_TIMEOUT| Kafka consumer message timeout, post which it terminates | Optional | Defaults to 30000ms |
|
||||
| TOTAL_CHAOS_DURATION | The time duration for chaos insertion (seconds) | Optional | Defaults to 15s |
|
||||
| Variables | Description | Type | Notes |
|
||||
| ------------------------ | ---------------------------------------------------------------------------- | --------- | -------------------------------------------------------- |
|
||||
| KAFKA_NAMESPACE | Namespace of Kafka Brokers | Mandatory | May be same as value for `spec.appinfo.appns` |
|
||||
| KAFKA_LABEL | Unique label of Kafka Brokers | Mandatory | May be same as value for `spec.appinfo.applabel` |
|
||||
| KAFKA_SERVICE | Headless service of the Kafka Statefulset | Mandatory | |
|
||||
| KAFKA_PORT | Port of the Kafka ClusterIP service | Mandatory | |
|
||||
| ZOOKEEPER_NAMESPACE | Namespace of Zookeeper Cluster | Mandatory | May be same as value for KAFKA_NAMESPACE or other |
|
||||
| ZOOKEEPER_LABEL | Unique label of Zokeeper statefulset | Mandatory | |
|
||||
| ZOOKEEPER_SERVICE | Headless service of the Zookeeper Statefulset | Mandatory | |
|
||||
| ZOOKEEPER_PORT | Port of the Zookeeper ClusterIP service | Mandatory | |
|
||||
| CLOUD_PLATFORM | Cloud platform type on which to inject disk loss | Mandatory | Supported platforms: GCP |
|
||||
| PROJECT_ID | GCP Project ID in which the Cluster is created | Mandatory | |
|
||||
| DISK_NAME | GCloud Disk attached to the Cluster Node where specified broker is scheduled | Mandatory | |
|
||||
| ZONE_NAME | Zone in which the Disks/Cluster are created | Mandatory | |
|
||||
| KAFKA_BROKER | Kafka broker pod which is using the specified disk | Mandatory | Experiment verifies this by mapping node details |
|
||||
| KAFKA_KIND | Kafka deployment type | Optional | Same as `spec.appinfo.appkind`. Supported: `statefulset` |
|
||||
| KAFKA_LIVENESS_STREAM | Kafka liveness message stream | Optional | Supported: `enabled`, `disabled` |
|
||||
| KAFKA_LIVENESS_IMAGE | Image used for liveness message stream | Optional | Image as `<registry_url>/<repository>/<image>:<tag>` |
|
||||
| KAFKA_REPLICATION_FACTOR | Number of partition replicas for liveness topic partition | Optional | Necessary if KAFKA_LIVENESS_STREAM is `enabled` |
|
||||
| KAFKA_INSTANCE_NAME | Name of the Kafka chroot path on zookeeper | Optional | Necessary if installation involves use of such path |
|
||||
| KAFKA_CONSUMER_TIMEOUT | Kafka consumer message timeout, post which it terminates | Optional | Defaults to 30000ms |
|
||||
| TOTAL_CHAOS_DURATION | The time duration for chaos insertion (seconds) | Optional | Defaults to 15s |
|
||||
|
||||
#### Sample ChaosEngine Manifest
|
||||
|
||||
|
|
@ -139,12 +148,12 @@ metadata:
|
|||
namespace: default
|
||||
spec:
|
||||
# It can be app/infra
|
||||
chaosType: 'app'
|
||||
chaosType: "app"
|
||||
#ex. values: ns1:name=percona,ns2:run=nginx
|
||||
auxiliaryAppInfo: ""
|
||||
appinfo:
|
||||
appns: default
|
||||
applabel: 'app=cp-kafka'
|
||||
applabel: "app=cp-kafka"
|
||||
appkind: statefulset
|
||||
chaosServiceAccount: kafka-sa
|
||||
monitoring: false
|
||||
|
|
@ -160,66 +169,66 @@ spec:
|
|||
components:
|
||||
# choose based on available kafka broker replicas
|
||||
- name: KAFKA_REPLICATION_FACTOR
|
||||
value: '3'
|
||||
value: "3"
|
||||
|
||||
# get via "kubectl get pods --show-labels -n <kafka-namespace>"
|
||||
- name: KAFKA_LABEL
|
||||
value: 'app=cp-kafka'
|
||||
value: "app=cp-kafka"
|
||||
|
||||
- name: KAFKA_NAMESPACE
|
||||
value: 'default'
|
||||
value: "default"
|
||||
|
||||
# get via "kubectl get svc -n <kafka-namespace>"
|
||||
- name: KAFKA_SERVICE
|
||||
value: 'kafka-cp-kafka-headless'
|
||||
value: "kafka-cp-kafka-headless"
|
||||
|
||||
# get via "kubectl get svc -n <kafka-namespace>
|
||||
- name: KAFKA_PORT
|
||||
value: '9092'
|
||||
value: "9092"
|
||||
|
||||
# in milliseconds
|
||||
- name: KAFKA_CONSUMER_TIMEOUT
|
||||
value: '70000'
|
||||
value: "70000"
|
||||
|
||||
# ensure to set the instance name if using KUDO operator
|
||||
- name: KAFKA_INSTANCE_NAME
|
||||
value: ''
|
||||
value: ""
|
||||
|
||||
- name: ZOOKEEPER_NAMESPACE
|
||||
value: 'default'
|
||||
value: "default"
|
||||
|
||||
# get via "kubectl get pods --show-labels -n <zk-namespace>"
|
||||
- name: ZOOKEEPER_LABEL
|
||||
value: 'app=cp-zookeeper'
|
||||
value: "app=cp-zookeeper"
|
||||
|
||||
# get via "kubectl get svc -n <zk-namespace>
|
||||
- name: ZOOKEEPER_SERVICE
|
||||
value: 'kafka-cp-zookeeper-headless'
|
||||
value: "kafka-cp-zookeeper-headless"
|
||||
|
||||
# get via "kubectl get svc -n <zk-namespace>
|
||||
- name: ZOOKEEPER_PORT
|
||||
value: '2181'
|
||||
value: "2181"
|
||||
|
||||
# get from google cloud console or "gcloud projects list"
|
||||
- name: PROJECT_ID
|
||||
value: 'argon-tractor-237811'
|
||||
value: "argon-tractor-237811"
|
||||
|
||||
# attached to (in use by) node where 'kafka-0' is scheduled
|
||||
- name: DISK_NAME
|
||||
value: 'disk-1'
|
||||
value: "disk-1"
|
||||
|
||||
- name: ZONE_NAME
|
||||
value: 'us-central1-a'
|
||||
value: "us-central1-a"
|
||||
|
||||
# Uses "disk-1" attached to the node on which it is scheduled
|
||||
- name: KAFKA_BROKER
|
||||
value: 'kafka-0'
|
||||
value: "kafka-0"
|
||||
|
||||
# set chaos duration (in sec) as desired
|
||||
- name: TOTAL_CHAOS_DURATION
|
||||
value: '60'
|
||||
|
||||
value: "60"
|
||||
```
|
||||
|
||||
### Create the ChaosEngine Resource
|
||||
|
||||
- Create the ChaosEngine manifest prepared in the previous step to trigger the Chaos.
|
||||
|
|
@ -254,4 +263,4 @@ spec:
|
|||
|
||||
- TODO: A sample recording of this experiment execution is provided here.
|
||||
|
||||
------
|
||||
---
|
||||
|
|
@ -1,5 +1,5 @@
|
|||
---
|
||||
id: version-1.0.0-kafka-broker-pod-failure
|
||||
id: kafka-broker-pod-failure
|
||||
title: Kafka Broker Pod Failure Experiment Details
|
||||
sidebar_label: Broker Pod Failure
|
||||
original_id: kafka-broker-pod-failure
|
||||
|
|
@ -7,9 +7,9 @@ original_id: kafka-broker-pod-failure
|
|||
|
||||
## Experiment Metadata
|
||||
|
||||
| Type | Description | Kafka Distribution | Tested K8s Platform
|
||||
| ----- | -------------------------------|----------------------|---------------------
|
||||
| Kafka | Fail kafka leader-broker pods | Confluent, Kudo-Kafka| AWS Konvoy, GKE
|
||||
| Type | Description | Kafka Distribution | Tested K8s Platform |
|
||||
| ----- | ----------------------------- | --------------------- | ------------------- |
|
||||
| Kafka | Fail kafka leader-broker pods | Confluent, Kudo-Kafka | AWS Konvoy, GKE |
|
||||
|
||||
## Prerequisites
|
||||
|
||||
|
|
@ -26,7 +26,6 @@ original_id: kafka-broker-pod-failure
|
|||
|
||||
- Ensure that the kafka-broker-disk failure experiment resource is available in the cluster. If not, install from [here](https://hub.litmuschaos.io/charts/kafka/experiments/kafka-broker-pod-failure)
|
||||
|
||||
|
||||
## Entry Criteria
|
||||
|
||||
- Kafka Cluster (comprising the Kafka-broker & Zookeeper Statefulsets) is healthy
|
||||
|
|
@ -77,12 +76,23 @@ metadata:
|
|||
labels:
|
||||
name: kafka-sa
|
||||
rules:
|
||||
- apiGroups: ["","litmuschaos.io","batch","apps"]
|
||||
resources: ["pods","deployments","jobs","pod/exec","statefulsets","configmaps","chaosengines","chaosexperiments","chaosresults"]
|
||||
verbs: ["create","list","get","patch","delete"]
|
||||
- apiGroups: [""]
|
||||
resources: ["nodes"]
|
||||
verbs : ["get","list"]
|
||||
- apiGroups: ["", "litmuschaos.io", "batch", "apps"]
|
||||
resources:
|
||||
[
|
||||
"pods",
|
||||
"deployments",
|
||||
"jobs",
|
||||
"pod/exec",
|
||||
"statefulsets",
|
||||
"configmaps",
|
||||
"chaosengines",
|
||||
"chaosexperiments",
|
||||
"chaosresults",
|
||||
]
|
||||
verbs: ["create", "list", "get", "patch", "delete"]
|
||||
- apiGroups: [""]
|
||||
resources: ["nodes"]
|
||||
verbs: ["get", "list"]
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
kind: ClusterRoleBinding
|
||||
|
|
@ -95,10 +105,9 @@ roleRef:
|
|||
kind: ClusterRole
|
||||
name: kafka-role
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: kafka-sa
|
||||
namespace: default
|
||||
|
||||
- kind: ServiceAccount
|
||||
name: kafka-sa
|
||||
namespace: default
|
||||
```
|
||||
|
||||
### Prepare ChaosEngine
|
||||
|
|
@ -108,27 +117,27 @@ subjects:
|
|||
|
||||
#### Supported Experiment Tunables
|
||||
|
||||
| Variables | Description |Type | Notes |
|
||||
| ----------------------| ------------------------------------------------------------ |-----------|---------------------------------------------------------|
|
||||
| KAFKA_NAMESPACE | Namespace of Kafka Brokers | Mandatory | May be same as value for `spec.appinfo.appns` |
|
||||
| KAFKA_LABEL | Unique label of Kafka Brokers | Mandatory | May be same as value for `spec.appinfo.applabel` |
|
||||
| KAFKA_SERVICE | Headless service of the Kafka Statefulset | Mandatory | |
|
||||
| KAFKA_PORT | Port of the Kafka ClusterIP service | Mandatory | |
|
||||
| ZOOKEEPER_NAMESPACE | Namespace of Zookeeper Cluster | Mandatory | May be same as value for KAFKA_NAMESPACE or other |
|
||||
| ZOOKEEPER_LABEL | Unique label of Zokeeper statefulset | Mandatory | |
|
||||
| ZOOKEEPER_SERVICE | Headless service of the Zookeeper Statefulset | Mandatory | |
|
||||
| ZOOKEEPER_PORT | Port of the Zookeeper ClusterIP service | Mandatory | |
|
||||
| KAFKA_KIND | Kafka deployment type | Optional | Same as `spec.appinfo.appkind`. Supported: `statefulset`|
|
||||
| KAFKA_LIVENESS_STREAM | Kafka liveness message stream | Optional | Supported: `enabled`, `disabled` |
|
||||
| KAFKA_LIVENESS_IMAGE | Image used for liveness message stream | Optional | Image as `<registry_url>/<repository>/<image>:<tag>` |
|
||||
| KAFKA_REPLICATION_FACTOR| Number of partition replicas for liveness topic partition | Optional | Necessary if KAFKA_LIVENESS_STREAM is `enabled` |
|
||||
| KAFKA_INSTANCE_NAME | Name of the Kafka chroot path on zookeeper | Optional | Necessary if installation involves use of such path |
|
||||
| KAFKA_CONSUMER_TIMEOUT| Kafka consumer message timeout, post which it terminates | Optional | Defaults to 30000ms |
|
||||
| KAFKA_BROKER | Kafka broker pod (name) to be deleted | Optional | A target selection mode (random/liveness-based/specific)|
|
||||
| TOTAL_CHAOS_DURATION | The time duration for chaos insertion (seconds) | Optional | Defaults to 15s |
|
||||
| CHAOS_INTERVAL | Time interval b/w two successive broker failures (sec) | Optional | Defaults to 5s |
|
||||
| LIB | The chaos lib used to inject the chaos | Optional | Defaults to `litmus`. Supported: `litmus`, `powerfulseal|
|
||||
| CHAOS_SERVICE_ACCOUNT | Service account used by the powerfulseal deployment | Optional | Defaults to `default` on namespace `spec.appinfo.appns` |
|
||||
| Variables | Description | Type | Notes |
|
||||
| ------------------------ | --------------------------------------------------------- | --------- | -------------------------------------------------------- |
|
||||
| KAFKA_NAMESPACE | Namespace of Kafka Brokers | Mandatory | May be same as value for `spec.appinfo.appns` |
|
||||
| KAFKA_LABEL | Unique label of Kafka Brokers | Mandatory | May be same as value for `spec.appinfo.applabel` |
|
||||
| KAFKA_SERVICE | Headless service of the Kafka Statefulset | Mandatory | |
|
||||
| KAFKA_PORT | Port of the Kafka ClusterIP service | Mandatory | |
|
||||
| ZOOKEEPER_NAMESPACE | Namespace of Zookeeper Cluster | Mandatory | May be same as value for KAFKA_NAMESPACE or other |
|
||||
| ZOOKEEPER_LABEL | Unique label of Zokeeper statefulset | Mandatory | |
|
||||
| ZOOKEEPER_SERVICE | Headless service of the Zookeeper Statefulset | Mandatory | |
|
||||
| ZOOKEEPER_PORT | Port of the Zookeeper ClusterIP service | Mandatory | |
|
||||
| KAFKA_KIND | Kafka deployment type | Optional | Same as `spec.appinfo.appkind`. Supported: `statefulset` |
|
||||
| KAFKA_LIVENESS_STREAM | Kafka liveness message stream | Optional | Supported: `enabled`, `disabled` |
|
||||
| KAFKA_LIVENESS_IMAGE | Image used for liveness message stream | Optional | Image as `<registry_url>/<repository>/<image>:<tag>` |
|
||||
| KAFKA_REPLICATION_FACTOR | Number of partition replicas for liveness topic partition | Optional | Necessary if KAFKA_LIVENESS_STREAM is `enabled` |
|
||||
| KAFKA_INSTANCE_NAME | Name of the Kafka chroot path on zookeeper | Optional | Necessary if installation involves use of such path |
|
||||
| KAFKA_CONSUMER_TIMEOUT | Kafka consumer message timeout, post which it terminates | Optional | Defaults to 30000ms |
|
||||
| KAFKA_BROKER | Kafka broker pod (name) to be deleted | Optional | A target selection mode (random/liveness-based/specific) |
|
||||
| TOTAL_CHAOS_DURATION | The time duration for chaos insertion (seconds) | Optional | Defaults to 15s |
|
||||
| CHAOS_INTERVAL | Time interval b/w two successive broker failures (sec) | Optional | Defaults to 5s |
|
||||
| LIB | The chaos lib used to inject the chaos | Optional | Defaults to `litmus`. Supported: `litmus`, `powerfulseal |
|
||||
| CHAOS_SERVICE_ACCOUNT | Service account used by the powerfulseal deployment | Optional | Defaults to `default` on namespace `spec.appinfo.appns` |
|
||||
|
||||
#### Sample ChaosEngine Manifest
|
||||
|
||||
|
|
@ -140,12 +149,12 @@ metadata:
|
|||
namespace: default
|
||||
spec:
|
||||
# It can be app/infra
|
||||
chaosType: 'app'
|
||||
chaosType: "app"
|
||||
#ex. values: ns1:name=percona,ns2:run=nginx
|
||||
auxiliaryAppInfo: ""
|
||||
appinfo:
|
||||
appns: default
|
||||
applabel: 'app=cp-kafka'
|
||||
applabel: "app=cp-kafka"
|
||||
appkind: statefulset
|
||||
chaosServiceAccount: kafka-sa
|
||||
monitoring: false
|
||||
|
|
@ -161,53 +170,53 @@ spec:
|
|||
components:
|
||||
# choose based on available kafka broker replicas
|
||||
- name: KAFKA_REPLICATION_FACTOR
|
||||
value: '3'
|
||||
value: "3"
|
||||
|
||||
# get via "kubectl get pods --show-labels -n <kafka-namespace>"
|
||||
- name: KAFKA_LABEL
|
||||
value: 'app=cp-kafka'
|
||||
value: "app=cp-kafka"
|
||||
|
||||
- name: KAFKA_NAMESPACE
|
||||
value: 'default'
|
||||
value: "default"
|
||||
|
||||
# get via "kubectl get svc -n <kafka-namespace>"
|
||||
- name: KAFKA_SERVICE
|
||||
value: 'kafka-cp-kafka-headless'
|
||||
value: "kafka-cp-kafka-headless"
|
||||
|
||||
# get via "kubectl get svc -n <kafka-namespace>
|
||||
- name: KAFKA_PORT
|
||||
value: '9092'
|
||||
value: "9092"
|
||||
|
||||
# in milliseconds
|
||||
- name: KAFKA_CONSUMER_TIMEOUT
|
||||
value: '70000'
|
||||
value: "70000"
|
||||
|
||||
# ensure to set the instance name if using KUDO operator
|
||||
- name: KAFKA_INSTANCE_NAME
|
||||
value: ''
|
||||
value: ""
|
||||
|
||||
- name: ZOOKEEPER_NAMESPACE
|
||||
value: 'default'
|
||||
value: "default"
|
||||
|
||||
# get via "kubectl get pods --show-labels -n <zk-namespace>"
|
||||
- name: ZOOKEEPER_LABEL
|
||||
value: 'app=cp-zookeeper'
|
||||
value: "app=cp-zookeeper"
|
||||
|
||||
# get via "kubectl get svc -n <zk-namespace>
|
||||
- name: ZOOKEEPER_SERVICE
|
||||
value: 'kafka-cp-zookeeper-headless'
|
||||
value: "kafka-cp-zookeeper-headless"
|
||||
|
||||
# get via "kubectl get svc -n <zk-namespace>
|
||||
- name: ZOOKEEPER_PORT
|
||||
value: '2181'
|
||||
value: "2181"
|
||||
|
||||
# set chaos duration (in sec) as desired
|
||||
- name: TOTAL_CHAOS_DURATION
|
||||
value: '60'
|
||||
value: "60"
|
||||
|
||||
# set chaos interval (in sec) as desired
|
||||
- name: CHAOS_INTERVAL
|
||||
value: '20'
|
||||
value: "20"
|
||||
|
||||
# pod failures without '--force' & default terminationGracePeriodSeconds
|
||||
- name: FORCE
|
||||
|
|
@ -237,4 +246,4 @@ spec:
|
|||
|
||||
- TODO: A sample recording of this experiment execution is provided here.
|
||||
|
||||
------
|
||||
---
|
||||
|
|
@ -1,22 +1,23 @@
|
|||
---
|
||||
id: version-1.0.0-node-drain
|
||||
id: node-drain
|
||||
title: Node Drain Experiment Details
|
||||
sidebar_label: Node Drain
|
||||
original_id: node-drain
|
||||
---
|
||||
------
|
||||
|
||||
---
|
||||
|
||||
## Experiment Metadata
|
||||
|
||||
| Type | Description | Tested K8s Platform |
|
||||
| ----------| -------------------------------------------- | ------------------------------------------------------------------|
|
||||
| Generic | Drain the node where application pod is scheduled. | GKE, AWS, Packet(Kubeadm), Konvoy(AWS)|
|
||||
| Type | Description | Tested K8s Platform |
|
||||
| ------- | -------------------------------------------------- | -------------------------------------- |
|
||||
| Generic | Drain the node where application pod is scheduled. | GKE, AWS, Packet(Kubeadm), Konvoy(AWS) |
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Ensure that the Litmus Chaos Operator is running by executing `kubectl get pods` in operator namespace (typically, `litmus`). If not, install from [here](https://raw.githubusercontent.com/litmuschaos/pages/master/docs/litmus-operator-latest.yaml)
|
||||
- Ensure that the `node-drain` experiment resource is available in the cluster by executing `kubectl get chaosexperiments` in the desired namespace. If not, install from [here](https://hub.litmuschaos.io/charts/generic/experiments/drain-node)
|
||||
- Ensure that the node specified in the experiment ENV variable `APP_NODE` (the node which will be drained) should be cordoned before execution of the chaos experiment (before applying the chaosengine manifest) to ensure that the litmus experiment runner pods are not scheduled on it / subjected to eviction. This can be achieved with the following steps:
|
||||
- Ensure that the node specified in the experiment ENV variable `APP_NODE` (the node which will be drained) should be cordoned before execution of the chaos experiment (before applying the chaosengine manifest) to ensure that the litmus experiment runner pods are not scheduled on it / subjected to eviction. This can be achieved with the following steps:
|
||||
|
||||
- Get node names against the applications pods: `kubectl get pods -o wide`
|
||||
- Cordon the node `kubectl cordon <nodename>`
|
||||
|
|
@ -34,7 +35,6 @@ original_id: node-drain
|
|||
- This experiment drains the node where application pod is running and verifies if it is scheduled on another available node.
|
||||
- In the end of experiment it uncordons the specified node so that it can be utilised in future.
|
||||
|
||||
|
||||
## Integrations
|
||||
|
||||
- Drain node can be effected using the chaos library: `litmus`
|
||||
|
|
@ -68,12 +68,21 @@ metadata:
|
|||
labels:
|
||||
name: nginx-sa
|
||||
rules:
|
||||
- apiGroups: ["","litmuschaos.io","batch","extensions"]
|
||||
resources: ["pods","jobs","chaosengines","daemonsets","pods/eviction","chaosexperiments","chaosresults"]
|
||||
verbs: ["create","list","get","patch","update","delete"]
|
||||
- apiGroups: [""]
|
||||
resources: ["nodes"]
|
||||
verbs: ["patch","get","list"]
|
||||
- apiGroups: ["", "litmuschaos.io", "batch", "extensions"]
|
||||
resources:
|
||||
[
|
||||
"pods",
|
||||
"jobs",
|
||||
"chaosengines",
|
||||
"daemonsets",
|
||||
"pods/eviction",
|
||||
"chaosexperiments",
|
||||
"chaosresults",
|
||||
]
|
||||
verbs: ["create", "list", "get", "patch", "update", "delete"]
|
||||
- apiGroups: [""]
|
||||
resources: ["nodes"]
|
||||
verbs: ["patch", "get", "list"]
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
kind: ClusterRoleBinding
|
||||
|
|
@ -86,9 +95,9 @@ roleRef:
|
|||
kind: ClusterRole
|
||||
name: nginx-sa
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: nginx-sa
|
||||
namespace: default
|
||||
- kind: ServiceAccount
|
||||
name: nginx-sa
|
||||
namespace: default
|
||||
```
|
||||
|
||||
### Prepare ChaosEngine
|
||||
|
|
@ -130,12 +139,12 @@ metadata:
|
|||
namespace: default
|
||||
spec:
|
||||
# It can be app/infra
|
||||
chaosType: 'infra'
|
||||
chaosType: "infra"
|
||||
#ex. values: ns1:name=percona,ns2:run=nginx
|
||||
auxiliaryAppInfo: ""
|
||||
appinfo:
|
||||
appns: default
|
||||
applabel: 'app=nginx'
|
||||
applabel: "app=nginx"
|
||||
appkind: deployment
|
||||
chaosServiceAccount: nginx-sa
|
||||
monitoring: false
|
||||
|
|
@ -149,9 +158,9 @@ spec:
|
|||
- name: node-drain
|
||||
spec:
|
||||
components:
|
||||
# set node name
|
||||
# set node name
|
||||
- name: APP_NODE
|
||||
value: 'node-1'
|
||||
value: "node-1"
|
||||
```
|
||||
|
||||
### Create the ChaosEngine Resource
|
||||
|
|
@ -1,16 +1,17 @@
|
|||
---
|
||||
id: version-1.0.0-openebs-pool-container-failure
|
||||
id: openebs-pool-container-failure
|
||||
title: OpenEBS Pool Container Failure Experiment Details
|
||||
sidebar_label: Pool Container Failure
|
||||
original_id: openebs-pool-container-failure
|
||||
---
|
||||
------
|
||||
|
||||
---
|
||||
|
||||
## Experiment Metadata
|
||||
|
||||
| Type | Description | Tested K8s Platform |
|
||||
| ----------| ------------------------ | ------------------------------------------------------------------|
|
||||
| OpenEBS | Kill the cstor pool pod container and check if gets created again | GKE, Konvoy(AWS), Packet(Kubeadm), Minikube, OpenShift(Baremetal) |
|
||||
| Type | Description | Tested K8s Platform |
|
||||
| ------- | ----------------------------------------------------------------- | ----------------------------------------------------------------- |
|
||||
| OpenEBS | Kill the cstor pool pod container and check if gets created again | GKE, Konvoy(AWS), Packet(Kubeadm), Minikube, OpenShift(Baremetal) |
|
||||
|
||||
## Prerequisites
|
||||
|
||||
|
|
@ -31,6 +32,7 @@ original_id: openebs-pool-container-failure
|
|||
dbpassword: k8sDem0
|
||||
dbname: test
|
||||
```
|
||||
|
||||
- Ensure that the chaosServiceAccount used for the experiment has cluster-scope permissions as the experiment may involve carrying out the chaos in the `openebs` namespace
|
||||
while performing application health checks in its respective namespace.
|
||||
|
||||
|
|
@ -74,13 +76,13 @@ If the experiment tunable DATA_PERSISTENCE is set to 'enabled':
|
|||
|
||||
#### Supported Experiment Tunables
|
||||
|
||||
| Variables | Description | Type | Notes |
|
||||
| ----------------------| ------------------------------------------------------------ |-----------|------------------------------------------------------------|
|
||||
| APP_PVC | The PersistentVolumeClaim used by the stateful application | Mandatory | PVC must use OpenEBS cStor storage class |
|
||||
| DEPLOY_TYPE | Type of Kubernetes resource used by the stateful application | Optional | Defaults to `deployment`. Supported: `deployment`, `statefulset`| |
|
||||
| LIB_IMAGE | The chaos library image used to inject the latency | Optional | Defaults to `gaiaadm/pumba:0.4.8`. Supported: `gaiaadm/pumba:0.4.8`|
|
||||
| TOTAL_CHAOS_DURATION | Amount of soak time for I/O post pod kill | Optional | Defaults to 600 seconds |
|
||||
| DATA_PERSISTENCE | Flag to perform data consistency checks on the application | Optional | Default value is disabled (empty/unset). Set to `enabled` to perform data checks. Ensure configmap with app details are created |
|
||||
| Variables | Description | Type | Notes |
|
||||
| -------------------- | ------------------------------------------------------------ | --------- | ------------------------------------------------------------------------------------------------------------------------------- | --- |
|
||||
| APP_PVC | The PersistentVolumeClaim used by the stateful application | Mandatory | PVC must use OpenEBS cStor storage class |
|
||||
| DEPLOY_TYPE | Type of Kubernetes resource used by the stateful application | Optional | Defaults to `deployment`. Supported: `deployment`, `statefulset` | |
|
||||
| LIB_IMAGE | The chaos library image used to inject the latency | Optional | Defaults to `gaiaadm/pumba:0.4.8`. Supported: `gaiaadm/pumba:0.4.8` |
|
||||
| TOTAL_CHAOS_DURATION | Amount of soak time for I/O post pod kill | Optional | Defaults to 600 seconds |
|
||||
| DATA_PERSISTENCE | Flag to perform data consistency checks on the application | Optional | Default value is disabled (empty/unset). Set to `enabled` to perform data checks. Ensure configmap with app details are created |
|
||||
|
||||
#### Sample ChaosEngine Manifest
|
||||
|
||||
|
|
@ -93,7 +95,7 @@ metadata:
|
|||
spec:
|
||||
appinfo:
|
||||
appns: default
|
||||
applabel: 'app=percona'
|
||||
applabel: "app=percona"
|
||||
appkind: deployment
|
||||
chaosServiceAccount: percona-sa
|
||||
monitoring: false
|
||||
|
|
@ -103,7 +105,7 @@ spec:
|
|||
spec:
|
||||
components:
|
||||
- name: APP_PVC
|
||||
value: 'pvc-c466262a-a5f2-4f0f-b594-5daddfc2e29d'
|
||||
value: "pvc-c466262a-a5f2-4f0f-b594-5daddfc2e29d"
|
||||
- name: DEPLOY_TYPE
|
||||
value: deployment
|
||||
```
|
||||
|
|
@ -1,16 +1,17 @@
|
|||
---
|
||||
id: version-1.0.0-openebs-pool-pod-failure
|
||||
id: openebs-pool-pod-failure
|
||||
title: OpenEBS Pool Pod Failure Experiment Details
|
||||
sidebar_label: Pool Pod Failure
|
||||
original_id: openebs-pool-pod-failure
|
||||
---
|
||||
------
|
||||
|
||||
---
|
||||
|
||||
## Experiment Metadata
|
||||
|
||||
| Type | Description | Tested K8s Platform |
|
||||
| ----------| ------------------------ | ------------------------------------------------------------------|
|
||||
| OpenEBS | Kill the cstor pool pod and check if gets created again | GKE, Konvoy(AWS), Packet(Kubeadm), Minikube, OpenShift(Baremetal) |
|
||||
| Type | Description | Tested K8s Platform |
|
||||
| ------- | ------------------------------------------------------- | ----------------------------------------------------------------- |
|
||||
| OpenEBS | Kill the cstor pool pod and check if gets created again | GKE, Konvoy(AWS), Packet(Kubeadm), Minikube, OpenShift(Baremetal) |
|
||||
|
||||
## Prerequisites
|
||||
|
||||
|
|
@ -31,6 +32,7 @@ original_id: openebs-pool-pod-failure
|
|||
dbpassword: k8sDem0
|
||||
dbname: test
|
||||
```
|
||||
|
||||
- Ensure that the chaosServiceAccount used for the experiment has cluster-scope permissions as the experiment may involve carrying out the chaos in the `openebs` namespace
|
||||
while performing application health checks in its respective namespace.
|
||||
|
||||
|
|
@ -72,12 +74,12 @@ If the experiment tunable DATA_PERSISTENCE is set to 'enabled':
|
|||
|
||||
#### Supported Experiment Tunables
|
||||
|
||||
| Variables | Description | Type | Notes |
|
||||
| ----------------------| ------------------------------------------------------------ |-----------|------------------------------------------------------------|
|
||||
| APP_PVC | The PersistentVolumeClaim used by the stateful application | Mandatory | PVC must use OpenEBS cStor storage class |
|
||||
| DEPLOY_TYPE | Type of Kubernetes resource used by the stateful application | Optional | Defaults to `deployment`. Supported: `deployment`, `statefulset`| |
|
||||
| TOTAL_CHAOS_DURATION | Amount of soak time for I/O post pod kill | Optional | Defaults to 600 seconds |
|
||||
| DATA_PERSISTENCE | Flag to perform data consistency checks on the application | Optional | Default value is disabled (empty/unset). Set to `enabled` to perform data checks. Ensure configmap with app details are created |
|
||||
| Variables | Description | Type | Notes |
|
||||
| -------------------- | ------------------------------------------------------------ | --------- | ------------------------------------------------------------------------------------------------------------------------------- | --- |
|
||||
| APP_PVC | The PersistentVolumeClaim used by the stateful application | Mandatory | PVC must use OpenEBS cStor storage class |
|
||||
| DEPLOY_TYPE | Type of Kubernetes resource used by the stateful application | Optional | Defaults to `deployment`. Supported: `deployment`, `statefulset` | |
|
||||
| TOTAL_CHAOS_DURATION | Amount of soak time for I/O post pod kill | Optional | Defaults to 600 seconds |
|
||||
| DATA_PERSISTENCE | Flag to perform data consistency checks on the application | Optional | Default value is disabled (empty/unset). Set to `enabled` to perform data checks. Ensure configmap with app details are created |
|
||||
|
||||
#### Sample ChaosEngine Manifest
|
||||
|
||||
|
|
@ -90,7 +92,7 @@ metadata:
|
|||
spec:
|
||||
appinfo:
|
||||
appns: default
|
||||
applabel: 'app=percona'
|
||||
applabel: "app=percona"
|
||||
appkind: deployment
|
||||
chaosServiceAccount: percona-sa
|
||||
monitoring: false
|
||||
|
|
@ -100,9 +102,9 @@ spec:
|
|||
spec:
|
||||
components:
|
||||
- name: FORCE
|
||||
value: 'true'
|
||||
value: "true"
|
||||
- name: APP_PVC
|
||||
value: 'pvc-c466262a-a5f2-4f0f-b594-5daddfc2e29d'
|
||||
value: "pvc-c466262a-a5f2-4f0f-b594-5daddfc2e29d"
|
||||
- name: DEPLOY_TYPE
|
||||
value: deployment
|
||||
```
|
||||
|
|
@ -1,16 +1,17 @@
|
|||
---
|
||||
id: version-1.0.0-openebs-target-container-failure
|
||||
id: openebs-target-container-failure
|
||||
title: OpenEBS Target Failure Experiment Details
|
||||
sidebar_label: Target Container Failure
|
||||
original_id: openebs-target-container-failure
|
||||
---
|
||||
------
|
||||
|
||||
---
|
||||
|
||||
## Experiment Metadata
|
||||
|
||||
| Type | Description | Tested K8s Platform |
|
||||
| ----------| ------------------------ | ------------------------------------------------------------------|
|
||||
| OpenEBS | Kill the cStor target/Jiva controller container | GKE, Konvoy(AWS), Packet(Kubeadm), Minikube, OpenShift(Baremetal) |
|
||||
| Type | Description | Tested K8s Platform |
|
||||
| ------- | ----------------------------------------------- | ----------------------------------------------------------------- |
|
||||
| OpenEBS | Kill the cStor target/Jiva controller container | GKE, Konvoy(AWS), Packet(Kubeadm), Minikube, OpenShift(Baremetal) |
|
||||
|
||||
## Prerequisites
|
||||
|
||||
|
|
@ -31,6 +32,7 @@ original_id: openebs-target-container-failure
|
|||
dbpassword: k8sDem0
|
||||
dbname: test
|
||||
```
|
||||
|
||||
- Ensure that the chaosServiceAccount used for the experiment has cluster-scope permissions as the experiment may involve carrying out the chaos in the `openebs` namespace
|
||||
while performing application health checks in its respective namespace.
|
||||
|
||||
|
|
@ -75,15 +77,15 @@ If the experiment tunable DATA_PERSISTENCE is set to 'enabled':
|
|||
|
||||
#### Supported Experiment Tunables
|
||||
|
||||
| Variables | Description | Type | Notes |
|
||||
| ----------------------| ------------------------------------------------------------ |-----------|------------------------------------------------------------|
|
||||
| APP_PVC | The PersistentVolumeClaim used by the stateful application | Mandatory | PVC may use either OpenEBS Jiva/cStor storage class |
|
||||
| DEPLOY_TYPE | Type of Kubernetes resource used by the stateful application | Optional | Defaults to `deployment`. Supported: `deployment`, `statefulset`|
|
||||
| CONTAINER_RUNTIME | The container runtime used in the Kubernetes Cluster | Optional | Defaults to `docker`. Supported: `docker`, `containerd` |
|
||||
| LIB_IMAGE | The chaos library image used to run the kill command | Optional | Defaults to `gaiaadm/pumba:0.4.8`. Supported: `{docker : gaiaadm/pumba:0.4.8, containerd: gprasath/crictl:ci}` |
|
||||
| TARGET_CONTAINER | The container to be killed in the storage controller pod | Optional | Defaults to `cstor-volume-mgmt` |
|
||||
| TOTAL_CHAOS_DURATION | Amount of soak time for I/O post container kill | Optional | Defaults to 60 seconds |
|
||||
| DATA_PERSISTENCE | Flag to perform data consistency checks on the application | Optional | Default value is disabled (empty/unset). Set to `enabled` to perform data checks. Ensure configmap with app details are created |
|
||||
| Variables | Description | Type | Notes |
|
||||
| -------------------- | ------------------------------------------------------------ | --------- | ------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| APP_PVC | The PersistentVolumeClaim used by the stateful application | Mandatory | PVC may use either OpenEBS Jiva/cStor storage class |
|
||||
| DEPLOY_TYPE | Type of Kubernetes resource used by the stateful application | Optional | Defaults to `deployment`. Supported: `deployment`, `statefulset` |
|
||||
| CONTAINER_RUNTIME | The container runtime used in the Kubernetes Cluster | Optional | Defaults to `docker`. Supported: `docker`, `containerd` |
|
||||
| LIB_IMAGE | The chaos library image used to run the kill command | Optional | Defaults to `gaiaadm/pumba:0.4.8`. Supported: `{docker : gaiaadm/pumba:0.4.8, containerd: gprasath/crictl:ci}` |
|
||||
| TARGET_CONTAINER | The container to be killed in the storage controller pod | Optional | Defaults to `cstor-volume-mgmt` |
|
||||
| TOTAL_CHAOS_DURATION | Amount of soak time for I/O post container kill | Optional | Defaults to 60 seconds |
|
||||
| DATA_PERSISTENCE | Flag to perform data consistency checks on the application | Optional | Default value is disabled (empty/unset). Set to `enabled` to perform data checks. Ensure configmap with app details are created |
|
||||
|
||||
#### Sample ChaosEngine Manifest
|
||||
|
||||
|
|
@ -96,7 +98,7 @@ metadata:
|
|||
spec:
|
||||
appinfo:
|
||||
appns: default
|
||||
applabel: 'app=percona'
|
||||
applabel: "app=percona"
|
||||
appkind: deployment
|
||||
chaosServiceAccount: percona-sa
|
||||
monitoring: false
|
||||
|
|
@ -106,9 +108,9 @@ spec:
|
|||
spec:
|
||||
components:
|
||||
- name: TARGET_CONTAINER
|
||||
value: 'cstor-istgt'
|
||||
value: "cstor-istgt"
|
||||
- name: APP_PVC
|
||||
value: 'pvc-c466262a-a5f2-4f0f-b594-5daddfc2e29d'
|
||||
value: "pvc-c466262a-a5f2-4f0f-b594-5daddfc2e29d"
|
||||
- name: DEPLOY_TYPE
|
||||
value: deployment
|
||||
```
|
||||
|
|
@ -135,4 +137,3 @@ spec:
|
|||
## OpenEBS Target Container Failure Demo [TODO]
|
||||
|
||||
- A sample recording of this experiment execution is provided here.
|
||||
|
||||
|
|
@ -1,16 +1,17 @@
|
|||
---
|
||||
id: version-1.0.0-openebs-target-network-delay
|
||||
id: openebs-target-network-delay
|
||||
title: OpenEBS Target Network Latency Experiment Details
|
||||
sidebar_label: Target Network Latency
|
||||
original_id: openebs-target-network-delay
|
||||
---
|
||||
------
|
||||
|
||||
---
|
||||
|
||||
## Experiment Metadata
|
||||
|
||||
| Type | Description | Tested K8s Platform |
|
||||
| ----------| ------------------------ | ------------------------------------------------------------------|
|
||||
| OpenEBS | Induce latency into the cStor target/Jiva controller container | GKE, Konvoy(AWS), Packet(Kubeadm), OpenShift(Baremetal) |
|
||||
| Type | Description | Tested K8s Platform |
|
||||
| ------- | -------------------------------------------------------------- | ------------------------------------------------------- |
|
||||
| OpenEBS | Induce latency into the cStor target/Jiva controller container | GKE, Konvoy(AWS), Packet(Kubeadm), OpenShift(Baremetal) |
|
||||
|
||||
## Prerequisites
|
||||
|
||||
|
|
@ -32,6 +33,7 @@ original_id: openebs-target-network-delay
|
|||
dbpassword: k8sDem0
|
||||
dbname: test
|
||||
```
|
||||
|
||||
- Ensure that the chaosServiceAccount used for the experiment has cluster-scope permissions as the experiment may involve carrying out the chaos in the `openebs` namespace
|
||||
while performing application health checks in its respective namespace.
|
||||
|
||||
|
|
@ -75,16 +77,16 @@ If the experiment tunable DATA_PERSISTENCE is set to 'enabled':
|
|||
|
||||
#### Supported Experiment Tunables
|
||||
|
||||
| Variables | Description | Type | Notes |
|
||||
| ----------------------| ------------------------------------------------------------ |-----------|------------------------------------------------------------|
|
||||
| APP_PVC | The PersistentVolumeClaim used by the stateful application | Mandatory | PVC may use either OpenEBS Jiva/cStor storage class |
|
||||
| DEPLOY_TYPE | Type of Kubernetes resource used by the stateful application | Optional | Defaults to `deployment`. Supported: `deployment`, `statefulset`|
|
||||
| CONTAINER_RUNTIME | The container runtime used in the Kubernetes Cluster | Optional | Defaults to `docker`. Supported: `docker` |
|
||||
| LIB_IMAGE | The chaos library image used to inject the latency | Optional | Defaults to `gaiaadm/pumba:0.4.8`. Supported: `gaiaadm/pumba:0.4.8`|
|
||||
| TARGET_CONTAINER | The container into which delays are injected in the storage controller pod | Optional | Defaults to `cstor-istgt` |
|
||||
| NETWORK_DELAY | Egress delay injected into the target container | Optional | Defaults to 60000 milliseconds (60s) |
|
||||
| TOTAL_CHAOS_DURATION | Total duration for which latency is injected | Optional | Defaults to 60000 milliseconds (60s) |
|
||||
| DATA_PERSISTENCE | Flag to perform data consistency checks on the application | Optional | Default value is disabled (empty/unset). Set to `enabled` to perform data checks. Ensure configmap with app details are created |
|
||||
| Variables | Description | Type | Notes |
|
||||
| -------------------- | -------------------------------------------------------------------------- | --------- | ------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| APP_PVC | The PersistentVolumeClaim used by the stateful application | Mandatory | PVC may use either OpenEBS Jiva/cStor storage class |
|
||||
| DEPLOY_TYPE | Type of Kubernetes resource used by the stateful application | Optional | Defaults to `deployment`. Supported: `deployment`, `statefulset` |
|
||||
| CONTAINER_RUNTIME | The container runtime used in the Kubernetes Cluster | Optional | Defaults to `docker`. Supported: `docker` |
|
||||
| LIB_IMAGE | The chaos library image used to inject the latency | Optional | Defaults to `gaiaadm/pumba:0.4.8`. Supported: `gaiaadm/pumba:0.4.8` |
|
||||
| TARGET_CONTAINER | The container into which delays are injected in the storage controller pod | Optional | Defaults to `cstor-istgt` |
|
||||
| NETWORK_DELAY | Egress delay injected into the target container | Optional | Defaults to 60000 milliseconds (60s) |
|
||||
| TOTAL_CHAOS_DURATION | Total duration for which latency is injected | Optional | Defaults to 60000 milliseconds (60s) |
|
||||
| DATA_PERSISTENCE | Flag to perform data consistency checks on the application | Optional | Default value is disabled (empty/unset). Set to `enabled` to perform data checks. Ensure configmap with app details are created |
|
||||
|
||||
#### Sample ChaosEngine Manifest
|
||||
|
||||
|
|
@ -97,7 +99,7 @@ metadata:
|
|||
spec:
|
||||
appinfo:
|
||||
appns: default
|
||||
applabel: 'app=percona'
|
||||
applabel: "app=percona"
|
||||
appkind: deployment
|
||||
chaosServiceAccount: percona-sa
|
||||
monitoring: false
|
||||
|
|
@ -107,15 +109,15 @@ spec:
|
|||
spec:
|
||||
components:
|
||||
- name: TARGET_CONTAINER
|
||||
value: 'cstor-istgt'
|
||||
value: "cstor-istgt"
|
||||
- name: APP_PVC
|
||||
value: 'pvc-c466262a-a5f2-4f0f-b594-5daddfc2e29d'
|
||||
value: "pvc-c466262a-a5f2-4f0f-b594-5daddfc2e29d"
|
||||
- name: DEPLOY_TYPE
|
||||
value: deployment
|
||||
- name: NETWORK_DELAY
|
||||
value: '30000'
|
||||
value: "30000"
|
||||
- name: TOTAL_CHAOS_DURATION
|
||||
value: '60000'
|
||||
value: "60000"
|
||||
```
|
||||
|
||||
### Create the ChaosEngine Resource
|
||||
|
|
@ -1,5 +1,5 @@
|
|||
---
|
||||
id: version-1.0.0-openebs-target-network-loss
|
||||
id: openebs-target-network-loss
|
||||
title: OpenEBS Target Network Loss Experiment Details
|
||||
sidebar_label: Target Network Loss
|
||||
original_id: openebs-target-network-loss
|
||||
|
|
@ -1,16 +1,17 @@
|
|||
---
|
||||
id: version-1.0.0-openebs-target-pod-failure
|
||||
id: openebs-target-pod-failure
|
||||
title: OpenEBS Target Pod Failure Experiment Details
|
||||
sidebar_label: Target Pod Failure
|
||||
original_id: openebs-target-pod-failure
|
||||
---
|
||||
------
|
||||
|
||||
---
|
||||
|
||||
## Experiment Metadata
|
||||
|
||||
| Type | Description | Tested K8s Platform |
|
||||
| ----------| ------------------------ | ------------------------------------------------------------------|
|
||||
| OpenEBS | Kill the cstor/jiva target/controller pod and check if gets created again | GKE, Konvoy(AWS), Packet(Kubeadm), Minikube, OpenShift(Baremetal) |
|
||||
| Type | Description | Tested K8s Platform |
|
||||
| ------- | ------------------------------------------------------------------------- | ----------------------------------------------------------------- |
|
||||
| OpenEBS | Kill the cstor/jiva target/controller pod and check if gets created again | GKE, Konvoy(AWS), Packet(Kubeadm), Minikube, OpenShift(Baremetal) |
|
||||
|
||||
## Prerequisites
|
||||
|
||||
|
|
@ -31,6 +32,7 @@ original_id: openebs-target-pod-failure
|
|||
dbpassword: k8sDem0
|
||||
dbname: test
|
||||
```
|
||||
|
||||
- Ensure that the chaosServiceAccount used for the experiment has cluster-scope permissions as the experiment may involve carrying out the chaos in the `openebs` namespace
|
||||
while performing application health checks in its respective namespace.
|
||||
|
||||
|
|
@ -72,12 +74,12 @@ If the experiment tunable DATA_PERSISTENCE is set to 'enabled':
|
|||
|
||||
#### Supported Experiment Tunables
|
||||
|
||||
| Variables | Description | Type | Notes |
|
||||
| ----------------------| ------------------------------------------------------------ |-----------|------------------------------------------------------------|
|
||||
| APP_PVC | The PersistentVolumeClaim used by the stateful application | Mandatory | PVC may use either OpenEBS Jiva/cStor storage class |
|
||||
| DEPLOY_TYPE | Type of Kubernetes resource used by the stateful application | Optional | Defaults to `deployment`. Supported: `deployment`, `statefulset`| |
|
||||
| TOTAL_CHAOS_DURATION | Amount of soak time for I/O post container kill | Optional | Defaults to 60 seconds |
|
||||
| DATA_PERSISTENCE | Flag to perform data consistency checks on the application | Optional | Default value is disabled (empty/unset). Set to `enabled` to perform data checks. Ensure configmap with app details are created |
|
||||
| Variables | Description | Type | Notes |
|
||||
| -------------------- | ------------------------------------------------------------ | --------- | ------------------------------------------------------------------------------------------------------------------------------- | --- |
|
||||
| APP_PVC | The PersistentVolumeClaim used by the stateful application | Mandatory | PVC may use either OpenEBS Jiva/cStor storage class |
|
||||
| DEPLOY_TYPE | Type of Kubernetes resource used by the stateful application | Optional | Defaults to `deployment`. Supported: `deployment`, `statefulset` | |
|
||||
| TOTAL_CHAOS_DURATION | Amount of soak time for I/O post container kill | Optional | Defaults to 60 seconds |
|
||||
| DATA_PERSISTENCE | Flag to perform data consistency checks on the application | Optional | Default value is disabled (empty/unset). Set to `enabled` to perform data checks. Ensure configmap with app details are created |
|
||||
|
||||
#### Sample ChaosEngine Manifest
|
||||
|
||||
|
|
@ -90,7 +92,7 @@ metadata:
|
|||
spec:
|
||||
appinfo:
|
||||
appns: default
|
||||
applabel: 'app=percona'
|
||||
applabel: "app=percona"
|
||||
appkind: deployment
|
||||
chaosServiceAccount: percona-sa
|
||||
monitoring: false
|
||||
|
|
@ -100,9 +102,9 @@ spec:
|
|||
spec:
|
||||
components:
|
||||
- name: FORCE
|
||||
value: 'true'
|
||||
value: "true"
|
||||
- name: APP_PVC
|
||||
value: 'pvc-c466262a-a5f2-4f0f-b594-5daddfc2e29d'
|
||||
value: "pvc-c466262a-a5f2-4f0f-b594-5daddfc2e29d"
|
||||
- name: DEPLOY_TYPE
|
||||
value: deployment
|
||||
```
|
||||
|
|
@ -0,0 +1,40 @@
|
|||
---
|
||||
id: plugins
|
||||
title: Using other chaos libraries as plugins
|
||||
sidebar_label: Plugins
|
||||
original_id: plugins
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
Litmus provides a way to use any chaos library or a tool to inject chaos. The chaos tool to be compatible with Litmus should satisfy the following requirements:
|
||||
|
||||
- Should be available as a Docker Image
|
||||
- Should take configuration through a `config-map`
|
||||
|
||||
The `plugins` or `chaos-libraries` host the core logic to inject chaos.
|
||||
|
||||
These plugins are hosted at https://github.com/litmuschaos/litmus-ansible/tree/master/chaoslib
|
||||
|
||||
Litmus project has integration into the following chaos-libraries.
|
||||
|
||||
| Chaos Library | Logo | Experiments covered |
|
||||
| ------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------- |
|
||||
| <a href="https://github.com/litmuschaos/litmus-ansible" target="_blank">Litmus</a> | <img src="https://camo.githubusercontent.com/953211f24c1c246f7017703f67b9779e4589bf76/68747470733a2f2f6c616e6473636170652e636e63662e696f2f6c6f676f732f6c69746d75732e737667" width="50"/> | Litmus native chaos libraries that encompasses the chaos capabilities for `pod-kill`, `container-kill`, `cpu-hog` |
|
||||
| <a href="https://github.com/alexei-led/pumba" target="_blank">Pumba</a> | <img src="https://github.com/alexei-led/pumba/raw/master/docs/img/pumba_logo.png" width="50"/> | Pumba provides chaos capabilities for `network-delay` |
|
||||
| <a href="https://github.com/bloomberg/powerfulseal" target="_blank">PowerfulSeal</a> | <img src="https://github.com/bloomberg/powerfulseal/raw/master/media/powerful-seal.png" width="50"/> | PowerfulSeal provides chaos capabilities for `pod-kill` |
|
||||
| | | |
|
||||
|
||||
Usage of plugins is a configuration parameter inside the chaos experiment.
|
||||
|
||||
> Add an example snippet here.
|
||||
|
||||
<br/>
|
||||
|
||||
<br/>
|
||||
|
||||
<hr/>
|
||||
|
||||
<br/>
|
||||
|
||||
<br/>
|
||||
|
|
@ -1,16 +1,17 @@
|
|||
---
|
||||
id: version-1.0.0-pod-cpu-hog
|
||||
id: pod-cpu-hog
|
||||
title: Pod CPU Hog Details
|
||||
sidebar_label: Pod CPU Hog
|
||||
original_id: pod-cpu-hog
|
||||
---
|
||||
------
|
||||
|
||||
---
|
||||
|
||||
## Experiment Metadata
|
||||
|
||||
| Type | Description | Tested K8s Platform |
|
||||
| ----------| -------------------------------------------- | ------------------------------------------------------------------|
|
||||
| Generic | Consume CPU resources on the application container| GKE, Packet(Kubeadm), Minikube |
|
||||
| Type | Description | Tested K8s Platform |
|
||||
| ------- | -------------------------------------------------- | ------------------------------ |
|
||||
| Generic | Consume CPU resources on the application container | GKE, Packet(Kubeadm), Minikube |
|
||||
|
||||
## Prerequisites
|
||||
|
||||
|
|
@ -32,7 +33,6 @@ original_id: pod-cpu-hog
|
|||
- It simulates conditions where app pods experience CPU spikes either due to expected/undesired processes thereby testing how the
|
||||
overall application stack behaves when this occurs.
|
||||
|
||||
|
||||
## Integrations
|
||||
|
||||
- Pod CPU can be effected using the chaos library: `litmus`
|
||||
|
|
@ -66,9 +66,10 @@ metadata:
|
|||
labels:
|
||||
name: nginx-sa
|
||||
rules:
|
||||
- apiGroups: ["","litmuschaos.io","batch"]
|
||||
resources: ["pods","jobs","chaosengines","chaosexperiments","chaosresults"]
|
||||
verbs: ["create","list","get","patch","update","delete"]
|
||||
- apiGroups: ["", "litmuschaos.io", "batch"]
|
||||
resources:
|
||||
["pods", "jobs", "chaosengines", "chaosexperiments", "chaosresults"]
|
||||
verbs: ["create", "list", "get", "patch", "update", "delete"]
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
kind: RoleBinding
|
||||
|
|
@ -81,9 +82,9 @@ roleRef:
|
|||
kind: Role
|
||||
name: nginx-sa
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: nginx-sa
|
||||
namespace: default
|
||||
- kind: ServiceAccount
|
||||
name: nginx-sa
|
||||
namespace: default
|
||||
```
|
||||
|
||||
### Prepare ChaosEngine
|
||||
|
|
@ -138,12 +139,12 @@ metadata:
|
|||
namespace: default
|
||||
spec:
|
||||
# It can be app/infra
|
||||
chaosType: 'app'
|
||||
chaosType: "app"
|
||||
#ex. values: ns1:name=percona,ns2:run=nginx
|
||||
auxiliaryAppInfo: ""
|
||||
appinfo:
|
||||
appns: default
|
||||
applabel: 'app=nginx'
|
||||
applabel: "app=nginx"
|
||||
appkind: deployment
|
||||
chaosServiceAccount: nginx-sa
|
||||
monitoring: false
|
||||
|
|
@ -158,7 +159,7 @@ spec:
|
|||
spec:
|
||||
components:
|
||||
- name: TARGET_CONTAINER
|
||||
value: 'nginx'
|
||||
value: "nginx"
|
||||
#number of cpu cores to be consumed
|
||||
#verify the resources the app has been launched with
|
||||
- name: CPU_CORES
|
||||
|
|
@ -166,7 +167,6 @@ spec:
|
|||
# in ms
|
||||
- name: TOTAL_CHAOS_DURATION
|
||||
value: "60000"
|
||||
|
||||
```
|
||||
|
||||
### Create the ChaosEngine Resource
|
||||
|
|
@ -1,21 +1,22 @@
|
|||
---
|
||||
id: version-1.0.0-pod-delete
|
||||
id: pod-delete
|
||||
title: Pod Delete Experiment Details
|
||||
sidebar_label: Pod Delete
|
||||
original_id: pod-delete
|
||||
---
|
||||
------
|
||||
|
||||
---
|
||||
|
||||
## Experiment Metadata
|
||||
|
||||
| Type | Description | Tested K8s Platform |
|
||||
| ----------| ------------------------ | ------------------------------------------------------------------|
|
||||
| Generic | Fail the application pod | GKE, Konvoy(AWS), Packet(Kubeadm), Minikube |
|
||||
| Type | Description | Tested K8s Platform |
|
||||
| ------- | ------------------------ | ------------------------------------------- |
|
||||
| Generic | Fail the application pod | GKE, Konvoy(AWS), Packet(Kubeadm), Minikube |
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Ensure that the Litmus Chaos Operator is running by executing `kubectl get pods` in operator namespace (typically, `litmus`).If not, install from [here](https://raw.githubusercontent.com/litmuschaos/pages/master/docs/litmus-operator-latest.yaml)
|
||||
- Ensure that the `pod-delete` experiment resource is available in the cluster by executing `kubectl get chaosexperiments` in the desired namespace. If not, install from [here](https://hub.litmuschaos.io/charts/generic/experiments/pod-delete)
|
||||
- Ensure that the `pod-delete` experiment resource is available in the cluster by executing `kubectl get chaosexperiments` in the desired namespace. If not, install from [here](https://hub.litmuschaos.io/charts/generic/experiments/pod-delete)
|
||||
|
||||
## Entry Criteria
|
||||
|
||||
|
|
@ -65,12 +66,21 @@ metadata:
|
|||
labels:
|
||||
name: nginx-sa
|
||||
rules:
|
||||
- apiGroups: ["","litmuschaos.io","batch","apps"]
|
||||
resources: ["pods","deployments","jobs","configmaps","chaosengines","chaosexperiments","chaosresults"]
|
||||
verbs: ["create","list","get","patch","update","delete"]
|
||||
- apiGroups: [""]
|
||||
resources: ["nodes"]
|
||||
verbs : ["get","list"]
|
||||
- apiGroups: ["", "litmuschaos.io", "batch", "apps"]
|
||||
resources:
|
||||
[
|
||||
"pods",
|
||||
"deployments",
|
||||
"jobs",
|
||||
"configmaps",
|
||||
"chaosengines",
|
||||
"chaosexperiments",
|
||||
"chaosresults",
|
||||
]
|
||||
verbs: ["create", "list", "get", "patch", "update", "delete"]
|
||||
- apiGroups: [""]
|
||||
resources: ["nodes"]
|
||||
verbs: ["get", "list"]
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
kind: RoleBinding
|
||||
|
|
@ -83,10 +93,9 @@ roleRef:
|
|||
kind: Role
|
||||
name: nginx-sa
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: nginx-sa
|
||||
namespace: default
|
||||
|
||||
- kind: ServiceAccount
|
||||
name: nginx-sa
|
||||
namespace: default
|
||||
```
|
||||
|
||||
### Prepare ChaosEngine
|
||||
|
|
@ -96,14 +105,13 @@ subjects:
|
|||
|
||||
#### Supported Experiment Tunables
|
||||
|
||||
| Variables | Description | Type | Notes |
|
||||
| ----------------------| ------------------------------------------------------------ |-----------|------------------------------------------------------------|
|
||||
| TOTAL_CHAOS_DURATION | The time duration for chaos insertion (seconds) | Optional | Defaults to 15s |
|
||||
| CHAOS_INTERVAL | Time interval b/w two successive pod failures (sec) | Optional | Defaults to 5s |
|
||||
| LIB | The chaos lib used to inject the chaos | Optional | Defaults to `litmus`. Supported: `litmus`, `powerfulseal` |
|
||||
| FORCE | Application Pod failures type | Optional | Default to `true`, With `terminationGracePeriodSeconds=0` |
|
||||
| KILL_COUNT | No. of application pods to be deleted | Optional | Default to `1`, kill_count > 1 is only supported by litmus lib , not by the powerfulseal |
|
||||
|
||||
| Variables | Description | Type | Notes |
|
||||
| -------------------- | --------------------------------------------------- | -------- | ---------------------------------------------------------------------------------------- |
|
||||
| TOTAL_CHAOS_DURATION | The time duration for chaos insertion (seconds) | Optional | Defaults to 15s |
|
||||
| CHAOS_INTERVAL | Time interval b/w two successive pod failures (sec) | Optional | Defaults to 5s |
|
||||
| LIB | The chaos lib used to inject the chaos | Optional | Defaults to `litmus`. Supported: `litmus`, `powerfulseal` |
|
||||
| FORCE | Application Pod failures type | Optional | Default to `true`, With `terminationGracePeriodSeconds=0` |
|
||||
| KILL_COUNT | No. of application pods to be deleted | Optional | Default to `1`, kill_count > 1 is only supported by litmus lib , not by the powerfulseal |
|
||||
|
||||
#### Sample ChaosEngine Manifest
|
||||
|
||||
|
|
@ -116,10 +124,10 @@ metadata:
|
|||
spec:
|
||||
appinfo:
|
||||
appns: default
|
||||
applabel: 'app=nginx'
|
||||
applabel: "app=nginx"
|
||||
appkind: deployment
|
||||
# It can be app/infra
|
||||
chaosType: 'app'
|
||||
chaosType: "app"
|
||||
#ex. values: ns1:name=percona,ns2:run=nginx
|
||||
auxiliaryAppInfo: ""
|
||||
chaosServiceAccount: nginx-sa
|
||||
|
|
@ -134,12 +142,12 @@ spec:
|
|||
- name: pod-delete
|
||||
spec:
|
||||
components:
|
||||
# set chaos duration (in sec) as desired
|
||||
# set chaos duration (in sec) as desired
|
||||
- name: TOTAL_CHAOS_DURATION
|
||||
value: '30'
|
||||
value: "30"
|
||||
# set chaos interval (in sec) as desired
|
||||
- name: CHAOS_INTERVAL
|
||||
value: '10'
|
||||
value: "10"
|
||||
# pod failures without '--force' & default terminationGracePeriodSeconds
|
||||
- name: FORCE
|
||||
value: "false"
|
||||
|
|
@ -1,21 +1,23 @@
|
|||
---
|
||||
id: version-1.0.0-pod-network-corruption
|
||||
id: pod-network-corruption
|
||||
title: Pod Network Corruption Experiment Details
|
||||
sidebar_label: Pod Network Corruption
|
||||
original_id: pod-network-corruption
|
||||
---
|
||||
------
|
||||
|
||||
---
|
||||
|
||||
## Experiment Metadata
|
||||
|
||||
| Type | Description | Tested K8s Platform |
|
||||
| ----------| ------------------------ | ------------------------------------------------------------------|
|
||||
| Generic | Inject Network Packet Corruption Into Application Pod | GKE, Packet(Kubeadm), Minikube > v1.6.0 |
|
||||
| Type | Description | Tested K8s Platform |
|
||||
| ------- | ----------------------------------------------------- | --------------------------------------- |
|
||||
| Generic | Inject Network Packet Corruption Into Application Pod | GKE, Packet(Kubeadm), Minikube > v1.6.0 |
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Ensure that the Litmus Chaos Operator is running by executing `kubectl get pods` in operator namespace (typically, `litmus`). If not, install from [here](https://raw.githubusercontent.com/litmuschaos/pages/master/docs/litmus-operator-latest.yaml)
|
||||
- Ensure that the `pod-network-corruption` experiment resource is available in the cluster by `kubectl get chaosexperiments` command. If not, install from [here](https://hub.litmuschaos.io/charts/generic/experiments/pod-network-corruption)
|
||||
- Cluster must run docker container runtime
|
||||
- Cluster must run docker container runtime
|
||||
|
||||
<div class="danger">
|
||||
<strong>NOTE</strong>:
|
||||
|
|
@ -32,7 +34,7 @@ original_id: pod-network-corruption
|
|||
|
||||
## Details
|
||||
|
||||
- The application pod should be healthy once chaos is stopped. Service-requests should be served despite chaos.
|
||||
- The application pod should be healthy once chaos is stopped. Service-requests should be served despite chaos.
|
||||
- Injects packet corruption on the specified container by starting a traffic control (tc) process with netem rules to add egress packet corruption
|
||||
- Corruption is injected via pumba library with command pumba netem corruption by passing the relevant network interface, packet-corruption-percentage, chaos duration and regex filter for container name
|
||||
- Can test the application's resilience to lossy/flaky network
|
||||
|
|
@ -66,9 +68,10 @@ metadata:
|
|||
labels:
|
||||
name: nginx-sa
|
||||
rules:
|
||||
- apiGroups: ["","litmuschaos.io","batch"]
|
||||
resources: ["pods","jobs","chaosengines","chaosexperiments","chaosresults"]
|
||||
verbs: ["create","list","get","patch","update","delete"]
|
||||
- apiGroups: ["", "litmuschaos.io", "batch"]
|
||||
resources:
|
||||
["pods", "jobs", "chaosengines", "chaosexperiments", "chaosresults"]
|
||||
verbs: ["create", "list", "get", "patch", "update", "delete"]
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
kind: RoleBinding
|
||||
|
|
@ -81,10 +84,9 @@ roleRef:
|
|||
kind: Role
|
||||
name: nginx-sa
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: nginx-sa
|
||||
namespace: default
|
||||
|
||||
- kind: ServiceAccount
|
||||
name: nginx-sa
|
||||
namespace: default
|
||||
```
|
||||
|
||||
### Prepare ChaosEngine
|
||||
|
|
@ -94,16 +96,16 @@ subjects:
|
|||
|
||||
#### Supported Experiment Tunables
|
||||
|
||||
| Variables | Description | Type | Notes |
|
||||
| ----------------------| ------------------------------------------------------------ |-----------|------------------------------------------------------------|
|
||||
| NETWORK_INTERFACE | Name of ethernet interface considered for shaping traffic | Mandatory | |
|
||||
| TARGET_CONTAINER | Name of container which is subjected to network latency | Mandatory | |
|
||||
| NETWORK_PACKET_CORRUPTION_PERCENTAGE | Packet corruption in percentage | Mandatory | Default (100)
|
||||
| LIB | The chaos lib used to inject the chaos eg. Pumba | Optional | only `pumba` supported currently |
|
||||
| CHAOSENGINE | ChaosEngine CR name associated with the experiment instance | Optional | |
|
||||
| CHAOS_SERVICE_ACCOUNT | Service account used by the pumba daemonset Optional | Optional | |
|
||||
| TOTAL_CHAOS_DURATION | The time duration for chaos insertion in milliseconds | Optional| Default (60000ms)|
|
||||
| LIB_IMAGE | The pumba image used to run the kill command | Optional | Defaults to `gaiaadm/pumba:0.6.5` |
|
||||
| Variables | Description | Type | Notes |
|
||||
| ------------------------------------ | ----------------------------------------------------------- | --------- | --------------------------------- |
|
||||
| NETWORK_INTERFACE | Name of ethernet interface considered for shaping traffic | Mandatory | |
|
||||
| TARGET_CONTAINER | Name of container which is subjected to network latency | Mandatory | |
|
||||
| NETWORK_PACKET_CORRUPTION_PERCENTAGE | Packet corruption in percentage | Mandatory | Default (100) |
|
||||
| LIB | The chaos lib used to inject the chaos eg. Pumba | Optional | only `pumba` supported currently |
|
||||
| CHAOSENGINE | ChaosEngine CR name associated with the experiment instance | Optional | |
|
||||
| CHAOS_SERVICE_ACCOUNT | Service account used by the pumba daemonset Optional | Optional | |
|
||||
| TOTAL_CHAOS_DURATION | The time duration for chaos insertion in milliseconds | Optional | Default (60000ms) |
|
||||
| LIB_IMAGE | The pumba image used to run the kill command | Optional | Defaults to `gaiaadm/pumba:0.6.5` |
|
||||
|
||||
#### Sample ChaosEngine Manifest
|
||||
|
||||
|
|
@ -111,13 +113,13 @@ subjects:
|
|||
apiVersion: litmuschaos.io/v1alpha1
|
||||
kind: ChaosEngine
|
||||
metadata:
|
||||
name: nginx-network-chaos
|
||||
name: nginx-network-chaos
|
||||
namespace: default
|
||||
spec:
|
||||
# It can be delete/retain
|
||||
jobCleanUpPolicy: delete
|
||||
# It can be app/infra
|
||||
chaosType: 'app'
|
||||
chaosType: "app"
|
||||
#ex. values: ns1:name=percona,ns2:run=nginx
|
||||
auxiliaryAppInfo: ""
|
||||
monitoring: false
|
||||
|
|
@ -135,15 +137,16 @@ spec:
|
|||
- name: pod-network-corruption
|
||||
spec:
|
||||
components:
|
||||
- name: ANSIBLE_STDOUT_CALLBACK
|
||||
value: default
|
||||
- name: TARGET_CONTAINER
|
||||
#Container name where chaos has to be injected
|
||||
value: "nginx"
|
||||
- name: NETWORK_INTERFACE
|
||||
#Network interface inside target container
|
||||
value: eth0
|
||||
- name: ANSIBLE_STDOUT_CALLBACK
|
||||
value: default
|
||||
- name: TARGET_CONTAINER
|
||||
#Container name where chaos has to be injected
|
||||
value: "nginx"
|
||||
- name: NETWORK_INTERFACE
|
||||
#Network interface inside target container
|
||||
value: eth0
|
||||
```
|
||||
|
||||
### Create the ChaosEngine Resource
|
||||
|
||||
- Create the ChaosEngine manifest prepared in the previous step to trigger the Chaos.
|
||||
|
|
@ -162,7 +165,6 @@ spec:
|
|||
|
||||
`kubectl describe chaosresult <ChaosEngine-Name>-<ChaosExperiment-Name> -n <application-namespace>`
|
||||
|
||||
|
||||
## Application Pod Network Packet Corruption Demo
|
||||
|
||||
- A sample recording of this experiment execution is provided [here](https://youtu.be/kSiLrIaILvs).
|
||||
|
|
@ -1,22 +1,23 @@
|
|||
---
|
||||
id: version-1.0.0-pod-network-latency
|
||||
id: pod-network-latency
|
||||
title: Pod Network Latency Experiment Details
|
||||
sidebar_label: Pod Network Latency
|
||||
original_id: pod-network-latency
|
||||
---
|
||||
------
|
||||
|
||||
---
|
||||
|
||||
## Experiment Metadata
|
||||
|
||||
| Type | Description | Tested K8s Platform |
|
||||
| ----------| ------------------------ | ------------------------------------------------------------------|
|
||||
| Generic | Inject Network Latency Into Application Pod | GKE, Packet(Kubeadm), Minikube > v1.6.0 |
|
||||
| Type | Description | Tested K8s Platform |
|
||||
| ------- | ------------------------------------------- | --------------------------------------- |
|
||||
| Generic | Inject Network Latency Into Application Pod | GKE, Packet(Kubeadm), Minikube > v1.6.0 |
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Ensure that the Litmus Chaos Operator is running by executing `kubectl get pods` in operator namespace (typically, `litmus`). If not, install from [here](https://raw.githubusercontent.com/litmuschaos/pages/master/docs/litmus-operator-latest.yaml)
|
||||
- Ensure that the `pod-network-latency` experiment resource is available in the cluster by executing kubectl `get chaosexperiments` in the desired namespace. . If not, install from [here](https://hub.litmuschaos.io/charts/generic/experiments/pod-network-latency)
|
||||
|
||||
|
||||
<div class="danger">
|
||||
<strong>NOTE</strong>:
|
||||
Experiment is supported only on Docker Runtime. Support for containerd/CRIO runtimes will be added in subsequent releases.
|
||||
|
|
@ -32,10 +33,10 @@ original_id: pod-network-latency
|
|||
|
||||
## Details
|
||||
|
||||
- The application pod should be healthy once chaos is stopped. Service-requests should be served despite chaos.
|
||||
- The application pod should be healthy once chaos is stopped. Service-requests should be served despite chaos.
|
||||
- Causes flaky access to application replica by injecting network delay using pumba.
|
||||
- Injects latency on the specified container by starting a traffic control (tc) process with netem rules to add egress delays
|
||||
- Latency is injected via pumba library with command pumba netem delay by passing the relevant network interface, latency, chaos duration and regex filter for container name
|
||||
- Injects latency on the specified container by starting a traffic control (tc) process with netem rules to add egress delays
|
||||
- Latency is injected via pumba library with command pumba netem delay by passing the relevant network interface, latency, chaos duration and regex filter for container name
|
||||
- Can test the application's resilience to lossy/flaky network
|
||||
|
||||
## Steps to Execute the Chaos Experiment
|
||||
|
|
@ -67,9 +68,10 @@ metadata:
|
|||
labels:
|
||||
name: nginx-sa
|
||||
rules:
|
||||
- apiGroups: ["","litmuschaos.io","batch"]
|
||||
resources: ["pods","jobs","chaosengines","chaosexperiments","chaosresults"]
|
||||
verbs: ["create","list","get","patch","update","delete"]
|
||||
- apiGroups: ["", "litmuschaos.io", "batch"]
|
||||
resources:
|
||||
["pods", "jobs", "chaosengines", "chaosexperiments", "chaosresults"]
|
||||
verbs: ["create", "list", "get", "patch", "update", "delete"]
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
kind: RoleBinding
|
||||
|
|
@ -82,10 +84,9 @@ roleRef:
|
|||
kind: Role
|
||||
name: nginx-sa
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: nginx-sa
|
||||
namespace: default
|
||||
|
||||
- kind: ServiceAccount
|
||||
name: nginx-sa
|
||||
namespace: default
|
||||
```
|
||||
|
||||
### Prepare ChaosEngine
|
||||
|
|
@ -95,15 +96,15 @@ subjects:
|
|||
|
||||
#### Supported Experiment Tunables
|
||||
|
||||
| Variables | Description | Type | Notes |
|
||||
| ----------------------| ------------------------------------------------------------ |-----------|------------------------------------------------------------|
|
||||
| NETWORK_INTERFACE | Name of ethernet interface considered for shaping traffic | Mandatory | |
|
||||
| TARGET_CONTAINER | Name of container which is subjected to network latency | Mandatory | |
|
||||
| TOTAL_CHAOS_DURATION | The time duration for chaos insertion in milliseconds | Optional| Default (60000ms)|
|
||||
| NETWORK_LATENCY | The latency/delay in milliseconds | Optional | Default (60000ms)
|
||||
| LIB | The chaos lib used to inject the chaos eg. Pumba | Optional | |
|
||||
| CHAOSENGINE | ChaosEngine CR name associated with the experiment instance | Optional | |
|
||||
| CHAOS_SERVICE_ACCOUNT | Service account used by the pumba daemonset Optional | Optional | |
|
||||
| Variables | Description | Type | Notes |
|
||||
| --------------------- | ----------------------------------------------------------- | --------- | ----------------- |
|
||||
| NETWORK_INTERFACE | Name of ethernet interface considered for shaping traffic | Mandatory | |
|
||||
| TARGET_CONTAINER | Name of container which is subjected to network latency | Mandatory | |
|
||||
| TOTAL_CHAOS_DURATION | The time duration for chaos insertion in milliseconds | Optional | Default (60000ms) |
|
||||
| NETWORK_LATENCY | The latency/delay in milliseconds | Optional | Default (60000ms) |
|
||||
| LIB | The chaos lib used to inject the chaos eg. Pumba | Optional | |
|
||||
| CHAOSENGINE | ChaosEngine CR name associated with the experiment instance | Optional | |
|
||||
| CHAOS_SERVICE_ACCOUNT | Service account used by the pumba daemonset Optional | Optional | |
|
||||
|
||||
#### Sample ChaosEngine Manifest
|
||||
|
||||
|
|
@ -111,13 +112,13 @@ subjects:
|
|||
apiVersion: litmuschaos.io/v1alpha1
|
||||
kind: ChaosEngine
|
||||
metadata:
|
||||
name: nginx-network-chaos
|
||||
name: nginx-network-chaos
|
||||
namespace: default
|
||||
spec:
|
||||
# It can be delete/retain
|
||||
jobCleanUpPolicy: delete
|
||||
# It can be app/infra
|
||||
chaosType: 'app'
|
||||
chaosType: "app"
|
||||
#ex. values: ns1:name=percona,ns2:run=nginx
|
||||
auxiliaryAppInfo: ""
|
||||
monitoring: false
|
||||
|
|
@ -135,23 +136,24 @@ spec:
|
|||
- name: pod-network-latency
|
||||
spec:
|
||||
components:
|
||||
- name: ANSIBLE_STDOUT_CALLBACK
|
||||
value: default
|
||||
- name: TARGET_CONTAINER
|
||||
#Container name where chaos has to be injected
|
||||
value: "nginx"
|
||||
- name: NETWORK_INTERFACE
|
||||
#Network interface inside target container
|
||||
value: eth0
|
||||
- name: LIB_IMAGE
|
||||
value: gaiaadm/pumba:0.6.5
|
||||
- name: NETWORK_LATENCY
|
||||
value: "2000"
|
||||
- name: TOTAL_CHAOS_DURATION
|
||||
value: "60000"
|
||||
- name: LIB
|
||||
value: pumba
|
||||
- name: ANSIBLE_STDOUT_CALLBACK
|
||||
value: default
|
||||
- name: TARGET_CONTAINER
|
||||
#Container name where chaos has to be injected
|
||||
value: "nginx"
|
||||
- name: NETWORK_INTERFACE
|
||||
#Network interface inside target container
|
||||
value: eth0
|
||||
- name: LIB_IMAGE
|
||||
value: gaiaadm/pumba:0.6.5
|
||||
- name: NETWORK_LATENCY
|
||||
value: "2000"
|
||||
- name: TOTAL_CHAOS_DURATION
|
||||
value: "60000"
|
||||
- name: LIB
|
||||
value: pumba
|
||||
```
|
||||
|
||||
### Create the ChaosEngine Resource
|
||||
|
||||
- Create the ChaosEngine manifest prepared in the previous step to trigger the Chaos.
|
||||
|
|
@ -170,7 +172,6 @@ spec:
|
|||
|
||||
`kubectl describe chaosresult <ChaosEngine-Name>-<ChaosExperiment-Name> -n <application-namespace>`
|
||||
|
||||
|
||||
## Application Pod Network Latency Demo
|
||||
|
||||
- A sample recording of this experiment execution is provided [here](https://youtu.be/QsQZyXVCcCw).
|
||||
|
|
@ -1,21 +1,22 @@
|
|||
---
|
||||
id: version-1.0.0-pod-network-loss
|
||||
id: pod-network-loss
|
||||
title: Pod Network Loss Experiment Details
|
||||
sidebar_label: Pod Network Loss
|
||||
original_id: pod-network-loss
|
||||
---
|
||||
------
|
||||
|
||||
---
|
||||
|
||||
## Experiment Metadata
|
||||
|
||||
| Type | Description | Tested K8s Platform |
|
||||
| ----------| ------------------------ | ------------------------------------------------------------------|
|
||||
| Generic | Inject Packet Loss Into Application Pod | GKE, Packet(Kubeadm), Minikube > v1.6.0 |
|
||||
| Type | Description | Tested K8s Platform |
|
||||
| ------- | --------------------------------------- | --------------------------------------- |
|
||||
| Generic | Inject Packet Loss Into Application Pod | GKE, Packet(Kubeadm), Minikube > v1.6.0 |
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Ensure that the Litmus Chaos Operator is running by executing `kubectl get pods` in operator namespace (typically, `litmus`). If not, install from [here](https://raw.githubusercontent.com/litmuschaos/pages/master/docs/litmus-operator-latest.yaml)
|
||||
- Ensure that the `pod-network-loss` experiment resource is available in the cluster by executing `kubectl get chaosexperiments` in the desired namespace. If not, install from [here](https://hub.litmuschaos.io/charts/generic/experiments/pod-network-loss)
|
||||
- Ensure that the `pod-network-loss` experiment resource is available in the cluster by executing `kubectl get chaosexperiments` in the desired namespace. If not, install from [here](https://hub.litmuschaos.io/charts/generic/experiments/pod-network-loss)
|
||||
<div class="danger">
|
||||
<strong>NOTE</strong>:
|
||||
Experiment is supported only on Docker Runtime. Support for containerd/CRIO runtimes will be added in subsequent releases.
|
||||
|
|
@ -32,10 +33,9 @@ original_id: pod-network-loss
|
|||
## Details
|
||||
|
||||
- Pod-network-loss injects chaos to disrupt network connectivity to kubernetes pods.
|
||||
- The application pod should be healthy once chaos is stopped. Service-requests should be served despite chaos.
|
||||
- The application pod should be healthy once chaos is stopped. Service-requests should be served despite chaos.
|
||||
- Causes loss of access to application replica by injecting packet loss using pumba
|
||||
|
||||
|
||||
## Steps to Execute the Chaos Experiment
|
||||
|
||||
- This Chaos Experiment can be triggered by creating a ChaosEngine resource on the cluster. To understand the values to provide in a ChaosEngine specification, refer [Getting Started](getstarted.md/#prepare-chaosengine)
|
||||
|
|
@ -65,9 +65,10 @@ metadata:
|
|||
labels:
|
||||
name: nginx-sa
|
||||
rules:
|
||||
- apiGroups: ["","litmuschaos.io","batch"]
|
||||
resources: ["pods","jobs","chaosengines","chaosexperiments","chaosresults"]
|
||||
verbs: ["create","list","get","patch","update","delete"]
|
||||
- apiGroups: ["", "litmuschaos.io", "batch"]
|
||||
resources:
|
||||
["pods", "jobs", "chaosengines", "chaosexperiments", "chaosresults"]
|
||||
verbs: ["create", "list", "get", "patch", "update", "delete"]
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
kind: RoleBinding
|
||||
|
|
@ -80,10 +81,9 @@ roleRef:
|
|||
kind: Role
|
||||
name: nginx-sa
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: nginx-sa
|
||||
namespace: default
|
||||
|
||||
- kind: ServiceAccount
|
||||
name: nginx-sa
|
||||
namespace: default
|
||||
```
|
||||
|
||||
### Prepare ChaosEngine
|
||||
|
|
@ -93,16 +93,16 @@ subjects:
|
|||
|
||||
#### Supported Experiment Tunables
|
||||
|
||||
| Variables | Description | Type | Notes |
|
||||
| ----------------------| ------------------------------------------------------------ |-----------|------------------------------------------------------------|
|
||||
| NETWORK_INTERFACE | Name of ethernet interface considered for shaping traffic | Mandatory | |
|
||||
| TARGET_CONTAINER | Name of container which is subjected to network latency | Mandatory | |
|
||||
| NETWORK_PACKET_LOSS_PERCENTAGE | The packet loss in percentage | Mandatory | |
|
||||
| TOTAL_CHAOS_DURATION | The time duration for chaos insertion in milliseconds | Optional | Default (60000ms) |
|
||||
| LIB | The chaos lib used to inject the chaos eg. Pumba | Optional | |
|
||||
| LIB_IMAGE | The image used by the chaoslib to inject the chaos | Optional | Default: `gaiaadm/pumba:0.6.5` |
|
||||
| CHAOSENGINE | ChaosEngine CR name associated with the experiment instance | Optional | |
|
||||
| CHAOS_SERVICE_ACCOUNT | Service account used by the pumba daemonset Optional | Optional | |
|
||||
| Variables | Description | Type | Notes |
|
||||
| ------------------------------ | ----------------------------------------------------------- | --------- | ------------------------------ |
|
||||
| NETWORK_INTERFACE | Name of ethernet interface considered for shaping traffic | Mandatory | |
|
||||
| TARGET_CONTAINER | Name of container which is subjected to network latency | Mandatory | |
|
||||
| NETWORK_PACKET_LOSS_PERCENTAGE | The packet loss in percentage | Mandatory | |
|
||||
| TOTAL_CHAOS_DURATION | The time duration for chaos insertion in milliseconds | Optional | Default (60000ms) |
|
||||
| LIB | The chaos lib used to inject the chaos eg. Pumba | Optional | |
|
||||
| LIB_IMAGE | The image used by the chaoslib to inject the chaos | Optional | Default: `gaiaadm/pumba:0.6.5` |
|
||||
| CHAOSENGINE | ChaosEngine CR name associated with the experiment instance | Optional | |
|
||||
| CHAOS_SERVICE_ACCOUNT | Service account used by the pumba daemonset Optional | Optional | |
|
||||
|
||||
#### Sample ChaosEngine Manifest
|
||||
|
||||
|
|
@ -117,7 +117,7 @@ spec:
|
|||
# It can be delete/retain
|
||||
jobCleanUpPolicy: delete
|
||||
# It can be app/infra
|
||||
chaosType: 'app'
|
||||
chaosType: "app"
|
||||
#ex. values: ns1:name=percona,ns2:run=nginx
|
||||
auxiliaryAppInfo: ""
|
||||
monitoring: false
|
||||
|
|
@ -135,23 +135,24 @@ spec:
|
|||
- name: pod-network-loss
|
||||
spec:
|
||||
components:
|
||||
- name: ANSIBLE_STDOUT_CALLBACK
|
||||
value: default
|
||||
- name: TARGET_CONTAINER
|
||||
#Container name where chaos has to be injected
|
||||
value: "nginx"
|
||||
- name: LIB_IMAGE
|
||||
value: gaiaadm/pumba:0.6.5
|
||||
- name: NETWORK_INTERFACE
|
||||
#Network interface inside target container
|
||||
value: eth0
|
||||
- name: NETWORK_PACKET_LOSS_PERCENTAGE
|
||||
value: "100"
|
||||
- name: TOTAL_CHAOS_DURATION
|
||||
value: "60000"
|
||||
- name: LIB
|
||||
value: pumba
|
||||
- name: ANSIBLE_STDOUT_CALLBACK
|
||||
value: default
|
||||
- name: TARGET_CONTAINER
|
||||
#Container name where chaos has to be injected
|
||||
value: "nginx"
|
||||
- name: LIB_IMAGE
|
||||
value: gaiaadm/pumba:0.6.5
|
||||
- name: NETWORK_INTERFACE
|
||||
#Network interface inside target container
|
||||
value: eth0
|
||||
- name: NETWORK_PACKET_LOSS_PERCENTAGE
|
||||
value: "100"
|
||||
- name: TOTAL_CHAOS_DURATION
|
||||
value: "60000"
|
||||
- name: LIB
|
||||
value: pumba
|
||||
```
|
||||
|
||||
### Create the ChaosEngine Resource
|
||||
|
||||
- Create the ChaosEngine manifest prepared in the previous step to trigger the Chaos.
|
||||
|
|
@ -170,7 +171,6 @@ spec:
|
|||
|
||||
`kubectl describe chaosresult <ChaosEngine-Name>-<ChaosExperiment-Name> -n <application-namespace>`
|
||||
|
||||
|
||||
## Application Pod Network Loss Demo
|
||||
|
||||
- A sample recording of this experiment execution is provided [here](https://youtu.be/jqvYy-nWc_I).
|
||||
|
|
@ -1,42 +1,40 @@
|
|||
---
|
||||
id: version-1.0.0-resources
|
||||
id: resources
|
||||
title: Resources related to Chaos Engineering on Kubernetes
|
||||
sidebar_label: Resources
|
||||
original_id: resources
|
||||
---
|
||||
------
|
||||
|
||||
---
|
||||
|
||||
## Chaos Demos
|
||||
|
||||
|
||||
|
||||
### Getting Started
|
||||
|
||||
Use this video to learn how to get started with Litmus. You will learn how to install Litmus, how to inject a fault into your application using one of the experiments available at ChaosHub.
|
||||
|
||||
<a href="https://asciinema.org/a/G9TcXpgikLuGTBY7btIUNSuWN" target="_blank">
|
||||
|
||||
<img src="/docs/assets/getstarted.svg" width="300">
|
||||
<img src={require("./assets/getstarted.svg").default} width="300"/>
|
||||
|
||||
</a>
|
||||
|
||||
<hr>
|
||||
<hr/>
|
||||
|
||||
## Reference Implementations
|
||||
|
||||
| Reference | Description |
|
||||
| ------------------ | ------------------------------------------------------------ |
|
||||
| Reference | Description |
|
||||
| ------------------ | ---------------------------------------------------------------------- |
|
||||
| https://openebs.ci | CNCF SandBox project uses Litmus chaos experiments in its CI pipelines |
|
||||
| | |
|
||||
| | |
|
||||
| | |
|
||||
| | |
|
||||
|
||||
<br>
|
||||
<br/>
|
||||
|
||||
<br>
|
||||
<br/>
|
||||
|
||||
<hr>
|
||||
<hr/>
|
||||
|
||||
<br>
|
||||
|
||||
<br>
|
||||
<br/>
|
||||
|
||||
<br/>
|
||||
|
|
@ -1,12 +1,13 @@
|
|||
---
|
||||
id: version-1.5.0-architecture
|
||||
original_id: architecture
|
||||
id: architecture
|
||||
title: Litmus Architecture
|
||||
sidebar_label: Architecture
|
||||
original_id: architecture
|
||||
---
|
||||
<hr>
|
||||
|
||||
<img src="/docs/assets/litmus-schematic.png" width="800">
|
||||
<hr />
|
||||
|
||||
<img src={require("./assets/architecture.png").default} width="800" />
|
||||
|
||||
**Chaos-Operator**
|
||||
|
||||
|
|
@ -22,34 +23,24 @@ During installation, the following three CRDs are installed on the Kubernetes cl
|
|||
|
||||
`chaosresults.litmuschaos.io`
|
||||
|
||||
|
||||
|
||||
**Chaos-Experiments**
|
||||
|
||||
Chaos Experiment is a CR and are available as YAML files on <a href=" https://hub.litmuschaos.io" target="_blank">Chaos Hub</a>. For more details visit Chaos Hub [documentation](chaoshub.md).
|
||||
|
||||
|
||||
Chaos Experiment is a CR and are available as YAML files on <a href="https://hub.litmuschaos.io" target="_blank">Chaos Hub</a>. For more details visit Chaos Hub [documentation](chaoshub.md).
|
||||
|
||||
**Chaos-Engine**
|
||||
|
||||
ChaosEngine CR links application to experiments. User has to create ChaosEngine YAML by specifying the application label and experiments and create the CR. The CR is watched by Chaos-Operator and chaos-experiments are executed on a given application.
|
||||
|
||||
|
||||
|
||||
**Chaos-Exporter**
|
||||
|
||||
Optionally metrics can be exported to a Prometheus database. Chaos-Exporter implements the Prometheus metrics endpoint.
|
||||
Optionally metrics can be exported to a Prometheus database. Chaos-Exporter implements the Prometheus metrics endpoint.
|
||||
|
||||
<br />
|
||||
|
||||
<br />
|
||||
|
||||
<br>
|
||||
|
||||
<br>
|
||||
|
||||
<hr>
|
||||
|
||||
<br>
|
||||
|
||||
<br>
|
||||
<hr />
|
||||
|
||||
<br />
|
||||
|
||||
<br />
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 33 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 15 KiB |
|
|
@ -0,0 +1,429 @@
|
|||
<?xml version="1.0"?>
|
||||
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="1349.33333328" height="862.4" font-family="Consolas, Menlo, 'Bitstream Vera Sans Mono', monospace, 'Powerline Symbols'" font-size="14px">
|
||||
<style>
|
||||
<!-- asciinema theme -->
|
||||
|
||||
.default-text-fill {fill: #cccccc}
|
||||
.default-bg-fill {fill: #121314}
|
||||
|
||||
.c-0 {fill: #000000}
|
||||
.c-1 {fill: #dd3c69}
|
||||
.c-2 {fill: #4ebf22}
|
||||
.c-3 {fill: #ddaf3c}
|
||||
.c-4 {fill: #26b0d7}
|
||||
.c-5 {fill: #b954e1}
|
||||
.c-6 {fill: #54e1b9}
|
||||
.c-7 {fill: #d9d9d9}
|
||||
.c-8 {fill: #4d4d4d}
|
||||
.c-9 {fill: #dd3c69}
|
||||
.c-10 {fill: #4ebf22}
|
||||
.c-11 {fill: #ddaf3c}
|
||||
.c-12 {fill: #26b0d7}
|
||||
.c-13 {fill: #b954e1}
|
||||
.c-14 {fill: #54e1b9}
|
||||
.c-15 {fill: #ffffff}
|
||||
.c-8, .c-9, .c-10, .c-11, .c-12, .c-13, .c-14, .c-15 {font-weight: bold}
|
||||
<!-- 256 colors -->
|
||||
|
||||
.c-16 {fill: #000000}
|
||||
.c-17 {fill: #00005f}
|
||||
.c-18 {fill: #000087}
|
||||
.c-19 {fill: #0000af}
|
||||
.c-20 {fill: #0000d7}
|
||||
.c-21 {fill: #0000ff}
|
||||
.c-22 {fill: #005f00}
|
||||
.c-23 {fill: #005f5f}
|
||||
.c-24 {fill: #005f87}
|
||||
.c-25 {fill: #005faf}
|
||||
.c-26 {fill: #005fd7}
|
||||
.c-27 {fill: #005fff}
|
||||
.c-28 {fill: #008700}
|
||||
.c-29 {fill: #00875f}
|
||||
.c-30 {fill: #008787}
|
||||
.c-31 {fill: #0087af}
|
||||
.c-32 {fill: #0087d7}
|
||||
.c-33 {fill: #0087ff}
|
||||
.c-34 {fill: #00af00}
|
||||
.c-35 {fill: #00af5f}
|
||||
.c-36 {fill: #00af87}
|
||||
.c-37 {fill: #00afaf}
|
||||
.c-38 {fill: #00afd7}
|
||||
.c-39 {fill: #00afff}
|
||||
.c-40 {fill: #00d700}
|
||||
.c-41 {fill: #00d75f}
|
||||
.c-42 {fill: #00d787}
|
||||
.c-43 {fill: #00d7af}
|
||||
.c-44 {fill: #00d7d7}
|
||||
.c-45 {fill: #00d7ff}
|
||||
.c-46 {fill: #00ff00}
|
||||
.c-47 {fill: #00ff5f}
|
||||
.c-48 {fill: #00ff87}
|
||||
.c-49 {fill: #00ffaf}
|
||||
.c-50 {fill: #00ffd7}
|
||||
.c-51 {fill: #00ffff}
|
||||
.c-52 {fill: #5f0000}
|
||||
.c-53 {fill: #5f005f}
|
||||
.c-54 {fill: #5f0087}
|
||||
.c-55 {fill: #5f00af}
|
||||
.c-56 {fill: #5f00d7}
|
||||
.c-57 {fill: #5f00ff}
|
||||
.c-58 {fill: #5f5f00}
|
||||
.c-59 {fill: #5f5f5f}
|
||||
.c-60 {fill: #5f5f87}
|
||||
.c-61 {fill: #5f5faf}
|
||||
.c-62 {fill: #5f5fd7}
|
||||
.c-63 {fill: #5f5fff}
|
||||
.c-64 {fill: #5f8700}
|
||||
.c-65 {fill: #5f875f}
|
||||
.c-66 {fill: #5f8787}
|
||||
.c-67 {fill: #5f87af}
|
||||
.c-68 {fill: #5f87d7}
|
||||
.c-69 {fill: #5f87ff}
|
||||
.c-70 {fill: #5faf00}
|
||||
.c-71 {fill: #5faf5f}
|
||||
.c-72 {fill: #5faf87}
|
||||
.c-73 {fill: #5fafaf}
|
||||
.c-74 {fill: #5fafd7}
|
||||
.c-75 {fill: #5fafff}
|
||||
.c-76 {fill: #5fd700}
|
||||
.c-77 {fill: #5fd75f}
|
||||
.c-78 {fill: #5fd787}
|
||||
.c-79 {fill: #5fd7af}
|
||||
.c-80 {fill: #5fd7d7}
|
||||
.c-81 {fill: #5fd7ff}
|
||||
.c-82 {fill: #5fff00}
|
||||
.c-83 {fill: #5fff5f}
|
||||
.c-84 {fill: #5fff87}
|
||||
.c-85 {fill: #5fffaf}
|
||||
.c-86 {fill: #5fffd7}
|
||||
.c-87 {fill: #5fffff}
|
||||
.c-88 {fill: #870000}
|
||||
.c-89 {fill: #87005f}
|
||||
.c-90 {fill: #870087}
|
||||
.c-91 {fill: #8700af}
|
||||
.c-92 {fill: #8700d7}
|
||||
.c-93 {fill: #8700ff}
|
||||
.c-94 {fill: #875f00}
|
||||
.c-95 {fill: #875f5f}
|
||||
.c-96 {fill: #875f87}
|
||||
.c-97 {fill: #875faf}
|
||||
.c-98 {fill: #875fd7}
|
||||
.c-99 {fill: #875fff}
|
||||
.c-100 {fill: #878700}
|
||||
.c-101 {fill: #87875f}
|
||||
.c-102 {fill: #878787}
|
||||
.c-103 {fill: #8787af}
|
||||
.c-104 {fill: #8787d7}
|
||||
.c-105 {fill: #8787ff}
|
||||
.c-106 {fill: #87af00}
|
||||
.c-107 {fill: #87af5f}
|
||||
.c-108 {fill: #87af87}
|
||||
.c-109 {fill: #87afaf}
|
||||
.c-110 {fill: #87afd7}
|
||||
.c-111 {fill: #87afff}
|
||||
.c-112 {fill: #87d700}
|
||||
.c-113 {fill: #87d75f}
|
||||
.c-114 {fill: #87d787}
|
||||
.c-115 {fill: #87d7af}
|
||||
.c-116 {fill: #87d7d7}
|
||||
.c-117 {fill: #87d7ff}
|
||||
.c-118 {fill: #87ff00}
|
||||
.c-119 {fill: #87ff5f}
|
||||
.c-120 {fill: #87ff87}
|
||||
.c-121 {fill: #87ffaf}
|
||||
.c-122 {fill: #87ffd7}
|
||||
.c-123 {fill: #87ffff}
|
||||
.c-124 {fill: #af0000}
|
||||
.c-125 {fill: #af005f}
|
||||
.c-126 {fill: #af0087}
|
||||
.c-127 {fill: #af00af}
|
||||
.c-128 {fill: #af00d7}
|
||||
.c-129 {fill: #af00ff}
|
||||
.c-130 {fill: #af5f00}
|
||||
.c-131 {fill: #af5f5f}
|
||||
.c-132 {fill: #af5f87}
|
||||
.c-133 {fill: #af5faf}
|
||||
.c-134 {fill: #af5fd7}
|
||||
.c-135 {fill: #af5fff}
|
||||
.c-136 {fill: #af8700}
|
||||
.c-137 {fill: #af875f}
|
||||
.c-138 {fill: #af8787}
|
||||
.c-139 {fill: #af87af}
|
||||
.c-140 {fill: #af87d7}
|
||||
.c-141 {fill: #af87ff}
|
||||
.c-142 {fill: #afaf00}
|
||||
.c-143 {fill: #afaf5f}
|
||||
.c-144 {fill: #afaf87}
|
||||
.c-145 {fill: #afafaf}
|
||||
.c-146 {fill: #afafd7}
|
||||
.c-147 {fill: #afafff}
|
||||
.c-148 {fill: #afd700}
|
||||
.c-149 {fill: #afd75f}
|
||||
.c-150 {fill: #afd787}
|
||||
.c-151 {fill: #afd7af}
|
||||
.c-152 {fill: #afd7d7}
|
||||
.c-153 {fill: #afd7ff}
|
||||
.c-154 {fill: #afff00}
|
||||
.c-155 {fill: #afff5f}
|
||||
.c-156 {fill: #afff87}
|
||||
.c-157 {fill: #afffaf}
|
||||
.c-158 {fill: #afffd7}
|
||||
.c-159 {fill: #afffff}
|
||||
.c-160 {fill: #d70000}
|
||||
.c-161 {fill: #d7005f}
|
||||
.c-162 {fill: #d70087}
|
||||
.c-163 {fill: #d700af}
|
||||
.c-164 {fill: #d700d7}
|
||||
.c-165 {fill: #d700ff}
|
||||
.c-166 {fill: #d75f00}
|
||||
.c-167 {fill: #d75f5f}
|
||||
.c-168 {fill: #d75f87}
|
||||
.c-169 {fill: #d75faf}
|
||||
.c-170 {fill: #d75fd7}
|
||||
.c-171 {fill: #d75fff}
|
||||
.c-172 {fill: #d78700}
|
||||
.c-173 {fill: #d7875f}
|
||||
.c-174 {fill: #d78787}
|
||||
.c-175 {fill: #d787af}
|
||||
.c-176 {fill: #d787d7}
|
||||
.c-177 {fill: #d787ff}
|
||||
.c-178 {fill: #d7af00}
|
||||
.c-179 {fill: #d7af5f}
|
||||
.c-180 {fill: #d7af87}
|
||||
.c-181 {fill: #d7afaf}
|
||||
.c-182 {fill: #d7afd7}
|
||||
.c-183 {fill: #d7afff}
|
||||
.c-184 {fill: #d7d700}
|
||||
.c-185 {fill: #d7d75f}
|
||||
.c-186 {fill: #d7d787}
|
||||
.c-187 {fill: #d7d7af}
|
||||
.c-188 {fill: #d7d7d7}
|
||||
.c-189 {fill: #d7d7ff}
|
||||
.c-190 {fill: #d7ff00}
|
||||
.c-191 {fill: #d7ff5f}
|
||||
.c-192 {fill: #d7ff87}
|
||||
.c-193 {fill: #d7ffaf}
|
||||
.c-194 {fill: #d7ffd7}
|
||||
.c-195 {fill: #d7ffff}
|
||||
.c-196 {fill: #ff0000}
|
||||
.c-197 {fill: #ff005f}
|
||||
.c-198 {fill: #ff0087}
|
||||
.c-199 {fill: #ff00af}
|
||||
.c-200 {fill: #ff00d7}
|
||||
.c-201 {fill: #ff00ff}
|
||||
.c-202 {fill: #ff5f00}
|
||||
.c-203 {fill: #ff5f5f}
|
||||
.c-204 {fill: #ff5f87}
|
||||
.c-205 {fill: #ff5faf}
|
||||
.c-206 {fill: #ff5fd7}
|
||||
.c-207 {fill: #ff5fff}
|
||||
.c-208 {fill: #ff8700}
|
||||
.c-209 {fill: #ff875f}
|
||||
.c-210 {fill: #ff8787}
|
||||
.c-211 {fill: #ff87af}
|
||||
.c-212 {fill: #ff87d7}
|
||||
.c-213 {fill: #ff87ff}
|
||||
.c-214 {fill: #ffaf00}
|
||||
.c-215 {fill: #ffaf5f}
|
||||
.c-216 {fill: #ffaf87}
|
||||
.c-217 {fill: #ffafaf}
|
||||
.c-218 {fill: #ffafd7}
|
||||
.c-219 {fill: #ffafff}
|
||||
.c-220 {fill: #ffd700}
|
||||
.c-221 {fill: #ffd75f}
|
||||
.c-222 {fill: #ffd787}
|
||||
.c-223 {fill: #ffd7af}
|
||||
.c-224 {fill: #ffd7d7}
|
||||
.c-225 {fill: #ffd7ff}
|
||||
.c-226 {fill: #ffff00}
|
||||
.c-227 {fill: #ffff5f}
|
||||
.c-228 {fill: #ffff87}
|
||||
.c-229 {fill: #ffffaf}
|
||||
.c-230 {fill: #ffffd7}
|
||||
.c-231 {fill: #ffffff}
|
||||
.c-232 {fill: #080808}
|
||||
.c-233 {fill: #121212}
|
||||
.c-234 {fill: #1c1c1c}
|
||||
.c-235 {fill: #262626}
|
||||
.c-236 {fill: #303030}
|
||||
.c-237 {fill: #3a3a3a}
|
||||
.c-238 {fill: #444444}
|
||||
.c-239 {fill: #4e4e4e}
|
||||
.c-240 {fill: #585858}
|
||||
.c-241 {fill: #626262}
|
||||
.c-242 {fill: #6c6c6c}
|
||||
.c-243 {fill: #767676}
|
||||
.c-244 {fill: #808080}
|
||||
.c-245 {fill: #8a8a8a}
|
||||
.c-246 {fill: #949494}
|
||||
.c-247 {fill: #9e9e9e}
|
||||
.c-248 {fill: #a8a8a8}
|
||||
.c-249 {fill: #b2b2b2}
|
||||
.c-250 {fill: #bcbcbc}
|
||||
.c-251 {fill: #c6c6c6}
|
||||
.c-252 {fill: #d0d0d0}
|
||||
.c-253 {fill: #dadada}
|
||||
.c-254 {fill: #e4e4e4}
|
||||
.c-255 {fill: #eeeeee}
|
||||
|
||||
.br { font-weight: bold }
|
||||
.it { font-style: italic }
|
||||
.un { text-decoration: underline }
|
||||
</style>
|
||||
|
||||
<rect width="100%" height="100%" class="default-bg-fill" />
|
||||
|
||||
<svg x="0.625%" y="1.136%" class="default-text-fill">
|
||||
<g style="shape-rendering: optimizeSpeed">
|
||||
<rect x="5.625%" y="6.818%" width="0.625%" height="19.7" class="c-7" />
|
||||
<rect x="0.000%" y="95.455%" width="98.750%" height="19.7" class="c-2" />
|
||||
</g>
|
||||
|
||||
<text class="default-text-fill">
|
||||
<tspan y="0.000%">
|
||||
<tspan dy="1em" x="0.000%">c</tspan><tspan x="0.625%">h</tspan><tspan x="1.250%">a</tspan><tspan x="1.875%">o</tspan><tspan x="2.500%">s</tspan><tspan x="3.125%">:</tspan><tspan x="3.750%">~</tspan><tspan x="4.375%">$</tspan><tspan x="5.625%">#</tspan><tspan x="6.875%">B</tspan><tspan x="7.500%">u</tspan><tspan x="8.125%">i</tspan><tspan x="8.750%">l</tspan><tspan x="9.375%">d</tspan><tspan x="10.625%">a</tspan><tspan x="11.250%">n</tspan><tspan x="11.875%">d</tspan><tspan x="13.125%">a</tspan><tspan x="13.750%">p</tspan><tspan x="14.375%">p</tspan><tspan x="15.000%">l</tspan><tspan x="15.625%">y</tspan><tspan x="16.875%">C</tspan><tspan x="17.500%">h</tspan><tspan x="18.125%">a</tspan><tspan x="18.750%">o</tspan><tspan x="19.375%">s</tspan><tspan x="20.000%">E</tspan><tspan x="20.625%">n</tspan><tspan x="21.250%">g</tspan><tspan x="21.875%">i</tspan><tspan x="22.500%">n</tspan><tspan x="23.125%">e</tspan><tspan x="24.375%">C</tspan><tspan x="25.000%">R</tspan><tspan x="26.250%">t</tspan><tspan x="26.875%">o</tspan><tspan x="28.125%">u</tspan><tspan x="28.750%">n</tspan><tspan x="29.375%">l</tspan><tspan x="30.000%">e</tspan><tspan x="30.625%">a</tspan><tspan x="31.250%">s</tspan><tspan x="31.875%">h</tspan><tspan x="33.125%">C</tspan><tspan x="33.750%">h</tspan><tspan x="34.375%">a</tspan><tspan x="35.000%">o</tspan><tspan x="35.625%">s</tspan><tspan x="49.375%" class="c-2">│</tspan><tspan x="50.000%">E</tspan><tspan x="50.625%">v</tspan><tspan x="51.250%">e</tspan><tspan x="51.875%">r</tspan><tspan x="52.500%">y</tspan><tspan x="53.750%">1</tspan><tspan x="54.375%">.</tspan><tspan x="55.000%">0</tspan><tspan x="55.625%">s</tspan><tspan x="56.250%">:</tspan><tspan x="57.500%">k</tspan><tspan x="58.125%">u</tspan><tspan x="58.750%">b</tspan><tspan x="59.375%">e</tspan><tspan x="60.000%">c</tspan><tspan x="60.625%">t</tspan><tspan x="61.250%">l</tspan><tspan x="62.500%">g</tspan><tspan x="63.125%">e</tspan><tspan x="63.750%">t</tspan><tspan x="65.000%">p</tspan><tspan x="65.625%">o</tspan><tspan x="83.750%">F</tspan><tspan x="84.375%">r</tspan><tspan x="85.000%">i</tspan><tspan x="86.250%">O</tspan><tspan x="86.875%">c</tspan><tspan x="87.500%">t</tspan><tspan x="89.375%">4</tspan><tspan x="90.625%">1</tspan><tspan x="91.250%">9</tspan><tspan x="91.875%">:</tspan><tspan x="92.500%">3</tspan><tspan x="93.125%">2</tspan><tspan x="93.750%">:</tspan><tspan x="94.375%">3</tspan><tspan x="95.000%">5</tspan><tspan x="96.250%">2</tspan><tspan x="96.875%">0</tspan><tspan x="97.500%">1</tspan><tspan x="98.125%">9</tspan>
|
||||
</tspan>
|
||||
<tspan y="2.273%">
|
||||
<tspan dy="1em" x="0.000%">c</tspan><tspan x="0.625%">h</tspan><tspan x="1.250%">a</tspan><tspan x="1.875%">o</tspan><tspan x="2.500%">s</tspan><tspan x="3.125%">:</tspan><tspan x="3.750%">~</tspan><tspan x="4.375%">$</tspan><tspan x="49.375%" class="c-2">│</tspan>
|
||||
</tspan>
|
||||
<tspan y="4.545%">
|
||||
<tspan dy="1em" x="0.000%">c</tspan><tspan x="0.625%">h</tspan><tspan x="1.250%">a</tspan><tspan x="1.875%">o</tspan><tspan x="2.500%">s</tspan><tspan x="3.125%">:</tspan><tspan x="3.750%">~</tspan><tspan x="4.375%">$</tspan><tspan x="5.625%">v</tspan><tspan x="6.250%">i</tspan><tspan x="7.500%">c</tspan><tspan x="8.125%">h</tspan><tspan x="8.750%">a</tspan><tspan x="9.375%">o</tspan><tspan x="10.000%">s</tspan><tspan x="10.625%">e</tspan><tspan x="11.250%">n</tspan><tspan x="11.875%">g</tspan><tspan x="12.500%">i</tspan><tspan x="13.125%">n</tspan><tspan x="13.750%">e</tspan><tspan x="14.375%">.</tspan><tspan x="15.000%">y</tspan><tspan x="15.625%">a</tspan><tspan x="16.250%">m</tspan><tspan x="16.875%">l</tspan><tspan x="49.375%" class="c-2">│</tspan><tspan x="50.000%">N</tspan><tspan x="50.625%">A</tspan><tspan x="51.250%">M</tspan><tspan x="51.875%">E</tspan><tspan x="69.375%">R</tspan><tspan x="70.000%">E</tspan><tspan x="70.625%">A</tspan><tspan x="71.250%">D</tspan><tspan x="71.875%">Y</tspan><tspan x="74.375%">S</tspan><tspan x="75.000%">T</tspan><tspan x="75.625%">A</tspan><tspan x="76.250%">T</tspan><tspan x="76.875%">U</tspan><tspan x="77.500%">S</tspan><tspan x="80.625%">R</tspan><tspan x="81.250%">E</tspan><tspan x="81.875%">S</tspan><tspan x="82.500%">T</tspan><tspan x="83.125%">A</tspan><tspan x="83.750%">R</tspan><tspan x="84.375%">T</tspan><tspan x="85.000%">S</tspan><tspan x="87.500%">A</tspan><tspan x="88.125%">G</tspan><tspan x="88.750%">E</tspan>
|
||||
</tspan>
|
||||
<tspan y="6.818%">
|
||||
<tspan dy="1em" x="0.000%">c</tspan><tspan x="0.625%">h</tspan><tspan x="1.250%">a</tspan><tspan x="1.875%">o</tspan><tspan x="2.500%">s</tspan><tspan x="3.125%">:</tspan><tspan x="3.750%">~</tspan><tspan x="4.375%">$</tspan><tspan x="49.375%" class="c-2">│</tspan><tspan x="50.000%">h</tspan><tspan x="50.625%">e</tspan><tspan x="51.250%">l</tspan><tspan x="51.875%">l</tspan><tspan x="52.500%">o</tspan><tspan x="53.125%">-</tspan><tspan x="53.750%">d</tspan><tspan x="54.375%">e</tspan><tspan x="55.000%">p</tspan><tspan x="55.625%">l</tspan><tspan x="56.250%">o</tspan><tspan x="56.875%">y</tspan><tspan x="57.500%">-</tspan><tspan x="58.125%">d</tspan><tspan x="58.750%">d</tspan><tspan x="59.375%">5</tspan><tspan x="60.000%">9</tspan><tspan x="60.625%">b</tspan><tspan x="61.250%">8</tspan><tspan x="61.875%">9</tspan><tspan x="62.500%">5</tspan><tspan x="63.125%">6</tspan><tspan x="63.750%">-</tspan><tspan x="64.375%">h</tspan><tspan x="65.000%">x</tspan><tspan x="65.625%">c</tspan><tspan x="66.250%">j</tspan><tspan x="66.875%">v</tspan><tspan x="69.375%">1</tspan><tspan x="70.000%">/</tspan><tspan x="70.625%">1</tspan><tspan x="74.375%">R</tspan><tspan x="75.000%">u</tspan><tspan x="75.625%">n</tspan><tspan x="76.250%">n</tspan><tspan x="76.875%">i</tspan><tspan x="77.500%">n</tspan><tspan x="78.125%">g</tspan><tspan x="80.625%">0</tspan><tspan x="87.500%">1</tspan><tspan x="88.125%">9</tspan><tspan x="88.750%">m</tspan>
|
||||
</tspan>
|
||||
<tspan y="9.091%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan>
|
||||
</tspan>
|
||||
<tspan y="11.364%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan>
|
||||
</tspan>
|
||||
<tspan y="13.636%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan>
|
||||
</tspan>
|
||||
<tspan y="15.909%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan>
|
||||
</tspan>
|
||||
<tspan y="18.182%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan>
|
||||
</tspan>
|
||||
<tspan y="20.455%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan>
|
||||
</tspan>
|
||||
<tspan y="22.727%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan>
|
||||
</tspan>
|
||||
<tspan y="25.000%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan>
|
||||
</tspan>
|
||||
<tspan y="27.273%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan>
|
||||
</tspan>
|
||||
<tspan y="29.545%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan>
|
||||
</tspan>
|
||||
<tspan y="31.818%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan>
|
||||
</tspan>
|
||||
<tspan y="34.091%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan>
|
||||
</tspan>
|
||||
<tspan y="36.364%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan>
|
||||
</tspan>
|
||||
<tspan y="38.636%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan>
|
||||
</tspan>
|
||||
<tspan y="40.909%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan>
|
||||
</tspan>
|
||||
<tspan y="43.182%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan>
|
||||
</tspan>
|
||||
<tspan y="45.455%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan>
|
||||
</tspan>
|
||||
<tspan y="47.727%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">├</tspan><tspan x="50.000%">─</tspan><tspan x="50.625%">─</tspan><tspan x="51.250%">─</tspan><tspan x="51.875%">─</tspan><tspan x="52.500%">─</tspan><tspan x="53.125%">─</tspan><tspan x="53.750%">─</tspan><tspan x="54.375%">─</tspan><tspan x="55.000%">─</tspan><tspan x="55.625%">─</tspan><tspan x="56.250%">─</tspan><tspan x="56.875%">─</tspan><tspan x="57.500%">─</tspan><tspan x="58.125%">─</tspan><tspan x="58.750%">─</tspan><tspan x="59.375%">─</tspan><tspan x="60.000%">─</tspan><tspan x="60.625%">─</tspan><tspan x="61.250%">─</tspan><tspan x="61.875%">─</tspan><tspan x="62.500%">─</tspan><tspan x="63.125%">─</tspan><tspan x="63.750%">─</tspan><tspan x="64.375%">─</tspan><tspan x="65.000%">─</tspan><tspan x="65.625%">─</tspan><tspan x="66.250%">─</tspan><tspan x="66.875%">─</tspan><tspan x="67.500%">─</tspan><tspan x="68.125%">─</tspan><tspan x="68.750%">─</tspan><tspan x="69.375%">─</tspan><tspan x="70.000%">─</tspan><tspan x="70.625%">─</tspan><tspan x="71.250%">─</tspan><tspan x="71.875%">─</tspan><tspan x="72.500%">─</tspan><tspan x="73.125%">─</tspan><tspan x="73.750%">─</tspan><tspan x="74.375%">─</tspan><tspan x="75.000%">─</tspan><tspan x="75.625%">─</tspan><tspan x="76.250%">─</tspan><tspan x="76.875%">─</tspan><tspan x="77.500%">─</tspan><tspan x="78.125%">─</tspan><tspan x="78.750%">─</tspan><tspan x="79.375%">─</tspan><tspan x="80.000%">─</tspan><tspan x="80.625%">─</tspan><tspan x="81.250%">─</tspan><tspan x="81.875%">─</tspan><tspan x="82.500%">─</tspan><tspan x="83.125%">─</tspan><tspan x="83.750%">─</tspan><tspan x="84.375%">─</tspan><tspan x="85.000%">─</tspan><tspan x="85.625%">─</tspan><tspan x="86.250%">─</tspan><tspan x="86.875%">─</tspan><tspan x="87.500%">─</tspan><tspan x="88.125%">─</tspan><tspan x="88.750%">─</tspan><tspan x="89.375%">─</tspan><tspan x="90.000%">─</tspan><tspan x="90.625%">─</tspan><tspan x="91.250%">─</tspan><tspan x="91.875%">─</tspan><tspan x="92.500%">─</tspan><tspan x="93.125%">─</tspan><tspan x="93.750%">─</tspan><tspan x="94.375%">─</tspan><tspan x="95.000%">─</tspan><tspan x="95.625%">─</tspan><tspan x="96.250%">─</tspan><tspan x="96.875%">─</tspan><tspan x="97.500%">─</tspan><tspan x="98.125%">─</tspan>
|
||||
</tspan>
|
||||
<tspan y="50.000%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan><tspan x="50.000%" class="br c-10">H</tspan><tspan x="50.625%" class="br c-10">e</tspan><tspan x="51.250%" class="br c-10">l</tspan><tspan x="51.875%" class="br c-10">l</tspan><tspan x="52.500%" class="br c-10">o</tspan><tspan x="53.750%" class="br c-10">W</tspan><tspan x="54.375%" class="br c-10">o</tspan><tspan x="55.000%" class="br c-10">r</tspan><tspan x="55.625%" class="br c-10">l</tspan><tspan x="56.250%" class="br c-10">d</tspan><tspan x="57.500%" class="br c-10">i</tspan><tspan x="58.125%" class="br c-10">s</tspan><tspan x="59.375%" class="br c-10">o</tspan><tspan x="60.000%" class="br c-10">n</tspan><tspan x="60.625%" class="br c-10">l</tspan><tspan x="61.250%" class="br c-10">i</tspan><tspan x="61.875%" class="br c-10">n</tspan><tspan x="62.500%" class="br c-10">e</tspan><tspan x="63.750%">H</tspan><tspan x="64.375%">T</tspan><tspan x="65.000%">T</tspan><tspan x="65.625%">P</tspan><tspan x="66.250%">/</tspan><tspan x="66.875%">2</tspan><tspan x="68.125%">2</tspan><tspan x="68.750%">0</tspan><tspan x="69.375%">0</tspan><tspan x="70.625%">O</tspan><tspan x="71.250%">K</tspan>
|
||||
</tspan>
|
||||
<tspan y="52.273%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan><tspan x="50.000%" class="br c-10">H</tspan><tspan x="50.625%" class="br c-10">e</tspan><tspan x="51.250%" class="br c-10">l</tspan><tspan x="51.875%" class="br c-10">l</tspan><tspan x="52.500%" class="br c-10">o</tspan><tspan x="53.750%" class="br c-10">W</tspan><tspan x="54.375%" class="br c-10">o</tspan><tspan x="55.000%" class="br c-10">r</tspan><tspan x="55.625%" class="br c-10">l</tspan><tspan x="56.250%" class="br c-10">d</tspan><tspan x="57.500%" class="br c-10">i</tspan><tspan x="58.125%" class="br c-10">s</tspan><tspan x="59.375%" class="br c-10">o</tspan><tspan x="60.000%" class="br c-10">n</tspan><tspan x="60.625%" class="br c-10">l</tspan><tspan x="61.250%" class="br c-10">i</tspan><tspan x="61.875%" class="br c-10">n</tspan><tspan x="62.500%" class="br c-10">e</tspan><tspan x="63.750%">H</tspan><tspan x="64.375%">T</tspan><tspan x="65.000%">T</tspan><tspan x="65.625%">P</tspan><tspan x="66.250%">/</tspan><tspan x="66.875%">2</tspan><tspan x="68.125%">2</tspan><tspan x="68.750%">0</tspan><tspan x="69.375%">0</tspan><tspan x="70.625%">O</tspan><tspan x="71.250%">K</tspan>
|
||||
</tspan>
|
||||
<tspan y="54.545%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan><tspan x="50.000%" class="br c-10">H</tspan><tspan x="50.625%" class="br c-10">e</tspan><tspan x="51.250%" class="br c-10">l</tspan><tspan x="51.875%" class="br c-10">l</tspan><tspan x="52.500%" class="br c-10">o</tspan><tspan x="53.750%" class="br c-10">W</tspan><tspan x="54.375%" class="br c-10">o</tspan><tspan x="55.000%" class="br c-10">r</tspan><tspan x="55.625%" class="br c-10">l</tspan><tspan x="56.250%" class="br c-10">d</tspan><tspan x="57.500%" class="br c-10">i</tspan><tspan x="58.125%" class="br c-10">s</tspan><tspan x="59.375%" class="br c-10">o</tspan><tspan x="60.000%" class="br c-10">n</tspan><tspan x="60.625%" class="br c-10">l</tspan><tspan x="61.250%" class="br c-10">i</tspan><tspan x="61.875%" class="br c-10">n</tspan><tspan x="62.500%" class="br c-10">e</tspan><tspan x="63.750%">H</tspan><tspan x="64.375%">T</tspan><tspan x="65.000%">T</tspan><tspan x="65.625%">P</tspan><tspan x="66.250%">/</tspan><tspan x="66.875%">2</tspan><tspan x="68.125%">2</tspan><tspan x="68.750%">0</tspan><tspan x="69.375%">0</tspan><tspan x="70.625%">O</tspan><tspan x="71.250%">K</tspan>
|
||||
</tspan>
|
||||
<tspan y="56.818%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan><tspan x="50.000%" class="br c-10">H</tspan><tspan x="50.625%" class="br c-10">e</tspan><tspan x="51.250%" class="br c-10">l</tspan><tspan x="51.875%" class="br c-10">l</tspan><tspan x="52.500%" class="br c-10">o</tspan><tspan x="53.750%" class="br c-10">W</tspan><tspan x="54.375%" class="br c-10">o</tspan><tspan x="55.000%" class="br c-10">r</tspan><tspan x="55.625%" class="br c-10">l</tspan><tspan x="56.250%" class="br c-10">d</tspan><tspan x="57.500%" class="br c-10">i</tspan><tspan x="58.125%" class="br c-10">s</tspan><tspan x="59.375%" class="br c-10">o</tspan><tspan x="60.000%" class="br c-10">n</tspan><tspan x="60.625%" class="br c-10">l</tspan><tspan x="61.250%" class="br c-10">i</tspan><tspan x="61.875%" class="br c-10">n</tspan><tspan x="62.500%" class="br c-10">e</tspan><tspan x="63.750%">H</tspan><tspan x="64.375%">T</tspan><tspan x="65.000%">T</tspan><tspan x="65.625%">P</tspan><tspan x="66.250%">/</tspan><tspan x="66.875%">2</tspan><tspan x="68.125%">2</tspan><tspan x="68.750%">0</tspan><tspan x="69.375%">0</tspan><tspan x="70.625%">O</tspan><tspan x="71.250%">K</tspan>
|
||||
</tspan>
|
||||
<tspan y="59.091%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan><tspan x="50.000%" class="br c-10">H</tspan><tspan x="50.625%" class="br c-10">e</tspan><tspan x="51.250%" class="br c-10">l</tspan><tspan x="51.875%" class="br c-10">l</tspan><tspan x="52.500%" class="br c-10">o</tspan><tspan x="53.750%" class="br c-10">W</tspan><tspan x="54.375%" class="br c-10">o</tspan><tspan x="55.000%" class="br c-10">r</tspan><tspan x="55.625%" class="br c-10">l</tspan><tspan x="56.250%" class="br c-10">d</tspan><tspan x="57.500%" class="br c-10">i</tspan><tspan x="58.125%" class="br c-10">s</tspan><tspan x="59.375%" class="br c-10">o</tspan><tspan x="60.000%" class="br c-10">n</tspan><tspan x="60.625%" class="br c-10">l</tspan><tspan x="61.250%" class="br c-10">i</tspan><tspan x="61.875%" class="br c-10">n</tspan><tspan x="62.500%" class="br c-10">e</tspan><tspan x="63.750%">H</tspan><tspan x="64.375%">T</tspan><tspan x="65.000%">T</tspan><tspan x="65.625%">P</tspan><tspan x="66.250%">/</tspan><tspan x="66.875%">2</tspan><tspan x="68.125%">2</tspan><tspan x="68.750%">0</tspan><tspan x="69.375%">0</tspan><tspan x="70.625%">O</tspan><tspan x="71.250%">K</tspan>
|
||||
</tspan>
|
||||
<tspan y="61.364%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan><tspan x="50.000%" class="br c-10">H</tspan><tspan x="50.625%" class="br c-10">e</tspan><tspan x="51.250%" class="br c-10">l</tspan><tspan x="51.875%" class="br c-10">l</tspan><tspan x="52.500%" class="br c-10">o</tspan><tspan x="53.750%" class="br c-10">W</tspan><tspan x="54.375%" class="br c-10">o</tspan><tspan x="55.000%" class="br c-10">r</tspan><tspan x="55.625%" class="br c-10">l</tspan><tspan x="56.250%" class="br c-10">d</tspan><tspan x="57.500%" class="br c-10">i</tspan><tspan x="58.125%" class="br c-10">s</tspan><tspan x="59.375%" class="br c-10">o</tspan><tspan x="60.000%" class="br c-10">n</tspan><tspan x="60.625%" class="br c-10">l</tspan><tspan x="61.250%" class="br c-10">i</tspan><tspan x="61.875%" class="br c-10">n</tspan><tspan x="62.500%" class="br c-10">e</tspan><tspan x="63.750%">H</tspan><tspan x="64.375%">T</tspan><tspan x="65.000%">T</tspan><tspan x="65.625%">P</tspan><tspan x="66.250%">/</tspan><tspan x="66.875%">2</tspan><tspan x="68.125%">2</tspan><tspan x="68.750%">0</tspan><tspan x="69.375%">0</tspan><tspan x="70.625%">O</tspan><tspan x="71.250%">K</tspan>
|
||||
</tspan>
|
||||
<tspan y="63.636%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan><tspan x="50.000%" class="br c-10">H</tspan><tspan x="50.625%" class="br c-10">e</tspan><tspan x="51.250%" class="br c-10">l</tspan><tspan x="51.875%" class="br c-10">l</tspan><tspan x="52.500%" class="br c-10">o</tspan><tspan x="53.750%" class="br c-10">W</tspan><tspan x="54.375%" class="br c-10">o</tspan><tspan x="55.000%" class="br c-10">r</tspan><tspan x="55.625%" class="br c-10">l</tspan><tspan x="56.250%" class="br c-10">d</tspan><tspan x="57.500%" class="br c-10">i</tspan><tspan x="58.125%" class="br c-10">s</tspan><tspan x="59.375%" class="br c-10">o</tspan><tspan x="60.000%" class="br c-10">n</tspan><tspan x="60.625%" class="br c-10">l</tspan><tspan x="61.250%" class="br c-10">i</tspan><tspan x="61.875%" class="br c-10">n</tspan><tspan x="62.500%" class="br c-10">e</tspan><tspan x="63.750%">H</tspan><tspan x="64.375%">T</tspan><tspan x="65.000%">T</tspan><tspan x="65.625%">P</tspan><tspan x="66.250%">/</tspan><tspan x="66.875%">2</tspan><tspan x="68.125%">2</tspan><tspan x="68.750%">0</tspan><tspan x="69.375%">0</tspan><tspan x="70.625%">O</tspan><tspan x="71.250%">K</tspan>
|
||||
</tspan>
|
||||
<tspan y="65.909%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan><tspan x="50.000%" class="br c-10">H</tspan><tspan x="50.625%" class="br c-10">e</tspan><tspan x="51.250%" class="br c-10">l</tspan><tspan x="51.875%" class="br c-10">l</tspan><tspan x="52.500%" class="br c-10">o</tspan><tspan x="53.750%" class="br c-10">W</tspan><tspan x="54.375%" class="br c-10">o</tspan><tspan x="55.000%" class="br c-10">r</tspan><tspan x="55.625%" class="br c-10">l</tspan><tspan x="56.250%" class="br c-10">d</tspan><tspan x="57.500%" class="br c-10">i</tspan><tspan x="58.125%" class="br c-10">s</tspan><tspan x="59.375%" class="br c-10">o</tspan><tspan x="60.000%" class="br c-10">n</tspan><tspan x="60.625%" class="br c-10">l</tspan><tspan x="61.250%" class="br c-10">i</tspan><tspan x="61.875%" class="br c-10">n</tspan><tspan x="62.500%" class="br c-10">e</tspan><tspan x="63.750%">H</tspan><tspan x="64.375%">T</tspan><tspan x="65.000%">T</tspan><tspan x="65.625%">P</tspan><tspan x="66.250%">/</tspan><tspan x="66.875%">2</tspan><tspan x="68.125%">2</tspan><tspan x="68.750%">0</tspan><tspan x="69.375%">0</tspan><tspan x="70.625%">O</tspan><tspan x="71.250%">K</tspan>
|
||||
</tspan>
|
||||
<tspan y="68.182%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan><tspan x="50.000%" class="br c-10">H</tspan><tspan x="50.625%" class="br c-10">e</tspan><tspan x="51.250%" class="br c-10">l</tspan><tspan x="51.875%" class="br c-10">l</tspan><tspan x="52.500%" class="br c-10">o</tspan><tspan x="53.750%" class="br c-10">W</tspan><tspan x="54.375%" class="br c-10">o</tspan><tspan x="55.000%" class="br c-10">r</tspan><tspan x="55.625%" class="br c-10">l</tspan><tspan x="56.250%" class="br c-10">d</tspan><tspan x="57.500%" class="br c-10">i</tspan><tspan x="58.125%" class="br c-10">s</tspan><tspan x="59.375%" class="br c-10">o</tspan><tspan x="60.000%" class="br c-10">n</tspan><tspan x="60.625%" class="br c-10">l</tspan><tspan x="61.250%" class="br c-10">i</tspan><tspan x="61.875%" class="br c-10">n</tspan><tspan x="62.500%" class="br c-10">e</tspan><tspan x="63.750%">H</tspan><tspan x="64.375%">T</tspan><tspan x="65.000%">T</tspan><tspan x="65.625%">P</tspan><tspan x="66.250%">/</tspan><tspan x="66.875%">2</tspan><tspan x="68.125%">2</tspan><tspan x="68.750%">0</tspan><tspan x="69.375%">0</tspan><tspan x="70.625%">O</tspan><tspan x="71.250%">K</tspan>
|
||||
</tspan>
|
||||
<tspan y="70.455%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan><tspan x="50.000%" class="br c-10">H</tspan><tspan x="50.625%" class="br c-10">e</tspan><tspan x="51.250%" class="br c-10">l</tspan><tspan x="51.875%" class="br c-10">l</tspan><tspan x="52.500%" class="br c-10">o</tspan><tspan x="53.750%" class="br c-10">W</tspan><tspan x="54.375%" class="br c-10">o</tspan><tspan x="55.000%" class="br c-10">r</tspan><tspan x="55.625%" class="br c-10">l</tspan><tspan x="56.250%" class="br c-10">d</tspan><tspan x="57.500%" class="br c-10">i</tspan><tspan x="58.125%" class="br c-10">s</tspan><tspan x="59.375%" class="br c-10">o</tspan><tspan x="60.000%" class="br c-10">n</tspan><tspan x="60.625%" class="br c-10">l</tspan><tspan x="61.250%" class="br c-10">i</tspan><tspan x="61.875%" class="br c-10">n</tspan><tspan x="62.500%" class="br c-10">e</tspan><tspan x="63.750%">H</tspan><tspan x="64.375%">T</tspan><tspan x="65.000%">T</tspan><tspan x="65.625%">P</tspan><tspan x="66.250%">/</tspan><tspan x="66.875%">2</tspan><tspan x="68.125%">2</tspan><tspan x="68.750%">0</tspan><tspan x="69.375%">0</tspan><tspan x="70.625%">O</tspan><tspan x="71.250%">K</tspan>
|
||||
</tspan>
|
||||
<tspan y="72.727%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan><tspan x="50.000%" class="br c-10">H</tspan><tspan x="50.625%" class="br c-10">e</tspan><tspan x="51.250%" class="br c-10">l</tspan><tspan x="51.875%" class="br c-10">l</tspan><tspan x="52.500%" class="br c-10">o</tspan><tspan x="53.750%" class="br c-10">W</tspan><tspan x="54.375%" class="br c-10">o</tspan><tspan x="55.000%" class="br c-10">r</tspan><tspan x="55.625%" class="br c-10">l</tspan><tspan x="56.250%" class="br c-10">d</tspan><tspan x="57.500%" class="br c-10">i</tspan><tspan x="58.125%" class="br c-10">s</tspan><tspan x="59.375%" class="br c-10">o</tspan><tspan x="60.000%" class="br c-10">n</tspan><tspan x="60.625%" class="br c-10">l</tspan><tspan x="61.250%" class="br c-10">i</tspan><tspan x="61.875%" class="br c-10">n</tspan><tspan x="62.500%" class="br c-10">e</tspan><tspan x="63.750%">H</tspan><tspan x="64.375%">T</tspan><tspan x="65.000%">T</tspan><tspan x="65.625%">P</tspan><tspan x="66.250%">/</tspan><tspan x="66.875%">2</tspan><tspan x="68.125%">2</tspan><tspan x="68.750%">0</tspan><tspan x="69.375%">0</tspan><tspan x="70.625%">O</tspan><tspan x="71.250%">K</tspan>
|
||||
</tspan>
|
||||
<tspan y="75.000%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan><tspan x="50.000%" class="br c-10">H</tspan><tspan x="50.625%" class="br c-10">e</tspan><tspan x="51.250%" class="br c-10">l</tspan><tspan x="51.875%" class="br c-10">l</tspan><tspan x="52.500%" class="br c-10">o</tspan><tspan x="53.750%" class="br c-10">W</tspan><tspan x="54.375%" class="br c-10">o</tspan><tspan x="55.000%" class="br c-10">r</tspan><tspan x="55.625%" class="br c-10">l</tspan><tspan x="56.250%" class="br c-10">d</tspan><tspan x="57.500%" class="br c-10">i</tspan><tspan x="58.125%" class="br c-10">s</tspan><tspan x="59.375%" class="br c-10">o</tspan><tspan x="60.000%" class="br c-10">n</tspan><tspan x="60.625%" class="br c-10">l</tspan><tspan x="61.250%" class="br c-10">i</tspan><tspan x="61.875%" class="br c-10">n</tspan><tspan x="62.500%" class="br c-10">e</tspan><tspan x="63.750%">H</tspan><tspan x="64.375%">T</tspan><tspan x="65.000%">T</tspan><tspan x="65.625%">P</tspan><tspan x="66.250%">/</tspan><tspan x="66.875%">2</tspan><tspan x="68.125%">2</tspan><tspan x="68.750%">0</tspan><tspan x="69.375%">0</tspan><tspan x="70.625%">O</tspan><tspan x="71.250%">K</tspan>
|
||||
</tspan>
|
||||
<tspan y="77.273%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan><tspan x="50.000%" class="br c-10">H</tspan><tspan x="50.625%" class="br c-10">e</tspan><tspan x="51.250%" class="br c-10">l</tspan><tspan x="51.875%" class="br c-10">l</tspan><tspan x="52.500%" class="br c-10">o</tspan><tspan x="53.750%" class="br c-10">W</tspan><tspan x="54.375%" class="br c-10">o</tspan><tspan x="55.000%" class="br c-10">r</tspan><tspan x="55.625%" class="br c-10">l</tspan><tspan x="56.250%" class="br c-10">d</tspan><tspan x="57.500%" class="br c-10">i</tspan><tspan x="58.125%" class="br c-10">s</tspan><tspan x="59.375%" class="br c-10">o</tspan><tspan x="60.000%" class="br c-10">n</tspan><tspan x="60.625%" class="br c-10">l</tspan><tspan x="61.250%" class="br c-10">i</tspan><tspan x="61.875%" class="br c-10">n</tspan><tspan x="62.500%" class="br c-10">e</tspan><tspan x="63.750%">H</tspan><tspan x="64.375%">T</tspan><tspan x="65.000%">T</tspan><tspan x="65.625%">P</tspan><tspan x="66.250%">/</tspan><tspan x="66.875%">2</tspan><tspan x="68.125%">2</tspan><tspan x="68.750%">0</tspan><tspan x="69.375%">0</tspan><tspan x="70.625%">O</tspan><tspan x="71.250%">K</tspan>
|
||||
</tspan>
|
||||
<tspan y="79.545%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan><tspan x="50.000%" class="br c-10">H</tspan><tspan x="50.625%" class="br c-10">e</tspan><tspan x="51.250%" class="br c-10">l</tspan><tspan x="51.875%" class="br c-10">l</tspan><tspan x="52.500%" class="br c-10">o</tspan><tspan x="53.750%" class="br c-10">W</tspan><tspan x="54.375%" class="br c-10">o</tspan><tspan x="55.000%" class="br c-10">r</tspan><tspan x="55.625%" class="br c-10">l</tspan><tspan x="56.250%" class="br c-10">d</tspan><tspan x="57.500%" class="br c-10">i</tspan><tspan x="58.125%" class="br c-10">s</tspan><tspan x="59.375%" class="br c-10">o</tspan><tspan x="60.000%" class="br c-10">n</tspan><tspan x="60.625%" class="br c-10">l</tspan><tspan x="61.250%" class="br c-10">i</tspan><tspan x="61.875%" class="br c-10">n</tspan><tspan x="62.500%" class="br c-10">e</tspan><tspan x="63.750%">H</tspan><tspan x="64.375%">T</tspan><tspan x="65.000%">T</tspan><tspan x="65.625%">P</tspan><tspan x="66.250%">/</tspan><tspan x="66.875%">2</tspan><tspan x="68.125%">2</tspan><tspan x="68.750%">0</tspan><tspan x="69.375%">0</tspan><tspan x="70.625%">O</tspan><tspan x="71.250%">K</tspan>
|
||||
</tspan>
|
||||
<tspan y="81.818%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan><tspan x="50.000%" class="br c-10">H</tspan><tspan x="50.625%" class="br c-10">e</tspan><tspan x="51.250%" class="br c-10">l</tspan><tspan x="51.875%" class="br c-10">l</tspan><tspan x="52.500%" class="br c-10">o</tspan><tspan x="53.750%" class="br c-10">W</tspan><tspan x="54.375%" class="br c-10">o</tspan><tspan x="55.000%" class="br c-10">r</tspan><tspan x="55.625%" class="br c-10">l</tspan><tspan x="56.250%" class="br c-10">d</tspan><tspan x="57.500%" class="br c-10">i</tspan><tspan x="58.125%" class="br c-10">s</tspan><tspan x="59.375%" class="br c-10">o</tspan><tspan x="60.000%" class="br c-10">n</tspan><tspan x="60.625%" class="br c-10">l</tspan><tspan x="61.250%" class="br c-10">i</tspan><tspan x="61.875%" class="br c-10">n</tspan><tspan x="62.500%" class="br c-10">e</tspan><tspan x="63.750%">H</tspan><tspan x="64.375%">T</tspan><tspan x="65.000%">T</tspan><tspan x="65.625%">P</tspan><tspan x="66.250%">/</tspan><tspan x="66.875%">2</tspan><tspan x="68.125%">2</tspan><tspan x="68.750%">0</tspan><tspan x="69.375%">0</tspan><tspan x="70.625%">O</tspan><tspan x="71.250%">K</tspan>
|
||||
</tspan>
|
||||
<tspan y="84.091%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan><tspan x="50.000%" class="br c-10">H</tspan><tspan x="50.625%" class="br c-10">e</tspan><tspan x="51.250%" class="br c-10">l</tspan><tspan x="51.875%" class="br c-10">l</tspan><tspan x="52.500%" class="br c-10">o</tspan><tspan x="53.750%" class="br c-10">W</tspan><tspan x="54.375%" class="br c-10">o</tspan><tspan x="55.000%" class="br c-10">r</tspan><tspan x="55.625%" class="br c-10">l</tspan><tspan x="56.250%" class="br c-10">d</tspan><tspan x="57.500%" class="br c-10">i</tspan><tspan x="58.125%" class="br c-10">s</tspan><tspan x="59.375%" class="br c-10">o</tspan><tspan x="60.000%" class="br c-10">n</tspan><tspan x="60.625%" class="br c-10">l</tspan><tspan x="61.250%" class="br c-10">i</tspan><tspan x="61.875%" class="br c-10">n</tspan><tspan x="62.500%" class="br c-10">e</tspan><tspan x="63.750%">H</tspan><tspan x="64.375%">T</tspan><tspan x="65.000%">T</tspan><tspan x="65.625%">P</tspan><tspan x="66.250%">/</tspan><tspan x="66.875%">2</tspan><tspan x="68.125%">2</tspan><tspan x="68.750%">0</tspan><tspan x="69.375%">0</tspan><tspan x="70.625%">O</tspan><tspan x="71.250%">K</tspan>
|
||||
</tspan>
|
||||
<tspan y="86.364%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan><tspan x="50.000%" class="br c-10">H</tspan><tspan x="50.625%" class="br c-10">e</tspan><tspan x="51.250%" class="br c-10">l</tspan><tspan x="51.875%" class="br c-10">l</tspan><tspan x="52.500%" class="br c-10">o</tspan><tspan x="53.750%" class="br c-10">W</tspan><tspan x="54.375%" class="br c-10">o</tspan><tspan x="55.000%" class="br c-10">r</tspan><tspan x="55.625%" class="br c-10">l</tspan><tspan x="56.250%" class="br c-10">d</tspan><tspan x="57.500%" class="br c-10">i</tspan><tspan x="58.125%" class="br c-10">s</tspan><tspan x="59.375%" class="br c-10">o</tspan><tspan x="60.000%" class="br c-10">n</tspan><tspan x="60.625%" class="br c-10">l</tspan><tspan x="61.250%" class="br c-10">i</tspan><tspan x="61.875%" class="br c-10">n</tspan><tspan x="62.500%" class="br c-10">e</tspan><tspan x="63.750%">H</tspan><tspan x="64.375%">T</tspan><tspan x="65.000%">T</tspan><tspan x="65.625%">P</tspan><tspan x="66.250%">/</tspan><tspan x="66.875%">2</tspan><tspan x="68.125%">2</tspan><tspan x="68.750%">0</tspan><tspan x="69.375%">0</tspan><tspan x="70.625%">O</tspan><tspan x="71.250%">K</tspan>
|
||||
</tspan>
|
||||
<tspan y="88.636%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan><tspan x="50.000%" class="br c-10">H</tspan><tspan x="50.625%" class="br c-10">e</tspan><tspan x="51.250%" class="br c-10">l</tspan><tspan x="51.875%" class="br c-10">l</tspan><tspan x="52.500%" class="br c-10">o</tspan><tspan x="53.750%" class="br c-10">W</tspan><tspan x="54.375%" class="br c-10">o</tspan><tspan x="55.000%" class="br c-10">r</tspan><tspan x="55.625%" class="br c-10">l</tspan><tspan x="56.250%" class="br c-10">d</tspan><tspan x="57.500%" class="br c-10">i</tspan><tspan x="58.125%" class="br c-10">s</tspan><tspan x="59.375%" class="br c-10">o</tspan><tspan x="60.000%" class="br c-10">n</tspan><tspan x="60.625%" class="br c-10">l</tspan><tspan x="61.250%" class="br c-10">i</tspan><tspan x="61.875%" class="br c-10">n</tspan><tspan x="62.500%" class="br c-10">e</tspan><tspan x="63.750%">H</tspan><tspan x="64.375%">T</tspan><tspan x="65.000%">T</tspan><tspan x="65.625%">P</tspan><tspan x="66.250%">/</tspan><tspan x="66.875%">2</tspan><tspan x="68.125%">2</tspan><tspan x="68.750%">0</tspan><tspan x="69.375%">0</tspan><tspan x="70.625%">O</tspan><tspan x="71.250%">K</tspan>
|
||||
</tspan>
|
||||
<tspan y="90.909%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan><tspan x="50.000%" class="br c-10">H</tspan><tspan x="50.625%" class="br c-10">e</tspan><tspan x="51.250%" class="br c-10">l</tspan><tspan x="51.875%" class="br c-10">l</tspan><tspan x="52.500%" class="br c-10">o</tspan><tspan x="53.750%" class="br c-10">W</tspan><tspan x="54.375%" class="br c-10">o</tspan><tspan x="55.000%" class="br c-10">r</tspan><tspan x="55.625%" class="br c-10">l</tspan><tspan x="56.250%" class="br c-10">d</tspan><tspan x="57.500%" class="br c-10">i</tspan><tspan x="58.125%" class="br c-10">s</tspan><tspan x="59.375%" class="br c-10">o</tspan><tspan x="60.000%" class="br c-10">n</tspan><tspan x="60.625%" class="br c-10">l</tspan><tspan x="61.250%" class="br c-10">i</tspan><tspan x="61.875%" class="br c-10">n</tspan><tspan x="62.500%" class="br c-10">e</tspan><tspan x="63.750%">H</tspan><tspan x="64.375%">T</tspan><tspan x="65.000%">T</tspan><tspan x="65.625%">P</tspan><tspan x="66.250%">/</tspan><tspan x="66.875%">2</tspan><tspan x="68.125%">2</tspan><tspan x="68.750%">0</tspan><tspan x="69.375%">0</tspan><tspan x="70.625%">O</tspan><tspan x="71.250%">K</tspan>
|
||||
</tspan>
|
||||
<tspan y="93.182%">
|
||||
<tspan dy="1em" x="49.375%" class="c-2">│</tspan>
|
||||
</tspan>
|
||||
<tspan y="95.455%">
|
||||
<tspan dy="1em" x="0.000%" class="c-0">[</tspan><tspan x="0.625%" class="c-0">d</tspan><tspan x="1.250%" class="c-0">e</tspan><tspan x="1.875%" class="c-0">m</tspan><tspan x="2.500%" class="c-0">o</tspan><tspan x="3.125%" class="c-0">]</tspan><tspan x="4.375%" class="c-0">0</tspan><tspan x="5.000%" class="c-0">:</tspan><tspan x="5.625%" class="c-0">s</tspan><tspan x="6.250%" class="c-0">s</tspan><tspan x="6.875%" class="c-0">h</tspan><tspan x="7.500%" class="c-0">*</tspan><tspan x="75.625%" class="c-0">"</tspan><tspan x="76.250%" class="c-0">r</tspan><tspan x="76.875%" class="c-0">a</tspan><tspan x="77.500%" class="c-0">h</tspan><tspan x="78.125%" class="c-0">u</tspan><tspan x="78.750%" class="c-0">l</tspan><tspan x="79.375%" class="c-0">-</tspan><tspan x="80.000%" class="c-0">T</tspan><tspan x="80.625%" class="c-0">h</tspan><tspan x="81.250%" class="c-0">i</tspan><tspan x="81.875%" class="c-0">n</tspan><tspan x="82.500%" class="c-0">k</tspan><tspan x="83.125%" class="c-0">P</tspan><tspan x="83.750%" class="c-0">a</tspan><tspan x="84.375%" class="c-0">d</tspan><tspan x="85.000%" class="c-0">-</tspan><tspan x="85.625%" class="c-0">E</tspan><tspan x="86.250%" class="c-0">4</tspan><tspan x="86.875%" class="c-0">9</tspan><tspan x="87.500%" class="c-0">0</tspan><tspan x="88.125%" class="c-0">"</tspan><tspan x="89.375%" class="c-0">0</tspan><tspan x="90.000%" class="c-0">1</tspan><tspan x="90.625%" class="c-0">:</tspan><tspan x="91.250%" class="c-0">0</tspan><tspan x="91.875%" class="c-0">2</tspan><tspan x="93.125%" class="c-0">0</tspan><tspan x="93.750%" class="c-0">5</tspan><tspan x="94.375%" class="c-0">-</tspan><tspan x="95.000%" class="c-0">O</tspan><tspan x="95.625%" class="c-0">c</tspan><tspan x="96.250%" class="c-0">t</tspan><tspan x="96.875%" class="c-0">-</tspan><tspan x="97.500%" class="c-0">1</tspan><tspan x="98.125%" class="c-0">9</tspan>
|
||||
</tspan>
|
||||
</text>
|
||||
|
||||
<g transform="translate(-50 -50)">
|
||||
<svg x="50%" y="50%" width="100" height="100">
|
||||
<svg version="1.1" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 866.0254037844387 866.0254037844387">
|
||||
<defs>
|
||||
<mask id="small-triangle-mask">
|
||||
<rect width="100%" height="100%" fill="white"/>
|
||||
<polygon points="508.01270189221935 433.01270189221935, 208.0127018922194 259.8076211353316, 208.01270189221927 606.217782649107" fill="black"></polygon>
|
||||
</mask>
|
||||
</defs>
|
||||
<polygon points="808.0127018922194 433.01270189221935, 58.01270189221947 -1.1368683772161603e-13, 58.01270189221913 866.0254037844386" mask="url(#small-triangle-mask)" fill="white"></polygon>
|
||||
<polyline points="481.2177826491071 333.0127018922194, 134.80762113533166 533.0127018922194" stroke="white" stroke-width="90"></polyline>
|
||||
</svg>
|
||||
</svg>
|
||||
</g>
|
||||
</svg>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 41 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 11 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 354 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 105 KiB |
|
|
@ -0,0 +1,71 @@
|
|||
---
|
||||
id: chaoshub
|
||||
title: Using and contributing to ChaosHub
|
||||
sidebar_label: ChaosHub
|
||||
original_id: chaoshub
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
**Important links**
|
||||
|
||||
Chaos Hub is maintained at https://hub.litmuschaos.io
|
||||
|
||||
To contribute new chaos charts visit: https://github.com/litmuschaos/chaos-charts
|
||||
|
||||
**Introduction**
|
||||
|
||||
Litmus chaos hub is a place where the chaos engineering community members publish their chaos experiments. A set of related chaos experiments are bundled into a `Chaos Chart`. Chaos Charts are classified into the following categories.
|
||||
|
||||
- [Generic Chaos](#generic-chaos)
|
||||
- [Application Chaos](#application-chaos)
|
||||
- [Platform Chaos](#platform-chaos)
|
||||
|
||||
### Generic Chaos
|
||||
|
||||
Chaos actions that apply to generic Kubernetes resources are classified into this category. Following chaos experiments are supported under Generic Chaos Chart
|
||||
|
||||
| Experiment name | Description | User guide link |
|
||||
| ---------------------- | ----------------------------------------------------- | --------------------------------------------------- |
|
||||
| Container Kill | Kill one container in the application pod | [container-kill](container-kill.md) |
|
||||
| Pod Delete | Fail the application pod | [pod-delete](pod-delete.md) |
|
||||
| Pod Network Latency | Experiment to inject network latency to the POD | [pod-network-latency](pod-network-latency.md) |
|
||||
| Pod Network Loss | Experiment to inject network loss to the POD | [pod-network-loss](pod-network-loss.md) |
|
||||
| Node CPU Hog | Exhaust CPU resources on the Kubernetes Node | [node-cpu-hog](node-cpu-hog.md) |
|
||||
| Disk Fill | Fillup Ephemeral Storage of a Resource | [disk-fill](disk-fill.md) |
|
||||
| Disk Loss | External disk loss from the node | [disk-loss](disk-loss.md) |
|
||||
| Node Drain | Drain the node where application pod is scheduled | [node-drain](node-drain.md) |
|
||||
| Pod CPU Hog | Consume CPU resources on the application container | [pod-cpu-hog](pod-cpu-hog.md) |
|
||||
| Pod Network Corruption | Inject Network Packet Corruption Into Application Pod | [pod-network-corruption](pod-network-corruption.md) |
|
||||
|
||||
### Application Chaos
|
||||
|
||||
While Chaos Experiments under the Generic category offer the ability to induce chaos into Kubernetes resources, it is difficult to analyze and conclude if the chaos induced found a weakness in a given application. The application specific chaos experiments are built with some checks on _pre-conditions_ and some expected outcomes after the chaos injection. The result of the chaos experiment is determined by matching the outcome with the expected outcome.
|
||||
|
||||
<div class="danger">
|
||||
<strong>NOTE:</strong> If the result of the chaos experiment is `pass`, it means that the application is resilient to that chaos.
|
||||
</div>
|
||||
|
||||
#### Benefits of contributing an application chaos experiment
|
||||
|
||||
Application developers write negative tests in their CI pipelines to test the resiliency of the applications. These negative can be converted into Litmus Chaos Experiments and contributed to ChaosHub, so that the users of the application can use them in staging/pre-production/production environments to check the resilience. Application environments vary considerably from where they are tested (CI pipelines) to where they are deployed (Production). Hence, running the same chaos tests in the user's environment will help determine the weaknesses of the deployment and fixing such weaknesses leads to increased resilience.
|
||||
|
||||
Following Application Chaos experiments are available on ChaosHub
|
||||
|
||||
| Application | Description | Chaos Experiments |
|
||||
| ----------- | ------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| OpenEBS | Container Attached Storage for Kubernetes | [openebs-pool-pod-failure](openebs-pool-pod-failure.md)<br/>[openebs-pool-container-failure](openebs-pool-container-failure.md)<br/>[openebs-target-pod-failure](openebs-target-pod-failure.md)<br/>[openebs-target-container-failure](openebs-target-container-failure.md)<br/>[openebs-target-network-delay](openebs-target-network-delay.md)<br/>[openebs-target-network-loss](openebs-target-network-loss.md) |
|
||||
| Kafka | Open-source stream processing software | [kafka-broker-pod-failure](kafka-broker-pod-failure.md)<br/>[kafka-broker-disk-failure](kafka-broker-disk-failure.md)<br/> |
|
||||
| CoreDns | CoreDNS is a fast and flexible DNS server that chains plugins | [coredns-pod-delete](coredns-pod-delete.md) |
|
||||
|
||||
### Platform Chaos
|
||||
|
||||
Chaos experiments that inject chaos into the platform resources of Kubernetes are classified into this category. Management of platform resources vary significantly from each other, Chaos Charts may be maintained separately for each platform (For example, AWS, GCP, Azure, etc)
|
||||
|
||||
Following Platform Chaos experiments are available on ChaosHub
|
||||
|
||||
| Platform | Description | Chaos Experiments |
|
||||
| -------- | ------------------------------------------- | ----------------- |
|
||||
| AWS | Amazon Web Services platform. Includes EKS. | None |
|
||||
| GCP | Google Cloud Platform. Includes GKE. | None |
|
||||
| Azure | Microsoft Azure platform. Includes AKS. | None |
|
||||
|
|
@ -0,0 +1,24 @@
|
|||
---
|
||||
id: community
|
||||
title: Join Litmus Community
|
||||
sidebar_label: Community
|
||||
original_id: community
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
Litmus community is a subset of the larger Kubernetes community. Have a question? Want to stay in touch with the happenings on Chaos Engineering on Kubernetes? Join `#litmus` channel on Kubernetes Slack.
|
||||
|
||||
<br/><br/>
|
||||
|
||||
<a href="https://kubernetes.slack.com/messages/CNXNB0ZTN" target="_blank"><img src={require("./assets/join-community.png").default} width="400"/></a>
|
||||
|
||||
<br/>
|
||||
|
||||
<br/>
|
||||
|
||||
<hr/>
|
||||
|
||||
<br/>
|
||||
|
||||
<br/>
|
||||
|
|
@ -1,10 +1,11 @@
|
|||
---
|
||||
id: version-1.1.0-container-kill
|
||||
id: container-kill
|
||||
title: Container Kill Experiment Details
|
||||
sidebar_label: Container Kill
|
||||
original_id: container-kill
|
||||
---
|
||||
------
|
||||
|
||||
---
|
||||
|
||||
## Experiment Metadata
|
||||
|
||||
|
|
@ -38,13 +39,13 @@ original_id: container-kill
|
|||
## Details
|
||||
|
||||
- Pumba chaoslib details
|
||||
- Kills one container in the specified application pod by sending SIGKILL termination signal to its docker socket (hence docker runtime is required)
|
||||
- Containers are killed using the `kill` command provided by [pumba](https://github.com/alexei-led/pumba)
|
||||
- Pumba is run as a daemonset on all nodes in dry-run mode to begin with the `kill` command is issued during experiment execution via `kubectl exec`
|
||||
- Kills one container in the specified application pod by sending SIGKILL termination signal to its docker socket (hence docker runtime is required)
|
||||
- Containers are killed using the `kill` command provided by [pumba](https://github.com/alexei-led/pumba)
|
||||
- Pumba is run as a daemonset on all nodes in dry-run mode to begin with the `kill` command is issued during experiment execution via `kubectl exec`
|
||||
- Containerd chaoslib details
|
||||
- Kills one container in the specified application pod by `crictl-chaos` Lib.
|
||||
- Containers are killed using the `crictl stop` command.
|
||||
- containerd-chaos is run as a daemonset on all nodes in dry-run mode to begin with the `stop` command is issued during experiment execution via `kubectl exec`
|
||||
- Kills one container in the specified application pod by `crictl-chaos` Lib.
|
||||
- Containers are killed using the `crictl stop` command.
|
||||
- containerd-chaos is run as a daemonset on all nodes in dry-run mode to begin with the `stop` command is issued during experiment execution via `kubectl exec`
|
||||
- Tests deployment sanity (replica availability & uninterrupted service) and recovery workflow of the application
|
||||
- Good for testing recovery of pods having side-car containers
|
||||
|
||||
|
|
@ -65,7 +66,8 @@ original_id: container-kill
|
|||
|
||||
#### Sample Rbac Manifest
|
||||
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/generic/container-kill/rbac.yaml yaml)
|
||||
[embedmd]: # "https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/generic/container-kill/rbac.yaml yaml"
|
||||
|
||||
```yaml
|
||||
---
|
||||
apiVersion: v1
|
||||
|
|
@ -84,9 +86,18 @@ metadata:
|
|||
labels:
|
||||
name: container-kill-sa
|
||||
rules:
|
||||
- apiGroups: ["","litmuschaos.io","batch","apps"]
|
||||
resources: ["pods","jobs","daemonsets","pods/exec","chaosengines","chaosexperiments","chaosresults"]
|
||||
verbs: ["create","list","get","patch","update","delete"]
|
||||
- apiGroups: ["", "litmuschaos.io", "batch", "apps"]
|
||||
resources:
|
||||
[
|
||||
"pods",
|
||||
"jobs",
|
||||
"daemonsets",
|
||||
"pods/exec",
|
||||
"chaosengines",
|
||||
"chaosexperiments",
|
||||
"chaosresults",
|
||||
]
|
||||
verbs: ["create", "list", "get", "patch", "update", "delete"]
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
kind: RoleBinding
|
||||
|
|
@ -100,10 +111,9 @@ roleRef:
|
|||
kind: Role
|
||||
name: container-kill-sa
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: container-kill-sa
|
||||
namespace: default
|
||||
|
||||
- kind: ServiceAccount
|
||||
name: container-kill-sa
|
||||
namespace: default
|
||||
```
|
||||
|
||||
### Prepare ChaosEngine
|
||||
|
|
@ -148,7 +158,8 @@ subjects:
|
|||
|
||||
#### Sample ChaosEngine Manifest
|
||||
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/generic/container-kill/engine.yaml yaml)
|
||||
[embedmd]: # "https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/generic/container-kill/engine.yaml yaml"
|
||||
|
||||
```yaml
|
||||
apiVersion: litmuschaos.io/v1alpha1
|
||||
kind: ChaosEngine
|
||||
|
|
@ -157,19 +168,19 @@ metadata:
|
|||
namespace: default
|
||||
spec:
|
||||
# It can be true/false
|
||||
annotationCheck: 'true'
|
||||
annotationCheck: "true"
|
||||
# It can be active/stop
|
||||
engineState: 'active'
|
||||
engineState: "active"
|
||||
#ex. values: ns1:name=percona,ns2:run=nginx
|
||||
auxiliaryAppInfo: ''
|
||||
auxiliaryAppInfo: ""
|
||||
appinfo:
|
||||
appns: 'default'
|
||||
applabel: 'app=nginx'
|
||||
appkind: 'deployment'
|
||||
appns: "default"
|
||||
applabel: "app=nginx"
|
||||
appkind: "deployment"
|
||||
chaosServiceAccount: container-kill-sa
|
||||
monitoring: false
|
||||
# It can be delete/retain
|
||||
jobCleanUpPolicy: 'delete'
|
||||
jobCleanUpPolicy: "delete"
|
||||
experiments:
|
||||
- name: container-kill
|
||||
spec:
|
||||
|
|
@ -177,7 +188,7 @@ spec:
|
|||
env:
|
||||
# specify the name of the container to be killed
|
||||
- name: TARGET_CONTAINER
|
||||
value: 'nginx'
|
||||
value: "nginx"
|
||||
```
|
||||
|
||||
### Create the ChaosEngine Resource
|
||||
|
|
@ -194,7 +205,7 @@ spec:
|
|||
|
||||
### Check Chaos Experiment Result
|
||||
|
||||
- Check whether the application is resilient to the container kill, once the experiment (job) is completed. The ChaosResult resource name is derived like this: `<ChaosEngine-Name>-<ChaosExperiment-Name>`.
|
||||
- Check whether the application is resilient to the container kill, once the experiment (job) is completed. The ChaosResult resource name is derived like this: `{"<ChaosEngine-Name>-<ChaosExperiment-Name>"}`.
|
||||
|
||||
`kubectl describe chaosresult nginx-chaos-container-kill -n <application-namespace>`
|
||||
|
||||
|
|
@ -1,10 +1,11 @@
|
|||
---
|
||||
id: version-1.1.0-coredns-pod-delete
|
||||
id: coredns-pod-delete
|
||||
title: CoreDNS Pod Delete Experiment Details
|
||||
sidebar_label: CoreDNS Pod Delete
|
||||
original_id: coredns-pod-delete
|
||||
---
|
||||
------
|
||||
|
||||
---
|
||||
|
||||
## Experiment Metadata
|
||||
|
||||
|
|
@ -22,6 +23,7 @@ original_id: coredns-pod-delete
|
|||
</table>
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Ensure that the Litmus Chaos Operator is running by executing `kubectl get pods` in operator namespace (typically, `litmus`). If not, install from [here](https://docs.litmuschaos.io/docs/getstarted/#install-litmus)
|
||||
- Ensure that the `coredns-pod-delete` experiment resource is available in the cluster by executing `kubectl get chaosexperiments` in the desired namespace. If not, install from [here](https://hub.litmuschaos.io/api/chaos?file=charts/coredns/coredns-pod-delete/experiment.yaml)
|
||||
|
||||
|
|
@ -52,11 +54,13 @@ original_id: coredns-pod-delete
|
|||
- Follow the steps in the sections below to create the chaosServiceAccount, prepare the ChaosEngine & execute the experiment.
|
||||
|
||||
## Prepare chaosServiceAccount
|
||||
|
||||
- Use this sample RBAC manifest to create a chaosServiceAccount in the desired (app) namespace. This example consists of the minimum necessary role permissions to execute the experiment.
|
||||
|
||||
### Sample Rbac Manifest
|
||||
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/coredns/coredns-pod-delete/rbac.yaml yaml)
|
||||
[embedmd]: # "https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/coredns/coredns-pod-delete/rbac.yaml yaml"
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
|
|
@ -73,9 +77,17 @@ metadata:
|
|||
labels:
|
||||
name: coredns-pod-delete-sa
|
||||
rules:
|
||||
- apiGroups: ["","litmuschaos.io","batch"]
|
||||
resources: ["services", "pods","jobs","chaosengines","chaosexperiments","chaosresults"]
|
||||
verbs: ["create","list","get","patch","update","delete"]
|
||||
- apiGroups: ["", "litmuschaos.io", "batch"]
|
||||
resources:
|
||||
[
|
||||
"services",
|
||||
"pods",
|
||||
"jobs",
|
||||
"chaosengines",
|
||||
"chaosexperiments",
|
||||
"chaosresults",
|
||||
]
|
||||
verbs: ["create", "list", "get", "patch", "update", "delete"]
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
kind: ClusterRoleBinding
|
||||
|
|
@ -88,14 +100,15 @@ roleRef:
|
|||
kind: ClusterRole
|
||||
name: coredns-pod-delete-sa
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: coredns-pod-delete-sa
|
||||
namespace: kube-system
|
||||
- kind: ServiceAccount
|
||||
name: coredns-pod-delete-sa
|
||||
namespace: kube-system
|
||||
```
|
||||
|
||||
### Prepare ChaosEngine
|
||||
|
||||
- Provide the application info in `spec.appinfo`
|
||||
|
||||
- It will be default as
|
||||
```
|
||||
appinfo:
|
||||
|
|
@ -143,7 +156,8 @@ subjects:
|
|||
|
||||
#### Sample ChaosEngine Manifest
|
||||
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/coredns/coredns-pod-delete/engine.yaml yaml)
|
||||
[embedmd]: # "https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/coredns/coredns-pod-delete/engine.yaml yaml"
|
||||
|
||||
```yaml
|
||||
apiVersion: litmuschaos.io/v1alpha1
|
||||
kind: ChaosEngine
|
||||
|
|
@ -152,19 +166,19 @@ metadata:
|
|||
namespace: kube-system
|
||||
spec:
|
||||
appinfo:
|
||||
appns: 'kube-system'
|
||||
applabel: 'k8s-app=kube-dns'
|
||||
appkind: 'deployment'
|
||||
appns: "kube-system"
|
||||
applabel: "k8s-app=kube-dns"
|
||||
appkind: "deployment"
|
||||
# It can be true/false
|
||||
annotationCheck: 'false'
|
||||
annotationCheck: "false"
|
||||
# It can be active/stop
|
||||
engineState: 'active'
|
||||
engineState: "active"
|
||||
#ex. values: ns1:name=percona,ns2:run=nginx
|
||||
auxiliaryAppInfo: ''
|
||||
auxiliaryAppInfo: ""
|
||||
chaosServiceAccount: coredns-pod-delete-sa
|
||||
monitoring: false
|
||||
# It can be delete/retain
|
||||
jobCleanUpPolicy: 'delete'
|
||||
jobCleanUpPolicy: "delete"
|
||||
experiments:
|
||||
- name: coredns-pod-delete
|
||||
spec:
|
||||
|
|
@ -172,14 +186,14 @@ spec:
|
|||
env:
|
||||
# set chaos duration (in sec) as desired
|
||||
- name: TOTAL_CHAOS_DURATION
|
||||
value: '30'
|
||||
value: "30"
|
||||
|
||||
# set chaos interval (in sec) as desired
|
||||
- name: CHAOS_INTERVAL
|
||||
value: '10'
|
||||
value: "10"
|
||||
|
||||
- name: CHAOS_NAMESPACE
|
||||
value: 'kube-system'
|
||||
value: "kube-system"
|
||||
```
|
||||
|
||||
### Create the ChaosEngine Resource
|
||||
|
|
@ -196,6 +210,6 @@ spec:
|
|||
|
||||
### Check Chaos Experiment Result
|
||||
|
||||
- Check whether the application is resilient to the coredns pod failure, once the experiment (job) is completed. The ChaosResult resource name is derived like this: `<ChaosEngine-Name>-<ChaosExperiment-Name>`.
|
||||
- Check whether the application is resilient to the coredns pod failure, once the experiment (job) is completed. The ChaosResult resource name is derived like this: `{"<ChaosEngine-Name>-<ChaosExperiment-Name>"}`.
|
||||
|
||||
`kubectl describe chaosresult engine-coredns-coredns-pod-delete -n <chaos-namespace>`
|
||||
|
|
@ -0,0 +1,179 @@
|
|||
---
|
||||
id: cpu-hog
|
||||
title: CPU Hog Experiment Details
|
||||
sidebar_label: CPU Hog
|
||||
original_id: cpu-hog
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
## Experiment Metadata
|
||||
|
||||
| Type | Description | Tested K8s Platform |
|
||||
| ------- | -------------------------------------------- | ------------------- |
|
||||
| Generic | Exhaust CPU resources on the Kubernetes Node | GKE |
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Ensure that the Litmus Chaos Operator is running by executing `kubectl get pods` in operator namespace (typically, `litmus`). If not, install from [here](https://raw.githubusercontent.com/litmuschaos/pages/master/docs/litmus-operator-latest.yaml)
|
||||
- Ensure that the `cpu-hog` experiment resource is available in the cluster by executing `kubectl get chaosexperiments` in the desired namespace. If not, install from [here](https://hub.litmuschaos.io/charts/generic/experiments/cpu-hog)
|
||||
- There should be administrative access to the platform on which the Kubernetes cluster is hosted, as the recovery of the affected node could be manual. For example, gcloud access to the GKE project
|
||||
|
||||
## Entry Criteria
|
||||
|
||||
- Application pods are healthy on the respective Nodes before chaos injection
|
||||
|
||||
## Exit Criteria
|
||||
|
||||
- Application pods may or may not be healthy post chaos injection
|
||||
|
||||
## Details
|
||||
|
||||
- This experiment causes CPU resource exhaustion on the Kubernetes node. The experiment aims to verify resiliency of applications whose replicas may be evicted on account on nodes turning unschedulable (Not Ready) due to lack of CPU resources.
|
||||
- The CPU chaos is injected using a daemonset running the linux stress tool (a workload generator). The chaos is effected for a period equalling the TOTAL_CHAOS_DURATION
|
||||
- Application implies services. Can be reframed as:
|
||||
Tests application resiliency upon replica evictions caused due to lack of CPU resources
|
||||
|
||||
## Integrations
|
||||
|
||||
- CPU Hog can be effected using the chaos library: `litmus`
|
||||
- The desired chaos library can be selected by setting `litmus` as value for the env variable `LIB`
|
||||
|
||||
## Steps to Execute the Chaos Experiment
|
||||
|
||||
- This Chaos Experiment can be triggered by creating a ChaosEngine resource on the cluster. To understand the values to provide in a ChaosEngine specification, refer [Getting Started](getstarted.md/#prepare-chaosengine)
|
||||
|
||||
- Follow the steps in the sections below to create the chaosServiceAccount, prepare the ChaosEngine & execute the experiment.
|
||||
|
||||
### Prepare chaosServiceAccount
|
||||
|
||||
- Use this sample RBAC manifest to create a chaosServiceAccount in the desired (app) namespace. This example consists of the minimum necessary role permissions to execute the experiment.
|
||||
|
||||
#### Sample Rbac Manifest
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: nginx-sa
|
||||
namespace: default
|
||||
labels:
|
||||
name: nginx-sa
|
||||
---
|
||||
# Source: openebs/templates/clusterrole.yaml
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
kind: ClusterRole
|
||||
metadata:
|
||||
name: nginx-sa
|
||||
labels:
|
||||
name: nginx-sa
|
||||
rules:
|
||||
- apiGroups: ["", "litmuschaos.io", "batch", "apps"]
|
||||
resources:
|
||||
[
|
||||
"pods",
|
||||
"daemonsets",
|
||||
"jobs",
|
||||
"pods/exec",
|
||||
"chaosengines",
|
||||
"chaosexperiments",
|
||||
"chaosresults",
|
||||
]
|
||||
verbs: ["create", "list", "get", "patch", "update", "delete"]
|
||||
- apiGroups: [""]
|
||||
resources: ["nodes"]
|
||||
verbs: ["get", "list"]
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
kind: ClusterRoleBinding
|
||||
metadata:
|
||||
name: nginx-sa
|
||||
labels:
|
||||
name: nginx-sa
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: ClusterRole
|
||||
name: nginx-sa
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: nginx-sa
|
||||
namespace: default
|
||||
```
|
||||
|
||||
### Prepare ChaosEngine
|
||||
|
||||
- Provide the application info in `spec.appinfo`
|
||||
- Provide the auxiliary applications info (ns & labels) in `spec.auxiliaryAppInfo`
|
||||
- Override the experiment tunables if desired
|
||||
|
||||
#### Supported Experiment Tunables
|
||||
|
||||
| Variables | Description | Type | Notes |
|
||||
| -------------------- | --------------------------------------------------- | --------- | ----------------------------------------- |
|
||||
| TOTAL_CHAOS_DURATION | The time duration for chaos insertion (seconds) | Optional | Defaults to 60s |
|
||||
| PLATFORM | The platform on which the chaos experiment will run | Mandatory | Defaults to GKE |
|
||||
| LIB | The chaos lib used to inject the chaos | Optional | Defaults to `litmus`. Supported: `litmus` |
|
||||
|
||||
#### Sample ChaosEngine Manifest
|
||||
|
||||
```yaml
|
||||
apiVersion: litmuschaos.io/v1alpha1
|
||||
kind: ChaosEngine
|
||||
metadata:
|
||||
name: nginx-chaos
|
||||
namespace: default
|
||||
spec:
|
||||
chaosType: 'infra' # It can be app/infra
|
||||
auxiliaryAppInfo: "ns1:name=percona,ns2:run=nginx"
|
||||
appinfo:
|
||||
appns: default
|
||||
applabel: 'app=nginx'
|
||||
appkind: deployment
|
||||
# It can be app/infra
|
||||
chaosType: 'infra'
|
||||
#ex. values: ns1:name=percona,ns2:run=nginx
|
||||
auxiliaryAppInfo: ""
|
||||
chaosServiceAccount: nginx-sa
|
||||
monitoring: false
|
||||
components:
|
||||
runner:
|
||||
image: "litmuschaos/chaos-executor:1.0.0"
|
||||
type: "go"
|
||||
# It can be delete/retain
|
||||
jobCleanUpPolicy: delete
|
||||
experiments:
|
||||
- name: cpu-hog
|
||||
spec:
|
||||
components:
|
||||
# set chaos duration (in sec) as desired
|
||||
- name: TOTAL_CHAOS_DURATION
|
||||
value: '60'
|
||||
# set chaos platform as desired
|
||||
- name: PLATFORM
|
||||
value: 'GKE'
|
||||
# chaos lib used to inject the chaos
|
||||
- name: LIB
|
||||
value: 'litmus'
|
||||
```
|
||||
|
||||
### Create the ChaosEngine Resource
|
||||
|
||||
- Create the ChaosEngine manifest prepared in the previous step to trigger the Chaos.
|
||||
|
||||
`kubectl apply -f chaosengine.yml`
|
||||
|
||||
### Watch Chaos progress
|
||||
|
||||
- Setting up a watch of the CPU consumed by nodes in the Kubernetes Cluster
|
||||
|
||||
`watch kubectl top nodes`
|
||||
|
||||
### Check Chaos Experiment Result
|
||||
|
||||
- Check whether the application is resilient to the CPU hog, once the experiment (job) is completed. The ChaosResult resource name is derived like this: `<ChaosEngine-Name>-<ChaosExperiment-Name>`.
|
||||
|
||||
`kubectl describe chaosresult nginx-chaos-cpu-hog -n <application-namespace>`
|
||||
|
||||
## Application Pod Failure Demo
|
||||
|
||||
- A sample recording of this experiment execution is provided here.
|
||||
|
|
@ -1,10 +1,11 @@
|
|||
---
|
||||
id: version-1.1.0-cStor-pool-chaos
|
||||
id: cStor-pool-chaos
|
||||
title: cStor Pool Chaos Experiment Details
|
||||
sidebar_label: cStor Pool Chaos
|
||||
original_id: cStor-pool-chaos
|
||||
---
|
||||
------
|
||||
|
||||
---
|
||||
|
||||
## Experiment Metadata
|
||||
|
||||
|
|
@ -73,12 +74,21 @@ metadata:
|
|||
labels:
|
||||
name: cStor-pool-chaos-sa
|
||||
rules:
|
||||
- apiGroups: ["","litmuschaos.io","batch","apps"]
|
||||
resources: ["pods","deployments","jobs","configmaps","chaosengines","chaosexperiments","chaosresults"]
|
||||
verbs: ["create","list","get","patch","update","delete"]
|
||||
- apiGroups: [""]
|
||||
resources: ["nodes"]
|
||||
verbs : ["get","list"]
|
||||
- apiGroups: ["", "litmuschaos.io", "batch", "apps"]
|
||||
resources:
|
||||
[
|
||||
"pods",
|
||||
"deployments",
|
||||
"jobs",
|
||||
"configmaps",
|
||||
"chaosengines",
|
||||
"chaosexperiments",
|
||||
"chaosresults",
|
||||
]
|
||||
verbs: ["create", "list", "get", "patch", "update", "delete"]
|
||||
- apiGroups: [""]
|
||||
resources: ["nodes"]
|
||||
verbs: ["get", "list"]
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
kind: RoleBinding
|
||||
|
|
@ -92,9 +102,9 @@ roleRef:
|
|||
kind: Role
|
||||
name: cStor-pool-chaos-sa
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: cStor-pool-chaos-sa
|
||||
namespace: openebs
|
||||
- kind: ServiceAccount
|
||||
name: cStor-pool-chaos-sa
|
||||
namespace: openebs
|
||||
```
|
||||
|
||||
### Prepare ChaosEngine
|
||||
|
|
@ -135,15 +145,15 @@ metadata:
|
|||
namespace: openebs
|
||||
spec:
|
||||
appinfo:
|
||||
appns: 'openebs'
|
||||
applabel: 'app=cstor-pool'
|
||||
appkind: 'deployment'
|
||||
appns: "openebs"
|
||||
applabel: "app=cstor-pool"
|
||||
appkind: "deployment"
|
||||
# It can be true/false
|
||||
annotationCheck: 'false'
|
||||
annotationCheck: "false"
|
||||
chaosServiceAccount: cStor-pool-chaos-sa
|
||||
monitoring: false
|
||||
# It can be delete/retain
|
||||
jobCleanUpPolicy: 'delete'
|
||||
jobCleanUpPolicy: "delete"
|
||||
experiments:
|
||||
- name: openebs-pool-pod-failure
|
||||
spec:
|
||||
|
|
@ -151,10 +161,10 @@ spec:
|
|||
env:
|
||||
# Namespace where openebs has been installed
|
||||
- name: OPENEBS_NS
|
||||
value: 'openebs'
|
||||
value: "openebs"
|
||||
# please leave it blank, for this experiment
|
||||
- name: APP_PVC
|
||||
value: ''
|
||||
value: ""
|
||||
```
|
||||
|
||||
### Create the ChaosEngine Resource
|
||||
|
|
@ -171,7 +181,7 @@ spec:
|
|||
|
||||
### Check Chaos Experiment Result
|
||||
|
||||
- Check whether the cStor pool pod is resilient to the pod failure, once the experiment (job) is completed. The ChaosResult resource name is derived like this: `<ChaosEngine-Name>-<ChaosExperiment-Name>`.
|
||||
- Check whether the cStor pool pod is resilient to the pod failure, once the experiment (job) is completed. The ChaosResult resource name is derived like this: `{"<ChaosEngine-Name>-<ChaosExperiment-Name>"}`.
|
||||
|
||||
`kubectl describe chaosresult cStor-pool-chaos-openebs-pool-pod-failure -n openebs`
|
||||
|
||||
|
|
@ -0,0 +1,67 @@
|
|||
---
|
||||
id: devguide
|
||||
title: Developer Guide for Chaos Charts
|
||||
sidebar_label: Developer Guide
|
||||
original_id: devguide
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
This page serves as a guide to develop either a new Chaos Chart or a new experiment in a Chaos Chart which are published at <a href="https://hub.litmuschaos.io" target="_blank">ChaosHub</a>.
|
||||
|
||||
Below are some key points to remember before understanding how to write a new chart or an experiment.
|
||||
|
||||
> Chaos Charts repository : https://github.com/litmuschaos/chaos-charts
|
||||
>
|
||||
> Litmusbooks repository : https://github.com/litmuschaos/litmus-ansible/tree/master/experiments
|
||||
>
|
||||
> Website rendering code repository: https://github.com/litmuschaos/charthub.litmuschaos.io
|
||||
|
||||
The experiments & chaos libraries are typically written in Ansible, though not mandatory. Ensure that
|
||||
the experiments can be executed in a container & can read/update the litmuschaos custom resources. For example,
|
||||
if you are writing an experiment in Go, use this [clientset](https://github.com/litmuschaos/chaos-operator/tree/master/pkg/client)
|
||||
|
||||
<hr/>
|
||||
|
||||
## Glossary
|
||||
|
||||
### Chaos Chart
|
||||
|
||||
A group of Choas Experiments put together in a YAML file. Each group or chart has a metadata manifest called `ChartServiceVersion`
|
||||
that holds data such as `ChartVersion`, `Contributors`, `Description`, `links` etc.., This metadata is rendered on the ChartHub.
|
||||
A chaos chart also consists of a `package` manifest that is an index of available experiments in the chart.
|
||||
|
||||
Here is an example of the [ChartServiceVersion](https://github.com/litmuschaos/chaos-charts/blob/master/charts/generic/generic.chartserviceversion.yaml) & [package](https://github.com/litmuschaos/chaos-charts/blob/master/charts/generic/generic.package.yaml) manifests of the generic chaos chart.
|
||||
|
||||
### Chaos Experiment
|
||||
|
||||
ChaosExperiment is a CRD that specifies the nature of a Chaos Experiment. The YAML file that constitutes a Chaos Experiment CR
|
||||
is stored under a Chaos Chart of ChaosHub and typically consists of low-level chaos parameters specific to that experiment, set
|
||||
to their default values.
|
||||
|
||||
Here is an example chaos experiment CR for a [pod-delete](https://github.com/litmuschaos/chaos-charts/blob/master/charts/generic/pod-delete/experiment.yaml) experiment
|
||||
|
||||
### Litmus Book
|
||||
|
||||
Litmus book is an `ansible` playbook that encompasses the logic of pre-checks, chaos-injection, post-checks, and result-updates.
|
||||
Typically, these are accompanied by a Kubernetes job that can execute the respective playbook.
|
||||
|
||||
Here is an example of the litmus book for the [pod-delete](https://github.com/litmuschaos/litmus-ansible/tree/master/experiments/generic/pod_delete) experiment.
|
||||
|
||||
### Chaos functions
|
||||
|
||||
The `ansible` business logic inside Litmus books can make use of readily available chaos functions. The chaos functions are available as `task-files` which are wrapped in one of the chaos libraries. See [plugins](plugins.md) for more details.
|
||||
|
||||
<hr/>
|
||||
|
||||
## Developing a Chaos Experiment
|
||||
|
||||
A detailed how-to guide on developing chaos experiments is available [here](https://github.com/litmuschaos/litmus-ansible/tree/master/contribute/developer_guide)
|
||||
|
||||
<br/>
|
||||
|
||||
<hr/>
|
||||
|
||||
<br/>
|
||||
|
||||
<br/>
|
||||
|
|
@ -1,10 +1,11 @@
|
|||
---
|
||||
id: version-1.1.0-disk-fill
|
||||
id: disk-fill
|
||||
title: Disk Fill Experiment Details
|
||||
sidebar_label: Disk Fill
|
||||
original_id: disk-fill
|
||||
---
|
||||
------
|
||||
|
||||
---
|
||||
|
||||
## Experiment Metadata
|
||||
|
||||
|
|
@ -58,24 +59,24 @@ spec:
|
|||
|
||||
## Entry-Criteria
|
||||
|
||||
- Application pods are healthy before chaos injection.
|
||||
- Application pods are healthy before chaos injection.
|
||||
|
||||
## Exit-Criteria
|
||||
|
||||
- Application pods are healthy post chaos injection.
|
||||
- Application pods are healthy post chaos injection.
|
||||
|
||||
## Details
|
||||
|
||||
- Causes Disk Stress by filling up the ephemeral storage of the pod (in the /var/lib/docker/container/{{container_id}}) on any given node.
|
||||
- Causes the application pod to get evicted if the capacity filled exceeds the pod's ephemeral storage limit.
|
||||
- Tests the Ephemeral Storage Limits, to ensure those parameters are sufficient.
|
||||
- Tests the application's resiliency to disk stress/replica evictions.
|
||||
- Causes Disk Stress by filling up the ephemeral storage of the pod (in the /var/lib/docker/container/{{container_id}}) on any given node.
|
||||
- Causes the application pod to get evicted if the capacity filled exceeds the pod's ephemeral storage limit.
|
||||
- Tests the Ephemeral Storage Limits, to ensure those parameters are sufficient.
|
||||
- Tests the application's resiliency to disk stress/replica evictions.
|
||||
|
||||
## Integrations
|
||||
|
||||
- Disk Fill can be effected using the chaos library: `litmus`, which makes use of `dd` to create a file of
|
||||
specified capacity on the node.
|
||||
- The desired chaoslib can be selected by setting the above options as value for the env variable `LIB`
|
||||
- Disk Fill can be effected using the chaos library: `litmus`, which makes use of `dd` to create a file of
|
||||
specified capacity on the node.
|
||||
- The desired chaoslib can be selected by setting the above options as value for the env variable `LIB`
|
||||
|
||||
## Steps to Execute the Chaos Experiment
|
||||
|
||||
|
|
@ -89,7 +90,8 @@ spec:
|
|||
|
||||
#### Sample Rbac Manifest
|
||||
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/generic/disk-fill/rbac.yaml yaml)
|
||||
[embedmd]: # "https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/generic/disk-fill/rbac.yaml yaml"
|
||||
|
||||
```yaml
|
||||
---
|
||||
apiVersion: v1
|
||||
|
|
@ -107,9 +109,18 @@ metadata:
|
|||
labels:
|
||||
name: disk-fill-sa
|
||||
rules:
|
||||
- apiGroups: ["","apps","litmuschaos.io","batch"]
|
||||
resources: ["pods","jobs","pods/exec","daemonsets","chaosengines","chaosexperiments","chaosresults"]
|
||||
verbs: ["create","list","get","patch","update","delete"]
|
||||
- apiGroups: ["", "apps", "litmuschaos.io", "batch"]
|
||||
resources:
|
||||
[
|
||||
"pods",
|
||||
"jobs",
|
||||
"pods/exec",
|
||||
"daemonsets",
|
||||
"chaosengines",
|
||||
"chaosexperiments",
|
||||
"chaosresults",
|
||||
]
|
||||
verbs: ["create", "list", "get", "patch", "update", "delete"]
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
kind: ClusterRoleBinding
|
||||
|
|
@ -122,9 +133,9 @@ roleRef:
|
|||
kind: ClusterRole
|
||||
name: disk-fill-sa
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: disk-fill-sa
|
||||
namespace: default
|
||||
- kind: ServiceAccount
|
||||
name: disk-fill-sa
|
||||
namespace: default
|
||||
```
|
||||
|
||||
### Prepare ChaosEngine
|
||||
|
|
@ -182,7 +193,8 @@ subjects:
|
|||
|
||||
#### Sample ChaosEngine Manifest
|
||||
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/generic/disk-fill/engine.yaml yaml)
|
||||
[embedmd]: # "https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/generic/disk-fill/engine.yaml yaml"
|
||||
|
||||
```yaml
|
||||
apiVersion: litmuschaos.io/v1alpha1
|
||||
kind: ChaosEngine
|
||||
|
|
@ -191,19 +203,19 @@ metadata:
|
|||
namespace: default
|
||||
spec:
|
||||
# It can be true/false
|
||||
annotationCheck: 'false'
|
||||
annotationCheck: "false"
|
||||
# It can be active/stop
|
||||
engineState: 'active'
|
||||
engineState: "active"
|
||||
#ex. values: ns1:name=percona,ns2:run=nginx
|
||||
auxiliaryAppInfo: ''
|
||||
auxiliaryAppInfo: ""
|
||||
appinfo:
|
||||
appns: 'default'
|
||||
applabel: 'app=nginx'
|
||||
appkind: 'deployment'
|
||||
appns: "default"
|
||||
applabel: "app=nginx"
|
||||
appkind: "deployment"
|
||||
chaosServiceAccount: disk-fill-sa
|
||||
monitoring: false
|
||||
# It can be delete/retain
|
||||
jobCleanUpPolicy: 'delete'
|
||||
jobCleanUpPolicy: "delete"
|
||||
experiments:
|
||||
- name: disk-fill
|
||||
spec:
|
||||
|
|
@ -211,9 +223,9 @@ spec:
|
|||
env:
|
||||
# specify the fill percentage according to the disk pressure required
|
||||
- name: FILL_PERCENTAGE
|
||||
value: '80'
|
||||
value: "80"
|
||||
- name: TARGET_CONTAINER
|
||||
value: 'nginx'
|
||||
value: "nginx"
|
||||
```
|
||||
|
||||
### Create the ChaosEngine Resource
|
||||
|
|
@ -234,7 +246,7 @@ spec:
|
|||
|
||||
### Check Chaos Experiment Result
|
||||
|
||||
- Check whether the application is resilient to the container kill, once the experiment (job) is completed. The ChaosResult resource name is derived like this: `<ChaosEngine-Name>-<ChaosExperiment-Name>`.
|
||||
- Check whether the application is resilient to the container kill, once the experiment (job) is completed. The ChaosResult resource name is derived like this: `{"<ChaosEngine-Name>-<ChaosExperiment-Name>"}`.
|
||||
|
||||
`kubectl describe chaosresult nginx-chaos-disk-fill -n <application-namespace>`
|
||||
|
||||
|
|
@ -1,10 +1,12 @@
|
|||
---
|
||||
id: version-1.1.0-disk-loss
|
||||
id: disk-loss
|
||||
title: Disk Loss Experiment Details
|
||||
sidebar_label: Disk Loss
|
||||
original_id: disk-loss
|
||||
---
|
||||
------
|
||||
|
||||
---
|
||||
|
||||
## Experiment Metadata
|
||||
|
||||
<table>
|
||||
|
|
@ -21,10 +23,11 @@ original_id: disk-loss
|
|||
</table>
|
||||
|
||||
## Prerequisites
|
||||
- Ensure that the Litmus Chaos Operator is running by executing `kubectl get pods` in operator namespace (typically, `litmus`). If not, install from [here](https://raw.githubusercontent.com/litmuschaos/pages/master/docs/litmus-operator-latest.yaml)
|
||||
- Ensure that the `disk-loss` experiment resource is available in the cluster by `kubectl get chaosexperiments` in the desired namespace. If not, install from <a href="https://hub.litmuschaos.io/charts/generic/experiments/disk-loss" target="_blank">here</a>
|
||||
- Ensure to create a Kubernetes secret having the gcloud/aws access configuration(key) in the namespace of `CHAOS_NAMESPACE`.
|
||||
- There should be administrative access to the platform on which the cluster is hosted, as the recovery of the affected node could be manual. Example gcloud access to the project
|
||||
|
||||
- Ensure that the Litmus Chaos Operator is running by executing `kubectl get pods` in operator namespace (typically, `litmus`). If not, install from [here](https://raw.githubusercontent.com/litmuschaos/pages/master/docs/litmus-operator-latest.yaml)
|
||||
- Ensure that the `disk-loss` experiment resource is available in the cluster by `kubectl get chaosexperiments` in the desired namespace. If not, install from <a href="https://hub.litmuschaos.io/charts/generic/experiments/disk-loss" target="_blank">here</a>
|
||||
- Ensure to create a Kubernetes secret having the gcloud/aws access configuration(key) in the namespace of `CHAOS_NAMESPACE`.
|
||||
- There should be administrative access to the platform on which the cluster is hosted, as the recovery of the affected node could be manual. Example gcloud access to the project
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
|
|
@ -39,30 +42,30 @@ stringData:
|
|||
|
||||
## Entry-Criteria
|
||||
|
||||
- The disk is healthy before chaos injection
|
||||
- The disk is healthy before chaos injection
|
||||
|
||||
## Exit-Criteria
|
||||
|
||||
- The disk is healthy post chaos injection
|
||||
- If `APP_CHECK` is true, the application pod health is checked post chaos injection
|
||||
- The disk is healthy post chaos injection
|
||||
- If `APP_CHECK` is true, the application pod health is checked post chaos injection
|
||||
|
||||
## Details
|
||||
|
||||
- In this experiment, the external disk is detached from the node for a period equal to the `TOTAL_CHAOS_DURATION`.
|
||||
- This chaos experiment is supported on GKE and AWS platforms.
|
||||
- If the disk is created as part of dynamic persistent volume, it is expected to re-attach automatically. The experiment re-attaches the disk if it is not already attached.
|
||||
- In this experiment, the external disk is detached from the node for a period equal to the `TOTAL_CHAOS_DURATION`.
|
||||
- This chaos experiment is supported on GKE and AWS platforms.
|
||||
- If the disk is created as part of dynamic persistent volume, it is expected to re-attach automatically. The experiment re-attaches the disk if it is not already attached.
|
||||
|
||||
<b>Note:</b> Especially with mounted disk. The remount of disk is a manual step that the user has to perform.
|
||||
|
||||
## Integrations
|
||||
|
||||
- Disk loss is effected using the litmus chaoslib that internally makes use of the aws/gcloud commands
|
||||
- Disk loss is effected using the litmus chaoslib that internally makes use of the aws/gcloud commands
|
||||
|
||||
## Steps to Execute the Chaos Experiment
|
||||
|
||||
- This Chaos Experiment can be triggered by creating a ChaosEngine resource on the cluster. To understand the values to provide in a ChaosEngine specification, refer [Getting Started](getstarted.md/#prepare-chaosengine)
|
||||
- This Chaos Experiment can be triggered by creating a ChaosEngine resource on the cluster. To understand the values to provide in a ChaosEngine specification, refer [Getting Started](getstarted.md/#prepare-chaosengine)
|
||||
|
||||
- Follow the steps in the sections below to create the chaosServiceAccount, prepare the ChaosEngine & execute the experiment.
|
||||
- Follow the steps in the sections below to create the chaosServiceAccount, prepare the ChaosEngine & execute the experiment.
|
||||
|
||||
### Prepare chaosServiceAccount
|
||||
|
||||
|
|
@ -70,7 +73,8 @@ stringData:
|
|||
|
||||
#### Sample Rbac Manifest
|
||||
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/generic/disk-loss/rbac.yaml yaml)
|
||||
[embedmd]: # "https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/generic/disk-loss/rbac.yaml yaml"
|
||||
|
||||
```yaml
|
||||
---
|
||||
apiVersion: v1
|
||||
|
|
@ -88,9 +92,17 @@ metadata:
|
|||
labels:
|
||||
name: nginx-sa
|
||||
rules:
|
||||
- apiGroups: ["","litmuschaos.io","batch"]
|
||||
resources: ["pods","jobs","secrets","chaosengines","chaosexperiments","chaosresults"]
|
||||
verbs: ["create","list","get","patch","update","delete"]
|
||||
- apiGroups: ["", "litmuschaos.io", "batch"]
|
||||
resources:
|
||||
[
|
||||
"pods",
|
||||
"jobs",
|
||||
"secrets",
|
||||
"chaosengines",
|
||||
"chaosexperiments",
|
||||
"chaosresults",
|
||||
]
|
||||
verbs: ["create", "list", "get", "patch", "update", "delete"]
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
kind: ClusterRoleBinding
|
||||
|
|
@ -103,16 +115,16 @@ roleRef:
|
|||
kind: ClusterRole
|
||||
name: nginx-sa
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: nginx-sa
|
||||
namespace: default
|
||||
- kind: ServiceAccount
|
||||
name: nginx-sa
|
||||
namespace: default
|
||||
```
|
||||
|
||||
### Prepare ChaosEngine
|
||||
|
||||
- Provide the application info in `spec.appinfo`
|
||||
- Provide the auxiliary applications info (ns & labels) in `spec.auxiliaryAppInfo`
|
||||
- Override the experiment tunables if desired
|
||||
- Provide the application info in `spec.appinfo`
|
||||
- Provide the auxiliary applications info (ns & labels) in `spec.auxiliaryAppInfo`
|
||||
- Override the experiment tunables if desired
|
||||
|
||||
### Supported Experiment Tunables for application
|
||||
|
||||
|
|
@ -211,7 +223,8 @@ subjects:
|
|||
|
||||
## Sample ChaosEngine Manifest
|
||||
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/generic/disk-loss/engine.yaml yaml)
|
||||
[embedmd]: # "https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/generic/disk-loss/engine.yaml yaml"
|
||||
|
||||
```yaml
|
||||
apiVersion: litmuschaos.io/v1alpha1
|
||||
kind: ChaosEngine
|
||||
|
|
@ -220,21 +233,21 @@ metadata:
|
|||
namespace: default
|
||||
spec:
|
||||
# It can be true/false
|
||||
annotationCheck: 'false'
|
||||
annotationCheck: "false"
|
||||
#ex. values: ns1:name=percona,ns2:run=nginx
|
||||
auxiliaryAppInfo: ''
|
||||
auxiliaryAppInfo: ""
|
||||
appinfo:
|
||||
appns: 'default'
|
||||
applabel: 'app=nginx'
|
||||
appkind: 'deployment'
|
||||
appns: "default"
|
||||
applabel: "app=nginx"
|
||||
appkind: "deployment"
|
||||
chaosServiceAccount: nginx-sa
|
||||
monitoring: false
|
||||
components:
|
||||
runner:
|
||||
image: 'litmuschaos/chaos-executor:1.0.0'
|
||||
type: 'go'
|
||||
image: "litmuschaos/chaos-executor:1.0.0"
|
||||
type: "go"
|
||||
# It can be retain/delete
|
||||
jobCleanUpPolicy: 'delete'
|
||||
jobCleanUpPolicy: "delete"
|
||||
experiments:
|
||||
- name: disk-loss
|
||||
spec:
|
||||
|
|
@ -242,52 +255,55 @@ spec:
|
|||
env:
|
||||
# set chaos duration (in sec) as desired
|
||||
- name: TOTAL_CHAOS_DURATION
|
||||
value: '60'
|
||||
value: "60"
|
||||
# set cloud platform name
|
||||
- name: CLOUD_PLATFORM
|
||||
value: 'GCP'
|
||||
value: "GCP"
|
||||
# set app_check to check application state
|
||||
- name: APP_CHECK
|
||||
value: 'true'
|
||||
value: "true"
|
||||
# This is a chaos namespace into which all infra chaos resources are created
|
||||
- name: CHAOS_NAMESPACE
|
||||
value: 'default'
|
||||
value: "default"
|
||||
# GCP project ID
|
||||
- name: PROJECT_ID
|
||||
value: 'litmus-demo-123'
|
||||
value: "litmus-demo-123"
|
||||
# Node name of the cluster
|
||||
- name: NODE_NAME
|
||||
value: 'demo-node-123'
|
||||
value: "demo-node-123"
|
||||
# Disk Name of the node, it must be an external disk.
|
||||
- name: DISK_NAME
|
||||
value: 'demo-disk-123'
|
||||
value: "demo-disk-123"
|
||||
# Enter the device name which you wanted to mount only for AWS.
|
||||
- name: DEVICE_NAME
|
||||
value: '/dev/sdb'
|
||||
value: "/dev/sdb"
|
||||
# Name of Zone in which node is present (GCP)
|
||||
# Use Region Name when running with AWS (ex: us-central1)
|
||||
- name: ZONE_NAME
|
||||
value: 'us-central1-a'
|
||||
value: "us-central1-a"
|
||||
# ChaosEngine CR name associated with the experiment instance
|
||||
- name: CHAOSENGINE
|
||||
value: ''
|
||||
value: ""
|
||||
# Service account used by the litmus
|
||||
- name: CHAOS_SERVICE_ACCOUNT
|
||||
value: ''
|
||||
value: ""
|
||||
```
|
||||
|
||||
## Create the ChaosEngine Resource
|
||||
- Create the ChaosEngine manifest prepared in the previous step to trigger the Chaos.
|
||||
|
||||
- Create the ChaosEngine manifest prepared in the previous step to trigger the Chaos.
|
||||
|
||||
`kubectl apply -f chaosengine.yml`
|
||||
|
||||
## Watch Chaos progress
|
||||
- Setting up a watch of the app which is using the disk in the Kubernetes Cluster
|
||||
|
||||
- Setting up a watch of the app which is using the disk in the Kubernetes Cluster
|
||||
|
||||
`watch -n 1 kubectl get pods`
|
||||
|
||||
## Check Chaos Experiment Result
|
||||
- Check whether the application is resilient to the disk loss, once the experiment (job) is completed. The ChaosResult resource name is derived like this: <ChaosEngine-Name>-<ChaosExperiment-Name>.
|
||||
|
||||
- Check whether the application is resilient to the disk loss, once the experiment (job) is completed. The ChaosResult resource name is derived like this: `{"<ChaosEngine-Name>-<ChaosExperiment-Name>"}`.
|
||||
|
||||
`kubectl describe chaosresult nginx-chaos-disk-loss -n <CHAOS_NAMESPACE>`
|
||||
|
||||
|
|
@ -1,10 +1,11 @@
|
|||
---
|
||||
id: version-1.1.0-faq-general
|
||||
id: faq-general
|
||||
title: The What, Why & How of Litmus
|
||||
sidebar_label: General
|
||||
original_id: faq-general
|
||||
---
|
||||
------
|
||||
|
||||
---
|
||||
|
||||
[Why should I use Litmus? What is its distinctive feature?](#why-should-i-use-litmus-what-is-its-distinctive-feature)
|
||||
|
||||
|
|
@ -36,11 +37,9 @@ original_id: faq-general
|
|||
|
||||
[Does Litmus track any usage metrics on the test clusters?](#does-litmus-track-any-usage-metrics-on-the-test-clusters)
|
||||
|
||||
<hr>
|
||||
<hr/>
|
||||
|
||||
|
||||
|
||||
### Why should I use Litmus? What is its distinctive feature?
|
||||
### Why should I use Litmus? What is its distinctive feature?
|
||||
|
||||
Litmus is a toolset to do cloud-native chaos engineering. Litmus provides tools to orchestrate chaos
|
||||
on Kubernetes to help developers and SREs find weaknesses in their application deployments. Litmus can
|
||||
|
|
@ -53,7 +52,7 @@ resources.
|
|||
|
||||
Litmus broadly defines Kubernetes chaos experiments into two categories: application or pod-level chaos
|
||||
experiments and platform or infra-level chaos experiments. The former includes pod-delete, container-kill,
|
||||
pod-cpu-hog, pod-network-loss etc., while the latter includes node-drain, disk-loss, node-cpu-hog etc.,
|
||||
pod-cpu-hog, pod-network-loss etc., while the latter includes node-drain, disk-loss, node-cpu-hog etc.,
|
||||
The infra chaos experiments typically have a higher blast radius and impacts more than one application
|
||||
deployed on the Kubernetes cluster. Litmus also categorizes experiments on the basis of the applications,
|
||||
with the experiments consisting of app-specific health checks. For example, OpenEBS, Kafka, CoreDNS.
|
||||
|
|
@ -83,7 +82,6 @@ with “chaosServiceAccounts” which are created by the developers with bare-mi
|
|||
execute the experiment in question. Visit the [chaos-charts](https://github.com/litmuschaos/chaos-charts) repo
|
||||
to view the experiment-specific rbac permissions. For example, here are the [permissions](https://github.com/litmuschaos/chaos-charts/blob/master/charts/generic/container-kill/rbac.yaml) for container-kill chaos.
|
||||
|
||||
|
||||
### What is the scope of a Litmus Chaos Experiment?
|
||||
|
||||
The chaos CRs (chaosexperiment, chaosengine, chaosresults) themselves are namespace scoped and are installed
|
||||
|
|
@ -128,7 +126,7 @@ necessary to run the experiments, but this depends on the experiment selected, c
|
|||
|
||||
### Is it mandatory to annotate application deployments for chaos?
|
||||
|
||||
Typically applications are expected to be annotated with `litmuschaos.io/chaos="true"` to lend themselves to chaos.
|
||||
Typically applications are expected to be annotated with `litmuschaos.io/chaos="true"` to lend themselves to chaos.
|
||||
This is in order to support selection of the right applications with similar labels in a namespaces, thereby isolating
|
||||
the application under test (AUT) & reduce the blast radius. It is also helpful for supporting automated execution
|
||||
(say, via cron) as a background service. However, in cases where the app deployment specifications are sacrosanct and
|
||||
|
|
@ -214,7 +212,3 @@ env:
|
|||
name: ANALYTICS
|
||||
value: 'FALSE'
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
|
|
@ -1,10 +1,11 @@
|
|||
---
|
||||
id: version-1.1.0-faq-troubleshooting
|
||||
title: Troubleshooting Litmus
|
||||
id: faq-troubleshooting
|
||||
title: "Troubleshooting Litmus"
|
||||
sidebar_label: Troubleshooting
|
||||
original_id: faq-troubleshooting
|
||||
---
|
||||
------
|
||||
|
||||
---
|
||||
|
||||
[The Litmus chaos operator is seen to be in CrashLoopBackOff state immediately after installation?](#the-litmus-chaos-operator-is-seen-to-be-in-crashloopbackOff-state-immediately-after-installation)
|
||||
|
||||
|
|
@ -14,9 +15,7 @@ original_id: faq-troubleshooting
|
|||
|
||||
[The experiment pod enters completed state w/o the desired chaos being injected?](#the-experiment-pod-enters-completed-state-wo-the-desired-chaos-being-injected)
|
||||
|
||||
<hr>
|
||||
|
||||
|
||||
<hr/>
|
||||
|
||||
### The Litmus chaos operator is seen to be in CrashLoopBackOff state immediately after installation?
|
||||
|
||||
|
|
@ -26,6 +25,7 @@ verified with the following commands:
|
|||
```console
|
||||
kubectl get crds | grep chaos
|
||||
```
|
||||
|
||||
```console
|
||||
kubectl api-resources | grep chaos
|
||||
```
|
||||
|
|
@ -60,6 +60,7 @@ Some of the possible reasons include:
|
|||
```console
|
||||
kubectl annotate <deploy-type>/<application_name> litmuschaos.io/chaos-
|
||||
```
|
||||
|
||||
- The ChaosEngine has the `.spec.engineState` set to `stop`, which causes the operator to refrain from creating chaos
|
||||
resources. While it is an unlikely scenario, it is possible to reuse a previously modified ChaosEngine manifest.
|
||||
|
||||
|
|
@ -87,7 +88,6 @@ Some of the possible reasons may include:
|
|||
- The chaosServiceAccount specified in the ChaosEngine CR doesn’t have sufficient permissions to create the experiment
|
||||
resources (For existing experiments, appropriate rbac manifests are already provided in chaos-charts/docs).
|
||||
|
||||
|
||||
### The experiment pod enters completed state w/o the desired chaos being injected?
|
||||
|
||||
If the experiment pod enters completed state immediately (or in a few seconds) after creation w/o injecting the desired chaos,
|
||||
|
|
@ -99,14 +99,10 @@ kubectl logs -f <experiment_name_(hash)_(hash)> -n <application_namespace>
|
|||
|
||||
Some of the possible reasons may include:
|
||||
|
||||
- The ChaosExperiment CR or the ChaosEngine CR doesn’t include mandatory ENVs (or consists of incorrect values/info)
|
||||
- The ChaosExperiment CR or the ChaosEngine CR doesn’t include mandatory ENVs (or consists of incorrect values/info)
|
||||
needed by the experiment. Note that each experiment (see docs) specifies a mandatory set of ENVs along with some
|
||||
optional ones, which are necessary for successful execution of the experiment.
|
||||
|
||||
- The chaosServiceAccount specified in the ChaosEngine CR doesn’t have sufficient permissions to create the experiment
|
||||
helper-resources (i.e., some experiments in turn create other K8s resources like jobs/daemonsets/deployments etc..,
|
||||
For existing experiments, appropriate rbac manifests are already provided in chaos-charts/docs).
|
||||
|
||||
|
||||
|
||||
|
||||
|
|
@ -1,10 +1,11 @@
|
|||
---
|
||||
id: version-1.1.0-getstarted
|
||||
id: getstarted
|
||||
title: Getting Started with Litmus
|
||||
sidebar_label: Introduction
|
||||
original_id: getstarted
|
||||
---
|
||||
------
|
||||
|
||||
---
|
||||
|
||||
## Pre-requisites
|
||||
|
||||
|
|
@ -28,11 +29,9 @@ Running chaos on your application involves the following steps:
|
|||
|
||||
[Observe chaos results](#observe-chaos-results)
|
||||
|
||||
<hr>
|
||||
<hr/>
|
||||
|
||||
|
||||
|
||||
### Install Litmus
|
||||
### Install Litmus
|
||||
|
||||
```
|
||||
kubectl apply -f https://litmuschaos.github.io/pages/litmus-operator-v1.1.0.yaml
|
||||
|
|
@ -48,14 +47,9 @@ The above command install all the CRDs, required service account configuration,
|
|||
kubectl get pods -n litmus
|
||||
```
|
||||
|
||||
Expected output:
|
||||
|
||||
|
||||
|
||||
|
||||
>chaos-operator-ce-554d6c8f9f-slc8k 1/1 Running 0 6m41s
|
||||
|
||||
Expected output:
|
||||
|
||||
> chaos-operator-ce-554d6c8f9f-slc8k 1/1 Running 0 6m41s
|
||||
|
||||
- Verify if chaos CRDs are installed
|
||||
|
||||
|
|
@ -65,15 +59,15 @@ kubectl get crds | grep chaos
|
|||
|
||||
Expected output:
|
||||
|
||||
> chaosengines.litmuschaos.io 2019-10-02T08:45:25Z
|
||||
> chaosengines.litmuschaos.io 2019-10-02T08:45:25Z
|
||||
>
|
||||
> chaosexperiments.litmuschaos.io 2019-10-02T08:45:26Z
|
||||
> chaosexperiments.litmuschaos.io 2019-10-02T08:45:26Z
|
||||
>
|
||||
> chaosresults.litmuschaos.io 2019-10-02T08:45:26Z
|
||||
> chaosresults.litmuschaos.io 2019-10-02T08:45:26Z
|
||||
|
||||
- Verify if the chaos api resources are successfully created in the desired (application) namespace.
|
||||
|
||||
*Note*: Sometimes, it can take a few seconds for the resources to be available post the CRD installation
|
||||
_Note_: Sometimes, it can take a few seconds for the resources to be available post the CRD installation
|
||||
|
||||
```
|
||||
kubectl api-resources | grep chaos
|
||||
|
|
@ -81,31 +75,29 @@ kubectl api-resources | grep chaos
|
|||
|
||||
Expected output:
|
||||
|
||||
> chaosengines litmuschaos.io true ChaosEngine
|
||||
> chaosengines litmuschaos.io true ChaosEngine
|
||||
>
|
||||
> chaosexperiments litmuschaos.io true ChaosExperiment
|
||||
> chaosexperiments litmuschaos.io true ChaosExperiment
|
||||
>
|
||||
> chaosresults litmuschaos.io true ChaosResult
|
||||
|
||||
|
||||
> chaosresults litmuschaos.io true ChaosResult
|
||||
|
||||
**NOTE**:
|
||||
|
||||
- In this guide, we shall describe the steps to inject container-kill chaos on an nginx application already deployed in the
|
||||
nginx namespace. It is a mandatory requirement to ensure that the chaos custom resources (chaosexperiment and chaosengine)
|
||||
and the experiment specific serviceaccount are created in the same namespace (typically, the same as the namespace of the
|
||||
application under test (AUT), in this case nginx). This is done to ensure that the developers/users of the experiment isolate
|
||||
the chaos to their respective work-namespaces in shared environments.
|
||||
nginx namespace. It is a mandatory requirement to ensure that the chaos custom resources (chaosexperiment and chaosengine)
|
||||
and the experiment specific serviceaccount are created in the same namespace (typically, the same as the namespace of the
|
||||
application under test (AUT), in this case nginx). This is done to ensure that the developers/users of the experiment isolate
|
||||
the chaos to their respective work-namespaces in shared environments.
|
||||
|
||||
- In all subsequent steps, please follow these instructions by replacing the nginx namespace and labels with that of your
|
||||
application.
|
||||
application.
|
||||
|
||||
### Install Chaos Experiments
|
||||
|
||||
Chaos experiments contain the actual chaos details. These experiments are installed on your cluster as Kubernetes CRs.
|
||||
The Chaos Experiments are grouped as Chaos Charts and are published on <a href=" https://hub.litmuschaos.io" target="_blank">Chaos Hub</a>.
|
||||
The Chaos Experiments are grouped as Chaos Charts and are published on <a href="https://hub.litmuschaos.io" target="_blank">Chaos Hub</a>.
|
||||
|
||||
The generic chaos experiments such as `pod-delete`, `container-kill`,` pod-network-latency` are available under Generic Chaos Chart.
|
||||
The generic chaos experiments such as `pod-delete`, `container-kill`,` pod-network-latency` are available under Generic Chaos Chart.
|
||||
This is the first chart you are recommended to install.
|
||||
|
||||
```
|
||||
|
|
@ -127,10 +119,10 @@ has just enough permissions needed to run the container-kill chaos experiment.
|
|||
**NOTE**:
|
||||
|
||||
- For rbac samples corresponding to other experiments such as, say, pod-delete, please refer the respective experiment folder in
|
||||
the [chaos-charts](https://github.com/litmuschaos/chaos-charts/tree/master/charts/generic/pod-delete) repository.
|
||||
the [chaos-charts](https://github.com/litmuschaos/chaos-charts/tree/master/charts/generic/pod-delete) repository.
|
||||
|
||||
[embedmd]: # "https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/generic/container-kill/rbac_nginx_getstarted.yaml yaml"
|
||||
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/generic/container-kill/rbac_nginx_getstarted.yaml yaml)
|
||||
```yaml
|
||||
---
|
||||
apiVersion: v1
|
||||
|
|
@ -149,9 +141,18 @@ metadata:
|
|||
labels:
|
||||
name: container-kill-sa
|
||||
rules:
|
||||
- apiGroups: ["","litmuschaos.io","batch","apps"]
|
||||
resources: ["pods","jobs","daemonsets","pods/exec","chaosengines","chaosexperiments","chaosresults"]
|
||||
verbs: ["create","list","get","patch","update","delete"]
|
||||
- apiGroups: ["", "litmuschaos.io", "batch", "apps"]
|
||||
resources:
|
||||
[
|
||||
"pods",
|
||||
"jobs",
|
||||
"daemonsets",
|
||||
"pods/exec",
|
||||
"chaosengines",
|
||||
"chaosexperiments",
|
||||
"chaosresults",
|
||||
]
|
||||
verbs: ["create", "list", "get", "patch", "update", "delete"]
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
kind: RoleBinding
|
||||
|
|
@ -165,10 +166,9 @@ roleRef:
|
|||
kind: Role
|
||||
name: container-kill-sa
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: container-kill-sa
|
||||
namespace: nginx
|
||||
|
||||
- kind: ServiceAccount
|
||||
name: container-kill-sa
|
||||
namespace: nginx
|
||||
```
|
||||
|
||||
### Annotate your application
|
||||
|
|
@ -193,7 +193,8 @@ ChaosEngine connects the application instance to a Chaos Experiment. Copy the fo
|
|||
`chaosengine.yaml` and update the values of `applabel` , `appns`, `appkind` and `experiments` as per your choice.
|
||||
Change the `chaosServiceAccount` to the name of service account created in above previous steps.
|
||||
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/generic/container-kill/engine_nginx_getstarted.yaml yaml)
|
||||
[embedmd]: # "https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/generic/container-kill/engine_nginx_getstarted.yaml yaml"
|
||||
|
||||
```yaml
|
||||
apiVersion: litmuschaos.io/v1alpha1
|
||||
kind: ChaosEngine
|
||||
|
|
@ -201,15 +202,15 @@ metadata:
|
|||
name: nginx-chaos
|
||||
namespace: nginx
|
||||
spec:
|
||||
annotationCheck: 'true'
|
||||
engineState: 'active'
|
||||
annotationCheck: "true"
|
||||
engineState: "active"
|
||||
appinfo:
|
||||
appns: 'nginx'
|
||||
applabel: 'app=nginx'
|
||||
appkind: 'deployment'
|
||||
appns: "nginx"
|
||||
applabel: "app=nginx"
|
||||
appkind: "deployment"
|
||||
chaosServiceAccount: container-kill-sa
|
||||
# use retain to keep the job for debug
|
||||
jobCleanUpPolicy: 'delete'
|
||||
jobCleanUpPolicy: "delete"
|
||||
experiments:
|
||||
- name: container-kill
|
||||
spec:
|
||||
|
|
@ -217,7 +218,7 @@ spec:
|
|||
env:
|
||||
# specify the name of the container to be killed
|
||||
- name: TARGET_CONTAINER
|
||||
value: 'nginx'
|
||||
value: "nginx"
|
||||
```
|
||||
|
||||
### Override Default Chaos Experiments Variables
|
||||
|
|
@ -237,20 +238,17 @@ experiments:
|
|||
value: nginx
|
||||
```
|
||||
|
||||
|
||||
|
||||
### Run Chaos
|
||||
|
||||
|
||||
```console
|
||||
kubectl apply -f chaosengine.yaml
|
||||
```
|
||||
|
||||
### Observe Chaos results
|
||||
|
||||
Describe the ChaosResult CR to know the status of each experiment. The ```spec.verdict``` is set to `Awaited` when the experiment is in progress, eventually changing to either `Pass` or `Fail`.
|
||||
Describe the ChaosResult CR to know the status of each experiment. The `spec.verdict` is set to `Awaited` when the experiment is in progress, eventually changing to either `Pass` or `Fail`.
|
||||
|
||||
<strong> NOTE:</strong> ChaosResult CR name will be `<chaos-engine-name>-<chaos-experiment-name>`
|
||||
<strong> NOTE:</strong> ChaosResult CR name will be `{"<chaos-engine-name>-<chaos-experiment-name>"}`
|
||||
|
||||
```console
|
||||
kubectl describe chaosresult nginx-chaos-container-kill -n nginx
|
||||
|
|
@ -1,5 +1,5 @@
|
|||
---
|
||||
id: version-1.1.0-kafka-broker-disk-failure
|
||||
id: kafka-broker-disk-failure
|
||||
title: Kafka Broker Disk Failure Experiment Details
|
||||
sidebar_label: Broker Disk Failure
|
||||
original_id: kafka-broker-disk-failure
|
||||
|
|
@ -72,7 +72,8 @@ original_id: kafka-broker-disk-failure
|
|||
|
||||
#### Sample Rbac Manifest
|
||||
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/kafka/kafka-broker-disk-failure/rbac.yaml yaml)
|
||||
[embedmd]: # "https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/kafka/kafka-broker-disk-failure/rbac.yaml yaml"
|
||||
|
||||
```yaml
|
||||
---
|
||||
apiVersion: v1
|
||||
|
|
@ -90,9 +91,19 @@ metadata:
|
|||
labels:
|
||||
name: kafka-broker-disk-failure-sa
|
||||
rules:
|
||||
- apiGroups: ["","litmuschaos.io","batch","apps"]
|
||||
resources: ["pods","jobs","pods/exec","statefulsets","secrets","chaosengines","chaosexperiments","chaosresults"]
|
||||
verbs: ["create","list","get","patch","delete"]
|
||||
- apiGroups: ["", "litmuschaos.io", "batch", "apps"]
|
||||
resources:
|
||||
[
|
||||
"pods",
|
||||
"jobs",
|
||||
"pods/exec",
|
||||
"statefulsets",
|
||||
"secrets",
|
||||
"chaosengines",
|
||||
"chaosexperiments",
|
||||
"chaosresults",
|
||||
]
|
||||
verbs: ["create", "list", "get", "patch", "delete"]
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
kind: ClusterRoleBinding
|
||||
|
|
@ -105,10 +116,9 @@ roleRef:
|
|||
kind: ClusterRole
|
||||
name: kafka-broker-disk-failure-sa
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: kafka-broker-disk-failure-sa
|
||||
namespace: default
|
||||
|
||||
- kind: ServiceAccount
|
||||
name: kafka-broker-disk-failure-sa
|
||||
namespace: default
|
||||
```
|
||||
|
||||
### Prepare ChaosEngine
|
||||
|
|
@ -219,7 +229,7 @@ subjects:
|
|||
<td> KAFKA_LIVENESS_IMAGE </td>
|
||||
<td> Image used for liveness message stream </td>
|
||||
<td> Optional </td>
|
||||
<td> Image as `<registry_url>/<repository>/<image>:<tag>` </td>
|
||||
<td> Image as `{"<registry_url>/<repository>/<image>:<tag>"}` </td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td> KAFKA_REPLICATION_FACTOR </td>
|
||||
|
|
@ -249,7 +259,8 @@ subjects:
|
|||
|
||||
#### Sample ChaosEngine Manifest
|
||||
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/kafka/kafka-broker-disk-failure/engine.yaml yaml)
|
||||
[embedmd]: # "https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/kafka/kafka-broker-disk-failure/engine.yaml yaml"
|
||||
|
||||
```yaml
|
||||
apiVersion: litmuschaos.io/v1alpha1
|
||||
kind: ChaosEngine
|
||||
|
|
@ -258,19 +269,19 @@ metadata:
|
|||
namespace: default
|
||||
spec:
|
||||
# It can be true/false
|
||||
annotationCheck: 'true'
|
||||
annotationCheck: "true"
|
||||
# It can be active/stop
|
||||
engineState: 'active'
|
||||
engineState: "active"
|
||||
#ex. values: ns1:name=percona,ns2:run=nginx
|
||||
auxiliaryAppInfo: ''
|
||||
auxiliaryAppInfo: ""
|
||||
appinfo:
|
||||
appns: 'default'
|
||||
applabel: 'app=cp-kafka'
|
||||
appkind: 'statefulset'
|
||||
appns: "default"
|
||||
applabel: "app=cp-kafka"
|
||||
appkind: "statefulset"
|
||||
chaosServiceAccount: kafka-broker-disk-failure-sa
|
||||
monitoring: false
|
||||
# It can be delete/retain
|
||||
jobCleanUpPolicy: 'delete'
|
||||
jobCleanUpPolicy: "delete"
|
||||
experiments:
|
||||
- name: kafka-broker-disk-failure
|
||||
spec:
|
||||
|
|
@ -278,65 +289,64 @@ spec:
|
|||
env:
|
||||
# choose based on available kafka broker replicas
|
||||
- name: KAFKA_REPLICATION_FACTOR
|
||||
value: '3'
|
||||
value: "3"
|
||||
|
||||
# get via 'kubectl get pods --show-labels -n <kafka-namespace>'
|
||||
- name: KAFKA_LABEL
|
||||
value: 'app=cp-kafka'
|
||||
value: "app=cp-kafka"
|
||||
|
||||
- name: KAFKA_NAMESPACE
|
||||
value: 'default'
|
||||
value: "default"
|
||||
|
||||
# get via 'kubectl get svc -n <kafka-namespace>'
|
||||
- name: KAFKA_SERVICE
|
||||
value: 'kafka-cp-kafka-headless'
|
||||
value: "kafka-cp-kafka-headless"
|
||||
|
||||
# get via 'kubectl get svc -n <kafka-namespace>'
|
||||
- name: KAFKA_PORT
|
||||
value: '9092'
|
||||
value: "9092"
|
||||
|
||||
# in milliseconds
|
||||
- name: KAFKA_CONSUMER_TIMEOUT
|
||||
value: '70000'
|
||||
value: "70000"
|
||||
|
||||
# ensure to set the instance name if using KUDO operator
|
||||
- name: KAFKA_INSTANCE_NAME
|
||||
value: ''
|
||||
value: ""
|
||||
|
||||
- name: ZOOKEEPER_NAMESPACE
|
||||
value: 'default'
|
||||
value: "default"
|
||||
|
||||
# get via 'kubectl get pods --show-labels -n <zk-namespace>'
|
||||
- name: ZOOKEEPER_LABEL
|
||||
value: 'app=cp-zookeeper'
|
||||
value: "app=cp-zookeeper"
|
||||
|
||||
# get via 'kubectl get svc -n <zk-namespace>
|
||||
- name: ZOOKEEPER_SERVICE
|
||||
value: 'kafka-cp-zookeeper-headless'
|
||||
value: "kafka-cp-zookeeper-headless"
|
||||
|
||||
# get via 'kubectl get svc -n <zk-namespace>
|
||||
- name: ZOOKEEPER_PORT
|
||||
value: '2181'
|
||||
value: "2181"
|
||||
|
||||
# get from google cloud console or 'gcloud projects list'
|
||||
- name: PROJECT_ID
|
||||
value: 'argon-tractor-237811'
|
||||
value: "argon-tractor-237811"
|
||||
|
||||
# attached to (in use by) node where 'kafka-0' is scheduled
|
||||
- name: DISK_NAME
|
||||
value: 'disk-1'
|
||||
value: "disk-1"
|
||||
|
||||
- name: ZONE_NAME
|
||||
value: 'us-central1-a'
|
||||
value: "us-central1-a"
|
||||
|
||||
# Uses 'disk-1' attached to the node on which it is scheduled
|
||||
- name: KAFKA_BROKER
|
||||
value: 'kafka-0'
|
||||
value: "kafka-0"
|
||||
|
||||
# set chaos duration (in sec) as desired
|
||||
- name: TOTAL_CHAOS_DURATION
|
||||
value: '60'
|
||||
|
||||
value: "60"
|
||||
```
|
||||
|
||||
### Create the ChaosEngine Resource
|
||||
|
|
@ -354,7 +364,7 @@ spec:
|
|||
### Check Chaos Experiment Result
|
||||
|
||||
- Check whether the kafka deployment is resilient to the broker disk failure, once the experiment (job) is completed.
|
||||
The ChaosResult resource name is derived like this: `<ChaosEngine-Name>-<ChaosExperiment-Name>`.
|
||||
The ChaosResult resource name is derived like this: `{"<ChaosEngine-Name>-<ChaosExperiment-Name>"}`.
|
||||
|
||||
`kubectl describe chaosresult kafka-chaos-kafka-broker-disk-failure -n <kafka-namespace>`
|
||||
|
||||
|
|
@ -373,4 +383,4 @@ spec:
|
|||
|
||||
- TODO: A sample recording of this experiment execution is provided here.
|
||||
|
||||
------
|
||||
---
|
||||
|
|
@ -1,5 +1,5 @@
|
|||
---
|
||||
id: version-1.1.0-kafka-broker-pod-failure
|
||||
id: kafka-broker-pod-failure
|
||||
title: Kafka Broker Pod Failure Experiment Details
|
||||
sidebar_label: Broker Pod Failure
|
||||
original_id: kafka-broker-pod-failure
|
||||
|
|
@ -37,7 +37,6 @@ original_id: kafka-broker-pod-failure
|
|||
|
||||
- Ensure that the kafka-broker-disk failure experiment resource is available in the cluster. If not, install from [here](https://hub.litmuschaos.io/charts/kafka/experiments/kafka-broker-pod-failure)
|
||||
|
||||
|
||||
## Entry Criteria
|
||||
|
||||
- Kafka Cluster (comprising the Kafka-broker & Zookeeper Statefulsets) is healthy
|
||||
|
|
@ -71,7 +70,8 @@ original_id: kafka-broker-pod-failure
|
|||
|
||||
#### Sample Rbac Manifest
|
||||
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/kafka/kafka-broker-pod-failure/rbac.yaml yaml)
|
||||
[embedmd]: # "https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/kafka/kafka-broker-pod-failure/rbac.yaml yaml"
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
|
|
@ -88,12 +88,23 @@ metadata:
|
|||
labels:
|
||||
name: kafka-broker-pod-failure-sa
|
||||
rules:
|
||||
- apiGroups: ["","litmuschaos.io","batch","apps"]
|
||||
resources: ["pods","deployments","jobs","pods/exec","statefulsets","configmaps","chaosengines","chaosexperiments","chaosresults"]
|
||||
verbs: ["create","list","get","patch","delete"]
|
||||
- apiGroups: [""]
|
||||
resources: ["nodes"]
|
||||
verbs : ["get","list"]
|
||||
- apiGroups: ["", "litmuschaos.io", "batch", "apps"]
|
||||
resources:
|
||||
[
|
||||
"pods",
|
||||
"deployments",
|
||||
"jobs",
|
||||
"pods/exec",
|
||||
"statefulsets",
|
||||
"configmaps",
|
||||
"chaosengines",
|
||||
"chaosexperiments",
|
||||
"chaosresults",
|
||||
]
|
||||
verbs: ["create", "list", "get", "patch", "delete"]
|
||||
- apiGroups: [""]
|
||||
resources: ["nodes"]
|
||||
verbs: ["get", "list"]
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
kind: ClusterRoleBinding
|
||||
|
|
@ -106,9 +117,9 @@ roleRef:
|
|||
kind: ClusterRole
|
||||
name: kafka-broker-pod-failure-sa
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: kafka-broker-pod-failure-sa
|
||||
namespace: default
|
||||
- kind: ServiceAccount
|
||||
name: kafka-broker-pod-failure-sa
|
||||
namespace: default
|
||||
```
|
||||
|
||||
### Prepare ChaosEngine
|
||||
|
|
@ -195,7 +206,7 @@ subjects:
|
|||
<td> KAFKA_LIVENESS_IMAGE </td>
|
||||
<td> Image used for liveness message stream </td>
|
||||
<td> Optional </td>
|
||||
<td> Image as `<registry_url>/<repository>/<image>:<tag>` </td>
|
||||
<td> Image as `{"<registry_url>/<repository>/<image>:<tag>"}` </td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td> KAFKA_REPLICATION_FACTOR </td>
|
||||
|
|
@ -243,7 +254,8 @@ subjects:
|
|||
|
||||
#### Sample ChaosEngine Manifest
|
||||
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/kafka/kafka-broker-pod-failure/engine.yaml yaml)
|
||||
[embedmd]: # "https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/kafka/kafka-broker-pod-failure/engine.yaml yaml"
|
||||
|
||||
```yaml
|
||||
apiVersion: litmuschaos.io/v1alpha1
|
||||
kind: ChaosEngine
|
||||
|
|
@ -252,19 +264,19 @@ metadata:
|
|||
namespace: default
|
||||
spec:
|
||||
# It can be true/false
|
||||
annotationCheck: 'true'
|
||||
annotationCheck: "true"
|
||||
# It can be active/stop
|
||||
engineState: 'active'
|
||||
engineState: "active"
|
||||
#ex. values: ns1:name=percona,ns2:run=nginx
|
||||
auxiliaryAppInfo: ''
|
||||
auxiliaryAppInfo: ""
|
||||
appinfo:
|
||||
appns: 'default'
|
||||
applabel: 'app=cp-kafka'
|
||||
appkind: 'statefulset'
|
||||
appns: "default"
|
||||
applabel: "app=cp-kafka"
|
||||
appkind: "statefulset"
|
||||
chaosServiceAccount: kafka-broker-pod-failure-sa
|
||||
monitoring: false
|
||||
# It can be delete/retain
|
||||
jobCleanUpPolicy: 'delete'
|
||||
jobCleanUpPolicy: "delete"
|
||||
experiments:
|
||||
- name: kafka-broker-pod-failure
|
||||
spec:
|
||||
|
|
@ -272,57 +284,57 @@ spec:
|
|||
env:
|
||||
# choose based on available kafka broker replicas
|
||||
- name: KAFKA_REPLICATION_FACTOR
|
||||
value: '3'
|
||||
value: "3"
|
||||
|
||||
# get via 'kubectl get pods --show-labels -n <kafka-namespace>'
|
||||
- name: KAFKA_LABEL
|
||||
value: 'app=cp-kafka'
|
||||
value: "app=cp-kafka"
|
||||
|
||||
- name: KAFKA_NAMESPACE
|
||||
value: 'default'
|
||||
value: "default"
|
||||
|
||||
# get via 'kubectl get svc -n <kafka-namespace>'
|
||||
- name: KAFKA_SERVICE
|
||||
value: 'kafka-cp-kafka-headless'
|
||||
value: "kafka-cp-kafka-headless"
|
||||
|
||||
# get via 'kubectl get svc -n <kafka-namespace>'
|
||||
- name: KAFKA_PORT
|
||||
value: '9092'
|
||||
value: "9092"
|
||||
|
||||
# in milliseconds
|
||||
- name: KAFKA_CONSUMER_TIMEOUT
|
||||
value: '70000'
|
||||
value: "70000"
|
||||
|
||||
# ensure to set the instance name if using KUDO operator
|
||||
- name: KAFKA_INSTANCE_NAME
|
||||
value: ''
|
||||
value: ""
|
||||
|
||||
- name: ZOOKEEPER_NAMESPACE
|
||||
value: 'default'
|
||||
value: "default"
|
||||
|
||||
# get via 'kubectl get pods --show-labels -n <zk-namespace>'
|
||||
- name: ZOOKEEPER_LABEL
|
||||
value: 'app=cp-zookeeper'
|
||||
value: "app=cp-zookeeper"
|
||||
|
||||
# get via 'kubectl get svc -n <zk-namespace>
|
||||
- name: ZOOKEEPER_SERVICE
|
||||
value: 'kafka-cp-zookeeper-headless'
|
||||
value: "kafka-cp-zookeeper-headless"
|
||||
|
||||
# get via 'kubectl get svc -n <zk-namespace>
|
||||
- name: ZOOKEEPER_PORT
|
||||
value: '2181'
|
||||
value: "2181"
|
||||
|
||||
# set chaos duration (in sec) as desired
|
||||
- name: TOTAL_CHAOS_DURATION
|
||||
value: '60'
|
||||
value: "60"
|
||||
|
||||
# set chaos interval (in sec) as desired
|
||||
- name: CHAOS_INTERVAL
|
||||
value: '20'
|
||||
value: "20"
|
||||
|
||||
# pod failures without '--force' & default terminationGracePeriodSeconds
|
||||
- name: FORCE
|
||||
value: 'false'
|
||||
value: "false"
|
||||
```
|
||||
|
||||
### Create the ChaosEngine Resource
|
||||
|
|
@ -340,7 +352,7 @@ spec:
|
|||
### Check Chaos Experiment Result
|
||||
|
||||
- Check whether the kafka deployment is resilient to the broker pod failure, once the experiment (job) is completed.
|
||||
The ChaosResult resource name is derived like this: `<ChaosEngine-Name>-<ChaosExperiment-Name>`.
|
||||
The ChaosResult resource name is derived like this: `{"<ChaosEngine-Name>-<ChaosExperiment-Name>"}`.
|
||||
|
||||
`kubectl describe chaosresult kafka-chaos-kafka-broker-pod-failure -n <kafka-namespace>`
|
||||
|
||||
|
|
@ -348,4 +360,4 @@ spec:
|
|||
|
||||
- TODO: A sample recording of this experiment execution is provided here.
|
||||
|
||||
------
|
||||
---
|
||||
|
|
@ -1,10 +1,11 @@
|
|||
---
|
||||
id: version-1.1.0-node-cpu-hog
|
||||
id: node-cpu-hog
|
||||
title: Node CPU Hog Experiment Details
|
||||
sidebar_label: Node CPU Hog
|
||||
original_id: node-cpu-hog
|
||||
---
|
||||
------
|
||||
|
||||
---
|
||||
|
||||
## Experiment Metadata
|
||||
|
||||
|
|
@ -24,8 +25,9 @@ original_id: node-cpu-hog
|
|||
## Prerequisites
|
||||
|
||||
- Ensure that the Litmus Chaos Operator is running by executing `kubectl get pods` in operator namespace (typically, `litmus`). If not, install from [here](https://raw.githubusercontent.com/litmuschaos/pages/master/docs/litmus-operator-latest.yaml)
|
||||
- Ensure that the `node-cpu-hog` experiment resource is available in the cluster by executing `kubectl get chaosexperiments` in the desired namespace. If not, install from [here](https://hub.litmuschaos.io/charts/generic/experiments/node-cpu-hog)
|
||||
- Ensure that the `node-cpu-hog` experiment resource is available in the cluster by executing `kubectl get chaosexperiments` in the desired namespace. If not, install from [here](https://hub.litmuschaos.io/charts/generic/experiments/node-cpu-hog)
|
||||
- There should be administrative access to the platform on which the Kubernetes cluster is hosted, as the recovery of the affected node could be manual. For example, gcloud access to the GKE project
|
||||
|
||||
## Entry Criteria
|
||||
|
||||
- Application pods are healthy on the respective Nodes before chaos injection
|
||||
|
|
@ -39,7 +41,7 @@ original_id: node-cpu-hog
|
|||
- This experiment causes CPU resource exhaustion on the Kubernetes node. The experiment aims to verify resiliency of applications whose replicas may be evicted on account on nodes turning unschedulable (Not Ready) due to lack of CPU resources.
|
||||
- The CPU chaos is injected using a daemonset running the linux stress tool (a workload generator). The chaos is effected for a period equalling the TOTAL_CHAOS_DURATION
|
||||
- Application implies services. Can be reframed as:
|
||||
Tests application resiliency upon replica evictions caused due to lack of CPU resources
|
||||
Tests application resiliency upon replica evictions caused due to lack of CPU resources
|
||||
|
||||
## Integrations
|
||||
|
||||
|
|
@ -58,7 +60,8 @@ Tests application resiliency upon replica evictions caused due to lack of CPU re
|
|||
|
||||
#### Sample Rbac Manifest
|
||||
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/generic/node-cpu-hog/rbac.yaml yaml)
|
||||
[embedmd]: # "https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/generic/node-cpu-hog/rbac.yaml yaml"
|
||||
|
||||
```yaml
|
||||
---
|
||||
apiVersion: v1
|
||||
|
|
@ -76,12 +79,21 @@ metadata:
|
|||
labels:
|
||||
name: node-cpu-hog-sa
|
||||
rules:
|
||||
- apiGroups: ["","litmuschaos.io","batch","apps"]
|
||||
resources: ["pods","daemonsets","jobs","pods/exec","chaosengines","chaosexperiments","chaosresults"]
|
||||
verbs: ["create","list","get","patch","update","delete"]
|
||||
- apiGroups: [""]
|
||||
resources: ["nodes"]
|
||||
verbs : ["get","list"]
|
||||
- apiGroups: ["", "litmuschaos.io", "batch", "apps"]
|
||||
resources:
|
||||
[
|
||||
"pods",
|
||||
"daemonsets",
|
||||
"jobs",
|
||||
"pods/exec",
|
||||
"chaosengines",
|
||||
"chaosexperiments",
|
||||
"chaosresults",
|
||||
]
|
||||
verbs: ["create", "list", "get", "patch", "update", "delete"]
|
||||
- apiGroups: [""]
|
||||
resources: ["nodes"]
|
||||
verbs: ["get", "list"]
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
kind: ClusterRoleBinding
|
||||
|
|
@ -94,9 +106,9 @@ roleRef:
|
|||
kind: ClusterRole
|
||||
name: node-cpu-hog-sa
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: node-cpu-hog-sa
|
||||
namespace: default
|
||||
- kind: ServiceAccount
|
||||
name: node-cpu-hog-sa
|
||||
namespace: default
|
||||
```
|
||||
|
||||
### Prepare ChaosEngine
|
||||
|
|
@ -142,7 +154,8 @@ subjects:
|
|||
|
||||
#### Sample ChaosEngine Manifest
|
||||
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/generic/node-cpu-hog/engine.yaml yaml)
|
||||
[embedmd]: # "https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/generic/node-cpu-hog/engine.yaml yaml"
|
||||
|
||||
```yaml
|
||||
apiVersion: litmuschaos.io/v1alpha1
|
||||
kind: ChaosEngine
|
||||
|
|
@ -151,19 +164,19 @@ metadata:
|
|||
namespace: default
|
||||
spec:
|
||||
# It can be true/false
|
||||
annotationCheck: 'false'
|
||||
annotationCheck: "false"
|
||||
# It can be active/stop
|
||||
engineState: 'active'
|
||||
engineState: "active"
|
||||
#ex. values: ns1:name=percona,ns2:run=nginx
|
||||
auxiliaryAppInfo: ''
|
||||
auxiliaryAppInfo: ""
|
||||
appinfo:
|
||||
appns: 'default'
|
||||
applabel: 'app=nginx'
|
||||
appkind: 'deployment'
|
||||
appns: "default"
|
||||
applabel: "app=nginx"
|
||||
appkind: "deployment"
|
||||
chaosServiceAccount: node-cpu-hog-sa
|
||||
monitoring: false
|
||||
# It can be delete/retain
|
||||
jobCleanUpPolicy: 'delete'
|
||||
jobCleanUpPolicy: "delete"
|
||||
experiments:
|
||||
- name: node-cpu-hog
|
||||
spec:
|
||||
|
|
@ -171,13 +184,13 @@ spec:
|
|||
env:
|
||||
# set chaos duration (in sec) as desired
|
||||
- name: TOTAL_CHAOS_DURATION
|
||||
value: '60'
|
||||
value: "60"
|
||||
# set chaos platform as desired
|
||||
- name: PLATFORM
|
||||
value: 'GKE'
|
||||
value: "GKE"
|
||||
# chaos lib used to inject the chaos
|
||||
- name: LIB
|
||||
value: 'litmus'
|
||||
value: "litmus"
|
||||
```
|
||||
|
||||
### Create the ChaosEngine Resource
|
||||
|
|
@ -194,7 +207,7 @@ spec:
|
|||
|
||||
### Check Chaos Experiment Result
|
||||
|
||||
- Check whether the application is resilient to the CPU hog, once the experiment (job) is completed. The ChaosResult resource name is derived like this: `<ChaosEngine-Name>-<ChaosExperiment-Name>`.
|
||||
- Check whether the application is resilient to the CPU hog, once the experiment (job) is completed. The ChaosResult resource name is derived like this: `{"<ChaosEngine-Name>-<ChaosExperiment-Name>"}`.
|
||||
|
||||
`kubectl describe chaosresult nginx-chaos-node-cpu-hog -n <application-namespace>`
|
||||
|
||||
|
|
@ -1,10 +1,11 @@
|
|||
---
|
||||
id: version-1.1.0-node-drain
|
||||
id: node-drain
|
||||
title: Node Drain Experiment Details
|
||||
sidebar_label: Node Drain
|
||||
original_id: node-drain
|
||||
---
|
||||
------
|
||||
|
||||
---
|
||||
|
||||
## Experiment Metadata
|
||||
|
||||
|
|
@ -25,7 +26,7 @@ original_id: node-drain
|
|||
|
||||
- Ensure that the Litmus Chaos Operator is running by executing `kubectl get pods` in operator namespace (typically, `litmus`). If not, install from [here](https://raw.githubusercontent.com/litmuschaos/pages/master/docs/litmus-operator-latest.yaml)
|
||||
- Ensure that the `node-drain` experiment resource is available in the cluster by executing `kubectl get chaosexperiments` in the desired namespace. If not, install from [here](https://hub.litmuschaos.io/charts/generic/experiments/node-drain)
|
||||
- Ensure that the node specified in the experiment ENV variable `APP_NODE` (the node which will be drained) should be cordoned before execution of the chaos experiment (before applying the chaosengine manifest) to ensure that the litmus experiment runner pods are not scheduled on it / subjected to eviction. This can be achieved with the following steps:
|
||||
- Ensure that the node specified in the experiment ENV variable `APP_NODE` (the node which will be drained) should be cordoned before execution of the chaos experiment (before applying the chaosengine manifest) to ensure that the litmus experiment runner pods are not scheduled on it / subjected to eviction. This can be achieved with the following steps:
|
||||
|
||||
- Get node names against the applications pods: `kubectl get pods -o wide`
|
||||
- Cordon the node `kubectl cordon <nodename>`
|
||||
|
|
@ -59,7 +60,8 @@ Use this sample RBAC manifest to create a chaosServiceAccount in the desired (ap
|
|||
|
||||
#### Sample Rbac Manifest
|
||||
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/generic/node-drain/rbac.yaml yaml)
|
||||
[embedmd]: # "https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/generic/node-drain/rbac.yaml yaml"
|
||||
|
||||
```yaml
|
||||
---
|
||||
apiVersion: v1
|
||||
|
|
@ -77,12 +79,21 @@ metadata:
|
|||
labels:
|
||||
name: node-drain-sa
|
||||
rules:
|
||||
- apiGroups: ["","litmuschaos.io","batch","extensions"]
|
||||
resources: ["pods","jobs","chaosengines","daemonsets","pods/eviction","chaosexperiments","chaosresults"]
|
||||
verbs: ["create","list","get","patch","update","delete"]
|
||||
- apiGroups: [""]
|
||||
resources: ["nodes"]
|
||||
verbs: ["patch","get","list"]
|
||||
- apiGroups: ["", "litmuschaos.io", "batch", "extensions"]
|
||||
resources:
|
||||
[
|
||||
"pods",
|
||||
"jobs",
|
||||
"chaosengines",
|
||||
"daemonsets",
|
||||
"pods/eviction",
|
||||
"chaosexperiments",
|
||||
"chaosresults",
|
||||
]
|
||||
verbs: ["create", "list", "get", "patch", "update", "delete"]
|
||||
- apiGroups: [""]
|
||||
resources: ["nodes"]
|
||||
verbs: ["patch", "get", "list"]
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
kind: ClusterRoleBinding
|
||||
|
|
@ -95,10 +106,9 @@ roleRef:
|
|||
kind: ClusterRole
|
||||
name: node-drain-sa
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: node-drain-sa
|
||||
namespace: default
|
||||
|
||||
- kind: ServiceAccount
|
||||
name: node-drain-sa
|
||||
namespace: default
|
||||
```
|
||||
|
||||
### Prepare ChaosEngine
|
||||
|
|
@ -138,7 +148,8 @@ subjects:
|
|||
|
||||
#### Sample ChaosEngine Manifest
|
||||
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/generic/node-drain/engine.yaml yaml)
|
||||
[embedmd]: # "https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/generic/node-drain/engine.yaml yaml"
|
||||
|
||||
```yaml
|
||||
apiVersion: litmuschaos.io/v1alpha1
|
||||
kind: ChaosEngine
|
||||
|
|
@ -147,19 +158,19 @@ metadata:
|
|||
namespace: default
|
||||
spec:
|
||||
# It can be true/false
|
||||
annotationCheck: 'false'
|
||||
annotationCheck: "false"
|
||||
# It can be active/stop
|
||||
engineState: 'active'
|
||||
engineState: "active"
|
||||
#ex. values: ns1:name=percona,ns2:run=nginx
|
||||
auxiliaryAppInfo: ''
|
||||
auxiliaryAppInfo: ""
|
||||
appinfo:
|
||||
appns: 'default'
|
||||
applabel: 'app=nginx'
|
||||
appkind: 'deployment'
|
||||
appns: "default"
|
||||
applabel: "app=nginx"
|
||||
appkind: "deployment"
|
||||
chaosServiceAccount: node-drain-sa
|
||||
monitoring: false
|
||||
# It can be delete/retain
|
||||
jobCleanUpPolicy: 'delete'
|
||||
jobCleanUpPolicy: "delete"
|
||||
experiments:
|
||||
- name: node-drain
|
||||
spec:
|
||||
|
|
@ -167,7 +178,7 @@ spec:
|
|||
env:
|
||||
# set node name
|
||||
- name: APP_NODE
|
||||
value: 'node-1'
|
||||
value: "node-1"
|
||||
```
|
||||
|
||||
### Create the ChaosEngine Resource
|
||||
|
|
@ -184,7 +195,7 @@ spec:
|
|||
|
||||
### Check Chaos Experiment Result
|
||||
|
||||
- Check whether the application is resilient to the node drain, once the experiment (job) is completed. The ChaosResult resource name is derived like this: `<ChaosEngine-Name>-<ChaosExperiment-Name>`.
|
||||
- Check whether the application is resilient to the node drain, once the experiment (job) is completed. The ChaosResult resource name is derived like this: `{"<ChaosEngine-Name>-<ChaosExperiment-Name>"}`.
|
||||
|
||||
`kubectl describe chaosresult nginx-chaos-node-drain -n <application-namespace>`
|
||||
|
||||
|
|
@ -1,10 +1,11 @@
|
|||
---
|
||||
id: version-1.2.0-openebs-control-plane-chaos
|
||||
id: openebs-control-plane-chaos
|
||||
title: OpenEBS Control Plane Chaos Experiment Details
|
||||
sidebar_label: Control Plane Chaos
|
||||
original_id: openebs-control-plane-chaos
|
||||
---
|
||||
------
|
||||
|
||||
---
|
||||
|
||||
## Experiment Metadata
|
||||
|
||||
|
|
@ -58,14 +59,14 @@ original_id: openebs-control-plane-chaos
|
|||
|
||||
- Follow the steps in the sections below to create the chaosServiceAccount, prepare the ChaosEngine & execute the experiment.
|
||||
|
||||
|
||||
### Prepare chaosServiceAccount
|
||||
|
||||
Use this sample RBAC manifest to create a chaosServiceAccount in the desired (openebs) namespace. This example consists of the minimum necessary role permissions to execute the experiment.
|
||||
|
||||
#### Sample Rbac Manifest
|
||||
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/openebs/openebs-control-plane-chaos/rbac.yaml)
|
||||
[embedmd]: # "https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/openebs/openebs-control-plane-chaos/rbac.yaml"
|
||||
|
||||
```yaml
|
||||
---
|
||||
apiVersion: v1
|
||||
|
|
@ -84,12 +85,21 @@ metadata:
|
|||
labels:
|
||||
name: control-plane-sa
|
||||
rules:
|
||||
- apiGroups: ["","litmuschaos.io","batch","apps"]
|
||||
resources: ["pods","deployments","pods/log","events","jobs","configmaps","chaosengines","chaosexperiments","chaosresults"]
|
||||
verbs: ["create","list","get","patch","update","delete"]
|
||||
- apiGroups: [""]
|
||||
resources: ["nodes"]
|
||||
verbs : ["get","list"]
|
||||
- apiGroups: ["", "litmuschaos.io", "batch", "apps"]
|
||||
resources:
|
||||
[
|
||||
"pods",
|
||||
"deployments",
|
||||
"jobs",
|
||||
"configmaps",
|
||||
"chaosengines",
|
||||
"chaosexperiments",
|
||||
"chaosresults",
|
||||
]
|
||||
verbs: ["create", "list", "get", "patch", "update", "delete"]
|
||||
- apiGroups: [""]
|
||||
resources: ["nodes"]
|
||||
verbs: ["get", "list"]
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
kind: RoleBinding
|
||||
|
|
@ -103,9 +113,9 @@ roleRef:
|
|||
kind: Role
|
||||
name: control-plane-sa
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: control-plane-sa
|
||||
namespace: openebs
|
||||
- kind: ServiceAccount
|
||||
name: control-plane-sa
|
||||
namespace: openebs
|
||||
```
|
||||
|
||||
### Prepare ChaosEngine
|
||||
|
|
@ -132,7 +142,8 @@ subjects:
|
|||
|
||||
#### Sample ChaosEngine Manifest
|
||||
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/openebs/openebs-control-plane-chaos/engine.yaml)
|
||||
[embedmd]: # "https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/openebs/openebs-control-plane-chaos/engine.yaml"
|
||||
|
||||
```yaml
|
||||
apiVersion: litmuschaos.io/v1alpha1
|
||||
kind: ChaosEngine
|
||||
|
|
@ -141,34 +152,34 @@ metadata:
|
|||
namespace: openebs
|
||||
spec:
|
||||
# It can be true/false
|
||||
annotationCheck: 'false'
|
||||
annotationCheck: "false"
|
||||
# It can be active/stop
|
||||
engineState: 'active'
|
||||
engineState: "active"
|
||||
appinfo:
|
||||
appns: 'openebs'
|
||||
applabel: 'name=maya-apiserver'
|
||||
appkind: 'deployment'
|
||||
appns: "openebs"
|
||||
applabel: "name=maya-apiserver"
|
||||
appkind: "deployment"
|
||||
chaosServiceAccount: control-plane-sa
|
||||
monitoring: false
|
||||
# It can be delete/retain
|
||||
jobCleanUpPolicy: 'delete'
|
||||
jobCleanUpPolicy: "delete"
|
||||
experiments:
|
||||
- name: openebs-control-plane-chaos
|
||||
spec:
|
||||
components:
|
||||
env:
|
||||
- name: OPENEBS_NAMESPACE
|
||||
value: 'openebs'
|
||||
value: "openebs"
|
||||
|
||||
## Period to wait before injection of chaos
|
||||
- name: RAMP_TIME
|
||||
value: '10'
|
||||
value: "10"
|
||||
|
||||
- name: FORCE
|
||||
value: ''
|
||||
value: ""
|
||||
|
||||
- name: LIB
|
||||
value: ''
|
||||
value: ""
|
||||
```
|
||||
|
||||
### Create the ChaosEngine Resource
|
||||
|
|
@ -185,7 +196,7 @@ spec:
|
|||
|
||||
### Check Chaos Experiment Result
|
||||
|
||||
- Check whether the OpenEBS control plane is resilient to the pod failure, once the experiment (job) is completed. The ChaosResult resource naming convention is: `<ChaosEngine-Name>-<ChaosExperiment-Name>`.
|
||||
- Check whether the OpenEBS control plane is resilient to the pod failure, once the experiment (job) is completed. The ChaosResult resource naming convention is: `{"<ChaosEngine-Name>-<ChaosExperiment-Name>"}`.
|
||||
|
||||
`kubectl describe chaosresult control-plane-chaos-openebs-control-plane-chaos -n openebs`
|
||||
|
||||
|
|
@ -1,10 +1,11 @@
|
|||
---
|
||||
id: version-1.1.0-openebs-pool-container-failure
|
||||
id: openebs-pool-container-failure
|
||||
title: OpenEBS Pool Container Failure Experiment Details
|
||||
sidebar_label: Pool Container Failure
|
||||
original_id: openebs-pool-container-failure
|
||||
---
|
||||
------
|
||||
|
||||
---
|
||||
|
||||
## Experiment Metadata
|
||||
|
||||
|
|
@ -28,34 +29,37 @@ original_id: openebs-pool-container-failure
|
|||
- Ensure that the Litmus Chaos Operator is running by executing `kubectl get pods` in operator namespace (typically, `litmus`). If not, install from [here](https://docs.litmuschaos.io/docs/getstarted/#install-litmus)
|
||||
- Ensure that the `openebs-pool-container-failure` experiment resource is available in the cluster. If not, install from [here](https://hub.litmuschaos.io/charts/openebs/experiments/openebs-pool-container-failure)
|
||||
- The DATA_PERSISTENCE can be enabled by provide the application's info in a configmap volume so that the experiment can perform necessary checks. Currently, LitmusChaos supports data consistency checks only for MySQL and Busybox.
|
||||
- For MYSQL data persistence check create a configmap as shown below in the application namespace (replace with actual credentials):
|
||||
|
||||
```
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: openebs-pool-container-failure
|
||||
data:
|
||||
parameters.yml: |
|
||||
dbuser: root
|
||||
dbpassword: k8sDem0
|
||||
dbname: test
|
||||
```
|
||||
- For Busybox data persistence check create a configmap as shown below in the application namespace (replace with actual credentials):
|
||||
- For MYSQL data persistence check create a configmap as shown below in the application namespace (replace with actual credentials):
|
||||
|
||||
```
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: openebs-pool-container-failure
|
||||
data:
|
||||
parameters.yml: |
|
||||
dbuser: root
|
||||
dbpassword: k8sDem0
|
||||
dbname: test
|
||||
```
|
||||
|
||||
- For Busybox data persistence check create a configmap as shown below in the application namespace (replace with actual credentials):
|
||||
|
||||
```
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: openebs-pool-container-failure
|
||||
data:
|
||||
parameters.yml: |
|
||||
blocksize: 4k
|
||||
blockcount: 1024
|
||||
testfile: exampleFile
|
||||
```
|
||||
|
||||
```
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: openebs-pool-container-failure
|
||||
data:
|
||||
parameters.yml: |
|
||||
blocksize: 4k
|
||||
blockcount: 1024
|
||||
testfile: exampleFile
|
||||
```
|
||||
- Ensure that the chaosServiceAccount used for the experiment has cluster-scope permissions as the experiment may involve carrying out the chaos in the `openebs` namespace
|
||||
while performing application health checks in its respective namespace.
|
||||
|
||||
|
|
@ -98,7 +102,8 @@ Use this sample RBAC manifest to create a chaosServiceAccount in the desired (ap
|
|||
|
||||
#### Sample Rbac Manifest
|
||||
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/openebs/openebs-pool-container-failure/rbac.yaml yaml)
|
||||
[embedmd]: # "https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/openebs/openebs-pool-container-failure/rbac.yaml yaml"
|
||||
|
||||
```yaml
|
||||
---
|
||||
apiVersion: v1
|
||||
|
|
@ -117,9 +122,32 @@ metadata:
|
|||
labels:
|
||||
name: pool-container-failure-sa
|
||||
rules:
|
||||
- apiGroups: ["","apps","litmuschaos.io","batch","extensions","storage.k8s.io","openebs.io"]
|
||||
resources: ["pods","jobs","daemonsets","replicasets","pods/exec","configmaps","secrets","persistentvolumeclaims","cstorvolumereplicas","chaosexperiments","chaosresults","chaosengines"]
|
||||
verbs: ["create","list","get","patch","update","delete"]
|
||||
- apiGroups:
|
||||
[
|
||||
"",
|
||||
"apps",
|
||||
"litmuschaos.io",
|
||||
"batch",
|
||||
"extensions",
|
||||
"storage.k8s.io",
|
||||
"openebs.io",
|
||||
]
|
||||
resources:
|
||||
[
|
||||
"pods",
|
||||
"jobs",
|
||||
"daemonsets",
|
||||
"replicasets",
|
||||
"pods/exec",
|
||||
"configmaps",
|
||||
"secrets",
|
||||
"persistentvolumeclaims",
|
||||
"cstorvolumereplicas",
|
||||
"chaosexperiments",
|
||||
"chaosresults",
|
||||
"chaosengines",
|
||||
]
|
||||
verbs: ["create", "list", "get", "patch", "update", "delete"]
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
kind: ClusterRoleBinding
|
||||
|
|
@ -132,9 +160,9 @@ roleRef:
|
|||
kind: ClusterRole
|
||||
name: pool-container-failure-sa
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: pool-container-failure-sa
|
||||
namespace: default
|
||||
- kind: ServiceAccount
|
||||
name: pool-container-failure-sa
|
||||
namespace: default
|
||||
```
|
||||
|
||||
### Prepare ChaosEngine
|
||||
|
|
@ -186,7 +214,8 @@ subjects:
|
|||
|
||||
#### Sample ChaosEngine Manifest
|
||||
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/openebs/openebs-pool-container-failure/engine.yaml yaml)
|
||||
[embedmd]: # "https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/openebs/openebs-pool-container-failure/engine.yaml yaml"
|
||||
|
||||
```yaml
|
||||
apiVersion: litmuschaos.io/v1alpha1
|
||||
kind: ChaosEngine
|
||||
|
|
@ -195,28 +224,28 @@ metadata:
|
|||
namespace: default
|
||||
spec:
|
||||
# It can be true/false
|
||||
annotationCheck: 'false'
|
||||
annotationCheck: "false"
|
||||
# It can be active/stop
|
||||
engineState: 'active'
|
||||
engineState: "active"
|
||||
#ex. values: ns1:name=percona,ns2:run=nginx
|
||||
auxiliaryAppInfo: ''
|
||||
auxiliaryAppInfo: ""
|
||||
appinfo:
|
||||
appns: 'default'
|
||||
applabel: 'app=nginx'
|
||||
appkind: 'deployment'
|
||||
appns: "default"
|
||||
applabel: "app=nginx"
|
||||
appkind: "deployment"
|
||||
chaosServiceAccount: pool-container-failure-sa
|
||||
monitoring: false
|
||||
# It can be delete/retain
|
||||
jobCleanUpPolicy: 'delete'
|
||||
jobCleanUpPolicy: "delete"
|
||||
experiments:
|
||||
- name: openebs-pool-container-failure
|
||||
spec:
|
||||
components:
|
||||
env:
|
||||
- name: APP_PVC
|
||||
value: 'demo-nginx-claim'
|
||||
value: "demo-nginx-claim"
|
||||
- name: DEPLOY_TYPE
|
||||
value: 'deployment'
|
||||
value: "deployment"
|
||||
```
|
||||
|
||||
### Create the ChaosEngine Resource
|
||||
|
|
@ -234,7 +263,7 @@ spec:
|
|||
### Check Chaos Experiment Result
|
||||
|
||||
- Check whether the application is resilient to the pool pod container failure, once the experiment (job) is completed. The ChaosResult resource naming convention
|
||||
is: `<ChaosEngine-Name>-<ChaosExperiment-Name>`.
|
||||
is: `{"<ChaosEngine-Name>-<ChaosExperiment-Name>"}`.
|
||||
|
||||
`kubectl describe chaosresult target-chaos-openebs-pool-container-failure -n <application-namespace>`
|
||||
|
||||
|
|
@ -1,10 +1,11 @@
|
|||
---
|
||||
id: version-1.2.0-openebs-pool-network-delay
|
||||
id: openebs-pool-network-delay
|
||||
title: OpenEBS Pool Network Latency Experiment Details
|
||||
sidebar_label: Pool Network Latency
|
||||
original_id: openebs-pool-network-delay
|
||||
---
|
||||
------
|
||||
|
||||
---
|
||||
|
||||
## Experiment Metadata
|
||||
|
||||
|
|
@ -47,6 +48,7 @@ data:
|
|||
dbpassword: k8sDem0
|
||||
dbname: test
|
||||
```
|
||||
|
||||
- For Busybox data persistence check create a configmap as shown below in the application namespace (replace with actual credentials):
|
||||
|
||||
```yaml
|
||||
|
|
@ -61,7 +63,8 @@ data:
|
|||
blockcount: 1024
|
||||
testfile: exampleFile
|
||||
```
|
||||
- Ensure that the chaosServiceAccount used for the experiment has cluster-scope permissions as the experiment may involve carrying out the chaos in the `openebs` namespace while performing application health checks in its respective namespace.
|
||||
|
||||
- Ensure that the chaosServiceAccount used for the experiment has cluster-scope permissions as the experiment may involve carrying out the chaos in the `openebs` namespace while performing application health checks in its respective namespace.
|
||||
|
||||
## Entry Criteria
|
||||
|
||||
|
|
@ -101,7 +104,9 @@ If the experiment tunable DATA_PERSISTENCE is set to 'mysql' or 'busybox':
|
|||
Use this sample RBAC manifest to create a chaosServiceAccount in the desired (app)namespace. This example consists of the minimum necessary cluster role permissions to execute the experiment.
|
||||
|
||||
#### Sample Rbac Manifest
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/openebs/openebs-pool-network-delay/rbac.yaml)
|
||||
|
||||
[embedmd]: # "https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/openebs/openebs-pool-network-delay/rbac.yaml"
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
|
|
@ -119,9 +124,34 @@ metadata:
|
|||
labels:
|
||||
name: pool-network-delay-sa
|
||||
rules:
|
||||
- apiGroups: ["","apps","litmuschaos.io","batch","extensions","storage.k8s.io","openebs.io"]
|
||||
resources: ["pods","pods/exec","pods/log","events","jobs","configmaps","services","persistentvolumeclaims","storageclasses","persistentvolumes","chaosengines","chaosexperiments","chaosresults","cstorpools","cstorvolumereplicas","replicasets"]
|
||||
verbs: ["create","list","get","patch","update","delete"]
|
||||
- apiGroups:
|
||||
[
|
||||
"",
|
||||
"apps",
|
||||
"litmuschaos.io",
|
||||
"batch",
|
||||
"extensions",
|
||||
"storage.k8s.io",
|
||||
"openebs.io",
|
||||
]
|
||||
resources:
|
||||
[
|
||||
"pods",
|
||||
"pods/exec",
|
||||
"jobs",
|
||||
"configmaps",
|
||||
"services",
|
||||
"persistentvolumeclaims",
|
||||
"storageclasses",
|
||||
"persistentvolumes",
|
||||
"chaosengines",
|
||||
"chaosexperiments",
|
||||
"chaosresults",
|
||||
"cstorpools",
|
||||
"cstorvolumereplicas",
|
||||
"replicasets",
|
||||
]
|
||||
verbs: ["create", "list", "get", "patch", "update", "delete"]
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
kind: ClusterRoleBinding
|
||||
|
|
@ -134,9 +164,9 @@ roleRef:
|
|||
kind: ClusterRole
|
||||
name: pool-network-delay-sa
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: pool-network-delay-sa
|
||||
namespace: default
|
||||
- kind: ServiceAccount
|
||||
name: pool-network-delay-sa
|
||||
namespace: default
|
||||
```
|
||||
|
||||
### Prepare ChaosEngine
|
||||
|
|
@ -187,7 +217,9 @@ subjects:
|
|||
</table>
|
||||
|
||||
#### Sample ChaosEngine Manifest
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/openebs/openebs-pool-network-delay/engine.yaml)
|
||||
|
||||
[embedmd]: # "https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/openebs/openebs-pool-network-delay/engine.yaml"
|
||||
|
||||
```yaml
|
||||
apiVersion: litmuschaos.io/v1alpha1
|
||||
kind: ChaosEngine
|
||||
|
|
@ -195,33 +227,32 @@ metadata:
|
|||
name: pool-chaos
|
||||
namespace: default
|
||||
spec:
|
||||
annotationCheck: 'false'
|
||||
annotationCheck: "false"
|
||||
appinfo:
|
||||
appns: 'default'
|
||||
applabel: 'app=nginx'
|
||||
appkind: 'deployment'
|
||||
appns: "default"
|
||||
applabel: "app=nginx"
|
||||
appkind: "deployment"
|
||||
chaosServiceAccount: pool-network-delay-sa
|
||||
monitoring: false
|
||||
# It can be delete/retain
|
||||
jobCleanUpPolicy: 'delete'
|
||||
jobCleanUpPolicy: "delete"
|
||||
experiments:
|
||||
- name: openebs-pool-network-delay
|
||||
spec:
|
||||
components:
|
||||
env:
|
||||
- name: APP_PVC
|
||||
value: 'demo-nginx-claim'
|
||||
value: "demo-nginx-claim"
|
||||
|
||||
- name: OPENEBS_NAMESPACE
|
||||
value: 'openebs'
|
||||
value: "openebs"
|
||||
|
||||
# in milliseconds
|
||||
- name: NETWORK_DELAY
|
||||
value: '60000'
|
||||
value: "60000"
|
||||
|
||||
- name: TOTAL_CHAOS_DURATION
|
||||
value: '60' # in seconds
|
||||
|
||||
value: "60" # in seconds
|
||||
```
|
||||
|
||||
### Create the ChaosEngine Resource
|
||||
|
|
@ -239,7 +270,7 @@ spec:
|
|||
|
||||
### Check Chaos Experiment Result
|
||||
|
||||
- Check whether the application is resilient to the pool network delays, once the experiment (job) is completed. The ChaosResult resource naming convention is: `<ChaosEngine-Name>-<ChaosExperiment-Name>`.
|
||||
- Check whether the application is resilient to the pool network delays, once the experiment (job) is completed. The ChaosResult resource naming convention is: `{"<ChaosEngine-Name>-<ChaosExperiment-Name>"}`.
|
||||
|
||||
`kubectl describe chaosresult pool-chaos-openebs-pool-network-delay -n <application-namespace>`
|
||||
|
||||
|
|
@ -1,10 +1,11 @@
|
|||
---
|
||||
id: version-1.2.0-openebs-pool-network-loss
|
||||
id: openebs-pool-network-loss
|
||||
title: OpenEBS Pool Network Loss Experiment Details
|
||||
sidebar_label: Pool Network Loss
|
||||
original_id: openebs-pool-network-loss
|
||||
---
|
||||
------
|
||||
|
||||
---
|
||||
|
||||
## Experiment Metadata
|
||||
|
||||
|
|
@ -47,6 +48,7 @@ data:
|
|||
dbpassword: k8sDem0
|
||||
dbname: test
|
||||
```
|
||||
|
||||
- For Busybox data persistence check create a configmap as shown below in the application namespace (replace with actual credentials):
|
||||
|
||||
```yaml
|
||||
|
|
@ -61,6 +63,7 @@ data:
|
|||
blockcount: 1024
|
||||
testfile: exampleFile
|
||||
```
|
||||
|
||||
- Ensure that the chaosServiceAccount used for the experiment has cluster-scope permissions as the experiment may involve carrying out the chaos in the `openebs` namespace while performing application health checks in its respective namespace.
|
||||
|
||||
## Entry Criteria
|
||||
|
|
@ -102,7 +105,8 @@ Use this sample RBAC manifest to create a chaosServiceAccount in the desired (ap
|
|||
|
||||
#### Sample Rbac Manifest
|
||||
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/openebs/openebs-pool-network-loss/rbac.yaml)
|
||||
[embedmd]: # "https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/openebs/openebs-pool-network-loss/rbac.yaml"
|
||||
|
||||
```yaml
|
||||
---
|
||||
apiVersion: v1
|
||||
|
|
@ -121,39 +125,37 @@ metadata:
|
|||
labels:
|
||||
name: pool-network-loss-sa
|
||||
rules:
|
||||
- apiGroups:
|
||||
- ""
|
||||
- "apps"
|
||||
- "litmuschaos.io"
|
||||
- "batch"
|
||||
- "extensions"
|
||||
- "storage.k8s.io"
|
||||
- "openebs.io"
|
||||
resources:
|
||||
- "pods"
|
||||
- "pods/exec"
|
||||
- "jobs"
|
||||
- "pods/log"
|
||||
- "events"
|
||||
- "configmaps"
|
||||
- "services"
|
||||
- "persistentvolumeclaims"
|
||||
- "storageclasses"
|
||||
- "persistentvolumeclaims"
|
||||
- "persistentvolumes"
|
||||
- "chaosengines"
|
||||
- "chaosexperiments"
|
||||
- "chaosresults"
|
||||
- "cstorpools"
|
||||
- "cstorvolumereplicas"
|
||||
- "replicasets"
|
||||
verbs:
|
||||
- "create"
|
||||
- "get"
|
||||
- "delete"
|
||||
- "list"
|
||||
- "patch"
|
||||
- "update"
|
||||
- apiGroups:
|
||||
- ""
|
||||
- "apps"
|
||||
- "litmuschaos.io"
|
||||
- "batch"
|
||||
- "extensions"
|
||||
- "storage.k8s.io"
|
||||
- "openebs.io"
|
||||
resources:
|
||||
- "pods"
|
||||
- "pods/exec"
|
||||
- "jobs"
|
||||
- "configmaps"
|
||||
- "services"
|
||||
- "persistentvolumeclaims"
|
||||
- "storageclasses"
|
||||
- "persistentvolumeclaims"
|
||||
- "persistentvolumes"
|
||||
- "chaosengines"
|
||||
- "chaosexperiments"
|
||||
- "chaosresults"
|
||||
- "cstorpools"
|
||||
- "cstorvolumereplicas"
|
||||
- "replicasets"
|
||||
verbs:
|
||||
- "create"
|
||||
- "get"
|
||||
- "delete"
|
||||
- "list"
|
||||
- "patch"
|
||||
- "update"
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
kind: ClusterRoleBinding
|
||||
|
|
@ -166,9 +168,9 @@ roleRef:
|
|||
kind: ClusterRole
|
||||
name: pool-network-loss-sa
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: pool-network-loss-sa
|
||||
namespace: default
|
||||
- kind: ServiceAccount
|
||||
name: pool-network-loss-sa
|
||||
namespace: default
|
||||
```
|
||||
|
||||
### Prepare ChaosEngine
|
||||
|
|
@ -219,7 +221,9 @@ subjects:
|
|||
</table>
|
||||
|
||||
#### Sample ChaosEngine Manifest
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/openebs/openebs-pool-network-loss/engine.yaml)
|
||||
|
||||
[embedmd]: # "https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/openebs/openebs-pool-network-loss/engine.yaml"
|
||||
|
||||
```yaml
|
||||
apiVersion: litmuschaos.io/v1alpha1
|
||||
kind: ChaosEngine
|
||||
|
|
@ -227,38 +231,37 @@ metadata:
|
|||
name: pool-chaos
|
||||
namespace: default
|
||||
spec:
|
||||
auxiliaryAppInfo: ''
|
||||
annotationCheck: 'false'
|
||||
auxiliaryAppInfo: ""
|
||||
annotationCheck: "false"
|
||||
# It can be active/stop
|
||||
engineState: 'active'
|
||||
engineState: "active"
|
||||
appinfo:
|
||||
appns: 'default'
|
||||
applabel: 'app=nginx'
|
||||
appkind: 'deployment'
|
||||
appns: "default"
|
||||
applabel: "app=nginx"
|
||||
appkind: "deployment"
|
||||
chaosServiceAccount: pool-network-loss-sa
|
||||
monitoring: false
|
||||
# It can be delete/retain
|
||||
jobCleanUpPolicy: 'delete'
|
||||
jobCleanUpPolicy: "delete"
|
||||
experiments:
|
||||
- name: openebs-pool-network-loss
|
||||
spec:
|
||||
components:
|
||||
env:
|
||||
- name: FORCE
|
||||
value: 'true'
|
||||
- name: FORCE
|
||||
value: "true"
|
||||
|
||||
- name: APP_PVC
|
||||
value: 'demo-nginx-claim'
|
||||
- name: APP_PVC
|
||||
value: "demo-nginx-claim"
|
||||
|
||||
- name: OPENEBS_NAMESPACE
|
||||
value: 'openebs'
|
||||
- name: OPENEBS_NAMESPACE
|
||||
value: "openebs"
|
||||
|
||||
- name: NETWORK_PACKET_LOSS_PERCENTAGE
|
||||
value: '100'
|
||||
|
||||
- name: TOTAL_CHAOS_DURATION
|
||||
value: '120' # in seconds
|
||||
- name: NETWORK_PACKET_LOSS_PERCENTAGE
|
||||
value: "100"
|
||||
|
||||
- name: TOTAL_CHAOS_DURATION
|
||||
value: "120" # in seconds
|
||||
```
|
||||
|
||||
### Create the ChaosEngine Resource
|
||||
|
|
@ -276,7 +279,7 @@ spec:
|
|||
|
||||
### Check Chaos Experiment Result
|
||||
|
||||
- Check whether the application is resilient to the pool network loss, once the experiment (job) is completed. The ChaosResult resource naming convention is: `<ChaosEngine-Name>-<ChaosExperiment-Name>`.
|
||||
- Check whether the application is resilient to the pool network loss, once the experiment (job) is completed. The ChaosResult resource naming convention is: `{"<ChaosEngine-Name>-<ChaosExperiment-Name>"}`.
|
||||
|
||||
`kubectl describe chaosresult pool-chaos-openebs-pool-network-loss -n <application-namespace>`
|
||||
|
||||
|
|
@ -1,10 +1,11 @@
|
|||
---
|
||||
id: version-1.1.0-openebs-pool-pod-failure
|
||||
id: openebs-pool-pod-failure
|
||||
title: OpenEBS Pool Pod Failure Experiment Details
|
||||
sidebar_label: Pool Pod Failure
|
||||
original_id: openebs-pool-pod-failure
|
||||
---
|
||||
------
|
||||
|
||||
---
|
||||
|
||||
## Experiment Metadata
|
||||
|
||||
|
|
@ -28,6 +29,7 @@ original_id: openebs-pool-pod-failure
|
|||
- Ensure that the Litmus Chaos Operator is running by executing `kubectl get pods` in operator namespace (typically, `litmus`). If not, install from [here](https://docs.litmuschaos.io/docs/getstarted/#install-litmus)
|
||||
- Ensure that the `openebs-pool-pod-failure` experiment resource is available in the cluster. If not, install from [here](https://hub.litmuschaos.io/charts/openebs/experiments/openebs-pool-pod-failure)
|
||||
- The DATA_PERSISTENCE can be enabled by provide the application's info in a configmap volume so that the experiment can perform necessary checks. Currently, LitmusChaos supports data consistency checks only for MySQL and Busybox.
|
||||
|
||||
- For MYSQL data persistence check create a configmap as shown below in the application namespace (replace with actual credentials):
|
||||
|
||||
```
|
||||
|
|
@ -42,6 +44,7 @@ original_id: openebs-pool-pod-failure
|
|||
dbpassword: k8sDem0
|
||||
dbname: test
|
||||
```
|
||||
|
||||
- For Busybox data persistence check create a configmap as shown below in the application namespace (replace with actual credentials):
|
||||
|
||||
```
|
||||
|
|
@ -56,6 +59,7 @@ original_id: openebs-pool-pod-failure
|
|||
blockcount: 1024
|
||||
testfile: exampleFile
|
||||
```
|
||||
|
||||
- Ensure that the chaosServiceAccount used for the experiment has cluster-scope permissions as the experiment may involve carrying out the chaos in the `openebs` namespace while performing application health checks in its respective namespace.
|
||||
|
||||
## Entry Criteria
|
||||
|
|
@ -95,7 +99,8 @@ Use this sample RBAC manifest to create a chaosServiceAccount in the desired (ap
|
|||
|
||||
#### Sample Rbac Manifest
|
||||
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/openebs/openebs-pool-pod-failure/rbac.yaml yaml)
|
||||
[embedmd]: # "https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/openebs/openebs-pool-pod-failure/rbac.yaml yaml"
|
||||
|
||||
```yaml
|
||||
---
|
||||
apiVersion: v1
|
||||
|
|
@ -114,12 +119,35 @@ metadata:
|
|||
labels:
|
||||
name: pool-pod-failure-sa
|
||||
rules:
|
||||
- apiGroups: ["","apps","litmuschaos.io","batch","extensions","storage.k8s.io","openebs.io"]
|
||||
resources: ["pods","jobs","deployments","configmaps","secrets","replicasets","persistentvolumeclaims","storageclasses","cstorvolumereplicas","chaosexperiments","chaosresults","chaosengines"]
|
||||
verbs: ["create","list","get","patch","update","delete"]
|
||||
- apiGroups: [""]
|
||||
resources: ["nodes"]
|
||||
verbs: ["get","list"]
|
||||
- apiGroups:
|
||||
[
|
||||
"",
|
||||
"apps",
|
||||
"litmuschaos.io",
|
||||
"batch",
|
||||
"extensions",
|
||||
"storage.k8s.io",
|
||||
"openebs.io",
|
||||
]
|
||||
resources:
|
||||
[
|
||||
"pods",
|
||||
"jobs",
|
||||
"deployments",
|
||||
"configmaps",
|
||||
"secrets",
|
||||
"replicasets",
|
||||
"persistentvolumeclaims",
|
||||
"storageclasses",
|
||||
"cstorvolumereplicas",
|
||||
"chaosexperiments",
|
||||
"chaosresults",
|
||||
"chaosengines",
|
||||
]
|
||||
verbs: ["create", "list", "get", "patch", "update", "delete"]
|
||||
- apiGroups: [""]
|
||||
resources: ["nodes"]
|
||||
verbs: ["get", "list"]
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
kind: ClusterRoleBinding
|
||||
|
|
@ -132,9 +160,9 @@ roleRef:
|
|||
kind: ClusterRole
|
||||
name: pool-pod-failure-sa
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: pool-pod-failure-sa
|
||||
namespace: default
|
||||
- kind: ServiceAccount
|
||||
name: pool-pod-failure-sa
|
||||
namespace: default
|
||||
```
|
||||
|
||||
### Prepare ChaosEngine
|
||||
|
|
@ -180,7 +208,8 @@ subjects:
|
|||
|
||||
#### Sample ChaosEngine Manifest
|
||||
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/openebs/openebs-pool-pod-failure/engine.yaml yaml)
|
||||
[embedmd]: # "https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/openebs/openebs-pool-pod-failure/engine.yaml yaml"
|
||||
|
||||
```yaml
|
||||
apiVersion: litmuschaos.io/v1alpha1
|
||||
kind: ChaosEngine
|
||||
|
|
@ -189,30 +218,30 @@ metadata:
|
|||
namespace: default
|
||||
spec:
|
||||
# It can be true/false
|
||||
annotationCheck: 'false'
|
||||
annotationCheck: "false"
|
||||
# It can be active/stop
|
||||
engineState: 'active'
|
||||
engineState: "active"
|
||||
#ex. values: ns1:name=percona,ns2:run=nginx
|
||||
auxiliaryAppInfo: ''
|
||||
auxiliaryAppInfo: ""
|
||||
appinfo:
|
||||
appns: 'default'
|
||||
applabel: 'app=nginx'
|
||||
appkind: 'deployment'
|
||||
appns: "default"
|
||||
applabel: "app=nginx"
|
||||
appkind: "deployment"
|
||||
chaosServiceAccount: pool-pod-failure-sa
|
||||
monitoring: false
|
||||
# It can be delete/retain
|
||||
jobCleanUpPolicy: 'delete'
|
||||
jobCleanUpPolicy: "delete"
|
||||
experiments:
|
||||
- name: openebs-pool-pod-failure
|
||||
spec:
|
||||
components:
|
||||
env:
|
||||
- name: FORCE
|
||||
value: 'true'
|
||||
value: "true"
|
||||
- name: APP_PVC
|
||||
value: 'demo-nginx-claim'
|
||||
value: "demo-nginx-claim"
|
||||
- name: DEPLOY_TYPE
|
||||
value: 'deployment'
|
||||
value: "deployment"
|
||||
```
|
||||
|
||||
### Create the ChaosEngine Resource
|
||||
|
|
@ -229,7 +258,7 @@ spec:
|
|||
|
||||
### Check Chaos Experiment Result
|
||||
|
||||
- Check whether the application is resilient to the pool pod failure, once the experiment (job) is completed. The ChaosResult resource naming convention is: `<ChaosEngine-Name>-<ChaosExperiment-Name>`.
|
||||
- Check whether the application is resilient to the pool pod failure, once the experiment (job) is completed. The ChaosResult resource naming convention is: `{"<ChaosEngine-Name>-<ChaosExperiment-Name>"}`.
|
||||
|
||||
`kubectl describe chaosresult target-chaos-openebs-pool-pod-failure -n <application-namespace>`
|
||||
|
||||
|
|
@ -1,10 +1,11 @@
|
|||
---
|
||||
id: version-1.1.0-openebs-target-container-failure
|
||||
id: openebs-target-container-failure
|
||||
title: OpenEBS Target Container Failure Experiment Details
|
||||
sidebar_label: Target Container Failure
|
||||
original_id: openebs-target-container-failure
|
||||
---
|
||||
------
|
||||
|
||||
---
|
||||
|
||||
## Experiment Metadata
|
||||
|
||||
|
|
@ -28,34 +29,37 @@ original_id: openebs-target-container-failure
|
|||
- Ensure that the Litmus Chaos Operator is running by executing `kubectl get pods` in operator namespace (typically, `litmus`). If not, install from [here](https://docs.litmuschaos.io/docs/getstarted/#install-litmus)
|
||||
- Ensure that the `openebs-target-container-failure` experiment resource is available in the cluster. If not, install from [here](https://hub.litmuschaos.io/charts/openebs/experiments/openebs-target-container-failure)
|
||||
- The DATA_PERSISTENCE can be enabled by provide the application's info in a configmap volume so that the experiment can perform necessary checks. Currently, LitmusChaos supports data consistency checks only for MySQL and Busybox.
|
||||
- For MYSQL data persistence check create a configmap as shown below in the application namespace (replace with actual credentials):
|
||||
|
||||
```
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: openebs-target-container-failure
|
||||
data:
|
||||
parameters.yml: |
|
||||
dbuser: root
|
||||
dbpassword: k8sDem0
|
||||
dbname: test
|
||||
```
|
||||
- For Busybox data persistence check create a configmap as shown below in the application namespace (replace with actual credentials):
|
||||
- For MYSQL data persistence check create a configmap as shown below in the application namespace (replace with actual credentials):
|
||||
|
||||
```
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: openebs-target-container-failure
|
||||
data:
|
||||
parameters.yml: |
|
||||
dbuser: root
|
||||
dbpassword: k8sDem0
|
||||
dbname: test
|
||||
```
|
||||
|
||||
- For Busybox data persistence check create a configmap as shown below in the application namespace (replace with actual credentials):
|
||||
|
||||
```
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: openebs-target-container-failure
|
||||
data:
|
||||
parameters.yml: |
|
||||
blocksize: 4k
|
||||
blockcount: 1024
|
||||
testfile: exampleFile
|
||||
```
|
||||
|
||||
```
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: openebs-target-container-failure
|
||||
data:
|
||||
parameters.yml: |
|
||||
blocksize: 4k
|
||||
blockcount: 1024
|
||||
testfile: exampleFile
|
||||
```
|
||||
- Ensure that the chaosServiceAccount used for the experiment has cluster-scope permissions as the experiment may involve carrying out the chaos in the `openebs` namespace
|
||||
while performing application health checks in its respective namespace.
|
||||
|
||||
|
|
@ -99,7 +103,8 @@ If the experiment tunable DATA_PERSISTENCE is set to 'enabled':
|
|||
|
||||
#### Sample Rbac Manifest
|
||||
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/openebs/openebs-target-container-failure/rbac.yaml yaml)
|
||||
[embedmd]: # "https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/openebs/openebs-target-container-failure/rbac.yaml yaml"
|
||||
|
||||
```yaml
|
||||
---
|
||||
apiVersion: v1
|
||||
|
|
@ -118,9 +123,23 @@ metadata:
|
|||
labels:
|
||||
name: target-container-failure-sa
|
||||
rules:
|
||||
- apiGroups: ["","litmuschaos.io","batch","apps","storage.k8s.io"]
|
||||
resources: ["pods","jobs","pods/exec","daemonsets","configmaps","secrets","persistentvolumeclaims","storageclasses","persistentvolumes","chaosengines","chaosexperiments","chaosresults"]
|
||||
verbs: ["create","list","get","patch","update","delete"]
|
||||
- apiGroups: ["", "litmuschaos.io", "batch", "apps", "storage.k8s.io"]
|
||||
resources:
|
||||
[
|
||||
"pods",
|
||||
"jobs",
|
||||
"pods/exec",
|
||||
"daemonsets",
|
||||
"configmaps",
|
||||
"secrets",
|
||||
"persistentvolumeclaims",
|
||||
"storageclasses",
|
||||
"persistentvolumes",
|
||||
"chaosengines",
|
||||
"chaosexperiments",
|
||||
"chaosresults",
|
||||
]
|
||||
verbs: ["create", "list", "get", "patch", "update", "delete"]
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
kind: ClusterRoleBinding
|
||||
|
|
@ -133,9 +152,9 @@ roleRef:
|
|||
kind: ClusterRole
|
||||
name: target-container-failure-sa
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: target-container-failure-sa
|
||||
namespace: default
|
||||
- kind: ServiceAccount
|
||||
name: target-container-failure-sa
|
||||
namespace: default
|
||||
```
|
||||
|
||||
### Prepare ChaosEngine
|
||||
|
|
@ -163,7 +182,7 @@ subjects:
|
|||
<td> LIB_IMAGE </td>
|
||||
<td> The chaos library image used to run the kill command </td>
|
||||
<td> Optional </td>
|
||||
<td> Defaults to `gaiaadm/pumba:0.4.8`. Supported: `{docker : gaiaadm/pumba:0.4.8, containerd: gprasath/crictl:ci}` </td>
|
||||
<td> Defaults to `gaiaadm/pumba:0.4.8`. Supported: `{"{docker : gaiaadm/pumba:0.4.8, containerd: gprasath/crictl:ci}"}` </td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td> CONTAINER_RUNTIME </td>
|
||||
|
|
@ -199,7 +218,8 @@ subjects:
|
|||
|
||||
#### Sample ChaosEngine Manifest
|
||||
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/openebs/openebs-target-container-failure/engine.yaml yaml)
|
||||
[embedmd]: # "https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/openebs/openebs-target-container-failure/engine.yaml yaml"
|
||||
|
||||
```yaml
|
||||
apiVersion: litmuschaos.io/v1alpha1
|
||||
kind: ChaosEngine
|
||||
|
|
@ -208,30 +228,30 @@ metadata:
|
|||
namespace: default
|
||||
spec:
|
||||
# It can be true/false
|
||||
annotationCheck: 'false'
|
||||
annotationCheck: "false"
|
||||
# It can be active/stop
|
||||
engineState: 'active'
|
||||
engineState: "active"
|
||||
#ex. values: ns1:name=percona,ns2:run=nginx
|
||||
auxiliaryAppInfo: ''
|
||||
auxiliaryAppInfo: ""
|
||||
appinfo:
|
||||
appns: 'default'
|
||||
applabel: 'app=nginx'
|
||||
appkind: 'deployment'
|
||||
appns: "default"
|
||||
applabel: "app=nginx"
|
||||
appkind: "deployment"
|
||||
chaosServiceAccount: target-container-failure-sa
|
||||
monitoring: false
|
||||
# It can be delete/retain
|
||||
jobCleanUpPolicy: 'delete'
|
||||
jobCleanUpPolicy: "delete"
|
||||
experiments:
|
||||
- name: openebs-target-container-failure
|
||||
spec:
|
||||
components:
|
||||
env:
|
||||
- name: TARGET_CONTAINER
|
||||
value: 'cstor-istgt'
|
||||
value: "cstor-istgt"
|
||||
- name: APP_PVC
|
||||
value: 'demo-nginx-claim'
|
||||
value: "demo-nginx-claim"
|
||||
- name: DEPLOY_TYPE
|
||||
value: 'deployment'
|
||||
value: "deployment"
|
||||
```
|
||||
|
||||
### Create the ChaosEngine Resource
|
||||
|
|
@ -249,7 +269,7 @@ spec:
|
|||
### Check Chaos Experiment Result
|
||||
|
||||
- Check whether the application is resilient to the target container kill, once the experiment (job) is completed. The ChaosResult resource naming convention
|
||||
is: `<ChaosEngine-Name>-<ChaosExperiment-Name>`.
|
||||
is: `{"<ChaosEngine-Name>-<ChaosExperiment-Name>"}`.
|
||||
|
||||
`kubectl describe chaosresult target-chaos-openebs-target-container-failure -n <application-namespace>`
|
||||
|
||||
|
|
@ -1,10 +1,11 @@
|
|||
---
|
||||
id: version-1.2.0-openebs-target-network-delay
|
||||
id: openebs-target-network-delay
|
||||
title: OpenEBS Target Network Latency Experiment Details
|
||||
sidebar_label: Target Network Latency
|
||||
original_id: openebs-target-network-delay
|
||||
---
|
||||
------
|
||||
|
||||
---
|
||||
|
||||
## Experiment Metadata
|
||||
|
||||
|
|
@ -29,34 +30,37 @@ original_id: openebs-target-network-delay
|
|||
- Ensure that the Litmus Chaos Operator is running by executing `kubectl get pods` in operator namespace (typically, `litmus`). If not, install from [here](https://docs.litmuschaos.io/docs/getstarted/#install-litmus)
|
||||
- Ensure that the `openebs-target-network-delay` experiment resource is available in the cluster. If not, install from [here](https://hub.litmuschaos.io/charts/openebs/experiments/openebs-target-network-delay)
|
||||
- The DATA_PERSISTENCE can be enabled by provide the application's info in a configmap volume so that the experiment can perform necessary checks. Currently, LitmusChaos supports data consistency checks only for MySQL and Busybox.
|
||||
- For MYSQL data persistence check create a configmap as shown below in the application namespace (replace with actual credentials):
|
||||
|
||||
```
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: openebs-target-network-delay
|
||||
data:
|
||||
parameters.yml: |
|
||||
dbuser: root
|
||||
dbpassword: k8sDem0
|
||||
dbname: test
|
||||
```
|
||||
- For Busybox data persistence check create a configmap as shown below in the application namespace (replace with actual credentials):
|
||||
- For MYSQL data persistence check create a configmap as shown below in the application namespace (replace with actual credentials):
|
||||
|
||||
```
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: openebs-target-network-delay
|
||||
data:
|
||||
parameters.yml: |
|
||||
dbuser: root
|
||||
dbpassword: k8sDem0
|
||||
dbname: test
|
||||
```
|
||||
|
||||
- For Busybox data persistence check create a configmap as shown below in the application namespace (replace with actual credentials):
|
||||
|
||||
```
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: openebs-target-network-delay
|
||||
data:
|
||||
parameters.yml: |
|
||||
blocksize: 4k
|
||||
blockcount: 1024
|
||||
testfile: exampleFile
|
||||
```
|
||||
|
||||
```
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: openebs-target-network-delay
|
||||
data:
|
||||
parameters.yml: |
|
||||
blocksize: 4k
|
||||
blockcount: 1024
|
||||
testfile: exampleFile
|
||||
```
|
||||
- Ensure that the chaosServiceAccount used for the experiment has cluster-scope permissions as the experiment may involve carrying out the chaos in the `openebs` namespace
|
||||
while performing application health checks in its respective namespace.
|
||||
|
||||
|
|
@ -99,7 +103,8 @@ Use this sample RBAC manifest to create a chaosServiceAccount in the desired (ap
|
|||
|
||||
#### Sample Rbac Manifest
|
||||
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/openebs/openebs-target-network-delay/rbac.yaml yaml)
|
||||
[embedmd]: # "https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/openebs/openebs-target-network-delay/rbac.yaml yaml"
|
||||
|
||||
```yaml
|
||||
---
|
||||
apiVersion: v1
|
||||
|
|
@ -118,9 +123,24 @@ metadata:
|
|||
labels:
|
||||
name: target-network-delay-sa
|
||||
rules:
|
||||
- apiGroups: ["","apps","litmuschaos.io","batch","extensions","storage.k8s.io"]
|
||||
resources: ["pods","pods/exec","pods/log","events","jobs","configmaps","secrets","services","persistentvolumeclaims","storageclasses","persistentvolumes","chaosexperiments","chaosresults","chaosengines"]
|
||||
verbs: ["create","list","get","patch","update","delete"]
|
||||
- apiGroups:
|
||||
["", "apps", "litmuschaos.io", "batch", "extensions", "storage.k8s.io"]
|
||||
resources:
|
||||
[
|
||||
"pods",
|
||||
"pods/exec",
|
||||
"jobs",
|
||||
"configmaps",
|
||||
"secrets",
|
||||
"services",
|
||||
"persistentvolumeclaims",
|
||||
"storageclasses",
|
||||
"persistentvolumes",
|
||||
"chaosexperiments",
|
||||
"chaosresults",
|
||||
"chaosengines",
|
||||
]
|
||||
verbs: ["create", "list", "get", "patch", "update", "delete"]
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
kind: ClusterRoleBinding
|
||||
|
|
@ -133,9 +153,9 @@ roleRef:
|
|||
kind: ClusterRole
|
||||
name: target-network-delay-sa
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: target-network-delay-sa
|
||||
namespace: default
|
||||
- kind: ServiceAccount
|
||||
name: target-network-delay-sa
|
||||
namespace: default
|
||||
```
|
||||
|
||||
### Prepare ChaosEngine
|
||||
|
|
@ -205,7 +225,8 @@ subjects:
|
|||
|
||||
#### Sample ChaosEngine Manifest
|
||||
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/openebs/openebs-target-network-delay/engine.yaml yaml)
|
||||
[embedmd]: # "https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/openebs/openebs-target-network-delay/engine.yaml yaml"
|
||||
|
||||
```yaml
|
||||
apiVersion: litmuschaos.io/v1alpha1
|
||||
kind: ChaosEngine
|
||||
|
|
@ -214,38 +235,38 @@ metadata:
|
|||
namespace: default
|
||||
spec:
|
||||
# It can be true/false
|
||||
annotationCheck: 'false'
|
||||
annotationCheck: "false"
|
||||
# It can be active/stop
|
||||
engineState: 'active'
|
||||
engineState: "active"
|
||||
#ex. values: ns1:name=percona,ns2:run=nginx
|
||||
auxiliaryAppInfo: ''
|
||||
auxiliaryAppInfo: ""
|
||||
appinfo:
|
||||
appns: 'default'
|
||||
applabel: 'app=nginx'
|
||||
appkind: 'deployment'
|
||||
appns: "default"
|
||||
applabel: "app=nginx"
|
||||
appkind: "deployment"
|
||||
chaosServiceAccount: target-network-delay-sa
|
||||
monitoring: false
|
||||
# It can be delete/retain
|
||||
jobCleanUpPolicy: 'delete'
|
||||
jobCleanUpPolicy: "delete"
|
||||
experiments:
|
||||
- name: openebs-target-network-delay
|
||||
spec:
|
||||
components:
|
||||
env:
|
||||
- name: TARGET_CONTAINER
|
||||
value: 'cstor-istgt'
|
||||
value: "cstor-istgt"
|
||||
|
||||
- name: APP_PVC
|
||||
value: 'demo-nginx-claim'
|
||||
value: "demo-nginx-claim"
|
||||
|
||||
- name: DEPLOY_TYPE
|
||||
value: 'deployment'
|
||||
value: "deployment"
|
||||
|
||||
- name: NETWORK_DELAY
|
||||
value: '30000'
|
||||
value: "30000"
|
||||
|
||||
- name: TOTAL_CHAOS_DURATION
|
||||
value: '60' # in seconds
|
||||
value: "60" # in seconds
|
||||
```
|
||||
|
||||
### Create the ChaosEngine Resource
|
||||
|
|
@ -264,7 +285,7 @@ spec:
|
|||
### Check Chaos Experiment Result
|
||||
|
||||
- Check whether the application is resilient to the target network delays, once the experiment (job) is completed. The ChaosResult resource naming
|
||||
convention is: `<ChaosEngine-Name>-<ChaosExperiment-Name>`.
|
||||
convention is: `{"<ChaosEngine-Name>-<ChaosExperiment-Name>"}`.
|
||||
|
||||
`kubectl describe chaosresult target-chaos-openebs-target-network-delay -n <application-namespace>`
|
||||
|
||||
|
|
@ -1,10 +1,11 @@
|
|||
---
|
||||
id: version-1.1.0-openebs-target-network-loss
|
||||
id: openebs-target-network-loss
|
||||
title: OpenEBS Target Network Loss Experiment Details
|
||||
sidebar_label: Target Network Loss
|
||||
original_id: openebs-target-network-loss
|
||||
---
|
||||
------
|
||||
|
||||
---
|
||||
|
||||
## Experiment Metadata
|
||||
|
||||
|
|
@ -29,34 +30,37 @@ original_id: openebs-target-network-loss
|
|||
- Ensure that the Litmus Chaos Operator is running by executing `kubectl get pods` in operator namespace (typically, `litmus`). If not, install from [here](https://docs.litmuschaos.io/docs/getstarted/#install-litmus)
|
||||
- Ensure that the `openebs-target-network-loss` experiment resource is available in the cluster. If not, install from [here](https://hub.litmuschaos.io/charts/openebs/experiments/openebs-target-network-loss)
|
||||
- The DATA_PERSISTENCE can be enabled by provide the application's info in a configmap volume so that the experiment can perform necessary checks. Currently, LitmusChaos supports data consistency checks only for MySQL and Busybox.
|
||||
- For MYSQL data persistence check create a configmap as shown below in the application namespace (replace with actual credentials):
|
||||
|
||||
```
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: openebs-target-network-loss
|
||||
data:
|
||||
parameters.yml: |
|
||||
dbuser: root
|
||||
dbpassword: k8sDem0
|
||||
dbname: test
|
||||
```
|
||||
- For Busybox data persistence check create a configmap as shown below in the application namespace (replace with actual credentials):
|
||||
- For MYSQL data persistence check create a configmap as shown below in the application namespace (replace with actual credentials):
|
||||
|
||||
```
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: openebs-target-network-loss
|
||||
data:
|
||||
parameters.yml: |
|
||||
dbuser: root
|
||||
dbpassword: k8sDem0
|
||||
dbname: test
|
||||
```
|
||||
|
||||
- For Busybox data persistence check create a configmap as shown below in the application namespace (replace with actual credentials):
|
||||
|
||||
```
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: openebs-target-network-loss
|
||||
data:
|
||||
parameters.yml: |
|
||||
blocksize: 4k
|
||||
blockcount: 1024
|
||||
testfile: exampleFile
|
||||
```
|
||||
|
||||
```
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: openebs-target-network-loss
|
||||
data:
|
||||
parameters.yml: |
|
||||
blocksize: 4k
|
||||
blockcount: 1024
|
||||
testfile: exampleFile
|
||||
```
|
||||
- Ensure that the chaosServiceAccount used for the experiment has cluster-scope permissions as the experiment may involve carrying out the chaos in the `openebs` namespace
|
||||
while performing application health checks in its respective namespace.
|
||||
|
||||
|
|
@ -99,7 +103,8 @@ Use this sample RBAC manifest to create a chaosServiceAccount in the desired (ap
|
|||
|
||||
#### Sample Rbac Manifest
|
||||
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/openebs/openebs-target-network-loss/rbac.yaml yaml)
|
||||
[embedmd]: # "https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/openebs/openebs-target-network-loss/rbac.yaml yaml"
|
||||
|
||||
```yaml
|
||||
---
|
||||
apiVersion: v1
|
||||
|
|
@ -118,9 +123,24 @@ metadata:
|
|||
labels:
|
||||
name: target-network-loss-sa
|
||||
rules:
|
||||
- apiGroups: ["","apps","litmuschaos.io","batch","extensions","storage.k8s.io"]
|
||||
resources: ["pods","pods/exec","jobs","configmaps","secrets","services","persistentvolumeclaims","storageclasses","persistentvolumes","chaosexperiments","chaosresults","chaosengines"]
|
||||
verbs: ["create","list","get","patch","update","delete"]
|
||||
- apiGroups:
|
||||
["", "apps", "litmuschaos.io", "batch", "extensions", "storage.k8s.io"]
|
||||
resources:
|
||||
[
|
||||
"pods",
|
||||
"pods/exec",
|
||||
"jobs",
|
||||
"configmaps",
|
||||
"secrets",
|
||||
"services",
|
||||
"persistentvolumeclaims",
|
||||
"storageclasses",
|
||||
"persistentvolumes",
|
||||
"chaosexperiments",
|
||||
"chaosresults",
|
||||
"chaosengines",
|
||||
]
|
||||
verbs: ["create", "list", "get", "patch", "update", "delete"]
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
kind: ClusterRoleBinding
|
||||
|
|
@ -133,9 +153,9 @@ roleRef:
|
|||
kind: ClusterRole
|
||||
name: target-network-loss-sa
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: target-network-loss-sa
|
||||
namespace: default
|
||||
- kind: ServiceAccount
|
||||
name: target-network-loss-sa
|
||||
namespace: default
|
||||
```
|
||||
|
||||
### Prepare ChaosEngine
|
||||
|
|
@ -193,7 +213,8 @@ subjects:
|
|||
|
||||
#### Sample ChaosEngine Manifest
|
||||
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/openebs/openebs-target-network-loss/engine.yaml yaml)
|
||||
[embedmd]: # "https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/openebs/openebs-target-network-loss/engine.yaml yaml"
|
||||
|
||||
```yaml
|
||||
apiVersion: litmuschaos.io/v1alpha1
|
||||
kind: ChaosEngine
|
||||
|
|
@ -202,35 +223,35 @@ metadata:
|
|||
namespace: default
|
||||
spec:
|
||||
# It can be true/false
|
||||
annotationCheck: 'false'
|
||||
annotationCheck: "false"
|
||||
# It can be active/stop
|
||||
engineState: 'active'
|
||||
engineState: "active"
|
||||
#ex. values: ns1:name=percona,ns2:run=nginx
|
||||
auxiliaryAppInfo: ''
|
||||
auxiliaryAppInfo: ""
|
||||
appinfo:
|
||||
appns: 'default'
|
||||
applabel: 'app=nginx'
|
||||
appkind: 'deployment'
|
||||
appns: "default"
|
||||
applabel: "app=nginx"
|
||||
appkind: "deployment"
|
||||
chaosServiceAccount: target-network-loss-sa
|
||||
monitoring: false
|
||||
# It can be delete/retain
|
||||
jobCleanUpPolicy: 'delete'
|
||||
jobCleanUpPolicy: "delete"
|
||||
experiments:
|
||||
- name: openebs-target-network-loss
|
||||
spec:
|
||||
components:
|
||||
env:
|
||||
- name: TARGET_CONTAINER
|
||||
value: 'cstor-istgt'
|
||||
value: "cstor-istgt"
|
||||
|
||||
- name: APP_PVC
|
||||
value: 'demo-nginx-claim'
|
||||
value: "demo-nginx-claim"
|
||||
|
||||
- name: DEPLOY_TYPE
|
||||
value: 'deployment'
|
||||
value: "deployment"
|
||||
|
||||
- name: TOTAL_CHAOS_DURATION
|
||||
value: '120' # in seconds
|
||||
value: "120" # in seconds
|
||||
```
|
||||
|
||||
### Create the ChaosEngine Resource
|
||||
|
|
@ -249,7 +270,7 @@ spec:
|
|||
### Check Chaos Experiment Result
|
||||
|
||||
- Check whether the application is resilient to the target network loss, once the experiment (job) is completed. The ChaosResult resource naming
|
||||
convention is: `<ChaosEngine-Name>-<ChaosExperiment-Name>`.
|
||||
convention is: `{"<ChaosEngine-Name>-<ChaosExperiment-Name>"}`.
|
||||
|
||||
`kubectl describe chaosresult target-chaos-openebs-target-network-loss -n <application-namespace>`
|
||||
|
||||
|
|
@ -1,10 +1,11 @@
|
|||
---
|
||||
id: version-1.2.0-openebs-target-pod-failure
|
||||
id: openebs-target-pod-failure
|
||||
title: OpenEBS Target Pod Failure Experiment Details
|
||||
sidebar_label: Target Pod Failure
|
||||
original_id: openebs-target-pod-failure
|
||||
---
|
||||
------
|
||||
|
||||
---
|
||||
|
||||
## Experiment Metadata
|
||||
|
||||
|
|
@ -28,34 +29,37 @@ original_id: openebs-target-pod-failure
|
|||
- Ensure that the Litmus Chaos Operator is running by executing `kubectl get pods` in operator namespace (typically, `litmus`). If not, install from [here](https://docs.litmuschaos.io/docs/getstarted/#install-litmus)
|
||||
- Ensure that the `openebs-target-pod-failure` experiment resource is available in the cluster. If not, install from [here](https://hub.litmuschaos.io/charts/openebs/experiments/openebs-target-pod-failure)
|
||||
- The DATA_PERSISTENCE can be enabled by provide the application's info in a configmap volume so that the experiment can perform necessary checks. Currently, LitmusChaos supports data consistency checks only for MySQL and Busybox.
|
||||
- For MYSQL data persistence check create a configmap as shown below in the application namespace (replace with actual credentials):
|
||||
|
||||
```
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: openebs-target-pod-failure
|
||||
data:
|
||||
parameters.yml: |
|
||||
dbuser: root
|
||||
dbpassword: k8sDem0
|
||||
dbname: test
|
||||
```
|
||||
- For Busybox data persistence check create a configmap as shown below in the application namespace (replace with actual credentials):
|
||||
- For MYSQL data persistence check create a configmap as shown below in the application namespace (replace with actual credentials):
|
||||
|
||||
```
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: openebs-target-pod-failure
|
||||
data:
|
||||
parameters.yml: |
|
||||
dbuser: root
|
||||
dbpassword: k8sDem0
|
||||
dbname: test
|
||||
```
|
||||
|
||||
- For Busybox data persistence check create a configmap as shown below in the application namespace (replace with actual credentials):
|
||||
|
||||
```
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: openebs-target-pod-failure
|
||||
data:
|
||||
parameters.yml: |
|
||||
blocksize: 4k
|
||||
blockcount: 1024
|
||||
testfile: exampleFile
|
||||
```
|
||||
|
||||
```
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: openebs-target-pod-failure
|
||||
data:
|
||||
parameters.yml: |
|
||||
blocksize: 4k
|
||||
blockcount: 1024
|
||||
testfile: exampleFile
|
||||
```
|
||||
- Ensure that the chaosServiceAccount used for the experiment has cluster-scope permissions as the experiment may involve carrying out the chaos in the `openebs` namespace
|
||||
while performing application health checks in its respective namespace.
|
||||
|
||||
|
|
@ -96,7 +100,8 @@ Use this sample RBAC manifest to create a chaosServiceAccount in the desired (ap
|
|||
|
||||
#### Sample Rbac Manifest
|
||||
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/openebs/openebs-target-pod-failure/rbac.yaml yaml)
|
||||
[embedmd]: # "https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/openebs/openebs-target-pod-failure/rbac.yaml yaml"
|
||||
|
||||
```yaml
|
||||
---
|
||||
apiVersion: v1
|
||||
|
|
@ -115,12 +120,28 @@ metadata:
|
|||
labels:
|
||||
name: target-pod-failure-sa
|
||||
rules:
|
||||
- apiGroups: ["","apps","litmuschaos.io","batch","extensions","storage.k8s.io"]
|
||||
resources: ["pods","jobs","pods/log","deployments","pods/exec","events","chaosexperiments","chaosresults","chaosengines","configmaps","secrets","services","persistentvolumeclaims","storageclasses","persistentvolumes"]
|
||||
verbs: ["create","list","get","patch","update","delete"]
|
||||
- apiGroups: [""]
|
||||
resources: ["nodes"]
|
||||
verbs: ["get","list"]
|
||||
- apiGroups:
|
||||
["", "apps", "litmuschaos.io", "batch", "extensions", "storage.k8s.io"]
|
||||
resources:
|
||||
[
|
||||
"pods",
|
||||
"jobs",
|
||||
"deployments",
|
||||
"pods/exec",
|
||||
"chaosexperiments",
|
||||
"chaosresults",
|
||||
"chaosengines",
|
||||
"configmaps",
|
||||
"secrets",
|
||||
"services",
|
||||
"persistentvolumeclaims",
|
||||
"storageclasses",
|
||||
"persistentvolumes",
|
||||
]
|
||||
verbs: ["create", "list", "get", "patch", "update", "delete"]
|
||||
- apiGroups: [""]
|
||||
resources: ["nodes"]
|
||||
verbs: ["get", "list"]
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
kind: ClusterRoleBinding
|
||||
|
|
@ -133,10 +154,9 @@ roleRef:
|
|||
kind: ClusterRole
|
||||
name: target-pod-failure-sa
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: target-pod-failure-sa
|
||||
namespace: default
|
||||
|
||||
- kind: ServiceAccount
|
||||
name: target-pod-failure-sa
|
||||
namespace: default
|
||||
```
|
||||
|
||||
### Prepare ChaosEngine
|
||||
|
|
@ -182,7 +202,8 @@ subjects:
|
|||
|
||||
#### Sample ChaosEngine Manifest
|
||||
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/openebs/openebs-target-pod-failure/engine.yaml yaml)
|
||||
[embedmd]: # "https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/openebs/openebs-target-pod-failure/engine.yaml yaml"
|
||||
|
||||
```yaml
|
||||
apiVersion: litmuschaos.io/v1alpha1
|
||||
kind: ChaosEngine
|
||||
|
|
@ -191,30 +212,30 @@ metadata:
|
|||
namespace: default
|
||||
spec:
|
||||
# It can be true/false
|
||||
annotationCheck: 'false'
|
||||
annotationCheck: "false"
|
||||
# It can be active/stop
|
||||
engineState: 'active'
|
||||
engineState: "active"
|
||||
#ex. values: ns1:name=percona,ns2:run=nginx
|
||||
auxiliaryAppInfo: ''
|
||||
auxiliaryAppInfo: ""
|
||||
appinfo:
|
||||
appns: 'default'
|
||||
applabel: 'app=nginx'
|
||||
appkind: 'deployment'
|
||||
appns: "default"
|
||||
applabel: "app=nginx"
|
||||
appkind: "deployment"
|
||||
chaosServiceAccount: target-pod-failure-sa
|
||||
monitoring: false
|
||||
# It can be delete/retain
|
||||
jobCleanUpPolicy: 'delete'
|
||||
jobCleanUpPolicy: "delete"
|
||||
experiments:
|
||||
- name: openebs-target-pod-failure
|
||||
spec:
|
||||
components:
|
||||
env:
|
||||
- name: FORCE
|
||||
value: 'true'
|
||||
value: "true"
|
||||
- name: APP_PVC
|
||||
value: 'demo-nginx-claim'
|
||||
value: "demo-nginx-claim"
|
||||
- name: DEPLOY_TYPE
|
||||
value: 'deployment'
|
||||
value: "deployment"
|
||||
```
|
||||
|
||||
### Create the ChaosEngine Resource
|
||||
|
|
@ -232,7 +253,7 @@ spec:
|
|||
### Check Chaos Experiment Result
|
||||
|
||||
- Check whether the application is resilient to the target container kill, once the experiment (job) is completed. The ChaosResult resource naming convention
|
||||
is: `<ChaosEngine-Name>-<ChaosExperiment-Name>`.
|
||||
is: `{"<ChaosEngine-Name>-<ChaosExperiment-Name>"}`.
|
||||
|
||||
`kubectl describe chaosresult target-chaos-openebs-target-pod-failure -n <application-namespace>`
|
||||
|
||||
|
|
@ -0,0 +1,40 @@
|
|||
---
|
||||
id: plugins
|
||||
title: Using other chaos libraries as plugins
|
||||
sidebar_label: Plugins
|
||||
original_id: plugins
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
Litmus provides a way to use any chaos library or a tool to inject chaos. The chaos tool to be compatible with Litmus should satisfy the following requirements:
|
||||
|
||||
- Should be available as a Docker Image
|
||||
- Should take configuration through a `config-map`
|
||||
|
||||
The `plugins` or `chaos-libraries` host the core logic to inject chaos.
|
||||
|
||||
These plugins are hosted at https://github.com/litmuschaos/litmus-ansible/tree/master/chaoslib
|
||||
|
||||
Litmus project has integration into the following chaos-libraries.
|
||||
|
||||
| Chaos Library | Logo | Experiments covered |
|
||||
| ------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------- |
|
||||
| <a href="https://github.com/litmuschaos/litmus-ansible" target="_blank">Litmus</a> | <img src="https://camo.githubusercontent.com/953211f24c1c246f7017703f67b9779e4589bf76/68747470733a2f2f6c616e6473636170652e636e63662e696f2f6c6f676f732f6c69746d75732e737667" width="50"/> | Litmus native chaos libraries that encompasses the chaos capabilities for `pod-kill`, `container-kill`, `cpu-hog` |
|
||||
| <a href="https://github.com/alexei-led/pumba" target="_blank">Pumba</a> | <img src="https://github.com/alexei-led/pumba/raw/master/docs/img/pumba_logo.png" width="50"/> | Pumba provides chaos capabilities for `network-delay` |
|
||||
| <a href="https://github.com/bloomberg/powerfulseal" target="_blank">PowerfulSeal</a> | <img src="https://github.com/bloomberg/powerfulseal/raw/master/media/powerful-seal.png" width="50"/> | PowerfulSeal provides chaos capabilities for `pod-kill` |
|
||||
| | | |
|
||||
|
||||
Usage of plugins is a configuration parameter inside the chaos experiment.
|
||||
|
||||
> Add an example snippet here.
|
||||
|
||||
<br/>
|
||||
|
||||
<br/>
|
||||
|
||||
<hr/>
|
||||
|
||||
<br/>
|
||||
|
||||
<br/>
|
||||
|
|
@ -1,10 +1,11 @@
|
|||
---
|
||||
id: version-1.1.0-pod-cpu-hog
|
||||
id: pod-cpu-hog
|
||||
title: Pod CPU Hog Details
|
||||
sidebar_label: Pod CPU Hog
|
||||
original_id: pod-cpu-hog
|
||||
---
|
||||
------
|
||||
|
||||
---
|
||||
|
||||
## Experiment Metadata
|
||||
|
||||
|
|
@ -41,7 +42,6 @@ original_id: pod-cpu-hog
|
|||
- It simulates conditions where app pods experience CPU spikes either due to expected/undesired processes thereby testing how the
|
||||
overall application stack behaves when this occurs.
|
||||
|
||||
|
||||
## Integrations
|
||||
|
||||
- Pod CPU can be effected using the chaos library: `litmus`
|
||||
|
|
@ -58,7 +58,8 @@ Use this sample RBAC manifest to create a chaosServiceAccount in the desired (ap
|
|||
|
||||
#### Sample Rbac Manifest
|
||||
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/generic/pod-cpu-hog/rbac.yaml yaml)
|
||||
[embedmd]: # "https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/generic/pod-cpu-hog/rbac.yaml yaml"
|
||||
|
||||
```yaml
|
||||
---
|
||||
apiVersion: v1
|
||||
|
|
@ -77,9 +78,10 @@ metadata:
|
|||
labels:
|
||||
name: pod-cpu-hog-sa
|
||||
rules:
|
||||
- apiGroups: ["","litmuschaos.io","batch"]
|
||||
resources: ["pods","jobs","chaosengines","chaosexperiments","chaosresults"]
|
||||
verbs: ["create","list","get","patch","update","delete"]
|
||||
- apiGroups: ["", "litmuschaos.io", "batch"]
|
||||
resources:
|
||||
["pods", "jobs", "chaosengines", "chaosexperiments", "chaosresults"]
|
||||
verbs: ["create", "list", "get", "patch", "update", "delete"]
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
kind: RoleBinding
|
||||
|
|
@ -93,9 +95,9 @@ roleRef:
|
|||
kind: Role
|
||||
name: pod-cpu-hog-sa
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: pod-cpu-hog-sa
|
||||
namespace: default
|
||||
- kind: ServiceAccount
|
||||
name: pod-cpu-hog-sa
|
||||
namespace: default
|
||||
```
|
||||
|
||||
### Prepare ChaosEngine
|
||||
|
|
@ -148,7 +150,8 @@ subjects:
|
|||
|
||||
#### Sample ChaosEngine Manifest
|
||||
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/generic/pod-cpu-hog/engine.yaml yaml)
|
||||
[embedmd]: # "https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/generic/pod-cpu-hog/engine.yaml yaml"
|
||||
|
||||
```yaml
|
||||
apiVersion: litmuschaos.io/v1alpha1
|
||||
kind: ChaosEngine
|
||||
|
|
@ -157,19 +160,19 @@ metadata:
|
|||
namespace: default
|
||||
spec:
|
||||
# It can be true/false
|
||||
annotationCheck: 'true'
|
||||
annotationCheck: "true"
|
||||
# It can be active/stop
|
||||
engineState: 'active'
|
||||
engineState: "active"
|
||||
#ex. values: ns1:name=percona,ns2:run=nginx
|
||||
auxiliaryAppInfo: ''
|
||||
auxiliaryAppInfo: ""
|
||||
appinfo:
|
||||
appns: 'default'
|
||||
applabel: 'app=nginx'
|
||||
appkind: 'deployment'
|
||||
appns: "default"
|
||||
applabel: "app=nginx"
|
||||
appkind: "deployment"
|
||||
chaosServiceAccount: pod-cpu-hog-sa
|
||||
monitoring: false
|
||||
# It can be delete/retain
|
||||
jobCleanUpPolicy: 'delete'
|
||||
jobCleanUpPolicy: "delete"
|
||||
experiments:
|
||||
- name: pod-cpu-hog
|
||||
spec:
|
||||
|
|
@ -178,16 +181,15 @@ spec:
|
|||
# Provide name of target container
|
||||
# where chaos has to be injected
|
||||
- name: TARGET_CONTAINER
|
||||
value: 'nginx'
|
||||
value: "nginx"
|
||||
|
||||
#number of cpu cores to be consumed
|
||||
#verify the resources the app has been launched with
|
||||
- name: CPU_CORES
|
||||
value: '1'
|
||||
value: "1"
|
||||
|
||||
- name: TOTAL_CHAOS_DURATION
|
||||
value: '60' # in seconds
|
||||
|
||||
value: "60" # in seconds
|
||||
```
|
||||
|
||||
### Create the ChaosEngine Resource
|
||||
|
|
@ -204,7 +206,7 @@ spec:
|
|||
|
||||
### Check Chaos Experiment Result
|
||||
|
||||
- Check whether the application stack is resilient to CPU spikes on the app replica, once the experiment (job) is completed. The ChaosResult resource name is derived like this: `<ChaosEngine-Name>-<ChaosExperiment-Name>`.
|
||||
- Check whether the application stack is resilient to CPU spikes on the app replica, once the experiment (job) is completed. The ChaosResult resource name is derived like this: `{"<ChaosEngine-Name>-<ChaosExperiment-Name>"}`.
|
||||
|
||||
`kubectl describe chaosresult nginx-chaos-pod-cpu-hog -n <application-namespace>`
|
||||
|
||||
|
|
@ -1,10 +1,11 @@
|
|||
---
|
||||
id: version-1.1.0-pod-delete
|
||||
id: pod-delete
|
||||
title: Pod Delete Experiment Details
|
||||
sidebar_label: Pod Delete
|
||||
original_id: pod-delete
|
||||
---
|
||||
------
|
||||
|
||||
---
|
||||
|
||||
## Experiment Metadata
|
||||
|
||||
|
|
@ -24,7 +25,7 @@ original_id: pod-delete
|
|||
## Prerequisites
|
||||
|
||||
- Ensure that the Litmus Chaos Operator is running by executing `kubectl get pods` in operator namespace (typically, `litmus`).If not, install from [here](https://docs.litmuschaos.io/docs/getstarted/#install-litmus)
|
||||
- Ensure that the `pod-delete` experiment resource is available in the cluster by executing `kubectl get chaosexperiments` in the desired namespace. If not, install from [here](https://hub.litmuschaos.io/charts/generic/experiments/pod-delete)
|
||||
- Ensure that the `pod-delete` experiment resource is available in the cluster by executing `kubectl get chaosexperiments` in the desired namespace. If not, install from [here](https://hub.litmuschaos.io/charts/generic/experiments/pod-delete)
|
||||
|
||||
## Entry Criteria
|
||||
|
||||
|
|
@ -57,7 +58,8 @@ original_id: pod-delete
|
|||
|
||||
#### Sample Rbac Manifest
|
||||
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/generic/pod-delete/rbac.yaml yaml)
|
||||
[embedmd]: # "https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/generic/pod-delete/rbac.yaml yaml"
|
||||
|
||||
```yaml
|
||||
---
|
||||
apiVersion: v1
|
||||
|
|
@ -76,12 +78,21 @@ metadata:
|
|||
labels:
|
||||
name: pod-delete-sa
|
||||
rules:
|
||||
- apiGroups: ["","litmuschaos.io","batch","apps"]
|
||||
resources: ["pods","deployments","jobs","configmaps","chaosengines","chaosexperiments","chaosresults"]
|
||||
verbs: ["create","list","get","patch","update","delete"]
|
||||
- apiGroups: [""]
|
||||
resources: ["nodes"]
|
||||
verbs : ["get","list"]
|
||||
- apiGroups: ["", "litmuschaos.io", "batch", "apps"]
|
||||
resources:
|
||||
[
|
||||
"pods",
|
||||
"deployments",
|
||||
"jobs",
|
||||
"configmaps",
|
||||
"chaosengines",
|
||||
"chaosexperiments",
|
||||
"chaosresults",
|
||||
]
|
||||
verbs: ["create", "list", "get", "patch", "update", "delete"]
|
||||
- apiGroups: [""]
|
||||
resources: ["nodes"]
|
||||
verbs: ["get", "list"]
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
kind: RoleBinding
|
||||
|
|
@ -95,10 +106,9 @@ roleRef:
|
|||
kind: Role
|
||||
name: pod-delete-sa
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: pod-delete-sa
|
||||
namespace: default
|
||||
|
||||
- kind: ServiceAccount
|
||||
name: pod-delete-sa
|
||||
namespace: default
|
||||
```
|
||||
|
||||
### Prepare ChaosEngine
|
||||
|
|
@ -155,7 +165,8 @@ subjects:
|
|||
|
||||
#### Sample ChaosEngine Manifest
|
||||
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/generic/pod-delete/engine.yaml yaml)
|
||||
[embedmd]: # "https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/generic/pod-delete/engine.yaml yaml"
|
||||
|
||||
```yaml
|
||||
apiVersion: litmuschaos.io/v1alpha1
|
||||
kind: ChaosEngine
|
||||
|
|
@ -164,19 +175,19 @@ metadata:
|
|||
namespace: default
|
||||
spec:
|
||||
appinfo:
|
||||
appns: 'default'
|
||||
applabel: 'app=nginx'
|
||||
appkind: 'deployment'
|
||||
appns: "default"
|
||||
applabel: "app=nginx"
|
||||
appkind: "deployment"
|
||||
# It can be true/false
|
||||
annotationCheck: 'true'
|
||||
annotationCheck: "true"
|
||||
# It can be active/stop
|
||||
engineState: 'active'
|
||||
engineState: "active"
|
||||
#ex. values: ns1:name=percona,ns2:run=nginx
|
||||
auxiliaryAppInfo: ''
|
||||
auxiliaryAppInfo: ""
|
||||
chaosServiceAccount: pod-delete-sa
|
||||
monitoring: false
|
||||
# It can be delete/retain
|
||||
jobCleanUpPolicy: 'delete'
|
||||
jobCleanUpPolicy: "delete"
|
||||
experiments:
|
||||
- name: pod-delete
|
||||
spec:
|
||||
|
|
@ -184,13 +195,13 @@ spec:
|
|||
env:
|
||||
# set chaos duration (in sec) as desired
|
||||
- name: TOTAL_CHAOS_DURATION
|
||||
value: '30'
|
||||
value: "30"
|
||||
# set chaos interval (in sec) as desired
|
||||
- name: CHAOS_INTERVAL
|
||||
value: '10'
|
||||
value: "10"
|
||||
# pod failures without '--force' & default terminationGracePeriodSeconds
|
||||
- name: FORCE
|
||||
value: 'false'
|
||||
value: "false"
|
||||
```
|
||||
|
||||
### Create the ChaosEngine Resource
|
||||
|
|
@ -207,7 +218,7 @@ spec:
|
|||
|
||||
### Check Chaos Experiment Result
|
||||
|
||||
- Check whether the application is resilient to the pod failure, once the experiment (job) is completed. The ChaosResult resource name is derived like this: `<ChaosEngine-Name>-<ChaosExperiment-Name>`.
|
||||
- Check whether the application is resilient to the pod failure, once the experiment (job) is completed. The ChaosResult resource name is derived like this: `{"<ChaosEngine-Name>-<ChaosExperiment-Name>"}`.
|
||||
|
||||
`kubectl describe chaosresult nginx-chaos-pod-delete -n <application-namespace>`
|
||||
|
||||
|
|
@ -1,10 +1,11 @@
|
|||
---
|
||||
id: version-1.1.0-pod-network-corruption
|
||||
id: pod-network-corruption
|
||||
title: Pod Network Corruption Experiment Details
|
||||
sidebar_label: Pod Network Corruption
|
||||
original_id: pod-network-corruption
|
||||
---
|
||||
------
|
||||
|
||||
---
|
||||
|
||||
## Experiment Metadata
|
||||
|
||||
|
|
@ -22,9 +23,10 @@ original_id: pod-network-corruption
|
|||
</table>
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Ensure that the Litmus Chaos Operator is running by executing `kubectl get pods` in operator namespace (typically, `litmus`). If not, install from [here](https://docs.litmuschaos.io/docs/getstarted/#install-litmus)
|
||||
- Ensure that the `pod-network-corruption` experiment resource is available in the cluster by `kubectl get chaosexperiments` command. If not, install from [here](https://hub.litmuschaos.io/charts/generic/experiments/pod-network-corruption)
|
||||
- Cluster must run docker container runtime
|
||||
- Cluster must run docker container runtime
|
||||
|
||||
<div class="danger">
|
||||
<strong>NOTE</strong>:
|
||||
|
|
@ -41,7 +43,7 @@ original_id: pod-network-corruption
|
|||
|
||||
## Details
|
||||
|
||||
- The application pod should be healthy once chaos is stopped. Service-requests should be served despite chaos.
|
||||
- The application pod should be healthy once chaos is stopped. Service-requests should be served despite chaos.
|
||||
- Injects packet corruption on the specified container by starting a traffic control (tc) process with netem rules to add egress packet corruption
|
||||
- Corruption is injected via pumba library with command pumba netem corruption by passing the relevant network interface, packet-corruption-percentage, chaos duration and regex filter for container name
|
||||
- Can test the application's resilience to lossy/flaky network
|
||||
|
|
@ -58,7 +60,8 @@ original_id: pod-network-corruption
|
|||
|
||||
#### Sample Rbac Manifest
|
||||
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/generic/pod-network-corruption/rbac.yaml yaml)
|
||||
[embedmd]: # "https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/generic/pod-network-corruption/rbac.yaml yaml"
|
||||
|
||||
```yaml
|
||||
---
|
||||
apiVersion: v1
|
||||
|
|
@ -77,9 +80,10 @@ metadata:
|
|||
labels:
|
||||
name: pod-network-corruption-sa
|
||||
rules:
|
||||
- apiGroups: ["","litmuschaos.io","batch"]
|
||||
resources: ["pods","jobs","chaosengines","chaosexperiments","chaosresults"]
|
||||
verbs: ["create","list","get","patch","update","delete"]
|
||||
- apiGroups: ["", "litmuschaos.io", "batch"]
|
||||
resources:
|
||||
["pods", "jobs", "chaosengines", "chaosexperiments", "chaosresults"]
|
||||
verbs: ["create", "list", "get", "patch", "update", "delete"]
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
kind: RoleBinding
|
||||
|
|
@ -93,9 +97,9 @@ roleRef:
|
|||
kind: Role
|
||||
name: pod-network-corruption-sa
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: pod-network-corruption-sa
|
||||
namespace: default
|
||||
- kind: ServiceAccount
|
||||
name: pod-network-corruption-sa
|
||||
namespace: default
|
||||
```
|
||||
|
||||
### Prepare ChaosEngine
|
||||
|
|
@ -158,44 +162,45 @@ subjects:
|
|||
|
||||
#### Sample ChaosEngine Manifest
|
||||
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/generic/pod-network-corruption/engine.yaml yaml)
|
||||
[embedmd]: # "https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/generic/pod-network-corruption/engine.yaml yaml"
|
||||
|
||||
```yaml
|
||||
apiVersion: litmuschaos.io/v1alpha1
|
||||
kind: ChaosEngine
|
||||
metadata:
|
||||
name: nginx-network-chaos
|
||||
name: nginx-network-chaos
|
||||
namespace: default
|
||||
spec:
|
||||
# It can be delete/retain
|
||||
jobCleanUpPolicy: 'delete'
|
||||
jobCleanUpPolicy: "delete"
|
||||
# It can be true/false
|
||||
annotationCheck: 'true'
|
||||
annotationCheck: "true"
|
||||
# It can be active/stop
|
||||
engineState: 'active'
|
||||
engineState: "active"
|
||||
#ex. values: ns1:name=percona,ns2:run=nginx
|
||||
auxiliaryAppInfo: ''
|
||||
auxiliaryAppInfo: ""
|
||||
monitoring: false
|
||||
appinfo:
|
||||
appns: 'default'
|
||||
appns: "default"
|
||||
# FYI, To see app label, apply kubectl get pods --show-labels
|
||||
applabel: 'app=nginx'
|
||||
appkind: 'deployment'
|
||||
applabel: "app=nginx"
|
||||
appkind: "deployment"
|
||||
chaosServiceAccount: pod-network-corruption-sa
|
||||
experiments:
|
||||
- name: pod-network-corruption
|
||||
spec:
|
||||
components:
|
||||
env:
|
||||
#Container name where chaos has to be injected
|
||||
#Container name where chaos has to be injected
|
||||
- name: TARGET_CONTAINER
|
||||
value: 'nginx'
|
||||
value: "nginx"
|
||||
|
||||
#Network interface inside target container
|
||||
- name: NETWORK_INTERFACE
|
||||
value: 'eth0'
|
||||
value: "eth0"
|
||||
|
||||
- name: TOTAL_CHAOS_DURATION
|
||||
value: '60' # in seconds
|
||||
value: "60" # in seconds
|
||||
```
|
||||
|
||||
### Create the ChaosEngine Resource
|
||||
|
|
@ -212,11 +217,10 @@ spec:
|
|||
|
||||
### Check Chaos Experiment Result
|
||||
|
||||
- Check whether the application is resilient to the Pod Network Packet Corruption, once the experiment (job) is completed. The ChaosResult resource name is derived like this: `<ChaosEngine-Name>-<ChaosExperiment-Name>`.
|
||||
- Check whether the application is resilient to the Pod Network Packet Corruption, once the experiment (job) is completed. The ChaosResult resource name is derived like this: `{"<ChaosEngine-Name>-<ChaosExperiment-Name>"}`.
|
||||
|
||||
`kubectl describe chaosresult <ChaosEngine-Name>-<ChaosExperiment-Name> -n <application-namespace>`
|
||||
|
||||
|
||||
## Application Pod Network Packet Corruption Demo
|
||||
|
||||
- A sample recording of this experiment execution is provided [here](https://youtu.be/kSiLrIaILvs).
|
||||
|
|
@ -1,10 +1,11 @@
|
|||
---
|
||||
id: version-1.1.0-pod-network-latency
|
||||
id: pod-network-latency
|
||||
title: Pod Network Latency Experiment Details
|
||||
sidebar_label: Pod Network Latency
|
||||
original_id: pod-network-latency
|
||||
---
|
||||
------
|
||||
|
||||
---
|
||||
|
||||
## Experiment Metadata
|
||||
|
||||
|
|
@ -22,10 +23,10 @@ original_id: pod-network-latency
|
|||
</table>
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Ensure that the Litmus Chaos Operator is running by executing `kubectl get pods` in operator namespace (typically, `litmus`). If not, install from [here](https://docs.litmuschaos.io/docs/getstarted/#install-litmus)
|
||||
- Ensure that the `pod-network-latency` experiment resource is available in the cluster by executing kubectl `get chaosexperiments` in the desired namespace. . If not, install from [here](https://hub.litmuschaos.io/charts/generic/experiments/pod-network-latency)
|
||||
|
||||
|
||||
<div class="danger">
|
||||
<strong>NOTE</strong>:
|
||||
Experiment is supported only on Docker Runtime. Support for containerd/CRIO runtimes will be added in subsequent releases.
|
||||
|
|
@ -41,10 +42,10 @@ original_id: pod-network-latency
|
|||
|
||||
## Details
|
||||
|
||||
- The application pod should be healthy once chaos is stopped. Service-requests should be served despite chaos.
|
||||
- The application pod should be healthy once chaos is stopped. Service-requests should be served despite chaos.
|
||||
- Causes flaky access to application replica by injecting network delay using pumba.
|
||||
- Injects latency on the specified container by starting a traffic control (tc) process with netem rules to add egress delays
|
||||
- Latency is injected via pumba library with command pumba netem delay by passing the relevant network interface, latency, chaos duration and regex filter for container name
|
||||
- Injects latency on the specified container by starting a traffic control (tc) process with netem rules to add egress delays
|
||||
- Latency is injected via pumba library with command pumba netem delay by passing the relevant network interface, latency, chaos duration and regex filter for container name
|
||||
- Can test the application's resilience to lossy/flaky network
|
||||
|
||||
## Steps to Execute the Chaos Experiment
|
||||
|
|
@ -59,7 +60,8 @@ original_id: pod-network-latency
|
|||
|
||||
#### Sample Rbac Manifest
|
||||
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/generic/pod-network-latency/rbac.yaml yaml)
|
||||
[embedmd]: # "https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/generic/pod-network-latency/rbac.yaml yaml"
|
||||
|
||||
```yaml
|
||||
---
|
||||
apiVersion: v1
|
||||
|
|
@ -78,9 +80,10 @@ metadata:
|
|||
labels:
|
||||
name: pod-network-latency-sa
|
||||
rules:
|
||||
- apiGroups: ["","litmuschaos.io","batch"]
|
||||
resources: ["pods","jobs","chaosengines","chaosexperiments","chaosresults"]
|
||||
verbs: ["create","list","get","patch","update","delete"]
|
||||
- apiGroups: ["", "litmuschaos.io", "batch"]
|
||||
resources:
|
||||
["pods", "jobs", "chaosengines", "chaosexperiments", "chaosresults"]
|
||||
verbs: ["create", "list", "get", "patch", "update", "delete"]
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
kind: RoleBinding
|
||||
|
|
@ -94,9 +97,9 @@ roleRef:
|
|||
kind: Role
|
||||
name: pod-network-latency-sa
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: pod-network-latency-sa
|
||||
namespace: default
|
||||
- kind: ServiceAccount
|
||||
name: pod-network-latency-sa
|
||||
namespace: default
|
||||
```
|
||||
|
||||
### Prepare ChaosEngine
|
||||
|
|
@ -159,28 +162,29 @@ subjects:
|
|||
|
||||
#### Sample ChaosEngine Manifest
|
||||
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/generic/pod-network-latency/engine.yaml yaml)
|
||||
[embedmd]: # "https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/generic/pod-network-latency/engine.yaml yaml"
|
||||
|
||||
```yaml
|
||||
apiVersion: litmuschaos.io/v1alpha1
|
||||
kind: ChaosEngine
|
||||
metadata:
|
||||
name: nginx-network-chaos
|
||||
name: nginx-network-chaos
|
||||
namespace: default
|
||||
spec:
|
||||
# It can be delete/retain
|
||||
jobCleanUpPolicy: 'delete'
|
||||
jobCleanUpPolicy: "delete"
|
||||
# It can be true/false
|
||||
annotationCheck: 'true'
|
||||
annotationCheck: "true"
|
||||
# It can be active/stop
|
||||
engineState: 'active'
|
||||
engineState: "active"
|
||||
#ex. values: ns1:name=percona,ns2:run=nginx
|
||||
auxiliaryAppInfo: ''
|
||||
auxiliaryAppInfo: ""
|
||||
monitoring: false
|
||||
appinfo:
|
||||
appns: 'default'
|
||||
appns: "default"
|
||||
# FYI, To see app label, apply kubectl get pods --show-labels
|
||||
applabel: 'app=nginx'
|
||||
appkind: 'deployment'
|
||||
applabel: "app=nginx"
|
||||
appkind: "deployment"
|
||||
chaosServiceAccount: pod-network-latency-sa
|
||||
experiments:
|
||||
- name: pod-network-latency
|
||||
|
|
@ -189,20 +193,20 @@ spec:
|
|||
env:
|
||||
#Container name where chaos has to be injected
|
||||
- name: TARGET_CONTAINER
|
||||
value: 'nginx'
|
||||
value: "nginx"
|
||||
|
||||
#Network interface inside target container
|
||||
- name: NETWORK_INTERFACE
|
||||
value: 'eth0'
|
||||
value: "eth0"
|
||||
|
||||
- name: LIB_IMAGE
|
||||
value: 'gaiaadm/pumba:0.6.5'
|
||||
value: "gaiaadm/pumba:0.6.5"
|
||||
|
||||
- name: NETWORK_LATENCY
|
||||
value: '60000'
|
||||
value: "60000"
|
||||
|
||||
- name: TOTAL_CHAOS_DURATION
|
||||
value: '60' # in seconds
|
||||
value: "60" # in seconds
|
||||
```
|
||||
|
||||
### Create the ChaosEngine Resource
|
||||
|
|
@ -219,11 +223,10 @@ spec:
|
|||
|
||||
### Check Chaos Experiment Result
|
||||
|
||||
- Check whether the application is resilient to the Pod Network Latency, once the experiment (job) is completed. The ChaosResult resource name is derived like this: `<ChaosEngine-Name>-<ChaosExperiment-Name>`.
|
||||
- Check whether the application is resilient to the Pod Network Latency, once the experiment (job) is completed. The ChaosResult resource name is derived like this: `{"<ChaosEngine-Name>-<ChaosExperiment-Name>"}`.
|
||||
|
||||
`kubectl describe chaosresult <ChaosEngine-Name>-<ChaosExperiment-Name> -n <application-namespace>`
|
||||
|
||||
|
||||
## Application Pod Network Latency Demo
|
||||
|
||||
- A sample recording of this experiment execution is provided [here](https://youtu.be/QsQZyXVCcCw).
|
||||
|
|
@ -1,10 +1,11 @@
|
|||
---
|
||||
id: version-1.1.0-pod-network-loss
|
||||
id: pod-network-loss
|
||||
title: Pod Network Loss Experiment Details
|
||||
sidebar_label: Pod Network Loss
|
||||
original_id: pod-network-loss
|
||||
---
|
||||
------
|
||||
|
||||
---
|
||||
|
||||
## Experiment Metadata
|
||||
|
||||
|
|
@ -24,7 +25,7 @@ original_id: pod-network-loss
|
|||
## Prerequisites
|
||||
|
||||
- Ensure that the Litmus Chaos Operator is running by executing `kubectl get pods` in operator namespace (typically, `litmus`). If not, install from [here](https://docs.litmuschaos.io/docs/getstarted/#install-litmus)
|
||||
- Ensure that the `pod-network-loss` experiment resource is available in the cluster by executing `kubectl get chaosexperiments` in the desired namespace. If not, install from [here](https://hub.litmuschaos.io/charts/generic/experiments/pod-network-loss)
|
||||
- Ensure that the `pod-network-loss` experiment resource is available in the cluster by executing `kubectl get chaosexperiments` in the desired namespace. If not, install from [here](https://hub.litmuschaos.io/charts/generic/experiments/pod-network-loss)
|
||||
<div class="danger">
|
||||
<strong>NOTE</strong>:
|
||||
Experiment is supported only on Docker Runtime. Support for containerd/CRIO runtimes will be added in subsequent releases.
|
||||
|
|
@ -41,7 +42,7 @@ original_id: pod-network-loss
|
|||
## Details
|
||||
|
||||
- Pod-network-loss injects chaos to disrupt network connectivity to kubernetes pods.
|
||||
- The application pod should be healthy once chaos is stopped. Service-requests should be served despite chaos.
|
||||
- The application pod should be healthy once chaos is stopped. Service-requests should be served despite chaos.
|
||||
- Causes loss of access to application replica by injecting packet loss using pumba
|
||||
|
||||
## Steps to Execute the Chaos Experiment
|
||||
|
|
@ -56,7 +57,8 @@ original_id: pod-network-loss
|
|||
|
||||
#### Sample Rbac Manifest
|
||||
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/generic/pod-network-loss/rbac.yaml yaml)
|
||||
[embedmd]: # "https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/generic/pod-network-loss/rbac.yaml yaml"
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
|
|
@ -74,9 +76,10 @@ metadata:
|
|||
labels:
|
||||
name: pod-network-loss-sa
|
||||
rules:
|
||||
- apiGroups: ["","litmuschaos.io","batch"]
|
||||
resources: ["pods","jobs","chaosengines","chaosexperiments","chaosresults"]
|
||||
verbs: ["create","list","get","patch","update","delete"]
|
||||
- apiGroups: ["", "litmuschaos.io", "batch"]
|
||||
resources:
|
||||
["pods", "jobs", "chaosengines", "chaosexperiments", "chaosresults"]
|
||||
verbs: ["create", "list", "get", "patch", "update", "delete"]
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
kind: RoleBinding
|
||||
|
|
@ -90,9 +93,9 @@ roleRef:
|
|||
kind: Role
|
||||
name: pod-network-loss-sa
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: pod-network-loss-sa
|
||||
namespace: default
|
||||
- kind: ServiceAccount
|
||||
name: pod-network-loss-sa
|
||||
namespace: default
|
||||
```
|
||||
|
||||
### Prepare ChaosEngine
|
||||
|
|
@ -155,7 +158,8 @@ subjects:
|
|||
|
||||
#### Sample ChaosEngine Manifest
|
||||
|
||||
[embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/generic/pod-network-loss/engine.yaml yaml)
|
||||
[embedmd]: # "https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/generic/pod-network-loss/engine.yaml yaml"
|
||||
|
||||
```yaml
|
||||
# chaosengine.yaml
|
||||
apiVersion: litmuschaos.io/v1alpha1
|
||||
|
|
@ -165,19 +169,19 @@ metadata:
|
|||
namespace: default
|
||||
spec:
|
||||
# It can be delete/retain
|
||||
jobCleanUpPolicy: 'delete'
|
||||
jobCleanUpPolicy: "delete"
|
||||
# It can be true/false
|
||||
annotationCheck: 'true'
|
||||
annotationCheck: "true"
|
||||
# It can be active/stop
|
||||
engineState: 'active'
|
||||
engineState: "active"
|
||||
#ex. values: ns1:name=percona,ns2:run=nginx
|
||||
auxiliaryAppInfo: ''
|
||||
auxiliaryAppInfo: ""
|
||||
monitoring: false
|
||||
appinfo:
|
||||
appns: 'default'
|
||||
appns: "default"
|
||||
# FYI, To see app label, apply kubectl get pods --show-labels
|
||||
applabel: 'app=nginx'
|
||||
appkind: 'deployment'
|
||||
applabel: "app=nginx"
|
||||
appkind: "deployment"
|
||||
chaosServiceAccount: pod-network-loss-sa
|
||||
experiments:
|
||||
- name: pod-network-loss
|
||||
|
|
@ -186,21 +190,20 @@ spec:
|
|||
env:
|
||||
#Container name where chaos has to be injected
|
||||
- name: TARGET_CONTAINER
|
||||
value: 'nginx'
|
||||
value: "nginx"
|
||||
|
||||
- name: LIB_IMAGE
|
||||
value: 'gaiaadm/pumba:0.6.5'
|
||||
value: "gaiaadm/pumba:0.6.5"
|
||||
|
||||
#Network interface inside target container
|
||||
- name: NETWORK_INTERFACE
|
||||
value: 'eth0'
|
||||
value: "eth0"
|
||||
|
||||
- name: NETWORK_PACKET_LOSS_PERCENTAGE
|
||||
value: '100'
|
||||
value: "100"
|
||||
|
||||
- name: TOTAL_CHAOS_DURATION
|
||||
value: '60' # in seconds
|
||||
|
||||
value: "60" # in seconds
|
||||
```
|
||||
|
||||
### Create the ChaosEngine Resource
|
||||
|
|
@ -217,11 +220,10 @@ spec:
|
|||
|
||||
### Check Chaos Experiment Result
|
||||
|
||||
- Check whether the application is resilient to the Pod Network Loss, once the experiment (job) is completed. The ChaosResult resource name is derived like this: `<ChaosEngine-Name>-<ChaosExperiment-Name>`.
|
||||
- Check whether the application is resilient to the Pod Network Loss, once the experiment (job) is completed. The ChaosResult resource name is derived like this: `{"<ChaosEngine-Name>-<ChaosExperiment-Name>"}`.
|
||||
|
||||
`kubectl describe chaosresult <ChaosEngine-Name>-<ChaosExperiment-Name> -n <application-namespace>`
|
||||
|
||||
|
||||
## Application Pod Network Loss Demo
|
||||
|
||||
- A sample recording of this experiment execution is provided [here](https://youtu.be/jqvYy-nWc_I).
|
||||
|
|
@ -0,0 +1,40 @@
|
|||
---
|
||||
id: resources
|
||||
title: Resources related to Chaos Engineering on Kubernetes
|
||||
sidebar_label: Resources
|
||||
original_id: resources
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
## Chaos Demos
|
||||
|
||||
### Getting Started
|
||||
|
||||
Use this video to learn how to get started with Litmus. You will learn how to install Litmus, how to inject a fault into your application using one of the experiments available at ChaosHub.
|
||||
|
||||
<a href="https://asciinema.org/a/G9TcXpgikLuGTBY7btIUNSuWN" target="_blank">
|
||||
|
||||
<img src={require("./assets/getstarted.svg").default} width="300"/>
|
||||
|
||||
</a>
|
||||
|
||||
<hr/>
|
||||
|
||||
## Reference Implementations
|
||||
|
||||
| Reference | Description |
|
||||
| ------------------ | ---------------------------------------------------------------------- |
|
||||
| https://openebs.ci | CNCF SandBox project uses Litmus chaos experiments in its CI pipelines |
|
||||
| | |
|
||||
| | |
|
||||
|
||||
<br/>
|
||||
|
||||
<br/>
|
||||
|
||||
<hr/>
|
||||
|
||||
<br/>
|
||||
|
||||
<br/>
|
||||
|
|
@ -1,12 +1,13 @@
|
|||
---
|
||||
id: architecture
|
||||
original_id: architecture
|
||||
title: Litmus Architecture
|
||||
sidebar_label: Architecture
|
||||
---
|
||||
|
||||
<hr/>
|
||||
|
||||
<img src={require('./assets/litmus-schematic.png').default} width="800" />
|
||||
<img src={require("./assets/litmus-schematic.png").default} width="800" />
|
||||
|
||||
**Chaos-Operator**
|
||||
|
||||
|
|
|
|||
|
|
@ -2,6 +2,7 @@
|
|||
id: chaosexperiment
|
||||
title: Constructing the ChaosExperiment
|
||||
sidebar_label: ChaosExperiment
|
||||
original_id: chaosexperiment
|
||||
---
|
||||
|
||||
---
|
||||
|
|
|
|||
|
|
@ -2,6 +2,7 @@
|
|||
id: chaoshub
|
||||
title: Using and contributing to ChaosHub
|
||||
sidebar_label: ChaosHub
|
||||
original_id: chaoshub
|
||||
---
|
||||
|
||||
---
|
||||
|
|
@ -73,6 +74,8 @@ Chaos experiments that inject chaos into the platform resources of Kubernetes ar
|
|||
|
||||
Following Platform Chaos experiments are available on ChaosHub
|
||||
|
||||
| Platform | Description | Chaos Experiments |
|
||||
| -------- | ------------------------------------------- | --------------------------------------------------------------------------- |
|
||||
| AWS | Amazon Web Services platform. Includes EKS. | [ec2-terminate](chaostoolkit-aws-ec2-terminate.md), [ebs-loss](ebs-loss.md) |
|
||||
| Platform | Description | Chaos Experiments |
|
||||
| -------- | ------------------------------------------- | ----------------- |
|
||||
| AWS | Amazon Web Services platform. Includes EKS. | None |
|
||||
| GCP | Google Cloud Platform. Includes GKE. | None |
|
||||
| Azure | Microsoft Azure platform. Includes AKS. | None |
|
||||
|
|
|
|||
|
|
@ -2,6 +2,7 @@
|
|||
id: chaosresult
|
||||
title: Constructing the ChaosResult
|
||||
sidebar_label: ChaosResult
|
||||
original_id: chaosresult
|
||||
---
|
||||
|
||||
---
|
||||
|
|
|
|||
|
|
@ -2,6 +2,7 @@
|
|||
id: chaosschedule
|
||||
title: Constructing the ChaosSchedule
|
||||
sidebar_label: ChaosSchedule (alpha)
|
||||
original_id: chaosschedule
|
||||
---
|
||||
|
||||
---
|
||||
|
|
@ -173,10 +174,7 @@ This section describes the fields in the ChaosSchedule spec and the possible val
|
|||
</tr>
|
||||
<tr>
|
||||
<th>Notes</th>
|
||||
<td> The <code>includedDays</code> in the spec specifies a (comma-separated) list
|
||||
of days of the week at which chaos is allowed to take place. {'{'}day_name{'}'} is to
|
||||
be specified with the first 3 letters of the name of day such as
|
||||
<code>Mon</code>, <code>Tue</code> etc.</td>
|
||||
<td>The <code>includedDays</code> in the spec specifies a (comma-separated) list of days of the week at which chaos is allowed to take place. {'{'}day_name{'}'} is to be specified with the first 3 letters of the name of day such as <code>Mon</code>, <code>Tue</code> etc.</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
|
|
@ -195,7 +193,7 @@ This section describes the fields in the ChaosSchedule spec and the possible val
|
|||
</tr>
|
||||
<tr>
|
||||
<th>Range</th>
|
||||
<td><i>{'{'}hour_number{'}'} will range from 0 to 23</i> (type: string)(pattern: {'{'}hour_number{'}'}-{'{'}hour_number{'}'}).</td>
|
||||
<td><i>{'{'}hour_number{'}'} will range from 0 to 23</i> (type: string)(pattern: {'{'}hour_number{'}'}-{'{'}hour_number{'}'}).</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th>Default</th>
|
||||
|
|
|
|||
|
|
@ -1,24 +1,24 @@
|
|||
---
|
||||
id: version-1.0.0-community
|
||||
id: community
|
||||
title: Join Litmus Community
|
||||
sidebar_label: Community
|
||||
original_id: community
|
||||
---
|
||||
------
|
||||
|
||||
---
|
||||
|
||||
Litmus community is a subset of the larger Kubernetes community. Have a question? Want to stay in touch with the happenings on Chaos Engineering on Kubernetes? Join `#litmus` channel on Kubernetes Slack.
|
||||
|
||||
<br><br>
|
||||
<br/><br/>
|
||||
|
||||
<a href="https://kubernetes.slack.com/messages/CNXNB0ZTN" target="_blank"><img src="/docs/assets/join-community.png" width="400"></a>
|
||||
<a href="https://kubernetes.slack.com/messages/CNXNB0ZTN" target="_blank"><img src="/docs/assets/join-community.png" width="400"/></a>
|
||||
|
||||
<br>
|
||||
<br/>
|
||||
|
||||
<br>
|
||||
<br/>
|
||||
|
||||
<hr>
|
||||
<hr/>
|
||||
|
||||
<br>
|
||||
|
||||
<br>
|
||||
<br/>
|
||||
|
||||
<br/>
|
||||
|
|
@ -2,6 +2,7 @@
|
|||
id: devguide
|
||||
title: Developer Guide for ChaosCharts
|
||||
sidebar_label: Developer Guide
|
||||
original_id: devguide
|
||||
---
|
||||
|
||||
---
|
||||
|
|
@ -12,15 +13,13 @@ Below are some key points to remember before understanding how to write a new ch
|
|||
|
||||
> ChaosCharts repository : https://github.com/litmuschaos/chaos-charts
|
||||
>
|
||||
> Litmus-Go repository : https://github.com/litmuschaos/litmus-go/tree/master/experiments
|
||||
> Litmusbooks repository : https://github.com/litmuschaos/litmus-ansible/tree/master/experiments
|
||||
>
|
||||
> Website rendering code repository: https://github.com/litmuschaos/charthub.litmuschaos.io
|
||||
|
||||
The experiments & chaos libraries are typically written in Go, though not mandatory. Ensure that
|
||||
The experiments & chaos libraries are typically written in Ansible, though not mandatory. Ensure that
|
||||
the experiments can be executed in a container & can read/update the litmuschaos custom resources. For example,
|
||||
if you are writing an experiment in Go, use this [clientset](https://github.com/litmuschaos/chaos-operator/tree/master/pkg/client).
|
||||
Litmus Experiment contains the logic of pre-checks, chaos-injection, litmus-probes, post-checks, and result-updates.
|
||||
Typically, these are accompanied by a Kubernetes job that can execute the respective experiment.
|
||||
if you are writing an experiment in Go, use this [clientset](https://github.com/litmuschaos/chaos-operator/tree/master/pkg/client)
|
||||
|
||||
<hr/>
|
||||
|
||||
|
|
@ -42,11 +41,22 @@ to their default values.
|
|||
|
||||
Here is an example chaos experiment CR for a [pod-delete](https://github.com/litmuschaos/chaos-charts/blob/master/charts/generic/pod-delete/experiment.yaml) experiment
|
||||
|
||||
### Litmus Book
|
||||
|
||||
Litmus book is an `ansible` playbook that encompasses the logic of pre-checks, chaos-injection, post-checks, and result-updates.
|
||||
Typically, these are accompanied by a Kubernetes job that can execute the respective playbook.
|
||||
|
||||
Here is an example of the litmus book for the [pod-delete](https://github.com/litmuschaos/litmus-ansible/tree/master/experiments/generic/pod_delete) experiment.
|
||||
|
||||
### Chaos functions
|
||||
|
||||
The `ansible` business logic inside Litmus books can make use of readily available chaos functions. The chaos functions are available as `task-files` which are wrapped in one of the chaos libraries. See [plugins](plugins.md) for more details.
|
||||
|
||||
<hr/>
|
||||
|
||||
## Developing a ChaosExperiment
|
||||
|
||||
A detailed how-to guide on developing chaos experiments is available [here](https://github.com/litmuschaos/litmus-go/tree/master/contribute/developer-guide)
|
||||
A detailed how-to guide on developing chaos experiments is available [here](https://github.com/litmuschaos/litmus-ansible/tree/master/contribute/developer_guide)
|
||||
|
||||
<br/>
|
||||
|
||||
|
|
|
|||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue