litmus/gcp-vm-instance-stop.md at 2.2.0

10 KiB

Raw Permalink Blame History

Introduction

It causes power-off of a GCP VM instance by instance name or list of instance names before bringing it back to the running state after the specified chaos duration.
It helps to check the performance of the application/process running on the VM instance.
When the AUTO_SCALING_GROUP is enable then the experiment will not try to start the instance post chaos, instead it will check the addition of the new node instances to the cluster.

!!! tip "Scenario: stop the gcp vm"

Uses

??? info "View the uses of the experiment" coming soon

Prerequisites

??? info "Verify the prerequisites" - Ensure that Kubernetes Version > 1.16 - Ensure that the Litmus Chaos Operator is running by executing kubectl get pods in operator namespace (typically, litmus).If not, install from here - Ensure that the gcp-vm-instance-stop experiment resource is available in the cluster by executing kubectl get chaosexperiments in the desired namespace. If not, install from here - Ensure that you have sufficient GCP permissions to stop and start the GCP VM instances. - Ensure to create a Kubernetes secret having the GCP service account credentials in the default namespace. A sample secret file looks like:

    ```yaml
    apiVersion: v1
    kind: Secret
    metadata:
      name: cloud-secret
    type: Opaque
    stringData:
      type: 
      project_id: 
      private_key_id: 
      private_key: 
      client_email: 
      client_id: 
      auth_uri: 
      token_uri: 
      auth_provider_x509_cert_url: 
      client_x509_cert_url: 
    ```

Default Validations

??? info "View the default validations" - VM instance should be in healthy state.

Minimal RBAC configuration example (optional)

!!! tip "NOTE"
If you are using this experiment as part of a litmus workflow scheduled constructed & executed from chaos-center, then you may be making use of the litmus-admin RBAC, which is pre installed in the cluster as part of the agent setup.

??? note "View the Minimal RBAC permissions"

    [embedmd]:# (https://raw.githubusercontent.com/litmuschaos/chaos-charts/master/charts/gcp/gcp-vm-instance-stop/rbac.yaml yaml)
    ```yaml
    ---
    apiVersion: v1
    kind: ServiceAccount
    metadata:
      name: gcp-vm-instance-stop-sa
      namespace: default
      labels:
        name: gcp-vm-instance-stop-sa
        app.kubernetes.io/part-of: litmus
    ---
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRole
    metadata:
      name: gcp-vm-instance-stop-sa
      labels:
        name: gcp-vm-instance-stop-sa
        app.kubernetes.io/part-of: litmus
    rules:
    - apiGroups: [""]
      resources: ["pods","events","secrets"]
      verbs: ["create","list","get","patch","update","delete","deletecollection"]
    - apiGroups: [""]
      resources: ["pods/exec","pods/log"]
      verbs: ["create","list","get"]
    - apiGroups: ["batch"]
      resources: ["jobs"]
      verbs: ["create","list","get","delete","deletecollection"]
    - apiGroups: ["litmuschaos.io"]
      resources: ["chaosengines","chaosexperiments","chaosresults"]
      verbs: ["create","list","get","patch","update"]
    - apiGroups: [""]
      resources: ["nodes"]
      verbs: ["get","list"]
    ---
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRoleBinding
    metadata:
      name: gcp-vm-instance-stop-sa
      labels:
        name: gcp-vm-instance-stop-sa
        app.kubernetes.io/part-of: litmus
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: ClusterRole
      name: gcp-vm-instance-stop-sa
    subjects:
    - kind: ServiceAccount
      name: gcp-vm-instance-stop-sa
      namespace: default
    ```
    Use this sample RBAC manifest to create a chaosServiceAccount in the desired (app) namespace. This example consists of the minimum necessary role permissions to execute the experiment.

Experiment tunables

??? info "check the experiment tunables"

Mandatory Fields

<table>
  <tr>
    <th> Variables </th>
    <th> Description </th>
    <th> Notes </th>
  </tr>
  <tr> 
    <td> GCP_PROJECT_ID </td>
    <td> GCP project ID to which the VM instances belong </td>
    <td> All the VM instances must belong to a single GCP project </td>
  </tr>
  <tr> 
    <td> VM_INSTANCE_NAMES </td>
    <td> Name of target VM instances </td>
    <td> Multiple instance names can be provided as instance1,instance2,... </td>
  </tr>
  <tr>
    <td> INSTANCE_ZONES </td>
    <td> The zones of the target VM instances </td>
    <td> Zone for every instance name has to be provided as zone1,zone2,... in the same order of <code>VM_INSTANCE_NAMES</code> </td>
  </tr>
</table>

<h2>Optional Fields</h2>

<table>
  <tr>
    <th> Variables </th>
    <th> Description </th>
    <th> Notes </th>
  </tr>
  <tr> 
    <td> TOTAL_CHAOS_DURATION </td>
    <td> The total time duration for chaos insertion (sec) </td>
    <td> Defaults to 30s </td>
  </tr>
   <tr> 
    <td> CHAOS_INTERVAL </td>
    <td> The interval (in sec) between successive instance termination </td>
    <td> Defaults to 30s </td>
  </tr>  
  <tr> 
    <td> AUTO_SCALING_GROUP </td>
    <td> Set to <code>enable</code> if the target instance is the part of a auto-scaling group </td>
    <td> Defaults to <code>disable</code> </td>
  </tr>  
  <tr>
    <td> SEQUENCE </td>
    <td> It defines sequence of chaos execution for multiple instance </td>
    <td> Default value: parallel. Supported: serial, parallel </td>
  </tr> 
  <tr>
    <td> RAMP_TIME </td>
    <td> Period to wait before and after injection of chaos in sec </td>
    <td> </td>
  </tr>
</table>

Experiment Examples

Common Experiment Tunables

Refer the common attributes to tune the common tunables for all the experiments.

Target GCP Instances

It will stop all the instances with the given VM_INSTANCE_NAMES instance names and corresponding INSTANCE_ZONES zone names in GCP_PROJECT_ID project.

NOTE: The VM_INSTANCE_NAMES contains multiple comma-separated vm instances. The comma-separated zone names should be provided in the same order as instance names.

Use the following example to tune this:

## details of the gcp instance
apiVersion: litmuschaos.io/v1alpha1
kind: ChaosEngine
metadata:
  name: engine-nginx
spec:
  engineState: "active"
  annotationCheck: "false"
  chaosServiceAccount: gcp-vm-instance-stop-sa
  experiments:
  - name: gcp-vm-instance-stop
    spec:
      components:
        env:
        # comma separated list of vm instance names
        - name: VM_INSTANCE_NAMES
          value: 'instance-01,instance-02'
        # comma separated list of zone names corresponds to the VM_INSTANCE_NAMES
        # it should be provided in same order of VM_INSTANCE_NAMES
        - name: INSTANCE_ZONES
          value: 'zone-01,zone-02'
        # gcp project id to which vm instance belongs
        - name: GCP_PROJECT_ID
          value: 'project-id'
        - name: TOTAL_CHAOS_DURATION
          VALUE: '60'

Autoscaling NodeGroup

If vm instances belong to the autoscaling group then provide the AUTO_SCALING_GROUP as enable else provided it as disable. The default value of AUTO_SCALING_GROUP is disable.

Use the following example to tune this:

## scale up and down to maintain the available instance counts
apiVersion: litmuschaos.io/v1alpha1
kind: ChaosEngine
metadata:
  name: engine-nginx
spec:
  engineState: "active"
  annotationCheck: "false"
  chaosServiceAccount: gcp-vm-instance-stop-sa
  experiments:
  - name: gcp-vm-instance-stop
    spec:
      components:
        env:
        # tells if instances are part of autoscaling group
        # supports: enable, disable. default: disable
        - name: AUTO_SCALING_GROUP
          value: 'enable'
        # comma separated list of vm instance names
        - name: VM_INSTANCE_NAMES
          value: 'instance-01,instance-02'
        # comma separated list of zone names corresponds to the VM_INSTANCE_NAMES
        # it should be provided in same order of VM_INSTANCE_NAMES
        - name: INSTANCE_ZONES
          value: 'zone-01,zone-02'
        # gcp project id to which vm instance belongs
        - name: GCP_PROJECT_ID
          value: 'project-id'
        - name: TOTAL_CHAOS_DURATION
          VALUE: '60'

Mutiple Iterations Of Chaos

The multiple iterations of chaos can be tuned via setting CHAOS_INTERVAL ENV. Which defines the delay between each iteration of chaos.

Use the following example to tune this:

# defines delay between each successive iteration of the chaos
apiVersion: litmuschaos.io/v1alpha1
kind: ChaosEngine
metadata:
  name: engine-nginx
spec:
  engineState: "active"
  annotationCheck: "false"
  chaosServiceAccount: gcp-vm-instance-stop-sa
  experiments:
  - name: gcp-vm-instance-stop
    spec:
      components:
        env:
        # delay between each iteration of chaos
        - name: CHAOS_INTERVAL
          value: '15'
        # time duration for the chaos execution
        - name: TOTAL_CHAOS_DURATION
          VALUE: '60'
        - name: VM_INSTANCE_NAMES
          value: 'instance-01,instance-02'
        - name: INSTANCE_ZONES
          value: 'zone-01,zone-02'
        - name: GCP_PROJECT_ID
          value: 'project-id'

10 KiB Raw Permalink Blame History

Introduction

Uses

Prerequisites

Default Validations

Minimal RBAC configuration example (optional)

Experiment tunables

Mandatory Fields

Experiment Examples

Common Experiment Tunables

Target GCP Instances

Autoscaling NodeGroup

Mutiple Iterations Of Chaos

10 KiB

Raw Permalink Blame History