Skip to content

EBS Loss By ID


  • It causes chaos to disrupt state of ebs volume by detaching it from the node/ec2 instance for a certain chaos duration using volume id.
  • In case of EBS persistent volumes, the volumes can get self-attached and experiment skips the re-attachment step. Tests deployment sanity (replica availability & uninterrupted service) and recovery workflows of the application pod.

Scenario: Detach EBS Volume

EBS Loss By ID


View the uses of the experiment

coming soon


Verify the prerequisites
  • Ensure that Kubernetes Version > 1.16
  • Ensure that the Litmus Chaos Operator is running by executing kubectl get pods in operator namespace (typically, litmus).If not, install from here
  • Ensure that the ebs-loss-by-id experiment resource is available in the cluster by executing kubectl get chaosexperiments in the desired namespace. If not, install from here
  • Ensure that you have sufficient AWS access to attach or detach an ebs volume for the instance.
  • Ensure to create a Kubernetes secret having the AWS access configuration(key) in the CHAOS_NAMESPACE. A sample secret file looks like:

    apiVersion: v1
    kind: Secret
      name: cloud-secret
    type: Opaque
      cloud_config.yml: |-
        # Add the cloud AWS credentials respectively
        aws_access_key_id = XXXXXXXXXXXXXXXXXXX
        aws_secret_access_key = XXXXXXXXXXXXXXX
  • If you change the secret key name (from cloud_config.yml) please also update the AWS_SHARED_CREDENTIALS_FILE ENV value on experiment.yamlwith the same name.

Default Validations

View the default validations
  • EBS volume is attached to the instance.

Minimal RBAC configuration example (optional)


If you are using this experiment as part of a litmus workflow scheduled constructed & executed from chaos-center, then you may be making use of the litmus-admin RBAC, which is pre installed in the cluster as part of the agent setup.

View the Minimal RBAC permissions
apiVersion: v1
kind: ServiceAccount
  name: ebs-loss-by-id-sa
  namespace: default
    name: ebs-loss-by-id-sa litmus
kind: ClusterRole
  name: ebs-loss-by-id-sa
    name: ebs-loss-by-id-sa litmus
  # Create and monitor the experiment & helper pods
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["create","delete","get","list","patch","update", "deletecollection"]
  # Performs CRUD operations on the events inside chaosengine and chaosresult
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["create","get","list","patch","update"]
  # Fetch configmaps & secrets details and mount it to the experiment pod (if specified)
  - apiGroups: [""]
    resources: ["secrets","configmaps"]
    verbs: ["get","list",]
  # Track and get the runner, experiment, and helper pods log 
  - apiGroups: [""]
    resources: ["pods/log"]
    verbs: ["get","list","watch"]  
  # for creating and managing to execute comands inside target container
  - apiGroups: [""]
    resources: ["pods/exec"]
    verbs: ["get","list","create"]
  # for configuring and monitor the experiment job by the chaos-runner pod
  - apiGroups: ["batch"]
    resources: ["jobs"]
    verbs: ["create","list","get","delete","deletecollection"]
  # for creation, status polling and deletion of litmus chaos resources used within a chaos workflow
  - apiGroups: [""]
    resources: ["chaosengines","chaosexperiments","chaosresults"]
    verbs: ["create","list","get","patch","update","delete"]
kind: ClusterRoleBinding
  name: ebs-loss-by-id-sa
    name: ebs-loss-by-id-sa litmus
  kind: ClusterRole
  name: ebs-loss-by-id-sa
- kind: ServiceAccount
  name: ebs-loss-by-id-sa
  namespace: default

Use this sample RBAC manifest to create a chaosServiceAccount in the desired (app) namespace. This example consists of the minimum necessary role permissions to execute the experiment.

Experiment tunables

check the experiment tunables

Mandatory Fields

Variables Description Notes
EBS_VOLUME_ID Comma separated list of volume IDs subjected to ebs detach chaos
REGION The region name for the target volumes

Optional Fields

Variables Description Notes
TOTAL_CHAOS_DURATION The time duration for chaos insertion (sec) Defaults to 30s
CHAOS_INTERVAL The time duration between the attachment and detachment of the volumes (sec) Defaults to 30s
SEQUENCE It defines sequence of chaos execution for multiple volumes Default value: parallel. Supported: serial, parallel
RAMP_TIME Period to wait before and after injection of chaos in sec

Experiment Examples

Common and AWS specific tunables

Refer the common attributes and AWS specific tunable to tune the common tunables for all experiments and aws specific tunables.

Detach Volumes By ID

It contains comma separated list of volume IDs subjected to ebs detach chaos. It can be tuned via EBS_VOLUME_ID ENV.

Use the following example to tune this:

# contains ebs volume id 
kind: ChaosEngine
  name: engine-nginx
  engineState: "active"
  annotationCheck: "false"
  chaosServiceAccount: ebs-loss-by-id-sa
  - name: ebs-loss-by-id
        # id of the ebs volume
        - name: EBS_VOLUME_ID
          value: 'ebs-vol-1'
        # region for the ebs volume
        - name: REGION
          value: '<region for EBS_VOLUME_ID>'
        - name: TOTAL_CHAOS_DURATION
          value: '60'