kubectl-rook-ceph

Provide common management and troubleshooting tools for the Rook Ceph storage provider as a Krew plugin.

Install

Note: This requires kubectl krew to be installed.

To install the plugin, run:

kubectl krew install rook-ceph

To check plugin version kubectl krew list this will list all krew plugin with their current version.

Update

kubectl krew upgrade rook-ceph

Usage

kubectl rook-ceph <root-args> <command> <command-args>

Root args

These are args currently supported:

1. -h|--help: this will print brief command help text.

   kubectl rook-ceph --help
   

2. -n|--namespace='rook-ceph': the Kubernetes namespace in which the CephCluster resides. (optional, default: rook-ceph)

   kubectl rook-ceph -o test-operator -n test-cluster rook version
   

3. -o|--operator-namespace : the Kubernetes namespace in which the rook operator resides, when the arg -n is passed but -o is not then -o will equal to the -n. (default: rook-ceph)

   kubectl rook-ceph -o test-operator -n test-cluster rook version
   

4. --context: the name of the Kubernetes context to be used (optional).

   kubectl rook-ceph --context=$(kubectl config current-context) mons
   

Commands

• ceph <args> : Run a Ceph CLI command. Supports any arguments the ceph command supports. See Ceph docs for more.

• ceph daemon <daemon.id> <args>: Run a Ceph daemon command by connecting to its admin socket.

• rados <args> : Run a Rados CLI command. Supports any arguments the rados command supports. See Rados docs for more.

• radosgw-admin <args> : Run an RGW CLI command. Supports any arguments the radosgw-admin command supports. See the radosgw-admin docs for more.

• rbd <args> : Call a 'rbd' CLI command with arbitrary args

• mons : Print mon endpoints

  • restore-quorum <mon-name> : Restore the mon quorum based on a single healthy mon since quorum was lost with the other mons

• health : check health of the cluster and common configuration issues

• operator

  • restart : Restart the Rook-Ceph operator
  • set <property> <value> : Set the property in the rook-ceph-operator-config configmap.

• rook

  • version : Print the version of Rook
  • status : Print the phase and/or conditions of every CR in the namespace
  • status all : Print the phase and conditions of all CRs
  • status <CR> : Print the phase and conditions of CRs of a specific type, such as cephobjectstore, cephfilesystem, etc
  • purge-osd <osd-id> [--force] : Permanently remove an OSD from the cluster. Multiple OSDs can be removed with a comma-separated list of IDs.

• maintenance : Perform maintenance operations on mons or OSDs. The mon or OSD deployment will be scaled down and replaced temporarily by a maintenance deployment.

  • start <deployment-name> [--alternate-image <alternate-image>] : Start a maintenance deployment with an optional alternative ceph container image
  • stop <deployment-name> : Stop the maintenance deployment and restore the mon or OSD deployment

• dr :

  • health [ceph status args]: Print the ceph status of a peer cluster in a mirroring-enabled environment thereby validating connectivity between ceph clusters. Ceph status args can be optionally passed, such as to change the log level: --debug-ms 1.

• restore-deleted <CRD> [CRName]: Restore the ceph resources which are stuck in deleting state due to underlying resources being present in the cluster

• help : Output help text

Documentation

Visit docs below for complete details about each command and their flags uses.

1. Running ceph commands
2. Running ceph daemon commands
3. Running rbd commands
4. Get mon endpoints
5. Get cluster health status
6. Update configmap rook-ceph-operator-config
7. Restart operator pod
8. Get rook version
9. Get all CR status
10. Get cephCluster CR status
11. Get specific CR status
12. To purge OSD
13. Perform maintenance for OSDs and Mons
14. Restore mon quorum
15. Disaster Recovery
16. Restore deleted CRs
17. Destroy cluster
18. Running rados commands

Examples

Run a Ceph Command

Any ceph command can be run with the plugin. This example gets the ceph status:

kubectl rook-ceph ceph status

 cluster:
   id:     a1ac6554-4cc8-4c3b-a8a3-f17f5ec6f529
   health: HEALTH_OK

 services:
   mon: 3 daemons, quorum a,b,c (age 11m)
   mgr: a(active, since 10m)
   mds: 1/1 daemons up, 1 hot standby
   osd: 3 osds: 3 up (since 10m), 3 in (since 8d)

 data:
   volumes: 1/1 healthy
   pools:   6 pools, 137 pgs
   objects: 34 objects, 4.1 KiB
   usage:   58 MiB used, 59 GiB / 59 GiB avail
   pgs:     137 active+clean

 io:
   client:   1.2 KiB/s rd, 2 op/s rd, 0 op/s wr

Restart the Operator

kubectl rook-ceph operator restart

deployment.apps/rook-ceph-operator restarted

Rook Version

kubectl rook-ceph rook version

rook: v1.8.0-alpha.0.267.g096dabfa6
go: go1.16.13

Ceph Versions

kubectl rook-ceph ceph versions

{
    "mon": {
        "ceph version 16.2.7 (dd0603118f56ab514f133c8d2e3adfc983942503) pacific (stable)": 1
    },
    "mgr": {
        "ceph version 16.2.7 (dd0603118f56ab514f133c8d2e3adfc983942503) pacific (stable)": 1
    },
    "osd": {
        "ceph version 16.2.7 (dd0603118f56ab514f133c8d2e3adfc983942503) pacific (stable)": 1
    },
    "mds": {},
    "overall": {
        "ceph version 16.2.7 (dd0603118f56ab514f133c8d2e3adfc983942503) pacific (stable)": 3
    }
}

Contributing

We welcome contributions. See the Rook Contributing Guide to get started.

Report a Bug

For filing bugs, suggesting improvements, or requesting new features, please open an issue.

Contact

Please use the following to reach members of the community:

• Slack: Join our slack channel
• Forums: rook-dev
• Twitter: @rook_io
• Email (general topics): cncf-rook-info@lists.cncf.io
• Email (security topics): cncf-rook-security@lists.cncf.io

Licensing

Rook is under the Apache 2.0 license.