From 8bc10be32d7fcb2967f9c03a3102bef8bf557b23 Mon Sep 17 00:00:00 2001 From: guoyao <1015105054@qq.com> Date: Mon, 29 Nov 2021 16:00:15 +0800 Subject: [PATCH] add velero integration doc Signed-off-by: guoyao <1015105054@qq.com> --- docs/working-with-velero.md | 163 ++++++++++++++++++++++++++++++++++++ 1 file changed, 163 insertions(+) create mode 100644 docs/working-with-velero.md diff --git a/docs/working-with-velero.md b/docs/working-with-velero.md new file mode 100644 index 000000000..2a2e46f69 --- /dev/null +++ b/docs/working-with-velero.md @@ -0,0 +1,163 @@ +# Integrate Velero to back up and restore Karmada resources + +[Velero](https://github.com/vmware-tanzu/velero) gives you tools to back up and restore +your Kubernetes cluster resources and persistent volumes. You can run Velero with a public +cloud platform or on-premises. + +Velero lets you: + +- Take backups of your cluster and restore in case of loss. +- Migrate cluster resources to other clusters. +- Replicate your production cluster to development and testing clusters. + +This document gives an example to demonstrate how to use the `Velero` to back up and restore Kubernetes cluster resources +and persistent volumes. Following example backups resources in cluster `member1`, and then restores those to cluster `member2`. + +## Start up Karamda clusters +You just need to clone Karamda repo, and run the following script in Karamda directory. +```shell +hack/local-up-karmada.sh +``` + +And then run the below command to switch to the member cluster `member1`. +```shell +export KUBECONFIG=/root/.kube/members.config +kubectl config use-context member1 +``` + +## Install MinIO +Velero uses Object Storage Services from different cloud providers to support backup and snapshot operations. +For simplicity, here takes, one object storage that runs locally on k8s clusters, for an example. + +Download the binary from the official site: +```shell +wget https://dl.min.io/server/minio/release/linux-amd64/minio +chmod +x minio +``` + +Run the below command to set `MinIO` username and password: +```shell +export MINIO_ROOT_USER=minio +export MINIO_ROOT_PASSWORD=minio123 +``` + +Run this command to start `MinIO`: +```shell +./minio server /data --console-address="0.0.0.0:20001" --address="0.0.0.0:9000" +``` + +Replace `/data` with the path to the drive or directory in which you want `MinIO` to store data. And now we can visit +`http://{SERVER_EXTERNAL_IP}/20001` in the browser to visit `MinIO` console UI. And `Velero` can use +`http://{SERVER_EXTERNAL_IP}/9000` to connect `MinIO`. The two configuration will make our follow-up work easier and more convenient. + +Please visit `MinIO` console to create region `minio` and bucket `velero`, these will be used by `Velero`. + +For more details about how to install `MinIO`, please run `minio server --help` for help, or you can visit +[MinIO Github Repo](https://github.com/minio/minio). + +## Install Velero +Velero consists of two components: +- ### A command-line client that runs locally. + 1. Download the [release](https://github.com/vmware-tanzu/velero/releases) tarball for your client platform + ```shell + wget https://github.com/vmware-tanzu/velero/releases/download/v1.7.0/velero-v1.7.0-linux-amd64.tar.gz + ``` + + 2. Extract the tarball: + ```shell + tar -zxvf velero-v1.7.0-linux-amd64.tar.gz + ``` + + 3. Move the extracted velero binary to somewhere in your $PATH (/usr/local/bin for most users). + ```shell + cp velero-v1.7.0-linux-amd64/velero /usr/local/bin/ + ``` + +- ### A server that runs on your cluster + We will use `velero install` to set up server components. + + For more details about how to use `MinIO` and `Velero` to backup resources, please ref: https://velero.io/docs/v1.7/contributions/minio/ + + 1. Create a Velero-specific credentials file (credentials-velero) in your local directory: + ```shell + [default] + aws_access_key_id = minio + aws_secret_access_key = minio123 + ``` + The two values should keep the same with `MinIO` username and password that we set when we install `MinIO` + + 2. Start the server. + + We need to install `Velero` in both `member1` and `member2`, so we should run the below command in shell for both two clusters, + this will start Velero server. Please run `kubectl config use-context member1` and `kubectl config use-context member2` + to switch to the different member clusters: `member1` or `member2`. + ```shell + velero install \ + --provider aws \ + --plugins velero/velero-plugin-for-aws:v1.2.1 \ + --bucket velero \ + --secret-file ./credentials-velero \ + --use-volume-snapshots=false \ + --backup-location-config region=minio,s3ForcePathStyle="true",s3Url=http://{SERVER_EXTERNAL_IP}:9000 + ``` + Replace `{SERVER_EXTERNAL_IP}` with your own server external IP. + + 3. Deploy the nginx application to cluster `member1`: + + Run the below command in the Karmada directory. + ```shell + kubectl apply -f samples/nginx/deployment.yaml + ``` + + And then you will find nginx is deployed successfully. + ```shell + # kubectl get deployment.apps + NAME READY UP-TO-DATE AVAILABLE AGE + nginx 2/2 2 2 17s + ``` + +### Back up and restore Karmada resources +Create a backup in `member1`: +```shell +velero backup create nginx-backup --selector app=nginx +``` + +Restore the backup in `member2` + +Run this command to switch to `member2` +```shell +export KUBECONFIG=/root/.kube/members.config +kubectl config use-context member2 +``` + +In `member2`, we can also get the backup that we created in `member1`: +```shell +# velero backup get +NAME STATUS ERRORS WARNINGS CREATED EXPIRES STORAGE LOCATION SELECTOR +nginx-backup Completed 0 0 2021-12-10 15:16:46 +0800 CST 29d default app=nginx +``` + +Restore `member1` resources to `member2`: +```shell +# velero restore create --from-backup nginx-backup +Restore request "nginx-backup-20211210151807" submitted successfully. +``` + +Watch restore result, you'll find that the status is Completed. +```shell +# velero restore get +NAME BACKUP STATUS STARTED COMPLETED ERRORS WARNINGS CREATED SELECTOR +nginx-backup-20211210151807 nginx-backup Completed 2021-12-10 15:18:07 +0800 CST 2021-12-10 15:18:07 +0800 CST 0 0 2021-12-10 15:18:07 +0800 CST +``` + +And then you can find deployment nginx will be restored successfully. +```shell +# kubectl get deployment.apps/nginx +NAME READY UP-TO-DATE AVAILABLE AGE +nginx 2/2 2 2 21s +``` + +## Reference +The above introductions about `Velero` and `MinIO` are only a summary from the official website and repos, for more details please refer to: +- Velero: https://velero.io/ +- MinIO: https://min.io/