265 lines
63 KiB
Markdown
265 lines
63 KiB
Markdown
# Karmada
|
|
|
|
Karmada (Kubernetes Armada) is a Kubernetes management system that enables you to run your cloud-native applications across multiple Kubernetes clusters and clouds, with no changes to your applications. By speaking Kubernetes-native APIs and providing advanced scheduling capabilities, Karmada enables truly open, multi-cloud Kubernetes.
|
|
|
|
Karmada aims to provide turnkey automation for multi-cluster application management in multi-cloud and hybrid cloud scenarios, with key features such as centralized multi-cloud management, high availability, failure recovery, and traffic scheduling.
|
|
|
|
## TL;DR
|
|
|
|
Switch to the `root` directory of the repo.
|
|
```console
|
|
$ helm install karmada -n karmada-system --create-namespace ./charts
|
|
```
|
|
|
|
## Prerequisites
|
|
|
|
- Kubernetes 1.16+
|
|
- helm v3+
|
|
|
|
## Installing the Chart
|
|
|
|
To install the chart with the release name `karmada` in namespace `karmada-system`:
|
|
|
|
Switch to the `root` directory of the repo.
|
|
```console
|
|
$ helm install karmada -n karmada-system --create-namespace ./charts
|
|
```
|
|
|
|
Get kubeconfig from the cluster:
|
|
|
|
```console
|
|
$ kubectl get secret -n karmada-system karmada-kubeconfig -o jsonpath={.data.kubeconfig} | base64 -d
|
|
```
|
|
|
|
> **Tip**: List all releases using `helm list`
|
|
|
|
For better scheduling effect, you can choose to install the `karmada-descheduler` component in the karmada controller plane. Link: [here](../docs/descheduler.md). The following is the specific installation guide.
|
|
|
|
Edited values.yaml for karmada-descheduler
|
|
```YAML
|
|
installMode: "component"
|
|
components: [
|
|
"descheduler"
|
|
]
|
|
```
|
|
Execute command (switch to the `root` directory of the repo, and sets the `current-context` in a kubeconfig file)
|
|
```console
|
|
$ kubectl config use-context host
|
|
$ helm install karmada-descheduler -n karmada-system ./charts
|
|
```
|
|
|
|
## Uninstalling the Chart
|
|
To uninstall/delete the `karmada` helm release in namespace `karmada-system`:
|
|
|
|
```console
|
|
$ helm uninstall karmada -n karmada-system
|
|
```
|
|
|
|
The command removes all the Kubernetes components associated with the chart and deletes the release.
|
|
> **Note**: There are some RBAC resources that are used by the `preJob` that can not be deleted by the `uninstall` command above. You might have to clean them manually with tools like `kubectl`. You can clean them by commands:
|
|
|
|
```console
|
|
$ kubectl delete sa/karmada-pre-job -nkarmada-system
|
|
$ kubectl delete clusterRole/karmada-pre-job
|
|
$ kubectl delete clusterRoleBinding/karmada-pre-job
|
|
$ kubectl delete ns karmada-system
|
|
```
|
|
|
|
## Example
|
|
### 1. Install agent
|
|
Edited values.yaml
|
|
```YAML
|
|
installMode: "agent"
|
|
agent:
|
|
clusterName: "member"
|
|
## kubeconfig of the karmada
|
|
kubeconfig:
|
|
caCrt: |
|
|
-----BEGIN CERTIFICATE-----
|
|
XXXXXXXXXXXXXXXXXXXXXXXXXXX
|
|
-----END CERTIFICATE-----
|
|
crt: |
|
|
-----BEGIN CERTIFICATE-----
|
|
XXXXXXXXXXXXXXXXXXXXXXXXXXX
|
|
-----END CERTIFICATE-----
|
|
key: |
|
|
-----BEGIN RSA PRIVATE KEY-----
|
|
XXXXXXXXXXXXXXXXXXXXXXXXXXX
|
|
-----END RSA PRIVATE KEY-----
|
|
server: "https://apiserver.karmada"
|
|
```
|
|
Execute command (switch to the `root` directory of the repo, and sets the `current-context` in a kubeconfig file)
|
|
```console
|
|
$ kubectl config use-context member
|
|
$ helm install karmada-agent -n karmada-system --create-namespace ./charts
|
|
```
|
|
### 2. Install component
|
|
Edited values.yaml for karmada-scheduler-estimator
|
|
```YAML
|
|
installMode: "component"
|
|
components: [
|
|
"schedulerEstimator"
|
|
]
|
|
schedulerEstimator:
|
|
clusterName: "member"
|
|
## kubeconfig of the member cluster
|
|
kubeconfig:
|
|
caCrt: |
|
|
-----BEGIN CERTIFICATE-----
|
|
XXXXXXXXXXXXXXXXXXXXXXXXXXX
|
|
-----END CERTIFICATE-----
|
|
crt: |
|
|
-----BEGIN CERTIFICATE-----
|
|
XXXXXXXXXXXXXXXXXXXXXXXXXXX
|
|
-----END CERTIFICATE-----
|
|
key: |
|
|
-----BEGIN RSA PRIVATE KEY-----
|
|
XXXXXXXXXXXXXXXXXXXXXXXXXXX
|
|
-----END RSA PRIVATE KEY-----
|
|
server: "https://apiserver.member"
|
|
```
|
|
Execute command (switch to the `root` directory of the repo, and sets the `current-context` in a kubeconfig file)
|
|
```console
|
|
$ kubectl config use-context host
|
|
$ helm install karmada-scheduler-estimator -n karmada-system ./charts
|
|
```
|
|
|
|
## Configuration
|
|
| Name | Description | Value |
|
|
|------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
|
| `installMode` | InstallMode "host", "agent" and "component" are provided, "host" means install karmada in the control-cluster, "agent" means install agent client in the member cluster, "component" means install selected components in the control-cluster | `"host"` |
|
|
| `clusterDomain` | Default cluster domain for karmada | `"cluster.local"` |
|
|
| `components` | Selected components list, selectable values: "schedulerEstimator" | `[]` |
|
|
| `certs.mode` | Mode "auto" and "custom" are provided, "auto" means auto generate certificate, "custom" means use user certificate | `"auto"` |
|
|
| `certs.auto.expiry` | Expiry of the certificate | `"43800h"` |
|
|
| `certs.auto.hosts` | Hosts of the certificate | `["kubernetes.default.svc","*.etcd.karmada-system.svc.cluster.local","*.karmada-system.svc.cluster.local","*.karmada-system.svc","localhost","127.0.0.1"]` |
|
|
| `certs.custom.caCrt` | CA CRT of the certificate | `""` |
|
|
| `certs.custom.crt` | CRT of the certificate | `""` |
|
|
| `certs.custom.key` | KEY of the certificate | `""` |
|
|
| `certs.custom.frontProxyCaCrt` | CA CRT of the front proxy certificate | `""` |
|
|
| `certs.custom.frontProxyCrt` | CRT of the front proxy certificate | `""` |
|
|
| `certs.custom.frontProxyKey` | KEY of the front proxy certificate | `""` |
|
|
| `etcd.mode` | Mode "external" and "internal" are provided, "external" means use external ectd, "internal" means install a etcd in the cluster | `"internal"` |
|
|
| `etcd.external.servers` | Servers of etcd | `""` |
|
|
| `etcd.external.registryPrefix` | Use to registry prefix of etcd | `"/registry/karmada"` |
|
|
| `etcd.external.certs.caCrt` | CA CRT of the etcd certificate | `""` |
|
|
| `etcd.external.certs.crt` | CRT of the etcd certificate | `""` |
|
|
| `etcd.external.certs.key` | KEY of the etcd certificate | `""` |
|
|
| `etcd.internal.replicaCount` | Target replicas of the etcd | `1` |
|
|
| `etcd.internal.image.repository` | Image of the etcd | `"k8s.gcr.io/etcd"` |
|
|
| `etcd.internal.image.pullPolicy` | Image pull policy of the etcd | `"IfNotPresent"` |
|
|
| `etcd.internal.image.tag` | Image tag of the etcd | `"3.4.13-0"` |
|
|
| `etcd.internal.storageType` | StorageType of the etcd, accepts "hostPath", "pvc" | `"hostPath"` |
|
|
| `etcd.internal.pvc.storageClass` | StorageClass of the etcd, takes effect when `etcd.internal.storageType` is "pvc" | `""` |
|
|
| `etcd.internal.pvc.size` | Storage size of the etcd, takes effect when `etcd.internal.storageType` is "pvc" | `""` |
|
|
| `etcd.internal.resources` | Resource quota of the etcd | `{}` |
|
|
| `agent.clusterName` | Name of the member cluster | `""` |
|
|
| `agent.kubeconfig.caCrt` | CA CRT of the karmada certificate | `""` |
|
|
| `agent.kubeconfig.crt` | CRT of the karmada certificate | `""` |
|
|
| `agent.kubeconfig.key` | KEY of the karmada certificate | `""` |
|
|
| `agent.kubeconfig.server` | API-server of the karmada | `""` |
|
|
| `agent.labels` | Labels of the agent deployment | `{"app": "karmada-agent"}` |
|
|
| `agent.replicaCount` | Target replicas of the agent | `1` |
|
|
| `agent.podLabels` | Labels of the agent pods | `{}` |
|
|
| `agent.podAnnotations` | Annotaions of the agent pods | `{}` |
|
|
| `agent.imagePullSecrets` | Image pull secret of the agent | `[]` |
|
|
| `agent.image.repository` | Image of the agent | `"swr.ap-southeast-1.myhuaweicloud.com/karmada/karmada-agent"` |
|
|
| `agent.image.tag` | Image tag of the agent | `"latest"` |
|
|
| `agent.image.pullPolicy` | Image pull policy of the agent | `"IfNotPresent"` |
|
|
| `agent.resources` | Resource quota of the agent | `{}` |
|
|
| `agent.nodeSelector` | Node selector of the agent | `{}` |
|
|
| `agent.affinity` | Affinity of the agent | `{}` |
|
|
| `agent.tolerations` | Tolerations of the agent | `[]` |
|
|
| `scheduler.labels` | Labels of the schedeler deployment | `{"app": "karmada-scheduler"}` |
|
|
| `scheduler.replicaCount` | Target replicas of the scheduler | `1` |
|
|
| `scheduler.podLabels` | Labels of the scheduler pods | `{}` |
|
|
| `scheduler.podAnnotations` | Annotaions of the scheduler pods | `{}` |
|
|
| `scheduler.imagePullSecrets` | Image pull secret of the scheduler | `[]` |
|
|
| `scheduler.image.repository` | Image of the scheduler | `"swr.ap-southeast-1.myhuaweicloud.com/karmada/karmada-scheduler"` |
|
|
| `scheduler.image.tag` | Image tag of the scheduler | `"latest"` |
|
|
| `scheduler.image.pullPolicy` | Image pull policy of the scheduler | `"IfNotPresent"` |
|
|
| `scheduler.resources` | Resource quota of the scheduler | `{}` |
|
|
| `scheduler.nodeSelector` | Node selector of the scheduler | `{}` |
|
|
| `scheduler.affinity` | Affinity of the scheduler | `{}` |
|
|
| `scheduler.tolerations` | Tolerations of the scheduler | `[]` |
|
|
| `webhook.labels` | Labels of the webhook deployment | `{"app": "karmada-webhook"}` |
|
|
| `webhook.replicaCount` | Target replicas of the webhook | `1` |
|
|
| `webhook.podLabels` | Labels of the webhook pods | `{}` |
|
|
| `webhook.podAnnotations` | Annotaions of the webhook pods | `{}` |
|
|
| `webhook.imagePullSecrets` | Image pull secret of the webhook | `[]` |
|
|
| `webhook.image.repository` | Image of the webhook | `"swr.ap-southeast-1.myhuaweicloud.com/karmada/karmada-webhook"` |
|
|
| `webhook.image.tag` | Image tag of the webhook | `"latest"` |
|
|
| `webhook.image.pullPolicy` | Image pull policy of the webhook | `"IfNotPresent"` |
|
|
| `webhook.resources` | Resource quota of the webhook | `{}` |
|
|
| `webhook.nodeSelector` | Node selector of the webhook | `{}` |
|
|
| `webhook.affinity` | Affinity of the webhook | `{}` |
|
|
| `webhook.tolerations` | Tolerations of the webhook | `[]` |
|
|
| `controllerManager.labels` | Labels of the karmada-controller-manager deployment | `{"app": "karmada-controller-manager"}` |
|
|
| `controllerManager.replicaCount` | Target replicas of the karmada-controller-manager | `1` |
|
|
| `controllerManager.podLabels` | Labels of the karmada-controller-manager pods | `{}` |
|
|
| `controllerManager.podAnnotations` | Annotaions of the karmada-controller-manager pods | `{}` |
|
|
| `controllerManager.imagePullSecrets` | Image pull secret of the karmada-controller-manager | `[]` |
|
|
| `controllerManager.image.repository` | Image of the karmada-controller-manager | `"swr.ap-southeast-1.myhuaweicloud.com/karmada/karmada-controller-manager"` |
|
|
| `controllerManager.image.tag` | Image tag of the karmada-controller-manager | `"latest"` |
|
|
| `controllerManager.image.pullPolicy` | Image pull policy of the karmada-controller-manager | `"IfNotPresent"` |
|
|
| `controllerManager.resources` | Resource quota of the karmada-controller-manager | `{}` |
|
|
| `controllerManager.nodeSelector` | Node selector of the karmada-controller-manager | `{}` |
|
|
| `controllerManager.affinity` | Affinity of the karmada-controller-manager | `{}` |
|
|
| `controllerManager.tolerations` | Tolerations of the karmada-controller-manager | `[]` |
|
|
| `apiServer.labels` | Labels of the karmada-apiserver deployment | `{"app": "karmada-apiserver"}` |
|
|
| `apiServer.replicaCount` | Target replicas of the karmada-apiserver | `1` |
|
|
| `apiServer.podLabels` | Labels of the karmada-apiserver pods | `{}` |
|
|
| `apiServer.podAnnotations` | Annotaions of the karmada-apiserver pods | `{}` |
|
|
| `apiServer.imagePullSecrets` | Image pull secret of the karmada-apiserver | `[]` |
|
|
| `apiServer.image.repository` | Image of the karmada-apiserver | `"k8s.gcr.io/kube-apiserver"` |
|
|
| `apiServer.image.tag` | Image tag of the karmada-apiserver | `"v1.19.1"` |
|
|
| `apiServer.image.pullPolicy` | Image pull policy of the karmada-apiserver | `"IfNotPresent"` |
|
|
| `apiServer.resources` | Resource quota of the karmada-apiserver | `{}` |
|
|
| `apiServer.hostNetwork` | Deploy karmada-apiserver with hostNetwork. If there are multiple karmadas in one cluster, you'd better set it to "false" | `"true"` |
|
|
| `apiServer.nodeSelector` | Node selector of the karmada-apiserver | `{}` |
|
|
| `apiServer.affinity` | Affinity of the karmada-apiserver | `{}` |
|
|
| `apiServer.tolerations` | Tolerations of the karmada-apiserver | `[]` |
|
|
| `apiServer.serviceType` | Service type of apiserver, accepts "ClusterIP", "NodePort", "LoadBalancer" | `"ClusterIP"` |
|
|
| `apiServer.nodePort` | Node port for apiserver, takes effect when `apiServer.serviceType` is "NodePort". If no port is specified, the nodePort will be automatically assigned. | `0` |
|
|
| `aggregatedApiServer.labels` | Labels of the karmada-aggregated-apiserver deployment | `{"app": "karmada-aggregated-apiserver"}` |
|
|
| `aggregatedApiServer.replicaCount` | Target replicas of the karmada-aggregated-apiserver | `1` |
|
|
| `aggregatedApiServer.podLabels` | Labels of the karmada-aggregated-apiserver pods | `{}` |
|
|
| `aggregatedApiServer.podAnnotations` | Annotaions of the karmada-aggregated-apiserver pods | `{}` |
|
|
| `aggregatedApiServer.imagePullSecrets` | Image pull secret of the karmada-aggregated-apiserver | `[]` |
|
|
| `aggregatedApiServer.image.repository` | Image of the karmada-aggregated-apiserver | `"swr.ap-southeast-1.myhuaweicloud.com/karmada/karmada-aggregated-apiserver"` |
|
|
| `aggregatedApiServer.image.tag` | Image tag of the karmada-aggregated-apiserver | `"latest"` |
|
|
| `aggregatedApiServer.image.pullPolicy` | Image pull policy of the karmada-aggregated-apiserver | `"IfNotPresent"` |
|
|
| `aggregatedApiServer.resources` | Resource quota of the karmada-aggregated-apiserver | `{requests: {cpu: 100m}}` |
|
|
| `aggregatedApiServer.nodeSelector` | Node selector of the karmada-aggregated-apiserver | `{}` |
|
|
| `aggregatedApiServer.affinity` | Affinity of the karmada-aggregated-apiserver | `{}` |
|
|
| `aggregatedApiServer.tolerations` | Tolerations of the karmada-aggregated-apiserver | `[]` |
|
|
| `kubeControllerManager.labels` | Labels of the kube-controller-manager deployment | `{"app": "kube-controller-manager"}` |
|
|
| `kubeControllerManager.replicaCount` | Target replicas of the kube-controller-manager | `1` |
|
|
| `kubeControllerManager.podLabels` | Labels of the kube-controller-manager pods | `{}` |
|
|
| `kubeControllerManager.podAnnotations` | Annotaions of the kube-controller-manager pods | `{}` |
|
|
| `kubeControllerManager.imagePullSecrets` | Image pull secret of the kube-controller-manager | `[]` |
|
|
| `kubeControllerManager.image.repository` | Image of the kube-controller-manager | `"k8s.gcr.io/kube-controller-manager"` |
|
|
| `kubeControllerManager.image.tag` | Image tag of the kube-controller-manager | `"v1.19.1"` |
|
|
| `kubeControllerManager.image.pullPolicy` | Image pull policy of the kube-controller-manager | `"IfNotPresent"` |
|
|
| `kubeControllerManager.resources` | Resource quota of the kube-controller-manager | `{}` |
|
|
| `kubeControllerManager.nodeSelector` | Node selector of the kube-controller-manager | `{}` |
|
|
| `kubeControllerManager.affinity` | Affinity of the kube-controller-manager | `{}` |
|
|
| `kubeControllerManager.tolerations` | Tolerations of the kube-controller-manager | `[]` |
|
|
| `schedulerEstimator.clusterName` | Name of the member cluster | `""` |
|
|
| `schedulerEstimator.kubeconfig.caCrt` | CA CRT of the certificate | `""` |
|
|
| `schedulerEstimator.kubeconfig.crt` | CRT of the certificate | `""` |
|
|
| `schedulerEstimator.kubeconfig.key` | KEY of the certificate | `""` |
|
|
| `schedulerEstimator.kubeconfig.server` | API-server of the member cluster | `""` |
|
|
| `schedulerEstimator.labels` | Labels of the scheduler-estimator deployment | `{}` |
|
|
| `schedulerEstimator.replicaCount` | Target replicas of the scheduler-estimator | `1` |
|
|
| `schedulerEstimator.podLabels` | Labels of the scheduler-estimator pods | `{}` |
|
|
| `schedulerEstimator.podAnnotations` | Annotaions of the scheduler-estimator pods | `{}` |
|
|
| `schedulerEstimator.imagePullSecrets` | Image pull secret of the scheduler-estimator | `[]` |
|
|
| `schedulerEstimator.image.repository` | Image of the scheduler-estimator | `"swr.ap-southeast-1.myhuaweicloud.com/karmada/karmada-scheduler-estimator"` |
|
|
| `schedulerEstimator.image.tag` | Image tag of the scheduler-estimator | `"latest"` |
|
|
| `schedulerEstimator.image.pullPolicy` | Image pull policy of the scheduler-estimator | `"IfNotPresent"` |
|
|
| `schedulerEstimator.resources` | Resource quota of the scheduler-estimator | `{}` |
|
|
| `schedulerEstimator.nodeSelector` | Node selector of the scheduler-estimator | `{}` |
|
|
| `schedulerEstimator.affinity` | Affinity of the scheduler-estimator | `{}` |
|
|
| `schedulerEstimator.tolerations` | Tolerations of the scheduler-estimator | `[]` |
|