# Karmada

Karmada (Kubernetes Armada) is a Kubernetes management system that enables you to run your cloud-native applications across multiple Kubernetes clusters and clouds, with no changes to your applications. By speaking Kubernetes-native APIs and providing advanced scheduling capabilities, Karmada enables truly open, multi-cloud Kubernetes.

Karmada aims to provide turnkey automation for multi-cluster application management in multi-cloud and hybrid cloud scenarios, with key features such as centralized multi-cloud management, high availability, failure recovery, and traffic scheduling.

## TL;DR

Switch to the `root` directory of the repo.
```console
$ helm install karmada -n karmada-system --create-namespace ./charts
```

## Prerequisites

- Kubernetes 1.16+
- helm v3+

## Installing the Chart

To install the chart with the release name `karmada` in namespace `karmada-system`:

Switch to the `root` directory of the repo.
```console
$ helm install karmada -n karmada-system --create-namespace ./charts
```

Get kubeconfig from the cluster:

```console
$ kubectl get secret -n karmada-system karmada-kubeconfig -o jsonpath={.data.kubeconfig} | base64 -d
```

> **Tip**: List all releases using `helm list`

## Uninstalling the Chart
To uninstall/delete the `karmada` helm release in namespace `karmada-system`:

```console
$ helm uninstall karmada -n karmada-system
```

The command removes all the Kubernetes components associated with the chart and deletes the release.

## Example
### 1. Install agent
Edited values.yaml
```YAML
installMode: "agent"
agent:
  clusterName: "member"
  ## kubeconfig of the karmada
  kubeconfig:
    caCrt: |
      -----BEGIN CERTIFICATE-----
      XXXXXXXXXXXXXXXXXXXXXXXXXXX
      -----END CERTIFICATE-----
    crt: |
      -----BEGIN CERTIFICATE-----
      XXXXXXXXXXXXXXXXXXXXXXXXXXX
      -----END CERTIFICATE-----
    key: |
      -----BEGIN RSA PRIVATE KEY-----
      XXXXXXXXXXXXXXXXXXXXXXXXXXX
      -----END RSA PRIVATE KEY-----
    server: "https://apiserver.karmada"
```
Execute command (switch to the `root` directory of the repo, and sets the `current-context` in a kubeconfig file)
```console
$ kubectl config use-context member
$ helm install karmada-agent -n karmada-system --create-namespace ./charts
```
### 2. Install component
Edited values.yaml
```YAML
installMode: "component"
components: [
  "schedulerEstimator"
]
schedulerEstimator:
  clusterName: "member"
  ## kubeconfig of the member cluster
  kubeconfig:
    caCrt: |
      -----BEGIN CERTIFICATE-----
      XXXXXXXXXXXXXXXXXXXXXXXXXXX
      -----END CERTIFICATE-----
    crt: |
      -----BEGIN CERTIFICATE-----
      XXXXXXXXXXXXXXXXXXXXXXXXXXX
      -----END CERTIFICATE-----
    key: |
      -----BEGIN RSA PRIVATE KEY-----
      XXXXXXXXXXXXXXXXXXXXXXXXXXX
      -----END RSA PRIVATE KEY-----
    server: "https://apiserver.member"
```
Execute command (switch to the `root` directory of the repo, and sets the `current-context` in a kubeconfig file)
```console
$ kubectl config use-context host
$ helm install karmada-scheduler-estimator -n karmada-system ./charts
```
## Configuration
| Name                      | Description                                     | Value |
| ------------------------- | ----------------------------------------------- | ----- |
| `installMode` | InstallMode "host", "agent" and "component" are provided, "host" means install karmada in the control-cluster, "agent" means install agent client in the member cluster, "component" means install selected components in the control-cluster   | `"host"`|
| `clusterDomain` | Default cluster domain for karmada | `"cluster.local"`  |
| `components` | Selected components list, selectable values: "schedulerEstimator"  | `[]`  |
|`certs.mode`| Mode "auto" and "custom" are provided, "auto" means auto generate certificate, "custom" means use user certificate |`"auto"`|
|`certs.auto.expiry`| Expiry of the certificate |`"43800h"`|
|`certs.auto.hosts`| Hosts of the certificate |`["kubernetes.default.svc","*.etcd.karmada-system.svc.cluster.local","*.karmada-system.svc.cluster.local","*.karmada-system.svc","localhost","127.0.0.1"]`|
|`certs.custom.caCrt`|CA CRT of the certificate|`""`|
|`certs.custom.crt`|CRT of the certificate|`""`|
|`certs.custom.key`|KEY of the certificate|`""`|
|`certs.custom.frontProxyCaCrt`|CA CRT of the front proxy certificate|`""`|
|`certs.custom.frontProxyCrt`|CRT of the front proxy certificate|`""`|
|`certs.custom.frontProxyKey`|KEY of the front proxy certificate|`""`|
|`etcd.mode`| Mode "external" and "internal" are provided, "external" means use external ectd, "internal" means install a etcd in the cluster |`"internal"`|
|`etcd.external.servers`| Servers of etcd |`""`|
|`etcd.external.registryPrefix`| Use to registry prefix of etcd |`"/registry/karmada"`|
|`etcd.external.certs.caCrt`| CA CRT of the etcd certificate |`""`|
|`etcd.external.certs.crt`| CRT of the etcd certificate |`""`|
|`etcd.external.certs.key`| KEY of the etcd certificate |`""`|
|`etcd.internal.replicaCount`| Target replicas of the etcd |`1`|
|`etcd.internal.image.repository`| Image of the etcd |`"k8s.gcr.io/etcd"`|
|`etcd.internal.image.pullPolicy`| Image pull policy of the etcd |`"IfNotPresent"`|
|`etcd.internal.image.tag`| Image tag of the etcd |`"3.4.13-0"`|
|`etcd.internal.storageType`| StorageType of the etcd, accepts "hostPath", "pvc" |`"hostPath"`|
|`etcd.internal.pvc.storageClass`| StorageClass of the etcd, takes effect when `etcd.internal.storageType` is "pvc" |`""`|
|`etcd.internal.pvc.size`| Storage size of the etcd, takes effect when `etcd.internal.storageType` is "pvc" |`""`|
|`etcd.internal.resources`| Resource quota of the etcd |`{}`|
|`agent.clusterName`| Name of the member cluster |`""`|
|`agent.kubeconfig.caCrt`| CA CRT of the karmada certificate |`""`|
|`agent.kubeconfig.crt`| CRT of the karmada certificate |`""`|
|`agent.kubeconfig.key`| KEY of the karmada certificate |`""`|
|`agent.kubeconfig.server`| API-server of the karmada |`""`|
|`agent.labels`| Labels of the agent deployment |`{"app": "karmada-agent"}`|
|`agent.replicaCount`| Target replicas of the agent |`1`|
|`agent.podLabels`| Labels of the agent pods |`{}`|
|`agent.podAnnotations`| Annotaions of the agent pods |`{}`|
|`agent.imagePullSecrets`| Image pull secret of the agent |`[]`|
|`agent.image.repository`| Image of the agent |`"swr.ap-southeast-1.myhuaweicloud.com/karmada/karmada-agent"`|
|`agent.image.tag`| Image tag of the agent |`"latest"`|
|`agent.image.pullPolicy`| Image pull policy of the agent |`"IfNotPresent"`|
|`agent.resources`| Resource quota of the agent |`{}`|
|`agent.nodeSelector`| Node selector of the agent |`{}`|
|`agent.affinity`| Affinity of the agent |`{}`|
|`agent.tolerations`| Tolerations of the agent |`[]`|
|`scheduler.labels`| Labels of the schedeler deployment |`{"app": "karmada-scheduler"}`|
|`scheduler.replicaCount`| Target replicas of the scheduler |`1`|
|`scheduler.podLabels`| Labels of the scheduler pods |`{}`|
|`scheduler.podAnnotations`| Annotaions of the scheduler pods |`{}`|
|`scheduler.imagePullSecrets`| Image pull secret of the scheduler |`[]`|
|`scheduler.image.repository`| Image of the scheduler |`"swr.ap-southeast-1.myhuaweicloud.com/karmada/karmada-scheduler"`|
|`scheduler.image.tag`| Image tag of the scheduler |`"latest"`|
|`scheduler.image.pullPolicy`| Image pull policy of the scheduler |`"IfNotPresent"`|
|`scheduler.resources`| Resource quota of the scheduler |`{}`|
|`scheduler.nodeSelector`| Node selector of the scheduler |`{}`|
|`scheduler.affinity`| Affinity of the scheduler |`{}`|
|`scheduler.tolerations`| Tolerations of the scheduler |`[]`|
|`webhook.labels`| Labels of the webhook deployment |`{"app": "karmada-webhook"}`|
|`webhook.replicaCount`| Target replicas of the webhook |`1`|
|`webhook.podLabels`| Labels of the webhook pods |`{}`|
|`webhook.podAnnotations`| Annotaions of the webhook pods |`{}`|
|`webhook.imagePullSecrets`| Image pull secret of the webhook |`[]`|
|`webhook.image.repository`| Image of the webhook |`"swr.ap-southeast-1.myhuaweicloud.com/karmada/karmada-webhook"`|
|`webhook.image.tag`| Image tag of the webhook |`"latest"`|
|`webhook.image.pullPolicy`| Image pull policy of the webhook |`"IfNotPresent"`|
|`webhook.resources`| Resource quota of the webhook |`{}`|
|`webhook.nodeSelector`| Node selector of the webhook |`{}`|
|`webhook.affinity`| Affinity of the webhook |`{}`|
|`webhook.tolerations`| Tolerations of the webhook |`[]`|
|`controllerManager.labels`| Labels of the karmada-controller-manager deployment |`{"app": "karmada-controller-manager"}`|
|`controllerManager.replicaCount`| Target replicas of the karmada-controller-manager |`1`|
|`controllerManager.podLabels`| Labels of the karmada-controller-manager pods |`{}`|
|`controllerManager.podAnnotations`| Annotaions of the karmada-controller-manager pods |`{}`|
|`controllerManager.imagePullSecrets`| Image pull secret of the karmada-controller-manager |`[]`|
|`controllerManager.image.repository`| Image of the karmada-controller-manager |`"swr.ap-southeast-1.myhuaweicloud.com/karmada/karmada-controller-manager"`|
|`controllerManager.image.tag`| Image tag of the karmada-controller-manager |`"latest"`|
|`controllerManager.image.pullPolicy`| Image pull policy of the karmada-controller-manager |`"IfNotPresent"`|
|`controllerManager.resources`| Resource quota of the karmada-controller-manager |`{}`|
|`controllerManager.nodeSelector`| Node selector of the karmada-controller-manager |`{}`|
|`controllerManager.affinity`| Affinity of the karmada-controller-manager |`{}`|
|`controllerManager.tolerations`| Tolerations of the karmada-controller-manager |`[]`|
|`apiServer.labels`| Labels of the karmada-apiserver deployment |`{"app": "karmada-apiserver"}`|
|`apiServer.replicaCount`| Target replicas of the karmada-apiserver |`1`|
|`apiServer.podLabels`| Labels of the karmada-apiserver pods |`{}`|
|`apiServer.podAnnotations`| Annotaions of the karmada-apiserver pods |`{}`|
|`apiServer.imagePullSecrets`| Image pull secret of the karmada-apiserver |`[]`|
|`apiServer.image.repository`| Image of the karmada-apiserver |`"k8s.gcr.io/kube-apiserver"`|
|`apiServer.image.tag`| Image tag of the karmada-apiserver |`"v1.19.1"`|
|`apiServer.image.pullPolicy`| Image pull policy of the karmada-apiserver |`"IfNotPresent"`|
|`apiServer.resources`| Resource quota of the karmada-apiserver |`{}`|
|`apiServer.hostNetwork`| Deploy karmada-apiserver with hostNetwork. If there are multiple karmadas in one cluster, you'd better set it to "false" |`"true"`|
|`apiServer.nodeSelector`| Node selector of the karmada-apiserver |`{}`|
|`apiServer.affinity`| Affinity of the karmada-apiserver |`{}`|
|`apiServer.tolerations`| Tolerations of the karmada-apiserver |`[]`|
|`apiServer.serviceType`| Service type of apiserver, accepts "ClusterIP", "NodePort", "LoadBalancer" |`"ClusterIP"`|
|`apiServer.nodePort`| Node port for apiserver, takes effect when `apiServer.serviceType` is "NodePort". If no port is specified, the nodePort will be automatically assigned. |`0`|
|`aggregatedApiServer.labels`| Labels of the karmada-aggregated-apiserver deployment |`{"app": "karmada-aggregated-apiserver"}`|
|`aggregatedApiServer.replicaCount`| Target replicas of the karmada-aggregated-apiserver |`1`|
|`aggregatedApiServer.podLabels`| Labels of the karmada-aggregated-apiserver pods |`{}`|
|`aggregatedApiServer.podAnnotations`| Annotaions of the karmada-aggregated-apiserver pods |`{}`|
|`aggregatedApiServer.imagePullSecrets`| Image pull secret of the karmada-aggregated-apiserver |`[]`|
|`aggregatedApiServer.image.repository`| Image of the karmada-aggregated-apiserver |`"swr.ap-southeast-1.myhuaweicloud.com/karmada/karmada-aggregated-apiserver"`|
|`aggregatedApiServer.image.tag`| Image tag of the karmada-aggregated-apiserver |`"latest"`|
|`aggregatedApiServer.image.pullPolicy`| Image pull policy of the karmada-aggregated-apiserver |`"IfNotPresent"`|
|`aggregatedApiServer.resources`| Resource quota of the karmada-aggregated-apiserver |`{requests: {cpu: 100m}}`|
|`aggregatedApiServer.nodeSelector`| Node selector of the karmada-aggregated-apiserver |`{}`|
|`aggregatedApiServer.affinity`| Affinity of the karmada-aggregated-apiserver |`{}`|
|`aggregatedApiServer.tolerations`| Tolerations of the karmada-aggregated-apiserver |`[]`|
|`kubeControllerManager.labels`| Labels of the kube-controller-manager deployment |`{"app": "kube-controller-manager"}`|
|`kubeControllerManager.replicaCount`| Target replicas of the kube-controller-manager |`1`|
|`kubeControllerManager.podLabels`| Labels of the kube-controller-manager pods |`{}`|
|`kubeControllerManager.podAnnotations`| Annotaions of the kube-controller-manager pods |`{}`|
|`kubeControllerManager.imagePullSecrets`| Image pull secret of the kube-controller-manager |`[]`|
|`kubeControllerManager.image.repository`| Image of the kube-controller-manager |`"k8s.gcr.io/kube-controller-manager"`|
|`kubeControllerManager.image.tag`| Image tag of the kube-controller-manager |`"v1.19.1"`|
|`kubeControllerManager.image.pullPolicy`| Image pull policy of the kube-controller-manager |`"IfNotPresent"`|
|`kubeControllerManager.resources`| Resource quota of the kube-controller-manager |`{}`|
|`kubeControllerManager.nodeSelector`| Node selector of the kube-controller-manager |`{}`|
|`kubeControllerManager.affinity`| Affinity of the kube-controller-manager |`{}`|
|`kubeControllerManager.tolerations`| Tolerations of the kube-controller-manager |`[]`|
|`schedulerEstimator.clusterName`| Name of the member cluster |`""`|
|`schedulerEstimator.kubeconfig.caCrt`| CA CRT of the certificate |`""`|
|`schedulerEstimator.kubeconfig.crt`| CRT of the certificate |`""`|
|`schedulerEstimator.kubeconfig.key`| KEY of the certificate |`""`|
|`schedulerEstimator.kubeconfig.server`| API-server of the member cluster |`""`|
|`schedulerEstimator.labels`| Labels of the scheduler-estimator deployment |`{}`|
|`schedulerEstimator.replicaCount`| Target replicas of the scheduler-estimator |`1`|
|`schedulerEstimator.podLabels`| Labels of the scheduler-estimator pods |`{}`|
|`schedulerEstimator.podAnnotations`| Annotaions of the scheduler-estimator pods |`{}`|
|`schedulerEstimator.imagePullSecrets`| Image pull secret of the scheduler-estimator |`[]`|
|`schedulerEstimator.image.repository`| Image of the scheduler-estimator |`"swr.ap-southeast-1.myhuaweicloud.com/karmada/karmada-scheduler-estimator"`|
|`schedulerEstimator.image.tag`| Image tag of the scheduler-estimator |`"latest"`|
|`schedulerEstimator.image.pullPolicy`| Image pull policy of the scheduler-estimator |`"IfNotPresent"`|
|`schedulerEstimator.resources`| Resource quota of the scheduler-estimator |`{}`|
|`schedulerEstimator.nodeSelector`| Node selector of the scheduler-estimator |`{}`|
|`schedulerEstimator.affinity`| Affinity of the scheduler-estimator |`{}`|
|`schedulerEstimator.tolerations`| Tolerations of the scheduler-estimator |`[]`|