karmada-io
/
karmada
mirror of https://github.com/karmada-io/karmada.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
karmada/docs/upgrading/v0.10-v1.0.md

225 lines
6.7 KiB
Markdown
Raw Blame History

# v0.10 to v1.0
Follow the [Regular Upgrading Process](./README.md).
## Upgrading Notable Changes
### Introduced `karmada-aggregated-apiserver` component
In the releases before v1.0.0, we are using CRD to extend the
[Cluster API](https://github.com/karmada-io/karmada/tree/24f586062e0cd7c9d8e6911e52ce399106f489aa/pkg/apis/cluster),
and starts v1.0.0 we use
[API Aggregation](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/apiserver-aggregation/)(AA) to
extend it.
Based on the above change, perform the following operations during the upgrade:
#### Step 1: Stop `karmada-apiserver`
You can stop `karmada-apiserver` by updating its replica to `0`.
#### Step 2: Remove Cluster CRD from ETCD
Remove the `Cluster CRD` from ETCD directly by running the following command.
```
etcdctl --cert="/etc/kubernetes/pki/etcd/karmada.crt" \
--key="/etc/kubernetes/pki/etcd/karmada.key" \
--cacert="/etc/kubernetes/pki/etcd/server-ca.crt" \
del /registry/apiextensions.k8s.io/customresourcedefinitions/clusters.cluster.karmada.io
```
> Note: This command only removed the `CRD` resource, all the `CR` (Cluster objects) not changed.
> That's the reason why we don't remove CRD by `karmada-apiserver`.
#### Step 3: Prepare the certificate for the `karmada-aggregated-apiserver`
To avoid [CA Reusage and Conflicts](https://kubernetes.io/docs/tasks/extend-kubernetes/configure-aggregation-layer/#ca-reusage-and-conflicts),
create CA signer and sign a certificate to enable the aggregation layer.
Update `karmada-cert-secret` secret in `karmada-system` namespace:
```diff
apiVersion: v1
kind: Secret
metadata:
name: karmada-cert-secret
namespace: karmada-system
type: Opaque
data:
...
+ front-proxy-ca.crt: |
+ {{front_proxy_ca_crt}}
+ front-proxy-client.crt: |
+ {{front_proxy_client_crt}}
+ front-proxy-client.key: |
+ {{front_proxy_client_key}}
```
Then update `karmada-apiserver` deployment's container command:
```diff
- - --proxy-client-cert-file=/etc/kubernetes/pki/karmada.crt
- - --proxy-client-key-file=/etc/kubernetes/pki/karmada.key
+ - --proxy-client-cert-file=/etc/kubernetes/pki/front-proxy-client.crt
+ - --proxy-client-key-file=/etc/kubernetes/pki/front-proxy-client.key
- - --requestheader-client-ca-file=/etc/kubernetes/pki/server-ca.crt
+ - --requestheader-client-ca-file=/etc/kubernetes/pki/front-proxy-ca.crt
```
After the update, restore the replicas of `karmada-apiserver` instances.
#### Step 4: Deploy `karmada-aggregated-apiserver`:
Deploy `karmada-aggregated-apiserver` instance to your `host cluster` by following manifests:
<details>
<summary>unfold me to see the yaml</summary>
```yaml
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: karmada-aggregated-apiserver
namespace: karmada-system
labels:
app: karmada-aggregated-apiserver
apiserver: "true"
spec:
selector:
matchLabels:
app: karmada-aggregated-apiserver
apiserver: "true"
replicas: 1
template:
metadata:
labels:
app: karmada-aggregated-apiserver
apiserver: "true"
spec:
automountServiceAccountToken: false
containers:
- name: karmada-aggregated-apiserver
image: swr.ap-southeast-1.myhuaweicloud.com/karmada/karmada-aggregated-apiserver:v1.0.0
imagePullPolicy: IfNotPresent
volumeMounts:
- name: k8s-certs
mountPath: /etc/kubernetes/pki
readOnly: true
- name: kubeconfig
subPath: kubeconfig
mountPath: /etc/kubeconfig
command:
- /bin/karmada-aggregated-apiserver
- --kubeconfig=/etc/kubeconfig
- --authentication-kubeconfig=/etc/kubeconfig
- --authorization-kubeconfig=/etc/kubeconfig
- --karmada-config=/etc/kubeconfig
- --etcd-servers=https://etcd-client.karmada-system.svc.cluster.local:2379
- --etcd-cafile=/etc/kubernetes/pki/server-ca.crt
- --etcd-certfile=/etc/kubernetes/pki/karmada.crt
- --etcd-keyfile=/etc/kubernetes/pki/karmada.key
- --tls-cert-file=/etc/kubernetes/pki/karmada.crt
- --tls-private-key-file=/etc/kubernetes/pki/karmada.key
- --audit-log-path=-
- --feature-gates=APIPriorityAndFairness=false
- --audit-log-maxage=0
- --audit-log-maxbackup=0
resources:
requests:
cpu: 100m
volumes:
- name: k8s-certs
secret:
secretName: karmada-cert-secret
- name: kubeconfig
secret:
secretName: kubeconfig
---
apiVersion: v1
kind: Service
metadata:
name: karmada-aggregated-apiserver
namespace: karmada-system
labels:
app: karmada-aggregated-apiserver
apiserver: "true"
spec:
ports:
- port: 443
protocol: TCP
targetPort: 443
selector:
app: karmada-aggregated-apiserver
```
</details>
Then, deploy `APIService` to `karmada-apiserver` by following manifests.
<details>
<summary>unfold me to see the yaml</summary>
```yaml
apiVersion: apiregistration.k8s.io/v1
kind: APIService
metadata:
name: v1alpha1.cluster.karmada.io
labels:
app: karmada-aggregated-apiserver
apiserver: "true"
spec:
insecureSkipTLSVerify: true
group: cluster.karmada.io
groupPriorityMinimum: 2000
service:
name: karmada-aggregated-apiserver
namespace: karmada-system
version: v1alpha1
versionPriority: 10
---
apiVersion: v1
kind: Service
metadata:
name: karmada-aggregated-apiserver
namespace: karmada-system
spec:
type: ExternalName
externalName: karmada-aggregated-apiserver.karmada-system.svc.cluster.local
```
</details>
#### Step 5: check clusters status
If everything goes well, you can see all your clusters just as before the upgrading.
```yaml
kubectl get clusters
```
### `karmada-agent` requires an extra `impersonate` verb
In order to proxy user's request, the `karmada-agent` now request an extra `impersonate` verb.
Please check the `ClusterRole` configuration or apply the following manifest.
```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: karmada-agent
rules:
- apiGroups: ['*']
resources: ['*']
verbs: ['*']
- nonResourceURLs: ['*']
verbs: ["get"]
```
### MCS feature now supports `Kubernetes v1.21+`
Since the `discovery.k8s.io/v1beta1` of `EndpointSlices` has been deprecated in favor of `discovery.k8s.io/v1`, in
[Kubernetes v1.21](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.21.md), Karmada adopt
this change at release v1.0.0.
Now the [MCS](https://github.com/karmada-io/karmada/blob/master/docs/multi-cluster-service.md) feature requires
member cluster version no less than v1.21.
Reference in New Issue View Git Blame Copy Permalink