From b7a4d76be5aab28f6add466bcb08900aaeed4e26 Mon Sep 17 00:00:00 2001 From: Indeed Date: Fri, 9 Jul 2021 08:10:40 -0700 Subject: [PATCH 1/2] match Leader Migration beta implementation. --- .../controller-manager-leader-migration.md | 22 +++++++++++++------ 1 file changed, 15 insertions(+), 7 deletions(-) diff --git a/content/en/docs/tasks/administer-cluster/controller-manager-leader-migration.md b/content/en/docs/tasks/administer-cluster/controller-manager-leader-migration.md index 8b141f9da0..3cb8ed3b08 100644 --- a/content/en/docs/tasks/administer-cluster/controller-manager-leader-migration.md +++ b/content/en/docs/tasks/administer-cluster/controller-manager-leader-migration.md @@ -10,7 +10,7 @@ content_type: task -{{< feature-state state="alpha" for_k8s_version="v1.21" >}} +{{< feature-state state="beta" for_k8s_version="v1.22" >}} {{< glossary_definition term_id="cloud-controller-manager" length="all" prepend="The cloud-controller-manager is">}} @@ -20,7 +20,7 @@ As part of the [cloud provider extraction effort](https://kubernetes.io/blog/201 Leader Migration provides a mechanism in which HA clusters can safely migrate "cloud specific" controllers between the `kube-controller-manager` and the `cloud-controller-manager` via a shared resource lock between the two components while upgrading the replicated control plane. For a single-node control plane, or if unavailability of controller managers can be tolerated during the upgrade, Leader Migration is not needed and this guide can be ignored. -Leader Migration is an alpha feature that is disabled by default and it requires `--enable-leader-migration` to be set on controller managers. It can be enabled by setting the feature gate `ControllerManagerLeaderMigration` plus `--enable-leader-migration` on `kube-controller-manager` or `cloud-controller-manager`. Leader Migration only applies during the upgrade and can be safely disabled or left enabled after the upgrade is complete. +Leader Migration can be enabled by setting `--enable-leader-migration` on `kube-controller-manager` or `cloud-controller-manager`. Leader Migration only applies during the upgrade and can be safely disabled or left enabled after the upgrade is complete. This guide walks you through the manual process of upgrading the control plane from `kube-controller-manager` with built-in cloud provider to running both `kube-controller-manager` and `cloud-controller-manager`. If you use a tool to administrator the cluster, please refer to the documentation of the tool and the cloud provider for more details. @@ -30,7 +30,7 @@ It is assumed that the control plane is running Kubernetes version N and to be u The control plane nodes should run `kube-controller-manager` with Leader Election enabled through `--leader-elect=true`. As of version N, an in-tree cloud privider must be set with `--cloud-provider` flag and `cloud-controller-manager` should not yet be deployed. -The out-of-tree cloud provider must have built a `cloud-controller-manager` with Leader Migration implmentation. If the cloud provider imports `k8s.io/cloud-provider` and `k8s.io/controller-manager` of version v0.21.0 or later, Leader Migration will be avaliable. +The out-of-tree cloud provider must have built a `cloud-controller-manager` with Leader Migration implmentation. If the cloud provider imports `k8s.io/cloud-provider` and `k8s.io/controller-manager` of version v0.21.0 or later, Leader Migration will be avaliable. However, for version before v0.22.0, Leader Migration is alpha and requires feature gate `ControllerManagerLeaderMigration` to be enabled. This guide assumes that kubelet of each control plane node starts `kube-controller-manager` and `cloud-controller-manager` as static pods defined by their manifests. If the components run in a different setting, please adjust the steps accordingly. @@ -52,11 +52,13 @@ Do the same to the `system::leader-locking-cloud-controller-manager` role. ### Initial Leader Migration configuration -Leader Migration requires a configuration file representing the state of controller-to-manager assignment. At this moment, with in-tree cloud provider, `kube-controller-manager` runs `route`, `service`, and `cloud-node-lifecycle`. The following example configuration shows the assignment. +Leader Migration optionally takes a configuration file representing the state of controller-to-manager assignment. At this moment, with in-tree cloud provider, `kube-controller-manager` runs `route`, `service`, and `cloud-node-lifecycle`. The following example configuration shows the assignment. + +Leader Migration can be enabled without a configuration. Please see [Default Configuration](#default-configuration) for details. ```yaml kind: LeaderMigrationConfiguration -apiVersion: controllermanager.config.k8s.io/v1alpha1 +apiVersion: controllermanager.config.k8s.io/v1beta1 leaderName: cloud-provider-extraction-migration resourceLock: leases controllerLeaders: @@ -70,7 +72,6 @@ controllerLeaders: On each control plane node, save the content to `/etc/leadermigration.conf`, and update the manifest of `kube-controller-manager` so that the file is mounted inside the container at the same location. Also, update the same manifest to add the following arguments: -- `--feature-gates=ControllerManagerLeaderMigration=true` to enable Leader Migration which is an alpha feature - `--enable-leader-migration` to enable Leader Migration on the controller manager - `--leader-migration-config=/etc/leadermigration.conf` to set configuration file @@ -82,7 +83,7 @@ In version N + 1, the desired state of controller-to-manager assignment can be r ```yaml kind: LeaderMigrationConfiguration -apiVersion: controllermanager.config.k8s.io/v1alpha1 +apiVersion: controllermanager.config.k8s.io/v1beta1 leaderName: cloud-provider-extraction-migration resourceLock: leases controllerLeaders: @@ -113,6 +114,13 @@ Now that the control plane has been upgraded to run both `kube-controller-manage In a rolling manager, update manifest of `cloud-controller-manager` to unset both `--enable-leader-migration` and `--leader-migration-config=` flag, also remove the mount of `/etc/leadermigration.conf`, and finally remove `/etc/leadermigration.conf`. To re-enable Leader Migration, recreate the configuration file and add its mount and the flags that enable Leader Migration back to `cloud-controller-manager`. +### Default Configuration + +Starting Kubernetes 1.22, Leader Migration provides a default configuration suitable for the default controller-to-manager assignment. +The default configuration can be enabled by setting `--enable-leader-migration` but without `--leader-migration-config=`. + +For `kube-controller-manager` and `cloud-controller-manager`, if there are no flags that enable any in-tree cloud provider or change ownership of controllers, the default configuration can be used to avoid manual creation of the configuration file. + ## {{% heading "whatsnext" %}} - Read the [Controller Manager Leader Migration](https://github.com/kubernetes/enhancements/tree/master/keps/sig-cloud-provider/2436-controller-manager-leader-migration) enhancement proposal From e4505847a6419f680066a2462554d3760eef532b Mon Sep 17 00:00:00 2001 From: Indeed Date: Mon, 19 Jul 2021 12:42:15 -0700 Subject: [PATCH 2/2] fix typo. --- .../controller-manager-leader-migration.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/content/en/docs/tasks/administer-cluster/controller-manager-leader-migration.md b/content/en/docs/tasks/administer-cluster/controller-manager-leader-migration.md index 3cb8ed3b08..ddfa8e592e 100644 --- a/content/en/docs/tasks/administer-cluster/controller-manager-leader-migration.md +++ b/content/en/docs/tasks/administer-cluster/controller-manager-leader-migration.md @@ -26,15 +26,15 @@ This guide walks you through the manual process of upgrading the control plane f ## {{% heading "prerequisites" %}} -It is assumed that the control plane is running Kubernetes version N and to be upgraded to version N + 1. Although it is possible to migrate within the same version, ideally the migration should be performed as part of a upgrade so that changes of configuration can be aligned to releases. The exact versions of N and N + 1 depend on each cloud provider. For example, if a cloud provider builds a `cloud-controller-manager` to work with Kubernetes 1.22, then N can be 1.21 and N + 1 can be 1.22. +It is assumed that the control plane is running Kubernetes version N and to be upgraded to version N + 1. Although it is possible to migrate within the same version, ideally the migration should be performed as part of an upgrade so that changes of configuration can be aligned to each release. The exact versions of N and N + 1 depend on each cloud provider. For example, if a cloud provider builds a `cloud-controller-manager` to work with Kubernetes 1.22, then N can be 1.21 and N + 1 can be 1.22. The control plane nodes should run `kube-controller-manager` with Leader Election enabled through `--leader-elect=true`. As of version N, an in-tree cloud privider must be set with `--cloud-provider` flag and `cloud-controller-manager` should not yet be deployed. -The out-of-tree cloud provider must have built a `cloud-controller-manager` with Leader Migration implmentation. If the cloud provider imports `k8s.io/cloud-provider` and `k8s.io/controller-manager` of version v0.21.0 or later, Leader Migration will be avaliable. However, for version before v0.22.0, Leader Migration is alpha and requires feature gate `ControllerManagerLeaderMigration` to be enabled. +The out-of-tree cloud provider must have built a `cloud-controller-manager` with Leader Migration implementation. If the cloud provider imports `k8s.io/cloud-provider` and `k8s.io/controller-manager` of version v0.21.0 or later, Leader Migration will be available. However, for version before v0.22.0, Leader Migration is alpha and requires feature gate `ControllerManagerLeaderMigration` to be enabled. This guide assumes that kubelet of each control plane node starts `kube-controller-manager` and `cloud-controller-manager` as static pods defined by their manifests. If the components run in a different setting, please adjust the steps accordingly. -For authorization, this guide assumes that the cluser uses RBAC. If another authorization mode grants permissions to `kube-controller-manager` and `cloud-controller-manager` components, please grant the needed access in a way that matches the mode. +For authorization, this guide assumes that the cluster uses RBAC. If another authorization mode grants permissions to `kube-controller-manager` and `cloud-controller-manager` components, please grant the needed access in a way that matches the mode.