Add docs for env from metadata in-place update (#2)

Signed-off-by: FillZpp <FillZpp.pub@gmail.com>

docs/installation.md

@@ -98,7 +98,8 @@ Feature-gate controls some influential features in Kruise:
 | `TemplateNoDefaults` | Whether to disable defaults injection for pod/pvc template in workloads | `false` | Should not close this feature if it has open |
 | `PodUnavailableBudgetDeleteGate` | Enables PodUnavailableBudget for pod deletion, eviction              | `false` | No protection for pod deletion, eviction |
 | `PodUnavailableBudgetUpdateGate` | Enables PodUnavailableBudget for pod.Spec update              | `false` | No protection for in-place update |
-| `WorkloadSpread`                 | Enables WorkloadSpread to manage multi-domain and elastic deploy  | `false` | WorkloadSpread disabled | 
+| `WorkloadSpread`                 | Enables WorkloadSpread to manage multi-domain and elastic deploy  | `false` | WorkloadSpread disabled |
+| `InPlaceUpdateEnvFromMetadata`   | Enables Kruise to in-place update a container in Pod when its env from labels/annotations changed and pod is in-place updating | `false` | Only container image can be in-place update |
 
 If you want to configure the feature-gate, just set the parameter when install or upgrade. Such as:
 
@@ -116,6 +117,12 @@ If you are in China and have problem to pull image from official DockerHub, you
 $ helm install kruise https://... --set  manager.image.repository=openkruise-registry.cn-hangzhou.cr.aliyuncs.com/openkruise/kruise-manager
 ```
 
+## Best Practices
+
+### Installation parameters for k3s
+
+Usually k3s has the different runtime path from the default `/var/run`. So you have to set `daemon.socketLocation` to the real runtime socket path on your k3s node (e.g. `/run/k3s` or `/var/run/k3s/`).
+
 ## Uninstall
 
 Note that this will lead to all resources created by Kruise, including webhook configurations, services, namespace, CRDs, CR instances and Pods managed by Kruise controller, to be deleted!

docs/reference/inplace-update.md

@@ -0,0 +1,59 @@
+---
+title: InPlace Update
+---
+
+In-place Update is one of the key features provided by OpenKruise.
+
+Workloads that support in-place update:
+
+- [CloneSet](/docs/user-manuals/cloneset)
+- [Advanced StatefulSet](/docs/user-manuals/advancedstatefulset)
+- [Advanced DaemonSet](/docs/user-manuals/advanceddaemonset)
+- [SidecarSet](/docs/user-manuals/advanceddaemonset)
+
+Currently `CloneSet`, `Advanced StatefulSet` and `Advanced DaemonSet` re-use the same code package [`./pkg/util/inplaceupdate`](https://github.com/openkruise/kruise/tree/master/pkg/util/inplaceupdate) and have similar behaviours of in-place update. In this article, we would like to introduce the usage and workflow of them.
+
+Note that the in-place update workflow of `SidecarSet` is a little different from the other workloads, such as it will not set Pod to not-ready before update. So the things we talk below do not totally go for `SidecarSet`.
+
+## What is in-place update?
+
+Once we are going to update image in a existing Pod, look at the comparation between *Recreate* and *InPlace* Update:
+
+![alt](/img/docs/reference/inplace-update-comparation.png)
+
+In **ReCreate** way we have to delete the old Pod and create a new Pod:
+
+- Pod name and uid all changed, because they are totally different Pod objects (such as Deployment update)
+- Or Pod name may not change but uid changed, because they are still different Pod objects, althrough re-use the same name (such as StatefulSet update)
+- Node name of the Pod changed, because the new Pod is almost impossible to be scheduled to the previous node.
+- Pod IP changed, because the new Pod is almost impossible to be allocated the previous IP.
+
+But for **InPlace** way we can re-use the Pod object but only modify the fields in it, so that:
+
+- Avoid additional cost of scheduling, allocating IP, allocating and mounting volumes
+- Faster image pulling, because of we can re-use most of image layers pulled by the old image and only to pull several new layers
+- When a container is in-place updating, the other containers in Pod will not be affected and remain running.
+
+## Understand *InPlaceIfPossible*
+
+The update type in Kruise workloads is named `InPlaceIfPossible`, which tells Kruise to update Pods in-place as possible, and it should go back to ReCreate Update if impossible.
+
+What changes does it consider to be possilble to in-place update?
+
+1. Update `spec.template.metadata.*` in workloads, such as labels and annotations, Kruise will only update the metadata to existing Pods without recreate them.
+2. Update `spec.template.spec.containers[x].image` in workloads, Kruise will in-place update the container image in Pods without recreate them.
+3. **Since Kruise v1.0 (including v1.0 alpha/beta)**, update `spec.template.metadata.labels/annotations` and there exists container env from the changed labels/annotations, Kruise will in-place update them to renew the env value in containers.
+
+Otherwise, the changes to other fields such as `spec.template.spec.containers[x].env` or `spec.template.spec.containers[x].resources` will go back to ReCreate Update.
+
+## Workflow overview
+
+You can see the whole workflow of in-place update below (*you may need to right click and open it in a new tab*):
+
+![alt](/img/docs/reference/inplace-update-workflow.png)
+
+## Requirements
+
+To use InPlace Update for env from metadata, you have to enable `kruise-daemon` (*defaults to be enabled*) and `InPlaceUpdateEnvFromMetadata` feature-gate when install or upgrade Kruise chart. 
+
+Note that if you have some nodes of virtual-kubelet type, kruise-daemon may not work on them and in-place update for env from metadata will not be executed.

docs/user-manuals/advancedstatefulset.md

@@ -64,15 +64,12 @@ Advanced StatefulSet adds a `podUpdatePolicy` field in `spec.updateStrategy.roll
 which controls recreate or in-place update for Pods.
 
 - `ReCreate` controller will delete old Pods and create new ones. This is the same behavior as default StatefulSet.
-- `InPlaceIfPossible` controller will try to in-place update Pod instead of recreating them if possible. Currently, only `spec.template.metadata.*` and `spec.template.spec.containers[x].image` field can be in-place updated.
-- `InPlaceOnly` controller will in-place update Pod instead of recreating them. With `InPlaceOnly` policy, user cannot modify any fields in `spec.template` other than `spec.template.spec.containers[x].image`.
+- `InPlaceIfPossible` controller will try to in-place update Pod instead of recreating them if possible. Please ready the reference doc below.
+- `InPlaceOnly` controller will in-place update Pod instead of recreating them. With `InPlaceOnly` policy, user cannot modify any fields other than the fields that supported to in-place update.
 
-When a Pod being in-place update, controller will firstly update Pod status to make it become not-ready using readinessGate,
-and then update images in Pod spec to trigger Kubelet recreate the container on Node.
-However, sometimes Kubelet recreate containers so fast that other controllers such as endpoints-controller in kcm
-have not noticed the Pod has turned to not-ready. This may lead to requests damaged.
+**You may need to read the [reference doc](../reference/inplace-update) for more details of in-place update.**
 
-So we bring **graceful period** into in-place update. Advanced StatefulSet has supported `gracePeriodSeconds`, which is a period
+We also bring **graceful period** into in-place update. Advanced StatefulSet has supported `gracePeriodSeconds`, which is a period
 duration between controller update pod status and update pod images.
 So that endpoints-controller could have enough time to remove this Pod from endpoints.
 

docs/user-manuals/cloneset.md

@@ -174,20 +174,17 @@ Don't worry. Even if you enable the `CloneSetShortHash`, CloneSet will still rec
 
 ## Update features
 
-### In-place update
+### Update types
 
-CloneSet provides three update policies just like `AdvancedStatefulSet`, defaults to `ReCreate`.
+CloneSet provides three update types, defaults to `ReCreate`.
 
 - `ReCreate`: controller will delete old Pods and PVCs and create new ones.
-- `InPlaceIfPossible`: controller will try to in-place update Pod instead of recreating them if possible. Currently, only `spec.template.metadata.*` and `spec.template.spec.containers[x].image` field can be in-place updated.
-- `InPlaceOnly`: controller will in-place update Pod instead of recreating them. With `InPlaceOnly` policy, user cannot modify any fields in `spec.template` other than `spec.template.spec.containers[x].image`.
+- `InPlaceIfPossible`: controller will try to in-place update Pod instead of recreating them if possible. Please ready the reference doc below.
+- `InPlaceOnly`: controller will in-place update Pod instead of recreating them. With `InPlaceOnly` policy, user cannot modify any fields other than the fields that supported to in-place update.
 
-When a Pod being in-place update, controller will firstly update Pod status to make it become not-ready using readinessGate,
-and then update images in Pod spec to trigger Kubelet recreate the container on Node.
-However, sometimes Kubelet recreate containers so fast that other controllers such as endpoints-controller in kcm
-have not noticed the Pod has turned to not-ready. This may lead to requests damaged.
+**You may need to read the [reference doc](../reference/inplace-update) for more details of in-place update.**
 
-So we bring **graceful period** into in-place update. CloneSet has supported `gracePeriodSeconds`, which is a period
+We also bring **graceful period** into in-place update. CloneSet has supported `gracePeriodSeconds`, which is a period
 duration between controller update pod status and update pod images.
 So that endpoints-controller could have enough time to remove this Pod from endpoints.
 

i18n/zh/code.json

@@ -197,10 +197,10 @@
     "It only restarts the specific container with the new image and the Pod will not be recreated, which leads to much faster update process and much less side effects on other sub-systems such as scheduler, CNI or CSI.": {
       "message": "它只会用新的镜像重建 Pod 中的特定容器，整个 Pod 以及其中的其他容器都不会被影响。因此它带来了更快的发布速度，以及避免了对其他 Scheduler、CNI、CSI 等组件的负面影响。"
     },
-    "Bypass Application Management": {
+    "Decoupled Application Management": {
       "message": "应用的旁路管理"
     },
-    "OpenKruise provides several bypass ways to manage sidecar container, multi-domain deployment for applications, which means you can manage these things without modifying the Workloads of applications.": {
+    "OpenKruise provides several decoupled ways to manage sidecar container, multi-domain deployment for applications, which means you can manage these things without modifying the Workloads of applications.": {
       "message": "OpenKruise 提供了多种通过旁路管理应用 sidecar 容器、多区域部署的方式，“旁路” 意味着你可以不需要修改应用的 Workloads 来实现它们。"
     },
     "For example, SidecarSet can help you inject sidecar containers into all matching Pods during creation and even update them in-place with no effect on other containers in Pod.": {

i18n/zh/docusaurus-plugin-content-docs/current.json

@@ -30,6 +30,12 @@
   "sidebar.docs.category.Application Protection": {
     "message": "应用安全防护"
   },
+  "sidebar.docs.category.Reference": {
+    "message": "参考"
+  },
+  "sidebar.docs.category.Best Practices": {
+    "message": "最佳实践"
+  },
   "sidebar.docs.category.Developer Manuals": {
     "message": "开发者手册"
   }

i18n/zh/docusaurus-plugin-content-docs/current/installation.md

@@ -95,6 +95,7 @@ Feature-gate 控制了 Kruise 中一些有影响性的功能：
 | `PodUnavailableBudgetDeleteGate` | 启用 PodUnavailableBudget 保护 pod 删除、驱逐   | `false` | 不防护 pod 删除、驱逐 |
 | `PodUnavailableBudgetUpdateGate` | 启用 PodUnavailableBudget 保护 pod 原地升级   | `false` | 不防护 pod 原地升级 |
 | `WorkloadSpread`                 | 启用 WorkloadSpread 管理应用多分区弹性与拓扑部署 | `false` | 不支持 WorkloadSpread | 
+| `InPlaceUpdateEnvFromMetadata`   | 启用 Kruise 原地升级容器当它存在 env from 的 labels/annotations 发生了变化 | `false` | 容器中只有 image 能够原地升级 |
 
 如果你要配置 feature-gate，只要在安装或升级时配置参数即可，比如：
 
@@ -112,6 +113,12 @@ $ helm install kruise https://... --set featureGates="ResourcesDeletionProtectio
 $ helm install kruise https://... --set  manager.image.repository=openkruise-registry.cn-hangzhou.cr.aliyuncs.com/openkruise/kruise-manager
 ```
 
+## 最佳实践
+
+### k3s 安装参数
+
+通常来说 k3s 有着与默认 `/var/run` 不同的 runtime socket 路径。所以你需要将 `daemon.socketLocation` 参数设置为你的 k3s 节点上真实的路径（比如 `/run/k3s` 或 `/var/run/k3s/`）。
+
 ## 卸载
 
 注意：卸载会导致所有 Kruise 下的资源都会删除掉，包括 webhook configurations, services, namespace, CRDs, CR instances 以及所有 Kruise workload 下的 Pod。 请务必谨慎操作！

i18n/zh/docusaurus-plugin-content-docs/current/reference/inplace-update.md

@@ -0,0 +1,59 @@
+---
+title: 原地升级
+---
+
+原地升级是 OpenKruise 提供的核心功能之一。
+
+目前支持原地升级的 Workload：
+
+- [CloneSet](/docs/user-manuals/cloneset)
+- [Advanced StatefulSet](/docs/user-manuals/advancedstatefulset)
+- [Advanced DaemonSet](/docs/user-manuals/advanceddaemonset)
+- [SidecarSet](/docs/user-manuals/advanceddaemonset)
+
+目前 `CloneSet`、`Advanced StatefulSet`、`Advanced DaemonSet` 是复用的同一个代码包 [`./pkg/util/inplaceupdate`](https://github.com/openkruise/kruise/tree/master/pkg/util/inplaceupdate) 并且有类似的原地升级行为。在本文中，我们会介绍它的用法和工作流程。
+
+注意，`SidecarSet` 的原地升级流程和其他 workloads 不太一样，比如它在升级 Pod 之前并不会把 Pod 设置为 not-ready 状态。因此，下文中讨论的内容并不完全适用于 `SidecarSet`。
+
+## 什么是原地升级
+
+当我们要升级一个存量 Pod 中的镜像时，这是 *重建升级* 和 *原地升级* 的区别：
+
+![alt](/img/docs/reference/inplace-update-comparation.png)
+
+**重建升级**时我们要删除旧 Pod、创建新 Pod：
+
+- Pod 名字和 uid 发生变化，因为它们是完全不同的两个 Pod 对象（比如 Deployment 升级）
+- Pod 名字可能不变、但 uid 变化，因为它们是不同的 Pod 对象，只是复用了同一个名字（比如 StatefulSet 升级）
+- Pod 所在 Node 名字发生变化，因为新 Pod 很大可能性是不会调度到之前所在的 Node 节点的
+- Pod IP 发生变化，因为新 Pod 很大可能性是不会被分配到之前的 IP 地址的
+
+但是对于**原地升级**，我们仍然复用同一个 Pod 对象，只是修改它里面的字段。因此：
+
+- 可以避免如 *调度*、*分配 IP*、*分配、挂载盘* 等额外的操作和代价
+- 更快的镜像拉取，因为开源复用已有旧镜像的大部分 layer 层，只需要拉取新镜像变化的一些 layer
+- 当一个容器在原地升级时，Pod 中的其他容器不会受到影响，仍然维持运行
+
+## 理解 *InPlaceIfPossible*
+
+这种 Kruise workload 的升级类型名为 `InPlaceIfPossible`，它意味着 Kruise 会尽量对 Pod 采取原地升级，如果不能则退化到重建升级。
+
+以下的改动会被允许执行原地升级：
+
+1. 更新 workload 中的 `spec.template.metadata.*`，比如 labels/annotations，Kruise 只会将 metadata 中的改动更新到存量 Pod 上。
+2. 更新 workload 中的 `spec.template.spec.containers[x].image`，Kruise 会原地升级 Pod 中这些容器的镜像，而不会重建整个 Pod。
+3. **从 Kruise v1.0 版本开始（包括 v1.0 alpha/beta）**，更新 `spec.template.metadata.labels/annotations` 并且 container 中有配置 env from 这些改动的 labels/anntations，Kruise 会原地升级这些容器来生效新的 env 值。
+
+否则，其他字段的改动，比如 `spec.template.spec.containers[x].env` 或 `spec.template.spec.containers[x].resources`，都是会回退为重建升级。
+
+## 工作流程总览
+
+可以在下图中看到原地升级的整体工作流程（*你可能需要右击在新标签页中打开*）：
+
+![alt](/img/docs/reference/inplace-update-workflow.png)
+
+## 使用要求
+
+如果要使用 env from metadata 原地升级能力，你需要在安装或升级 Kruise chart 的时候打开 `kruise-daemon`（默认打开）和 `InPlaceUpdateEnvFromMetadata` 两个 feature-gate。
+
+注意，如果你有一些 virtual-kubelet 类型的 Node 节点，kruise-daemon 可能是无法在上面运行的，因此也无法使用 env from metadata 原地升级。

i18n/zh/docusaurus-plugin-content-docs/current/user-manuals/advancedstatefulset.md

@@ -53,13 +53,12 @@ spec:
 Advanced StatefulSet 增加了 `podUpdatePolicy` 来允许用户指定重建升级还是原地升级。
 
 - `ReCreate`: 控制器会删除旧 Pod 和它的 PVC，然后用新版本重新创建出来。
-- `InPlaceIfPossible`: 控制器会优先尝试原地升级 Pod，如果不行再采用重建升级。目前，只有修改 `spec.template.metadata.*` 和 `spec.template.spec.containers[x].image` 这些字段才可以走原地升级。
+- `InPlaceIfPossible`: 控制器会优先尝试原地升级 Pod，如果不行再采用重建升级。具体参考下方阅读文档。
 - `InPlaceOnly`: 控制器只允许采用原地升级。因此，用户只能修改上一条中的限制字段，如果尝试修改其他字段会被 Kruise 拒绝。
 
-当一个 Pod 被原地升级时，控制器会先把 Pod status 中修改为 not-ready 状态，然后再更新 Pod spec 中的 image 字段来触发 Kubelet 重建对应的容器。
-不过这样可能存在的一个风险：有时候 Kubelet 重建容器太快，还没等到其他控制器如 endpoints-controller 感知到 Pod not-ready，可能会导致流量受损。
+**请阅读[该文档](../reference/inplace-update)了解更多原地升级的细节。**
 
-因此我们又在原地升级中提供了 **graceful period** 选项，作为优雅原地升级的策略。用户如果配置了 `gracePeriodSeconds` 这个字段，控制器在原地升级的过程中会先把 Pod status 改为 not-ready，然后等一段时间（`gracePeriodSeconds`），最后再去修改 Pod spec 中的镜像版本。
+我们还在原地升级中提供了 **graceful period** 选项，作为优雅原地升级的策略。用户如果配置了 `gracePeriodSeconds` 这个字段，控制器在原地升级的过程中会先把 Pod status 改为 not-ready，然后等一段时间（`gracePeriodSeconds`），最后再去修改 Pod spec 中的镜像版本。
 这样，就为 endpoints-controller 这些控制器留出了充足的时间来将 Pod 从 endpoints 端点列表中去除。
 
 ```yaml

i18n/zh/docusaurus-plugin-content-docs/current/user-manuals/cloneset.md

@@ -165,18 +165,17 @@ metadata:
 
 ## 升级功能
 
-### 原地升级
+### 升级类型
 
-CloneSet 提供了和 `Advanced StatefulSet` 相同的 3 个升级方式，默认为 `ReCreate`：
+CloneSet 提供了 3 种升级方式，默认为 `ReCreate`：
 
 - `ReCreate`: 控制器会删除旧 Pod 和它的 PVC，然后用新版本重新创建出来。
-- `InPlaceIfPossible`: 控制器会优先尝试原地升级 Pod，如果不行再采用重建升级。目前，只有修改 `spec.template.metadata.*` 和 `spec.template.spec.containers[x].image` 这些字段才可以走原地升级。
+- `InPlaceIfPossible`: 控制器会优先尝试原地升级 Pod，如果不行再采用重建升级。具体参考下方阅读文档。
 - `InPlaceOnly`: 控制器只允许采用原地升级。因此，用户只能修改上一条中的限制字段，如果尝试修改其他字段会被 Kruise 拒绝。
 
-当一个 Pod 被原地升级时，控制器会先利用 readinessGates 把 Pod status 中修改为 not-ready 状态，然后再更新 Pod spec 中的 image 字段来触发 Kubelet 重建对应的容器。
-不过这样可能存在的一个风险：有时候 Kubelet 重建容器太快，还没等到其他控制器如 endpoints-controller 感知到 Pod not-ready，可能会导致流量受损。
+**请阅读[该文档](../reference/inplace-update)了解更多原地升级的细节。**
 
-因此我们又在原地升级中提供了 **graceful period** 选项，作为优雅原地升级的策略。用户如果配置了 `gracePeriodSeconds` 这个字段，控制器在原地升级的过程中会先把 Pod status 改为 not-ready，然后等一段时间（`gracePeriodSeconds`），最后再去修改 Pod spec 中的镜像版本。
+我们还在原地升级中提供了 **graceful period** 选项，作为优雅原地升级的策略。用户如果配置了 `gracePeriodSeconds` 这个字段，控制器在原地升级的过程中会先把 Pod status 改为 not-ready，然后等一段时间（`gracePeriodSeconds`），最后再去修改 Pod spec 中的镜像版本。
 这样，就为 endpoints-controller 这些控制器留出了充足的时间来将 Pod 从 endpoints 端点列表中去除。
 
 ```yaml

sidebars.js

@@ -61,6 +61,14 @@ module.exports = {
         },
       ],
     },
+    {
+      type: 'category',
+      label: 'Reference',
+      collapsed: true,
+      items: [
+        'reference/inplace-update',
+      ],
+    },
     {
       type: 'category',
       label: 'Best Practices',

src/css/custom.css

@@ -60,15 +60,19 @@ table td {
 }
 
 .markdown {
-  font-size: 1.0rem;
+  font-size: 0.9rem;
 }
 
 .markdown h1:first-child {
-  --ifm-h1-font-size: 2.6rem;
+  --ifm-h1-font-size: 2.0rem;
+}
+
+.markdown > h2 {
+  --ifm-h2-font-size: 1.8rem;
 }
 
 .theme-doc-sidebar-menu {
-  font-size: 0.9rem;
+  font-size: 0.8rem;
 }
 
 .blogPostTitle_node_modules-\@docusaurus-theme-classic-lib-next-theme-BlogPostItem-styles-module:first-child {