Update task for DaemonSet history and rollback (#4098)
* Update task for DaemonSet history and rollback Also remove mentions of templateGeneration field because it's deprecated * Address comments
This commit is contained in:
parent
43b9e97774
commit
9f6d8f4788
|
|
@ -148,6 +148,7 @@ toc:
|
|||
- title: Manage Cluster Daemons
|
||||
section:
|
||||
- docs/tasks/manage-daemon/update-daemon-set.md
|
||||
- docs/tasks/manage-daemon/rollback-daemon-set.md
|
||||
|
||||
- title: Manage GPUs
|
||||
section:
|
||||
|
|
|
|||
|
|
@ -0,0 +1,154 @@
|
|||
---
|
||||
assignees:
|
||||
- janetkuo
|
||||
title: Performing a Rollback on a DaemonSet
|
||||
---
|
||||
|
||||
{% capture overview %}
|
||||
|
||||
This page shows how to perform a rollback on a DaemonSet.
|
||||
|
||||
{% endcapture %}
|
||||
|
||||
|
||||
{% capture prerequisites %}
|
||||
|
||||
* The DaemonSet rollout history and DaemonSet rollback features are only
|
||||
supported in `kubectl` in Kubernetes version 1.7 or later.
|
||||
* Make sure you know how to [perform a rolling update on a
|
||||
DaemonSet](/docs/tasks/manage-daemon/update-daemon-set/)
|
||||
|
||||
{% endcapture %}
|
||||
|
||||
|
||||
{% capture steps %}
|
||||
|
||||
## Performing a Rollback on a DaemonSet
|
||||
|
||||
### Step 1: Find the DaemonSet revision you want to roll back to
|
||||
|
||||
You can skip this step if you just want to roll back to the last revision.
|
||||
|
||||
List all revisions of a DaemonSet:
|
||||
|
||||
```shell
|
||||
kubectl rollout history daemonset <daemonset-name>
|
||||
```
|
||||
|
||||
This returns a list of DaemonSet revisions:
|
||||
|
||||
```shell
|
||||
daemonsets "<daemonset-name>"
|
||||
REVISION CHANGE-CAUSE
|
||||
1 ...
|
||||
2 ...
|
||||
...
|
||||
```
|
||||
|
||||
* Change cause is copied from DaemonSet annotation `kubernetes.io/change-cause`
|
||||
to its revisions upon creation. You may specify `--record=true` in `kubectl`
|
||||
to record the command executed in the change cause annotation.
|
||||
|
||||
To see the details of a specific revision:
|
||||
|
||||
```shell
|
||||
kubectl rollout history daemonset <daemonset-name> --revision=1
|
||||
```
|
||||
|
||||
This returns the details of that revision:
|
||||
|
||||
```shell
|
||||
daemonsets "<daemonset-name>" with revision #1
|
||||
Pod Template:
|
||||
Labels: foo=bar
|
||||
Containers:
|
||||
app:
|
||||
Image: ...
|
||||
Port: ...
|
||||
Environment: ...
|
||||
Mounts: ...
|
||||
Volumes: ...
|
||||
```
|
||||
|
||||
### Step 2: Roll back to a specific revision
|
||||
|
||||
```shell
|
||||
# Specify the revision number you get from Step 1 in --to-revision
|
||||
kubectl rollout undo daemonset <daemonset-name> --to-revision=<revision>
|
||||
```
|
||||
|
||||
If it succeeds, the command returns:
|
||||
|
||||
```shell
|
||||
daemonset "<daemonset-name>" rolled back
|
||||
```
|
||||
|
||||
If `--to-revision` flag is not specified, the last revision will be picked.
|
||||
|
||||
### Step 3: Watch the progress of the DaemonSet rollback
|
||||
|
||||
`kubectl rollout undo daemonset` tells the server to start rolling back the
|
||||
DaemonSet. The real rollback is done asynchronously on the server side.
|
||||
|
||||
To watch the progress of the rollback:
|
||||
|
||||
```shell
|
||||
kubectl rollout status ds/<daemonset-name>
|
||||
```
|
||||
|
||||
When the rollback is complete, the output is similar to this:
|
||||
|
||||
```shell
|
||||
daemon set "<daemonset-name>" successfully rolled out
|
||||
```
|
||||
|
||||
{% endcapture %}
|
||||
|
||||
|
||||
{% capture discussion %}
|
||||
|
||||
## Understanding DaemonSet Revisions
|
||||
|
||||
In the previous `kubectl rollout history` step, you got a list of DaemonSet
|
||||
revisions. Each revision is stored in a resource named `ControllerRevision`.
|
||||
`ControllerRevision` is a resource only available in Kubernetes release 1.7 or
|
||||
later.
|
||||
|
||||
To see what is stored in each revision, find the DaemonSet revision raw
|
||||
resources:
|
||||
|
||||
```shell
|
||||
kubectl get controllerrevision -l <daemonset-selector-key>=<daemonset-selector-value>
|
||||
```
|
||||
|
||||
This returns a list of `ControllerRevisions`:
|
||||
|
||||
```shell
|
||||
NAME CONTROLLER REVISION AGE
|
||||
<daemonset-name>-<revision-hash> DaemonSet/<daemonset-name> 1 1h
|
||||
<daemonset-name>-<revision-hash> DaemonSet/<daemonset-name> 2 1h
|
||||
```
|
||||
|
||||
Each `ControllerRevision` stores the annotations and template of a DaemonSet
|
||||
revision.
|
||||
|
||||
`kubectl rollout undo` takes a specific `ControllerRevision` and replaces
|
||||
DaemonSet template with the template stored in the `ControllerRevision`.
|
||||
`kubectl rollout undo` is equivalent to updating DaemonSet template to a
|
||||
previous revision through other commands, such as `kubectl edit` or `kubectl
|
||||
apply`.
|
||||
|
||||
Note that DaemonSet revisions only roll forward. That is to say, after a
|
||||
rollback is complete, the revision number (`.revision` field) of the
|
||||
`ControllerRevision` being rolled back to will advance. For example, if you
|
||||
have revision 1 and 2 in the system, and roll back from revision 2 to revision
|
||||
1, the `ControllerRevision` with `.revision: 1` will become `.revision: 3`.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
* See [troubleshooting DaemonSet rolling
|
||||
update](/docs/tasks/manage-daemon/update-daemon-set/#troubleshooting)
|
||||
|
||||
{% endcapture %}
|
||||
|
||||
{% include templates/task.md %}
|
||||
|
|
@ -33,46 +33,18 @@ DaemonSet has two update strategy types :
|
|||
DaemonSet template, old DaemonSet pods will be killed, and new DaemonSet pods
|
||||
will be created automatically, in a controlled fashion.
|
||||
|
||||
## Limitations
|
||||
|
||||
* DaemonSet rollout history is not supported yet.
|
||||
* DaemonSet rollback is not directly supported in `kubectl` yet. You can rollback
|
||||
by updating DaemonSet template to match the previous version.
|
||||
|
||||
## Caveat: Updating DaemonSet created from Kubernetes version 1.5 or before
|
||||
|
||||
If you try to rolling update a DaemonSet that was created from Kubernetes
|
||||
If you try a rolling update on a DaemonSet that was created from Kubernetes
|
||||
version 1.5 or before, a rollout will be triggered when you *first* change the
|
||||
DaemonSet update strategy to `RollingUpdate`, even when DaemonSet template isn't
|
||||
modified. All existing DaemonSet pods will be restarted.
|
||||
DaemonSet update strategy to `RollingUpdate`, no matter if DaemonSet template is
|
||||
modified or not. If the DaemonSet template is not changed, all existing DaemonSet
|
||||
pods will be restarted (deleted and created).
|
||||
|
||||
To avoid this restart, first find the DaemonSet's current
|
||||
`.spec.templateGeneration`:
|
||||
Therefore, make sure you want to trigger a rollout before you first switch the
|
||||
strategy to `RollingUpdate`.
|
||||
|
||||
```shell{% raw %}
|
||||
kubectl get ds/<daemonset-name> -o go-template='{{.spec.templateGeneration}}{{"\n"}}'
|
||||
{% endraw %}```
|
||||
|
||||
The output should be a number *N*. If the output shows `<no value>`, N = 0.
|
||||
|
||||
Then, simply label the existing DaemonSet pods with `pod-template-generation=<N>`
|
||||
before changing DaemonSet `.spec.updateStrategy` to `RollingUpdate`:
|
||||
|
||||
```shell
|
||||
# Replace N with DaemonSet `.spec.templateGeneration`
|
||||
# Only run this on pods created from current DaemonSet template
|
||||
kubectl label pods -l <daemonset-selector-key>=<daemonset-selector-value> pod-template-generation=<N>
|
||||
```
|
||||
|
||||
This tells the DaemonSet that the labeled DaemonSet pods are created from current
|
||||
DaemonSet template. Therefore, you should only run this command to existing DaemonSet
|
||||
pods that are generated from current DaemonSet template.
|
||||
|
||||
Note that you only need to do this when you first change the update strategy of
|
||||
a DaemonSet created from Kubernetes version 1.5 or before to `RollingUpdate`,
|
||||
but don't want to update its template and start a new rollout yet.
|
||||
|
||||
## Setting DaemonSet update strategy for rolling update
|
||||
## Performing a Rolling Update
|
||||
|
||||
To enable the rolling update feature of a DaemonSet, you must set its
|
||||
`.spec.updateStrategy.type` to `RollingUpdate`.
|
||||
|
|
@ -87,15 +59,15 @@ First, check the update strategy of your DaemonSet, and make sure it's set to
|
|||
RollingUpdate:
|
||||
|
||||
```shell{% raw %}
|
||||
kubectl get ds/<daemonset-name> -o go-template='{{.spec.updateStrategy.type}}{{"\n"}}'{% endraw %}
|
||||
```
|
||||
kubectl get ds/<daemonset-name> -o go-template='{{.spec.updateStrategy.type}}{{"\n"}}'
|
||||
``` {% endraw %}
|
||||
|
||||
If you haven't created the DaemonSet in the system, check your DaemonSet
|
||||
manifest with the following command instead:
|
||||
|
||||
```shell{% raw %}
|
||||
kubectl create -f ds.yaml --dry-run -o go-template='{{.spec.updateStrategy.type}}{{"\n"}}'
|
||||
{% endraw %}```
|
||||
``` {% endraw %}
|
||||
|
||||
The output from both commands should be:
|
||||
|
||||
|
|
@ -224,7 +196,9 @@ progress.
|
|||
|
||||
{% capture whatsnext %}
|
||||
|
||||
*TODO: Link to "Task: Creating a DaemonSet to adopt existing DaemonSet pods"*
|
||||
* See [Task: Performing a rollback on a
|
||||
DaemonSet](/docs/tasks/manage-daemon/rollback-daemon-set/)
|
||||
* *TODO: Link to "Task: Creating a DaemonSet to adopt existing DaemonSet pods"*
|
||||
|
||||
{% endcapture %}
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue