diff --git a/contributors/design-proposals/daemonset-update.md b/contributors/design-proposals/daemonset-update.md index b3cd7384d..b50beb867 100644 --- a/contributors/design-proposals/daemonset-update.md +++ b/contributors/design-proposals/daemonset-update.md @@ -67,7 +67,7 @@ type DaemonSetUpdateStrategy struct { // RollingUpdate. //--- // TODO: Update this to follow our convention for oneOf, whatever we decide it - // to be. Same as DeploymentStrategy.RollingUpdate. + // to be. Same as Deployment `strategy.rollingUpdate`. // See https://github.com/kubernetes/kubernetes/issues/35345 // +optional RollingUpdate *RollingUpdateDaemonSet @@ -118,10 +118,16 @@ type DaemonSetSpec struct { // +optional MinReadySeconds int32 `json:"minReadySeconds,omitempty"` + // DEPRECATED. // A sequence number representing a specific generation of the template. // Populated by the system. Can be set at creation time. Read-only otherwise. // +optional TemplateGeneration int64 `json:"templateGeneration,omitempty"` + + // The number of old history to retain to allow rollback. + // This is a pointer to distinguish between explicit zero and not specified. + // Defaults to 10. + RevisionHistoryLimit *int32 `json:"revisionHistoryLimit,omitempty"` } // DaemonSetStatus represents the current status of a daemon set. @@ -146,13 +152,25 @@ type DaemonSetStatus struct { // (ready for at least minReadySeconds) // +optional NumberUnavailable int32 `json:"numberUnavailable"` + + // Count of hash collisions for the DaemonSet. The DaemonSet controller + // uses this field as a collision avoidance mechanism when it needs to + // create the name for the newest ControllerRevision. + // +optional + CollisionCount *int64 `json:"collisionCount,omitempty"` } const ( + // DEPRECATED: DefaultDeploymentUniqueLabelKey is used instead. // DaemonSetTemplateGenerationKey is the key of the labels that is added - // to daemon set pods to distinguish between old and new pod templates + // to daemon set pods to distinguish between old and new pod // during DaemonSet template update. DaemonSetTemplateGenerationKey string = "pod-template-generation" + + // DefaultDaemonSetUniqueLabelKey is the default label key that is added + // to existing DaemonSet pods to distinguish between old and new + // DaemonSet pods during DaemonSet template updates. + DefaultDaemonSetUniqueLabelKey string = "daemonset-controller-hash" ) ``` @@ -163,27 +181,61 @@ const ( The DaemonSet Controller will make DaemonSet updates happen. It will watch DaemonSets on the apiserver. +DaemonSet controller manages [`ControllerRevisions`](controller_history.md) for +DaemonSet revision introspection and rollback. It's referred to as "history" +throughout the rest of this proposal. + For each pending DaemonSet updates, it will: -1. Find all pods whose labels are matched by DaemonSet `.spec.selector`. - - If `OwnerReference` is implemented for DaemonSets, filter out pods that - aren't controlled by this DaemonSet too. +1. Reconstruct DaemonSet history: + - List existing DaemonSet history controlled by this DaemonSet + - Find the history of DaemonSet's current target state, and create one if + not found: + - The `.name` of this history will be unique, generated from pod template + hash with hash collision resolution. If history creation failed: + - If it's because of name collision: + - Compare history with DaemonSet current target state: + - If they're the same, we've already created the history + - Otherwise, bump DaemonSet `.status.collisionCount` by 1, exit and + retry in the next sync loop + - Otherwise, exit and retry again in the next sync loop. + - The history will be labeled with `DefaultDaemonSetUniqueLabelKey`. + - DaemonSet controller will add a ControllerRef in the history + `.ownerReferences`. + - Current history should have the largest `.revision` number amonst all + existing history. Update `.revision` if it's not (e.g. after a rollback.) + - If more than one current history is found, remove duplicates and relabel + their pods' `DefaultDaemonSetUniqueLabelKey`. +1. Sync nodes: + - Find all nodes that should run these pods created by this DaemonSet. + - Create daemon pods on nodes when they should have those pods running but not + yet. Otherwise, delete running daemon pods that shouldn't be running on nodes. + - Label new pods with current `.spec.templateGeneration` and + `DefaultDaemonSetUniqueLabelKey` value of current history when creating them. 1. Check `DaemonSetUpdateStrategy`: - If `OnDelete`: do nothing - If `RollingUpdate`: - - Find all daemon pods that belong to this DaemonSet using label selectors. - - Pods with "pod-template-generation" value equal to the DaemonSet's - `.spec.templateGeneration` are new pods, otherwise they're old pods. - - Note that pods without "pod-template-generation" labels (e.g. DaemonSet - pods created before RollingUpdate strategy is implemented) will be - seen as old pods. + - For all pods owned by this DaemonSet: + - If its `pod-template-generation` label value equals to DaemonSet's + `.spec.templateGeneration`, it's a new pod (don't compare + `DefaultDaemonSetUniqueLabelKey`, for backward compatibility). + - Add `DefaultDaemonSetUniqueLabelKey` label to the new pod based on current + history, if the pod doesn't have this label set yet. + - Otherwise, if the value doesn't match, or the pod doesn't have a + `pod-template-generation` label, check its `DefaultDaemonSetUniqueLabelKey` label: + - If the value matches any of the history's `DefaultDaemonSetUniqueLabelKey` label, + it's a pod generated from that history. + - If that history matches the current target state of the DaemonSet, + it's a new pod. + - Otherwise, it's an old pod. + - Otherwise, if the pod doesn't have a `DefaultDaemonSetUniqueLabelKey` label, or no + matching history is found, it's an old pod. - If there are old pods found, compare `MaxUnavailable` with DaemonSet `.status.numberUnavailable` to see how many old daemon pods can be killed. Then, kill those pods in the order that unhealthy pods (failed, pending, not ready) are killed first. -1. Find all nodes that should run these pods created by this DaemonSet. -1. Create daemon pods on nodes when they should have those pods running but not - yet. Otherwise, delete running daemon pods that shouldn't be running on nodes. +1. Clean up old history based on `.spec.revisionHistoryLimit` + - Always keep live history and current history 1. Cleanup, update DaemonSet status - `.status.numberAvailable` = the total number of DaemonSet pods that have become `Ready` for `MinReadySeconds` @@ -198,6 +250,8 @@ In DaemonSet strategy (pkg/registry/extensions/daemonset/strategy.go#PrepareForU increase DaemonSet's `.spec.templateGeneration` by 1 if any changes is made to DaemonSet's `.spec.template`. +This was originally implmeneted in 1.6, and kept in 1.7 for backward compatibility. + ### kubectl #### kubectl rollout @@ -206,6 +260,9 @@ Users can use `kubectl rollout` to monitor DaemonSet updates: - `kubectl rollout status daemonset/`: to see the DaemonSet upgrade status +- `kubectl rollout history daemonset/`: to view the history of + DaemonSet updates. +- `kubectl rollout undo daemonset/`: to rollback a DaemonSet ## Updating DaemonSets mid-way @@ -216,7 +273,7 @@ one will begin rolling out. ## Deleting DaemonSets -Deleting a DaemonSet (with cascading) will delete all its pods. +Deleting a DaemonSet (with cascading) will delete all its pods and history. ## DaemonSet Strategies @@ -281,57 +338,13 @@ In the future, we may: - Support pausing DaemonSet rolling update - Support auto-rollback DaemonSets -### DaemonSet History +### API -In the future, we will support DaemonSet history, so that users can view -previous revisions and roll back when necessary. +Implement a subresource for DaemonSet history (`daemonsets/foo/history`) that +summarizes the information in the history. -One possible approach is to create a new resource called `DaemonSetRevision` to -store DaemonSet history. - -Another way to implement DaemonSet history is through creating `PodTemplates` as -snapshots of DaemonSet templates, and then create them in DaemonSet controller: - -- Find existing PodTemplates whose labels are matched by DaemonSet - `.spec.selector`. - - Sort those PodTemplates by creation timestamp and only retain at most - `.spec.revisionHistoryLimit` latest PodTemplates (remove the rest). - - Find the PodTemplate whose `.template` is the same as the DaemonSet's - `.spec.template`. - - If not found, create a new PodTemplate from DaemonSet's - `.spec.template`: - - The name will be `-` - - PodTemplate `.metadata.labels` will have a "pod-template-generation" - label, value be the same as DaemonSet's `.spec.templateGeneration`. - - PodTemplate will have revision information to avoid triggering - unnecessary restarts on rollback, since we only roll forward and only - increase templateGeneration. - - PodTemplate `.metadata.annotations` will be copied from DaemonSet - `.metadata.annotations`. - - If the PodTemplate is found, sync its "pod-template-generation" and - revision information with current DaemonSet. - - DaemonSet creates pods with "pod-template-generation" label. - -PodTemplate may need to be made an admin-only or read only resource if it's used -to store DaemonSet history. - -#### History Clean Up Policy - -We also need to specify a clean up policy for the number of history to keep in -the system, and keep only limited number of history by default (for example, 3). - -#### kubectl - -`kubectl` should also support DaemonSet history: - -- `kubectl rollout undo daemonset/` to roll back -- `kubectl rollout history daemonset/`: to view the history of - DaemonSet updates - -#### API - -Implement a subresource for DaemonSet history (e.g. `daemonsets/foo/history`) -that summarizes the information in the history. +Implement a subresource for DaemonSet rollback (`daemonsets/foo/rollback`) that +triggers a DaemonSet rollback. ### Tests