kubernetes
/
community
mirror of https://github.com/kubernetes/community.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
community/contributors/design-proposals/cluster-lifecycle/draft-20171020-bootstrap-ch...

5.2 KiB
Raw Blame History

Kubernetes Bootstrap Checkpointing Proposal

Metadata

kep-number: draft-20171029-boostrap-checkpointing.md
title: Kubernetes Bootstrap Checkpointing Proposal
authors:
  - name: Timothy St. Clair
    github: @timothysc
    email: timothysc@gmail.com
owning-sig: sig-cluster-lifecycle 
participating-sigs:
  - sig-node
reviewers:
  - @yujuhong 
  - @luxas 
  - @roberthbailey
approvers:
  - @yujuhong
  - @roberthbailey
editor:
  name: @timothysc
creation-date: 2017-10-20
last-updated: 2017-10-20
status: final-review

Table of Contents

Summary

There are several methods to deploy a kubernetes cluster, one method that offers some unique advantages is self hosting. The purpose of this proposal is to outline a method to checkpoint specific annotated pods, namely the control plane components, for the purpose of enabling self hosting.

The details of self hosting are beyond the scope of this proposal, and are outlined in the references listed below:

Extra details on this proposal, and its history, can be found in the links below:

Objectives

The scope of this proposal is bounded, but has the potential for broader reuse in the future. The reader should be mindful of the explicitly stated Non-Goals that are listed below.

Goals

  • Provide a basic framework for recording annotated Pods to the filesystem.
  • Ensure that a restart of the kubelet checks for existence of these files and loads them on startup.

Non-Goals

  • This is not a generic checkpointing mechanism for arbitrary resources. (e.g. Secrets) Such changes require wider discussions.
  • This will not checkpoint internal kubelet state.
  • This proposal does not cover self hosted kubelet(s). It is beyond the scope of this proposal, and comes with it's own unique set of challenges.

Proposal

The enablement of this feature is gated by a single command line flag that is passed to the kubelet on startup, --bootstrap-checkpoint-path , and will be denoted that it is [Alpha].

User Stories

Pod Submission to Running

  • On submission of a Pod, via kubeadm or an operator, an annotation node.kubernetes.io/bootstrap-checkpoint=true is added to that Pod, which indicates that it should be checkpointed by the kubelet. When the kubelet receives a notification from the apiserver that a new pod is to run, it will inspect the --bootstrap-checkpoint-path flag to determine if checkpointing is enabled. Finally, the kubelet will perform an atomic write of a Pod_UID.yaml file when the afore mentioned annotation exists. The scope of this annotation is bounded and will not be promoted to a field.

Pod Deletion

  • On detected deletion of a Pod, the kubelet will remove the associated checkpoint from the filesystem. Any failure to remove a pod, or file, will result in an error notification in the kubelet logs.

Cold Start

  • On a cold start, the kubelet will check the value of --bootstrap-checkpoint-path. If the value is specified, it will read in the contents of the that directory and startup the appropriate Pod. Lastly, the kubelet will then pull the list of pods from the api-server and rectify what is supposed to be running according to what is bound, and will go through its normal startup procedure.

Implementation Constraints

Due to its opt-in behavior, administrators will need to take the same precautions necessary in segregating master nodes, when enabling the bootstrap annotation.

Please see WIP Implementation for more details.

Graduation Criteria

Graduating this feature is a responsibility of sig-cluster-lifecycle and sig-node to determine over the course of the 1.10 and 1.11 releases. History has taught us that initial implementations often have a tendency overlook use cases and require refinement. It is the goal of this proposal to have an initial alpha implementation of of bootstrap checkpoining in the 1.9 cycle, and further refinement will occur after we have validated it across several deployments.

Testing

Testing of this feature will occur in three parts.

  • Unit testing of standard code behavior
  • Simple node-e2e test to ensure restart recovery
  • (TODO) E2E test w/kubeadm self hosted master restart recovery of an apiserver.

Implementation History

  • 20171020 - 1.9 draft proposal
  • ? - accepted proposal
  • ? - alpha implementation code complete

Unresolved Questions

  • None at this time.