kubernetes
/
community
mirror of https://github.com/kubernetes/community.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

proposal for dynamic admission controller configuration

This commit is contained in:
Chao Xu 2017-05-02 13:29:30 -07:00
parent 2a9aa2e5b0
commit b81ec0a141
1 changed files with 202 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

202
contributors/design-proposals/dynamic-admission-control-configuration.md Normal file
View File

@ -0,0 +1,202 @@
# Dynamic admission control configuration
## Background
[#132](https://github.com/kubernetes/community/pull/132) proposed making
admission control extensible. In the proposal, the `initializer admission
controller` and the `generic webhook admission controller` are the two
controllers that set default initializers and external admission hooks for
resources newly created. These two admission controllers are in the same binary
as the apiserver. This [section](https://github.com/smarterclayton/community/blob/be132e88f7597ab3927b788a3de6d5ab6de673d2/contributors/design-proposals/admission_control_extension.md#dynamic-configuration)
of #132 gave a preliminary design of the dynamic configuration of the list of
the default admission controls. This document hashes out the implementation
details.
## Goals
* Admins are able to predict what initializers/webhooks will be applied to newly
created objects.
* Do not block the entire cluster if the intializers/webhooks are not ready
after registration.
## Specification
We assume initializers could be "fail open". We need to update #132 if this is
accepted.
The schema is copied from
[#132](https://github.com/kubernetes/community/pull/132) with a few
modifications.
```golang
type AdmissionControlConfiguration struct {
TypeMeta // although this object could simply be serialized like ComponentConfig
// ResourceInitializers is a list of resources and their default initializers
ResourceInitializers []ResourceDefaultInitializer
ExternalAdmissionHooks []ExternalAdmissionHook
}
// Because the order of initializers matters, and each resource might need
// differnt order, the ResourceDefaultInitializers are indexed by Resource.
type ResourceDefaultInitializer struct {
// Resource identifies the type of resource to be initialized that should be
// initialized
Resource GroupResource
// Initializers are the default names that will be registered to this resource
Initializers []Initializer
}
type Initializer struct {
// Name is the string that will be registered to the resource that needs
// initialization.
Name string
// **Optional for alpha implement**
// FailurePolicy defines what happens if there is no initializer controller
// takes action. Allowed values are Ignore, or Fail. If "Ignore" is set,
// apiserver removes initilizer from the initializers list of the resource
// if the timeout is reached; If "Fail" is set, apiserver returns timeout
// error if the timeout is reached.
FailurePolicy FailurePolicyType
// **Optional for alpha implement**
// If timeout is reached, the intializer is removed from the resource's
// initializer list by the apiserver.
// Default to XXX seconds.
Timeout *int64
}
type ExternalAdmissionHook struct {
// Operations is the list of operations this hook will be invoked on - Create, Update, or *
// for all operations. Defaults to '*'.
Operations []string
// Resources are the resources this hook should be invoked on. '*' is all resources.
Resources []string
// Subresources are the list of subresources this hook should be invoked on. '*' is all resources.
Subresources []string
// ClientConfig defines how to talk to the hook.
ClientConfig AdmissionHookClientConfig
// FailurePolicy defines how unrecognized errors from the admission endpoint are handled -
// allowed values are Ignore, Retry, Fail. Default value is Fail
FailurePolicy FailurePolicyType
}
// AdmissionHookClientConfig contains the information to make a TLS
// connection with the webhook
// **very similar to the schema of kubeconfig**
type AdmissionHookClientConfig struct {
// Address of the external admission hook, could be a host string,
// a host:port pair, or a URL.
Address string
// ClientCertificate is the path to a client cert file for TLS.
ClientCertificate string
// ClientCertificateData contains PEM-encoded data from a client cert file
for TLS. Overrides ClientCertificate
ClientCertificateData []byte
// ClientKey is the path to a client key file for TLS.
ClientKey string
// ClientKeyData contains PEM-encoded data from a client key file for TLS. Overrides ClientKey
ClientKeyData []byte
// CertificateAuthority is the path to a cert file for the certificate authority.
CertificateAuthority string
// CertificateAuthorityData contains PEM-encoded certificate authority certificates. Overrides CertificateAuthority
CertificateAuthorityData []byte
}
```
## Synchronization of AdmissionControlConfiguration (**optional for alpha implement**)
If the `initializer admission controller` and the `generic webhook admission
controller` watch the `AdmissionControlConfiguration` and act upon deltas, their
cached version of the configuration might be arbitrarily delayed. This makes it
impossible to predicate what initializer/hooks will be applied to newly created
objects.
We considered a few ways to make the behavior of the `initializer admission
controller` and the `generic webhook admission controller` predictable.
(I prefer #2. #1 is inefficient, #3 requires complex schema and is not intuitive)
#### 1. Always do consistent read
The `initializer admission controller` and the `generic webhook admission
controller` always do consistent read of the `AdmissionControlConfiguration`
before applying the configuration to the incoming objects. This adds latency to
every CREATE request. Because the two admission controllers are in the same
process as the apiserver, the latency mainly consists of the consistent read
latency of the backend storage (etcd), and the proto unmarshalling.
#### 2. Optimized version of #1, do consistent read of a smaller object
Instead of having the two controllers do consistent read of the entire
`AdmissionControlConfiguration` object, we let the registry store the
resourceVersion of the `AdmissionControlConfiguration` (perhaps in a configMap),
and let the two controllers always do consistent read of the resourceVersion and
only read the entire `AdmissionControlConfiguration` if the local version is
lower than the stored one.
#### 3. Don't synchronize, but report what is the cached version
The main goal is *NOT* to always apply the latest
`AdmissionControlConfiguration`, but to make it predictable what
initializers/hooks will be applied. If we introduce the
`generation/observedGeneration` concept to the `AdmissionControlConfiguration`,
then a human (e.g., a cluster admin) can compare the generation with the
observedGeneration and predict if all the initializer/hooks listed in the
`AdmissionControlConfiguration` will be applied.
In the HA setup, the `observedGeneration` reported by of every apiserver's
`initializer admission controller` and `generic webhook admission controller`
are different, so the API needs to record multiple `observedGeneration`.
A tentative schema:
```golang
Type AdmissionControlConfiguration struct {
...
// Generation is set by the registry
Geneartion int64
// ObserverdGenerations is set by `initializer admission controller` and
// `generic webhook admission controller` in each apiserver.
ObserverdGenerations []ObservedGenerationByServer
}
type ObservedGenerationByServer struct {
// Address of this server
// This can be a hostname, hostname:port, IP or IP:port
Server string
// The entity that reports the observedGeneration
AdmissionController AdmissionControllerType
ObservedGeneration int64
}
type AdmissionControllerType string
const (
InitializerAdmissionController AdmissionControllerType = "initializer admission controller"
GenericWebhookAdmissionController AdmissionControllerType = "generic webhook admission controller"
)
```
## What if an initializer controller/webhook is not ready after registered? (**optional for alpha implement**)
This will block the entire cluster. We have a few options:
1. only allow initializers/webhooks to be created as "fail open". They can
upgrade themselves to "fail closed" via the normal Update operation. A human
can also update them to "fail closed" later.
2. less preferred: add readiness check to initializer and webhooks, `initializer
admission controller` and `generic webhook admission controller` only apply
those have passed readiness check. Specifically, we add `readiness` fields to
`AdmissionControllerConfiguration`; then we either create yet another
controller to probe for the readiness and update the
`AdmissionControllerConfiguration`, or ask each initializer/webhook to update
their readiness in the `AdmissionControllerConfigure`. The former is complex.
The latter is essentially the same as the first approach, except that we need
to introduce the additional concept of "readiness".