kubernetes
/
community
mirror of https://github.com/kubernetes/community.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #524 from deads2k/tpr-to-beta 

Proposal: TPR to beta proposal
This commit is contained in:
Anthony Yeh 2017-04-25 11:05:19 -07:00 committed by GitHub
parent 953840e419 f68ad715d9
commit c359e8b365
1 changed files with 253 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

253
contributors/design-proposals/thirdpartyresources.md Normal file
View File

@ -0,0 +1,253 @@
# Moving ThirdPartyResources to beta
## Background
There are a number of important issues with the alpha version of
ThirdPartyResources that we wish to address to move TPR to beta. The list is
tracked [here](https://github.com/kubernetes/features/issues/95), and also
includes feedback from existing Kubernetes ThirdPartyResource users. This
proposal covers the steps we believe are necessary to move TPR to beta and to
prevent future challenges in upgrading.
## Goals
1. Ensure ThirdPartyResource APIs operate consistently with first party
Kubernetes APIs.
2. Enable ThirdPartyResources to specify how they will appear in API
discovery to be consistent with other resources and avoid naming confilcts
3. Move TPR into their own API group to allow the extensions group to be
[removed](https://github.com/kubernetes/kubernetes/issues/43214)
4. Support cluster scoped TPR resources
5. Identify other features required for TPR to become beta
6. Minimize the impact to alpha ThirdPartyResources consumers and define a
process for how TPR migrations / breaking changes can be accomplished (for
both the cluster and for end users)
Non-goals
1. Solve automatic conversion of TPR between versions or automatic migration of
existing TPR
### Desired API Semantics
TPRs are intended to look like normal kube-like resources to external clients.
In order to do that effectively, they should respect the normal get, list,
watch, create, patch, update, and delete semantics.
In "normal" Kubernetes APIs, if I have a persisted resource in the same group
with the same name in v1 and v2, they are backed by the same underlying object.
A change made to one is reflected in the other. API clients, garbage collection,
namespace cleanup, version negotiation, and controllers all build on this.
The convertibility of Kubernetes APIs provides a seamless interaction between
versions. A TPR does not have the ability to convert between versions, which
focuses on the primary role of TPR as an easily extensible and simple mechanism
for adding new APIs. Conversion primarily allows structural, but not backwards
incompatible, changes. By not supporting conversion, all TPR use cases are
preserved, but a large amount of complexity is avoided for consumers of TPR.
Allowing a single, user specified version for a given TPR will provide this
semantic by preventing server-side versioning altogether. All instances of a
single TPR must have the same version or the Kubernetes API semantic of always
returning a resource encoded to the matching version will not be maintained.
Since conversions (even native Kubernetes conversions) cannot be used to handle
behavioral changes, the same effect can be achieved for TPRs client-side with
overlapping serialization changes.
### Avoiding Naming Problems
There are several identifiers that a Kubernetes API resource has which share
value-spaces within an API group and must not conflict. They are:
1. Resource-type value space
1. plural resource-type name - like "configmaps"
2. singular resource-type name - like "configmap"
3. short names - like "cm"
2. Kind-type value space - for group "example.com"
1. Kind name - like "ConfigMap"
2. ListKind name - like "ConfigMapList"
If these values conflict within their value-spaces then no client will be able
to properly distinguish intent.
The actual name of the TPR-registration (resource that describes the TPR to
create) resource can only protect one of these values from conflict. Since
Kubernetes API types are accessed via a URL that looks like `/apis/<group>/<version>/namespaces/<namespace-name>/<plural-resource-type>`,
the name of the TPR-registration object will be `<plural-resource-type>.<group>`.
Conflicts with other parts of the value-space can not be detected with static
validation, so there will be a spec/status split with `status.conditions` that
reflect the acceptance status of a TPR-registration. For instance, you cannot
determine whether two TPRs in the same group have the same short name without
inspecting the current state of existing TPRs.
Parts of the value-space will be "claimed" by making an entry in TPR.status to
include the accepted names which will be served. This prevents a new TPR from
disabling an existing TPR's name.
## New API
In order to:
1. eliminate opaquely derived information - deriving camel-cased kind names
from lower-case dash-delimited values as for instance.
1. allow the expression of complex transformations - not all plurals are easily
determined (ox and oxen) and not all are English. Fields for complete
specification eliminates ambiguity.
1. handle TPR-registration value-space conflicts
1. [stop using the extensions API group](https://github.com/kubernetes/kubernetes/issues/43214)
We can create a type `ThirdPartyResource.apiextension.k8s.io`.
```go
// ThirdPartyResourceSpec describe how a user wants their resource to appear
type ThirdPartyResourceSpec struct {
// Group is the group this resource belongs in
Group string `json:"group" protobuf:"bytes,1,opt,name=group"`
// Version is the version this resource belongs in
Version string `json:"version" protobuf:"bytes,2,opt,name=version"`
// Names holds the information about the resource and kind you have chosen which is
// surfaced through discovery.
Names ThirdPartyResourceNames
// Scope indicates whether this resource is cluster or namespace scoped. Default is namespaced
Scope ResourceScope `json:"scope" protobuf:"bytes,8,opt,name=scope,casttype=ResourceScope"`
}
type ThirdPartyResourceNames struct {
// Plural is the plural name of the resource to serve. It must match the name of the TPR-registration
// too: plural.group
Plural string `json:"plural" protobuf:"bytes,3,opt,name=plural"`
// Singular is the singular name of the resource. Defaults to lowercased <kind>
Singular string `json:"singular,omitempty" protobuf:"bytes,4,opt,name=singular"`
// ShortNames are short names for the resource.
ShortNames []string `json:"shortNames,omitempty" protobuf:"bytes,5,opt,name=shortNames"`
// Kind is the serialized kind of the resource
Kind string `json:"kind" protobuf:"bytes,6,opt,name=kind"`
// ListKind is the serialized kind of the list for this resource. Defaults to <kind>List
ListKind string `json:"listKind,omitempty" protobuf:"bytes,7,opt,name=listKind"`
}
type ResourceScope string
const (
ClusterScoped ResourceScope = "Cluster"
NamespaceScoped ResourceScope = "Namespaced"
)
type ConditionStatus string
// These are valid condition statuses. "ConditionTrue" means a resource is in the condition.
// "ConditionFalse" means a resource is not in the condition. "ConditionUnknown" means kubernetes
// can't decide if a resource is in the condition or not. In the future, we could add other
// intermediate conditions, e.g. ConditionDegraded.
const (
ConditionTrue ConditionStatus = "True"
ConditionFalse ConditionStatus = "False"
ConditionUnknown ConditionStatus = "Unknown"
)
// ThirdPartyResourceConditionType is a valid value for ThirdPartyResourceCondition.Type
type ThirdPartyResourceConditionType string
const (
// NameConflict means the resource or kind names chosen for this ThirdPartyResource conflict with others in the group.
// The first TPR in the group to have the name reflected in status "wins" the name.
NameConflict ThirdPartyResourceConditionType = "NameConflict"
// Terminating means that the ThirdPartyResource has been deleted and is cleaning up.
Terminating ThirdPartyResourceConditionType = "Terminating"
)
// ThirdPartyResourceCondition contains details for the current condition of this ThirdPartyResource.
type ThirdPartyResourceCondition struct {
// Type is the type of the condition.
Type ThirdPartyResourceConditionType `json:"type" protobuf:"bytes,1,opt,name=type,casttype=ThirdPartyResourceConditionType"`
// Status is the status of the condition.
// Can be True, False, Unknown.
Status ConditionStatus `json:"status" protobuf:"bytes,2,opt,name=status,casttype=ConditionStatus"`
// Last time the condition transitioned from one status to another.
// +optional
LastTransitionTime metav1.Time `json:"lastTransitionTime,omitempty" protobuf:"bytes,4,opt,name=lastTransitionTime"`
// Unique, one-word, CamelCase reason for the condition's last transition.
// +optional
Reason string `json:"reason,omitempty" protobuf:"bytes,5,opt,name=reason"`
// Human-readable message indicating details about last transition.
// +optional
Message string `json:"message,omitempty" protobuf:"bytes,6,opt,name=message"`
}
// ThirdPartyResourceStatus indicates the state of the ThirdPartyResource
type ThirdPartyResourceStatus struct {
// Conditions indicate state for particular aspects of a ThirdPartyResource
Conditions []ThirdPartyResourceCondition `json:"conditions" protobuf:"bytes,1,opt,name=conditions"`
// AcceptedNames are the names that are actually being used to serve discovery
// They may not be the same as names in spec.
AcceptedNames ThirdPartyResourceNames
}
// +genclient=true
// ThirdPartyResource represents a resource that should be exposed on the API server. Its name MUST be in the format
// <.spec.plural>.<.spec.group>.
type ThirdPartyResource struct {
metav1.TypeMeta `json:",inline"`
metav1.ObjectMeta `json:"metadata,omitempty" protobuf:"bytes,1,opt,name=metadata"`
// Spec describes how the user wants the resources to appear
Spec ThirdPartyResourceSpec `json:"spec,omitempty" protobuf:"bytes,2,opt,name=spec"`
// Status indicates the actual state of the ThirdPartyResource
Status ThirdPartyResourceStatus `json:"status,omitempty" protobuf:"bytes,3,opt,name=status"`
}
// ThirdPartyResourceList is a list of ThirdPartyResource objects.
type ThirdPartyResourceList struct {
metav1.TypeMeta `json:",inline"`
metav1.ListMeta `json:"metadata,omitempty" protobuf:"bytes,1,opt,name=metadata"`
// Items individual ThirdParties
Items []ThirdPartyResource `json:"items" protobuf:"bytes,2,rep,name=items"`
}
```
## Behavior
### Create
When a new TPR is created, no synchronous action is taken.
A controller will run to confirm that value-space of the reserved names doesn't
collide and sets the "KindNameConflict" condition to `false`.
A custom `http.Handler` will look at request and use the parsed out
GroupVersionResource information to match it to a ThirdPartyResource. The ThirdPartyResource
will be checked to make sure its valid enough in .Status to serve and will
response appropriated. If there is no ThirdPartyResource defined, it will delegate
to the next handler in the chain.
### Delete
When a TPR-registration is deleted, it will be handled as a finalizer like a
namespace is done today. The `Terminating` condition will be updated (like
namespaces) and that will cause mutating requests to be rejected by the REST
handler (see above). The finalizer will remove all the associated storage.
Once the finalizer is done, it will delete the TPR-registration itself.
## Migration from existing TPR
Because of the changes required to meet the goals, there is not a silent
auto-migration from the existing TPR to the new TPR. It will be possible, but
it will be manual. At a high level, you simply:
1. Stop all clients from writing to TPR (revoke edit rights for all users) and
stop controllers.
2. Get all your TPR-data.
`$ kubectl get TPR --all-namespaces -o yaml > data.yaml`
3. Delete the old TPR-data. Be sure you orphan!
`$ kubectl delete TPR --all --all-namespaces --cascade=false`
4. Delete the old TPR-registration.
`$ kubectl delete TPR/name`
5. Create a new TPR-registration with the same GroupVersionKind as before.
`$ kubectl create -f new_tpr.name`
6. Recreate your new TPR-data.
`$ kubectl create -f data.yaml`
7. Restart controllers.
There are a couple things that you'll need to consider:
1. Garbage collection. You may have created links that weren't respected by
the GC collector in 1.6. Since you orphaned your dependents, you'll probably
want to re-adopt them like the Kubernetes controllers do with their resources.
2. Controllers will observe deletes. Part of this migration actually deletes
the resource. Your controller will see the delete. You ought to shut down
your TPR controller while you migrate your data. If you do this, your
controller will never see a delete.