kubernetes
/
community
mirror of https://github.com/kubernetes/community.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Proposal: Alternate API representations for resources 

Supports server side handling of transformation of resources.
This commit is contained in:
Clayton Coleman 2016-12-01 13:11:03 -05:00
parent 483ac937f4
commit 6786b34ef3
No known key found for this signature in database
GPG Key ID: 3D16906B4F1C5CB3
1 changed files with 433 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

433
contributors/design-proposals/api-machinery/alternate-api-representations.md Normal file
View File

@ -0,0 +1,433 @@
# Alternate representations of API resources
## Abstract
Naive clients benefit from allowing the server to returning resource information in a form
that is easy to represent or is more efficient when dealing with resources in bulk. It
should be possible to ask an API server to return a representation of one or more resources
of the same type in a way useful for:
* Retrieving a subset of object metadata in a list or watch of a resource, such as the
metadata needed by the generic Garbage Collector or the Namespace Lifecycle Controller
* Dealing with generic operations like `Scale` correctly from a client across multiple API
groups, versions, or servers
* Return a simple tabular representation of an object or list of objects for naive
web or command-line clients to display (for `kubectl get`)
* Return a simple description of an object that can be displayed in a wide range of clients
(for `kubectl describe`)
* Return the object with fields set by the server cleared (as `kubectl export`) which
is dependent on the schema, not on user input.
The server should allow a common mechanism for a client to request a resource be returned
in one of a number of possible forms. In general, many of these forms are simply alternate
versions of the existing content and are not intended to support arbitrary parameterization.
Also, the server today contains a number of objects which are common across multiple groups,
but which clients must be able to deal with in a generic fashion. These objects - Status,
ListMeta, ObjectMeta, List, ListOptions, ExportOptions, and Scale - are embedded into each
group version but are actually part of a a shared API group. It must be possible for a naive
client to translate the Scale response returned by two different API group versions.
## Motivation
Currently it is difficult for a naive client (dealing only with the list of resources
presented by API discovery) to properly handle new and extended API groups, especially
as versions of those groups begin to evolve. It must be possible for a naive client to
perform a set of common operations across a wide range of groups and versions and leverage
a predictable schema.
We also foresee increasing difficulty in building clients that must deal with extensions -
there are at least 6 known web-ui or CLI implementations that need to display some
information about third party resources or additional API groups registered with a server
without requiring each of them to change. Providing a server side implementation will
allow clients to retrieve meaningful information for the `get` and `describe` style
operations even for new API groups.
## Implementation
The HTTP spec and the common REST paradigm provide mechanisms for clients to [negotiate
alternative representations of objects (RFC2616 14.1)](http://www.w3.org/Protocols/rfc2616/rfc2616.txt)
and for the server to correctly indicate a requested mechanism was chosen via the `Accept`
and `Content-Type` headers. This is a standard request response protocol intended to allow
clients to request the server choose a representation to return to the client based on the
server's capabilities. In RESTful terminology, a representation is simply a known schema that
the client is capable of handling - common schemas are HTML, JSON, XML, or protobuf, with the
possibility of the client and server further refining the requested output via either query
parameters or media type parameters.
In order to ensure that generic clients can properly deal with many different group versions,
we introduce the `meta.k8s.io` group with version `v1` that grandfathers all existing resources
currently described as "unversioned". A generic client may request that responses be applied
in this version. The contents of a particular API group version would continue to be bound into
other group versions (`status.v1.meta.k8s.io` would be bound as `Status` into all existing
API groups). We would remove the `unversioned` package and properly home these resources in
a real API group.
### Considerations around choosing an implementation
* We wish to avoid creating new resource *locations* (URLs) for existing resources
* New resource locations complicate access control, caching, and proxying
* We are still retrieving the same resource, just in an alternate representation,
which matches our current use of the protobuf, JSON, and YAML serializations
* We do not wish to alter the mechanism for authorization - a user with access
to a particular resource in a given namespace should be limited regardless of
the representation in use.
* Allowing "all namespaces" to be listed would require us to create "fake" resources
which would complicate authorization
* We wish to support retrieving object representations in multiple schemas - JSON for
simple clients and Protobuf for clients concerned with efficiency.
* Most clients will wish to retrieve a newer format, but for older servers will desire
to fall back to the implict resource represented by the endpoint.
* Over time, clients may need to request results in multiple API group versions
because of breaking changes (when we introduce v2, clients that know v2 will want
to ask for v2, then v1)
* The Scale resource is an example - a generic client may know v1 Scale, but when
v2 Scale is introduced the generic client will still only request v1 Scale from
any given resource, and the server that no longer recognizes v1 Scale must
indicate that to the client.
* We wish to preserve the greatest possible query parameter space for sub resources
and special cases, which encourages us to avoid polluting the API with query
parameters that can be otherwise represented as alternate forms.
* We do not wish to allow deep orthogonal parameterization - a list of pods is a list
of pods regardless of the form, and the parameters passed to the JSON representation
should not vary significantly to the tabular representation.
* Because we expect not all extensions will implement protobuf, an efficient client
must continue to be able to "fall-back" to JSON, such as for third party
resources.
* We do not wish to create fake content-types like `application/json+kubernetes+v1+meta.k8s.io`
because the list of combinations is unbounded and our ability to encode specific values
(like slashes) into the value is limited.
### Client negotiation of response representation
When a client wishes to request an alternate representation of an object, it should form
a valid `Accept` header containing one or more accepted representations, where each
representation is represented by a media-type and [media-type parameters](https://tools.ietf.org/html/rfc6838#section-4.3).
The server should omit representations that are unrecognized or in error - if no representations
are left after omission the server should return a `406 Not Acceptable` HTTP response.
The supported parameters are:
| Name | Value | Default | Description |
| ---- | ----- | ------- | ----------- |
| g | The group name of the desired response | Current group | The group the response is expected in. |
| v | The version of the desired response | Current version | The version the response is expected in. Note that this is separate from Group because `/` is not a valid character in Accept headers. |
| as | Kind name | None | If specified, transform the resource into the following kind (including the group and version parameters). |
| sv | The server group (`meta.k8s.io`) version that should be applied to generic resources returned by this endpoint | Matching server version for the current group and version | If specified, the server should transform generic responses into this version of the server API group. |
| export | `1` | None | If specified, transform the resource prior to returning to omit defaulted fields. Additional arguments allowed in the query parameter. For legacy reasons, `?export=1` will continue to be supported on the request |
| pretty | `0`/`1` | `1` | If specified, apply formatting to the returned response that makes the serialization readable (for JSON, use indentation) |
Examples:
```
# Request a PodList in an alternate form
GET /v1/pods
Accept: application/json;as=Table;g=meta.k8s.io;v=v1
# Request a PodList in an alternate form, with pretty JSON formatting
GET /v1/pods
Accept: application/json;as=Table;g=meta.k8s.io;v=v1;pretty=1
# Request that status messages be of the form meta.k8s.io/v2 on the response
GET /v1/pods
Accept: application/json;sv=v2
{
"kind": "Status",
"apiVersion": "meta.k8s.io/v2",
...
}
```
For both export and the more complicated server side `kubectl get` cases, it's likely that
more parameters are required and should be specified as query parameters. However, the core
behavior is best represented as a variation on content-type. Supporting both is not limiting
in the short term as long as we can validate correctly.
As a simplification for common use, we should create **media-type aliases** which may show up in lists of mime-types supported
and simplify use for clients. For example, the following aliases would be reasonable:
* `application/json+vnd.kubernetes.export` would return the requested object in export form
* `application/json+vnd.kubernetes.as+meta.k8s.io+v1+TabularOutput` would return the requested object in a tabular form
* `text/csv` would return the requested object in a tabular form in the comma-separated-value (CSV) format
### Example: Partial metadata retrieval
The client may request to the server to return the list of namespaces as a
`PartialObjectMetadata` kind, which is an object containing only `ObjectMeta` and
can be serialized as protobuf or JSON. This is expected to be significantly more
performant when controllers like the Garbage collector retrieve multiple objects.
GET /api/v1/namespaces
Accept: application/json;g=meta.k8s.io,v=v1,as=PartialObjectMetadata, application/json
The server would respond with
200 OK
Content-Type: application/json;g=meta.k8s.io,v=v1,as=PartialObjectMetadata
{
"apiVersion": "meta.k8s.io/v1",
"kind": "PartialObjectMetadataList",
"items": [
{
"apiVersion": "meta.k8s.io/v1",
"kind": "PartialObjectMetadata",
"metadata": {
"name": "foo",
"resourceVersion": "10",
...
}
},
...
]
}
In this example PartialObjectMetadata is a real registered type, and each API group
provides an efficient transformation from their schema to the partial schema directly.
The client upon retrieving this type can act as a generic resource.
Note that the `as` parameter indicates to the server the Kind of the resource, but
the Kubernetes API convention of returning a List with a known schema continues. An older
server could ignore the presence of the `as` parameter on the media type and merely return
a `NamespaceList` and the client would either use the content-type or the object Kind
to distinguish. Because all responses are expected to be self-describing, an existing
Kubernetes client would be expected to differentiate on Kind.
An old server, not recognizing these parameters, would respond with:
200 OK
Content-Type: application/json
{
"apiVersion": "v1",
"kind": "NamespaceList",
"items": [
{
"apiVersion": "v1",
"kind": "Namespace",
"metadata": {
"name": "foo",
"resourceVersion": "10",
...
}
},
...
]
}
### Example: Retrieving a known version of the Scale resource
Each API group that supports resources that can be scaled must expose a subresource on
their object that accepts GET or PUT with a `Scale` kind resource. This subresource acts
as a generic interface that a client that knows nothing about the underlying object can
use to modify the scale value of that resource. However, clients *must* be able to understand
the response the server provides, and over time the response may change and should therefore
be versioned. Our current API provides no way for a client to discover whether a `Scale`
response returned by `batch/v2alpha1` is the same as the `Scale` resource returned by
`autoscaling/v1`.
Under this proposal, to scale a generic resource a client would perform the following
operations:
GET /api/v1/namespace/example/replicasets/test/scale
Accept: application/json;g=meta.k8s.io,v=v1,as=Scale, application/json
200 OK
Content-Type: application/json;g=meta.k8s.io,v=v1,as=Scale
{
"apiVersion": "meta.k8s.io/v1",
"kind": "Scale",
"spec": {
"replicas": 1
}
...
}
The client, seeing that a generic response was returned (`meta.k8s.io/v1`), knows that
the server supports accepting that resource as well, and performs a PUT:
PUT /apis/extensions/v1beta1/namespace/example/replicasets/test/scale
Accept: application/json;g=meta.k8s.io,v=v1,as=Scale, application/json
Content-Type: application/json
{
"apiVersion": "meta.k8s.io/v1",
"kind": "Scale",
"spec": {
"replicas": 2
}
}
200 OK
Content-Type: application/json;g=meta.k8s.io,v=v1,as=Scale
{
"apiVersion": "meta.k8s.io/v1",
"kind": "Scale",
"spec": {
"replicas": 2
}
...
}
Note that the client still asks for the common Scale as the response so that it
can access the value it wants.
### Example: Retrieving an alternative representation of the resource for use in `kubectl get`
As new extension groups are added to the server, all clients must implement simple "view" logic
for each resource. However, these views are specific to the resource in question, which only
the server is aware of. To make clients more tolerant of extension and third party resources,
it should be possible for clients to ask the server to present a resource or list of resources
in a tabular / descriptive format rather than raw JSON.
While the design of serverside tabular support is outside the scope of this proposal, a few
knows apply. The server must return a structured resource usable by both command line and
rich clients (web or IDE), which implies a schema, which implies JSON, and which means the
server should return a known Kind. For this example we will call that kind `TabularOutput`
to demonstrate the concept.
A server side resource would implement a transformation from their resource to `TabularOutput`
and the API machinery would translate a single item or a list of items (or a watch) into
the tabular resource.
A generic client wishing to display a tabular list for resources of type `v1.ReplicaSets` would
make the following call:
GET /api/v1/namespaces/example/replicasets
Accept: application/json;g=meta.k8s.io,v=v1,as=TabularOutput, application/json
200 OK
Content-Type: application/json;g=meta.k8s.io,v=v1,as=TabularOutput
{
"apiVersion": "meta.k8s.io/v1",
"kind": "TabularOutput",
"columns": [
{"name": "Name", "description": "The name of the resource"},
{"name": "Resource Version", "description": "The version of the resource"},
...
],
"items": [
{"columns": ["name", "10", ...]},
...
]
}
The client can then present that information as necessary. If the server returns the
resource list `v1.ReplicaSetList` the client knows that the server does not support tabular
output and so must fall back to a generic output form (perhaps using the existing
compiled in listers).
Note that `kubectl get` supports a number of parameters for modifying the response,
including whether to filter resources, whether to show a "wide" list, or whether to
turn certain labels into columns. Those options are best represented as query parameters
and transformed into a known type.
### Example: Versioning a ListOptions call to a generic API server
When retrieving lists of resources, the server transforms input query parameters like
`labels` and `fields` into a `ListOptions` type. It should be possible for a generic
client dealing with the server to be able to specify the version of ListOptions it
is sending to detect version skew.
Since this is an input and list is implemented with GET, it is not possible to send
a body and no Content-Type is possible. For this approach, we recommend that the kind
and API version be specifiable via the GET call for further clarification:
New query parameters:
| Name | Value | Default | Description |
| ---- | ----- | ------- | ----------- |
| kind | The kind of parameters being sent | `ListOptions` (GET), `DeleteOptions` (DELETE) | The kind of the serialized struct, defaults to ListOptions on GET and DeleteOptions on DELETE. |
| queryVersion / apiVersion | The API version of the parameter struct | `meta.k8s.io/v1` | May be altered to match the expected version. Because we have not yet versioned ListOptions, this is safe to alter. |
To send ListOptions in the v2 future format, where the serialization of `resourceVersion`
is changed to `rv`, clients would provide:
GET /api/v1/namespaces/example/replicasets?apiVersion=meta.k8s.io/v2&rv=10
Before we introduce a second API group version, we would have to ensure old servers
properly reject apiVersions they do not understand.
### Impact on web infrastructure
In the past, web infrastructure and old browsers have coped poorly with the `Accept`
header. However, most modern caching infrastructure properly supports `Vary: Accept`
and caching of responses has not been a significant requirement for Kubernetes APIs
to this point.
### Considerations for discoverability
To ensure clients can discover these endpoints, the Swagger and OpenAPI documents
should also include a set of example mime-types for each endpoint that are supported.
Specifically, the `produces` field on an individual operation can be used to list a
set of well known types. The description of the operation can include a stanza about
retrieving alternate representations.
## Alternatives considered
* Implement only with query parameters
To properly implement alternative resource versions must support multiple version
support (ask for v2, then v1). The Accept mechanism already handles this sort of
multi-version negotiation, while any approach based on query parameters would
have to implement this option as well. In addition, some serializations may not
be valid in all content types, so the client asking for TabularOutput in protobuf
may also ask for TabularOutput in JSON - if TabularOutput is not valid in protobuf
the server call fall back to JSON.
* Use new resource paths - `/apis/autoscaling/v1/namespaces/example/horizontalpodautoscalermetadata`
This leads to a proliferation of paths which will confuse automated tools and end
users. Authorization, logging, audit may all need a way to map the two resources
as equivalent, while clients would need a discovery mechanism that identifies a
"same underlying object" relationship that is different from subresources.
* Use a special HTTP header to denote the alternative representation
Given the need to support multiple versions, this would be reimplementing Accept
in a slightly different way, so we prefer to reuse Accept.
* For partial object retrieval, support complex field selectors
From an efficiency perspective, calculating subpaths and filtering out sub fields
from the underlying object is complex. In practice, almost all filtering falls into
a few limited subsets, and thus retrieving an object into a few known schemas can be made
much more efficient. In addition, arbitrary transformation of the object provides
opportunities for supporting forward "partial" migration - for instance, returning a
ReplicationController as a ReplicaSet to simplify a transition across resource types.
While this is not under explicit consideration, allowing a caller to move objects across
schemas will eventually be a required behavior when dramatic changes occur in an API
schema.
## Backwards Compatibility
### Old clients
Old clients would not be affected by the new Accept path.
If servers begin returning Status in version `meta.k8s.io/v1`, old clients would likely error
as that group has never been used. We would continue to return the group version of the calling
API group on server responses unless the `sv` mime-type parameter is set.
### Old servers
Because old Kubernetes servers are not selective about the content type parameters they
accept, we may wish to patch server versions to explicitly bypass content
types they do not recognize the parameters to. As a special consideration, this would allow
new clients to more strictly handle Accept (so that the server returns errors if the content
type is not recognized).
As part of introducing the new API group `meta.k8s.io`, some opaque calls where we assume the
empty API group-version for the resource (GET parameters) could be defaulted to this group.
## Future items
* ???