Automatic merge from submit-queue (batch tested with PRs 50280, 52529, 53093, 53108, 53168). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add 201/Status to the list of returned objects.
**What this PR does / why we need it**:
Adds 201/202 to the list of codes/objects returned by `POST` and 201 to the list of codes returned by `PUT` requests in Swagger/OpenAPI
**Special notes for your reviewer**:
This helps fix
https://github.com/kubernetes-client/csharp/issues/29
**Release note**:
```release-note
Adds 201/202 to the list of codes/objects returned by `POST` and 201 to the list of codes returned by `PUT` requests in Swagger/OpenAPI
```
Kubernetes-commit: ba4f5ced3cfd30649588b36dea99c8c55433f0b4
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>..
Collapse all metrics handlers into common code
Remove the MonitorRequest method and replace with a method that takes
request.RequestInfo, which is our default way to talk about API objects.
Preserves existing semantics for calls.
Not for 1.8, but fixes the ugliness and code duplication in #52237
Kubernetes-commit: 30f015a6fc13979193a6105890ce91ace651eb5c
Automatic merge from submit-queue (batch tested with PRs 51067, 52319, 52803, 52961, 51972). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>..
ProducesObject should only update the returned API object resource documentation
This should be a no-op for Kubernetes (the only rest.Storage that specifies ProducesObject() is pod logs, on GET), but will allow issues such as openshift/origin#8345 to be resolved (where generated API documentation is currently incorrect due to the asymmetry between the object kind sent and received by the API)
```release-note NONE```
Kubernetes-commit: 50fa5637b6dc9538c908fbdd632ff1d3c59f65da
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>..
Update gophercloud to Handle New Identity Endpoints
Currently openstack cloud provider just support keystone v2.0 and v3
The latest Identity Service is publishing an ID of v3.8, we should
update gophercloud to recognize v3.8 as a valid version id.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#52830
**Release note**:
```release-note
NONE
```
Kubernetes-commit: 4f2d7b93da2464a3147e0a7e71d896dd2bade9ad
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>..
bazel: build/test almost everything
**What this PR does / why we need it**: Miscellaneous cleanups and bug fixes. The main motivating idea here was to make `bazel build //...` and `bazel test //...` mostly work. (There's a few reasons these still don't work, but we're a lot closer.)
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/assign @BenTheElder @mikedanese @spxtr
Kubernetes-commit: 7c9e614cbb507b094d82301c5ea4d26f075eed82
Automatic merge from submit-queue (batch tested with PRs 51438, 52182, 51607, 47912, 51595). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>..
Ensure we log the flag apiserver starts with.
Trying to make sure we always log the flags an instance of apiserver
starts with.
This can be especially valuable for emailed logs or e2e/kubemark tests.
**What this PR does / why we need it**: Ensures we log the flags an apiserver was started with.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #none
**Special notes for your reviewer**:
**Release note**:```release-note NONE
```
Kubernetes-commit: d2cf4d01c69881e65f8a9d38b9baad70d1eacdb3
Automatic merge from submit-queue (batch tested with PRs 51031, 51705, 51888, 51727, 51684). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>..
Avoid hard-coding location for auto-generated cert files
**What this PR does / why we need it**:
This fixes the confusing/incorrect help message for "--tls-cert-file" in apiserver package.
**Which issue this PR fixes**: fixes#51887
**Special notes for your reviewer**:
Please advise if a release note is needed.
**Release note**:
```release-note
NONE
```
Kubernetes-commit: 3484ae9e496eb0b4e24fb646db39e51440c1c135
Automatic merge from submit-queue (batch tested with PRs 52843, 52710, 52821, 52844). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>..
update-staging-godeps: only mangle staging repos in staging Godeps.json
- this re-adds non-staging k8s.io/* repos to the staging Godeps.jsons
- x-outs instead of removing of staging dependencies in order to get a
precise trigger for a complete godep restore+save run in the publisher
bot.
The first breaks k8s.io/kube-aggregator's staging export.
The second potentially leads to inconsistent godeps in our exported staging repos.
Kubernetes-commit: 3ba46ee9fab32ccaeeee4af4a80375e6e1d71ace
Automatic merge from submit-queue (batch tested with PRs 52477, 52790, 52798). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>..
conversion-gen: make staging dirs independent of living in vendor/
The `+k8s:conversion-gen` tags included the package directory inside of kube's vendor dir. This
makes them invalid when we publish staging repos.
Without this PR our sample-apiserver example code-generation is broken once published.
Kubernetes-commit: 74ff323f9e546e62288d331f62a37e0ec58f6b35
Currently openstack cloud provider just support keystone v2.0 and v3
The latest Identity Service is publishing an ID of v3.8, we should
update gophercloud to recognize v3.8 as a valid version id.
Kubernetes-commit: f98326b7f9364f6a5c3e336af1279833713e73c9
Automatic merge from submit-queue (batch tested with PRs 51824, 50476, 52451, 52009, 52237)
Improve apiserver metrics reporting
Normalize "WATCHLIST" to "WATCH", add "scope" to the other metrics (listing 50k pods is != listing pods in a namespace), and add a new scope "resource" to cover individual resource calls.
This roughly aligns metrics with our ACL model (technically resource scope is GET, but POST to a subresource and POST to a namespace are not the same thing).
```release-note
WATCHLIST calls are now reported as WATCH verbs in prometheus for the apiserver_request_* series. A new "scope" label is added to all apiserver_request_* values that is either 'cluster', 'resource', or 'namespace' depending on which level the query is performed at.
```
Kubernetes-commit: 93ddb7be5f547f5ebb0a60bd1135234fdf4e3c07
Automatic merge from submit-queue (batch tested with PRs 51900, 51782, 52030)
apiservers: stratify versioned informer construction
The versioned share informer factory has been part of the GenericApiServer config,
but its construction depended on other fields of that config (e.g. the loopback
client config). Hence, the order of changes to the config mattered.
This PR stratifies this by moving the SharedInformerFactory from the generic Config
to the CompleteConfig struct. Hence, it is only filled during completion when it is
guaranteed that the loopback client config is set.
While doing this, the CompletedConfig construction is made more type-safe again,
i.e. the use of SkipCompletion() is considereably reduced. This is archieved by
splitting the derived apiserver Configs into the GenericConfig and the ExtraConfig
part. Then the completion is structural again because CompleteConfig is again
of the same structure: generic CompletedConfig and local completed ExtraConfig.
Fixes#50661.
Kubernetes-commit: 63d6bdb58c864de4f42ff139cb0a044a35f7a36f
Automatic merge from submit-queue
fix format of forbidden messages
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#51813
**Special notes for your reviewer**:
/assign @deads2k @liggitt
**Release note**:
```release-note
None
```
Kubernetes-commit: 45fe0a9e0417eb148c3082c53ae16da7377a2c32
Automatic merge from submit-queue (batch tested with PRs 52097, 52054)
Provide field info in storage configuration
Makes debugging how storage was configured difficult
@kubernetes/sig-api-machinery-bugs
Kubernetes-commit: 310b24bfcdc9958304973bf61f3aa23f516e82d9
Automatic merge from submit-queue (batch tested with PRs 49133, 51557, 51749, 50842, 52018)
enhance unit tests of advance audit feature
This change addresses comments from @crassirostris
https://github.com/kubernetes/kubernetes/pull/49115#discussion_r133416080
It does three things:
1. use auditinternal for unit test in filter stage
2. add a seperate unit test for Audit-ID http header
3. add unit test for audit log backend
**Release note**:
```
NONE
```
Kubernetes-commit: d369160c7900368c96391738561dbc932fa9737d
Automatic merge from submit-queue
set AdvancedAuditing feature gate to true by default
All feature commits are merged. The types are updated already to beta. This only enable the feature gate by default.
**Release note**:
```
Promote the AdvancedAuditing feature to beta and enable the feature gate by default.
```
Kubernetes-commit: 7be29bd9b6913e2f39b44a72a5f46e6f33244410
Automatic merge from submit-queue (batch tested with PRs 51984, 51351, 51873, 51795, 51634)
Bug Fix - Adding an allowed address pair wipes port security groups
**What this PR does / why we need it**:
Fix for cloud routes enabled instances will have their security groups
removed when the allowed address pair is added to the instance's port.
Upstream bug report is in:
https://github.com/gophercloud/gophercloud/issues/509
Upstream bug fix is in:
https://github.com/gophercloud/gophercloud/pull/510
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
Fixes#51755
**Special notes for your reviewer**:
Just an fix in vendored code. minimal changes needed in OpenStack cloud provider
**Release note**:
```release-note
NONE
```
Kubernetes-commit: 9a8cb435b77085fa7d518c4428a02eae316b1003
Automatic merge from submit-queue (batch tested with PRs 51186, 50350, 51751, 51645, 51837)
Update Cadvisor Dependency
Fixes: https://github.com/kubernetes/kubernetes/issues/51832
This is the worst dependency update ever...
The root of the problem is the [name change of Sirupsen -> sirupsen](https://github.com/sirupsen/logrus/issues/570#issuecomment-313933276). This means that in order to update cadvisor, which venders the lowercase, we need to update all dependencies to use the lower-cased version. With that being said, this PR updates the following packages:
`github.com/docker/docker`
- `github.com/docker/distribution`
- `github.com/opencontainers/go-digest`
- `github.com/opencontainers/image-spec`
- `github.com/opencontainers/runtime-spec`
- `github.com/opencontainers/selinux`
- `github.com/opencontainers/runc`
- `github.com/mrunalp/fileutils`
- `golang.org/x/crypto`
- `golang.org/x/sys`
- `github.com/docker/go-connections`
- `github.com/docker/go-units`
- `github.com/docker/libnetwork`
- `github.com/docker/libtrust`
- `github.com/sirupsen/logrus`
- `github.com/vishvananda/netlink`
`github.com/google/cadvisor`
- `github.com/euank/go-kmsg-parser`
`github.com/json-iterator/go`
Fixed https://github.com/kubernetes/kubernetes/issues/51832
```release-note
Fix journalctl leak on kubelet restart
Fix container memory rss
Add hugepages monitoring support
Fix incorrect CPU usage metrics with 4.7 kernel
Add tmpfs monitoring support
```
Kubernetes-commit: 99aa992ce845fe947a406ac4d3f99d2208f0416b
Automatic merge from submit-queue (batch tested with PRs 51833, 51936)
fix docstring of advanced audit policy
I found this when I write doc for advanced audit beta version.
**Release note**:
```
NONE
```
Kubernetes-commit: 3a5622db737d4910d20cb2ce73fea986b8b43a64
Automatic merge from submit-queue
Provide a way to omit Event stages in audit policy
This provide a way to omit some stages for each audit policy rule.
For example:
```
apiVersion: audit.k8s.io/v1beta1
kind: Policy
- level: Metadata
resources:
- group: "rbac.authorization.k8s.io"
resources: ["roles"]
omitStages:
- "RequestReceived"
```
RequestReceived stage will not be emitted to audit backends with previous config.
**Release note**:
```
None
```
#
Kubernetes-commit: 9d29ce135658be851a96aad9dd337c5afd531d2e
Automatic merge from submit-queue (batch tested with PRs 50579, 50875, 51797, 51807, 51803)
oidc auth: make the OIDC claims prefix configurable
Add the following flags to control the prefixing of usernames and
groups authenticated using OpenID Connect tokens.
--oidc-username-prefix
--oidc-groups-prefix
```release-note
The OpenID Connect authenticator can now use a custom prefix, or omit the default prefix, for username and groups claims through the --oidc-username-prefix and --oidc-groups-prefix flags. For example, the authenticator can map a user with the username "jane" to "google:jane" by supplying the "google:" username prefix.
```
Closes https://github.com/kubernetes/kubernetes/issues/50408
Ref https://github.com/kubernetes/kubernetes/issues/31380
cc @grillz @kubernetes/sig-auth-pr-reviews @thomastaylor312 @gtaylor
Kubernetes-commit: d970eb8f9450e985073734790d2dfd93693db1da
Automatic merge from submit-queue (batch tested with PRs 51301, 50497, 50112, 48184, 50993)
audit newest impersonated user info in the ResponseStarted, ResponseComplete audit stage
Impersonation will automatically add system:authenticated, system:serviceaccounts group to the impersonated user info. This pr use the newest impersonated user info in the second audit event. This will help users to debug rbac problems.
**Release note**:
```
[advanced audit] audit newest impersonated user info in the ResponseStarted, ResponseComplete audit stage
```
@liggitt @sttts
Kubernetes-commit: 134b667d0a7f7606775631e63675a2b6b2e1f59f
Automatic merge from submit-queue (batch tested with PRs 51583, 51283, 51374, 51690, 51716)
Unify initializer name validation
Unify the validation rules on initializer names. Fix https://github.com/kubernetes/kubernetes/issues/51843.
```release-note
Action required: validation rule on metadata.initializers.pending[x].name is tightened. The initializer name needs to contain at least three segments separated by dots. If you create objects with pending initializers, (i.e., not relying on apiserver adding pending initializers according to initializerconfiguration), you need to update the initializer name in existing objects and in configuration files to comply to the new validation rule.
```
Kubernetes-commit: 12f96e2e3542dc8d6d415a31698bdb272f138095
Automatic merge from submit-queue
basic logging and additional documentation for healthz installer
@pmorie upstream logging
- InstallHandler is the public interface through which all interaction
occurs.
- It is good to know whether the default ping is occurring to know due
to manual installation or automatic installation.
- It is good to know how many handlers are installed to see whether
code changes are taking effect.
**What this PR does / why we need it**:
add logging for external users of the genericapiserver
**Special notes for your reviewer**:
This should run exactly once at startup.
**Release note**:
```release-note
NONE
```
Kubernetes-commit: fe3e7482764ace362b465405c45780d03a8c6706
Automatic merge from submit-queue (batch tested with PRs 45724, 48051, 46444, 51056, 51605)
Add selfsubjectrulesreview in authorization
**What this PR does / why we need it**:
**Which issue this PR fixes**: fixes#47834#31292
**Special notes for your reviewer**:
**Release note**:
```release-note
Add selfsubjectrulesreview API for allowing users to query which permissions they have in a given namespace.
```
/cc @deads2k @liggitt
Kubernetes-commit: c84b3132a2f1c784447ed6f405be9fe9a601d423
Automatic merge from submit-queue (batch tested with PRs 51632, 51055, 51676, 51560, 50007)
Split APIVersion into APIGroup and APIVersion in audit events
audit.Event.ObjectRef.APIVersion currently holds both the the API group and
version, separated by a /. This change break these out into separate fields.
**Release note**:
```
NONE
```
Kubernetes-commit: 92db97dfcc5eb45dcb0f686526041438d0b31a2e
Automatic merge from submit-queue
Use json-iterator instead of ugorji for JSON.
@smarterclayton @wojtek-t
Fixes#36120
xref #18762
```release-note
Switch JSON marshal/unmarshal to json-iterator library. Performance should be close to previous with no generated code.
```
Kubernetes-commit: 6a845c67f097cd76f9d90ab3c9c5b5603c70afe4
Automatic merge from submit-queue (batch tested with PRs 51574, 51534, 49257, 44680, 48836)
audit policy: support subresources and resource names
Updates #48561
policy:
- level: Metadata
resources:
- group: ""
resources ["pods/logs"]
- level: None
resources:
- group: ""
resources: ["configmaps"]
resourceNames: ["controller-leader"]
The top level resource no longer matches the subresource. For example "pods"
no longer matches requests to the logs subresource on pods.
```release-note
Audit policy supports matching subresources and resource names, but the top level resource no longer matches the subresouce. For example "pods" no longer matches requests to the logs subresource of pods. Use "pods/logs" to match subresources.
```
/cc @sttts @soltysh @crassirostris @ihmccreery @timstclair
Kubernetes-commit: 58aa139f9926afd8d8d067677698a46d72fcb7b2
Automatic merge from submit-queue (batch tested with PRs 51480, 49616, 50123, 50846, 50404)
Kubectl to use http caching to cache openapi responses from the server
**What this PR does / why we need it**:
This PR is trying to address the problems raised in #50254
> * uses a disk-based cache that is not safe between processes (does not use atomic fs operations)
> * writes get/list responses to disk that should not be cached (like kubectl get secrets)
> * is vulnerable to partially written cache responses being used as responses to future requests
> * breaks uses of the client transport that make use of websockets
> * defaults to enabling the cache for any client builder using RecommendedConfigOverrideFlags or DefaultClientConfig which affects more components than just kubectl
All of these points are addressed by this pull-request:
1. It now uses atomic fs operations
2. Doesn't cache by default, only if requested by the client (and it's only done by openapi client)
3. Fixed because of atomic fs operations
4. Found the reason for the bug: Cache wrapper couldn't be unwrapped. I implemented the `WrappedRoundTripper` interface.
5. Since 2. is fixed, I think that should be fine
@smarterclayton @liggitt
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#50254
**Special notes for your reviewer**:
**Release note**:
```release-note
Allows kubectl to use http caching mechanism for the OpenAPI schema. The cache directory can be configured through `--cache-dir` command line flag to kubectl. If set to empty string, caching will be disabled.
```
Kubernetes-commit: 0f2c2bd847ca7a66035b2289e83359ef4c923617
Automatic merge from submit-queue (batch tested with PRs 51298, 51510, 51511)
modifying the comment of BeforeDelete function to improve readability
**What this PR does / why we need it**:
modifying the comment of `BeforeDelete` function in `staging/src/k8s.io/apiserver/pkg/registry/rest/delete.go` to improve readability.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
`NONE`
Kubernetes-commit: ce559394653a8e5041f44d49391b851ec5465538
Automatic merge from submit-queue (batch tested with PRs 50932, 49610, 51312, 51415, 50705)
Add --request-timeout to kube-apiserver to make global request timeout configurable
**What this PR does / why we need it**:
Make the currently hard coded 60 global request timeout in apiserver configurable via a --request-timeout command line flag.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
https://github.com/kubernetes/kubernetes/issues/51355
**Special notes for your reviewer**:
We plan to backport this to 1.6 and 1.7. To ease the backport work, we've kept this change to a minimum. After this PR we will submit an additional PR to rename `--min-request-timeout` to something that more clearly describes it's purpose, such as `--long-running-request-timeout-min`.
**Release note**:
```release-note
Add --request-timeout to kube-apiserver to make global request timeout configurable.
```
Kubernetes-commit: 0b0b2a22c5837975824add628df6ecb2dc297d57
Automatic merge from submit-queue
Fix forbidden message format
Before this change:
$ kubectl get pods --as=tom
Error from server (Forbidden): pods "" is forbidden: User "tom" cannot list pods in the namespace "default".
After this change:
$ kubectl get pods --as=tom
Error from server (Forbidden): pods is forbidden: User "tom" cannot list pods in the namespace "default".
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```
Fix forbidden message format, remove extra ""
```
Kubernetes-commit: 562d2be7a08c5cfb0f78331b0225acdf73bf9d94
Automatic merge from submit-queue
Remove null -> [] slice hack
Closes#44593
When 1.6 added protobuf storage, the storage layer lost the ability to persist slice fields with empty but non-null values.
As a workaround, we tried to convert empty slice fields to `[]`, rather than `null`. Compressing `null` -> `[]` was just as much of an API breakage as `[]` -> `null`, but was hoped to cause fewer problems in clients that don't do null checks.
Because of conversion optimizations around converting lists of objects, the `null` -> `[]` hack was discovered to only apply to individual get requests, not to a list of objects. 1.6 and 1.7 was released with this behavior, and the world didn't explode. 1.7 documented the breaking API change that `null` and `[]` should be considered equivalent, unless otherwise noted on a particular field.
This PR:
* Reverts the earlier attempt (https://github.com/kubernetes/kubernetes/pull/43422) at ensuring non-null json slice output in conversion
* Makes results of `get` consistent with the results of `list` (which helps naive clients that do deepequal comparisons of objects obtained via list/watch and get), and allows empty slice fields to be returned as `null`
```release-note
Protobuf serialization does not distinguish between `[]` and `null`.
API fields previously capable of storing and returning either `[]` and `null` via JSON API requests (for example, the Endpoints `subsets` field) can now store only `null` when created using the protobuf content-type or stored in etcd using protobuf serialization (the default in 1.6+). JSON API clients should tolerate `null` values for such fields, and treat `null` and `[]` as equivalent in meaning unless specifically documented otherwise for a particular field.
```
Kubernetes-commit: 217513e27a6e54eb92d09165293cf811d5bdf878
Automatic merge from submit-queue (batch tested with PRs 51134, 51122, 50562, 50971, 51327)
set --audit-log-format default to json
Updates: https://github.com/kubernetes/kubernetes/issues/48561
**Release note**:
```
set --audit-log-format default to json for kube-apiserver
```
Kubernetes-commit: b5bb8099e79813aa8d73a49725efa54323d9279b
Kubernetes Publisher