PATCH verb is used when creating a namespace using server-side apply,
while POST verb is used when creating a namespace using client-side
apply.
The difference in path between the two ways to create a namespace led to
an inconsistency when calling webhooks. When server-side apply is used,
the request sent to webhooks has the field "namespace" populated with
the name of namespace being created. On the other hand, when using
client-side apply the "namespace" field is omitted.
This commit aims to make the behaviour consistent and populates the
"namespace" field when creating a namespace using POST verb (i.e.
client-side apply).
Kubernetes-commit: 3cb510e33eecbdc37aad14f121396ccfbf5268cb
apiserver_request_duration_seconds does not take into account the
time a request spends in the server filters. If a filter takes longer
then the latency incurred will not be reflected in the apiserver
latency metrics.
For example, the amount of time a request spends in priority and
fairness machineries or in shuffle queues will not be accounted for.
- Add a server filter that attaches request received timestamp to the
request context very early in in the handler chain (as soon as
net/http hands over control to us).
- Use the above received timestamp in the apiserver latency metrics
apiserver_request_duration_seconds.
- Use the above received timestamp in the audit layer to set
RequestReceivedTimestamp.
Kubernetes-commit: d74ab9e1a4929be208d4529fd12b76d3fcd5d546
If a request tries to change managedFields, the response returns the
managedField of the live object.
Kubernetes-commit: c522ee08a3d248ec1097e3673119ffa7a4e1ef7b
previously all sorts of errors including a data race were possible because deferredResponseWriter resets the writer and returns it to the pool.
an attempt to write to a nil writer will lead to "invalid memory address or nil pointer dereference"
sharing the same instance of deferredResponseWriter might lead to "index out of range [43] with length 30" and "recovered from err index > windowEnd" errors
Kubernetes-commit: e6f98311d00f083c1b980ed7434d2e9769fa921f
Currently if a group is specified for an impersonated user,
'system:authenticated' is not added to the 'Groups' list inside the
request context.
This causes priority and fairness match to fail. The catch-all flow
schema needs the user to be in the 'system:authenticated' or in the
'system:unauthenticated' group. An impersonated user with a specified
group is in neither.
As a general rule, if an impersonated user has passed authorization
checks, we should consider him authenticated.
Kubernetes-commit: 01619cfaf6d2b1bcd96c65239e40add5c046f1e4
- Test that client-side apply users don't encounter a conflict with
server-side apply for objects that previously didn't track managedFields
- Test that we stop tracking managed fields with `managedFields: []`
- Test that we stop tracking managed fields when the feature is disabled
Kubernetes-commit: f2deb2417a6c542c54606ab17376b26ef1552b87
Introduce min, average, and standard deviation for the number of
executing mutating and readOnly requests.
Introduce min, max, average, and standard deviation for the number
waiting and number waiting per priority level.
Later:
Revised to use a series of windows
Use three individuals instead of array of powers
Later:
Add coarse queue count metrics, removed windowed avg and stddev
Add metrics for number of queued mutating and readOnly requests,
to complement metrics for number executing.
Later:
Removed windowed average and standard deviation because consumers can
derive such from integrals of consumer's chosen window.
Also replaced "requestKind" Prometheus label with "request_kind".
Later:
Revised to focus on sampling
Make the clock intrinsic to a TimedObserver
... so that the clock can be read while holding the observer's lock;
otherwise, forward progress is not guaranteed (and violations were
observed in testing).
Bug fixes and histogram buckets revision
SetX1 to 1 when queue length limit is zero, beause dividing by zero is nasty.
Remove obsolete argument in gen_test.go.
Add a bucket boundary at 0 for sample-and-water-mark histograms, to
distinguish zeroes from non-zeros.
This includes adding Integrator test.
Simplified test code.
More pervasively used "ctlr" instead of "ctl" as abbreviation for
"controller".
Kubernetes-commit: 57ecea22296797a93b0157169db0ff2e477f58d0
- Allow client-side to server-side apply upgrade.
Ensure that a user can change management of an object from client-side apply to
server-side apply without conflicts.
- Allow server-side apply to client-side downgrade.
For an object managed with client-side apply, a user may upgrade to
managing the object with server-side apply, then decide to downgrade.
We can support this downgrade by keeping the last-applied-configuration
annotation for client-side apply updated with server-side apply.
Kubernetes-commit: e4368eb67e363d3d03f81214a8929268d2fe88ff
* fix a number of unbounded dimensions in request metrics
* add test suite for cleanVerb and cleanContentType
* Properly validate that the content-type and charset (if applicable) are RFC compliant
* add additional test case
* truncate list of content-types
Change-Id: Ia5fe0d2e2c602e4def4b8e0849cc19f3f9251818
Kubernetes-commit: 6c588c3f441252f42fd37526297ed92d1e1f3acf
This change adds the generic ability for request handlers that run
before WithAudit to set annotations in the audit.Event.Annotations
map.
Note that this change does not use this capability yet. Determining
which handlers should set audit annotations and what keys and values
should be used requires further discussion (this data will become
part of our public API).
Signed-off-by: Monis Khan <mok@vmware.com>
Kubernetes-commit: 0bc62112adf270ef4efada37286319c229324c7b
This change removes support for basic authn in v1.19 via the
--basic-auth-file flag. This functionality was deprecated in v1.16
in response to ATR-K8S-002: Non-constant time password comparison.
Similar functionality is available via the --token-auth-file flag
for development purposes.
Signed-off-by: Monis Khan <mok@vmware.com>
Kubernetes-commit: df292749c9d063b06861d0f4f1741c37b815a2fa
Beta OS/arch labels have been deprecated since 1.14.
This change replaces these labels with the GA ones.
Kubernetes-commit: bcd975aa6575ae37ec3be3481e44cd0dccd02337
I've also moved the deserialization of the object outside the benchmark
since we're not trying to benchmark the yaml parser.
Kubernetes-commit: a52776fbfb305374d87bb553739f712e055b2206
Lowers probability of managedField population on create/update to 0%
until serialization/normalization issues are resolved
Kubernetes-commit: ba23aa98f6574bd1f9781f0d3e61d0496f16fc53
Instead of returning an error on the watch stream, if we can't properly
negotiate a watch serialization format we should error and return that
error to the client.
Kubernetes-commit: 9aad6aa54d824ba93a6670cd5a0cab6ad337e9f0
Currently we record request metrics during the normal request flow and
we also manually invoke `Record` in the timeout handler to record
timeouts. This means that we effectively double count whenever we
timeout. This PR renames the `Record` function to `RecordRequestError`
to more accurately reflect the intended side-effect of the function
call.
Change-Id: Ie37fd0c1e501bd525640a434433d364a5fd6dde2
Kubernetes-commit: 4c6e7247878477a1f2efc26df7f141258010374f
This is super expensive and not needed at all since we don't have to
reparse the entire object. Remove all allocations but the first one.
Kubernetes-commit: 31c644a1e79c685b52683ed1e84964186a37f3ff
We don't have a lot of data on allocations and how much time it takes to
run apply or update on objects, so adding some benchmark will help us
investigate possible improvements.
Kubernetes-commit: 92cf3764f979e63317c8f483d8e841e0358599f4
The previous HTTP compression implementation functioned as a filter, which
required it to deal with a number of special cases that complicated the
implementation.
Instead, when we write an API object to a response, handle only that one
case. This will allow a more limited implementation that does not impact
other code flows.
Also, to prevent excessive CPU use on small objects, compression is
disabled on responses smaller than 128Kb in size.
Kubernetes-commit: 4ed2b9875d0498b5c577095075bda341e96fcec2
Normal files should have permissions 644 by default,
and does not require the last bit to be
executable
Signed-off-by: Odin Ugedal <odin@ugedal.com>
Kubernetes-commit: 35cb87f9cf71776e99a970dfff751cd29ba7ebfb
Correctly ensure CRDs can be watched using protobuf when transformed to
PartialObjectMetadata. To do this we add a set of serializers allowed to
be used for "normal" requests (that return CRDs) while the serializers
supported by the infrastructure is broader and includes protobuf. During
negotatiation we check for transformation requests and protobuf is
excluded from non-transform requests.
As part of the change, correct an error message when the server returns
a 406 but the client doesn't accept the format to avoid confusing users
who set impossible Accept rules for CRDs (the dynamic client doesn't
support Protobuf, so if the server responds with a protobuf status the
message from the server is lost and the generic error was confusing).
Kubernetes-commit: 89e752add07f443248f66e4798d160f2d7529a19
conflict.
Adding unit test verify that deleteValidation is retried.
adding e2e test verifying the webhook can intercept configmap and custom
resource deletion, and the existing object is sent via the
admissionreview.OldObject.
update the admission integration test to verify that the existing object
is passed to the deletion admission webhook as oldObject, in case of an
immediate deletion and in case of an update-on-delete.
Kubernetes-commit: 7bb4a3bace048cb9cd93d0221a7bf7c4accbf6be
Errors on updates are bad because they usually come from controllers and
it's very hard to take actions on them. We also don't want to start
breaking kubernetes clusters if something in a schema happens a way we
didn't foresee (even though we've tried to be diligent and test as much
as possible, these can still happen).
Log an identifiable error when they happen. Ideally people can look in
the logs to find these and report them, or providers can look for these
in logs and make sure they don't happen.
Only conversion to internal types are going to be logged and ignored.
It means that we're still failing for:
- Version conversions. If we can't convert the object from one version
to another,
- Unions. If we can't normalize the union,
- Invalid MangedFields sent in the object. If something has changed the
ManagedFields to an invalid value.
- Failure to serialize the manager information, this really shouldn't
happen.
- Encoding the ManagedFields
Kubernetes-commit: 4e32d183d0257c9f6c7f8342d1f9aa7f28458f2f
Typo during setting up PartialObjectMetadataList, it should be a slice
of `PartialObjectMetadata`, not a slice of `*PartialObjectMetadata`.
Kubernetes-commit: f25efd12e63f1d7db5f29fe28831ad0126200c0b
Now that internal types are equivalent, allow the apiserver to serve
metav1 and metav1beta1 depending on the client. Test that in the
apiserver integration test and ensure we get the appropriate responses.
Register the metav1 type in the appropriate external locations.
Kubernetes-commit: 33a3e325f754d179b25558dee116fca1c67d353a
We have an existing helper function for this: runtime.SerializerInfoForMediaType()
This is common prep-work for encoding runtime.Objects into JSON/YAML for transmission over the wire or writing to ComponentConfigs.
Kubernetes-commit: 47e52d2981dc2a5c5950042f50688cf24dd92eda
Clarifies that requesting no conversion is part of the codec factory, and
future refactors will make the codec factory less opionated about conversion.
Kubernetes-commit: 7f9dfe58f4cbe1e1b9e80f52addff70bac87bed4
RequestScope is a large struct and causes stack growth when we pass
it by value into multiple stack levels. Avoid the allocations for
this read only struct by passing a pointer.
Kubernetes-commit: 8fede0b18a81a6fb1acc1a48857f482857c25286
A self link should only require one allocation, and we should skip
url.PathEscape() except when the path actually needs it.
Add a fuzz test to build random strings and verify them against
the optimized implementation. Add a new BenchmarkWatchHTTP_UTF8 that
covers when we have unicode names in the self link.
```
> before
BenchmarkGet-12 10000 118863 ns/op 17482 B/op 130 allocs/op
BenchmarkWatchHTTP-12 30000 38346 ns/op 1893 B/op 29 allocs/op
> after
BenchmarkGet-12 10000 116218 ns/op 17456 B/op 130 allocs/op
BenchmarkWatchHTTP-12 50000 35988 ns/op 1571 B/op 26 allocs/op
BenchmarkWatchHTTP_UTF8-12 50000 41467 ns/op 1928 B/op 28 allocs/op
```
Saves 3 allocations in the fast path and 1 in the slow path (the
slow path has to build the buffer and then call url.EscapedPath
which always allocates).
Kubernetes-commit: 389a8436b52db4936b56e08f07984da362c91f6b
There was no reason to have two types and this avoids ~10% of allocations
on the GET code path.
```
BenchmarkGet-12 100000 109045 ns/op 17608 B/op 146 allocs/op
BenchmarkGet-12 100000 108850 ns/op 15942 B/op 132 allocs/op
```
Kubernetes-commit: 0489d0b1cf139253b82f73b072578073bc5616d6
ReplaceAllStrings always allocates, while scanning a relatively short
regex twice is slightly more CPU immediately but less later.
```
BenchmarkGet-12 100000 108824 ns/op 17818 B/op 152 allocs/op
BenchmarkGet-12 100000 108013 ns/op 17732 B/op 149 allocs/op
```
Kubernetes-commit: 83c41eab1d6f55c9ccf61744bfc345c95d3a0664
IsListType was causing ~100 allocations for a non list object. It is
used in a wide range of code, so perform a more targeted check.
The use of `%#v` in a hot return path for `fmt.Errorf()` was the main
victim.
Replace `%#v` with a typed error and create a cache of types that are
lists with a bounded size (probably not necessary, but safer).
```
BenchmarkGet-12 100000 119635 ns/op 20110 B/op 206 allocs/op
BenchmarkWatchHTTP-12 100000 65761 ns/op 7296 B/op 139 allocs/op
BenchmarkGet-12 100000 109085 ns/op 17831 B/op 152 allocs/op
BenchmarkWatchHTTP-12 200000 33966 ns/op 1913 B/op 30 allocs/op
```
Kubernetes-commit: 58fb665646aa4c1b63f1322a50e3af7a60e39886
Clean up the code paths that lead to objects being transformed and output with negotiation.
Remove some duplicate code that was not consistent. Now, watch will respond correctly to
Table and PartialObjectMetadata requests. Add unit and integration tests.
When transforming responses to Tables, only the first watch event for a given type will
include the columns. Columns will not change unless the watch is restarted.
Add a volume attachment printer and tighten up table validation error cases.
Disable protobuf from table conversion because Tables don't have protobuf because they
use `interface{}`
Kubernetes-commit: 3230a0b4fd14a6166f8362d4732e199e8779c426
What is the problem being solved?
https://github.com/kubernetes/kubernetes/pull/75135
Currently version compatibility is not being checked in server side apply between the patch object and the live object. This is causing a merge that will error to be run and the apiserver returns a 500 error. The request should fail if the apiVersion provided in the object is different from the apiVersion in the url, but it should fail before trying to merge, and be a 4xx error. Probably a bad request error.
Why is this the best approach?
The approach of serializing the patch byte array and then checking for version equality with the already serialized live object is the simplest and most straightforward solution.
Kubernetes-commit: d5bd17cda0c134e5ef5c03c3eac79a9ce4e18003
And add a corresponding flag in kubectl (for apply), even though the
value is defaulted in kubectl with "kubectl".
The flag is required for Apply patch-type, and optional for other PATCH,
CREATE and UPDATE (in which case we fallback on the user-agent).
Kubernetes-commit: eb904d8fa89da491f400614f99458ed3f0d529fb
Andrea Nodari