This patch adds few unit tests to assert that the webhook accessors are
only recreate when they are update in the api-server.
In order to test this feature we had to make few changes to wb manager
that allows us to mock `NewValidatingWebhookAccessor` external function.
Kubernetes-commit: 7d3d44af77679ed488b28dc839d02a8258fd3adc
This patch fixes the deadlock issue by using a map to cache already
initiated Webhooks instead of using `needRefresh` map.
Kubernetes-commit: c6f36e8702a9e90350c585298f1fc6e908699b12
* add quantity library to CEL
* add more tests to quantity
* use 1.29 env for quantity
* set CEL default env to 1.28 for 1.28 release
* add compare function
* docs and arith lib
* fixup addInt and subInt overload, add docs
* more tests
* cleanup docs
* remove old comments
* remove unnecessary cast
* add isInteger
* add overflow tests
* boilerplate
* refactor expectedResult for tests
* doc typo fix
* returns bool
* add docs link
* different dos link
* add isInteger true case
* expand iff
* add quantity back to 1.28 version, and revert change to DefaultCompatibilityVersion
* formatting
Kubernetes-commit: 423f4dfc7982136c958fc78e187c911a8896ba1b
* [API REVIEW] ValidatingAdmissionPolicyStatucController config.
worker count.
* ValidatingAdmissionPolicyStatus controller.
* remove CEL typechecking from API server.
* fix initializer tests.
* remove type checking integration tests
from API server integration tests.
* validatingadmissionpolicy-status options.
* grant access to VAP controller.
* add defaulting unit test.
* generated: ./hack/update-codegen.sh
* add OWNERS for VAP status controller.
* type checking test case.
Kubernetes-commit: 049614f884e61d87fc5e277cf9fd7cb2e6571217
Change name to make it compliant with prometheus guidelines.
Calculate it on demand instead of periodic to comply with prometheus standards.
Replace "endpoint" with "server" label to make it semantically consistent with storage factory
Kubernetes-commit: 7a63997c8a1a9ba14f2bdc478fdf33cf88f48d80
Request bookmark every 100ms when there is at least one request blocked on revision not present in watch cache.
Kubernetes-commit: 39bb8f4bb1d013937aceac6c387563ffe13545c5
This avoids the surprise of identical authorization checks within a
policy evaluating to different decisions during the same admission
pass, and reduces the overhead of repeatedly referencing the same
authorization check.
Kubernetes-commit: f1700e4b95b404b37312084800ab8022f7069fee
This is a duplicate of
`apiserver_flowcontrol_request_concurrency_in_use` but with a better
name. Hopefully we can later remove the copy with the inferior name.
Signed-off-by: Mike Spreitzer <mspreitz@us.ibm.com>
Kubernetes-commit: 65e818d4ecfb7bf2a165897fb1caf29bf42f4f83
TestTimeoutRequestHeaders and TestTimeoutWithLogging are designed to
catch data races on request headers and include an HTTP handler that
triggers timeout then repeatedly mutates request headers. Sometimes,
the request header mutation loop could complete before the timeout
filter observed the timeout, resulting in a test failure. The mutation
loop now runs until the test ends.
Kubernetes-commit: e5a15c87e9d83ee19ba93aa356dfbb7b33a013c8
dry-run and non-dry-run are currently a little different since dry-run
was using the destination object to get the current status. That causes
a weird duplication bug with the HorizontalPodAutoscaler conversion
code. Addresses the bug by using an empty object for the current state
and keep the destination for its actual "out" purpose.
Kubernetes-commit: 20866b3f85ac50a094a4400469ebcac381cbc7e9
Doing this allows us to implement some more nuanced cacher manipulations
to be used in testing. For ex: implementing a test-only compaction method
for the watch cache.
Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>
Kubernetes-commit: 6d66fbc6b670f1120a9041873bb8d1a0655bbefc
This commit prepares for when cacher tests are moved here
from the `tests` package. Tests in that package redeclare
some of the testing utils that exist here, so dedup-ing them.
This commit also adapts to any changes in test util signatures.
There are still some utils that can be reused but currently are
highly specific to some tests. (ex: watch_cache_test.go)
Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>
Kubernetes-commit: 70978e4af619819787a4eb544ffd732aa7954d76
Since cachingObject has the encoded data cached and they are not
supposed to change. It's memory efficient to just copy the slice
references.
Signed-off-by: Eric Lin <exlin@google.com>
Kubernetes-commit: 3085b57869a2a7bf5290ab97facaf17fedfa88a0
There exists a storage test to test for rv=0 and production
of ADDED events. This commit adapts the test to be used for
the watch cache as well.
Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>
Kubernetes-commit: 4d85a1f00cb0f1350cf8495925be0e8bfed59a15
When a http2 connection dies due to ping timeout, http2 client gets an
error of "http2: client connection lost". This is similar to
ConnectionReset case so it should be retryable.
Signed-off-by: Eric Lin <exlin@google.com>
Kubernetes-commit: 2658a2b627f80ba46e81667278c884acee3988e9
This was making my eyes bleed as I read over code.
I used the following in vim. I made them up on the fly, but they seemed
to pass manual inspection.
:g/},\n\s*{$/s//}, {/
:w
:g/{$\n\s*{$/s//{{/
:w
:g/^\(\s*\)},\n\1},$/s//}},/
:w
:g/^\(\s*\)},$\n\1}$/s//}}/
:w
Kubernetes-commit: d55b67b349021b6c46fc6ce78f2a36bd4217145f
it is required for Server-SIde-Apply to function correctly (SSA is based on OpenAPI schemas)
Kubernetes-commit: 302daa889c5ddb9c862cd0101b94071e42a3081d
This touches cases where FromInt() is used on numeric constants, or
values which are already int32s, or int variables which are defined
close by and can be changed to int32s with little impact.
Signed-off-by: Stephen Kitt <skitt@redhat.com>
Kubernetes-commit: 94410ee8078971b8894e5c400ce3fc79f02bc510
If the cacher hasn't seen any event (when lastProcessedResourceVersion is zero) and
the bookmarkTimer has ticked then we shouldn't popExpiredWatchers. This is
because the watchers wont' be re-added and will miss future bookmark events when
the cacher finally receives an event via the c.incoming chan.
Kubernetes-commit: 6db4cbfde7babfb34f5cd1059c769ec2d870f12a
* cacher: remove locking from watcherBookmarkTimeBuckets
it turns out that the watcherBookmarkTimeBuckets
is called from only three places/methods: startDispatching, finishDispatching and Watch.
All these methods acquire c.Lock() before touching watcherBookmarkTimeBuckets.
Thus we could remove explicit locking in
watcherBookmarkTimeBuckets since the access is already synced.
* cacher: rename watcherBookmarkTimeBuckets methods to indicate that proper synchronisation must be used
Kubernetes-commit: eab66a687b282266f0520b79166f7f55828ffd28
waitUntilWatchCacheFreshAndForceAllEvents must be called without
a read lock held otherwise the watchcache won't be able to make
progress (i.e. the watchCache.processEvent method that requries acquiring an exclusive lock)
the deadlock can happen only when the alpha watchlist feature flag is on
and the client specifically requests streaming.
Kubernetes-commit: 476e407ffd2ab393840d3f7a9fd01b71698738a3
this check needs to go after any mutations. After the mutating admission chain, rest.BeforeUpdate (which is responsible for reverting updates to immutable timestamp fields, among other things.) is called in the store.Update function. Without moving this check, it will be possible for an object to be written to etcd with only a change to its managed fields timestamp.
Kubernetes-commit: 2b01f63b115e19e8ac9f8ee8e00dde65c5f40290
Fixes up a few misspellings of gorestful in the Director field docstring
for APIServerHandler.
Signed-off-by: hasheddan <georgedanielmangum@gmail.com>
Kubernetes-commit: b1e3326eaeee982d3d5c1618022c306d50fe631e
Change admission ApplyTo() to take in clients instead of a rest.Config.
Signed-off-by: Andy Goldstein <andy.goldstein@redhat.com>
Kubernetes-commit: 364b66ddd6554a898724b6781fd90a15a38ddb41
* ftr(watch-cache): add benchmarks
* ftr(kube-apiserver): faster watch-cache getlist
* refine: testcase name
* - refine var name make it easier to convey meaning
- add comment to explain why we need to apply for a slice of runtime.Object instead of making a slice of ListObject.Items directly.
Kubernetes-commit: 75f17eb38fc8bbcb360d43dffce6e27a7159d43f
Prior to this change, we wait until the DEK is used to perform an
encryption before validating the response. This means that the
plugin could report healthy but all TransformToStorage calls would
fail. Now we correctly cause the plugin to become unhealthy and do
not attempt to use the newly generated DEK.
Signed-off-by: Monis Khan <mok@microsoft.com>
Kubernetes-commit: 5469c198e5d074c7e88e14c3dcbc3ebb2b37cfa8
This matches the logic we have for the Authorization header as well
as the impersonation headers.
Signed-off-by: Monis Khan <mok@microsoft.com>
Kubernetes-commit: e9866d2794675aa8dc82ba2637ae45f9f3a27dff
* Add custom match conditions for CEL admission
This PR is based off of, and dependent on the following PR:
https://github.com/kubernetes/kubernetes/pull/116261
Signed-off-by: Max Smythe <smythe@google.com>
* run `make update`
Signed-off-by: Max Smythe <smythe@google.com>
* Fix unit tests
Signed-off-by: Max Smythe <smythe@google.com>
* Fix unit tests
Signed-off-by: Max Smythe <smythe@google.com>
* Update compatibility test data
Signed-off-by: Max Smythe <smythe@google.com>
* Revert "Update compatibility test data"
This reverts commit 312ba7f9e74e0ec4a7ac1f07bf575479c608af28.
* Allow params during validation; make match conditions optional
Signed-off-by: Max Smythe <smythe@google.com>
* Add conditional ignoring of matcher CEL expression validation on update
Signed-off-by: Max Smythe <smythe@google.com>
* Run codegen
Signed-off-by: Max Smythe <smythe@google.com>
* Add more validation tests
Signed-off-by: Max Smythe <smythe@google.com>
* Short-circuit CEL matcher when no matchers specified
Signed-off-by: Max Smythe <smythe@google.com>
* Run codegen
Signed-off-by: Max Smythe <smythe@google.com>
* Address review comments
Signed-off-by: Max Smythe <smythe@google.com>
---------
Signed-off-by: Max Smythe <smythe@google.com>
Kubernetes-commit: e5fd204c33e90a7e8f5a0ee70242f1296a5ec7af
* api changes adding match conditions
* feature gate and registry strategy to drop fields
* matchConditions logic for admission webhooks
* feedback
* update test
* import order
* bears.com
* update fail policy ignore behavior
* update docs and matcher to hold fail policy as non-pointer
* update matcher error aggregation, fix early fail failpolicy ignore, update docs
* final cleanup
* openapi gen
Kubernetes-commit: 5e5b3029f3bbfc93c3569f07ad300a5c6057fc58
It is possible for a KMSv2 plugin to return a static value as Ciphertext
and store the actual encrypted DEK in the annotations. In this case,
using the encDEK will not work. Instead, we are now using a combination
of the encDEK, keyID and annotations to generate the cache key.
Signed-off-by: Anish Ramasekar <anish.ramasekar@gmail.com>
Kubernetes-commit: 8eacf09649ac9042c7e998b5c24ac59d68ae7e6c
Note that this fixes a bug in the existing `toBytes` implementation
which does not correctly set the capacity on the returned slice.
Signed-off-by: Monis Khan <mok@microsoft.com>
Kubernetes-commit: aa80f8fb856bb2b645c90457f9b1dd75e4e57c73
This change updates KMS v2 to not create a new DEK for every
encryption. Instead, we re-use the DEK while the key ID is stable.
Specifically:
We no longer use a random 12 byte nonce per encryption. Instead, we
use both a random 4 byte nonce and an 8 byte nonce set via an atomic
counter. Since each DEK is randomly generated and never re-used,
the combination of DEK and counter are always unique. Thus there
can never be a nonce collision. AES GCM strongly encourages the use
of a 12 byte nonce, hence the additional 4 byte random nonce. We
could leave those 4 bytes set to all zeros, but there is no harm in
setting them to random data (it may help in some edge cases such as
live VM migration).
If the plugin is not healthy, the last DEK will be used for
encryption for up to three minutes (there is no difference on the
behavior of reads which have always used the DEK cache). This will
reduce the impact of a short plugin outage while making it easy to
perform storage migration after a key ID change (i.e. simply wait
ten minutes after the key ID change before starting the migration).
The DEK rotation cycle is performed in sync with the KMS v2 status
poll thus we always have the correct information to determine if a
read is stale in regards to storage migration.
Signed-off-by: Monis Khan <mok@microsoft.com>
Kubernetes-commit: 832d6f0e19f13b9dd22b1fe9d705817e9e64f4f1
* apiserver: add latency tracker for priority & fairness queue wait time
Signed-off-by: Andrew Sy Kim <andrewsy@google.com>
* apiserver: exclude priority & fairness wait times to SLO/SLI latency metrics
Signed-off-by: Andrew Sy Kim <andrewsy@google.com>
* apiserver: update TestLatencyTrackersFrom to check latency from PriorityAndFairnessTracker
Signed-off-by: Andrew Sy Kim <andrewsy@google.com>
* flowcontrol: add helper function observeQueueWaitTime to consolidate metric and latency tracker calls
Signed-off-by: Andrew Sy Kim <andrewsy@google.com>
* flowcontrol: replace time.Now() / time.Since() with clock.Now() / clock.Since() for better testability
Signed-off-by: Andrew Sy Kim <andrewsy@google.com>
* flowcontrol: add unit test TestQueueWaitTimeLatencyTracker to validate queue wait times recorded by latency tracker
Signed-off-by: Andrew Sy Kim <andrewsy@google.com>
---------
Signed-off-by: Andrew Sy Kim <andrewsy@google.com>
Kubernetes-commit: ee18f602523e11a80823a659bed8f70f98a12914
The pod_logs subsystem was inadvertently made redundant in the following
kube-apiserver metrics:
- kube_apiserver_pod_logs_pods_logs_backend_tls_failure_total
- kube_apiserver_pod_logs_pods_logs_insecure_backend_total
To safely rename them, it is required to deprecate them in 1.27 whilst
introducing the new metrics replacing them.
Signed-off-by: Damien Grisonnet <dgrisonn@redhat.com>
Kubernetes-commit: 1efa1a65ee26c68de3f972f4e079338889a3e5e9
this method waits until cache is at least
as fresh as given requestedWatchRV if sendInitialEvents was requested.
Additionally, it instructs the caller whether it should ask for
all events from the cache (full state) or not.
Kubernetes-commit: 21fb98105043d1a15ef48089ef231931851d2d15
This change helps users understand the state of their encryption
config if storage migration is not consistently run with key ID
rotation.
Signed-off-by: Monis Khan <mok@microsoft.com>
Kubernetes-commit: 539f734bfd0d828a5b3dfbfe30df9d4229e6492c
if old less than new, Inc function should be called for `watchCacheCapacityIncreaseTotal` instead of `watchCacheCapacity`
Signed-off-by: joey <zchengjoey@gmail.com>
Kubernetes-commit: 96b9531f3e3f489e47493297987eee14d2a08855
* cacher allow context cancellation if not ready
Replace the sync.Cond variable with a channel so we can use the
context cancellation signal.
Co-authored-by: Wojciech Tyczy<C5><84>ski <wojtekt@google.com>
Change-Id: I2f75313a6337feee440ece4c1e873c32a12560dd
* wait again on pending state
Change-Id: I1ad79253a5a5d56a4d9611125825b1f7ad552be8
---------
Co-authored-by: Wojciech Tyczy<C5><84>ski <wojtekt@google.com>
Kubernetes-commit: 3b17aece1fa492e98aa82b948597b3641961195f
1. Define ContainerResizePolicy and add it to Container struct.
2. Add ResourcesAllocated and Resources fields to ContainerStatus struct.
3. Define ResourcesResizeStatus and add it to PodStatus struct.
4. Add InPlacePodVerticalScaling feature gate and drop disabled fields.
5. ResizePolicy validation & defaulting and Resources mutability for CPU/Memory.
6. Various fixes from code review feedback (originally committed on Apr 12, 2022)
KEP: /enhancements/keps/sig-node/1287-in-place-update-pod-resources
Kubernetes-commit: 76962b0fa7862727e93ef591f4b0822c8d80534b
This reverts commit 7efa62dfdf96890f7f3cf95d957c7561e09055c4, reversing
changes made to c48a7971e35a56cc2c996e174e3f76e9d2e82eaa.
Kubernetes-commit: 6165a12c718936a8edcc5b7e1a2e6b0f5d6e279e
This commit extends the test cases of RunTestList
to include the things tested by TestListDeprecated
and subsequently deletes the test.
This additionally adds a test case for checking that
the list return the modified version of an object.
Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>
Kubernetes-commit: 7474d9b719619caa356dfb71a06482f33b3ed1ce
The original design was to honour strong consistency
semantics for when the RV is unset, i.e. serve the
watch by doing a quorum read.
However, the implementation did not match the intent,
in that, the Cacher did not distinguish between set
and unset RV. This commit rectifies that behaviour by
serving the watch from the underlying storage if the
RV is unset.
This commit subsequently also adds a test for the same.
Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>
Kubernetes-commit: 610b67031c79c6c38964631d27dd59df357c6d2e
This commit allows injecting errors for the
Watch() method of the dummy storage impl.
As a consequence of this, a race is introduced
between when the injected error is written and
read whenever a Watch() is invoked using the
dummy storage. This commit adds locking in order
to mitigate this.
Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>
Kubernetes-commit: 2593671337ad449f51b9dcc0b63aa190dd07ab68
* Make policy decision object public
Signed-off-by: Max Smythe <smythe@google.com>
* Separate version conversion from validation
Signed-off-by: Max Smythe <smythe@google.com>
* Address review comments
Signed-off-by: Max Smythe <smythe@google.com>
* Fix variable name
Signed-off-by: Max Smythe <smythe@google.com>
---------
Signed-off-by: Max Smythe <smythe@google.com>
Kubernetes-commit: 0ed74145fb00626ce0e900812a54ca3de5406f2e
The --contention-profiling flag enables block profiling by calling
SetBlockProfileRate(). It, however does not call SetMutexProfileFraction
which enables mutex profiling.
Mutex profiling gives us information about lock contention whereas
block profiling on the other hand gives us information on gorotuines
being blocked on sync primitives.
Updating the docs to "block profiling" in order to make it more accurate.
Eventhough a block profile may reflect contention points on locks to some
extent, it also talks about other sources where goroutines may be blocked.
Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>
Kubernetes-commit: f236ab24dfcb84906a1d453877a79bd4a94f897d
reduces memory and cpu when things like configmap are used as a param
cannot be shared due to limitatoins of sharedinformerfactory
Kubernetes-commit: b969dfec9fd33f8bfff47e54f2995a4865839ea6
Each of these scripts is basically identical, and all were too brittle.
Now they should be more resilient and easier to manage. The script
still needs to be updated if we add new ones, which I do not love.
More cleanup to follow.
Kubernetes-commit: e0ecccff3f5148cc167117ac73233b4edc1640d8
The condition methods will eventually all take a context. Since we
have been provided one, alter the accepted condition type and
change the four references in tree.
Collers of ExponentialBackoffWithContext should use a condition
aware function (ConditionWithContextFunc). If the context can be
ignored the helper ConditionFunc.WithContext can be used to convert
an existing function to the new type.
Kubernetes-commit: 34bfdc3635cb621d94eebde5d8f4b9c0b933c68e
Amine