Doing this allows us to implement some more nuanced cacher manipulations
to be used in testing. For ex: implementing a test-only compaction method
for the watch cache.
Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>
Kubernetes-commit: 6d66fbc6b670f1120a9041873bb8d1a0655bbefc
This commit prepares for when cacher tests are moved here
from the `tests` package. Tests in that package redeclare
some of the testing utils that exist here, so dedup-ing them.
This commit also adapts to any changes in test util signatures.
There are still some utils that can be reused but currently are
highly specific to some tests. (ex: watch_cache_test.go)
Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>
Kubernetes-commit: 70978e4af619819787a4eb544ffd732aa7954d76
Since cachingObject has the encoded data cached and they are not
supposed to change. It's memory efficient to just copy the slice
references.
Signed-off-by: Eric Lin <exlin@google.com>
Kubernetes-commit: 3085b57869a2a7bf5290ab97facaf17fedfa88a0
There exists a storage test to test for rv=0 and production
of ADDED events. This commit adapts the test to be used for
the watch cache as well.
Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>
Kubernetes-commit: 4d85a1f00cb0f1350cf8495925be0e8bfed59a15
If the cacher hasn't seen any event (when lastProcessedResourceVersion is zero) and
the bookmarkTimer has ticked then we shouldn't popExpiredWatchers. This is
because the watchers wont' be re-added and will miss future bookmark events when
the cacher finally receives an event via the c.incoming chan.
Kubernetes-commit: 6db4cbfde7babfb34f5cd1059c769ec2d870f12a
* cacher: remove locking from watcherBookmarkTimeBuckets
it turns out that the watcherBookmarkTimeBuckets
is called from only three places/methods: startDispatching, finishDispatching and Watch.
All these methods acquire c.Lock() before touching watcherBookmarkTimeBuckets.
Thus we could remove explicit locking in
watcherBookmarkTimeBuckets since the access is already synced.
* cacher: rename watcherBookmarkTimeBuckets methods to indicate that proper synchronisation must be used
Kubernetes-commit: eab66a687b282266f0520b79166f7f55828ffd28
waitUntilWatchCacheFreshAndForceAllEvents must be called without
a read lock held otherwise the watchcache won't be able to make
progress (i.e. the watchCache.processEvent method that requries acquiring an exclusive lock)
the deadlock can happen only when the alpha watchlist feature flag is on
and the client specifically requests streaming.
Kubernetes-commit: 476e407ffd2ab393840d3f7a9fd01b71698738a3
* ftr(watch-cache): add benchmarks
* ftr(kube-apiserver): faster watch-cache getlist
* refine: testcase name
* - refine var name make it easier to convey meaning
- add comment to explain why we need to apply for a slice of runtime.Object instead of making a slice of ListObject.Items directly.
Kubernetes-commit: 75f17eb38fc8bbcb360d43dffce6e27a7159d43f
Prior to this change, we wait until the DEK is used to perform an
encryption before validating the response. This means that the
plugin could report healthy but all TransformToStorage calls would
fail. Now we correctly cause the plugin to become unhealthy and do
not attempt to use the newly generated DEK.
Signed-off-by: Monis Khan <mok@microsoft.com>
Kubernetes-commit: 5469c198e5d074c7e88e14c3dcbc3ebb2b37cfa8
It is possible for a KMSv2 plugin to return a static value as Ciphertext
and store the actual encrypted DEK in the annotations. In this case,
using the encDEK will not work. Instead, we are now using a combination
of the encDEK, keyID and annotations to generate the cache key.
Signed-off-by: Anish Ramasekar <anish.ramasekar@gmail.com>
Kubernetes-commit: 8eacf09649ac9042c7e998b5c24ac59d68ae7e6c
This change updates KMS v2 to not create a new DEK for every
encryption. Instead, we re-use the DEK while the key ID is stable.
Specifically:
We no longer use a random 12 byte nonce per encryption. Instead, we
use both a random 4 byte nonce and an 8 byte nonce set via an atomic
counter. Since each DEK is randomly generated and never re-used,
the combination of DEK and counter are always unique. Thus there
can never be a nonce collision. AES GCM strongly encourages the use
of a 12 byte nonce, hence the additional 4 byte random nonce. We
could leave those 4 bytes set to all zeros, but there is no harm in
setting them to random data (it may help in some edge cases such as
live VM migration).
If the plugin is not healthy, the last DEK will be used for
encryption for up to three minutes (there is no difference on the
behavior of reads which have always used the DEK cache). This will
reduce the impact of a short plugin outage while making it easy to
perform storage migration after a key ID change (i.e. simply wait
ten minutes after the key ID change before starting the migration).
The DEK rotation cycle is performed in sync with the KMS v2 status
poll thus we always have the correct information to determine if a
read is stale in regards to storage migration.
Signed-off-by: Monis Khan <mok@microsoft.com>
Kubernetes-commit: 832d6f0e19f13b9dd22b1fe9d705817e9e64f4f1
this method waits until cache is at least
as fresh as given requestedWatchRV if sendInitialEvents was requested.
Additionally, it instructs the caller whether it should ask for
all events from the cache (full state) or not.
Kubernetes-commit: 21fb98105043d1a15ef48089ef231931851d2d15
This change helps users understand the state of their encryption
config if storage migration is not consistently run with key ID
rotation.
Signed-off-by: Monis Khan <mok@microsoft.com>
Kubernetes-commit: 539f734bfd0d828a5b3dfbfe30df9d4229e6492c
if old less than new, Inc function should be called for `watchCacheCapacityIncreaseTotal` instead of `watchCacheCapacity`
Signed-off-by: joey <zchengjoey@gmail.com>
Kubernetes-commit: 96b9531f3e3f489e47493297987eee14d2a08855
* cacher allow context cancellation if not ready
Replace the sync.Cond variable with a channel so we can use the
context cancellation signal.
Co-authored-by: Wojciech Tyczy<C5><84>ski <wojtekt@google.com>
Change-Id: I2f75313a6337feee440ece4c1e873c32a12560dd
* wait again on pending state
Change-Id: I1ad79253a5a5d56a4d9611125825b1f7ad552be8
---------
Co-authored-by: Wojciech Tyczy<C5><84>ski <wojtekt@google.com>
Kubernetes-commit: 3b17aece1fa492e98aa82b948597b3641961195f
This commit extends the test cases of RunTestList
to include the things tested by TestListDeprecated
and subsequently deletes the test.
This additionally adds a test case for checking that
the list return the modified version of an object.
Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>
Kubernetes-commit: 7474d9b719619caa356dfb71a06482f33b3ed1ce
The original design was to honour strong consistency
semantics for when the RV is unset, i.e. serve the
watch by doing a quorum read.
However, the implementation did not match the intent,
in that, the Cacher did not distinguish between set
and unset RV. This commit rectifies that behaviour by
serving the watch from the underlying storage if the
RV is unset.
This commit subsequently also adds a test for the same.
Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>
Kubernetes-commit: 610b67031c79c6c38964631d27dd59df357c6d2e
This commit allows injecting errors for the
Watch() method of the dummy storage impl.
As a consequence of this, a race is introduced
between when the injected error is written and
read whenever a Watch() is invoked using the
dummy storage. This commit adds locking in order
to mitigate this.
Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>
Kubernetes-commit: 2593671337ad449f51b9dcc0b63aa190dd07ab68
Each of these scripts is basically identical, and all were too brittle.
Now they should be more resilient and easier to manage. The script
still needs to be updated if we add new ones, which I do not love.
More cleanup to follow.
Kubernetes-commit: e0ecccff3f5148cc167117ac73233b4edc1640d8
This change updates the API server code to load the encryption
config once at start up instead of multiple times. Previously the
code would set up the storage transformers and the etcd healthz
checks in separate parse steps. This is problematic for KMS v2 key
ID based staleness checks which need to be able to assert that the
API server has a single view into the KMS plugin's current key ID.
Signed-off-by: Monis Khan <mok@microsoft.com>
Kubernetes-commit: f507bc255382b2e2095351053bc17e74f7100d35
- Moves kms proto apis to the staging repo
- Updates generate and verify kms proto scripts to check staging repo
Signed-off-by: Anish Ramasekar <anish.ramasekar@gmail.com>
Kubernetes-commit: c3794e2377016b1c18b1dcb63dc61d686c8ebcbf
return the last request error, instead of last error received
The rate limit allows 1 event per healthcheck timeout / 2
Kubernetes-commit: 510a85c53a5138babb1650fadd328e6f34baa03b
Use GroupResource instead of object reflection when recording the
following metrics:
- etcd_request_duration_seconds
- etcd_bookmark_counts
Add GroupResource to logs and traces where only reflection-based typing
was previously used.
Both of these changes allow us to disginguish between different CRDs,
all of which are represented as *unstructured.Unstructured.
Signed-off-by: Andy Goldstein <andy.goldstein@redhat.com>
Kubernetes-commit: 305fa2add60ad507417304d865f001006d5175fe
Use the group resource instead of objectType in watch cache metrics,
because all CustomResources are grouped together as
*unstructured.Unstructured, instead of 1 entry per type.
Signed-off-by: Andy Goldstein <andy.goldstein@redhat.com>
Kubernetes-commit: d08b69e8d35a5aa73a178c508f9b0e1ad74b882d
All CustomResources are treated as *unstructured.Unstructured, leading
the watch cache to log anything related to CRs as Unstructured. This
change uses the schema.GroupResource instead of object type for all type
related log messages in the watch cache, resulting in distinct output
for each CR type.
Signed-off-by: Andy Goldstein <andy.goldstein@redhat.com>
Kubernetes-commit: 397533a4c2df9639ff4422c907d06fae195a1835
- add feature gate
- add encrypted object and run generated_files
- generate protobuf for encrypted object and add unit tests
- move parse endpoint to util and refactor
- refactor interface and remove unused interceptor
- add protobuf generate to update-generated-kms.sh
- add integration tests
- add defaulting for apiVersion in kmsConfiguration
- handle v1/v2 and default in encryption config parsing
- move metrics to own pkg and reuse for v2
- use Marshal and Unmarshal instead of serializer
- add context for all service methods
- check version and keyid for healthz
Signed-off-by: Anish Ramasekar <anish.ramasekar@gmail.com>
Kubernetes-commit: f19f3f409938ff9ac8a61966e47fbe9c6075ec90
Replicated from https://github.com/etcd-io/etcd/blob/v3.5.4/client/v3/logger.go#L47
The logic of this function doesn't make a lot of sense to me, but
copying it will avoid any behaviour change.
Signed-off-by: Nic Cope <nicc@rk0n.org>
Kubernetes-commit: c1aa7a0fe73cbcab8e70f7b73a845ae9394f9a71
Currently the API server creates one etcd client per CRD. If clients
aren't provided a logger they'll each create their own. These loggers
can account for ~20% of API server memory consumption on a cluster with
hundreds of CRDs.
Signed-off-by: Nic Cope <nicc@rk0n.org>
Kubernetes-commit: 0c81eabb853e581abbcb37ebf094af3316e1012e
This logger is responsible for 20% of the API server's memory usage when
many CRDs are installed. See the below issue for more context.
https://github.com/kubernetes/kubernetes/issues/111476
Signed-off-by: Nic Cope <nicc@rk0n.org>
Kubernetes-commit: 0e5401c93940126beac45264aa056507b0950075
- Run hack/update-codegen.sh
- Run hack/update-generated-device-plugin.sh
- Run hack/update-generated-protobuf.sh
- Run hack/update-generated-runtime.sh
- Run hack/update-generated-swagger-docs.sh
- Run hack/update-openapi-spec.sh
- Run hack/update-gofmt.sh
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
Kubernetes-commit: a9593d634c6a053848413e600dadbf974627515f
v1.43.0 marked grpc.WithInsecure() deprecated so this commit moves to use
what is the recommended replacement:
grpc.WithTransportCredentials(insecure.NewCredentials())
Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
Kubernetes-commit: 2c8bfad9106039aa15233b5bf7282b25a7b7e0a0
expiredBookmarkWatchers allows us to schedule the next bookmark event after dispatching not before as it was previously.
It opens a new functionality in which a watcher might decide to change when the next bookmark should be delivered based on some internal state.
Kubernetes-commit: 0576f6a011cba8f0c8550fd3dd31111376c9dcd0
Using a Pod type in a GetList() call in a test
can panic at worst and error out at best. Here,
neither happened because the error condition
being tested for (cacher being stopped or not)
gets returned before the list pointer can be
enforced.
This commit changes the above to use PodList.
Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>
Kubernetes-commit: 487761f4e2543114db158f0d59e598dedc481882
Our tests are mostly error based and explicit error typing allows
us to test against error types directly. Having made this change also
makes it obvious that our test coverage was lacking in two branches,
specifically, we were previously not testing empty start keys nor were
we testing for invalid start RVs.
Kubernetes-commit: 213e380a2e48830db6c71d2da5485d4226d95625
Marek Siarkowicz
Madhav Jivrajani
Dr. Stefan Schimanski
Eric Lin
Monis Khan
Anish Ramasekar
Daniel Smith
Lukasz Szaszkiewicz
kkkkun
Wojciech Tyczyński
Tim Hockin
iyear
Rita Zhang
scott
Jordan Liggitt
Patrick Ohly
joey
Antonio Ojea
ZhangYu
baomingwang
Krzysztof Ostrowski
sanwishe
Tim Allclair
David Ashpole
Han Kang
Oscar Utbult
lixiaobing1
Harsha Narayana
viveksahu26
Antonio Ojea
Andy Goldstein
Sally O'Malley
Nic Cope
Marcel Zięba
Maciej Wyrzuc
Davanum Srinivas
Mikko Ylinen
Artur Żyliński
Abirdcfly