Previously, this test assumed that:
- a global watch would return only an event for the key in question
- only the delete event in question would be returned
Neither of these assumptions are correct for an etcd backend as long
as any other clients are interacting with the system. This commit
makes the watch more specific and extracts the correct event.
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>
Kubernetes-commit: 2631c0a0f959bd67aa455045dce33e77150ab5f8
Some of these changes are cosmetic (repeatedly calling klog.V instead of
reusing the result), others address real issues:
- Logging a message only above a certain verbosity threshold without
recording that verbosity level (if klog.V().Enabled() { klog.Info... }):
this matters when using a logging backend which records the verbosity
level.
- Passing a format string with parameters to a logging function that
doesn't do string formatting.
All of these locations where found by the enhanced logcheck tool from
https://github.com/kubernetes/klog/pull/297.
In some cases it reports false positives, but those can be suppressed with
source code comments.
Kubernetes-commit: edffc700a43e610f641907290a5152ca593bad79
Without these select statements, this test runs until the package-global
timeout and causes a panic. This change makes the test fail faster and
more legibly.
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>
Kubernetes-commit: fc33d0176a5afb81927430d075165152f953c54e
When tests attempt to validate behavior in the case that a client asks
for a resource version that is "too large" for the underlying storage,
the previous implementation would simply add 1 to the latest revision
seen. This is only appropriate for storage backends that
a) provide a continuous monotonic logical clock
b) have no other events occurring while the test runs
For instance, when using a singe etcd backend as a shared fixture for
these tests, adding 1 to a previously-seen revision is not suffcient to
ensure that the resulting revision is "too large". By instead using the
largest possible integer value, we can be certain of this.
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>
Kubernetes-commit: b973cdc57cc6ee57684455cdb76db13a8c82cefa
When the original commit created the lease manager, this comment was
added to set the default test reuse time to 1s. Even at that time, the
comment claimed it was setting 10s. Instead of using this value, though,
new tests that did not call `testSetup()` started to use the default
configuration for production. This commit clarifies the intent of this
comment, moves it next to the code block that it actually applies to,
and makes use of this test-specific logic everywhere.
x-ref: f230b000db
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>
Kubernetes-commit: 6aa37eb06247fb95a6a4ef61cbd50885e52055a0
This commit simply modernizes the comparisons made in the storage tests
to use `cmp.Diff()` so that pointer comparisons and length checks do not
have to be made by hand. We also get nice diffs in the test output this
way instead of large pasted blobs.
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>
Kubernetes-commit: dfdd486f09321e9105fa747a8d1ac5a9a2a7a94a
Modernize the comparisons used in the watch tests to use `cmp.Diff()` for
readability.
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>
Kubernetes-commit: d17a19b39d2dbdaf2cbbaad46de403d6d7ce0602
This was the last test to not use sub-tests, so we can also remove the
indices that the expectation functions take as parameters now.
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>
Kubernetes-commit: 9f7bb4264e0b79cbe7979c09f0e4c75a434a27bb
In this test, the current implementation uses a nebulous "RV 1" for some
queries. The intent of this absolute choice is to probe etcd at a
version before any writes ocurred for the test. The particular test
fixture for etcd that is used starts at revision 1, so 1 is used.
This choice is hard to understand the meaning of for readers, though,
and is not valid for any other etcd fixture used for the tests. In order
to improve readability of the test as well as to make it more resilient
to the underlying store, this change updates the test to read the
revision of the underlying storage before making any writes and using
that revision when querying the storage in the tests.
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>
Kubernetes-commit: d2b42b6369ab8db9d0aa0b58dcdf6548ff489d70
This test, as written, is *extremely* cryptic and hard to parse. Add a
comment and stop intentionally ignoring an error that only needs to be
ignored if we're being cryptic.
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>
Kubernetes-commit: 50eed81923495f5ee1ac44436676ddbaf2a380fe
When an envelope transformer calls out to KMS (for instance), it will be
very helpful to pass a `context.Context` to allow for cancellation. This
patch does that, while passing the previously-expected additional data
via a context value.
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>
Kubernetes-commit: 27312feb9983c18d1daf00afba788727d024cdd0
This test case was a duplicate of the previous one.
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>
Kubernetes-commit: 921e7525c074750a47818fdf89a4fe5c0b058f0f
* Remove linter warnings.
* Cancel contexts to avoid leaks.
* Rename a few XXXThreadUnsafe to XXXLocked to
maintain consistency.
* A few are still called XXXThreadUnsafe mainly
because those are safe to be called from the
perspective that only one gorotuine will access
them - not really called under a lock.
Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>
Kubernetes-commit: c3081b48759db1f05a446f2acca7e05c4511ce2e
- Modify GetAllEventsSinceThreadUnsafe to return a watchCacheInterval
- Modify Watch() to compute a watchCacheInterval rather than a slice
of all "initEvents" and pass this interval to process()
- Use interval::Next() to obtain events to process rather than obtain
them all at once
- Modify tests accordingly to use interval
- On invalidation, stop processing and stop the watch.
- Make indexValidator injectable for testing
- Add unit test for verifying the behaviour of stopping the watch.
Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>
Kubernetes-commit: 7f2aa7ad3a61a52d0a780f904b291d063399c28a
watchCacheInterval serves as an abstraction over a source
of watchCacheEvents. It maintains a window of events over
an underlying source and these events can be served using
the exposed Next() API. The main intent for doing things
this way is to introduce an upper bound of memory usage
for starting a watch and reduce the maximum possible time
interval for which the lock would be held while events are
copied over.
The source of events for the interval is typically either
the watchCache circular buffer, if events being retrieved
need to be for resource versions > 0 or the underlying
implementation of Store, if resource version = 0.
Furthermore, an interval can be either valid or invalid at
any given point of time. The notion of validity makes sense
only in cases where the window of events in the underlying
source can change over time - i.e. for watchCache circular
buffer. When the circular buffer is full and an event needs
to be popped off, watchCache::startIndex is incremented. In
this case, an interval tracking that popped event is valid
only if it has already been copied to its internal buffer.
However, for efficiency we perform that lazily and we mark
an interval as invalid iff we need to copy events from the
watchCache and we end up needing events that have already
been popped off. This translates to the following condition:
watchCacheInterval::startIndex >= watchCache::startIndex.
When this condition becomes false, the interval is no longer
valid and should not be used to retrieve and serve elements
from the underlying source.
Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>
Kubernetes-commit: 347607e97139959f33024a691d0561b1479aeeef
In the following code pattern, the log message will get logged with v=0 in JSON
output although conceptually it has a higher verbosity:
if klog.V(5).Enabled() {
klog.Info("hello world")
}
Having the actual verbosity in the JSON output is relevant, for example for
filtering out only the important info messages. The solution is to use
klog.V(5).Info or something similar.
Whether the outer if is necessary at all depends on how complex the parameters
are. The return value of klog.V can be captured in a variable and be used
multiple times to avoid the overhead for that function call and to avoid
repeating the verbosity level.
Kubernetes-commit: 9eaa2dc554e0c3d4485d4c916dfdbc2f517db2e0
Split process() function into processEvents() and process().
This is done in anticipation of GetAllEventsSinceThreadUnsafe()
returning an entity using which events can be constructed and
not the events itself.
Subsequently, this commit also moves updating resource version
for initEvents from Watch() to the processEvents() func.
Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>
Kubernetes-commit: aab7cd3d8a66f425022ca5b2a2bd0d3019efe526
This PR enables unaryClientInterceptor in conjunction with Prometheus interceptor.
Previously it was simply overwritten by the Prometheus interceptor.
As a result etcd client didn't attempt to retry certain errors.
The unaryClientInterceptor is important because it knows how to retry all sorts of errors from the etcd cluster. It will make the API server more resilient to failures - end users won't see certain errors.
The full list of retriable (codes.Unavailable) errors can be found at https://github.com/etcd-io/etcd/blob/main/api/v3rpc/rpctypes/error.go#L72
Kubernetes-commit: 83171562b0954b2e19eb69943f01a44779cc7a8f
Steve Kuznetsov