It allows us to allocate a single buffer for the entire watch session and release it when a watch connection is closed.
Previously memory was allocated for every object serialization putting a lot of pressure on GC and consuming more memory than needed.
Kubernetes-commit: eda1b0c68ec166ee52c50e4a6ab682ce7227b6a5
Modernize the comparisons used in the watch tests to use `cmp.Diff()` for
readability.
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>
Kubernetes-commit: d17a19b39d2dbdaf2cbbaad46de403d6d7ce0602
This was the last test to not use sub-tests, so we can also remove the
indices that the expectation functions take as parameters now.
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>
Kubernetes-commit: 9f7bb4264e0b79cbe7979c09f0e4c75a434a27bb
In this test, the current implementation uses a nebulous "RV 1" for some
queries. The intent of this absolute choice is to probe etcd at a
version before any writes ocurred for the test. The particular test
fixture for etcd that is used starts at revision 1, so 1 is used.
This choice is hard to understand the meaning of for readers, though,
and is not valid for any other etcd fixture used for the tests. In order
to improve readability of the test as well as to make it more resilient
to the underlying store, this change updates the test to read the
revision of the underlying storage before making any writes and using
that revision when querying the storage in the tests.
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>
Kubernetes-commit: d2b42b6369ab8db9d0aa0b58dcdf6548ff489d70
This test, as written, is *extremely* cryptic and hard to parse. Add a
comment and stop intentionally ignoring an error that only needs to be
ignored if we're being cryptic.
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>
Kubernetes-commit: 50eed81923495f5ee1ac44436676ddbaf2a380fe
When an envelope transformer calls out to KMS (for instance), it will be
very helpful to pass a `context.Context` to allow for cancellation. This
patch does that, while passing the previously-expected additional data
via a context value.
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>
Kubernetes-commit: 27312feb9983c18d1daf00afba788727d024cdd0
This test case was a duplicate of the previous one.
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>
Kubernetes-commit: 921e7525c074750a47818fdf89a4fe5c0b058f0f
Previously, if a request is clusterscoped, the context that
was returned has no namespace, ideally the context should
contain a metav1.NamespaceNone as the namespace even for
cluster scoped requests.
Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>
Kubernetes-commit: 412626f4024b1acdd856c9047f97d387ee40f884
* Remove linter warnings.
* Cancel contexts to avoid leaks.
* Rename a few XXXThreadUnsafe to XXXLocked to
maintain consistency.
* A few are still called XXXThreadUnsafe mainly
because those are safe to be called from the
perspective that only one gorotuine will access
them - not really called under a lock.
Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>
Kubernetes-commit: c3081b48759db1f05a446f2acca7e05c4511ce2e
- Modify GetAllEventsSinceThreadUnsafe to return a watchCacheInterval
- Modify Watch() to compute a watchCacheInterval rather than a slice
of all "initEvents" and pass this interval to process()
- Use interval::Next() to obtain events to process rather than obtain
them all at once
- Modify tests accordingly to use interval
- On invalidation, stop processing and stop the watch.
- Make indexValidator injectable for testing
- Add unit test for verifying the behaviour of stopping the watch.
Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>
Kubernetes-commit: 7f2aa7ad3a61a52d0a780f904b291d063399c28a
This reverts commit 6faa4f001008a5a29476f5722f66430c35f48229, reversing
changes made to 33a2c50bce334467640e016f68cf19e9382ba1a7.
Kubernetes-commit: 8fb33338635565f2f755a4557b94c26039c175d9
watchCacheInterval serves as an abstraction over a source
of watchCacheEvents. It maintains a window of events over
an underlying source and these events can be served using
the exposed Next() API. The main intent for doing things
this way is to introduce an upper bound of memory usage
for starting a watch and reduce the maximum possible time
interval for which the lock would be held while events are
copied over.
The source of events for the interval is typically either
the watchCache circular buffer, if events being retrieved
need to be for resource versions > 0 or the underlying
implementation of Store, if resource version = 0.
Furthermore, an interval can be either valid or invalid at
any given point of time. The notion of validity makes sense
only in cases where the window of events in the underlying
source can change over time - i.e. for watchCache circular
buffer. When the circular buffer is full and an event needs
to be popped off, watchCache::startIndex is incremented. In
this case, an interval tracking that popped event is valid
only if it has already been copied to its internal buffer.
However, for efficiency we perform that lazily and we mark
an interval as invalid iff we need to copy events from the
watchCache and we end up needing events that have already
been popped off. This translates to the following condition:
watchCacheInterval::startIndex >= watchCache::startIndex.
When this condition becomes false, the interval is no longer
valid and should not be used to retrieve and serve elements
from the underlying source.
Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>
Kubernetes-commit: 347607e97139959f33024a691d0561b1479aeeef
In the following code pattern, the log message will get logged with v=0 in JSON
output although conceptually it has a higher verbosity:
if klog.V(5).Enabled() {
klog.Info("hello world")
}
Having the actual verbosity in the JSON output is relevant, for example for
filtering out only the important info messages. The solution is to use
klog.V(5).Info or something similar.
Whether the outer if is necessary at all depends on how complex the parameters
are. The return value of klog.V can be captured in a variable and be used
multiple times to avoid the overhead for that function call and to avoid
repeating the verbosity level.
Kubernetes-commit: 9eaa2dc554e0c3d4485d4c916dfdbc2f517db2e0
The effective layering of ResponseWriters is today, from outside to
inside, httplog(timeout(audit(metrics(original)))). From
6e3fd91e1aa3259d7bd67e0a65693e346ade347d, calls to http.Error in the
apiserver's root healthz handler use an unwrapped ResponseWriter --
effectively timeout(audit(metrics(original))) -- to avoid logging
stack traces for those requests.
From 0d50c969c587c8a6c16e0962118305ac652c5a6b, the same call to
http.Error receives a completely-unwrapped ResponseWriter. This has
the effect of bypassing not only the httplog wrapper, but also
timeout, audit, and metrics. The timeout wrapper defends against
the (disallowed) use of underyling ResponseWriter after the completion
of its request's ServeHTTP call. Since that defensive behavior is
being bypassed, it's possible for the root healthz handler to panic
when health probes time out.
Instead of continuing to use a wrapper-aware means of disabling stack
traces, this commit adds a new function to httplog that allows
customization of the stack trace logging predicate on a per-request
basis.
Kubernetes-commit: ff849fe8b688606d5173d5ee0213a96cffae23c0
In 18177e2bdeafbddeb3d66fec0b8cb88794cd69ff, PostGoRestful field has
been renamed to NonGoRestfulMux, but the documentation change did not
follow.
This commit fixes that to avoid potential confusion.
Signed-off-by: Mateusz Gozdek <mgozdek@microsoft.com>
Kubernetes-commit: 36ede8de945adcc06bde21c49dc157e9c741e0d2
This PR removes Serve function and uses all required places
ServeWithListenerStopped which takes place new Serve function.
This function returns ListenerStopped channel can be used to drain
requests before shutting down the server.
Kubernetes-commit: a8d2b3a7926394b1c53621804cdeb93e4a61b7c8
Print information from both the original path registration and the new
path registration stack traces when encountering a duplicate. This helps
the developer determine where the duplication is coming from and makes
it much easier to resolve.
Signed-off-by: Andy Goldstein <andy.goldstein@redhat.com>
Kubernetes-commit: 04aa8f9dcdbc575fde37e25e45315359b0aa1ca6
Implements server side field validation behind the
`ServerSideFieldValidation` feature gate. With the
feature enabled, any create/update/patch request
with the `fieldValidation` query param set to
"Strict" will error if the object in the request
body have unknown fields. A value of "Warn"
(also the default when the feautre is enabled)
will succeed the request with a warning.
When the feature is disabled (or the query param
has a value of "Ignore"), the request will succeed
as it previously had with no indications of any
unknown or duplicate fields.
Kubernetes-commit: e50e2bbc889eb274ad1463a54188a2805767bfde
addresses feedback by adding a test to make sure that the manager pipeline will not return the same instance used as input, and that the output does not input managedFields
Kubernetes-commit: 68e175ef68eff1351917ac206fb29abe42128062
David Eads