Commit Graph

108 Commits

Author SHA1 Message Date
Madhav Jivrajani db00da87d8 cacher: Fix watch behaviour for unset RV
The original design was to honour strong consistency
semantics for when the RV is unset, i.e. serve the
watch by doing a quorum read.

However, the implementation did not match the intent,
in that, the Cacher did not distinguish between set
and unset RV. This commit rectifies that behaviour by
serving the watch from the underlying storage if the
RV is unset.

This commit subsequently also adds a test for the same.

Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>

Kubernetes-commit: 610b67031c79c6c38964631d27dd59df357c6d2e
2023-01-16 11:10:05 +05:30
Wojciech Tyczyński 299c158ba3 Reuse generic List test for watchcache
Kubernetes-commit: ef8c4fbca8e5bed1e7edc162b95c412a7f1a758e
2022-11-04 22:01:37 +01:00
Wojciech Tyczyński 8cd19efa73 Fix inconsistency in returning list results both in etcd3 and watchcache
Kubernetes-commit: da8d197a7205c01fdf376246995b0a1b3ab53859
2022-11-08 11:47:28 +01:00
David Ashpole aa161f2fc0 migrate apiserver utiltrace usage to component-base/tracing
Kubernetes-commit: de26b9023f2872c5cd7e15fad5dd5ab649222c13
2022-10-20 18:15:38 +00:00
Tim Allclair bd7c7f52c2 Consolidate AuditContext
Kubernetes-commit: f1d684b7b60b39b7dc1eb4156307c593f0ba74e1
2022-07-12 11:53:57 -07:00
viveksahu26 6cb2fda0ab fix warnings or linter errors
Signed-off-by: viveksahu26 <vivekkumarsahu650@gmail.com>

Kubernetes-commit: 386bc4a7fa3e975cb247d300fdc5ad1b14a15605
2022-09-05 19:10:43 +05:30
Andy Goldstein 7eb011f596 watch cache: metrics: objectType -> group resource
Use the group resource instead of objectType in watch cache metrics,
because all CustomResources are grouped together as
*unstructured.Unstructured, instead of 1 entry per type.

Signed-off-by: Andy Goldstein <andy.goldstein@redhat.com>

Kubernetes-commit: d08b69e8d35a5aa73a178c508f9b0e1ad74b882d
2022-08-11 15:51:21 -04:00
Andy Goldstein ecf3a57374 watch cache: log GroupResource, not objectType
All CustomResources are treated as *unstructured.Unstructured, leading
the watch cache to log anything related to CRs as Unstructured. This
change uses the schema.GroupResource instead of object type for all type
related log messages in the watch cache, resulting in distinct output
for each CR type.

Signed-off-by: Andy Goldstein <andy.goldstein@redhat.com>

Kubernetes-commit: 397533a4c2df9639ff4422c907d06fae195a1835
2022-08-11 15:45:04 -04:00
Lukasz Szaszkiewicz 684cf11215 cacher: add support for consistent streaming
design details https://github.com/kubernetes/enhancements/tree/master/keps/sig-api-machinery/3157-watch-list#design-details

Kubernetes-commit: debace151cfffc1b5d805341efcbf5f0062e95ed
2022-07-05 15:48:30 +02:00
Marcel Zięba 3c024b4916 Add option to retry internal api error in reflector.
Kubernetes-commit: 0b2b6489de8f75d5299f54180617601126bb8878
2022-07-25 08:02:54 +00:00
Davanum Srinivas 7e94033a61 Generate and format files
- Run hack/update-codegen.sh
- Run hack/update-generated-device-plugin.sh
- Run hack/update-generated-protobuf.sh
- Run hack/update-generated-runtime.sh
- Run hack/update-generated-swagger-docs.sh
- Run hack/update-openapi-spec.sh
- Run hack/update-gofmt.sh

Signed-off-by: Davanum Srinivas <davanum@gmail.com>

Kubernetes-commit: a9593d634c6a053848413e600dadbf974627515f
2022-07-19 20:54:13 -04:00
Wojciech Tyczyński 2049cfbb9e Adjust watch channel sizes in watchcache
Kubernetes-commit: 0db5c05bdb8bbc510307a48cbade712583bb009e
2022-04-28 11:56:41 +02:00
Lukasz Szaszkiewicz 27a7c443bd cacher: with expiredBookmarkWatchers
expiredBookmarkWatchers allows us to schedule the next bookmark event after dispatching not before as it was previously.
It opens a new functionality in which a watcher might decide to change when the next bookmark should be delivered based on some internal state.

Kubernetes-commit: 0576f6a011cba8f0c8550fd3dd31111376c9dcd0
2022-07-15 15:28:50 +02:00
Artur Żyliński e34c622d49 Add audit-id to storage traces
Refactor GetAuditIDTruncated to use context instead of request

Kubernetes-commit: b1e12b01b6c578da3eb593805b48e9d4a69efe54
2022-06-20 17:09:32 +02:00
Antonio Ojea 00730c9649 apiserver cacher: don't accept requests if stopped
The cacher blocks requests until it is ready, however, the
ready variable doesn't differentiate if the cacher was stopped.

The cacher is using a condition variable based on sync.Cond to
handle the readiness, however, this was not taking into account
if it was not ready because it was waiting to be ready or it was
stopped.

Add a new condition to the condition variable to handle the
stop condition, and returning an error to signal the goroutines
that they should stop waiting and bail out.

Kubernetes-commit: 2cb3a56e83ae33464edb174b1b6373ba50600759
2022-03-03 12:01:59 +01:00
Han Kang 24d28aad81 refactor watch cache metrics so that they are in a single place
Change-Id: I7874e90d4ec111e82450273f27f8e50556cb096a

Kubernetes-commit: a569829f599acf7ac3acbd3902f41d2c74fc2bd7
2022-04-22 14:21:25 -07:00
Sanskar Jaiswal d1bd5277fb Update comment and declaration of `storage.GuaranteedUpdate` to be clearer.
Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>

Kubernetes-commit: f8df26ae803103f82edbf1efe17b1b169801b256
2022-04-16 17:19:06 +05:30
Wojciech Tyczyński 58b4aa562f Fix potential race in dispatching watch event
Kubernetes-commit: 779f157ecfb24d0ee944f18e481bfa8cc8c94f6f
2022-02-22 17:14:15 +01:00
Wojciech Tyczyński 0b8c9f5ebf Avoid perfoming deep copies in watchcache if not needed
Kubernetes-commit: 0cb3a02bbede2f547ddd20e23e2bbc782ac3956c
2022-02-21 15:18:25 +01:00
Wojciech Tyczyński b028a2ea8d Unify merge GetToList and List storage methods to single GetList
Kubernetes-commit: f3d56369f71f5106eeec8300e77bd69b172733da
2022-02-10 10:15:29 +01:00
Wojciech Tyczyński 9f8f72fea9 Change storage.Interface to utilize recurisve parameter
Kubernetes-commit: 3016b3d8f868a3041c7cb86695de09d7ab27cf3b
2022-02-10 09:58:31 +01:00
Madhav Jivrajani ffc1b941d4 cacher: Minor cleanup and refactor of code and tests
* Remove linter warnings.
* Cancel contexts to avoid leaks.
* Rename a few XXXThreadUnsafe to XXXLocked to
  maintain consistency.
  * A few are still called XXXThreadUnsafe mainly
    because those are safe to be called from the
    perspective that only one gorotuine will access
    them - not really called under a lock.

Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>

Kubernetes-commit: c3081b48759db1f05a446f2acca7e05c4511ce2e
2022-02-10 13:44:34 +05:30
Madhav Jivrajani 4f2ee5cf42 cacher: Use watchCacheInterval to reduce lock contention
- Modify GetAllEventsSinceThreadUnsafe to return a watchCacheInterval
- Modify Watch() to compute a watchCacheInterval rather than a slice
  of all "initEvents" and pass this interval to process()
- Use interval::Next() to obtain events to process rather than obtain
  them all at once
- Modify tests accordingly to use interval
- On invalidation, stop processing and stop the watch.
- Make indexValidator injectable for testing
- Add unit test for verifying the behaviour of stopping the watch.

Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>

Kubernetes-commit: 7f2aa7ad3a61a52d0a780f904b291d063399c28a
2021-10-05 15:52:09 +05:30
Wojciech Tyczyński 067689be43 Graduate EfficientWatchResumption feature to GA
Kubernetes-commit: b7257a4d6a3f2382f4753322ec6d3458410b04f9
2021-12-10 10:20:51 +01:00
Wojciech Tyczyński b4935b065f Slightly improve reliability of storage List tests.
Kubernetes-commit: 8ca6c5922b9a0bafa8c83617d6930b981430334e
2021-12-10 12:05:32 +01:00
Wojciech Tyczyński 0002a597cd Unify List and GetToList functions of cacher.
Kubernetes-commit: c86543d9cb302859510604256fdb81ea1b0d46fb
2021-11-18 16:41:36 +01:00
Wojciech Tyczyński 498feb7a9e Add watchcache metrics to tracking its progress
Kubernetes-commit: ba5e08223d3c199a8efb69248728d41e71d70cd0
2021-11-30 11:12:54 +01:00
Wojciech Tyczyński 71bfcaaea1 Simplify storage.Interface by merging Watch and WatchList functions.
Kubernetes-commit: 53653e9b0a7145c3b6e133c5639194ff79b3719d
2021-11-18 12:40:26 +01:00
Madhav Jivrajani af19f84a96 cacher: refactor process() function
Split process() function into processEvents() and process().
This is done in anticipation of GetAllEventsSinceThreadUnsafe()
returning an entity using which events can be constructed and
not the events itself.

Subsequently, this commit also moves updating resource version
for initEvents from Watch() to the processEvents() func.

Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>

Kubernetes-commit: aab7cd3d8a66f425022ca5b2a2bd0d3019efe526
2021-10-29 14:41:10 +05:30
Dr. Stefan Schimanski c0ff3feb85 apiserver/watchcache: doc watch event fan-out to blocked watchers better
Kubernetes-commit: 94b2bb1d3dcfe7973329cc56024f449adde4086c
2021-10-08 17:05:27 +02:00
Mike Spreitzer 889106eff7 Add metrics about handling LIST requests
Add metrics that illuminate the costs of handling LIST requests.

Kubernetes-commit: bf424292db185546c474b27b963fd27bdbafc72f
2021-09-01 18:18:23 -04:00
wojtekt b898581360 Migrate to k8s.io/utils/clock in apiserver
Kubernetes-commit: 859a98c0358610e2c127cd2fba1be601ca975188
2021-09-14 20:36:07 +02:00
Abu Kashem bd2ef9810a apf: estimate list width
Kubernetes-commit: 296c18ec323328bf1c6a621a9ca3a094d78bb9d0
2021-06-22 19:38:00 -04:00
Antonio Ojea 42a99cd0a2 decouple timeBudget from real clock
instead of using a goroutine refreshing the budget, obtain
the value from the last time the budget was accessed.

Kubernetes-commit: dd2c38306000eeb1720afc8346165a6caab09259
2021-07-20 13:54:59 +02:00
wojtekt cd8b524e3f Avoid code duplication in watchcache
Kubernetes-commit: 2df05df6982df93e4ced37d1236f0366069d5838
2021-07-07 09:40:58 +02:00
wojtekt 5290a24345 Implement support for watch initialization in P&F
Kubernetes-commit: 0cc217647ca8be0820973b970124a072c27b6575
2021-05-07 12:49:06 +02:00
Maciej Borsz 5d1582cdd6 Print queue lengths on watch termination
Kubernetes-commit: 8f8beeabb7deb04624c70444012ebc8ba59d990f
2021-01-20 09:32:00 +01:00
Maciej Borsz 8313c177f0 Improve logging in cacheWatcher
Kubernetes-commit: b78f0b31063b7d47781a1ce9ee4ed6c118fb949f
2021-01-11 12:33:52 +01:00
Maciej Borsz 255da953aa Add 'apiserver_terminated_watchers_total'
Kubernetes-commit: 342ad48feb2629ebc832c0b85c52b4eb04d81498
2021-01-08 10:55:44 +01:00
wojtekt 3de94d4883 Unify variable naming between GuaranteedUpdate and Delete in storage
Kubernetes-commit: d0726e4b1354b1c8c3978b96ab7b01d13a2b6340
2020-12-16 09:28:44 +01:00
wojtekt ddcc69d4f7 Use current state from watchcache to avoid etcd get for deletions
Kubernetes-commit: 7bab6a9c6ed7d4e54da3e010a7bf0a30786d28f9
2020-11-03 17:30:26 +01:00
wojtekt 8c25869e43 Add suggestion to storage interface Delete method
Kubernetes-commit: c2d61896f45967cdea15072b7a12faeb635b5dc5
2020-11-03 13:21:00 +01:00
chenyw1990 e6daad296e Fix cacheWatcher leak when time jump to the future and jump back
Kubernetes-commit: 36d5db95f8598bc64071c663c6f44a453c662102
2020-11-05 22:02:41 +08:00
wojtekt 942f220e85 Remove variadic argument from storage interface
Kubernetes-commit: 8b98305858b107369f2c9b9fd8ef1c5b0da078c0
2020-11-02 15:52:51 +01:00
wojtekt e0edc885d5 Fix cacher test flakiness
Kubernetes-commit: 6eb71c4089f69c7cc35ef2fd13bba4329b5716b9
2020-10-26 15:29:51 +01:00
wojtekt cdc93483b8 Watch bookmarks may contain version of objects of other types
Kubernetes-commit: 0bd8104809922aae504fc9be2e853650ecd17b8a
2020-09-29 12:27:20 +02:00
wojtekt c4fb001eda Enable progress notify events in watchcache
Kubernetes-commit: a94fb5369d5e77b3fcafd1296bac072a1d6e13fe
2020-08-31 13:15:36 +02:00
Joe Betz 97937c66f2 Revert nested trace PR#88936
Kubernetes-commit: 02cf58102a61b6d1e021e256381ff750573ce55d
2020-07-20 09:55:05 -07:00
Joe Betz 7a467399ac Enable nested tracing, add request filter chain tracing incl. authn/authz tracing
Kubernetes-commit: b12ac0abc64adb71d97fbde12f373b1424631f20
2020-03-06 16:11:21 -08:00
Joe Betz ee219411ed Add ResourceVersionMatch parameter to make Resource Version semantics consistent for list
Kubernetes-commit: e214f2408b59c745c199645547948a8ad2a87ac2
2020-05-29 10:44:26 -07:00