Commit Graph

37 Commits

Author SHA1 Message Date
Wojciech Tyczyński 012ace9f52 Fix lack of sorting of list results in watchcache
Kubernetes-commit: aaaee155dd3c9402e674d066f4d6ca30cc971c8e
2022-11-08 13:06:56 +01:00
David Ashpole aa161f2fc0 migrate apiserver utiltrace usage to component-base/tracing
Kubernetes-commit: de26b9023f2872c5cd7e15fad5dd5ab649222c13
2022-10-20 18:15:38 +00:00
Andy Goldstein 7eb011f596 watch cache: metrics: objectType -> group resource
Use the group resource instead of objectType in watch cache metrics,
because all CustomResources are grouped together as
*unstructured.Unstructured, instead of 1 entry per type.

Signed-off-by: Andy Goldstein <andy.goldstein@redhat.com>

Kubernetes-commit: d08b69e8d35a5aa73a178c508f9b0e1ad74b882d
2022-08-11 15:51:21 -04:00
Wojciech Tyczyński 2049cfbb9e Adjust watch channel sizes in watchcache
Kubernetes-commit: 0db5c05bdb8bbc510307a48cbade712583bb009e
2022-04-28 11:56:41 +02:00
Han Kang 24d28aad81 refactor watch cache metrics so that they are in a single place
Change-Id: I7874e90d4ec111e82450273f27f8e50556cb096a

Kubernetes-commit: a569829f599acf7ac3acbd3902f41d2c74fc2bd7
2022-04-22 14:21:25 -07:00
Madhav Jivrajani ffc1b941d4 cacher: Minor cleanup and refactor of code and tests
* Remove linter warnings.
* Cancel contexts to avoid leaks.
* Rename a few XXXThreadUnsafe to XXXLocked to
  maintain consistency.
  * A few are still called XXXThreadUnsafe mainly
    because those are safe to be called from the
    perspective that only one gorotuine will access
    them - not really called under a lock.

Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>

Kubernetes-commit: c3081b48759db1f05a446f2acca7e05c4511ce2e
2022-02-10 13:44:34 +05:30
Madhav Jivrajani 4f2ee5cf42 cacher: Use watchCacheInterval to reduce lock contention
- Modify GetAllEventsSinceThreadUnsafe to return a watchCacheInterval
- Modify Watch() to compute a watchCacheInterval rather than a slice
  of all "initEvents" and pass this interval to process()
- Use interval::Next() to obtain events to process rather than obtain
  them all at once
- Modify tests accordingly to use interval
- On invalidation, stop processing and stop the watch.
- Make indexValidator injectable for testing
- Add unit test for verifying the behaviour of stopping the watch.

Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>

Kubernetes-commit: 7f2aa7ad3a61a52d0a780f904b291d063399c28a
2021-10-05 15:52:09 +05:30
Madhav Jivrajani 74fdb4a93b cacher: watchCacheInterval to reduce lock contention
watchCacheInterval serves as an abstraction over a source
of watchCacheEvents. It maintains a window of events over
an underlying source and these events can be served using
the exposed Next() API. The main intent for doing things
this way is to introduce an upper bound of memory usage
for starting a watch and reduce the maximum possible time
interval for which the lock would be held while events are
copied over.

The source of events for the interval is typically either
the watchCache circular buffer, if events being retrieved
need to be for resource versions > 0 or the underlying
implementation of Store, if resource version = 0.

Furthermore, an interval can be either valid or invalid at
any given point of time. The notion of validity makes sense
only in cases where the window of events in the underlying
source can change over time - i.e. for watchCache circular
buffer. When the circular buffer is full and an event needs
to be popped off, watchCache::startIndex is incremented. In
this case, an interval tracking that popped event is valid
only if it has already been copied to its internal buffer.
However, for efficiency we perform that lazily and we mark
an interval as invalid iff we need to copy events from the
watchCache and we end up needing events that have already
been popped off. This translates to the following condition:
  watchCacheInterval::startIndex >= watchCache::startIndex.
When this condition becomes false, the interval is no longer
valid and should not be used to retrieve and serve elements
from the underlying source.

Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>

Kubernetes-commit: 347607e97139959f33024a691d0561b1479aeeef
2021-12-05 23:17:00 +05:30
Wojciech Tyczyński 93c06a1426 Minor watchcache cleanup
Kubernetes-commit: ace55542575fb098b3e413692bbe2bc20d2348ba
2021-10-29 14:06:49 +02:00
wojtekt a7cc30c9b3 Optimize watchcache by not starting a gorotuine for all Get/List requests setting RV=0
Kubernetes-commit: 37f93fc63d988c2f1ea11f467d2f73a43d1fa5b9
2021-09-24 14:14:19 +02:00
Mike Spreitzer 889106eff7 Add metrics about handling LIST requests
Add metrics that illuminate the costs of handling LIST requests.

Kubernetes-commit: bf424292db185546c474b27b963fd27bdbafc72f
2021-09-01 18:18:23 -04:00
wojtekt b898581360 Migrate to k8s.io/utils/clock in apiserver
Kubernetes-commit: 859a98c0358610e2c127cd2fba1be601ca975188
2021-09-14 20:36:07 +02:00
yuanhuaiwang ceb54fd8a6 improve watchCache metrics
Kubernetes-commit: 58e4fae1daafd88dbcc5bd38601397f13d391744
2020-12-02 16:19:20 +08:00
wojtekt cdc93483b8 Watch bookmarks may contain version of objects of other types
Kubernetes-commit: 0bd8104809922aae504fc9be2e853650ecd17b8a
2020-09-29 12:27:20 +02:00
wojtekt c4fb001eda Enable progress notify events in watchcache
Kubernetes-commit: a94fb5369d5e77b3fcafd1296bac072a1d6e13fe
2020-08-31 13:15:36 +02:00
gongguan 5dbb27a1df remove deprecated cacher CacheCapacity config
Kubernetes-commit: 12c838136b09005e0cbc831096643195ef24cb89
2020-06-05 21:34:36 +08:00
gongguan 1da8aeae17 improve Clock as Cacher Config
Kubernetes-commit: 0f7de876a5e931a57ba6347b81e8e28d360c788a
2020-06-15 23:57:11 +08:00
Wojciech Tyczynski 54e40d4b2d Revert "Revert "Rely on default watch cache capacity and ignore its requested size""
Kubernetes-commit: 7787ebc85b87eb7bc00f234f4e52eed3d987d5dd
2020-05-27 14:19:38 +02:00
wojtekt 8371df6187 Watch should return "too old RV" if it can't serve it.
Kubernetes-commit: ef1e5b6d3ac25431d890bfe4f540d9aa956a856d
2020-05-25 20:46:27 +02:00
Jordan Liggitt d501f7b597 Revert "Rely on default watch cache capacity and ignore its requested size"
This reverts PR 91260

Kubernetes-commit: 6249f287e1d3d9fb63b60961c8625a82d7db360e
2020-05-23 09:21:22 -04:00
wojtekt 31ffec3302 Rely on default watch cache capacity and ignore its requested size
Kubernetes-commit: 99dff90d613e2014873dc2fae0a00acaf95c3bd2
2020-05-19 18:13:28 +02:00
Davanum Srinivas 5879417a28 switch over k/k to use klog v2
Signed-off-by: Davanum Srinivas <davanum@gmail.com>

Kubernetes-commit: 442a69c3bdf6fe8e525b05887e57d89db1e2f3a5
2020-04-17 15:25:06 -04:00
wojtekt 6332f98216 Update duration to be kept in watchcache
Kubernetes-commit: d1de6295bb8047cbfd2a4ecbb0d264197f4cd8a4
2020-05-04 17:21:40 +02:00
louisgong 4c8b97679c Use dynamic size watch-cache.
If all cached events occur inside eventFreshDuration, increase cache capacity by 2x.
Decrease cache capacity by 2x when recent half events occur outside eventFreshDuration.

Kubernetes-commit: 56407b656c7acf6039cead0192070429e53a0c70
2020-04-12 17:22:38 +08:00
shaloulcy 2c701b633b validate storage cache indexers
Signed-off-by: shaloulcy <lcy041536@gmail.com>

Kubernetes-commit: fa9ba80a67deeb9089e1be8ebac21063ebc07904
2020-02-05 12:08:14 +08:00
shaloulcy 6099d76e39 add indexer for storage cacher
Signed-off-by: shaloulcy <lcy041536@gmail.com>

Kubernetes-commit: 87582e2c3c1cb8c8e10ec05a4700f865c759b7b2
2019-11-19 16:52:07 +08:00
Joe Betz 1ee386df74 Add HTTP 410 (Gone) status code checks to reflector and relist with RV=''
Kubernetes-commit: e2fe126d485af243d45f06bafd2ca83931881429
2019-10-09 16:45:32 -07:00
wojtekt a76d249b96 Tweak use of caching objects
Kubernetes-commit: c5170dfb66749b60a4a1611dc7b4a39e64b33e2e
2019-10-16 09:05:13 +02:00
Joe Betz 1b75e5416d Use same 'minimum resource version' semantics both when watch cache is enabled and disabled
Kubernetes-commit: 1406f7adf20a9ceaf5b8270ec9a61dd9e450ae8a
2018-12-04 16:55:59 -08:00
Wojciech Tyczynski 20ecceebd3 Cache serializations
Kubernetes-commit: 25a728ae5eb0ae067d21679fad915c0555242470
2019-08-25 09:55:52 +02:00
wojtekt a3fd374f10 Reduce contention in watchcache by not calling event handler under lock
Kubernetes-commit: e6e43828284c6e83cf2b4658fa377b4cc7dec0c3
2019-04-17 12:53:36 +02:00
fansong.cfs 58fb117373 add watch bookmark
Kubernetes-commit: d70edd3d39d4430d71c4b7c9adba8df5ba7f16c8
2019-03-19 18:16:23 +08:00
wojtekt f8c5a84403 Slightly simplify watchcache
Kubernetes-commit: c93f889833a4e9b519409c60f326c9970b4ee02a
2019-04-09 14:20:18 +02:00
Andrew Kim 2b43930b32 replace k8s.io/apiserver/pkg/util/trace with k8s.io/utils/trace
Kubernetes-commit: 93b086f6eafc1e94214fc1d946a1917482751cf3
2019-01-24 11:50:30 -05:00
Jordan Liggitt 80029a760c Deprecate and remove use of alpha metadata.initializers field, remove IncludeUninitialized options
Kubernetes-commit: 17aa60686ebe5fd04b4fe6f442dc36a8d70b6730
2019-01-17 09:19:28 -05:00
wojtekt d3536986da Fix unnecessary too-old-errors from watch cache
Kubernetes-commit: 12021725922efc3a80c8a0673b28826a524eb0a0
2018-08-30 12:28:44 +02:00
fisherxu 9ffd3e56ff move cacher in separate dir
Kubernetes-commit: dbb448bbdcb9e440eee57024ffa5f1698956a054
2018-05-30 20:44:31 +08:00