Commit Graph

937 Commits

Author SHA1 Message Date
Marek Siarkowicz b5a6bf9a62 Test requests send to etcd for all LIST requests
Kubernetes-commit: 709280ef6e785196c62ce73d65d0ddc7b4dca718
2025-06-12 14:50:23 +02:00
Marek Siarkowicz c057db759a Ensure keys used in storage and cacher start with resourcePrefix
Kubernetes-commit: 6d5aece4d81cc2b23308dd034d0c9276feb5cc55
2025-09-11 13:42:19 +02:00
Marek Siarkowicz ce9b4fddcb Extract the prepareKey function
Kubernetes-commit: 0b10735cd72fbf5416a20d2bb7e525b7d8a1e716
2025-09-17 16:41:27 +02:00
Marek Siarkowicz 9d438d9fe5 Ensure consistent key schema requirements between cacher and etcd3
Kubernetes-commit: 91a3a30eb5ca401ca700cb42cea416d83750cade
2025-09-15 14:18:03 +02:00
Marek Siarkowicz 35644ed688 Unify directory protection for recursive requests in storage
Kubernetes-commit: 74faf1527fd444b90e5f5622b440c17ac96bc166
2025-09-15 13:46:43 +02:00
Richa Banker 3218fe46fe Specify the deprecated version of apiserver_storage_objects metric
Kubernetes-commit: 476325f6e5febeb1c5beb673aa01474518310811
2025-09-11 16:16:20 -07:00
Marek Siarkowicz 73c7acd623 Fix cacher resource prefix not having a "/" at the end in tests
Kubernetes-commit: 1e742f01ba6749bf7e356097c034c8cf9eb56d85
2025-09-11 15:03:05 +02:00
Marek Siarkowicz 9e9ff9a0ae Fix flaking RunTestDelayedWatchDelivery
Kubernetes-commit: 1baa7936d893e39dd42cec3de1c025c5313bd07e
2025-08-20 14:10:33 +02:00
Marek Siarkowicz 549d9cb681 Fix tests not using proper resource paths
Kubernetes-commit: 4f6598e8a31558f253109351a36e9a72d0d78ab7
2025-09-11 12:23:35 +02:00
Marek Siarkowicz 677aaf0c38 Fix tests to to only accesses keys from under resourcePrefix
Kubernetes-commit: 1ac067ae0e68b14a7a6eff464aacdaf7d70e838d
2025-09-03 12:23:07 +02:00
Anish Ramasekar 6065eecf08 kmsv2: run TestKMSv2ProviderKeyIDStaleness in parallel
This change updates the NowFunc to be per KMS provider instead of global
to the API server. This allows integration tests that use distinct
provider names to run in parallel when simulating key expiry.

Signed-off-by: Anish Ramasekar <anish.ramasekar@gmail.com>

Kubernetes-commit: 480fad996d58af5f6700389a68c662287e746992
2025-09-09 13:08:56 -07:00
Marek Siarkowicz aa97561e5f Use increaseRV in TestWatchStreamSeparation to imply external RV increase
Kubernetes-commit: 31e035ec39e27f14d01dbe3cd466416998ba1939
2025-09-10 14:08:38 +02:00
Marek Siarkowicz 89a0518f28 Cleanup enabling resource size estimate
Kubernetes-commit: edc8dafc6993dd7928cc148687a23d7eaaa23f99
2025-09-04 09:38:18 +02:00
Jordan Liggitt 4d892e4443 Revert "protect against race between deletion and adding finalizers"
This reverts commit 1dd33c8e6de2428bb0bb50142518158764fbb942.

Kubernetes-commit: 229954d1138c13346546715ef34abd29989f76ac
2025-09-09 16:33:42 -04:00
Jordan Liggitt 0829991ecb protect against race between deletion and adding finalizers
Kubernetes-commit: 1dd33c8e6de2428bb0bb50142518158764fbb942
2025-01-22 17:19:03 -05:00
Marek Siarkowicz 3127846a6e Enforce that all resources set resourcePrefix
Kubernetes-commit: 683a76dd159bc329a840778b474059ed0b9d8c11
2025-09-03 10:23:29 +02:00
Marek Siarkowicz 80746a8efd Disable estimating resource size for resources with watch cache disabled
Listing all keys from etcd turned out to be too expensive, negativly
impacting events POST latency. Events resource is the only resource that
by default has watch cache disabled and which includes very
large number of small objects making it very costly to list keys.

Expected impact:
* No apiserver_resource_size_estimate_bytes metric for events.
* APF overestimating LIST request cost to events. Fallback assumes
  object size of 1.5MB, meaning LIST events will always get maxSeats

Kubernetes-commit: 3e84e16315ac2b3cb9729a98ed1ae636d8900db6
2025-09-03 15:42:41 +02:00
Marek Siarkowicz ba83a7fa01 Disable collecting stats for resources not setting prefix to prevent error logs
Kubernetes-commit: 588276255bab9900b976aad19f58a6c821bd39eb
2025-09-01 10:21:52 +02:00
Richa Banker e8d163a6bc Register apiserver_resource_objects in TestDeleteStoreStats
Kubernetes-commit: 3995b6cf72bbd5ec8ed964d7fcbb9fcf440d1d36
2025-08-06 16:57:37 -07:00
Richa Banker 45fdfc0ca4 Add sig-instrumentation as OWNERS for apiserver/pkg/storage/etcd3/metrics
Kubernetes-commit: a28eda435f2c2d5f4cd4461466fea87897d9822b
2025-08-08 09:58:02 -07:00
Marek Siarkowicz 7f5c70221d Fix storage counting all objects instead just objects for resource
Kubernetes-commit: d49e3d67cb39eb2dee6e9216ffed0e3ae13501a5
2025-08-19 15:09:44 +02:00
Richa Banker b64541f9b2 Remove DeprecatedVersion for apiserver_storage_objects, adjust help text
Kubernetes-commit: cbd6fb867ba26f977df605fe6a84f8570ed7abdc
2025-08-07 14:02:46 -07:00
Richa Banker 646afb740b Use Delete() instead of DeleteLabelValues() for etcd metrics
Kubernetes-commit: 2629fe25faa73837201dcfc7c4de039112ca125f
2025-08-06 21:24:27 -07:00
Marek Siarkowicz 83a4b98448 Add jitter to priodically executed process in storage to avoid too concurrent executions
Kubernetes-commit: 79dc0b8a4146a411d8c106705558deedd8f7f8bf
2025-07-18 10:57:49 +02:00
Marek Siarkowicz e1b2eb304d Deprecate apiserver_storage_objects and replace it with apiserver_resource_objects metric using labels consistent with other metrics
Kubernetes-commit: cf68f75fc3393e2579ba8158d3936c275a6d9be0
2025-07-15 19:38:39 +02:00
Marek Siarkowicz 478396c06d Add apiserver_resource_size_estimate_bytes metric
Kubernetes-commit: a091fa607eaecfa7b80cb0437db1d210aa587f98
2025-07-11 16:38:43 +02:00
Sascha Grunert ad12519854 Convert `k8s.io/kms/apis` from gogo to protoc
Use standard protoc for the kms APIs instead of gogo.

Part of kubernetes#96564

Signed-off-by: Sascha Grunert <sgrunert@redhat.com>

Kubernetes-commit: 8e6651520e68a32d2b6a77e10ca20a5e4ccb0adb
2025-07-09 07:46:16 +02:00
Marek Siarkowicz 4b361d95e8 Graduate ListFromCacheSnapshot to Beta
Kubernetes-commit: fd23fa3d423a933f7817d205308013af4b0e9b53
2025-07-11 21:52:17 +02:00
Marek Siarkowicz d2387bc56c Inconsistency resets cache snapshots and prevents collection of new ones until cache is marked consistent again
Kubernetes-commit: 32afbd9ea5d8225fdcce7ac17afcce2f4f8cf169
2025-07-14 22:59:25 +02:00
Marek Siarkowicz d8d8f4abb2 Enable running cache inconsistency detection by default
Kubernetes-commit: e6cf7dbf063b1ed94bcf1678f73352e584456024
2025-07-10 18:26:41 +02:00
Marek Siarkowicz b5092551f9 Graduate ConsistentListFromCache to GA
Kubernetes-commit: 5bae9fa8939efcae2b1a757f5ef3599205fa6e08
2025-07-01 11:33:00 +02:00
Marek Siarkowicz b82f4b24a0 Fix TestProgressNotify for etcd v3.6.2
Kubernetes-commit: e3836172845c484bf3269b827f7c6b81b2178fe6
2025-07-13 12:42:45 +02:00
Marek Siarkowicz 19e8aed081 Compact watch cache based on last observed etcd compaction
Kubernetes-commit: bfeaae3310fcc4c20ed3c8f2c1afebf059699e22
2025-07-10 15:17:26 +02:00
Marek Siarkowicz dda2fcb797 Expose compaction revision from compactor
Kubernetes-commit: 03e32bd260fbd77351e1b528f22a69a8eaece9ee
2025-07-09 16:22:19 +02:00
Marek Siarkowicz 34f5472b35 Add test for compactor
Kubernetes-commit: 5e3ea8e279c51a0b4fcc996d244d2c5593935816
2025-07-09 16:29:55 +02:00
Marek Siarkowicz 2f077fcbca Add testing compaction in storage List tests
Kubernetes-commit: c53b41e98c2e9248916704f36443de87cbee64e0
2025-07-08 11:19:59 +02:00
Benjamin Elder bf8c1a6d90 storage/etcd3: add back missing errcheck
lost in recent refactoring.

Kubernetes-commit: 838f3afc5219640725922266b689c7d24fbdf804
2025-07-07 17:37:36 -07:00
PatrickLaabs d0ee71ec17 chore: second depr. pointer pkg replacement for apiserver
Kubernetes-commit: fe61e825e046d793d64fc0686110212dacf8231e
2025-07-06 11:09:16 +02:00
liuxu ab47d6f39c remove apiserver_storage_objects metrics after crd deleted
Kubernetes-commit: 8c6c7df736dff27903bc03a1b9a42c1e2c84c7ee
2025-07-03 09:57:15 +08:00
Marek Siarkowicz 67c4a8a9ee Add benchmark for SizeBasedListCostEstimate feature
Kubernetes-commit: 1639b090839b37ab08c1d90d62f2c7a4c44faba4
2025-06-25 15:50:34 +02:00
Marek Siarkowicz df850b5a51 Override getKeys when cacher is enabled
Kubernetes-commit: e2c6b7fdf97206cc60d6c5f0e1cb306652ec189a
2025-06-23 16:35:06 +02:00
Marek Siarkowicz b805c4d851 Run background cleanup goroutine
Kubernetes-commit: 7cb241799935745277125db033ae217fe64e322c
2025-06-18 17:24:17 +02:00
Marek Siarkowicz cf27dab482 Estimate average size of objects in etcd and plug it into request cost estimator
Kubernetes-commit: ec78b8305ad392f6faf4e5247ea33ceabb484c3f
2025-06-13 16:34:42 +02:00
Marek Siarkowicz eaf6e48501 Handle consistent LIST in watch cache to avoid incorrect semantics while setting ResourceVersion on options
Kubernetes-commit: 292679a28ae472da29c5d860afdb5c2250637d31
2025-06-06 17:16:46 +02:00
Lukasz Szaszkiewicz a77ab41bc0 client-go/reflector: stop exposing UseWatchList (#132453)
* client-go/reflector: stop exposing UseWatchList

* apiserver/cacher: stop setting reflector.UseWatchList

* test/integration/watchlist: fix TestReflectorWatchListFallback

Kubernetes-commit: b8b3984874e930c92057589fd1a7668dbdffc117
2025-06-25 13:45:59 +00:00
PatrickLaabs fdccb8b2dc fixing large resourceversion and limit for storages
Kubernetes-commit: ccdef28acd3a286e8d62222ddf804ae4042764e5
2025-06-18 16:22:13 +02:00
Lukasz Szaszkiewicz 0d3a31d996 apiserver/storage/cacher/listwatcher: error when the WatchList FG is disabled
Kubernetes-commit: 7e0d71fc14bcbe1fae42ccc91f2d48fc9b99a049
2025-06-25 08:50:03 +02:00
Lukasz Szaszkiewicz 0e36545f56 apiserver/cacher: properly wire listwatch options to the listwatcher
Kubernetes-commit: dce69afadeaf3fc8c98d1dfd56394fb77d52225b
2025-06-24 10:55:06 +02:00
Marek Siarkowicz baef40d3ec Move UnsafeCorruptObjectDeletion outside of etcd3.New function
By returning *store instead of storage.Interface we can expose Close()
function so in the future we can register it to destroyFunc in
newETCD3Storage.

Kubernetes-commit: 72305f82f9a20315d7ff4904a840bfd56478d44a
2025-06-23 10:50:06 +02:00
Marek Siarkowicz 9b695a5efa Validate requests sent to etcd in TestList "test List with limit" scenario
This adds a regression test to detect fallback to etcd as discovered in https://github.com/kubernetes/kubernetes/issues/132132.

Kubernetes-commit: 4cb6d3d77617f141fefd4994910380c095dac1ad
2025-06-12 11:33:02 +02:00