Commit Graph

63 Commits

Author SHA1 Message Date
Lukasz Szaszkiewicz 90ba08909c storage/etcd: the watcher supports the API streaming
Kubernetes-commit: ca562fd280a9dd5db952eb8c14b93fcf6668cc49
2023-08-22 14:02:39 +02:00
Lukasz Szaszkiewicz fe7ae035ef storage/etcd3/watcher_test: refactor TestWatchErrorWhenNoNewFunc to a table test
Kubernetes-commit: 4b60c18183f11207d0cb2d89bf28090156222c61
2023-09-19 10:35:09 +02:00
Lukasz Szaszkiewicz 957c305d94 storage/etcd: add TestWatchDispatchBookmarkEvents unit test
Kubernetes-commit: 875b00137fdfbc74756a0fc5b1c9b2adbeb78e55
2023-09-01 13:45:44 +02:00
wackxu f40bea647b paginate initial list inside the storage watcher
Signed-off-by: wackxu <xushiwei5@huawei.com>

Kubernetes-commit: f5d6c65186d63647a46400762f849d500e6ac591
2023-07-28 16:43:34 +08:00
Lukasz Szaszkiewicz 206231e0d0 storage/etcd3: error when progressNotify option set and newFunc was provided for a registry
Kubernetes-commit: 1988c31fc8115bf9eec1adf2bffed3fd677d1a9f
2023-08-28 17:50:42 +02:00
Lukasz Szaszkiewicz 7ecfbf7914 storage/etcd: simplify passing the transformer
Kubernetes-commit: a5600b6925169a7d99376d52472e5f8e8635082b
2023-08-21 12:27:41 +02:00
Lukasz Szaszkiewicz decaec94cd storage/etcd: skip SendInitialEvents if the request is backward compatible
otherwise an error will be returned.
backward compatibility is defined as RV = "" || RV = "O" and AllowWatchBookmark is set to false.
in that case we rely on 267eb25e60/staging/src/k8s.io/apiserver/pkg/storage/etcd3/watcher.go (L260)

Kubernetes-commit: f2de1a00b8fd7a22f9d0b2c1ace69be41d304f83
2023-05-08 13:04:31 +02:00
Wojciech Tyczyński 68a91d7597 Enable a bunch of generic storage tests for watchcache.
Kubernetes-commit: 88e2c34439016e5feddfed529698ed71e81904d9
2023-04-17 19:03:51 +02:00
Wojciech Tyczyński ca97e27fb8 Refactor watch bookmark tests to allow sharing between etcd3 and watchcache
Kubernetes-commit: 0297329795efc0f2bd24751e18f66b0aa79c222d
2023-04-17 14:22:52 +02:00
Wojciech Tyczyński 072d278e39 Generalize few watch tests from cacher
Kubernetes-commit: 45e836a968acc113cb03768cd8c730bea89bd332
2023-04-17 21:48:08 +02:00
Jordan Liggitt 8d0e8f8cb8 Partition watchers by namespace/name scope
Kubernetes-commit: 1c2fa0c7f791cbe28ee54d957d6f8a1120d91018
2023-02-05 18:46:19 -05:00
Wojciech Tyczyński a47bbc6347 Minor cleanup in etcd3 tests
Kubernetes-commit: 83399ccfb8822cc31a6a3a66e8352591d88feaa8
2022-10-31 18:02:40 +01:00
Wojciech Tyczyński a24d8963d1 Refactor remaining etcd3 tests to make them generic
Kubernetes-commit: e6ddb69e6fd06bf9a02fa731f12472f3c375762d
2022-10-31 18:02:19 +01:00
Wojciech Tyczyński 07db92af49 Fix setting resource version on deletion
Kubernetes-commit: bbcf5e38776f2b18026539a0fbcf3aa505386c1f
2022-10-26 21:37:19 +02:00
Wojciech Tyczyński 499bbb88dc Refactor WatchError test to make it generic
Kubernetes-commit: b3f9272d57a71a9c50b38754b7309ddf68adb93f
2022-10-28 11:35:58 +02:00
Wojciech Tyczyński f6802e074e Refactor storage tests using compaction
Kubernetes-commit: b02f172cbdf7e824d13a6a9c3a9b9fe4f3f92afb
2022-10-26 20:47:19 +02:00
Wojciech Tyczyński 834cf0fc14 Refactor compaction in etcd3 tests
Kubernetes-commit: 7da7ddd779f9ea835f0c57deae05e050c543066b
2022-10-26 16:34:28 +02:00
Wojciech Tyczyński d9a3685d6f Minor cleanup of storage tests
Kubernetes-commit: 5344bc5e1b7e50d5ff359c714d522b421b28bd12
2022-10-25 14:09:36 +02:00
Steve Kuznetsov 03363cd42b storage: move some list tests to generic package
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 162450c01c9004ded53bb77f651f9a3f30920b30
2022-05-12 15:16:07 -07:00
Steve Kuznetsov 80256820ce storage: move the APIObjectVersioner definition to storage
The means by which we extract and parse the version of an API object is
not specific to etcd3. In order to allow for a generic suite of tests
against any storage.Interface imlpementation, we need this logic to live
outside of the etcd3 package, or import cycles will exist.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 3939f3003e9605c06f65e64d1fc6f94b294f9d97
2022-05-11 07:44:21 -07:00
Steve Kuznetsov a4bcbc5187 storage/testing: move cancelled watch test to generic package
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 1f24bd91c30a3f0b623c452cffa9f043f202d98b
2022-05-09 08:23:58 -07:00
Steve Kuznetsov fe17bc776e etcd3/store: call a generic cancelled watch test
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: c0fc8172559f7796560a9b80fd23df92095adb14
2022-05-09 08:23:03 -07:00
Steve Kuznetsov 7b82bba67e etcd3/store: update cancelled watch test to be generic
There's no reason to create the watch using the underlying watcher.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 774870611c1d2b405d67eb73ade628ced5e0e994
2022-05-09 08:22:04 -07:00
Steve Kuznetsov b393e91916 storage/testing: wire things back together with imports
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 4ff560e4fa3c716a11f7260f33ec318faad59394
2022-05-05 11:32:05 -07:00
Steve Kuznetsov ebf7c468b8 storage/etcd3: move reusable tests to importable files
These can't live in `_test.go` files, or others won't be able to re-use
them as you can't import from test files.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 0bfeb728786726cd28b4c64d48874d45a2724165
2022-05-05 09:27:16 -07:00
Steve Kuznetsov cf5cf85de0 storage/etcd3: factor tests to accept `storage.Interface`
This commit simply factors the test functionality into functions that
accept `storage.Interface`.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 117f674cab1f598251483811fb769046dc3458a3
2022-05-05 09:18:08 -07:00
Steve Kuznetsov 169ec42cd5 storage/etcd3: continue unifying test setup
Previous work by liggitt in 01760927b82 improved the boilerplate
required to run an embedded etcd server for tests as well as set up the
`*etcd3.store{}` for testing. A number of tests were not ported to use the
new helpers, though, either due to custom setup or due to inconsistent
use of setup options. A follow-up by stevekuznetsov in 6aa37eb0624
removed much of the inconsistency, meaning that most callers to
`newStore()` were simply using the default boilerplate and options that
`testSetup()` used.

This patch moves all users to testSetup(), adding options as necessary
to enable some fringe setup use-cases. With a unified setup, new tests
will not copy boilerplate they do not need and it will be immediately
obvious when reading a test if the client or storage setup is *not*
default, improving readability.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 138faa3799341d02df9fc4bedc1371d338c34887
2022-04-22 07:39:32 -07:00
Steve Kuznetsov 90420e863e pkg/storage/etcd3: validate revision invariant
We must ensure that we notice if the etcd behavior on linearized reads
changes.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: ed5fd905f2b42e9919d99c40a1cb25014f0a7f89
2022-04-13 10:57:02 -07:00
Steve Kuznetsov 41c1f65bf6 pkg/storage/etcd3: correctly validate resourceVersions
In a number of tests, the underlying storage backend interaction will
return the revision (logical clock underpinning the MVCC implementation)
at the call-time of the RPC. Previously, the tests validated that this
returned revision was exactly equal to some previously seen revision.
This assertion is only true in systems where no other events are
advancing the logical clock. For instance, when using a single etcd
cluster as a shared fixture for these tests, the assertion is not valid
any longer. By checking that the returned revision is no older than the
previously seen revision, the validation logic is correct in all cases.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: eba25cdbbcc5d35e707516194f64d8ed363c2773
2022-03-23 12:44:49 -08:00
Steve Kuznetsov 42854c1734 pkg/storage/etcd3: be more precise in watch test
Previously, this test assumed that:
 - a global watch would return only an event for the key in question
 - only the delete event in question would be returned

Neither of these assumptions are correct for an etcd backend as long
as any other clients are interacting with the system. This commit
makes the watch more specific and extracts the correct event.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 2631c0a0f959bd67aa455045dce33e77150ab5f8
2022-03-23 12:59:53 -08:00
Steve Kuznetsov 25c879eec5 pkg/storage/etcd3: update test to time out, not panic
Without these select statements, this test runs until the package-global
timeout and causes a panic. This change makes the test fail faster and
more legibly.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: fc33d0176a5afb81927430d075165152f953c54e
2022-03-23 12:22:35 -08:00
Steve Kuznetsov edb0c0eba4 pkg/storage/etcd3: use correct diff in tests
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 8903372e8b8d32f8b244f5087e3f2907b86f03bd
2022-03-23 12:33:57 -08:00
Steve Kuznetsov 3d5978d57e storage: etcd: clarify lease manager configurations
When the original commit created the lease manager, this comment was
added to set the default test reuse time to 1s. Even at that time, the
comment claimed it was setting 10s. Instead of using this value, though,
new tests that did not call `testSetup()` started to use the default
configuration for production. This commit clarifies the intent of this
comment, moves it next to the code block that it actually applies to,
and makes use of this test-specific logic everywhere.

x-ref: f230b000db

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 6aa37eb06247fb95a6a4ef61cbd50885e52055a0
2022-02-18 08:10:57 -08:00
Steve Kuznetsov fb315d8c17 store: etcd: TestWatch: use cmp.Diff
Modernize the comparisons used in the watch tests to use `cmp.Diff()` for
readability.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: d17a19b39d2dbdaf2cbbaad46de403d6d7ce0602
2022-02-18 06:51:36 -08:00
Steve Kuznetsov 557ecdf10d store: etcd: TestWatch: use sub-tests
This was the last test to not use sub-tests, so we can also remove the
indices that the expectation functions take as parameters now.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 9f7bb4264e0b79cbe7979c09f0e4c75a434a27bb
2022-02-18 06:50:05 -08:00
Steve Kuznetsov 4074cf3cb4 storage: etcd: TestWatchError: improve readability
This test, as written, is *extremely* cryptic and hard to parse. Add a
comment and stop intentionally ignoring an error that only needs to be
ignored if we're being cryptic.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 50eed81923495f5ee1ac44436676ddbaf2a380fe
2022-02-17 15:46:05 -08:00
Wojciech Tyczyński 71bfcaaea1 Simplify storage.Interface by merging Watch and WatchList functions.
Kubernetes-commit: 53653e9b0a7145c3b6e133c5639194ff79b3719d
2021-11-18 12:40:26 +01:00
Mike Spreitzer 31ec50ec43 Plumb the schema.GroupResource into etcd3 Store struct
Kubernetes-commit: 6f8019aae95db85552381f7e7066a52fef61be7a
2021-09-13 16:54:34 -04:00
Jordan Liggitt 4bb06f3c4d Simplify running embedded etcd server in tests
Kubernetes-commit: 01760927b82cf28bc6ba5b6364fda3c9f8c5c0b1
2021-06-15 12:07:46 -04:00
Jordan Liggitt 6365bf9124 Switch to go.etcd.io/etcd/client/v3
Kubernetes-commit: 2979c3325e3ac4023dd19c435f3a995074328751
2021-03-23 11:01:52 -04:00
wojtekt 5290a24345 Implement support for watch initialization in P&F
Kubernetes-commit: 0cc217647ca8be0820973b970124a072c27b6575
2021-05-07 12:49:06 +02:00
Ling Samuel bd7da186cc apiserver add metric etcd_lease_object_counts
Signed-off-by: Ling Samuel <lingsamuelgrace@gmail.com>

Kubernetes-commit: 7e9fe39cd7c3ee67fcc29e6ca0836917c5a4e9e1
2020-12-07 10:45:44 +08:00
wojtekt 8c25869e43 Add suggestion to storage interface Delete method
Kubernetes-commit: c2d61896f45967cdea15072b7a12faeb635b5dc5
2020-11-03 13:21:00 +01:00
Ling Samuel f673ce24f7 apiserver add --lease-reuse-duration-seconds to config lease reuse duration
Signed-off-by: Ling Samuel <lingsamuelgrace@gmail.com>

Kubernetes-commit: c99567005db15d220f2b792e98af735c5aa24f6e
2020-12-02 17:35:10 +08:00
Ken Sipe 5d58b175c8 fix S1021 var declaration
Signed-off-by: Ken Sipe <kensipe@gmail.com>

Kubernetes-commit: 6c49299739a9819c3672248517ab3d6636d1d8c6
2020-06-25 17:10:34 -05:00
wojtekt 942f220e85 Remove variadic argument from storage interface
Kubernetes-commit: 8b98305858b107369f2c9b9fd8ef1c5b0da078c0
2020-11-02 15:52:51 +01:00
wojtekt e40efde837 Implement etcd3 progress-notify feature in etcd3 layer
Kubernetes-commit: 56e72841b6005740453828a9f4f7a9a1b9a831f6
2020-08-31 12:40:41 +02:00
wojtekt a5000473c1 Pipe newFunc to etcd3 storage layer
Kubernetes-commit: fbd65a265a47ffe081aaac2f794a55034333d11d
2020-08-31 11:58:45 +02:00
Joe Betz 5b75cf0ff9 Add GetOptions and ListOptions to storage interface
Kubernetes-commit: 4c99949ae6b4a74e4ffda0caba86c151c23d579e
2020-05-29 10:48:33 -07:00
Monis Khan f5c1e085ed storage: confirm that paging and predicate filtering work together
This change adds the TestListContinuationWithFilter test which
confirms that paging with a predicate that does not match everything
results in the correct amount of calls to TransformFromStorage and
KV.Get.  The partial result of each paging call is also asserted.

Signed-off-by: Monis Khan <mok@vmware.com>

Kubernetes-commit: 002c75442d768d2bcc51047667354ff16bbfa2e8
2020-02-28 15:35:22 -05:00