Commit Graph

110 Commits

Author SHA1 Message Date
Wojciech Tyczyński 324d2bc3cf Cleanup paging parameters in etcd3 store
Kubernetes-commit: 3f4d3b67682335db510f85deb65b322127a3a0a1
2023-10-20 15:38:59 +02:00
Siyuan Zhang 77032c52b8 k8s.io/apiserver/storage: add 3 new unit tests for delete.
Signed-off-by: Siyuan Zhang <sizhang@google.com>

Kubernetes-commit: 26a4e06c92c248748dd3c50c74d75f8adc3c6823
2023-10-11 10:38:01 -07:00
Lukasz Szaszkiewicz f75c503352 storage/factory: extend the Create method by newList and resourcePrefix params
Kubernetes-commit: ccabc01093a1344ebb27c32c946e9da3b8e91fd2
2023-07-28 09:53:01 +02:00
Lukasz Szaszkiewicz 7ecfbf7914 storage/etcd: simplify passing the transformer
Kubernetes-commit: a5600b6925169a7d99376d52472e5f8e8635082b
2023-08-21 12:27:41 +02:00
Marek Siarkowicz df9896fd00 Test consistent List
Kubernetes-commit: b36fdd68b72cd5c64ea5be3917846067644da983
2023-07-03 17:22:22 +02:00
scott d681755704 ftr(etcd): add benchmarks
Kubernetes-commit: db92cc1388021495e47dbe257a9e6559031cfe51
2023-03-29 21:52:38 +08:00
Wojciech Tyczyński 299c158ba3 Reuse generic List test for watchcache
Kubernetes-commit: ef8c4fbca8e5bed1e7edc162b95c412a7f1a758e
2022-11-04 22:01:37 +01:00
Tim Allclair 3268b0561b Validate etcd paths
Kubernetes-commit: f1693a02c46f79f8c90a4ef17c4a750034f21484
2022-10-10 18:15:22 -07:00
Wojciech Tyczyński c38aa93533 Reuse generic TestGet in cache tests.
Kubernetes-commit: 75a1ef87b32213a7d3d1c2027dd515e4f74c1777
2022-10-27 16:00:30 +02:00
Wojciech Tyczyński 499bbb88dc Refactor WatchError test to make it generic
Kubernetes-commit: b3f9272d57a71a9c50b38754b7309ddf68adb93f
2022-10-28 11:35:58 +02:00
Wojciech Tyczyński 3f5a3e0ae0 Refactor etcd3 list consistency test
Kubernetes-commit: cd5da36c92f3ed52debc22c39a7cd9d369b0eecd
2022-10-27 10:19:09 +02:00
Wojciech Tyczyński c527f6432a Minor cleanup of etcd3 tests
Kubernetes-commit: bbe1ebc82aa019118c91e8447517f91b9d036c1e
2022-10-27 09:48:06 +02:00
Wojciech Tyczyński f6802e074e Refactor storage tests using compaction
Kubernetes-commit: b02f172cbdf7e824d13a6a9c3a9b9fe4f3f92afb
2022-10-26 20:47:19 +02:00
Wojciech Tyczyński 834cf0fc14 Refactor compaction in etcd3 tests
Kubernetes-commit: 7da7ddd779f9ea835f0c57deae05e050c543066b
2022-10-26 16:34:28 +02:00
Wojciech Tyczyński 3afe8ebf5f Minor cleanup of etcd3 tests
Kubernetes-commit: e04fe81dfcff189ef5162b9b26b55760f613a010
2022-10-26 13:05:49 +02:00
Wojciech Tyczyński 73a664638b Refactor pagination tests
Kubernetes-commit: 6c8ce894e18a2e2cae20edd2f6e79db9407ebce3
2022-10-26 12:18:21 +02:00
Wojciech Tyczyński dec6f492e8 Refactor GetReads
Kubernetes-commit: 8472e1bc13b0fa8aa3a67e5fcf2f13bfd7974cd3
2022-10-26 11:30:32 +02:00
Wojciech Tyczyński 5a8fdf8480 Move GuaranteedUpdateChecksData test to generic package
Kubernetes-commit: afc5ded83931cdb8a2643486d6834eb8d9b04588
2022-10-25 14:31:20 +02:00
Wojciech Tyczyński d9a3685d6f Minor cleanup of storage tests
Kubernetes-commit: 5344bc5e1b7e50d5ff359c714d522b421b28bd12
2022-10-25 14:09:36 +02:00
Wojciech Tyczyński 1960fa837f Move GuaranteedUpdate test to generic test package
Kubernetes-commit: 012676acc3dd2b235bd601e4105294d2715f155b
2022-10-25 14:04:00 +02:00
Abirdcfly 6be1a93b55 fix: --chunk-size with selector returns missing result
Signed-off-by: Abirdcfly <fp544037857@gmail.com>

Kubernetes-commit: fba297629d82866078faa6dc076cb5f81633e557
2022-06-20 00:54:57 +08:00
Steve Kuznetsov 03363cd42b storage: move some list tests to generic package
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 162450c01c9004ded53bb77f651f9a3f30920b30
2022-05-12 15:16:07 -07:00
Steve Kuznetsov 289ed89252 storage/etcd3: make some list tests generic
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: a8067f8e865475cbdb11d812c915927714690676
2022-05-12 15:13:23 -07:00
Steve Kuznetsov ae4cc8bfc8 storage/etcd3: factor out non-paginated list tests
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 8fcf00ef9171cfb623525d2a6aea1d721c3b5e74
2022-05-11 07:18:05 -07:00
Steve Kuznetsov 139068ca91 storage/etcd3: factor store setup into a separate function
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 31d7a28c65c34f2e2a28cfc41c2c517d370e7045
2022-05-11 07:17:26 -07:00
Steve Kuznetsov 67344c728d storage: move continue token definition to storage
The means by which we encode and decode the continue token during a
paginated LIST call is not specific to etcd3. In order to allow for a
generic suite of tests against any storage.Interface implementation, we
need this logic to live outside of the etcd3 package, or import cycles
will exist.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: eb3aa5be10393968d8083c79f5958501fc029e8d
2022-05-11 07:52:02 -07:00
Steve Kuznetsov 80256820ce storage: move the APIObjectVersioner definition to storage
The means by which we extract and parse the version of an API object is
not specific to etcd3. In order to allow for a generic suite of tests
against any storage.Interface imlpementation, we need this logic to live
outside of the etcd3 package, or import cycles will exist.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 3939f3003e9605c06f65e64d1fc6f94b294f9d97
2022-05-11 07:44:21 -07:00
Steve Kuznetsov 3fd88b5da7 storage/testing: move creation test to generic package
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 2e118f42465efb390bec005f1853a964dd339f97
2022-05-09 08:04:53 -07:00
Steve Kuznetsov daf507c2ff etcd3/store: make creation test validation generic
Different callers to this test may need to do different backend-specific
validation on the stored data, so we allow them a callback for this.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 6d25e96cedaad249fe75aac4b1fe08bb69829a61
2022-05-09 08:03:41 -07:00
Steve Kuznetsov f54afb0566 etcd3/store: update creation test to use storage client
There is no functional difference between checking for an empty key
using the database client and doing so with the storage interface. Using
the latter allows this test to be more portable.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: f894f8196d9266915424e2cefc9e4eb480ae6f5e
2022-05-09 07:59:38 -07:00
Steve Kuznetsov b393e91916 storage/testing: wire things back together with imports
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 4ff560e4fa3c716a11f7260f33ec318faad59394
2022-05-05 11:32:05 -07:00
Steve Kuznetsov ebf7c468b8 storage/etcd3: move reusable tests to importable files
These can't live in `_test.go` files, or others won't be able to re-use
them as you can't import from test files.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 0bfeb728786726cd28b4c64d48874d45a2724165
2022-05-05 09:27:16 -07:00
Steve Kuznetsov cf5cf85de0 storage/etcd3: factor tests to accept `storage.Interface`
This commit simply factors the test functionality into functions that
accept `storage.Interface`.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 117f674cab1f598251483811fb769046dc3458a3
2022-05-05 09:18:08 -07:00
Steve Kuznetsov 169ec42cd5 storage/etcd3: continue unifying test setup
Previous work by liggitt in 01760927b82 improved the boilerplate
required to run an embedded etcd server for tests as well as set up the
`*etcd3.store{}` for testing. A number of tests were not ported to use the
new helpers, though, either due to custom setup or due to inconsistent
use of setup options. A follow-up by stevekuznetsov in 6aa37eb0624
removed much of the inconsistency, meaning that most callers to
`newStore()` were simply using the default boilerplate and options that
`testSetup()` used.

This patch moves all users to testSetup(), adding options as necessary
to enable some fringe setup use-cases. With a unified setup, new tests
will not copy boilerplate they do not need and it will be immediately
obvious when reading a test if the client or storage setup is *not*
default, improving readability.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 138faa3799341d02df9fc4bedc1371d338c34887
2022-04-22 07:39:32 -07:00
Steve Kuznetsov 41c1f65bf6 pkg/storage/etcd3: correctly validate resourceVersions
In a number of tests, the underlying storage backend interaction will
return the revision (logical clock underpinning the MVCC implementation)
at the call-time of the RPC. Previously, the tests validated that this
returned revision was exactly equal to some previously seen revision.
This assertion is only true in systems where no other events are
advancing the logical clock. For instance, when using a single etcd
cluster as a shared fixture for these tests, the assertion is not valid
any longer. By checking that the returned revision is no older than the
previously seen revision, the validation logic is correct in all cases.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: eba25cdbbcc5d35e707516194f64d8ed363c2773
2022-03-23 12:44:49 -08:00
Maciej Borsz f4edb394b0 Implement page size progressing for rare selectors.
Kubernetes-commit: a2ad9f9e4aba6aae6657a3189bdced6dbc8ba4b5
2022-03-07 15:44:59 +00:00
Steve Kuznetsov de7922ba23 pkg/storage/etcd3: refactor "too large" resourceVersions
When tests attempt to validate behavior in the case that a client asks
for a resource version that is "too large" for the underlying storage,
the previous implementation would simply add 1 to the latest revision
seen. This is only appropriate for storage backends that
  a) provide a continuous monotonic logical clock
  b) have no other events occurring while the test runs

For instance, when using a singe etcd backend as a shared fixture for
these tests, adding 1 to a previously-seen revision is not suffcient to
ensure that the resulting revision is "too large". By instead using the
largest possible integer value, we can be certain of this.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: b973cdc57cc6ee57684455cdb76db13a8c82cefa
2022-03-23 12:27:16 -08:00
astraw99 b0232c830b Fix function name of IsExist err
Kubernetes-commit: 79471695c2a16cb7c1fb17adff7765f07e2c9057
2022-03-19 23:06:51 +08:00
Steve Kuznetsov 3d5978d57e storage: etcd: clarify lease manager configurations
When the original commit created the lease manager, this comment was
added to set the default test reuse time to 1s. Even at that time, the
comment claimed it was setting 10s. Instead of using this value, though,
new tests that did not call `testSetup()` started to use the default
configuration for production. This commit clarifies the intent of this
comment, moves it next to the code block that it actually applies to,
and makes use of this test-specific logic everywhere.

x-ref: f230b000db

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 6aa37eb06247fb95a6a4ef61cbd50885e52055a0
2022-02-18 08:10:57 -08:00
Steve Kuznetsov 1501fbd930 storage: etcd: use cmp.Diff for comparisons
This commit simply modernizes the comparisons made in the storage tests
to use `cmp.Diff()` so that pointer comparisons and length checks do not
have to be made by hand. We also get nice diffs in the test output this
way instead of large pasted blobs.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: dfdd486f09321e9105fa747a8d1ac5a9a2a7a94a
2022-02-17 07:55:49 -08:00
Wojciech Tyczyński b028a2ea8d Unify merge GetToList and List storage methods to single GetList
Kubernetes-commit: f3d56369f71f5106eeec8300e77bd69b172733da
2022-02-10 10:15:29 +01:00
Wojciech Tyczyński 9f8f72fea9 Change storage.Interface to utilize recurisve parameter
Kubernetes-commit: 3016b3d8f868a3041c7cb86695de09d7ab27cf3b
2022-02-10 09:58:31 +01:00
Steve Kuznetsov 9eba72652e storage: etcd: TestList: improve RV choices
In this test, the current implementation uses a nebulous "RV 1" for some
queries. The intent of this absolute choice is to probe etcd at a
version before any writes ocurred for the test. The particular test
fixture for etcd that is used starts at revision 1, so 1 is used.

This choice is hard to understand the meaning of for readers, though,
and is not valid for any other etcd fixture used for the tests. In order
to improve readability of the test as well as to make it more resilient
to the underlying store, this change updates the test to read the
revision of the underlying storage before making any writes and using
that revision when querying the storage in the tests.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: d2b42b6369ab8db9d0aa0b58dcdf6548ff489d70
2022-02-17 08:36:30 -08:00
Steve Kuznetsov 67dd3e671d storage: etcd: pass contexts correctly in TestConsistentList
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 7272aca69d2602b0bcc97bb5cdc8139ba4687ceb
2022-02-16 13:18:27 -08:00
Steve Kuznetsov af1cb1cefe storage: transformers: pass a context.Context
When an envelope transformer calls out to KMS (for instance), it will be
very helpful to pass a `context.Context` to allow for cancellation. This
patch does that, while passing the previously-expected additional data
via a context value.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 27312feb9983c18d1daf00afba788727d024cdd0
2022-02-17 07:29:44 -08:00
Steve Kuznetsov dbc2925b64 storage: etcd: shorten sub-test names
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 4e166ff74d64042cbeabf72d908ffbf0243d4715
2022-02-16 14:10:43 -08:00
Steve Kuznetsov eaea50a0de storage: etcd: TestConditionalDelete: use sub-tests
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 7729691f5c08dd3aa05e3019190653648a52aba2
2022-02-16 13:29:16 -08:00
Steve Kuznetsov ce3e7f36a8 storage: etcd: TestUnconditionalDelete: use sub-tests
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: dbb00694f69a1f061d15fc3440a39c8ea360977d
2022-02-16 13:27:52 -08:00
Steve Kuznetsov a1c2b74171 storage: etcd: TestGuaranteedUpdate: use sub-tests
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: b6a66252174c59335d82e5b4990b930c8265a69c
2022-02-16 13:26:00 -08:00
Steve Kuznetsov cbb630c2dd storage: etcd: TestGetToList: use sub-tests
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: f1ded3b0c34c70c7a993d3c979571d96a29d56bc
2022-02-16 13:23:49 -08:00