Commit Graph

52 Commits

Author SHA1 Message Date
Wojciech Tyczyński a47bbc6347 Minor cleanup in etcd3 tests
Kubernetes-commit: 83399ccfb8822cc31a6a3a66e8352591d88feaa8
2022-10-31 18:02:40 +01:00
Wojciech Tyczyński a24d8963d1 Refactor remaining etcd3 tests to make them generic
Kubernetes-commit: e6ddb69e6fd06bf9a02fa731f12472f3c375762d
2022-10-31 18:02:19 +01:00
Wojciech Tyczyński 07db92af49 Fix setting resource version on deletion
Kubernetes-commit: bbcf5e38776f2b18026539a0fbcf3aa505386c1f
2022-10-26 21:37:19 +02:00
Wojciech Tyczyński 499bbb88dc Refactor WatchError test to make it generic
Kubernetes-commit: b3f9272d57a71a9c50b38754b7309ddf68adb93f
2022-10-28 11:35:58 +02:00
Wojciech Tyczyński f6802e074e Refactor storage tests using compaction
Kubernetes-commit: b02f172cbdf7e824d13a6a9c3a9b9fe4f3f92afb
2022-10-26 20:47:19 +02:00
Wojciech Tyczyński 834cf0fc14 Refactor compaction in etcd3 tests
Kubernetes-commit: 7da7ddd779f9ea835f0c57deae05e050c543066b
2022-10-26 16:34:28 +02:00
Wojciech Tyczyński d9a3685d6f Minor cleanup of storage tests
Kubernetes-commit: 5344bc5e1b7e50d5ff359c714d522b421b28bd12
2022-10-25 14:09:36 +02:00
Steve Kuznetsov 03363cd42b storage: move some list tests to generic package
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 162450c01c9004ded53bb77f651f9a3f30920b30
2022-05-12 15:16:07 -07:00
Steve Kuznetsov 80256820ce storage: move the APIObjectVersioner definition to storage
The means by which we extract and parse the version of an API object is
not specific to etcd3. In order to allow for a generic suite of tests
against any storage.Interface imlpementation, we need this logic to live
outside of the etcd3 package, or import cycles will exist.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 3939f3003e9605c06f65e64d1fc6f94b294f9d97
2022-05-11 07:44:21 -07:00
Steve Kuznetsov a4bcbc5187 storage/testing: move cancelled watch test to generic package
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 1f24bd91c30a3f0b623c452cffa9f043f202d98b
2022-05-09 08:23:58 -07:00
Steve Kuznetsov fe17bc776e etcd3/store: call a generic cancelled watch test
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: c0fc8172559f7796560a9b80fd23df92095adb14
2022-05-09 08:23:03 -07:00
Steve Kuznetsov 7b82bba67e etcd3/store: update cancelled watch test to be generic
There's no reason to create the watch using the underlying watcher.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 774870611c1d2b405d67eb73ade628ced5e0e994
2022-05-09 08:22:04 -07:00
Steve Kuznetsov b393e91916 storage/testing: wire things back together with imports
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 4ff560e4fa3c716a11f7260f33ec318faad59394
2022-05-05 11:32:05 -07:00
Steve Kuznetsov ebf7c468b8 storage/etcd3: move reusable tests to importable files
These can't live in `_test.go` files, or others won't be able to re-use
them as you can't import from test files.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 0bfeb728786726cd28b4c64d48874d45a2724165
2022-05-05 09:27:16 -07:00
Steve Kuznetsov cf5cf85de0 storage/etcd3: factor tests to accept `storage.Interface`
This commit simply factors the test functionality into functions that
accept `storage.Interface`.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 117f674cab1f598251483811fb769046dc3458a3
2022-05-05 09:18:08 -07:00
Steve Kuznetsov 169ec42cd5 storage/etcd3: continue unifying test setup
Previous work by liggitt in 01760927b82 improved the boilerplate
required to run an embedded etcd server for tests as well as set up the
`*etcd3.store{}` for testing. A number of tests were not ported to use the
new helpers, though, either due to custom setup or due to inconsistent
use of setup options. A follow-up by stevekuznetsov in 6aa37eb0624
removed much of the inconsistency, meaning that most callers to
`newStore()` were simply using the default boilerplate and options that
`testSetup()` used.

This patch moves all users to testSetup(), adding options as necessary
to enable some fringe setup use-cases. With a unified setup, new tests
will not copy boilerplate they do not need and it will be immediately
obvious when reading a test if the client or storage setup is *not*
default, improving readability.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 138faa3799341d02df9fc4bedc1371d338c34887
2022-04-22 07:39:32 -07:00
Steve Kuznetsov 90420e863e pkg/storage/etcd3: validate revision invariant
We must ensure that we notice if the etcd behavior on linearized reads
changes.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: ed5fd905f2b42e9919d99c40a1cb25014f0a7f89
2022-04-13 10:57:02 -07:00
Steve Kuznetsov 41c1f65bf6 pkg/storage/etcd3: correctly validate resourceVersions
In a number of tests, the underlying storage backend interaction will
return the revision (logical clock underpinning the MVCC implementation)
at the call-time of the RPC. Previously, the tests validated that this
returned revision was exactly equal to some previously seen revision.
This assertion is only true in systems where no other events are
advancing the logical clock. For instance, when using a single etcd
cluster as a shared fixture for these tests, the assertion is not valid
any longer. By checking that the returned revision is no older than the
previously seen revision, the validation logic is correct in all cases.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: eba25cdbbcc5d35e707516194f64d8ed363c2773
2022-03-23 12:44:49 -08:00
Steve Kuznetsov 42854c1734 pkg/storage/etcd3: be more precise in watch test
Previously, this test assumed that:
 - a global watch would return only an event for the key in question
 - only the delete event in question would be returned

Neither of these assumptions are correct for an etcd backend as long
as any other clients are interacting with the system. This commit
makes the watch more specific and extracts the correct event.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 2631c0a0f959bd67aa455045dce33e77150ab5f8
2022-03-23 12:59:53 -08:00
Steve Kuznetsov 25c879eec5 pkg/storage/etcd3: update test to time out, not panic
Without these select statements, this test runs until the package-global
timeout and causes a panic. This change makes the test fail faster and
more legibly.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: fc33d0176a5afb81927430d075165152f953c54e
2022-03-23 12:22:35 -08:00
Steve Kuznetsov edb0c0eba4 pkg/storage/etcd3: use correct diff in tests
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 8903372e8b8d32f8b244f5087e3f2907b86f03bd
2022-03-23 12:33:57 -08:00
Steve Kuznetsov 3d5978d57e storage: etcd: clarify lease manager configurations
When the original commit created the lease manager, this comment was
added to set the default test reuse time to 1s. Even at that time, the
comment claimed it was setting 10s. Instead of using this value, though,
new tests that did not call `testSetup()` started to use the default
configuration for production. This commit clarifies the intent of this
comment, moves it next to the code block that it actually applies to,
and makes use of this test-specific logic everywhere.

x-ref: f230b000db

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 6aa37eb06247fb95a6a4ef61cbd50885e52055a0
2022-02-18 08:10:57 -08:00
Steve Kuznetsov fb315d8c17 store: etcd: TestWatch: use cmp.Diff
Modernize the comparisons used in the watch tests to use `cmp.Diff()` for
readability.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: d17a19b39d2dbdaf2cbbaad46de403d6d7ce0602
2022-02-18 06:51:36 -08:00
Steve Kuznetsov 557ecdf10d store: etcd: TestWatch: use sub-tests
This was the last test to not use sub-tests, so we can also remove the
indices that the expectation functions take as parameters now.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 9f7bb4264e0b79cbe7979c09f0e4c75a434a27bb
2022-02-18 06:50:05 -08:00
Steve Kuznetsov 4074cf3cb4 storage: etcd: TestWatchError: improve readability
This test, as written, is *extremely* cryptic and hard to parse. Add a
comment and stop intentionally ignoring an error that only needs to be
ignored if we're being cryptic.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 50eed81923495f5ee1ac44436676ddbaf2a380fe
2022-02-17 15:46:05 -08:00
Wojciech Tyczyński 71bfcaaea1 Simplify storage.Interface by merging Watch and WatchList functions.
Kubernetes-commit: 53653e9b0a7145c3b6e133c5639194ff79b3719d
2021-11-18 12:40:26 +01:00
Mike Spreitzer 31ec50ec43 Plumb the schema.GroupResource into etcd3 Store struct
Kubernetes-commit: 6f8019aae95db85552381f7e7066a52fef61be7a
2021-09-13 16:54:34 -04:00
Jordan Liggitt 4bb06f3c4d Simplify running embedded etcd server in tests
Kubernetes-commit: 01760927b82cf28bc6ba5b6364fda3c9f8c5c0b1
2021-06-15 12:07:46 -04:00
Jordan Liggitt 6365bf9124 Switch to go.etcd.io/etcd/client/v3
Kubernetes-commit: 2979c3325e3ac4023dd19c435f3a995074328751
2021-03-23 11:01:52 -04:00
wojtekt 5290a24345 Implement support for watch initialization in P&F
Kubernetes-commit: 0cc217647ca8be0820973b970124a072c27b6575
2021-05-07 12:49:06 +02:00
Ling Samuel bd7da186cc apiserver add metric etcd_lease_object_counts
Signed-off-by: Ling Samuel <lingsamuelgrace@gmail.com>

Kubernetes-commit: 7e9fe39cd7c3ee67fcc29e6ca0836917c5a4e9e1
2020-12-07 10:45:44 +08:00
wojtekt 8c25869e43 Add suggestion to storage interface Delete method
Kubernetes-commit: c2d61896f45967cdea15072b7a12faeb635b5dc5
2020-11-03 13:21:00 +01:00
Ling Samuel f673ce24f7 apiserver add --lease-reuse-duration-seconds to config lease reuse duration
Signed-off-by: Ling Samuel <lingsamuelgrace@gmail.com>

Kubernetes-commit: c99567005db15d220f2b792e98af735c5aa24f6e
2020-12-02 17:35:10 +08:00
Ken Sipe 5d58b175c8 fix S1021 var declaration
Signed-off-by: Ken Sipe <kensipe@gmail.com>

Kubernetes-commit: 6c49299739a9819c3672248517ab3d6636d1d8c6
2020-06-25 17:10:34 -05:00
wojtekt 942f220e85 Remove variadic argument from storage interface
Kubernetes-commit: 8b98305858b107369f2c9b9fd8ef1c5b0da078c0
2020-11-02 15:52:51 +01:00
wojtekt e40efde837 Implement etcd3 progress-notify feature in etcd3 layer
Kubernetes-commit: 56e72841b6005740453828a9f4f7a9a1b9a831f6
2020-08-31 12:40:41 +02:00
wojtekt a5000473c1 Pipe newFunc to etcd3 storage layer
Kubernetes-commit: fbd65a265a47ffe081aaac2f794a55034333d11d
2020-08-31 11:58:45 +02:00
Joe Betz 5b75cf0ff9 Add GetOptions and ListOptions to storage interface
Kubernetes-commit: 4c99949ae6b4a74e4ffda0caba86c151c23d579e
2020-05-29 10:48:33 -07:00
Monis Khan f5c1e085ed storage: confirm that paging and predicate filtering work together
This change adds the TestListContinuationWithFilter test which
confirms that paging with a predicate that does not match everything
results in the correct amount of calls to TransformFromStorage and
KV.Get.  The partial result of each paging call is also asserted.

Signed-off-by: Monis Khan <mok@vmware.com>

Kubernetes-commit: 002c75442d768d2bcc51047667354ff16bbfa2e8
2020-02-28 15:35:22 -05:00
Wenjia Zhang 41da2ced4c Replace github.com/coreos/etcd by go.etcd.io/etcd
Kubernetes-commit: 3b274fad2a719dc1fd0eaef6c55a0d344db10242
2019-10-15 22:38:43 -07:00
Chao Xu 5c9333e4d3 Run deleteValidation at the storage layer so that it will be retried on
conflict.

Adding unit test verify that deleteValidation is retried.

adding e2e test verifying the webhook can intercept configmap and custom
resource deletion, and the existing object is sent via the
admissionreview.OldObject.

update the admission integration test to verify that the existing object
is passed to the deletion admission webhook as oldObject, in case of an
immediate deletion and in case of an update-on-delete.

Kubernetes-commit: 7bb4a3bace048cb9cd93d0221a7bf7c4accbf6be
2019-04-09 13:49:16 -07:00
Jordan Liggitt 80029a760c Deprecate and remove use of alpha metadata.initializers field, remove IncludeUninitialized options
Kubernetes-commit: 17aa60686ebe5fd04b4fe6f442dc36a8d70b6730
2019-01-17 09:19:28 -05:00
Jordan Liggitt bd604a62aa Remove deprecated --etcd-quorum-read flag
Kubernetes-commit: cff79c542130831f4a212099974570244a0c9586
2018-10-08 11:04:28 -04:00
David Eads cb5cac48ee make package name match all the import aliases
Kubernetes-commit: d3bd0eb1d5cefc25e4476d8dc086ebd90439ef4e
2018-08-01 10:01:32 -04:00
fisherxu dd4ba939ef collapse the resource version parse
Kubernetes-commit: 68ec856b4506d4440b327141051bdf31fc92be42
2018-06-23 17:29:06 +08:00
hzxuzhonghu 6ab99203b7 Replace "golang.org/x/net/context" with "context"
Kubernetes-commit: 70e45eccf27726f0e63dd1024924ccc7e2cd35a0
2018-02-28 12:20:22 +08:00
Daniel Smith 4b163fbe32 resource version parsing should all be in one place
Kubernetes-commit: 023895d597be6539a1a16fa54d60e47a17d85dc1
2018-01-10 10:43:59 -08:00
Clayton Coleman 460257fd61 Server side implementation of paging for etcd3
Add a feature gate in the apiserver to control whether paging can be
used. Add controls to the storage factory that allow it to be disabled
per resource. Use a JSON encoded continuation token that can be
versioned. Create a 410 error if the continuation token is expired.

Adds GetContinue() to ListMeta.

Kubernetes-commit: 8952a0cb722b77459cf2701632a30f5b264f5aba
2017-09-03 14:04:12 +00:00
Clayton Coleman 5fa08b8c5e Allow initialization of resources
Add support for creating resources that are not immediately visible to
naive clients, but must first be initialized by one or more privileged
cluster agents. These controllers can mark the object as initialized,
allowing others to see them.

Permission to override initialization defaults or modify an initializing
object is limited per resource to a virtual subresource "RESOURCE/initialize"
via RBAC.

Initialization is currently alpha.

Kubernetes-commit: 331eea67d8000e5c4b37e2234a90903c15881c2f
2017-06-13 20:47:30 +00:00
Monis Khan 6794013a5b Panic server on watch errors in test environment
This change makes it so that errors during watch decoding panic the
server if it is in a test environment.  This allows us to catch coder
errors related to storing incompatible types at the same location in
etcd.

Signed-off-by: Monis Khan <mkhan@redhat.com>

Kubernetes-commit: a13f026fd012859f04467e6007e2cafe4a788927
2017-06-13 20:47:29 +00:00