Commit Graph

496 Commits

Author SHA1 Message Date
Maciej Wyrzuc cb0bb2af35 Add additional etcd check to readyz with 2 seconds timeout.
Kubernetes-commit: b42045a64fd07fb948660839b6c7c14440bee9df
2022-07-25 13:08:50 +00:00
Davanum Srinivas 7e94033a61 Generate and format files
- Run hack/update-codegen.sh
- Run hack/update-generated-device-plugin.sh
- Run hack/update-generated-protobuf.sh
- Run hack/update-generated-runtime.sh
- Run hack/update-generated-swagger-docs.sh
- Run hack/update-openapi-spec.sh
- Run hack/update-gofmt.sh

Signed-off-by: Davanum Srinivas <davanum@gmail.com>

Kubernetes-commit: a9593d634c6a053848413e600dadbf974627515f
2022-07-19 20:54:13 -04:00
Mikko Ylinen 12a8b7fef3 grpc: move to use grpc.WithTransportCredentials()
v1.43.0 marked grpc.WithInsecure() deprecated so this commit moves to use
what is the recommended replacement:

grpc.WithTransportCredentials(insecure.NewCredentials())

Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>

Kubernetes-commit: 2c8bfad9106039aa15233b5bf7282b25a7b7e0a0
2022-05-11 12:13:28 +03:00
Wojciech Tyczyński 2049cfbb9e Adjust watch channel sizes in watchcache
Kubernetes-commit: 0db5c05bdb8bbc510307a48cbade712583bb009e
2022-04-28 11:56:41 +02:00
Lukasz Szaszkiewicz 27a7c443bd cacher: with expiredBookmarkWatchers
expiredBookmarkWatchers allows us to schedule the next bookmark event after dispatching not before as it was previously.
It opens a new functionality in which a watcher might decide to change when the next bookmark should be delivered based on some internal state.

Kubernetes-commit: 0576f6a011cba8f0c8550fd3dd31111376c9dcd0
2022-07-15 15:28:50 +02:00
Artur Żyliński e34c622d49 Add audit-id to storage traces
Refactor GetAuditIDTruncated to use context instead of request

Kubernetes-commit: b1e12b01b6c578da3eb593805b48e9d4a69efe54
2022-06-20 17:09:32 +02:00
zk ad01c712d9 Update etcdRequestLatency metrics bucket size
Kubernetes-commit: fb372d07982463984dfdab7cf24d937b91b0bb83
2022-01-09 23:45:09 +08:00
Madhav Jivrajani 218ed2fae0 cacher: Use PodList type for use in GetList
Using a Pod type in a GetList() call in a test
can panic at worst and error out at best. Here,
neither happened because the error condition
being tested for (cacher being stopped or not)
gets returned before the list pointer can be
enforced.

This commit changes the above to use PodList.

Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>

Kubernetes-commit: 487761f4e2543114db158f0d59e598dedc481882
2022-07-15 12:22:04 +05:30
Anish Ramasekar c6c1465ed7 Add KMS v2alpha1 API
Signed-off-by: Anish Ramasekar <anish.ramasekar@gmail.com>

Kubernetes-commit: 907545445ab8b4e34c1068ab9828a930c30cbfc4
2022-05-24 23:43:09 +00:00
Abirdcfly 6be1a93b55 fix: --chunk-size with selector returns missing result
Signed-off-by: Abirdcfly <fp544037857@gmail.com>

Kubernetes-commit: fba297629d82866078faa6dc076cb5f81633e557
2022-06-20 00:54:57 +08:00
Han Kang a414002089 cleanup deprecated metrics and usages
Kubernetes-commit: f223b900907b71431d7b6ceefa1642bb44fd9d84
2022-06-01 11:55:14 -07:00
Han Kang de98becc1d add explicit typing for continue tests
Our tests are mostly error based and explicit error typing allows
us to test against error types directly. Having made this change also
makes it obvious that our test coverage was lacking in two branches,
specifically, we were previously not testing empty start keys nor were
we testing for invalid start RVs.

Kubernetes-commit: 213e380a2e48830db6c71d2da5485d4226d95625
2022-05-31 10:23:07 -07:00
Steve Kuznetsov 03363cd42b storage: move some list tests to generic package
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 162450c01c9004ded53bb77f651f9a3f30920b30
2022-05-12 15:16:07 -07:00
Steve Kuznetsov 289ed89252 storage/etcd3: make some list tests generic
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: a8067f8e865475cbdb11d812c915927714690676
2022-05-12 15:13:23 -07:00
Steve Kuznetsov ae4cc8bfc8 storage/etcd3: factor out non-paginated list tests
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 8fcf00ef9171cfb623525d2a6aea1d721c3b5e74
2022-05-11 07:18:05 -07:00
Steve Kuznetsov 139068ca91 storage/etcd3: factor store setup into a separate function
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 31d7a28c65c34f2e2a28cfc41c2c517d370e7045
2022-05-11 07:17:26 -07:00
Steve Kuznetsov b2d4efbca3 apiserver/storage: add myself to the reviewers list
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 0d616946c036d5ead9dd03e312c6ac16f9a91c3e
2022-05-12 15:40:33 -07:00
Steve Kuznetsov 6bff431f4a storage: move test utiltiies out of test files
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: e50cb6c8d36ca73c99c935126978cbec01f9be01
2022-05-12 11:52:28 -07:00
Steve Kuznetsov 67344c728d storage: move continue token definition to storage
The means by which we encode and decode the continue token during a
paginated LIST call is not specific to etcd3. In order to allow for a
generic suite of tests against any storage.Interface implementation, we
need this logic to live outside of the etcd3 package, or import cycles
will exist.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: eb3aa5be10393968d8083c79f5958501fc029e8d
2022-05-11 07:52:02 -07:00
Steve Kuznetsov 0a2d58a552 storage: isolate the test resource
This test resource only exists to test the versioning logic, but it
cannot live in the general testing package, or import cycles will exist.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: fa2a6d633984aaa36ef988ecf4410c20e27897ea
2022-05-11 09:05:18 -07:00
Steve Kuznetsov 80256820ce storage: move the APIObjectVersioner definition to storage
The means by which we extract and parse the version of an API object is
not specific to etcd3. In order to allow for a generic suite of tests
against any storage.Interface imlpementation, we need this logic to live
outside of the etcd3 package, or import cycles will exist.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 3939f3003e9605c06f65e64d1fc6f94b294f9d97
2022-05-11 07:44:21 -07:00
Wojciech Tyczyński 0a7c4bcca1 Cleanup etcd healthcheck on shutdown
Kubernetes-commit: cb80082f666e0e5fe220df32e31a8face18e9393
2022-05-10 11:12:08 +02:00
Antonio Ojea 00730c9649 apiserver cacher: don't accept requests if stopped
The cacher blocks requests until it is ready, however, the
ready variable doesn't differentiate if the cacher was stopped.

The cacher is using a condition variable based on sync.Cond to
handle the readiness, however, this was not taking into account
if it was not ready because it was waiting to be ready or it was
stopped.

Add a new condition to the condition variable to handle the
stop condition, and returning an error to signal the goroutines
that they should stop waiting and bail out.

Kubernetes-commit: 2cb3a56e83ae33464edb174b1b6373ba50600759
2022-03-03 12:01:59 +01:00
Steve Kuznetsov a4bcbc5187 storage/testing: move cancelled watch test to generic package
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 1f24bd91c30a3f0b623c452cffa9f043f202d98b
2022-05-09 08:23:58 -07:00
Steve Kuznetsov fe17bc776e etcd3/store: call a generic cancelled watch test
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: c0fc8172559f7796560a9b80fd23df92095adb14
2022-05-09 08:23:03 -07:00
Steve Kuznetsov 7b82bba67e etcd3/store: update cancelled watch test to be generic
There's no reason to create the watch using the underlying watcher.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 774870611c1d2b405d67eb73ade628ced5e0e994
2022-05-09 08:22:04 -07:00
Steve Kuznetsov 3fd88b5da7 storage/testing: move creation test to generic package
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 2e118f42465efb390bec005f1853a964dd339f97
2022-05-09 08:04:53 -07:00
Steve Kuznetsov daf507c2ff etcd3/store: make creation test validation generic
Different callers to this test may need to do different backend-specific
validation on the stored data, so we allow them a callback for this.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 6d25e96cedaad249fe75aac4b1fe08bb69829a61
2022-05-09 08:03:41 -07:00
Steve Kuznetsov f54afb0566 etcd3/store: update creation test to use storage client
There is no functional difference between checking for an empty key
using the database client and doing so with the storage interface. Using
the latter allows this test to be more portable.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: f894f8196d9266915424e2cefc9e4eb480ae6f5e
2022-05-09 07:59:38 -07:00
Daniel Smith 088283fa36 Remove ClusterName
Kubernetes-commit: 331525670b772eb8956b7f5204078c51c00aaef3
2022-04-20 23:50:27 +00:00
Steve Kuznetsov b393e91916 storage/testing: wire things back together with imports
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 4ff560e4fa3c716a11f7260f33ec318faad59394
2022-05-05 11:32:05 -07:00
Steve Kuznetsov ebf7c468b8 storage/etcd3: move reusable tests to importable files
These can't live in `_test.go` files, or others won't be able to re-use
them as you can't import from test files.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 0bfeb728786726cd28b4c64d48874d45a2724165
2022-05-05 09:27:16 -07:00
Steve Kuznetsov cf5cf85de0 storage/etcd3: factor tests to accept `storage.Interface`
This commit simply factors the test functionality into functions that
accept `storage.Interface`.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 117f674cab1f598251483811fb769046dc3458a3
2022-05-05 09:18:08 -07:00
Han Kang 24d28aad81 refactor watch cache metrics so that they are in a single place
Change-Id: I7874e90d4ec111e82450273f27f8e50556cb096a

Kubernetes-commit: a569829f599acf7ac3acbd3902f41d2c74fc2bd7
2022-04-22 14:21:25 -07:00
Maciej Borsz 17c48c0f65 Improve Create/Update/Patch traces.
Kubernetes-commit: 56769e794b757c541d1c7fef36a5507e52e3e7b2
2022-04-20 07:30:12 +00:00
Steve Kuznetsov d96402b9d6 storage/etcd3: clarify the pagingation flow in LIST
It is not possible for the nil-check to ever return anything different
from what the explicit boolean used to, but this is only something that
a reader can come to the conclusion on if they very, very carefuly read
the code. Instead of having this implicit flow that is difficult to
follow, let's keep the boolean.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 809fd64b289add1b378b45c748c23b7278c366f1
2022-04-22 11:50:24 -07:00
Steve Kuznetsov 169ec42cd5 storage/etcd3: continue unifying test setup
Previous work by liggitt in 01760927b82 improved the boilerplate
required to run an embedded etcd server for tests as well as set up the
`*etcd3.store{}` for testing. A number of tests were not ported to use the
new helpers, though, either due to custom setup or due to inconsistent
use of setup options. A follow-up by stevekuznetsov in 6aa37eb0624
removed much of the inconsistency, meaning that most callers to
`newStore()` were simply using the default boilerplate and options that
`testSetup()` used.

This patch moves all users to testSetup(), adding options as necessary
to enable some fringe setup use-cases. With a unified setup, new tests
will not copy boilerplate they do not need and it will be immediately
obvious when reading a test if the client or storage setup is *not*
default, improving readability.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 138faa3799341d02df9fc4bedc1371d338c34887
2022-04-22 07:39:32 -07:00
Han Kang 546d4052ee remove stutter from alpha metric
Change-Id: I6669225943a4196cfe70659fa296a0f81a0ab682

Kubernetes-commit: a9665c2d5834d3db881a524f3f64622949465a1d
2022-04-20 16:56:00 -07:00
Sanskar Jaiswal d1bd5277fb Update comment and declaration of `storage.GuaranteedUpdate` to be clearer.
Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>

Kubernetes-commit: f8df26ae803103f82edbf1efe17b1b169801b256
2022-04-16 17:19:06 +05:30
Steve Kuznetsov 90420e863e pkg/storage/etcd3: validate revision invariant
We must ensure that we notice if the etcd behavior on linearized reads
changes.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: ed5fd905f2b42e9919d99c40a1cb25014f0a7f89
2022-04-13 10:57:02 -07:00
Steve Kuznetsov 41c1f65bf6 pkg/storage/etcd3: correctly validate resourceVersions
In a number of tests, the underlying storage backend interaction will
return the revision (logical clock underpinning the MVCC implementation)
at the call-time of the RPC. Previously, the tests validated that this
returned revision was exactly equal to some previously seen revision.
This assertion is only true in systems where no other events are
advancing the logical clock. For instance, when using a single etcd
cluster as a shared fixture for these tests, the assertion is not valid
any longer. By checking that the returned revision is no older than the
previously seen revision, the validation logic is correct in all cases.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: eba25cdbbcc5d35e707516194f64d8ed363c2773
2022-03-23 12:44:49 -08:00
Anish Ramasekar e442eafb33 feat: prepare KMS data encryption for migration to AES-GCM
Signed-off-by: Anish Ramasekar <anish.ramasekar@gmail.com>
Co-authored-by: Monis Khan <mok@vmware.com>
Signed-off-by: Anish Ramasekar <anish.ramasekar@gmail.com>

Kubernetes-commit: 90b42f91fd904b71fd52ca9ae55a5de73e6b779a
2022-03-16 17:54:10 +00:00
Steve Kuznetsov 42854c1734 pkg/storage/etcd3: be more precise in watch test
Previously, this test assumed that:
 - a global watch would return only an event for the key in question
 - only the delete event in question would be returned

Neither of these assumptions are correct for an etcd backend as long
as any other clients are interacting with the system. This commit
makes the watch more specific and extracts the correct event.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 2631c0a0f959bd67aa455045dce33e77150ab5f8
2022-03-23 12:59:53 -08:00
Abu Kashem 1b651c5994 add latency tracker for storage and transform
Kubernetes-commit: eca90856940e9251ecf3fde95c5e4d2d16f5ad68
2022-02-01 18:13:03 -05:00
Patrick Ohly ba3b8e9322 enhance and fix log calls
Some of these changes are cosmetic (repeatedly calling klog.V instead of
reusing the result), others address real issues:

- Logging a message only above a certain verbosity threshold without
  recording that verbosity level (if klog.V().Enabled() { klog.Info... }):
  this matters when using a logging backend which records the verbosity
  level.

- Passing a format string with parameters to a logging function that
  doesn't do string formatting.

All of these locations where found by the enhanced logcheck tool from
https://github.com/kubernetes/klog/pull/297.

In some cases it reports false positives, but those can be suppressed with
source code comments.

Kubernetes-commit: edffc700a43e610f641907290a5152ca593bad79
2022-02-16 12:17:47 +01:00
Steve Kuznetsov 25c879eec5 pkg/storage/etcd3: update test to time out, not panic
Without these select statements, this test runs until the package-global
timeout and causes a panic. This change makes the test fail faster and
more legibly.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: fc33d0176a5afb81927430d075165152f953c54e
2022-03-23 12:22:35 -08:00
Maciej Borsz f4edb394b0 Implement page size progressing for rare selectors.
Kubernetes-commit: a2ad9f9e4aba6aae6657a3189bdced6dbc8ba4b5
2022-03-07 15:44:59 +00:00
Steve Kuznetsov edb0c0eba4 pkg/storage/etcd3: use correct diff in tests
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 8903372e8b8d32f8b244f5087e3f2907b86f03bd
2022-03-23 12:33:57 -08:00
Steve Kuznetsov de7922ba23 pkg/storage/etcd3: refactor "too large" resourceVersions
When tests attempt to validate behavior in the case that a client asks
for a resource version that is "too large" for the underlying storage,
the previous implementation would simply add 1 to the latest revision
seen. This is only appropriate for storage backends that
  a) provide a continuous monotonic logical clock
  b) have no other events occurring while the test runs

For instance, when using a singe etcd backend as a shared fixture for
these tests, adding 1 to a previously-seen revision is not suffcient to
ensure that the resulting revision is "too large". By instead using the
largest possible integer value, we can be certain of this.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: b973cdc57cc6ee57684455cdb76db13a8c82cefa
2022-03-23 12:27:16 -08:00
Daniel Smith 7b0bc4f4b3 remove unneeded references
Kubernetes-commit: 2831f9a343ec405efce60d09da482a654971018e
2022-03-17 18:35:00 +00:00