Commit Graph

169 Commits

Author SHA1 Message Date
Steve Kuznetsov 42854c1734 pkg/storage/etcd3: be more precise in watch test
Previously, this test assumed that:
 - a global watch would return only an event for the key in question
 - only the delete event in question would be returned

Neither of these assumptions are correct for an etcd backend as long
as any other clients are interacting with the system. This commit
makes the watch more specific and extracts the correct event.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 2631c0a0f959bd67aa455045dce33e77150ab5f8
2022-03-23 12:59:53 -08:00
Abu Kashem 1b651c5994 add latency tracker for storage and transform
Kubernetes-commit: eca90856940e9251ecf3fde95c5e4d2d16f5ad68
2022-02-01 18:13:03 -05:00
Patrick Ohly ba3b8e9322 enhance and fix log calls
Some of these changes are cosmetic (repeatedly calling klog.V instead of
reusing the result), others address real issues:

- Logging a message only above a certain verbosity threshold without
  recording that verbosity level (if klog.V().Enabled() { klog.Info... }):
  this matters when using a logging backend which records the verbosity
  level.

- Passing a format string with parameters to a logging function that
  doesn't do string formatting.

All of these locations where found by the enhanced logcheck tool from
https://github.com/kubernetes/klog/pull/297.

In some cases it reports false positives, but those can be suppressed with
source code comments.

Kubernetes-commit: edffc700a43e610f641907290a5152ca593bad79
2022-02-16 12:17:47 +01:00
Steve Kuznetsov 25c879eec5 pkg/storage/etcd3: update test to time out, not panic
Without these select statements, this test runs until the package-global
timeout and causes a panic. This change makes the test fail faster and
more legibly.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: fc33d0176a5afb81927430d075165152f953c54e
2022-03-23 12:22:35 -08:00
Maciej Borsz f4edb394b0 Implement page size progressing for rare selectors.
Kubernetes-commit: a2ad9f9e4aba6aae6657a3189bdced6dbc8ba4b5
2022-03-07 15:44:59 +00:00
Steve Kuznetsov edb0c0eba4 pkg/storage/etcd3: use correct diff in tests
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 8903372e8b8d32f8b244f5087e3f2907b86f03bd
2022-03-23 12:33:57 -08:00
Steve Kuznetsov de7922ba23 pkg/storage/etcd3: refactor "too large" resourceVersions
When tests attempt to validate behavior in the case that a client asks
for a resource version that is "too large" for the underlying storage,
the previous implementation would simply add 1 to the latest revision
seen. This is only appropriate for storage backends that
  a) provide a continuous monotonic logical clock
  b) have no other events occurring while the test runs

For instance, when using a singe etcd backend as a shared fixture for
these tests, adding 1 to a previously-seen revision is not suffcient to
ensure that the resulting revision is "too large". By instead using the
largest possible integer value, we can be certain of this.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: b973cdc57cc6ee57684455cdb76db13a8c82cefa
2022-03-23 12:27:16 -08:00
astraw99 b0232c830b Fix function name of IsExist err
Kubernetes-commit: 79471695c2a16cb7c1fb17adff7765f07e2c9057
2022-03-19 23:06:51 +08:00
Steve Kuznetsov 3d5978d57e storage: etcd: clarify lease manager configurations
When the original commit created the lease manager, this comment was
added to set the default test reuse time to 1s. Even at that time, the
comment claimed it was setting 10s. Instead of using this value, though,
new tests that did not call `testSetup()` started to use the default
configuration for production. This commit clarifies the intent of this
comment, moves it next to the code block that it actually applies to,
and makes use of this test-specific logic everywhere.

x-ref: f230b000db

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 6aa37eb06247fb95a6a4ef61cbd50885e52055a0
2022-02-18 08:10:57 -08:00
Steve Kuznetsov 1501fbd930 storage: etcd: use cmp.Diff for comparisons
This commit simply modernizes the comparisons made in the storage tests
to use `cmp.Diff()` so that pointer comparisons and length checks do not
have to be made by hand. We also get nice diffs in the test output this
way instead of large pasted blobs.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: dfdd486f09321e9105fa747a8d1ac5a9a2a7a94a
2022-02-17 07:55:49 -08:00
Steve Kuznetsov fb315d8c17 store: etcd: TestWatch: use cmp.Diff
Modernize the comparisons used in the watch tests to use `cmp.Diff()` for
readability.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: d17a19b39d2dbdaf2cbbaad46de403d6d7ce0602
2022-02-18 06:51:36 -08:00
Steve Kuznetsov 557ecdf10d store: etcd: TestWatch: use sub-tests
This was the last test to not use sub-tests, so we can also remove the
indices that the expectation functions take as parameters now.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 9f7bb4264e0b79cbe7979c09f0e4c75a434a27bb
2022-02-18 06:50:05 -08:00
Wojciech Tyczyński b028a2ea8d Unify merge GetToList and List storage methods to single GetList
Kubernetes-commit: f3d56369f71f5106eeec8300e77bd69b172733da
2022-02-10 10:15:29 +01:00
Wojciech Tyczyński 9f8f72fea9 Change storage.Interface to utilize recurisve parameter
Kubernetes-commit: 3016b3d8f868a3041c7cb86695de09d7ab27cf3b
2022-02-10 09:58:31 +01:00
Steve Kuznetsov 9eba72652e storage: etcd: TestList: improve RV choices
In this test, the current implementation uses a nebulous "RV 1" for some
queries. The intent of this absolute choice is to probe etcd at a
version before any writes ocurred for the test. The particular test
fixture for etcd that is used starts at revision 1, so 1 is used.

This choice is hard to understand the meaning of for readers, though,
and is not valid for any other etcd fixture used for the tests. In order
to improve readability of the test as well as to make it more resilient
to the underlying store, this change updates the test to read the
revision of the underlying storage before making any writes and using
that revision when querying the storage in the tests.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: d2b42b6369ab8db9d0aa0b58dcdf6548ff489d70
2022-02-17 08:36:30 -08:00
Steve Kuznetsov 4074cf3cb4 storage: etcd: TestWatchError: improve readability
This test, as written, is *extremely* cryptic and hard to parse. Add a
comment and stop intentionally ignoring an error that only needs to be
ignored if we're being cryptic.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 50eed81923495f5ee1ac44436676ddbaf2a380fe
2022-02-17 15:46:05 -08:00
Steve Kuznetsov 67dd3e671d storage: etcd: pass contexts correctly in TestConsistentList
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 7272aca69d2602b0bcc97bb5cdc8139ba4687ceb
2022-02-16 13:18:27 -08:00
Steve Kuznetsov af1cb1cefe storage: transformers: pass a context.Context
When an envelope transformer calls out to KMS (for instance), it will be
very helpful to pass a `context.Context` to allow for cancellation. This
patch does that, while passing the previously-expected additional data
via a context value.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 27312feb9983c18d1daf00afba788727d024cdd0
2022-02-17 07:29:44 -08:00
Steve Kuznetsov dbc2925b64 storage: etcd: shorten sub-test names
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 4e166ff74d64042cbeabf72d908ffbf0243d4715
2022-02-16 14:10:43 -08:00
Steve Kuznetsov eaea50a0de storage: etcd: TestConditionalDelete: use sub-tests
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 7729691f5c08dd3aa05e3019190653648a52aba2
2022-02-16 13:29:16 -08:00
Steve Kuznetsov ce3e7f36a8 storage: etcd: TestUnconditionalDelete: use sub-tests
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: dbb00694f69a1f061d15fc3440a39c8ea360977d
2022-02-16 13:27:52 -08:00
Steve Kuznetsov a1c2b74171 storage: etcd: TestGuaranteedUpdate: use sub-tests
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: b6a66252174c59335d82e5b4990b930c8265a69c
2022-02-16 13:26:00 -08:00
Steve Kuznetsov cbb630c2dd storage: etcd: TestGetToList: use sub-tests
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: f1ded3b0c34c70c7a993d3c979571d96a29d56bc
2022-02-16 13:23:49 -08:00
Steve Kuznetsov ba213514c9 storage: etcd: remove a duplicate test case
This test case was a duplicate of the previous one.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 921e7525c074750a47818fdf89a4fe5c0b058f0f
2022-02-16 13:13:03 -08:00
Wojciech Tyczyński 46eb50ac00 Cleanup apiserver storage selflink references where possible
Kubernetes-commit: 9b2908ea3b98edbb1a8bcaf4e3e429283b2debfc
2022-01-19 21:13:25 +01:00
Davanum Srinivas c56bc7b872 OWNERS cleanup - Jan 2021 Week 1
Signed-off-by: Davanum Srinivas <davanum@gmail.com>

Kubernetes-commit: 9682b7248fb69733c2a0ee53618856e87b067f16
2022-01-03 10:59:47 -05:00
Patrick Ohly ec795ae204 avoid klog Info calls without verbosity
In the following code pattern, the log message will get logged with v=0 in JSON
output although conceptually it has a higher verbosity:

   if klog.V(5).Enabled() {
       klog.Info("hello world")
   }

Having the actual verbosity in the JSON output is relevant, for example for
filtering out only the important info messages. The solution is to use
klog.V(5).Info or something similar.

Whether the outer if is necessary at all depends on how complex the parameters
are. The return value of klog.V can be captured in a variable and be used
multiple times to avoid the overhead for that function call and to avoid
repeating the verbosity level.

Kubernetes-commit: 9eaa2dc554e0c3d4485d4c916dfdbc2f517db2e0
2021-12-11 12:10:21 +01:00
Wojciech Tyczyński b4935b065f Slightly improve reliability of storage List tests.
Kubernetes-commit: 8ca6c5922b9a0bafa8c83617d6930b981430334e
2021-12-10 12:05:32 +01:00
Davanum Srinivas b840d63feb Cleanup OWNERS files (No Activity in the last year)
Signed-off-by: Davanum Srinivas <davanum@gmail.com>

Kubernetes-commit: 497e9c1971c9e7d0193bc6d11503ec4ad527f1d5
2021-12-10 15:18:50 -05:00
Wojciech Tyczyński a55c2c834e Unify List and GetToList functions of etcd3
Kubernetes-commit: 27db8e34cfc930972beb8b5a33c443e134bcd614
2021-11-22 13:30:25 +01:00
Davanum Srinivas 56a3a30ae1 Check in OWNERS modified by update-yamlfmt.sh
Signed-off-by: Davanum Srinivas <davanum@gmail.com>

Kubernetes-commit: 9405e9b55ebcd461f161859a698b949ea3bde31d
2021-12-09 21:31:26 -05:00
Wojciech Tyczyński 71bfcaaea1 Simplify storage.Interface by merging Watch and WatchList functions.
Kubernetes-commit: 53653e9b0a7145c3b6e133c5639194ff79b3719d
2021-11-18 12:40:26 +01:00
Dr. Stefan Schimanski 2f0da0d2ea apiserver/storage: free etcd kv early in list decoding loop
Kubernetes-commit: dbf98e42664c8a3a104fe06e0e362497fca2902d
2021-10-08 14:05:26 +02:00
Mike Spreitzer 889106eff7 Add metrics about handling LIST requests
Add metrics that illuminate the costs of handling LIST requests.

Kubernetes-commit: bf424292db185546c474b27b963fd27bdbafc72f
2021-09-01 18:18:23 -04:00
Mike Spreitzer 31ec50ec43 Plumb the schema.GroupResource into etcd3 Store struct
Kubernetes-commit: 6f8019aae95db85552381f7e7066a52fef61be7a
2021-09-13 16:54:34 -04:00
Jordan Liggitt 29227fd4ee Avoid spurious calls to update/delete validation
Kubernetes-commit: 561ce035b5b7d577ddb5bb30657bc1f8b399e109
2021-08-05 17:21:40 -04:00
Ryan Phillips 7afffd8ef3 Revert "apiserver: add callback to get notified of object count"
Kubernetes-commit: d95b14e1abfb5ec87248e5dd826b89d0c738af42
2021-07-08 13:56:39 -05:00
Abu Kashem d929410e13 apiserver: add callback to get notified of object count
Kubernetes-commit: 1002b0d163dd948334f10b0e25d0e91d253791e8
2021-06-18 10:44:07 -04:00
Jordan Liggitt 4bb06f3c4d Simplify running embedded etcd server in tests
Kubernetes-commit: 01760927b82cf28bc6ba5b6364fda3c9f8c5c0b1
2021-06-15 12:07:46 -04:00
Jordan Liggitt aafb67c9b2 Quiet grpc info logs in apiserver
Kubernetes-commit: 503f5e1b52812f1bc5bb66d9b02a8f32cc5dccd9
2021-06-15 18:50:54 -04:00
Jordan Liggitt aaf5f32453 Quiet embedded etcd logs
Kubernetes-commit: f283deee6b65be2b435ec192b6444f969f9d2a42
2021-06-04 00:43:11 -04:00
Jordan Liggitt 8492f8d104 Drop use of deprecated clientv3.SetLogger
Kubernetes-commit: 52b629efbc8be30901192880b8005e7362f9a69f
2021-05-19 09:10:05 -04:00
Jordan Liggitt 6365bf9124 Switch to go.etcd.io/etcd/client/v3
Kubernetes-commit: 2979c3325e3ac4023dd19c435f3a995074328751
2021-03-23 11:01:52 -04:00
Jordan Liggitt 5d270fd014 Drop etcd v2 support from test util
Kubernetes-commit: 77ab4bdbbcc84ebb10645d810a087d272f16f803
2021-03-23 10:54:43 -04:00
wojtekt 5290a24345 Implement support for watch initialization in P&F
Kubernetes-commit: 0cc217647ca8be0820973b970124a072c27b6575
2021-05-07 12:49:06 +02:00
Maciej Borsz 770eb2bc44 Update watcher.go
Kubernetes-commit: 532e35ba31e4df0a2df7dd5f2930035ce9379559
2021-05-07 16:07:34 +02:00
卢振兴10069964 549cbbf8de fix broken link in some files
Kubernetes-commit: b29a5fb0746f772b38da570cd8fdc77396ffca31
2021-04-13 08:43:24 +08:00
Han Kang d21d96225a bump the deprecated version to 1.22
Change-Id: Ibefaa94151704fcaaa920541bbb9a8ad714c1d24

Kubernetes-commit: e7ee76efc0813da419045c80cc06a513fba6fc37
2021-03-29 07:54:12 -07:00
Han Kang 4ac82c26cf rename apiserver_storage_object_counts to conform to best practices (apiserver_storage_objects)
Change-Id: I8c08a37cd3ed7c943a6691185f750e55d389dd3a

Kubernetes-commit: 50e6f15deec3f9b9ffe52accd6aab5f9560b88ad
2021-03-10 04:32:27 -08:00
Yu Yi 09fb3448ff metrics: promote apiserver_storage_object_counts to stable
- rename etcd_object_counts to apiserver_storage_object_counts

CONTEXT https://github.com/kubernetes/kubernetes/issues/98270
FIXES https://github.com/kubernetes/kubernetes/issues/98270

Kubernetes-commit: 77950ba79b8b56bae34291093c69006975c7b775
2021-03-04 14:42:42 +00:00