Commit Graph

117 Commits

Author SHA1 Message Date
HirazawaUi bc8676d59a Add decoding time to the audit log
Kubernetes-commit: 20fe2a3539e90f7554f94359ac3b4058a5bbb363
2023-10-25 22:52:11 +08:00
HirazawaUi 88cd5253fa make the decode function respect the timeout context
Kubernetes-commit: 3fb16abfca43f53048a28758f2366cb52e31d2b7
2023-11-08 23:39:10 +08:00
Siyuan Zhang cef6db2cfb k8s.io/apiserver/storage/etcd: refactor etcd GetList.
Extract the logic to determine withRev to a separate method for better readability.

Signed-off-by: Siyuan Zhang <sizhang@google.com>

Kubernetes-commit: 624169c5b50ee8a6e9a761e9488134985334817e
2023-10-20 12:59:46 -07:00
Wojciech Tyczynski 6caf326620 Revert "Make the decode function respect the timeout context"
Kubernetes-commit: 98a2f22e740ccd2c30711f1b21d6383f1b91595e
2023-10-31 16:27:17 +01:00
Siyuan Zhang ef409f941b k8s.io/apiserver/storage/etcd: refactor etcd GetList.
Reorder some code.

Signed-off-by: Siyuan Zhang <sizhang@google.com>

Kubernetes-commit: a968f51fa2f87ed57f9e48ba436e11421c403b27
2023-10-20 12:39:51 -07:00
Siyuan Zhang 49886c205c k8s.io/apiserver/storage/etcd: refactor etcd GetList.
reduce redundant update of withRev after request.

Signed-off-by: Siyuan Zhang <sizhang@google.com>

Kubernetes-commit: 84ec5e2eccbc07b17f3b3e3e00dc3996105e0346
2023-10-20 12:06:46 -07:00
HirazawaUi be73f76247 Make the decode function respect the timeout context
Kubernetes-commit: f78b367db6393a449b8f456e725cbe155d9b90e6
2023-10-25 23:12:16 +08:00
Wojciech Tyczyński 324d2bc3cf Cleanup paging parameters in etcd3 store
Kubernetes-commit: 3f4d3b67682335db510f85deb65b322127a3a0a1
2023-10-20 15:38:59 +02:00
Siyuan Zhang 70af178d56 k8s.io/apiserver/storage: add a new TestCreate case.
Add a test case of create with rv set.

Signed-off-by: Siyuan Zhang <sizhang@google.com>

Kubernetes-commit: 2f923e356e042d9dce88e8f78abf30f414051e71
2023-10-11 10:24:31 -07:00
Siyuan Zhang e15d4d2e0b k8s.io/apiserver/storage/etcd: refactor getCurrentState.
Extract getCurrentState as a separate method that can be reused.

Signed-off-by: Siyuan Zhang <sizhang@google.com>

Kubernetes-commit: ebca5d438d9cb2c82d0b99dbcb0aeca8879db441
2023-10-11 09:56:07 -07:00
Lukasz Szaszkiewicz 90ba08909c storage/etcd: the watcher supports the API streaming
Kubernetes-commit: ca562fd280a9dd5db952eb8c14b93fcf6668cc49
2023-08-22 14:02:39 +02:00
Marek Siarkowicz 8b7824a7e8 Avoid creating local variables that don't change
Having local variables gives false impression that this is overwritten
in the function block.

Kubernetes-commit: e01bd641447a315e28fab8148e99ac6afba9bcd7
2023-08-30 16:51:40 +02:00
Marek Siarkowicz 90adbb75eb Flatten switch case
Kubernetes-commit: 10553a1966892e305252c65ebeb9043416304f48
2023-08-30 16:42:34 +02:00
Marek Siarkowicz 5f9b5dda94 Refactor common WithRange case
From API call WithRange and WithPrefix work the same, they just set the range end.
The difference is when the range end is provided:
* WithRange(end) requires providing the end while calling
* WithPrefix() calculates the end based on key provided to the Get.

For example, those are equal:
* client.Get(ctx, "/pods/", WithPrefix())
* client.Get(ctx, "/pods/", WithRange(GetPrfixRangeEnd("/pods/")))

As keyPrefix is equal preparedKey there should not be a difference.

Kubernetes-commit: 1f4f2a5d6014dc8f98b25a9484d4a6064a6ae18e
2023-08-30 16:26:20 +02:00
Marek Siarkowicz a8fba27b08 Remove duplicated returnRV variable
returnRV was was equal to withRev, but updated at different time.

When preparing the request they are set equal to each other.

The only difference was during the for loop.
returnRV was always set no matter if pagination was enabled, while withRev only when paginating.

Kubernetes-commit: be4692864bb983e94e8d7b6b6aa1a9c22fe23bce
2023-08-30 17:36:05 +02:00
Lukasz Szaszkiewicz f75c503352 storage/factory: extend the Create method by newList and resourcePrefix params
Kubernetes-commit: ccabc01093a1344ebb27c32c946e9da3b8e91fd2
2023-07-28 09:53:01 +02:00
Wojciech Tyczyński f56b9ee7f5 Graduate RemainingItemCount to GA
Kubernetes-commit: 4e2e059c7b205d2e4b246a262128223258a49498
2023-07-21 15:22:51 +02:00
Lukasz Szaszkiewicz 7ecfbf7914 storage/etcd: simplify passing the transformer
Kubernetes-commit: a5600b6925169a7d99376d52472e5f8e8635082b
2023-08-21 12:27:41 +02:00
Lukasz Szaszkiewicz dc7ff6e034 storage/etcd: remove newWatcher function
Kubernetes-commit: 6161752ecbe434b8fb157dd3a30bb5c0cef7a23e
2023-08-21 12:13:12 +02:00
Lukasz Szaszkiewicz 2d9be35745 storage/etcd: no-op, refactor watcher.Watch method signature
Kubernetes-commit: f7e659db236286ca15707bf08acb08dc0ac4ab0e
2023-07-24 12:33:03 +02:00
Marek Siarkowicz cd751eb82e Implement conditionalProgressRequester that allows requesting watch progress notification if watch cache is not fresh
Kubernetes-commit: 98461be8ffa7383152c442414a16adb217e98080
2023-07-10 18:10:49 +02:00
Marek Siarkowicz 96d1438818 Refactor WithRequireLeader to make it part of the etcd store
Kubernetes-commit: a9af2de8fdd1f2799b2017b19027958c69ffb04d
2023-06-21 14:21:24 +02:00
Lukasz Szaszkiewicz decaec94cd storage/etcd: skip SendInitialEvents if the request is backward compatible
otherwise an error will be returned.
backward compatibility is defined as RV = "" || RV = "O" and AllowWatchBookmark is set to false.
in that case we rely on 267eb25e60/staging/src/k8s.io/apiserver/pkg/storage/etcd3/watcher.go (L260)

Kubernetes-commit: f2de1a00b8fd7a22f9d0b2c1ace69be41d304f83
2023-05-08 13:04:31 +02:00
iyear b1805a9a4d apiserver/etcd: add request and request_error metrics (#117222)
* apiserver/etcd: add request and request_error metrics

Signed-off-by: iyear <ljyngup@gmail.com>

* apiserver/etcd: rename etcdRequestCounts metric

Signed-off-by: iyear <ljyngup@gmail.com>

---------

Signed-off-by: iyear <ljyngup@gmail.com>

Kubernetes-commit: 4f6714ef42dcdb1b35bb2d8581c1ef83d59f02d1
2023-04-12 22:02:45 +08:00
Lukasz Szaszkiewicz 10e39104a7 Disable SendInitialEvents for etcd and watch cache
Kubernetes-commit: 7b67bbbcf361983e563663424e13f2b6cf6d5392
2023-03-01 11:00:12 +01:00
baomingwang 3f56cdd970 Added serialization from etcd error metric
Kubernetes-commit: 92c490f0bca03b6caea896ea10415a0d77c9efdc
2022-12-08 19:34:09 -08:00
Wojciech Tyczyński 8cd19efa73 Fix inconsistency in returning list results both in etcd3 and watchcache
Kubernetes-commit: da8d197a7205c01fdf376246995b0a1b3ab53859
2022-11-08 11:47:28 +01:00
Tim Allclair 3268b0561b Validate etcd paths
Kubernetes-commit: f1693a02c46f79f8c90a4ef17c4a750034f21484
2022-10-10 18:15:22 -07:00
Wojciech Tyczyński 07db92af49 Fix setting resource version on deletion
Kubernetes-commit: bbcf5e38776f2b18026539a0fbcf3aa505386c1f
2022-10-26 21:37:19 +02:00
Wojciech Tyczyński 499bbb88dc Refactor WatchError test to make it generic
Kubernetes-commit: b3f9272d57a71a9c50b38754b7309ddf68adb93f
2022-10-28 11:35:58 +02:00
David Ashpole aa161f2fc0 migrate apiserver utiltrace usage to component-base/tracing
Kubernetes-commit: de26b9023f2872c5cd7e15fad5dd5ab649222c13
2022-10-20 18:15:38 +00:00
Tim Allclair bd7c7f52c2 Consolidate AuditContext
Kubernetes-commit: f1d684b7b60b39b7dc1eb4156307c593f0ba74e1
2022-07-12 11:53:57 -07:00
viveksahu26 6cb2fda0ab fix warnings or linter errors
Signed-off-by: viveksahu26 <vivekkumarsahu650@gmail.com>

Kubernetes-commit: 386bc4a7fa3e975cb247d300fdc5ad1b14a15605
2022-09-05 19:10:43 +05:30
Andy Goldstein 8587d29223 etcd3: include GroupResource in logs/metrics
Use GroupResource instead of object reflection when recording the
following metrics:

- etcd_request_duration_seconds
- etcd_bookmark_counts

Add GroupResource to logs and traces where only reflection-based typing
was previously used.

Both of these changes allow us to disginguish between different CRDs,
all of which are represented as *unstructured.Unstructured.

Signed-off-by: Andy Goldstein <andy.goldstein@redhat.com>

Kubernetes-commit: 305fa2add60ad507417304d865f001006d5175fe
2022-08-25 13:55:55 -04:00
Artur Żyliński e34c622d49 Add audit-id to storage traces
Refactor GetAuditIDTruncated to use context instead of request

Kubernetes-commit: b1e12b01b6c578da3eb593805b48e9d4a69efe54
2022-06-20 17:09:32 +02:00
Abirdcfly 6be1a93b55 fix: --chunk-size with selector returns missing result
Signed-off-by: Abirdcfly <fp544037857@gmail.com>

Kubernetes-commit: fba297629d82866078faa6dc076cb5f81633e557
2022-06-20 00:54:57 +08:00
Steve Kuznetsov 67344c728d storage: move continue token definition to storage
The means by which we encode and decode the continue token during a
paginated LIST call is not specific to etcd3. In order to allow for a
generic suite of tests against any storage.Interface implementation, we
need this logic to live outside of the etcd3 package, or import cycles
will exist.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: eb3aa5be10393968d8083c79f5958501fc029e8d
2022-05-11 07:52:02 -07:00
Steve Kuznetsov 80256820ce storage: move the APIObjectVersioner definition to storage
The means by which we extract and parse the version of an API object is
not specific to etcd3. In order to allow for a generic suite of tests
against any storage.Interface imlpementation, we need this logic to live
outside of the etcd3 package, or import cycles will exist.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 3939f3003e9605c06f65e64d1fc6f94b294f9d97
2022-05-11 07:44:21 -07:00
Maciej Borsz 17c48c0f65 Improve Create/Update/Patch traces.
Kubernetes-commit: 56769e794b757c541d1c7fef36a5507e52e3e7b2
2022-04-20 07:30:12 +00:00
Steve Kuznetsov d96402b9d6 storage/etcd3: clarify the pagingation flow in LIST
It is not possible for the nil-check to ever return anything different
from what the explicit boolean used to, but this is only something that
a reader can come to the conclusion on if they very, very carefuly read
the code. Instead of having this implicit flow that is difficult to
follow, let's keep the boolean.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 809fd64b289add1b378b45c748c23b7278c366f1
2022-04-22 11:50:24 -07:00
Sanskar Jaiswal d1bd5277fb Update comment and declaration of `storage.GuaranteedUpdate` to be clearer.
Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>

Kubernetes-commit: f8df26ae803103f82edbf1efe17b1b169801b256
2022-04-16 17:19:06 +05:30
Maciej Borsz f4edb394b0 Implement page size progressing for rare selectors.
Kubernetes-commit: a2ad9f9e4aba6aae6657a3189bdced6dbc8ba4b5
2022-03-07 15:44:59 +00:00
Wojciech Tyczyński b028a2ea8d Unify merge GetToList and List storage methods to single GetList
Kubernetes-commit: f3d56369f71f5106eeec8300e77bd69b172733da
2022-02-10 10:15:29 +01:00
Wojciech Tyczyński 9f8f72fea9 Change storage.Interface to utilize recurisve parameter
Kubernetes-commit: 3016b3d8f868a3041c7cb86695de09d7ab27cf3b
2022-02-10 09:58:31 +01:00
Steve Kuznetsov af1cb1cefe storage: transformers: pass a context.Context
When an envelope transformer calls out to KMS (for instance), it will be
very helpful to pass a `context.Context` to allow for cancellation. This
patch does that, while passing the previously-expected additional data
via a context value.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 27312feb9983c18d1daf00afba788727d024cdd0
2022-02-17 07:29:44 -08:00
Wojciech Tyczyński a55c2c834e Unify List and GetToList functions of etcd3
Kubernetes-commit: 27db8e34cfc930972beb8b5a33c443e134bcd614
2021-11-22 13:30:25 +01:00
Wojciech Tyczyński 71bfcaaea1 Simplify storage.Interface by merging Watch and WatchList functions.
Kubernetes-commit: 53653e9b0a7145c3b6e133c5639194ff79b3719d
2021-11-18 12:40:26 +01:00
Dr. Stefan Schimanski 2f0da0d2ea apiserver/storage: free etcd kv early in list decoding loop
Kubernetes-commit: dbf98e42664c8a3a104fe06e0e362497fca2902d
2021-10-08 14:05:26 +02:00
Mike Spreitzer 889106eff7 Add metrics about handling LIST requests
Add metrics that illuminate the costs of handling LIST requests.

Kubernetes-commit: bf424292db185546c474b27b963fd27bdbafc72f
2021-09-01 18:18:23 -04:00
Mike Spreitzer 31ec50ec43 Plumb the schema.GroupResource into etcd3 Store struct
Kubernetes-commit: 6f8019aae95db85552381f7e7066a52fef61be7a
2021-09-13 16:54:34 -04:00