Commit Graph

136 Commits

Author SHA1 Message Date
Mike Spreitzer 889106eff7 Add metrics about handling LIST requests
Add metrics that illuminate the costs of handling LIST requests.

Kubernetes-commit: bf424292db185546c474b27b963fd27bdbafc72f
2021-09-01 18:18:23 -04:00
Mike Spreitzer 31ec50ec43 Plumb the schema.GroupResource into etcd3 Store struct
Kubernetes-commit: 6f8019aae95db85552381f7e7066a52fef61be7a
2021-09-13 16:54:34 -04:00
Jordan Liggitt 29227fd4ee Avoid spurious calls to update/delete validation
Kubernetes-commit: 561ce035b5b7d577ddb5bb30657bc1f8b399e109
2021-08-05 17:21:40 -04:00
Ryan Phillips 7afffd8ef3 Revert "apiserver: add callback to get notified of object count"
Kubernetes-commit: d95b14e1abfb5ec87248e5dd826b89d0c738af42
2021-07-08 13:56:39 -05:00
Abu Kashem d929410e13 apiserver: add callback to get notified of object count
Kubernetes-commit: 1002b0d163dd948334f10b0e25d0e91d253791e8
2021-06-18 10:44:07 -04:00
Jordan Liggitt 4bb06f3c4d Simplify running embedded etcd server in tests
Kubernetes-commit: 01760927b82cf28bc6ba5b6364fda3c9f8c5c0b1
2021-06-15 12:07:46 -04:00
Jordan Liggitt aafb67c9b2 Quiet grpc info logs in apiserver
Kubernetes-commit: 503f5e1b52812f1bc5bb66d9b02a8f32cc5dccd9
2021-06-15 18:50:54 -04:00
Jordan Liggitt aaf5f32453 Quiet embedded etcd logs
Kubernetes-commit: f283deee6b65be2b435ec192b6444f969f9d2a42
2021-06-04 00:43:11 -04:00
Jordan Liggitt 8492f8d104 Drop use of deprecated clientv3.SetLogger
Kubernetes-commit: 52b629efbc8be30901192880b8005e7362f9a69f
2021-05-19 09:10:05 -04:00
Jordan Liggitt 6365bf9124 Switch to go.etcd.io/etcd/client/v3
Kubernetes-commit: 2979c3325e3ac4023dd19c435f3a995074328751
2021-03-23 11:01:52 -04:00
Jordan Liggitt 5d270fd014 Drop etcd v2 support from test util
Kubernetes-commit: 77ab4bdbbcc84ebb10645d810a087d272f16f803
2021-03-23 10:54:43 -04:00
wojtekt 5290a24345 Implement support for watch initialization in P&F
Kubernetes-commit: 0cc217647ca8be0820973b970124a072c27b6575
2021-05-07 12:49:06 +02:00
Maciej Borsz 770eb2bc44 Update watcher.go
Kubernetes-commit: 532e35ba31e4df0a2df7dd5f2930035ce9379559
2021-05-07 16:07:34 +02:00
卢振兴10069964 549cbbf8de fix broken link in some files
Kubernetes-commit: b29a5fb0746f772b38da570cd8fdc77396ffca31
2021-04-13 08:43:24 +08:00
Han Kang d21d96225a bump the deprecated version to 1.22
Change-Id: Ibefaa94151704fcaaa920541bbb9a8ad714c1d24

Kubernetes-commit: e7ee76efc0813da419045c80cc06a513fba6fc37
2021-03-29 07:54:12 -07:00
Han Kang 4ac82c26cf rename apiserver_storage_object_counts to conform to best practices (apiserver_storage_objects)
Change-Id: I8c08a37cd3ed7c943a6691185f750e55d389dd3a

Kubernetes-commit: 50e6f15deec3f9b9ffe52accd6aab5f9560b88ad
2021-03-10 04:32:27 -08:00
Yu Yi 09fb3448ff metrics: promote apiserver_storage_object_counts to stable
- rename etcd_object_counts to apiserver_storage_object_counts

CONTEXT https://github.com/kubernetes/kubernetes/issues/98270
FIXES https://github.com/kubernetes/kubernetes/issues/98270

Kubernetes-commit: 77950ba79b8b56bae34291093c69006975c7b775
2021-03-04 14:42:42 +00:00
Han Kang f16dfd3478 add myself to owners for etcd metrics
Kubernetes-commit: 5bc56ef4e6a255a0ca7ec86fbe9a078c284eab1a
2021-03-04 07:46:34 -08:00
Nikhita Raghunath 2b88377b23 *: remove madhusudancs from reviewers
Kubernetes-commit: 6b12c96a9b7fe4d7c03d0dfed447edd6b1055067
2021-02-16 10:40:42 +05:30
Jiaxin Shan dfad5032fb Fix ALPHA stability level reference link
Kubernetes-commit: e01a21469b9719f7d0e84021c032cd8f0016b5d2
2021-01-31 15:37:07 -08:00
Ling Samuel 59be6d306d api-server add --lease-max-object-count
Signed-off-by: Ling Samuel <lingsamuelgrace@gmail.com>

Kubernetes-commit: c8db72c38c34a965800bd9d570c495c45f142186
2020-12-07 10:45:44 +08:00
Wenjia 15c145e78b Remove unused expectedRV in store_test TestGet
expectedRV is defined in tests struct  but not set in test cases, removing the unnecessary checking

Kubernetes-commit: f8f36c672776bb00b2d53c5d49e92b1bfb608516
2021-01-21 11:27:57 -08:00
pacoxu b74ce63ea5 fix static check for vendor/k8s.io/apiserver/pkg/storage/etcd3/
Kubernetes-commit: aaa7084a4a548795c987b29dffb48749531b60a4
2021-01-07 11:21:50 +08:00
Ling Samuel bd7da186cc apiserver add metric etcd_lease_object_counts
Signed-off-by: Ling Samuel <lingsamuelgrace@gmail.com>

Kubernetes-commit: 7e9fe39cd7c3ee67fcc29e6ca0836917c5a4e9e1
2020-12-07 10:45:44 +08:00
Abu Kashem 2d4e2517cc reduce buckets for etcd_request_duration_seconds
Kubernetes-commit: a7bd404d9b29adae2160acd31081a3aeb49ccaa5
2020-11-20 09:48:24 -05:00
wojtekt 3de94d4883 Unify variable naming between GuaranteedUpdate and Delete in storage
Kubernetes-commit: d0726e4b1354b1c8c3978b96ab7b01d13a2b6340
2020-12-16 09:28:44 +01:00
wojtekt ddcc69d4f7 Use current state from watchcache to avoid etcd get for deletions
Kubernetes-commit: 7bab6a9c6ed7d4e54da3e010a7bf0a30786d28f9
2020-11-03 17:30:26 +01:00
wojtekt 8c25869e43 Add suggestion to storage interface Delete method
Kubernetes-commit: c2d61896f45967cdea15072b7a12faeb635b5dc5
2020-11-03 13:21:00 +01:00
Ling Samuel f673ce24f7 apiserver add --lease-reuse-duration-seconds to config lease reuse duration
Signed-off-by: Ling Samuel <lingsamuelgrace@gmail.com>

Kubernetes-commit: c99567005db15d220f2b792e98af735c5aa24f6e
2020-12-02 17:35:10 +08:00
Ken Sipe 5d58b175c8 fix S1021 var declaration
Signed-off-by: Ken Sipe <kensipe@gmail.com>

Kubernetes-commit: 6c49299739a9819c3672248517ab3d6636d1d8c6
2020-06-25 17:10:34 -05:00
wojtekt 942f220e85 Remove variadic argument from storage interface
Kubernetes-commit: 8b98305858b107369f2c9b9fd8ef1c5b0da078c0
2020-11-02 15:52:51 +01:00
wojtekt e40efde837 Implement etcd3 progress-notify feature in etcd3 layer
Kubernetes-commit: 56e72841b6005740453828a9f4f7a9a1b9a831f6
2020-08-31 12:40:41 +02:00
wojtekt a5000473c1 Pipe newFunc to etcd3 storage layer
Kubernetes-commit: fbd65a265a47ffe081aaac2f794a55034333d11d
2020-08-31 11:58:45 +02:00
Abu Kashem bd49a32a3f count of etcd object should be limited to the specified resource
Currently count includes keys from different resource(s) if the keys
are a prefix of the specified resource/key.

Consider the following keys:
A: <storage-prefix>//foo.bar.io/machines
B: <storage-prefix>//foo.bar.io/machinesets
If we ask for the count of key A, the result will also include the
keys from key B since key B shares the same prefix as key A.

Append a separator to mark the end of the key, this will exclude all
other keys from a different resource that is a prefix of the specified
key.

Kubernetes-commit: 7e445867aa4d37a67591faf6e5508abaea69d216
2020-09-11 17:07:09 -04:00
Abu Kashem 2819f102d4 Make similar buckets for api and etcd request duration histogram
Make similar buckets for the apiserver_request_duration_seconds and
the etcd_request_duration_seconds histogram so that the result is
more comparable side by side.

etcd_request_duration_seconds uses the default buckets provided by
prometheus client library:
DefBuckets = []float64{.005, .01, .025, .05, .1, .25, .5, 1, 2.5, 5, 10}

apiserver_request_duration_seconds on the other hand uses more fine
grained buckets, and the maximum bucket size is 60s. Both histograms
should use similar bucket sizes so they are more comparable side by side.

Kubernetes-commit: 9d8441f17d90c46eca6390a522e8771bed10e0ba
2020-08-20 11:49:15 -04:00
wojtekt 6e6045833c Fix bug for inconsistent lists served from etcd
Kubernetes-commit: a5b60c352d4457d96a90ddf1479f0833ef36f7d4
2020-08-14 14:51:56 +02:00
Joe Betz 97937c66f2 Revert nested trace PR#88936
Kubernetes-commit: 02cf58102a61b6d1e021e256381ff750573ce55d
2020-07-20 09:55:05 -07:00
Joe Betz 7a467399ac Enable nested tracing, add request filter chain tracing incl. authn/authz tracing
Kubernetes-commit: b12ac0abc64adb71d97fbde12f373b1424631f20
2020-03-06 16:11:21 -08:00
Joe Betz ee219411ed Add ResourceVersionMatch parameter to make Resource Version semantics consistent for list
Kubernetes-commit: e214f2408b59c745c199645547948a8ad2a87ac2
2020-05-29 10:44:26 -07:00
AkashSarda e0d29c69be Migrate API server logs to Structured logging framewrok
Kubernetes-commit: af60e7b34d59d635740880a03e7e2587ec9ba041
2020-06-17 21:06:35 +00:00
Joe Betz 5b75cf0ff9 Add GetOptions and ListOptions to storage interface
Kubernetes-commit: 4c99949ae6b4a74e4ffda0caba86c151c23d579e
2020-05-29 10:48:33 -07:00
Davanum Srinivas 5879417a28 switch over k/k to use klog v2
Signed-off-by: Davanum Srinivas <davanum@gmail.com>

Kubernetes-commit: 442a69c3bdf6fe8e525b05887e57d89db1e2f3a5
2020-04-17 15:25:06 -04:00
Michael Gasch 0cdbc006ea Add etcd WithRequireLeader option to API watches
Watches against etcd in the API server can hang forever if the etcd
cluster loses quorum, e.g. the majority of nodes crashes. This fix
improves responsiveness (detection and reaction time) of API server
watches against etcd in some rare (but still possible) edge cases so
that watches are terminated with `"etcdserver: no leader"
(ErrNoLeader)`.

Implementation behavior described by jingyih:

```
The etcd server waits until it cannot find a leader for 3 election
timeouts to cancel existing streams. 3 is currently a hard coded
constant. The election timeout defaults to 1000ms.

If the cluster is healthy, when the leader is stopped, the leadership
transfer should be smooth. (leader transfers its leadership before
stopping). If leader is hard killed, other servers will take an election
timeout to realize leader lost and start campaign.
```

For further details, discussion and validation see
https://github.com/kubernetes/kubernetes/issues/89488#issuecomment-606491110
and https://github.com/etcd-io/etcd/issues/8980.

Closes: https://github.com/kubernetes/kubernetes/issues/89488

Signed-off-by: Michael Gasch <mgasch@vmware.com>

Kubernetes-commit: 70c9f770d7aa2194bfd3f58fe01756a7d200b866
2020-03-31 12:01:53 +02:00
jingyih 9303178e27 Add a metric exposing etcd database size
Kubernetes-commit: 922ec728de9248657f026eb6cfb8fdaeb11049ac
2020-03-16 07:55:38 -07:00
Jordan Liggitt 1a93b794ed Preserve target apiVersion when decoding into unstructured lists
Kubernetes-commit: fa12441ab99cac81b0034208fd10d8a4fc3d5bd0
2020-03-10 03:13:20 -04:00
Monis Khan f5c1e085ed storage: confirm that paging and predicate filtering work together
This change adds the TestListContinuationWithFilter test which
confirms that paging with a predicate that does not match everything
results in the correct amount of calls to TransformFromStorage and
KV.Get.  The partial result of each paging call is also asserted.

Signed-off-by: Monis Khan <mok@vmware.com>

Kubernetes-commit: 002c75442d768d2bcc51047667354ff16bbfa2e8
2020-02-28 15:35:22 -05:00
danielqsj 400e5a4291 remove deprecated metrics of etcd
Kubernetes-commit: 447b17587b6df7cf5828cf32fb333f0a3b7957cd
2019-04-12 15:54:31 +08:00
Ted Yu 006e524c38 Log mvcc revision compaction as warning
Kubernetes-commit: 9fec783d805ebdb05f1936dffd9621702eb0f711
2019-10-31 07:24:17 -07:00
danielqsj 76c4ab8bb5 unify alias of api errors under pkg and staging
Kubernetes-commit: 5bc0e26c1902e7e28abfc828de19a2dbb3e492c2
2019-11-12 16:26:59 +08:00
RainbowMango de8c866328 Hide apiserver metrics that have been deprecated in 1.14
Update E2E test accordingly.

Kubernetes-commit: 5e0695e339ff0887ebebddfa8630de191f7fa235
2019-10-12 20:25:10 +08:00