Commit Graph

343 Commits

Author SHA1 Message Date
Patrik Cyvoct 29daf09387 fix case when HC timeout is 0
Signed-off-by: Patrik Cyvoct <patrik@ptrk.io>

Kubernetes-commit: 6af838c3d1027f41d286ef21b2e52ae60315a358
2020-10-28 11:45:50 +01:00
Patrik Cyvoct ddd0b07a5a Allow configuration of etcd healthcheck timeout
Signed-off-by: Patrik Cyvoct <patrik@ptrk.io>

Kubernetes-commit: 2e430ba622817b534642fd93098f87ea7f983aee
2020-07-20 09:22:13 +02:00
wojtekt 942f220e85 Remove variadic argument from storage interface
Kubernetes-commit: 8b98305858b107369f2c9b9fd8ef1c5b0da078c0
2020-11-02 15:52:51 +01:00
wojtekt 0f83389bec Fix cacher test after bumping fakeBudget timeout to 2 seconds
Kubernetes-commit: f62e61c28f11a39c1fb4cfda5842bb648ee24eb9
2020-10-27 15:25:31 +01:00
wojtekt e0edc885d5 Fix cacher test flakiness
Kubernetes-commit: 6eb71c4089f69c7cc35ef2fd13bba4329b5716b9
2020-10-26 15:29:51 +01:00
wojtekt cdc93483b8 Watch bookmarks may contain version of objects of other types
Kubernetes-commit: 0bd8104809922aae504fc9be2e853650ecd17b8a
2020-09-29 12:27:20 +02:00
Daniel Smith 7e305c53b5 fix goroutine that lives too long
Kubernetes-commit: 9a2e0fcb18bc9feb327122edc95a968a8d423af7
2020-09-28 10:50:47 -07:00
wojtekt c4fb001eda Enable progress notify events in watchcache
Kubernetes-commit: a94fb5369d5e77b3fcafd1296bac072a1d6e13fe
2020-08-31 13:15:36 +02:00
wojtekt e40efde837 Implement etcd3 progress-notify feature in etcd3 layer
Kubernetes-commit: 56e72841b6005740453828a9f4f7a9a1b9a831f6
2020-08-31 12:40:41 +02:00
wojtekt a5000473c1 Pipe newFunc to etcd3 storage layer
Kubernetes-commit: fbd65a265a47ffe081aaac2f794a55034333d11d
2020-08-31 11:58:45 +02:00
Abu Kashem bd49a32a3f count of etcd object should be limited to the specified resource
Currently count includes keys from different resource(s) if the keys
are a prefix of the specified resource/key.

Consider the following keys:
A: <storage-prefix>//foo.bar.io/machines
B: <storage-prefix>//foo.bar.io/machinesets
If we ask for the count of key A, the result will also include the
keys from key B since key B shares the same prefix as key A.

Append a separator to mark the end of the key, this will exclude all
other keys from a different resource that is a prefix of the specified
key.

Kubernetes-commit: 7e445867aa4d37a67591faf6e5508abaea69d216
2020-09-11 17:07:09 -04:00
Abu Kashem 2819f102d4 Make similar buckets for api and etcd request duration histogram
Make similar buckets for the apiserver_request_duration_seconds and
the etcd_request_duration_seconds histogram so that the result is
more comparable side by side.

etcd_request_duration_seconds uses the default buckets provided by
prometheus client library:
DefBuckets = []float64{.005, .01, .025, .05, .1, .25, .5, 1, 2.5, 5, 10}

apiserver_request_duration_seconds on the other hand uses more fine
grained buckets, and the maximum bucket size is 60s. Both histograms
should use similar bucket sizes so they are more comparable side by side.

Kubernetes-commit: 9d8441f17d90c46eca6390a522e8771bed10e0ba
2020-08-20 11:49:15 -04:00
Nikhita Raghunath 3151c1befa apimachinery: remove inactive members from OWNERS
Kubernetes-commit: db495ea9837cf840639a4e02085253fb341e12b5
2020-07-11 19:28:24 +05:30
wojtekt 6e6045833c Fix bug for inconsistent lists served from etcd
Kubernetes-commit: a5b60c352d4457d96a90ddf1479f0833ef36f7d4
2020-08-14 14:51:56 +02:00
Joe Betz 97937c66f2 Revert nested trace PR#88936
Kubernetes-commit: 02cf58102a61b6d1e021e256381ff750573ce55d
2020-07-20 09:55:05 -07:00
Joe Betz 7a467399ac Enable nested tracing, add request filter chain tracing incl. authn/authz tracing
Kubernetes-commit: b12ac0abc64adb71d97fbde12f373b1424631f20
2020-03-06 16:11:21 -08:00
wojtekt cd5511f16a Fix bug in reflector not recovering from "Too large resource version" errors
Kubernetes-commit: 3704174f95c7311e025284ef30bb56945fa6e7cc
2020-06-26 09:45:29 +02:00
Joe Betz ee219411ed Add ResourceVersionMatch parameter to make Resource Version semantics consistent for list
Kubernetes-commit: e214f2408b59c745c199645547948a8ad2a87ac2
2020-05-29 10:44:26 -07:00
AkashSarda e0d29c69be Migrate API server logs to Structured logging framewrok
Kubernetes-commit: af60e7b34d59d635740880a03e7e2587ec9ba041
2020-06-17 21:06:35 +00:00
wojtekt 8a4e4b61c9 Add initialization logs to watchcache
Kubernetes-commit: 34b2cf7d2e1c09e10d97ec83e413c41ea7364d80
2020-06-18 13:55:23 +02:00
gongguan 5dbb27a1df remove deprecated cacher CacheCapacity config
Kubernetes-commit: 12c838136b09005e0cbc831096643195ef24cb89
2020-06-05 21:34:36 +08:00
gongguan 915ec02699 restore event Gone test
Kubernetes-commit: f3cbfc3f7e4bf38978b118933d4e626125a9d80f
2020-06-16 11:07:07 +08:00
gongguan 1da8aeae17 improve Clock as Cacher Config
Kubernetes-commit: 0f7de876a5e931a57ba6347b81e8e28d360c788a
2020-06-15 23:57:11 +08:00
Wojciech Tyczynski 54e40d4b2d Revert "Revert "Rely on default watch cache capacity and ignore its requested size""
Kubernetes-commit: 7787ebc85b87eb7bc00f234f4e52eed3d987d5dd
2020-05-27 14:19:38 +02:00
Gaurav Singh 72110f9b1f Code review changes
Signed-off-by: Gaurav Singh <gaurav1086@gmail.com>

Kubernetes-commit: 6a28f1995a4d0749d4871f80c0d8e2989e833a78
2020-05-19 16:42:20 -04:00
Gaurav Singh 7dd66e9272 Fix bug
Signed-off-by: Gaurav Singh <gaurav1086@gmail.com>

Kubernetes-commit: 549214ca8c6f7e05fcbb0c8c045251819b039947
2020-05-17 17:54:28 -04:00
Gaurav Singh 168bf11368 Fix data race in cacher whitebox test
Signed-off-by: Gaurav Singh <gaurav1086@gmail.com>

Kubernetes-commit: 3dc7098cae4c699e30ed9ae85005446f8322fe99
2020-05-17 17:51:49 -04:00
Joe Betz 5b75cf0ff9 Add GetOptions and ListOptions to storage interface
Kubernetes-commit: 4c99949ae6b4a74e4ffda0caba86c151c23d579e
2020-05-29 10:48:33 -07:00
wojtekt 8371df6187 Watch should return "too old RV" if it can't serve it.
Kubernetes-commit: ef1e5b6d3ac25431d890bfe4f540d9aa956a856d
2020-05-25 20:46:27 +02:00
Tim Allclair 5d41955c07 Remove tallclair from some OWNERS files
Kubernetes-commit: 029a144ae9e143ea3f648a8d14fdddb7c845246d
2020-02-20 15:07:06 -08:00
Jordan Liggitt d501f7b597 Revert "Rely on default watch cache capacity and ignore its requested size"
This reverts PR 91260

Kubernetes-commit: 6249f287e1d3d9fb63b60961c8625a82d7db360e
2020-05-23 09:21:22 -04:00
wojtekt 31ffec3302 Rely on default watch cache capacity and ignore its requested size
Kubernetes-commit: 99dff90d613e2014873dc2fae0a00acaf95c3bd2
2020-05-19 18:13:28 +02:00
Davanum Srinivas 5879417a28 switch over k/k to use klog v2
Signed-off-by: Davanum Srinivas <davanum@gmail.com>

Kubernetes-commit: 442a69c3bdf6fe8e525b05887e57d89db1e2f3a5
2020-04-17 15:25:06 -04:00
wojtekt 6332f98216 Update duration to be kept in watchcache
Kubernetes-commit: d1de6295bb8047cbfd2a4ecbb0d264197f4cd8a4
2020-05-04 17:21:40 +02:00
Wojciech Tyczynski 4589c74d5e Send watch bookmarks every minute
Kubernetes-commit: 9f1e4620379028754f2da4223adf4d1a445e90f4
2020-04-27 18:41:31 +02:00
Wojciech Tyczynski 7686f3528b Revert "Send watch bookmarks every minute"
Kubernetes-commit: 1cb98ed2376b1f7777ca3d7bfac98cbb5f8b9ce3
2020-04-27 17:07:27 +02:00
wojtekt f1c77ba823 Send watch bookmarks every minute
Kubernetes-commit: d4b532e7190c2ad12a0317ff946e2cd0b33f0ada
2020-04-17 19:42:26 +02:00
louisgong 4c8b97679c Use dynamic size watch-cache.
If all cached events occur inside eventFreshDuration, increase cache capacity by 2x.
Decrease cache capacity by 2x when recent half events occur outside eventFreshDuration.

Kubernetes-commit: 56407b656c7acf6039cead0192070429e53a0c70
2020-04-12 17:22:38 +08:00
louisgong 1117ccdc72 move watchCache metrics to a seperate file
Kubernetes-commit: 21ba510ef0e02f64a813cb460212cb832f8f4d78
2020-04-16 15:45:16 +08:00
Michael Gasch 0cdbc006ea Add etcd WithRequireLeader option to API watches
Watches against etcd in the API server can hang forever if the etcd
cluster loses quorum, e.g. the majority of nodes crashes. This fix
improves responsiveness (detection and reaction time) of API server
watches against etcd in some rare (but still possible) edge cases so
that watches are terminated with `"etcdserver: no leader"
(ErrNoLeader)`.

Implementation behavior described by jingyih:

```
The etcd server waits until it cannot find a leader for 3 election
timeouts to cancel existing streams. 3 is currently a hard coded
constant. The election timeout defaults to 1000ms.

If the cluster is healthy, when the leader is stopped, the leadership
transfer should be smooth. (leader transfers its leadership before
stopping). If leader is hard killed, other servers will take an election
timeout to realize leader lost and start campaign.
```

For further details, discussion and validation see
https://github.com/kubernetes/kubernetes/issues/89488#issuecomment-606491110
and https://github.com/etcd-io/etcd/issues/8980.

Closes: https://github.com/kubernetes/kubernetes/issues/89488

Signed-off-by: Michael Gasch <mgasch@vmware.com>

Kubernetes-commit: 70c9f770d7aa2194bfd3f58fe01756a7d200b866
2020-03-31 12:01:53 +02:00
jingyih 9303178e27 Add a metric exposing etcd database size
Kubernetes-commit: 922ec728de9248657f026eb6cfb8fdaeb11049ac
2020-03-16 07:55:38 -07:00
Abu Kashem 6d0de7f4cb Fix data race issue in unit test
TestDispatchingBookmarkEventsWithConcurrentStop can use processEvent
instead of `dispatchEvent` to avoid data race conditions with
`Cacher.watchersBuffer`.

Kubernetes-commit: 7dc075673c24ffd8bde08a4c7ec55a8b633b20e2
2020-03-16 17:43:32 -04:00
Jordan Liggitt 1a93b794ed Preserve target apiVersion when decoding into unstructured lists
Kubernetes-commit: fa12441ab99cac81b0034208fd10d8a4fc3d5bd0
2020-03-10 03:13:20 -04:00
Monis Khan f5c1e085ed storage: confirm that paging and predicate filtering work together
This change adds the TestListContinuationWithFilter test which
confirms that paging with a predicate that does not match everything
results in the correct amount of calls to TransformFromStorage and
KV.Get.  The partial result of each paging call is also asserted.

Signed-off-by: Monis Khan <mok@vmware.com>

Kubernetes-commit: 002c75442d768d2bcc51047667354ff16bbfa2e8
2020-02-28 15:35:22 -05:00
immutablet 66b663f223 Instrument DEK cache fill and request inter-arrival times.
Kubernetes-commit: 684d6fb0ade6ac088af391cedd70bc847941a54f
2020-02-18 16:39:53 -08:00
shaloulcy 2c701b633b validate storage cache indexers
Signed-off-by: shaloulcy <lcy041536@gmail.com>

Kubernetes-commit: fa9ba80a67deeb9089e1be8ebac21063ebc07904
2020-02-05 12:08:14 +08:00
shaloulcy 6099d76e39 add indexer for storage cacher
Signed-off-by: shaloulcy <lcy041536@gmail.com>

Kubernetes-commit: 87582e2c3c1cb8c8e10ec05a4700f865c759b7b2
2019-11-19 16:52:07 +08:00
Davanum Srinivas cde2338e26 update generated files
Kubernetes-commit: b3853138a4f1a0637ec3c38a5c59f8228765b261
2020-01-13 17:56:56 -05:00
danielqsj fdd895e164 remove last part of deprecated metrics
Kubernetes-commit: 606e4503cf4cea9f05cfd467b88c88cf9c5648d9
2019-12-27 14:23:11 +08:00
danielqsj 669e87efbe clean SinceInMicroseconds, convert to SinceInSeconds
Kubernetes-commit: ab182552b485ceed96c3b05d284ae9480377daf8
2019-04-15 16:18:01 +08:00