Commit Graph

109 Commits

Author SHA1 Message Date
Davanum Srinivas b887c9ebec Drop usages of deprecated otelgrpc methods
Signed-off-by: Davanum Srinivas <davanum@gmail.com>

Kubernetes-commit: 7c0f968ab256486b524ea37014ccf580b12c73e4
2025-05-23 19:40:36 -07:00
Karl Isenberg 46dd96ca03 refactor: Stop using ioutil in apiserver
- The ioutl package is deprecated. Migrate to os package functions.

Kubernetes-commit: f93e4645c18c6f56bfddc158ef7b3f674b3c41dd
2025-05-08 11:28:52 -07:00
Antoni Zawodny 0dcf3e9d26 Configure watch cache history window based on request timeout
Kubernetes-commit: 4a2b7ee5699331df31b7483be082c201a1e7f51f
2024-12-13 15:40:03 +01:00
Abu Kashem 1f27993c44 implement unsafe deletion, and wire it
- implement unsafe deletion, and wire it
- aggregate corrupt object error(s) from the storage LIST operation
- extend storage error:
a) add a new type ErrCodeCorruptObj to represent a corrupt object:
b) add a new member 'InnerErr error' to StorageError to hold
   the inner error
- add API status error

Kubernetes-commit: 5d4b4a160dc551dc8979012eeabea1a098945603
2024-09-20 17:36:27 -04:00
Marek Siarkowicz 37f422396a Update recorders to wrap kubernetes.Client
Kubernetes-commit: 066c1c05d73690b48c872f3fbc23b7722cd44fe3
2024-07-05 14:19:49 +02:00
Abu Kashem 14881364b3 refactor: extract decode functions into an interface for etcd3 store
Kubernetes-commit: 1d1a656d8de1cdb99deaa6ec771aa354616eaa16
2024-09-24 07:19:19 -04:00
古九 e312f49d45 fix short circuit if the compaction request from apiserver is disabled
Kubernetes-commit: 9fef30117f89830cc8b17610c359141b663844f1
2024-08-12 10:21:49 +08:00
TommyStarK 2c2eb3836c kube-apiserver: remove deprecated otel NewNoopTracerProvider
Signed-off-by: TommyStarK <thomasmilox@gmail.com>

Kubernetes-commit: cce8551272de44882a3cd4fc7c06805247941086
2024-06-07 19:57:37 +02:00
Marek Siarkowicz 0e8e550153 Upgrade etcd libraries to v3.5.13
Add otelgrpc.WithMessageEvents(otelgrpc.ReceivedEvents, otelgrpc.SentEvents) to tracing options due to https://github.com/open-telemetry/opentelemetry-go-contrib/pull/3964

Kubernetes-commit: 3e5b03eb433ee359782f5aa6e9368ab2a0d0370c
2024-04-23 11:10:37 +02:00
Wojciech Tyczyński 324d2bc3cf Cleanup paging parameters in etcd3 store
Kubernetes-commit: 3f4d3b67682335db510f85deb65b322127a3a0a1
2023-10-20 15:38:59 +02:00
Wojciech Tyczyński 5bf4f58ab8 Remove storageConfig.Paging parameter
Kubernetes-commit: b386120da239bf9652fc02b2d2cbbd0fcc3cd121
2023-10-20 15:35:58 +02:00
Lukasz Szaszkiewicz f75c503352 storage/factory: extend the Create method by newList and resourcePrefix params
Kubernetes-commit: ccabc01093a1344ebb27c32c946e9da3b8e91fd2
2023-07-28 09:53:01 +02:00
Antonio Ojea 49f7deccb5 fix race on etcd client constructor for healthchecks
Change-Id: Id29b5b377989dcb5377316cfcdea367071a47365

Kubernetes-commit: 77b3bb0f69bfa4e5eb56ba484f724476304616cc
2023-08-08 13:55:14 +00:00
Marek Siarkowicz 573a8d6d05 Improve apiserver storage size metric to allow it's graduation
Change name to make it compliant with prometheus guidelines.
Calculate it on demand instead of periodic to comply with prometheus standards.
Replace "endpoint" with "server" label to make it semantically consistent with storage factory

Kubernetes-commit: 7a63997c8a1a9ba14f2bdc478fdf33cf88f48d80
2023-06-22 11:56:09 +02:00
Marek Siarkowicz fb965cd2c4 Make etcd component status consistent with health probes
Co-authored-by: Antonio Ojea <antonio.ojea.garcia@gmail.com>

Kubernetes-commit: a60314c47ef28dd04b086a2ad4b4bfd40b6f86a4
2023-06-05 16:26:12 +02:00
Daniel Smith 6abce7684c lavalamp is taking a long break
Kubernetes-commit: 1ffe3f467e8b8033312b7c68943d58125fd27663
2023-05-11 16:43:38 +00:00
kkkkun f4c24327c3 Fix etcd3 config changed
Kubernetes-commit: 57969c1b52ce0c3c27ccbf02ecc52ebc1b8533d9
2023-05-06 19:40:40 +08:00
Monis Khan 8d68e6f323 Load encryption config once
This change updates the API server code to load the encryption
config once at start up instead of multiple times.  Previously the
code would set up the storage transformers and the etcd healthz
checks in separate parse steps.  This is problematic for KMS v2 key
ID based staleness checks which need to be able to assert that the
API server has a single view into the KMS plugin's current key ID.

Signed-off-by: Monis Khan <mok@microsoft.com>

Kubernetes-commit: f507bc255382b2e2095351053bc17e74f7100d35
2022-08-29 17:25:48 -04:00
Antonio Ojea 1239036585 rate limit /healthz etcd healthchecks
return the last request error, instead of last error received
The rate limit allows 1 event per healthcheck timeout / 2

Kubernetes-commit: 510a85c53a5138babb1650fadd328e6f34baa03b
2022-08-29 11:09:58 +02:00
Antonio Ojea d2581bb0e0 fix etcd unit tests
stop leaking goroutines

reduce etcd test duration

Kubernetes-commit: dd6d3d95cdeb0e165e8365212d85d0f3b972d3e8
2022-08-28 23:13:45 +02:00
Antonio Ojea fb26cb4f0c don't serialize etcd healthchecks
Kubernetes-commit: 5a67248115ec0bc762c0351a73810117ac0bb814
2022-08-29 17:57:01 +02:00
Sally O'Malley 4f9e133507 kubelet tracing
Signed-off-by: Sally O'Malley <somalley@redhat.com>
Co-authored-by: David Ashpole <dashpole@google.com>

Kubernetes-commit: 47e7d8034ff3be8e198dde6a671d05a11c30e333
2021-10-10 09:17:27 -04:00
Nic Cope 9b243e9d90 Copy etcd client debug level logic from upstream
Replicated from https://github.com/etcd-io/etcd/blob/v3.5.4/client/v3/logger.go#L47

The logic of this function doesn't make a lot of sense to me, but
copying it will avoid any behaviour change.

Signed-off-by: Nic Cope <nicc@rk0n.org>

Kubernetes-commit: c1aa7a0fe73cbcab8e70f7b73a845ae9394f9a71
2022-07-29 14:26:31 -07:00
Nic Cope a105c2570c Give etcd client logger a name
Logic copied from https://github.com/etcd-io/etcd/blob/v3.5.4/client/v3/client.go#L374

Signed-off-by: Nic Cope <nicc@rk0n.org>

Kubernetes-commit: f54d2606336e2e8130339d2a0bc04fac6906aa78
2022-07-29 14:24:17 -07:00
Nic Cope 463756f91d Share a single etcd3 client logger across all clients
Currently the API server creates one etcd client per CRD. If clients
aren't provided a logger they'll each create their own. These loggers
can account for ~20% of API server memory consumption on a cluster with
hundreds of CRDs.

Signed-off-by: Nic Cope <nicc@rk0n.org>

Kubernetes-commit: 0c81eabb853e581abbcb37ebf094af3316e1012e
2022-07-28 19:51:55 -07:00
Nic Cope ef17269e10 Disable the etcd3 client logger
This logger is responsible for 20% of the API server's memory usage when
many CRDs are installed. See the below issue for more context.

https://github.com/kubernetes/kubernetes/issues/111476

Signed-off-by: Nic Cope <nicc@rk0n.org>

Kubernetes-commit: 0e5401c93940126beac45264aa056507b0950075
2022-07-27 14:44:49 -07:00
Maciej Wyrzuc cb0bb2af35 Add additional etcd check to readyz with 2 seconds timeout.
Kubernetes-commit: b42045a64fd07fb948660839b6c7c14440bee9df
2022-07-25 13:08:50 +00:00
Wojciech Tyczyński 0a7c4bcca1 Cleanup etcd healthcheck on shutdown
Kubernetes-commit: cb80082f666e0e5fe220df32e31a8face18e9393
2022-05-10 11:12:08 +02:00
Abu Kashem 1b651c5994 add latency tracker for storage and transform
Kubernetes-commit: eca90856940e9251ecf3fde95c5e4d2d16f5ad68
2022-02-01 18:13:03 -05:00
Davanum Srinivas c56bc7b872 OWNERS cleanup - Jan 2021 Week 1
Signed-off-by: Davanum Srinivas <davanum@gmail.com>

Kubernetes-commit: 9682b7248fb69733c2a0ee53618856e87b067f16
2022-01-03 10:59:47 -05:00
Davanum Srinivas b840d63feb Cleanup OWNERS files (No Activity in the last year)
Signed-off-by: Davanum Srinivas <davanum@gmail.com>

Kubernetes-commit: 497e9c1971c9e7d0193bc6d11503ec4ad527f1d5
2021-12-10 15:18:50 -05:00
Davanum Srinivas 56a3a30ae1 Check in OWNERS modified by update-yamlfmt.sh
Signed-off-by: Davanum Srinivas <davanum@gmail.com>

Kubernetes-commit: 9405e9b55ebcd461f161859a698b949ea3bde31d
2021-12-09 21:31:26 -05:00
Lukasz Szaszkiewicz 8fa48ed99b etcd-client starts retrying transient errors from the etcd cluster
This PR enables unaryClientInterceptor in conjunction with Prometheus interceptor.
Previously it was simply overwritten by the Prometheus interceptor.
As a result etcd client didn't attempt to retry certain errors.

The unaryClientInterceptor is important because it knows how to retry all sorts of errors from the etcd cluster. It will make the API server more resilient to failures -  end users won't see certain errors.
The full list of retriable (codes.Unavailable) errors can be found at https://github.com/etcd-io/etcd/blob/main/api/v3rpc/rpctypes/error.go#L72

Kubernetes-commit: 83171562b0954b2e19eb69943f01a44779cc7a8f
2021-09-15 16:40:44 +02:00
Mike Spreitzer 31ec50ec43 Plumb the schema.GroupResource into etcd3 Store struct
Kubernetes-commit: 6f8019aae95db85552381f7e7066a52fef61be7a
2021-09-13 16:54:34 -04:00
Mike Spreitzer b225af44fe Introduce storagebackend.ConfigForResource
This is a Config specialized for a GroupResource.
It will support generating new resource-specific metrics.

Kubernetes-commit: 85bcd243aa3c8769a5904a1aea44ce704f5e7174
2021-08-29 01:06:12 -04:00
Abu Kashem feb4eefe1c apiserver: add callback to get notified of object count
Kubernetes-commit: 2c60feffbee690af4632d068158e640abe10f678
2021-07-14 16:44:34 -04:00
Paco Xu d0159b4bbf Revert "use PermitWithoutStream=true for etcd: send pings even without active stream"
Kubernetes-commit: 1280a365e4a2407efb9aba9e701eeb1a5a69a2f0
2021-07-12 11:47:55 +08:00
Ryan Phillips 7afffd8ef3 Revert "apiserver: add callback to get notified of object count"
Kubernetes-commit: d95b14e1abfb5ec87248e5dd826b89d0c738af42
2021-07-08 13:56:39 -05:00
David Ashpole 4d188ce1c3 Add distributed tracing to the etcd client
Kubernetes-commit: 71f810bb71ed7b87e02c91ebb879363f32dcfaac
2021-06-25 11:54:53 -07:00
pacoxu 6640c869f9 use PermitWithoutStream=true for etcd: send pings even without active streams
Signed-off-by: pacoxu <paco.xu@daocloud.io>

Kubernetes-commit: 849dbe034b3e4398922c383e09900b464a60db69
2021-04-29 16:27:08 +08:00
Abu Kashem d929410e13 apiserver: add callback to get notified of object count
Kubernetes-commit: 1002b0d163dd948334f10b0e25d0e91d253791e8
2021-06-18 10:44:07 -04:00
Jordan Liggitt 4bb06f3c4d Simplify running embedded etcd server in tests
Kubernetes-commit: 01760927b82cf28bc6ba5b6364fda3c9f8c5c0b1
2021-06-15 12:07:46 -04:00
Jordan Liggitt 2f8b9e4dbc Fix etcd egress dialer addr parsing
Kubernetes-commit: a26c392de176494f2c425f712bc49fc399e9ce6d
2021-06-14 18:02:59 -04:00
Jordan Liggitt 6365bf9124 Switch to go.etcd.io/etcd/client/v3
Kubernetes-commit: 2979c3325e3ac4023dd19c435f3a995074328751
2021-03-23 11:01:52 -04:00
caozhiyuan 5a479e600e fix hardcoding and format error log
Kubernetes-commit: c468a02718402bf42ffc0cd8cdbdf805fdf903a2
2021-04-14 20:19:41 +08:00
Ling Samuel bd7da186cc apiserver add metric etcd_lease_object_counts
Signed-off-by: Ling Samuel <lingsamuelgrace@gmail.com>

Kubernetes-commit: 7e9fe39cd7c3ee67fcc29e6ca0836917c5a4e9e1
2020-12-07 10:45:44 +08:00
Ling Samuel f673ce24f7 apiserver add --lease-reuse-duration-seconds to config lease reuse duration
Signed-off-by: Ling Samuel <lingsamuelgrace@gmail.com>

Kubernetes-commit: c99567005db15d220f2b792e98af735c5aa24f6e
2020-12-02 17:35:10 +08:00
Patrik Cyvoct 29daf09387 fix case when HC timeout is 0
Signed-off-by: Patrik Cyvoct <patrik@ptrk.io>

Kubernetes-commit: 6af838c3d1027f41d286ef21b2e52ae60315a358
2020-10-28 11:45:50 +01:00
Patrik Cyvoct ddd0b07a5a Allow configuration of etcd healthcheck timeout
Signed-off-by: Patrik Cyvoct <patrik@ptrk.io>

Kubernetes-commit: 2e430ba622817b534642fd93098f87ea7f983aee
2020-07-20 09:22:13 +02:00
wojtekt a5000473c1 Pipe newFunc to etcd3 storage layer
Kubernetes-commit: fbd65a265a47ffe081aaac2f794a55034333d11d
2020-08-31 11:58:45 +02:00