Commit Graph

92 Commits

Author SHA1 Message Date
Monis Khan 8d68e6f323 Load encryption config once
This change updates the API server code to load the encryption
config once at start up instead of multiple times.  Previously the
code would set up the storage transformers and the etcd healthz
checks in separate parse steps.  This is problematic for KMS v2 key
ID based staleness checks which need to be able to assert that the
API server has a single view into the KMS plugin's current key ID.

Signed-off-by: Monis Khan <mok@microsoft.com>

Kubernetes-commit: f507bc255382b2e2095351053bc17e74f7100d35
2022-08-29 17:25:48 -04:00
Antonio Ojea 1239036585 rate limit /healthz etcd healthchecks
return the last request error, instead of last error received
The rate limit allows 1 event per healthcheck timeout / 2

Kubernetes-commit: 510a85c53a5138babb1650fadd328e6f34baa03b
2022-08-29 11:09:58 +02:00
Antonio Ojea d2581bb0e0 fix etcd unit tests
stop leaking goroutines

reduce etcd test duration

Kubernetes-commit: dd6d3d95cdeb0e165e8365212d85d0f3b972d3e8
2022-08-28 23:13:45 +02:00
Antonio Ojea fb26cb4f0c don't serialize etcd healthchecks
Kubernetes-commit: 5a67248115ec0bc762c0351a73810117ac0bb814
2022-08-29 17:57:01 +02:00
Sally O'Malley 4f9e133507 kubelet tracing
Signed-off-by: Sally O'Malley <somalley@redhat.com>
Co-authored-by: David Ashpole <dashpole@google.com>

Kubernetes-commit: 47e7d8034ff3be8e198dde6a671d05a11c30e333
2021-10-10 09:17:27 -04:00
Nic Cope 9b243e9d90 Copy etcd client debug level logic from upstream
Replicated from https://github.com/etcd-io/etcd/blob/v3.5.4/client/v3/logger.go#L47

The logic of this function doesn't make a lot of sense to me, but
copying it will avoid any behaviour change.

Signed-off-by: Nic Cope <nicc@rk0n.org>

Kubernetes-commit: c1aa7a0fe73cbcab8e70f7b73a845ae9394f9a71
2022-07-29 14:26:31 -07:00
Nic Cope a105c2570c Give etcd client logger a name
Logic copied from https://github.com/etcd-io/etcd/blob/v3.5.4/client/v3/client.go#L374

Signed-off-by: Nic Cope <nicc@rk0n.org>

Kubernetes-commit: f54d2606336e2e8130339d2a0bc04fac6906aa78
2022-07-29 14:24:17 -07:00
Nic Cope 463756f91d Share a single etcd3 client logger across all clients
Currently the API server creates one etcd client per CRD. If clients
aren't provided a logger they'll each create their own. These loggers
can account for ~20% of API server memory consumption on a cluster with
hundreds of CRDs.

Signed-off-by: Nic Cope <nicc@rk0n.org>

Kubernetes-commit: 0c81eabb853e581abbcb37ebf094af3316e1012e
2022-07-28 19:51:55 -07:00
Nic Cope ef17269e10 Disable the etcd3 client logger
This logger is responsible for 20% of the API server's memory usage when
many CRDs are installed. See the below issue for more context.

https://github.com/kubernetes/kubernetes/issues/111476

Signed-off-by: Nic Cope <nicc@rk0n.org>

Kubernetes-commit: 0e5401c93940126beac45264aa056507b0950075
2022-07-27 14:44:49 -07:00
Maciej Wyrzuc cb0bb2af35 Add additional etcd check to readyz with 2 seconds timeout.
Kubernetes-commit: b42045a64fd07fb948660839b6c7c14440bee9df
2022-07-25 13:08:50 +00:00
Wojciech Tyczyński 0a7c4bcca1 Cleanup etcd healthcheck on shutdown
Kubernetes-commit: cb80082f666e0e5fe220df32e31a8face18e9393
2022-05-10 11:12:08 +02:00
Abu Kashem 1b651c5994 add latency tracker for storage and transform
Kubernetes-commit: eca90856940e9251ecf3fde95c5e4d2d16f5ad68
2022-02-01 18:13:03 -05:00
Davanum Srinivas c56bc7b872 OWNERS cleanup - Jan 2021 Week 1
Signed-off-by: Davanum Srinivas <davanum@gmail.com>

Kubernetes-commit: 9682b7248fb69733c2a0ee53618856e87b067f16
2022-01-03 10:59:47 -05:00
Davanum Srinivas b840d63feb Cleanup OWNERS files (No Activity in the last year)
Signed-off-by: Davanum Srinivas <davanum@gmail.com>

Kubernetes-commit: 497e9c1971c9e7d0193bc6d11503ec4ad527f1d5
2021-12-10 15:18:50 -05:00
Davanum Srinivas 56a3a30ae1 Check in OWNERS modified by update-yamlfmt.sh
Signed-off-by: Davanum Srinivas <davanum@gmail.com>

Kubernetes-commit: 9405e9b55ebcd461f161859a698b949ea3bde31d
2021-12-09 21:31:26 -05:00
Lukasz Szaszkiewicz 8fa48ed99b etcd-client starts retrying transient errors from the etcd cluster
This PR enables unaryClientInterceptor in conjunction with Prometheus interceptor.
Previously it was simply overwritten by the Prometheus interceptor.
As a result etcd client didn't attempt to retry certain errors.

The unaryClientInterceptor is important because it knows how to retry all sorts of errors from the etcd cluster. It will make the API server more resilient to failures -  end users won't see certain errors.
The full list of retriable (codes.Unavailable) errors can be found at https://github.com/etcd-io/etcd/blob/main/api/v3rpc/rpctypes/error.go#L72

Kubernetes-commit: 83171562b0954b2e19eb69943f01a44779cc7a8f
2021-09-15 16:40:44 +02:00
Mike Spreitzer 31ec50ec43 Plumb the schema.GroupResource into etcd3 Store struct
Kubernetes-commit: 6f8019aae95db85552381f7e7066a52fef61be7a
2021-09-13 16:54:34 -04:00
Mike Spreitzer b225af44fe Introduce storagebackend.ConfigForResource
This is a Config specialized for a GroupResource.
It will support generating new resource-specific metrics.

Kubernetes-commit: 85bcd243aa3c8769a5904a1aea44ce704f5e7174
2021-08-29 01:06:12 -04:00
Abu Kashem feb4eefe1c apiserver: add callback to get notified of object count
Kubernetes-commit: 2c60feffbee690af4632d068158e640abe10f678
2021-07-14 16:44:34 -04:00
Paco Xu d0159b4bbf Revert "use PermitWithoutStream=true for etcd: send pings even without active stream"
Kubernetes-commit: 1280a365e4a2407efb9aba9e701eeb1a5a69a2f0
2021-07-12 11:47:55 +08:00
Ryan Phillips 7afffd8ef3 Revert "apiserver: add callback to get notified of object count"
Kubernetes-commit: d95b14e1abfb5ec87248e5dd826b89d0c738af42
2021-07-08 13:56:39 -05:00
David Ashpole 4d188ce1c3 Add distributed tracing to the etcd client
Kubernetes-commit: 71f810bb71ed7b87e02c91ebb879363f32dcfaac
2021-06-25 11:54:53 -07:00
pacoxu 6640c869f9 use PermitWithoutStream=true for etcd: send pings even without active streams
Signed-off-by: pacoxu <paco.xu@daocloud.io>

Kubernetes-commit: 849dbe034b3e4398922c383e09900b464a60db69
2021-04-29 16:27:08 +08:00
Abu Kashem d929410e13 apiserver: add callback to get notified of object count
Kubernetes-commit: 1002b0d163dd948334f10b0e25d0e91d253791e8
2021-06-18 10:44:07 -04:00
Jordan Liggitt 4bb06f3c4d Simplify running embedded etcd server in tests
Kubernetes-commit: 01760927b82cf28bc6ba5b6364fda3c9f8c5c0b1
2021-06-15 12:07:46 -04:00
Jordan Liggitt 2f8b9e4dbc Fix etcd egress dialer addr parsing
Kubernetes-commit: a26c392de176494f2c425f712bc49fc399e9ce6d
2021-06-14 18:02:59 -04:00
Jordan Liggitt 6365bf9124 Switch to go.etcd.io/etcd/client/v3
Kubernetes-commit: 2979c3325e3ac4023dd19c435f3a995074328751
2021-03-23 11:01:52 -04:00
caozhiyuan 5a479e600e fix hardcoding and format error log
Kubernetes-commit: c468a02718402bf42ffc0cd8cdbdf805fdf903a2
2021-04-14 20:19:41 +08:00
Ling Samuel bd7da186cc apiserver add metric etcd_lease_object_counts
Signed-off-by: Ling Samuel <lingsamuelgrace@gmail.com>

Kubernetes-commit: 7e9fe39cd7c3ee67fcc29e6ca0836917c5a4e9e1
2020-12-07 10:45:44 +08:00
Ling Samuel f673ce24f7 apiserver add --lease-reuse-duration-seconds to config lease reuse duration
Signed-off-by: Ling Samuel <lingsamuelgrace@gmail.com>

Kubernetes-commit: c99567005db15d220f2b792e98af735c5aa24f6e
2020-12-02 17:35:10 +08:00
Patrik Cyvoct 29daf09387 fix case when HC timeout is 0
Signed-off-by: Patrik Cyvoct <patrik@ptrk.io>

Kubernetes-commit: 6af838c3d1027f41d286ef21b2e52ae60315a358
2020-10-28 11:45:50 +01:00
Patrik Cyvoct ddd0b07a5a Allow configuration of etcd healthcheck timeout
Signed-off-by: Patrik Cyvoct <patrik@ptrk.io>

Kubernetes-commit: 2e430ba622817b534642fd93098f87ea7f983aee
2020-07-20 09:22:13 +02:00
wojtekt a5000473c1 Pipe newFunc to etcd3 storage layer
Kubernetes-commit: fbd65a265a47ffe081aaac2f794a55034333d11d
2020-08-31 11:58:45 +02:00
Davanum Srinivas 5879417a28 switch over k/k to use klog v2
Signed-off-by: Davanum Srinivas <davanum@gmail.com>

Kubernetes-commit: 442a69c3bdf6fe8e525b05887e57d89db1e2f3a5
2020-04-17 15:25:06 -04:00
jingyih 9303178e27 Add a metric exposing etcd database size
Kubernetes-commit: 922ec728de9248657f026eb6cfb8fdaeb11049ac
2020-03-16 07:55:38 -07:00
Wenjia Zhang e518fe04f6 Resolve uncompatibility from update: etcd CAFile -> TrustedCAFIle
Kubernetes-commit: 9ead9373f350c7ad438257a9e8b8977a67b900eb
2019-10-23 11:15:43 -07:00
Wenjia Zhang 41da2ced4c Replace github.com/coreos/etcd by go.etcd.io/etcd
Kubernetes-commit: 3b274fad2a719dc1fd0eaef6c55a0d344db10242
2019-10-15 22:38:43 -07:00
Ted Yu 0026dd975b etcd health check key should have proper prefix
Kubernetes-commit: f4941e265e1a41a1b8846eae61a4049785cabb7e
2019-10-21 17:03:24 -07:00
Walter Fender da748a626b Add support for konnectivity service to the etcd3 client.
If konnectivity service is enabled, the etcd client will now use it.
This did require moving a few methods to break circular dependencies.

Factored in feedback from lavalamp and wenjiaswe.

Kubernetes-commit: edbb0fa2fe2084d5d9ce0cf9dc0d1b2d820bb392
2019-08-27 15:58:06 -07:00
Gyuho Lee 1d75397d46 k8s/apiextensions-apiserver/test/integration: block etcd client creation until connection is up
The new etcd balancer (>3.3.14, 3.4.0) uses an asynchronous resolver for
endpoints. Without "WithBlock", the client may return before the
connection is up.

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>

Kubernetes-commit: a254d0e2a67645948c9631d4bf11ef60aa26f5ae
2019-08-14 17:28:54 -07:00
Han Kang 17306f6993 add comment about explicitly registering grpcprom client metrics
Kubernetes-commit: 1700a315c188e4b9d434d1e51de75426aa9d7867
2019-08-27 10:30:57 -07:00
Han Kang b9084e350a migrate kube-apiserver metrics to stability framework
Kubernetes-commit: 466980dd747e06e55451301c624eecccfa505123
2019-08-22 15:38:42 -07:00
wojtekt 9698803236 Move etcd/testing to etcd3/testing
Kubernetes-commit: 7497260e54c555c1fb1def741267f9dc23fe7dce
2019-07-01 10:38:19 +02:00
Jordan Liggitt 24677e9349 Enable paging by default in etcd options, by feature flag in sample-apiserver
Kubernetes-commit: 90cd672ab690cb387684603316dcd4550af1006b
2019-04-30 17:37:46 -04:00
Jordan Liggitt c20c25cef4 Remove unused quorum field
Kubernetes-commit: 35757a4b96b49fc21bb9bc8b47c5a86f005b166a
2019-04-30 17:39:31 -04:00
Chao Xu d99ef88606 Expose storage version hash
Kubernetes-commit: 3b618af0d435628feedf06f97bd1c69340d07d95
2019-01-14 19:31:25 -08:00
Justin SB bf98046128 Remove executable file permission from OWNERS files
Kubernetes-commit: dd19b923b7c26420af39fcf4eedfa213b236c8d3
2019-01-03 12:18:20 -05:00
Roy Lenferink 4c9524b9fb Updated OWNERS files to include link to docs
Kubernetes-commit: b43c04452f3b563473b5c2a765d4ac18cc0ff58f
2019-01-30 20:05:00 +01:00
Lorenz Brun 278dec697f Consider prefix in health check
Kubernetes-commit: df3033a24095551db25dbab385ac3e630c8bd280
2019-01-15 21:22:07 +01:00
Lorenz Brun 901324b169 Fix etcd healthcheck for consensus failures
Kubernetes-commit: 19b2758919daa563522aec93956e326e786ff79f
2019-01-14 20:47:41 +01:00