Commit Graph

542 Commits

Author SHA1 Message Date
Wojciech Tyczyński a47bbc6347 Minor cleanup in etcd3 tests
Kubernetes-commit: 83399ccfb8822cc31a6a3a66e8352591d88feaa8
2022-10-31 18:02:40 +01:00
Wojciech Tyczyński a24d8963d1 Refactor remaining etcd3 tests to make them generic
Kubernetes-commit: e6ddb69e6fd06bf9a02fa731f12472f3c375762d
2022-10-31 18:02:19 +01:00
Wojciech Tyczyński c38aa93533 Reuse generic TestGet in cache tests.
Kubernetes-commit: 75a1ef87b32213a7d3d1c2027dd515e4f74c1777
2022-10-27 16:00:30 +02:00
Wojciech Tyczyński 07db92af49 Fix setting resource version on deletion
Kubernetes-commit: bbcf5e38776f2b18026539a0fbcf3aa505386c1f
2022-10-26 21:37:19 +02:00
Wojciech Tyczyński 499bbb88dc Refactor WatchError test to make it generic
Kubernetes-commit: b3f9272d57a71a9c50b38754b7309ddf68adb93f
2022-10-28 11:35:58 +02:00
Wojciech Tyczyński 3f5a3e0ae0 Refactor etcd3 list consistency test
Kubernetes-commit: cd5da36c92f3ed52debc22c39a7cd9d369b0eecd
2022-10-27 10:19:09 +02:00
Wojciech Tyczyński c527f6432a Minor cleanup of etcd3 tests
Kubernetes-commit: bbe1ebc82aa019118c91e8447517f91b9d036c1e
2022-10-27 09:48:06 +02:00
Wojciech Tyczyński f6802e074e Refactor storage tests using compaction
Kubernetes-commit: b02f172cbdf7e824d13a6a9c3a9b9fe4f3f92afb
2022-10-26 20:47:19 +02:00
Wojciech Tyczyński 834cf0fc14 Refactor compaction in etcd3 tests
Kubernetes-commit: 7da7ddd779f9ea835f0c57deae05e050c543066b
2022-10-26 16:34:28 +02:00
David Ashpole aa161f2fc0 migrate apiserver utiltrace usage to component-base/tracing
Kubernetes-commit: de26b9023f2872c5cd7e15fad5dd5ab649222c13
2022-10-20 18:15:38 +00:00
Tim Allclair bd7c7f52c2 Consolidate AuditContext
Kubernetes-commit: f1d684b7b60b39b7dc1eb4156307c593f0ba74e1
2022-07-12 11:53:57 -07:00
Wojciech Tyczyński 3afe8ebf5f Minor cleanup of etcd3 tests
Kubernetes-commit: e04fe81dfcff189ef5162b9b26b55760f613a010
2022-10-26 13:05:49 +02:00
Wojciech Tyczyński 73a664638b Refactor pagination tests
Kubernetes-commit: 6c8ce894e18a2e2cae20edd2f6e79db9407ebce3
2022-10-26 12:18:21 +02:00
Wojciech Tyczyński dec6f492e8 Refactor GetReads
Kubernetes-commit: 8472e1bc13b0fa8aa3a67e5fcf2f13bfd7974cd3
2022-10-26 11:30:32 +02:00
Wojciech Tyczyński 5a8fdf8480 Move GuaranteedUpdateChecksData test to generic package
Kubernetes-commit: afc5ded83931cdb8a2643486d6834eb8d9b04588
2022-10-25 14:31:20 +02:00
Wojciech Tyczyński d9a3685d6f Minor cleanup of storage tests
Kubernetes-commit: 5344bc5e1b7e50d5ff359c714d522b421b28bd12
2022-10-25 14:09:36 +02:00
Wojciech Tyczyński 1960fa837f Move GuaranteedUpdate test to generic test package
Kubernetes-commit: 012676acc3dd2b235bd601e4105294d2715f155b
2022-10-25 14:04:00 +02:00
Han Kang e9f139f163 swap name for storage metric for underlying storage db
Change-Id: Ic7cd9bf5ef29d9c51a728ef37c94d77172fbf678

Kubernetes-commit: 3a2d1bab26bfb9e7d887207dec0c51645c7017fe
2022-10-24 15:27:05 -04:00
Oscar Utbult 9d3d7b483a grammar: replace all occurrences of "the the" with "the"
Kubernetes-commit: e4f776f23098ecf942cafa898777195adbc800f1
2022-09-17 22:58:01 +02:00
lixiaobing1 c144979a82 replace WithInsecure() with WithTransportCredentials()
Kubernetes-commit: 7892175acdb329d44cf1f34230f78e608b3cb736
2022-10-15 16:41:53 +08:00
Monis Khan f4ac8fc8bb prefixTransformers: allocate after transformation length is known
Signed-off-by: Monis Khan <mok@microsoft.com>

Kubernetes-commit: 59e1a32fc8ed35e328a3971d3a1d640ffc28ff55
2022-10-17 16:56:40 -04:00
Monis Khan be9579fc15 k8s.io/apiserver/pkg/storage/value: allow encryption-at-rest approval
Signed-off-by: Monis Khan <mok@microsoft.com>

Kubernetes-commit: b68bc0678d5876e4c11c7d5289f777b6e37c4917
2022-10-17 17:07:56 -04:00
Harsha Narayana 1da54ec21a kmsv2: enable logging for kmsv2 enc/dec operations
Kubernetes-commit: 79d741f1f8efcfc75cecd22898c7b6b689449f0a
2022-08-31 22:08:55 +05:30
Monis Khan 8d68e6f323 Load encryption config once
This change updates the API server code to load the encryption
config once at start up instead of multiple times.  Previously the
code would set up the storage transformers and the etcd healthz
checks in separate parse steps.  This is problematic for KMS v2 key
ID based staleness checks which need to be able to assert that the
API server has a single view into the KMS plugin's current key ID.

Signed-off-by: Monis Khan <mok@microsoft.com>

Kubernetes-commit: f507bc255382b2e2095351053bc17e74f7100d35
2022-08-29 17:25:48 -04:00
Anish Ramasekar 1411f0e151 kmsv2: validate annotations key and size
Signed-off-by: Anish Ramasekar <anish.ramasekar@gmail.com>

Kubernetes-commit: d1fb258ff2d009f202cff3fdd25e6fd2bbda08ef
2022-09-14 21:58:17 +00:00
Monis Khan c602291fa1 encryption config: no-op refactor to prepare for single loading
Signed-off-by: Monis Khan <mok@microsoft.com>

Kubernetes-commit: db850931a8699e780dd794e1763fd0e54b4239b5
2022-08-29 17:25:48 -04:00
Anish Ramasekar c027ae3881 Add staging directory for kms
- Moves kms proto apis to the staging repo
- Updates generate and verify kms proto scripts to check staging repo

Signed-off-by: Anish Ramasekar <anish.ramasekar@gmail.com>

Kubernetes-commit: c3794e2377016b1c18b1dcb63dc61d686c8ebcbf
2022-08-23 20:22:09 +00:00
Anish Ramasekar ec520ccd91 [KMS]: update envelope caching unit tests
Signed-off-by: Anish Ramasekar <anish.ramasekar@gmail.com>

Kubernetes-commit: 92dce5de71f752c8c136ec7c7417a73d50317cf5
2022-09-15 18:01:48 +00:00
viveksahu26 6cb2fda0ab fix warnings or linter errors
Signed-off-by: viveksahu26 <vivekkumarsahu650@gmail.com>

Kubernetes-commit: 386bc4a7fa3e975cb247d300fdc5ad1b14a15605
2022-09-05 19:10:43 +05:30
Antonio Ojea 1239036585 rate limit /healthz etcd healthchecks
return the last request error, instead of last error received
The rate limit allows 1 event per healthcheck timeout / 2

Kubernetes-commit: 510a85c53a5138babb1650fadd328e6f34baa03b
2022-08-29 11:09:58 +02:00
Antonio Ojea d2581bb0e0 fix etcd unit tests
stop leaking goroutines

reduce etcd test duration

Kubernetes-commit: dd6d3d95cdeb0e165e8365212d85d0f3b972d3e8
2022-08-28 23:13:45 +02:00
Antonio Ojea fb26cb4f0c don't serialize etcd healthchecks
Kubernetes-commit: 5a67248115ec0bc762c0351a73810117ac0bb814
2022-08-29 17:57:01 +02:00
Monis Khan 70b4742ce2 kms: fix go routine leak in gRPC connection
Signed-off-by: Monis Khan <mok@microsoft.com>

Kubernetes-commit: 4e68e9b5ad70ae074b3fb20f0fb2ba25d0792274
2022-08-24 01:51:19 +00:00
Andy Goldstein 8587d29223 etcd3: include GroupResource in logs/metrics
Use GroupResource instead of object reflection when recording the
following metrics:

- etcd_request_duration_seconds
- etcd_bookmark_counts

Add GroupResource to logs and traces where only reflection-based typing
was previously used.

Both of these changes allow us to disginguish between different CRDs,
all of which are represented as *unstructured.Unstructured.

Signed-off-by: Andy Goldstein <andy.goldstein@redhat.com>

Kubernetes-commit: 305fa2add60ad507417304d865f001006d5175fe
2022-08-25 13:55:55 -04:00
Andy Goldstein 7eb011f596 watch cache: metrics: objectType -> group resource
Use the group resource instead of objectType in watch cache metrics,
because all CustomResources are grouped together as
*unstructured.Unstructured, instead of 1 entry per type.

Signed-off-by: Andy Goldstein <andy.goldstein@redhat.com>

Kubernetes-commit: d08b69e8d35a5aa73a178c508f9b0e1ad74b882d
2022-08-11 15:51:21 -04:00
Andy Goldstein ecf3a57374 watch cache: log GroupResource, not objectType
All CustomResources are treated as *unstructured.Unstructured, leading
the watch cache to log anything related to CRs as Unstructured. This
change uses the schema.GroupResource instead of object type for all type
related log messages in the watch cache, resulting in distinct output
for each CR type.

Signed-off-by: Andy Goldstein <andy.goldstein@redhat.com>

Kubernetes-commit: 397533a4c2df9639ff4422c907d06fae195a1835
2022-08-11 15:45:04 -04:00
Anish Ramasekar bdd7082eed chore(kms): remove unused plugin name and migrate from deprecated `io/ioutil` pkg
Signed-off-by: Anish Ramasekar <anish.ramasekar@gmail.com>

Kubernetes-commit: 7db7a63959162d743f771183bf4e88e82afef868
2022-08-23 22:55:22 +00:00
Anish Ramasekar 225e26ac4a Implement KMS v2alpha1
- add feature gate
- add encrypted object and run generated_files
- generate protobuf for encrypted object and add unit tests
- move parse endpoint to util and refactor
- refactor interface and remove unused interceptor
- add protobuf generate to update-generated-kms.sh
- add integration tests
- add defaulting for apiVersion in kmsConfiguration
- handle v1/v2 and default in encryption config parsing
- move metrics to own pkg and reuse for v2
- use Marshal and Unmarshal instead of serializer
- add context for all service methods
- check version and keyid for healthz

Signed-off-by: Anish Ramasekar <anish.ramasekar@gmail.com>

Kubernetes-commit: f19f3f409938ff9ac8a61966e47fbe9c6075ec90
2022-06-29 20:51:35 +00:00
Sally O'Malley 4f9e133507 kubelet tracing
Signed-off-by: Sally O'Malley <somalley@redhat.com>
Co-authored-by: David Ashpole <dashpole@google.com>

Kubernetes-commit: 47e7d8034ff3be8e198dde6a671d05a11c30e333
2021-10-10 09:17:27 -04:00
Nic Cope 9b243e9d90 Copy etcd client debug level logic from upstream
Replicated from https://github.com/etcd-io/etcd/blob/v3.5.4/client/v3/logger.go#L47

The logic of this function doesn't make a lot of sense to me, but
copying it will avoid any behaviour change.

Signed-off-by: Nic Cope <nicc@rk0n.org>

Kubernetes-commit: c1aa7a0fe73cbcab8e70f7b73a845ae9394f9a71
2022-07-29 14:26:31 -07:00
Nic Cope a105c2570c Give etcd client logger a name
Logic copied from https://github.com/etcd-io/etcd/blob/v3.5.4/client/v3/client.go#L374

Signed-off-by: Nic Cope <nicc@rk0n.org>

Kubernetes-commit: f54d2606336e2e8130339d2a0bc04fac6906aa78
2022-07-29 14:24:17 -07:00
Nic Cope 463756f91d Share a single etcd3 client logger across all clients
Currently the API server creates one etcd client per CRD. If clients
aren't provided a logger they'll each create their own. These loggers
can account for ~20% of API server memory consumption on a cluster with
hundreds of CRDs.

Signed-off-by: Nic Cope <nicc@rk0n.org>

Kubernetes-commit: 0c81eabb853e581abbcb37ebf094af3316e1012e
2022-07-28 19:51:55 -07:00
Nic Cope ef17269e10 Disable the etcd3 client logger
This logger is responsible for 20% of the API server's memory usage when
many CRDs are installed. See the below issue for more context.

https://github.com/kubernetes/kubernetes/issues/111476

Signed-off-by: Nic Cope <nicc@rk0n.org>

Kubernetes-commit: 0e5401c93940126beac45264aa056507b0950075
2022-07-27 14:44:49 -07:00
Wojciech Tyczyński 6ab5ae374a Fix draining cacher tests
Kubernetes-commit: a530a6898a36d43eef766f041eab4e4b6d5adedb
2022-07-28 10:20:23 +02:00
Lukasz Szaszkiewicz 684cf11215 cacher: add support for consistent streaming
design details https://github.com/kubernetes/enhancements/tree/master/keps/sig-api-machinery/3157-watch-list#design-details

Kubernetes-commit: debace151cfffc1b5d805341efcbf5f0062e95ed
2022-07-05 15:48:30 +02:00
Marcel Zięba 3c024b4916 Add option to retry internal api error in reflector.
Kubernetes-commit: 0b2b6489de8f75d5299f54180617601126bb8878
2022-07-25 08:02:54 +00:00
Maciej Wyrzuc cb0bb2af35 Add additional etcd check to readyz with 2 seconds timeout.
Kubernetes-commit: b42045a64fd07fb948660839b6c7c14440bee9df
2022-07-25 13:08:50 +00:00
Davanum Srinivas 7e94033a61 Generate and format files
- Run hack/update-codegen.sh
- Run hack/update-generated-device-plugin.sh
- Run hack/update-generated-protobuf.sh
- Run hack/update-generated-runtime.sh
- Run hack/update-generated-swagger-docs.sh
- Run hack/update-openapi-spec.sh
- Run hack/update-gofmt.sh

Signed-off-by: Davanum Srinivas <davanum@gmail.com>

Kubernetes-commit: a9593d634c6a053848413e600dadbf974627515f
2022-07-19 20:54:13 -04:00
Mikko Ylinen 12a8b7fef3 grpc: move to use grpc.WithTransportCredentials()
v1.43.0 marked grpc.WithInsecure() deprecated so this commit moves to use
what is the recommended replacement:

grpc.WithTransportCredentials(insecure.NewCredentials())

Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>

Kubernetes-commit: 2c8bfad9106039aa15233b5bf7282b25a7b7e0a0
2022-05-11 12:13:28 +03:00
Wojciech Tyczyński 2049cfbb9e Adjust watch channel sizes in watchcache
Kubernetes-commit: 0db5c05bdb8bbc510307a48cbade712583bb009e
2022-04-28 11:56:41 +02:00