Commit Graph

115 Commits

Author SHA1 Message Date
ZhangYu 7e4c7c03fd change unnecessary switch to if in metrics.go
Kubernetes-commit: 48b9f7bb6c8298a00cdc04e394af15afe5804a3e
2022-11-30 11:04:03 +08:00
Rita Zhang f471919cab kmsv2: add metrics
Signed-off-by: Rita Zhang <rita.z.zhang@gmail.com>

Kubernetes-commit: bd0f7f8ee8f7f1c7809e17fa60804bb37f65c495
2023-01-29 22:40:18 -08:00
Anish Ramasekar 0304cd9f08 logcheck.conf: ensure that kms and value/encrypt uses structured logging
Signed-off-by: Anish Ramasekar <anish.ramasekar@gmail.com>

Kubernetes-commit: b6b00e65bcfe40fdaca9ce8b214baf5015ad7c5c
2023-02-10 00:17:54 +00:00
Anish Ramasekar c9656aa265 [KMS] Use structured logging in value/encrypt
Signed-off-by: Anish Ramasekar <anish.ramasekar@gmail.com>

Kubernetes-commit: 6db7496db60816c36b519a940fa1b0af1c0898da
2023-02-09 19:54:37 +00:00
Anish Ramasekar 63ca46e40a [KMSv2] Add metrics for grpc service
Signed-off-by: Anish Ramasekar <anish.ramasekar@gmail.com>

Kubernetes-commit: de3b2d525b42d6eedeaa0b157f0f5ab98a4821f4
2023-02-09 07:46:29 +00:00
Anish Ramasekar 421ef770de [KMSv2] store hash of encrypted DEK as key in cache
Signed-off-by: Anish Ramasekar <anish.ramasekar@gmail.com>

Kubernetes-commit: f72cf5c510cf2cf7b8ee375f5c2ec835e3ed225a
2023-01-27 00:19:32 +00:00
Anish Ramasekar 9fb6b944f0 kmsv2: implement expire cache with clock
Signed-off-by: Anish Ramasekar <anish.ramasekar@gmail.com>

Kubernetes-commit: 4804baa01187b4251bd632e07721d875f567d6f1
2022-09-14 20:01:45 +00:00
Tim Hockin 979698ec2b Set proto go_package: kms API
This creates some diff to the *.pb.go files to note that
in the "options".

You can dump the gzipped blob with the following program (thanks
StackOverflow!):

```go
package main

import (
	"bytes"
	"compress/gzip"
	"encoding/json"
	"fmt"
	"os"

	"io/ioutil"

	proto "github.com/golang/protobuf/proto"
	dpb "github.com/golang/protobuf/protoc-gen-go/descriptor"
)

func main() {
	m := map[string][]byte{
		"before": blobv1,
		"after": blobv2,
	}
	arg := os.Args[1]
	dump(m[arg])
}

func dump(bytes []byte) {
	fd, err := decodeFileDesc(bytes)
	if err != nil {
		panic(err)
	}
	b, err := json.MarshalIndent(fd, "", "  ")
	if err != nil {
		panic(err)
	}
	fmt.Println(string(b))
}

// decompress does gzip decompression.
func decompress(b []byte) ([]byte, error) {
	r, err := gzip.NewReader(bytes.NewReader(b))
	if err != nil {
		return nil, fmt.Errorf("bad gzipped descriptor: %v", err)
	}
	out, err := ioutil.ReadAll(r)
	if err != nil {
		return nil, fmt.Errorf("bad gzipped descriptor: %v", err)
	}
	return out, nil
}

func decodeFileDesc(enc []byte) (*dpb.FileDescriptorProto, error) {
	raw, err := decompress(enc)
	if err != nil {
		return nil, fmt.Errorf("failed to decompress enc: %v", err)
	}

	fd := new(dpb.FileDescriptorProto)
	if err := proto.Unmarshal(raw, fd); err != nil {
		return nil, fmt.Errorf("bad descriptor: %v", err)
	}
	return fd, nil
}
```

Kubernetes-commit: ab11d8a4495aa0ff03b94c8d1a5345abfcca2c59
2023-01-14 10:21:53 -08:00
Patrick Ohly 8f8c30ff8f logging: fix names of keys
The stricter checking with the upcoming logcheck v0.4.1 pointed out these names
which don't comply with our recommendations in
https://github.com/kubernetes/community/blob/master/contributors/devel/sig-instrumentation/migration-to-structured-logging.md#name-arguments.

Kubernetes-commit: bc6c7fa91201348d010b638fbadf32007c0ac546
2023-01-16 15:04:01 +01:00
Tim Hockin 72929fc438 Call update-proto-bindings from update-codegen
One script to bring them all ...

Kubernetes-commit: 4dae505d531e149881788dc36148602967419c75
2023-01-05 15:41:51 -08:00
Tim Hockin 05e000c1bd Merge 5 fragile proto-bindings scripts into 1
Each of these scripts is basically identical, and all were too brittle.
Now they should be more resilient and easier to manage.  The script
still needs to be updated if we add new ones, which I do not love.

More cleanup to follow.

Kubernetes-commit: e0ecccff3f5148cc167117ac73233b4edc1640d8
2023-01-05 13:53:59 -08:00
Rita Zhang 19c40b040a kmsv2: use status key ID to update staleness of encrypted data
Signed-off-by: Rita Zhang <rita.z.zhang@gmail.com>

Kubernetes-commit: 510ac9b3919b62cb52e166192a7d0d09a9b2dfb5
2022-12-02 13:32:34 -08:00
Krzysztof Ostrowski a85078bf03 apiserver/kmsv2: mv Service interface into kmsv2
Signed-off-by: Krzysztof Ostrowski <kostrows@redhat.com>

Kubernetes-commit: b7701b00eaa8cdc2103beb8ab78f625cc3b62d90
2023-01-09 14:36:06 +01:00
lixiaobing1 c144979a82 replace WithInsecure() with WithTransportCredentials()
Kubernetes-commit: 7892175acdb329d44cf1f34230f78e608b3cb736
2022-10-15 16:41:53 +08:00
Monis Khan f4ac8fc8bb prefixTransformers: allocate after transformation length is known
Signed-off-by: Monis Khan <mok@microsoft.com>

Kubernetes-commit: 59e1a32fc8ed35e328a3971d3a1d640ffc28ff55
2022-10-17 16:56:40 -04:00
Monis Khan be9579fc15 k8s.io/apiserver/pkg/storage/value: allow encryption-at-rest approval
Signed-off-by: Monis Khan <mok@microsoft.com>

Kubernetes-commit: b68bc0678d5876e4c11c7d5289f777b6e37c4917
2022-10-17 17:07:56 -04:00
Harsha Narayana 1da54ec21a kmsv2: enable logging for kmsv2 enc/dec operations
Kubernetes-commit: 79d741f1f8efcfc75cecd22898c7b6b689449f0a
2022-08-31 22:08:55 +05:30
Monis Khan 8d68e6f323 Load encryption config once
This change updates the API server code to load the encryption
config once at start up instead of multiple times.  Previously the
code would set up the storage transformers and the etcd healthz
checks in separate parse steps.  This is problematic for KMS v2 key
ID based staleness checks which need to be able to assert that the
API server has a single view into the KMS plugin's current key ID.

Signed-off-by: Monis Khan <mok@microsoft.com>

Kubernetes-commit: f507bc255382b2e2095351053bc17e74f7100d35
2022-08-29 17:25:48 -04:00
Anish Ramasekar 1411f0e151 kmsv2: validate annotations key and size
Signed-off-by: Anish Ramasekar <anish.ramasekar@gmail.com>

Kubernetes-commit: d1fb258ff2d009f202cff3fdd25e6fd2bbda08ef
2022-09-14 21:58:17 +00:00
Monis Khan c602291fa1 encryption config: no-op refactor to prepare for single loading
Signed-off-by: Monis Khan <mok@microsoft.com>

Kubernetes-commit: db850931a8699e780dd794e1763fd0e54b4239b5
2022-08-29 17:25:48 -04:00
Anish Ramasekar c027ae3881 Add staging directory for kms
- Moves kms proto apis to the staging repo
- Updates generate and verify kms proto scripts to check staging repo

Signed-off-by: Anish Ramasekar <anish.ramasekar@gmail.com>

Kubernetes-commit: c3794e2377016b1c18b1dcb63dc61d686c8ebcbf
2022-08-23 20:22:09 +00:00
Anish Ramasekar ec520ccd91 [KMS]: update envelope caching unit tests
Signed-off-by: Anish Ramasekar <anish.ramasekar@gmail.com>

Kubernetes-commit: 92dce5de71f752c8c136ec7c7417a73d50317cf5
2022-09-15 18:01:48 +00:00
Monis Khan 70b4742ce2 kms: fix go routine leak in gRPC connection
Signed-off-by: Monis Khan <mok@microsoft.com>

Kubernetes-commit: 4e68e9b5ad70ae074b3fb20f0fb2ba25d0792274
2022-08-24 01:51:19 +00:00
Anish Ramasekar bdd7082eed chore(kms): remove unused plugin name and migrate from deprecated `io/ioutil` pkg
Signed-off-by: Anish Ramasekar <anish.ramasekar@gmail.com>

Kubernetes-commit: 7db7a63959162d743f771183bf4e88e82afef868
2022-08-23 22:55:22 +00:00
Anish Ramasekar 225e26ac4a Implement KMS v2alpha1
- add feature gate
- add encrypted object and run generated_files
- generate protobuf for encrypted object and add unit tests
- move parse endpoint to util and refactor
- refactor interface and remove unused interceptor
- add protobuf generate to update-generated-kms.sh
- add integration tests
- add defaulting for apiVersion in kmsConfiguration
- handle v1/v2 and default in encryption config parsing
- move metrics to own pkg and reuse for v2
- use Marshal and Unmarshal instead of serializer
- add context for all service methods
- check version and keyid for healthz

Signed-off-by: Anish Ramasekar <anish.ramasekar@gmail.com>

Kubernetes-commit: f19f3f409938ff9ac8a61966e47fbe9c6075ec90
2022-06-29 20:51:35 +00:00
Mikko Ylinen 12a8b7fef3 grpc: move to use grpc.WithTransportCredentials()
v1.43.0 marked grpc.WithInsecure() deprecated so this commit moves to use
what is the recommended replacement:

grpc.WithTransportCredentials(insecure.NewCredentials())

Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>

Kubernetes-commit: 2c8bfad9106039aa15233b5bf7282b25a7b7e0a0
2022-05-11 12:13:28 +03:00
Anish Ramasekar c6c1465ed7 Add KMS v2alpha1 API
Signed-off-by: Anish Ramasekar <anish.ramasekar@gmail.com>

Kubernetes-commit: 907545445ab8b4e34c1068ab9828a930c30cbfc4
2022-05-24 23:43:09 +00:00
Anish Ramasekar e442eafb33 feat: prepare KMS data encryption for migration to AES-GCM
Signed-off-by: Anish Ramasekar <anish.ramasekar@gmail.com>
Co-authored-by: Monis Khan <mok@vmware.com>
Signed-off-by: Anish Ramasekar <anish.ramasekar@gmail.com>

Kubernetes-commit: 90b42f91fd904b71fd52ca9ae55a5de73e6b779a
2022-03-16 17:54:10 +00:00
Steve Kuznetsov af1cb1cefe storage: transformers: pass a context.Context
When an envelope transformer calls out to KMS (for instance), it will be
very helpful to pass a `context.Context` to allow for cancellation. This
patch does that, while passing the previously-expected additional data
via a context value.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>

Kubernetes-commit: 27312feb9983c18d1daf00afba788727d024cdd0
2022-02-17 07:29:44 -08:00
Davanum Srinivas 56a3a30ae1 Check in OWNERS modified by update-yamlfmt.sh
Signed-off-by: Davanum Srinivas <davanum@gmail.com>

Kubernetes-commit: 9405e9b55ebcd461f161859a698b949ea3bde31d
2021-12-09 21:31:26 -05:00
tiloso ab3cca3647 Fix staticcheck in apiserver and client-go pkgs
Kubernetes-commit: 830a137d2ea70663cd94403595313b95ac40ffe8
2021-06-19 22:03:46 +02:00
Stephen Augustus 771ffe6475 generated: Run hack/update-gofmt.sh
Signed-off-by: Stephen Augustus <foo@auggie.dev>

Kubernetes-commit: 481cf6fbe753b9eb2a47ced179211206b0a99540
2021-08-12 17:13:11 -04:00
Davanum Srinivas fe1610f3fe switch from golang-lru to the one in k8s.io/utils
Signed-off-by: Davanum Srinivas <davanum@gmail.com>

Kubernetes-commit: 79d0c6cdc10293c9bfe644ce31dc186a936579b0
2021-07-07 13:45:07 -04:00
Shihang Zhang 4a6863aa9a the last upperbound of kms latency metric is too small
Kubernetes-commit: 6d7c83f2cd19455107bc02bc98fed2296bb46dca
2021-02-23 14:19:25 -08:00
卢振兴10069964 549cbbf8de fix broken link in some files
Kubernetes-commit: b29a5fb0746f772b38da570cd8fdc77396ffca31
2021-04-13 08:43:24 +08:00
Jiaxin Shan dfad5032fb Fix ALPHA stability level reference link
Kubernetes-commit: e01a21469b9719f7d0e84021c032cd8f0016b5d2
2021-01-31 15:37:07 -08:00
Davanum Srinivas 5879417a28 switch over k/k to use klog v2
Signed-off-by: Davanum Srinivas <davanum@gmail.com>

Kubernetes-commit: 442a69c3bdf6fe8e525b05887e57d89db1e2f3a5
2020-04-17 15:25:06 -04:00
immutablet 66b663f223 Instrument DEK cache fill and request inter-arrival times.
Kubernetes-commit: 684d6fb0ade6ac088af391cedd70bc847941a54f
2020-02-18 16:39:53 -08:00
Davanum Srinivas cde2338e26 update generated files
Kubernetes-commit: b3853138a4f1a0637ec3c38a5c59f8228765b261
2020-01-13 17:56:56 -05:00
danielqsj fdd895e164 remove last part of deprecated metrics
Kubernetes-commit: 606e4503cf4cea9f05cfd467b88c88cf9c5648d9
2019-12-27 14:23:11 +08:00
danielqsj 669e87efbe clean SinceInMicroseconds, convert to SinceInSeconds
Kubernetes-commit: ab182552b485ceed96c3b05d284ae9480377daf8
2019-04-15 16:18:01 +08:00
danielqsj b31ce5f20e remove deprecated metrics of apiserver
Kubernetes-commit: 060756babbb6991cf9426c38be595d23e7124b4e
2019-04-12 16:15:13 +08:00
immutablet 5cec6b4746 Add defaulting logic for EncryptionConfiguration.
Kubernetes-commit: a151aa35dc21881d178e498141e5f58df13fb400
2019-11-14 22:53:18 -08:00
RainbowMango 5f15b07078 Hide apiserver metrics that have been deprecated in 1.15
Kubernetes-commit: 627a9abc9edb2b06c3b30e53ccd422e15c1f6ea1
2019-10-12 21:17:01 +08:00
RainbowMango de8c866328 Hide apiserver metrics that have been deprecated in 1.14
Update E2E test accordingly.

Kubernetes-commit: 5e0695e339ff0887ebebddfa8630de191f7fa235
2019-10-12 20:25:10 +08:00
immutablet 29f5d9ba4a Move the common logic of checking for kms-plugin's version into gRPC client interceptor.
Kubernetes-commit: d2b4723302e61efdd942d59801f18ae3ec24887a
2019-10-25 15:08:52 -07:00
immutablet 3079381054 Use single kms-plugin mock in unit and integration tests.
Kubernetes-commit: 4d24b41410f2253c7b2f9e2b6d56910894016c61
2019-10-11 15:25:05 -07:00
RainbowMango dfdab3a0c8 Remove direct reference to prometheus from apiserver/pkg/storage/value.
Kubernetes-commit: 78d0f1827acfb8400d6e1622d55bdfed64b047ed
2019-10-09 21:58:51 +08:00
immutablet 5035dae3d5 Replace deprecated methods in the logic involved in the construction of gRPC connection to kms-plugin.
Kubernetes-commit: e50c264c35a32200febde3b10838b2ef2f986c39
2019-10-07 15:57:47 -07:00
RainbowMango 7687c3a4e8 Migrate prometheus bucket functionality to metrics stability framework.
Kubernetes-commit: 1f9ab1b85c5560a155b1702d1b5d28aa2d95a1b7
2019-09-12 17:10:44 +08:00