istio
/
istio.io
mirror of https://github.com/istio/istio.io.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

add multi-signer docs (#11419) 

* addess review comments

* address review comments

* address review comments
This commit is contained in:
Iris 2022-07-16 07:15:58 +08:00 committed by GitHub
parent e465d49209
commit e202a0995d
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 201 additions and 181 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

374
content/en/docs/tasks/security/cert-management/custom-ca-k8s/index.md
View File

@ -16,173 +16,161 @@ This feature requires Kubernetes version >= 1.18.
This task shows how to provision Workload Certificates
using a custom certificate authority that integrates with the
[Kubernetes CSR API](https://kubernetes.io/docs/reference/access-authn-authz/certificate-signing-requests/).
[Kubernetes CSR API](https://kubernetes.io/docs/reference/access-authn-authz/certificate-signing-requests/). Different workloads can get their certificates signed from different cert-signers. Each cert-signer is effectively a different CA. It is expected that workloads whose certificates are issued from the same cert-signer can talk MTLS to each other while workloads signed by different signers cannot.
This feature leverages [Chiron](/blog/2019/dns-cert/), a lightweight component linked with Istiod that signs certificates using the Kubernetes CSR API.
This task is split into two parts. The first part demonstrates how to use the Kubernetes CA itself to sign workload certificates.
The second part demonstrates how to use a custom CA that integrates with the Kubernetes CSR API to sign your certificates.
For this example, we use [open-source cert-manager](https://cert-manager.io).
Cert-manager has added [experimental Support for Kubernetes `CertificateSigningRequests`](https://cert-manager.io/docs/usage/kube-csr/) starting with version 1.4.
## Part 1: Using Kubernetes CA
## Deploy Custom CA controller in the Kubernetes cluster
1. Deploy cert-manager according to the [installation doc](https://cert-manager.io/docs/installation/).
{{< warning >}}
Note that this example should only be used for basic evaluation. The use of the `kubernetes.io/legacy-unknown` signer is NOT recommended in production environments.
Note: Make sure to enable feature gate: `--feature-gates=ExperimentalCertificateSigningRequestControllers=true`
{{< /warning >}}
### Deploying Istio with Kubernetes CA
1. Deploy Istio on the cluster using `istioctl` with the following configuration.
1. Create three self signed cluster issuers `istio-system`, `foo` and `bar` for cert-manager.
Note: Namespace issuers and other types of issuers can also be used.
{{< text bash >}}
$ cat <<EOF > ./istio.yaml
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
components:
pilot:
k8s:
env:
# Indicate to Istiod that we use a Custom Certificate Authority
- name: EXTERNAL_CA
value: ISTIOD_RA_KUBERNETES_API
# Tells Istiod to use the Kubernetes legacy CA Signer
- name: K8S_SIGNER
value: kubernetes.io/legacy-unknown
EOF
$ istioctl install --set profile=demo -f ./istio.yaml
{{< /text >}}
1. Deploy the `bookinfo` sample application in the bookinfo namespace.
Ensure that the following commands are executed in the Istio root directory.
{{< text bash >}}
$ kubectl create ns bookinfo
$ kubectl apply -f <(istioctl kube-inject -f samples/bookinfo/platform/kube/bookinfo.yaml) -n bookinfo
{{< /text >}}
### Verify that the certificates installed are correct
When the workloads are deployed, above, they send CSR Requests to Istiod which forwards them to the Kubernetes CA for signing.
If all goes well, the signed certificates are sent back to the workloads where they are then installed.
To verify that they have been signed by the Kubernetes CA, you need to first extract the signed certificates.
1. Dump all pods running in the namespace.
{{< text bash >}}
$ kubectl get pods -n bookinfo
{{< /text >}}
Pick any one of the running pods for the next step.
1. Get the certificate chain and CA root certificate used by the Istio proxies for mTLS.
{{< text bash >}}
$ istioctl pc secret <pod-name> -o json > proxy_secret
{{< /text >}}
The proxy_secret json file contains the CA root certificate for mTLS in the `trustedCA` field. Note that this certificate is base64 encoded.
1. The certificate used by the Kubernetes CA (specifically the `kubernetes.io/legacy-unknown` signer) is loaded onto the secret associated with every service account in the bookinfo namespace.
{{< text bash >}}
$ kubectl get secrets -n bookinfo
{{< /text >}}
Pick a secret-name that is associated with any of the service-accounts. These have a "token" in their name.
{{< text bash >}}
$ kubectl get secrets -n bookinfo <secret-name> -o json
{{< /text >}}
The `ca.crt` field in the output contains the base64 encoded Kubernetes CA certificate.
1. Compare the `ca.cert` obtained in the previous step with the contents of the `TrustedCA` field in the step before. These two should be the same.
1. (Optional) Follow the rest of the steps in the [bookinfo example](/docs/examples/bookinfo/) to ensure that communication between services is working as expected.
### Cleanup Part 1
* Remove the `istio-system` and `bookinfo` namespaces:
{{< text bash >}}
$ kubectl delete ns istio-system
$ kubectl delete ns bookinfo
{{< /text >}}
## Part 2: Using Custom CA
This assumes that the custom CA implements a controller that has the necessary permissions to read and sign Kubernetes CSR Requests.
Refer to the [Kubernetes CSR documentation](https://kubernetes.io/docs/reference/access-authn-authz/certificate-signing-requests/) for more details. Note that the steps below are dependent on an external-source and may change.
### Deploy Custom CA controller in the Kubernetes cluster
1. For this example, we use an [open-source Certificate Authority implementation](https://github.com/cert-manager/signer-ca).
This code builds a controller that reads the CSR resources on the Kubernetes cluster and creates certificates using local keys. Follow the instructions on the page to:
1. Build the Certificate-Controller docker image
1. Upload the image to a Docker Registry
1. Generate the Kubernetes manifest to deploy it
1. Deploy the Kubernetes manifest generated in the previous step on your local cluster in the signer-ca-system namespace.
{{< text bash >}}
$ kubectl apply -f local-ca.yaml
{{< /text >}}
Ensure that all the services are running.
{{< text bash >}}
$ kubectl get services -n signer-ca-system
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
signer-ca-controller-manager-metrics-service ClusterIP 10.8.9.25 none 8443/TCP 72s
{{< /text >}}
1. Get the public key of the CA. This is encoded in the secret "signer-ca-*" in the signer-ca-system namespace.
{{< text bash >}}
$ kubectl get secrets signer-ca-5hff5h74hm -n signer-ca-system -o json
{{< /text >}}
The `tls.crt` field contains the base64 encoded public key file. Record this for future use.
### Load the CA root certificate into a secret that istiod can access
1. Load the secret into the istiod namespace.
{{< text bash >}}
$ cat <<EOF > ./external-ca-secret.yaml
apiVersion: v1
kind: Secret
$ cat <<EOF > ./selfsigned-issuer.yaml
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: external-ca-cert
namespace: istio-system
data:
root-cert.pem: <tls.cert from the step above>
name: selfsigned-bar-issuer
spec:
selfSigned: {}
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: bar-ca
namespace: cert-manager
spec:
isCA: true
commonName: bar
secretName: bar-ca-selfsigned
issuerRef:
name: selfsigned-bar-issuer
kind: ClusterIssuer
group: cert-manager.io
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: bar
spec:
ca:
secretName: bar-ca-selfsigned
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: selfsigned-foo-issuer
spec:
selfSigned: {}
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: foo-ca
namespace: cert-manager
spec:
isCA: true
commonName: foo
secretName: foo-ca-selfsigned
issuerRef:
name: selfsigned-foo-issuer
kind: ClusterIssuer
group: cert-manager.io
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: foo
spec:
ca:
secretName: foo-ca-selfsigned
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: selfsigned-istio-issuer
spec:
selfSigned: {}
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: istio-ca
namespace: cert-manager
spec:
isCA: true
commonName: istio-system
secretName: istio-ca-selfsigned
issuerRef:
name: selfsigned-istio-issuer
kind: ClusterIssuer
group: cert-manager.io
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: istio-system
spec:
ca:
secretName: istio-ca-selfsigned
EOF
$ kubectl apply -f external-ca-secret.yaml
$ kubectl apply -f ./selfsigned-issuer.yaml
{{< /text >}}
This step is necessary for Istio to verify that the workload certificates have been signed by the correct certificate authority and to add the root-cert to the trust bundle for mTLS to work.
## Export root certificates for each cluster issuer
### Deploying Istio
{{< text bash >}}
$ export istioca=$(kubectl get clusterissuers istio-system -o jsonpath='{.spec.ca.secretName}' | xargs kubectl get secret -n cert-manager -o jsonpath='{.data.ca\.crt}' | base64 -d)
1. Deploy Istio on the cluster using `istioctl` with the following configuration.
$ export fooca=$(kubectl get clusterissuers foo -o jsonpath='{.spec.ca.secretName}' | xargs kubectl get secret -n cert-manager -o jsonpath='{.data.ca\.crt}' | base64 -d)
$ export barca=$(kubectl get clusterissuers bar -o jsonpath='{.spec.ca.secretName}' | xargs kubectl get secret -n cert-manager -o jsonpath='{.data.ca\.crt}' | base64 -d)
{{< /text >}}
## Deploy Istio with default cert-signer info
1. Deploy Istio on the cluster using `istioctl` with the following configuration. The `ISTIO_META_CERT_SIGNER` is the default cert-signer for workloads.
{{< text bash >}}
$ cat <<EOF > ./istio.yaml
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
meshConfig:
defaultConfig:
proxyMetadata:
ISTIO_META_CERT_SIGNER: istio-system
caCertificates:
- pem: |
$istioca
certSigners:
- clusterissuers.cert-manager.io/istio-system
- pem: |
$fooca
certSigners:
- clusterissuers.cert-manager.io/foo
- pem: |
$barca
certSigners:
- clusterissuers.cert-manager.io/bar
components:
pilot:
k8s:
env:
# Indicate to Istiod that we use an external signer
- name: CERT_SIGNER_DOMAIN
value: clusterissuers.cert-manager.io
- name: EXTERNAL_CA
value: ISTIOD_RA_KUBERNETES_API
# Indicate to Istiod the external k8s Signer Name
- name: K8S_SIGNER
value: example.com/foo
- name: PILOT_CERT_PROVIDER
value: k8s.io/clusterissuers.cert-manager.io/istio-system
overlays:
# Amend ClusterRole to add permission for istiod to approve certificate signing by custom signer
- kind: ClusterRole
name: istiod-clusterrole-istio-system
patches:
@ -191,72 +179,104 @@ Refer to the [Kubernetes CSR documentation](https://kubernetes.io/docs/reference
apiGroups:
- certificates.k8s.io
resourceNames:
- example.com/foo
- clusterissuers.cert-manager.io/foo
- clusterissuers.cert-manager.io/bar
- clusterissuers.cert-manager.io/istio-system
resources:
- signers
verbs:
- approve
- kind: Deployment
name: istiod
patches:
- path: spec.template.spec.containers[0].volumeMounts[-1]
value: |
# Mount external CA certificate into Istiod
name: external-ca-cert
mountPath: /etc/external-ca-cert
readOnly: true
- path: spec.template.spec.volumes[-1]
value: |
name: external-ca-cert
secret:
secretName: external-ca-cert
optional: true
EOF
$ istioctl install --set profile=demo -f ./istio.yaml
$ istioctl install -f ./istio.yaml
{{< /text >}}
1. Deploy the `bookinfo` sample application in the bookinfo namespace.
1. Deploy the `proxyconfig-bar.yaml` in the `bar` namespace to define cert-signer for workloads in the `bar` namespace.
{{< text bash >}}
$ kubectl create ns bookinfo
$ kubectl apply -f <(istioctl kube-inject -f samples/bookinfo/platform/kube/bookinfo.yaml) -n bookinfo
$ cat <<EOF > ./proxyconfig-bar.yaml
apiVersion: networking.istio.io/v1beta1
kind: ProxyConfig
metadata:
name: barpc
namespace: bar
spec:
environmentVariables:
ISTIO_META_CERT_SIGNER: bar
EOF
$ kubectl apply -f ./proxyconfig-bar.yaml
{{< /text >}}
### Verify that Custom CA certificates installed are correct
When the workloads are deployed, above, they send CSR Requests to Istiod which forwards them to the Kubernetes CA for signing. If all goes well, the signed certificates are sent back to the workloads where they are then installed. To verify that they have indeed been signed by the Kubernetes CA, you need to first extract the signed certificates.
1. Dump all pods running in the namespace.
1. Deploy the `proxyconfig-foo.yaml` in the foo namespace to define cert-signer for workloads in the `foo` namespace.
{{< text bash >}}
$ kubectl get pods -n bookinfo
$ cat <<EOF > ./proxyconfig-bar.yaml
apiVersion: networking.istio.io/v1beta1
kind: ProxyConfig
metadata:
name: foopc
namespace: foo
spec:
environmentVariables:
ISTIO_META_CERT_SIGNER: foo
EOF
$ kubectl apply -f ./proxyconfig-foo.yaml
{{< /text >}}
Pick any of the running pods for the next step.
1. Get the certificate chain and CA root certificate used by the Istio proxies for mTLS.
1. Deploy the `httpbin` and `sleep` sample application in the `foo` and `bar` namespaces.
{{< text bash >}}
$ istioctl pc secret <pod-name> -n bookinfo -o json > proxy_secret
$ kubectl label ns foo istio-injection=enabled
$ kubectl label ns bar istio-injection=enabled
$ kubectl apply -f samples/httpbin/httpbin.yaml -n foo
$ kubectl apply -f samples/sleep/sleep.yaml -n foo
$ kubectl apply -f samples/httpbin/httpbin.yaml -n bar
$ kubectl apply -f samples/sleep/sleep.yaml -n bar
{{< /text >}}
The `proxy_secret` json file contains the CA root certificate for mTLS in the `trustedCA` field. Note that this certificate is base64 encoded.
## Verify the network connectivity between `httpbin` and `sleep` within the same namespace
1. Compare the CA root certificate obtained in the step above with "root-cert.pem" value in external-ca-cert. These two should be the same.
When the workloads are deployed, they send CSR Requests with related signer info. Istiod forwards the CSR request to the custom CA for signing. The custom CA will use the correct cluster issuer or issuer to sign the cert back. Workloads under `foo` namespace will use `foo` cluster issuers while workloads under `bar` namespace will use the `bar` cluster issuers. To verify that they have indeed been signed by correct cluster issuers, We can verify workloads under the same namespace can communicate will while workloads under the different namespace cannot communicate.
1. (Optional) Follow the rest of the steps in the [bookinfo example](/docs/examples/bookinfo/) to ensure that communication between services is working as expected.
1. Check network connectivity between service `sleep` and `httpbin` in the `foo` namespace.
### Cleanup Part 2
{{< text bash >}}
$ export SLEEP_POD_FOO=$(kubectl get pod -n foo -l app=sleep -o jsonpath={.items..metadata.name})
$ kubectl exec -it $SLEEP_POD_FOO -n foo -c sleep curl http://httpbin.foo:8000/html
<!DOCTYPE html>
<html>
<head>
</head>
<body>
<h1>Herman Melville - Moby-Dick</h1>
* Remove the `istio-system` and `bookinfo` namespaces:
<div>
<p>
Availing himself of the mild...
</p>
</div>
</body>
{{< /text >}}
1. Check network connectivity between service `sleep` in the `foo` namespace and `httpbin` in the `bar` namespace.
{{< text bash >}}
$ export SLEEP_POD_FOO=$(kubectl get pod -n foo -l app=sleep -o jsonpath={ .items..metadata.name})
$ kubectl exec -it $SLEEP_POD_FOO -n foo -c sleep curl http://httpbin.bar:8000/html
upstream connect error or disconnect/reset before headers. reset reason: connection failure, transport failure reason: TLS error: 268435581:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED
{{< /text >}}
## Cleanup
* Remove the `istio-system`, `foo` and `bar` namespaces:
{{< text bash >}}
$ kubectl delete ns istio-system
$ kubectl delete ns bookinfo
$ kubectl delete ns foo
$ kubectl delete ns bar
{{< /text >}}
## Reasons to use this feature
* Added Security - Unlike `plugin-ca-cert` or the default `self-signed` option, enabling this feature means that the CA private keys need not be present in the Kubernetes cluster.
* Custom CA Integration - By specifying a Signer name in the Kubernetes CSR Request, this feature allows Istio to integrate with custom Certificate Authorities using the Kubernetes CSR API interface. This does require the custom CA to implement a Kubernetes controller to watch the `CertificateSigningRequest` Resources and act on them.
* Custom CA Integration - By specifying a Signer name in the Kubernetes CSR Request, this feature allows Istio to integrate with custom Certificate Authorities using the Kubernetes CSR API interface. This does require the custom CA to implement a Kubernetes controller to watch the `CertificateSigningRequest` and `Certificate` Resources and act on them.
* Better multi-tenancy - By specifying a different cert-signer for different workloads, certificates for different tenant's workloads can be signed by different CAs.