Update mTLS certificate rotation guidance (#2208)

* Update mTLS rotation guidance

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Update daprdocs/content/en/operations/security/mtls.md

Co-authored-by: Mark Fussell <markfussell@gmail.com>
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Update daprdocs/content/en/operations/security/mtls.md

Co-authored-by: Mark Fussell <markfussell@gmail.com>
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Update daprdocs/content/en/operations/security/mtls.md

Co-authored-by: Mark Fussell <markfussell@gmail.com>
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Update daprdocs/content/en/operations/security/mtls.md

Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Refine downtime warning

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

Co-authored-by: Mark Fussell <markfussell@gmail.com>
Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
This commit is contained in:
Bernd Verst 2022-02-22 17:56:33 -08:00 committed by GitHub
parent 428d8c2e00
commit e8e9b2c101
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 65 additions and 7 deletions

View File

@ -62,6 +62,7 @@ kubectl rollout restart statefulsets/dapr-placement-server -n <DAPR_NAMESPACE>
*Note: the control plane Sidecar Injector service does not need to be redeployed*
### Disabling mTLS with Helm
*The control plane will continue to use mTLS*
```bash
kubectl create ns dapr-system
@ -74,6 +75,7 @@ helm install \
```
### Disabling mTLS with the CLI
*The control plane will continue to use mTLS*
```
dapr init --kubernetes --enable-mtls=false
@ -86,7 +88,6 @@ In order to view the Sentry service logs, run the following command:
```
kubectl logs --selector=app=dapr-sentry --namespace <DAPR_NAMESPACE>
```
### Bringing your own certificates
Using Helm, you can provide the PEM encoded root cert, issuer cert and private key that will be populated into the Kubernetes secret used by the Sentry service.
@ -163,9 +164,52 @@ helm install \
If the Root or Issuer certs are about to expire, you can update them and restart the required system services.
{{% alert title="Avoiding downtime when rotating certificates" color="warning" %}}
To avoid downtime when rotating expiring certificates your new certificates must be signed with the same private root key as the previous certificates.
To avoid downtime when rotating expiring certificates your new certificates must be signed with the same private root key as the previous certificates. This is not currently possible using self-signed certificates generated by Dapr.
{{% /alert %}}
#### Dapr-generated self-signed certificates
1. Clear the existing Dapr Trust Bundle secret by saving the following YAML to a file (e.g. `clear-trust-bundle.yaml`) and applying this secret.
```yaml
apiVersion: v1
kind: Secret
metadata:
name: dapr-trust-bundle
labels:
app: dapr-sentry
data:
```
```bash
kubectl apply -f `clear-trust-bundle.yaml` -n <DAPR_NAMESPACE>
```
2. Restart the Dapr Sentry service. This will generate a new certificate bundle and update the `dapr-trust-bundle` Kubernetes secret.
```bash
kubectl rollout restart -n <DAPR_NAMESPACE> deployment/dapr-sentry
```
3. Once the Sentry service has been restarted, restart the rest of the Dapr control plane to pick up the new Dapr Trust Bundle.
```bash
kubectl rollout restart deploy/dapr-sentry -n <DAPR_NAMESPACE>
kubectl rollout restart deploy/dapr-operator -n <DAPR_NAMESPACE>
kubectl rollout restart statefulsets/dapr-placement-server -n <DAPR_NAMESPACE>
```
4. Restart your Dapr applications to pick up the latest trust bundle.
{{% alert title="Potential application downtime with mTLS enabled." color="warning" %}}
Restarts of deployments using service to service invocation using mTLS will fail until the callee service has also been restarted (thereby loading the new Dapr Trust Bundle). Additionally, the placement service will not be able to assign new actors (while existing actors remain unaffected) until applications have been restarted to load the new Dapr Trust Bundle.
{{% /alert %}}
```bash
kubectl rollout restart deployment/mydaprservice1 kubectl deployment/myotherdaprservice2
```
#### Custom certificates (bring your own)
First, issue new certificates using the step above in [Bringing your own certificates](#bringing-your-own-certificates).
Now that you have the new certificates, use Helm to upgrade the certificates:
@ -189,14 +233,22 @@ kubectl edit secret dapr-trust-bundle -n <DAPR_NAMESPACE>
Replace the `ca.crt`, `issuer.crt` and `issuer.key` keys in the Kubernetes secret with their corresponding values from the new certificates.
*__Note: The values must be base64 encoded__*
If you signed the new cert root with the same private key the Dapr Sentry service will pick up the new certificates automatically. You can restart your application deployments using `kubectl rollout restart` with zero downtime. It is not necessary to restart all deployments at once, as long as deployments are restarted before original certificate expiration.
If you signed the new cert root with the **same private key** the Dapr Sentry service will pick up the new certificates automatically. You can restart your application deployments using `kubectl rollout restart` with zero downtime. It is not necessary to restart all deployments at once, as long as deployments are restarted before original certificate expiration.
If you signed the new cert root with a different private key, you must restart the Dapr sentry service.
If you signed the new cert root with a **different private key**, you must restart the Dapr Sentry service, followed by the remainder of the Dapr control plane service.
```bash
kubectl rollout restart deploy/dapr-sentry -n <DAPR_NAMESPACE>
```
Once Sentry has been completely restarted run:
```bash
kubectl rollout restart deploy/dapr-sentry -n <DAPR_NAMESPACE>
kubectl rollout restart deploy/dapr-operator -n <DAPR_NAMESPACE>
kubectl rollout restart statefulsets/dapr-placement-server -n <DAPR_NAMESPACE>
```
Next, you must restart all Dapr-enabled pods.
The recommended way to do this is to perform a rollout restart of your deployment:
@ -342,9 +394,15 @@ Place `ca.crt`, `issuer.crt` and `issuer.key` in a desired path (`$HOME/.dapr/ce
If the Root or Issuer certs are about to expire, you can update them and restart the required system services.
First, issue new certificates using the step above in [Bringing your own certificates](#bringing-your-own-certificates).
To have Dapr generate new certificates, delete the existing certificates at `$HOME/.dapr/certs` and restart the sentry service to generate new certificates.
```bash
./sentry --issuer-credentials $HOME/.dapr/certs --trust-domain cluster.local --config=./config.yaml
```
To replace with your own certificates, first generate new certificates using the step above in [Bringing your own certificates](#bringing-your-own-certificates).
Copy `ca.crt`, `issuer.crt` and `issuer.key` to the filesystem path of every configured system service, and restart the process or container.
By default, system services will look for the credentials in `/var/run/dapr/credentials`.
By default, system services will look for the credentials in `/var/run/dapr/credentials`. The examples above use `$HOME/.dapr/certs` as a custom location.
*Note:If you signed the cert root with a different private key, restart the Dapr instances.*
*Note: If you signed the cert root with a different private key, restart the Dapr instances.*