mirror of https://github.com/istio/istio.io.git
Cleanup health checking doc (#7961)
* Cleanup health checking doc * regen * lint * tweak * remove DR * regen
This commit is contained in:
Frank Budinsky
GitHub
parent
6024905e61
commit
2ed05b13ce
|
|
@ -15,76 +15,57 @@ test: yes
|
|||
---
|
||||
|
||||
[Kubernetes liveness and readiness probes](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/)
|
||||
offer three different options:
|
||||
describes several ways to configure liveness and readiness probes including:
|
||||
|
||||
1. Command
|
||||
1. TCP request
|
||||
1. HTTP request
|
||||
|
||||
This guide shows how to use these approaches in Istio with mutual TLS enabled.
|
||||
The command approach works with Istio regardless of whether or not mutual TLS is enabled.
|
||||
|
||||
Command and TCP type probes work with Istio regardless of whether or not mutual TLS is enabled. The HTTP request approach requires different Istio configuration with
|
||||
mutual TLS enabled.
|
||||
The HTTP request approach, on the other hand, requires special Istio configuration when mutual TLS is enabled.
|
||||
This is because the health check requests to the `liveness-http` service are sent by Kubelet,
|
||||
which does not have an Istio issued certificate. Therefore when mutual TLS is enabled,
|
||||
the health check requests will fail.
|
||||
|
||||
## Before you begin
|
||||
Istio solves this problem by rewriting the application `PodSpec` readiness/liveness probe,
|
||||
so that the probe request is sent to [Pilot agent](/docs/reference/commands/pilot-agent/).
|
||||
Pilot agent then redirects the request to the application, strips the response body, only returning the response code.
|
||||
|
||||
* Understand [Kubernetes liveness and readiness probes](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/), Istio
|
||||
[authentication policy](/docs/concepts/security/#authentication-policies) and [mutual TLS authentication](/docs/concepts/security/#mutual-tls-authentication) concepts.
|
||||
This feature is enabled by default in all built-in Istio [configuration profiles](/docs/setup/additional-setup/config-profiles/)
|
||||
but can be disabled as described below.
|
||||
|
||||
* Have a Kubernetes cluster with Istio installed, without global mutual TLS enabled.
|
||||
## Liveness and readiness probes using the command approach
|
||||
|
||||
## Liveness and readiness probes with command option
|
||||
|
||||
First, you need to configure health checking with mutual TLS enabled.
|
||||
|
||||
To enable mutual TLS for services, you must configure an authentication policy and a destination rule.
|
||||
Follow these steps to complete the configuration:
|
||||
|
||||
Run the following command to create namespace:
|
||||
Istio provides a [liveness sample]({{< github_file >}}/samples/health-check/liveness-command.yaml) that
|
||||
implements this approach. To demonstrate it working with mutual TLS enabled,
|
||||
first create a namespace for the example:
|
||||
|
||||
{{< text bash >}}
|
||||
$ kubectl create ns istio-io-health
|
||||
{{< /text >}}
|
||||
|
||||
1. To configure the authentication policy, run:
|
||||
To configure strict mutual TLS, run:
|
||||
|
||||
{{< text bash >}}
|
||||
$ kubectl apply -f - <<EOF
|
||||
apiVersion: "security.istio.io/v1beta1"
|
||||
kind: "PeerAuthentication"
|
||||
metadata:
|
||||
name: "default"
|
||||
namespace: "istio-io-health"
|
||||
spec:
|
||||
mtls:
|
||||
mode: STRICT
|
||||
EOF
|
||||
{{< /text >}}
|
||||
{{< text bash >}}
|
||||
$ kubectl apply -f - <<EOF
|
||||
apiVersion: "security.istio.io/v1beta1"
|
||||
kind: "PeerAuthentication"
|
||||
metadata:
|
||||
name: "default"
|
||||
namespace: "istio-io-health"
|
||||
spec:
|
||||
mtls:
|
||||
mode: STRICT
|
||||
EOF
|
||||
{{< /text >}}
|
||||
|
||||
1. To configure the destination rule, run:
|
||||
|
||||
{{< text bash >}}
|
||||
$ kubectl apply -f - <<EOF
|
||||
apiVersion: "networking.istio.io/v1alpha3"
|
||||
kind: "DestinationRule"
|
||||
metadata:
|
||||
name: "default"
|
||||
namespace: "istio-io-health"
|
||||
spec:
|
||||
host: "*.default.svc.cluster.local"
|
||||
trafficPolicy:
|
||||
tls:
|
||||
mode: ISTIO_MUTUAL
|
||||
EOF
|
||||
{{< /text >}}
|
||||
|
||||
Run the following command to deploy the service:
|
||||
Next, run the following command to deploy the sample service:
|
||||
|
||||
{{< text bash >}}
|
||||
$ kubectl -n istio-io-health apply -f <(istioctl kube-inject -f @samples/health-check/liveness-command.yaml@)
|
||||
{{< /text >}}
|
||||
|
||||
Repeat the check status command to verify that the liveness probes work:
|
||||
To confirm that the liveness probes are working, check the status of the sample pod to verify that it is running.
|
||||
|
||||
{{< text bash >}}
|
||||
$ kubectl -n istio-io-health get pod
|
||||
|
|
@ -92,23 +73,17 @@ NAME READY STATUS RESTARTS AGE
|
|||
liveness-6857c8775f-zdv9r 2/2 Running 0 4m
|
||||
{{< /text >}}
|
||||
|
||||
## Liveness and readiness probes with HTTP request option
|
||||
## Liveness and readiness probes using the HTTP request approach
|
||||
|
||||
This section shows how to configure health checking with the HTTP request option when mutual TLS is enabled.
|
||||
As stated previously, Istio uses probe rewrite to implement HTTP probes by default. You can disable this
|
||||
feature either for specific pods, or globally.
|
||||
|
||||
Kubernetes HTTP health check request is sent from Kubelet, which does not have Istio issued certificate to the `liveness-http` service. So when mutual TLS is enabled, the health check request will fail. We have the probe rewrite option to solve the problem.
|
||||
### Disable the HTTP probe rewrite for a pod
|
||||
|
||||
### Probe rewrite
|
||||
|
||||
This approach rewrites the application `PodSpec` readiness/liveness probe, such that the probe request will be sent to
|
||||
[Pilot agent](/docs/reference/commands/pilot-agent/). Pilot agent then redirects the
|
||||
request to application, and strips the response body only returning the response code.
|
||||
|
||||
This feature is enabled by default when installing with any of our [profiles](/docs/setup/additional-setup/config-profiles/).
|
||||
|
||||
### Disable the probe rewrite option for your pod
|
||||
|
||||
You can [annotate the pod](/docs/reference/config/annotations/) with `sidecar.istio.io/rewriteAppHTTPProbers: "false"` to disable the probe rewrite option. Make sure you add the annotation to the [pod resource](https://kubernetes.io/docs/concepts/workloads/pods/pod-overview/) because it will be ignored anywhere else (for example, on an enclosing deployment resource).
|
||||
You can [annotate the pod](/docs/reference/config/annotations/) with `sidecar.istio.io/rewriteAppHTTPProbers: "false"`
|
||||
to disable the probe rewrite option. Make sure you add the annotation to the
|
||||
[pod resource](https://kubernetes.io/docs/concepts/workloads/pods/pod-overview/) because it will be ignored
|
||||
anywhere else (for example, on an enclosing deployment resource).
|
||||
|
||||
{{< text yaml >}}
|
||||
apiVersion: apps/v1
|
||||
|
|
@ -141,20 +116,22 @@ spec:
|
|||
periodSeconds: 5
|
||||
{{< /text >}}
|
||||
|
||||
This approach allows you to disable the health check probe rewrite gradually on each deployment without reinstalling Istio.
|
||||
This approach allows you to disable the health check probe rewrite gradually on individual deployments,
|
||||
without reinstalling Istio.
|
||||
|
||||
### Disable the probe rewrite option globally
|
||||
### Disable the probe rewrite globally
|
||||
|
||||
[Install Istio](/docs/setup/install/istioctl/) with `--set values.sidecarInjectorWebhook.rewriteAppHTTPProbe=false` to disable the probe rewrite globally. **Alternatively**, update the configuration map of Istio sidecar injection:
|
||||
[Install Istio](/docs/setup/install/istioctl/) using `--set values.sidecarInjectorWebhook.rewriteAppHTTPProbe=false`
|
||||
to disable the probe rewrite globally. **Alternatively**, update the configuration map for the Istio sidecar injector:
|
||||
|
||||
{{< text bash >}}
|
||||
$ kubectl get cm istio-sidecar-injector -n istio-system -o yaml | sed -e 's/"rewriteAppHTTPProbe": true/"rewriteAppHTTPProbe": false/' | kubectl apply -f -
|
||||
{{< /text >}}
|
||||
|
||||
### Cleanup
|
||||
## Cleanup
|
||||
|
||||
Remove the mutual TLS policy and corresponding destination rule added in the steps above:
|
||||
Remove the namespace used for the examples:
|
||||
|
||||
{{< text bash >}}
|
||||
$ kubectl delete ns istio-io-health istio-same-port istio-sep-port
|
||||
$ kubectl delete ns istio-io-health
|
||||
{{< /text >}}
|
||||
|
|
|
|||
|
|
@ -20,11 +20,11 @@
|
|||
# docs/ops/configuration/mesh/app-health-check/index.md
|
||||
####################################################################################################
|
||||
|
||||
snip_liveness_and_readiness_probes_with_command_option_1() {
|
||||
snip_liveness_and_readiness_probes_using_the_command_approach_1() {
|
||||
kubectl create ns istio-io-health
|
||||
}
|
||||
|
||||
snip_liveness_and_readiness_probes_with_command_option_2() {
|
||||
snip_liveness_and_readiness_probes_using_the_command_approach_2() {
|
||||
kubectl apply -f - <<EOF
|
||||
apiVersion: "security.istio.io/v1beta1"
|
||||
kind: "PeerAuthentication"
|
||||
|
|
@ -37,35 +37,20 @@ spec:
|
|||
EOF
|
||||
}
|
||||
|
||||
snip_liveness_and_readiness_probes_with_command_option_3() {
|
||||
kubectl apply -f - <<EOF
|
||||
apiVersion: "networking.istio.io/v1alpha3"
|
||||
kind: "DestinationRule"
|
||||
metadata:
|
||||
name: "default"
|
||||
namespace: "istio-io-health"
|
||||
spec:
|
||||
host: "*.default.svc.cluster.local"
|
||||
trafficPolicy:
|
||||
tls:
|
||||
mode: ISTIO_MUTUAL
|
||||
EOF
|
||||
}
|
||||
|
||||
snip_liveness_and_readiness_probes_with_command_option_4() {
|
||||
snip_liveness_and_readiness_probes_using_the_command_approach_3() {
|
||||
kubectl -n istio-io-health apply -f <(istioctl kube-inject -f samples/health-check/liveness-command.yaml)
|
||||
}
|
||||
|
||||
snip_liveness_and_readiness_probes_with_command_option_5() {
|
||||
snip_liveness_and_readiness_probes_using_the_command_approach_4() {
|
||||
kubectl -n istio-io-health get pod
|
||||
}
|
||||
|
||||
! read -r -d '' snip_liveness_and_readiness_probes_with_command_option_5_out <<\ENDSNIP
|
||||
! read -r -d '' snip_liveness_and_readiness_probes_using_the_command_approach_4_out <<\ENDSNIP
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
liveness-6857c8775f-zdv9r 2/2 Running 0 4m
|
||||
ENDSNIP
|
||||
|
||||
! read -r -d '' snip_disable_the_probe_rewrite_option_for_your_pod_1 <<\ENDSNIP
|
||||
! read -r -d '' snip_disable_the_http_probe_rewrite_for_a_pod_1 <<\ENDSNIP
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
|
|
@ -96,10 +81,10 @@ spec:
|
|||
periodSeconds: 5
|
||||
ENDSNIP
|
||||
|
||||
snip_disable_the_probe_rewrite_option_globally_1() {
|
||||
snip_disable_the_probe_rewrite_globally_1() {
|
||||
kubectl get cm istio-sidecar-injector -n istio-system -o yaml | sed -e 's/"rewriteAppHTTPProbe": true/"rewriteAppHTTPProbe": false/' | kubectl apply -f -
|
||||
}
|
||||
|
||||
snip_cleanup_1() {
|
||||
kubectl delete ns istio-io-health istio-same-port istio-sep-port
|
||||
kubectl delete ns istio-io-health
|
||||
}
|
||||
|
|
|
|||
|
|
@ -21,26 +21,24 @@ set -o pipefail
|
|||
|
||||
# @setup profile=default
|
||||
|
||||
snip_liveness_and_readiness_probes_with_command_option_1
|
||||
snip_liveness_and_readiness_probes_using_the_command_approach_1
|
||||
|
||||
snip_liveness_and_readiness_probes_with_command_option_2
|
||||
|
||||
snip_liveness_and_readiness_probes_with_command_option_3
|
||||
snip_liveness_and_readiness_probes_using_the_command_approach_2
|
||||
|
||||
_wait_for_istio peerauthentication istio-io-health default
|
||||
_wait_for_istio destinationrule istio-io-health default
|
||||
|
||||
snip_liveness_and_readiness_probes_with_command_option_4
|
||||
snip_liveness_and_readiness_probes_using_the_command_approach_3
|
||||
|
||||
_wait_for_deployment istio-io-health liveness
|
||||
|
||||
_verify_like snip_liveness_and_readiness_probes_with_command_option_5 "$snip_liveness_and_readiness_probes_with_command_option_5_out"
|
||||
_verify_like snip_liveness_and_readiness_probes_using_the_command_approach_4 "$snip_liveness_and_readiness_probes_using_the_command_approach_4_out"
|
||||
|
||||
kubectl -n istio-io-health delete -f samples/health-check/liveness-command.yaml
|
||||
|
||||
snip_disable_the_probe_rewrite_option_globally_1
|
||||
snip_disable_the_probe_rewrite_globally_1
|
||||
|
||||
# TODO test annotation approach and verify both disable approaches work.
|
||||
|
||||
# @cleanup
|
||||
set +e # ignore cleanup errors
|
||||
snip_cleanup_1
|
||||
kubectl delete ns health-annotate
|
||||
|
|
|
|||
Loading…
Reference in New Issue