Add tips for easier external control plane setup (#11609)

* Add tips for easier  external control plane setup

* fix func name

* fix lint

* fix indent

* fix test

* rename yaml file

* remove nested text in tips

* tweak indent

* regen
This commit is contained in:
Frank Budinsky 2022-07-21 08:16:43 -04:00 committed by GitHub
parent f889f18538
commit cef3426a49
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
3 changed files with 183 additions and 69 deletions

View File

@ -138,13 +138,40 @@ and installing the sidecar injector webhook configuration on the remote cluster
although in this example you will only deploy a single external istiod in the `external-istiod` namespace.
{{< /tip >}}
1. Configure your environment to expose the Istio ingress gateway service using a public hostname with TLS. Set the `EXTERNAL_ISTIOD_ADDR` environment variable to the hostname and `SSL_SECRET_NAME` environment variable to the secret that holds the TLS certs:
1. Configure your environment to expose the Istio ingress gateway service using a public hostname with TLS.
Set the `EXTERNAL_ISTIOD_ADDR` environment variable to the hostname and `SSL_SECRET_NAME` environment variable to the secret that holds the TLS certs:
{{< text syntax=bash snip_id=none >}}
$ export EXTERNAL_ISTIOD_ADDR=<your external istiod host>
$ export SSL_SECRET_NAME=<your external istiod secret>
{{< /text >}}
These instructions assume that you are exposing the external cluster's gateway using a hostname with properly signed DNS certs
as this is the recommended approach in a production environment.
Refer to the [secure ingress task](/docs/tasks/traffic-management/ingress/secure-ingress/#configure-a-tls-ingress-gateway-for-a-single-host)
for more information on exposing a secure gateway.
Your environment variables should look something like this:
{{< text bash >}}
$ echo "$EXTERNAL_ISTIOD_ADDR" "$SSL_SECRET_NAME"
myhost.example.com myhost-example-credential
{{< /text >}}
{{< tip >}}
If you don't have a DNS hostname but want to experiment with an external control plane in a test environment,
you can access the gateway using its external load balancer IP address:
{{< text bash >}}
$ export EXTERNAL_ISTIOD_ADDR=$(kubectl -n istio-system --context="${CTX_EXTERNAL_CLUSTER}" get svc istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
$ export SSL_SECRET_NAME=NONE
{{< /text >}}
Doing this will also require a few other changes in the configuration. Make sure to follow all of the related steps
in the instructions below.
{{< /tip >}}
#### Set up the remote config cluster
1. Use the `external` profile to configure the remote cluster's Istio installation. This installs an injection
@ -167,13 +194,29 @@ and installing the sidecar injector webhook configuration on the remote cluster
pilot:
configMap: true
istiodRemote:
injectionURL: https://${EXTERNAL_ISTIOD_ADDR}:15017/inject/:ENV:cluster=${REMOTE_CLUSTER_NAME}:ENV:net=network1
injectionURL: https://${EXTERNAL_ISTIOD_ADDR}:15017/inject/cluster/${REMOTE_CLUSTER_NAME}/net/network1
base:
validationURL: https://${EXTERNAL_ISTIOD_ADDR}:15017/validate
EOF
{{< /text >}}
Then, install the configuration on the remote cluster:
1. If you are using an IP address for the `EXTERNAL_ISTIOD_ADDR`, instead of a proper DNS hostname,
modify the configuration to specify the discovery address and paths, instead of URLs:
{{< warning >}}
This is not recommended in a production environment.
{{< /warning >}}
{{< text bash >}}
$ sed -i'.bk' \
-e "s|injectionURL: https://${EXTERNAL_ISTIOD_ADDR}:15017|injectionPath: |" \
-e "/istioNamespace:/a\\
remotePilotAddress: ${EXTERNAL_ISTIOD_ADDR}" \
-e '/base/,+1d' \
remote-config-cluster.yaml; rm remote-config-cluster.yaml.bk
{{< /text >}}
1. Install the configuration on the remote cluster:
{{< text bash >}}
$ kubectl create namespace external-istiod --context="${CTX_REMOTE_CLUSTER}"
@ -281,7 +324,22 @@ and installing the sidecar injector webhook configuration on the remote cluster
EOF
{{< /text >}}
Then, apply the Istio configuration on the external cluster:
1. If you are using an IP address for the `EXTERNAL_ISTIOD_ADDR`, instead of a proper DNS hostname,
delete the proxy metadata and webhook config environment variables from the configuration:
{{< warning >}}
This is not recommended in a production environment.
{{< /warning >}}
{{< text bash >}}
$ sed -i'.bk' \
-e '/proxyMetadata:/,+2d' \
-e '/INJECTION_WEBHOOK_CONFIG_NAME/,+1d' \
-e '/VALIDATION_WEBHOOK_CONFIG_NAME/,+1d' \
external-istiod.yaml ; rm external-istiod.yaml.bk
{{< /text >}}
1. Apply the Istio configuration on the external cluster:
{{< text bash >}}
$ istioctl manifest generate -f external-istiod.yaml | kubectl apply --context="${CTX_EXTERNAL_CLUSTER}" -f -
@ -377,7 +435,25 @@ and installing the sidecar injector webhook configuration on the remote cluster
EOF
{{< /text >}}
Then, apply the configuration on the external cluster:
1. If you are using an IP address for the `EXTERNAL_ISTIOD_ADDR`, instead of a proper DNS hostname,
modify the configuration.
Delete the `DestinationRule`, don't terminate TLS in the `Gateway`, and use TLS routing in the `VirtualService`:
{{< warning >}}
This is not recommended in a production environment.
{{< /warning >}}
{{< text bash >}}
$ sed -i'.bk' \
-e '55,$d' \
-e 's/mode: SIMPLE/mode: PASSTHROUGH/' -e '/credentialName:/d' -e "s/${EXTERNAL_ISTIOD_ADDR}/\"*\"/" \
-e 's/http:/tls:/' -e 's/https/tls/' -e '/route:/i\
sniHosts:\
- "*"' \
external-istiod-gw.yaml; rm external-istiod-gw.yaml.bk
{{< /text >}}
1. Apply the configuration on the external cluster:
{{< text bash >}}
$ kubectl apply -f external-istiod-gw.yaml --context="${CTX_EXTERNAL_CLUSTER}"
@ -560,8 +636,8 @@ $ export SECOND_CLUSTER_NAME=<your second remote cluster name>
1. Create the remote Istio install configuration, which installs the injection webhook that uses the
external control plane's injector, instead of a locally deployed one:
{{< text syntax=bash snip_id=get_second_config_cluster_iop >}}
$ cat <<EOF > second-config-cluster.yaml
{{< text syntax=bash snip_id=get_second_remote_cluster_iop >}}
$ cat <<EOF > second-remote-cluster.yaml
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
metadata:
@ -572,10 +648,25 @@ $ export SECOND_CLUSTER_NAME=<your second remote cluster name>
global:
istioNamespace: external-istiod
istiodRemote:
injectionURL: https://${EXTERNAL_ISTIOD_ADDR}:15017/inject/:ENV:cluster=${SECOND_CLUSTER_NAME}:ENV:net=network2
injectionURL: https://${EXTERNAL_ISTIOD_ADDR}:15017/inject/cluster/${SECOND_CLUSTER_NAME}/net/network2
EOF
{{< /text >}}
1. If you are using an IP address for the `EXTERNAL_ISTIOD_ADDR`, instead of a proper DNS hostname,
modify the configuration to specify the discovery address and path, instead of an injection URL:
{{< warning >}}
This is not recommended in a production environment.
{{< /warning >}}
{{< text bash >}}
$ sed -i'.bk' \
-e "s|injectionURL: https://${EXTERNAL_ISTIOD_ADDR}:15017|injectionPath: |" \
-e "/istioNamespace:/a\\
remotePilotAddress: ${EXTERNAL_ISTIOD_ADDR}" \
second-remote-cluster.yaml; rm second-remote-cluster.yaml.bk
{{< /text >}}
1. Create and annotate the system namespace on the remote cluster:
{{< text bash >}}
@ -590,7 +681,7 @@ $ export SECOND_CLUSTER_NAME=<your second remote cluster name>
1. Install the configuration on the remote cluster:
{{< text bash >}}
$ istioctl manifest generate -f second-config-cluster.yaml | kubectl apply --context="${CTX_SECOND_CLUSTER}" -f -
$ istioctl manifest generate -f second-remote-cluster.yaml | kubectl apply --context="${CTX_SECOND_CLUSTER}" -f -
{{< /text >}}
1. Confirm that the remote cluster's webhook configuration has been installed:

View File

@ -60,6 +60,19 @@ istio-ingressgateway-9d4c7f5c7-7qpzz 1/1 Running 0 29s
istiod-68488cd797-mq8dn 1/1 Running 0 38s
ENDSNIP
snip_set_up_a_gateway_in_the_external_cluster_5() {
echo "$EXTERNAL_ISTIOD_ADDR" "$SSL_SECRET_NAME"
}
! read -r -d '' snip_set_up_a_gateway_in_the_external_cluster_5_out <<\ENDSNIP
myhost.example.com myhost-example-credential
ENDSNIP
snip_set_up_a_gateway_in_the_external_cluster_6() {
export EXTERNAL_ISTIOD_ADDR=$(kubectl -n istio-system --context="${CTX_EXTERNAL_CLUSTER}" get svc istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
export SSL_SECRET_NAME=NONE
}
snip_get_remote_config_cluster_iop() {
cat <<EOF > remote-config-cluster.yaml
apiVersion: install.istio.io/v1alpha1
@ -75,22 +88,31 @@ spec:
pilot:
configMap: true
istiodRemote:
injectionURL: https://${EXTERNAL_ISTIOD_ADDR}:15017/inject/:ENV:cluster=${REMOTE_CLUSTER_NAME}:ENV:net=network1
injectionURL: https://${EXTERNAL_ISTIOD_ADDR}:15017/inject/cluster/${REMOTE_CLUSTER_NAME}/net/network1
base:
validationURL: https://${EXTERNAL_ISTIOD_ADDR}:15017/validate
EOF
}
snip_set_up_the_remote_config_cluster_2() {
sed -i'.bk' \
-e "s|injectionURL: https://${EXTERNAL_ISTIOD_ADDR}:15017|injectionPath: |" \
-e "/istioNamespace:/a\\
remotePilotAddress: ${EXTERNAL_ISTIOD_ADDR}" \
-e '/base/,+1d' \
remote-config-cluster.yaml; rm remote-config-cluster.yaml.bk
}
snip_set_up_the_remote_config_cluster_3() {
kubectl create namespace external-istiod --context="${CTX_REMOTE_CLUSTER}"
istioctl manifest generate -f remote-config-cluster.yaml | kubectl apply --context="${CTX_REMOTE_CLUSTER}" -f -
}
snip_set_up_the_remote_config_cluster_3() {
snip_set_up_the_remote_config_cluster_4() {
kubectl get mutatingwebhookconfiguration --context="${CTX_REMOTE_CLUSTER}"
}
! read -r -d '' snip_set_up_the_remote_config_cluster_3_out <<\ENDSNIP
! read -r -d '' snip_set_up_the_remote_config_cluster_4_out <<\ENDSNIP
NAME WEBHOOKS AGE
istio-sidecar-injector-external-istiod 4 6m24s
ENDSNIP
@ -175,14 +197,22 @@ EOF
}
snip_set_up_the_control_plane_in_the_external_cluster_4() {
istioctl manifest generate -f external-istiod.yaml | kubectl apply --context="${CTX_EXTERNAL_CLUSTER}" -f -
sed -i'.bk' \
-e '/proxyMetadata:/,+2d' \
-e '/INJECTION_WEBHOOK_CONFIG_NAME/,+1d' \
-e '/VALIDATION_WEBHOOK_CONFIG_NAME/,+1d' \
external-istiod.yaml ; rm external-istiod.yaml.bk
}
snip_set_up_the_control_plane_in_the_external_cluster_5() {
istioctl manifest generate -f external-istiod.yaml | kubectl apply --context="${CTX_EXTERNAL_CLUSTER}" -f -
}
snip_set_up_the_control_plane_in_the_external_cluster_6() {
kubectl get po -n external-istiod --context="${CTX_EXTERNAL_CLUSTER}"
}
! read -r -d '' snip_set_up_the_control_plane_in_the_external_cluster_5_out <<\ENDSNIP
! read -r -d '' snip_set_up_the_control_plane_in_the_external_cluster_6_out <<\ENDSNIP
NAME READY STATUS RESTARTS AGE
istiod-779bd6fdcf-bd6rg 1/1 Running 0 70s
ENDSNIP
@ -266,7 +296,17 @@ spec:
EOF
}
snip_set_up_the_control_plane_in_the_external_cluster_7() {
snip_set_up_the_control_plane_in_the_external_cluster_8() {
sed -i'.bk' \
-e '55,$d' \
-e 's/mode: SIMPLE/mode: PASSTHROUGH/' -e '/credentialName:/d' -e "s/${EXTERNAL_ISTIOD_ADDR}/\"*\"/" \
-e 's/http:/tls:/' -e 's/https/tls/' -e '/route:/i\
sniHosts:\
- "*"' \
external-istiod-gw.yaml; rm external-istiod-gw.yaml.bk
}
snip_set_up_the_control_plane_in_the_external_cluster_9() {
kubectl apply -f external-istiod-gw.yaml --context="${CTX_EXTERNAL_CLUSTER}"
}
@ -374,8 +414,8 @@ curl -s "http://${GATEWAY_URL}/hello"
Hello version: v1, instance: helloworld-v1-776f57d5f6-s7zfc
ENDSNIP
snip_get_second_config_cluster_iop() {
cat <<EOF > second-config-cluster.yaml
snip_get_second_remote_cluster_iop() {
cat <<EOF > second-remote-cluster.yaml
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
metadata:
@ -386,29 +426,37 @@ spec:
global:
istioNamespace: external-istiod
istiodRemote:
injectionURL: https://${EXTERNAL_ISTIOD_ADDR}:15017/inject/:ENV:cluster=${SECOND_CLUSTER_NAME}:ENV:net=network2
injectionURL: https://${EXTERNAL_ISTIOD_ADDR}:15017/inject/cluster/${SECOND_CLUSTER_NAME}/net/network2
EOF
}
snip_register_the_new_cluster_2() {
sed -i'.bk' \
-e "s|injectionURL: https://${EXTERNAL_ISTIOD_ADDR}:15017|injectionPath: |" \
-e "/istioNamespace:/a\\
remotePilotAddress: ${EXTERNAL_ISTIOD_ADDR}" \
second-remote-cluster.yaml; rm second-remote-cluster.yaml.bk
}
snip_register_the_new_cluster_3() {
kubectl create namespace external-istiod --context="${CTX_SECOND_CLUSTER}"
kubectl annotate namespace external-istiod "topology.istio.io/controlPlaneClusters=${REMOTE_CLUSTER_NAME}" --context="${CTX_SECOND_CLUSTER}"
}
snip_register_the_new_cluster_3() {
istioctl manifest generate -f second-config-cluster.yaml | kubectl apply --context="${CTX_SECOND_CLUSTER}" -f -
snip_register_the_new_cluster_4() {
istioctl manifest generate -f second-remote-cluster.yaml | kubectl apply --context="${CTX_SECOND_CLUSTER}" -f -
}
snip_register_the_new_cluster_4() {
snip_register_the_new_cluster_5() {
kubectl get mutatingwebhookconfiguration --context="${CTX_SECOND_CLUSTER}"
}
! read -r -d '' snip_register_the_new_cluster_4_out <<\ENDSNIP
! read -r -d '' snip_register_the_new_cluster_5_out <<\ENDSNIP
NAME WEBHOOKS AGE
istio-sidecar-injector-external-istiod 4 4m13s
ENDSNIP
snip_register_the_new_cluster_5() {
snip_register_the_new_cluster_6() {
istioctl x create-remote-secret \
--context="${CTX_SECOND_CLUSTER}" \
--name="${SECOND_CLUSTER_NAME}" \

View File

@ -26,34 +26,6 @@ kubectl_get_egress_gateway_for_remote_cluster() {
echo "$response"
}
# Override some snip functions to configure the istiod gateway using TLS passthrough in the test environemnt.
snip_get_external_istiod_iop_modified() {
snip_get_external_istiod_iop
# Update config file: delete CA certificates and meshID, and update pilot vars
# TODO(https://github.com/istio/istio/issues/31690) remove 'env' replace
sed -i \
-e '/proxyMetadata:/,+2d' \
-e '/INJECTION_WEBHOOK_CONFIG_NAME/,+1d' \
-e "/VALIDATION_WEBHOOK_CONFIG_NAME/,+1d" \
external-istiod.yaml
}
snip_get_external_istiod_gateway_config_modified() {
TMP="$EXTERNAL_ISTIOD_ADDR"
EXTERNAL_ISTIOD_ADDR='"*"'
snip_get_external_istiod_gateway_config
# Update config file: delete the DestinationRule, don't terminate TLS in the Gateway, and use TLS routing in the VirtualService
sed -i \
-e '55,$d' \
-e 's/mode: SIMPLE/mode: PASSTHROUGH/' -e '/credentialName:/d' \
-e 's/http:/tls:/' -e 's/https/tls/' -e "/route:/i\ sniHosts:\n - ${EXTERNAL_ISTIOD_ADDR}" \
external-istiod-gw.yaml
EXTERNAL_ISTIOD_ADDR="$TMP"
}
# Set the CTX_EXTERNAL_CLUSTER, CTX_REMOTE_CLUSTER, and REMOTE_CLUSTER_NAME env variables.
_set_kube_vars # helper function to initialize KUBECONFIG_FILES and KUBE_CONTEXTS
@ -68,34 +40,35 @@ echo y | snip_set_up_a_gateway_in_the_external_cluster_2
_verify_like snip_set_up_a_gateway_in_the_external_cluster_3 "$snip_set_up_a_gateway_in_the_external_cluster_3_out"
export SSL_SECRET_NAME="UNUSED"
export EXTERNAL_ISTIOD_ADDR=$(kubectl \
--context="${CTX_EXTERNAL_CLUSTER}" \
-n istio-system get svc istio-ingressgateway \
-o jsonpath='{.status.loadBalancer.ingress[0].ip}')
snip_set_up_a_gateway_in_the_external_cluster_6
# Set up the remote cluster.
snip_get_remote_config_cluster_iop
snip_set_up_the_remote_config_cluster_2
#set +e #ignore failures here
echo y | snip_set_up_the_remote_config_cluster_2
echo y | snip_set_up_the_remote_config_cluster_3
#set -e
_verify_like snip_set_up_the_remote_config_cluster_3 "$snip_set_up_the_remote_config_cluster_3_out"
_verify_like snip_set_up_the_remote_config_cluster_4 "$snip_set_up_the_remote_config_cluster_4_out"
# Install istiod on the external cluster.
snip_set_up_the_control_plane_in_the_external_cluster_1
snip_set_up_the_control_plane_in_the_external_cluster_2
snip_get_external_istiod_iop_modified
echo y | istioctl manifest generate -f external-istiod.yaml --set values.pilot.env.ISTIOD_CUSTOM_HOST="${EXTERNAL_ISTIOD_ADDR}" | kubectl apply --context="${CTX_EXTERNAL_CLUSTER}" -f -
snip_get_external_istiod_iop
snip_set_up_the_control_plane_in_the_external_cluster_4
_verify_like snip_set_up_the_control_plane_in_the_external_cluster_5 "$snip_set_up_the_control_plane_in_the_external_cluster_5_out"
echo y | snip_set_up_the_control_plane_in_the_external_cluster_5
snip_get_external_istiod_gateway_config_modified
snip_set_up_the_control_plane_in_the_external_cluster_7
_verify_like snip_set_up_the_control_plane_in_the_external_cluster_6 "$snip_set_up_the_control_plane_in_the_external_cluster_6_out"
snip_get_external_istiod_gateway_config
snip_set_up_the_control_plane_in_the_external_cluster_8
snip_set_up_the_control_plane_in_the_external_cluster_9
# Validate the installation.
@ -130,15 +103,17 @@ _verify_contains snip_test_the_ingress_gateway_4 "Hello version: v1"
export CTX_SECOND_CLUSTER="${KUBE_CONTEXTS[1]}"
export SECOND_CLUSTER_NAME="${CTX_SECOND_CLUSTER}"
snip_get_second_config_cluster_iop
snip_get_second_remote_cluster_iop
snip_register_the_new_cluster_2
echo y | snip_register_the_new_cluster_3
snip_register_the_new_cluster_3
echo y | snip_register_the_new_cluster_4
# Confirm remote clusters webhook configuration has been installed
_verify_like snip_register_the_new_cluster_4 "$snip_register_the_new_cluster_4_out"
_verify_like snip_register_the_new_cluster_5 "$snip_register_the_new_cluster_5_out"
# Create a secret with credentials to allow the control plane to access the endpoints on the second remote cluster and install it
snip_register_the_new_cluster_5
snip_register_the_new_cluster_6
# Setup east-west gateways
snip_setup_eastwest_gateways_1
@ -172,7 +147,7 @@ kubectl delete ns sample --context="${CTX_SECOND_CLUSTER}"
kubectl delete -f external-istiod-gw.yaml --context="${CTX_EXTERNAL_CLUSTER}"
istioctl manifest generate -f remote-config-cluster.yaml | kubectl delete --context="${CTX_REMOTE_CLUSTER}" -f -
istioctl manifest generate -f second-config-cluster.yaml | kubectl delete --context="${CTX_SECOND_CLUSTER}" -f -
istioctl manifest generate -f second-remote-cluster.yaml | kubectl delete --context="${CTX_SECOND_CLUSTER}" -f -
istioctl manifest generate -f eastwest-gateway-1.yaml | kubectl delete --context="${CTX_REMOTE_CLUSTER}" -f -
istioctl manifest generate -f eastwest-gateway-2.yaml | kubectl delete --context="${CTX_SECOND_CLUSTER}" -f -
istioctl manifest generate -f external-istiod.yaml | kubectl delete --context="${CTX_EXTERNAL_CLUSTER}" -f -
@ -189,4 +164,4 @@ kubectl delete ns istio-system external-istiod --context="${CTX_EXTERNAL_CLUSTER
kubectl delete ns external-istiod --context="${CTX_REMOTE_CLUSTER}"
kubectl delete ns external-istiod --context="${CTX_SECOND_CLUSTER}"
rm external-istiod-gw.yaml remote-config-cluster.yaml external-istiod.yaml controlplane-gateway.yaml eastwest-gateway-1.yaml eastwest-gateway-2.yaml second-config-cluster.yaml
rm external-istiod-gw.yaml remote-config-cluster.yaml external-istiod.yaml controlplane-gateway.yaml eastwest-gateway-1.yaml eastwest-gateway-2.yaml second-remote-cluster.yaml