diff --git a/content/en/docs/setup/install/external-controlplane/index.md b/content/en/docs/setup/install/external-controlplane/index.md index c4c66a55a7..039565d461 100644 --- a/content/en/docs/setup/install/external-controlplane/index.md +++ b/content/en/docs/setup/install/external-controlplane/index.md @@ -138,13 +138,40 @@ and installing the sidecar injector webhook configuration on the remote cluster although in this example you will only deploy a single external istiod in the `external-istiod` namespace. {{< /tip >}} -1. Configure your environment to expose the Istio ingress gateway service using a public hostname with TLS. Set the `EXTERNAL_ISTIOD_ADDR` environment variable to the hostname and `SSL_SECRET_NAME` environment variable to the secret that holds the TLS certs: +1. Configure your environment to expose the Istio ingress gateway service using a public hostname with TLS. + + Set the `EXTERNAL_ISTIOD_ADDR` environment variable to the hostname and `SSL_SECRET_NAME` environment variable to the secret that holds the TLS certs: {{< text syntax=bash snip_id=none >}} $ export EXTERNAL_ISTIOD_ADDR= $ export SSL_SECRET_NAME= {{< /text >}} + These instructions assume that you are exposing the external cluster's gateway using a hostname with properly signed DNS certs + as this is the recommended approach in a production environment. + Refer to the [secure ingress task](/docs/tasks/traffic-management/ingress/secure-ingress/#configure-a-tls-ingress-gateway-for-a-single-host) + for more information on exposing a secure gateway. + + Your environment variables should look something like this: + + {{< text bash >}} + $ echo "$EXTERNAL_ISTIOD_ADDR" "$SSL_SECRET_NAME" + myhost.example.com myhost-example-credential + {{< /text >}} + + {{< tip >}} + If you don't have a DNS hostname but want to experiment with an external control plane in a test environment, + you can access the gateway using its external load balancer IP address: + + {{< text bash >}} + $ export EXTERNAL_ISTIOD_ADDR=$(kubectl -n istio-system --context="${CTX_EXTERNAL_CLUSTER}" get svc istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}') + $ export SSL_SECRET_NAME=NONE + {{< /text >}} + + Doing this will also require a few other changes in the configuration. Make sure to follow all of the related steps + in the instructions below. + {{< /tip >}} + #### Set up the remote config cluster 1. Use the `external` profile to configure the remote cluster's Istio installation. This installs an injection @@ -167,13 +194,29 @@ and installing the sidecar injector webhook configuration on the remote cluster pilot: configMap: true istiodRemote: - injectionURL: https://${EXTERNAL_ISTIOD_ADDR}:15017/inject/:ENV:cluster=${REMOTE_CLUSTER_NAME}:ENV:net=network1 + injectionURL: https://${EXTERNAL_ISTIOD_ADDR}:15017/inject/cluster/${REMOTE_CLUSTER_NAME}/net/network1 base: validationURL: https://${EXTERNAL_ISTIOD_ADDR}:15017/validate EOF {{< /text >}} - Then, install the configuration on the remote cluster: +1. If you are using an IP address for the `EXTERNAL_ISTIOD_ADDR`, instead of a proper DNS hostname, + modify the configuration to specify the discovery address and paths, instead of URLs: + + {{< warning >}} + This is not recommended in a production environment. + {{< /warning >}} + + {{< text bash >}} + $ sed -i'.bk' \ + -e "s|injectionURL: https://${EXTERNAL_ISTIOD_ADDR}:15017|injectionPath: |" \ + -e "/istioNamespace:/a\\ + remotePilotAddress: ${EXTERNAL_ISTIOD_ADDR}" \ + -e '/base/,+1d' \ + remote-config-cluster.yaml; rm remote-config-cluster.yaml.bk + {{< /text >}} + +1. Install the configuration on the remote cluster: {{< text bash >}} $ kubectl create namespace external-istiod --context="${CTX_REMOTE_CLUSTER}" @@ -281,7 +324,22 @@ and installing the sidecar injector webhook configuration on the remote cluster EOF {{< /text >}} - Then, apply the Istio configuration on the external cluster: +1. If you are using an IP address for the `EXTERNAL_ISTIOD_ADDR`, instead of a proper DNS hostname, + delete the proxy metadata and webhook config environment variables from the configuration: + + {{< warning >}} + This is not recommended in a production environment. + {{< /warning >}} + + {{< text bash >}} + $ sed -i'.bk' \ + -e '/proxyMetadata:/,+2d' \ + -e '/INJECTION_WEBHOOK_CONFIG_NAME/,+1d' \ + -e '/VALIDATION_WEBHOOK_CONFIG_NAME/,+1d' \ + external-istiod.yaml ; rm external-istiod.yaml.bk + {{< /text >}} + +1. Apply the Istio configuration on the external cluster: {{< text bash >}} $ istioctl manifest generate -f external-istiod.yaml | kubectl apply --context="${CTX_EXTERNAL_CLUSTER}" -f - @@ -377,7 +435,25 @@ and installing the sidecar injector webhook configuration on the remote cluster EOF {{< /text >}} - Then, apply the configuration on the external cluster: +1. If you are using an IP address for the `EXTERNAL_ISTIOD_ADDR`, instead of a proper DNS hostname, + modify the configuration. + Delete the `DestinationRule`, don't terminate TLS in the `Gateway`, and use TLS routing in the `VirtualService`: + + {{< warning >}} + This is not recommended in a production environment. + {{< /warning >}} + + {{< text bash >}} + $ sed -i'.bk' \ + -e '55,$d' \ + -e 's/mode: SIMPLE/mode: PASSTHROUGH/' -e '/credentialName:/d' -e "s/${EXTERNAL_ISTIOD_ADDR}/\"*\"/" \ + -e 's/http:/tls:/' -e 's/https/tls/' -e '/route:/i\ + sniHosts:\ + - "*"' \ + external-istiod-gw.yaml; rm external-istiod-gw.yaml.bk + {{< /text >}} + +1. Apply the configuration on the external cluster: {{< text bash >}} $ kubectl apply -f external-istiod-gw.yaml --context="${CTX_EXTERNAL_CLUSTER}" @@ -560,8 +636,8 @@ $ export SECOND_CLUSTER_NAME= 1. Create the remote Istio install configuration, which installs the injection webhook that uses the external control plane's injector, instead of a locally deployed one: - {{< text syntax=bash snip_id=get_second_config_cluster_iop >}} - $ cat < second-config-cluster.yaml + {{< text syntax=bash snip_id=get_second_remote_cluster_iop >}} + $ cat < second-remote-cluster.yaml apiVersion: install.istio.io/v1alpha1 kind: IstioOperator metadata: @@ -572,10 +648,25 @@ $ export SECOND_CLUSTER_NAME= global: istioNamespace: external-istiod istiodRemote: - injectionURL: https://${EXTERNAL_ISTIOD_ADDR}:15017/inject/:ENV:cluster=${SECOND_CLUSTER_NAME}:ENV:net=network2 + injectionURL: https://${EXTERNAL_ISTIOD_ADDR}:15017/inject/cluster/${SECOND_CLUSTER_NAME}/net/network2 EOF {{< /text >}} +1. If you are using an IP address for the `EXTERNAL_ISTIOD_ADDR`, instead of a proper DNS hostname, + modify the configuration to specify the discovery address and path, instead of an injection URL: + + {{< warning >}} + This is not recommended in a production environment. + {{< /warning >}} + + {{< text bash >}} + $ sed -i'.bk' \ + -e "s|injectionURL: https://${EXTERNAL_ISTIOD_ADDR}:15017|injectionPath: |" \ + -e "/istioNamespace:/a\\ + remotePilotAddress: ${EXTERNAL_ISTIOD_ADDR}" \ + second-remote-cluster.yaml; rm second-remote-cluster.yaml.bk + {{< /text >}} + 1. Create and annotate the system namespace on the remote cluster: {{< text bash >}} @@ -590,7 +681,7 @@ $ export SECOND_CLUSTER_NAME= 1. Install the configuration on the remote cluster: {{< text bash >}} - $ istioctl manifest generate -f second-config-cluster.yaml | kubectl apply --context="${CTX_SECOND_CLUSTER}" -f - + $ istioctl manifest generate -f second-remote-cluster.yaml | kubectl apply --context="${CTX_SECOND_CLUSTER}" -f - {{< /text >}} 1. Confirm that the remote cluster's webhook configuration has been installed: diff --git a/content/en/docs/setup/install/external-controlplane/snips.sh b/content/en/docs/setup/install/external-controlplane/snips.sh index f6bbc0d961..845fa9f9ba 100644 --- a/content/en/docs/setup/install/external-controlplane/snips.sh +++ b/content/en/docs/setup/install/external-controlplane/snips.sh @@ -60,6 +60,19 @@ istio-ingressgateway-9d4c7f5c7-7qpzz 1/1 Running 0 29s istiod-68488cd797-mq8dn 1/1 Running 0 38s ENDSNIP +snip_set_up_a_gateway_in_the_external_cluster_5() { +echo "$EXTERNAL_ISTIOD_ADDR" "$SSL_SECRET_NAME" +} + +! read -r -d '' snip_set_up_a_gateway_in_the_external_cluster_5_out <<\ENDSNIP +myhost.example.com myhost-example-credential +ENDSNIP + +snip_set_up_a_gateway_in_the_external_cluster_6() { +export EXTERNAL_ISTIOD_ADDR=$(kubectl -n istio-system --context="${CTX_EXTERNAL_CLUSTER}" get svc istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}') +export SSL_SECRET_NAME=NONE +} + snip_get_remote_config_cluster_iop() { cat < remote-config-cluster.yaml apiVersion: install.istio.io/v1alpha1 @@ -75,22 +88,31 @@ spec: pilot: configMap: true istiodRemote: - injectionURL: https://${EXTERNAL_ISTIOD_ADDR}:15017/inject/:ENV:cluster=${REMOTE_CLUSTER_NAME}:ENV:net=network1 + injectionURL: https://${EXTERNAL_ISTIOD_ADDR}:15017/inject/cluster/${REMOTE_CLUSTER_NAME}/net/network1 base: validationURL: https://${EXTERNAL_ISTIOD_ADDR}:15017/validate EOF } snip_set_up_the_remote_config_cluster_2() { +sed -i'.bk' \ + -e "s|injectionURL: https://${EXTERNAL_ISTIOD_ADDR}:15017|injectionPath: |" \ + -e "/istioNamespace:/a\\ + remotePilotAddress: ${EXTERNAL_ISTIOD_ADDR}" \ + -e '/base/,+1d' \ + remote-config-cluster.yaml; rm remote-config-cluster.yaml.bk +} + +snip_set_up_the_remote_config_cluster_3() { kubectl create namespace external-istiod --context="${CTX_REMOTE_CLUSTER}" istioctl manifest generate -f remote-config-cluster.yaml | kubectl apply --context="${CTX_REMOTE_CLUSTER}" -f - } -snip_set_up_the_remote_config_cluster_3() { +snip_set_up_the_remote_config_cluster_4() { kubectl get mutatingwebhookconfiguration --context="${CTX_REMOTE_CLUSTER}" } -! read -r -d '' snip_set_up_the_remote_config_cluster_3_out <<\ENDSNIP +! read -r -d '' snip_set_up_the_remote_config_cluster_4_out <<\ENDSNIP NAME WEBHOOKS AGE istio-sidecar-injector-external-istiod 4 6m24s ENDSNIP @@ -175,14 +197,22 @@ EOF } snip_set_up_the_control_plane_in_the_external_cluster_4() { -istioctl manifest generate -f external-istiod.yaml | kubectl apply --context="${CTX_EXTERNAL_CLUSTER}" -f - +sed -i'.bk' \ + -e '/proxyMetadata:/,+2d' \ + -e '/INJECTION_WEBHOOK_CONFIG_NAME/,+1d' \ + -e '/VALIDATION_WEBHOOK_CONFIG_NAME/,+1d' \ + external-istiod.yaml ; rm external-istiod.yaml.bk } snip_set_up_the_control_plane_in_the_external_cluster_5() { +istioctl manifest generate -f external-istiod.yaml | kubectl apply --context="${CTX_EXTERNAL_CLUSTER}" -f - +} + +snip_set_up_the_control_plane_in_the_external_cluster_6() { kubectl get po -n external-istiod --context="${CTX_EXTERNAL_CLUSTER}" } -! read -r -d '' snip_set_up_the_control_plane_in_the_external_cluster_5_out <<\ENDSNIP +! read -r -d '' snip_set_up_the_control_plane_in_the_external_cluster_6_out <<\ENDSNIP NAME READY STATUS RESTARTS AGE istiod-779bd6fdcf-bd6rg 1/1 Running 0 70s ENDSNIP @@ -266,7 +296,17 @@ spec: EOF } -snip_set_up_the_control_plane_in_the_external_cluster_7() { +snip_set_up_the_control_plane_in_the_external_cluster_8() { +sed -i'.bk' \ + -e '55,$d' \ + -e 's/mode: SIMPLE/mode: PASSTHROUGH/' -e '/credentialName:/d' -e "s/${EXTERNAL_ISTIOD_ADDR}/\"*\"/" \ + -e 's/http:/tls:/' -e 's/https/tls/' -e '/route:/i\ + sniHosts:\ + - "*"' \ + external-istiod-gw.yaml; rm external-istiod-gw.yaml.bk +} + +snip_set_up_the_control_plane_in_the_external_cluster_9() { kubectl apply -f external-istiod-gw.yaml --context="${CTX_EXTERNAL_CLUSTER}" } @@ -374,8 +414,8 @@ curl -s "http://${GATEWAY_URL}/hello" Hello version: v1, instance: helloworld-v1-776f57d5f6-s7zfc ENDSNIP -snip_get_second_config_cluster_iop() { -cat < second-config-cluster.yaml +snip_get_second_remote_cluster_iop() { +cat < second-remote-cluster.yaml apiVersion: install.istio.io/v1alpha1 kind: IstioOperator metadata: @@ -386,29 +426,37 @@ spec: global: istioNamespace: external-istiod istiodRemote: - injectionURL: https://${EXTERNAL_ISTIOD_ADDR}:15017/inject/:ENV:cluster=${SECOND_CLUSTER_NAME}:ENV:net=network2 + injectionURL: https://${EXTERNAL_ISTIOD_ADDR}:15017/inject/cluster/${SECOND_CLUSTER_NAME}/net/network2 EOF } snip_register_the_new_cluster_2() { +sed -i'.bk' \ + -e "s|injectionURL: https://${EXTERNAL_ISTIOD_ADDR}:15017|injectionPath: |" \ + -e "/istioNamespace:/a\\ + remotePilotAddress: ${EXTERNAL_ISTIOD_ADDR}" \ + second-remote-cluster.yaml; rm second-remote-cluster.yaml.bk +} + +snip_register_the_new_cluster_3() { kubectl create namespace external-istiod --context="${CTX_SECOND_CLUSTER}" kubectl annotate namespace external-istiod "topology.istio.io/controlPlaneClusters=${REMOTE_CLUSTER_NAME}" --context="${CTX_SECOND_CLUSTER}" } -snip_register_the_new_cluster_3() { -istioctl manifest generate -f second-config-cluster.yaml | kubectl apply --context="${CTX_SECOND_CLUSTER}" -f - +snip_register_the_new_cluster_4() { +istioctl manifest generate -f second-remote-cluster.yaml | kubectl apply --context="${CTX_SECOND_CLUSTER}" -f - } -snip_register_the_new_cluster_4() { +snip_register_the_new_cluster_5() { kubectl get mutatingwebhookconfiguration --context="${CTX_SECOND_CLUSTER}" } -! read -r -d '' snip_register_the_new_cluster_4_out <<\ENDSNIP +! read -r -d '' snip_register_the_new_cluster_5_out <<\ENDSNIP NAME WEBHOOKS AGE istio-sidecar-injector-external-istiod 4 4m13s ENDSNIP -snip_register_the_new_cluster_5() { +snip_register_the_new_cluster_6() { istioctl x create-remote-secret \ --context="${CTX_SECOND_CLUSTER}" \ --name="${SECOND_CLUSTER_NAME}" \ diff --git a/content/en/docs/setup/install/external-controlplane/test.sh b/content/en/docs/setup/install/external-controlplane/test.sh index 1b345e62ba..d9ce8b0602 100644 --- a/content/en/docs/setup/install/external-controlplane/test.sh +++ b/content/en/docs/setup/install/external-controlplane/test.sh @@ -26,34 +26,6 @@ kubectl_get_egress_gateway_for_remote_cluster() { echo "$response" } -# Override some snip functions to configure the istiod gateway using TLS passthrough in the test environemnt. - -snip_get_external_istiod_iop_modified() { - snip_get_external_istiod_iop - - # Update config file: delete CA certificates and meshID, and update pilot vars - # TODO(https://github.com/istio/istio/issues/31690) remove 'env' replace - sed -i \ - -e '/proxyMetadata:/,+2d' \ - -e '/INJECTION_WEBHOOK_CONFIG_NAME/,+1d' \ - -e "/VALIDATION_WEBHOOK_CONFIG_NAME/,+1d" \ - external-istiod.yaml -} - -snip_get_external_istiod_gateway_config_modified() { - TMP="$EXTERNAL_ISTIOD_ADDR" - EXTERNAL_ISTIOD_ADDR='"*"' - snip_get_external_istiod_gateway_config - - # Update config file: delete the DestinationRule, don't terminate TLS in the Gateway, and use TLS routing in the VirtualService - sed -i \ - -e '55,$d' \ - -e 's/mode: SIMPLE/mode: PASSTHROUGH/' -e '/credentialName:/d' \ - -e 's/http:/tls:/' -e 's/https/tls/' -e "/route:/i\ sniHosts:\n - ${EXTERNAL_ISTIOD_ADDR}" \ - external-istiod-gw.yaml - EXTERNAL_ISTIOD_ADDR="$TMP" -} - # Set the CTX_EXTERNAL_CLUSTER, CTX_REMOTE_CLUSTER, and REMOTE_CLUSTER_NAME env variables. _set_kube_vars # helper function to initialize KUBECONFIG_FILES and KUBE_CONTEXTS @@ -68,34 +40,35 @@ echo y | snip_set_up_a_gateway_in_the_external_cluster_2 _verify_like snip_set_up_a_gateway_in_the_external_cluster_3 "$snip_set_up_a_gateway_in_the_external_cluster_3_out" -export SSL_SECRET_NAME="UNUSED" -export EXTERNAL_ISTIOD_ADDR=$(kubectl \ - --context="${CTX_EXTERNAL_CLUSTER}" \ - -n istio-system get svc istio-ingressgateway \ - -o jsonpath='{.status.loadBalancer.ingress[0].ip}') +snip_set_up_a_gateway_in_the_external_cluster_6 # Set up the remote cluster. snip_get_remote_config_cluster_iop +snip_set_up_the_remote_config_cluster_2 #set +e #ignore failures here -echo y | snip_set_up_the_remote_config_cluster_2 +echo y | snip_set_up_the_remote_config_cluster_3 #set -e -_verify_like snip_set_up_the_remote_config_cluster_3 "$snip_set_up_the_remote_config_cluster_3_out" +_verify_like snip_set_up_the_remote_config_cluster_4 "$snip_set_up_the_remote_config_cluster_4_out" # Install istiod on the external cluster. snip_set_up_the_control_plane_in_the_external_cluster_1 snip_set_up_the_control_plane_in_the_external_cluster_2 -snip_get_external_istiod_iop_modified -echo y | istioctl manifest generate -f external-istiod.yaml --set values.pilot.env.ISTIOD_CUSTOM_HOST="${EXTERNAL_ISTIOD_ADDR}" | kubectl apply --context="${CTX_EXTERNAL_CLUSTER}" -f - +snip_get_external_istiod_iop +snip_set_up_the_control_plane_in_the_external_cluster_4 -_verify_like snip_set_up_the_control_plane_in_the_external_cluster_5 "$snip_set_up_the_control_plane_in_the_external_cluster_5_out" +echo y | snip_set_up_the_control_plane_in_the_external_cluster_5 -snip_get_external_istiod_gateway_config_modified -snip_set_up_the_control_plane_in_the_external_cluster_7 +_verify_like snip_set_up_the_control_plane_in_the_external_cluster_6 "$snip_set_up_the_control_plane_in_the_external_cluster_6_out" + +snip_get_external_istiod_gateway_config +snip_set_up_the_control_plane_in_the_external_cluster_8 + +snip_set_up_the_control_plane_in_the_external_cluster_9 # Validate the installation. @@ -130,15 +103,17 @@ _verify_contains snip_test_the_ingress_gateway_4 "Hello version: v1" export CTX_SECOND_CLUSTER="${KUBE_CONTEXTS[1]}" export SECOND_CLUSTER_NAME="${CTX_SECOND_CLUSTER}" -snip_get_second_config_cluster_iop +snip_get_second_remote_cluster_iop snip_register_the_new_cluster_2 -echo y | snip_register_the_new_cluster_3 + +snip_register_the_new_cluster_3 +echo y | snip_register_the_new_cluster_4 # Confirm remote cluster’s webhook configuration has been installed -_verify_like snip_register_the_new_cluster_4 "$snip_register_the_new_cluster_4_out" +_verify_like snip_register_the_new_cluster_5 "$snip_register_the_new_cluster_5_out" # Create a secret with credentials to allow the control plane to access the endpoints on the second remote cluster and install it -snip_register_the_new_cluster_5 +snip_register_the_new_cluster_6 # Setup east-west gateways snip_setup_eastwest_gateways_1 @@ -172,7 +147,7 @@ kubectl delete ns sample --context="${CTX_SECOND_CLUSTER}" kubectl delete -f external-istiod-gw.yaml --context="${CTX_EXTERNAL_CLUSTER}" istioctl manifest generate -f remote-config-cluster.yaml | kubectl delete --context="${CTX_REMOTE_CLUSTER}" -f - -istioctl manifest generate -f second-config-cluster.yaml | kubectl delete --context="${CTX_SECOND_CLUSTER}" -f - +istioctl manifest generate -f second-remote-cluster.yaml | kubectl delete --context="${CTX_SECOND_CLUSTER}" -f - istioctl manifest generate -f eastwest-gateway-1.yaml | kubectl delete --context="${CTX_REMOTE_CLUSTER}" -f - istioctl manifest generate -f eastwest-gateway-2.yaml | kubectl delete --context="${CTX_SECOND_CLUSTER}" -f - istioctl manifest generate -f external-istiod.yaml | kubectl delete --context="${CTX_EXTERNAL_CLUSTER}" -f - @@ -189,4 +164,4 @@ kubectl delete ns istio-system external-istiod --context="${CTX_EXTERNAL_CLUSTER kubectl delete ns external-istiod --context="${CTX_REMOTE_CLUSTER}" kubectl delete ns external-istiod --context="${CTX_SECOND_CLUSTER}" -rm external-istiod-gw.yaml remote-config-cluster.yaml external-istiod.yaml controlplane-gateway.yaml eastwest-gateway-1.yaml eastwest-gateway-2.yaml second-config-cluster.yaml +rm external-istiod-gw.yaml remote-config-cluster.yaml external-istiod.yaml controlplane-gateway.yaml eastwest-gateway-1.yaml eastwest-gateway-2.yaml second-remote-cluster.yaml