From 2525eb5ea5292eb99f15e02f3bb70a1eb689fb29 Mon Sep 17 00:00:00 2001 From: John Howard Date: Mon, 6 Jun 2022 15:36:31 -0700 Subject: [PATCH] Wildcard egress: remove arbitrary domain section (#11291) * Wildcard egress: remove arbitrary domain section This doc has been a nuisance for many years. It recommends an extremely complex and dangerous pattern, relying on deploying nginx, extremely complex EnvoyFilters enabling unsupported, custom, alpha Envoy c++ filters, and a number of other scary practices. IMO this does not belong in Istio docs at all, and certainly not in our top level taks. * Add back single wildcard * Update content/en/docs/tasks/traffic-management/egress/wildcard-egress-hosts/index.md Co-authored-by: Frank Budinsky Co-authored-by: Frank Budinsky --- .../egress/wildcard-egress-hosts/index.md | 390 ------------------ .../egress/wildcard-egress-hosts/snips.sh | 305 -------------- .../egress/wildcard-egress-hosts/test.sh | 44 -- 3 files changed, 739 deletions(-) diff --git a/content/en/docs/tasks/traffic-management/egress/wildcard-egress-hosts/index.md b/content/en/docs/tasks/traffic-management/egress/wildcard-egress-hosts/index.md index 52ca689435..ffbbc08b3c 100644 --- a/content/en/docs/tasks/traffic-management/egress/wildcard-egress-hosts/index.md +++ b/content/en/docs/tasks/traffic-management/egress/wildcard-egress-hosts/index.md @@ -118,18 +118,6 @@ the default) is used in the service entry below. $ kubectl delete serviceentry wikipedia {{< /text >}} -## Configure egress gateway traffic to a wildcard host - -The configuration for accessing a wildcard host via an egress gateway depends on whether or not -the set of wildcard domains are served by a single common host. -This is the case for _*.wikipedia.org_. All of the language-specific sites are served by every -one of the _wikipedia.org_ servers. You can route the traffic to an IP of any _*.wikipedia.org_ site, -including _www.wikipedia.org_, and it will [manage to serve](https://en.wikipedia.org/wiki/Virtual_hosting) -any specific site. - -In the general case, where all the domain names of a wildcard are not served by a single hosting server, -a more complex configuration is required. - ### Wildcard configuration for a single hosting server When all wildcard hosts are served by a single server, the configuration for @@ -254,384 +242,6 @@ $ kubectl delete virtualservice direct-wikipedia-through-egress-gateway $ kubectl delete destinationrule egressgateway-for-wikipedia {{< /text >}} -### Wildcard configuration for arbitrary domains - -The configuration in the previous section worked because all the _*.wikipedia.org_ sites can -be served by any one of the _wikipedia.org_ servers. However, this is not always the case. -For example, you may want to configure egress control for access to more general -wildcard domains like `*.com` or `*.org`. - -Configuring traffic to arbitrary wildcard domains introduces a challenge for Istio gateways. In the previous section -you directed the traffic to _www.wikipedia.org_, which was made known to your gateway during configuration. -The gateway, however, would not know the IP address of any arbitrary host it receives in a request. -This is due to a limitation of [Envoy](https://www.envoyproxy.io), the proxy used by the default Istio egress gateway. -Envoy routes traffic either to predefined hosts, predefined IP addresses, or to the original destination IP address of -the request. In the gateway case, the original destination IP of the request is lost since the request is first routed -to the egress gateway and its destination IP address is the IP address of the gateway. - -Consequently, the Istio gateway based on Envoy cannot route traffic to an arbitrary host that is not preconfigured, -and therefore is unable to perform traffic control for arbitrary wildcard domains. -To enable such traffic control for HTTPS, and for any TLS, you need to deploy an SNI forward proxy in addition to Envoy. -Envoy will route the requests destined for a wildcard domain to the SNI forward proxy, which, in turn, will forward the -requests to the destination specified by the SNI value. - -The egress gateway with SNI proxy and the related parts of the Istio architecture are shown in the following diagram: - -{{< image width="80%" link="./EgressGatewayWithSNIProxyCP.svg" caption="Egress Gateway with SNI proxy" >}} - -The following sections show you how to redeploy the egress gateway with an SNI proxy and then configure Istio to route -HTTPS traffic through the gateway to arbitrary wildcard domains. - -#### Setup egress gateway with SNI proxy - -In this section you deploy an egress gateway with an SNI proxy in addition to the standard Istio Envoy proxy. -This example uses [Nginx](http://nginx.org) for the SNI proxy, although any SNI proxy that is capable of routing traffic -according to arbitrary, not-preconfigured, SNI values would do. -The SNI proxy will listen on port `8443`, although you can use any port other than the ports specified for -the egress `Gateway` and for the `VirtualServices` bound to it. -The SNI proxy will forward the traffic to port `443`. - -1. Create a configuration file for the Nginx SNI proxy. You may want to edit the file to specify additional Nginx - settings, if required. Note that the `listen` directive of the `server` specifies port `8443`, its `proxy_pass` - directive uses `ssl_preread_server_name` with port `443` and `ssl_preread` is `on` to enable `SNI` reading. - - {{< text bash >}} - $ cat < ./sni-proxy.conf - # setup custom path that do not require root access - pid /tmp/nginx.pid; - - events { - } - - stream { - log_format log_stream '\$remote_addr [\$time_local] \$protocol [\$ssl_preread_server_name]' - '\$status \$bytes_sent \$bytes_received \$session_time'; - - access_log /var/log/nginx/access.log log_stream; - error_log /var/log/nginx/error.log; - - # tcp forward proxy by SNI - server { - resolver 8.8.8.8 ipv6=off; - listen 127.0.0.1:18443; - proxy_pass \$ssl_preread_server_name:443; - ssl_preread on; - } - } - EOF - {{< /text >}} - -1. Create a Kubernetes [ConfigMap](https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/) - to hold the configuration of the Nginx SNI proxy: - - {{< text bash >}} - $ kubectl create configmap egress-sni-proxy-configmap -n istio-system --from-file=nginx.conf=./sni-proxy.conf - {{< /text >}} - -1. Create an `IstioOperator` CR to add a new egress gateway with SNI proxy: - - {{< text bash >}} - $ istioctl manifest generate -f - < ./egressgateway-with-sni-proxy.yaml - apiVersion: install.istio.io/v1alpha1 - kind: IstioOperator - spec: - # Only generate a gateway component defined below. - # Using this with "istioctl install" will reconcile and remove existing control-plane components. - # Instead use "istioctl manifest generate" or "kubectl create" if using the istio operator. - profile: empty - components: - egressGateways: - - name: istio-egressgateway-with-sni-proxy - enabled: true - label: - app: istio-egressgateway-with-sni-proxy - istio: egressgateway-with-sni-proxy - k8s: - service: - ports: - - port: 443 - targetPort: 8443 - name: https - overlays: - - kind: Deployment - name: istio-egressgateway-with-sni-proxy - patches: - - path: spec.template.spec.containers[-1] - value: | - name: sni-proxy - image: nginx - volumeMounts: - - name: sni-proxy-config - mountPath: /etc/nginx - readOnly: true - securityContext: - runAsNonRoot: true - runAsUser: 101 - - path: spec.template.spec.volumes[-1] - value: | - name: sni-proxy-config - configMap: - name: egress-sni-proxy-configmap - defaultMode: 292 # 0444 - EOF - {{< /text >}} - -1. Deploy the new gateway: - - {{< text bash >}} - $ kubectl apply -f ./egressgateway-with-sni-proxy.yaml - {{< /text >}} - -1. Verify that the new egress gateway is running. Note that the pod has two containers (one is the Envoy proxy and the - second one is the SNI proxy). - - {{< text bash >}} - $ kubectl get pod -l istio=egressgateway-with-sni-proxy -n istio-system - NAME READY STATUS RESTARTS AGE - istio-egressgateway-with-sni-proxy-79f6744569-pf9t2 2/2 Running 0 17s - {{< /text >}} - -1. Create a service entry with a static address equal to 127.0.0.1 (`localhost`), and disable mutual TLS for traffic directed to the new - service entry: - - {{< text bash >}} - $ kubectl apply -f - <}} - -#### Configure traffic through egress gateway with SNI proxy - -1. Define a `ServiceEntry` for `*.wikipedia.org`: - - {{< text bash >}} - $ cat <}} - -1. Create an egress `Gateway` for _*.wikipedia.org_, port 443, protocol TLS, and a virtual service to direct the - traffic destined for _*.wikipedia.org_ through the gateway. - - {{< text bash >}} - $ kubectl apply -f - <}} - -1. Add an `EnvoyFilter` to the gateway, to prevent it from being deceived. - - {{< text bash >}} - $ kubectl apply -n istio-system -f - <}} - -1. Send HTTPS requests to - [https://en.wikipedia.org](https://en.wikipedia.org) and [https://de.wikipedia.org](https://de.wikipedia.org): - - {{< text bash >}} - $ kubectl exec "$SOURCE_POD" -c sleep -- sh -c 'curl -s https://en.wikipedia.org/wiki/Main_Page | grep -o ".*"; curl -s https://de.wikipedia.org/wiki/Wikipedia:Hauptseite | grep -o ".*"' - Wikipedia, the free encyclopedia - Wikipedia – Die freie Enzyklopädie - {{< /text >}} - -1. Check the log of the egress gateway's Envoy proxy. If Istio is deployed in the `istio-system` namespace, the command to - print the log is: - - {{< text bash >}} - $ kubectl logs -l istio=egressgateway-with-sni-proxy -c istio-proxy -n istio-system - {{< /text >}} - - You should see lines similar to the following: - - {{< text plain >}} - [2019-01-02T16:34:23.312Z] "- - -" 0 - 578 79141 624 - "-" "-" "-" "-" "127.0.0.1:18443" outbound|18443||sni-proxy.local 127.0.0.1:55018 172.30.109.84:443 172.30.109.112:45346 en.wikipedia.org - [2019-01-02T16:34:24.079Z] "- - -" 0 - 586 65770 638 - "-" "-" "-" "-" "127.0.0.1:18443" outbound|18443||sni-proxy.local 127.0.0.1:55034 172.30.109.84:443 172.30.109.112:45362 de.wikipedia.org - {{< /text >}} - -1. Check the logs of the SNI proxy. If Istio is deployed in the `istio-system` namespace, the command to print the - log is: - - {{< text bash >}} - $ kubectl logs -l istio=egressgateway-with-sni-proxy -n istio-system -c sni-proxy - 127.0.0.1 [01/Aug/2018:15:32:02 +0000] TCP [en.wikipedia.org]200 81513 280 0.600 - 127.0.0.1 [01/Aug/2018:15:32:03 +0000] TCP [de.wikipedia.org]200 67745 291 0.659 - {{< /text >}} - -#### Cleanup wildcard configuration for arbitrary domains - -1. Delete the configuration items for _*.wikipedia.org_: - - {{< text bash >}} - $ kubectl delete serviceentry wikipedia - $ kubectl delete gateway istio-egressgateway-with-sni-proxy - $ kubectl delete virtualservice direct-wikipedia-through-egress-gateway - $ kubectl delete destinationrule egressgateway-for-wikipedia - $ kubectl delete --ignore-not-found=true envoyfilter forward-downstream-sni - $ kubectl delete --ignore-not-found=true envoyfilter -n istio-system egress-gateway-sni-verifier - {{< /text >}} - -1. Delete the configuration items for the `egressgateway-with-sni-proxy` deployment: - - {{< text bash >}} - $ kubectl delete serviceentry sni-proxy - $ kubectl delete destinationrule disable-mtls-for-sni-proxy - $ kubectl delete configmap egress-sni-proxy-configmap -n istio-system - $ kubectl delete -f ./egressgateway-with-sni-proxy.yaml - {{< /text >}} - -1. Remove the configuration files you created: - - {{< text bash >}} - $ rm ./sni-proxy.conf ./egressgateway-with-sni-proxy.yaml - {{< /text >}} - ## Cleanup * Shutdown the [sleep]({{< github_tree >}}/samples/sleep) service: diff --git a/content/en/docs/tasks/traffic-management/egress/wildcard-egress-hosts/snips.sh b/content/en/docs/tasks/traffic-management/egress/wildcard-egress-hosts/snips.sh index dfeb4df6cc..ee4a3d26b5 100644 --- a/content/en/docs/tasks/traffic-management/egress/wildcard-egress-hosts/snips.sh +++ b/content/en/docs/tasks/traffic-management/egress/wildcard-egress-hosts/snips.sh @@ -173,311 +173,6 @@ kubectl delete virtualservice direct-wikipedia-through-egress-gateway kubectl delete destinationrule egressgateway-for-wikipedia } -snip_setup_egress_gateway_with_sni_proxy_1() { -cat < ./sni-proxy.conf -# setup custom path that do not require root access -pid /tmp/nginx.pid; - -events { -} - -stream { - log_format log_stream '\$remote_addr [\$time_local] \$protocol [\$ssl_preread_server_name]' - '\$status \$bytes_sent \$bytes_received \$session_time'; - - access_log /var/log/nginx/access.log log_stream; - error_log /var/log/nginx/error.log; - - # tcp forward proxy by SNI - server { - resolver 8.8.8.8 ipv6=off; - listen 127.0.0.1:18443; - proxy_pass \$ssl_preread_server_name:443; - ssl_preread on; - } -} -EOF -} - -snip_setup_egress_gateway_with_sni_proxy_2() { -kubectl create configmap egress-sni-proxy-configmap -n istio-system --from-file=nginx.conf=./sni-proxy.conf -} - -snip_setup_egress_gateway_with_sni_proxy_3() { -istioctl manifest generate -f - < ./egressgateway-with-sni-proxy.yaml -apiVersion: install.istio.io/v1alpha1 -kind: IstioOperator -spec: - # Only generate a gateway component defined below. - # Using this with "istioctl install" will reconcile and remove existing control-plane components. - # Instead use "istioctl manifest generate" or "kubectl create" if using the istio operator. - profile: empty - components: - egressGateways: - - name: istio-egressgateway-with-sni-proxy - enabled: true - label: - app: istio-egressgateway-with-sni-proxy - istio: egressgateway-with-sni-proxy - k8s: - service: - ports: - - port: 443 - targetPort: 8443 - name: https - overlays: - - kind: Deployment - name: istio-egressgateway-with-sni-proxy - patches: - - path: spec.template.spec.containers[-1] - value: | - name: sni-proxy - image: nginx - volumeMounts: - - name: sni-proxy-config - mountPath: /etc/nginx - readOnly: true - securityContext: - runAsNonRoot: true - runAsUser: 101 - - path: spec.template.spec.volumes[-1] - value: | - name: sni-proxy-config - configMap: - name: egress-sni-proxy-configmap - defaultMode: 292 # 0444 -EOF -} - -snip_setup_egress_gateway_with_sni_proxy_4() { -kubectl apply -f ./egressgateway-with-sni-proxy.yaml -} - -snip_setup_egress_gateway_with_sni_proxy_5() { -kubectl get pod -l istio=egressgateway-with-sni-proxy -n istio-system -} - -! read -r -d '' snip_setup_egress_gateway_with_sni_proxy_5_out <<\ENDSNIP -NAME READY STATUS RESTARTS AGE -istio-egressgateway-with-sni-proxy-79f6744569-pf9t2 2/2 Running 0 17s -ENDSNIP - -snip_setup_egress_gateway_with_sni_proxy_6() { -kubectl apply -f - <.*"; curl -s https://de.wikipedia.org/wiki/Wikipedia:Hauptseite | grep -o ".*"' -} - -! read -r -d '' snip_configure_traffic_through_egress_gateway_with_sni_proxy_4_out <<\ENDSNIP -Wikipedia, the free encyclopedia -Wikipedia – Die freie Enzyklopädie -ENDSNIP - -snip_configure_traffic_through_egress_gateway_with_sni_proxy_5() { -kubectl logs -l istio=egressgateway-with-sni-proxy -c istio-proxy -n istio-system -} - -! read -r -d '' snip_configure_traffic_through_egress_gateway_with_sni_proxy_6 <<\ENDSNIP -[2019-01-02T16:34:23.312Z] "- - -" 0 - 578 79141 624 - "-" "-" "-" "-" "127.0.0.1:18443" outbound|18443||sni-proxy.local 127.0.0.1:55018 172.30.109.84:443 172.30.109.112:45346 en.wikipedia.org -[2019-01-02T16:34:24.079Z] "- - -" 0 - 586 65770 638 - "-" "-" "-" "-" "127.0.0.1:18443" outbound|18443||sni-proxy.local 127.0.0.1:55034 172.30.109.84:443 172.30.109.112:45362 de.wikipedia.org -ENDSNIP - -snip_configure_traffic_through_egress_gateway_with_sni_proxy_7() { -kubectl logs -l istio=egressgateway-with-sni-proxy -n istio-system -c sni-proxy -} - -! read -r -d '' snip_configure_traffic_through_egress_gateway_with_sni_proxy_7_out <<\ENDSNIP -127.0.0.1 [01/Aug/2018:15:32:02 +0000] TCP [en.wikipedia.org]200 81513 280 0.600 -127.0.0.1 [01/Aug/2018:15:32:03 +0000] TCP [de.wikipedia.org]200 67745 291 0.659 -ENDSNIP - -snip_cleanup_wildcard_configuration_for_arbitrary_domains_1() { -kubectl delete serviceentry wikipedia -kubectl delete gateway istio-egressgateway-with-sni-proxy -kubectl delete virtualservice direct-wikipedia-through-egress-gateway -kubectl delete destinationrule egressgateway-for-wikipedia -kubectl delete --ignore-not-found=true envoyfilter forward-downstream-sni -kubectl delete --ignore-not-found=true envoyfilter -n istio-system egress-gateway-sni-verifier -} - -snip_cleanup_wildcard_configuration_for_arbitrary_domains_2() { -kubectl delete serviceentry sni-proxy -kubectl delete destinationrule disable-mtls-for-sni-proxy -kubectl delete configmap egress-sni-proxy-configmap -n istio-system -kubectl delete -f ./egressgateway-with-sni-proxy.yaml -} - -snip_cleanup_wildcard_configuration_for_arbitrary_domains_3() { -rm ./sni-proxy.conf ./egressgateway-with-sni-proxy.yaml -} - snip_cleanup_1() { kubectl delete -f samples/sleep/sleep.yaml } diff --git a/content/en/docs/tasks/traffic-management/egress/wildcard-egress-hosts/test.sh b/content/en/docs/tasks/traffic-management/egress/wildcard-egress-hosts/test.sh index bb80f6c8f2..59fcbac9f2 100755 --- a/content/en/docs/tasks/traffic-management/egress/wildcard-egress-hosts/test.sh +++ b/content/en/docs/tasks/traffic-management/egress/wildcard-egress-hosts/test.sh @@ -56,55 +56,11 @@ _verify_contains snip_wildcard_configuration_for_a_single_hosting_server_4 "outb snip_cleanup_wildcard_configuration_for_a_single_hosting_server_1 -snip_setup_egress_gateway_with_sni_proxy_1 - -snip_setup_egress_gateway_with_sni_proxy_2 - -snip_setup_egress_gateway_with_sni_proxy_3 - -snip_setup_egress_gateway_with_sni_proxy_4 -_wait_for_deployment istio-system istio-egressgateway-with-sni-proxy - -_verify_like snip_setup_egress_gateway_with_sni_proxy_5 "$snip_setup_egress_gateway_with_sni_proxy_5_out" - -snip_setup_egress_gateway_with_sni_proxy_6 -_wait_for_istio serviceentry default sni-proxy -_wait_for_istio destinationrule default disable-mtls-for-sni-proxy - -snip_configure_traffic_through_egress_gateway_with_sni_proxy_1 -_wait_for_istio serviceentry default wikipedia - -snip_configure_traffic_through_egress_gateway_with_sni_proxy_2 -_wait_for_istio gateway default istio-egressgateway-with-sni-proxy -_wait_for_istio destinationrule default egressgateway-for-wikipedia -_wait_for_istio virtualservice default direct-wikipedia-through-egress-gateway -_wait_for_istio envoyfilter default forward-downstream-sni - -snip_configure_traffic_through_egress_gateway_with_sni_proxy_3 -_wait_for_istio envoyfilter istio-system egress-gateway-sni-verifier - -_verify_same snip_configure_traffic_through_egress_gateway_with_sni_proxy_4 "$snip_configure_traffic_through_egress_gateway_with_sni_proxy_4_out" - -_verify_lines snip_configure_traffic_through_egress_gateway_with_sni_proxy_5 " -+ outbound|18443||sni-proxy.local -+ en.wikipedia.org -+ de.wikipedia.org -" - -_verify_lines snip_configure_traffic_through_egress_gateway_with_sni_proxy_7 " -+ TCP [en.wikipedia.org]200 -+ TCP [de.wikipedia.org]200 -" - # @cleanup snip_cleanup_direct_traffic_to_a_wildcard_host_1 snip_cleanup_wildcard_configuration_for_a_single_hosting_server_1 -snip_cleanup_wildcard_configuration_for_arbitrary_domains_1 -snip_cleanup_wildcard_configuration_for_arbitrary_domains_2 -snip_cleanup_wildcard_configuration_for_arbitrary_domains_3 - snip_cleanup_1 echo y | snip_cleanup_2