Fix and test wildcard egress task (#8050)

* Fix and test wildcard egress task

* change startup

* change startup

* lint

* more lint

* fixes from vadim

* fixes

* fix gateway filter

* add EOF

* regen snips
This commit is contained in:
Frank Budinsky 2020-09-01 09:17:09 -04:00 committed by GitHub
parent 3c07d89870
commit 28d609fc84
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
7 changed files with 785 additions and 211 deletions

View File

@ -4,6 +4,11 @@
* Setup Istio by following the instructions in the [Installation guide](/docs/setup/).
{{< tip >}}
The egress gateway and access logging will be enabled if you install the `demo`
[configuration profile](/docs/setup/additional-setup/config-profiles/).
{{< /tip >}}
* Deploy the [sleep]({{< github_tree >}}/samples/sleep) sample app to use as a test source for sending requests.
If you have
[automatic sidecar injection](/docs/setup/additional-setup/sidecar-injection/#automatic-sidecar-injection)

View File

@ -41,6 +41,14 @@ controlled way.
* [Enable Envoys access logging](/docs/tasks/observability/logs/access-log/#enable-envoy-s-access-logging)
{{< warning >}}
The instructions in this task create a destination rule for the egress gateway in the `default` namespace
and assume that the client, `SOURCE_POD`, is also running in the `default` namespace.
If not, the destination rule will not be found on the
[destination rule lookup path](/docs/ops/best-practices/traffic-management/#cross-namespace-configuration)
and the client requests will fail.
{{< /warning >}}
## Deploy Istio egress gateway
1. Check if the Istio egress gateway is deployed:
@ -58,15 +66,6 @@ controlled way.
--set values.gateways.istio-egressgateway.enabled=true
{{< /text >}}
{{< warning >}}
The following instructions create a destination rule for the egress gateway in the `default` namespace
and assume that the client, `SOURCE_POD`, is also running in the `default` namespace.
If not, the destination rule will not be found on the
[destination rule lookup path](/docs/ops/best-practices/traffic-management/#cross-namespace-configuration)
and the client requests will fail.
{{< /warning >}}
## Egress gateway for HTTP traffic
First create a `ServiceEntry` to allow direct traffic to an external service.

File diff suppressed because one or more lines are too long

Before

Width:  |  Height:  |  Size: 72 KiB

File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 126 KiB

View File

@ -6,7 +6,7 @@ weight: 50
aliases:
- /docs/examples/advanced-gateways/wildcard-egress-hosts/
owner: istio/wg-networking-maintainers
test: no
test: yes
---
The [Control Egress Traffic](/docs/tasks/traffic-management/egress/) task and
@ -23,11 +23,39 @@ Each version of `wikipedia.org` in a particular language has its own hostname, e
You want to enable egress traffic by common configuration items for all the Wikipedia sites,
without the need to specify every language's site separately.
{{< boilerplate before-you-begin-egress >}}
## Before you begin
* [Deploy Istio egress gateway](/docs/tasks/traffic-management/egress/egress-gateway/#deploy-istio-egress-gateway).
* Install Istio using the `demo` [configuration profile](/docs/setup/additional-setup/config-profiles/)
and with the blocking-by-default outbound traffic policy:
* [Enable Envoys access logging](/docs/tasks/observability/logs/access-log/#enable-envoy-s-access-logging)
{{< text bash >}}
$ istioctl install --set profile=demo --set meshConfig.outboundTrafficPolicy.mode=REGISTRY_ONLY
{{< /text >}}
* Deploy the [sleep]({{< github_tree >}}/samples/sleep) sample app to use as a test source for sending requests.
If you have
[automatic sidecar injection](/docs/setup/additional-setup/sidecar-injection/#automatic-sidecar-injection)
enabled, run the following command to deploy the sample app:
{{< text bash >}}
$ kubectl apply -f @samples/sleep/sleep.yaml@
{{< /text >}}
Otherwise, manually inject the sidecar before deploying the `sleep` application with the following command:
{{< text bash >}}
$ kubectl apply -f <(istioctl kube-inject -f @samples/sleep/sleep.yaml@)
{{< /text >}}
{{< tip >}}
You can use any pod with `curl` installed as a test source.
{{< /tip >}}
* Set the `SOURCE_POD` environment variable to the name of your source pod:
{{< text bash >}}
$ export SOURCE_POD=$(kubectl get pod -l app=sleep -o jsonpath={.items..metadata.name})
{{< /text >}}
## Configure direct traffic to a wildcard host
@ -37,7 +65,17 @@ When calling services directly (i.e., not via an egress gateway), the configurat
a wildcard host is no different than that of any other (e.g., fully qualified) host,
only much more convenient when there are many hosts within the common domain.
1. Define a `ServiceEntry` and corresponding `VirtualSevice` for `*.wikipedia.org`:
{{< warning >}}
Note that the configuration below can be easily bypassed by a malicious application. For a secure egress traffic control,
direct the traffic through an egress gateway.
{{< /warning >}}
{{< warning >}}
Note that the `DNS` resolution cannot be used for wildcard hosts. This is why the `NONE` resolution (omitted since it is
the default) is used in the service entry below.
{{< /warning >}}
1. Define a `ServiceEntry` for `*.wikipedia.org`:
{{< text bash >}}
$ kubectl apply -f - <<EOF
@ -50,26 +88,8 @@ only much more convenient when there are many hosts within the common domain.
- "*.wikipedia.org"
ports:
- number: 443
name: tls
protocol: TLS
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: wikipedia
spec:
hosts:
- "*.wikipedia.org"
tls:
- match:
- port: 443
sniHosts:
- "*.wikipedia.org"
route:
- destination:
host: "*.wikipedia.org"
port:
number: 443
name: https
protocol: HTTPS
EOF
{{< /text >}}
@ -77,7 +97,7 @@ only much more convenient when there are many hosts within the common domain.
[https://en.wikipedia.org](https://en.wikipedia.org) and [https://de.wikipedia.org](https://de.wikipedia.org):
{{< text bash >}}
$ kubectl exec -it $SOURCE_POD -c sleep -- sh -c 'curl -s https://en.wikipedia.org/wiki/Main_Page | grep -o "<title>.*</title>"; curl -s https://de.wikipedia.org/wiki/Wikipedia:Hauptseite | grep -o "<title>.*</title>"'
$ kubectl exec "$SOURCE_POD" -c sleep -- sh -c 'curl -s https://en.wikipedia.org/wiki/Main_Page | grep -o "<title>.*</title>"; curl -s https://de.wikipedia.org/wiki/Wikipedia:Hauptseite | grep -o "<title>.*</title>"'
<title>Wikipedia, the free encyclopedia</title>
<title>Wikipedia Die freie Enzyklopädie</title>
{{< /text >}}
@ -86,7 +106,6 @@ only much more convenient when there are many hosts within the common domain.
{{< text bash >}}
$ kubectl delete serviceentry wikipedia
$ kubectl delete virtualservice wikipedia
{{< /text >}}
## Configure egress gateway traffic to a wildcard host
@ -124,8 +143,8 @@ the set of domains.
servers:
- port:
number: 443
name: tls
protocol: TLS
name: https
protocol: HTTPS
hosts:
- "*.wikipedia.org"
tls:
@ -192,8 +211,8 @@ the set of domains.
- www.wikipedia.org
ports:
- number: 443
name: tls
protocol: TLS
name: https
protocol: HTTPS
resolution: DNS
EOF
{{< /text >}}
@ -202,7 +221,7 @@ the set of domains.
[https://en.wikipedia.org](https://en.wikipedia.org) and [https://de.wikipedia.org](https://de.wikipedia.org):
{{< text bash >}}
$ kubectl exec -it $SOURCE_POD -c sleep -- sh -c 'curl -s https://en.wikipedia.org/wiki/Main_Page | grep -o "<title>.*</title>"; curl -s https://de.wikipedia.org/wiki/Wikipedia:Hauptseite | grep -o "<title>.*</title>"'
$ kubectl exec "$SOURCE_POD" -c sleep -- sh -c 'curl -s https://en.wikipedia.org/wiki/Main_Page | grep -o "<title>.*</title>"; curl -s https://de.wikipedia.org/wiki/Wikipedia:Hauptseite | grep -o "<title>.*</title>"'
<title>Wikipedia, the free encyclopedia</title>
<title>Wikipedia Die freie Enzyklopädie</title>
{{< /text >}}
@ -212,7 +231,7 @@ the set of domains.
counter is:
{{< text bash >}}
$ kubectl exec -it $(kubectl get pod -l istio=egressgateway -n istio-system -o jsonpath='{.items[0].metadata.name}') -c istio-proxy -n istio-system -- pilot-agent request GET clusters | grep '^outbound|443||www.wikipedia.org.*cx_total:'
$ kubectl exec "$(kubectl get pod -l istio=egressgateway -n istio-system -o jsonpath='{.items[0].metadata.name}')" -c istio-proxy -n istio-system -- pilot-agent request GET clusters | grep '^outbound|443||www.wikipedia.org.*cx_total:'
outbound|443||www.wikipedia.org::208.80.154.224:443::cx_total::2
{{< /text >}}
@ -248,7 +267,7 @@ requests to the destination specified by the SNI value.
The egress gateway with SNI proxy and the related parts of the Istio architecture are shown in the following diagram:
{{< image width="80%" link="./EgressGatewayWithSNIProxy.svg" caption="Egress Gateway with SNI proxy" >}}
{{< image width="80%" link="./EgressGatewayWithSNIProxyCP.svg" caption="Egress Gateway with SNI proxy" >}}
The following sections show you how to redeploy the egress gateway with an SNI proxy and then configure Istio to route
HTTPS traffic through the gateway to arbitrary wildcard domains.
@ -298,61 +317,66 @@ The SNI proxy will forward the traffic to port `443`.
$ kubectl create configmap egress-sni-proxy-configmap -n istio-system --from-file=nginx.conf=./sni-proxy.conf
{{< /text >}}
1. The following command will generate `istio-egressgateway-with-sni-proxy.yaml` which you can optionally edit and then deploy.
1. Create an `IstioOperator` CR to add a new egress gateway with SNI proxy:
{{< text bash >}}
$ cat <<EOF | istioctl manifest generate --set values.global.istioNamespace=istio-system -f - > ./istio-egressgateway-with-sni-proxy.yaml
gateways:
enabled: true
istio-ingressgateway:
enabled: false
istio-egressgateway:
enabled: false
istio-egressgateway-with-sni-proxy:
enabled: true
labels:
app: istio-egressgateway-with-sni-proxy
istio: egressgateway-with-sni-proxy
replicaCount: 1
autoscaleMin: 1
autoscaleMax: 5
cpu:
targetAverageUtilization: 80
serviceAnnotations: {}
type: ClusterIP
ports:
- port: 443
name: https
secretVolumes:
- name: egressgateway-certs
secretName: istio-egressgateway-certs
mountPath: /etc/istio/egressgateway-certs
- name: egressgateway-ca-certs
secretName: istio-egressgateway-ca-certs
mountPath: /etc/istio/egressgateway-ca-certs
configVolumes:
- name: sni-proxy-config
configMapName: egress-sni-proxy-configmap
additionalContainers:
- name: sni-proxy
image: nginx
volumeMounts:
- name: sni-proxy-config
mountPath: /etc/nginx
readOnly: true
$ cat > ./egressgateway-with-sni-proxy.yaml <<EOF
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
profile: demo
meshConfig:
outboundTrafficPolicy:
mode: REGISTRY_ONLY
components:
egressGateways:
- name: istio-egressgateway-with-sni-proxy
enabled: true
label:
app: istio-egressgateway-with-sni-proxy
istio: egressgateway-with-sni-proxy
k8s:
service:
ports:
- port: 443
name: https
EOF
{{< /text >}}
1. Deploy the new egress gateway:
1. Deploy the new gateway:
{{< text bash >}}
$ kubectl apply -f ./istio-egressgateway-with-sni-proxy.yaml
serviceaccount "istio-egressgateway-with-sni-proxy-service-account" created
role "istio-egressgateway-with-sni-proxy-istio-system" created
rolebinding "istio-egressgateway-with-sni-proxy-istio-system" created
service "istio-egressgateway-with-sni-proxy" created
deployment "istio-egressgateway-with-sni-proxy" created
horizontalpodautoscaler "istio-egressgateway-with-sni-proxy" created
$ istioctl install -f ./egressgateway-with-sni-proxy.yaml --set values.gateways.istio-egressgateway.runAsRoot=true
{{< /text >}}
1. Patch the deployment with a second pod container for the SNI proxy:
{{< text bash >}}
$ cat <<EOF > ./egressgateway-with-sni-proxy-patch.yaml
spec:
template:
spec:
volumes:
- name: sni-proxy-config
configMap:
name: egress-sni-proxy-configmap
defaultMode: 292 # 0444
containers:
- name: sni-proxy
image: nginx
volumeMounts:
- name: sni-proxy-config
mountPath: /etc/nginx
readOnly: true
securityContext:
runAsNonRoot: false
runAsUser: 0
EOF
{{< /text >}}
{{< text bash >}}
$ kubectl patch deployment istio-egressgateway-with-sni-proxy -n istio-system --patch "$(cat ./egressgateway-with-sni-proxy-patch.yaml)"
deployment.apps/istio-egressgateway-with-sni-proxy patched
{{< /text >}}
1. Verify that the new egress gateway is running. Note that the pod has two containers (one is the Envoy proxy and the
@ -420,13 +444,6 @@ The SNI proxy will forward the traffic to port `443`.
1. Create an egress `Gateway` for _*.wikipedia.org_, port 443, protocol TLS, and a virtual service to direct the
traffic destined for _*.wikipedia.org_ through the gateway.
Choose the instructions corresponding to whether or not you want to enable
[mutual TLS Authentication](/docs/tasks/security/authentication/authn-policy/) between the source pod and the egress gateway.
{{< tabset category-name="mtls" >}}
{{< tab name="mutual TLS enabled" category-value="enabled" >}}
{{< text bash >}}
$ kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
@ -444,10 +461,7 @@ The SNI proxy will forward the traffic to port `443`.
hosts:
- "*.wikipedia.org"
tls:
mode: MUTUAL
serverCertificate: /etc/certs/cert-chain.pem
privateKey: /etc/certs/key.pem
caCertificates: /etc/certs/root-cert.pem
mode: ISTIO_MUTUAL
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
@ -502,124 +516,70 @@ The SNI proxy will forward the traffic to port `443`.
number: 8443
weight: 100
---
# The following filter is used to forward the original SNI (sent by the application) as the SNI of the mutual TLS
# connection.
# The forwarded SNI will be reported to Mixer so that policies will be enforced based on the original SNI value.
# The following filter is used to forward the original SNI (sent by the application) as the SNI of the
# mutual TLS connection.
# The forwarded SNI will be will be used to enforce policies based on the original SNI value.
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: forward-downstream-sni
spec:
filters:
- listenerMatch:
portNumber: 443
listenerType: SIDECAR_OUTBOUND
filterName: forward_downstream_sni
filterType: NETWORK
filterConfig: {}
---
# The following filter verifies that the SNI of the mutual TLS connection (the SNI reported to Mixer) is
# identical to the original SNI issued by the application (the SNI used for routing by the SNI proxy).
# The filter prevents Mixer from being deceived by a malicious application: routing to one SNI while
# reporting some other value of SNI. If the original SNI does not match the SNI of the mutual TLS connection, the
# filter will block the connection to the external service.
configPatches:
- applyTo: NETWORK_FILTER
match:
context: SIDECAR_OUTBOUND
listener:
portNumber: 443
filterChain:
filter:
name: istio.stats
patch:
operation: INSERT_BEFORE
value:
name: forward_downstream_sni
config: {}
EOF
{{< /text >}}
1. Add an `EnvoyFilter` to the gateway, to prevent if from being deceived.
{{< text bash >}}
$ kubectl apply -n istio-system -f - <<EOF
# The following filter verifies that the SNI of the mutual TLS connection is
# identical to the original SNI issued by the client (the SNI used for routing by the SNI proxy).
# The filter prevents the gateway from being deceived by a malicious client: routing to one SNI while
# reporting some other value of SNI. If the original SNI does not match the SNI of the mutual TLS connection,
# the filter will block the connection to the external service.
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: egress-gateway-sni-verifier
spec:
workloadLabels:
app: istio-egressgateway-with-sni-proxy
filters:
- listenerMatch:
portNumber: 443
listenerType: GATEWAY
filterName: sni_verifier
filterType: NETWORK
filterConfig: {}
workloadSelector:
labels:
app: istio-egressgateway-with-sni-proxy
configPatches:
- applyTo: NETWORK_FILTER
match:
context: GATEWAY
listener:
portNumber: 443
filterChain:
filter:
name: istio.stats
patch:
operation: INSERT_BEFORE
value:
name: sni_verifier
config: {}
EOF
{{< /text >}}
{{< /tab >}}
{{< tab name="mutual TLS disabled" category-value="disabled" >}}
{{< text bash >}}
$ kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: istio-egressgateway-with-sni-proxy
spec:
selector:
istio: egressgateway-with-sni-proxy
servers:
- port:
number: 443
name: tls
protocol: TLS
hosts:
- "*.wikipedia.org"
tls:
mode: PASSTHROUGH
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: egressgateway-for-wikipedia
spec:
host: istio-egressgateway-with-sni-proxy.istio-system.svc.cluster.local
subsets:
- name: wikipedia
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: direct-wikipedia-through-egress-gateway
spec:
hosts:
- "*.wikipedia.org"
gateways:
- mesh
- istio-egressgateway-with-sni-proxy
tls:
- match:
- gateways:
- mesh
port: 443
sniHosts:
- "*.wikipedia.org"
route:
- destination:
host: istio-egressgateway-with-sni-proxy.istio-system.svc.cluster.local
subset: wikipedia
port:
number: 443
weight: 100
- match:
- gateways:
- istio-egressgateway-with-sni-proxy
port: 443
sniHosts:
- "*.wikipedia.org"
route:
- destination:
host: sni-proxy.local
port:
number: 8443
weight: 100
EOF
{{< /text >}}
{{< /tab >}}
{{< /tabset >}}
1. Send HTTPS requests to
[https://en.wikipedia.org](https://en.wikipedia.org) and [https://de.wikipedia.org](https://de.wikipedia.org):
{{< text bash >}}
$ kubectl exec -it $SOURCE_POD -c sleep -- sh -c 'curl -s https://en.wikipedia.org/wiki/Main_Page | grep -o "<title>.*</title>"; curl -s https://de.wikipedia.org/wiki/Wikipedia:Hauptseite | grep -o "<title>.*</title>"'
$ kubectl exec "$SOURCE_POD" -c sleep -- sh -c 'curl -s https://en.wikipedia.org/wiki/Main_Page | grep -o "<title>.*</title>"; curl -s https://de.wikipedia.org/wiki/Wikipedia:Hauptseite | grep -o "<title>.*</title>"'
<title>Wikipedia, the free encyclopedia</title>
<title>Wikipedia Die freie Enzyklopädie</title>
{{< /text >}}
@ -664,21 +624,20 @@ The SNI proxy will forward the traffic to port `443`.
{{< text bash >}}
$ kubectl delete serviceentry sni-proxy
$ kubectl delete destinationrule disable-mtls-for-sni-proxy
$ kubectl delete -f ./istio-egressgateway-with-sni-proxy.yaml
$ kubectl delete configmap egress-sni-proxy-configmap -n istio-system
{{< /text >}}
1. Remove the configuration files you created:
{{< text bash >}}
$ rm ./istio-egressgateway-with-sni-proxy.yaml
$ rm ./sni-proxy.conf
$ rm ./sni-proxy.conf ./egressgateway-with-sni-proxy.yaml ./egressgateway-with-sni-proxy-patch.yaml
{{< /text >}}
## Cleanup
Shutdown the [sleep]({{< github_tree >}}/samples/sleep) service:
* Shutdown the [sleep]({{< github_tree >}}/samples/sleep) service:
{{< text bash >}}
$ kubectl delete -f @samples/sleep/sleep.yaml@
{{< /text >}}
{{< text bash >}}
$ kubectl delete -f @samples/sleep/sleep.yaml@
$ istioctl x uninstall
{{< /text >}}

View File

@ -0,0 +1,496 @@
#!/bin/bash
# shellcheck disable=SC2034,SC2153,SC2155,SC2164
# Copyright Istio Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
####################################################################################################
# WARNING: THIS IS AN AUTO-GENERATED FILE, DO NOT EDIT. PLEASE MODIFY THE ORIGINAL MARKDOWN FILE:
# docs/tasks/traffic-management/egress/wildcard-egress-hosts/index.md
####################################################################################################
snip_before_you_begin_1() {
istioctl install --set profile=demo --set meshConfig.outboundTrafficPolicy.mode=REGISTRY_ONLY
}
snip_before_you_begin_2() {
kubectl apply -f samples/sleep/sleep.yaml
}
snip_before_you_begin_3() {
kubectl apply -f <(istioctl kube-inject -f samples/sleep/sleep.yaml)
}
snip_before_you_begin_4() {
export SOURCE_POD=$(kubectl get pod -l app=sleep -o jsonpath={.items..metadata.name})
}
snip_configure_direct_traffic_to_a_wildcard_host_1() {
kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: wikipedia
spec:
hosts:
- "*.wikipedia.org"
ports:
- number: 443
name: https
protocol: HTTPS
EOF
}
snip_configure_direct_traffic_to_a_wildcard_host_2() {
kubectl exec "$SOURCE_POD" -c sleep -- sh -c 'curl -s https://en.wikipedia.org/wiki/Main_Page | grep -o "<title>.*</title>"; curl -s https://de.wikipedia.org/wiki/Wikipedia:Hauptseite | grep -o "<title>.*</title>"'
}
! read -r -d '' snip_configure_direct_traffic_to_a_wildcard_host_2_out <<\ENDSNIP
<title>Wikipedia, the free encyclopedia</title>
<title>Wikipedia Die freie Enzyklopädie</title>
ENDSNIP
snip_cleanup_direct_traffic_to_a_wildcard_host_1() {
kubectl delete serviceentry wikipedia
}
snip_wildcard_configuration_for_a_single_hosting_server_1() {
kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: istio-egressgateway
spec:
selector:
istio: egressgateway
servers:
- port:
number: 443
name: https
protocol: HTTPS
hosts:
- "*.wikipedia.org"
tls:
mode: PASSTHROUGH
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: egressgateway-for-wikipedia
spec:
host: istio-egressgateway.istio-system.svc.cluster.local
subsets:
- name: wikipedia
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: direct-wikipedia-through-egress-gateway
spec:
hosts:
- "*.wikipedia.org"
gateways:
- mesh
- istio-egressgateway
tls:
- match:
- gateways:
- mesh
port: 443
sniHosts:
- "*.wikipedia.org"
route:
- destination:
host: istio-egressgateway.istio-system.svc.cluster.local
subset: wikipedia
port:
number: 443
weight: 100
- match:
- gateways:
- istio-egressgateway
port: 443
sniHosts:
- "*.wikipedia.org"
route:
- destination:
host: www.wikipedia.org
port:
number: 443
weight: 100
EOF
}
snip_wildcard_configuration_for_a_single_hosting_server_2() {
kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: www-wikipedia
spec:
hosts:
- www.wikipedia.org
ports:
- number: 443
name: https
protocol: HTTPS
resolution: DNS
EOF
}
snip_wildcard_configuration_for_a_single_hosting_server_3() {
kubectl exec "$SOURCE_POD" -c sleep -- sh -c 'curl -s https://en.wikipedia.org/wiki/Main_Page | grep -o "<title>.*</title>"; curl -s https://de.wikipedia.org/wiki/Wikipedia:Hauptseite | grep -o "<title>.*</title>"'
}
! read -r -d '' snip_wildcard_configuration_for_a_single_hosting_server_3_out <<\ENDSNIP
<title>Wikipedia, the free encyclopedia</title>
<title>Wikipedia Die freie Enzyklopädie</title>
ENDSNIP
snip_wildcard_configuration_for_a_single_hosting_server_4() {
kubectl exec "$(kubectl get pod -l istio=egressgateway -n istio-system -o jsonpath='{.items[0].metadata.name}')" -c istio-proxy -n istio-system -- pilot-agent request GET clusters | grep '^outbound|443||www.wikipedia.org.*cx_total:'
}
! read -r -d '' snip_wildcard_configuration_for_a_single_hosting_server_4_out <<\ENDSNIP
outbound|443||www.wikipedia.org::208.80.154.224:443::cx_total::2
ENDSNIP
snip_cleanup_wildcard_configuration_for_a_single_hosting_server_1() {
kubectl delete serviceentry www-wikipedia
kubectl delete gateway istio-egressgateway
kubectl delete virtualservice direct-wikipedia-through-egress-gateway
kubectl delete destinationrule egressgateway-for-wikipedia
}
snip_setup_egress_gateway_with_sni_proxy_1() {
cat <<EOF > ./sni-proxy.conf
user www-data;
events {
}
stream {
log_format log_stream '\$remote_addr [\$time_local] \$protocol [\$ssl_preread_server_name]'
'\$status \$bytes_sent \$bytes_received \$session_time';
access_log /var/log/nginx/access.log log_stream;
error_log /var/log/nginx/error.log;
# tcp forward proxy by SNI
server {
resolver 8.8.8.8 ipv6=off;
listen 127.0.0.1:8443;
proxy_pass \$ssl_preread_server_name:443;
ssl_preread on;
}
}
EOF
}
snip_setup_egress_gateway_with_sni_proxy_2() {
kubectl create configmap egress-sni-proxy-configmap -n istio-system --from-file=nginx.conf=./sni-proxy.conf
}
snip_setup_egress_gateway_with_sni_proxy_3() {
cat > ./egressgateway-with-sni-proxy.yaml <<EOF
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
profile: demo
meshConfig:
outboundTrafficPolicy:
mode: REGISTRY_ONLY
components:
egressGateways:
- name: istio-egressgateway-with-sni-proxy
enabled: true
label:
app: istio-egressgateway-with-sni-proxy
istio: egressgateway-with-sni-proxy
k8s:
service:
ports:
- port: 443
name: https
EOF
}
snip_setup_egress_gateway_with_sni_proxy_4() {
istioctl install -f ./egressgateway-with-sni-proxy.yaml --set values.gateways.istio-egressgateway.runAsRoot=true
}
snip_setup_egress_gateway_with_sni_proxy_5() {
cat <<EOF > ./egressgateway-with-sni-proxy-patch.yaml
spec:
template:
spec:
volumes:
- name: sni-proxy-config
configMap:
name: egress-sni-proxy-configmap
defaultMode: 292 # 0444
containers:
- name: sni-proxy
image: nginx
volumeMounts:
- name: sni-proxy-config
mountPath: /etc/nginx
readOnly: true
securityContext:
runAsNonRoot: false
runAsUser: 0
EOF
}
snip_setup_egress_gateway_with_sni_proxy_6() {
kubectl patch deployment istio-egressgateway-with-sni-proxy -n istio-system --patch "$(cat ./egressgateway-with-sni-proxy-patch.yaml)"
}
! read -r -d '' snip_setup_egress_gateway_with_sni_proxy_6_out <<\ENDSNIP
deployment.apps/istio-egressgateway-with-sni-proxy patched
ENDSNIP
snip_setup_egress_gateway_with_sni_proxy_7() {
kubectl get pod -l istio=egressgateway-with-sni-proxy -n istio-system
}
! read -r -d '' snip_setup_egress_gateway_with_sni_proxy_7_out <<\ENDSNIP
NAME READY STATUS RESTARTS AGE
istio-egressgateway-with-sni-proxy-79f6744569-pf9t2 2/2 Running 0 17s
ENDSNIP
snip_setup_egress_gateway_with_sni_proxy_8() {
kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: sni-proxy
spec:
hosts:
- sni-proxy.local
location: MESH_EXTERNAL
ports:
- number: 8443
name: tcp
protocol: TCP
resolution: STATIC
endpoints:
- address: 127.0.0.1
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: disable-mtls-for-sni-proxy
spec:
host: sni-proxy.local
trafficPolicy:
tls:
mode: DISABLE
EOF
}
snip_configure_traffic_through_egress_gateway_with_sni_proxy_1() {
cat <<EOF | kubectl create -f -
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: wikipedia
spec:
hosts:
- "*.wikipedia.org"
ports:
- number: 443
name: tls
protocol: TLS
EOF
}
snip_configure_traffic_through_egress_gateway_with_sni_proxy_2() {
kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: istio-egressgateway-with-sni-proxy
spec:
selector:
istio: egressgateway-with-sni-proxy
servers:
- port:
number: 443
name: tls-egress
protocol: TLS
hosts:
- "*.wikipedia.org"
tls:
mode: ISTIO_MUTUAL
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: egressgateway-for-wikipedia
spec:
host: istio-egressgateway-with-sni-proxy.istio-system.svc.cluster.local
subsets:
- name: wikipedia
trafficPolicy:
loadBalancer:
simple: ROUND_ROBIN
portLevelSettings:
- port:
number: 443
tls:
mode: ISTIO_MUTUAL
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: direct-wikipedia-through-egress-gateway
spec:
hosts:
- "*.wikipedia.org"
gateways:
- mesh
- istio-egressgateway-with-sni-proxy
tls:
- match:
- gateways:
- mesh
port: 443
sniHosts:
- "*.wikipedia.org"
route:
- destination:
host: istio-egressgateway-with-sni-proxy.istio-system.svc.cluster.local
subset: wikipedia
port:
number: 443
weight: 100
tcp:
- match:
- gateways:
- istio-egressgateway-with-sni-proxy
port: 443
route:
- destination:
host: sni-proxy.local
port:
number: 8443
weight: 100
---
# The following filter is used to forward the original SNI (sent by the application) as the SNI of the
# mutual TLS connection.
# The forwarded SNI will be will be used to enforce policies based on the original SNI value.
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: forward-downstream-sni
spec:
configPatches:
- applyTo: NETWORK_FILTER
match:
context: SIDECAR_OUTBOUND
listener:
portNumber: 443
filterChain:
filter:
name: istio.stats
patch:
operation: INSERT_BEFORE
value:
name: forward_downstream_sni
config: {}
EOF
}
snip_configure_traffic_through_egress_gateway_with_sni_proxy_3() {
kubectl apply -n istio-system -f - <<EOF
# The following filter verifies that the SNI of the mutual TLS connection is
# identical to the original SNI issued by the client (the SNI used for routing by the SNI proxy).
# The filter prevents the gateway from being deceived by a malicious client: routing to one SNI while
# reporting some other value of SNI. If the original SNI does not match the SNI of the mutual TLS connection,
# the filter will block the connection to the external service.
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: egress-gateway-sni-verifier
spec:
workloadSelector:
labels:
app: istio-egressgateway-with-sni-proxy
configPatches:
- applyTo: NETWORK_FILTER
match:
context: GATEWAY
listener:
portNumber: 443
filterChain:
filter:
name: istio.stats
patch:
operation: INSERT_BEFORE
value:
name: sni_verifier
config: {}
EOF
}
snip_configure_traffic_through_egress_gateway_with_sni_proxy_4() {
kubectl exec "$SOURCE_POD" -c sleep -- sh -c 'curl -s https://en.wikipedia.org/wiki/Main_Page | grep -o "<title>.*</title>"; curl -s https://de.wikipedia.org/wiki/Wikipedia:Hauptseite | grep -o "<title>.*</title>"'
}
! read -r -d '' snip_configure_traffic_through_egress_gateway_with_sni_proxy_4_out <<\ENDSNIP
<title>Wikipedia, the free encyclopedia</title>
<title>Wikipedia Die freie Enzyklopädie</title>
ENDSNIP
snip_configure_traffic_through_egress_gateway_with_sni_proxy_5() {
kubectl logs -l istio=egressgateway-with-sni-proxy -c istio-proxy -n istio-system
}
! read -r -d '' snip_configure_traffic_through_egress_gateway_with_sni_proxy_6 <<\ENDSNIP
[2019-01-02T16:34:23.312Z] "- - -" 0 - 578 79141 624 - "-" "-" "-" "-" "127.0.0.1:8443" outbound|8443||sni-proxy.local 127.0.0.1:55018 172.30.109.84:443 172.30.109.112:45346 en.wikipedia.org
[2019-01-02T16:34:24.079Z] "- - -" 0 - 586 65770 638 - "-" "-" "-" "-" "127.0.0.1:8443" outbound|8443||sni-proxy.local 127.0.0.1:55034 172.30.109.84:443 172.30.109.112:45362 de.wikipedia.org
ENDSNIP
snip_configure_traffic_through_egress_gateway_with_sni_proxy_7() {
kubectl logs -l istio=egressgateway-with-sni-proxy -n istio-system -c sni-proxy
}
! read -r -d '' snip_configure_traffic_through_egress_gateway_with_sni_proxy_7_out <<\ENDSNIP
127.0.0.1 [01/Aug/2018:15:32:02 +0000] TCP [en.wikipedia.org]200 81513 280 0.600
127.0.0.1 [01/Aug/2018:15:32:03 +0000] TCP [de.wikipedia.org]200 67745 291 0.659
ENDSNIP
snip_cleanup_wildcard_configuration_for_arbitrary_domains_1() {
kubectl delete serviceentry wikipedia
kubectl delete gateway istio-egressgateway-with-sni-proxy
kubectl delete virtualservice direct-wikipedia-through-egress-gateway
kubectl delete destinationrule egressgateway-for-wikipedia
kubectl delete --ignore-not-found=true envoyfilter forward-downstream-sni egress-gateway-sni-verifier
}
snip_cleanup_wildcard_configuration_for_arbitrary_domains_2() {
kubectl delete serviceentry sni-proxy
kubectl delete destinationrule disable-mtls-for-sni-proxy
kubectl delete configmap egress-sni-proxy-configmap -n istio-system
}
snip_cleanup_wildcard_configuration_for_arbitrary_domains_3() {
rm ./sni-proxy.conf ./egressgateway-with-sni-proxy.yaml ./egressgateway-with-sni-proxy-patch.yaml
}
snip_cleanup_1() {
kubectl delete -f samples/sleep/sleep.yaml
istioctl x uninstall
}

View File

@ -0,0 +1,115 @@
#!/usr/bin/env bash
# shellcheck disable=SC1090,SC2154,SC2155
# Copyright 2020 Istio Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# @setup profile=none
set -e
set -u
set -o pipefail
snip_before_you_begin_1
_wait_for_deployment istio-system istiod
kubectl label namespace default istio-injection=enabled --overwrite
snip_before_you_begin_2
snip_before_you_begin_4
confirm_blocking() {
kubectl exec "$SOURCE_POD" -c sleep -- curl -I https://www.google.com | grep "HTTP/"; kubectl exec "$SOURCE_POD" -c sleep -- curl -I https://edition.cnn.com | grep "HTTP/"
}
_verify_contains confirm_blocking "command terminated with exit code 35"
snip_configure_direct_traffic_to_a_wildcard_host_1
_wait_for_istio serviceentry default wikipedia
_verify_same snip_configure_direct_traffic_to_a_wildcard_host_2 "$snip_configure_direct_traffic_to_a_wildcard_host_2_out"
snip_cleanup_direct_traffic_to_a_wildcard_host_1
snip_wildcard_configuration_for_a_single_hosting_server_1
_wait_for_istio gateway default istio-egressgateway
_wait_for_istio destinationrule default egressgateway-for-wikipedia
_wait_for_istio virtualservice default direct-wikipedia-through-egress-gateway
snip_wildcard_configuration_for_a_single_hosting_server_2
_wait_for_istio serviceentry default www-wikipedia
_verify_same snip_wildcard_configuration_for_a_single_hosting_server_3 "$snip_wildcard_configuration_for_a_single_hosting_server_3_out"
_verify_contains snip_wildcard_configuration_for_a_single_hosting_server_4 "outbound|443||www.wikipedia.org"
snip_cleanup_wildcard_configuration_for_a_single_hosting_server_1
snip_setup_egress_gateway_with_sni_proxy_1
snip_setup_egress_gateway_with_sni_proxy_2
snip_setup_egress_gateway_with_sni_proxy_3
snip_setup_egress_gateway_with_sni_proxy_4
snip_setup_egress_gateway_with_sni_proxy_5
_verify_same snip_setup_egress_gateway_with_sni_proxy_6 "$snip_setup_egress_gateway_with_sni_proxy_6_out"
_verify_like snip_setup_egress_gateway_with_sni_proxy_7 "$snip_setup_egress_gateway_with_sni_proxy_7_out"
_wait_for_deployment istio-system istio-egressgateway-with-sni-proxy
snip_setup_egress_gateway_with_sni_proxy_8
_wait_for_istio serviceentry default sni-proxy
_wait_for_istio destinationrule default disable-mtls-for-sni-proxy
snip_configure_traffic_through_egress_gateway_with_sni_proxy_1
_wait_for_istio serviceentry default wikipedia
snip_configure_traffic_through_egress_gateway_with_sni_proxy_2
_wait_for_istio gateway default istio-egressgateway-with-sni-proxy
_wait_for_istio destinationrule default egressgateway-for-wikipedia
_wait_for_istio virtualservice default direct-wikipedia-through-egress-gateway
_wait_for_istio envoyfilter default forward-downstream-sni
snip_configure_traffic_through_egress_gateway_with_sni_proxy_3
_wait_for_istio envoyfilter istio-system egress-gateway-sni-verifier
_verify_same snip_configure_traffic_through_egress_gateway_with_sni_proxy_4 "$snip_configure_traffic_through_egress_gateway_with_sni_proxy_4_out"
_verify_lines snip_configure_traffic_through_egress_gateway_with_sni_proxy_5 "
+ outbound|8443||sni-proxy.local
+ en.wikipedia.org
+ de.wikipedia.org
"
_verify_lines snip_configure_traffic_through_egress_gateway_with_sni_proxy_7 "
+ TCP [en.wikipedia.org]200
+ TCP [de.wikipedia.org]200
"
# @cleanup
set +e # ignore cleanup errors
snip_cleanup_direct_traffic_to_a_wildcard_host_1
snip_cleanup_wildcard_configuration_for_a_single_hosting_server_1
snip_cleanup_wildcard_configuration_for_arbitrary_domains_1
snip_cleanup_wildcard_configuration_for_arbitrary_domains_2
snip_cleanup_wildcard_configuration_for_arbitrary_domains_3
snip_cleanup_1
kubectl delete ns istio-system
kubectl label namespace default istio-injection-