Use standard ports for telemetry exposure (#7069)

* Use standard ports for telemetry exposure

Blocker for https://github.com/istio/istio/issues/22911

* Full update and include HTTPS

* Fix link

* Remove from ports table

* Apply suggestions from code review

Co-Authored-By: Rachael Graham <rachael.graham@ibm.com>

Co-authored-by: Rachael Graham <rachael.graham@ibm.com>
This commit is contained in:
John Howard 2020-04-16 13:20:33 -07:00 committed by GitHub
parent 83de8ae304
commit 9af39e2f0e
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 75 additions and 157 deletions

View File

@ -73,10 +73,6 @@ The following ports and protocols are used by Istio.
| 8080 | HTTP | Istiod | Debug interface | | 8080 | HTTP | Istiod | Debug interface |
| 443 | HTTPS | Istiod | Webhooks | | 443 | HTTPS | Istiod | Webhooks |
| 15014 | HTTP | Citadel, Citadel agent, Galley, Mixer, Istiod, Sidecar Injector | Control plane monitoring | | 15014 | HTTP | Citadel, Citadel agent, Galley, Mixer, Istiod, Sidecar Injector | Control plane monitoring |
| 15029 | HTTP | Kiali | Kiali User Interface |
| 15030 | HTTP | Prometheus | Prometheus User Interface |
| 15031 | HTTP | Grafana | Grafana User Interface |
| 15032 | HTTP | Tracing | Tracing User Interface |
| 15443 | TLS | Ingress and Egress Gateways | SNI | | 15443 | TLS | Ingress and Egress Gateways | SNI |
| 9090 | HTTP | Prometheus | Prometheus | | 9090 | HTTP | Prometheus | Prometheus |
| 42422 | TCP | Mixer | Telemetry - Prometheus | | 42422 | TCP | Mixer | Telemetry - Prometheus |

View File

@ -17,21 +17,7 @@ two basic access methods: secure (via HTTPS) and insecure (via HTTP). The secure
recommended* for any production or sensitive environment. Insecure access is simpler to set up, but recommended* for any production or sensitive environment. Insecure access is simpler to set up, but
will not protect any credentials or data transmitted outside of your cluster. will not protect any credentials or data transmitted outside of your cluster.
### Option 1: Secure access (HTTPS) For both options, first follow these steps:
A server certificate is required for secure access. Follow these steps to install and configure
server certificates for a domain that you control.
You may use self-signed certificates instead. Visit our
[Securing Gateways with HTTPS](/docs/tasks/traffic-management/ingress/secure-ingress/)
for general information on using self-signed certificates to access in-cluster services.
{{< warning >}}
This option covers securing the transport layer *only*. You should also configure the telemetry
addons to require authentication when exposing them externally.
{{< /warning >}}
1. [Install cert-manager](/docs/ops/integrations/certmanager/) to manage certificates automatically.
1. [Install Istio](/docs/setup/install/istioctl) in your cluster. 1. [Install Istio](/docs/setup/install/istioctl) in your cluster.
@ -42,71 +28,43 @@ addons to require authentication when exposing them externally.
* Prometheus: `--set values.prometheus.enabled=true` * Prometheus: `--set values.prometheus.enabled=true`
* Tracing: `--set values.tracing.enabled=true` * Tracing: `--set values.tracing.enabled=true`
1. Configure the DNS records for your domain. 1. Set up the domain to expose addons. In this example, you expose each addon on a subdomain, such as `grafana.example.com`.
1. Get the external IP address of the `istio-ingressgateway`. * If you have an existing domain pointing to the external IP address of `istio-ingressgateway`:
{{< text bash >}} {{< text bash >}}
$ kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}' $ export INGRESS_DOMAIN=<your.desired.domain>
<IP ADDRESS OF CLUSTER INGRESS>
{{< /text >}} {{< /text >}}
1. Set an environment variable to hold your target domain. * If you do not have a domain, you may use [`nip.io`](https://nip.io/) which will automatically resolve to the IP address provided. This is not recommended for production usage.
{{< text bash >}} {{< text bash >}}
$ TELEMETRY_DOMAIN=<your.desired.domain> $ export INGRESS_HOST=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
$ export INGRESS_DOMAIN=${INGRESS_HOST}.nip.io
{{< /text >}} {{< /text >}}
1. Point your desired domain at that external IP address via your domain provider. ### Option 1: Secure access (HTTPS)
The mechanism for achieving this step varies by provider. Here are a few example documentation links: A server certificate is required for secure access. Follow these steps to install and configure
server certificates for a domain that you control.
* Bluehost: [DNS Management Add Edit or Delete DNS Entries](https://my.bluehost.com/hosting/help/559) {{< warning >}}
* GoDaddy: [Add an A record](https://www.godaddy.com/help/add-an-a-record-19238) This option covers securing the transport layer *only*. You should also configure the telemetry
* Google Domains: [Resource Records](https://support.google.com/domains/answer/3290350?hl=en) addons to require authentication when exposing them externally.
* Name.com: [Adding an A record](https://www.name.com/support/articles/115004893508-Adding-an-A-record) {{< /warning >}}
1. Verify that the DNS records are correct. This example uses self-signed certificates, which may not be appropriate for production usages. For these cases, consider using [cert-manager](/docs/ops/integrations/certmanager/) or other tools to provision certificates. You may also visit the [Securing Gateways with HTTPS](/docs/tasks/traffic-management/ingress/secure-ingress/) task for general information on using HTTPS on the gateway.
{{< text bash >}} 1. Setup the certificates. This example uses `openssl` to self sign.
$ dig +short $TELEMETRY_DOMAIN
<IP ADDRESS OF CLUSTER INGRESS>
{{< /text >}}
1. Generate a server certificate {{< text bash >}}
$ CERT_DIR=/tmp/certs
{{< text bash >}} $ mkdir -p ${CERT_DIR}
$ cat <<EOF | kubectl apply -f - $ openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 -subj '/O=example Inc./CN=*.${INGRESS_DOMAIN}' -keyout ${CERT_DIR}/ca.key -out ${CERT_DIR}/ca.crt
apiVersion: cert-manager.io/v1alpha2 $ openssl req -out ${CERT_DIR}/cert.csr -newkey rsa:2048 -nodes -keyout ${CERT_DIR}/tls.key -subj "/CN=*.${INGRESS_DOMAIN}/O=example organization"
kind: Certificate $ openssl x509 -req -days 365 -CA ${CERT_DIR}/ca.crt -CAkey ${CERT_DIR}/ca.key -set_serial 0 -in ${CERT_DIR}/cert.csr -out ${CERT_DIR}/tls.crt
metadata: $ kubectl create -n istio-system secret tls telemetry-gw-cert --key=${CERT_DIR}/tls.key --cert=${CERT_DIR}/tls.crt
name: telemetry-gw-cert {{< /text >}}
namespace: istio-system
spec:
secretName: telemetry-gw-cert
issuerRef:
name: letsencrypt
kind: ClusterIssuer
commonName: $TELEMETRY_DOMAIN
dnsNames:
- $TELEMETRY_DOMAIN
acme:
config:
- http01:
ingressClass: istio
domains:
- $TELEMETRY_DOMAIN
---
EOF
certificate.cert-manager.io "telemetry-gw-cert" created
{{< /text >}}
1. Wait until the server certificate is ready.
{{< text syntax="bash" expandlinks="false" >}}
$ JSONPATH='{range .items[*]}{@.metadata.name}:{range @.status.conditions[*]}{@.type}={@.status}{end}{end}' && kubectl -n istio-system get certificates.cert-manager.io -o jsonpath="$JSONPATH"
telemetry-gw-cert:Ready=True
{{< /text >}}
1. Apply networking configuration for the telemetry addons. 1. Apply networking configuration for the telemetry addons.
@ -124,16 +82,14 @@ addons to require authentication when exposing them externally.
istio: ingressgateway istio: ingressgateway
servers: servers:
- port: - port:
number: 15031 number: 443
name: https-grafana name: https-grafana
protocol: HTTPS protocol: HTTPS
tls: tls:
mode: SIMPLE mode: SIMPLE
serverCertificate: sds
privateKey: sds
credentialName: telemetry-gw-cert credentialName: telemetry-gw-cert
hosts: hosts:
- "$TELEMETRY_DOMAIN" - "grafana.${INGRESS_DOMAIN}"
--- ---
apiVersion: networking.istio.io/v1alpha3 apiVersion: networking.istio.io/v1alpha3
kind: VirtualService kind: VirtualService
@ -142,13 +98,11 @@ addons to require authentication when exposing them externally.
namespace: istio-system namespace: istio-system
spec: spec:
hosts: hosts:
- "$TELEMETRY_DOMAIN" - "grafana.${INGRESS_DOMAIN}"
gateways: gateways:
- grafana-gateway - grafana-gateway
http: http:
- match: - route:
- port: 15031
route:
- destination: - destination:
host: grafana host: grafana
port: port:
@ -185,16 +139,14 @@ addons to require authentication when exposing them externally.
istio: ingressgateway istio: ingressgateway
servers: servers:
- port: - port:
number: 15029 number: 443
name: https-kiali name: https-kiali
protocol: HTTPS protocol: HTTPS
tls: tls:
mode: SIMPLE mode: SIMPLE
serverCertificate: sds
privateKey: sds
credentialName: telemetry-gw-cert credentialName: telemetry-gw-cert
hosts: hosts:
- "$TELEMETRY_DOMAIN" - "kiali.${INGRESS_DOMAIN}"
--- ---
apiVersion: networking.istio.io/v1alpha3 apiVersion: networking.istio.io/v1alpha3
kind: VirtualService kind: VirtualService
@ -203,13 +155,11 @@ addons to require authentication when exposing them externally.
namespace: istio-system namespace: istio-system
spec: spec:
hosts: hosts:
- "$TELEMETRY_DOMAIN" - "kiali.${INGRESS_DOMAIN}"
gateways: gateways:
- kiali-gateway - kiali-gateway
http: http:
- match: - route:
- port: 15029
route:
- destination: - destination:
host: kiali host: kiali
port: port:
@ -246,16 +196,14 @@ addons to require authentication when exposing them externally.
istio: ingressgateway istio: ingressgateway
servers: servers:
- port: - port:
number: 15030 number: 443
name: https-prom name: https-prom
protocol: HTTPS protocol: HTTPS
tls: tls:
mode: SIMPLE mode: SIMPLE
serverCertificate: sds
privateKey: sds
credentialName: telemetry-gw-cert credentialName: telemetry-gw-cert
hosts: hosts:
- "$TELEMETRY_DOMAIN" - "prometheus.${INGRESS_DOMAIN}"
--- ---
apiVersion: networking.istio.io/v1alpha3 apiVersion: networking.istio.io/v1alpha3
kind: VirtualService kind: VirtualService
@ -264,13 +212,11 @@ addons to require authentication when exposing them externally.
namespace: istio-system namespace: istio-system
spec: spec:
hosts: hosts:
- "$TELEMETRY_DOMAIN" - "prometheus.${INGRESS_DOMAIN}"
gateways: gateways:
- prometheus-gateway - prometheus-gateway
http: http:
- match: - route:
- port: 15030
route:
- destination: - destination:
host: prometheus host: prometheus
port: port:
@ -307,16 +253,14 @@ addons to require authentication when exposing them externally.
istio: ingressgateway istio: ingressgateway
servers: servers:
- port: - port:
number: 15032 number: 443
name: https-tracing name: https-tracing
protocol: HTTPS protocol: HTTPS
tls: tls:
mode: SIMPLE mode: SIMPLE
serverCertificate: sds
privateKey: sds
credentialName: telemetry-gw-cert credentialName: telemetry-gw-cert
hosts: hosts:
- "$TELEMETRY_DOMAIN" - "tracing.${INGRESS_DOMAIN}"
--- ---
apiVersion: networking.istio.io/v1alpha3 apiVersion: networking.istio.io/v1alpha3
kind: VirtualService kind: VirtualService
@ -325,13 +269,11 @@ addons to require authentication when exposing them externally.
namespace: istio-system namespace: istio-system
spec: spec:
hosts: hosts:
- "$TELEMETRY_DOMAIN" - "tracing.${INGRESS_DOMAIN}"
gateways: gateways:
- tracing-gateway - tracing-gateway
http: http:
- match: - route:
- port: 15032
route:
- destination: - destination:
host: tracing host: tracing
port: port:
@ -356,22 +298,17 @@ addons to require authentication when exposing them externally.
1. Visit the telemetry addons via your browser. 1. Visit the telemetry addons via your browser.
* Kiali: `https://$TELEMETRY_DOMAIN:15029/` {{< warning >}}
* Prometheus: `https://$TELEMETRY_DOMAIN:15030/` If you used self signed certificates, your browser will likely mark them as insecure.
* Grafana: `https://$TELEMETRY_DOMAIN:15031/` {{< /warning >}}
* Tracing: `https://$TELEMETRY_DOMAIN:15032/`
* Kiali: `https://kiali.${INGRESS_DOMAIN}`
* Prometheus: `https://prometheus.${INGRESS_DOMAIN}`
* Grafana: `https://grafana.${INGRESS_DOMAIN}`
* Tracing: `https://tracing.${INGRESS_DOMAIN}`
### Option 2: Insecure access (HTTP) ### Option 2: Insecure access (HTTP)
1. [Install Istio](/docs/setup/install/istioctl) in your cluster with your desired telemetry addons.
To additionally install the telemetry addons, use the following installation options:
* Grafana: `--set values.grafana.enabled=true`
* Kiali: `--set values.kiali.enabled=true`
* Prometheus: `--set values.prometheus.enabled=true`
* Tracing: `--set values.tracing.enabled=true`
1. Apply networking configuration for the telemetry addons. 1. Apply networking configuration for the telemetry addons.
1. Apply the following configuration to expose Grafana: 1. Apply the following configuration to expose Grafana:
@ -388,11 +325,11 @@ addons to require authentication when exposing them externally.
istio: ingressgateway istio: ingressgateway
servers: servers:
- port: - port:
number: 15031 number: 80
name: http-grafana name: http-grafana
protocol: HTTP protocol: HTTP
hosts: hosts:
- "*" - "grafana.${INGRESS_DOMAIN}"
--- ---
apiVersion: networking.istio.io/v1alpha3 apiVersion: networking.istio.io/v1alpha3
kind: VirtualService kind: VirtualService
@ -401,13 +338,11 @@ addons to require authentication when exposing them externally.
namespace: istio-system namespace: istio-system
spec: spec:
hosts: hosts:
- "*" - "grafana.${INGRESS_DOMAIN}"
gateways: gateways:
- grafana-gateway - grafana-gateway
http: http:
- match: - route:
- port: 15031
route:
- destination: - destination:
host: grafana host: grafana
port: port:
@ -444,11 +379,11 @@ addons to require authentication when exposing them externally.
istio: ingressgateway istio: ingressgateway
servers: servers:
- port: - port:
number: 15029 number: 80
name: http-kiali name: http-kiali
protocol: HTTP protocol: HTTP
hosts: hosts:
- "*" - "kiali.${INGRESS_DOMAIN}"
--- ---
apiVersion: networking.istio.io/v1alpha3 apiVersion: networking.istio.io/v1alpha3
kind: VirtualService kind: VirtualService
@ -457,13 +392,11 @@ addons to require authentication when exposing them externally.
namespace: istio-system namespace: istio-system
spec: spec:
hosts: hosts:
- "*" - "kiali.${INGRESS_DOMAIN}"
gateways: gateways:
- kiali-gateway - kiali-gateway
http: http:
- match: - route:
- port: 15029
route:
- destination: - destination:
host: kiali host: kiali
port: port:
@ -500,11 +433,11 @@ addons to require authentication when exposing them externally.
istio: ingressgateway istio: ingressgateway
servers: servers:
- port: - port:
number: 15030 number: 80
name: http-prom name: http-prom
protocol: HTTP protocol: HTTP
hosts: hosts:
- "*" - "prometheus.${INGRESS_DOMAIN}"
--- ---
apiVersion: networking.istio.io/v1alpha3 apiVersion: networking.istio.io/v1alpha3
kind: VirtualService kind: VirtualService
@ -513,13 +446,11 @@ addons to require authentication when exposing them externally.
namespace: istio-system namespace: istio-system
spec: spec:
hosts: hosts:
- "*" - "prometheus.${INGRESS_DOMAIN}"
gateways: gateways:
- prometheus-gateway - prometheus-gateway
http: http:
- match: - route:
- port: 15030
route:
- destination: - destination:
host: prometheus host: prometheus
port: port:
@ -556,11 +487,11 @@ addons to require authentication when exposing them externally.
istio: ingressgateway istio: ingressgateway
servers: servers:
- port: - port:
number: 15032 number: 80
name: http-tracing name: http-tracing
protocol: HTTP protocol: HTTP
hosts: hosts:
- "*" - "tracing.${INGRESS_DOMAIN}"
--- ---
apiVersion: networking.istio.io/v1alpha3 apiVersion: networking.istio.io/v1alpha3
kind: VirtualService kind: VirtualService
@ -569,13 +500,11 @@ addons to require authentication when exposing them externally.
namespace: istio-system namespace: istio-system
spec: spec:
hosts: hosts:
- "*" - "tracing.${INGRESS_DOMAIN}"
gateways: gateways:
- tracing-gateway - tracing-gateway
http: http:
- match: - route:
- port: 15032
route:
- destination: - destination:
host: tracing host: tracing
port: port:
@ -600,10 +529,10 @@ addons to require authentication when exposing them externally.
1. Visit the telemetry addons via your browser. 1. Visit the telemetry addons via your browser.
* Kiali: `http://<IP ADDRESS OF CLUSTER INGRESS>:15029/` * Kiali: `http://kiali.${INGRESS_DOMAIN}`
* Prometheus: `http://<IP ADDRESS OF CLUSTER INGRESS>:15030/` * Prometheus: `http://prometheus.${INGRESS_DOMAIN}`
* Grafana: `http://<IP ADDRESS OF CLUSTER INGRESS>:15031/` * Grafana: `http://grafana.${INGRESS_DOMAIN}`
* Tracing: `http://<IP ADDRESS OF CLUSTER INGRESS>:15032/` * Tracing: `http://tracing.${INGRESS_DOMAIN}`
## Cleanup ## Cleanup
@ -626,10 +555,3 @@ addons to require authentication when exposing them externally.
virtualservice.networking.istio.io "prometheus-vs" deleted virtualservice.networking.istio.io "prometheus-vs" deleted
virtualservice.networking.istio.io "tracing-vs" deleted virtualservice.networking.istio.io "tracing-vs" deleted
{{< /text >}} {{< /text >}}
* If installed, remove the gateway certificate:
{{< text bash >}}
$ kubectl -n istio-system delete certificates.cert-manager.io telemetry-gw-cert
certificate.cert-manager.io "telemetry-gw-cert" deleted
{{< /text >}}