From 6ba9f2c86ebc2834db4a23d29aee4399670652ac Mon Sep 17 00:00:00 2001 From: Douglas Reid Date: Wed, 26 Feb 2020 14:55:17 -0800 Subject: [PATCH] fix(observability docs): restructure docs for v2 telemetry (and remove problematic Mixer refs) (#6533) * wip: setup observability tasks for v2 Signed-off-by: Douglas Reid * continue work Signed-off-by: Douglas Reid * lint fix Signed-off-by: Douglas Reid * remove mixer ref from what-is-istio Signed-off-by: Douglas Reid * further cleanup Signed-off-by: Douglas Reid * lint fix Signed-off-by: Douglas Reid * when will the linting stop? Signed-off-by: Douglas Reid * Update content/en/docs/tasks/observability/mixer/_index.md Co-Authored-By: Frank Budinsky Co-authored-by: Frank Budinsky --- content/en/about/bugs/index.md | 8 -- content/en/about/feature-stages/index.md | 4 +- .../egress-monitoring-access-control/index.md | 2 +- .../en/docs/concepts/observability/index.md | 18 ++--- .../en/docs/concepts/what-is-istio/index.md | 4 - .../microservices-istio/logs-istio/index.md | 47 +---------- .../in-proxy-service-telemetry/index.md | 77 ------------------- .../en/docs/setup/getting-started/index.md | 2 +- .../docs/tasks/observability/logs/_index.md | 2 +- .../tasks/observability/metrics/_index.md | 2 +- .../metrics/querying-metrics/index.md | 16 +--- .../docs/tasks/observability/mixer/_index.md | 8 ++ .../tasks/observability/mixer/logs/_index.md | 6 ++ .../{ => mixer}/logs/collecting-logs/index.md | 3 +- .../{ => mixer}/logs/fluentd/index.md | 3 +- .../observability/mixer/metrics/_index.md | 6 ++ .../metrics/collecting-metrics/index.md | 1 + .../{ => mixer}/metrics/tcp-metrics/index.md | 3 +- .../tcp-metrics/istio-tcp-attribute-flow.svg | 0 .../news/releases/0.x/announcing-0.5/index.md | 2 +- .../releases/1.3.x/announcing-1.3.1/index.md | 2 +- .../announcing-1.4/change-notes/index.md | 2 +- 22 files changed, 47 insertions(+), 171 deletions(-) delete mode 100644 content/en/docs/ops/configuration/telemetry/in-proxy-service-telemetry/index.md create mode 100644 content/en/docs/tasks/observability/mixer/_index.md create mode 100644 content/en/docs/tasks/observability/mixer/logs/_index.md rename content/en/docs/tasks/observability/{ => mixer}/logs/collecting-logs/index.md (98%) rename content/en/docs/tasks/observability/{ => mixer}/logs/fluentd/index.md (99%) create mode 100644 content/en/docs/tasks/observability/mixer/metrics/_index.md rename content/en/docs/tasks/observability/{ => mixer}/metrics/collecting-metrics/index.md (99%) rename content/en/docs/tasks/observability/{ => mixer}/metrics/tcp-metrics/index.md (98%) rename content/en/docs/tasks/observability/{ => mixer}/metrics/tcp-metrics/istio-tcp-attribute-flow.svg (100%) diff --git a/content/en/about/bugs/index.md b/content/en/about/bugs/index.md index efcf454517..03c8c7d765 100644 --- a/content/en/about/bugs/index.md +++ b/content/en/about/bugs/index.md @@ -66,14 +66,6 @@ containing: * Current and previous logs from all Istio components and sidecar -* Mixer logs: - - {{< text bash >}} - $ kubectl logs -n istio-system -l istio=mixer -c mixer - $ kubectl logs -n istio-system -l istio=policy -c mixer - $ kubectl logs -n istio-system -l istio=telemetry -c mixer - {{< /text >}} - * Pilot logs: {{< text bash >}} diff --git a/content/en/about/feature-stages/index.md b/content/en/about/feature-stages/index.md index 803e20995d..d9e2114ace 100644 --- a/content/en/about/feature-stages/index.md +++ b/content/en/about/feature-stages/index.md @@ -54,7 +54,7 @@ Below is our list of existing features and their current phases. This informatio | Feature | Phase |-------------------|------------------- | [Prometheus Integration](/docs/tasks/observability/metrics/querying-metrics/) | Stable -| [Local Logging (STDIO)](/docs/tasks/observability/logs/collecting-logs/) | Stable +| [Local Logging (STDIO)](/docs/tasks/observability/mixer/logs/collecting-logs/) | Stable | [Statsd Integration](/docs/reference/config/policy-and-telemetry/adapters/statsd/) | Stable | [Client and Server Telemetry Reporting](/docs/reference/config/policy-and-telemetry/) | Stable | [Service Dashboard in Grafana](/docs/tasks/observability/metrics/using-istio-dashboard/) | Stable @@ -62,7 +62,7 @@ Below is our list of existing features and their current phases. This informatio | [Distributed Tracing](/docs/tasks/observability/distributed-tracing/) | Stable | [Stackdriver Integration](/docs/reference/config/policy-and-telemetry/adapters/stackdriver/) | Beta | [Distributed Tracing to Zipkin / Jaeger](/docs/tasks/observability/distributed-tracing/) | Beta -| [Logging with Fluentd](/docs/tasks/observability/logs/fluentd/) | Beta +| [Logging with Fluentd](/docs/tasks/observability/mixer/logs/fluentd/) | Beta | [Trace Sampling](/docs/tasks/observability/distributed-tracing/overview/#trace-sampling) | Beta ### Security and policy enforcement diff --git a/content/en/blog/2018/egress-monitoring-access-control/index.md b/content/en/blog/2018/egress-monitoring-access-control/index.md index 503a348236..60929b97de 100644 --- a/content/en/blog/2018/egress-monitoring-access-control/index.md +++ b/content/en/blog/2018/egress-monitoring-access-control/index.md @@ -40,7 +40,7 @@ will prevent any possibility for a malicious application to access the forbidden * The [Egress Gateway with TLS Origination](/docs/tasks/traffic-management/egress/egress-gateway-tls-origination/) example demonstrates how to allow applications to send HTTP requests to external servers that require HTTPS, while directing traffic through egress gateway. -* The [Collecting Metrics](/docs/tasks/observability/metrics/collecting-metrics/) task describes how to configure metrics for services in a mesh. +* The [Collecting Metrics](/docs/tasks/observability/mixer/metrics/collecting-metrics/) task describes how to configure metrics for services in a mesh. * The [Visualizing Metrics with Grafana](/docs/tasks/observability/metrics/using-istio-dashboard/) describes the Istio Dashboard to monitor mesh traffic. * The [Basic Access Control](/docs/tasks/policy-enforcement/denial-and-list/) task shows how to control access to diff --git a/content/en/docs/concepts/observability/index.md b/content/en/docs/concepts/observability/index.md index c5ecf407f4..abb1507f3d 100644 --- a/content/en/docs/concepts/observability/index.md +++ b/content/en/docs/concepts/observability/index.md @@ -37,9 +37,6 @@ behaviors such as the overall volume of traffic, the error rates within the traf In addition to monitoring the behavior of services within a mesh, it is also important to monitor the behavior of the mesh itself. Istio components export metrics on their own internal behaviors to provide insight on the health and function of the mesh control plane. -Istio metrics collection is driven by operator configuration. Operators select how and when to collect metrics, as well as how detailed the metrics themselves -should be. This enables operators to flexibly tune metrics collection to meet their individual needs. - ### Proxy-level metrics Istio metrics collection begins with the sidecar proxies (Envoy). Each proxy generates a rich set of metrics about all traffic passing through the proxy (both @@ -77,11 +74,8 @@ In addition to the proxy-level metrics, Istio provides a set of service-oriented basic service monitoring needs: latency, traffic, errors, and saturation. Istio ships with a default set of [dashboards](/docs/tasks/observability/metrics/using-istio-dashboard/) for monitoring service behaviors based on these metrics. -The [default Istio metrics](/docs/reference/config/policy-and-telemetry/metrics/) are defined by a set of configuration artifacts that ship with Istio and are -exported to [Prometheus](/docs/reference/config/policy-and-telemetry/adapters/prometheus/) by default. Operators are free to modify the -shape and content of these metrics, as well as to change their collection mechanism, to meet their individual monitoring needs. - -The [Collecting Metrics](/docs/tasks/observability/metrics/collecting-metrics/) task provides more information on customizing Istio metrics generation. +The [standard Istio metrics](/docs/reference/config/policy-and-telemetry/metrics/) are +exported to [Prometheus](/docs/reference/config/policy-and-telemetry/adapters/prometheus/) by default. Use of the service-level metrics is entirely optional. Operators may choose to turn off generation and collection of these metrics to meet their individual needs. @@ -92,6 +86,8 @@ Example service-level metric: istio_requests_total{ connection_security_policy="mutual_tls", destination_app="details", + destination_canonical_service="details", + destination_canonical_revision="v1", destination_principal="cluster.local/ns/default/sa/default", destination_service="details.default.svc.cluster.local", destination_service_name="details", @@ -104,6 +100,8 @@ istio_requests_total{ response_code="200", response_flags="-", source_app="productpage", + source_canonical_service="productpage", + source_canonical_revision="v1", source_principal="cluster.local/ns/default/sa/default", source_version="v1", source_workload="productpage-v1", @@ -149,10 +147,6 @@ Access logs provide a way to monitor and understand behavior from the perspectiv Istio can generate access logs for service traffic in a configurable set of formats, providing operators with full control of the how, what, when and where of logging. Istio exposes a full set of source and destination metadata to the access logging mechanisms, allowing detailed audit of network transactions. -Access logs may be generated locally or exported to custom backends, including [Fluentd](/docs/tasks/observability/logs/fluentd/). - -More information on access logging is provided in the [Collecting Logs](/docs/tasks/observability/logs/collecting-logs/) and the [Getting Envoy's Access Logs](/docs/tasks/observability/logs/access-log/) tasks. - Example Istio access log (formatted in JSON): {{< text json >}} diff --git a/content/en/docs/concepts/what-is-istio/index.md b/content/en/docs/concepts/what-is-istio/index.md index a52fa5ac54..d959d8efbf 100644 --- a/content/en/docs/concepts/what-is-istio/index.md +++ b/content/en/docs/concepts/what-is-istio/index.md @@ -92,10 +92,6 @@ Istio’s robust tracing, monitoring, and logging features give you deep insight impacts things upstream and downstream with Istio’s monitoring features, while its custom dashboards provide visibility into the performance of all your services and let you see how that performance is affecting your other processes. -Istio’s Mixer component is responsible for policy controls and telemetry collection. It provides backend abstraction and intermediation, insulating the rest of -Istio from the implementation details of individual infrastructure backends, and giving operators fine-grained control over all interactions between the mesh -and infrastructure backends. - All these features let you more effectively set, monitor, and enforce SLOs on services. Of course, the bottom line is that you can detect and fix issues quickly and efficiently. diff --git a/content/en/docs/examples/microservices-istio/logs-istio/index.md b/content/en/docs/examples/microservices-istio/logs-istio/index.md index 9b565b4ec2..dd8cc29ac2 100644 --- a/content/en/docs/examples/microservices-istio/logs-istio/index.md +++ b/content/en/docs/examples/microservices-istio/logs-istio/index.md @@ -1,51 +1,14 @@ --- -title: Logging with Istio -overview: Collecting and querying logs. +title: Monitoring with Istio +overview: Collecting and querying mesh metrics. weight: 72 --- -Logs and monitoring are important aspects of microservice architecture and are crucial to support transitioning to the microservices architecture style. Other requirements include rapid provisioning and rapid deployment, according to [this article](https://aadrake.com/posts/2017-05-20-enough-with-the-microservices.html). +Monitoring is crucial to support transitioning to the microservices architecture style. Other requirements include rapid provisioning and rapid deployment, according to [this article](https://aadrake.com/posts/2017-05-20-enough-with-the-microservices.html). -With Istio, you gain monitoring and logging of the traffic between microservices by default. +With Istio, you gain monitoring of the traffic between microservices by default. You can use the Istio Dashboard for monitoring your microservices in real time. -In this module, you will learn how to inspect and query logs related to the traffic between your microservices. - -1. Store the name of your namespace in the `NAMESPACE` environment variable. - You will need it to recognize your microservices in the logs: - - {{< text bash >}} - $ export NAMESPACE=$(kubectl config view -o jsonpath="{.contexts[?(@.name == \"$(kubectl config current-context)\")].context.namespace}") - $ echo $NAMESPACE - tutorial - {{< /text >}} - -1. View the access log entries, related to your namespace: - - {{< text bash >}} - $ kubectl -n istio-system logs -l istio-mixer-type=telemetry -c mixer | grep "accesslog.instance.istio-system" | grep "\"destinationNamespace\": \"$NAMESPACE\"" | grep '"reporter": "destination"' - ... - {"level":"info","time":"2019-02-17T06:49:14.078599Z","instance":"accesslog.logentry.istio-system","apiClaims":"","apiKey":"","clientTraceId":"","connection_security_policy":"mutual_tls","destinationApp":"details","destinationIp":"AAAAAAAAAAAAAP//rB5taA==","destinationName":"details-v1-58c68b9ff-l287q","destinationNamespace":"tutorial","destinationOwner":"kubernetes://apis/apps/v1/namespaces/tutorial/deployments/details-v1","destinationPrincipal":"cluster.local/ns/tutorial/sa/default","destinationServiceHost":"details.tutorial.svc.cluster.local","destinationWorkload":"details-v1","grpcMessage":"","grpcStatus":"","httpAuthority":"details:9080","latency":"2.403666ms","method":"GET","permissiveResponseCode":"none","permissiveResponsePolicyID":"none","protocol":"http","receivedBytes":1023,"referer":"","reporter":"destination","requestId":"b2094a51-367f-9e6c-8cfb-9f9a1166076c","requestSize":0,"requestedServerName":"outbound_.9080_._.details.tutorial.svc.cluster.local","responseCode":200,"responseFlags":"-","responseSize":178,"responseTimestamp":"2019-02-17T06:49:14.080840Z","sentBytes":313,"sourceApp":"productpage","sourceIp":"AAAAAAAAAAAAAP//rB5tTg==","sourceName":"productpage-v1-59b4f9f8d5-5z6r9","sourceNamespace":"tutorial","sourceOwner":"kubernetes://apis/apps/v1/namespaces/tutorial/deployments/productpage-v1","sourcePrincipal":"cluster.local/ns/tutorial/sa/default","sourceWorkload":"productpage-v1","url":"/details/0","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.0 Safari/605.1.15","xForwardedFor":"0.0.0.0"} - {"level":"info","time":"2019-02-17T06:49:14.094427Z","instance":"accesslog.logentry.istio-system","apiClaims":"","apiKey":"","clientTraceId":"","connection_security_policy":"mutual_tls","destinationApp":"ratings","destinationIp":"AAAAAAAAAAAAAP//rB6STA==","destinationName":"ratings-v1-b7b7fbbc9-c6lh5","destinationNamespace":"tutorial","destinationOwner":"kubernetes://apis/apps/v1/namespaces/tutorial/deployments/ratings-v1","destinationPrincipal":"cluster.local/ns/tutorial/sa/default","destinationServiceHost":"ratings.tutorial.svc.cluster.local","destinationWorkload":"ratings-v1","grpcMessage":"","grpcStatus":"","httpAuthority":"ratings:9080","latency":"956.847µs","method":"GET","permissiveResponseCode":"none","permissiveResponsePolicyID":"none","protocol":"http","receivedBytes":1039,"referer":"","reporter":"destination","requestId":"b2094a51-367f-9e6c-8cfb-9f9a1166076c","requestSize":0,"requestedServerName":"outbound_.9080_._.ratings.tutorial.svc.cluster.local","responseCode":200,"responseFlags":"-","responseSize":48,"responseTimestamp":"2019-02-17T06:49:14.095225Z","sentBytes":166,"sourceApp":"reviews","sourceIp":"AAAAAAAAAAAAAP//rB6SSA==","sourceName":"reviews-v3-6cf47594fd-5svrx","sourceNamespace":"tutorial","sourceOwner":"kubernetes://apis/apps/v1/namespaces/tutorial/deployments/reviews-v3","sourcePrincipal":"cluster.local/ns/tutorial/sa/default","sourceWorkload":"reviews-v3","url":"/ratings/0","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.0 Safari/605.1.15","xForwardedFor":"0.0.0.0"} - {"level":"info","time":"2019-02-17T06:49:14.085630Z","instance":"accesslog.logentry.istio-system","apiClaims":"","apiKey":"","clientTraceId":"","connection_security_policy":"mutual_tls","destinationApp":"reviews","destinationIp":"AAAAAAAAAAAAAP//rB6SSA==","destinationName":"reviews-v3-6cf47594fd-5svrx","destinationNamespace":"tutorial","destinationOwner":"kubernetes://apis/apps/v1/namespaces/tutorial/deployments/reviews-v3","destinationPrincipal":"cluster.local/ns/tutorial/sa/default","destinationServiceHost":"reviews.tutorial.svc.cluster.local","destinationWorkload":"reviews-v3","grpcMessage":"","grpcStatus":"","httpAuthority":"reviews:9080","latency":"13.630935ms","method":"GET","permissiveResponseCode":"none","permissiveResponsePolicyID":"none","protocol":"http","receivedBytes":1023,"referer":"","reporter":"destination","requestId":"b2094a51-367f-9e6c-8cfb-9f9a1166076c","requestSize":0,"requestedServerName":"outbound_.9080_.v3_.reviews.tutorial.svc.cluster.local","responseCode":200,"responseFlags":"-","responseSize":375,"responseTimestamp":"2019-02-17T06:49:14.099090Z","sentBytes":555,"sourceApp":"productpage","sourceIp":"AAAAAAAAAAAAAP//rB5tTg==","sourceName":"productpage-v1-59b4f9f8d5-5z6r9","sourceNamespace":"tutorial","sourceOwner":"kubernetes://apis/apps/v1/namespaces/tutorial/deployments/productpage-v1","sourcePrincipal":"cluster.local/ns/tutorial/sa/default","sourceWorkload":"productpage-v1","url":"/reviews/0","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.0 Safari/605.1.15","xForwardedFor":"0.0.0.0"} - {"level":"info","time":"2019-02-17T06:49:14.071252Z","instance":"accesslog.logentry.istio-system","apiClaims":"","apiKey":"","clientTraceId":"","connection_security_policy":"none","destinationApp":"productpage","destinationIp":"AAAAAAAAAAAAAP//rB5tTg==","destinationName":"productpage-v1-59b4f9f8d5-5z6r9","destinationNamespace":"tutorial","destinationOwner":"kubernetes://apis/apps/v1/namespaces/tutorial/deployments/productpage-v1","destinationPrincipal":"","destinationServiceHost":"productpage.tutorial.svc.cluster.local","destinationWorkload":"productpage-v1","grpcMessage":"","grpcStatus":"","httpAuthority":"tutorial.bookinfo.com","latency":"30.980332ms","method":"GET","permissiveResponseCode":"none","permissiveResponsePolicyID":"none","protocol":"http","receivedBytes":737,"referer":"","reporter":"destination","requestId":"b2094a51-367f-9e6c-8cfb-9f9a1166076c","requestSize":0,"requestedServerName":"","responseCode":200,"responseFlags":"-","responseSize":5845,"responseTimestamp":"2019-02-17T06:49:14.102102Z","sentBytes":6062,"sourceApp":"","sourceIp":"AAAAAAAAAAAAAP//AAAAAA==","sourceName":"unknown","sourceNamespace":"default","sourceOwner":"unknown","sourcePrincipal":"","sourceWorkload":"unknown","url":"/productpage","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.0 Safari/605.1.15","xForwardedFor":"10.127.220.66"} - {{< /text >}} - - Notice the `sourceName` and `destinationName` attributes that identify who called whom, for example `productpage` - called `reviews`. (For `productpage`, `sourceName` is a pod of `istio-ingressgateway`). - Notice the HTTP-related attributes: `responseCode`, `url`, `method`. - Also observe general communication attributes: `responseSize`, `responseTimestamp`, `latency`. - - Note that the log entries from all of Bookinfo microservices appear in one place. - You do not have to go after each and every microservice and to display their logs one by one. - -While you can work with logs as text files, you can do something smarter. Use -a log database, where you can query your logs, similar to querying structured -data in SQL databases. Querying a database is more efficient than searching -text, so you can run sophisticated queries to quickly get precise results. You -can also let the database process the results, for example to aggregate some -metrics. - Istio is integrated out-of-the-box with [Prometheus time series database and monitoring system](https://prometheus.io). Prometheus collects various traffic-related metrics and provides @@ -91,6 +54,4 @@ The queries above use the `istio_requests_total` metric, which is a standard Ist other metrics, in particular, the ones of Envoy ([Envoy](https://www.envoyproxy.io) is the sidecar proxy of Istio). You can see the collected metrics in the _insert metric at cursor_ drop-down menu. -You can even [define your own metrics](/docs/tasks/observability/metrics/collecting-metrics/). - You are ready to [enable mutual TLS authentication with Istio](/docs/examples/microservices-istio/add-mtls). diff --git a/content/en/docs/ops/configuration/telemetry/in-proxy-service-telemetry/index.md b/content/en/docs/ops/configuration/telemetry/in-proxy-service-telemetry/index.md deleted file mode 100644 index 2dc59ec2b7..0000000000 --- a/content/en/docs/ops/configuration/telemetry/in-proxy-service-telemetry/index.md +++ /dev/null @@ -1,77 +0,0 @@ ---- -title: Generate Istio Metrics Without Mixer [Alpha] -description: How to enable in-proxy generation of HTTP service-level metrics. -weight: 20 -aliases: - - /docs/ops/telemetry/in-proxy-service-telemetry ---- - -Istio 1.4 adds alpha support to generate service-level HTTP metrics -directly in the Envoy proxies. This feature lets you continue to monitor your -service meshes using the tools Istio provides without needing Mixer. - -The in-proxy generation of service-level metrics replaces the following HTTP -metrics that Mixer currently generates: - -- `istio_requests_total` -- `istio_request_duration_seconds` -- `istio_request_size` - -## Enable service-level metrics generation in Envoy - -To generate service-level metrics directly in the Envoy proxies, set the following values: - - {{< text bash >}} - $ istioctl manifest apply --set values.telemetry.enabled=true,values.telemetry.v1.enabled=false,values.telemetry.v2.enabled=true,values.telemetry.v2.prometheus.enabled=true - {{< /text >}} - -Go to the **Istio Mesh** Grafana dashboard. Verify that the dashboard displays the same telemetry as before but without - any requests flowing through Istio's Mixer. - -## Differences with Mixer-based generation - -Small differences between the in-proxy generation and Mixer-based generation of service-level metrics -persist in Istio 1.3. We won't consider the functionality stable until in-proxy generation has full feature-parity with -Mixer-based generation. - -Until then, please consider these differences: - -- The `istio_request_duration_seconds` latency metric has the new name: `istio_request_duration_milliseconds`. - The new metric uses milliseconds instead of seconds. We updated the Grafana dashboards to - account for these changes. -- The `istio_request_duration_milliseconds` metric uses more granular buckets inside the proxy, providing - increased accuracy in latency reporting. - -## Performance impact - -{{< warning >}} - -As this work is currently experimental, our primary focus has been on establishing -the base functionality. We have identified several performance optimizations based -on our initial experimentation, and expect to continue to improve the performance -and scalability of this feature as it develops. - -We won't consider this feature for promotion to **Beta** or **Stable** [status](/about/feature-stages/#feature-phase-definitions) -until performance and scalability assessments and improvements have been made. - -The performance of your mesh depends on your configuration. -To learn more, see our [performance best practices post](/blog/2019/performance-best-practices/). - -{{< /warning >}} - -Here's what we've measured so far: - -- All new filters together use 10% less CPU resources for the `istio-proxy` containers - than the Mixer filter. -- The new filters add ~5ms P90 latency at 1000 rps compared to Envoy proxies - configured with no telemetry filters. -- If you only use the `istio-telemetry` service to generate service-level metrics, - you can switch off the `istio-telemetry` service. This could save up to ~0.5 vCPU per - 1000 rps of mesh traffic, and could halve the CPU consumed by Istio while collecting - [standard metrics](/docs/reference/config/policy-and-telemetry/metrics/). - -## Known limitations - -- We only provide support for exporting these metrics via Prometheus. -- We provide no support to generate TCP metrics. -- We provide no proxy-side customization or configuration of the generated metrics. diff --git a/content/en/docs/setup/getting-started/index.md b/content/en/docs/setup/getting-started/index.md index 448465eac1..f04df76956 100644 --- a/content/en/docs/setup/getting-started/index.md +++ b/content/en/docs/setup/getting-started/index.md @@ -181,7 +181,7 @@ see the following topics to explore more Istio features: - [Traffic shifting](/docs/tasks/traffic-management/traffic-shifting/) - [Querying metrics](/docs/tasks/observability/metrics/querying-metrics/) - [Visualizing metrics](/docs/tasks/observability/metrics/using-istio-dashboard/) -- [Collecting logs](/docs/tasks/observability/logs/collecting-logs/) +- [Collecting logs](/docs/tasks/observability/logs/access-log/) - [Rate limiting](/docs/tasks/policy-enforcement/rate-limiting/) - [Ingress gateways](/docs/tasks/traffic-management/ingress/ingress-control/) - [Accessing external services](/docs/tasks/traffic-management/egress/egress-control/) diff --git a/content/en/docs/tasks/observability/logs/_index.md b/content/en/docs/tasks/observability/logs/_index.md index 927d725aa1..09f95c6c5a 100644 --- a/content/en/docs/tasks/observability/logs/_index.md +++ b/content/en/docs/tasks/observability/logs/_index.md @@ -1,6 +1,6 @@ --- title: Logs -description: Demonstrates the configuration, collection, and processing of Istio mesh logs. +description: Demonstrates the collection of logs within Istio. weight: 20 aliases: - /docs/tasks/telemetry/logs/ diff --git a/content/en/docs/tasks/observability/metrics/_index.md b/content/en/docs/tasks/observability/metrics/_index.md index 018cb16f5e..3078f6b35f 100644 --- a/content/en/docs/tasks/observability/metrics/_index.md +++ b/content/en/docs/tasks/observability/metrics/_index.md @@ -1,6 +1,6 @@ --- title: Metrics -description: Demonstrates the configuration, collection, and processing of Istio mesh metrics. +description: Demonstrates the collection and querying of metrics within Istio. weight: 1 aliases: - /docs/tasks/telemetry/metrics/ diff --git a/content/en/docs/tasks/observability/metrics/querying-metrics/index.md b/content/en/docs/tasks/observability/metrics/querying-metrics/index.md index 455dad5395..2323ab07e9 100644 --- a/content/en/docs/tasks/observability/metrics/querying-metrics/index.md +++ b/content/en/docs/tasks/observability/metrics/querying-metrics/index.md @@ -87,22 +87,10 @@ Other queries to try: ### About the Prometheus add-on -Mixer comes with a built-in [Prometheus](https://prometheus.io) adapter that -exposes an endpoint serving generated metric values. The Prometheus add-on is a -Prometheus server that comes preconfigured to scrape Mixer endpoints to collect -the exposed metrics. It provides a mechanism for persistent storage and querying +The Prometheus add-on is a Prometheus server that comes preconfigured to scrape +Istio endpoints to collect metrics. It provides a mechanism for persistent storage and querying of Istio metrics. -The configured Prometheus add-on scrapes the following endpoints: - -1. `istio-telemetry.istio-system:42422`: The `istio-mesh` job returns all Mixer-generated metrics. -1. `istio-telemetry.istio-system:15014`: The `istio-telemetry` job returns all Mixer-specific metrics. Use this endpoint to monitor Mixer itself. -1. `istio-proxy:15090`: The `envoy-stats` job returns raw stats generated by Envoy. Prometheus is configured to look for pods with the `envoy-prom` endpoint exposed. The add-on configuration filters out a large number of envoy metrics during collection in an attempt to limit the scale of data by the add-on processes. -1. `istio-pilot.istio-system:15014`: The `pilot` job returns the Pilot-generated metrics. -1. `istio-galley.istio-system:15014`: The `galley` job returns the Galley-generated metrics. -1. `istio-policy.istio-system:15014`: The `istio-policy` job returns all policy-related metrics. -1. `istio-citadel.istio-system:15014`: The `istio-citadel` job returns all Citadel-generated metrics. - For more on querying Prometheus, please read their [querying docs](https://prometheus.io/docs/querying/basics/). diff --git a/content/en/docs/tasks/observability/mixer/_index.md b/content/en/docs/tasks/observability/mixer/_index.md new file mode 100644 index 0000000000..39368b0677 --- /dev/null +++ b/content/en/docs/tasks/observability/mixer/_index.md @@ -0,0 +1,8 @@ +--- +title: Using Mixer for Telemetry (deprecated) +description: Demonstrates how to collect telemetry information from the mesh using Mixer. +weight: 89 +aliases: + - /docs/examples/telemetry/ + - /docs/tasks/telemetry/ +--- diff --git a/content/en/docs/tasks/observability/mixer/logs/_index.md b/content/en/docs/tasks/observability/mixer/logs/_index.md new file mode 100644 index 0000000000..c82d6c3a01 --- /dev/null +++ b/content/en/docs/tasks/observability/mixer/logs/_index.md @@ -0,0 +1,6 @@ +--- +title: Logs +description: Demonstrates the configuration, collection, and processing of Istio mesh logs. +weight: 20 +aliases: +--- \ No newline at end of file diff --git a/content/en/docs/tasks/observability/logs/collecting-logs/index.md b/content/en/docs/tasks/observability/mixer/logs/collecting-logs/index.md similarity index 98% rename from content/en/docs/tasks/observability/logs/collecting-logs/index.md rename to content/en/docs/tasks/observability/mixer/logs/collecting-logs/index.md index a2c6d678b4..626a284a7a 100644 --- a/content/en/docs/tasks/observability/logs/collecting-logs/index.md +++ b/content/en/docs/tasks/observability/mixer/logs/collecting-logs/index.md @@ -1,9 +1,10 @@ --- -title: Collecting Logs +title: Collecting Logs with Mixer description: This task shows you how to configure Istio to collect and customize logs. weight: 10 keywords: [telemetry,logs] aliases: + - /docs/tasks/observability/logs/collecting-logs/ - /docs/tasks/telemetry/logs/collecting-logs/ --- diff --git a/content/en/docs/tasks/observability/logs/fluentd/index.md b/content/en/docs/tasks/observability/mixer/logs/fluentd/index.md similarity index 99% rename from content/en/docs/tasks/observability/logs/fluentd/index.md rename to content/en/docs/tasks/observability/mixer/logs/fluentd/index.md index e673819bdd..59e93342a1 100644 --- a/content/en/docs/tasks/observability/logs/fluentd/index.md +++ b/content/en/docs/tasks/observability/mixer/logs/fluentd/index.md @@ -1,9 +1,10 @@ --- -title: Logging with Fluentd +title: Logging with Mixer and Fluentd description: This task shows you how to configure Istio to log to a Fluentd daemon. weight: 90 keywords: [telemetry,logging] aliases: + - /docs/tasks/observability/logs/fluentd/ - /docs/tasks/telemetry/fluentd/ - /docs/tasks/telemetry/logs/fluentd/ --- diff --git a/content/en/docs/tasks/observability/mixer/metrics/_index.md b/content/en/docs/tasks/observability/mixer/metrics/_index.md new file mode 100644 index 0000000000..bfb8e145a6 --- /dev/null +++ b/content/en/docs/tasks/observability/mixer/metrics/_index.md @@ -0,0 +1,6 @@ +--- +title: Metrics +description: Demonstrates the configuration, collection, and processing of Istio mesh metrics using Mixer. +weight: 1 +aliases: +--- \ No newline at end of file diff --git a/content/en/docs/tasks/observability/metrics/collecting-metrics/index.md b/content/en/docs/tasks/observability/mixer/metrics/collecting-metrics/index.md similarity index 99% rename from content/en/docs/tasks/observability/metrics/collecting-metrics/index.md rename to content/en/docs/tasks/observability/mixer/metrics/collecting-metrics/index.md index df03b99427..4c30071e87 100644 --- a/content/en/docs/tasks/observability/metrics/collecting-metrics/index.md +++ b/content/en/docs/tasks/observability/mixer/metrics/collecting-metrics/index.md @@ -7,6 +7,7 @@ aliases: - /docs/tasks/metrics-logs.html - /docs/tasks/telemetry/metrics-logs/ - /docs/tasks/telemetry/metrics/collecting-metrics/ + - /docs/tasks/observability/metrics/collecting-metrics/ --- This task shows how to configure Istio to automatically gather telemetry for diff --git a/content/en/docs/tasks/observability/metrics/tcp-metrics/index.md b/content/en/docs/tasks/observability/mixer/metrics/tcp-metrics/index.md similarity index 98% rename from content/en/docs/tasks/observability/metrics/tcp-metrics/index.md rename to content/en/docs/tasks/observability/mixer/metrics/tcp-metrics/index.md index f8e6aebdce..98849e7cfd 100644 --- a/content/en/docs/tasks/observability/metrics/tcp-metrics/index.md +++ b/content/en/docs/tasks/observability/mixer/metrics/tcp-metrics/index.md @@ -140,8 +140,7 @@ In this task, you added Istio configuration that instructed Mixer to automatically generate and report a new metric for all traffic to a TCP service within the mesh. -Similar to the [Collecting Metrics and -Logs](/docs/tasks/observability/metrics/collecting-metrics/) Task, the new +Similar to the [Collecting Metrics](/docs/tasks/observability/mixer/metrics/collecting-metrics/) Task, the new configuration consisted of _instances_, a _handler_, and a _rule_. Please see that Task for a complete description of the components of metric collection. diff --git a/content/en/docs/tasks/observability/metrics/tcp-metrics/istio-tcp-attribute-flow.svg b/content/en/docs/tasks/observability/mixer/metrics/tcp-metrics/istio-tcp-attribute-flow.svg similarity index 100% rename from content/en/docs/tasks/observability/metrics/tcp-metrics/istio-tcp-attribute-flow.svg rename to content/en/docs/tasks/observability/mixer/metrics/tcp-metrics/istio-tcp-attribute-flow.svg diff --git a/content/en/news/releases/0.x/announcing-0.5/index.md b/content/en/news/releases/0.x/announcing-0.5/index.md index 47d5dfc53a..51f0c281c0 100644 --- a/content/en/news/releases/0.x/announcing-0.5/index.md +++ b/content/en/news/releases/0.x/announcing-0.5/index.md @@ -41,7 +41,7 @@ providing a flexible fine-grained access control mechanism. [Learn more](https:/ [Learn more](/docs/concepts/security/#authorization) - **Fluentd**. Mixer now has an adapter for log collection through [Fluentd](https://www.fluentd.org). -[Learn more](/docs/tasks/observability/logs/fluentd/) +[Learn more](/docs/tasks/observability/mixer/logs/fluentd/) - **Stdio**. The stdio adapter now lets you log to files with support for log rotation & backup, along with a host of controls. diff --git a/content/en/news/releases/1.3.x/announcing-1.3.1/index.md b/content/en/news/releases/1.3.x/announcing-1.3.1/index.md index c53d58101b..7b7b0be16c 100644 --- a/content/en/news/releases/1.3.x/announcing-1.3.1/index.md +++ b/content/en/news/releases/1.3.x/announcing-1.3.1/index.md @@ -31,5 +31,5 @@ This release includes bug fixes to improve robustness. This release note describ - **Added** annotations for Datadog tracing that controls extra features in sidecars. - **Added** the `pilot_xds_push_time` metric to report Pilot xDS push time. - **Added** `istioctl experimental analyze` to support multi-resource analysis and validation. -- **Added** support for running metadata exchange and stats extensions in a WebAssembly sandbox. Follow [these](/docs/ops/configuration/telemetry/in-proxy-service-telemetry/) instructions to try it out. +- **Added** support for running metadata exchange and stats extensions in a WebAssembly sandbox. - **Removed** time diff info in the proxy-status command. diff --git a/content/en/news/releases/1.4.x/announcing-1.4/change-notes/index.md b/content/en/news/releases/1.4.x/announcing-1.4/change-notes/index.md index 9e07d70449..539ca31355 100644 --- a/content/en/news/releases/1.4.x/announcing-1.4/change-notes/index.md +++ b/content/en/news/releases/1.4.x/announcing-1.4/change-notes/index.md @@ -25,7 +25,7 @@ weight: 10 ## Telemetry - **Added** experimental in-proxy telemetry reporting to [Stackdriver](https://github.com/istio/proxy/blob/{{< source_branch_name >}}/extensions/stackdriver/README.md). -- **Improved** support for [in-proxy](/docs/ops/configuration/telemetry/in-proxy-service-telemetry/) Prometheus generation of HTTP service metrics (from experimental to alpha). +- **Improved** support for in-proxy Prometheus generation of HTTP service metrics (from experimental to alpha). - **Improved** telemetry collection for [blocked and passthrough external service traffic](/blog/2019/monitoring-external-service-traffic/). - **Added** the option to configure [stat patterns](/docs/reference/config/istio.mesh.v1alpha1/#MeshConfig) for Envoy stats. - **Added** the `inbound` and `outbound` prefixes to the Envoy HTTP stats to specify traffic direction.