istio.io/content/docs/tasks/telemetry/metrics/querying-metrics/index.md

title	description	weight	keywords	aliases
Querying Metrics from Prometheus	This task shows you how to query for Istio Metrics using Prometheus.	30	telemetry metrics	/docs/tasks/telemetry/querying-metrics/

This task shows you how to query for Istio Metrics using Prometheus. As part of this task, you will use the web-based interface for querying metric values.

The Bookinfo sample application is used as the example application throughout this task.

Before you begin

Install Istio in your cluster and deploy an application.

Querying Istio Metrics

1. Verify that the prometheus service is running in your cluster.

   In Kubernetes environments, execute the following command:

   {{< text bash >}} $ kubectl -n istio-system get svc prometheus NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE prometheus 10.59.241.54 9090/TCP 2m {{< /text >}}

2. Send traffic to the mesh.

   For the Bookinfo sample, visit http://$GATEWAY_URL/productpage in your web browser or issue the following command:

   {{< text bash >}} $ curl http://$GATEWAY_URL/productpage {{< /text >}}

   {{< tip >}} $GATEWAY_URL is the value set in the Bookinfo example. {{< /tip >}}

3. Open the Prometheus UI.

   In Kubernetes environments, execute the following command:

   {{< text bash >}} kubectl -n istio-system port-forward(kubectl -n istio-system get pod -l app=prometheus -o jsonpath='{.items[0].metadata.name}') 9090:9090 & {{< /text >}}

   Visit http://localhost:9090/graph in your web browser.

4. Execute a Prometheus query.

   In the "Expression" input box at the top of the web page, enter the text: istio_requests_total. Then, click the Execute button.

The results will be similar to:

{{< image link="./prometheus_query_result.png" caption="Prometheus Query Result" >}}

Other queries to try:

• Total count of all requests to the productpage service:

  {{< text plain >}} istio_requests_total{destination_service="productpage.default.svc.cluster.local"} {{< /text >}}

• Total count of all requests to v3 of the reviews service:

  {{< text plain >}} istio_requests_total{destination_service="reviews.default.svc.cluster.local", destination_version="v3"} {{< /text >}}

  This query returns the current total count of all requests to the v3 of the reviews service.

• Rate of requests over the past 5 minutes to all instances of the productpage service:

  {{< text plain >}} rate(istio_requests_total{destination_service=~"productpage.*", response_code="200"}[5m]) {{< /text >}}

About the Prometheus add-on

Mixer comes with a built-in Prometheus adapter that exposes an endpoint serving generated metric values. The Prometheus add-on is a Prometheus server that comes preconfigured to scrape Mixer endpoints to collect the exposed metrics. It provides a mechanism for persistent storage and querying of Istio metrics.

The configured Prometheus add-on scrapes the following endpoints:

1. istio-telemetry.istio-system:42422: The istio-mesh job returns all Mixer-generated metrics.
2. istio-telemetry.istio-system:10514: The istio-telemetry job returns all Mixer-specific metrics. Use this endpoint to monitor Mixer itself.
3. istio-proxy:15090: The envoy-stats job returns raw stats generated by Envoy. Prometheus is configured to look for pods with the envoy-prom endpoint exposed. The add-on configuration filters out a large number of envoy metrics during collection in an attempt to limit the scale of data by the add-on processes.
4. istio-pilot.istio-system:10514: The pilot job returns the Pilot-generated metrics.
5. istio-galley.istio-system:10514: The galley job returns the Galley-generated metrics.
6. istio-policy.istio-system:10514: The istio-policy job returns all policy-related metrics.

For more on querying Prometheus, please read their querying docs.

Cleanup

• Remove any kubectl port-forward processes that may still be running:

  {{< text bash >}} $ killall kubectl {{< /text >}}

• If you are not planning to explore any follow-on tasks, refer to the Bookinfo cleanup instructions to shutdown the application.