istio.io/_docs/tasks/telemetry/querying-metrics.md

title	overview	order	layout	type
Querying Metrics from Prometheus	This task shows you how to query for Istio Metrics using Prometheus.	30	docs	markdown

{% include home.html %}

This task shows you how to query for Istio Metrics using Prometheus. As part of this task, you will install the Prometheus Istio addon and use the web-based interface for querying metric values.

The Bookinfo sample application is used as the example application throughout this task.

Before you begin

• Install Istio in your cluster and deploy an application.

Querying Istio Metrics

1. To query the metrics provided by Mixer, first install the Prometheus add-on.

   In Kubernetes environments, execute the following command:

   kubectl apply -f install/kubernetes/addons/prometheus.yaml
   

2. Verify that the service is running in your cluster.

   In Kubernetes environments, execute the following command:

   kubectl -n istio-system get svc prometheus
   

   The output will be similar to:

   NAME         CLUSTER-IP     EXTERNAL-IP   PORT(S)    AGE
   prometheus   10.59.241.54   <none>        9090/TCP   2m
   

3. Send traffic to the mesh.

   For the Bookinfo sample, visit http://$GATEWAY_URL/productpage in your web browser or issue the following command:

   curl http://$GATEWAY_URL/productpage
   

   Note: $GATEWAY_URL is the value set in the Bookinfo guide.

4. Open the Prometheus UI.

   In Kubernetes environments, execute the following command:

   kubectl -n istio-system port-forward $(kubectl -n istio-system get pod -l app=prometheus -o jsonpath='{.items[0].metadata.name}') 9090:9090 &
   

   Visit http://localhost:9090/graph in your web browser.

5. Execute a Prometheus query.

   In the "Expression" input box at the top of the web page, enter the text: istio_request_count. Then, click the Execute button.

   The results will be similar to:

{% include figure.html width='100%' ratio='39.36%' img='./img/prometheus_query_result.png' alt='Prometheus Query Result' title='Prometheus Query Result' caption='Prometheus Query Result' %}

Other queries to try:

- Total count of all requests to `productpage` service:

  ```
  istio_request_count{destination_service="productpage.default.svc.cluster.local"}
  ```

- Total count of all requests to `v3` of the `reviews` service:

  ```
  istio_request_count{destination_service="reviews.default.svc.cluster.local", destination_version="v3"}
  ```

  This query returns the current total count of all requests to the v3 of the reviews service.

- Rate of requests over the past 5 minutes to all `productpage` services:

  ```
  rate(istio_request_count{destination_service=~"productpage.*", response_code="200"}[5m])
  ```

About the Prometheus Add-on

Mixer comes with a built-in Prometheus adapter that exposes an endpoint serving generated metric values. The Prometheus add-on is a Prometheus server that comes preconfigured to scrape Mixer endpoints to collect the exposed metrics. It provides a mechanism for persistent storage and querying of Istio metrics.

The configured Prometheus add-on scrapes three endpoints:

1. istio-mesh (istio-mixer.istio-system:42422): all Mixer-generated mesh metrics.
2. mixer (istio-mixer.istio-system:9093): all Mixer-specific metrics. Used to monitor Mixer itself.
3. envoy (istio-mixer.istio-system:9102): raw stats generated by Envoy (and translated from statsd to prometheus).

For more on querying Prometheus, please read their querying docs.

Cleanup

• In Kubernetes environments, execute the following command to remove the Prometheus add-on:

  kubectl delete -f install/kubernetes/addons/prometheus.yaml
  

• Remove any kubectl port-forward processes that may still be running:

  killall kubectl
  

• If you are not planning to explore any follow-on tasks, refer to the Bookinfo cleanup instructions to shutdown the application.

What's next

• Refer to the In-Depth Telemetry guide.