5.6 KiB
type | title | linkTitle | weight | description |
---|---|---|---|---|
docs | Observe metrics with Grafana | Grafana | 5000 | How to view Dapr metrics in a Grafana dashboard. |
Pre-requisites
- [Setup Prometheus]({{}})
Setup on Kubernetes
Install Grafana
-
Install Grafana
helm install grafana stable/grafana -n dapr-monitoring
If you are Minikube user or want to disable persistent volume for development purpose, you can disable it by using the following command:
helm install grafana stable/grafana -n dapr-monitoring --set persistence.enabled=false
-
Retrieve the admin password for Grafana login
kubectl get secret --namespace dapr-monitoring grafana -o jsonpath="{. data.admin-password}" | base64 --decode cj3m0OfBNx8SLzUlTx91dEECgzRlYJb60D2evof1%
{{% alert title="Note" color="info" %}} Remove the
%
character from the password that this command returns. For example, the admin password iscj3m0OfBNx8SLzUlTx91dEECgzRlYJb60D2evof1
. {{% /alert %}} -
Validation
Ensure Grafana is running in your cluster (see last line below)
kubectl get pods -n dapr-monitoring NAME READY STATUS RESTARTS AGE dapr-prom-kube-state-metrics-9849d6cc6-t94p8 1/1 Running 0 4m58s dapr-prom-prometheus-alertmanager-749cc46f6-9b5t8 2/2 Running 0 4m58s dapr-prom-prometheus-node-exporter-5jh8p 1/1 Running 0 4m58s dapr-prom-prometheus-node-exporter-88gbg 1/1 Running 0 4m58s dapr-prom-prometheus-node-exporter-bjp9f 1/1 Running 0 4m58s dapr-prom-prometheus-pushgateway-688665d597-h4xx2 1/1 Running 0 4m58s dapr-prom-prometheus-server-694fd8d7c-q5d59 2/2 Running 0 4m58s grafana-c49889cff-x56vj 1/1 Running 0 5m10s
Configure Prometheus as data source
First you need to connect Prometheus as a data source to Grafana.
-
Port-forward to svc/grafana
$ kubectl port-forward svc/grafana 8080:80 -n dapr-monitoring Forwarding from 127.0.0.1:8080 -> 3000 Forwarding from [::1]:8080 -> 3000 Handling connection for 8080 Handling connection for 8080
-
Browse
http://localhost:8080
-
Login with admin and password
-
Click Configuration Settings -> Data Sources
-
Add Prometheus as a data soruce.
-
Enter Promethesus server address in your cluster.
You can get the Prometheus server address by running the following command.
kubectl get svc -n dapr-monitoring NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE dapr-prom-kube-state-metrics ClusterIP 10.0.174.177 <none> 8080/TCP 7d9h dapr-prom-prometheus-alertmanager ClusterIP 10.0.255.199 <none> 80/TCP 7d9h dapr-prom-prometheus-node-exporter ClusterIP None <none> 9100/TCP 7d9h dapr-prom-prometheus-pushgateway ClusterIP 10.0.190.59 <none> 9091/TCP 7d9h dapr-prom-prometheus-server ClusterIP 10.0.172.191 <none> 80/TCP 7d9h elasticsearch-master ClusterIP 10.0.36.146 <none> 9200/TCP,9300/TCP 7d10h elasticsearch-master-headless ClusterIP None <none> 9200/TCP,9300/TCP 7d10h grafana ClusterIP 10.0.15.229 <none> 80/TCP 5d5h kibana-kibana ClusterIP 10.0.188.224 <none> 5601/TCP 7d10h
In this Howto, the server is
dapr-prom-prometheus-server
.You now need to set up Prometheus data source with the following settings:
- Name:
Dapr
- HTTP URL:
http://dapr-prom-prometheus-server.dapr-monitoring
- Default: On
- Name:
-
Click
Save & Test
button to verify that the connection succeeded.
Import dashboards in Grafana
Next you import the Dapr dashboards into Grafana.
In the upper left, click the "+" then "Import".
You can now import built-in Grafana dashboard templates.
The Grafana dashboards are part of release assets with this URL https://github.com/dapr/dapr/releases/
You can find grafana-actor-dashboard.json
, grafana-sidecar-dashboard.json
and grafana-system-services-dashboard.json
in release assets location.
- Find the dashboard that you imported and enjoy!
References
- Set up Prometheus and Grafana
- Prometheus Installation
- Prometheus on Kubernetes
- Prometheus Kubernetes Operator
- Prometheus Query Language