Add Prometheus auto service discovery instructions to

prometheus.md

- Updated prometheus.md with instructions for setting up Prometheus auto service discovery for Dapr and sidecar targets.

Signed-off-by: Maulin Desai <mdesai@bosleo.com>
This commit is contained in:
Maulin Desai 2024-08-17 15:22:30 +05:30
parent 3b38ae3ecc
commit bcfa46244a
3 changed files with 111 additions and 0 deletions

View File

@ -93,13 +93,108 @@ helm install dapr-prom prometheus-community/prometheus -n dapr-monitoring
--set alertmanager.persistence.enabled=false --set pushgateway.persistentVolume.enabled=false --set server.persistentVolume.enabled=false
```
For automatic discovery of Dapr targets (Service Discovery), use:
```bash
helm install dapr-prom prometheus-community/prometheus -f values.yaml -n dapr-monitoring --create-namespace
```
### `values.yaml` File
```yaml
alertmanager:
persistence:
enabled: false
pushgateway:
persistentVolume:
enabled: false
server:
persistentVolume:
enabled: false
# Adds additional scrape configurations to prometheus.yml
# Uses service discovery to find Dapr and Dapr sidecar targets
extraScrapeConfigs: |-
- job_name: dapr-sidecars
kubernetes_sd_configs:
- role: pod
relabel_configs:
- action: keep
regex: "true"
source_labels:
- __meta_kubernetes_pod_annotation_dapr_io_enabled
- action: keep
regex: "true"
source_labels:
- __meta_kubernetes_pod_annotation_dapr_io_enable_metrics
- action: replace
replacement: ${1}
source_labels:
- __meta_kubernetes_namespace
target_label: namespace
- action: replace
replacement: ${1}
source_labels:
- __meta_kubernetes_pod_name
target_label: pod
- action: replace
regex: (.*);daprd
replacement: ${1}-dapr
source_labels:
- __meta_kubernetes_pod_annotation_dapr_io_app_id
- __meta_kubernetes_pod_container_name
target_label: service
- action: replace
replacement: ${1}:9090
source_labels:
- __meta_kubernetes_pod_ip
target_label: __address__
- job_name: dapr
kubernetes_sd_configs:
- role: pod
relabel_configs:
- action: keep
regex: dapr
source_labels:
- __meta_kubernetes_pod_label_app_kubernetes_io_name
- action: keep
regex: dapr
source_labels:
- __meta_kubernetes_pod_label_app_kubernetes_io_part_of
- action: replace
replacement: ${1}
source_labels:
- __meta_kubernetes_pod_label_app
target_label: app
- action: replace
replacement: ${1}
source_labels:
- __meta_kubernetes_namespace
target_label: namespace
- action: replace
replacement: ${1}
source_labels:
- __meta_kubernetes_pod_name
target_label: pod
- action: replace
replacement: ${1}:9090
source_labels:
- __meta_kubernetes_pod_ip
target_label: __address__
```
3. Validation
Ensure Prometheus is running in your cluster.
```bash
kubectl get pods -n dapr-monitoring
```
Expected output:
```bash
NAME READY STATUS RESTARTS AGE
dapr-prom-kube-state-metrics-9849d6cc6-t94p8 1/1 Running 0 4m58s
dapr-prom-prometheus-alertmanager-749cc46f6-9b5t8 2/2 Running 0 4m58s
@ -110,6 +205,22 @@ dapr-prom-prometheus-pushgateway-688665d597-h4xx2 1/1 Running 0
dapr-prom-prometheus-server-694fd8d7c-q5d59 2/2 Running 0 4m58s
```
### Access the Prometheus Dashboard
To view the Prometheus dashboard and check service discovery:
```bash
kubectl port-forward svc/dapr-prom-prometheus-server 9090:80 -n dapr-monitoring
```
Open a browser and visit `http://localhost:9090`. Navigate to `Status -> Service Discovery` to verify that the Dapr targets are discovered correctly.
<img src="/images/prometheus-web-ui.png" alt="Prometheus Web UI" width="1200">
You can see the `job_name` and its discovered targets.
<img src="/images/prometheus-service-discovery.png" alt="Prometheus Service Discovery" width="1200">
## Example
<div class="embed-responsive embed-responsive-16by9">

Binary file not shown.

After

Width:  |  Height:  |  Size: 401 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 232 KiB