dapr
/
docs
mirror of https://github.com/dapr/docs.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'v1.11' into aacrawfi/split-workflow

This commit is contained in:
Mark Fussell 2023-06-16 20:42:28 -07:00 committed by GitHub
parent ae422e42f8 ee25a6a753
commit 1cd9f4c4b0
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 90 additions and 88 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

178
daprdocs/content/en/operations/monitoring/metrics/azure-monitor.md
View File

@ -15,118 +15,120 @@ description: "Enable Dapr metrics and logs with Azure Monitor for Azure Kubernet
## Enable Prometheus metric scrape using config map ## Enable Prometheus metric scrape using config map
1. Make sure that omsagents are running 1. Make sure that Azure Monitor Agents (AMA) are running.
```bash ```bash
$ kubectl get pods -n kube-system $ kubectl get pods -n kube-system
NAME READY STATUS RESTARTS AGE NAME READY STATUS RESTARTS AGE
... ...
omsagent-75qjs 1/1 Running 1 44h ama-logs-48kpv 2/2 Running 0 2d13h
omsagent-c7c4t 1/1 Running 0 44h ama-logs-mx24c 2/2 Running 0 2d13h
omsagent-rs-74f488997c-dshpx 1/1 Running 1 44h ama-logs-rs-f9bbb9898-vbt6k 1/1 Running 0 30h
omsagent-smtk7 1/1 Running 1 44h ama-logs-sm2mz 2/2 Running 0 2d13h
... ama-logs-z7p4c 2/2 Running 0 2d13h
``` ...
```
2. Apply config map to enable Prometheus metrics endpoint scrape. 1. Apply config map to enable Prometheus metrics endpoint scrape.
You can use [azm-config-map.yaml](/docs/azm-config-map.yaml) to enable prometheus metrics endpoint scrape. You can use [azm-config-map.yaml](/docs/azm-config-map.yaml) to enable Prometheus metrics endpoint scrape.
If you installed Dapr to the different namespace, you need to change the `monitor_kubernetes_pod_namespaces` array values. For example: If you installed Dapr to a different namespace, you need to change the `monitor_kubernetes_pod_namespaces` array values. For example:
```yaml ```yaml
... ...
prometheus-data-collection-settings: |- prometheus-data-collection-settings: |-
[prometheus_data_collection_settings.cluster] [prometheus_data_collection_settings.cluster]
interval = "1m" interval = "1m"
monitor_kubernetes_pods = true monitor_kubernetes_pods = true
monitor_kubernetes_pods_namespaces = ["dapr-system", "default"] monitor_kubernetes_pods_namespaces = ["dapr-system", "default"]
[prometheus_data_collection_settings.node] [prometheus_data_collection_settings.node]
interval = "1m" interval = "1m"
... ...
``` ```
Apply config map: Apply config map:
```bash ```bash
kubectl apply -f ./azm-config.map.yaml kubectl apply -f ./azm-config.map.yaml
``` ```
## Install Dapr with JSON formatted logs ## Install Dapr with JSON formatted logs
1. Install Dapr with enabling JSON-formatted logs 1. Install Dapr with enabling JSON-formatted logs.
```bash ```bash
helm install dapr dapr/dapr --namespace dapr-system --set global.logAsJson=true helm install dapr dapr/dapr --namespace dapr-system --set global.logAsJson=true
``` ```
2. Enable JSON formatted log in Dapr sidecar and add Prometheus annotations. 1. Enable JSON formatted log in Dapr sidecar and add Prometheus annotations.
> Note: OMS Agent scrapes the metrics only if replicaset has Prometheus annotations. > Note: The Azure Monitor Agents (AMA) only sends the metrics if the Prometheus annotations are set.
Add `dapr.io/log-as-json: "true"` annotation to your deployment yaml. Add `dapr.io/log-as-json: "true"` annotation to your deployment yaml.
Example: Example:
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: pythonapp
namespace: default
labels:
app: python
spec:
replicas: 1
selector:
matchLabels:
app: python
template:
metadata:
labels:
app: python
annotations:
dapr.io/enabled: "true"
dapr.io/app-id: "pythonapp"
dapr.io/log-as-json: "true"
prometheus.io/scrape: "true"
prometheus.io/port: "9090"
prometheus.io/path: "/"
... ```yaml
``` apiVersion: apps/v1
kind: Deployment
metadata:
name: pythonapp
namespace: default
labels:
app: python
spec:
replicas: 1
selector:
matchLabels:
app: python
template:
metadata:
labels:
app: python
annotations:
dapr.io/enabled: "true"
dapr.io/app-id: "pythonapp"
dapr.io/log-as-json: "true"
prometheus.io/scrape: "true"
prometheus.io/port: "9090"
prometheus.io/path: "/"
...
```
## Search metrics and logs with Azure Monitor ## Search metrics and logs with Azure Monitor
1. Go to Azure Monitor 1. Go to Azure Monitor in the Azure portal.
2. Search Dapr logs 1. Search Dapr **Logs**.
Here is an example query, to parse JSON formatted logs and query logs from dapr system processes. Here is an example query, to parse JSON formatted logs and query logs from Dapr system processes.
``` ```
ContainerLog ContainerLog
| extend parsed=parse_json(LogEntry) | extend parsed=parse_json(LogEntry)
| project Time=todatetime(parsed['time']), app_id=parsed['app_id'], scope=parsed['scope'],level=parsed['level'], msg=parsed['msg'], type=parsed['type'], ver=parsed['ver'], instance=parsed['instance'] | project Time=todatetime(parsed['time']), app_id=parsed['app_id'], scope=parsed['scope'],level=parsed['level'], msg=parsed['msg'], type=parsed['type'], ver=parsed['ver'], instance=parsed['instance']
| where level != "" | where level != ""
| sort by Time | sort by Time
``` ```
3. Search metrics 1. Search **Metrics**.
This query, queries process_resident_memory_bytes Prometheus metrics for Dapr system processes and renders timecharts This query, queries `process_resident_memory_bytes` Prometheus metrics for Dapr system processes and renders timecharts.
``` ```
InsightsMetrics InsightsMetrics
| where Namespace == "prometheus" and Name == "process_resident_memory_bytes" | where Namespace == "prometheus" and Name == "process_resident_memory_bytes"
| extend tags=parse_json(Tags) | extend tags=parse_json(Tags)
| project TimeGenerated, Name, Val, app=tostring(tags['app']) | project TimeGenerated, Name, Val, app=tostring(tags['app'])
| summarize memInBytes=percentile(Val, 99) by bin(TimeGenerated, 1m), app | summarize memInBytes=percentile(Val, 99) by bin(TimeGenerated, 1m), app
| where app startswith "dapr-" | where app startswith "dapr-"
| render timechart | render timechart
``` ```
# References ## References
* [Configure scraping of Prometheus metrics with Azure Monitor for containers](https://docs.microsoft.com/azure/azure-monitor/insights/container-insights-prometheus-integration) - [Configure scraping of Prometheus metrics with Azure Monitor for containers](https://docs.microsoft.com/azure/azure-monitor/insights/container-insights-prometheus-integration)
* [Configure agent data collection for Azure Monitor for containers](https://docs.microsoft.com/azure/azure-monitor/insights/container-insights-agent-config) - [Configure agent data collection for Azure Monitor for containers](https://docs.microsoft.com/azure/azure-monitor/insights/container-insights-agent-config)
* [Azure Monitor Query](https://docs.microsoft.com/azure/azure-monitor/log-query/query-language) - [Azure Monitor Query](https://docs.microsoft.com/azure/azure-monitor/log-query/query-language)