chore: tier 1 freshness: content/config/daemon/prometheus.md

Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
This commit is contained in:
David Karlsson 2023-10-27 15:34:54 +02:00
parent 0d707a2c9a
commit aad4419941
1 changed files with 24 additions and 25 deletions

View File

@ -3,9 +3,9 @@ description: Collecting Docker metrics with Prometheus
keywords: prometheus, metrics keywords: prometheus, metrics
title: Collect Docker metrics with Prometheus title: Collect Docker metrics with Prometheus
aliases: aliases:
- /engine/admin/prometheus/ - /engine/admin/prometheus/
- /config/thirdparty/monitoring/ - /config/thirdparty/monitoring/
- /config/thirdparty/prometheus/ - /config/thirdparty/prometheus/
--- ---
[Prometheus](https://prometheus.io/) is an open-source systems monitoring and [Prometheus](https://prometheus.io/) is an open-source systems monitoring and
@ -19,13 +19,13 @@ container, and monitor your Docker instance using Prometheus.
> development and may change at any time. > development and may change at any time.
{ .warning } { .warning }
Currently, you can only monitor Docker itself. You cannot currently monitor your Currently, you can only monitor Docker itself. You can't currently monitor your
application using the Docker target. application using the Docker target.
## Prerequisites ## Prerequisites
1. One or more Docker engines are joined into a Docker swarm, using `docker 1. One or more Docker engines are joined into a Docker Swarm, using `docker
swarm init` on one manager and `docker swarm join` on other managers and swarm init` on one manager and `docker swarm join` on other managers and
worker nodes. worker nodes.
2. You need an internet connection to pull the Prometheus image. 2. You need an internet connection to pull the Prometheus image.
@ -33,23 +33,22 @@ application using the Docker target.
To configure the Docker daemon as a Prometheus target, you need to specify the To configure the Docker daemon as a Prometheus target, you need to specify the
`metrics-address`. The best way to do this is via the `daemon.json`, which is `metrics-address`. The best way to do this is via the `daemon.json`, which is
located at one of the following locations by default. If the file does not located at one of the following locations by default. If the file doesn't
exist, create it. exist, create it.
- **Linux**: `/etc/docker/daemon.json` - **Linux**: `/etc/docker/daemon.json`
- **Windows Server**: `C:\ProgramData\docker\config\daemon.json` - **Windows Server**: `C:\ProgramData\docker\config\daemon.json`
- **Docker Desktop for Mac / Docker Desktop for Windows**: Click the Docker icon in the toolbar, - **Docker Desktop**: Open the Docker Desktop settings and select **Docker Engine**.
select **Settings**, then select **Docker Engine**.
If the file is currently empty, paste the following: If the file is currently empty, paste the following:
```json ```json
{ {
"metrics-addr" : "127.0.0.1:9323" "metrics-addr": "127.0.0.1:9323"
} }
``` ```
If the file is not empty, add the new key, making sure that the resulting If the file isn't empty, add the new key, making sure that the resulting
file is valid JSON. Be careful that every line ends with a comma (`,`) except file is valid JSON. Be careful that every line ends with a comma (`,`) except
for the last line. for the last line.
@ -69,14 +68,14 @@ except for the addition of the Docker job definition at the bottom of the file.
```yml ```yml
# my global config # my global config
global: global:
scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute. scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute. evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
# scrape_timeout is set to the global default (10s). # scrape_timeout is set to the global default (10s).
# Attach these labels to any time series or alerts when communicating with # Attach these labels to any time series or alerts when communicating with
# external systems (federation, remote storage, Alertmanager). # external systems (federation, remote storage, Alertmanager).
external_labels: external_labels:
monitor: 'codelab-monitor' monitor: "codelab-monitor"
# Load rules once and periodically evaluate them according to the global 'evaluation_interval'. # Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files: rule_files:
@ -87,20 +86,21 @@ rule_files:
# Here it's Prometheus itself. # Here it's Prometheus itself.
scrape_configs: scrape_configs:
# The job name is added as a label `job=<job_name>` to any timeseries scraped from this config. # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
- job_name: 'prometheus' - job_name: "prometheus"
# metrics_path defaults to '/metrics' # metrics_path defaults to '/metrics'
# scheme defaults to 'http'. # scheme defaults to 'http'.
static_configs: static_configs:
- targets: ['host.docker.internal:9090'] - targets: ["host.docker.internal:9090"]
- job_name: 'docker' - job_name:
# metrics_path defaults to '/metrics' "docker"
# scheme defaults to 'http'. # metrics_path defaults to '/metrics'
# scheme defaults to 'http'.
static_configs: static_configs:
- targets: ['localhost:9323'] - targets: ["localhost:9323"]
``` ```
Next, start a single-replica Prometheus service using this configuration. Next, start a single-replica Prometheus service using this configuration.
@ -124,16 +124,16 @@ Next, start a single-replica Prometheus service using this configuration.
prom/prometheus prom/prometheus
``` ```
Verify that the Docker target is listed at http://localhost:9090/targets/. Verify that the Docker target is listed at `http://localhost:9090/targets/`.
![Prometheus targets page](images/prometheus-targets.png) ![Prometheus targets page](images/prometheus-targets.png)
You can't access the endpoint URLs directly if you use Docker Desktop You can't access the endpoint URLs directly if you use Docker Desktop
for Mac or Docker Desktop for Windows. for Mac or Docker Desktop for Windows.
## Use Prometheus ## Use Prometheus
Create a graph. Click the **Graphs** link in the Prometheus UI. Choose a metric Create a graph. Select the **Graphs** link in the Prometheus UI. Choose a metric
from the combo box to the right of the **Execute** button, and click from the combo box to the right of the **Execute** button, and click
**Execute**. The screenshot below shows the graph for **Execute**. The screenshot below shows the graph for
`engine_daemon_network_actions_seconds_count`. `engine_daemon_network_actions_seconds_count`.
@ -160,7 +160,7 @@ your graph.
![Prometheus engine_daemon_network_actions_seconds_count report](images/prometheus-graph_load.png) ![Prometheus engine_daemon_network_actions_seconds_count report](images/prometheus-graph_load.png)
When you are ready, stop and remove the `ping_service` service, so that you When you are ready, stop and remove the `ping_service` service, so that you
are not flooding a host with pings for no reason. aren't flooding a host with pings for no reason.
```console ```console
$ docker service remove ping_service $ docker service remove ping_service
@ -169,8 +169,7 @@ $ docker service remove ping_service
Wait a few minutes and you should see that the graph falls back to the idle Wait a few minutes and you should see that the graph falls back to the idle
level. level.
## Next steps ## Next steps
- Read the [Prometheus documentation](https://prometheus.io/docs/introduction/overview/) - Read the [Prometheus documentation](https://prometheus.io/docs/introduction/overview/)
- Set up some [alerts](https://prometheus.io/docs/alerting/overview/) - Set up some [alerts](https://prometheus.io/docs/alerting/overview/)