diff --git a/content/scout/images/scout-metrics-grafana.png b/content/scout/images/scout-metrics-grafana.png new file mode 100644 index 0000000000..0e49733e0d Binary files /dev/null and b/content/scout/images/scout-metrics-grafana.png differ diff --git a/content/scout/images/scout-metrics-prom-target.png b/content/scout/images/scout-metrics-prom-target.png new file mode 100644 index 0000000000..6b040fcec6 Binary files /dev/null and b/content/scout/images/scout-metrics-prom-target.png differ diff --git a/content/scout/metrics-exporter.md b/content/scout/metrics-exporter.md new file mode 100644 index 0000000000..758016ceb9 --- /dev/null +++ b/content/scout/metrics-exporter.md @@ -0,0 +1,183 @@ +--- +title: Docker Scout metrics exporter +description: | + Learn how to scrape data from Docker Scout using Prometheus to create your own + vulnerability and policy dashboards wiht Grafana +keywords: scout, exporter, prometheus, grafana, metrics, dashboard, api, compose +--- + +Docker Scout exposes a metrics HTTP endpoint that lets you scrape vulnerability +and policy data from Docker Scout, using Prometheus. With this you can create +your own, self-hosted Docker Scout dashboards for visualizing supply chain +metrics. + +## Metrics + +The metrics endpoint exposes the following metrics: + +| Metric | Description | Labels | Type | +| ------------------------------- | --------------------------------------------------- | --------------------------------- | ----- | +| `scout_stream_vulnerabilities` | Vulnerabilities in a stream | `streamName`, `severity` | Gauge | +| `scout_policy_compliant_images` | Compliant images for a policy in a stream | `id`, `displayName`, `streamName` | Gauge | +| `scout_policy_evaluated_images` | Total images evaluated against a policy in a stream | `id`, `displayName`, `streamName` | Gauge | + +> **Streams** +> +> In Docker Scout, the streams concept is a superset of [environments](./integrations/environment/_index.md). +> Streams include all runtime environments that you've defined, +> as well as the special `latest-indexed` stream. +> The `latest-indexed` stream contains the most recently pushed (and analyzed) tag for each repository. +> +> Streams is mostly an internal concept in Docker Scout, +> with the exception of the data exposed through this metrics endpoint. +{ .tip } + +## Creating an access token + +To export metrics from your organization, first make sure your organization is enrolled in Docker Scout. +Then, create a Personal Access Token (PAT) - a secret token that allows the exporter to authenticate with the Docker Scout API. + +The PAT does not require any specific permissions, but it must be created by a user who is an owner of the Docker organization. +To create a PAT, follow the steps in [Create an access token](/security/for-developers/access-tokens/#create-an-access-token). + +Once you have created the PAT, store it in a secure location. +You will need to provide this token to the exporter when scraping metrics. + +## Configure Prometheus + +### Add a job for your organization + +In the Prometheus configuration file, add a new job for your organization. +The job should include the following configuration; +replace `ORG` with your organization name: + +```yaml +scrape_configs: + - job_name: + metrics_path: /v1/exporter/org/ + scheme: https + static_configs: + - targets: + - api.scout.docker.com +``` + +The address in the `targets` field is set to the domain name of the Docker Scout API, `api.scout.docker.com`. +Make sure that there's no firewall rule in place preventing the server from communicating with this endpoint. + +### Scrape interval + +By default, Prometheus scrapes the metrics every 15 seconds. +You can change the scrape interval by setting the `scrape_interval` field in the Prometheus configuration file at the global or job level. +A scraping interval of 60 minutes or higher is recommended. + +Because of the own nature of vulnerability data, the metrics exposed through this API are unlikely to change at a high frequency. +For this reason, the metrics endpoint has a 60-minute cache by default. +If you set the scrape interval to less than 60 minutes, you will see the same data in the metrics for multiple scrapes during that time window. + +### Add bearer token authentication + +To scrape metrics from the Docker Scout Exporter endpoint using Prometheus, you need to configure Prometheus to use the PAT as a bearer token. +The exporter requires the PAT to be passed in the `Authorization` header of the request. + +Update the Prometheus configuration file to include the `authorization` configuration block. +This block defines the PAT as a bearer token stored in a file: + +```yaml +scrape_configs: + - job_name: $ORG + authorization: + type: Bearer + credentials_file: /etc/prometheus/token +``` + +The content of the file should be the PAT in plain text: + +```console +dckr_pat_... +``` + +If you are running Prometheus in a Docker container or Kubernetes pod, mount the file into the container using a volume or secret. + +Finally, restart Prometheus to apply the changes. + +## Sample project + +If you don't have a Prometheus server set up, you can run a [sample project](https://github.com/dockersamples/scout-metrics-exporter) using Docker Compose. +The sample includes a Prometheus server that scrapes metrics for a Docker organization enrolled in Docker Scout, +alongside Grafana with a pre-configured dashboard to visualize the vulnerability and policy metrics. + +1. Clone the starter template for bootstrapping a set of Compose services + for scraping and visualizing the Docker Scout metrics endpoint: + + ```console + $ git clone git@github.com:dockersamples/scout-metrics-exporter.git + ``` + +2. [Create a Docker access token](/security/for-developers/access-tokens/#create-an-access-token) + and store it in a plain text file at `prometheus/token` under the template directory. + + ```plaintext {title=token} + $ echo $DOCKER_PAT > ./prometheus/token + ``` + +3. In the Prometheus configuration file at `prometheus/prometheus.yml`, + replace `ORG` in the `metrics_path` property on line 6 with the namespace of your Docker organization. + + ```yaml {title="prometheus/prometheus.yml",hl_lines="6",linenos=1} + global: + scrape_interval: 60s + scrape_timeout: 40s + scrape_configs: + - job_name: Docker Scout policy + metrics_path: /v1/exporter/org/ + scheme: https + static_configs: + - targets: + - api.scout.docker.com + authorization: + type: Bearer + credentials_file: /etc/prometheus/token + ``` + +4. Start the compose services. + + ```console + docker compose up -d + ``` + + This command starts two services: the Prometheus server and Grafana. + Prometheus scrapes metrics from the Docker Scout endpoint, + and Grafana visualizes the metrics using a pre-configured dashboard. + +To stop the demo and clean up any resources created, run: + +```console +docker compose down -v +``` + +### Access to Prometheus + +After starting the services, you can access the Prometheus expression browser by visiting . +The Prometheus server runs in a Docker container and is accessible on port 9090. + +After a few seconds, you should see the metrics endpoint as a target in the +Prometheus UI at . + +![Docker Scout metrics exporter Prometheus target](./images/scout-metrics-prom-target.png "Docker Scout metrics exporter Prometheus target") + +### Viewing the metrics in Grafana + +To view the Grafana dashboards, go to , +and sign in using the credentials defined in the Docker Compose file (username: `admin`, password: `grafana`). + +![Policy dashboard in Grafana](./images/scout-metrics-grafana.png "Policy dashboard in Grafana") + +The dashboards are pre-configured to visualize the vulnerability and policy metrics scraped by Prometheus. + +### Revoke an access token + +If you suspect that your PAT has been compromised or is no longer needed, you can revoke it at any time. +To revoke a PAT, follow the steps in the [Create and manage access tokens](/security/for-developers/access-tokens/#modify-existing-tokens). + +Revoking a PAT immediately invalidates the token, and prevents Prometheus from scraping metrics using that token. +You will need to create a new PAT and update the Prometheus configuration to use the new token. diff --git a/data/redirects.yml b/data/redirects.yml index 92bf746986..370a05fb7e 100644 --- a/data/redirects.yml +++ b/data/redirects.yml @@ -644,6 +644,8 @@ - "/go/scout-github/" "/scout/guides/vex/": - "/go/vex-guide/" +"/scout/metrics-exporter/": + - "/go/scout-metrics-exporter" # Build links (internal) "/build/bake/reference/": diff --git a/data/toc.yaml b/data/toc.yaml index 5506541bd6..6721431f5a 100644 --- a/data/toc.yaml +++ b/data/toc.yaml @@ -1404,6 +1404,8 @@ Manuals: title: SBOM - path: /scout/env-vars/ title: Environment variables + - path: /scout/metrics-exporter/ + title: Metrics exporter - sectiontitle: Policy Evaluation section: - path: /scout/policy/ diff --git a/layouts/_default/_markup/render-image.html b/layouts/_default/_markup/render-image.html index 8ca7d7576b..d41c920b11 100644 --- a/layouts/_default/_markup/render-image.html +++ b/layouts/_default/_markup/render-image.html @@ -9,7 +9,7 @@ {{ $border := index $params "border" }} -
+ {{ with .Title }} +
{{ . }}
+ {{ end }} -
+