From baaead7cdd1777f41fe8061bcf9c47badb4637a3 Mon Sep 17 00:00:00 2001 From: Misty Stanley-Jones Date: Wed, 14 Jun 2017 14:37:37 -0700 Subject: [PATCH] Update logging driver options from code --- engine/admin/logging/awslogs.md | 66 ++++++++++++++++++++++++++++--- engine/admin/logging/fluentd.md | 59 +++++++++++++++++++++------ engine/admin/logging/gcplogs.md | 63 ++++++++++++++++++----------- engine/admin/logging/gelf.md | 38 +++++++++++++----- engine/admin/logging/journald.md | 36 +++++++++++------ engine/admin/logging/json-file.md | 34 ++++++++-------- engine/admin/logging/splunk.md | 55 ++++++++++++++++---------- engine/admin/logging/syslog.md | 56 ++++++++++++++------------ 8 files changed, 285 insertions(+), 122 deletions(-) diff --git a/engine/admin/logging/awslogs.md b/engine/admin/logging/awslogs.md index 963fe9d355..e8d296dfcd 100644 --- a/engine/admin/logging/awslogs.md +++ b/engine/admin/logging/awslogs.md @@ -14,10 +14,25 @@ and Command Line Tools](http://docs.aws.amazon.com/cli/latest/reference/logs/ind ## Usage -You can configure the default logging driver by passing the `--log-driver` -option to the Docker daemon: +To use the `awslogs` driver as the default logging driver, set the `log-driver` +and `log-opt` keys to appropriate values in the `daemon.json` file, which is +located in `/etc/docker/` on Linux hosts or +`C:\ProgramData\docker\config\daemon.json` on Windows Server. For more about +configuring Docker using `daemon.json`, see +[daemon.json](/engine/reference/commandline/dockerd.md#daemon-configuration-file). +The following example sets the log driver to `awslogs` and sets the +`awslogs-region` option. - dockerd --log-driver=awslogs +```json +{ + "log-driver": "awslogs", + "log-opts": { + "awslogs-region": "us-east-1" + } +} +``` + +Restart Docker for the changes to take effect. You can set the logging driver for a specific container by using the `--log-driver` option to `docker run`: @@ -26,7 +41,9 @@ You can set the logging driver for a specific container by using the ## Amazon CloudWatch Logs options -You can use the `--log-opt NAME=VALUE` flag to specify Amazon CloudWatch Logs logging driver options. +You can add logging options to the `daemon.json` to set Docker-wide defaults, +or use the `--log-opt NAME=VALUE` flag to specify Amazon CloudWatch Logs +logging driver options when starting a container. ### awslogs-region @@ -77,6 +94,43 @@ $ docker run --log-driver=awslogs \ > **Note**: > Your AWS IAM policy must include the `logs:CreateLogGroup` permission before you attempt to use `awslogs-create-group`. +### aws-datetime-format + +The `aws-datetime-format` option defines a multiline start pattern in [Python +`strftime` format](http://strftime.org). One example of a use case for using +this format is for parsing output such as a stack dump, which might otherwise +be logged in multiple entries. The correct pattern allows it to be captured in a +single entry. + +This option always takes precedence if both `awslogs-datetime-format` and +`awslogs-multiline-pattern` are configured. + +For example: + +```bash +$ docker run -it --rm \ + --log-driver=awslogs \ + --log-opt awslogs-group=test \ + --log-opt awslogs-datetime-format='%Y-%m-%d' \ + awslogtest:latest /test2.sh +``` + +### aws-multiline-pattern + +The `aws-multiline-pattern` option defines a multiline start pattern using a +regular expression. This option is ignored if `awslogs-datetime-format` is also +configured. + +For example: + +```bash +$ docker run -it --rm \ + --log-driver=awslogs \ + --log-opt awslogs-group=test \ + --log-opt awslogs-multiline-pattern='^ABCD' \ + awslogtest:latest /test1.sh + +``` ### tag @@ -90,9 +144,9 @@ If not specified, the container ID is used as the log stream. {% raw %} > **Note**: > The CloudWatch log API doesn't support `:` in the log name. This can cause some issues when using the `{{ .ImageName }}` as a tag, since a docker image has a format of `IMAGE:TAG`, such as `alpine:latest`. -> Template markup can be used to get the proper format. +> Template markup can be used to get the proper format. > To get the image name and the first 12 characters of the container id, you can use: `--log-opt tag='{{ with split .ImageName ":" }}{{join . "_"}}{{end}}-{{.ID}}'` -> the output will be something like: `alpine_latest-bf0072049c76` +> the output will be something like: `alpine_latest-bf0072049c76` {% endraw %} diff --git a/engine/admin/logging/fluentd.md b/engine/admin/logging/fluentd.md index 45a5aef71d..e686baa32d 100644 --- a/engine/admin/logging/fluentd.md +++ b/engine/admin/logging/fluentd.md @@ -16,11 +16,11 @@ destinations. In addition to the log message itself, the `fluentd` log driver sends the following metadata in the structured log message: -| Field | Description | --------------------|-------------------------------------| -| `container_id` | The full 64-character container ID. | -| `container_name` | The container name at the time it was started. If you use `docker rename` to rename a container, the new name is not reflected in the journal entries. | -| `source` | `stdout` or `stderr` | +| Field | Description | +|:-----------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------| +| `container_id` | The full 64-character container ID. | +| `container_name` | The container name at the time it was started. If you use `docker rename` to rename a container, the new name is not reflected in the journal entries. | +| `source` | `stdout` or `stderr` | The `docker logs` command is not available for this logging driver. @@ -32,10 +32,26 @@ Some options are supported by specifying `--log-opt` as many times as needed: - `tag`: specify tag for fluentd message, which interpret some markup, ex {% raw %}`{{.ID}}`, `{{.FullID}}` or `{{.Name}}` `docker.{{.ID}}`{% endraw %} -Configure the default logging driver by passing the -`--log-driver` option to the Docker daemon: + To use the `fluentd` driver as the default logging driver, set the `log-driver` + and `log-opt` keys to appropriate values in the `daemon.json` file, which is + located in `/etc/docker/` on Linux hosts or + `C:\ProgramData\docker\config\daemon.json` on Windows Server. For more about + +configuring Docker using `daemon.json`, see + +[daemon.json](/engine/reference/commandline/dockerd.md#daemon-configuration-file). - dockerd --log-driver=fluentd +The following example sets the log driver to `fluentd` and sets the +`fluentd-address` option. + + ```json + { + "log-driver": "fluentd", + "log-opts": { + "fluentd-address": "fluentdhost:24224" + } + } + ``` + + Restart Docker for the changes to take effect. To set the logging driver for a specific container, pass the `--log-driver` option to `docker run`: @@ -73,13 +89,34 @@ Refer to the [log tag option documentation](log_tags.md) for customizing the log tag format. -### labels and env +### labels, env, and env-regex -The `labels` and `env` options each take a comma-separated list of keys. If there is collision between `label` and `env` keys, the value of the `env` takes precedence. Both options add additional fields to the extra attributes of a logging message. +The `labels` and `env` options each take a comma-separated list of keys. If +there is collision between `label` and `env` keys, the value of the `env` takes +precedence. Both options add additional fields to the extra attributes of a +logging message. + +The `env-regex` option is similar to and compatible with `env`. Its value is a +regular expression to match logging-related environment variables. It is used +for advanced [log tag options](log_tags.md). ### fluentd-async-connect -Docker connects to Fluentd in the background. Messages are buffered until the connection is established. +Docker connects to Fluentd in the background. Messages are buffered until the +connection is established. + +### fluentd-buffer-limit + +The amount of data to buffer before flushing to disk. Defaults the amount of RAM +available to the container. + +### fluentd-retry-wait + +How long to wait between retries. Defaults to 1 second. + +### fluentd-max-retries + +The maximum number of retries. Defaults to 10. ## Fluentd daemon management with Docker diff --git a/engine/admin/logging/gcplogs.md b/engine/admin/logging/gcplogs.md index 474fb00544..3d8cb1c440 100644 --- a/engine/admin/logging/gcplogs.md +++ b/engine/admin/logging/gcplogs.md @@ -4,16 +4,32 @@ keywords: gcplogs, google, docker, logging, driver title: Google Cloud Logging driver --- -The Google Cloud Logging driver sends container logs to Google Cloud -Logging. +The Google Cloud Logging driver sends container logs to +[Google Cloud Logging](https://cloud.google.com/logging/docs/){: target="_blank" class="_" } +Logging. ## Usage -You can configure the default logging driver by passing the `--log-driver` -option to the Docker daemon: +To use the `gcplogs` driver as the default logging driver, set the `log-driver` +and `log-opt` keys to appropriate values in the `daemon.json` file, which is +located in `/etc/docker/` on Linux hosts or +`C:\ProgramData\docker\config\daemon.json` on Windows Server. For more about +configuring Docker using `daemon.json`, see +[daemon.json](/engine/reference/commandline/dockerd.md#daemon-configuration-file). - dockerd --log-driver=gcplogs +The following example sets the log driver to `gcplogs` and sets the +`gcp-meta-name` option. + +```json +{ + "log-driver": "gcplogs", + "log-opts": { + "gcp-meta-name": "example-instance-12345" + } +} +``` + +Restart Docker for the changes to take effect. You can set the logging driver for a specific container by using the `--log-driver` option to `docker run`: @@ -24,15 +40,15 @@ This log driver does not implement a reader so it is incompatible with `docker logs`. If Docker detects that it is running in a Google Cloud Project, it will discover -configuration from the instance metadata service. Otherwise, the user must specify +configuration from the +[instance metadata service](https://cloud.google.com/compute/docs/metadata){: target="_blank" class="_"} +Otherwise, the user must specify which project to log to using the `--gcp-project` log option and Docker will -attempt to obtain credentials from the Google Application Default Credential. The `--gcp-project` -takes precedence over information discovered from the metadata server so a -Docker daemon running in a Google Cloud Project can be overridden to log to a -different Google Cloud Project using `--gcp-project`. +attempt to obtain credentials from the +[Google Application Default Credential](https://developers.google.com/identity/protocols/application-default-credentials){: target="_blank" class="_"}. +The `--gcp-project` flag takes precedence over information discovered from the +metadata server so a Docker daemon running in a Google Cloud Project can be +overridden to log to a different Google Cloud Project using `--gcp-project`. Docker fetches the values for zone, instance name and instance id from Google Cloud metadata server. Those values can be provided via options if metadata @@ -43,15 +59,16 @@ server is not available. They will not override the values from metadata server. You can use the `--log-opt NAME=VALUE` flag to specify these additional Google Cloud Logging driver options: -| Option | Required | Description | -|-----------------------------|----------|---------------------------------------------------------------------------------------------------------------------------------------------| -| `gcp-project` | optional | Which GCP project to log to. Defaults to discovering this value from the GCE metadata service. | -| `gcp-log-cmd` | optional | Whether to log the command that the container was started with. Defaults to false. | -| `labels` | optional | Comma-separated list of keys of labels, which should be included in message, if these labels are specified for container. | -| `env` | optional | Comma-separated list of keys of environment variables, which should be included in message, if these variables are specified for container. | -| `gcp-meta-zone` | optional | Zone name for the instance. | -| `gcp-meta-name` | optional | Instance name. | -| `gcp-meta-id` | optional | Instance ID. | +| Option | Required | Description | +|:----------------|:---------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `gcp-project` | optional | Which GCP project to log to. Defaults to discovering this value from the GCE metadata service. | +| `gcp-log-cmd` | optional | Whether to log the command that the container was started with. Defaults to false. | +| `labels` | optional | Comma-separated list of keys of labels, which should be included in message, if these labels are specified for container. | +| `env` | optional | Comma-separated list of keys of environment variables, which should be included in message, if these variables are specified for container. | +| `env-regex` | optional | Similar to and compatible with `env`. A regular expression to match logging-related environment variables. Used for advanced [log tag options](log_tags.md). | +| `gcp-meta-zone` | optional | Zone name for the instance. | +| `gcp-meta-name` | optional | Instance name. | +| `gcp-meta-id` | optional | Instance ID. | If there is collision between `label` and `env` keys, the value of the `env` takes precedence. Both options add additional fields to the attributes of a diff --git a/engine/admin/logging/gelf.md b/engine/admin/logging/gelf.md index 783dd8c734..5590a3cbb2 100644 --- a/engine/admin/logging/gelf.md +++ b/engine/admin/logging/gelf.md @@ -20,6 +20,26 @@ In GELF, every log message is a dict with the following fields: ## Usage +To use the `gelf` driver as the default logging driver, set the `log-driver` and +`log-opt` keys to appropriate values in the `daemon.json` file, which is located +in `/etc/docker/` on Linux hosts or `C:\ProgramData\docker\config\daemon.json` +on Windows Server. For more about configuring Docker using `daemon.json`, see +[daemon.json](/engine/reference/commandline/dockerd.md#daemon-configuration-file). + +The following example sets the log driver to `gelf` and sets the `gelf-address` +option. + +```json +{ + "log-driver": "gelf", + "log-opts": { + "gelf-address": "udp://1.2.3.4:12201" + } +} +``` + +Restart Docker for the changes to take effect. + To use `gelf` as the default logging driver for new containers, pass the `--log-driver` and `--log-opt` options to the Docker daemon: @@ -52,15 +72,15 @@ $ docker run \ The `gelf` logging driver supports the following options: -| Option | Description | Example value | -|----------------------|--------------------------|-------------------------------------------| -| `gelf-address` | The address of the GELF server. `udp` is the only supported URI specifier and you must specify the port.| `--log-opt gelf-address=udp://192.168.0.42:12201` | -| `gelf-compression-type` | The type of compression the GELF driver uses to compress each log message. Allowed values are `gzip`, `zlib` and `none`. The default is `gzip`. | `--log-opt gelf-compression-type=gzip` | -| `gelf-compression-level` | The level of compression when `gzip` or `zlib` is the `gelf-compression-type`. An integer in the range of `-1` to `9` (BestCompression). Default value is 1 (BestSpeed). Higher levels provide more compression at lower speed.| `--log-opt gelf-compression-level=2` | -| `tag` | A string that is appended to the `APP-NAME` in the `gelf` message. By default, Docker uses the first 12 characters of the container ID to tag log messages. Refer to the [log tag option documentation](log_tags.md) for customizing the log tag format. | `--log-opt tag=mailer` | -| `labels` | Applies when starting the Docker daemon. A comma-separated list of logging-related labels this daemon will accept. Adds additional key on the `extra` fields, prefixed by an underscore (`_`). Used for advanced [log tag options](log_tags.md).| `--log-opt labels=production_status,geo` | -| `env` | Applies when starting the Docker daemon. A comma-separated list of logging-related environment variables this daemon will accept. Adds additional key on the `extra` fields, prefixed by an underscore (`_`). Used for advanced [log tag options](log_tags.md). | `--log-opt env=os,customer` | - +| Option | Description | Example value | +| :------------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :-------------------------------------------------- | +| `gelf-address` | The address of the GELF server. `udp` is the only supported URI specifier and you must specify the port. | `--log-opt gelf-address=udp://192.168.0.42:12201` | +| `gelf-compression-type` | The type of compression the GELF driver uses to compress each log message. Allowed values are `gzip`, `zlib` and `none`. The default is `gzip`. | `--log-opt gelf-compression-type=gzip` | +| `gelf-compression-level` | The level of compression when `gzip` or `zlib` is the `gelf-compression-type`. An integer in the range of `-1` to `9` (BestCompression). Default value is 1 (BestSpeed). Higher levels provide more compression at lower speed. Either `-1` or `0` disables compression. | `--log-opt gelf-compression-level=2` | +| `tag` | A string that is appended to the `APP-NAME` in the `gelf` message. By default, Docker uses the first 12 characters of the container ID to tag log messages. Refer to the [log tag option documentation](log_tags.md) for customizing the log tag format. | `--log-opt tag=mailer` | +| `labels` | Applies when starting the Docker daemon. A comma-separated list of logging-related labels this daemon will accept. Adds additional key on the `extra` fields, prefixed by an underscore (`_`). Used for advanced [log tag options](log_tags.md). | `--log-opt labels=production_status,geo` | +| `env` | Applies when starting the Docker daemon. A comma-separated list of logging-related environment variables this daemon will accept. Adds additional key on the `extra` fields, prefixed by an underscore (`_`). Used for advanced [log tag options](log_tags.md). | `--log-opt env=os,customer` | +| `env-regex` | Similar to and compatible with `env`. A regular expression to match logging-related environment variables. Used for advanced [log tag options](log_tags.md). | `--log-opt env-regex=^(os | customer).` | ### Examples diff --git a/engine/admin/logging/journald.md b/engine/admin/logging/journald.md index 9002e3f913..d61da4117b 100644 --- a/engine/admin/logging/journald.md +++ b/engine/admin/logging/journald.md @@ -14,23 +14,33 @@ Log entries can be retrieved using the `journalctl` command, through use of the In addition to the text of the log message itself, the `journald` log driver stores the following metadata in the journal with each message: -| Field | Description | -------------------------------|-------------| -| `CONTAINER_ID` | The container ID truncated to 12 characters. | -| `CONTAINER_ID_FULL` | The full 64-character container ID. | +| Field | Description | +|:----------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------| +| `CONTAINER_ID` | The container ID truncated to 12 characters. | +| `CONTAINER_ID_FULL` | The full 64-character container ID. | | `CONTAINER_NAME` | The container name at the time it was started. If you use `docker rename` to rename a container, the new name is not reflected in the journal entries. | -| `CONTAINER_TAG` | The container tag ([log tag option documentation](log_tags.md)). | -| `CONTAINER_PARTIAL_MESSAGE` | A field that flags log integrity. Improve logging of long log lines. | +| `CONTAINER_TAG` | The container tag ([log tag option documentation](log_tags.md)). | +| `CONTAINER_PARTIAL_MESSAGE` | A field that flags log integrity. Improve logging of long log lines. | ## Usage -Configure the default logging driver by passing the `--log-driver` option to the -Docker daemon: +To use the `journald` driver as the default logging driver, set the `log-driver` +and `log-opt` keys to appropriate values in the `daemon.json` file, which is +located in `/etc/docker/` on Linux hosts or +`C:\ProgramData\docker\config\daemon.json` on Windows Server. For more about ++configuring Docker using `daemon.json`, see ++[daemon.json](/engine/reference/commandline/dockerd.md#daemon-configuration-file). -```bash -$ dockerd --log-driver=journald +The following example sets the log driver to `journald`: + +```json +{ + "log-driver": "journald" +} ``` +Restart Docker for the changes to take effect. + To configure the logging driver for a specific container, use the `--log-driver` flag on the `docker run` command. @@ -48,13 +58,17 @@ driver options. Specify template to set `CONTAINER_TAG` value in `journald` logs. Refer to [log tag option documentation](log_tags.md) to customize the log tag format. -### `labels` and `env` +### `labels`, `env`, and `eng-regex` The `labels` and `env` options each take a comma-separated list of keys. If there is collision between `label` and `env` keys, the value of the `env` takes precedence. Each option adds additional metadata to the journal with each message. +`env-regex` is similar to and compatible with `env`. Set it to a regular +expression to match logging-related environment variables. It is used for +advanced [log tag options](log_tags.md). + ## Note regarding container names The value logged in the `CONTAINER_NAME` field is the name of the container that diff --git a/engine/admin/logging/json-file.md b/engine/admin/logging/json-file.md index ae304a24ef..b7a628da6d 100644 --- a/engine/admin/logging/json-file.md +++ b/engine/admin/logging/json-file.md @@ -14,24 +14,25 @@ only one container. ## Usage -Docker uses the JSON File logging driver by default. To explicitly set it, -use the `--log-driver` and `--log-opt` flags when starting Docker: +To use the `syslog` driver as the default logging driver, set the `log-driver` +and `log-opt` keys to appropriate values in the `daemon.json` file, which is +located in `/etc/docker/` on Linux hosts or +`C:\ProgramData\docker\config\daemon.json` on Windows Server. For more about ++configuring Docker using `daemon.json`, see ++[daemon.json](/engine/reference/commandline/dockerd.md#daemon-configuration-file). -```bash -dockerd - -–log-driver json-file –-log-opt max-size=10m -``` - -To make the change persistent, configure it in `/etc/docker/daemon.json`: +The following example sets the log driver to `json-file` and sets the `max-size` +option. ```json { "log-driver": "json-file", - "log-opts": { + "log-opts": { "max-size": "10m" } } -``` + +Restart Docker for the changes to take effect. You can set the logging driver for a specific container by using the `--log-driver` flag to `docker create` or `docker run`: @@ -46,12 +47,13 @@ $ docker run \ The `json-file` logging driver supports the following logging options: -| Option | Description | Example value | -|------------|--------------------------|-------------------------------------------| -| `max-size` | The maximum size of the log before it is rolled. A positive integer plus a modifier representing the unit of measure (`k`, `m`, or `g`). | `--log-opt max-size=10m` | -| `max-file` | The maximum number of log files that can be present. If rolling the logs creates excess files, the oldest file is removed. **Only effective when `max-size` is also set.** A positive integer. | `--log-opt max-file=3` | -| `labels` | Applies when starting the Docker daemon. A comma-separated list of logging-related labels this daemon will accept. Used for advanced [log tag options](log_tags.md).| `--log-opt labels=production_status,geo` | -| `env` | Applies when starting the Docker daemon. A comma-separated list of logging-related environment variables this daemon will accept. Used for advanced [log tag options](log_tags.md). | `--log-opt env=os,customer` | +| Option | Description | Example value | +|:------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------------| +| `max-size` | The maximum size of the log before it is rolled. A positive integer plus a modifier representing the unit of measure (`k`, `m`, or `g`). | `--log-opt max-size=10m` | +| `max-file` | The maximum number of log files that can be present. If rolling the logs creates excess files, the oldest file is removed. **Only effective when `max-size` is also set.** A positive integer. | `--log-opt max-file=3` | +| `labels` | Applies when starting the Docker daemon. A comma-separated list of logging-related labels this daemon will accept. Used for advanced [log tag options](log_tags.md). | `--log-opt labels=production_status,geo` | +| `env` | Applies when starting the Docker daemon. A comma-separated list of logging-related environment variables this daemon will accept. Used for advanced [log tag options](log_tags.md). | `--log-opt env=os,customer` | +| `env-regex` | Similar to and compatible with `env`. A regular expression to match logging-related environment variables. Used for advanced [log tag options](log_tags.md). | `--log-opt env-regex=^(os|customer).` | > **Note**: If `max-size` and `max-file` are set, `docker logs` only returns the > log lines from the newest log file. diff --git a/engine/admin/logging/splunk.md b/engine/admin/logging/splunk.md index 3287f39328..df66251c3d 100644 --- a/engine/admin/logging/splunk.md +++ b/engine/admin/logging/splunk.md @@ -12,10 +12,22 @@ in Splunk Enterprise and Splunk Cloud. ## Usage -You can configure the default logging driver by passing the `--log-driver` -option to the Docker daemon: +To use the `splunk` driver as the default logging driver, set the `log-driver` +and `log-opt` keys to appropriate values in the `daemon.json` file, which is +located in `/etc/docker/` on Linux hosts or +`C:\ProgramData\docker\config\daemon.json` on Windows Server. For more about ++configuring Docker using `daemon.json`, see ++[daemon.json](/engine/reference/commandline/dockerd.md#daemon-configuration-file). - dockerd --log-driver=splunk +The following example sets the log driver to `splunk`. + +```json +{ + "log-driver": "splunk" +} +``` + +Restart Docker for the changes to take effect. You can set the logging driver for a specific container by using the `--log-driver` option to `docker run`: @@ -27,23 +39,24 @@ You can set the logging driver for a specific container by using the You can use the `--log-opt NAME=VALUE` flag to specify these additional Splunk logging driver options: -| Option | Required | Description | -|-----------------------------|----------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `splunk-token` | required | Splunk HTTP Event Collector token. | -| `splunk-url` | required | Path to your Splunk Enterprise, self-service Splunk Cloud instance, or Splunk Cloud managed cluster (including port and scheme used by HTTP Event Collector) in one of the following formats: `https://your_splunk_instance:8088` or `https://input-prd-p-XXXXXXX.cloud.splunk.com:8088` or `https://http-inputs-XXXXXXXX.splunkcloud.com`. | -| `splunk-source` | optional | Event source. | -| `splunk-sourcetype` | optional | Event source type. | -| `splunk-index` | optional | Event index. | -| `splunk-capath` | optional | Path to root certificate. | -| `splunk-caname` | optional | Name to use for validating server certificate; by default the hostname of the `splunk-url` will be used. | -| `splunk-insecureskipverify` | optional | Ignore server certificate validation. | -| `splunk-format` | optional | Message format. Can be `inline`, `json` or `raw`. Defaults to `inline`. | -| `splunk-verify-connection` | optional | Verify on start, that docker can connect to Splunk server. Defaults to true. | -| `splunk-gzip` | optional | Enable/disable gzip compression to send events to Splunk Enterprise or Splunk Cloud instance. Defaults to false. | -| `splunk-gzip-level` | optional | Set compression level for gzip. Valid values are -1 (default), 0 (no compression), 1 (best speed) ... 9 (best compression). Defaults to [DefaultCompression](https://golang.org/pkg/compress/gzip/#DefaultCompression). | -| `tag` | optional | Specify tag for message, which interpret some markup. Default value is {% raw %}`{{.ID}}`{% endraw %} (12 characters of the container ID). Refer to the [log tag option documentation](log_tags.md) for customizing the log tag format. | -| `labels` | optional | Comma-separated list of keys of labels, which should be included in message, if these labels are specified for container. | -| `env` | optional | Comma-separated list of keys of environment variables, which should be included in message, if these variables are specified for container. | +| Option | Required | Description | +|:----------------------------|:---------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `splunk-token` | required | Splunk HTTP Event Collector token. | +| `splunk-url` | required | Path to your Splunk Enterprise, self-service Splunk Cloud instance, or Splunk Cloud managed cluster (including port and scheme used by HTTP Event Collector) in one of the following formats: `https://your_splunk_instance:8088` or `https://input-prd-p-XXXXXXX.cloud.splunk.com:8088` or `https://http-inputs-XXXXXXXX.splunkcloud.com`. | +| `splunk-source` | optional | Event source. | +| `splunk-sourcetype` | optional | Event source type. | +| `splunk-index` | optional | Event index. | +| `splunk-capath` | optional | Path to root certificate. | +| `splunk-caname` | optional | Name to use for validating server certificate; by default the hostname of the `splunk-url` will be used. | +| `splunk-insecureskipverify` | optional | Ignore server certificate validation. | +| `splunk-format` | optional | Message format. Can be `inline`, `json` or `raw`. Defaults to `inline`. | +| `splunk-verify-connection` | optional | Verify on start, that docker can connect to Splunk server. Defaults to true. | +| `splunk-gzip` | optional | Enable/disable gzip compression to send events to Splunk Enterprise or Splunk Cloud instance. Defaults to false. | +| `splunk-gzip-level` | optional | Set compression level for gzip. Valid values are -1 (default), 0 (no compression), 1 (best speed) ... 9 (best compression). Defaults to [DefaultCompression](https://golang.org/pkg/compress/gzip/#DefaultCompression). | +| `tag` | optional | Specify tag for message, which interpret some markup. Default value is {% raw %}`{{.ID}}`{% endraw %} (12 characters of the container ID). Refer to the [log tag option documentation](log_tags.md) for customizing the log tag format. | +| `labels` | optional | Comma-separated list of keys of labels, which should be included in message, if these labels are specified for container. | +| `env` | optional | Comma-separated list of keys of environment variables, which should be included in message, if these variables are specified for container. | +| `env-regex` | optional | Similar to and compatible with `env`. A regular expression to match logging-related environment variables. Used for advanced [log tag options](log_tags.md). | If there is collision between `label` and `env` keys, the value of the `env` takes precedence. Both options add additional fields to the attributes of a logging message. @@ -143,7 +156,7 @@ MyImage/MyContainer env1=val1 label1=label1 {"foo": "bar"} Splunk Logging Driver allows you to configure few advanced options by specifying next environment variables for the Docker daemon. | Environment variable name | Default value | Description | -|--------------------------------------------------|---------------|----------------------------------------------------------------------------------------------------------------------------------------------------| +|:-------------------------------------------------|:--------------|:---------------------------------------------------------------------------------------------------------------------------------------------------| | `SPLUNK_LOGGING_DRIVER_POST_MESSAGES_FREQUENCY` | `5s` | If there is nothing to batch how often driver will post messages. You can think about this as the maximum time to wait for more messages to batch. | | `SPLUNK_LOGGING_DRIVER_POST_MESSAGES_BATCH_SIZE` | `1000` | How many messages driver should wait before sending them in one batch. | | `SPLUNK_LOGGING_DRIVER_BUFFER_MAX` | `10 * 1000` | If driver cannot connect to remote server, what is the maximum amount of messages it can hold in buffer for retries. | diff --git a/engine/admin/logging/syslog.md b/engine/admin/logging/syslog.md index 900d9b4426..f5ddfed251 100644 --- a/engine/admin/logging/syslog.md +++ b/engine/admin/logging/syslog.md @@ -32,28 +32,29 @@ The format is defined in [RFC 5424](https://tools.ietf.org/html/rfc5424) and Doc ## Usage -To use the `syslog` driver as the default logging driver, you can use the `--log-driver` and `--log-opt` flags when starting Docker: - -```bash -dockerd \ - --log-driver syslog \ - --log-opt syslog-address=udp://1.2.3.4:1111 -``` - -> **Note**: The syslog-address supports both UDP and TCP. - -To make the changes persistent, add the toptions to `/etc/docker/daemon.json`: +To use the `syslog` driver as the default logging driver, set the `log-driver` +and `log-opt` keys to appropriate values in the `daemon.json` file, which is +located in `/etc/docker/` on Linux hosts or +`C:\ProgramData\docker\config\daemon.json` on Windows Server. For more about ++configuring Docker using `daemon.json`, see ++[daemon.json](/engine/reference/commandline/dockerd.md#daemon-configuration-file). +The following example sets the log driver to `syslog` and sets the +`syslog-address` option. ```json { "log-driver": "syslog", - "log-opts": { + "log-opts": { "syslog": "udp://1.2.3.4:1111" } } ``` +Restart Docker for the changes to take effect. + +> **Note**: The syslog-address supports both UDP and TCP. + You can set the logging driver for a specific container by using the `--log-driver` flag to `docker create` or `docker run`: @@ -65,17 +66,22 @@ docker run \ ## Options -The following logging options are supported for the `syslog` logging driver: +The following logging options are supported as options for the `syslog` logging +driver. They can be set as defaults in the `daemon.json`, by adding them as +key-value pairs to the `log-opts` JSON array. They can also be set on a given +container by adding a `--log-opt =` flag for each option when +starting the container. -| Option | Description | Example value | -|----------------------|--------------------------|-------------------------------------------| -| `syslog-address` | The address of an external `syslog` server. The URI specifier may be `[tcp|udp|tcp+tls]://host:port`, `unix://path`, or `unixgram://path`. If the transport is `tcp`, `udp`, or `tcp+tls`, the default port is `514`.| `--log-opt syslog-address=tcp+tls://192.168.1.3:514`, `--log-opt syslog-address=unix:///tmp/syslog.sock` | -| `syslog-facility` | The `syslog` facility to use. Can be the number or name for any valid `syslog` facility. See the [syslog documentation](https://tools.ietf.org/html/rfc5424#section-6.2.1). | `--log-opt syslog-facility=daemon` | -| `syslog-tls-ca-cert` | The absolute path to the trust certificates signed by the CA. **Ignored if the address protocol is not `tcp+tls`.** | `--log-opt syslog-tls-ca-cert=/etc/ca-certificates/custom/ca.pem` | -| `syslog-tls-cert` | The absolute path to the TLS certificate file. **Ignored if the address protocol is not `tcp+tls`**. | `--log-opt syslog-tls-cert=/etc/ca-certificates/custom/cert.pem` | -| `syslog-tls-key` | The absolute path to the TLS key file. **Ignored if the address protocol is not `tcp+tls`.** | `--log-opt syslog-tls-key=/etc/ca-certificates/custom/key.pem` | -| `syslog-tls-skip` | If set to `true`, TLS verification is skipped when connecting to the `syslog` daemon. Defaults to `false`. **Ignored if the address protocol is not `tcp+tls`.** | `--log-opt syslog-tls-skip-verify=true` | -| `tag` | A string that is appended to the `APP-NAME` in the `syslog` message. By default, Docker uses the first 12 characters of the container ID to tag log messages. Refer to the [log tag option documentation](log_tags.md) for customizing the log tag format. | `--log-opt tag=mailer` | -| `syslog-format` | The `syslog` message format to use. If not specified the local UNIX syslog format is used, without a specified hostname. Specify `rfc3164` for the RFC-3164 compatible format, `rfc5424` for RFC-5424 compatible format, or `rfc5424micro` for RFC-5424 compatible format with microsecond timestamp resolution. | `--log-opt syslog-format=rfc5424micro` | -| `labels` | Applies when starting the Docker daemon. A comma-separated list of logging-related labels this daemon will accept. Used for advanced [log tag options](log_tags.md).| `--log-opt labels=production_status,geo` | -| `env` | Applies when starting the Docker daemon. A comma-separated list of logging-related environment variables this daemon will accept. Used for advanced [log tag options](log_tags.md). | `--log-opt env=os,customer` | +| Option | Description | Example value | +|:-------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `syslog-address` | The address of an external `syslog` server. The URI specifier may be `[tcp | udp|tcp+tls]://host:port`, `unix://path`, or `unixgram://path`. If the transport is `tcp`, `udp`, or `tcp+tls`, the default port is `514`.| `--log-opt syslog-address=tcp+tls://192.168.1.3:514`, `--log-opt syslog-address=unix:///tmp/syslog.sock` | +| `syslog-facility` | The `syslog` facility to use. Can be the number or name for any valid `syslog` facility. See the [syslog documentation](https://tools.ietf.org/html/rfc5424#section-6.2.1). | `--log-opt syslog-facility=daemon` | +| `syslog-tls-ca-cert` | The absolute path to the trust certificates signed by the CA. **Ignored if the address protocol is not `tcp+tls`.** | `--log-opt syslog-tls-ca-cert=/etc/ca-certificates/custom/ca.pem` | +| `syslog-tls-cert` | The absolute path to the TLS certificate file. **Ignored if the address protocol is not `tcp+tls`**. | `--log-opt syslog-tls-cert=/etc/ca-certificates/custom/cert.pem` | +| `syslog-tls-key` | The absolute path to the TLS key file. **Ignored if the address protocol is not `tcp+tls`.** | `--log-opt syslog-tls-key=/etc/ca-certificates/custom/key.pem` | +| `syslog-tls-skip-verify` | If set to `true`, TLS verification is skipped when connecting to the `syslog` daemon. Defaults to `false`. **Ignored if the address protocol is not `tcp+tls`.** | `--log-opt syslog-tls-skip-verify=true` | +| `tag` | A string that is appended to the `APP-NAME` in the `syslog` message. By default, Docker uses the first 12 characters of the container ID to tag log messages. Refer to the [log tag option documentation](log_tags.md) for customizing the log tag format. | `--log-opt tag=mailer` | +| `syslog-format` | The `syslog` message format to use. If not specified the local UNIX syslog format is used, without a specified hostname. Specify `rfc3164` for the RFC-3164 compatible format, `rfc5424` for RFC-5424 compatible format, or `rfc5424micro` for RFC-5424 compatible format with microsecond timestamp resolution. | `--log-opt syslog-format=rfc5424micro` | +| `labels` | Applies when starting the Docker daemon. A comma-separated list of logging-related labels this daemon will accept. Used for advanced [log tag options](log_tags.md). | `--log-opt labels=production_status,geo` | +| `env` | Applies when starting the Docker daemon. A comma-separated list of logging-related environment variables this daemon will accept. Used for advanced [log tag options](log_tags.md). | `--log-opt env=os,customer` | +| `env-regex` | Applies when starting the Docker daemon. Similar to and compatible with `env`. A regular expression to match logging-related environment variables. Used for advanced [log tag options](log_tags.md). | `--log-opt env-regex=^(os|customer). |