From 64de46831787717cd6e2b5ba3faf81f52dbfc2de Mon Sep 17 00:00:00 2001 From: Lorenzo Fontana Date: Wed, 10 May 2017 19:56:13 +0200 Subject: [PATCH] Splitting all the logging drivers (#2975) * Splitting logging drivers Signed-off-by: Lorenzo Fontana --- _data/toc.yaml | 6 +++ engine/admin/logging/gelf.md | 77 +++++++++++++++++++++++++++++ engine/admin/logging/index.md | 3 ++ engine/admin/logging/journald.md | 2 +- engine/admin/logging/json-file.md | 66 +++++++++++++++++++++++++ engine/admin/logging/overview.md | 9 +++- engine/admin/logging/syslog.md | 81 +++++++++++++++++++++++++++++++ 7 files changed, 241 insertions(+), 3 deletions(-) create mode 100644 engine/admin/logging/gelf.md create mode 100644 engine/admin/logging/json-file.md create mode 100644 engine/admin/logging/syslog.md diff --git a/_data/toc.yaml b/_data/toc.yaml index fe68aa8c8b..c257bc1315 100644 --- a/_data/toc.yaml +++ b/_data/toc.yaml @@ -247,6 +247,12 @@ guides: title: Log tags for logging driver - path: /engine/admin/logging/logentries/ title: Logentries logging driver + - path: /engine/admin/logging/json-file/ + title: JSON File logging driver + - path: /engine/admin/logging/gelf/ + title: Graylog Extended Format (GELF) logging driver + - path: /engine/admin/logging/syslog/ + title: Syslog logging driver - path: /engine/admin/logging/awslogs/ title: Amazon CloudWatch logs logging driver - path: /engine/admin/logging/etwlogs/ diff --git a/engine/admin/logging/gelf.md b/engine/admin/logging/gelf.md new file mode 100644 index 0000000000..783dd8c734 --- /dev/null +++ b/engine/admin/logging/gelf.md @@ -0,0 +1,77 @@ +--- +description: Describes how to use the Graylog Extended Format logging driver. +keywords: graylog, gelf, logging, driver +redirect_from: +- /engine/reference/logging/gelf/ +title: Graylog Extended Format logging driver +--- + +The `gelf` logging driver is a convenient format that is understood by a number of tools such as +[Graylog](https://www.graylog.org/),[Logstash](https://www.elastic.co/products/logstash), and +[Fluentd](http://www.fluentd.org/). Many tools use this format. + +In GELF, every log message is a dict with the following fields: + +- version +- host (who sent the message in the first place) +- timestamp +- short and long version of the message +- any custom fields you configure yourself + +## Usage + +To use `gelf` as the default logging driver for new containers, pass the `--log-driver` +and `--log-opt` options to the Docker daemon: + +```bash +dockerd + -–log-driver gelf –-log-opt gelf-address=udp://1.2.3.4:12201 \ +``` + +To make the configuration permanent, you can configure it in `/etc/docker/daemon.json`: + +```json +{ + "log-driver": "gelf", + "log-opts": { + "gelf-address": "udp://1.2.3.4:12201" + } +} +``` + +You can set the logging driver for a specific container by setting the +`--log-driver` flag when using `docker create` or `docker run`: + +```bash +$ docker run \ + -–log-driver gelf –-log-opt gelf-address=udp://1.2.3.4:12201 \ + alpine echo hello world +``` + +### GELF Options + +The `gelf` logging driver supports the following options: + +| Option | Description | Example value | +|----------------------|--------------------------|-------------------------------------------| +| `gelf-address` | The address of the GELF server. `udp` is the only supported URI specifier and you must specify the port.| `--log-opt gelf-address=udp://192.168.0.42:12201` | +| `gelf-compression-type` | The type of compression the GELF driver uses to compress each log message. Allowed values are `gzip`, `zlib` and `none`. The default is `gzip`. | `--log-opt gelf-compression-type=gzip` | +| `gelf-compression-level` | The level of compression when `gzip` or `zlib` is the `gelf-compression-type`. An integer in the range of `-1` to `9` (BestCompression). Default value is 1 (BestSpeed). Higher levels provide more compression at lower speed.| `--log-opt gelf-compression-level=2` | +| `tag` | A string that is appended to the `APP-NAME` in the `gelf` message. By default, Docker uses the first 12 characters of the container ID to tag log messages. Refer to the [log tag option documentation](log_tags.md) for customizing the log tag format. | `--log-opt tag=mailer` | +| `labels` | Applies when starting the Docker daemon. A comma-separated list of logging-related labels this daemon will accept. Adds additional key on the `extra` fields, prefixed by an underscore (`_`). Used for advanced [log tag options](log_tags.md).| `--log-opt labels=production_status,geo` | +| `env` | Applies when starting the Docker daemon. A comma-separated list of logging-related environment variables this daemon will accept. Adds additional key on the `extra` fields, prefixed by an underscore (`_`). Used for advanced [log tag options](log_tags.md). | `--log-opt env=os,customer` | + + +### Examples + +This example configures the container to use the GELF server running at +`192.168.0.42` on port `12201`. + +```bash +$ docker run -dit \ + --log-driver=gelf \ + --log-opt gelf-address=udp://192.168.0.42:12201 \ + alpine sh +``` + + diff --git a/engine/admin/logging/index.md b/engine/admin/logging/index.md index 7b5c982ff6..063d4f838d 100644 --- a/engine/admin/logging/index.md +++ b/engine/admin/logging/index.md @@ -8,8 +8,11 @@ title: Logging drivers * [Configuring logging drivers](overview.md) * [Configuring log tags](log_tags.md) +* [JSON File logging driver](json-file.md) * [Fluentd logging driver](fluentd.md) * [Journald logging driver](journald.md) +* [Syslog logging driver](syslog.md) +* [Graylog Extended Format logging driver](gelf.md) * [Amazon CloudWatch Logs logging driver](awslogs.md) * [Splunk logging driver](splunk.md) * [ETW logging driver](etwlogs.md) diff --git a/engine/admin/logging/journald.md b/engine/admin/logging/journald.md index bed5a303ce..9002e3f913 100644 --- a/engine/admin/logging/journald.md +++ b/engine/admin/logging/journald.md @@ -1,5 +1,5 @@ --- -description: Describes how to use the fluentd logging driver. +description: Describes how to use the Journald logging driver. keywords: Journald, docker, logging, driver redirect_from: - /engine/reference/logging/journald/ diff --git a/engine/admin/logging/json-file.md b/engine/admin/logging/json-file.md new file mode 100644 index 0000000000..ae304a24ef --- /dev/null +++ b/engine/admin/logging/json-file.md @@ -0,0 +1,66 @@ +--- +description: Describes how to use the json-file logging driver. +keywords: json-file, docker, logging, driver +redirect_from: +- /engine/reference/logging/json-file/ +title: JSON File logging driver +--- + + +By default, Docker captures the standard output (and standard error) of all your containers, +and writes them in files using the JSON format. The JSON format annotates each line with its +origin (`stdout` or `stderr`) and its timestamp. Each log file containers information about +only one container. + +## Usage + +Docker uses the JSON File logging driver by default. To explicitly set it, +use the `--log-driver` and `--log-opt` flags when starting Docker: + +```bash +dockerd + -–log-driver json-file –-log-opt max-size=10m +``` + +To make the change persistent, configure it in `/etc/docker/daemon.json`: + +```json +{ + "log-driver": "json-file", + "log-opts": { + "max-size": "10m" + } +} +``` + +You can set the logging driver for a specific container by using the +`--log-driver` flag to `docker create` or `docker run`: + +```bash +$ docker run \ + -–log-driver json-file –-log-opt max-size=10m \ + alpine echo hello world +``` + +### Options + +The `json-file` logging driver supports the following logging options: + +| Option | Description | Example value | +|------------|--------------------------|-------------------------------------------| +| `max-size` | The maximum size of the log before it is rolled. A positive integer plus a modifier representing the unit of measure (`k`, `m`, or `g`). | `--log-opt max-size=10m` | +| `max-file` | The maximum number of log files that can be present. If rolling the logs creates excess files, the oldest file is removed. **Only effective when `max-size` is also set.** A positive integer. | `--log-opt max-file=3` | +| `labels` | Applies when starting the Docker daemon. A comma-separated list of logging-related labels this daemon will accept. Used for advanced [log tag options](log_tags.md).| `--log-opt labels=production_status,geo` | +| `env` | Applies when starting the Docker daemon. A comma-separated list of logging-related environment variables this daemon will accept. Used for advanced [log tag options](log_tags.md). | `--log-opt env=os,customer` | + +> **Note**: If `max-size` and `max-file` are set, `docker logs` only returns the +> log lines from the newest log file. + +### Examples + +This example starts an `alpine` container which can have a maximum of 3 log +files no larger than 10 megabytes each. + +```bash +$ docker run -it --log-opt max-size=10m --log-opt max-file=3 alpine ash +``` diff --git a/engine/admin/logging/overview.md b/engine/admin/logging/overview.md index c8a71bef85..8d3ad87c5c 100644 --- a/engine/admin/logging/overview.md +++ b/engine/admin/logging/overview.md @@ -1,6 +1,6 @@ --- description: Configure logging driver. -keywords: docker, logging, driver, Fluentd +keywords: docker, logging, driver redirect_from: - /engine/reference/logging/overview/ title: Configure logging drivers @@ -136,7 +136,7 @@ $ docker run -it --log-driver none alpine ash ## `json-file` `json-file` is the default logging driver, and returns logging output in JSON -format. +format. For detailed information on working with this logging driver, see [the json-file](json-file.md) reference documentation. ### Options @@ -164,6 +164,9 @@ $ docker run -it --log-opt max-size=10m --log-opt max-file=3 alpine ash ## `syslog` +The `syslog` logging driver routes logs to a `syslog` server. +For detailed information on working with this logging driver, see [the syslog logging driver](syslog.md) reference documentation. + ### Options The following logging options are supported for the `syslog` logging driver: @@ -234,6 +237,8 @@ $ docker run \ ## `gelf` +The `gelf` logging driver is a convenient format that is understood by a number of tools like [Graylog](https://www.graylog.org/), [Logstash](https://www.elastic.co/products/logstash), [Fluentd](http://www.fluentd.org/), and many more. For detailed information on working with this logging driver, see [the Graylog Extended Format logging driver](gelf.md) reference documentation. + ### Options The `gelf` logging driver supports the following options: diff --git a/engine/admin/logging/syslog.md b/engine/admin/logging/syslog.md new file mode 100644 index 0000000000..5eb47858fa --- /dev/null +++ b/engine/admin/logging/syslog.md @@ -0,0 +1,81 @@ +--- +description: Describes how to use the syslog logging driver. +keywords: syslog, docker, logging, driver +redirect_from: +- /engine/reference/logging/syslog/ +title: Syslog logging driver +--- + +The `syslog` logging driver routes logs to a `syslog` server. The `syslog` protocol uses +a raw string as the log message and supports a limited set of metadata. The syslog +message mut be formatted in a specific way to be valid. From a valid message, the +receiver can extract the following information: + +- **priority**: the logging level, such as `debug`, `warning`, `error`, `info`. +- **timestamp**: when the event occurred. +- **hostname**: where the event happened. +- **facility**: which subsystem logged the message, such as `mail` or `kernel`. +- **process name** and **process ID (PID)**: The name and ID of the process that generated the log. + +The format is defined in [RFC 5424](https://tools.ietf.org/html/rfc5424) and Docker's syslog driver implements the +[ABNF reference](https://tools.ietf.org/html/rfc5424#section-6) in the following way: + +```none + TIMESTAMP SP HOSTNAME SP APP-NAME SP PROCID SP MSGID + + + + | + + | | | | | + | | | | | + +------------+ +----+ | +----+ +---------+ + v v v v v +2017-04-01T17:41:05.616647+08:00 a.vm {taskid:aa,version:} 1787791 {taskid:aa,version:} +``` + +## Usage + +To use the `syslog` driver as the default logging driver, you can use the `--log-driver` and `--log-opt` flags when starting Docker: + +```bash +dockerd \ + --log-driver syslog \ + --log-opt syslog-address=udp://1.2.3.4:1111 +``` + +> **Note**: The syslog-address supports both UP and TCP. + +To make the changes persistent, add the toptions to `/etc/docker/daemon.json`: + + +```json +{ + "log-driver": "syslog", + "log-opts": { + "syslog": "udp://1.2.3.4:1111" + } +} +``` + +You can set the logging driver for a specific container by using the +`--log-driver` flag to `docker create` or `docker run`: + +```bash +docker run \ + -–log-driver syslog –-log-opt syslog=udp://1.2.3.4:1111 \ + alpine echo hello world +``` + +## Options + +The following logging options are supported for the `syslog` logging driver: + +| Option | Description | Example value | +|----------------------|--------------------------|-------------------------------------------| +| `syslog-address` | The address of an external `syslog` server. The URI specifier may be `[tcp|udp|tcp+tls]://host:port`, `unix://path`, or `unixgram://path`. If the transport is `tcp`, `udp`, or `tcp+tls`, the default port is `514`.| `--log-opt syslog-address=tcp+tls://192.168.1.3:514`, `--log-opt syslog-address=unix:///tmp/syslog.sock` | +| `syslog-facility` | The `syslog` facility to use. Can be the number or name for any valid `syslog` facility. See the [syslog documentation](https://tools.ietf.org/html/rfc5424#section-6.2.1). | `--log-opt syslog-facility=daemon` | +| `syslog-tls-ca-cert` | The absolute path to the trust certificates signed by the CA. **Ignored if the address protocol is not `tcp+tls`.** | `--log-opt syslog-tls-ca-cert=/etc/ca-certificates/custom/ca.pem` | +| `syslog-tls-cert` | The absolute path to the TLS certificate file. **Ignored if the address protocol is not `tcp+tls`**. | `--log-opt syslog-tls-cert=/etc/ca-certificates/custom/cert.pem` | +| `syslog-tls-key` | The absolute path to the TLS key file. **Ignored if the address protocol is not `tcp+tls`.** | `--log-opt syslog-tls-key=/etc/ca-certificates/custom/key.pem` | +| `syslog-tls-skip` | If set to `true`, TLS verification is skipped when connecting to the `syslog` daemon. Defaults to `false`. **Ignored if the address protocol is not `tcp+tls`.** | `--log-opt syslog-tls-skip-verify=true` | +| `tag` | A string that is appended to the `APP-NAME` in the `syslog` message. By default, Docker uses the first 12 characters of the container ID to tag log messages. Refer to the [log tag option documentation](log_tags.md) for customizing the log tag format. | `--log-opt tag=mailer` | +| `syslog-format` | The `syslog` message format to use. If not specified the local UNIX syslog format is used, without a specified hostname. Specify `rfc3164` for the RFC-3164 compatible format, `rfc5424` for RFC-5424 compatible format, or `rfc5424micro` for RFC-5424 compatible format with microsecond timestamp resolution. | `--log-opt syslog-format=rfc5424micro` | +| `labels` | Applies when starting the Docker daemon. A comma-separated list of logging-related labels this daemon will accept. Used for advanced [log tag options](log_tags.md).| `--log-opt labels=production_status,geo` | +| `env` | Applies when starting the Docker daemon. A comma-separated list of logging-related environment variables this daemon will accept. Used for advanced [log tag options](log_tags.md). | `--log-opt env=os,customer` |