docs/certify-plugins-logging.md at 30e0fd3f18b79f1041bd7c15201d1e45feec0dd0

27 KiB

Raw Blame History

description	keywords	title
Run certification tests against your images	Docker, docker, store, certified content, logging	Certify Docker logging plugins

Introduction

Content that qualifies as Docker Certified must conform to best practices and pass certain baseline tests.

Docker Store lets you publish certified images as well as plugins for logging, volumes, and networks. You must certify your own images and logging plugins with the inspect tools as explained in these docs. Currently, Docker Store certifies your volume and networking plugins for you upon submission.

This page explains how publishers can successfully test their Docker logging plugins. Also available: Certify your Docker images.

Content that requires a non-certified infrastructure environment cannot be published as certified.

Certify your logging plugins

You must use the tool, inspectDockerLoggingPlugin, to certify your content for publication on Docker Store by ensuring that your Docker logging plugins conform to best practices.

The inspectDockerLoggingPlugin command verifies that your Docker logging plugin can be installed and works on Docker Enterprise Edition. It also runs a container from an official Docker image of alpine:latest and outputs the contents of a file named quotes.txt (included in this repo). In sum, the inspectDockerLoggingPlugin command:

Inspects and displays the Docker logging plugin.
Installs the Docker logging plugin on Docker EE.
Runs a Docker service container with the Docker logging plugin, reads a file named quotes.txt, echos its contents to stdout, and logs the file's content.
Displays the container logs and compares it to quotes.txt. If they match, the test is successful.

The syntax for running a specific logging plugin is: docker container run --log-driver.

No parameters are passed to the logging plugin. If parameters are required for the Docker logging plugin to work correctly, then a custom test script must be written and used. The default docker container run command is:

docker container run -it --log-driver xxxxxxxxxxxxxxxxxxxxx \
--volume \"$(pwd)/quotes.txt:/quotes.txt\" alpine:latest \
sh -c 'cat /quotes.txt;sleep 20

The custom script must log the contents of the quotes.txt file. It should also cleanup (remove the container and docker image). Refer to the --test-script command argument in the command help.

Docker container logs

Best practices require Docker logging plugins to support the ReadLogs API so that the logs can be retrieved with the docker container logs command. If the ReadLogs API is not supported, a custom script is needed to retrieve the logs and print them to stdout. Refer to the --get-logs-script command argument in the command help.

External server

To send data to an external server, use the http_api_endpoint HTTP Server for your docker logging plugin. You can run the http_api_endpoint HTTP Server and have the docker logging plugin send its logging data to the http_api_endpoint HTTP Server. The data can then be retrieved using a curl command. Refer to the http_api_endpoint HTTP Server for more details.

Prerequisites

Your Docker EE installation must be running on the server used to verify your submissions. If necessary, request entitlement to a specific Docker Enterprise Edition.

Docker EE (on the server for verifying submissions)
git client
inspectDockerLoggingPlugin tool

Set up testing environment

There are three steps: (1) install git, (2) configure credentials, and (3) configure endpoints.

Install git (required for inspectDockerLoggingPlugin):

Ubuntu
```
sudo apt-get update -qq
sudo apt-get install git -y
```
RHEL/CentOS
```
sudo yum makecache fast
sudo yum install git -y
```
Windows

To download and install git for Windows: https://git-scm.com/download/win.

Configure your Docker Registry credentials by either defining environment variables or passing them as arguments to inspectDockerLoggingPlugin.

a. Define environment variables for registry credentials, DOCKER_USER and DOCKER_PASSWORD:

Linux

export DOCKER_USER="my_docker_registry_user_account"
export DOCKER_PASSWORD="my_docker_registry_user_account_password"

Windows command prompt

set DOCKER_USER="my_docker_registry_user_account"
set DOCKER_PASSWORD="my_docker_registry_user_account_password"

Windows powershell

$env:DOCKER_USER="my_docker_registry_user_account"
$env:DOCKER_PASSWORD="my_docker_registry_user_account_password"

b. Pass arguments to inspectDockerLoggingPlugin (or be prompted for them):

--docker-user
--docker-password

Configure endpoints (and override default values) by either defining environment variables or passing them as arguments to inspectDockerLoggingPlugin.

By default, inspectDockerLoggingPlugin uses these two endpoints to communicate with the Docker Hub Registry:
- Registry Authentication Endpoint: https://auth.docker.io
- Registry API Endpoint: https://registry-1.docker.io
You may want to use your private registry for initial testing and override the defaults.

a. Define environment variables, DOCKER_REGISTRY_AUTH_ENDPOINT and DOCKER_REGISTRY_API_ENDPOINT:

Linux or MacOS
```
export DOCKER_REGISTRY_AUTH_ENDPOINT="https://my_docker_registry_authentication_endpoint"
export DOCKER_REGISTRY_API_ENDPOINT="https://my_docker_registry_api_enpoint"
```
Windows command prompt
```
set DOCKER_REGISTRY_AUTH_ENDPOINT="https://my_docker_registry_authentication_endpoint"
set DOCKER_REGISTRY_API_ENDPOINT="https://my_docker_registry_api_enpoint"
```
Windows powershell
```
$env:DOCKER_REGISTRY_AUTH_ENDPOINT="https://my_docker_registry_authentication_endpoint"
$env:DOCKER_REGISTRY_API_ENDPOINT="https://my_docker_registry_api_enpoint"
```
b. Pass your endpoints as arguments to inspectDockerLoggingPlugin:
```
--docker-registry-auth-endpoint
--docker-registry-api-endpoint
```

Syntax

Download inspectDockerLoggingPlugin command.

OS/Architecture	Download Link
Windows/X86	https://s3.amazonaws.com/store-logos-us-east-1/certification/windows/inspectDockerLoggingPlugin.exe
Windows/X86	https://s3.amazonaws.com/store-logos-us-east-1/certification/windows/http_api_endpoint.exe
Linux/X86	https://s3.amazonaws.com/store-logos-us-east-1/certification/linux/inspectDockerLoggingPlugin
Linux/X86	https://s3.amazonaws.com/store-logos-us-east-1/certification/linux/http_api_endpoint
Linux/IBMZ	https://s3.amazonaws.com/store-logos-us-east-1/certification/zlinux/inspectDockerLoggingPlugin
Linux/IBMPOWER	https://s3.amazonaws.com/store-logos-us-east-1/certification/power/inspectDockerLoggingPlugin

Set permissions on inspectDockerLoggingPlugin so that it is executable:
```
chmod u+x inspectDockerLoggingPlugin
```

Capture the product ID you'd like to reference for the certification test.

Inspects a Docker logging plugin to see if it conforms to best practices.

Syntax: inspectDockerLoggingPlugin [options] dockerLoggingPlugin

Options:
  -docker-password string
    	 Docker Password.  This overrides the DOCKER_PASSWORD environment variable.
  -docker-registry-api-endpoint string
    	 Docker Registry API Endpoint. This overrides the DOCKER_REGISTRY_API_ENDPOINT environment variable. (default "https://registry-1.docker.io")
  -docker-registry-auth-endpoint string
    	 Docker Registry Authentication Endpoint. This overrides the DOCKER_REGISTRY_AUTH_ENDPOINT environment variable. (default "https://auth.docker.io")
  -docker-user string
    	 Docker User ID.  This overrides the DOCKER_USER environment variable.
  -get-logs-script string
    	 An optional custom script used to retrieve the logs.
  -help
    	 Help on the command.
  -html
    	 Generate HTML output.
  -json
    	 Generate JSON output.
  -product-id string
    	 Optional Product identifier from Docker Store for this plugin. Please include it when you want the output sent to docker store for certification.
  -test-script string
    	 An optional custom script used to test the Docker logging plugin. The script gets passed 1 parameter - the Docker logging plugin name.
  -verbose
    	 Displays more verbose output.

dockerLoggingPlugin
The Docker logging plugin to inspect. This argument is required.

Inspection Output

By default, inspectDockerLoggingPlugin displays output locally to stdout (the default), JSON, and HTML. You can also upload output to Docker Store, which is recommended for admnistrator verification.

Upload to Docker Store (by entering product-id at the commandline).
Send message to stdout. This is the default.
JSON sent to stdout. Use the --json option to override and replace the messages sent to stdout.
HTML local file. Use the --html option to generate an HTML report. Both --json and --html can be specified at the same time.

Inspection Examples

Inspect a Docker logging plugin with messages sent to stdout
Inspect a Docker logging plugin with JSON output
Inspect a Docker logging plugin with HTML output
Send data to API endpoint on external server

Inspect a Docker logging plugin with messages sent to stdout

To inspect the Docker logging plugin "gforghetti/docker-log-driver-test:latest":

gforghetti:~:$ ./inspectDockerLoggingPlugin gforghetti/docker-log-driver-test:latest

Output:

**************************************************************************************************************************************************************************************************
* Docker logging plugin: gforghetti/docker-log-driver-test:latest
**************************************************************************************************************************************************************************************************

**************************************************************************************************************************************************************************************************
* Step #1 Inspecting the Docker logging plugin: gforghetti/docker-log-driver-test:latest ...
**************************************************************************************************************************************************************************************************
Passed:   Docker logging plugin image gforghetti/docker-log-driver-test:latest has been inspected.

**************************************************************************************************************************************************************************************************
* Step #2 Docker logging plugin information
**************************************************************************************************************************************************************************************************
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Docker logging plugin:  | gforghetti/docker-log-driver-test:latest                                                                                                                             |
| Description:            | jsonfilelog as plugin                                                                                                                                                |
| Documentation:          | -                                                                                                                                                                    |
| Digest:                 | sha256:870c81b9b8872956eec83311707e52ccd4af40683ba64aac3c1700d252a07624                                                                                              |
| Base layer digest:      | sha256:8b0c5cbf1339dacef5f56717567aeee37d9e4f196f0874457d46c01592a30d70                                                                                              |
| Docker version:         | 17.06.0-ce                                                                                                                                                           |
| Interface Socket:       | jsonfile.sock                                                                                                                                                        |
| Interface Socket Types: | docker.logdriver/1.0                                                                                                                                                 |
| IpcHost:                | false                                                                                                                                                                |
| PidHost:                | false                                                                                                                                                                |
| Entrypoint:             | /usr/bin/docker-log-driver                                                                                                                                           |
| WorkDir:                | /tmp                                                                                                                                                                 |
| User:                   |                                                                                                                                                                      |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+

**************************************************************************************************************************************************************************************************
* Step #3 Checking to see if the Docker logging plugin is already installed.
**************************************************************************************************************************************************************************************************
The Docker logging plugin gforghetti/docker-log-driver-test:latest needs to be installed.

**************************************************************************************************************************************************************************************************
* Step #4 Installing the Docker logging plugin gforghetti/docker-log-driver-test:latest ...
**************************************************************************************************************************************************************************************************
Passed:   Docker logging plugin gforghetti/docker-log-driver-test:latest has been installed successfully.

**************************************************************************************************************************************************************************************************
* Step #5 Testing the Docker logging plugin: gforghetti/docker-log-driver-test:latest ...
**************************************************************************************************************************************************************************************************
Starting a Docker container to test the docker logging plugin gforghetti/docker-log-driver-test:latest

**************************************************************************************************************************************************************************************************
* Step #6 Retrieving the Docker Logs ...
**************************************************************************************************************************************************************************************************
Retrieving the Docker logs using the "docker container logs c0bc7de01b155203d1ab72a1496979c4715c9b9f81c7f5a1a37ada3aa333e1d5" command

**************************************************************************************************************************************************************************************************
* Step #7 Verifying that the contents retrieved matches what was sent to the Docker Logging plugin.
**************************************************************************************************************************************************************************************************
Passed:   Docker logging plugin Test was successful.

**************************************************************************************************************************************************************************************************
* Step #8 Removing the Docker container and any associated volumes.
**************************************************************************************************************************************************************************************************
Passed:   Docker container and any associated volumes removed.

**************************************************************************************************************************************************************************************************
* Step #9 Removing the Docker logging plugin
**************************************************************************************************************************************************************************************************
Passed:   Docker logging plugin gforghetti/docker-log-driver-test:latest was removed.

**************************************************************************************************************************************************************************************************
* Summary of the inspection for the Docker logging plugin: gforghetti/docker-log-driver-test:latest
**************************************************************************************************************************************************************************************************

Report Date: Tue Sep 19 12:40:11 2017
Operating System: Operating System: MacOS darwin Version: 10.12.6
Docker version 17.09.0-ce-rc2, build 363a3e7


Passed:   Docker logging plugin image gforghetti/docker-log-driver-test:latest has been inspected.
Passed:   Docker logging plugin gforghetti/docker-log-driver-test:latest has been installed successfully.
Passed:   Docker logging plugin Test was successful.
Passed:   Docker container and any associated volumes removed.
Passed:   Docker logging plugin gforghetti/docker-log-driver-test:latest was removed.

The inspection of the Docker logging plugin gforghetti/docker-log-driver-test:latest has completed.
gforghetti:~/$

Inspect a Docker logging plugin with JSON Output

To inspect the Docker logging plugin `gforghetti/docker-log-driver-test:latest` with JSON Output:

gforghetti:~:$ ./inspectDockerLoggingPlugin --json gforghetti/docker-log-driver-test:latest | jq

Note: The output was piped to the jq command to display it "nicely".

Output:

{
  "Date": "Tue Sep 19 12:41:49 2017",
  "SystemOperatingSystem": "Operating System: MacOS darwin Version: 10.12.6",
  "SystemDockerVersion": "Docker version 17.09.0-ce-rc2, build 363a3e7",
  "DockerLogginPlugin": "gforghetti/docker-log-driver-test:latest",
  "Description": "jsonfilelog as plugin",
  "Documentation": "-",
  "DockerLoggingPluginDigest": "sha256:870c81b9b8872956eec83311707e52ccd4af40683ba64aac3c1700d252a07624",
  "BaseLayerImageDigest": "sha256:8b0c5cbf1339dacef5f56717567aeee37d9e4f196f0874457d46c01592a30d70",
  "DockerVersion": "17.06.0-ce",
  "Entrypoint": "/usr/bin/docker-log-driver",
  "InterfaceSocket": "jsonfile.sock",
  "InterfaceSocketTypes": "docker.logdriver/1.0",
  "WorkDir": "/tmp",
  "User": "",
  "IpcHost": false,
  "PidHost": false,
  "Errors": 0,
  "Warnings": 0,
  "HTMLReportFile": "",
  "VulnerabilitiesScanURL": "",
  "Results": [
    {
      "Status": "Passed",
      "Message": "Docker logging plugin image gforghetti/docker-log-driver-test:latest has been inspected."
    },
    {
      "Status": "Passed",
      "Message": "Docker logging plugin gforghetti/docker-log-driver-test:latest has been installed successfully."
    },
    {
      "Status": "Passed",
      "Message": "Docker logging plugin Test was successful."
    },
    {
      "Status": "Passed",
      "Message": "Docker container and any associated volumes removed."
    },
    {
      "Status": "Passed",
      "Message": "Docker logging plugin gforghetti/docker-log-driver-test:latest was removed."
    }
  ]
}
gforghetti:~/$

Inspect a Docker logging plugin with HTML output

To inspect the Docker logging plugin `gforghetti/docker-log-driver-test:latest` with HTML output:

gforghetti:~:$ ./inspectDockerLoggingPlugin --html gforghetti/docker-log-driver-test:latest

Output:

Note: The majority of the stdout message output has been intentionally omitted below.

The inspection of the Docker logging plugin cpuguy83/docker-logdriver-test:latest has completed.
An HTML report has been generated in the file cpuguy83-docker-logdriver-test-latest_inspection_report.html
gforghetti:~/$

Send data to API endpoint on external server

Introduction

The http_api_endpoint is an HTTP Server that can be used to test docker logging plugins that do not support the read log api and instead send data to an API Endpoint running on an external server. The Sumo Logic Logging Plugin is one example.

You can configure those docker logging plugins to send their logging data to the http_api_endpoint HTTP Server for testing the plugin and then code a script to retrieve the logs using a curl command.

Syntax

./http_api_endpoint [options]

Options:

--port (The port for the http_api_endpoint HTTP Server to listen on. Defaults to port 80)
--debug (write debugging information)
--help (display the command help)

Using and testing the http_api_endpoint HTTP Server

The curl command can be used to test and use the http_api_endpoint HTTP Server.

Issue the following curl command to send new data to the http_api_endpoint:
```
# DATA='Hello World!'
# curl -s -X POST -d "${DATA}" http://127.0.0.1:80
```
Note: if any data was previously sent, it will be replaced.
Issue the following curl command to send data to the http_api_endpoint and append that data to the already collected data:
```
# DATA='Hello World!'
# curl -s -X POST -d "${DATA}" http://127.0.0.1:80
```
Issue the following curl command to retrieve the data from the http_api_endpoint:
```
# curl -s -X GET http://127.0.0.1:80
```
```
Hello World!
```
Issue the following curl command to erase any data currently collected by the http_api_endpoint:
```
# curl -s -X DELETE http://127.0.0.1:80
```
To Terminate:
```
# curl -s http://127.0.0.1:80/EXIT
```

Example of using the http_api_endpoint HTTP Server for a Logging Plugin

Script to run a container to test the Logging Plugin

# cat test_new_plugin.sh

#!/usr/bin/env bash

#######################################################################################################################################
# This bash script tests a Docker logging plugin that does not support the read log api and instead sends data to an API Endpoint running on an external server.
#
#######################################################################################################################################
# Docker Inc.
#######################################################################################################################################

#######################################################################################################################################
# Make sure the Docker logging plugin was specified on the command line
#######################################################################################################################################
DOCKER_LOGGING_PLUGIN=$1
if [[ -z $DOCKER_LOGGING_PLUGIN ]]; then
    printf 'You must specify the Docker Loggin Plugin!\n'
    exit 1
fi

HTTP_API_ENDPOINT='http://localhost:80'

#######################################################################################################################################
# Check to make sure the http_api_endpoint HTTP Server is running
#######################################################################################################################################
curl -s -X POST "${HTTP_API_ENDPOINT}"
if [[ $? -ne 0 ]]; then
    printf 'Unable to connect to the HTTP API Endpoint: '"${HTTP_API_ENDPOINT}"'!\n'
    exit 1
fi

#######################################################################################################################################
# Run a alpine container with the plugin and send data to it
#######################################################################################################################################
docker container run \
--rm \
--log-driver="${DOCKER_LOGGING_PLUGIN}" \
--log-opt sumo-url="${HTTP_API_ENDPOINT}" \
--log-opt sum-sending-interval=5s \
--log-opt sumo-compress=false \
--volume $(pwd)/quotes.txt:/quotes.txt \
alpine:latest \
sh -c 'cat /quotes.txt;sleep 10'

exit $?

Script to retrieve the logging data from the http_api_endpoint HTTP Server

# cat get_plugin_logs.sh

#!/usr/bin/env sh

#######################################################################################################################################
# This bash script retrieves any data logged to the http_api_endpoint HTTP Server.
#######################################################################################################################################
# Docker Inc.
#######################################################################################################################################

curl -s -X GET http://127.0.0.1:80

To test the Docker logging plugin

./inspectDockerLoggingPlugin --verbose --html --test-script ./test_plugin.sh --get-logs-script ./get_plugin_logs.sh myNamespace/docker-logging-driver:1.0.2

27 KiB Raw Blame History

Introduction

Certify your logging plugins

Docker container logs

External server

Prerequisites

Set up testing environment

Syntax

Inspection Output

Inspection Examples

Inspect a Docker logging plugin with messages sent to stdout

To inspect the Docker logging plugin "gforghetti/docker-log-driver-test:latest":

Output:

Inspect a Docker logging plugin with JSON Output

To inspect the Docker logging plugin gforghetti/docker-log-driver-test:latest with JSON Output:

Output:

Inspect a Docker logging plugin with HTML output

To inspect the Docker logging plugin gforghetti/docker-log-driver-test:latest with HTML output:

Output:

Send data to API endpoint on external server

Introduction

Syntax

Using and testing the http_api_endpoint HTTP Server

Example of using the http_api_endpoint HTTP Server for a Logging Plugin

Script to run a container to test the Logging Plugin

Script to retrieve the logging data from the http_api_endpoint HTTP Server

To test the Docker logging plugin

27 KiB

Raw Blame History

To inspect the Docker logging plugin `gforghetti/docker-log-driver-test:latest` with JSON Output:

To inspect the Docker logging plugin `gforghetti/docker-log-driver-test:latest` with HTML output: