From 701d20bbfb032d6887268dad7589e029daee15ce Mon Sep 17 00:00:00 2001 From: Dan Moran Date: Fri, 26 Feb 2021 13:20:28 -0500 Subject: [PATCH] Rearrange InfluxDB docs so upgrade info is immediately visible. --- influxdb/content.md | 456 ++++++++++++++++++++++---------------------- 1 file changed, 228 insertions(+), 228 deletions(-) diff --git a/influxdb/content.md b/influxdb/content.md index 277ceed83..25b3a738e 100644 --- a/influxdb/content.md +++ b/influxdb/content.md @@ -10,231 +10,8 @@ InfluxDB is a time series database built from the ground up to handle high write The `latest` tag for this image now points to the latest released implementation of InfluxDB 2.x. If you are using the `latest` tag and would like to stay on the InfluxDB 1.x line, please update your environment to reference the `1.8` tag. -## Upgrading from quay.io-hosted InfluxDB 2.x image - -Early Docker builds of InfluxDB 2.x were hosted at `quay.io/influxdb/influxdb`. The builds were very bare-bones, containing the `influx` and `influxd` binaries without any default configuration or helper scripts. By default, the `influxd` process stored data under `/root/.influxdbv2`. - -Starting with `v2.0.4`, we've restored our DockerHub build. This build defaults to storing data in `/var/lib/influxdb2`. Upgrading directly from `quay.io/influxdb/influxdb` to `influxdb:2.0.4` without modifying any settings will appear to cause data loss, as the new process won't be able to find your existing data files. - -To avoid this problem when migrating from `quay.io/influxdb/influxdb` to `influxdb:2.0`, you can use one of the following approaches. - -### Change volume mount point - -If you don't mind using the new default path, you can switch the mount-point for the volume containing your data: - -```console -# Migrate from this: -$ docker run -p 8086:8086 \ - -v $PWD:/root/.influxdbv2 \ - quay.io/influxdb/influxdb:v2.0.3 - -# To this: -docker run -p 8086:8086 \ - -v $PWD:/var/lib/influxdb2 \ - %%IMAGE%% -``` - -### Override default configs - -If you'd rather keep your data files in the home directory, you can override the container's default config: - -```console -# Migrate from this: -$ docker run -p 8086:8086 \ - -v $PWD:/root/.influxdbv2 \ - quay.io/influxdb/influxdb:v2.0.3 - -# To this: -docker run -p 8086:8086 \ - -e INFLUXD_BOLT_PATH=/root/.influxdbv2/influxd.bolt \ - -e INFLUXD_ENGINE_PATH=/root/.influxdbv2/engine \ - -v $PWD:/root/.influxdbv2 \ - %%IMAGE%% -``` - -See the section about configuration below for more ways to override the data paths. - ## Using this Image - InfluxDB 2.x -### Running the container - -The InfluxDB image exposes a shared volume under `/var/lib/influxdb2`. You can mount a host directory to that point to access persisted container data. A typical invocation of the container might be: - -```console -$ docker run -p 8086:8086 \ - -v $PWD:/var/lib/influxdb2 \ - %%IMAGE%% -``` - -Modify `$PWD` to the directory where you want to store data associated with the InfluxDB container. - -You can also have Docker control the volume mountpoint by using a named volume. - -```console -$ docker run -p 8086:8086 \ - -v influxdb2:/var/lib/influxdb2 \ - %%IMAGE%% -``` - -### Exposed Ports - -The following ports are important and are used by InfluxDB. - -- 8086 HTTP UI and API port - -The HTTP port will be automatically exposed when using `docker run -P`. - -Find more about API Endpoints & Ports [here](https://docs.influxdata.com/influxdb/v2.0/reference/api/). - -### Configuration - -InfluxDB can be configured using a mix of a config file, environment variables, and CLI options. To mount a configuration file and use it with the server, you can use this command to generate the default configuration file: - -```console -$ docker run --rm %%IMAGE%% influxd print-config > config.yml -``` - -Modify the default configuration, which will now be available under `$PWD`. Then start the InfluxDB container: - -```console -$ docker run -p 8086:8086 \ - -v $PWD/config.yml:/etc/influxdb2/config.yml:ro \ - %%IMAGE%% -``` - -Modify `$PWD` to be the directory where you want to store the configuration file. - -Individual config settings can be overridden by environment variables. The variables must be named using the format `INFLUXD_${SNAKE_CASE_NAME}`. The `SNAKE_CASE_NAME` for an option will be the option's name with all dashes (`-`) replaced by underscores (`_`), in all caps. - -Examples: - -```console -# Config setting: bolt-path -INFLUXD_BOLT_PATH=/root/influxdb.bolt -# Config setting: no-tasks -INFLUXD_NO_TASKS=true -# Config setting: storage-wal-fsync-delay -INFLUXD_STORAGE_WAL_FSYNC_DELAY=15m -``` - -Finally, all config options can be passed as CLI options: - -```console -$ docker run -p 8086:8086 \ - %%IMAGE%% --storage-wal-fsync-delay=15m -``` - -CLI options take precedence over environment variables. - -Find more about configuring InfluxDB [here](https://docs.influxdata.com/influxdb/v2.0/reference/config-options/). - -### Database Setup - -InfluxDB 2.x requires authentication. A special API exists to bootstrap the first super-user in the database, along with an initial organization and bucket. It's possible to access this API manually, or to run it automatically via environment variables. - -#### Manual Setup - -If your InfluxDB container is running locally (or on a host exposed to the network), you can perform initial setup from outside the container using either the UI or the `influx` CLI. Find more about setting up InfluxDB using these methods [here](https://docs.influxdata.com/influxdb/v2.0/get-started/#set-up-influxdb). - -It's also possible to perform manual setup from within the container using `docker exec`. For example, if you start the container: - -```console -$ docker run -d -p 8086:8086 \ - --name influxdb2 \ - -v $PWD:/var/lib/influxdb2 \ - %%IMAGE%% -``` - -You can then run the `influx` client in the container: - -```console -$ docker exec influxdb2 influx setup \ - --username $USERNAME \ - --password $PASSWORD \ - --org $ORGANIZATION \ - --bucket $BUCKET -``` - -Running setup from within the container will cause CLI configs to be written to `/etc/influxdb2/influx-configs`. You can then use the `influx` CLI from within the container to extract the generated admin token: - -```console -# Using table output + cut -$ docker exec influxdb2 influx auth list \ - --user $USERNAME \ - --hide-headers | cut -f 3 - -# Using JSON output + jq -$ docker exec influxdb2 influx auth list \ - --user $USERNAME \ - --json | jq -r '.[].token' -``` - -Alternatively, you could configure your initial InfluxDB run to mount `/etc/influxdb2` as a volume: - -```console -$ docker run -d -p 8086:8086 \ - --name influxdb2 \ - -v $PWD/data:/var/lib/influxdb2 \ - -v $PWD/config:/etc/influxdb2 \ - %%IMAGE%% -``` - -This will make the generated CLI configs available to the host. - -#### Automated Setup - -The InfluxDB image contains some extra functionality to automatically bootstrap the system. This functionality is enabled by setting the `DOCKER_INFLUXDB_INIT_MODE` environment variable to the value `setup` when running the container. Additional environment variables are used to configure the setup logic: - -- `DOCKER_INFLUXDB_INIT_USERNAME`: The username to set for the system's initial super-user (**Required**). -- `DOCKER_INFLUXDB_INIT_PASSWORD`: The password to set for the system's inital super-user (**Required**). -- `DOCKER_INFLUXDB_INIT_ORG`: The name to set for the system's initial organization (**Required**). -- `DOCKER_INFLUXDB_INIT_BUCKET`: The name to set for the system's initial bucket (**Required**). -- `DOCKER_INFLUXDB_INIT_RETENTION`: The duration the system's initial bucket should retain data. If not set, the initial bucket will retain data forever. -- `DOCKER_INFLUXDB_INIT_ADMIN_TOKEN`: The authentication token to associate with the system's initial super-user. If not set, a token will be auto-generated by the system. - -Automated setup will generate metadata files and CLI configurations. It's recommended to mount volumes at both paths to avoid losing data. - -For example, a minimal invocation of automated setup is: - -```console -$ docker run -d -p 8086:8086 \ - -v $PWD/data:/var/lib/influxdb2 \ - -v $PWD/config:/etc/influxdb2 \ - -e DOCKER_INFLUXDB_INIT_MODE=setup \ - -e DOCKER_INFLUXDB_INIT_USERNAME=my-user \ - -e DOCKER_INFLUXDB_INIT_PASSWORD=my-password \ - -e DOCKER_INFLUXDB_INIT_ORG=my-org \ - -e DOCKER_INFLUXDB_INIT_BUCKET=my-bucket \ - %%IMAGE%% -``` - -And an example using all available options is: - -```console -$ docker run -d -p 8086:8086 \ - -v $PWD/data:/var/lib/influxdb2 \ - -v $PWD/config:/etc/influxdb2 \ - -e DOCKER_INFLUXDB_INIT_MODE=setup \ - -e DOCKER_INFLUXDB_INIT_USERNAME=my-user \ - -e DOCKER_INFLUXDB_INIT_PASSWORD=my-password \ - -e DOCKER_INFLUXDB_INIT_ORG=my-org \ - -e DOCKER_INFLUXDB_INIT_BUCKET=my-bucket \ - -e DOCKER_INFLUXDB_INIT_RETENTION=1w \ - -e DOCKER_INFLUXDB_INIT_ADMIN_TOKEN=my-super-secret-auth-token \ - %%IMAGE%% -``` - -**NOTE:** Automated setup will not run if an existing boltdb file is found at the configured path. This behavior allows for the InfluxDB container to reboot post-setup without encountering "DB is already set up" errors. - -### Interacting with InfluxDB - -Once an InfluxDB instance has completed initial setup, its APIs will unlock. See the main documentation site for reference information and examples on: - -- [Writing data](https://docs.influxdata.com/influxdb/v2.0/write-data/) -- [Reading data](https://docs.influxdata.com/influxdb/v2.0/query-data/) -- [Configuring security](https://docs.influxdata.com/influxdb/v2.0/security/) -- [And more!](https://docs.influxdata.com/influxdb/v2.0/) - ### Upgrading from InfluxDB 1.x InfluxDB 2.x provides a 1.x-compatible API, but expects a different storage layout on disk. To bridge this mismatch, the InfluxDB image contains extra functionality to migrate 1.x data and config into 2.x layouts automatically before booting the `influxd` server. @@ -289,7 +66,7 @@ $ docker run -p 8086:8086 \ -e DOCKER_INFLUXDB_INIT_PASSWORD=my-password \ -e DOCKER_INFLUXDB_INIT_ORG=my-org \ -e DOCKER_INFLUXDB_INIT_BUCKET=my-bucket \ - %%IMAGE%% + %%IMAGE%%:2.0 ``` #### Upgrade Example - Custom InfluxDB 1.x Config @@ -315,7 +92,7 @@ $ docker run -p 8086:8086 \ -e DOCKER_INFLUXDB_INIT_PASSWORD=my-password \ -e DOCKER_INFLUXDB_INIT_ORG=my-org \ -e DOCKER_INFLUXDB_INIT_BUCKET=my-bucket \ - %%IMAGE%% + %%IMAGE%%:2.0 ``` #### Upgrade Example - Custom Paths @@ -342,7 +119,7 @@ $ docker run -p 8086:8086 \ -e DOCKER_INFLUXDB_INIT_ORG=my-org \ -e DOCKER_INFLUXDB_INIT_BUCKET=my-bucket \ -e DOCKER_INFLUXDB_INIT_UPGRADE_V1_CONFIG=/root/influxdb/influxdb.conf \ - %%IMAGE%% + %%IMAGE%%:2.0 ``` To retain your custom paths, you'd run: @@ -361,9 +138,232 @@ $ docker run -p 8086:8086 \ -e DOCKER_INFLUXDB_CONFIG_PATH=/root/influxdb2/config.toml \ -e DOCKER_INFLUXDB_BOLT_PATH=/root/influxdb2/influxdb.bolt \ -e DOCKER_INFLUXDB_ENGINE_PATH=/root/influxdb2/engine \ - %%IMAGE%% + %%IMAGE%%:2.0 ``` +### Upgrading from quay.io-hosted InfluxDB 2.x image + +Early Docker builds of InfluxDB 2.x were hosted at `quay.io/influxdb/influxdb`. The builds were very bare-bones, containing the `influx` and `influxd` binaries without any default configuration or helper scripts. By default, the `influxd` process stored data under `/root/.influxdbv2`. + +Starting with `v2.0.4`, we've restored our DockerHub build. This build defaults to storing data in `/var/lib/influxdb2`. Upgrading directly from `quay.io/influxdb/influxdb` to `influxdb:2.0.4` without modifying any settings will appear to cause data loss, as the new process won't be able to find your existing data files. + +To avoid this problem when migrating from `quay.io/influxdb/influxdb` to `influxdb:2.0`, you can use one of the following approaches. + +#### Change volume mount point + +If you don't mind using the new default path, you can switch the mount-point for the volume containing your data: + +```console +# Migrate from this: +$ docker run -p 8086:8086 \ + -v $PWD:/root/.influxdbv2 \ + quay.io/influxdb/influxdb:v2.0.3 + +# To this: +docker run -p 8086:8086 \ + -v $PWD:/var/lib/influxdb2 \ + %%IMAGE%%:2.0 +``` + +#### Override default configs + +If you'd rather keep your data files in the home directory, you can override the container's default config: + +```console +# Migrate from this: +$ docker run -p 8086:8086 \ + -v $PWD:/root/.influxdbv2 \ + quay.io/influxdb/influxdb:v2.0.3 + +# To this: +docker run -p 8086:8086 \ + -e INFLUXD_BOLT_PATH=/root/.influxdbv2/influxd.bolt \ + -e INFLUXD_ENGINE_PATH=/root/.influxdbv2/engine \ + -v $PWD:/root/.influxdbv2 \ + %%IMAGE%%:2.0 +``` + +See the section about configuration below for more ways to override the data paths. + +### Running the container + +The InfluxDB image exposes a shared volume under `/var/lib/influxdb2`. You can mount a host directory to that point to access persisted container data. A typical invocation of the container might be: + +```console +$ docker run -p 8086:8086 \ + -v $PWD:/var/lib/influxdb2 \ + %%IMAGE%%:2.0 +``` + +Modify `$PWD` to the directory where you want to store data associated with the InfluxDB container. + +You can also have Docker control the volume mountpoint by using a named volume. + +```console +$ docker run -p 8086:8086 \ + -v influxdb2:/var/lib/influxdb2 \ + %%IMAGE%%:2.0 +``` + +### Exposed Ports + +The following ports are important and are used by InfluxDB. + +- 8086 HTTP UI and API port + +The HTTP port will be automatically exposed when using `docker run -P`. + +Find more about API Endpoints & Ports [here](https://docs.influxdata.com/influxdb/v2.0/reference/api/). + +### Configuration + +InfluxDB can be configured using a mix of a config file, environment variables, and CLI options. To mount a configuration file and use it with the server, you can use this command to generate the default configuration file: + +```console +$ docker run --rm %%IMAGE%%:2.0 influxd print-config > config.yml +``` + +Modify the default configuration, which will now be available under `$PWD`. Then start the InfluxDB container: + +```console +$ docker run -p 8086:8086 \ + -v $PWD/config.yml:/etc/influxdb2/config.yml:ro \ + %%IMAGE%%:2.0 +``` + +Modify `$PWD` to be the directory where you want to store the configuration file. + +Individual config settings can be overridden by environment variables. The variables must be named using the format `INFLUXD_${SNAKE_CASE_NAME}`. The `SNAKE_CASE_NAME` for an option will be the option's name with all dashes (`-`) replaced by underscores (`_`), in all caps. + +Examples: + +```console +# Config setting: bolt-path +INFLUXD_BOLT_PATH=/root/influxdb.bolt +# Config setting: no-tasks +INFLUXD_NO_TASKS=true +# Config setting: storage-wal-fsync-delay +INFLUXD_STORAGE_WAL_FSYNC_DELAY=15m +``` + +Finally, all config options can be passed as CLI options: + +```console +$ docker run -p 8086:8086 \ + %%IMAGE%%:2.0 --storage-wal-fsync-delay=15m +``` + +CLI options take precedence over environment variables. + +Find more about configuring InfluxDB [here](https://docs.influxdata.com/influxdb/v2.0/reference/config-options/). + +### Database Setup + +InfluxDB 2.x requires authentication. A special API exists to bootstrap the first super-user in the database, along with an initial organization and bucket. It's possible to access this API manually, or to run it automatically via environment variables. + +#### Manual Setup + +If your InfluxDB container is running locally (or on a host exposed to the network), you can perform initial setup from outside the container using either the UI or the `influx` CLI. Find more about setting up InfluxDB using these methods [here](https://docs.influxdata.com/influxdb/v2.0/get-started/#set-up-influxdb). + +It's also possible to perform manual setup from within the container using `docker exec`. For example, if you start the container: + +```console +$ docker run -d -p 8086:8086 \ + --name influxdb2 \ + -v $PWD:/var/lib/influxdb2 \ + %%IMAGE%%:2.0 +``` + +You can then run the `influx` client in the container: + +```console +$ docker exec influxdb2 influx setup \ + --username $USERNAME \ + --password $PASSWORD \ + --org $ORGANIZATION \ + --bucket $BUCKET +``` + +Running setup from within the container will cause CLI configs to be written to `/etc/influxdb2/influx-configs`. You can then use the `influx` CLI from within the container to extract the generated admin token: + +```console +# Using table output + cut +$ docker exec influxdb2 influx auth list \ + --user $USERNAME \ + --hide-headers | cut -f 3 + +# Using JSON output + jq +$ docker exec influxdb2 influx auth list \ + --user $USERNAME \ + --json | jq -r '.[].token' +``` + +Alternatively, you could configure your initial InfluxDB run to mount `/etc/influxdb2` as a volume: + +```console +$ docker run -d -p 8086:8086 \ + --name influxdb2 \ + -v $PWD/data:/var/lib/influxdb2 \ + -v $PWD/config:/etc/influxdb2 \ + %%IMAGE%%:2.0 +``` + +This will make the generated CLI configs available to the host. + +#### Automated Setup + +The InfluxDB image contains some extra functionality to automatically bootstrap the system. This functionality is enabled by setting the `DOCKER_INFLUXDB_INIT_MODE` environment variable to the value `setup` when running the container. Additional environment variables are used to configure the setup logic: + +- `DOCKER_INFLUXDB_INIT_USERNAME`: The username to set for the system's initial super-user (**Required**). +- `DOCKER_INFLUXDB_INIT_PASSWORD`: The password to set for the system's inital super-user (**Required**). +- `DOCKER_INFLUXDB_INIT_ORG`: The name to set for the system's initial organization (**Required**). +- `DOCKER_INFLUXDB_INIT_BUCKET`: The name to set for the system's initial bucket (**Required**). +- `DOCKER_INFLUXDB_INIT_RETENTION`: The duration the system's initial bucket should retain data. If not set, the initial bucket will retain data forever. +- `DOCKER_INFLUXDB_INIT_ADMIN_TOKEN`: The authentication token to associate with the system's initial super-user. If not set, a token will be auto-generated by the system. + +Automated setup will generate metadata files and CLI configurations. It's recommended to mount volumes at both paths to avoid losing data. + +For example, a minimal invocation of automated setup is: + +```console +$ docker run -d -p 8086:8086 \ + -v $PWD/data:/var/lib/influxdb2 \ + -v $PWD/config:/etc/influxdb2 \ + -e DOCKER_INFLUXDB_INIT_MODE=setup \ + -e DOCKER_INFLUXDB_INIT_USERNAME=my-user \ + -e DOCKER_INFLUXDB_INIT_PASSWORD=my-password \ + -e DOCKER_INFLUXDB_INIT_ORG=my-org \ + -e DOCKER_INFLUXDB_INIT_BUCKET=my-bucket \ + %%IMAGE%%:2.0 +``` + +And an example using all available options is: + +```console +$ docker run -d -p 8086:8086 \ + -v $PWD/data:/var/lib/influxdb2 \ + -v $PWD/config:/etc/influxdb2 \ + -e DOCKER_INFLUXDB_INIT_MODE=setup \ + -e DOCKER_INFLUXDB_INIT_USERNAME=my-user \ + -e DOCKER_INFLUXDB_INIT_PASSWORD=my-password \ + -e DOCKER_INFLUXDB_INIT_ORG=my-org \ + -e DOCKER_INFLUXDB_INIT_BUCKET=my-bucket \ + -e DOCKER_INFLUXDB_INIT_RETENTION=1w \ + -e DOCKER_INFLUXDB_INIT_ADMIN_TOKEN=my-super-secret-auth-token \ + %%IMAGE%%:2.0 +``` + +**NOTE:** Automated setup will not run if an existing boltdb file is found at the configured path. This behavior allows for the InfluxDB container to reboot post-setup without encountering "DB is already set up" errors. + +### Interacting with InfluxDB + +Once an InfluxDB instance has completed initial setup, its APIs will unlock. See the main documentation site for reference information and examples on: + +- [Writing data](https://docs.influxdata.com/influxdb/v2.0/write-data/) +- [Reading data](https://docs.influxdata.com/influxdb/v2.0/query-data/) +- [Configuring security](https://docs.influxdata.com/influxdb/v2.0/security/) +- [And more!](https://docs.influxdata.com/influxdb/v2.0/) + ### Custom Initialization Scripts The InfluxDB image supports running arbitrary initialization scripts after initial system setup, on both the `setup` and `upgrade` paths. Scripts must have extension `.sh` and be mounted inside of the `/docker-entrypoint-initdb.d` directory. When multiple scripts are present, they will be executed in lexical sort order by name. @@ -412,7 +412,7 @@ $ docker run -p 8086:8086 \ -e V1_RP_NAME=v1-rp \ -e V1_AUTH_USERNAME=v1-user \ -e V1_AUTH_PASSWORD=v1-password \ - %%IMAGE%% + %%IMAGE%%:2.0 ``` **NOTE:** Custom scripts will not run if an existing boltdb file is found at the configured path (causing `setup` or `upgrade` to be skipped). This behavior allows for the InfluxDB container to reboot post-initialization without encountering errors from non-idempotent script commands.