docker
/
docs
mirror of https://github.com/docker/docs.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

ENGDOCS-1499 (#17713) 

* initial structure'

* flesh out overview page

* flesh out overview page

* structure pages

* tidy up and refine

* tidy

* fix build

* tidy

* review edits

---------

Co-authored-by: aevesdocker <alliesadler@f693mt7fh6.home>
This commit is contained in:
Allie Sadler 2023-07-19 10:49:19 +01:00 committed by GitHub
parent cd4761b399
commit 2c0682d8e2
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
8 changed files with 538 additions and 456 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
_data/toc.yaml
View File

@ -1850,6 +1850,16 @@ manuals:
title: Change pre-defined environment variables
- path: /compose/profiles/
title: Using service profiles
- sectiontitle: Working with multiple Compose files
section:
- path: /compose/multiple-compose-files/
title: Overview
- path: /compose/multiple-compose-files/extends/
title: Extend
- path: /compose/multiple-compose-files/merge/
title: Merge
- path: /compose/multiple-compose-files/include/
title: Include
- path: /compose/gpu-support/
title: GPU support in Compose
- path: /compose/extends/

453
compose/extends.md
View File

@ -1,453 +0,0 @@
---
description: How to use Docker Compose's extends keyword to share configuration between files and projects
keywords: fig, composition, compose, docker, orchestration, documentation, docs
title: Share Compose configurations between files and projects
---
{% include compose-eol.md %}
Compose supports two methods of sharing common configuration:
1. Extend an entire Compose file by
[using multiple Compose files](extends.md#multiple-compose-files)
2. Extend individual services with [the `extends` field](extends.md#extending-services)
## Multiple Compose files
Using multiple Compose files lets you customize a Compose application
for different environments or different workflows.
### Understanding multiple Compose files
By default, Compose reads two files, a `docker-compose.yml` and an optional
`docker-compose.override.yml` file. By convention, the `docker-compose.yml`
contains your base configuration. The override file, as its name implies, can
contain configuration overrides for existing services or entirely new
services.
If a service is defined in both files, Compose merges the configurations using
the rules described in
[Adding and overriding configuration](extends.md#adding-and-overriding-configuration).
To use multiple override files, or an override file with a different name, you
can use the `-f` option to specify the list of files. Compose merges files in
the order they're specified on the command line. See the
[`docker compose` command reference](reference/index.md) for more information
about using `-f`.
When you use multiple configuration files, you must make sure all paths in the
files are relative to the base Compose file (the first Compose file specified
with `-f`). This is required because override files need not be valid
Compose files. Override files can contain small fragments of configuration.
Tracking which fragment of a service is relative to which path is difficult and
confusing, so to keep paths easier to understand, all paths must be defined
relative to the base file.
### Example use case
In this section, there are two common use cases for multiple Compose files: changing a
Compose app for different environments, and running administrative tasks
against a Compose app.
#### Different environments
A common use case for multiple files is changing a development Compose app
for a production-like environment (which may be production, staging or CI).
To support these differences, you can split your Compose configuration into
a few different files:
Start with a base file that defines the canonical configuration for the
services.
**docker-compose.yml**
```yaml
services:
web:
image: example/my_web_app:latest
depends_on:
- db
- cache
db:
image: postgres:latest
cache:
image: redis:latest
```
In this example the development configuration exposes some ports to the
host, mounts our code as a volume, and builds the web image.
**docker-compose.override.yml**
```yaml
services:
web:
build: .
volumes:
- '.:/code'
ports:
- 8883:80
environment:
DEBUG: 'true'
db:
command: '-d'
ports:
- 5432:5432
cache:
ports:
- 6379:6379
```
When you run `docker compose up` it reads the overrides automatically.
Now, it would be nice to use this Compose app in a production environment. So,
create another override file (which might be stored in a different git
repo or managed by a different team).
**docker-compose.prod.yml**
```yaml
services:
web:
ports:
- 80:80
environment:
PRODUCTION: 'true'
cache:
environment:
TTL: '500'
```
To deploy with this production Compose file you can run
```console
$ docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d
```
This deploys all three services using the configuration in
`docker-compose.yml` and `docker-compose.prod.yml` (but not the
dev configuration in `docker-compose.override.yml`).
See [production](production.md) for more information about Compose in
production.
#### Administrative tasks
Another common use case is running one off or administrative tasks against one
or more services in a Compose app. This example demonstrates running a
database backup.
Start with a **docker-compose.yml**.
```yaml
services:
web:
image: example/my_web_app:latest
depends_on:
db
db:
image: postgres:latest
```
In a **docker-compose.admin.yml** add a new service to run the database
export or backup.
```yaml
services:
dbadmin:
build: database_admin/
depends_on:
- db
```
To start a normal environment run `docker compose up -d`. To run a database
backup, include the `docker-compose.admin.yml` as well.
```console
$ docker compose -f docker-compose.yml -f docker-compose.admin.yml \
run dbadmin db-backup
```
## Extending services
Docker Compose's `extends` keyword enables the sharing of common configurations
among different files, or even different projects entirely. Extending services
is useful if you have several services that reuse a common set of configuration
options. Using `extends` you can define a common set of service options in one
place and refer to it from anywhere.
Keep in mind that `volumes_from` and `depends_on` are never shared between
services using `extends`. These exceptions exist to avoid implicit
dependencies; you always define `volumes_from` locally. This ensures
dependencies between services are clearly visible when reading the current file.
Defining these locally also ensures that changes to the referenced file don't
break anything.
### Understand the extends configuration
When defining any service in `docker-compose.yml`, you can declare that you are
extending another service like this:
```yaml
services:
web:
extends:
file: common-services.yml
service: webapp
```
This instructs Compose to re-use the configuration for the `webapp` service
defined in the `common-services.yml` file. Suppose that `common-services.yml`
looks like this:
```yaml
services:
webapp:
build: .
ports:
- "8000:8000"
volumes:
- "/data"
```
In this case, you get exactly the same result as if you wrote
`docker-compose.yml` with the same `build`, `ports` and `volumes` configuration
values defined directly under `web`.
You can go further and define (or re-define) configuration locally in
`docker-compose.yml`:
```yaml
services:
web:
extends:
file: common-services.yml
service: webapp
environment:
- DEBUG=1
cpu_shares: 5
important_web:
extends: web
cpu_shares: 10
```
You can also write other services and link your `web` service to them:
```yaml
services:
web:
extends:
file: common-services.yml
service: webapp
environment:
- DEBUG=1
cpu_shares: 5
depends_on:
- db
db:
image: postgres
```
### Example use case
Extending an individual service is useful when you have multiple services that
have a common configuration. The example below is a Compose app with
two services: a web application and a queue worker. Both services use the same
codebase and share many configuration options.
In a **common.yml** we define the common configuration:
```yaml
services:
app:
build: .
environment:
CONFIG_FILE_PATH: /code/config
API_KEY: xxxyyy
cpu_shares: 5
```
In a **docker-compose.yml** we define the concrete services which use the
common configuration:
```yaml
services:
webapp:
extends:
file: common.yml
service: app
command: /code/run_web_app
ports:
- 8080:8080
depends_on:
- queue
- db
queue_worker:
extends:
file: common.yml
service: app
command: /code/run_worker
depends_on:
- queue
```
## Adding and overriding configuration
Compose copies configurations from the original service over to the local one.
If a configuration option is defined in both the original service and the local
service, the local value *replaces* or *extends* the original value.
For single-value options like `image`, `command` or `mem_limit`, the new value
replaces the old value.
original service:
```yaml
services:
myservice:
# ...
command: python app.py
```
local service:
```yaml
services:
myservice:
# ...
command: python otherapp.py
```
result:
```yaml
services:
myservice:
# ...
command: python otherapp.py
```
For the **multi-value options** `ports`, `expose`, `external_links`, `dns`,
`dns_search`, and `tmpfs`, Compose concatenates both sets of values:
original service:
```yaml
services:
myservice:
# ...
expose:
- "3000"
```
local service:
```yaml
services:
myservice:
# ...
expose:
- "4000"
- "5000"
```
result:
```yaml
services:
myservice:
# ...
expose:
- "3000"
- "4000"
- "5000"
```
In the case of `environment`, `labels`, `volumes`, and `devices`, Compose
"merges" entries together with locally-defined values taking precedence. For
`environment` and `labels`, the environment variable or label name determines
which value is used:
original service:
```yaml
services:
myservice:
# ...
environment:
- FOO=original
- BAR=original
```
local service:
```yaml
services:
myservice:
# ...
environment:
- BAR=local
- BAZ=local
```
result
```yaml
services:
myservice:
# ...
environment:
- FOO=original
- BAR=local
- BAZ=local
```
Entries for `volumes` and `devices` are merged using the mount path in the
container:
original service:
```yaml
services:
myservice:
# ...
volumes:
- ./original:/foo
- ./original:/bar
```
local service:
```yaml
services:
myservice:
# ...
volumes:
- ./local:/bar
- ./local:/baz
```
result:
```yaml
services:
myservice:
# ...
volumes:
- ./original:/foo
- ./local:/bar
- ./local:/baz
```
## Reference information
[`extends`](compose-file/05-services.md#extends)

2
compose/features-uses.md
View File

@ -85,7 +85,7 @@ See [Variable substitution](compose-file/compose-file-v3.md#variable-substitutio
details.
You can extend a Compose file using the `extends` field or by creating multiple
Compose files. See [extends](extends.md) for more details.
Compose files. For more details, see [Working with multiple Compose files](multiple-compose-files/index.md).
## Common use cases of Docker Compose

193
compose/multiple-compose-files/extends.md Normal file
View File

@ -0,0 +1,193 @@
---
description: How to use Docker Compose's extends keyword to share configuration between files and projects
keywords: fig, composition, compose, docker, orchestration, documentation, docs
title: Extend your Compose file
redirect:
- /compose/extends/
---
Docker Compose's [`extends` attribute](../compose-file/05-services.md#extends) lets you share common configurations
among different files, or even different projects entirely.
Extending services
is useful if you have several services that reuse a common set of configuration
options. With `extends` you can define a common set of service options in one
place and refer to it from anywhere. You can refer to another Compose file and select a service you want to also use in your own application, with the ability to override some attributes for your own needs.
> **Important**
>
> When you use multiple Compose files, you must make sure all paths in the
files are relative to the base Compose file. This is required because extend files need not be valid
Compose files. Extend files can contain small fragments of configuration.
Tracking which fragment of a service is relative to which path is difficult and
confusing, so to keep paths easier to understand, all paths must be defined
relative to the base file.
{: .important}
## How it works
When defining any service in your `compose.yaml` file, you can declare that you are
extending another service:
```yaml
services:
web:
extends:
file: common-services.yml
service: webapp
```
This instructs Compose to re-use the configuration for the `webapp` service
defined in the `common-services.yaml` file. Suppose that `common-services.yaml`
looks like this:
```yaml
services:
webapp:
build: .
ports:
- "8000:8000"
volumes:
- "/data"
```
In this case, you get exactly the same result as if you wrote
`docker-compose.yml` with the same `build`, `ports` and `volumes` configuration
values defined directly under `web`.
You can go further and define, or re-define, configuration locally in
`compose.yaml`:
```yaml
services:
web:
extends:
file: common-services.yml
service: webapp
environment:
- DEBUG=1
cpu_shares: 5
important_web:
extends: web
cpu_shares: 10
```
You can also write other services and link your `web` service to them:
```yaml
services:
web:
extends:
file: common-services.yml
service: webapp
environment:
- DEBUG=1
cpu_shares: 5
depends_on:
- db
db:
image: postgres
```
## Further examples
### Example one
Extending an individual service is useful when you have multiple services that
have a common configuration. The example below is a Compose app with
two services, a web application and a queue worker. Both services use the same
codebase and share many configuration options.
The `common.yaml` file defines the common configuration:
```yaml
services:
app:
build: .
environment:
CONFIG_FILE_PATH: /code/config
API_KEY: xxxyyy
cpu_shares: 5
```
The `docker-compose.yaml` defines the concrete services which use the
common configuration:
```yaml
services:
webapp:
extends:
file: common.yaml
service: app
command: /code/run_web_app
ports:
- 8080:8080
depends_on:
- queue
- db
queue_worker:
extends:
file: common.yaml
service: app
command: /code/run_worker
depends_on:
- queue
```
### Example two
Another common use case for `extends` is running one off or administrative tasks against one
or more services in a Compose app. This example demonstrates running a
database backup.
The `docker-compose.yml` defines the base configuration.
```yaml
services:
web:
image: example/my_web_app:latest
depends_on:
db
db:
image: postgres:latest
```
`docker-compose.admin.yml` adds a new service to run the database
export or backup.
```yaml
services:
dbadmin:
build: database_admin/
depends_on:
- db
```
To start a normal environment, run `docker compose up -d`. To run a database
backup, include the `docker-compose.admin.yml` as well.
```console
$ docker compose -f docker-compose.yml -f docker-compose.admin.yml \
run dbadmin db-backup
```
Compose extends files in
the order they're specified on the command line.
## Exceptions and limitations
`volumes_from` and `depends_on` are never shared between
services using `extends`. These exceptions exist to avoid implicit
dependencies; you always define `volumes_from` locally. This ensures
dependencies between services are clearly visible when reading the current file.
Defining these locally also ensures that changes to the referenced file don't
break anything.
`extends` is useful if you only need a single service to be shared and you are familiar with the file you're extending to, so you can to tweak the configuration. But this isn’t an acceptable solution when you want to re-use someone else's unfamiliar configurations and you don’t know about its own dependencies.
## Reference information
- [`extends`](../compose-file/05-services.md#extends)

37
compose/multiple-compose-files/include.md Normal file
View File

@ -0,0 +1,37 @@
---
description: How to use Docker Compose's include top-level element
keywords: compose, docker, include, compose file
title: Include
---
With Docker Compose version 2.20 and later, you can include a whole Compose file directly in your local Compose file using the [`include` top-level element](../compose-file/14-include.md). This solves the relative path problem that [`extends`](extends.md) and [merge](merge.md) present.
`include` makes it easier to modularize complex applications into sub-Compose files. This allows application configurations to be made simpler and more explicit. This also helps to reflect in the config file organization the engineering team responsible for the code.
Each path listed in the `include` section loads as an individual Compose application model, with its own project directory, in order to resolve relative paths.
Once the included Compose application loads, all resources are copied into the current Compose application model.
> **Note**
>
> `include` applies recursively so an included Compose file which declares its own `include` section, results in those other files being included as well.
## Example
```yaml
include:
- my-compose-include.yaml #with serviceB declared
services:
serviceA:
build: .
depends_on:
- serviceB #use serviceB directly as if it was declared in this Compose file
```
`my-compose-include.yaml` manages `serviceB` which details some replicas, web UI to inspect data, isolated networks, volumes for data persistence, etc. The application relying on `serviceB` doesn’t need to know about the infrastructure details, and consumes the Compose file as a building block it can rely on.
This means the team managing `serviceB` can refactor its own database component to introduce additional services without impacting any dependent teams. It also means that the dependent teams don't need to include additional flags on each Compose command they run.
## Reference information
[`include` top-level element](../compose-file/14-include.md)

18
compose/multiple-compose-files/index.md Normal file
View File

@ -0,0 +1,18 @@
---
description: General overview for the different ways you can work with multiple compose files in Docker Compose
keywords: compose, compose file, merge, extends, include, docker compose
title: Overview
---
{% include compose-eol.md %}
This section contains information on the three key ways you can use multiple Compose files in your Compose application.
Using multiple Compose files lets you customize a Compose application for different environments or workflows. This is useful for large applications that may use dozens of containers, with ownership distributed across multiple teams.
For example, if your organization or team uses a monorepo, each team may have their own “local” Compose file to run a subset of the application. They then need to rely on other teams to provide a reference Compose file that defines the expected way to run their own subset. Complexity moves from the code in to the infrastructure and the configuration file.
Docker Compose provides three ways to manage this complexity when working with multiple Compose files. Depending on your project's needs, you can:
- [Extend a Compose file](extends.md) by referring to another Compose file and selecting the bits you want to use in your own application, with the ability to override some attributes.
- [Merge a set of Compose files](merge.md) together to create a composite Compose file.
- [Include other Compose files](include.md) directly in to your Compose file.

278
compose/multiple-compose-files/merge.md Normal file
View File

@ -0,0 +1,278 @@
---
description: How merging Compose files works
keywords: compose, docker, merge, compose file
title: Merge Compose files
---
Docker Compose lets you merge and override a set of Compose files together to create a composite Compose file.
By default, Compose reads two files, a `compose.yml` and an optional
`compose.override.yml` file. By convention, the `compose.yml`
contains your base configuration. The override file can
contain configuration overrides for existing services or entirely new
services.
If a service is defined in both files, Compose merges the configurations using
the rules described below and in the
[Compose Specification](../compose-file/13-merge.md).
To use multiple override files, or an override file with a different name, you
can use the `-f` option to specify the list of files. Compose merges files in
the order they're specified on the command line. See the
[`docker compose` command reference](../reference/index.md) for more information
about using `-f`.
> **Important**
>
> When you use multiple Compose files, you must make sure all paths in the
files are relative to the base Compose file (the first Compose file specified
with `-f`). This is required because override files need not be valid
Compose files. Override files can contain small fragments of configuration.
Tracking which fragment of a service is relative to which path is difficult and
confusing, so to keep paths easier to understand, all paths must be defined
relative to the base file.
{: .important}
## Merging rules
Compose copies configurations from the original service over to the local one.
If a configuration option is defined in both the original service and the local
service, the local value replaces or extends the original value.
For single-value options like `image`, `command` or `mem_limit`, the new value
replaces the old value.
original service:
```yaml
services:
myservice:
# ...
command: python app.py
```
local service:
```yaml
services:
myservice:
# ...
command: python otherapp.py
```
result:
```yaml
services:
myservice:
# ...
command: python otherapp.py
```
For the multi-value options `ports`, `expose`, `external_links`, `dns`,
`dns_search`, and `tmpfs`, Compose concatenates both sets of values:
original service:
```yaml
services:
myservice:
# ...
expose:
- "3000"
```
local service:
```yaml
services:
myservice:
# ...
expose:
- "4000"
- "5000"
```
result:
```yaml
services:
myservice:
# ...
expose:
- "3000"
- "4000"
- "5000"
```
In the case of `environment`, `labels`, `volumes`, and `devices`, Compose
"merges" entries together with locally defined values taking precedence. For
`environment` and `labels`, the environment variable or label name determines
which value is used:
original service:
```yaml
services:
myservice:
# ...
environment:
- FOO=original
- BAR=original
```
local service:
```yaml
services:
myservice:
# ...
environment:
- BAR=local
- BAZ=local
```
result:
```yaml
services:
myservice:
# ...
environment:
- FOO=original
- BAR=local
- BAZ=local
```
Entries for `volumes` and `devices` are merged using the mount path in the
container:
original service:
```yaml
services:
myservice:
# ...
volumes:
- ./original:/foo
- ./original:/bar
```
local service:
```yaml
services:
myservice:
# ...
volumes:
- ./local:/bar
- ./local:/baz
```
result:
```yaml
services:
myservice:
# ...
volumes:
- ./original:/foo
- ./local:/bar
- ./local:/baz
```
For more merging rules, see [Merge and override](../compose-file/13-merge.md) in the Compose Specification.
## Example
A common use case for multiple files is changing a development Compose app
for a production-like environment (which may be production, staging or CI).
To support these differences, you can split your Compose configuration into
a few different files:
Start with a base file that defines the canonical configuration for the
services.
`compose.yml`
```yaml
services:
web:
image: example/my_web_app:latest
depends_on:
- db
- cache
db:
image: postgres:latest
cache:
image: redis:latest
```
In this example the development configuration exposes some ports to the
host, mounts our code as a volume, and builds the web image.
`compose.override.yml`
```yaml
services:
web:
build: .
volumes:
- '.:/code'
ports:
- 8883:80
environment:
DEBUG: 'true'
db:
command: '-d'
ports:
- 5432:5432
cache:
ports:
- 6379:6379
```
When you run `docker compose up` it reads the overrides automatically.
To use this Compose app in a production environment, another override file is created, which might be stored in a different git
repo or managed by a different team.
`compose.prod.yml`
```yaml
services:
web:
ports:
- 80:80
environment:
PRODUCTION: 'true'
cache:
environment:
TTL: '500'
```
To deploy with this production Compose file you can run
```console
$ docker compose -f compose.yml -f compose.prod.yml up -d
```
This deploys all three services using the configuration in
`compose.yml` and `compose.prod.yml` but not the
dev configuration in `compose.override.yml`.
For more information, see [Using Compose in production](../production.md).
## Limitations
Docker Compose supports relative paths for the many resources to be included in the application model: build context for service images, location of file defining environment variables, path to a local directory used in a bind-mounted volume.
With such a constraint, code organization in a monorepo can become hard as a natural choice would be to have dedicated folders per team or component, but then the Compose files relative paths become irrelevant.
## Reference information
- [Merge rules](../compose-file/13-merge.md)

3
compose/production.md
View File

@ -38,8 +38,7 @@ Once you have a second configuration file, you can use it with the
$ docker compose -f docker-compose.yml -f production.yml up -d
```
See [Using multiple compose files](extends.md#different-environments) for a more
complete example.
See [Using multiple compose files](multiple-compose-files/index.md) for a more complete example, and other options.
### Deploying changes