rewrite storage-driver section

- Reduce use of "Community Edition", and some sections that were written
  for Docker Enterprise.
- Add notes about deprecation of AuFS in the overview.
- Update the storage driver index page to use BuildKit, and rephrase some
  bits that made assumptions about each Dockerfile instruction producing
  a new layer.
- Updated some examples to be usable on Docker Desktop, which does not
  provide direct access to the daemon's storage directoreis.
- Add some notes linking to relevant sections about formatting output.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This commit is contained in:
Sebastiaan van Stijn 2021-07-28 12:17:13 +02:00
parent 74cb51dcc3
commit f5e49b158b
No known key found for this signature in database
GPG Key ID: 76698F39D527CE8C
2 changed files with 342 additions and 254 deletions

View File

@ -14,14 +14,22 @@ stores images, and how these images are used by containers. You can use this
information to make informed choices about the best way to persist data from
your applications and avoid performance problems along the way.
Storage drivers allow you to create data in the writable layer of your container.
The files won't be persisted after the container is deleted, and both read and
write speeds are lower than native file system performance.
## Storage drivers versus Docker volumes
> **Note**: Operations that are known to be problematic include write-intensive database storage,
particularly when pre-existing data exists in the read-only layer. More details are provided in this document.
Docker uses storage drivers to store image layers, and to store data in the
writable layer of a container. The container's writable layer does not persist
after the container is deleted, but is suitable for storing ephemeral data that
is generated at runtime. Storage drivers are optimized for space efficiency, but
(depending on the storage driver) write speeds are lower than native file system
performance, especially for storage drivers that a use copy-on-write filesystem.
Write-intensive applications, such as database storage, are impacted by a
performance overhead, particularly if pre-existing data exists in the read-only
layer.
[Learn how to use volumes](../volumes.md) to persist data and improve performance.
Use Docker volumes for write-intensive data, data that must persist beyond the
container's lifespan, and data that must be shared between containers. Refer to
the [volumes section](../volumes.md) to learn how to use volumes to persist data
and improve performance.
## Images and layers
@ -32,24 +40,37 @@ read-only. Consider the following Dockerfile:
```dockerfile
# syntax=docker/dockerfile:1
FROM ubuntu:18.04
LABEL org.opencontainers.image.authors="org@example.com"
COPY . /app
RUN make /app
RUN rm -r $HOME/.cache
CMD python /app/app.py
```
This Dockerfile contains four commands, each of which creates a layer. The
`FROM` statement starts out by creating a layer from the `ubuntu:18.04` image.
The `COPY` command adds some files from your Docker client's current directory.
The `RUN` command builds your application using the `make` command. Finally,
the last layer specifies what command to run within the container.
This Dockerfile contains four commands. Commands that modify the filesystem create
a layer. The`FROM` statement starts out by creating a layer from the `ubuntu:18.04`
image. The `LABEL` command only modifies the image's metadata, and does not produce
a new layer. The `COPY` command adds some files from your Docker client's current
directory. The first `RUN` command builds your application using the `make` command,
and writes the result to a new layer. The second `RUN` command removes a cache
directory, and writes the result to a new layer. Finally, the `CMD` instruction
specifies what command to run within the container, which only modifies the
image's metadata, which does not produce an image layer.
Each layer is only a set of differences from the layer before it. The layers are
stacked on top of each other. When you create a new container, you add a new
writable layer on top of the underlying layers. This layer is often called the
"container layer". All changes made to the running container, such as writing
new files, modifying existing files, and deleting files, are written to this thin
writable container layer. The diagram below shows a container based on the Ubuntu
15.04 image.
Each layer is only a set of differences from the layer before it. Note that both
_adding_, and _removing_ files will result in a new layer. In the example above,
the `$HOME/.cache` directory is removed, but will still be available in the
previous layer and add up to the image's total size. Refer to the
[Best practices for writing Dockerfiles](../../develop/develop-images/dockerfile_best-practices.md)
and [use multi-stage builds](../../develop/develop-images/multistage-build.md)
sections to learn how to optimize your Dockerfiles for efficient images.
The layers are stacked on top of each other. When you create a new container,
you add a new writable layer on top of the underlying layers. This layer is often
called the "container layer". All changes made to the running container, such as
writing new files, modifying existing files, and deleting files, are written to
this thin writable container layer. The diagram below shows a container based
on an `ubuntu:15.04` image.
![Layers of a container based on the Ubuntu image](images/container-layers.jpg)
@ -71,15 +92,18 @@ multiple containers sharing the same Ubuntu 15.04 image.
![Containers sharing same image](images/sharing-layers.jpg)
> **Note**: If you need multiple images to have shared access to the exact
> same data, store this data in a Docker volume and mount it into your
> containers.
Docker uses storage drivers to manage the contents of the image layers and the
writable container layer. Each storage driver handles the implementation
differently, but all drivers use stackable image layers and the copy-on-write
(CoW) strategy.
> **Note**
>
> Use Docker volumes if you need multiple containers to have shared access to
> the exact same data. Refer to the [volumes section](../volumes.md) to learn
> about volumes.
## Container size on disk
To view the approximate size of a running container, you can use the `docker ps -s`
@ -87,7 +111,6 @@ command. Two different columns relate to size.
- `size`: the amount of data (on disk) that is used for the writable layer of
each container.
- `virtual size`: the amount of data used for the read-only image data
used by the container plus the container's writable layer `size`.
Multiple containers may share some or all read-only
@ -106,9 +129,9 @@ these containers would be SUM (`size` of containers) plus one image size
This also does not count the following additional ways a container can take up
disk space:
- Disk space used for log files if you use the `json-file` logging driver. This
can be non-trivial if your container generates a large amount of logging data
and log rotation is not configured.
- Disk space used for log files stored by the [logging-driver](../../config/containers/logging/index.md).
This can be non-trivial if your container generates a large amount of logging
data and log rotation is not configured.
- Volumes and bind mounts used by the container.
- Disk space used for the container's configuration files, which are typically
small.
@ -133,7 +156,7 @@ pulled down separately, and stored in Docker's local storage area, which is
usually `/var/lib/docker/` on Linux hosts. You can see these layers being pulled
in this example:
```bash
```console
$ docker pull ubuntu:18.04
18.04: Pulling from library/ubuntu
f476d66f5408: Pull complete
@ -149,7 +172,7 @@ local storage area. To examine the layers on the filesystem, list the contents
of `/var/lib/docker/<storage-driver>`. This example uses the `overlay2`
storage driver:
```bash
```console
$ ls /var/lib/docker/overlay2
16802227a96c24dcbeab5b37821e2b67a9f921749cd9a2e386d5a6d5bc6fc6d3
377d73dbb466e0bc7c9ee23166771b35ebdbe02ef17753d79fd3571d4ce659d7
@ -158,16 +181,15 @@ ec1ec45792908e90484f7e629330666e7eee599f08729c93890a7205a6ba35f5
l
```
The directory names do not correspond to the layer IDs (this has been true since
Docker 1.10).
The directory names do not correspond to the layer IDs.
Now imagine that you have two different Dockerfiles. You use the first one to
create an image called `acme/my-base-image:1.0`.
```dockerfile
# syntax=docker/dockerfile:1
FROM ubuntu:18.04
COPY . /app
FROM alpine
RUN apk add --no-cache bash
```
The second one is based on `acme/my-base-image:1.0`, but has some additional
@ -176,16 +198,18 @@ layers:
```dockerfile
# syntax=docker/dockerfile:1
FROM acme/my-base-image:1.0
COPY . /app
RUN chmod +x /app/hello.sh
CMD /app/hello.sh
```
The second image contains all the layers from the first image, plus a new layer
with the `CMD` instruction, and a read-write container layer. Docker already
has all the layers from the first image, so it does not need to pull them again.
The two images share any layers they have in common.
The second image contains all the layers from the first image, plus new layers
created by the `COPY` and `RUN` instructions, and a read-write container layer.
Docker already has all the layers from the first image, so it does not need to
pull them again. The two images share any layers they have in common.
If you build images from the two Dockerfiles, you can use `docker image ls` and
`docker history` commands to verify that the cryptographic IDs of the shared
`docker image history` commands to verify that the cryptographic IDs of the shared
layers are the same.
1. Make a new directory `cow-test/` and change into it.
@ -193,16 +217,10 @@ layers are the same.
2. Within `cow-test/`, create a new file called `hello.sh` with the following contents:
```bash
#!/bin/sh
#!/usr/bin/env bash
echo "Hello world"
```
Save the file, and make it executable:
```bash
chmod +x hello.sh
```
3. Copy the contents of the first Dockerfile above into a new file called
`Dockerfile.base`.
@ -215,14 +233,23 @@ layers are the same.
```console
$ docker build -t acme/my-base-image:1.0 -f Dockerfile.base .
Sending build context to Docker daemon 812.4MB
Step 1/2 : FROM ubuntu:18.04
---> d131e0fa2585
Step 2/2 : COPY . /app
---> Using cache
---> bd09118bcef6
Successfully built bd09118bcef6
Successfully tagged acme/my-base-image:1.0
[+] Building 6.0s (11/11) FINISHED
=> [internal] load build definition from Dockerfile.base 0.4s
=> => transferring dockerfile: 116B 0.0s
=> [internal] load .dockerignore 0.3s
=> => transferring context: 2B 0.0s
=> resolve image config for docker.io/docker/dockerfile:1 1.5s
=> [auth] docker/dockerfile:pull token for registry-1.docker.io 0.0s
=> CACHED docker-image://docker.io/docker/dockerfile:1@sha256:9e2c9eca7367393aecc68795c671... 0.0s
=> [internal] load .dockerignore 0.0s
=> [internal] load build definition from Dockerfile.base 0.0s
=> [internal] load metadata for docker.io/library/alpine:latest 0.0s
=> CACHED [1/2] FROM docker.io/library/alpine 0.0s
=> [2/2] RUN apk add --no-cache bash 3.1s
=> exporting to image 0.2s
=> => exporting layers 0.2s
=> => writing image sha256:da3cf8df55ee9777ddcd5afc40fffc3ead816bda99430bad2257de4459625eaa 0.0s
=> => naming to docker.io/acme/my-base-image:1.0 0.0s
```
6. Build the second image.
@ -230,15 +257,25 @@ layers are the same.
```console
$ docker build -t acme/my-final-image:1.0 -f Dockerfile .
Sending build context to Docker daemon 4.096kB
Step 1/2 : FROM acme/my-base-image:1.0
---> bd09118bcef6
Step 2/2 : CMD /app/hello.sh
---> Running in a07b694759ba
---> dbf995fc07ff
Removing intermediate container a07b694759ba
Successfully built dbf995fc07ff
Successfully tagged acme/my-final-image:1.0
[+] Building 3.6s (12/12) FINISHED
=> [internal] load build definition from Dockerfile 0.1s
=> => transferring dockerfile: 156B 0.0s
=> [internal] load .dockerignore 0.1s
=> => transferring context: 2B 0.0s
=> resolve image config for docker.io/docker/dockerfile:1 0.5s
=> CACHED docker-image://docker.io/docker/dockerfile:1@sha256:9e2c9eca7367393aecc68795c671... 0.0s
=> [internal] load .dockerignore 0.0s
=> [internal] load build definition from Dockerfile 0.0s
=> [internal] load metadata for docker.io/acme/my-base-image:1.0 0.0s
=> [internal] load build context 0.2s
=> => transferring context: 340B 0.0s
=> [1/3] FROM docker.io/acme/my-base-image:1.0 0.2s
=> [2/3] COPY . /app 0.1s
=> [3/3] RUN chmod +x /app/hello.sh 0.4s
=> exporting to image 0.1s
=> => exporting layers 0.1s
=> => writing image sha256:8bd85c42fa7ff6b33902ada7dcefaaae112bf5673873a089d73583b0074313dd 0.0s
=> => naming to docker.io/acme/my-final-image:1.0 0.0s
```
7. Check out the sizes of the images:
@ -246,47 +283,99 @@ layers are the same.
```console
$ docker image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
acme/my-final-image 1.0 dbf995fc07ff 58 seconds ago 103MB
acme/my-base-image 1.0 bd09118bcef6 3 minutes ago 103MB
REPOSITORY TAG IMAGE ID CREATED SIZE
acme/my-final-image 1.0 8bd85c42fa7f About a minute ago 7.75MB
acme/my-base-image 1.0 da3cf8df55ee 2 minutes ago 7.75MB
```
8. Check out the layers that comprise each image:
8. Check out the history of each image:
```console
$ docker history bd09118bcef6
IMAGE CREATED CREATED BY SIZE COMMENT
bd09118bcef6 4 minutes ago /bin/sh -c #(nop) COPY dir:35a7eb158c1504e... 100B
d131e0fa2585 3 months ago /bin/sh -c #(nop) CMD ["/bin/bash"] 0B
<missing> 3 months ago /bin/sh -c mkdir -p /run/systemd && echo '... 7B
<missing> 3 months ago /bin/sh -c sed -i 's/^#\s*\(deb.*universe\... 2.78kB
<missing> 3 months ago /bin/sh -c rm -rf /var/lib/apt/lists/* 0B
<missing> 3 months ago /bin/sh -c set -xe && echo '#!/bin/sh' >... 745B
<missing> 3 months ago /bin/sh -c #(nop) ADD file:eef57983bd66e3a... 103MB
$ docker image history acme/my-base-image:1.0
IMAGE CREATED CREATED BY SIZE COMMENT
da3cf8df55ee 5 minutes ago RUN /bin/sh -c apk add --no-cache bash # bui… 2.15MB buildkit.dockerfile.v0
<missing> 7 weeks ago /bin/sh -c #(nop) CMD ["/bin/sh"] 0B
<missing> 7 weeks ago /bin/sh -c #(nop) ADD file:f278386b0cef68136… 5.6MB
```
Some steps do not have a size (`0B`), and are metadata-only changes, which do
not produce an image layer and do not take up any size, other than the metadata
itself. The output above shows that this image consists of 2 image layers.
```console
$ docker history dbf995fc07ff
$ docker image history acme/my-final-image:1.0
IMAGE CREATED CREATED BY SIZE COMMENT
dbf995fc07ff 3 minutes ago /bin/sh -c #(nop) CMD ["/bin/sh" "-c" "/a... 0B
bd09118bcef6 5 minutes ago /bin/sh -c #(nop) COPY dir:35a7eb158c1504e... 100B
d131e0fa2585 3 months ago /bin/sh -c #(nop) CMD ["/bin/bash"] 0B
<missing> 3 months ago /bin/sh -c mkdir -p /run/systemd && echo '... 7B
<missing> 3 months ago /bin/sh -c sed -i 's/^#\s*\(deb.*universe\... 2.78kB
<missing> 3 months ago /bin/sh -c rm -rf /var/lib/apt/lists/* 0B
<missing> 3 months ago /bin/sh -c set -xe && echo '#!/bin/sh' >... 745B
<missing> 3 months ago /bin/sh -c #(nop) ADD file:eef57983bd66e3a... 103MB
IMAGE CREATED CREATED BY SIZE COMMENT
8bd85c42fa7f 3 minutes ago CMD ["/bin/sh" "-c" "/app/hello.sh"] 0B buildkit.dockerfile.v0
<missing> 3 minutes ago RUN /bin/sh -c chmod +x /app/hello.sh # buil… 39B buildkit.dockerfile.v0
<missing> 3 minutes ago COPY . /app # buildkit 222B buildkit.dockerfile.v0
<missing> 4 minutes ago RUN /bin/sh -c apk add --no-cache bash # bui… 2.15MB buildkit.dockerfile.v0
<missing> 7 weeks ago /bin/sh -c #(nop) CMD ["/bin/sh"] 0B
<missing> 7 weeks ago /bin/sh -c #(nop) ADD file:f278386b0cef68136… 5.6MB
```
Notice that all the layers are identical except the top layer of the second
image. All the other layers are shared between the two images, and are only
stored once in `/var/lib/docker/`. The new layer actually doesn't take any
room at all, because it is not changing any files, but only running a command.
Notice that all steps of the first image are also included in the final
image. The final image includes the two layers from the first image, and
two layers that were added in the second image.
> **Note**: The `<missing>` lines in the `docker history` output indicate
> that those layers were built on another system and are not available
> locally. This can be ignored.
> What are the `<missing>` steps?
>
> The `<missing>` lines in the `docker history` output indicate that those
> steps were either built on another system and part of the `alpine` image
> that was pulled from Docker Hub, or were built with BuildKit as builder.
> Before BuildKit, the "classic" builder would produce a new "intermediate"
> image for each step for caching purposes, and the `IMAGE` column would show
> the ID of that image.
> BuildKit uses its own caching mechanism, and no longer requires intermediate
> images for caching. Refer to [build images with BuildKit](../../develop/develop-images/build_enhancements.md)
> to learn more about other enhancements made in BuildKit.
9. Check out the layers for each image
Use the `docker image inspect` command to view the cryptographic IDs of the
layers in each image:
{% raw %}
```console
$ docker image inspect --format "{{json .RootFS.Layers}}" acme/my-final-image:1.0
[
"sha256:72e830a4dff5f0d5225cdc0a320e85ab1ce06ea5673acfe8d83a7645cbd0e9cf",
"sha256:07b4a9068b6af337e8b8f1f1dae3dd14185b2c0003a9a1f0a6fd2587495b204a"
]
```
{% endraw %}
{% raw %}
```console
$ docker image inspect --format "{{json .RootFS.Layers}}" acme/my-base-image:1.0
[
"sha256:72e830a4dff5f0d5225cdc0a320e85ab1ce06ea5673acfe8d83a7645cbd0e9cf",
"sha256:07b4a9068b6af337e8b8f1f1dae3dd14185b2c0003a9a1f0a6fd2587495b204a",
"sha256:cc644054967e516db4689b5282ee98e4bc4b11ea2255c9630309f559ab96562e",
"sha256:e84fb818852626e89a09f5143dbc31fe7f0e0a6a24cd8d2eb68062b904337af4"
]
```
{% endraw %}
Notice that the first two layers are identical in both images. The second
image adds two additional layers. Shared image layers are only stored once
in `/var/lib/docker/` and are also shared when pushing and pulling and image
to an image registry. Shared image layers can therefore reduce network
bandwidth and storage.
> Tip: format output of Docker commands with the `--format` option
>
> The examples above use the `docker image inspect` command with the `--format`
> option to view the layer IDs, formatted as a JSON array. The `--format`
> option on Docker commands can be a powerful feature that allows you to
> extract and format specific information from the output, without requiring
> additional tools such as `awk` or `sed`. To learn more about formatting
> the output of docker commands using the `--format` flag, refer to the
> [format command and log output section](../../config/formatting.md).
> We also pretty-printed the JSON output using the [`jq` utility](https://stedolan.github.io/jq/){: target="_blank" rel="noopener" class="_" }
> for readability.
### Copying makes containers efficient
@ -297,16 +386,14 @@ layer. This means that the writable layer is as small as possible.
When an existing file in a container is modified, the storage driver performs a
copy-on-write operation. The specifics steps involved depend on the specific
storage driver. For the `aufs`, `overlay`, and `overlay2` drivers, the
storage driver. For the `overlay2`, `overlay`, and `aufs` drivers, the
copy-on-write operation follows this rough sequence:
* Search through the image layers for the file to update. The process starts
at the newest layer and works down to the base layer one layer at a time.
When results are found, they are added to a cache to speed future operations.
* Perform a `copy_up` operation on the first copy of the file that is found, to
copy the file to the container's writable layer.
* Any modifications are made to this copy of the file, and the container cannot
see the read-only copy of the file that exists in the lower layer.
@ -316,13 +403,20 @@ descriptions.
Containers that write a lot of data consume more space than containers
that do not. This is because most write operations consume new space in the
container's thin writable top layer.
container's thin writable top layer. Note that changing the metadata of files,
for example, changing file permissions or ownership of a file, can also result
in a `copy_up` operation, therefore duplicating the file to the writable layer.
> **Note**: for write-heavy applications, you should not store the data in
> the container. Instead, use Docker volumes, which are independent of the
> running container and are designed to be efficient for I/O. In addition,
> volumes can be shared among containers and do not increase the size of your
> container's writable layer.
> Tip: Use volumes for write-heavy applications
>
> For write-heavy applications, you should not store the data in the container.
> Applications, such as write-intensive database storage, are known to be
> problematic particularly when pre-existing data exists in the read-only layer.
>
> Instead, use Docker volumes, which are independent of the running container,
> and designed to be efficient for I/O. In addition, volumes can be shared among
> containers and do not increase the size of your container's writable layer.
> Refer to the [use volumes](../volumes.md) section to learn about volumes.
A `copy_up` operation can incur a noticeable performance overhead. This overhead
is different depending on which storage driver is in use. Large files,
@ -334,72 +428,107 @@ To verify the way that copy-on-write works, the following procedures spins up 5
containers based on the `acme/my-final-image:1.0` image we built earlier and
examines how much room they take up.
> **Note**: This procedure doesn't work on Docker Desktop for Mac or Docker Desktop for Windows.
1. From a terminal on your Docker host, run the following `docker run` commands.
The strings at the end are the IDs of each container.
```bash
```console
$ docker run -dit --name my_container_1 acme/my-final-image:1.0 bash \
&& docker run -dit --name my_container_2 acme/my-final-image:1.0 bash \
&& docker run -dit --name my_container_3 acme/my-final-image:1.0 bash \
&& docker run -dit --name my_container_4 acme/my-final-image:1.0 bash \
&& docker run -dit --name my_container_5 acme/my-final-image:1.0 bash
c36785c423ec7e0422b2af7364a7ba4da6146cbba7981a0951fcc3fa0430c409
dcad7101795e4206e637d9358a818e5c32e13b349e62b00bf05cd5a4343ea513
1e7264576d78a3134fbaf7829bc24b1d96017cf2bc046b7cd8b08b5775c33d0c
38fa94212a419a082e6a6b87a8e2ec4a44dd327d7069b85892a707e3fc818544
1a174fc216cccf18ec7d4fe14e008e30130b11ede0f0f94a87982e310cf2e765
40ebdd7634162eb42bdb1ba76a395095527e9c0aa40348e6c325bd0aa289423c
a5ff32e2b551168b9498870faf16c9cd0af820edf8a5c157f7b80da59d01a107
3ed3c1a10430e09f253704116965b01ca920202d52f3bf381fbb833b8ae356bc
939b3bf9e7ece24bcffec57d974c939da2bdcc6a5077b5459c897c1e2fa37a39
cddae31c314fbab3f7eabeb9b26733838187abc9a2ed53f97bd5b04cd7984a5a
```
2. Run the `docker ps` command with the `--size` option to verify the 5 containers
are running, and to see each container's size.
2. Run the `docker ps` command to verify the 5 containers are running.
{% raw %}
```console
$ docker ps --size --format "table {{.ID}}\t{{.Image}}\t{{.Names}}\t{{.Size}}"
```bash
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
1a174fc216cc acme/my-final-image:1.0 "bash" About a minute ago Up About a minute my_container_5
38fa94212a41 acme/my-final-image:1.0 "bash" About a minute ago Up About a minute my_container_4
1e7264576d78 acme/my-final-image:1.0 "bash" About a minute ago Up About a minute my_container_3
dcad7101795e acme/my-final-image:1.0 "bash" About a minute ago Up About a minute my_container_2
c36785c423ec acme/my-final-image:1.0 "bash" About a minute ago Up About a minute my_container_1
CONTAINER ID IMAGE NAMES SIZE
cddae31c314f acme/my-final-image:1.0 my_container_5 0B (virtual 7.75MB)
939b3bf9e7ec acme/my-final-image:1.0 my_container_4 0B (virtual 7.75MB)
3ed3c1a10430 acme/my-final-image:1.0 my_container_3 0B (virtual 7.75MB)
a5ff32e2b551 acme/my-final-image:1.0 my_container_2 0B (virtual 7.75MB)
40ebdd763416 acme/my-final-image:1.0 my_container_1 0B (virtual 7.75MB)
```
{% endraw %}
The output above shows that all containers share the image's read-only layers
(7.75MB), but no data was written to the container's filesystem, so no additional
storage is used for the containers.
3. List the contents of the local storage area.
> Advanced: metadata and logs storage used for containers
>
> **Note**: This step requires a Linux machine, and does not work on Docker
> Desktop for Mac or Docker Desktop for Windows, as it requires access to
> the Docker Daemon's file storage.
>
> While the output of `docker ps` provides you information about disk space
> consumed by a container's writable layer, it does not include information
> about metadata and log-files stored for each container.
>
> More details can be obtained my exploring the Docker Daemon's storage location
> (`/var/lib/docker` by default).
>
> ```console
> $ sudo du -sh /var/lib/docker/containers/*
>
> 36K /var/lib/docker/containers/3ed3c1a10430e09f253704116965b01ca920202d52f3bf381fbb833b8ae356bc
> 36K /var/lib/docker/containers/40ebdd7634162eb42bdb1ba76a395095527e9c0aa40348e6c325bd0aa289423c
> 36K /var/lib/docker/containers/939b3bf9e7ece24bcffec57d974c939da2bdcc6a5077b5459c897c1e2fa37a39
> 36K /var/lib/docker/containers/a5ff32e2b551168b9498870faf16c9cd0af820edf8a5c157f7b80da59d01a107
> 36K /var/lib/docker/containers/cddae31c314fbab3f7eabeb9b26733838187abc9a2ed53f97bd5b04cd7984a5a
> ```
>
> Each of these containers only takes up 36k of space on the filesystem.
```bash
$ sudo ls /var/lib/docker/containers
3. Per-container storage
1a174fc216cccf18ec7d4fe14e008e30130b11ede0f0f94a87982e310cf2e765
1e7264576d78a3134fbaf7829bc24b1d96017cf2bc046b7cd8b08b5775c33d0c
38fa94212a419a082e6a6b87a8e2ec4a44dd327d7069b85892a707e3fc818544
c36785c423ec7e0422b2af7364a7ba4da6146cbba7981a0951fcc3fa0430c409
dcad7101795e4206e637d9358a818e5c32e13b349e62b00bf05cd5a4343ea513
To demonstrate this, run the following command to write the word 'hello' to
a file on the container's writable layer in containers `my_container_1`,
`my_container_2`, and `my_container_3`:
```console
$ for i in {1..3}; do docker exec my_container_$i sh -c 'printf hello > /out.txt'; done
```
Running the `docker ps` command again afterward shows that those containers
now consume 5 bytes each. This data is unique to each container, and not
shared. The read-only layers of the containers are not affected, and are still
shared by all containers.
4. Now check out their sizes:
{% raw %}
```console
$ docker ps --size --format "table {{.ID}}\t{{.Image}}\t{{.Names}}\t{{.Size}}"
```bash
$ sudo du -sh /var/lib/docker/containers/*
32K /var/lib/docker/containers/1a174fc216cccf18ec7d4fe14e008e30130b11ede0f0f94a87982e310cf2e765
32K /var/lib/docker/containers/1e7264576d78a3134fbaf7829bc24b1d96017cf2bc046b7cd8b08b5775c33d0c
32K /var/lib/docker/containers/38fa94212a419a082e6a6b87a8e2ec4a44dd327d7069b85892a707e3fc818544
32K /var/lib/docker/containers/c36785c423ec7e0422b2af7364a7ba4da6146cbba7981a0951fcc3fa0430c409
32K /var/lib/docker/containers/dcad7101795e4206e637d9358a818e5c32e13b349e62b00bf05cd5a4343ea513
CONTAINER ID IMAGE NAMES SIZE
cddae31c314f acme/my-final-image:1.0 my_container_5 0B (virtual 7.75MB)
939b3bf9e7ec acme/my-final-image:1.0 my_container_4 0B (virtual 7.75MB)
3ed3c1a10430 acme/my-final-image:1.0 my_container_3 5B (virtual 7.75MB)
a5ff32e2b551 acme/my-final-image:1.0 my_container_2 5B (virtual 7.75MB)
40ebdd763416 acme/my-final-image:1.0 my_container_1 5B (virtual 7.75MB)
```
{% endraw %}
Each of these containers only takes up 32k of space on the filesystem.
Not only does copy-on-write save space, but it also reduces start-up time.
When you start a container (or multiple containers from the same image), Docker
only needs to create the thin writable container layer.
The examples above illustrate how copy-on-write filesystems help making containers
efficient. Not only does copy-on-write save space, but it also reduces container
start-up time. When you create a container (or multiple containers from the same
image), Docker only needs to create the thin writable container layer.
If Docker had to make an entire copy of the underlying image stack each time it
started a new container, container start times and disk space used would be
created a new container, container create times and disk space used would be
significantly increased. This would be similar to the way that virtual machines
work, with one or more virtual disks per virtual machine.
work, with one or more virtual disks per virtual machine. The [`vfs` storage](vfs-driver.md)
does not provide a CoW filesystem or other optimizations. When using this storage
driver, a full copy of the image's data is created for each container.
## Related information

View File

@ -13,90 +13,63 @@ use Docker volumes to write data. However, some workloads require you to be able
to write to the container's writable layer. This is where storage drivers come
in.
Docker supports several different storage drivers, using a pluggable
architecture. The storage driver controls how images and containers are stored
and managed on your Docker host.
After you have read the [storage driver overview](index.md), the
next step is to choose the best storage driver for your workloads. In making
this decision, there are three high-level factors to consider:
If multiple storage drivers are supported in your kernel, Docker has a prioritized
list of which storage driver to use if no storage driver is explicitly configured,
assuming that the storage driver meets the prerequisites.
Use the storage driver with the best overall performance and stability in the most
usual scenarios.
Docker supports several storage drivers, using a pluggable architecture. The
storage driver controls how images and containers are stored and managed on your
Docker host. After you have read the [storage driver overview](index.md), the
next step is to choose the best storage driver for your workloads. Use the storage
driver with the best overall performance and stability in the most usual scenarios.
Docker supports the following storage drivers:
The Docker Engine provides the following storage drivers on Linux:
* `overlay2` is the preferred storage driver, for all currently supported
Linux distributions, and requires no extra configuration.
* `aufs` was the preferred storage driver for Docker 18.06 and older, when
running on Ubuntu 14.04 on kernel 3.13 which had no support for `overlay2`.
* `fuse-overlayfs` is preferred only for running Rootless Docker
on a host that does not provide support for rootless `overlay2`.
On Ubuntu and Debian 10, the `fuse-overlayfs` driver does not need to be
used `overlay2` works even in rootless mode.
See [Rootless mode documentation](../../engine/security/rootless.md).
* `devicemapper` is supported, but requires `direct-lvm` for production
environments, because `loopback-lvm`, while zero-configuration, has very
poor performance. `devicemapper` was the recommended storage driver for
CentOS and RHEL, as their kernel version did not support `overlay2`. However,
current versions of CentOS and RHEL now have support for `overlay2`,
which is now the recommended driver.
* The `btrfs` and `zfs` storage drivers are used if they are the backing
filesystem (the filesystem of the host on which Docker is installed).
These filesystems allow for advanced options, such as creating "snapshots",
but require more maintenance and setup. Each of these relies on the backing
filesystem being configured correctly.
* The `vfs` storage driver is intended for testing purposes, and for situations
where no copy-on-write filesystem can be used. Performance of this storage
driver is poor, and is not generally recommended for production use.
| Driver | Description |
|--------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `overlay2` | `overlay2` is the preferred storage driver for all currently supported Linux distributions, and requires no extra configuration. |
| `fuse-overlayfs` | `fuse-overlayfs`is preferred only for running Rootless Docker on a host that does not provide support for rootless `overlay2`. On Ubuntu and Debian 10, the `fuse-overlayfs` driver does not need to be used, and `overlay2` works even in rootless mode. Refer to the [rootless mode documentation](../../engine/security/rootless.md) for details. |
| `btrfs` and `zfs` | The `btrfs` and `zfs` storage drivers allow for advanced options, such as creating "snapshots", but require more maintenance and setup. Each of these relies on the backing filesystem being configured correctly. |
| `vfs` | The `vfs` storage driver is intended for testing purposes, and for situations where no copy-on-write filesystem can be used. Performance of this storage driver is poor, and is not generally recommended for production use. |
| `aufs` | The `aufs` storage driver Was the preferred storage driver for Docker 18.06 and older, when running on Ubuntu 14.04 on kernel 3.13 which had no support for `overlay2`. However, current versions of Ubuntu and Debian now have support for `overlay2`, which is now the recommended driver. |
| `devicemapper` | The `devicemapper` storage driver requires `direct-lvm` for production environments, because `loopback-lvm`, while zero-configuration, has very poor performance. `devicemapper` was the recommended storage driver for CentOS and RHEL, as their kernel version did not support `overlay2`. However, current versions of CentOS and RHEL now have support for `overlay2`, which is now the recommended driver. |
| `overlay` | The legacy `overlay` driver was used for kernels that did not support the "multiple-lowerdir" feature required for `overlay2` All currently supported Linux distributions now provide support for this, and it is therefore deprecated. |
Docker's source code defines the selection order. You can see the order at
[the source code for Docker Engine {{ site.docker_ce_version }}](https://github.com/moby/moby/blob/{{ site.docker_ce_version }}/daemon/graphdriver/driver_linux.go#L52-L53)
If you run a different version of Docker, you can use the branch selector at the top of the file viewer to choose a different branch.
The Docker Engine has a prioritized list of which storage driver to use if no
storage driver is explicitly configured, assuming that the storage driver meets
the prerequisites, and automatically selects a compatible storage driver. You
can see the order in the [source code for Docker Engine {{ site.docker_ce_version }}](https://github.com/moby/moby/blob/{{ site.docker_ce_version }}/daemon/graphdriver/driver_linux.go#L52-L53).
{: id="storage-driver-order" }
Some storage drivers require you to use a specific format for the backing filesystem. If you have external
requirements to use a specific backing filesystem, this may limit your choices. See [Supported backing filesystems](#supported-backing-filesystems).
Some storage drivers require you to use a specific format for the backing filesystem.
If you have external requirements to use a specific backing filesystem, this may
limit your choices. See [Supported backing filesystems](#supported-backing-filesystems).
After you have narrowed down which storage drivers you can choose from, your choice is determined by the
characteristics of your workload and the level of stability you need. See [Other considerations](#other-considerations)
for help in making the final decision.
> ***NOTE***: Your choice may be limited by your operating system and distribution.
> For instance, `aufs` is only supported on Ubuntu and Debian, and may require extra packages
> to be installed, while `btrfs` is only supported on SLES, which is only supported with Docker
> Enterprise. See [Support storage drivers per Linux distribution](#supported-storage-drivers-per-linux-distribution)
> for more information.
After you have narrowed down which storage drivers you can choose from, your choice
is determined by the characteristics of your workload and the level of stability
you need. See [Other considerations](#other-considerations) for help in making
the final decision.
## Supported storage drivers per Linux distribution
At a high level, the storage drivers you can use is partially determined by
the Docker edition you use.
> Docker Desktop, and Docker in Rootless mode
>
> Modifying the storage-driver is not supported on Docker Desktop for Mac and
> Docker Desktop for Windows, and only the default storage driver can be used.
> The comparison table below is also not applicable for Rootless mode. For the
> drivers available in rootless mode, see the [Rootless mode documentation](../../engine/security/rootless.md).
In addition, Docker does not recommend any configuration that requires you to
disable security features of your operating system, such as the need to disable
`selinux` if you use the `overlay` or `overlay2` driver on CentOS.
Your operating system and kernel may not support every storage driver. For
instance, `aufs` is only supported on Ubuntu and Debian, and may require extra
packages to be installed, while `btrfs` is only supported if your system uses
`btrfs` as storage. In general, the following configurations work on recent
versions of the Linux distribution:
### Docker Engine - Community
For Docker Engine - Community, only some configurations are tested, and your operating
system's kernel may not support every storage driver. In general, the following
configurations work on recent versions of the Linux distribution:
| Linux distribution | Recommended storage drivers | Alternative drivers |
|:--------------------|:-----------------------------------------------------------------------|:--------------------------------------------------|
| Docker Engine - Community on Ubuntu | `overlay2` or `aufs` (for Ubuntu 14.04 running on kernel 3.13) | `overlay`¹, `devicemapper`², `zfs`, `vfs` |
| Docker Engine - Community on Debian | `overlay2` (Debian Stretch), `aufs` or `devicemapper` (older versions) | `overlay`¹, `vfs` |
| Docker Engine - Community on CentOS | `overlay2` | `overlay`¹, `devicemapper`², `zfs`, `vfs` |
| Docker Engine - Community on Fedora | `overlay2` | `overlay`¹, `devicemapper`², `zfs`, `vfs` |
| Docker Engine - Community on SLES 15 | `overlay2` | `overlay`¹, `devicemapper`², `vfs` |
| Docker Engine - Community on RHEL | `overlay2` | `overlay`¹, `devicemapper`², `vfs` |
| Linux distribution | Recommended storage drivers | Alternative drivers |
|:--------------------|:------------------------------|:---------------------------------------------------|
| Ubuntu | `overlay2` | `overlay`¹, `devicemapper`², `aufs`³, `zfs`, `vfs` |
| Debian | `overlay2` | `overlay`¹, `devicemapper`², `aufs`³, `vfs` |
| CentOS | `overlay2` | `overlay`¹, `devicemapper`², `zfs`, `vfs` |
| Fedora | `overlay2` | `overlay`¹, `devicemapper`², `zfs`, `vfs` |
| SLES 15 | `overlay2` | `overlay`¹, `devicemapper`², `vfs` |
| RHEL | `overlay2` | `overlay`¹, `devicemapper`², `vfs` |
¹) The `overlay` storage driver is deprecated, and will be removed in a future
release. It is recommended that users of the `overlay` storage driver migrate to `overlay2`.
@ -105,49 +78,33 @@ release. It is recommended that users of the `overlay` storage driver migrate to
release. It is recommended that users of the `devicemapper` storage driver migrate
to `overlay2`.
> **Note**
>
> The comparison table above is not applicable for Rootless mode.
> For the drivers available in Rootless mode, see [the Rootless mode documentation](../../engine/security/rootless.md).
When possible, `overlay2` is the recommended storage driver. When installing
Docker for the first time, `overlay2` is used by default. Previously, `aufs` was
used by default when available, but this is no longer the case. If you want to
use `aufs` on new installations going forward, you need to explicitly configure
it, and you may need to install extra packages, such as `linux-image-extra`.
See [aufs](aufs-driver.md).
On existing installations using `aufs`, it is still used.
³) The `aufs` storage driver is deprecated, and will be removed in a future
release. It is recommended that users of the `aufs` storage driver migrate
to `overlay2`.
When in doubt, the best all-around configuration is to use a modern Linux
distribution with a kernel that supports the `overlay2` storage driver, and to
use Docker volumes for write-heavy workloads instead of relying on writing data
to the container's writable layer.
The `vfs` storage driver is usually not the best choice. Before using the `vfs`
storage driver, be sure to read about
The `vfs` storage driver is usually not the best choice, and primarily intended
for debugging purposes in situations where no other storage-driver is supported.
Before using the `vfs` storage driver, be sure to read about
[its performance and storage characteristics and limitations](vfs-driver.md).
> **Expectations for non-recommended storage drivers**: Commercial support is
> not available for Docker Engine - Community, and you can technically use any storage driver
> that is available for your platform. For instance, you can use `btrfs` with
> Docker Engine - Community, even though it is not recommended on any platform for
> Docker Engine - Community, and you do so at your own risk.
>
> The recommendations in the table above are based on automated regression
> testing and the configurations that are known to work for a large number of
> users. If you use a recommended configuration and find a reproducible issue,
> it is likely to be fixed very quickly. If the driver that you want to use is
> not recommended according to this table, you can run it at your own risk. You
> can and should still report any issues you run into. However, such issues
> have a lower priority than issues encountered when using a recommended
> configuration.
The recommendations in the table above are known to work for a large number of
users. If you use a recommended configuration and find a reproducible issue,
it is likely to be fixed very quickly. If the driver that you want to use is
not recommended according to this table, you can run it at your own risk. You
can and should still report any issues you run into. However, such issues
have a lower priority than issues encountered when using a recommended
configuration.
### Docker Desktop for Mac and Docker Desktop for Windows
Docker Desktop for Mac and Docker Desktop for Windows are intended for development, rather
than production. Modifying the storage driver on these platforms is not
possible.
Depending on your Linux distribution, other storage-drivers, such as `btrfs` may
be available. These storage drivers can have advantages for specific use-cases,
but may require additional set-up or maintenance, which make them not recommended
for common scenarios. Refer to the documentation for those storage drivers for
details.
## Supported backing filesystems
@ -205,8 +162,8 @@ specific shared storage system.
For some users, stability is more important than performance. Though Docker
considers all of the storage drivers mentioned here to be stable, some are newer
and are still under active development. In general, `overlay2`, `aufs`, `overlay`,
and `devicemapper` are the choices with the highest stability.
and are still under active development. In general, `overlay2`, `aufs`, and
`devicemapper` are the choices with the highest stability.
### Test with your own workloads
@ -223,7 +180,7 @@ set-up steps to use a given storage driver.
To see what storage driver Docker is currently using, use `docker info` and look
for the `Storage Driver` line:
```bash
```console
$ docker info
Containers: 0
@ -237,11 +194,13 @@ To change the storage driver, see the specific instructions for the new storage
driver. Some drivers require additional configuration, including configuration
to physical or logical disks on the Docker host.
> **Important**: When you change the storage driver, any existing images and
> containers become inaccessible. This is because their layers cannot be used
> by the new storage driver. If you revert your changes, you can
> access the old images and containers again, but any that you pulled or
> created using the new driver are then inaccessible.
> **Important**
>
> When you change the storage driver, any existing images and containers become
> inaccessible. This is because their layers cannot be used by the new storage
> driver. If you revert your changes, you can access the old images and containers
> again, but any that you pulled or created using the new driver are then
> inaccessible.
{:.important}
## Related information