docker
/
docs
mirror of https://github.com/docker/docs.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

engine: freshness updates for storage mounts (bind, volume, tmpfs) 

Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
This commit is contained in:
David Karlsson 2024-11-13 14:50:05 +01:00
parent 81998229fb
commit ba7c7c6d44
8 changed files with 420 additions and 417 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

216
content/manuals/engine/storage/_index.md
View File

@ -9,120 +9,65 @@ aliases:
---
By default all files created inside a container are stored on a writable
container layer. This means that:
container layer that sits on top of the read-only, immutable image layers.
- The data doesn't persist when that container no longer exists, and it can be
difficult to get the data out of the container if another process needs it.
- A container's writable layer is tightly coupled to the host machine
where the container is running. You can't easily move the data somewhere else.
- Writing into a container's writable layer requires a
[storage driver](/engine/storage/drivers/) to manage the
filesystem. The storage driver provides a union filesystem, using the Linux
kernel. This extra abstraction reduces performance as compared to using
_data volumes_, which write directly to the host filesystem.
Data written to the container layer doesn't persist when the container is
destroyed. This means that it can be difficult to get the data out of the
container if another process needs it.
Docker has two options for containers to store files on the host machine, so
that the files are persisted even after the container stops: volumes, and
bind mounts.
The writable layer is unique per container. You can't easily extract the data
from the writeable layer to the host, or to another container.
Docker also supports containers storing files in-memory on the host machine. Such files are not persisted.
If you're running Docker on Linux, `tmpfs` mount is used to store files in the host's system memory.
If you're running Docker on Windows, named pipe is used to store files in the host's system memory.
## Storage mount options
## Choose the right type of mount
Docker supports the following types of storage mounts for storing data outside
of the writable layer of the container:
- [Volume mounts](#volume-mounts)
- [Bind mounts](#bind-mounts)
- [tmpfs mounts](#tmpfs-mounts)
- [Named pipes](#named-pipes)
No matter which type of mount you choose to use, the data looks the same from
within the container. It is exposed as either a directory or an individual file
in the container's filesystem.
An easy way to visualize the difference among volumes, bind mounts, and `tmpfs`
mounts is to think about where the data lives on the Docker host.
### Volume mounts
![Types of mounts and where they live on the Docker host](images/types-of-mounts.webp?w=450&h=300)
Volumes are persistent storage mechanisms managed by the Docker daemon. They
retain data even after the containers using them are removed. Volume data is
stored on the filesystem on the host, but in order to interact with the data in
the volume, you must mount the volume to a container. Directly accessing or
interacting with the volume data is unsupported, undefined behavior, and may
result in the volume or its data breaking in unexpected ways.
- Volumes are stored in a part of the host filesystem which is _managed by
Docker_ (`/var/lib/docker/volumes/` on Linux). Non-Docker processes should not
modify this part of the filesystem. Volumes are the best way to persist data
in Docker.
- Bind mounts may be stored anywhere on the host system. They may even be
important system files or directories. Non-Docker processes on the Docker host
or a Docker container can modify them at any time.
- `tmpfs` mounts are stored in the host system's memory only, and are never
written to the host system's filesystem.
Bind mounts and volumes can both be mounted into containers using the `-v` or
`--volume` flag, but the syntax for each is slightly different. For `tmpfs`
mounts, you can use the `--tmpfs` flag. We recommend using the `--mount` flag
for both containers and services, for bind mounts, volumes, or `tmpfs` mounts,
as the syntax is more clear.
### Volumes
Volumes are created and managed by Docker. You can create a volume explicitly
using the `docker volume create` command, or Docker can create a volume during
container or service creation.
When you create a volume, it's stored within a directory on the Docker
host. When you mount the volume into a container, this directory is what's
mounted into the container. This is similar to the way that bind mounts work,
except that volumes are managed by Docker and are isolated from the core
functionality of the host machine.
A given volume can be mounted into multiple containers simultaneously. When no
running container is using a volume, the volume is still available to Docker
and isn't removed automatically. You can remove unused volumes using `docker
volume prune`.
When you mount a volume, it may be named or anonymous. Anonymous volumes are
given a random name that's guaranteed to be unique within a given Docker host.
Just like named volumes, anonymous volumes persist even if you remove the
container that uses them, except if you use the `--rm` flag when creating the
container, in which case the anonymous volume is destroyed.
See [Remove anonymous volumes](volumes.md#remove-anonymous-volumes).
If you create multiple containers after each other that use anonymous volumes,
each container creates its own volume.
Anonymous volumes aren't reused or shared between containers automatically.
To share an anonymous volume between two or more containers,
you must mount the anonymous volume using the random volume ID.
Volumes also support the use of volume drivers, which allow you to store
your data on remote hosts or cloud providers, among other possibilities.
Volumes are ideal for performance-critical data processing and long-term
storage needs. Since the storage location is managed on the daemon host,
volumes provide the same raw file performance as accessing the host filesystem
directly.
### Bind mounts
Bind mounts have limited functionality compared to volumes. When you use a bind
mount, a file or directory on the host machine is mounted into a container. The
file or directory is referenced by its full path on the host machine. The file
or directory doesn't need to exist on the Docker host already. It is created on
demand if it doesn't yet exist. Bind mounts are fast, but they rely on the host
machine's filesystem having a specific directory structure available. If you
are developing new Docker applications, consider using named volumes instead.
You can't use Docker CLI commands to directly manage bind mounts.
Bind mounts create a direct link between a host system path and a container,
allowing access to files or directories stored anywhere on the host. Since they
aren't isolated by Docker, both non-Docker processes on the host and container
processes can modify the mounted files simultaneously.
> [!IMPORTANT]
>
> Bind mounts allow write access to files on the host by default.
>
> One side effect of using bind mounts is that you can change the host
> filesystem via processes running in a container, including creating,
> modifying, or deleting important system files or directories. This is a
> powerful ability which can have security implications, including impacting
> non-Docker processes on the host system.
Use bind mounts when you need to be able to access files from both the
container and the host.
> [!TIP]
>
> Working with large repositories or monorepos, or with virtual file systems that are no longer scaling with your codebase?
> Check out [Synchronized file shares](/manuals/desktop/features/synchronized-file-sharing.md). It provides fast and flexible host-to-VM file sharing by enhancing bind mount performance through the use of synchronized filesystem caches.
### tmpfs mounts
### tmpfs
A tmpfs mount stores files directly in the host machine's memory, ensuring the
data is not written to disk. This storage is ephemeral: the data is lost when
the container is stopped or restarted, or when the host is rebooted. tmpfs
mounts do not persist data either on the Docker host or within the container's
filesystem.
A `tmpfs` mount isn't persisted on disk, either on the Docker host or within a
container. It can be used by a container during the lifetime of the container,
to store non-persistent state or sensitive information. For instance,
internally, Swarm services use `tmpfs` mounts to mount
[secrets](/manuals/engine/swarm/secrets.md) into a service's containers.
These mounts are suitable for scenarios requiring temporary, in-memory storage,
such as caching intermediate data, handling sensitive information like
credentials, or reducing disk I/O. Use tmpfs mounts only when the data does not
need to persist beyond the current container session.
### Named pipes
@ -131,85 +76,6 @@ can be used for communication between the Docker host and a container. Common
use case is to run a third-party tool inside of a container and connect to the
Docker Engine API using a named pipe.
## Good use cases for volumes
Volumes are the preferred way to persist data in Docker containers and services.
Some use cases for volumes include:
- Sharing data among multiple running containers. If you don't explicitly create
it, a volume is created the first time it is mounted into a container. When
that container stops or is removed, the volume still exists. Multiple
containers can mount the same volume simultaneously, either read-write or
read-only. Volumes are only removed when you explicitly remove them.
- When the Docker host is not guaranteed to have a given directory or file
structure. Volumes help you decouple the configuration of the Docker host
from the container runtime.
- When you want to store your container's data on a remote host or a cloud
provider, rather than locally.
- When you need to back up, restore, or migrate data from one Docker
host to another, volumes are a better choice. You can stop containers using
the volume, then back up the volume's directory
(such as `/var/lib/docker/volumes/<volume-name>`).
- When your application requires high-performance I/O on Docker Desktop. Volumes
are stored in the Linux VM rather than the host, which means that the reads and writes
have much lower latency and higher throughput.
- When your application requires fully native file system behavior on Docker
Desktop. For example, a database engine requires precise control over disk
flushing to guarantee transaction durability. Volumes are stored in the Linux
VM and can make these guarantees, whereas bind mounts are remoted to macOS or
Windows, where the file systems behave slightly differently.
## Good use cases for bind mounts
In general, you should use volumes where possible. Bind mounts are appropriate
for the following types of use case:
- Sharing configuration files from the host machine to containers. This is how
Docker provides DNS resolution to containers by default, by mounting
`/etc/resolv.conf` from the host machine into each container.
- Sharing source code or build artifacts between a development environment on
the Docker host and a container. For instance, you may mount a Maven `target/`
directory into a container, and each time you build the Maven project on the
Docker host, the container gets access to the rebuilt artifacts.
If you use Docker for development this way, your production Dockerfile would
copy the production-ready artifacts directly into the image, rather than
relying on a bind mount.
- When the file or directory structure of the Docker host is guaranteed to be
consistent with the bind mounts the containers require.
## Good use cases for tmpfs mounts
`tmpfs` mounts are best used for cases when you do not want the data to persist
either on the host machine or within the container. This may be for security
reasons or to protect the performance of the container when your application
needs to write a large volume of non-persistent state data.
## Tips for using bind mounts or volumes
If you use either bind mounts or volumes, keep the following in mind:
- If you mount an **empty volume** into a directory in the container in which files
or directories exist, these files or directories are propagated (copied)
into the volume. Similarly, if you start a container and specify a volume which
does not already exist, an empty volume is created for you.
This is a good way to pre-populate data that another container needs.
- If you mount a **bind mount or non-empty volume** into a directory in the container
in which some files or directories exist, these files or directories are
obscured by the mount, just as if you saved files into `/mnt` on a Linux host
and then mounted a USB drive into `/mnt`. The contents of `/mnt` would be
obscured by the contents of the USB drive until the USB drive was unmounted.
The obscured files are not removed or altered, but are not accessible while the
bind mount or volume is mounted.
## Next steps
- Learn more about [volumes](./volumes.md).

247
content/manuals/engine/storage/bind-mounts.md
View File

@ -4,91 +4,150 @@ title: Bind mounts
weight: 20
keywords: storage, persistence, data persistence, mounts, bind mounts
aliases:
- /engine/admin/volumes/bind-mounts/
- /storage/bind-mounts/
- /engine/admin/volumes/bind-mounts/
- /storage/bind-mounts/
---
Bind mounts have been around since the early days of Docker. Bind mounts have
limited functionality compared to [volumes](volumes.md). When you use a bind
mount, a file or directory on the host machine is mounted into a container.
The file or directory is referenced by its absolute path on the host
machine. By contrast, when you use a volume, a new directory is created within
Docker's storage directory on the host machine, and Docker manages that
directory's contents.
When you use a bind mount, a file or directory on the host machine is mounted
from the host into a container. By contrast, when you use a volume, a new
directory is created within Docker's storage directory on the host machine, and
Docker manages that directory's contents.
The file or directory does not need to exist on the Docker host already. It is
created on demand if it does not yet exist. Bind mounts are very performant, but
they rely on the host machine's filesystem having a specific directory structure
available. If you are developing new Docker applications, consider using
[named volumes](volumes.md) instead. You can't use Docker CLI commands to directly
manage bind mounts.
## When to use bind mounts
![Bind mounts on the Docker host](images/types-of-mounts-bind.webp?w=450&h=300)
Bind mounts are appropriate for the following types of use case:
> [!TIP]
>
> Working with large repositories or monorepos, or with virtual file systems that are no longer scaling with your codebase?
> Check out [Synchronized file shares](/manuals/desktop/features/synchronized-file-sharing.md). It provides fast and flexible host-to-VM file sharing by enhancing bind mount performance through the use of synchronized filesystem caches.
- Sharing source code or build artifacts between a development environment on
the Docker host and a container.
## Choose the -v or --mount flag
- When you want to create or generate files in a container and persist the
files onto the host's filesystem.
In general, `--mount` is more explicit and verbose. The biggest difference is that
the `-v` syntax combines all the options together in one field, while the `--mount`
syntax separates them. Here is a comparison of the syntax for each flag.
- Sharing configuration files from the host machine to containers. This is how
Docker provides DNS resolution to containers by default, by mounting
`/etc/resolv.conf` from the host machine into each container.
> Tip
>
> New users should use the `--mount` syntax. Experienced users may
> be more familiar with the `-v` or `--volume` syntax, but are encouraged to
> use `--mount`, because research has shown it to be easier to use.
Bind mounts are also available for builds: you can bind mount source code from
the host into the build container to test, lint, or compile a project.
- `-v` or `--volume`: Consists of three fields, separated by colon characters
(`:`). The fields must be in the correct order, and the meaning of each field
is not immediately obvious.
- In the case of bind mounts, the first field is the path to the file or
directory on the **host machine**.
- The second field is the path where the file or directory is mounted in
the container.
- The third field is optional, and is a comma-separated list of options, such
as `ro`, `z`, and `Z`. These options
are discussed below.
## Bind-mounting over existing data
- `--mount`: Consists of multiple key-value pairs, separated by commas and each
consisting of a `<key>=<value>` tuple. The `--mount` syntax is more verbose
than `-v` or `--volume`, but the order of the keys is not significant, and
the value of the flag is easier to understand.
- The `type` of the mount, which can be `bind`, `volume`, or `tmpfs`. This
topic discusses bind mounts, so the type is always `bind`.
- The `source` of the mount. For bind mounts, this is the path to the file
or directory on the Docker daemon host. May be specified as `source` or
`src`.
- The `destination` takes as its value the path where the file or directory
is mounted in the container. May be specified as `destination`, `dst`,
or `target`.
- The `readonly` option, if present, causes the bind mount to be [mounted into
the container as read-only](#use-a-read-only-bind-mount).
- The `bind-propagation` option, if present, changes the
[bind propagation](#configure-bind-propagation). May be one of `rprivate`,
`private`, `rshared`, `shared`, `rslave`, `slave`.
- The `--mount` flag does not support `z` or `Z` options for modifying
selinux labels.
If you bind mount file or directory into a directory in the container in which
files or directories exist, the pre-existing files are obscured by the mount.
This is similar to if you were to save files into `/mnt` on a Linux host, and
then mounted a USB drive into `/mnt`. The contents of `/mnt` would be obscured
by the contents of the USB drive until the USB drive was unmounted.
The examples below show both the `--mount` and `-v` syntax where possible, and
`--mount` is presented first.
With containers, there's no straightforward way of removing a mount to reveal
the obscured files again. Your best option is to recreate the container without
the mount.
### Differences between `-v` and `--mount` behavior
## Considerations and constraints
Because the `-v` and `--volume` flags have been a part of Docker for a long
time, their behavior cannot be changed. This means that there is one behavior
that is different between `-v` and `--mount`.
- Bind mounts have write access to files on the host by default.
If you use `-v` or `--volume` to bind-mount a file or directory that does not
yet exist on the Docker host, `-v` creates the endpoint for you. It is
always created as a directory.
One side effect of using bind mounts is that you can change the host
filesystem via processes running in a container, including creating,
modifying, or deleting important system files or directories. This capability
can have security implications. For example, it may affect non-Docker
processes on the host system.
If you use `--mount` to bind-mount a file or directory that does not
yet exist on the Docker host, Docker does not automatically create it for
you, but generates an error.
You can use the `readonly` or `ro` option to prevent the container from
writing to the mount.
- Bind mounts are created to the Docker daemon host, not the client.
If you're using a remote Docker daemon, you can't create a bind mount to
access files on the client machine in a container.
For Docker Desktop, the daemon runs inside a Linux VM, not directly on the
native host. Docker Desktop has built-in mechanisms that transparently handle
bind mounts, allowing you to share native host filesystem paths with
containers running in the virtual machine.
- Containers with bind mounts are strongly tied to the host.
Bind mounts rely on the host machine's filesystem having a specific directory
structure available. This reliance means that containers with bind mounts may
fail if run on a different host without the same directory structure.
## Syntax
To create a bind mount, you can use either the `--mount` or `--volume` flag.
```console
$ docker run --mount type=bind,src=<host-path>,dst=<container-path>
$ docker run --volume <host-path>:<container-path>
```
In general, `--mount` is preferred. The main difference is that the `--mount`
flag is more explicit and supports all the available options.
If you use `--volume` to bind-mount a file or directory that does not yet
exist on the Docker host, Docker automatically creates the directory on the
host for you. It's always created as a directory.
`--mount` does not automatically create a directory if the specified mount
path does not exist on the host. Instead, it produces an error:
```console
$ docker run --mount type=bind,src=/dev/noexist,dst=/mnt/foo alpine
docker: Error response from daemon: invalid mount config for type "bind": bind source path does not exist: /dev/noexist.
```
### Options for --mount
The `--mount` flag consists of multiple key-value pairs, separated by commas
and each consisting of a `<key>=<value>` tuple. The order of the keys isn't
significant.
```console
$ docker run --mount type=bind,src=<host-path>,dst=<container-path>[,<key>=<value>...]
```
Valid options for `--mount type=bind` include:
| Option | Description |
| ------------------------------ | --------------------------------------------------------------------------------------------------------------- |
| `source`, `src` | The location of the file or directory on the host. This can be an absolute or relative path. |
| `destination`, `dst`, `target` | The path where the file or directory is mounted in the container. Must be an absolute path. |
| `readonly`, `ro` | If present, causes the bind mount to be [mounted into the container as read-only](#use-a-read-only-bind-mount). |
| `bind-propagation` | If present, changes the [bind propagation](#configure-bind-propagation). |
```console {title="Example"}
$ docker run --mount type=bind,src=.,dst=/project,ro,bind-propagation=rshared
```
### Options for --volume
The `--volume` or `-v` flag consists of three fields, separated by colon
characters (`:`). The fields must be in the correct order.
```console
$ docker run -v <host-path>:<container-path>[:opts]
```
The first field is the path on the host to bind mount into the container. The
second field is the path where the file or directory is mounted in the
container.
The third field is optional, and is a comma-separated list of options. Valid
options for `--volume` with a bind mount include:
| Option | Description |
| -------------------- | ------------------------------------------------------------------------------------------------------------------ |
| `readonly`, `ro` | If present, causes the bind mount to be [mounted into the container as read-only](#use-a-read-only-bind-mount). |
| `z`, `Z` | Configures SELinux labeling. See [Configure the SELinux label](#configure-the-selinux-label) |
| `rprivate` (default) | Sets bind propagation to `rprivate` for this mount. See [Configure bind propagation](#configure-bind-propagation). |
| `private` | Sets bind propagation to `private` for this mount. See [Configure bind propagation](#configure-bind-propagation). |
| `rshared` | Sets bind propagation to `rshared` for this mount. See [Configure bind propagation](#configure-bind-propagation). |
| `shared` | Sets bind propagation to `shared` for this mount. See [Configure bind propagation](#configure-bind-propagation). |
| `rslave` | Sets bind propagation to `rslave` for this mount. See [Configure bind propagation](#configure-bind-propagation). |
| `slave` | Sets bind propagation to `slave` for this mount. See [Configure bind propagation](#configure-bind-propagation). |
```console {title="Example"}
$ docker run -v .:/project:ro,rshared
```
## Start a container with a bind mount
@ -102,9 +161,9 @@ directory into your container at `/app/`. Run the command from within the
directory on Linux or macOS hosts.
If you're on Windows, see also [Path conversions on Windows](/manuals/desktop/troubleshoot-and-support/troubleshoot/topics.md).
The `--mount` and `-v` examples below produce the same result. You
can't run them both unless you remove the `devtest` container after running the
first one.
The following `--mount` and `-v` examples produce the same result. You can't
run them both unless you remove the `devtest` container after running the first
one.
{{< tabs >}}
{{< tab name="`--mount`" >}}
@ -151,21 +210,19 @@ This shows that the mount is a `bind` mount, it shows the correct source and
destination, it shows that the mount is read-write, and that the propagation is
set to `rprivate`.
Stop the container:
Stop and remove the container:
```console
$ docker container stop devtest
$ docker container rm devtest
$ docker container rm -fv devtest
```
### Mount into a non-empty directory on the container
If you bind-mount a directory into a non-empty directory on the container, the directory's
existing contents are obscured by the bind mount. This can be beneficial,
such as when you want to test a new version of your application without
building a new image. However, it can also be surprising and this behavior
differs from that of [docker volumes](volumes.md).
If you bind-mount a directory into a non-empty directory on the container, the
directory's existing contents are obscured by the bind mount. This can be
beneficial, such as when you want to test a new version of your application
without building a new image. However, it can also be surprising and this
behavior differs from that of [volumes](volumes.md).
This example is contrived to be extreme, but replaces the contents of the
container's `/usr/` directory with the `/tmp/` directory on the host machine. In
@ -216,7 +273,7 @@ For some development applications, the container needs to
write into the bind mount, so changes are propagated back to the
Docker host. At other times, the container only needs read access.
This example modifies the one above but mounts the directory as a read-only
This example modifies the previous one, but mounts the directory as a read-only
bind mount, by adding `ro` to the (empty by default) list of options, after the
mount point within the container. Where multiple options are present, separate
them by commas.
@ -264,12 +321,10 @@ correctly. Look for the `Mounts` section:
],
```
Stop the container:
Stop and remove the container:
```console
$ docker container stop devtest
$ docker container rm devtest
$ docker container rm -fv devtest
```
## Recursive mounts
@ -290,7 +345,7 @@ be read-only on a kernel version earlier than 5.12, using the
Supported values for the `bind-recursive` option are:
| Value | Description |
|:--------------------|:------------------------------------------------------------------------------------------------------------------|
| :------------------ | :---------------------------------------------------------------------------------------------------------------- |
| `enabled` (default) | Read-only mounts are made recursively read-only if kernel is v5.12 or later. Otherwise, submounts are read-write. |
| `disabled` | Submounts are ignored (not included in the bind mount). |
| `writable` | Submounts are read-write. |
@ -310,12 +365,11 @@ propagation setting has a recursive counterpoint. In the case of recursion,
consider that `/tmp/a` is also mounted as `/foo`. The propagation settings
control whether `/mnt/a` and/or `/tmp/a` would exist.
> [!WARNING]
>
> [!NOTE]
> Mount propagation doesn't work with Docker Desktop.
| Propagation setting | Description |
|:--------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| :------------------ | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `shared` | Sub-mounts of the original mount are exposed to replica mounts, and sub-mounts of replica mounts are also propagated to the original mount. |
| `slave` | similar to a shared mount, but only in one direction. If the original mount exposes a sub-mount, the replica mount can see it. However, if the replica mount exposes a sub-mount, the original mount cannot see it. |
| `private` | The mount is private. Sub-mounts within it are not exposed to replica mounts, and sub-mounts of replica mounts are not exposed to the original mount. |
@ -364,9 +418,9 @@ $ docker run -d \
Now if you create `/app/foo/`, `/app2/foo/` also exists.
## Configure the selinux label
## Configure the SELinux label
If you use `selinux` you can add the `z` or `Z` options to modify the selinux
If you use SELinux, you can add the `z` or `Z` options to modify the SELinux
label of the host file or directory being mounted into the container. This
affects the file or directory on the host machine itself and can have
consequences outside of the scope of Docker.
@ -381,14 +435,14 @@ inoperable and you may need to relabel the host machine files by hand.
> [!IMPORTANT]
>
> When using bind mounts with services, selinux labels
> When using bind mounts with services, SELinux labels
> (`:Z` and `:z`), as well as `:ro` are ignored. See
> [moby/moby #32579](https://github.com/moby/moby/issues/32579) for details.
This example sets the `z` option to specify that multiple containers can share
the bind mount's contents:
It is not possible to modify the selinux label using the `--mount` flag.
It is not possible to modify the SELinux label using the `--mount` flag.
```console
$ docker run -d \
@ -398,8 +452,7 @@ $ docker run -d \
nginx:latest
```
## Use a bind mount with compose
## Use a bind mount with Docker Compose
A single Docker Compose service with a bind mount looks like this:

BIN
content/manuals/engine/storage/images/types-of-mounts-bind.webp
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 13 KiB

BIN
content/manuals/engine/storage/images/types-of-mounts-tmpfs.webp
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 13 KiB

BIN
content/manuals/engine/storage/images/types-of-mounts-volume.webp
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 14 KiB

BIN
content/manuals/engine/storage/images/types-of-mounts.webp
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 13 KiB

120
content/manuals/engine/storage/tmpfs.md
View File

@ -4,61 +4,91 @@ title: tmpfs mounts
weight: 30
keywords: storage, persistence, data persistence, tmpfs
aliases:
- /engine/admin/volumes/tmpfs/
- /storage/tmpfs/
- /engine/admin/volumes/tmpfs/
- /storage/tmpfs/
---
[Volumes](volumes.md) and [bind mounts](bind-mounts.md) let you share files
between the host machine and container so that you can persist data even after
the container is stopped.
If you're running Docker on Linux, you have a third option: `tmpfs` mounts.
When you create a container with a `tmpfs` mount, the container can create
If you're running Docker on Linux, you have a third option: tmpfs mounts.
When you create a container with a tmpfs mount, the container can create
files outside the container's writable layer.
As opposed to volumes and bind mounts, a `tmpfs` mount is temporary, and only
persisted in the host memory. When the container stops, the `tmpfs` mount is
As opposed to volumes and bind mounts, a tmpfs mount is temporary, and only
persisted in the host memory. When the container stops, the tmpfs mount is
removed, and files written there won't be persisted.
![tmpfs on the Docker host](images/types-of-mounts-tmpfs.webp?w=450&h=300)
tmpfs mounts are best used for cases when you do not want the data to persist
either on the host machine or within the container. This may be for security
reasons or to protect the performance of the container when your application
needs to write a large volume of non-persistent state data.
This is useful to temporarily store sensitive files that you don't want to
persist in either the host or the container writable layer.
> [!IMPORTANT]
> tmpfs mounts in Docker map directly to
> [tmpfs](https://en.wikipedia.org/wiki/Tmpfs) in the Linux kernel. As such,
> the temporary data may be written to a swap file, and thereby persisted to
> the filesystem.
## Mounting over existing data
If you create a tmpfs mount into a directory in the container in which files or
directories exist, the pre-existing files are obscured by the mount. This is
similar to if you were to save files into `/mnt` on a Linux host, and then
mounted a USB drive into `/mnt`. The contents of `/mnt` would be obscured by
the contents of the USB drive until the USB drive was unmounted.
With containers, there's no straightforward way of removing a mount to reveal
the obscured files again. Your best option is to recreate the container without
the mount.
## Limitations of tmpfs mounts
* Unlike volumes and bind mounts, you can't share `tmpfs` mounts between
containers.
* This functionality is only available if you're running Docker on Linux.
* Setting permissions on tmpfs may cause them to [reset after container restart](https://github.com/docker/for-linux/issues/138). In some cases [setting the uid/gid](https://github.com/docker/compose/issues/3425#issuecomment-423091370) can serve as a workaround.
- Unlike volumes and bind mounts, you can't share tmpfs mounts between containers.
- This functionality is only available if you're running Docker on Linux.
- Setting permissions on tmpfs may cause them to [reset after container restart](https://github.com/docker/for-linux/issues/138). In some cases [setting the uid/gid](https://github.com/docker/compose/issues/3425#issuecomment-423091370) can serve as a workaround.
## Choose the --tmpfs or --mount flag
## Syntax
In general, `--mount` is more explicit and verbose. The biggest difference is
that the `--tmpfs` flag does not support any configurable options.
To mount a tmpfs with the `docker run` command, you can use either the
`--mount` or `--tmpfs` flag.
- `--tmpfs`: Mounts a `tmpfs` mount without allowing you to specify any
configurable options, and can only be used with standalone containers.
```console
$ docker run --mount type=tmpfs,dst=<mount-path>
$ docker run --tmpfs <mount-path>
```
- `--mount`: Consists of multiple key-value pairs, separated by commas and each
consisting of a `<key>=<value>` tuple. The `--mount` syntax is more verbose
than `--tmpfs`:
- The `type` of the mount, which can be [`bind`](bind-mounts.md), `volume`, or
[`tmpfs`](tmpfs.md). This topic discusses `tmpfs`, so the type is always
`tmpfs`.
- The `destination` takes as its value the path where the `tmpfs` mount
is mounted in the container. May be specified as `destination`, `dst`,
or `target`.
- The `tmpfs-size` and `tmpfs-mode` options. See
[tmpfs options](#specify-tmpfs-options).
In general, `--mount` is preferred. The main difference is that the `--mount`
flag is more explicit and supports all the available options.
The examples below show both the `--mount` and `--tmpfs` syntax where possible,
and `--mount` is presented first.
The `--tmpfs` flag cannot be used with swarm services. You must use `--mount`.
### Differences between `--tmpfs` and `--mount` behavior
### Options for --mount
- The `--tmpfs` flag does not allow you to specify any configurable options.
- The `--tmpfs` flag cannot be used with swarm services. You must use `--mount`.
The `--mount` flag consists of multiple key-value pairs, separated by commas
and each consisting of a `<key>=<value>` tuple. The order of the keys isn't
significant.
```console
$ docker run --mount type=tmpfs,dst=<mount-path>[,<key>=<value>...]
```
Valid options for `--mount type=tmpfs` include:
| Option | Description |
| :----------------------------- | :--------------------------------------------------------------------------------------------------------------------- |
| `destination`, `dst`, `target` | Size of the tmpfs mount in bytes. If unset, the default maximum size of a tmpfs volume is 50% of the host's total RAM. |
| `tmpfs-size` | Size of the tmpfs mount in bytes. If unset, the default maximum size of a tmpfs volume is 50% of the host's total RAM. |
| `tmpfs-mode` | File mode of the tmpfs in octal. For instance, `700` or `0770`. Defaults to `1777` or world-writable. |
```console {title="Example"}
$ docker run --mount type=tmpfs,dst=/app,tmpfs-size=21474836480,tmpfs-mode=1770
```
### Options for --tmpfs
The `--tmpfs` flag does not let you specify any options.
## Use a tmpfs mount in a container
@ -108,28 +138,6 @@ $ docker stop tmptest
$ docker rm tmptest
```
### Specify tmpfs options
`tmpfs` mounts allow for two configuration options, neither of which is
required. If you need to specify these options, you must use the `--mount` flag,
as the `--tmpfs` flag does not support them.
| Option | Description |
|:-------------|:--------------------------------------------------------------------------------------------------------------------------|
| `tmpfs-size` | Size of the tmpfs mount in bytes. If unset, the default maximum size of a tmpfs volume is 50% of the host's total RAM. |
| `tmpfs-mode` | File mode of the tmpfs in octal. For instance, `700` or `0770`. Defaults to `1777` or world-writable. |
The following example sets the `tmpfs-mode` to `1770`, so that it is not
world-readable within the container.
```console
docker run -d \
-it \
--name tmptest \
--mount type=tmpfs,destination=/app,tmpfs-mode=1770 \
nginx:latest
```
## Next steps
- Learn about [volumes](volumes.md)

254
content/manuals/engine/storage/volumes.md
View File

@ -1,38 +1,55 @@
---
description: Learn how to create, manage, and use volumes instead of bind mounts for
description:
Learn how to create, manage, and use volumes instead of bind mounts for
persisting data generated and used by Docker.
title: Volumes
weight: 10
keywords: docker compose volumes, docker volumes, docker compose volume, docker volume
keywords:
docker compose volumes, docker volumes, docker compose volume, docker volume
mount, docker mount volume, docker volume create, docker volume location
aliases:
- /userguide/dockervolumes/
- /engine/tutorials/dockervolumes/
- /engine/userguide/dockervolumes/
- /engine/admin/volumes/volumes/
- /storage/volumes/
- /userguide/dockervolumes/
- /engine/tutorials/dockervolumes/
- /engine/userguide/dockervolumes/
- /engine/admin/volumes/volumes/
- /storage/volumes/
---
Volumes are persistent data stores for containers, created and managed by
Docker. You can create a volume explicitly using the `docker volume create`
command, or Docker can create a volume during container or service creation.
When you create a volume, it's stored within a directory on the Docker
host. When you mount the volume into a container, this directory is what's
mounted into the container. This is similar to the way that bind mounts work,
except that volumes are managed by Docker and are isolated from the core
functionality of the host machine.
## When to use volumes
Volumes are the preferred mechanism for persisting data generated by and used
by Docker containers. While [bind mounts](bind-mounts.md) are dependent on the
directory structure and OS of the host machine, volumes are completely managed by
Docker. Volumes have several advantages over bind mounts:
Docker. Volumes are a good choice for the following use cases:
- Volumes are easier to back up or migrate than bind mounts.
- You can manage volumes using Docker CLI commands or the Docker API.
- Volumes work on both Linux and Windows containers.
- Volumes can be more safely shared among multiple containers.
- Volume drivers let you store volumes on remote hosts or cloud providers, encrypt the contents of volumes, or add other functionality.
- New volumes can have their content pre-populated by a container.
- Volumes on Docker Desktop have much higher performance than bind mounts from
Mac and Windows hosts.
- New volumes can have their content pre-populated by a container or build.
- When your application requires high-performance I/O.
In addition, volumes are often a better choice than persisting data in a
container's writable layer, because a volume doesn't increase the size of the
containers using it, and the volume's contents exist outside the lifecycle of a
given container.
Volumes are not a good choice if you need to access the files from the host, as
the volume is completely managed by Docker. Use [bind mounts](bind-mounts.md)
if you need to access files or directories from both containers and the host.
![Volumes on the Docker host](images/types-of-mounts-volume.webp?w=450&h=300)
Volumes are often a better choice than writing data directly to a container,
because a volume doesn't increase the size of the containers using it. Using a
volume is also faster; writing into a container's writable layer requires a
[storage driver](/manuals/engine/storage/drivers/_index.md) to manage the
filesystem. The storage driver provides a union filesystem, using the Linux
kernel. This extra abstraction reduces performance as compared to using
volumes, which write directly to the host filesystem.
If your container generates non-persistent state data, consider using a
[tmpfs mount](tmpfs.md) to avoid storing the data anywhere permanently, and to
@ -42,74 +59,122 @@ writable layer.
Volumes use `rprivate` bind propagation, and bind propagation isn't
configurable for volumes.
## Choose the -v or --mount flag
## A volume's lifecycle
In general, `--mount` is more explicit and verbose. The biggest difference is that
the `-v` syntax combines all the options together in one field, while the `--mount`
syntax separates them. Here is a comparison of the syntax for each flag.
A volume's contents exist outside the lifecycle of a given container. When a
container is destroyed, the writable layer is destroyed with it. Using a volume
ensures that the data is persisted even if the container using it is removed.
If you need to specify volume driver options, you must use `--mount`.
A given volume can be mounted into multiple containers simultaneously. When no
running container is using a volume, the volume is still available to Docker
and isn't removed automatically. You can remove unused volumes using `docker
volume prune`.
- `-v` or `--volume`: Consists of three fields, separated by colon characters
(`:`). The fields must be in the correct order, and the meaning of each field
isn't immediately obvious.
## Mounting a volume over existing data
- In the case of named volumes, the first field is the name of the volume, and is
unique on a given host machine. For anonymous volumes, the first field is
omitted.
- The second field is the path where the file or directory is mounted in
the container.
- The third field is optional, and is a comma-separated list of options, such
as `ro`. These options are discussed below.
If you mount a _non-empty volume_ into a directory in the container in which
files or directories exist, the pre-existing files are obscured by the mount.
This is similar to if you were to save files into `/mnt` on a Linux host, and
then mounted a USB drive into `/mnt`. The contents of `/mnt` would be obscured
by the contents of the USB drive until the USB drive was unmounted.
- `--mount`: Consists of multiple key-value pairs, separated by commas and each
consisting of a `<key>=<value>` tuple. The `--mount` syntax is more verbose
than `-v` or `--volume`, but the order of the keys isn't significant, and
the value of the flag is easier to understand.
- The `type` of the mount, which can be [`bind`](bind-mounts.md), `volume`, or
[`tmpfs`](tmpfs.md). This topic discusses volumes, so the type is always
`volume`.
- The `source` of the mount. For named volumes, this is the name of the volume.
For anonymous volumes, this field is omitted. Can be specified as `source`
or `src`.
- The `destination` takes as its value the path where the file or directory
is mounted in the container. Can be specified as `destination`, `dst`,
or `target`.
- The `volume-subpath` option takes a path to a subdirectory within the
volume to mount into the container. The subdirectory must exist in the
volume before the volume is mounted to a container.
See [Mount a volume subdirectory](#mount-a-volume-subdirectory).
- The `readonly` option, if present, causes the bind mount to be [mounted into
the container as read-only](#use-a-read-only-volume). Can be specified as `readonly` or `ro`.
- The `volume-opt` option, which can be specified more than once, takes a
key-value pair consisting of the option name and its value.
With containers, there's no straightforward way of removing a mount to reveal
the obscured files again. Your best option is to recreate the container without
the mount.
> [!WARNING]
>
> If your volume driver accepts a comma-separated list as an option,
> you must escape the value from the outer CSV parser. To escape a `volume-opt`,
> surround it with double quotes (`"`) and surround the entire mount parameter
> with single quotes (`'`).
>
> For example, the `local` driver accepts mount options as a comma-separated
> list in the `o` parameter. This example shows the correct way to escape the list.
>
> ```console
> $ docker service create \
> --mount 'type=volume,src=<VOLUME-NAME>,dst=<CONTAINER-PATH>,volume-driver=local,volume-opt=type=nfs,volume-opt=device=<nfs-server>:<nfs-path>,"volume-opt=o=addr=<nfs-address>,vers=4,soft,timeo=180,bg,tcp,rw"'
> --name myservice \
> <IMAGE>
> ```
If you mount an _empty volume_ into a directory in the container in which files
or directories exist, these files or directories are propagated (copied) into
the volume by default. Similarly, if you start a container and specify a volume
which does not already exist, an empty volume is created for you. This is a
good way to pre-populate data that another container needs.
The examples below show both the `--mount` and `-v` syntax where possible, with
`--mount` first.
To prevent Docker from copying a container's pre-existing files into an empty
volume, use the `volume-nocopy` option, see [Options for --mount](#options-for---mount).
### Differences between `-v` and `--mount` behavior
## Named and anonymous volumes
As opposed to bind mounts, all options for volumes are available for both
`--mount` and `-v` flags.
A volume may be named or anonymous. Anonymous volumes are given a random name
that's guaranteed to be unique within a given Docker host. Just like named
volumes, anonymous volumes persist even if you remove the container that uses
them, except if you use the `--rm` flag when creating the container, in which
case the anonymous volume associated with the container is destroyed. See
[Remove anonymous volumes](volumes.md#remove-anonymous-volumes).
Volumes used with services, only support `--mount`.
If you create multiple containers consecutively that each use anonymous
volumes, each container creates its own volume. Anonymous volumes aren't reused
or shared between containers automatically. To share an anonymous volume
between two or more containers, you must mount the anonymous volume using the
random volume ID.
## Syntax
To mount a volume with the `docker run` command, you can use either the
`--mount` or `--volume` flag.
```console
$ docker run --mount type=volume,src=<volume-name>,dst=<mount-path>
$ docker run --volume <volume-name>:<mount-path>
```
In general, `--mount` is preferred. The main difference is that the `--mount`
flag is more explicit and supports all the available options.
You must use `--mount` if you want to:
- Specify [volume driver options](#use-a-volume-driver)
- Mount a [volume subdirectory](#mount-a-volume-subdirectory)
- Mount a volume into a Swarm service
### Options for --mount
The `--mount` flag consists of multiple key-value pairs, separated by commas
and each consisting of a `<key>=<value>` tuple. The order of the keys isn't
significant.
```console
$ docker run --mount type=volume[,src=<volume-name>],dst=<mount-path>[,<key>=<value>...]
```
Valid options for `--mount type=volume` include:
| Option | Description |
| ------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `source`, `src` | The source of the mount. For named volumes, this is the name of the volume. For anonymous volumes, this field is omitted. |
| `destination`, `dst`, `target` | The path where the file or directory is mounted in the container. |
| `volume-subpath` | A path to a subdirectory within the volume to mount into the container. The subdirectory must exist in the volume before the volume is mounted to a container. See [Mount a volume subdirectory](#mount-a-volume-subdirectory). |
| `readonly`, `ro` | If present, causes the volume to be [mounted into the container as read-only](#use-a-read-only-volume). |
| `volume-nocopy` | If present, data at the destination isn't copied into the volume if the volume is empty. By default, content at the target destination gets copied into a mounted volume if empty. |
| `volume-opt` | Can be specified more than once, takes a key-value pair consisting of the option name and its value. |
```console {title="Example"}
$ docker run --mount type=volume,src=myvolume,dst=/data,ro,volume-subpath=/foo
```
### Options for --volume
The `--volume` or `-v` flag consists of three fields, separated by colon
characters (`:`). The fields must be in the correct order.
```console
$ docker run -v [<volume-name>:]<mount-path>[:opts]
```
In the case of named volumes, the first field is the name of the volume, and is
unique on a given host machine. For anonymous volumes, the first field is
omitted. The second field is the path where the file or directory is mounted in
the container.
The third field is optional, and is a comma-separated list of options. Valid
options for `--volume` with a data volume include:
| Option | Description |
| ---------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `readonly`, `ro` | If present, causes the volume to be [mounted into the container as read-only](#use-a-read-only-volume). |
| `volume-nocopy` | If present, data at the destination isn't copied into the volume if the volume is empty. By default, content at the target destination gets copied into a mounted volume if empty. |
```console {title="Example"}
$ docker run -v myvolume:/data:ro
```
## Create and manage volumes
@ -158,8 +223,8 @@ If you start a container with a volume that doesn't yet exist, Docker creates
the volume for you. The following example mounts the volume `myvol2` into
`/app/` in the container.
The `-v` and `--mount` examples below produce the same result. You can't run
them both unless you remove the `devtest` container and the `myvol2` volume
The following `-v` and `--mount` examples produce the same result. You can't
run them both unless you remove the `devtest` container and the `myvol2` volume
after running the first one.
{{< tabs >}}
@ -219,7 +284,7 @@ $ docker volume rm myvol2
## Use a volume with Docker Compose
The example below shows a single Docker Compose service with a volume:
The following example shows a single Docker Compose service with a volume:
```yaml
services:
@ -286,12 +351,6 @@ $ docker service rm devtest-service
Removing the service doesn't remove any volumes created by the service.
Volume removal is a separate step.
#### Syntax differences for services
The `docker service create` command doesn't support the `-v` or `--volume` flag.
When mounting a volume into a service's containers, you must use the `--mount`
flag.
### Populate a volume using a container
If you start a container which creates a new volume, and the container
@ -349,7 +408,7 @@ the container only needs read access to the data. Multiple
containers can mount the same volume. You can simultaneously mount a
single volume as `read-write` for some containers and as `read-only` for others.
The following example changes the one above. It mounts the directory as a read-only
The following example changes the previous one. It mounts the directory as a read-only
volume, by adding `ro` to the (empty by default) list of options, after the
mount point within the container. Where multiple options are present, you can separate
them using commas.
@ -457,7 +516,7 @@ One is to add logic to your application to store files on a cloud object
storage system like Amazon S3. Another is to create volumes with a driver that
supports writing files to an external storage system like NFS or Amazon S3.
Volume drivers allow you to abstract the underlying storage system from the
Volume drivers let you abstract the underlying storage system from the
application logic. For example, if your services use a volume with an NFS
driver, you can update the services to use a different driver. For example, to
store data in the cloud, without changing the application logic.
@ -470,6 +529,23 @@ The following examples use the `vieux/sshfs` volume driver, first when creating
a standalone volume, and then when starting a container which creates a new
volume.
> [!NOTE]
>
> If your volume driver accepts a comma-separated list as an option,
> you must escape the value from the outer CSV parser. To escape a `volume-opt`,
> surround it with double quotes (`"`) and surround the entire mount parameter
> with single quotes (`'`).
>
> For example, the `local` driver accepts mount options as a comma-separated
> list in the `o` parameter. This example shows the correct way to escape the list.
>
> ```console
> $ docker service create \
> --mount 'type=volume,src=<VOLUME-NAME>,dst=<CONTAINER-PATH>,volume-driver=local,volume-opt=type=nfs,volume-opt=device=<nfs-server>:<nfs-path>,"volume-opt=o=addr=<nfs-address>,vers=4,soft,timeo=180,bg,tcp,rw"'
> --name myservice \
> <IMAGE>
> ```
### Initial setup
The following example assumes that you have two nodes, the first of which is a Docker
@ -579,7 +655,7 @@ $ mount -t <mount.volume-opt.type> <mount.volume-opt.device> <mount.dst> -o <mou
```
To explain this further, consider the following `mount` command example.
This command mounts the `/dev/loop5` device to the path `/external-drive` on the system.
This command mounts the `/dev/loop5` device to the path `/external-drive` on the system.
```console
$ mount -t ext4 /dev/loop5 /external-drive
@ -662,7 +738,7 @@ In the next command:
- Launch a new container and mount the volume from the `dbstore` container
- Mount a local host directory as `/backup`
- Pass a command that tars the contents of the `dbdata` volume to a `backup.tar` file inside our `/backup` directory.
- Pass a command that tars the contents of the `dbdata` volume to a `backup.tar` file inside the `/backup` directory.
```console
$ docker run --rm --volumes-from dbstore -v $(pwd):/backup ubuntu tar cvf /backup/backup.tar /dbdata
@ -688,7 +764,7 @@ Then, un-tar the backup file in the new container’s data volume:
$ docker run --rm --volumes-from dbstore2 -v $(pwd):/backup ubuntu bash -c "cd /dbdata && tar xvf /backup/backup.tar --strip 1"
```
You can use the techniques above to automate backup, migration, and restore
You can use these techniques to automate backup, migration, and restore
testing using your preferred tools.
## Remove volumes