From ba8ae8535f9be85325de3999bdec31fc4eda682c Mon Sep 17 00:00:00 2001 From: CrazyMax Date: Fri, 4 Nov 2022 10:39:02 +0100 Subject: [PATCH] build: context page Signed-off-by: CrazyMax --- _data/toc.yaml | 2 + _includes/gha-tutorial.md | 2 +- build/building/cache/index.md | 8 +- build/building/context.md | 206 ++++++++++++++++++ build/building/packaging.md | 28 +-- build/buildkit/index.md | 4 +- compose/compose-file/build.md | 2 + develop/develop-images/baseimages.md | 4 +- .../dockerfile_best-practices.md | 40 +--- language/dotnet/build-images.md | 9 +- language/java/build-images.md | 6 +- language/nodejs/build-images.md | 6 +- samples/dotnetcore.md | 12 +- 13 files changed, 248 insertions(+), 81 deletions(-) create mode 100644 build/building/context.md diff --git a/_data/toc.yaml b/_data/toc.yaml index 4a10304a1e..fa8d679d01 100644 --- a/_data/toc.yaml +++ b/_data/toc.yaml @@ -1505,6 +1505,8 @@ manuals: section: - path: /build/building/packaging/ title: Packaging your software + - path: /build/building/context/ + title: Build context - sectiontitle: Build drivers section: - path: /build/building/drivers/ diff --git a/_includes/gha-tutorial.md b/_includes/gha-tutorial.md index 3f51c7ba06..c401d77cad 100644 --- a/_includes/gha-tutorial.md +++ b/_includes/gha-tutorial.md @@ -114,7 +114,7 @@ The previous YAML snippet contains a sequence of steps that: The `with` key lists a number of input parameters that configures the step: - - `context`: the build context. + - `context`: the [build context](/build/building/context/). - `file`: filepath to the Dockerfile. - `push`: tells the action to upload the image to a registry after building it. diff --git a/build/building/cache/index.md b/build/building/cache/index.md index f6d063f99c..608f203fd3 100644 --- a/build/building/cache/index.md +++ b/build/building/cache/index.md @@ -126,10 +126,10 @@ To get started, here are a few tips and tricks: Be considerate of what files you add to the image. -Running a command like `COPY . /src` will `COPY` your entire build context into -the image. If you've got logs, package manager artifacts, or even previous build -results in your current directory, those will also be copied over. This could -make your image larger than it needs to be, especially as those files are +Running a command like `COPY . /src` will `COPY` your entire [build context](../context.md) +into the image. If you've got logs, package manager artifacts, or even previous +build results in your current directory, those will also be copied over. This +could make your image larger than it needs to be, especially as those files are usually not useful. Avoid adding unnecessary files to your builds by explicitly stating the files or diff --git a/build/building/context.md b/build/building/context.md new file mode 100644 index 0000000000..58ad1f0b34 --- /dev/null +++ b/build/building/context.md @@ -0,0 +1,206 @@ +--- +title: Build context +description: Learn how to use the build context to access files from your Dockerfile +keywords: build, buildx, buildkit, context, git, tarball, stdin +--- + +The [`docker build`](../../engine/reference/commandline/build.md) or +[`docker buildx build`](../../engine/reference/commandline/buildx_build.md) +commands build Docker images from a [Dockerfile](../../engine/reference/builder.md) +and a "context". + +A build's context is the set of files located at the `PATH` or `URL` specified +as the positional argument to the build command: + +```console +$ docker build [OPTIONS] PATH | URL | - + ^^^^^^^^^^^^^^ +``` + +The build process can refer to any of the files in the context. For example, +your build can use a [`COPY` instruction](../../engine/reference/builder.md#copy) +to reference a file in the context or a [`RUN --mount=type=bind` instruction](../../engine/reference/builder.md#run---mounttypebind) +for better performance with [BuildKit](../buildkit/index.md). The build context +is processed recursively. So, a `PATH` includes any subdirectories and the +`URL` includes the repository and its submodules. + +## `PATH` context + +This example shows a build command that uses the current directory (`.`) as a +build context: + +```console +$ docker build . +... +#16 [internal] load build context +#16 sha256:23ca2f94460dcbaf5b3c3edbaaa933281a4e0ea3d92fe295193e4df44dc68f85 +#16 transferring context: 13.16MB 2.2s done +... +``` + +With the following Dockerfile: + +```dockerfile +FROM busybox +WORKDIR /src +COPY foo . +``` + +And this directory structure: + +``` +. +├── Dockerfile +├── bar +├── foo +└── node_modules +``` + +The legacy builder sends the entire directory to the daemon, including `bar` +and `node_modules` directories, even though the `Dockerfile` does not use +them. When using [BuildKit](../buildkit/index.md), the client only sends the +files required by the `COPY` instructions, in this case `foo`. + +In some cases you may want to send the entire context: + +```dockerfile +FROM busybox +WORKDIR /src +COPY . . +``` + +You can use a [`.dockerignore`](../../engine/reference/builder.md#dockerignore-file) +file to exclude some files or directories from being sent: + +```gitignore +# .dockerignore +node_modules +bar +``` + +> **Warning** +> +> Avoid using your root directory, `/`, as the `PATH` for your build context, +> as it causes the build to transfer the entire contents of your hard drive to +> the daemon. +{:.warning} + +## `URL` context + +The `URL` parameter can refer to three kinds of resources: +* [Git repositories](#git-repositories) +* Pre-packaged [tarball contexts](#tarball-contexts) +* Plain [text files](#text-files) + +### Git repositories + +When the `URL` parameter points to the location of a Git repository, the +repository acts as the build context. The builder recursively pulls the +repository and its submodules. A shallow clone is performed and therefore pulls +down just the latest commits, not the entire history. A repository is first +pulled into a temporary directory on your host. After that succeeds, the +directory is sent to the daemon as the context. Local copy gives you the ability +to access private repositories using local user credentials, VPN's, and so forth. + +> **Note** +> +> If the `URL` parameter contains a fragment the system will recursively clone +> the repository and its submodules using a `git clone --recursive` command. + +Git URLs accept a context configuration parameter in the form of a URL fragment, +separated by a colon (`:`). The first part represents the reference that Git +will check out, and can be either a branch, a tag, or a remote reference. The +second part represents a subdirectory inside the repository that will be used +as a build context. + +For example, run this command to use a directory called `docker` in the branch +`container`: + +```console +$ docker build https://github.com/user/myrepo.git#container:docker +``` + +The following table represents all the valid suffixes with their build +contexts: + +| Build Syntax Suffix | Commit Used | Build Context Used | +|--------------------------------|-----------------------|--------------------| +| `myrepo.git` | `refs/heads/master` | `/` | +| `myrepo.git#mytag` | `refs/tags/mytag` | `/` | +| `myrepo.git#mybranch` | `refs/heads/mybranch` | `/` | +| `myrepo.git#pull/42/head` | `refs/pull/42/head` | `/` | +| `myrepo.git#:myfolder` | `refs/heads/master` | `/myfolder` | +| `myrepo.git#master:myfolder` | `refs/heads/master` | `/myfolder` | +| `myrepo.git#mytag:myfolder` | `refs/tags/mytag` | `/myfolder` | +| `myrepo.git#mybranch:myfolder` | `refs/heads/mybranch` | `/myfolder` | + +By default `.git` directory is not kept on Git checkouts. You can set the +[BuildKit built-in arg `BUILDKIT_CONTEXT_KEEP_GIT_DIR=1`](../../engine/reference/builder.md#buildkit-built-in-build-args) +to keep it. It can be useful to keep it around if you want to retrieve Git +information during your build: + +```dockerfile +# syntax=docker/dockerfile:1 +FROM alpine +WORKDIR /src +RUN --mount=target=. \ + make REVISION=$(git rev-parse HEAD) build +``` + +```console +$ docker build --build-arg BUILDKIT_CONTEXT_KEEP_GIT_DIR=1 https://github.com/user/myrepo.git#main +``` + +### Tarball contexts + +If you pass a `URL` to a remote tarball, the `URL` itself is sent to the daemon: + +```console +$ docker build http://server/context.tar.gz +#1 [internal] load remote build context +#1 DONE 0.2s + +#2 copy /context / +#2 DONE 0.1s +... +``` + +The download operation will be performed on the host the daemon is running on, +which is not necessarily the same host from which the build command is being +issued. The daemon will fetch `context.tar.gz` and use it as the build context. +Tarball contexts must be tar archives conforming to the standard `tar` UNIX +format and can be compressed with any one of the `xz`, `bzip2`, `gzip` or +`identity` (no compression) formats. + +### Text files + +Instead of specifying a context, you can pass a single `Dockerfile` in the +`URL` or pipe the file in via `STDIN`. To pipe a `Dockerfile` from `STDIN`: + +```console +$ docker build - < Dockerfile +``` + +With Powershell on Windows, you can run: + +```powershell +Get-Content Dockerfile | docker build - +``` + +If you use `STDIN` or specify a `URL` pointing to a plain text file, the system +places the contents into a file called `Dockerfile`, and any `-f`, `--file` +option is ignored. In this scenario, there is no context. + +The following example builds an image using a `Dockerfile` that is passed +through stdin. No files are sent as build context to the daemon. + +```bash +docker build -t myimage:latest -< **Warning** -> -> Avoid using your root directory, `/`, as the `PATH` for your build context, -> as it causes the build to transfer the entire contents of your hard drive to -> the daemon. -{:.warning} - -So, in accordance with the build command issued and how build context works, -your Dockerfile and python app need to be in the same directory. +So, in accordance with the build command issued and how [build context](context.md) +works, your Dockerfile and python app need to be in the same directory. Now run your newly built image: diff --git a/build/buildkit/index.md b/build/buildkit/index.md index f4cf766554..097dcdb88c 100644 --- a/build/buildkit/index.md +++ b/build/buildkit/index.md @@ -14,8 +14,8 @@ complex scenarios: - Detect and skip executing unused build stages - Parallelize building independent build stages -- Incrementally transfer only the changed files in your build context between builds -- Detect and skip transferring unused files in your build context +- Incrementally transfer only the changed files in your [build context](../building/context.md) between builds +- Detect and skip transferring unused files in your [build context](../building/context.md) - Use [Dockerfile frontend](dockerfile-frontend.md) implementations with many new features - Avoid side effects with rest of the API (intermediate images and containers) - Prioritize your build cache for automatic pruning diff --git a/compose/compose-file/build.md b/compose/compose-file/build.md index 5cbcb3f7a6..03b4ea34cc 100644 --- a/compose/compose-file/build.md +++ b/compose/compose-file/build.md @@ -106,6 +106,8 @@ build: context: ./dir ``` +See [Build context](../../build/building/context.md) page for more information. + ### dockerfile `dockerfile` allows to set an alternate Dockerfile. A relative path MUST be resolved from the build context. diff --git a/develop/develop-images/baseimages.md b/develop/develop-images/baseimages.md index 64053044a5..8d3395e2a8 100644 --- a/develop/develop-images/baseimages.md +++ b/develop/develop-images/baseimages.md @@ -78,8 +78,8 @@ image using this `docker build` command: $ docker build --tag hello . ``` -Don't forget the `.` character at the end, which sets the build context to the -current directory. +Don't forget the `.` character at the end, which sets the [build context](../../build/building/context.md) +to the current directory. > **Note**: Because Docker Desktop for Mac and Docker Desktop for Windows use a Linux VM, > you need a Linux binary, rather than a Mac or Windows binary. diff --git a/develop/develop-images/dockerfile_best-practices.md b/develop/develop-images/dockerfile_best-practices.md index 569d6f9ba1..b584cdc026 100644 --- a/develop/develop-images/dockerfile_best-practices.md +++ b/develop/develop-images/dockerfile_best-practices.md @@ -60,45 +60,7 @@ stateless fashion. ### Understand build context -When you issue a `docker build` command, the current working directory is called -the _build context_. By default, the Dockerfile is assumed to be located here, -but you can specify a different location with the file flag (`-f`). Regardless -of where the `Dockerfile` actually lives, all recursive contents of files and -directories in the current directory are sent to the Docker daemon as the build -context. - -> Build context example -> -> Create a directory for the build context and `cd` into it. Write "hello" into -> a text file named `hello` and create a Dockerfile that runs `cat` on it. Build -> the image from within the build context (`.`): -> -> ```console -> $ mkdir myproject && cd myproject -> $ echo "hello" > hello -> $ echo -e "FROM busybox\nCOPY /hello /\nRUN cat /hello" > Dockerfile -> $ docker build -t helloapp:v1 . -> ``` -> -> Move `Dockerfile` and `hello` into separate directories and build a second -> version of the image (without relying on cache from the last build). Use `-f` -> to point to the Dockerfile and specify the directory of the build context: -> -> ```console -> $ mkdir -p dockerfiles context -> $ mv Dockerfile dockerfiles && mv hello context -> $ docker build --no-cache -t helloapp:v2 -f dockerfiles/Dockerfile context -> ``` - -Inadvertently including files that are not necessary for building an image -results in a larger build context and larger image size. This can increase the -time to build the image, time to pull and push it, and the container runtime -size. To see how big your build context is, look for a message like this when -building your `Dockerfile`: - -```none -Sending build context to Docker daemon 187.8MB -``` +See [Build context](../../build/building/context.md) page for more information. ### Pipe Dockerfile through `stdin` diff --git a/language/dotnet/build-images.md b/language/dotnet/build-images.md index eb7fe0d631..ced7cd4c55 100644 --- a/language/dotnet/build-images.md +++ b/language/dotnet/build-images.md @@ -197,18 +197,15 @@ ENTRYPOINT ["dotnet", "myWebApp.dll"] ## .dockerignore file -To make your build context as small as possible, add a .dockerignore file to your `dotnet-docker` folder and copy the following into it. +To make your [build context](../../build/building/context.md) as small as +possible, add a [`.dockerignore` file](../../engine/reference/builder.md#dockerignore-file) +to your `dotnet-docker` folder and copy the following into it. ```shell **/bin/ **/obj/ ``` -> **Note** -> -> To learn more about .dockerignore, see [.dockerignore file](../../engine/reference/builder.md/#dockerignore-file). - - ### Directory structure Just to recap, we created a directory in our local machine called `dotnet-docker` and created a simple .NET application in the `src` folder. We also created a Dockerfile containing the commands to build an image as well as a .dockerignore file. The `dotnet-docker` directory structure should now look like: diff --git a/language/java/build-images.md b/language/java/build-images.md index 94a0c86d04..de9dbee04a 100644 --- a/language/java/build-images.md +++ b/language/java/build-images.md @@ -156,8 +156,10 @@ To increase the performance of the build, and as a general best practice, we rec target ``` -This line excludes the `target` directory, which contains output from Maven, from the Docker build context. -There are many good reasons to carefully structure a `.dockerignore` file, but this one-line file is good enough for now. +This line excludes the `target` directory, which contains output from Maven, +from the Docker [build context](../../build/building/context.md). There are many +good reasons to carefully structure a `.dockerignore` file, but this one-line +file is good enough for now. ## Build an image diff --git a/language/nodejs/build-images.md b/language/nodejs/build-images.md index f181121e94..0e20f81df1 100644 --- a/language/nodejs/build-images.md +++ b/language/nodejs/build-images.md @@ -182,7 +182,11 @@ CMD [ "node", "server.js" ] ## Create a .dockerignore file -To use a file in the build context, the Dockerfile refers to the file specified in an instruction, for example, a COPY instruction. To increase the build’s performance, exclude files and directories by adding a .dockerignore file to the context directory. To improve the context load time create a `.dockerignore` file and add `node_modules` directory in it. +To use a file in the [build context](../../build/building/context.md), the +Dockerfile refers to the file specified in an instruction, for example, a +COPY instruction. A `.dockerignore` file lets you specify files and directories +to be excluded from the build context. To improve the build's performance, +create a `.dockerignore` file and add the `node_modules` directory in it: ```.dockerignore node_modules diff --git a/samples/dotnetcore.md b/samples/dotnetcore.md index 34dfe5120f..81138207e7 100644 --- a/samples/dotnetcore.md +++ b/samples/dotnetcore.md @@ -63,8 +63,8 @@ clone our [ASP.NET Docker Sample](https://github.com/dotnet/dotnet-docker/tree/m ENTRYPOINT ["dotnet", "aspnetapp.dll"] ``` -4. To make your build context as small as possible add a [`.dockerignore` - file](/engine/reference/builder/#dockerignore-file) +4. To make your [build context](../build/building/context.md) as small as + possible add a [`.dockerignore` file](../engine/reference/builder.md#dockerignore-file) to your project folder and copy the following into it. ```dockerignore @@ -95,10 +95,10 @@ obj/ ENTRYPOINT ["dotnet", "aspnetapp.dll"] ``` -4. To make your build context as small as possible add a [`.dockerignore` - file](/engine/reference/builder/#dockerignore-file) - to your project folder. - +4. To make your [build context](../build/building/context.md) as small as + possible add a [`.dockerignore`file](../engine/reference/builder.md#dockerignore-file) + to your project folder. + ## Build and run the Docker image 1. Open a command prompt and navigate to your project folder.