From 69d375a480fea55a980ec7cdfb129c78d082734b Mon Sep 17 00:00:00 2001 From: David Karlsson <35727626+dvdksn@users.noreply.github.com> Date: Wed, 5 Jun 2024 14:42:06 +0200 Subject: [PATCH] build: consolidate building best practices Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com> --- content/build/building/base-images.md | 2 +- .../building/best-practices.md} | 172 +++++++++++++++++- content/build/guide/layers.md | 2 +- .../dockerfile_best-practices.md | 64 ------- content/develop/develop-images/guidelines.md | 156 ---------------- content/develop/security-best-practices.md | 2 +- content/docker-hub/builds/_index.md | 2 +- content/get-started/09_image_best.md | 2 +- content/language/_index.md | 2 +- content/scout/policy/remediation.md | 4 +- content/storage/storagedriver/_index.md | 2 +- .../trusted-content/official-images/_index.md | 2 +- data/redirects.yml | 6 +- data/toc.yaml | 10 +- 14 files changed, 182 insertions(+), 246 deletions(-) rename content/{develop/develop-images/instructions.md => build/building/best-practices.md} (72%) delete mode 100644 content/develop/develop-images/dockerfile_best-practices.md delete mode 100644 content/develop/develop-images/guidelines.md diff --git a/content/build/building/base-images.md b/content/build/building/base-images.md index 258d1b77a1..f17b341f66 100644 --- a/content/build/building/base-images.md +++ b/content/build/building/base-images.md @@ -110,5 +110,5 @@ There are lots of resources available to help you write your `Dockerfile`. * There's a [complete guide to all the instructions](../../reference/dockerfile.md) available for use in a `Dockerfile` in the reference section. * To help you write a clear, readable, maintainable `Dockerfile`, we've also - written a [Dockerfile best practices guide](../../develop/develop-images/dockerfile_best-practices.md). + written a [Dockerfile best practices guide](../../build/building/best-practices.md). * If your goal is to create a new Docker Official Image, read [Docker Official Images](../../trusted-content/official-images/_index.md). diff --git a/content/develop/develop-images/instructions.md b/content/build/building/best-practices.md similarity index 72% rename from content/develop/develop-images/instructions.md rename to content/build/building/best-practices.md index 17077aa9b8..459d1157ab 100644 --- a/content/develop/develop-images/instructions.md +++ b/content/build/building/best-practices.md @@ -1,11 +1,173 @@ --- -description: Hints, tips and guidelines for writing clean, reliable Dockerfile instructions -keywords: parent image, images, dockerfile, best practices, hub, official image -title: Best practices for Dockerfile instructions +description: Hints, tips and guidelines for writing clean, reliable Dockerfiles +keywords: base images, dockerfile, best practices, hub, official image +title: Building best practices +tags: [Best practices] +aliases: + - /articles/dockerfile_best-practices/ + - /engine/articles/dockerfile_best-practices/ + - /docker-cloud/getting-started/intermediate/optimize-dockerfiles/ + - /docker-cloud/tutorials/optimize-dockerfiles/ + - /engine/userguide/eng-image/dockerfile_best-practices/ + - /develop/develop-images/dockerfile_best-practices/ + - /develop/develop-images/guidelines/ + - /develop/develop-images/instructions/ --- -These recommendations are designed to help you create an efficient and -maintainable Dockerfile. +## Use multi-stage builds + +Multi-stage builds let you reduce the size of your final image, by creating a +cleaner separation between the building of your image and the final output. +Split your Dockerfile instructions into distinct stages to make sure that the +resulting output only contains the files that's needed to run the application. + +Using multiple stages can also let you build more efficiently by executing +build steps in parallel. + +See [Multi-stage builds](../../build/building/multi-stage.md) for more +information. + +## Exclude with .dockerignore + +To exclude files not relevant to the build, without restructuring your source +repository, use a `.dockerignore` file. This file supports exclusion patterns +similar to `.gitignore` files. For information on creating one, see +[Dockerignore file](../../build/building/context.md#dockerignore-files). + +## Create ephemeral containers + +The image defined by your Dockerfile should generate containers that are as +ephemeral as possible. Ephemeral means that the container can be stopped +and destroyed, then rebuilt and replaced with an absolute minimum set up and +configuration. + +Refer to [Processes](https://12factor.net/processes) under _The Twelve-factor App_ +methodology to get a feel for the motivations of running containers in such a +stateless fashion. + +## Don't install unnecessary packages + +Avoid installing extra or unnecessary packages just because they might be nice to have. For example, you don’t need to include a text editor in a database image. + +When you avoid installing extra or unnecessary packages, your images have reduced complexity, reduced dependencies, reduced file sizes, and reduced build times. + +## Decouple applications + +Each container should have only one concern. Decoupling applications into +multiple containers makes it easier to scale horizontally and reuse containers. +For instance, a web application stack might consist of three separate +containers, each with its own unique image, to manage the web application, +database, and an in-memory cache in a decoupled manner. + +Limiting each container to one process is a good rule of thumb, but it's not a +hard and fast rule. For example, not only can containers be +[spawned with an init process](../../engine/reference/run.md#specify-an-init-process), +some programs might spawn additional processes of their own accord. For +instance, [Celery](https://docs.celeryproject.org/) can spawn multiple worker +processes, and [Apache](https://httpd.apache.org/) can create one process per +request. + +Use your best judgment to keep containers as clean and modular as possible. If +containers depend on each other, you can use [Docker container networks](../../network/index.md) +to ensure that these containers can communicate. + +## Sort multi-line arguments + +Whenever possible, sort multi-line arguments alphanumerically to make maintenance easier. +This helps to avoid duplication of packages and make the +list much easier to update. This also makes PRs a lot easier to read and +review. Adding a space before a backslash (`\`) helps as well. + +Here’s an example from the [buildpack-deps image](https://github.com/docker-library/buildpack-deps): + +```dockerfile +RUN apt-get update && apt-get install -y \ + bzr \ + cvs \ + git \ + mercurial \ + subversion \ + && rm -rf /var/lib/apt/lists/* +``` + +## Leverage build cache + +When building an image, Docker steps through the instructions in your +Dockerfile, executing each in the order specified. For each instruction, Docker +checks whether it can reuse the instruction from the build cache. + +Understanding how the build cache works, and how cache invalidation occurs, +is critical for ensuring faster builds. +For more information about the Docker build cache and how to optimize your builds, +see [Docker build cache](../../build/cache/_index.md). + +## Pin base image versions + +Image tags are mutable, meaning a publisher can update a tag to point to a new +image. This is useful because it lets publishers update tags to point to +newer versions of an image. And as an image consumer, it means you +automatically get the new version when you re-build your image. + +For example, if you specify `FROM alpine:3.19` in your Dockerfile, `3.19` +resolves to the latest patch version for `3.19`. + +```dockerfile +# syntax=docker/dockerfile:1 +FROM alpine:3.19 +``` + +At one point in time, the `3.19` tag might point to version 3.19.1 of the +image. If you rebuild the image 3 months later, the same tag might point to a +different version, such as 3.19.4. This publishing workflow is best practice, +and most publishers use this tagging strategy, but it isn't enforced. + +The downside with this is that you're not guaranteed to get the same for every +build. This could result in breaking changes, and it means you also don't have +an audit trail of the exact image versions that you're using. + +To fully secure your supply chain integrity, you can pin the image version to a +specific digest. By pinning your images to a digest, you're guaranteed to +always use the same image version, even if a publisher replaces the tag with a +new image. For example, the following Dockerfile pins the Alpine image to the +same tag as earlier, `3.19`, but this time with a digest reference as well. + +```dockerfile +# syntax=docker/dockerfile:1 +FROM alpine:3.19@sha256:13b7e62e8df80264dbb747995705a986aa530415763a6c58f84a3ca8af9a5bcd +``` + +With this Dockerfile, even if the publisher updates the `3.19` tag, your builds +would still use the pinned image version: +`13b7e62e8df80264dbb747995705a986aa530415763a6c58f84a3ca8af9a5bcd`. + +While this helps you avoid unexpected changes, it's also more tedious to have +to look up and include the image digest for base image versions manually each +time you want to update it. And you're opting out of automated security fixes, +which is likely something you want to get. + +Docker Scout has a built-in [**Outdated base images** +policy](../../scout/policy/_index.md#outdated-base-images) that checks for +whether the base image version you're using is in fact the latest version. This +policy also checks if pinned digests in your Dockerfile correspond to the +correct version. If a publisher updates an image that you've pinned, the policy +evaluation returns a non-compliant status, indicating that you should update +your image. + +Docker Scout also supports an automated remediation workflow for keeping your +base images up-to-date. When a new image digest is available, Docker Scout can +automatically raise a pull request on your repository to update your +Dockerfiles to use the latest version. This is better than using a tag that +changes the version automatically, because you're in control and you have an +audit trail of when and how the change occurred. + +For more information about automatically updating your base images with Docker +Scout, see +[Remediation](../../scout/policy/remediation.md#automatic-base-image-updates) + +## Dockerfile instructions + +Follow these recommendations on how to properly use the [Dockerfile instructions](../../reference/dockerfile.md) +to create an efficient and maintainable Dockerfile. ### FROM diff --git a/content/build/guide/layers.md b/content/build/guide/layers.md index 8d2412ce07..72a6270dd6 100644 --- a/content/build/guide/layers.md +++ b/content/build/guide/layers.md @@ -71,7 +71,7 @@ work at build time. Related information: - [Docker build cache](../cache/_index.md) -- [Dockerfile best practices](../../develop/develop-images/dockerfile_best-practices.md) +- [Dockerfile best practices](../../build/building/best-practices.md) ## Next steps diff --git a/content/develop/develop-images/dockerfile_best-practices.md b/content/develop/develop-images/dockerfile_best-practices.md deleted file mode 100644 index f9be76c157..0000000000 --- a/content/develop/develop-images/dockerfile_best-practices.md +++ /dev/null @@ -1,64 +0,0 @@ ---- -description: Overview of a Dockerfile and introduction to best practices -keywords: parent image, images, dockerfile, best practices, hub, official image -title: Overview of best practices for writing Dockerfiles -aliases: -- /articles/dockerfile_best-practices/ -- /engine/articles/dockerfile_best-practices/ -- /docker-cloud/getting-started/intermediate/optimize-dockerfiles/ -- /docker-cloud/tutorials/optimize-dockerfiles/ -- /engine/userguide/eng-image/dockerfile_best-practices/ -tags: [Best practices] ---- - -This topic covers recommended best practices and methods for building -efficient images. It provides [general guidelines for your Dockerfiles](guidelines.md) and more [specific best practices for each Dockerfile instruction](instructions.md). - -## What is a Dockerfile? - -Docker builds images automatically by reading the instructions from a -Dockerfile which is a text file that contains all commands, in order, needed to -build a given image. A Dockerfile adheres to a specific format and set of -instructions which you can find at [Dockerfile reference](../../reference/dockerfile.md). - -A Docker image consists of read-only layers each of which represents a -Dockerfile instruction. The layers are stacked and each one is a delta of the -changes from the previous layer. - -```dockerfile -# syntax=docker/dockerfile:1 - -FROM ubuntu:22.04 -COPY . /app -RUN make /app -CMD python /app/app.py -``` - -In the example above, each instruction creates one layer: - -- `FROM` creates a layer from the `ubuntu:22.04` Docker image. -- `COPY` adds files from your Docker client's current directory. -- `RUN` builds your application with `make`. -- `CMD` specifies what command to run within the container. - -When you run an image and generate a container, you add a new writable layer, also called the container layer, on top of the underlying layers. All changes made to -the running container, such as writing new files, modifying existing files, and -deleting files, are written to this writable container layer. - -## Additional resources - -* [Dockerfile reference](../../reference/dockerfile.md) -* [More about Automated builds](../../docker-hub/builds/index.md) -* [Guidelines for creating Docker Official Images](../../trusted-content/official-images/_index.md) -* [Best practices to containerize Node.js web applications with Docker](https://snyk.io/blog/10-best-practices-to-containerize-nodejs-web-applications-with-docker) -* [More about base images](../../build/building/base-images.md) -* [More on image layers and how Docker builds and stores images](../../storage/storagedriver/index.md). - -## Examples of Docker Official Images - -These Official Images have exemplary Dockerfiles: - -* [Go](https://hub.docker.com/_/golang/) -* [Perl](https://hub.docker.com/_/perl/) -* [Hy](https://hub.docker.com/_/hylang/) -* [Ruby](https://hub.docker.com/_/ruby/) diff --git a/content/develop/develop-images/guidelines.md b/content/develop/develop-images/guidelines.md deleted file mode 100644 index 2f712ffab8..0000000000 --- a/content/develop/develop-images/guidelines.md +++ /dev/null @@ -1,156 +0,0 @@ ---- -description: Hints, tips and guidelines for writing clean, reliable Dockerfiles -keywords: parent image, images, dockerfile, best practices, hub, official image -title: General best practices for writing Dockerfiles -tags: [Best practices] ---- - -## Use multi-stage builds - -Multi-stage builds let you reduce the size of your final image, by creating a -cleaner separation between the building of your image and the final output. -Split your Dockerfile instructions into distinct stages to make sure that the -resulting output only contains the files that's needed to run the application. - -Using multiple stages can also let you build more efficiently by executing -build steps in parallel. - -See [Multi-stage builds](../../build/building/multi-stage.md) for more -information. - -## Exclude with .dockerignore - -To exclude files not relevant to the build, without restructuring your source -repository, use a `.dockerignore` file. This file supports exclusion patterns -similar to `.gitignore` files. For information on creating one, see -[Dockerignore file](../../build/building/context.md#dockerignore-files). - -## Create ephemeral containers - -The image defined by your Dockerfile should generate containers that are as -ephemeral as possible. Ephemeral means that the container can be stopped -and destroyed, then rebuilt and replaced with an absolute minimum set up and -configuration. - -Refer to [Processes](https://12factor.net/processes) under _The Twelve-factor App_ -methodology to get a feel for the motivations of running containers in such a -stateless fashion. - -## Don't install unnecessary packages - -Avoid installing extra or unnecessary packages just because they might be nice to have. For example, you don’t need to include a text editor in a database image. - -When you avoid installing extra or unnecessary packages, your images have reduced complexity, reduced dependencies, reduced file sizes, and reduced build times. - -## Decouple applications - -Each container should have only one concern. Decoupling applications into -multiple containers makes it easier to scale horizontally and reuse containers. -For instance, a web application stack might consist of three separate -containers, each with its own unique image, to manage the web application, -database, and an in-memory cache in a decoupled manner. - -Limiting each container to one process is a good rule of thumb, but it's not a -hard and fast rule. For example, not only can containers be -[spawned with an init process](../../engine/reference/run.md#specify-an-init-process), -some programs might spawn additional processes of their own accord. For -instance, [Celery](https://docs.celeryproject.org/) can spawn multiple worker -processes, and [Apache](https://httpd.apache.org/) can create one process per -request. - -Use your best judgment to keep containers as clean and modular as possible. If -containers depend on each other, you can use [Docker container networks](../../network/index.md) -to ensure that these containers can communicate. - -## Sort multi-line arguments - -Whenever possible, sort multi-line arguments alphanumerically to make maintenance easier. -This helps to avoid duplication of packages and make the -list much easier to update. This also makes PRs a lot easier to read and -review. Adding a space before a backslash (`\`) helps as well. - -Here’s an example from the [buildpack-deps image](https://github.com/docker-library/buildpack-deps): - -```dockerfile -RUN apt-get update && apt-get install -y \ - bzr \ - cvs \ - git \ - mercurial \ - subversion \ - && rm -rf /var/lib/apt/lists/* -``` - -## Leverage build cache - -When building an image, Docker steps through the instructions in your -Dockerfile, executing each in the order specified. For each instruction, Docker -checks whether it can reuse the instruction from the build cache. - -Understanding how the build cache works, and how cache invalidation occurs, -is critical for ensuring faster builds. -For more information about the Docker build cache and how to optimize your builds, -see [Docker build cache](../../build/cache/_index.md). - -## Pin base image versions - -Image tags are mutable, meaning a publisher can update a tag to point to a new -image. This is useful because it lets publishers update tags to point to -newer versions of an image. And as an image consumer, it means you -automatically get the new version when you re-build your image. - -For example, if you specify `FROM alpine:3.19` in your Dockerfile, `3.19` -resolves to the latest patch version for `3.19`. - -```dockerfile -# syntax=docker/dockerfile:1 -FROM alpine:3.19 -``` - -At one point in time, the `3.19` tag might point to version 3.19.1 of the -image. If you rebuild the image 3 months later, the same tag might point to a -different version, such as 3.19.4. This publishing workflow is best practice, -and most publishers use this tagging strategy, but it isn't enforced. - -The downside with this is that you're not guaranteed to get the same for every -build. This could result in breaking changes, and it means you also don't have -an audit trail of the exact image versions that you're using. - -To fully secure your supply chain integrity, you can pin the image version to a -specific digest. By pinning your images to a digest, you're guaranteed to -always use the same image version, even if a publisher replaces the tag with a -new image. For example, the following Dockerfile pins the Alpine image to the -same tag as earlier, `3.19`, but this time with a digest reference as well. - -```dockerfile -# syntax=docker/dockerfile:1 -FROM alpine:3.19@sha256:13b7e62e8df80264dbb747995705a986aa530415763a6c58f84a3ca8af9a5bcd -``` - -With this Dockerfile, even if the publisher updates the `3.19` tag, your builds -would still use the pinned image version: -`13b7e62e8df80264dbb747995705a986aa530415763a6c58f84a3ca8af9a5bcd`. - -While this helps you avoid unexpected changes, it's also more tedious to have -to look up and include the image digest for base image versions manually each -time you want to update it. And you're opting out of automated security fixes, -which is likely something you want to get. - -Docker Scout has a built-in [**Outdated base images** -policy](../../scout/policy/_index.md#outdated-base-images) that checks for -whether the base image version you're using is in fact the latest version. This -policy also checks if pinned digests in your Dockerfile correspond to the -correct version. If a publisher updates an image that you've pinned, the policy -evaluation returns a non-compliant status, indicating that you should update -your image. - -Docker Scout also supports an automated remediation workflow for keeping your -base images up-to-date. When a new image digest is available, Docker Scout can -automatically raise a pull request on your repository to update your -Dockerfiles to use the latest version. This is better than using a tag that -changes the version automatically, because you're in control and you have an -audit trail of when and how the change occurred. - -For more information about automatically updating your base images with Docker -Scout, see -[Remediation](../../scout/policy/remediation.md#automatic-base-image-updates) diff --git a/content/develop/security-best-practices.md b/content/develop/security-best-practices.md index 698b865a4d..5293663bc8 100644 --- a/content/develop/security-best-practices.md +++ b/content/develop/security-best-practices.md @@ -113,7 +113,7 @@ Consider the following best practices when rebuilding an image: vulnerabilities. Based on that, automate the rebuilding of images if necessary. For detailed best practices and methods for building efficient images, see -[Dockerfile best practices](develop-images/dockerfile_best-practices.md). +[Image-building best practices](/build/building/best-practices). ## Check your image for vulnerabilities diff --git a/content/docker-hub/builds/_index.md b/content/docker-hub/builds/_index.md index 6591f53ac6..3d6af1d682 100644 --- a/content/docker-hub/builds/_index.md +++ b/content/docker-hub/builds/_index.md @@ -70,7 +70,7 @@ when the tests succeed. 9. For each branch or tag, enable or disable the **Build Caching** toggle. - [Build caching](../../develop/develop-images/dockerfile_best-practices.md#leverage-build-cache) + [Build caching](../../build/building/best-practices.md#leverage-build-cache) can save time if you are building a large image frequently or have many dependencies. Leave the build caching disabled to make sure all of your dependencies are resolved at build time, or if diff --git a/content/get-started/09_image_best.md b/content/get-started/09_image_best.md index 4baa02cbb3..861ecf8b57 100644 --- a/content/get-started/09_image_best.md +++ b/content/get-started/09_image_best.md @@ -212,7 +212,7 @@ Related information: - [.dockerignore](../build/building/context.md#dockerignore-files) - [Dockerfile reference](../reference/dockerfile.md) - [Build with Docker guide](../build/guide/index.md) - - [Dockerfile best practices](../develop/develop-images/dockerfile_best-practices.md) + - [Dockerfile best practices](../build/building/best-practices.md) ## Next steps diff --git a/content/language/_index.md b/content/language/_index.md index 2262c1320c..0da63105c9 100644 --- a/content/language/_index.md +++ b/content/language/_index.md @@ -16,7 +16,7 @@ The language-specific guides walk you through the process of: In addition to the language-specific modules, Docker documentation also provides guidelines to build images and efficiently manage your development environment. For more information, refer to the following topics: -* [Best practices for writing Dockerfiles](../develop/develop-images/dockerfile_best-practices.md) +* [Building best practices](../build/building/best-practices.md) * [Docker development best practices](../develop/dev-best-practices.md) * [Build images with BuildKit](../build/buildkit/index.md#getting-started) * [Build with Docker](../build/guide/_index.md) diff --git a/content/scout/policy/remediation.md b/content/scout/policy/remediation.md index 346d3783a3..18835273d0 100644 --- a/content/scout/policy/remediation.md +++ b/content/scout/policy/remediation.md @@ -100,7 +100,7 @@ determine if it's up-to-date. If there's a policy violation, the recommended actions show how to update your base image version to the latest version, while also pinning the base image version to a specific digest. For more information, see [Pin base image -versions](../../develop/develop-images/guidelines.md#pin-base-image-versions). +versions](../../build/building/best-practices.md#pin-base-image-versions). ### GitHub integration enabled @@ -121,7 +121,7 @@ image to a digest is important for reproducibility, and helps avoid unwanted changes from making their way into your supply chain. For more information about base image pinning, see [Pin base image -versions](../../develop/develop-images/guidelines.md#pin-base-image-versions). +versions](../../build/building/best-practices.md#pin-base-image-versions).