Moves ParseVolume() to a new internal package to remove the dependency
on cli/compose/loader in cli/command/container/opts.go
refactor to keep types isolated
- rename the package to "volumespec" to reuse the name of the package
as part of the name (parsevolume.ParseVolume() -> volumespec.Parse())
- move the related compose types to the internal package as well,
and rename them to be more generic (not associated with "compose");
- ServiceVolumeConfig -> VolumeConfig
- ServiceVolumeBind -> BindOpts
- ServiceVolumeVolume -> VolumeOpts
- ServiceVolumeImage -> ImageOpts
- ServiceVolumeTmpfs -> TmpFsOpts
- ServiceVolumeCluster -> ClusterOpts
- alias the internal types inside cli/compose/types to keep backward
compatibility (for any external consumers); even though the implementation
is internal, Go allows aliasing types to use them externally.
Signed-off-by: Michael Tews <michael@tews.dev>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
internal/registry/errors.go:26:43: use-any: since Go 1.18 'interface{}' can be replaced by 'any' (revive)
func invalidParamf(format string, args ...interface{}) error {
^
internal/registry/registry_mock_test.go:52:51: use-any: since Go 1.18 'interface{}' can be replaced by 'any' (revive)
func writeResponse(w http.ResponseWriter, message interface{}, code int) {
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The registry.ServiceConfig struct in the API types was meant for the
registry configuration on the daemon side; it has variuos fields we
don't use, defines methods for (un)marshaling JSON, and a custom version
of `net.IPNet`, also to (un)marshal JSON.
None of that is needed, so let's change it to a local type, and implement
a constructor (as we now only have "insecure registries" to care
about).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It was written to be used as validate-func for command-line flags, which
we don't use it for (which for CLI-flags includes normalizing the value).
The validation itself didn't add much; it only checked the registry didn't
start or end with a hyphen (which would still fail when parsing).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
now that we no longer need to account for mirrors, these were
identical, so just use a single one.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The CLI does not have information about mirrors, and doesn't
configure them, so we can remove these parts.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This adds an internal fork of [github.com/docker/docker/registry], taken
at commit [moby@f651a5d]. Git history was not preserved in this fork,
but can be found using the URLs provided.
This fork was created to remove the dependency on the "Moby" codebase,
and because the CLI only needs a subset of its features. The original
package was written specifically for use in the daemon code, and includes
functionality that cannot be used in the CLI.
[github.com/docker/docker/registry]: https://pkg.go.dev/github.com/docker/docker@v28.3.2+incompatible/registry
[moby@49306c6]: 49306c607b/registry
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This was deprecated in 6f46cd2f4b,
which is part of v28.x, and no longer used, so we can remove it.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This utility was only used in the CLI, but the implementation was
based on it being used on the daemon side, so included resolving
the host's IP-address, mirrors, etc.
The only reason it's used in the CLI is to provide credentials for
the registry that's being searched, so reduce it to just that.
There's more cleaning up to do in this area, so to make our lives
easier, it's implemented locally as non-exported functions; likely
to be replaced with something else.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Docker Content Trust is currently only implemented for the classic
builder, but is known to not work with multi-stage builds, and
requires rewriting the Dockerfile, which is brittle because the
Dockerfile syntax evolved with the introduction of BuildKit as
default builder.
Given that the classic builder is deprecated, and only used for
Windows images, which are not verified by content trust;
# docker pull --disable-content-trust=false mcr.microsoft.com/windows/servercore:ltsc2025
Error: remote trust data does not exist for mcr.microsoft.com/windows/servercore: mcr.microsoft.com does not have trust data for mcr.microsoft.com/windows/servercore
With content trust not implemented in BuildKit, and not implemented
in docker compose, this resulted in an inconsistent behavior.
This patch removes content-trust support for "docker build". As this
is a client-side feature, users who require this feature can still
use an older CLI to to start the build.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
gitutils_test.go:210: git-http-backend: git: 'http-backend' is not a git command. See 'git --help'.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Adds a local fork of this package for use in the classic builder.
Code was taken at commit [d33d46d01656e1d9ee26743f0c0d7779f685dd4e][1].
Migration was done using the following steps:
# install filter-repo (https://github.com/newren/git-filter-repo/blob/main/INSTALL.md)
brew install git-filter-repo
# create a temporary clone of docker
cd ~/Projects
git clone https://github.com/docker/docker.git build_context_temp
cd build_context_temp
# commit taken from
git rev-parse --verify HEAD
d33d46d01656e1d9ee26743f0c0d7779f685dd4e
git filter-repo --analyze
# remove all code, except for the remotecontext packages, and move to build/internal docs and previous locations of it
git filter-repo \
--path 'builder/remotecontext/git' \
--path 'builder/remotecontext/urlutil' \
--path-rename builder/remotecontext:cli/command/image/build/internal
# go to the target repository
cd ~/go/src/github.com/docker/cli
# create a branch to work with
git checkout -b fork_remotecontext
# add the temporary repository as an upstream and make sure it's up-to-date
git remote add build_context_temp ~/Projects/build_context_temp
git fetch build_context_temp
# merge the upstream code
git merge --allow-unrelated-histories --signoff -S build_context_temp/master
[1]: https://github.com/docker/docker/d33d46d01656e1d9ee26743f0c0d7779f685dd4e
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Removes direct imports of github.com/docker/docker/builder in
the image package, to be moved later.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- use consistent name for MountOpt vars
- cleanup some comments and make them a GoDoc
- remove import alias
- use subtests for tests that were prepared for it.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It's part of the presentation logic of the cli, and only used internally.
We can consider providing utilities for these, but better as part of
separate packages.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Also rename "ctx" argument; we shouldn't use this as name for things
that are not a context.Context.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It's part of the presentation logic of the cli, and only used internally.
We can consider providing utilities for these, but better as part of
separate packages.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This patch deprecates the unused `RegistryAuthenticationPrivilegedFunc`.
The function would prompt the user when the registry returns a 403 after trying
the initial auth value set in `RegistryAuth`.
Signed-off-by: Alano Terblanche <18033717+Benehiko@users.noreply.github.com>
docker build --call outline .
TARGET: binary
BUILD ARG VALUE DESCRIPTION
BASE_VARIANT alpine
ALPINE_VERSION 3.21 sets the version of the alpine base image to use, including for the golang image.
GO_VERSION 1.24.5
XX_VERSION 1.6.1
GOVERSIONINFO_VERSION v1.4.1
GO_LINKMODE static defines if static or dynamic binary should be produced
GO_BUILDTAGS defines additional build tags
GO_STRIP strips debugging symbols if set
CGO_ENABLED manually sets if cgo is used
VERSION sets the version for the produced binary
PACKAGER_NAME sets the company that produced the windows binary
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This patch removes the interactive prompts from `docker push/pull`.
The prompt would only execute on a response status code 403 from the registry
after trying the value set in `RegistryAuth`. Docker Hub could return 404
instead or 429, which would never execute the prompt.
The UX regarding the prompt is also questionable since the user might
not actually want to authenticate with a registry and the CLI could fail fast
instead. The user can always run `docker login` or set the `DOCKER_AUTH_CONFIG`
environment variable to get authenticated.
Signed-off-by: Alano Terblanche <18033717+Benehiko@users.noreply.github.com>
client.ContainerDiff already validates the given container name/ID, and
produces an error when empty, so we don't have to check for this;
abba330bbf/client/container_diff.go (L13-L16)
While updating, also;
- remove the diffOptions type, as there were no other options, and make
the container name/ID a string argument.
- fix camelCase nameing of dockerCLI
Before this patch:
docker diff ""
Container name cannot be empty
With this patch:
docker diff ""
invalid container name or ID: value is empty
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It was the only utility we consumed from the package, and it's trivial
to implement, so let's create local copies of it.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This utility was only used for testing, and to generate a random
suffix for Dockerfiles. As we don't need the same contract as
pkg/stringid.GenerateRandomID() (not allow all-numeric IDs as they
would not be usable for hostnames), we can use a local test-utility,
and local implementation for the random suffix instead.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
We were depending on pkg/stringid to truncate IDs for presentation. While
traditionally, we used a fixed length for "truncated" IDs, this is not
a strict requirement (any ID-prefix should work, but conflicts may
happen on shorter IDs).
This patch adds a local `TruncateID()` utility in the formatter package;
it's currently using the same implementation and length as the
`stringid.TruncateID` function, but may diverge in future.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The `AllowOverwriteDirWithFile` option was added when reimplementing the
CLI using the API Client lib in [moby@1b2b91b]. Before that refactor, the
`noOverwriteDirNonDir` query argument [would be set unconditionally][1]
by the CLI, with no options to control the behavior.
It's unclear why the `noOverwriteDirNonDir` was implemented as opt-in (not
opt-out), as overwriting a file with a directory (or vice-versa) would
generally be unexpected behavior.
We're considering making `noOverwriteDirNonDir` unconditional on the daemon
side, and to deprecate the `AllowOverwriteDirWithFile` option. This patch
removes its use, as it was set to the default either way, and there's no
options to configure it from the CLI.
[1]: 8c9ad7b818/api/client/cp.go (L345-L346)
[moby@1b2b91b]: 1b2b91ba43
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Make the test slightly more permissive; we're looking for a trailing
newline, not necessarily an empty line.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- https://github.com/golang/go/issues?q=milestone%3AGo1.24.5+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.24.4...go1.24.5
This minor releases include 1 security fixes following the security policy:
- cmd/go: unexpected command execution in untrusted VCS repositories
Various uses of the Go toolchain in untrusted VCS repositories can result in
unexpected code execution. When using the Go toolchain in directories fetched
using various VCS tools (such as directly cloning Git or Mercurial repositories)
can cause the toolchain to execute unexpected commands, if said directory
contains multiple VCS configuration metadata (such as a '.hg' directory in a Git
repository). This is due to how the Go toolchain attempts to resolve which VCS
is being used in order to embed build information in binaries and determine
module versions.
The toolchain will now abort attempting to resolve which VCS is being used if it
detects multiple VCS configuration metadata in a module directory or nested VCS
configuration metadata (such as a '.git' directoy in a parent directory and a
'.hg' directory in a child directory). This will not prevent the toolchain from
building modules, but will result in binaries omitting VCS related build
information.
If this behavior is expected by the user, the old behavior can be re-enabled by
setting GODEBUG=allowmultiplevcs=1. This should only be done in trusted
repositories.
Thanks to RyotaK (https://ryotak.net) of GMO Flatt Security Inc for reporting
this issue.
This is CVE-2025-4674 and https://go.dev/issue/74380.
View the release notes for more information:
https://go.dev/doc/devel/release#go1.24.5
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
I noticed this in a ticket in the compose issue tracker; with debug logging
enabled, the OTEL error-logger may be logging even if there's no error;
DEBU[0000] Executing bake with args: [bake --file - --progress rawjson --metadata-file /tmp/compose-build-metadataFile-1203980021.json --allow fs.read=/home/user/dev/project --allow fs.read=/home/user/dev/project --allow fs.read=/home/user/dev/project/nginx --allow fs.read=/home/user/dev/project]
TRAC[0000] Plugin server listening on @docker_cli_d8df486f78df3b7357995be71bf0cef6
DEBU[0005] otel error error="<nil>"
^CTRAC[0055] Closing plugin server
TRAC[0055] Closing plugin server
DEBU[0055] otel error error="<nil>"
DEBU[0055] otel error error="<nil>"
Update the error-handler to not log if there's no error.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Make it a hidden file. From the [CodeCov docs][1]:
> Can I name the file .codecov.yml?
>
> Yes, you can name the file `codecov.yml` or `.codecov.yml`. However, the
> file must still be located in the repository root, `dev/`, or `.github/`
> directories
[1]: https://docs.codecov.com/docs/codecov-yaml#can-i-name-the-file-codecovyml
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The strslice.StrSlice type is a string-slice with a custom JSON Unmarshal
function to provide backward-compatibility with older API requests (see
[moby@17d6f00] and [moby@ea4a067]).
Given that the type is assigned implicitly through the fields on HostConfig,
we can just use a regular []string instead.
[moby@17d6f00]: 17d6f00ec2
[moby@ea4a067]: ea4a06740b
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Adding image tags that follow the semver major and minor versions (e.g., `28`
and `28.3`) for the moby-bin images.
This makes it easier for users to reference the latest build within a
major or minor version series without having to know the exact
minor/patch version.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
When connecting to a remote daemon through an ssh:// connection,
the CLI connects with the remote host using ssh, executing the
`docker system dial-stdio` command on the remote host to connect
to the daemon API's unix socket.
By default, the `docker system dial-stdio` command connects with the
daemon using the default location (/var/run/docker.sock), or the
location as configured on the remote host.
Commit 25ebf0ec9c (included in docker
CLI v24.0.0-rc.2 and higher) introduced a feature to allow the location
of the socket to be specified through the host connection string, for
example:
DOCKER_HOST='ssh://example.test/run/custom-docker.sock'
The custom path is included as part of the ssh command executed from
the client machine to connect with the remote host. THe example above
would execute the following command from the client machine;
ssh -o ConnectTimeout=30 -T -- example.test docker --host unix:///run/custom-docker.sock system dial-stdio
ssh executes remote commands in a shell environment, and no quoting
was in place, which allowed for a connection string to include additional
content, which would be expanded / executed on the remote machine.
For example, the following example would execute `echo hello > /hello.txt`
on the remote machine;
export DOCKER_HOST='ssh://example.test/var/run/docker.sock $(echo hello > /hello.txt)'
docker info
# (output of docker info from the remote machine)
While this doesn't allow the user to do anything they're not already
able to do so (by directly using the same SSH connection), the behavior
is not expected, so this patch adds quoting to prevent such URLs from
resulting in expansion.
This patch updates the cli/connhelper and cli/connhelper/ssh package to
quote parameters used in the ssh command to prevent code execution and
expansion of variables on the remote machine. Quoting is also applied to
other parameters that are obtained from the DOCKER_HOST url, such as username
and hostname.
- The existing `Spec.Args()` method inthe cli/connhelper/ssh package now
quotes arguments, and returns a nil slice when failing to quote. Users
of this package should therefore check the returned arguments before
consuming. This method did not provide an error-return, and adding
one would be a breaking change.
- A new `Spec.Command` method is introduced, which (unlike the `Spec.Args()`
method) provides an error return. Users are recommended to use this new
method instead of the `Spec.Args()` method.
Some minor additional changes in behavior are included in this patch;
- Connection URLs with a trailing slash (e.g. `ssh://example.test/`)
would previously result in `unix:///` being used as custom socket
path. After this patch, the trailing slash is ignored, and no custom
socket path is used.
- Specifying a remote command is now required. When passing an empty
remote command, `Spec.Args()` now results in a `nil` value to be
returned (or an `no remote command specified` error when using
`Spec.Comnmand()`.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
cli/connhelper/internal/syntax/parser.go:31:2: Duplicate words (the) found (dupword)
// Note that it shares some features with Bash, due to the the shared
^
cli/connhelper/internal/syntax/quote.go:48:1: cyclomatic complexity 35 of func `Quote` is high (> 16) (gocyclo)
func Quote(s string, lang LangVariant) (string, error) {
^
cli/connhelper/internal/syntax/quote.go:103:3: shadow: declaration of "offs" shadows declaration at line 56 (govet)
offs := 0
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These comments were added to enforce using the correct import path for
our packages ("github.com/docker/docker", not "github.com/moby/moby").
However, when working in go module mode (not GOPATH / vendor), they have
no effect, so their impact is limited.
Remove these imports in preparation of migrating our code to become an
actual go module.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Also renamed some vars for clarity, renamed a error-returns to prevent
shadowing, and fixed some linter warnings about unhandled errors.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
If we have an error type that we're checking a substring against, we
should really be checking using ErrorContains to indicate the right
semantics to assert.
Mostly done using these transforms:
find . -type f -name "*_test.go" | \
xargs gofmt -w -r 'assert.Assert(t, is.ErrorContains(e, s)) -> assert.ErrorContains(t, e, s)'
find . -type f -name "*_test.go" | \
xargs gofmt -w -r 'assert.Assert(t, is.Contains(err.Error(), s)) -> assert.ErrorContains(t, err, s)'
find . -type f -name "*_test.go" | \
xargs gofmt -w -r 'assert.Check(t, is.Contains(err.Error(), s)) -> assert.Check(t, is.ErrorContains(err, s))'
As well as some small fixups to helpers that were doing
strings.Contains explicitly.
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
builder/remotecontext/git/gitutils_test.go:116:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This is a partial revert of 389ada7188, which
switched from os/exec to the golang.org/x/sys/execabs package to mitigate
security issues (mainly on Windows) with lookups resolving to binaries in the
current directory.
from the go1.19 release notes https://go.dev/doc/go1.19#os-exec-path
> ## PATH lookups
>
> Command and LookPath no longer allow results from a PATH search to be found
> relative to the current directory. This removes a common source of security
> problems but may also break existing programs that depend on using, say,
> exec.Command("prog") to run a binary named prog (or, on Windows, prog.exe) in
> the current directory. See the os/exec package documentation for information
> about how best to update such programs.
>
> On Windows, Command and LookPath now respect the NoDefaultCurrentDirectoryInExePath
> environment variable, making it possible to disable the default implicit search
> of “.” in PATH lookups on Windows systems.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
cmd.Environ() is new in go1.19, and not needed for this specific case.
Without this, trying to use this package in code that uses go1.18 will fail;
builder/remotecontext/git/gitutils.go:216:23: cmd.Environ undefined (type *exec.Cmd has no field or method Environ)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Setting cmd.Env overrides the default of passing through the parent
process' environment, which works out fine most of the time, except when
it doesn't. For whatever reason, leaving out all the environment causes
git-for-windows sh.exe subprocesses to enter an infinite loop of
access violations during Cygwin initialization in certain environments
(specifically, our very own dev container image).
Signed-off-by: Cory Snider <csnider@mirantis.com>
While it is undesirable for the system or user git config to be used
when the daemon clones a Git repo, it could break workflows if it was
unconditionally applied to docker/cli as well.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Prevent git commands we run from reading the user or system
configuration, or cloning submodules from the local filesystem.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Keep It Simple! Set the working directory for git commands by...setting
the git process's working directory. Git commands can be run in the
parent process's working directory by passing the empty string.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Make the test more debuggable by logging all git command output and
running each table-driven test case as a subtest.
Signed-off-by: Cory Snider <csnider@mirantis.com>
Older versions of Go don't format comments, so committing this as
a separate commit, so that we can already make these changes before
we upgrade to Go 1.19.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Simplify some of the logic, and add documentation about the package,
as well as warnings that this package should not be used as a general-
purpose utility.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
pkg/urlutil (despite its poorly chosen name) is not really intended as a generic
utility to handle URLs, and should only be used by the builder to handle (remote)
build contexts.
- IsURL() only does a very rudimentary check for http(s):// prefixes, without any
other validation, but due to its name may give incorrect expectations.
- IsGitURL() is written specifically with docker build remote git contexts in
mind, and has handling for backward-compatibility, where strings that are
not URLs, but start with "github.com/" are accepted.
Because of the above, this patch:
- moves the package inside builder/remotecontext, close to where it's intended
to be used (ideally this would be part of build/remotecontext itself, but this
package imports many other dependencies, which would introduce those as extra
dependencies in the CLI).
- deprecates pkg/urlutil, but adds aliases as there are some external consumers.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The io/ioutil package has been deprecated in Go 1.16. This commit
replaces the existing io/ioutil functions with their new definitions in
io and os packages.
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
urlutil.IsUrl() was merely checking if the url had a http(s)://
prefix, which is just as well handled through using url.Parse()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This function was only used inside gitutils,
and is written specifically for the requirements
there.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
`docker build` accepts remote repositories
using either the `git://` notation, or `git@`.
Docker attempted to parse both as an URL, however,
`git@` is not an URL, but an argument to `git clone`.
Go 1.7 silently ignored this, and managed to
extract the needed information from these
remotes, however, Go 1.8 does a more strict
validation, and invalidated these.
This patch adds a different path for `git@` remotes,
to prevent them from being handled as URL (and
invalidated).
A test is also added, because there were no
tests for handling of `git@` remotes.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>