1. Replace raw `docker inspect -f xxx` with `inspectField`, to make code
cleaner and more consistent
2. assert the error in function `inspectField*` so we don't need to
assert the return value of it every time, this will make inspect easier.
Signed-off-by: Zhang Wei <zhangwei555@huawei.com>
TestBuildEnvUsage2 was flagged in https://github.com/docker/docker/issues/19425
as one of the slowest integration tests. It's slow because it has some
comprehensive builder test cases that end up creating a lot of layers.
Even with a busybox base image, this can be expensive. It's not possible
to build "FROM scratch" because the test cases need the shell to ensure
environment variables are set correctly.
Some of the ENV and RUN statements can be combined. This causes fewer
layers to get created. Doing this produces a marginal improvement in the
runtime.
Before:
PASS: docker_cli_build_test.go:3956: DockerSuite.TestBuildEnvUsage2 43.619s
After:
PASS: docker_cli_build_test.go:3956: DockerSuite.TestBuildEnvUsage2 31.286s
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
if create a container with -w to specify the working directory and
the directory does not exist in the container rootfs, the directory
will be created until the container start. It make docker export of
a created container and a running container inconsistent.
Signed-off-by: Lei Jitang <leijitang@huawei.com>
- Stop serializing JSONMessage in favor of events.Message.
- Keep backwards compatibility with JSONMessage for container events.
Signed-off-by: David Calavera <david.calavera@gmail.com>
We keep only one logic to test event related behavior that will help us
diagnose flacky event errors.
Signed-off-by: David Calavera <david.calavera@gmail.com>
Right now, the quiet (-q, --quiet) flag ignores the output
generated from within the container.
However, it ought to be quiet in a way that all kind
of diagnostic output should be ignored, unless the build
process fails.
This patch makes the quiet flag behave in the following way:
1. If the build process succeeds, stdout contains the image ID
and stderr is empty.
2. If the build process fails, stdout is empty and stderr
has the error message and the diagnostic output of that process.
If the quiet flag is not set, then everything goes to stdout
and error messages, if there are any, go to stderr.
Signed-off-by: Boaz Shuster <ripcurld.github@gmail.com>
A TopicFunc is an interface to let the pubisher decide whether it needs
to send a message to a subscriber or not. It returns true if the
publisher must send the message and false otherwise.
Users of the pubsub package can create a subscriber with a topic
function by calling `pubsub.SubscribeTopic`.
Message delivery has also been modified to use concurrent channels per
subscriber. That way, topic verification and message delivery is not
o(N+M) anymore, based on the number of subscribers and topic verification
complexity.
Using pubsub topics, the API stops controlling the message delivery,
delegating that function to a topic generated with the filtering
provided by the user. The publisher sends every message to the
subscriber if there is no filter, but the api doesn't have to select
messages to return anymore.
Signed-off-by: David Calavera <david.calavera@gmail.com>
Add distribution package for managing pulls and pushes. This is based on
the old code in the graph package, with major changes to work with the
new image/layer model.
Add v1 migration code.
Update registry, api/*, and daemon packages to use the reference
package's types where applicable.
Update daemon package to use image/layer/tag stores instead of the graph
package
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Closes#17781
This allows for env vars in EXPOSE to be parsed for spaces so that each
"word" is then treated independently instead of as a single word/arg.
Signed-off-by: Doug Davis <dug@us.ibm.com>
Fixes#17290
Fixes following issues:
- Cache checksums turning off while walking a broken symlink.
- Cache checksums were taken from symlinks while targets were actually copied.
- Copying a symlink pointing to a file to a directory used the basename of the target as a destination basename, instead of basename of the symlink.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
This is useful for cluster systems such as swarm to sync the image
state when new images are successfully built.
Signed-off-by: Shijiang Wei <mountkin@gmail.com>
Adds support for the daemon to handle user namespace maps as a
per-daemon setting.
Support for handling uid/gid mapping is added to the builder,
archive/unarchive packages and functions, all graphdrivers (except
Windows), and the test suite is updated to handle user namespace daemon
rootgraph changes.
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
Fixes an issue where `VOLUME some_name:/foo` would be parsed as a named
volume, allowing access from the builder to any volume on the host.
This makes sure that named volumes must always be passed in as a bind.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
- The build-time variables are passed as environment-context for command(s)
run as part of the RUN primitve. These variables are not persisted in environment of
intermediate and final images when passed as context for RUN. The build environment
is prepended to the intermediate continer's command string for aiding cache lookups.
It also helps with build traceability. But this also makes the feature less secure from
point of view of passing build time secrets.
- The build-time variables also get used to expand the symbols used in certain
Dockerfile primitves like ADD, COPY, USER etc, without an explicit prior definiton using a
ENV primitive. These variables get persisted in the intermediate and final images
whenever they are expanded.
- The build-time variables are only expanded or passed to the RUN primtive if they
are defined in Dockerfile using the ARG primitive or belong to list of built-in variables.
HTTP_PROXY, HTTPS_PROXY, http_proxy, https_proxy, FTP_PROXY and NO_PROXY are built-in
variables that needn't be explicitly defined in Dockerfile to use this feature.
Signed-off-by: Madhav Puri <madhav.puri@gmail.com>
Builds where the base images have been resolved to trusted digest
references will now be tagged with the original tag reference from
the Dockerfile on a successful build.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Clean up tests to remove duplicate code
Add tests which run pull and create in an isolated configuration directory.
Add build test for untrusted tag
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Build cache uses pgk/tarsum to get a digest of content which is
ADD'd or COPY'd during a build. The builder has always used v0 of
the tarsum algorithm which includes mtimes however since the whole
file is hashed anyway, the mtime doesn't really provide any extra
information about whether the file has changed and many version
control tools like Git strip mtime from files when they are cloned.
This patch updates the build subsystem to use v1 of Tarsum which
explicitly ignores mtime when calculating a digest. Now ADD and
COPY will result in a cache hit if only the mtime and not the file
contents have changed.
NOTE: Tarsum is NOT a meant to be a cryptographically secure hash
function. It is a best-effort approach to determining if two sets of
filesystem content are different.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Change CLI error msg because it was too specific and didn't make sense
when there were errors not related to inaccessible files.
Removed some log.Error() calls since they're not really errors we should
log. Returning the error will be enough.
Closes: #13417
Signed-off-by: Doug Davis <dug@us.ibm.com>
Previous fix used %q which incorrectly go-escaped things. For example:
```
RUN echoo A \& B C
```
would result in the user seeing:
```
INFO[0000] The command '/bin/sh -c echoo A \\& B\tC' returned a non-zero code: 127
```
Note the double-\ and the \t instead of a tab character
The testcase had to double escape things due to logrus getting in the way
but I'm going to fix that in another PR because its a change to the UX.
Signed-off-by: Doug Davis <dug@us.ibm.com>
When RUN returns with a non-zero return code it prints the command
that was executed as a Go []string:
```
INFO[0000] The command &{[/bin/sh -c noop a1 a2]} returned a non-zero code: 127
```
instead it should look like this:
```
INFO[0000] The command "/bin/sh -c noop a1 a2" returned a non-zero code: 127
```
Signed-off-by: Doug Davis <dug@us.ibm.com>
This fixes an issue where the build output for the "Steps" would look like:
```
Step 1: RUN echo hi echo hi
```
instead of
```
Step 1: RUN echo hi
```
Also, I noticed that there were no checks to make sure invalid Dockerfile
cmd flags were caught on cmds that didn't use cmd flags at all. They would
have been caught on the cmds that had flags, but cmds that didn't bother
to add a new code for flags would have just ignored them. So, I added
checks to each cmd to flag it.
Added testcases for issues.
Signed-off-by: Doug Davis <dug@us.ibm.com>
Sebastiaan van Stijn
David Calavera
Zhang Wei
Darren Stahl
Aaron Lehmann
Antonio Murdaca
John Howard
Alexander Morozov
Riyaz Faizullabhoy
Lei Jitang
Doug Davis
Mauricio Garavaglia
Boaz Shuster
Tonis Tiigi
Justas Brazauskas
Tibor Vass
Aidan Hobson Sayers
Stefan Scherer
Brian Goff
Shijiang Wei
Antonio Murdaca
Jessica Frazelle
Phil Estes
Tibor Vass
Antonio Murdaca
Madhav Puri
Soshi Katsuta
Veres Lajos
Arnaud Porterie
Alexander Morozov
Josh Hawn
Michael Crosby
Derek McGowan
Shishir Mahajan
Ben Firshman
Vincent Demeester
Moysés Borges
Jessie Frazelle
Antonio Murdaca
Qiang Huang
buddhamagnet
David Mackey
Nathan LeClaire
Marianna
Srini Brahmaroutu