I can never get it to work for me when its just 3 seconds.
With this change it generates the OOM message around 17 seconds, but
I increased the timeout to 30 for people with slower machines
Signed-off-by: Doug Davis <dug@us.ibm.com>
Set cpuset to "0" so that it works on single core machines.
W/o this (and set to "1") we'll see something like this error
when running:
System error: write /cgroup/cpuset/docker/66689499bbd08cd8dccc9b7bfd1d6b34e85d73ce8c84d3c69b5e91944322da60/docker/79d7c548b58c85c4cfad6cd01eb7c3b30db254d1014c496137edd93ddc528a6f/cpuset.cpus: invalid argument"
Signed-off-by: Doug Davis <dug@us.ibm.com>
The validation script from #10681 is too pedantic, and does not handle
well situations like:
```
cat <<EOF # or <<-EOF
Whether the leading whitespace is stripped out or not by bash
it should still be considered as valid.
EOF
```
This reverts commit 4e65c1c319.
Signed-off-by: Tibor Vass <tibor@docker.com>
Add the capability to cancel the build by disconnecting the client.
This adds a `cancelled` channel which is used to signal that a build
should halt. The build is halted by sending a Kill signal and noticing
that the cancellation channel is closed.
This first pass implementation does not allow cancellation during a
pull, but that will come in a subsequent PR.
* Add documentation of cancellation to cli and API
* Protect job cancellation with sync.Once
* Add TestBuildCancelationKillsSleep
* Add test case for build cancellation of RUN statements.
Signed-off-by: Peter Waller <p@pwaller.net>
`TestBuildSpaces` is used to do string manipulation to redact timestamps
from compared cmd output. However it was assuming there is going to be
only one `time=".."` in the output --not the case on windows. Therefore
I wrote a regexp to redact all log timestamps.
Also the test used to have a copy/paste bug (see err1/err2). This fixes
that, too.
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
With this Dockerfile
```
FROM ubuntu
ENV ABC=""
VOLUME $ABC
```
It builds ok but then at run time I get this error:
FATA[0002] Error response from daemon: Cannot start container 8902b4a7aaf5c4e4b11a38070d392db465fa97ad88c91c8b38dda5ab8149ccac: [8] System error: no such file or directory
Because the Volume config shows "" as the path. This PR checks for "" as
the path and stops it at build time.
Signed-off-by: Doug Davis <dug@us.ibm.com>
This shows a warning message about adjusted file/directory permission bits
when the `docker build` cli command is executed on windows.
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
w/o this I would see:
```
unexpected file format for /proc/self/cgroup - ""
```
while running the cgroup tests
Signed-off-by: Doug Davis <dug@us.ibm.com>
Thanks to @tiborvass's review of another PR it was noticed that
we didn't have a testcase to make sure the cache checking for LABEL
was being done properly. This adds a test for it.
Signed-off-by: Doug Davis <dug@us.ibm.com>
Closes#10191
Allow `docker build` to set --cpu-shares, --cpuset, --memory,
--memory-swap for all containers created by the build.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
When the deamon starts up with log level set to INFO it will show something
like this:
```
INFO[0000] Loading containers: start.
................................................................
INFO[0000] Loading containers: done.
```
where the dots represent containers in the system.
When you run with log level set to "error" it will still show the dots
w/o the "Loading..." lines before and after which looks really odd.
This PR will fix it so that the dots are only shown IFF the "Loading..."
lines are also shown
Signed-off-by: Doug Davis <dug@us.ibm.com>
Currently the daemon will not stop on error because the serve API job is
blocking the channel wait for daemon init. A better way is to run the
blocking serve API job as a goroutine and make sure that error
notification gets back to the main daemon thread (using the already
existing channel) so that clean shutdown can occur on error.
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
Add ability to refer to an image by repository name and digest using the
format repository@digest. Works for pull, push, run, build, and rmi.
Signed-off-by: Andy Goldstein <agoldste@redhat.com>
Copied code from CmdSave into CmdExport. This should work, not an expert in the API calls being made. But it does make more sense to have a consistent export/save flag.
Signed-off-by: Joseph Kern <joseph.a.kern@gmail.com>
checkpoint before edits on the export functions
Signed-off-by: Joseph Kern <joseph.a.kern@gmail.com>
Added an --output flag to docker export and created tests.
Signed-off-by: Joseph Kern <joseph.a.kern@gmail.com>
White space cleanup.
Signed-off-by: Joseph Kern <joseph.a.kern@gmail.com>
Docker-DCO-1.1-Signed-off-by: Joseph Kern <jkern@semafour.net> (github: jfrazelle)
checkpoint before edits on the export functions
Signed-off-by: Joseph Kern <joseph.a.kern@gmail.com>
White space cleanup.
Signed-off-by: Joseph Kern <joseph.a.kern@gmail.com>
Added text to reflect a new output option for the export command.
Signed-off-by: Joseph Kern <joseph.a.kern@gmail.com>
Whitespace clean up
Signed-off-by: Joseph Kern <joseph.a.kern@gmail.com>
Added man page documentation for the new --output flag in export
Signed-off-by: Joseph Kern <joseph.a.kern@gmail.com>
This modifies iptables.Exists so that it must be called with an explicit
table and chain. This allows us (a) to generate an appropriate command
line for "iptables -C", which was not previously possible, and (b) it
allows us to limit our strings.Contains() search to just the table and
chain in question, preventing erroneous matches against unrelated rules.
Resolves#10781
Signed-off-by: Lars Kellogg-Stedman <lars@redhat.com>
Update pull code to consider any layer download or new tag as an update.
Update hello-world frozen image to be explicitly tagged as frozen, to not interfere with pull tests. The hello-world is used by pull tests because of its small size and there is no other official image with such a size.
fixes#11383
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Instead of just printing a warning and going on, this will generate
an error and stop processing.
This used to be part of #10561 but I decided it might need its own
independent discussion/PR as to not derail #10561.
Signed-off-by: Doug Davis <dug@us.ibm.com>
Save "LABEL" field in Dockerfile into image content.
This will allow a user to save user data into an image, which
can later be retrieved using:
docker inspect IMAGEID
I have copied this from the "Comment" handling in docker images.
We want to be able to add Name/Value data to an image to describe the image,
and then be able to use other tools to look at this data, to be able to do
security checks based on this data.
We are thinking about adding version names,
Perhaps listing the content of the dockerfile.
Descriptions of where the code came from etc.
This LABEL field should also be allowed to be specified in the
docker import --change LABEL:Name=Value
docker commit --change LABEL:Name=Value
Docker-DCO-1.1-Signed-off-by: Dan Walsh <dwalsh@redhat.com> (github: rhatdan)
As we started running CI machines on Azure cloud and Azure
doesn't have ICMP stack implemented by replacing
`ping 8.8.8.8` with `nslookup google.com`.
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
Fixes#11315
After rename occured the graphdb was updated but the container struct
was never commited back to disk, so on daemon restart it loads the old
name again.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This fixes the `docker events`-related tests as they have been
failing due to clock skew between CI machine and test daemon on
some other machine (even 1-2 seconds of diff causes races as
we pass local time to --since/--until).
If we're running in same host, we keep using time.Now(), otherwise
we read the system time of the daemon from `/info` endpoint.
Fixes pretty much all events-related tests on windows CI.
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
FROM scratch
ADD echo /
RUN [ "echo" ]
will die with
exec: "/bin/sh": stat /bin/sh: no such file or directory
Signed-off-by: Doug Davis <dug@us.ibm.com>
delete unecessary blank after "[OPTIONS] "
and add a blank to the log between options and signature.
To make the code style consistency.
Signed-off-by: Zen Lin(Zhinan Lin) <linzhinan@huawei.com>
This change enables `fakeGIT()` to use the new `fakeStorage`
server which is automatically starting a container on the remote test
daemon machine using the git repo directory (when requested).
Fixes the following tests:
- `TestBuildApiLowerDockerfile`
- `TestBuildApiBuildGitWithF`
- `TestBuildApiDoubleDockerfile` (skipped on windows: NTFS case-insensitive)
- `TestBuildFromGIT` (still needs local server)
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
Cgroup resources are host dependent, they should be in hostConfig.
For backward compatibility, we just copy it to hostConfig, and leave it in
Config for now, so there is no regressions, but the right way to use this
throught json is to put it in HostConfig, like:
{
"Hostname": "",
...
"HostConfig": {
"CpuShares": 512,
"Memory": 314572800,
...
}
}
As we will add CpusetMems, CpusetCpus is definitely a better name, but some
users are already using Cpuset in their http APIs, we also make it compatible.
The main idea is keep using Cpuset in Config Struct, and make it has the same
value as CpusetCpus, but not always, some scenarios:
- Users use --cpuset in docker command, it can setup cpuset.cpus and can
get Cpuset field from docker inspect or other http API which will get
config info.
- Users use --cpuset-cpus in docker command, ditto.
- Users use Cpuset field in their http APIs, ditto.
- Users use CpusetCpus field in their http APIs, they won't get Cpuset field
in Config info, because by then, they should already know what happens
to Cpuset.
Signed-off-by: Qiang Huang <h.huangqiang@huawei.com>
We should warn users who use the `--dns` command line option to point
DNS to a localhost address, either IPv4 or IPv6. Unless they have
specifically set up the container as a DNS server or are using
--net=host (which is why this should be allowed, but warned on because
those are pretty unique cases) a localhost address as a resolver will
not reach what they might expect (e.g. expecting it will hit localhost
on the Docker daemon/host).
Added a test for the message, and fixed up tests to separate stdout and
stderr that were using `--dns=127.0.0.1` to test the options.
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
Thanks to @ahmetalpbalkan for noticing... we had an old check in this
testcase that no longer applied (due to stuff being removing recently).
However, while in there I added a check to make sure that the file referenced
by the query parameter isn't used at all.
Signed-off-by: Doug Davis <dug@us.ibm.com>
Manifest is now generated during a v2 push, not relying on previously generated hashes. When pushing a layer, the hash is directly calculated from the tar contents which will be pushed. Computing the hash on push ensures that the hash contents always match what is seen by the registry. This also mitigates issues with tarsum differences and permits using pure SHA digests.
Additionally the new manifest function is moved to the unit tests since it is no longer called outside the tests.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Implemented a FakeStorage alternative that supports spinning
up a remote container on DOCKER_TEST_HOST to serve files over
an offline-compiled Go static web server image so that tests which
use URLs in Dockerfile can build them over at the daemon side.
`fakeStorage` function now automatically chooses if it should
use a local httptest.Server or a remote container.
This fixes the following tests when running against a remote
daemon:
- `TestBuildCacheADD`
- `TestBuildCopyWildcardNoFind`
- `TestBuildCopyWildcardCache`
- `TestBuildADDRemoteFileWithCache`
- `TestBuildADDRemoteFileWithoutCache`
- `TestBuildADDRemoteFileMTime`
- `TestBuildADDLocalAndRemoteFilesWithCache`
- `TestBuildADDLocalAndRemoteFilesWithoutCache`
- `TestBuildFromURLWithF`
- `TestBuildApiDockerFileRemote`
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
Windows CI fails to dial remote test host over tcp in the test cases where
we clear environment variables during `exec(dockerBinary, ...)` in the
absence of `SystemRoot` environment variable (typically points to `c:\windows`).
This fixes tests:
- `TestRunEnvironmentErase`
- `TestRunEnvironmentOverride`
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
`TestCliProxyDisableProxyUnixSock` verifies that HTTP_PROXY env var is
used when DOCKER_HOST is not set and cli connects to `unix://...` by
default.
In the windows/darwin CLI CIs, daemon is on remote host and we always
connect via `tcp://`, therefore this test fails very straight. Skipping
this test to run only on "same host daemon" assumption.
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
TestRunCidFileCleanupIfEmpty fails on windows/mac because the test runs
the command `docker run scratch` and it gives the following error:
Unable to find image 'scratch:latest' locally
Pulling repository scratch
511136ea3c5a: Download complete
FATA[0004] 'scratch' is a reserved name
I am not entirely sure if this is a test issue or not but I had a quick
workaround by creating another image using `FROM scratch` and using that.
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
The tests end up overwriting the `dockerfile` with `Dockerfile` since
windows filesystems are case-insensitive. The following methods are
skipped:
- TestBuildRenamedDockerfile
- TestBuildFromMixedcaseDockerfile
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
Added link to original issue and clarified text so someone without any
background on the original issue can understand why the test exists.
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
The overlay filesystem does not support inotify at this time. The
resolv.conf updater test was passing on overlay-based Jenkins because of
a fluke--because it was DIND, /etc/resolv.conf on the "host" was really
a bind-mounted resolv.conf from the outer container, which means a watch
directly on that file worked as it was not overlay backed. The new test
(from #10703) unmounts the bind-mounted copy to test create and modify
code-paths, which caused us to hit the issue.
This PR also adds a note to the docs about the lack of auto-update when
using the overlay storage driver.
See https://lkml.org/lkml/2012/2/28/223 for more info on inotify and
overlay.
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
We could use EXPOSE ip:hostPort:containerPort,
but actually it did as EXPOSE ::containerPort
commit 2275c833 already warned user on daemon side.
This patch will print warning message on client side.
Signed-off-by: Chen Hanxiao <chenhanxiao@cn.fujitsu.com>
This change modifies the chmod bits of build context archives built on
windows to preserve the execute bit and remove the r/w bits from
grp/others.
Also adjusted integ-cli tests to verify permissions based on the platform
the tests are running.
Fixes#11047.
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
See: https://github.com/docker/docker/issues/10867
While looking at #10867 I noticed that the error message generated for
a blank image ID wasn't very helpful so this fixes that.
Signed-off-by: Doug Davis <dug@us.ibm.com>
Closes#10807
Adds support for `dockerfile` ONLY when `Dockerfile` can't be found.
If we're building from a Dockerfile via stdin/URL then always download
it a `Dockerfile` and ignore the -f flag.
Signed-off-by: Doug Davis <dug@us.ibm.com>
and add a testcase to catch this in the future.
While in there I also:
- removed extra periods from the few options that had them (new test)
- made the --filter option consistent across all command
Signed-off-by: Doug Davis <dug@us.ibm.com>
If the tests are running outside a container (i.e.
executed without `make test`), we are using a `busybox`
pulled from Docker Hub (not jpatezzo's docker-busybox).
That one adds an extra
`PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin`
env var all the time and that messes the test `TestCommitChange`.
That's currently breaking the windows CI.
I'm keeping the same PATH here but making it explicit
so that it's always set and we verify what we set. It's actually the same
thing if I set `ENV PATH foo` here but I thought it may lead to some
problems hard to debug in the future.
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
Some pull/push tests are launching `registry-v2`
binary which is compiled and installed if the tests
are running in-container through `make test-integration-cli`.
However, registry is not supported to run on non-linux
platforms and we can't really spin up any registry-v2
containers in the remote DOCKER_TEST_HOST at this point.
Just skipping those with the new TestRequirement called
`RegistryHosting`.
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
`TestRunBindMounts` requires daemon to be on the same host.
Running this cli test on Linux is fair enough coverage for
this functionality and we can skip this for platforms where
daemon cannot run side-by-side with the cli for now.
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
Use the new `runCommandPipelineWithOutput` helper to
remove bash dependency required for piping in
`TestSaveDirectoryPermissions`.
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
Move the stats structs from the api/stats package into a new package
api/types that will contain all the api structs for docker's api request
and responses.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
In addition to config env, `commit` now will also accepts a `changes` env which
is a string contains new-line separated Dockerfile instructions.
`commit` will evaluate `changes` into `runconfig.Config` and merge it with
`config` env, and then finally commit a new image with the changed config
Docker-DCO-1.1-Signed-off-by: Daniel, Dao Quang Minh <dqminh89@gmail.com> (github: dqminh)
Docker-DCO-1.1-Signed-off-by: Daniel, Dao Quang Minh <dqminh89@gmail.com> (github: rhatdan)
Some integration-cli tests assume daemon and cli are running
on the same machine and therefore they examine side effects
of executed docker commands on docker host by reading files
or running other sort of commands.
In case of windows/darwin CLI tests these provide little
or no value and should be OK to skip.
List of skipped tests:
- `TestContainerNetworkMode`
- `TestCpVolumePath`
- `TestCreateVolumesCreated`
- `TestBuildContextCleanup`
- `TestBuildContextCleanupFailedBuild`
- `TestLinksEtcHostsContentMatch`
- `TestRmContainerWithRemovedVolume`
- `TestRunModeIpcHost`
- `TestRunModeIpcContainer`
- `TestRunModePidHost`
- `TestRunNetHost`
- `TestRunDeallocatePortOnMissingIptablesRule`
- `TestRunPortInUse`
- `TestRunPortProxy`
- `TestRunMountOrdering`
- `TestRunModeHostname`
- `TestRunDnsDefaultOptions`
- `TestRunDnsOptionsBasedOnHostResolvConf`
- `TestRunResolvconfUpdater`
- `TestRunVolumesNotRecreatedOnStart`
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
Skip tests based on remote daemon's exec support (to exclude
these tests from `make test` ran in LXC case). Makes use of
`test_no_exec` build tag passed by build scripts.
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
Shout out to @estesp for the idea. Some use cases of
`readContainerFile` can be replaced with `docker exec $id cat $file`.
This helper method can eliminate the requirement that
host/cli should be on the same machine.
TestRunMutableNetworkFiles and TestRunResolvconfUpdater still
need to access the docker host filesystem as they modify
the file directly from there assuming cli and daemon are
on the same machine.
This fixes TestLinksUpdateOnRestart and TestLinksHostsFilesInject
for Windows CI.
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
This fixes a few misuses of `deleteAllContainers()` cleanup
method in integration-cli suite by moving call to the
beginning of the method and guaranteeing their execution
(including panics) with `defer`s.
Also added some forgotten cleanup calls while I'm at it.
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
`docker build -q .` where Dockerfile contains a RUN cmd will hang on the
RUN. It waits for the output stream to close but because of -q we never
attached to the container and end up waiting forever.
The fact that no one noticed this tells me that people may not actually
use -q and if so I wonder if it would make sense to make -q work the may
it does for other commands (like `docker ps`) and make it so it only
shows the container ID at the end. A -q/quiet option that only hides the
container RUN output apparently isn't really that useful since no one is
using it. See: https://github.com/docker/docker/issues/4094
Signed-off-by: Doug Davis <dug@us.ibm.com>
TestBuildAddBadLinks used to build a path by
concenating unix-style forward slashes. Fixed that
by providing a windows-equivalent using `runtime.GOOS`.
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
Although this doesn't fix the test (os.Symlink is not yet
implemented for Windows), this prevents unix-style paths
from being passed to os.Symlink. Also makes code cleaner
for linux.
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
Skipping some of the tests closely tied to running in a
unix environment. Windows does not support chmod/chown
and this causes some tests to fail creating desired
behavior.
- `TestBuildWithInaccessibleFilesInContext`: uses chown/chmod
- `TestBuildDockerfileOutsideContext`: uses os.Symlink, not implemented on
windows
- `TestCpUnprivilegedUser`: uses chmod, and requires 'unprivilegeduser'
created by Dockerfile (and thus requires to run inside container)
- `TestBuildChownSingleFile`: uses chown
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
NetworkManager updates resolve.conf by replacing the current file
with an updated one. This change enables docker to listen for these
events.
Signed-off-by: Sami Wagiaalla <swagiaal@redhat.com>
The most obvious use case is when one wants to make sure as fast
as possible that a command is a valid Dockerfile command.
Signed-off-by: kargakis <kargakis@users.noreply.github.com>
This fixes `TestVolumesNoCopyData` for test execution on
windows by passing a unix-style path as volume even though
it's running on windows.
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
estesp noticed that when $HOME is / the ~ substitutions messes up
becuase it tries to replace all paths that start with "/" with "~".
This fixes it so that it will only replace it when $HOME isn't "/".
Signed-off-by: Doug Davis <dug@us.ibm.com>
André Martins
Alexander Morozov
Doug Davis
Deng Guangxing
Mabin
Jessie Frazelle
Brian Goff
Michael Crosby
Yuan Sun
Vincent Demeester
Lei Jitang
Srini Brahmaroutu
Peter Choi
Pradeep Chhetri
Jake Champlin
paul
Doug Davis
Tibor Vass
Jessica Frazelle
Arnaud Porterie
Alexander Morozov
Antonio Murdaca
Tibor Vass
George MacRorie
bobby abbott
Peter Waller
Ahmet Alp Balkan
Mitch Capper
Vishnu Kannan
Tibor Vass
Phil Estes
Andy Goldstein
Phil Estes
Joseph Kern
Lars Kellogg-Stedman
Derek McGowan
Qiang Huang
Darren Shepherd
Dan Walsh
Zen Lin(Zhinan Lin)
moxiegirl
Tianon Gravi
Chen Hanxiao
Michael Crosby
Daniel, Dao Quang Minh
Abin Shahab
Ahmet Alp Balkan
Sami Wagiaalla
kargakis