As we started running CI machines on Azure cloud and Azure
doesn't have ICMP stack implemented by replacing
`ping 8.8.8.8` with `nslookup google.com`.
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
Cgroup resources are host dependent, they should be in hostConfig.
For backward compatibility, we just copy it to hostConfig, and leave it in
Config for now, so there is no regressions, but the right way to use this
throught json is to put it in HostConfig, like:
{
"Hostname": "",
...
"HostConfig": {
"CpuShares": 512,
"Memory": 314572800,
...
}
}
As we will add CpusetMems, CpusetCpus is definitely a better name, but some
users are already using Cpuset in their http APIs, we also make it compatible.
The main idea is keep using Cpuset in Config Struct, and make it has the same
value as CpusetCpus, but not always, some scenarios:
- Users use --cpuset in docker command, it can setup cpuset.cpus and can
get Cpuset field from docker inspect or other http API which will get
config info.
- Users use --cpuset-cpus in docker command, ditto.
- Users use Cpuset field in their http APIs, ditto.
- Users use CpusetCpus field in their http APIs, they won't get Cpuset field
in Config info, because by then, they should already know what happens
to Cpuset.
Signed-off-by: Qiang Huang <h.huangqiang@huawei.com>
We should warn users who use the `--dns` command line option to point
DNS to a localhost address, either IPv4 or IPv6. Unless they have
specifically set up the container as a DNS server or are using
--net=host (which is why this should be allowed, but warned on because
those are pretty unique cases) a localhost address as a resolver will
not reach what they might expect (e.g. expecting it will hit localhost
on the Docker daemon/host).
Added a test for the message, and fixed up tests to separate stdout and
stderr that were using `--dns=127.0.0.1` to test the options.
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
Windows CI fails to dial remote test host over tcp in the test cases where
we clear environment variables during `exec(dockerBinary, ...)` in the
absence of `SystemRoot` environment variable (typically points to `c:\windows`).
This fixes tests:
- `TestRunEnvironmentErase`
- `TestRunEnvironmentOverride`
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
TestRunCidFileCleanupIfEmpty fails on windows/mac because the test runs
the command `docker run scratch` and it gives the following error:
Unable to find image 'scratch:latest' locally
Pulling repository scratch
511136ea3c5a: Download complete
FATA[0004] 'scratch' is a reserved name
I am not entirely sure if this is a test issue or not but I had a quick
workaround by creating another image using `FROM scratch` and using that.
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
Added link to original issue and clarified text so someone without any
background on the original issue can understand why the test exists.
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
The overlay filesystem does not support inotify at this time. The
resolv.conf updater test was passing on overlay-based Jenkins because of
a fluke--because it was DIND, /etc/resolv.conf on the "host" was really
a bind-mounted resolv.conf from the outer container, which means a watch
directly on that file worked as it was not overlay backed. The new test
(from #10703) unmounts the bind-mounted copy to test create and modify
code-paths, which caused us to hit the issue.
This PR also adds a note to the docs about the lack of auto-update when
using the overlay storage driver.
See https://lkml.org/lkml/2012/2/28/223 for more info on inotify and
overlay.
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
`TestRunBindMounts` requires daemon to be on the same host.
Running this cli test on Linux is fair enough coverage for
this functionality and we can skip this for platforms where
daemon cannot run side-by-side with the cli for now.
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
Some integration-cli tests assume daemon and cli are running
on the same machine and therefore they examine side effects
of executed docker commands on docker host by reading files
or running other sort of commands.
In case of windows/darwin CLI tests these provide little
or no value and should be OK to skip.
List of skipped tests:
- `TestContainerNetworkMode`
- `TestCpVolumePath`
- `TestCreateVolumesCreated`
- `TestBuildContextCleanup`
- `TestBuildContextCleanupFailedBuild`
- `TestLinksEtcHostsContentMatch`
- `TestRmContainerWithRemovedVolume`
- `TestRunModeIpcHost`
- `TestRunModeIpcContainer`
- `TestRunModePidHost`
- `TestRunNetHost`
- `TestRunDeallocatePortOnMissingIptablesRule`
- `TestRunPortInUse`
- `TestRunPortProxy`
- `TestRunMountOrdering`
- `TestRunModeHostname`
- `TestRunDnsDefaultOptions`
- `TestRunDnsOptionsBasedOnHostResolvConf`
- `TestRunResolvconfUpdater`
- `TestRunVolumesNotRecreatedOnStart`
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
This fixes a few misuses of `deleteAllContainers()` cleanup
method in integration-cli suite by moving call to the
beginning of the method and guaranteeing their execution
(including panics) with `defer`s.
Also added some forgotten cleanup calls while I'm at it.
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
NetworkManager updates resolve.conf by replacing the current file
with an updated one. This change enables docker to listen for these
events.
Signed-off-by: Sami Wagiaalla <swagiaal@redhat.com>
This fixes `TestVolumesNoCopyData` for test execution on
windows by passing a unix-style path as volume even though
it's running on windows.
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
This change enables TestRunSetMacAddress to run on windows
without `bash` dependency. Also `defer`red call of cleanup
method.
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
Or else we can violate array range boundaries in:
out = strings.Split(out, ":")[1]
and get runtime error.
We got this runtime error when run TestRunPortFromDockerRangeInUse
Somehow docker goes silently if it cannot publish port because
of no bridge.
Signed-off-by: Pavel Tikhomirov <ptikhomirov@parallels.com>
For Windows, we run integration-cli with DOCKER_TEST_HOST env var b/c
daemon is on some remote machine. This keeps the DOCKER_HOST set by
bash scripts in the env.
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
Handles missed comments in PR#10622 and adds an API test to validate
error returned properly for duplicate bind mounts for the same
container target path.
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
Addresses: #10618
Given that the user has no notification that they tried to bind mount
different directories on the same target in the container, this errors
out in that case, without changing the current code allowing for
--volumes-from to trump -v/VOLUME specifications.
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
The cmd.Lookup should be "-attach" and not "attach", as seen in
docker/docker/runconfig/parse.go
Signed-off-by: André Martins <martins@noironetworks.com>
This fixes various tests by checking for non zero exit code, accounting for lxc-specific base-diffs, and by removing lxc specific environment vars.
It also adds the --share-ipc option to lxc-start for shared ipc namespaces.
Signed-off-by: Abin Shahab <ashahab@altiscale.com> (github: ashahab-altiscale)
Docker-DCO-1.1-Signed-off-by: Abin Shahab <ashahab@altiscale.com> (github: ashahab-altiscale)
Addresses #5811
This cleans up an error in the logic which removes localhost resolvers
from the host resolv.conf at container creation start time. Specifically
when the determination is made if any nameservers are left after
removing localhost resolvers, it was using a string match on the word
"nameserver", which could have been anywhere (including commented out)
leading to incorrect situations where no nameservers were left but the
default ones were not added.
This also adds some complexity to the regular expressions for finding
nameservers in general, as well as matching on localhost resolvers due
to the recent addition of IPv6 support. Because of IPv6 support now
available in the Docker daemon, the resolvconf code is now aware of
IPv6 enable/disable state and uses that for both filter/cleaning of
nameservers as well as adding default Google DNS (IPv4 only vs. IPv4
and IPv6 if IPv6 enabled). For all these changes, tests have been
added/strengthened to test these additional capabilities.
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
See #10141 for more info, but the main point of this is to make sure
that if you do "docker run -e FOO ..." that FOO from the current env
is passed into the container. This means that if there's a value, its
set. But it also means that if FOO isn't set then it should be unset in
the container too - even if it has to remove it from the env. So,
unset HOSTNAME
docker run -e HOSTNAME busybox env
should _NOT_ show HOSTNAME in the list at all
Closes#10141
Signed-off-by: Doug Davis <dug@us.ibm.com>
To run shell(and not exit), lxc needs STDIN. Without STDIN open, it will exit 0.
Signed-off-by: Abin Shahab <ashahab@altiscale.com> (github: ashahab-altiscale)
Fixes#9709
In cases where the volumes-from container is removed and the consuming
container is restarted, docker was trying to re-apply volumes from that
now missing container, which is uneccessary since the volumes are
already applied.
Also cleaned up the volumes-from parsing function, which was doing way more than
it should have been.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Sending capability ids instead of capability names ot LXC for --cap-add and --cap-drop.
Also fixed tests.
Docker-DCO-1.1-Signed-off-by: Abin Shahab <ashahab@altiscale.com> (github: ashahab-altiscale)
Add a --readonly flag to allow the container's root filesystem to be
mounted as readonly. This can be used in combination with volumes to
force a container's process to only write to locations that will be
persisted. This is useful in many cases where the admin controls where
they would like developers to write files and error on any other
locations.
Closes#7923Closes#8752
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Only modifies non-running containers resolv.conf bind mount, and only if
the container has an unmodified resolv.conf compared to its contents at
container start time (so we don't overwrite manual/automated changes
within the container runtime). For containers which are running when
the host resolv.conf changes, the update will only be applied to the
container version of resolv.conf when the container is "bounced" down
and back up (e.g. stop/start or restart)
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
Forbid `docker run -t` with a redirected stdin (such as `echo test |
docker run -ti busybox cat`). Forbid `docker exec -t` with a redirected
stdin. Forbid `docker attach` with a redirect stdin toward a tty enabled
container.
Signed-off-by: Arnaud Porterie <arnaud.porterie@docker.com>
Sometimes other programs can bind on ports from our range, so we just
skip this ports on allocation.
Fixes#9293
Probably fixes#8714
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
I also needed to add a mflag.IsSet() function that allows you to check
to see if a certain flag was actually specified on the cmd line.
Per #9221 - also tweaked the docs to fix a typo.
Closes#9221
Signed-off-by: Doug Davis <dug@us.ibm.com>
This adds an integration test for checking that the network namespace
fds are the same when a container joins another container's network
namespace.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Common patterns:
- Multiple images were built with same name but only one cleanup.
- Containers were deleted after images.
- Images not removed after retagging.
Signed-off-by: Tõnis Tiigi <tonistiigi@gmail.com> (github: tonistiigi)
Next steps, in another PR, would be:
- make all logging go through the logrus stuff
- I'd like to see if we can remove the env var stuff (like DEBUG) but we'll see
Closes#5198
Signed-off-by: Doug Davis <dug@us.ibm.com>
Some workloads rely on IPC for communications with other processes. We
would like to split workloads between two container but still allow them
to communicate though shared IPC.
This patch mimics the --net code to allow --ipc=host to not split off
the IPC Namespace. ipc=container:CONTAINERID to share ipc between containers
If you share IPC between containers, then you need to make sure SELinux labels
match.
Docker-DCO-1.1-Signed-off-by: Dan Walsh <dwalsh@redhat.com> (github: rhatdan)
Never close attached stream before both stdout and stderr have written
all their buffered contents. Remove stdinCloser because it is not needed
any more as the stream is closed anyway after attach has finished.
Fixes#3631
Signed-off-by: Andy Goldstein <agoldste@redhat.com>
Stable IPs causes some regressions in the way people use Docker, see GH#8493.
Reverting it for 1.3, we'll enable it back for the next release.
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
When a container is restarted all the volume configs are parsed again.
Even if the volume was already handled in a previous start it was still
calling "FindOrCreateVolume" on the volume repo causing a new volume to
be created.
This wasn't being detected because as part of the mount initialization
it checks to see if the the _mount_ was already initialized, but this
happens after the parsing of the configs.
So a check is added during parsing to skip a volume which was already
created for that container.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
When stdout/stderr is closed prematurely, the proxy's writes to stdout/stderr
(i.e. `log.Errorf/log.Printf`) will returns with EPIPE error, and go runtime
will terminate the proxy when stdout/stderr writes trigger 10 EPIPE errors.
instead of using stdout/stderr as the status handler, we pass an extra file to
the child process and write `0\n` or `1\nerror message` to it and close it
after. This allow the child process to handle stdout/stderr as normal.
Docker-DCO-1.1-Signed-off-by: Daniel, Dao Quang Minh <dqminh89@gmail.com> (github: dqminh)
Prior to the volumes re-factor, data was not being copied on
volumes-from or host-mounted volumes.
After the re-factor, data was being copied for volumes-from.
This reverts this unintentional change in behavior.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Needed to check if the mode was invalid and return error, not valid and
return error.
This didn't get picked up because the existing integration-cli tests
were all either expecting errors when a valid mode was passed in (e.g.
"ro" passed in, we expected an error because it was testing write). So
modified a test which was testing for "rw" to actually pass in "rw"
instead of assuming the "rw"
Docker-DCO-1.1-Signed-off-by: Brian Goff <bgoff@cpuguy83-mbp.home> (github: cpuguy83)
This change will allocate network settings (IP and public ports) at
container creation rather than start and keep them throughout the
lifetime of the container (i.e. until it gets destroyed) instead of
discarding them when the container is stopped.
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
volumes.Get was not checking for symlinked paths meanwhile when adding a
new volume it was following the symlink.
So when trying to use a bind-mount that is a symlink, the volume is
added with the correct path, but when another container tries to use the
same volume it got a "Volume exists" error because volumes.Get returned
nil and as such attempted to create a new volume.
Docker-DCO-1.1-Signed-off-by: Brian Goff <cpuguy83@gmail.com> (github: cpuguy83)
This adds a --add-host host:ip flag which appends lines to /etc/hosts. This is needed in places where you want the container to get a different name resolution than it would through DNS. This was submitted before as #5525, closed, and now I am re-opening. It has come up 2 or 3 times in the last couple days.
Signed-off-by: Tim Hockin <thockin@google.com>
We add a --device flag which can be used like:
docker run --device /dev/sda:/dev/xvda:rwm ubuntu /bin/bash
To allow the container to have read write permissions to access the host's /dev/sda via a node named /dev/xvda in the container.
Note: Much of this code was written by Dinesh Subhraveti dineshs@altiscale.com (github: dineshs-altiscale) and so he deserves a ton of credit.
Docker-DCO-1.1-Signed-off-by: Timothy <timothyhobbs@seznam.cz> (github: timthelion)
In that case /etc/resolv.conf will be generated with no search
option. Usage: --dns-search=.
Docker-DCO-1.1-Signed-off-by: Fabio Falci <fabiofalci@gmail.com> (github: fabiofalci)
We discussed this at the docker plumbers meetup and for tools and
working on the system for things like boot2docker and coreos this is
needed. You can already bypass this check so we felt it is ok to start
allowing this feature.
Docker-DCO-1.1-Signed-off-by: Michael Crosby <michael@docker.com> (github: crosbymichael)
We now have one place that keeps track of (most) devices that are allowed and created within the container. That place is pkg/libcontainer/devices/devices.go
This fixes several inconsistencies between which devices were created in the lxc backend and the native backend. It also fixes inconsistencies between wich devices were created and which were allowed. For example, /dev/full was being created but it was not allowed within the cgroup. It also declares the file modes and permissions of the default devices, rather than copying them from the host. This is in line with docker's philosphy of not being host dependent.
Docker-DCO-1.1-Signed-off-by: Timothy Hobbs <timothyhobbs@seznam.cz> (github: https://github.com/timthelion)
It has been pointed out that some files in /proc and /sys can be used
to break out of containers. However, if those filesystems are mounted
read-only, most of the known exploits are mitigated, since they rely
on writing some file in those filesystems.
This does not replace security modules (like SELinux or AppArmor), it
is just another layer of security. Likewise, it doesn't mean that the
other mitigations (shadowing parts of /proc or /sys with bind mounts)
are useless. Those measures are still useful. As such, the shadowing
of /proc/kcore is still enabled with both LXC and native drivers.
Special care has to be taken with /proc/1/attr, which still needs to
be mounted read-write in order to enable the AppArmor profile. It is
bind-mounted from a private read-write mount of procfs.
All that enforcement is done in dockerinit. The code doing the real
work is in libcontainer. The init function for the LXC driver calls
the function from libcontainer to avoid code duplication.
Docker-DCO-1.1-Signed-off-by: Jérôme Petazzoni <jerome@docker.com> (github: jpetazzo)
Without this patch, containers inherit the open file descriptors of the daemon, so my "exec 42>&2" allows us to "echo >&42 some nasty error with some bad advice" directly into the daemon log. :)
Also, "hack/dind" was already doing this due to issues caused by the inheritance, so I'm removing that hack too since this patch obsoletes it by generalizing it for all containers.
Docker-DCO-1.1-Signed-off-by: Andrew Page <admwiggin@gmail.com> (github: tianon)
This also migrates the volumes from integration tests into the new cli
integration test framework.
Docker-DCO-1.1-Signed-off-by: Michael Crosby <michael@crosbymichael.com> (github: crosbymichael)
As explained in https://github.com/dotcloud/docker/issues/4979
--volumes-from fails with ENOFILE errors.
This is because the code tries to look at the "from" volume without
ensuring that it is mounted yet. We fix this by mounting the containers
before stating in it.
Also includes a regression test.
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
Ahmet Alp Balkan
Qiang Huang
Jessie Frazelle
Phil Estes
Lei Jitang
Phil Estes
Jessica Frazelle
Sami Wagiaalla
Steve Francia
Pavel Tikhomirov
Michael Crosby
Tibor Vass
André Martins
Abin Shahab
Alexander Morozov
Doug Davis
Brian Goff
Michael Crosby
Alexander Morozov
Arnaud Porterie
Srini Brahmaroutu
Ian Bishop
Yohei Ueda
Erik Hollensbe
Daehyeok Mun
unclejack
Tonis Tiigi
Dan Walsh
Malte Janduda
Andy Goldstein
Andrea Luzzardi
Daniel, Dao Quang Minh
Brian Goff
Tibor Vass
Victor Vieux
unclejack
Tim Hockin
Vishnu Kannan
Victor Vieux
Michael Crosby
Tianon Gravi
Victor Vieux
Matthew Heon
Timothy
Fabio Falci
Michael Crosby
Brandon Philips
Jérôme Petazzoni
Alexander Larsson