This PR adds a "request ID" to each event generated, the 'docker events'
stream now looks like this:
```
2015-09-10T15:02:50.000000000-07:00 [reqid: c01e3534ddca] de7c5d4ca927253cf4e978ee9c4545161e406e9b5a14617efb52c658b249174a: (from ubuntu) create
```
Note the `[reqID: c01e3534ddca]` part, that's new.
Each HTTP request will generate its own unique ID. So, if you do a
`docker build` you'll see a series of events all with the same reqID.
This allow for log processing tools to determine which events are all related
to the same http request.
I didn't propigate the context to all possible funcs in the daemon,
I decided to just do the ones that needed it in order to get the reqID
into the events. I'd like to have people review this direction first, and
if we're ok with it then I'll make sure we're consistent about when
we pass around the context - IOW, make sure that all funcs at the same level
have a context passed in even if they don't call the log funcs - this will
ensure we're consistent w/o passing it around for all calls unnecessarily.
ping @icecrime @calavera @crosbymichael
Signed-off-by: Doug Davis <dug@us.ibm.com>
Using @mavenugo's patch for enabling the libcontainer pre-start hook to
be used for network namespace initialization (correcting the conflict
with user namespaces); updated the boolean check to the more generic
SupportsHooks() name, and fixed the hook state function signature.
Signed-off-by: Madhu Venugopal <madhu@docker.com>
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
This changeset creates /dev/shm and /dev/mqueue mounts for each container under
/var/lib/containers/<id>/ and bind mounts them into the container. When --ipc:container<id/name>
is used, then the /dev/shm and /dev/mqueue of the ipc container are used instead of creating
new ones for the container.
Signed-off-by: Mrunal Patel <mrunalp@gmail.com>
Docker-DCO-1.1-Signed-off-by: Dan Walsh <dwalsh@redhat.com> (github: rhatdan)
(cherry picked from commit d88fe447df)
This changeset creates /dev/shm and /dev/mqueue mounts for each container under
/var/lib/containers/<id>/ and bind mounts them into the container. When --ipc:container<id/name>
is used, then the /dev/shm and /dev/mqueue of the ipc container are used instead of creating
new ones for the container.
Signed-off-by: Mrunal Patel <mrunalp@gmail.com>
Docker-DCO-1.1-Signed-off-by: Dan Walsh <dwalsh@redhat.com> (github: rhatdan)
Signed-off-by: Stefan J. Wernli <swernli@microsoft.com>
Windows: add support for images stored in alternate location.
Signed-off-by: Stefan J. Wernli <swernli@microsoft.com>
Introduce a write denial for files at the root of /proc.
This prohibits root users from performing a chmod of those
files. The rules for denials in proc are also cleaned up,
making the rules better match their targets.
Locally tested on:
- Ubuntu precise (12.04) with AppArmor 2.7
- Ubuntu trusty (14.04) with AppArmor 2.8.95
Signed-off-by: Eric Windisch <eric@windisch.us>
This reverts commit 40b71adee3.
Original commit (for which this is effectively a rebased version) is
72a500e9e5 and was provided by Lei Jitang
<leijitang@huawei.com>.
Signed-off-by: Tim Dettrick <t.dettrick@uq.edu.au>
The 'deny ptrace' statement was supposed to only ignore
ptrace failures in the AUDIT log. However, ptrace was implicitly
allowed from unconfined processes (such as the docker daemon and
its integration tests) due to the abstractions/base include.
This rule narrows the definition such that it will only ignore
the failures originating inside of the container and will not
cause denials when the daemon or its tests ptrace inside processes.
Introduces positive and negative tests for ptrace /w apparmor.
Signed-off-by: Eric Windisch <eric@windisch.us>
Integration tests were failing due to proc filter behavior
changes with new apparmor policies.
Also include the missing docker-unconfined policy resolving
potential startup errors. This policy is complain-only so
it should behave identically to the standard unconfined policy,
but will not apply system path-based policies within containers.
Signed-off-by: Eric Windisch <eric@windisch.us>
Will attempt to load profiles automatically. If loading fails
but the profiles are already loaded, execution will continue.
A hard failure will only occur if Docker cannot load
the profiles *and* they have not already been loaded via
some other means.
Also introduces documentation for AppArmor.
Signed-off-by: Eric Windisch <eric@windisch.us>
By using the 'unconfined' policy for privileged
containers, we have inherited the host's apparmor
policies, which really make no sense in the
context of the container's filesystem.
For instance, policies written against
the paths of binaries such as '/usr/sbin/tcpdump'
can be easily circumvented by moving the binary
within the container filesystem.
Fixes GH#5490
Signed-off-by: Eric Windisch <eric@windisch.us>
It's introduced in
68ba5f0b69 (Execdriver implementation on new libcontainer API)
But I don't see reson why we need it.
Signed-off-by: Qiang Huang <h.huangqiang@huawei.com>
The automatic installation of AppArmor policies prevents the
management of custom, site-specific apparmor policies for the
default container profile. Furthermore, this change will allow
a future policy for the engine itself to be written without demanding
the engine be able to arbitrarily create and manage AppArmor policies.
- Add deb package suggests for apparmor.
- Ubuntu postinst use aa-status & fix policy path
- Add the policies to the debian packages.
- Add apparmor tests for writing proc files
Additional restrictions against modifying files in proc
are enforced by AppArmor. Ensure that AppArmor is preventing
access to these files, not simply Docker's configuration of proc.
- Remove /proc/k?mem from AA policy
The path to mem and kmem are in /dev, not /proc
and cannot be restricted successfully through AppArmor.
The device cgroup will need to be sufficient here.
- Load contrib/apparmor during integration tests
Note that this is somewhat dirty because we
cannot restore the host to its original configuration.
However, it should be noted that prior to this patch
series, the Docker daemon itself was loading apparmor
policy from within the tests, so this is no dirtier or
uglier than the status-quo.
Signed-off-by: Eric Windisch <eric@windisch.us>
Replaced github.com/docker/libcontainer with
github.com/opencontainers/runc/libcontaier.
Also I moved AppArmor profile generation to docker.
Main idea of this update is to fix mounting cgroups inside containers.
After updating docker on CI we can even remove dind.
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
Memory swappiness option takes 0-100, and helps to tune swappiness
behavior per container.
For example, When a lower value of swappiness is chosen
the container will see minimum major faults. When no value is
specified for memory-swappiness in docker UI, it is inherited from
parent cgroup. (generally 60 unless it is changed).
Signed-off-by: Raghavendra K T <raghavendra.kt@linux.vnet.ibm.com>
Libcontainer already supported mount container's own cgroup into
container, with this patch, we can see container's own cgroup info
in container.
Signed-off-by: Qiang Huang <h.huangqiang@huawei.com>
If a container is read-only, also set /proc, /sys,
& /dev to read-only. This should apply to both privileged and
unprivileged containers.
Note that when /dev is read-only, device files may still be
written to. This change will simply prevent the device paths
from being modified, or performing mknod of new devices within
the /dev path.
Tests are included for all cases. Also adds a test to ensure
that /dev/pts is always mounted read/write, even in the case of a
read-write rootfs. The kernel restricts writes here naturally and
bad things will happen if we mount it ro.
Signed-off-by: Eric Windisch <eric@windisch.us>
By default, the cgroup setting in libcontainer's configs.Cgroup for
memory swappiness will default to 0, which is a valid choice for memory
swappiness, but that means by default every container's memory
swappiness will be set to zero instead of the default 60, which is
probably not what users are expecting.
When the swappiness UI PR comes into Docker, there will be docker run
controls to set this per container, but for now we want to make sure
*not* to change the default, as well as work around an older kernel
issue that refuses to allow it to be set when cgroup hiearchies are in
use.
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
This is breaking various setups where the host's rootfs is mount shared
correctly and breaks live migration with bind mounts.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
* Don't AllocateNetwork when network is disabled
* Don't createNetwork in execdriver when network is disabled
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
As part of this some generic packages like iptables, etchosts and resolvconf
have also been moved to libnetwork. Even though they can still be
consumed in a generic fashion they will reside and be maintained
from within the libnetwork project.
Signed-off-by: Jana Radhakrishnan <mrjana@docker.com>
- Updated Dockerfile to satisfy libnetwork GOPATH requirements.
- Reworked daemon to allocate network resources using libnetwork.
- Reworked remove link code to also update network resources in libnetwork.
- Adjusted the exec driver command population to reflect libnetwork design.
- Adjusted the exec driver create command steps.
- Updated a few test cases to reflect the change in design.
- Removed the dns setup code from docker as resolv.conf is entirely managed
in libnetwork.
- Integrated with lxc exec driver.
Signed-off-by: Jana Radhakrishnan <mrjana@docker.com>
Generation based on CAP_LAST_CAP, I hardcoded
capability.CAP_BLOCK_SUSPEND as last for systems which has no
/proc/sys/kernel/cap_last_cap
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Add tests for mounting into /proc and /sys
These two locations should be prohibited from mounting volumes into
those destinations.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
`lxc-stop` does not support sending arbitrary signals.
By default, `lxc-stop -n <id>` would send `SIGPWR`.
The lxc driver was always sending `lxc-stop -n <id> -k`, which always
sends `SIGKILL`. In this case `lxc-start` returns an exit code of `0`,
regardless of what the container actually exited with.
Because of this we must send signals directly to the process when we
can.
Also need to set quiet mode on `lxc-start` otherwise it reports an error
on `stderr` when the container exits cleanly (ie, we didn't SIGKILL it),
this error is picked up in the container logs... and isn't really an
error.
Also cleaned up some potential races for waitblocked test.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
The `--userland-proxy` daemon flag makes it possible to rely on hairpin
NAT and additional iptables routes instead of userland proxy for port
publishing and inter-container communication.
Usage of the userland proxy remains the default as hairpin NAT is
unsupported by older kernels.
Signed-off-by: Arnaud Porterie <arnaud.porterie@docker.com>
To help avoid version mismatches between libcontainer and Docker, this updates libcontainer to be the source of truth for which version of logrus the project is using. This should help avoid potential incompatibilities in the future, too. 👍
Signed-off-by: Andrew "Tianon" Page <admwiggin@gmail.com>
This also moves `exec -i` test to _unix_test.go because it seems to need a
pty to reliably reproduce the behavior.
Signed-off-by: Daniel, Dao Quang Minh <dqminh89@gmail.com>
Updates most of the instances of HTTP urls in the engine's
comments. Does not account for any use in the code itself,
documentation, contrib, or project files.
Signed-off-by: Eric Windisch <eric@windisch.us>
Preventing the test execution to pollute the deterministic runtime environment
by seeding the global rand.Random.
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
When working with Go channels you must not set it to nil or else the
channel will block forever. It will not panic reading from a nil chan
but it blocks. The correct way to do this is to create the channel then
close it as the correct results to the caller will be returned.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
container.
docker run -v /dev:/dev should stop mounting other default mounts in i
libcontainer otherwise directories and devices like /dev/ptx get mishandled.
We want to be able to run libvirtd for launching vms and it needs
access to the hosts /dev. This is a key componant of OpenStack.
Docker-DCO-1.1-Signed-off-by: Dan Walsh <dwalsh@redhat.com> (github: rhatdan)
This ensures that the libcontainer state is fully removed for a
container after it is terminated.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
The default for rlimit handling should be to inherit the rlimit of the
daemon unless explicitly set.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Cgroup resources are host dependent, they should be in hostConfig.
For backward compatibility, we just copy it to hostConfig, and leave it in
Config for now, so there is no regressions, but the right way to use this
throught json is to put it in HostConfig, like:
{
"Hostname": "",
...
"HostConfig": {
"CpuShares": 512,
"Memory": 314572800,
...
}
}
As we will add CpusetMems, CpusetCpus is definitely a better name, but some
users are already using Cpuset in their http APIs, we also make it compatible.
The main idea is keep using Cpuset in Config Struct, and make it has the same
value as CpusetCpus, but not always, some scenarios:
- Users use --cpuset in docker command, it can setup cpuset.cpus and can
get Cpuset field from docker inspect or other http API which will get
config info.
- Users use --cpuset-cpus in docker command, ditto.
- Users use Cpuset field in their http APIs, ditto.
- Users use CpusetCpus field in their http APIs, they won't get Cpuset field
in Config info, because by then, they should already know what happens
to Cpuset.
Signed-off-by: Qiang Huang <h.huangqiang@huawei.com>
This fixes various tests by checking for non zero exit code, accounting for lxc-specific base-diffs, and by removing lxc specific environment vars.
It also adds the --share-ipc option to lxc-start for shared ipc namespaces.
Signed-off-by: Abin Shahab <ashahab@altiscale.com> (github: ashahab-altiscale)
Docker-DCO-1.1-Signed-off-by: Abin Shahab <ashahab@altiscale.com> (github: ashahab-altiscale)
Set lxc.auto.mount = proc:mixed in unprivilged mode. This ensures that lxc mounts sys and proc/sysrq-trigger as readonly.
Signed-off-by: Abin Shahab <ashahab@altiscale.com> (github: ashahab-altiscale)
Docker-DCO-1.1-Signed-off-by: Abin Shahab <ashahab@altiscale.com> (github: ashahab-altiscale)
Sending capability ids instead of capability names ot LXC for --cap-add and --cap-drop.
Also fixed tests.
Docker-DCO-1.1-Signed-off-by: Abin Shahab <ashahab@altiscale.com> (github: ashahab-altiscale)
Add a --readonly flag to allow the container's root filesystem to be
mounted as readonly. This can be used in combination with volumes to
force a container's process to only write to locations that will be
persisted. This is useful in many cases where the admin controls where
they would like developers to write files and error on any other
locations.
Closes#7923Closes#8752
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
We want to be able to use container without the PID namespace. We basically
want containers that can manage the host os, which I call Super Privileged
Containers. We eventually would like to get to the point where the only
namespace we use is the MNT namespace to bring the Apps userspace with it.
By eliminating the PID namespace we can get better communication between the
host and the clients and potentially tools like strace and gdb become easier
to use. We also see tools like libvirtd running within a container telling
systemd to place a VM in a particular cgroup, we need to have communications of the PID.
I don't see us needing to share PID namespaces between containers, since this
is really what docker exec does.
So currently I see us just needing docker run --pid=host
Docker-DCO-1.1-Signed-off-by: Dan Walsh <dwalsh@redhat.com> (github: rhatdan)
This fixes the issue where an lxc.conf override of lxc.network.ipv4 was not being honored.
Docker-DCO-1.1-Signed-off-by: Abin Shahab <ashahab@altiscale.com> (github: ashahab-altiscale)
This commit contains changes for docker:
* user.GetGroupFile to user.GetGroupPath docker/libcontainer#301
* Add systemd support for OOM docker/libcontainer#307
* Support for custom namespaces docker/libcontainer#279, docker/libcontainer#312
* Fixes#9699docker/libcontainer#308
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
Mrunal Patel
Michael Crosby
David Calavera
Jess Frazelle
Doug Davis
Darren Stahl
qhuang
John Howard
Antonio Murdaca
Hu Keping
Madhu Venugopal
Qiang Huang
Sebastiaan van Stijn
Stefan J. Wernli
John Starks
Eric Windisch
Tim Dettrick
Arnaud Porterie
Yibai Zhang
Veres Lajos
Jessica Frazelle
Tibor Vass
Alexey Guskov
Alexander Morozov
Jessica Frazelle
Phil Estes
Alexander Morozov
Raghavendra K T
Vincent Demeester
unclejack
Phil Estes
Alex Samorukov
Antonio Murdaca
Jana Radhakrishnan
Darren Shepherd
Ma Shimiao
Brian Goff
Tianon Gravi
Jessica Frazelle
Daniel, Dao Quang Minh
Lei Jitang
Michael Crosby
Evan Hazlett
Yan Feng
Ahmet Alp Balkan
guoxiuyan
Dan Walsh
Michal Fojtik
Vishnu Kannan
Abin Shahab
Malte Janduda