Previously the cache was only updated once on startup, because the graph
code only check for filesystems on startup. However this breaks the API as it
was supposed and so unit tests.
Fixes#13142
Signed-off-by: Jörg Thalheim <joerg@higgsboson.tk>
Add handler for SIGUSR1 based on feedback regarding when to dump
goroutine stacks. This will also dump goroutine stack traces on SIGQUIT
followed by a hard-exit from the daemon.
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
Added --since argument to `docker logs` command. Accept unix
timestamps and shows logs only created after the specified date.
Default value is 0 and passing default value or not specifying
the value in the request causes parameter to be ignored (behavior
prior to this change).
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
If firewalld is not installed (or I suppose not running), firewalld was
producing an error in the daemon init logs, even though firewalld is not
required for iptables stuff to function.
The firewalld library code was also logging directly to logrus instead
of returning errors.
Moved logging code higher up in the stack and changed firewalld code to
return errors where appropriate.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Generation based on CAP_LAST_CAP, I hardcoded
capability.CAP_BLOCK_SUSPEND as last for systems which has no
/proc/sys/kernel/cap_last_cap
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
The docker graph call driver.Exists() on initialisation for each filesystem in
the graph. This results will results in a lot `zfs get all` commands. To reduce
this, retrieve all descend filesystem at startup and cache it for later checks
Signed-off-by: Jörg Thalheim <joerg@higgsboson.tk>
instead of let zfs automaticly mount datasets, mount them on demand using mount(2).
This speed up this graph driver in 2 ways:
- less zfs processes needed to start a container
- /proc/mounts get smaller, so zfs userspace tools has less to read (which can
a significant amount of data as the number of layer grows)
This ways it can be also ensured that the correct mountpoint is always used.
Signed-off-by: Jörg Thalheim <joerg@higgsboson.tk>
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Add tests for mounting into /proc and /sys
These two locations should be prohibited from mounting volumes into
those destinations.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
The lxc code here is doing the exact same thing on calling
execdriver.Terminate, so let's just use that.
Also removes some dead comments originally introduced
50144aeb42 but no longer relevant since we
have restart policies.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
`lxc-stop` does not support sending arbitrary signals.
By default, `lxc-stop -n <id>` would send `SIGPWR`.
The lxc driver was always sending `lxc-stop -n <id> -k`, which always
sends `SIGKILL`. In this case `lxc-start` returns an exit code of `0`,
regardless of what the container actually exited with.
Because of this we must send signals directly to the process when we
can.
Also need to set quiet mode on `lxc-start` otherwise it reports an error
on `stderr` when the container exits cleanly (ie, we didn't SIGKILL it),
this error is picked up in the container logs... and isn't really an
error.
Also cleaned up some potential races for waitblocked test.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
The `--userland-proxy` daemon flag makes it possible to rely on hairpin
NAT and additional iptables routes instead of userland proxy for port
publishing and inter-container communication.
Usage of the userland proxy remains the default as hairpin NAT is
unsupported by older kernels.
Signed-off-by: Arnaud Porterie <arnaud.porterie@docker.com>
To help avoid version mismatches between libcontainer and Docker, this updates libcontainer to be the source of truth for which version of logrus the project is using. This should help avoid potential incompatibilities in the future, too. 👍
Signed-off-by: Andrew "Tianon" Page <admwiggin@gmail.com>
Adds a `stream` query param to the stats API which allows API users to
only collect one stats entry and disconnect instead of keeping the
connection alive to stream more stats.
Also adds a `--no-stream` flag to `docker stats` which does the same
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Right now devicemapper mounts thin device using online discards by default
and passes mount option "discard". Generally people discourage usage of
online discards as they can be a drain on performance. Instead it is
recommended to use fstrim once in a while to reclaim the space.
In case of containers, we recommend to keep data volumes separate. So
there might not be lot of rm, unlink operations going on and there might
not be lot of space being freed by containers. So it might not matter
much if we don't reclaim that free space in pool.
User can still pass mount option explicitly using dm.mountopt=discard to
enable discards if they would like to.
So this is more like setting the containers by default for better performance
instead of better space efficiency in pool. And user can change the behavior
if they don't like default behavior.
Reported-by: Mike Snitzer <snitzer@redhat.com>
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
This patch modifies the journald log driver to store the container ID in
a field named CONTAINER_ID, rather than (ab)using the MESSAGE_ID field.
Additionally, this adds the CONTAINER_ID_FULL field containing the
complete container ID and CONTAINER_NAME, containing the container name.
When using the journald log driver, this permits you to see log messages
from a particular container like this:
# journalctl CONTAINER_ID=a9238443e193
Example output from "journalctl -o verbose" includes the following:
CONTAINER_ID=27aae7361e67
CONTAINER_ID_FULL=27aae7361e67e2b4d3864280acd2b80e78daf8ec73786d8b68f3afeeaabbd4c4
CONTAINER_NAME=web
Closes: #12864
Signed-off-by: Lars Kellogg-Stedman <lars@redhat.com>
prioritize the ports with static mapping before dynamic mapping. This removes
the port conflicts when we allocate static port in the reserved range
together with dynamic ones.
When static port is allocated first, Docker will skip those when determining
free ports for dynamic ones.
Signed-off-by: Daniel, Dao Quang Minh <dqminh89@gmail.com>
When a container has errors on removal, it gets flagged as dead.
If you `docker rm -f` a dead container the container is dereffed from
the daemon and doesn't show up on `docker ps` anymore... except that the
container JSON file may still be lingering around and becomes undead
when you restart the daemon.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
statsCollector.publishers must be protected to prevent
modifications during the iteration in run().
Being locked for a long time is bad, so pairs of containers &
publishers (pointers) are copied to release the lock fast.
Signed-off-by: Anton Tiurin <noxiouz@yandex.ru>
Due to the importance of path safety, the internal sanitisation wrappers
for volumes and containers should be exposed so other parts of Docker
can benefit from proper path sanitisation.
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com> (github: cyphar)
Assuming that docker kill is trying to actually kill the container
is a mistake. If the container is not running we should report it
back to the caller as a error.
Docker-DCO-1.1-Signed-off-by: Dan Walsh <dwalsh@redhat.com> (github: rhatdan)
Docker-DCO-1.1-Signed-off-by: Regan McCooey <rmccooey27@aol.com> (github: rmccooey27)
Docker-DCO-1.1-Signed-off-by: Regan McCooey <rmccooey27@aol.com> (github: rhatdan)
This also moves `exec -i` test to _unix_test.go because it seems to need a
pty to reliably reproduce the behavior.
Signed-off-by: Daniel, Dao Quang Minh <dqminh89@gmail.com>
- Mount struct now called volumeMount
- Merged volume creation for each volume type (volumes-from, binds, normal
volumes) so this only happens in once place
- Simplified container copy of volumes (for when `docker cp` is a
volume)
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
If device is being reactivated before it could go away and deferred
deactivation is scheduled on it, cancel it.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
This will help with debugging as one could just do "docker info" and figure
out of deferred removal is enabled or not.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
Provide a new command line knob dm.deferred_device_removal which will enable
deferred device deactivation if driver and library support it.
This patch also checks for library support and driver version.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
When firewalld (or iptables service) restarts/reloads,
all previously added docker firewall rules are flushed.
With firewalld we can react to its Reloaded() [1]
D-Bus signal and recreate the firewall rules.
Also when firewalld gets restarted (stopped & started)
we can catch the NameOwnerChanged signal [2].
To specify which signals we want to react to we use AddMatch [3].
Libvirt has been doing this for quite a long time now.
Docker changes firewall rules on basically 3 places.
1) daemon/networkdriver/portmapper/mapper.go - port mappings
Portmapper fortunatelly keeps list of mapped ports,
so we can easily recreate firewall rules on firewalld restart/reload
New ReMapAll() function does that
2) daemon/networkdriver/bridge/driver.go
When setting a bridge, basic firewall rules are created.
This is done at once during start, it's parametrized and nowhere
tracked so how can one know what and how to set it again when
there's been firewalld restart/reload ?
The only solution that came to my mind is using of closures [4],
i.e. I keep list of references to closures (anonymous functions
together with a referencing environment) and when there's firewalld
restart/reload I re-call them in the same order.
3) links/links.go - linking containers
Link is added in Enable() and removed in Disable().
In Enable() we add a callback function, which creates the link,
that's OK so far.
It'd be ideal if we could remove the same function from
the list in Disable(). Unfortunatelly that's not possible AFAICT,
because we don't know the reference to that function
at that moment, so we can only add a reference to function,
which removes the link. That means that after creating and
removing a link there are 2 functions in the list,
one adding and one removing the link and after
firewalld restart/reload both are called.
It works, but it's far from ideal.
[1] https://jpopelka.fedorapeople.org/firewalld/doc/firewalld.dbus.html#FirewallD1.Signals.Reloaded
[2] http://dbus.freedesktop.org/doc/dbus-specification.html#bus-messages-name-owner-changed
[3] http://dbus.freedesktop.org/doc/dbus-specification.html#message-bus-routing-match-rules
[4] https://en.wikipedia.org/wiki/Closure_%28computer_programming%29
Signed-off-by: Jiri Popelka <jpopelka@redhat.com>
Firewalld [1] is a firewall managing daemon with D-Bus interface.
What sort of problem are we trying to solve with this ?
Firewalld internally also executes iptables/ip6tables to change firewall settings.
It might happen on systems where both docker and firewalld are running
concurrently, that both of them try to call iptables at the same time.
The result is that the second one fails because the first one is holding a xtables lock.
One workaround is to use --wait/-w option in both
docker & firewalld when calling iptables.
It's already been done in both upstreams:
b315c380f4b3b451d6f8
But it'd still be better if docker used firewalld when it's running.
Other problem the firewalld support would solve is that
iptables/firewalld service's restart flushes all firewall rules
previously added by docker.
See next patch for possible solution.
This patch utilizes firewalld's D-Bus interface.
If firewalld is running, we call direct.passthrough() [2] method instead
of executing iptables directly.
direct.passthrough() takes the same arguments as iptables tool itself
and passes them through to iptables tool.
It might be better to use other methods, like direct.addChain and
direct.addRule [3] so it'd be more intergrated with firewalld, but
that'd make the patch much bigger.
If firewalld is not running, everything works as before.
[1] http://www.firewalld.org/
[2] https://jpopelka.fedorapeople.org/firewalld/doc/firewalld.dbus.html#FirewallD1.direct.Methods.passthrough
[3] https://jpopelka.fedorapeople.org/firewalld/doc/firewalld.dbus.html#FirewallD1.direct.Methods.addChainhttps://jpopelka.fedorapeople.org/firewalld/doc/firewalld.dbus.html#FirewallD1.direct.Methods.addRule
Signed-off-by: Jiri Popelka <jpopelka@redhat.com>
Two main things
- Create a real struct Info for all of the data with the proper types
- Add test for REST API get info
Signed-off-by: Hu Keping <hukeping@huawei.com>
This patch changes two things
1. Set facility to LOG_DAEMON
2. Remove ": " from tag so that the tag + pid become a single column in
the log
Signed-off-by: Darren Shepherd <darren@rancher.com>
Before this, a storage driver would be defaulted to based on the
priority list, and only print a warning if there is state from other
drivers.
This meant a reordering of priority list would "break" users in an
upgrade of docker, such that there images in the prior driver's state
were now invisible.
With this change, prior state is scanned, and if present that driver is
preferred.
As such, we can reorder the priority list, and after an upgrade,
existing installs with prior drivers can have a contiguous experience,
while fresh installs may default to a driver in the new priority list.
Ref: https://github.com/docker/docker/pull/11962#issuecomment-88274858
Signed-off-by: Vincent Batts <vbatts@redhat.com>
Signed-off-by: Megan Kostick <mkostick@us.ibm.com>
Alphabetize FSMagic list to make more human-readable.
Signed-off-by: Megan Kostick <mkostick@us.ibm.com>
Check whether the swap limit capabilities are disabled or not only when memory swap is set to greater than 0.
Signed-off-by: David Calavera <david.calavera@gmail.com>
Jessie Frazelle
Jessica Frazelle
Antonio Murdaca
Alexander Morozov
Brian Goff
John Howard
Sebastiaan van Stijn
Jörg Thalheim
Phil Estes
Tibor Vass
Alexander Morozov
Ahmet Alp Balkan
Gianluca Borello
Qiang Huang
Bharath Thiruveedula
Ma Shimiao
Michael Crosby
wlan0
David Calavera
Burke Libbey
Anthony Baire
Lei Jitang
Arnaud Porterie
Vincent Batts
mauriyouth
Phil Estes
Tianon Gravi
unclejack
HuKeping
Vivek Goyal
Michael Crosby
Lars Kellogg-Stedman
Daniel, Dao Quang Minh
Dan Walsh
Jessica Frazelle
David Mackey
Aidan Hobson Sayers
jmzwcn
Anton Tiurin
Aleksa Sarai
Regan McCooey
jianbosun
Gosuke Miyashita
Srini Brahmaroutu
Sylvain Baubeau
Jiri Popelka
Ankush Agarwal
Doug Davis
Simei He
Megan Kostick
Yan Feng
Darren Shepherd
Deng Guangxing
Vincent Batts
Evan Hazlett