- Use a common struct for Volumes
- Split out some functionality in intializeVolume into separate functions
- Removes some duplicate code
- In general much easier to grok the code now
Docker-DCO-1.1-Signed-off-by: Brian Goff <cpuguy83@gmail.com> (github: cpuguy83)
This PR moves the userland proxies for TCP and UDP traffic out of the
main docker daemon's process ( from goroutines per proxy ) to be a
separate reexec of the docker binary. This reduces the cpu and memory
needed by the daemon and if the proxy processes crash for some reason
the daemon is unaffected. This also displays in the standard process
tree so that a user can clearly see if there is a userland proxy that is
bound to a certain ip and port.
```bash
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
5d349506feb6 busybox:buildroot-2014.02 "sh" 13 minutes ago Up 1 seconds 0.0.0.0:49153->81/tcp, 0.0.0.0:49154->90/tcp hungry_pike
root@1cbfdcedc5a7:/go/src/github.com/docker/docker# ps aux
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.0 0.1 18168 3100 ? Ss 21:09 0:00 bash
root 8328 0.7 0.6 329072 13420 ? Sl 22:03 0:00 docker -d -s vfs
root 8373 1.0 0.5 196500 10548 ? Sl 22:03 0:00 userland-proxy -proto tcp -host-ip 0.0.0.0 -host-port 49153 -container-ip 10.0.0.2 -container-port 81
root 8382 1.0 0.5 270232 10576 ? Sl 22:03 0:00 userland-proxy -proto tcp -host-ip 0.0.0.0 -host-port 49154 -container-ip 10.0.0.2 -container-port 90
root 8385 1.2 0.0 3168 184 pts/0 Ss+ 22:03 0:00 sh
root 8408 0.0 0.1 15568 2112 ? R+ 22:03 0:00 ps aux
```
This also helps us to cleanly cleanup the proxy processes by stopping
these commands instead of trying to terminate a goroutine.
Signed-off-by: Michael Crosby <michael@docker.com>
* starting with filtering for exit codes. `docker ps -a --filter 'exited=1'`
* API doc for filter parameter
* formatting filters for help usage
* tweaks for review
This requires https://github.com/dotcloud/docker/pull/4430
Docker-DCO-1.1-Signed-off-by: Vincent Batts <vbatts@redhat.com> (github: vbatts)
This changes the way the exec drivers work by not specifing a -driver
flag on reexec. For each of the exec drivers they register their own
functions that will be matched aginst the argv 0 on exec and called if
they match.
This also allows any functionality to be added to docker so that the
binary can be reexec'd and any type of function can be called. I moved
the flag parsing on docker exec to the specific initializers so that the
implementations do not bleed into one another. This also allows for
more flexability within reexec initializers to specify their own flags
and options.
Signed-off-by: Michael Crosby <michael@docker.com>
`rm -f` was originally deprecated in favor of `rm --stop/--kill` since `rm
-f` was sending SIGTERM and potentially very slow.
Instead this will bring back `rm -f` but use SIGKILL isntead
Docker-DCO-1.1-Signed-off-by: Brian Goff <cpuguy83@gmail.com> (github: cpuguy83)
This removes a shim `daemon.Server` interface which was used to start
separating Daemon from Server *gradually*, without getting cyclic
dependency errors.
Now that the last Daemon->Server dependency has been removed, we can
finally remove the shim. Yay!
Signed-off-by: Solomon Hykes <solomon@docker.com>
This disentangles the following functions, which were previously all mixed together:
* 1) Waiting for jobs to terminate when shutting down
* 2) Handling signals in the Docker daemon
* 3) Per-subsystem cleanup handlers
* 4) pidfile management
Responsibilities are dispatched as follows:
* Signal traps are set in `main`, and trigger `engine.Shutdown`
* `engine.Shutdown` coordinates cleanup by waiting for jobs to complete, and calling shutdown handlers
* To perform cleanup at shutdown, each subsystem registers handlers with `engine.OnShutdown`
* `daemon` is one subsystem, so it registers cleanup via `engine.OnShutdown`.
* `daemon` owns the pidfile, which is used to lock access to `/var/lib/docker`. Part of its cleanup is to remove the pidfile.
Signed-off-by: Solomon Hykes <solomon@docker.com>
* Events subsystem merged from `server/events.go` and
`utils/jsonmessagepublisher.go` and moved to `events/events.go`
* Only public interface for this subsystem is engine jobs
* There is two new engine jobs - `log_event` and `subscribers_count`
* There is auxiliary function `container.LogEvent` for logging events for
containers
Docker-DCO-1.1-Signed-off-by: Alexandr Morozov <lk4d4math@gmail.com> (github: LK4D4)
[solomon@docker.com: resolve merge conflicts]
Signed-off-by: Solomon Hykes <solomon@docker.com>
Note: this cannot yet be moved to graph/ because of a lingering
dependency on daemon. This has been noted in a FIXME.
Signed-off-by: Solomon Hykes <solomon@docker.com>
/tmp is often a tmpfs file system and large temporary files could cause
docker commands to fail. Also using /tmp potentially allows users on the
system to get access to content, or even attack the content. Moving the tmpdir to
/var/lib/container/tmp will protect the data.
Docker-DCO-1.1-Signed-off-by: Dan Walsh <dwalsh@redhat.com> (github: rhatdan)
Conflicts:
docker/docker.go
This is part of an effort to break apart the deprecated server/ package
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
This is part of an effort to break apart the deprecated server/ package
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
This is part of an effort to break apart the deprecated server/ package
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
Fix issues with renaming changes to container_changes
Docker-DCO-1.1-Signed-off-by: Tibor Vass <teabee89@gmail.com> (github: tiborvass)
This is part of an effort to break apart the deprecated server/ package
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
This is part of an effort to break apart the deprecated server/ package
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
Fix issues with renaming container_delete to delete
Docker-DCO-1.1-Signed-off-by: Tibor Vass <teabee89@gmail.com> (github: tiborvass)
This is part of an effort to break apart the deprecated server/ package
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
This is part of an effort to break apart the deprecated server/ package
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
This is part of an effort to break apart the deprecated server/ package
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
This is part of an effort to break apart the deprecated server/ package
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
This is part of an effort to break apart the deprecated server/ package
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
This is part of an effort to break apart the deprecated server/ package
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
This is part of an effort to break apart the deprecated server/ package
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
This is part of an effort to break apart the deprecated server/ package
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
This is part of an effort to break apart the deprecated server/ package
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
This is part of an effort to break apart the deprecated server/ package
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
This is part of an effort to break apart the deprecated server/ package
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
This is part of an effort to break apart the deprecated server/ package
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
This is part of an effort to break apart the legacy server package. Help wanted!
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
functions to pkg/parsers/kernel, and parsing filters to
pkg/parsers/filter. Adjust imports and package references.
Docker-DCO-1.1-Signed-off-by: Erik Hollensbe <github@hollensbe.org> (github: erikh)
Defining err as named return parameter will make sure the variable gets
assigned before returning and thus avoid masking
Docker-DCO-1.1-Signed-off-by: Johannes 'fish' Ziemke <github@freigeist.org> (github: discordianfish)
lxc is special in that we cannot create the master outside of the
container without opening the slave because we have nothing to provide to the
cmd. We have to open both then do the crazy setup on command right now instead of
passing the console path to lxc and telling it to open up that console. we save a couple of
openfiles in the native driver because we can do this.
Docker-DCO-1.1-Signed-off-by: Michael Crosby <michael@docker.com> (github: crosbymichael)
This uses "," instead of spaces so that the flags are parsed correctly
and also does not do a strings.Split on an empty string because
strings.Split will return a slice with one element, and empty string
causing parsing to fail when it validates that the cap exists.
Docker-DCO-1.1-Signed-off-by: Michael Crosby <michael@docker.com> (github: crosbymichael)
Here was possible race with inspect where we changing HostConfig.Links
Docker-DCO-1.1-Signed-off-by: Alexandr Morozov <lk4d4math@gmail.com> (github: LK4D4)
We add a --device flag which can be used like:
docker run --device /dev/sda:/dev/xvda:rwm ubuntu /bin/bash
To allow the container to have read write permissions to access the host's /dev/sda via a node named /dev/xvda in the container.
Note: Much of this code was written by Dinesh Subhraveti dineshs@altiscale.com (github: dineshs-altiscale) and so he deserves a ton of credit.
Docker-DCO-1.1-Signed-off-by: Timothy <timothyhobbs@seznam.cz> (github: timthelion)
Commit 09ee269d ("devmapper: Add option for specifying the thin pool
blocksize") also switched the default dm-thin-pool blocksize from 64K to
512K. That change unfortunately breaks the activation of dm-thin-pool
devices that were previously created using a 64K blocksize. Here is an
example of the dm-thin-pool activation failure users may experience:
device-mapper: thin: 253:4: pool target (204800 blocks) too small: expected 1638400
device-mapper: table: 253:4: thin-pool: preresume failed, error = -22
The reason for this is docker is passing 512K as the blocksize for a
dm-thin-pool that was previously created using a 64K blocksize. Docker
doesn't record the blocksize the is used when it creates a dm-thin-pool.
Until now it never had a need to do so because the blocksize was always
hardcoded. The dm-thin-pool blocksize must be the same every time a
dm-thin-pool is activated.
As a stop-gap fix, revert to using 64K for the default blocksize.
But we do need a proper fix for this now that 'dm.blocksize' is exposed
as a proper storage option. One possible fix would be to record the
blocksize for each dm-thin-pool that docker creates and to pass that
recorded blocksize down in the dmsetup table load each time the
dm-thin-pool is activated (this would be comparable to what lvm2 does).
Docker-DCO-1.1-Signed-off-by: Mike Snitzer <snitzer@redhat.com> (github: snitm)
The Docker btrfs graph driver does not interact well with SELinux at present.
If btrfs mounts the same file in several locations, the same SELinux label will
be applied to all mountpoints. In the context of the graph driver, things such
as shared libraries become inaccessible to containers due to SELInux, causing
all dynamically linked applications to fail when run in a container.
Consequently, error when we detect the daemon is being run with SELinux enabled
and the btrfs driver. Documentation has been added for this behavior.
Docker-DCO-1.1-Signed-off-by: Matthew Heon <mheon@redhat.com> (github: mheon)
It became slightly faster and lighter
possibly fixes#5923 problems
Docker-DCO-1.1-Signed-off-by: Alexandr Morozov <lk4d4math@gmail.com> (github: LK4D4)
Initiates a pause before committing a container,
adds a pause option to the commit command, defaulting to 'true'.
Fixes bug: #6267
Fixes bug: #3675
Docker-DCO-1.1-Signed-off-by: Eric Windisch <ewindisch@docker.com> (github: ewindisch)
Port allocation status is stored in a global map: a port detected in use will remain as such for the lifetime of the daemon. Change the behavior to only mark as allocated ports which are claimed by Docker itself (which we can trust to properly remove from the allocation map once released). Ports allocated by other applications will always be retried to account for the eventually of the port having been released.
Docker-DCO-1.1-Signed-off-by: Arnaud Porterie <icecrime@gmail.com> (github: icecrime)
Add dm.blocksize option that you can use with --storage-opt to set a
specific blocksize for the thin provisioning pool.
Also change the default dm-thin-pool blocksize from 64K to 512K. This
strikes a balance between the desire to have smaller blocksize given
docker's use of snapshots versus the desire to have more performance
that comes with using a larger blocksize. But if very small files will
be used on average the user is encouraged to override this default.
Docker-DCO-1.1-Signed-off-by: Mike Snitzer <snitzer@redhat.com> (github: snitm)
Device Mapper needs device sizes in binary (1024) multiples. Otherwise
kernel checks can find that the specified thin-pool device sizes aren't
a multiple of the specified thin-pool blocksize.
The name for "RAMInBytes" is likely too narrow given the new consumers
but... Also add "tebibyte" support to RAMInBytes.
Docker-DCO-1.1-Signed-off-by: Mike Snitzer <snitzer@redhat.com> (github: snitm)
createPool() and reloadPool() should be consistent with the thin-pool
table params they use.
Since createPool() specifies '1 skip_block_zeroing' reloadPool() should
too. Otherwise, if the pool is reloaded (as is done when resizing
loopback devices) block zeroing will be enabled after the reload
completes.
Docker-DCO-1.1-Signed-off-by: Mike Snitzer <snitzer@redhat.com> (github: snitm)
This will allow us to _know_ what the user's -H settings are, which may
be useful for debugging later.
Docker-DCO-1.1-Signed-off-by: Sven Dowideit <SvenDowideit@fosiki.com> (github: SvenDowideit)
This patch updates container.getResourcePath and container.getRootResourcePath
to return the error from symlink.FollowSymlinkInScope (rather than using utils).
Docker-DCO-1.1-Signed-off-by: Aleksa Sarai <cyphar@cyphar.com> (github: cyphar)
Remove Inject to help rebase
Docker-DCO-1.1-Signed-off-by: Tibor Vass <teabee89@gmail.com> (github: tiborvass)
Docker-DCO-1.1-Signed-off-by: cyphar <cyphar@cyphar.com> (github: tiborvass)
as a maintainer.
Best of luck on your e-commerce business Guillaume, and thanks for all
the great contributions!
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <solomon@docker.com> (github: shykes)
If this is at the root directory for the daemon you could unmount
somones filesystem when you stop docker and this is actually only needed
for the palces that the graph drivers mount the container's root
filesystems.
Docker-DCO-1.1-Signed-off-by: Michael Crosby <michael@crosbymichael.com> (github: crosbymichael)
The blkdiscard hack we do on container/image delete is pretty slow, but
required to restore space to the "host" root filesystem. However, it
is pretty useless on raw devices, and you may not need it in development
either.
In a simple test of the devicemapper backend on loopback the time to
delete 20 container went from 11 seconds to 0.4 seconds with
--storage-opt blkdiscard=false.
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
This adds dm.datadev and dm.metadatadev options that you can use with
--storage-opt to set to specific devices to use for the thin
provisioning pool.
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
This adds the following --storage-opts for the daemon:
dm.fs: The filesystem to use for the base image
dm.mkfsarg: Add an argument to the mkfs command for the base image
dm.mountopt: Add a mount option for devicemapper mount
Currently supported filesystems are xfs and ext4.
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
This allows setting these settings to be passed:
dm.basesize
dm.loopdatasize
dm.loopmetadatasize
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
If we can't even get the current device mapper driver version, then
we cleanly fail the devmapper driver as not supported and fall back
on the next one.
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
This patch adds pause/unpause to the command line, api, and drivers
for use on containers. This is implemented using the cgroups/freeze
utility in libcontainer and lxc freeze/unfreeze.
Co-Authored-By: Eric Windisch <ewindisch@docker.com>
Co-Authored-By: Chris Alfonso <calfonso@redhat.com>
Docker-DCO-1.1-Signed-off-by: Ian Main <imain@redhat.com> (github: imain)
This only works if the file or dir is already created in
the image before setting it to be a volume. There is no way around this
because we don't have the data avaliable to set the volume at the
beginning of the dockerfile
Docker-DCO-1.1-Signed-off-by: Michael Crosby <michael@crosbymichael.com> (github: crosbymichael)
This is a fix for a race condition in the LXC driver. This is described
more in issue #6092.
Closes#6092
Docker-DCO-1.1-Signed-off-by: Shane Canon <scanon@lbl.gov> (github: scanon)
This also makes sure that devices are pointers to avoid copies
Docker-DCO-1.1-Signed-off-by: Michael Crosby <michael@crosbymichael.com> (github: crosbymichael)
We now have one place that keeps track of (most) devices that are allowed and created within the container. That place is pkg/libcontainer/devices/devices.go
This fixes several inconsistencies between which devices were created in the lxc backend and the native backend. It also fixes inconsistencies between wich devices were created and which were allowed. For example, /dev/full was being created but it was not allowed within the cgroup. It also declares the file modes and permissions of the default devices, rather than copying them from the host. This is in line with docker's philosphy of not being host dependent.
Docker-DCO-1.1-Signed-off-by: Timothy Hobbs <timothyhobbs@seznam.cz> (github: https://github.com/timthelion)
There are two cases where we can't use a graphdriver:
1) the graphdriver itself isn't supported by the system
2) the graphdriver is supported by some configuration/prerequisites are
missing
This introduces a new error for the 2) case and uses it when trying to
run docker with btrfs backend on a non-btrfs filesystem.
Docker-DCO-1.1-Signed-off-by: Johannes 'fish' Ziemke <github@freigeist.org> (github: discordianfish)
We don't need ordered set anymore, also some cleanings and simple
benchmark.
Docker-DCO-1.1-Signed-off-by: Alexandr Morozov <lk4d4math@gmail.com> (github: LK4D4)
Josiah Kiehl
Victor Vieux
Michael Crosby
Tibor Vass
Brian Goff
Solomon Hykes
Vincent Batts
Alexandr Morozov
Victor Vieux
Erik Hollensbe
Tianon Gravi
Eric Windisch
Marc Tamsky
Dan Walsh
Rajdeep Dua
unclejack
Vishnu Kannan
Johannes 'fish' Ziemke
Timothy
Kohei Tsuruta
Albert Zhang
Bryan Bess
Mike Snitzer
Matthew Heon
SvenDowideit
Eric Windisch
Kato Kazuyoshi
Arnaud Porterie
Travis Cline
unclejack
Senthil Kumar Selvaraj
Sven Dowideit
Dinesh Subhraveti
Rick van de Loo
cyphar
Jérôme Petazzoni
Michael Crosby
Alexander Larsson
Chris Alfonso
Ian Main
Victor Marmol
Michael Crosby
Shane Canon
William Thurston