Fixed the following errors:
1. Request(0) causes a dead loop when the map is full and map.last == BEGIN.
2. When map.last is the only available port (or ip), Request(0) returns ErrAllPortsAllocated (or ErrNoAvailableIPs). Exception is when map.last == BEGIN.
Signed-off-by: shuai-z <zs.broccoli@gmail.com>
Fixes#8832
All stdio streams need to finish writing before the
connection can be closed.
Signed-off-by: Tõnis Tiigi <tonistiigi@gmail.com> (github: tonistiigi)
If we need to raise an error, make sure the internal state is clean, because
a successful driver.Get() may have its internal state changed (eg. counting,
or mounts), while callers will only do that after a succussful Mount().
Signed-off-by: shuai-z <zs.broccoli@gmail.com>
In an effort to make layer content 'stable' between import
and export from two different graph drivers, we must resolve
an issue where AUFS produces metadata files in its layers
which other drivers explicitly ignore when importing.
The issue presents itself like this:
- Generate a layer using AUFS
- On commit of that container, the new stored layer contains
AUFS metadata files/dirs. The stored layer content has some
tarsum value: '1234567'
- `docker save` that image to a USB drive and `docker load`
into another docker engine instance which uses another
graph driver, say 'btrfs'
- On load, this graph driver explicitly ignores any AUFS metadata
that it encounters. The stored layer content now has some
different tarsum value: 'abcdefg'.
The only (apparent) useful aufs metadata to keep are the psuedo link
files located at `/.wh..wh.plink/`. Thes files hold information at the
RW layer about hard linked files between this layer and another layer.
The other graph drivers make sure to copy up these psuedo linked files
but I've tested out a few different situations and it seems that this
is unnecessary (In my test, AUFS already copies up the other hard linked
files to the RW layer).
This changeset adds explicit exclusion of the AUFS metadata files and
directories (NOTE: not the whiteout files!) on commit of a container
using the AUFS storage driver.
Also included is a change to the archive package. It now explicitly
ignores the root directory from being included in the resulting tar archive
for 2 reasons: 1) it's unnecessary. 2) It's another difference between
what other graph drivers produce when exporting a layer to a tar archive.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
This backend uses the overlayfs union filesystem for containers
plus hard link file sharing for images.
Each container/image can have a "root" subdirectory which is a plain
filesystem hierarchy, or they can use overlayfs.
If they use overlayfs there is a "upper" directory and a "lower-id"
file, as well as "merged" and "work" directories. The "upper"
directory has the upper layer of the overlay, and "lower-id" contains
the id of the parent whose "root" directory shall be used as the lower
layer in the overlay. The overlay itself is mounted in the "merged"
directory, and the "work" dir is needed for overlayfs to work.
When a overlay layer is created there are two cases, either the
parent has a "root" dir, then we start out with a empty "upper"
directory overlaid on the parents root. This is typically the
case with the init layer of a container which is based on an image.
If there is no "root" in the parent, we inherit the lower-id from
the parent and start by making a copy if the parents "upper" dir.
This is typically the case for a container layer which copies
its parent -init upper layer.
Additionally we also have a custom implementation of ApplyLayer
which makes a recursive copy of the parent "root" layer using
hardlinks to share file data, and then applies the layer on top
of that. This means all chile images share file (but not directory)
data with the parent.
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
The vfs storage driver currently shells out to the `cp` binary on the host
system to perform an 'archive' copy of the base image to a new directory.
The archive option preserves the modified time of the files which are created
but there was an issue where it was unable to preserve the modified time of
copied symbolic links on some host systems with an outdated version of `cp`.
This change no longer relies on the host system implementation and instead
utilizes the `CopyWithTar` function found in `pkg/archive` which is used
to copy from source to destination directory using a Tar archive, which
should correctly preserve file attributes.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
If we first request port 49153 (BeginPortRange) explicitly, and later some time request the next free port (of same ip/proto) by calling RequestPort() with port number 0, we will again get 49153 returned, even if it's currently in use. Because findPort() blindly retured BeginPortRange the first run, without checking if it has already been taken.
Signed-off-by: shuai-z <zs.broccoli@gmail.com>
when a container failed to start, saves the error message into State.Error so
that it can be retrieved when calling `docker inspect` instead of having to
look at the log
Docker-DCO-1.1-Signed-off-by: Daniel, Dao Quang Minh <dqminh89@gmail.com> (github: dqminh)
Never close attached stream before both stdout and stderr have written
all their buffered contents. Remove stdinCloser because it is not needed
any more as the stream is closed anyway after attach has finished.
Fixes#3631
Signed-off-by: Andy Goldstein <agoldste@redhat.com>
Fixes#1992
Right now when you `docker cp` a path which is in a volume, the cp
itself works, however you end up getting files that are in the
container's fs rather than the files in the volume (which is not in the
container's fs).
This makes it so when you `docker cp` a path that is in a volume it
follows the volume to the real path on the host.
archive.go has been modified so that when you do `docker cp mydata:/foo
.`, and /foo is the volume, the outputed folder is called "foo" instead
of the volume ID (because we are telling it to tar up
`/var/lib/docker/vfs/dir/<some id>` and not "foo", but the user would be
expecting "foo", not the ID
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Stable IPs causes some regressions in the way people use Docker, see GH#8493.
Reverting it for 1.3, we'll enable it back for the next release.
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
When a container is restarted all the volume configs are parsed again.
Even if the volume was already handled in a previous start it was still
calling "FindOrCreateVolume" on the volume repo causing a new volume to
be created.
This wasn't being detected because as part of the mount initialization
it checks to see if the the _mount_ was already initialized, but this
happens after the parsing of the configs.
So a check is added during parsing to skip a volume which was already
created for that container.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Volume refs were not being restored on daemon restart.
This made it possible to remove a volume being used by other containers
after a daemon restart.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
When stdout/stderr is closed prematurely, the proxy's writes to stdout/stderr
(i.e. `log.Errorf/log.Printf`) will returns with EPIPE error, and go runtime
will terminate the proxy when stdout/stderr writes trigger 10 EPIPE errors.
instead of using stdout/stderr as the status handler, we pass an extra file to
the child process and write `0\n` or `1\nerror message` to it and close it
after. This allow the child process to handle stdout/stderr as normal.
Docker-DCO-1.1-Signed-off-by: Daniel, Dao Quang Minh <dqminh89@gmail.com> (github: dqminh)
Currently if you start the docker -d on a system with 127.0.0.1 in /etc/resolv.conf
It will set the default dns to 8.8.8.8 8.8.4.4 permanently.
This causes a problem at boot on Fedora machines where NetworkManager has not
populated /etc/resolv.conf before docker gets started.
This fix checks /etc/resolv.conf on every docker run. And only populates
daemon.config.Dns if the user specified it on the command line.
Docker-DCO-1.1-Signed-off-by: Dan Walsh <dwalsh@redhat.com> (github: rhatdan)
The defer logic was a little tricky and was hiding one bug: `err` was
being redefined (with `:=`) and thus it escaped the defer error checking
logic.
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
Prior to the volumes re-factor, data was not being copied on
volumes-from or host-mounted volumes.
After the re-factor, data was being copied for volumes-from.
This reverts this unintentional change in behavior.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Needed to check if the mode was invalid and return error, not valid and
return error.
This didn't get picked up because the existing integration-cli tests
were all either expecting errors when a valid mode was passed in (e.g.
"ro" passed in, we expected an error because it was testing write). So
modified a test which was testing for "rw" to actually pass in "rw"
instead of assuming the "rw"
Docker-DCO-1.1-Signed-off-by: Brian Goff <bgoff@cpuguy83-mbp.home> (github: cpuguy83)
This change will allocate network settings (IP and public ports) at
container creation rather than start and keep them throughout the
lifetime of the container (i.e. until it gets destroyed) instead of
discarding them when the container is stopped.
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
Since we are moving network allocation outside of container scope (it
will be managed by create/destroy), those functions need to be
accessible from the outside.
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
Right now, MAC addresses are randomly generated by the kernel when
creating the veth interfaces.
This causes different issues related to ARP, such as #4581, #5737 and #8269.
This change adds support for consistent MAC addresses, guaranteeing that
an IP address will always end up with the same MAC address, no matter
what.
Since IP addresses are already guaranteed to be unique by the
IPAllocator, MAC addresses will inherit this property as well for free.
Consistent mac addresses is also a requirement for stable networking (#8297)
since re-using the same IP address on a different MAC address triggers the ARP
issue.
Finally, this change makes the MAC address accessible through docker
inspect, which fixes#4033.
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
Add support for pulling signed images from a version 2 registry.
Only official images within the library namespace will be pull from the
new registry and check the build signature.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Now that the archive package does not depend on any docker-specific
packages, only those in pkg and vendor, it can be safely moved into pkg.
Signed-off-by: Rafe Colton <rafael.colton@gmail.com>
This is the first of two steps to break the archive package's dependence
on utils so that archive may be moved into pkg. Also, the `Go()`
function is small, concise, and not specific to the docker internals, so
it is a good candidate for pkg.
Signed-off-by: Rafe Colton <rafael.colton@gmail.com>
security-opts will allow you to customise the security subsystem.
For example the labeling system like SELinux will run on a container.
--security-opt="label:user:USER" : Set the label user for the container
--security-opt="label:role:ROLE" : Set the label role for the container
--security-opt="label:type:TYPE" : Set the label type for the container
--security-opt="label:level:LEVEL" : Set the label level for the container
--security-opt="label:disabled" : Turn off label confinement for the container
Since we are passing a list of string options instead of a space separated
string of options, I will change function calls to use InitLabels instead of
GenLabels. Genlabels interface is Depracated.
Docker-DCO-1.1-Signed-off-by: Dan Walsh <dwalsh@redhat.com> (github: rhatdan)
This also removes dead code in the native driver for a past feature that
was never fully implemented.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Since RemoveLocalDns patch will remove all localhost entries
from resolv.conf we no longer need anything more then
!bytes.Contains(resolvConf, []byte("nameserver")
To check for no nameserver entry in dns config.
Docker-DCO-1.1-Signed-off-by: Dan Walsh <dwalsh@redhat.com> (github: rhatdan)
We have a bug report complaining about docker dumping the contents of the
hosts resolv.conf if it container 127.0.0.1. They asked that instead
of dropping the file altogether, that we just remove the line.
This patch removes the 127.0.0.1 lines, if they exist and then
checks if any nameserver lines exist.
Docker-DCO-1.1-Signed-off-by: Dan Walsh <dwalsh@redhat.com> (github: rhatdan)
Currently, the HostConfig is only passed from the CLI to Docker only
when issuing a docker create, but not when doing a docker run.
In the near future, in order to allocate ports at creation time rather
than start time, we will need to have the HostConfig readily available
at container creation.
This PR makes the client always pass the HostConfig when creating a
container (regardless of whether it's for a run or create).
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
Since it is possible to request a specific IP, IPAllocator has to verify
that the request is within boundaries.
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
On Fedora and RHEL we ship selinux-enabled flag in the docker.service config,
but if people setup the /var/lib/docker as btrfs and disable SELinux,
we should not block the daemon from running.
Docker-DCO-1.1-Signed-off-by: Dan Walsh <dwalsh@redhat.com> (github: rhatdan)
If `--storage-opt dm.datadev=/dev/loop0 --storage-opt
dm.metadatadev=/dev/loop1 ` were provided, the information was not
reflected in the information output.
Closes: #7137
Signed-off-by: Vincent Batts <vbatts@redhat.com>
There were a new areas in the brige driver that did not need to have log
output. Those were removed. Also set the engine's logging to false
when running the integration tests.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
For the cases where --bip option is used it is sometimes best to disable
IP masquerading as the provided bridge IP range may be routable.
Signed-off-by: Eugene Yakubovich <eugene.yakubovich@coreos.com>
This adds a --add-host host:ip flag which appends lines to /etc/hosts. This is needed in places where you want the container to get a different name resolution than it would through DNS. This was submitted before as #5525, closed, and now I am re-opening. It has come up 2 or 3 times in the last couple days.
Signed-off-by: Tim Hockin <thockin@google.com>
This exposes the already existing "create container" operation. It is
very similar to "docker run -d" except it doesn't actually start the
container, but just prepares it. It can then be manually started using
"docker start" at any point.
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
Conflicts:
api/client/commands.go
runconfig/parse.go
server/container.go
Docker-DCO-1.1-Signed-off-by: Tibor Vass <teabee89@gmail.com> (github: tiborvass)
Some graphdrivers are Differs and type assertions are made
in various places throughout the project. Differ offers some
convenience in generating/applying diffs of filesystem layers
but for most graphdrivers another code path is taken.
This patch brings all of the logic related to filesystem
diffs in one place, and simplifies the implementation of some
common types like Image, Daemon, and Container.
Signed-off-by: Josh Hawn <josh.hawn@docker.com>
Use utils.RFC3339NanoFixed ("2006-01-02T15:04:05.000000000Z07:00")
instead of time.RFC3339Nano to format our log timestamps - this way
things are aligned, in particular the nano seconds are padded with zeros
Signed-off-by: Doug Davis <dug@us.ibm.com>
1. /container/<name>/exec - Creates a new exec command instance in the daemon and container '<name>'. Returns an unique ID for each exec command.
2. /exec/<name>/start - Starts an existing exec command instance. Removes the exec command from the daemon once it completes.
Adding /exec/<name>/resize to resize tty session of an exec command.
Docker-DCO-1.1-Signed-off-by: Vishnu Kannan <vishnuk@google.com> (github: vishh)
Adds support for a --registry-mirror=scheme://<host>[:port]
daemon flag. The flag may be present multiple times. If
provided, mirrors are prepended to the list of endpoints used
for image pull. Note that only mirrors of the public
index.docker.io registry are supported, and image/tag resolution
is still performed via the official index.
Docker-DCO-1.1-Signed-off-by: Tim Smith <timbot@google.com> (github: timbot)
commit 4aa5da278f moves `Console` from Command to
ProcessConfig, but missed the change in lxc_template. Therefore creating a
container with tty using lxc driver with fail with error
template: lxc:60:20: executing "lxc" at <.Console>: Console is not a field of
struct type struct { *execdriver.Command; AppArmor bool; ProcessLabel string; MountLabel string }
This changes lxc_console template to refers to `.ProcessConfig.Console`
Docker-DCO-1.1-Signed-off-by: Daniel, Dao Quang Minh <dqminh89@gmail.com> (github: dqminh)
Since these will be shared between containers we want to label
them as svirt_sandbox_file_t:s0. That will allow multiple containers
to write to them.
Currently we are allowing container domains to read/write all content in
/var/lib/docker because of container volumes. This is a big security hole
in our SELinux story.
This patch will allow us to tighten up the security of docker containers.
Docker-DCO-1.1-Signed-off-by: Dan Walsh <dwalsh@redhat.com> (github: rhatdan)
This commit fixes following FIXMEs:
// FIXME: rename "delete" to "rm" for consistency with the CLI command
// FIXME: rename ContainerDestroy to ContainerRm for consistency with the CLI command
Signed-off-by: lim seong yeol <seongyeol37@gmail.com>
Reset the time increment if the container's execution time is greater
than 10s or else as a container runs and is restarted the time will grow
overtime.
Signed-off-by: Michael Crosby <michael@docker.com>
We need to do this so that when a user asks docker to stop the container
and it is currently in the restart loop we don't want to have to wait
for the duration of the restart time increment before ack. the stop.
Signed-off-by: Michael Crosby <michael@docker.com>
- Use a common struct for Volumes
- Split out some functionality in intializeVolume into separate functions
- Removes some duplicate code
- In general much easier to grok the code now
Docker-DCO-1.1-Signed-off-by: Brian Goff <cpuguy83@gmail.com> (github: cpuguy83)
This PR moves the userland proxies for TCP and UDP traffic out of the
main docker daemon's process ( from goroutines per proxy ) to be a
separate reexec of the docker binary. This reduces the cpu and memory
needed by the daemon and if the proxy processes crash for some reason
the daemon is unaffected. This also displays in the standard process
tree so that a user can clearly see if there is a userland proxy that is
bound to a certain ip and port.
```bash
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
5d349506feb6 busybox:buildroot-2014.02 "sh" 13 minutes ago Up 1 seconds 0.0.0.0:49153->81/tcp, 0.0.0.0:49154->90/tcp hungry_pike
root@1cbfdcedc5a7:/go/src/github.com/docker/docker# ps aux
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.0 0.1 18168 3100 ? Ss 21:09 0:00 bash
root 8328 0.7 0.6 329072 13420 ? Sl 22:03 0:00 docker -d -s vfs
root 8373 1.0 0.5 196500 10548 ? Sl 22:03 0:00 userland-proxy -proto tcp -host-ip 0.0.0.0 -host-port 49153 -container-ip 10.0.0.2 -container-port 81
root 8382 1.0 0.5 270232 10576 ? Sl 22:03 0:00 userland-proxy -proto tcp -host-ip 0.0.0.0 -host-port 49154 -container-ip 10.0.0.2 -container-port 90
root 8385 1.2 0.0 3168 184 pts/0 Ss+ 22:03 0:00 sh
root 8408 0.0 0.1 15568 2112 ? R+ 22:03 0:00 ps aux
```
This also helps us to cleanly cleanup the proxy processes by stopping
these commands instead of trying to terminate a goroutine.
Signed-off-by: Michael Crosby <michael@docker.com>
* starting with filtering for exit codes. `docker ps -a --filter 'exited=1'`
* API doc for filter parameter
* formatting filters for help usage
* tweaks for review
This requires https://github.com/dotcloud/docker/pull/4430
Docker-DCO-1.1-Signed-off-by: Vincent Batts <vbatts@redhat.com> (github: vbatts)
shuai-z
Tibor Vass
Alexandr Morozov
Malte Janduda
Jessie Frazelle
Srini Brahmaroutu
Erik Hollensbe
Lei Jitang
unclejack
Victor Vieux
Vincent Batts
Tonis Tiigi
Jessica Frazelle
Josh Hawn
Alexander Larsson
Andrea Luzzardi
Michael Crosby
Phil Estes
Daniel, Dao Quang Minh
Andy Goldstein
Brian Goff
Albert Zhang
Dan Walsh
Brian Goff
Alexandr Morozov
Victor Vieux
Derek McGowan
unclejack
Vishnu Kannan
ArikaChen
Rafe Colton
Tianon Gravi
Michael Crosby
Oh Jinkyun
Frederick F. Kautz IV
Eugene Yakubovich
Tim Hockin
Doug Davis
Tim Smith
Hu Tao
Marc Tamsky
Ward Vandewege
Chris Armstrong
Victor Vieux
Vivek Dasgupta
limsy
Solomon Hykes
Michael Crosby
Josiah Kiehl