This passed the --net=container:CONTINER_ID to lxc-start as --share-net
Docker-DCO-1.1-Signed-off-by: Abin Shahab <ashahab@altiscale.com> (github: ashahab-altiscale)
Running parseVolumesFromSpec on all VolumesFrom specs before initialize
any mounts endures that we don't leave container.Volumes in an
inconsistent (partially initialized) if one of out mount groups is not
available (e.g. the container we're trying to mount from does not
exist).
Keeping container.Volumes in a consistent state ensures that next time
we Start() the container, it'll run prepareVolumes() again.
The attached test demonstrates that when a container fails to start due
to a missing container specified in VolumesFrom, it "remembers" a Volume
that worked.
Fixes: #8726
Signed-off-by: Thomas Orozco <thomas@orozco.fr>
Since the containers can handle the out of memory kernel kills gracefully, docker
will only provide out of memory information as an additional metadata as part of
container status.
Docker-DCO-1.1-Signed-off-by: Vishnu Kannan <vishnuk@google.com> (github: vishh)
In previous patch I had introduce json:"-" tags to be on safer side to make
sure certain fields are not marshalled/unmarshalled. But struct fields
starting with small letter are not exported so they will not be marshalled
anyway. So remove json:"-" tags from there.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
Lxc driver was throwing errors for mounts where the mount point does not exist in the container.
This adds a create=dir/file mount option to the lxc template, to alleviate this issue.
Docker-DCO-1.1-Signed-off-by: Abin Shahab <ashahab@altiscale.com> (github: ashahab-altiscale)
We removed the syncpipe package and replaced it with specific calls to
create a new *os.File from a specified fd passed to the process. This
reduced code and an extra object to manage the container's init
lifecycle.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
My pull request failed the build due to gofmat issues. I have run gofmt
on specified files and this commit fixes it.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
The way thin-pool right now is designed, user space is supposed to keep
track of what device ids have already been used. If user space tries to
create a new thin/snap device and device id has already been used, thin
pool retuns -EEXIST.
Upon receiving -EEXIST, current docker implementation simply tries the
NextDeviceId++ and keeps on doing this till it finds a free device id.
This approach has two issues.
- It is little suboptimal.
- If device id already exists, current kenrel implementation spits out
a messsage on console.
[17991.140135] device-mapper: thin: Creation of new snapshot 33 of device 3 failed.
Here kenrel is trying to tell user that device id 33 has already been used.
And this shows up for every device id docker tries till it reaches a point
where device ids are not used. So if there are thousands of container and
one is trying to create a new container after fresh docker start, expect
thousands of such warnings to flood console.
This patch saves the NextDeviceId in a file in
/var/lib/docker/devmapper/metadata/deviceset-metadata and reads it back
when docker starts. This way we don't retry lots of device ids which
have already been used.
There might be some device ids which are free but we will get back to them
once device numbers wrap around (24bit limit on device ids).
This patch should cut down on number of kernel warnings.
Notice that I am creating a deviceset metadata file which is a global file
for this pool. So down the line if we need to save more data we should be
able to do that.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
I was trying to save nextDeviceId to a file but it would not work and
json.Marshal() will do nothing. Then some search showed that I need to
make first letter of struct field capital, exporting this field and
now json.Marshal() works.
This is a preparatory patch for the next one.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
Currently we save device metadata and have a helper function saveMetadata()
which converts data in json format as well as saves it to file. For
converting data in json format, one needs to know what is being saved.
Break this function down in two functions. One function only has file
write capability and takes in argument about byte array of json data.
Now this function does not have to know what data is being saved. It
only knows about a stream of json data is being saved to a file.
This allows me to reuse this function to save a different type of
metadata. In this case I am planning to save NextDeviceId so that
docker can use this device Id upon next restart. Otherwise docker
starts from 0 which is suboptimal.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
Fixed the following errors:
1. Request(0) causes a dead loop when the map is full and map.last == BEGIN.
2. When map.last is the only available port (or ip), Request(0) returns ErrAllPortsAllocated (or ErrNoAvailableIPs). Exception is when map.last == BEGIN.
Signed-off-by: shuai-z <zs.broccoli@gmail.com>
Fixes#8832
All stdio streams need to finish writing before the
connection can be closed.
Signed-off-by: Tõnis Tiigi <tonistiigi@gmail.com> (github: tonistiigi)
If we need to raise an error, make sure the internal state is clean, because
a successful driver.Get() may have its internal state changed (eg. counting,
or mounts), while callers will only do that after a succussful Mount().
Signed-off-by: shuai-z <zs.broccoli@gmail.com>
In an effort to make layer content 'stable' between import
and export from two different graph drivers, we must resolve
an issue where AUFS produces metadata files in its layers
which other drivers explicitly ignore when importing.
The issue presents itself like this:
- Generate a layer using AUFS
- On commit of that container, the new stored layer contains
AUFS metadata files/dirs. The stored layer content has some
tarsum value: '1234567'
- `docker save` that image to a USB drive and `docker load`
into another docker engine instance which uses another
graph driver, say 'btrfs'
- On load, this graph driver explicitly ignores any AUFS metadata
that it encounters. The stored layer content now has some
different tarsum value: 'abcdefg'.
The only (apparent) useful aufs metadata to keep are the psuedo link
files located at `/.wh..wh.plink/`. Thes files hold information at the
RW layer about hard linked files between this layer and another layer.
The other graph drivers make sure to copy up these psuedo linked files
but I've tested out a few different situations and it seems that this
is unnecessary (In my test, AUFS already copies up the other hard linked
files to the RW layer).
This changeset adds explicit exclusion of the AUFS metadata files and
directories (NOTE: not the whiteout files!) on commit of a container
using the AUFS storage driver.
Also included is a change to the archive package. It now explicitly
ignores the root directory from being included in the resulting tar archive
for 2 reasons: 1) it's unnecessary. 2) It's another difference between
what other graph drivers produce when exporting a layer to a tar archive.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
This backend uses the overlayfs union filesystem for containers
plus hard link file sharing for images.
Each container/image can have a "root" subdirectory which is a plain
filesystem hierarchy, or they can use overlayfs.
If they use overlayfs there is a "upper" directory and a "lower-id"
file, as well as "merged" and "work" directories. The "upper"
directory has the upper layer of the overlay, and "lower-id" contains
the id of the parent whose "root" directory shall be used as the lower
layer in the overlay. The overlay itself is mounted in the "merged"
directory, and the "work" dir is needed for overlayfs to work.
When a overlay layer is created there are two cases, either the
parent has a "root" dir, then we start out with a empty "upper"
directory overlaid on the parents root. This is typically the
case with the init layer of a container which is based on an image.
If there is no "root" in the parent, we inherit the lower-id from
the parent and start by making a copy if the parents "upper" dir.
This is typically the case for a container layer which copies
its parent -init upper layer.
Additionally we also have a custom implementation of ApplyLayer
which makes a recursive copy of the parent "root" layer using
hardlinks to share file data, and then applies the layer on top
of that. This means all chile images share file (but not directory)
data with the parent.
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
The vfs storage driver currently shells out to the `cp` binary on the host
system to perform an 'archive' copy of the base image to a new directory.
The archive option preserves the modified time of the files which are created
but there was an issue where it was unable to preserve the modified time of
copied symbolic links on some host systems with an outdated version of `cp`.
This change no longer relies on the host system implementation and instead
utilizes the `CopyWithTar` function found in `pkg/archive` which is used
to copy from source to destination directory using a Tar archive, which
should correctly preserve file attributes.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
If we first request port 49153 (BeginPortRange) explicitly, and later some time request the next free port (of same ip/proto) by calling RequestPort() with port number 0, we will again get 49153 returned, even if it's currently in use. Because findPort() blindly retured BeginPortRange the first run, without checking if it has already been taken.
Signed-off-by: shuai-z <zs.broccoli@gmail.com>
when a container failed to start, saves the error message into State.Error so
that it can be retrieved when calling `docker inspect` instead of having to
look at the log
Docker-DCO-1.1-Signed-off-by: Daniel, Dao Quang Minh <dqminh89@gmail.com> (github: dqminh)
Never close attached stream before both stdout and stderr have written
all their buffered contents. Remove stdinCloser because it is not needed
any more as the stream is closed anyway after attach has finished.
Fixes#3631
Signed-off-by: Andy Goldstein <agoldste@redhat.com>
Fixes#1992
Right now when you `docker cp` a path which is in a volume, the cp
itself works, however you end up getting files that are in the
container's fs rather than the files in the volume (which is not in the
container's fs).
This makes it so when you `docker cp` a path that is in a volume it
follows the volume to the real path on the host.
archive.go has been modified so that when you do `docker cp mydata:/foo
.`, and /foo is the volume, the outputed folder is called "foo" instead
of the volume ID (because we are telling it to tar up
`/var/lib/docker/vfs/dir/<some id>` and not "foo", but the user would be
expecting "foo", not the ID
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Stable IPs causes some regressions in the way people use Docker, see GH#8493.
Reverting it for 1.3, we'll enable it back for the next release.
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
When a container is restarted all the volume configs are parsed again.
Even if the volume was already handled in a previous start it was still
calling "FindOrCreateVolume" on the volume repo causing a new volume to
be created.
This wasn't being detected because as part of the mount initialization
it checks to see if the the _mount_ was already initialized, but this
happens after the parsing of the configs.
So a check is added during parsing to skip a volume which was already
created for that container.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Volume refs were not being restored on daemon restart.
This made it possible to remove a volume being used by other containers
after a daemon restart.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
When stdout/stderr is closed prematurely, the proxy's writes to stdout/stderr
(i.e. `log.Errorf/log.Printf`) will returns with EPIPE error, and go runtime
will terminate the proxy when stdout/stderr writes trigger 10 EPIPE errors.
instead of using stdout/stderr as the status handler, we pass an extra file to
the child process and write `0\n` or `1\nerror message` to it and close it
after. This allow the child process to handle stdout/stderr as normal.
Docker-DCO-1.1-Signed-off-by: Daniel, Dao Quang Minh <dqminh89@gmail.com> (github: dqminh)
Currently if you start the docker -d on a system with 127.0.0.1 in /etc/resolv.conf
It will set the default dns to 8.8.8.8 8.8.4.4 permanently.
This causes a problem at boot on Fedora machines where NetworkManager has not
populated /etc/resolv.conf before docker gets started.
This fix checks /etc/resolv.conf on every docker run. And only populates
daemon.config.Dns if the user specified it on the command line.
Docker-DCO-1.1-Signed-off-by: Dan Walsh <dwalsh@redhat.com> (github: rhatdan)
The defer logic was a little tricky and was hiding one bug: `err` was
being redefined (with `:=`) and thus it escaped the defer error checking
logic.
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
Prior to the volumes re-factor, data was not being copied on
volumes-from or host-mounted volumes.
After the re-factor, data was being copied for volumes-from.
This reverts this unintentional change in behavior.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Needed to check if the mode was invalid and return error, not valid and
return error.
This didn't get picked up because the existing integration-cli tests
were all either expecting errors when a valid mode was passed in (e.g.
"ro" passed in, we expected an error because it was testing write). So
modified a test which was testing for "rw" to actually pass in "rw"
instead of assuming the "rw"
Docker-DCO-1.1-Signed-off-by: Brian Goff <bgoff@cpuguy83-mbp.home> (github: cpuguy83)
This change will allocate network settings (IP and public ports) at
container creation rather than start and keep them throughout the
lifetime of the container (i.e. until it gets destroyed) instead of
discarding them when the container is stopped.
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
Since we are moving network allocation outside of container scope (it
will be managed by create/destroy), those functions need to be
accessible from the outside.
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
Right now, MAC addresses are randomly generated by the kernel when
creating the veth interfaces.
This causes different issues related to ARP, such as #4581, #5737 and #8269.
This change adds support for consistent MAC addresses, guaranteeing that
an IP address will always end up with the same MAC address, no matter
what.
Since IP addresses are already guaranteed to be unique by the
IPAllocator, MAC addresses will inherit this property as well for free.
Consistent mac addresses is also a requirement for stable networking (#8297)
since re-using the same IP address on a different MAC address triggers the ARP
issue.
Finally, this change makes the MAC address accessible through docker
inspect, which fixes#4033.
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
Add support for pulling signed images from a version 2 registry.
Only official images within the library namespace will be pull from the
new registry and check the build signature.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Now that the archive package does not depend on any docker-specific
packages, only those in pkg and vendor, it can be safely moved into pkg.
Signed-off-by: Rafe Colton <rafael.colton@gmail.com>
This is the first of two steps to break the archive package's dependence
on utils so that archive may be moved into pkg. Also, the `Go()`
function is small, concise, and not specific to the docker internals, so
it is a good candidate for pkg.
Signed-off-by: Rafe Colton <rafael.colton@gmail.com>
security-opts will allow you to customise the security subsystem.
For example the labeling system like SELinux will run on a container.
--security-opt="label:user:USER" : Set the label user for the container
--security-opt="label:role:ROLE" : Set the label role for the container
--security-opt="label:type:TYPE" : Set the label type for the container
--security-opt="label:level:LEVEL" : Set the label level for the container
--security-opt="label:disabled" : Turn off label confinement for the container
Since we are passing a list of string options instead of a space separated
string of options, I will change function calls to use InitLabels instead of
GenLabels. Genlabels interface is Depracated.
Docker-DCO-1.1-Signed-off-by: Dan Walsh <dwalsh@redhat.com> (github: rhatdan)
This also removes dead code in the native driver for a past feature that
was never fully implemented.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Since RemoveLocalDns patch will remove all localhost entries
from resolv.conf we no longer need anything more then
!bytes.Contains(resolvConf, []byte("nameserver")
To check for no nameserver entry in dns config.
Docker-DCO-1.1-Signed-off-by: Dan Walsh <dwalsh@redhat.com> (github: rhatdan)
We have a bug report complaining about docker dumping the contents of the
hosts resolv.conf if it container 127.0.0.1. They asked that instead
of dropping the file altogether, that we just remove the line.
This patch removes the 127.0.0.1 lines, if they exist and then
checks if any nameserver lines exist.
Docker-DCO-1.1-Signed-off-by: Dan Walsh <dwalsh@redhat.com> (github: rhatdan)
Currently, the HostConfig is only passed from the CLI to Docker only
when issuing a docker create, but not when doing a docker run.
In the near future, in order to allocate ports at creation time rather
than start time, we will need to have the HostConfig readily available
at container creation.
This PR makes the client always pass the HostConfig when creating a
container (regardless of whether it's for a run or create).
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
Since it is possible to request a specific IP, IPAllocator has to verify
that the request is within boundaries.
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
On Fedora and RHEL we ship selinux-enabled flag in the docker.service config,
but if people setup the /var/lib/docker as btrfs and disable SELinux,
we should not block the daemon from running.
Docker-DCO-1.1-Signed-off-by: Dan Walsh <dwalsh@redhat.com> (github: rhatdan)
If `--storage-opt dm.datadev=/dev/loop0 --storage-opt
dm.metadatadev=/dev/loop1 ` were provided, the information was not
reflected in the information output.
Closes: #7137
Signed-off-by: Vincent Batts <vbatts@redhat.com>
There were a new areas in the brige driver that did not need to have log
output. Those were removed. Also set the engine's logging to false
when running the integration tests.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
For the cases where --bip option is used it is sometimes best to disable
IP masquerading as the provided bridge IP range may be routable.
Signed-off-by: Eugene Yakubovich <eugene.yakubovich@coreos.com>
This adds a --add-host host:ip flag which appends lines to /etc/hosts. This is needed in places where you want the container to get a different name resolution than it would through DNS. This was submitted before as #5525, closed, and now I am re-opening. It has come up 2 or 3 times in the last couple days.
Signed-off-by: Tim Hockin <thockin@google.com>
This exposes the already existing "create container" operation. It is
very similar to "docker run -d" except it doesn't actually start the
container, but just prepares it. It can then be manually started using
"docker start" at any point.
Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
Conflicts:
api/client/commands.go
runconfig/parse.go
server/container.go
Docker-DCO-1.1-Signed-off-by: Tibor Vass <teabee89@gmail.com> (github: tiborvass)
Some graphdrivers are Differs and type assertions are made
in various places throughout the project. Differ offers some
convenience in generating/applying diffs of filesystem layers
but for most graphdrivers another code path is taken.
This patch brings all of the logic related to filesystem
diffs in one place, and simplifies the implementation of some
common types like Image, Daemon, and Container.
Signed-off-by: Josh Hawn <josh.hawn@docker.com>
Use utils.RFC3339NanoFixed ("2006-01-02T15:04:05.000000000Z07:00")
instead of time.RFC3339Nano to format our log timestamps - this way
things are aligned, in particular the nano seconds are padded with zeros
Signed-off-by: Doug Davis <dug@us.ibm.com>
1. /container/<name>/exec - Creates a new exec command instance in the daemon and container '<name>'. Returns an unique ID for each exec command.
2. /exec/<name>/start - Starts an existing exec command instance. Removes the exec command from the daemon once it completes.
Adding /exec/<name>/resize to resize tty session of an exec command.
Docker-DCO-1.1-Signed-off-by: Vishnu Kannan <vishnuk@google.com> (github: vishh)
Adds support for a --registry-mirror=scheme://<host>[:port]
daemon flag. The flag may be present multiple times. If
provided, mirrors are prepended to the list of endpoints used
for image pull. Note that only mirrors of the public
index.docker.io registry are supported, and image/tag resolution
is still performed via the official index.
Docker-DCO-1.1-Signed-off-by: Tim Smith <timbot@google.com> (github: timbot)
commit 4aa5da278f moves `Console` from Command to
ProcessConfig, but missed the change in lxc_template. Therefore creating a
container with tty using lxc driver with fail with error
template: lxc:60:20: executing "lxc" at <.Console>: Console is not a field of
struct type struct { *execdriver.Command; AppArmor bool; ProcessLabel string; MountLabel string }
This changes lxc_console template to refers to `.ProcessConfig.Console`
Docker-DCO-1.1-Signed-off-by: Daniel, Dao Quang Minh <dqminh89@gmail.com> (github: dqminh)
Since these will be shared between containers we want to label
them as svirt_sandbox_file_t:s0. That will allow multiple containers
to write to them.
Currently we are allowing container domains to read/write all content in
/var/lib/docker because of container volumes. This is a big security hole
in our SELinux story.
This patch will allow us to tighten up the security of docker containers.
Docker-DCO-1.1-Signed-off-by: Dan Walsh <dwalsh@redhat.com> (github: rhatdan)
This commit fixes following FIXMEs:
// FIXME: rename "delete" to "rm" for consistency with the CLI command
// FIXME: rename ContainerDestroy to ContainerRm for consistency with the CLI command
Signed-off-by: lim seong yeol <seongyeol37@gmail.com>
Daniel, Dao Quang Minh
unclejack
Tibor Vass
Abin Shahab
Michael Crosby
Thomas Orozco
unclejack
Vishnu Kannan
Alexandr Morozov
Vincent Batts
Vivek Goyal
Doug Davis
Michael Crosby
Erik Hollensbe
Blake Geno
Alexandr Morozov
shuai-z
Malte Janduda
Jessie Frazelle
Srini Brahmaroutu
Lei Jitang
Victor Vieux
Tonis Tiigi
Jessica Frazelle
Josh Hawn
Alexander Larsson
Andrea Luzzardi
Phil Estes
Andy Goldstein
Brian Goff
Albert Zhang
Dan Walsh
Brian Goff
Victor Vieux
Derek McGowan
ArikaChen
Rafe Colton
Tianon Gravi
Oh Jinkyun
Frederick F. Kautz IV
Eugene Yakubovich
Tim Hockin
Tim Smith
Hu Tao
Marc Tamsky
Ward Vandewege
Chris Armstrong
Victor Vieux
Vivek Dasgupta
limsy
Solomon Hykes