Commit Graph

3431 Commits

Author SHA1 Message Date
Alexander Larsson e40f5c7cb9 lxc: Work around lxc-start need for private mounts
lxc-start requires / to be mounted private, otherwise the changes
it does inside the container (both mounts and unmounts) will propagate
out to the host.

We work around this by starting up lxc-start in its own namespace where
we set / to rprivate.

Unfortunately go can't really execute any code between clone and exec,
so we can't do this in a nice way. Instead we have a horrible hack that
use the unshare command, the shell and the mount command...
2013-09-26 15:09:32 +00:00
Alexander Larsson d80be57c15 Utils: Add ShellQuoteArguments 2013-09-26 15:09:32 +00:00
Alexander Larsson 20bac716b5 Container: Inject into the mount, not the rwPath
For device-mapper setups we can't just push the file into the rwPath.
2013-09-26 15:09:32 +00:00
Alexander Larsson 2566e2604c api_test: Fix PostContainersCreate
We can't look for the created file in the rwpath, because that
doesn't exist in the device-mapper world, instead look in the
RootfsPath.
2013-09-26 15:09:32 +00:00
Alexander Larsson e1c418cac3 Runtime: Only remove device on destroy if it exists 2013-09-26 15:09:32 +00:00
Alexander Larsson 3343b3f8f8 graph test: Unmount image via image.Unmount()
This helps us track the unmount
2013-09-26 15:09:32 +00:00
Alexander Larsson c6e8813c97 deviceset: Cleanup device sets on test end
We unmount all mounts and deactivate all device mapper devices to
make sure we're left with no leftovers after the test.
2013-09-26 15:09:32 +00:00
Alexander Larsson 251a7ed437 DeviceSet: Add UnmountDevice()
Right now this does nothing but add a new layer, but it means
that all DeviceMounts are paired with DeviceUnmounts so that we
can track (and cleanup) active mounts.
2013-09-26 15:09:31 +00:00
Alexander Larsson 261b0b01df Always start tests from a clean set of loopback images
This way we don't get any issues with leftovers
2013-09-26 15:09:31 +00:00
Alexander Larsson a7fd1fce5d tests: Store the loopback images for test outside unit-tests
This directory is copied to each test prefix which is really
slow with the large loopback mounts.
2013-09-26 15:09:31 +00:00
Alexander Larsson 6938a36c69 Allow specifying the docker client path in _DOCKER_INIT_PATH
I currently need this to get the tests running, otherwise it will
mount the docker.test binary inside the containers, which doesn't
work due to the libdevmapper.so dependency.
2013-09-26 15:09:31 +00:00
Alexander Larsson bc7fa7b957 Limit the amount of prints during normal runs
This removes some Debugf() calls and chages some direct prints to
Debugf(). This means we don't get a bunch of spew when running the
tests.
2013-09-26 15:09:31 +00:00
Alexander Larsson d47c18c5fb Reuse a single DeviceSetDM for all the tests
We wrap the "real" DeviceSet for each test so that we get only
a single device-mapper pool and loopback mounts, but still
separate out the IDs in the tests. This makes the test run
much faster.
2013-09-26 15:09:31 +00:00
Alexander Larsson 0e686fa2f4 Add DeviceSetWrapper
This wraps an existing DeviceSet and just adds a prefix to all ids in
it. This will be useful for reusing a single DeviceSet for all the tests
(but with separate ids)
2013-09-26 15:09:31 +00:00
Alexander Larsson 3f3f5f0bba Delete corresponding Devices when deleting Images
If an image is deleted and there is a corresponding device
for that image we also delete the image.
2013-09-26 15:09:31 +00:00
Alexander Larsson 19ba0b851b Runtime: Delete corresponding devices when deleting container 2013-09-26 15:08:55 +00:00
Alexander Larsson 94fa3c7bb5 Implement container.ExportRW() on device-mapper 2013-09-26 15:08:55 +00:00
Alexander Larsson 223280f319 Make TarFilter more useful
There are a few changes:
* Callers can specify if they want recursive behaviour or not
* All file listings to tar are sent on stdin, to handle long lists better
* We can pass in a list of filenames which will be created as empty
  files in the tarball

This is exactly what we want for the creation of layer tarballs given
a container fs, a set of files to add and a set of whiteout files to create.
2013-09-26 15:08:55 +00:00
Alexander Larsson 8f23945f7f Archive: Fix up tar commandline arguments in TarFilter()
There is no need to duplicate the compression flags for
every element in the filter.
2013-09-26 15:08:55 +00:00
Alexander Larsson 8e8ef7cb5b Implement docker diff for device-mapper
To do diffing we just compare file metadata, so this relies
on things like size and mtime/ctime to catch any changes.
Its *possible* to trick this by updating a file without
changing the size and setting back the mtime/ctime, but
that seems pretty unlikely to happen in reality, and lets
us avoid comparing the actual file data.
2013-09-26 15:08:55 +00:00
Alexander Larsson 8f343ea65a devmapper: Base the device-mapper names on the root dir name
This means the default is "docker-*", but for tests we get separate
prefixes for each test.
2013-09-26 15:08:55 +00:00
Alexander Larsson b125f2334c Image: Always create a .docker-id file in the devices we create
Without this there is really no way to map back from the device-mapper
devices to the actual docker image/container ids in case the json file
somehow got lost
2013-09-26 15:08:55 +00:00
Alexander Larsson a89a51128e Image: Deactivate image device when unmounting container
There is no need to keep all the device-mapper devices active, we
can just activate them on demand if needed.
2013-09-26 15:08:54 +00:00
Alexander Larsson fcd41fe51a Image: Initial support for device-mapper mounts
This supports creating images from layers and mounting them
for running a container.

Not supported yet are:
* Creating diffs between images/containers
* Creating layers for new images from a device-mapper container
2013-09-26 15:08:54 +00:00
Alexander Larsson 53851474c0 Runtime: Add MountMethod to allow AUFS and device-mapper to coexist 2013-09-26 15:08:54 +00:00
Alexander Larsson f317a6b6fe Runtime: Add DeviceSet singleton
This adds a DeviceSet singleton to the Runtime object which will be used for
any DeviceMapper dependent code.
2013-09-26 15:08:54 +00:00
Alexander Larsson 87e248f524 Server: Pass in device-mapper DeviceSet to server
This makes docker (but not docker-init) link to libdevmapper and will
allow it to use the DeviceSet
2013-09-26 15:08:54 +00:00
Alexander Larsson ac194fc696 Add DeviceSet interface
This interface matches the device-mapper implementation (DeviceSetDM)
but is free from any dependencies. This allows core docker code
to refer to a DeviceSet without having an explicit dependency on
the devmapper package.

This is important, because the devmapper package has external
dependencies which are not wanted in the docker client app, as it
needs to run with minimal dependencies in the docker image.
2013-09-26 15:08:54 +00:00
Alexander Larsson 8637ba710e Image: Add runtime and container id args to Mount()
We will later need the runtime to get access to the VolumeSet
singleton, and the container id to have a name for the volume
for the container
2013-09-26 15:08:54 +00:00
Alexander Larsson 0f5ccf934e Runtime: Automatically use docker-init if it exists
In some builds the main docker binary is not statically linked,
and as such not usable in as the .dockerinit binary, for those
cases we look for a separately shipped docker-init binary and
use that instead.
2013-09-26 15:08:54 +00:00
Alexander Larsson 250bc3f615 Add a separate docker-init binary
This may be used for the .dockerinit case if the main binary is not
statically linked.
2013-09-26 15:08:54 +00:00
Alexander Larsson 2b1dc8a8a3 devmapper: Add simple tool to test the DeviceSet commands 2013-09-26 15:08:54 +00:00
Alexander Larsson 0b12702c0c devmapper: Add DeviceSet device-mapper helper
This is a module that uses the device-mapper create CoW snapshots
You instantiate a DeviceSetDM object on a specified root (/var/lib/docker),
and it will create a subdirectory there called "loopback". It will
contain two sparse files which are loopback mounted into
a thin-pool device-mapper device called "docker-pool".

We then create a base snapshot in the pool with an empty filesystem
which can be used as a base for docker snapshots. It also keeps track
of the mapping between docker image ids and the snapshots in the pool.

Typical use of is something like (without error checking):

devices = NewDeviceSetDM("/var/lib/docker")
devices.AddDevice(imageId, "") // "" is the base image id
devices.MountDevice(imageId, "/mnt/image")
 ... extract base image to /mnt/image
devices.AddDevice(containerId, imageId)
devices.MountDevice(containerId, "/mnt/container")
... start container at /mnt/container
2013-09-26 15:08:54 +00:00
Alexander Larsson 739af0a17f Add libdevmapper wrapper 2013-09-26 15:08:54 +00:00
Andy Rothfusz 06c1f000e8 Merge pull request #2000 from tianon/deprecated-docker-latest-tgz
Replace deprecated upgrading reference to docker-latest.tgz, which hasn't been updated since 0.5.3
2013-09-25 13:47:48 -07:00
Andy Rothfusz 00b3acb8ab Merge pull request #2001 from tianon/gentoo-tree
Update Gentoo installation documentation now that we're in the portage tree proper
2013-09-25 13:45:51 -07:00
Victor Vieux 420ba9c85a Merge pull request #2004 from kencochrane/master
fix the error message so it is the same as the regex issue #1999
2013-09-25 09:46:15 -07:00
Ken Cochrane 0f829bf5cf fix the error message so it is the same as the regex issue #1999 2013-09-25 11:33:09 -04:00
Victor Vieux 1363de0934 Merge pull request #1989 from alexlarsson/setsid
Container: Always create a new session for the container
2013-09-25 07:40:51 -07:00
Tianon Gravi d06116d2e8 Update Gentoo installation documentation now that we're in the portage tree proper
We're officially a first-class Gentoo citizen now, which is very exciting.  Many thanks to @gregkh for helping us get here.

I started just adapting sections of language in this document, and realized several bits needed to just be rewritten entirely to be more clear.
2013-09-24 23:26:36 -06:00
Tianon Gravi 595210a370 Replace deprecated upgrading reference to docker-latest.tgz, which hasn't been updated since 0.5.3 2013-09-24 21:25:45 -06:00
Michael Crosby c9b916b293 Update VERSION to 0.6.3-dev 2013-09-24 19:54:13 -07:00
Michael Crosby 1e6370fd4b Merge branch 'release' 2013-09-24 19:54:06 -07:00
Solomon Hykes a59a66528b Merge pull request #1968 from tianon/hack-build-instructions
* Hack: Add several of the small make.sh fixes from #1920, and make the output more consistent and contributor-friendly
2013-09-24 17:38:39 -07:00
Andy Rothfusz b048e9dffc Merge pull request #1983 from blissdev/patch-1
various command fixes in postgres example
2013-09-24 14:03:48 -07:00
blissdev c000e6a7fc revert removal of slash betwixt user/repo 2013-09-24 15:51:21 -05:00
Tianon Gravi aa3de0b849 Add several of the small make.sh fixes from #1920, and make the output more consistent and contributor-friendly, since release instructions already exist in release.sh 2013-09-24 14:36:20 -06:00
Michael Crosby 648d759517 Merge pull request #1972 from dotcloud/bump_0.6.3
Bump to version v0.6.3
2013-09-24 11:20:27 -07:00
Solomon Hykes 03fe5632d0 Merge pull request #1970 from dotcloud/cleanup-hack
Cleanup and reorganize docs and tooling for contributors and maintainers
2013-09-24 10:38:21 -07:00
Victor Vieux 7447867edd Merge pull request #1982 from modcloth-labs/spelling-fix-for-the-word-protocol
Minor spelling correction of protocoll -> protocol
2013-09-24 07:40:21 -07:00