Commit Graph

3720 Commits

Author SHA1 Message Date
Michael Crosby 723de87681 Move run -rm to the cli only 2013-09-26 14:52:37 -07:00
David Sissitka c22e377a6d Used tabs for alignment when I should've used spaces. Fixed. 2013-09-26 13:56:59 -07:00
Colin Dunklau 3f0d0075f8 Hint for Remote API doc details in header links 2013-09-26 15:30:26 -05:00
David Sissitka f8c9f11f06 Configured FPM to make /etc/init/docker.conf a config file. 2013-09-26 12:52:43 -07:00
Alexander Larsson 80319add55 lxc: Allow set_file_cap capability in container
This means you're able to set the bits for capabilities on files
inside the container. This is needed for e.g. many fedora packages
as they use finegrained capabilities rather than setuid binaries.

This is safe as we're not adding capabilities really, since the
container is already allowed to create setuid binaries. Setuid
binaries are strictly more powerful that any capabilities (as root implies
all capabilities).

This doesn't mean the container can *gain* capabilities that it
doesn't already have though. The actual set of caps are strictly
decreasing.
2013-09-26 21:41:45 +02:00
Victor Vieux ebfa24acb0 go fmt and aufs support removed 2013-09-26 15:40:13 +00:00
Victor Vieux 5e1d540209 Revert "add a -mount-method flag"
This reverts commit e52d756f40c9ccf8b37ca496cb72be057c909ed7.
2013-09-26 15:14:03 +00:00
Victor Vieux c1e25d7273 add a -mount-method flag 2013-09-26 15:10:01 +00:00
Alexander Larsson d263aa6ca9 RootIsShared: Fix root detection
Column 4 is the mount position, column 3 will not always be
"/" for the root. On one of my system its "/root".
2013-09-26 15:09:33 +00:00
Alexander Larsson 03320f0d1c Tests: Clean up any old devmapper leftovers before starting tests 2013-09-26 15:09:33 +00:00
Alexander Larsson 6c7ae06435 Image.Changes: Deactivate image device after unmounting it
There is no need to keep the image device around if we were the
onces creating the device.
2013-09-26 15:09:33 +00:00
Alexander Larsson 395bce4c41 Add DeviceSet.HasActivatedDevice()
This lets you see if a device has been activated
2013-09-26 15:09:33 +00:00
Alexander Larsson 41399ac005 image: Unmount before removing device in error paths
The device remove fails unless we unmount first
2013-09-26 15:09:33 +00:00
Alexander Larsson 67788723c9 runtime test: Ensure all containers are unmounted at nuke()
Otherwise we may leave around e.g. devmapper mounts
2013-09-26 15:09:33 +00:00
Alexander Larsson f99f39abaa ShellQuoteArguments: Fix quoting
This accidentally used two quotes to start/end each quoted string.
2013-09-26 15:09:33 +00:00
Alexander Larsson 009d0f9d81 Image: unmount device before removing it on failures
If we don't do this the remove will fail due to EBUSY
2013-09-26 15:09:32 +00:00
Alexander Larsson ed65815613 image: Handle systems that don't support O_PATH when updating timestamp
Older kernel can't handle O_PATH in open() so this will
fail on dirs and symlinks. For dirs wa can fallback to
the normal Utimes, but for symlinks there is not much to do
but ignore their timestamps.
2013-09-26 15:09:32 +00:00
Alexander Larsson cc28829429 devmapper: Fix loopback mount code
Typo in the loop-control code made it always fall back to the
old method of opening loopback devices.
2013-09-26 15:09:32 +00:00
Alexander Larsson 062a2b32e9 applyLayer() use btrfs reflinks if availible
We use the new file copy helper which uses btrfs reflinks if availible.
2013-09-26 15:09:32 +00:00
Alexander Larsson cda8754013 Add CopyFile that can use btrfs reflinks if availible 2013-09-26 15:09:32 +00:00
Alexander Larsson 5415804c9d Remove accidental commit that enabled MountMethodFilesystem 2013-09-26 15:09:32 +00:00
Alexander Larsson adae684987 Add trivial copy-based CoW backend
This creates a container by copying the corresponding files
from the layers into the containers. This is not gonna be very useful
on a developer setup, as there is no copy-on-write or general diskspace
sharing. It also makes container instantiation slower.

However, it may be useful in deployment where we don't always have a lot
of containers running (long-running daemons) and where we don't
do a lot of docker commits.
2013-09-26 15:09:32 +00:00
Alexander Larsson ad0a6a03e3 Add Changes.ChangesLayers()
This calculates the difference between a set of layers and a
directory tree.
2013-09-26 15:09:32 +00:00
Alexander Larsson 36603e68e3 Changes: Better metadata comparison
Change the comparison to better handle files that are copied during
container creation but not actually changed:

* Inode - this will change during a copy
* ctime - this will change during a copy (as we can't set it back)
* blocksize - this will change for sparse files during copy
* size for directories - this can change anytime but doesn't
  necessarily reflect an actual contents change
* Compare mtimes at microsecond precision (as this is what utimes has)
2013-09-26 15:09:32 +00:00
Alexander Larsson 99c7d129f4 Image.applyLayer: Be better at creating identical files
There are some changes here that make the file metadata better match
the layer files:

* Set the mode of the file after the chown, as otherwise the per-group/uid
  specific flags and e.g. sticky bit is lost
* Use lchown instead of chown
* Delay mtime updates to after all other changes so that later file
  creation doesn't change the mtime for the parent directory
* Use Futimes in combination with O_PATH|O_NOFOLLOW to set mtime on symlinks
2013-09-26 15:09:32 +00:00
Alexander Larsson 02b5f1369c Change how ChangesDirs() works
Rather than scan the files in the old directory twice to detect the
deletions we now scan both directories twice and then do all the
diffing on the in-memory structure.

This is more efficient, but it also lets us diff more complex things
later that are not exact on-disk trees.
2013-09-26 15:09:32 +00:00
Alexander Larsson d478a4bb54 RootIsShared() - Fix array out of bounds error
This happened for me on the last (empty) line, but better safe than sorry
so we make the check general.
2013-09-26 15:09:32 +00:00
Alexander Larsson c199ed228b devmapper: Move init layer to top rather than bottom
The init layer needs to be topmost to make sure certain files
are always there (for instance, the ubuntu:12.10 image wrongly
has /dev/shm being a symlink to /run/shm, and we need to override
that). However, previously the devmapper code implemented the
init layer by putting it in the base devmapper device, which meant
layers above it could override these files (so that ubuntu:12.10
broke).

So, instead we put the base layer in *each* images devmapper device.
This is "safe" because we still have the pristine layer data
in the layer directory. Also, it means we diff the container
against the image with the init layer applied, so it won't show
up in diffs/commits.
2013-09-26 15:09:32 +00:00
Alexander Larsson e40f5c7cb9 lxc: Work around lxc-start need for private mounts
lxc-start requires / to be mounted private, otherwise the changes
it does inside the container (both mounts and unmounts) will propagate
out to the host.

We work around this by starting up lxc-start in its own namespace where
we set / to rprivate.

Unfortunately go can't really execute any code between clone and exec,
so we can't do this in a nice way. Instead we have a horrible hack that
use the unshare command, the shell and the mount command...
2013-09-26 15:09:32 +00:00
Alexander Larsson d80be57c15 Utils: Add ShellQuoteArguments 2013-09-26 15:09:32 +00:00
Alexander Larsson 20bac716b5 Container: Inject into the mount, not the rwPath
For device-mapper setups we can't just push the file into the rwPath.
2013-09-26 15:09:32 +00:00
Alexander Larsson 2566e2604c api_test: Fix PostContainersCreate
We can't look for the created file in the rwpath, because that
doesn't exist in the device-mapper world, instead look in the
RootfsPath.
2013-09-26 15:09:32 +00:00
Alexander Larsson e1c418cac3 Runtime: Only remove device on destroy if it exists 2013-09-26 15:09:32 +00:00
Alexander Larsson 3343b3f8f8 graph test: Unmount image via image.Unmount()
This helps us track the unmount
2013-09-26 15:09:32 +00:00
Alexander Larsson c6e8813c97 deviceset: Cleanup device sets on test end
We unmount all mounts and deactivate all device mapper devices to
make sure we're left with no leftovers after the test.
2013-09-26 15:09:32 +00:00
Alexander Larsson 251a7ed437 DeviceSet: Add UnmountDevice()
Right now this does nothing but add a new layer, but it means
that all DeviceMounts are paired with DeviceUnmounts so that we
can track (and cleanup) active mounts.
2013-09-26 15:09:31 +00:00
Alexander Larsson 261b0b01df Always start tests from a clean set of loopback images
This way we don't get any issues with leftovers
2013-09-26 15:09:31 +00:00
Alexander Larsson a7fd1fce5d tests: Store the loopback images for test outside unit-tests
This directory is copied to each test prefix which is really
slow with the large loopback mounts.
2013-09-26 15:09:31 +00:00
Alexander Larsson 6938a36c69 Allow specifying the docker client path in _DOCKER_INIT_PATH
I currently need this to get the tests running, otherwise it will
mount the docker.test binary inside the containers, which doesn't
work due to the libdevmapper.so dependency.
2013-09-26 15:09:31 +00:00
Alexander Larsson bc7fa7b957 Limit the amount of prints during normal runs
This removes some Debugf() calls and chages some direct prints to
Debugf(). This means we don't get a bunch of spew when running the
tests.
2013-09-26 15:09:31 +00:00
Alexander Larsson d47c18c5fb Reuse a single DeviceSetDM for all the tests
We wrap the "real" DeviceSet for each test so that we get only
a single device-mapper pool and loopback mounts, but still
separate out the IDs in the tests. This makes the test run
much faster.
2013-09-26 15:09:31 +00:00
Alexander Larsson 0e686fa2f4 Add DeviceSetWrapper
This wraps an existing DeviceSet and just adds a prefix to all ids in
it. This will be useful for reusing a single DeviceSet for all the tests
(but with separate ids)
2013-09-26 15:09:31 +00:00
Alexander Larsson 3f3f5f0bba Delete corresponding Devices when deleting Images
If an image is deleted and there is a corresponding device
for that image we also delete the image.
2013-09-26 15:09:31 +00:00
Alexander Larsson 19ba0b851b Runtime: Delete corresponding devices when deleting container 2013-09-26 15:08:55 +00:00
Alexander Larsson 94fa3c7bb5 Implement container.ExportRW() on device-mapper 2013-09-26 15:08:55 +00:00
Alexander Larsson 223280f319 Make TarFilter more useful
There are a few changes:
* Callers can specify if they want recursive behaviour or not
* All file listings to tar are sent on stdin, to handle long lists better
* We can pass in a list of filenames which will be created as empty
  files in the tarball

This is exactly what we want for the creation of layer tarballs given
a container fs, a set of files to add and a set of whiteout files to create.
2013-09-26 15:08:55 +00:00
Alexander Larsson 8f23945f7f Archive: Fix up tar commandline arguments in TarFilter()
There is no need to duplicate the compression flags for
every element in the filter.
2013-09-26 15:08:55 +00:00
Alexander Larsson 8e8ef7cb5b Implement docker diff for device-mapper
To do diffing we just compare file metadata, so this relies
on things like size and mtime/ctime to catch any changes.
Its *possible* to trick this by updating a file without
changing the size and setting back the mtime/ctime, but
that seems pretty unlikely to happen in reality, and lets
us avoid comparing the actual file data.
2013-09-26 15:08:55 +00:00
Alexander Larsson 8f343ea65a devmapper: Base the device-mapper names on the root dir name
This means the default is "docker-*", but for tests we get separate
prefixes for each test.
2013-09-26 15:08:55 +00:00
Alexander Larsson b125f2334c Image: Always create a .docker-id file in the devices we create
Without this there is really no way to map back from the device-mapper
devices to the actual docker image/container ids in case the json file
somehow got lost
2013-09-26 15:08:55 +00:00