Commit Graph

532 Commits

Author SHA1 Message Date
Dan Walsh 1716d497a4 Relabel BTRFS Content on container Creation
This change will allow us to run SELinux in a container with
BTRFS back end.  We continue to work on fixing the kernel/BTRFS
but this change will allow SELinux Security separation on BTRFS.

It basically relabels the content on container creation.

Just relabling -init directory in BTRFS use case. Everything looks like it
works. I don't believe tar/achive stores the SELinux labels, so we are good
as far as docker commit.

Tested Speed on startup with BTRFS on top of loopback directory. BTRFS
not on loopback should get even better perfomance on startup time.  The
more inodes inside of the container image will increase the relabel time.

This patch will give people who care more about security the option of
runnin BTRFS with SELinux.  Those who don't want to take the slow down
can disable SELinux either in individual containers or for all containers
by continuing to disable SELinux in the daemon.

Without relabel:

> time docker run --security-opt label:disable fedora echo test
test

real    0m0.918s
user    0m0.009s
sys    0m0.026s

With Relabel

test

real    0m1.942s
user    0m0.007s
sys    0m0.030s

Signed-off-by: Dan Walsh <dwalsh@redhat.com>

Signed-off-by: Dan Walsh <dwalsh@redhat.com>
2015-11-11 14:49:27 -05:00
Vivek Goyal 07ff17fb85 devmapper: Switch to xfs as default filesystem if supported
If platform supports xfs filesystem then use xfs as default filesystem 
for container rootfs instead of ext4. Reason being that ext4 is pre-allcating
lot of metadata (around 1.8GB on 100G thin volume) and that can take long
enough on AWS storage that systemd times out and docker fails to start.

If one disables pre-allocation of ext4 metadata, then it will be allocated
when containers are mounted and we will have multiple copies of metadata
per container. For a 100G thin device, it was around 1.5GB of metadata
per container.

ext4 has an optimization to skip zeroing if discards are issued and
underlying device guarantees that zero will be returned when discarded
blocks are read back. devicemapper thin devices don't offer that guarantee
so ext4 optimization does not kick in. In fact given discards are optional
and can be dropped on the floor if need be, it looks like it might not be
possible to guarantee that all the blocks got discarded and if read back
zero will be returned.

Signed-off-by: Anusha Ragunathan <anusha@docker.com>
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
2015-11-11 12:07:35 -05:00
Vivek Goyal 83a34e000b devmapper: Warn if user specified a filesytem and base device already has fs
If user wants to use a filesystem it can be specified using dm.fs=<filesystem>
option. It is possible that docker already had base image and a filesystem
on that. Later if user wants to change file system using dm.fs= option
and restarts docker, that's not possible. Warn user about it.

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
2015-11-11 12:07:35 -05:00
Vincent Batts 1066725395 Merge pull request #16774 from tonistiigi/15643-aufs-plnk
Copy aufs hardlinks to top layer
2015-11-11 07:59:41 +01:00
Lei Jitang e035d27223 Show warning when user specify dm.basesize for already initialized devicemapper drive
Signed-off-by: Lei Jitang <leijitang@huawei.com>
2015-11-10 14:50:19 +08:00
David Calavera 58b270c338 Merge pull request #17431 from vdemeester/hope-it-does-not-broke-everything-again
Another try at dockerversion placeholder for library import
2015-11-09 13:15:50 -08:00
Vincent Demeester 8054a30387 dockerversion placeholder for library import
- Add a *version* file placeholder.
- Update autogen and builds to use it and an autogen build flag

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2015-11-09 19:32:46 +01:00
Vincent Batts 10df5647bd Merge pull request #17757 from rhvgoyal/log-blkid-failure
devmapper: Provide more error information if blkid fails
2015-11-07 09:16:14 -05:00
David Calavera 1ba280a8db Merge pull request #17576 from Microsoft/10662-graphtest
Windows: Fix daemon\graphdriver\graphtest
2015-11-06 15:13:31 -08:00
Vivek Goyal 2c8b7c597a devmapper: Provide more error information if blkid fails
Right now if blkid fails we are just logging a debug message and don;t return
the actual error to caller. Caller gets the error message that thin pool
base device UUID verification failed and it might give impression that thin
pool changed. But that's not the case. Thin pool is in such a state that we
could not even query the thin device UUID. Retrun error message appropriately
to make situation more clear.

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
2015-11-06 08:21:20 -05:00
David Calavera 3b5fac462d Remove LXC support.
The LXC driver was deprecated in Docker 1.8.
Following the deprecation rules, we can remove a deprecated feature
after two major releases. LXC won't be supported anymore starting on Docker 1.10.

Signed-off-by: David Calavera <david.calavera@gmail.com>
2015-11-05 17:09:57 -05:00
John Howard 8764967381 Windows: Fix daemon\graphdriver\graphtest
Signed-off-by: John Howard <jhoward@microsoft.com>
2015-10-31 21:14:18 -07:00
Brian Goff b78ca243d9 Revert "dockerversion placeholder for library-import"
This reverts commit d5cd032a86.

Commit caused issues on systems with case-insensitive filesystems.
Revert for now

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2015-10-27 21:23:53 -04:00
Vincent Demeester d5cd032a86 dockerversion placeholder for library-import
- Move autogen/dockerversion to version
- Update autogen and "builds" to use this package and a build flag

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2015-10-27 20:36:07 +01:00
Vivek Goyal 2f16895ee9 devmapper: Drop devices lock before returning from function
cleanupDeleted() takes devices.Lock() but does not drop it if there are
no deleted devices. Hence docker deadlocks if one is using deferred
device deletion feature. (--storage-opt dm.use_deferred_deletion=true).

Fix it. Drop the lock before returning.

Also added a unit test case to make sure in future this can be easily
detected if somebody changes the function.

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
2015-10-19 17:51:17 -04:00
Vincent Demeester 5ecbc9747f Merge pull request #16303 from coolljt0725/add_docker_info_show_base_size
Add docker info show base filesystem size of container/image when use devicemapper
2015-10-13 14:43:52 +02:00
Tonis Tiigi ef05b83417 Copy aufs hardlinks to top layer
Show warning if old method is used.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2015-10-12 20:44:32 -07:00
Tonis Tiigi 504e67b867 Calculate hash based image IDs on pull
Generate a hash chain involving the image configuration, layer digests,
and parent image hashes. Use the digests to compute IDs for each image
in a manifest, instead of using the remotely specified IDs.

To avoid breaking users' caches, check for images already in the graph
under old IDs, and avoid repulling an image if the version on disk under
the legacy ID ends up with the same digest that was computed from the
manifest for that image.

When a calculated ID already exists in the graph but can't be verified,
continue trying SHA256(digest) until a suitable ID is found.

"save" and "load" are not changed to use a similar scheme. "load" will
preserve the IDs present in the tar file.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-10-12 10:51:28 -07:00
Hu Keping a066b94ef0 Comment: add some lines back
Signed-off-by: Hu Keping <hukeping@huawei.com>
2015-10-12 19:34:58 +08:00
Lei Jitang 5c374c7137 Add docker info show base filesystem size of container/image when use devicemapper
Signed-off-by: Lei Jitang <leijitang@huawei.com>
2015-10-10 22:52:05 +08:00
Phil Estes 442b45628e Add user namespace (mapping) support to the Docker engine
Adds support for the daemon to handle user namespace maps as a
per-daemon setting.

Support for handling uid/gid mapping is added to the builder,
archive/unarchive packages and functions, all graphdrivers (except
Windows), and the test suite is updated to handle user namespace daemon
rootgraph changes.

Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
2015-10-09 17:47:37 -04:00
Brian Goff 3b16cb15b4 Do not probe plugins for graph init unless `-s`
When `-s` is not specified, there is no need to ask if there is a plugin
with the specified name.

This speeds up unit tests dramatically since they don't need to wait the
timeout period for each call to `graphdriver.New`.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2015-10-09 12:02:54 -04:00
Michael Crosby 6d9a84bcd0 Merge pull request #16152 from chlunde/devmapper-mount-flags
Use pkg/mount to support more flags in dm.mountopt
2015-10-08 16:02:50 -07:00
Jess Frazelle 4c55464dd3 Merge pull request #13777 from cpuguy83/graphdriver_extpoints
Create extpoint for graphdrivers
2015-10-08 13:46:42 -07:00
Arnaud Porterie b3b9fe8c77 Merge pull request #15868 from crosbymichael/aufs-data
Add aufs data structure for added mount information
2015-10-08 11:17:20 -07:00
Alexander Morozov 6654b0e05f Merge pull request #16852 from estesp/overlay-cleanup
Simplify dir removal in overlay driver
2015-10-08 08:44:50 -07:00
Vincent Batts c3e5364813 Merge pull request #16823 from rhvgoyal/check-thin-pool
devmapper: Make sure device is a thin pool device
2015-10-08 11:27:53 -04:00
Phil Estes 6ed11b5374 Simplify dir removal in overlay driver
There is no need to call `os.Stat` on the driver filesystem path of a
container as `os.RemoveAll` already handles (properly) the case where
the path no longer exists.

Given the results of the stat() were not even being used,  there is no
value in erroring out because of the stat call failure, and worse, it
prevents daemon cleanup of containers in "Dead" state unless you re-create
directories that were already removed via a manual cleanup after a
failure.  This brings removal in overlay in line with aufs/devicemapper
drivers which don't error out if the filesystem path no longer exists.

Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
2015-10-08 11:04:00 -04:00
Vincent Demeester 53d2429164 Merge pull request #16780 from Microsoft/jjh/vfs-build-tags
Fix VFS build tags
2015-10-08 09:43:06 +02:00
Vivek Goyal 6d2d0a74e8 devmapper: Make sure device is a thin pool device
Right now we check for the existence of device but don't make sure it is
a thin pool device. We assume it is a thin pool device and call poolStatus()
on the device which returns an error EOF. And that error does not tell
anything.

So before we reach the stage of calling poolStatus() make sure we are working
with a thin pool device otherwise error out.

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
2015-10-07 16:28:49 -04:00
Vivek Goyal 87de04005d devmapper: Implement a goroutine to cleanup deleted devices
Start a goroutine which runs every 30 seconds and if there are deferred
deleted devices, it tries to clean those up.

Also it moves the call to cleanupDeletedDevices() into goroutine and
moves the locking completely inside the function. Now function does not
assume that device lock is held at the time of entry.

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
2015-10-06 17:37:21 -04:00
Vivek Goyal d295dc6652 devmapper: Keep track of number of deleted devices
Keep track of number of deleted devices and export this information through
"docker info".

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
2015-10-06 17:37:21 -04:00
Vivek Goyal d929589c1f devmapper: Implement deferred deletion functionality
Finally here is the patch to implement deferred deletion functionality.
Deferred deleted devices are marked as "Deleted" in device meta file. 

First we try to delete the device and only if deletion fails and user has
enabled deferred deletion, device is marked for deferred deletion.

When docker starts up again, we go through list of deleted devices and
try to delete these again.

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
2015-10-06 17:37:21 -04:00
Vivek Goyal 51e059e7e9 devmapper: Provide option to enabled deferred device deletion
Provide a command line option dm.use_deferred_deletion to enable deferred
device deletion feature. By default feature will be turned off.

Not sure if there is much value in deferred deletion being turned on
without deferred removal being turned on. So for now, this feature can
be enabled only if deferred removal is on.

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
2015-10-06 17:37:21 -04:00
Vivek Goyal 6b8b4feaa1 devmapper: construct used device ID map from device Hash map
Currently during startup we walk through all the device files and read
their device ID and mark in a bitmap that device id is used.

We are anyway going through all device files. So we can as well load all
that data into device hash map. This will save us little time when
container is actually launched later.

Also this will help with later patches where cleanup deferred device
wants to go through all the devices and see which have been marked for
deletion and delete these.

So re-organize the code a bit.

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
2015-10-06 17:37:21 -04:00
Vivek Goyal 0fcd485626 devmapper: Move UUID setup and verification code in a function
Simplify setupBaseImage() even further. Move some more code in a separate
function. Pure code reorganization. No functionality change.

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
2015-10-06 08:38:03 -04:00
Vivek Goyal 69051ec0a5 devmapper: Move thin pool related checks in a function
Move thin pool related checks in a separate function. Pure code reorganization.
Makes reading code easier.

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
2015-10-06 08:38:03 -04:00
Vivek Goyal efc1ddd7e3 devmapper: Move base device creation in separate function
This moves base device creation function in a separate function. Pure
code reorganization. Makes reading code little easier.

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
2015-10-06 08:38:03 -04:00
John Howard 7db08b4e62 Fix VFS build tags
Signed-off-by: John Howard <jhoward@microsoft.com>
2015-10-05 15:52:09 -07:00
Vivek Goyal 482eca3099 devmapper: Few code cleanups
This patch does three things. Following are the descriptions.

===
Create a separate function for delete transactions so that parent function
is little smaller.

Also close transaction if an error happens.
===
When docker is being shutdown, save deviceset metadata first before
trying to remove the devices. Generally caller gives only 10 seconds
for shutdown to complete and then kills it after that. So if some device
is busy, we will wait 20 seconds for it removal and never be able to save
metadata. So first save metadata and then deal with device removal.
===
Move issue discard operation in a separate function. This makes reading code
little easier.

Also don't issue discards if device is still open. That means devices is
still probably being used and issuing discards is not a good idea.

This is especially true in case of deferred deletion. We want to issue
discards when device is not open. At that time device can be deleted too.
Otherwise we will issue discards and deletion will actually fail. Later
we will try deletion again and issue discards again and deletion will
fail again as device is open and busy.

So this will ensure that discards are issued once when device is not open
and it can actually be deleted.

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
2015-10-05 09:02:31 -04:00
Jess Frazelle 134fefbaa2 Merge pull request #16490 from Microsoft/10662-mtimefix
Fixed file modified time not changing on windows
2015-10-02 12:06:03 -07:00
Jess Frazelle aa2e3247cb Merge pull request #16470 from tonistiigi/fix-aufs-opq
Add basic support for .wh..wh..opq
2015-10-02 11:28:33 -07:00
Darren Stahl 40b77af234 Fixed file modified time not changing on Windows
Signed-off-by: Darren Stahl <darst@microsoft.com>
2015-10-01 10:45:32 -07:00
Vivek Goyal 289145ecc6 devmapper: Get rid of metaData.devicesLock
Right now we seem to have 3 locks. 

- devinfo.lock
  This is a per device lock

- metaData.devicesLock

  This is supposedely protecting map of devices.

- Global DeviceSet lock

  This is protecting map of devices as well as serializing calls to libdevmapper.

Semantics of per devices lock and global deviceset lock seem to be very clear.
Even ordering between these two locks has been defined properly.

What is not clear is the need and ordering of metaData.devicesLock. Looks like
this lock is not necessary and global DeviceSet lock should be used to
protect map of devices as it is part of DeviceSet.

This patchset gets rid of metaData.devicesLock and instead uses DeviceSet
lock to protect map of devices.

Also at couple of places during initialization takes devices.Lock(). That
is not strictly necessary as there is supposed to be one thread of execution
during initializaiton. Still it makes the code clearer.

I think this makes code more clear and easier to understand and easier to
make further changes.

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
2015-10-01 13:02:55 -04:00
Vivek Goyal 73f8b46d84 devmapper: Get rid of unused function HasActivatedDevice()
Looks like nobody is calling HasActivatedDevice(). Get rid of it.

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
2015-10-01 07:59:12 -04:00
Vivek Goyal 94caae2477 devmapper: Move maxDeviceID check in loadMetadata
maxDeviceID is upper limit on device Id thin pool can support. Right now
we have this check only during startup. It is a good idea to move this
check in loadMetadata so that any time a device file is loaded and if it
is corrupted and device Id is more than maxDevieceID, it will be detected
right then and there.

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
2015-09-30 18:54:06 -04:00
Vivek Goyal 39081eb3aa devmapper: Use deactivateDevice() instead of removeDevice() in deleteDevice()
Use deactivateDevice() instead of removeDevice() directly. This will make
sure for device deletion, deferred removal is used if user has configured
it in. Also this makes reading code litle easier as there is single function
to remove a device and that is deactivateDevice().

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
2015-09-30 18:54:06 -04:00
Vivek Goyal e97e46b737 devmapper: Fail device deletion early if device is still mounted
If a device is still mounted at the time of DeleteDevice(), that means
higher layers have not called Put() properly on the device and are trying
to delete it. This is a bug in the code where Get() and Put() have not been
properly paired up. Fail device deletion if it is still mounted.

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
2015-09-30 15:21:22 -04:00
Vivek Goyal f5c0eb9ffe devmapper: Fix comments and for HasDevice() and Exists()
Exists() and HasDevice() just check if device file exists or not. It does
not say anything about if device is mounted or not. Fix comments.

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
2015-09-30 15:21:22 -04:00
Vivek Goyal ba02bf31cb devmapper: Do not load transaction meta file in device Hash map
device has map (device.Devices), contains valid devices and we skip all
the files which are not device files. transaction metadata file is not
device file. Skip this file when devices files are being read and loaded
into map.

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
2015-09-30 15:21:22 -04:00
David Calavera 1fffc7a89d Fix golint issues on the windows graph driver.
Signed-off-by: David Calavera <david.calavera@gmail.com>
2015-09-29 18:32:45 -04:00
Tonis Tiigi 2fb5d0c323 Add constants for AUFS whiteout files
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2015-09-29 11:21:45 -07:00
Tonis Tiigi 00e3277107 Add basic support for .wh..wh..opq
This fixes the case where directory is removed in
aufs and then the same layer is imported to a
different graphdriver.

Currently when you do `rm -rf /foo && mkdir /foo`
in a layer in aufs the files under `foo` would
only be be hidden on aufs.

The problems with this fix:

1) When a new diff is recreated from non-aufs driver
the `opq` files would not be there. This should not
mean layer differences for the user but still
different content in the tar (one would have one
`opq` file, the others would have `.wh.*` for every
file inside that folder). This difference also only
happens if the tar-split file isn’t stored for the
layer.

2) New files that have the filenames before `.wh..wh..opq`
when they are sorted do not get picked up by non-aufs
graphdrivers. Fixing this would require a bigger
refactoring that is planned in the future.


Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2015-09-29 08:22:25 -07:00
Carl Henrik Lunde 9a64f2bbb3 Use pkg/mount to support more flags in dm.mountopt
The mount syscall does not handle string flags like "noatime",
we must use bitmasks like MS_NOATIME instead.

pkg/mount.Mount already handles this work.

Signed-off-by: Carl Henrik Lunde <chlunde@ping.uio.no>
2015-09-24 21:14:41 +02:00
Chun Chen 2458452a3b Try to resize data and metadata loopback file when initiating devicemapper
Signed-off-by: Chun Chen <ramichen@tencent.com>
2015-09-24 09:31:00 +08:00
Brian Goff b78e4216a2 Create extpoint for graphdrivers
Allows people to create out-of-process graphdrivers that can be used
with Docker.

Extensions must be started before Docker otherwise Docker will fail to
start.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2015-09-09 20:24:35 -04:00
Vincent Demeester 6990b76a69 Lint package pkg/devicemapper
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2015-09-05 23:15:13 +02:00
Shishir Mahajan 4870fb36d4 Warning message for lvm devmapper running on top of loopback devices
Signed-off-by: Shishir Mahajan <shishir.mahajan@redhat.com>
2015-08-31 10:35:48 -04:00
Miguel Morales 95e3a4ca6d Typo fix then -> than
Signed-off-by: Miguel Morales <mimoralea@hp.com>
2015-08-28 10:44:39 -05:00
Michael Crosby 20bb065535 Add aufs data structure for added mount information
This adds a data struct in the aufs driver for including more
information about active mounts along with their reference count.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2015-08-26 16:38:13 -07:00
Jessica Frazelle bd06432ba3 cleanup and fix btrfs subvolume recursion deletion
Signed-off-by: Jessica Frazelle <acidburn@docker.com>
2015-08-25 13:00:41 -07:00
Ma Shimiao dea78fc2ce fix 9939: docker does not remove btrfs subvolumes when destroying container
Signed-off-by: Ma Shimiao <mashimiao.fnst@cn.fujitsu.com>
2015-08-24 14:52:07 -07:00
Alexander Morozov 1544c5edb6 Merge pull request #15735 from tonistiigi/graph-register-readcloser
Make graph.Register take in io.Reader
2015-08-24 10:40:04 -07:00
Jessie Frazelle 90801ab939 Merge pull request #15708 from Microsoft/sjw/graphdriver_missing_parent
Windows: Graphdriver should reject create of layer w/o parent
2015-08-21 16:34:03 -07:00
Stefan J. Wernli a456f20b44 Windows: Graphdriver should reject create of layer w/o parent
Signed-off-by: Stefan J. Wernli <swernli@microsoft.com>
2015-08-21 13:23:04 -07:00
Tonis Tiigi 2d1158790d Make graph.Register take in io.Reader
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2015-08-20 20:37:11 -07:00
Doug Davis 90ebc3b455 Merge pull request #14928 from brahmaroutu/lint_daemon_graphdriver
daemon/graphdriver fix lint errors/warnings
2015-08-20 03:25:00 -07:00
Laszlo Meszaros 78676f19c1 deviceset.go: fixed link in warning about udex sync is not supported
Signed-off-by: Laszlo Meszaros <lacienator@gmail.com>
2015-08-19 14:16:40 +02:00
Srini Brahmaroutu 9e1a41aae5 daemon/graphdriver fix lint errors/warnings
Addresses #14756

Signed-off-by: Srini Brahmaroutu <srbrahma@us.ibm.com>
2015-08-17 19:27:36 +00:00
Stefan J. Wernli dfbb5520e3 Windows: Graph remove custom interface and add central store
Signed-off-by: Stefan J. Wernli <swernli@microsoft.com>

Windows: add support for images stored in alternate location.

Signed-off-by: Stefan J. Wernli <swernli@microsoft.com>
2015-08-14 23:45:53 -07:00
Kir Kolyshkin 15a232fd06 graphdriver/zfs: fix GetMetadata() comment
Commit e27c904 added a wrong and misleading comment
to GetMetadata(). Fix it using the wording from
commit 407a626 which introduced GetMetadata().

Signed-off-by: Kir Kolyshkin <kir@openvz.org>
2015-08-11 18:16:11 -07:00
Kir Kolyshkin f5f7fee2ec graphdriver/zfs: privatize mountPath and zfsPath
These functions are not part of the graphdriver.Driver
interface and should therefore be private.

Also, remove comments added by commit e27c904 as they are
* pretty obvious
* no longer required by golint

Signed-off-by: Kir Kolyshkin <kir@openvz.org>
2015-08-11 18:16:08 -07:00
Alexander Morozov 0bfad28b86 Merge pull request #15422 from kolyshkin/graphtest-fix
graphtest: filter out lost+found dir entry
2015-08-10 12:21:21 -07:00
Alexander Morozov 6f89a8ee1b Merge pull request #15404 from vbatts/vbatts-dm-zero-sized-field
devicemapper: fix zero-sized field access
2015-08-10 09:16:11 -07:00
Vincent Batts f83d05c3be devicemapper: fix zero-sized field access
Fixes: #15279

Due to
7904946eeb
the devices field is dropped.

This solution works on go1.4 and go1.5

Signed-off-by: Vincent Batts <vbatts@redhat.com>
2015-08-10 11:11:58 -04:00
Qiang Huang 81cc8ebc93 Change all docker -d to docker daemon
Signed-off-by: Qiang Huang <h.huangqiang@huawei.com>
2015-08-10 20:48:08 +08:00
Kir Kolyshkin 158c536267 graphtest: filter out lost+found dir entry
Ploop graph driver provides its own ext4 filesystem to every
container. It so happens that ext4 root comes with lost+found
directory, causing failures from DriverTestCreateEmpty() and
DriverTestCreateBase() tests on ploop.

While I am not yet ready to submit ploop graph driver for review,
this change looks simple enough to push.

Note that filtering is done without any additional allocations,
as described in https://github.com/golang/go/wiki/SliceTricks.

[v2: added a comment about lost+found]

Signed-off-by: Kir Kolyshkin <kir@openvz.org>
2015-08-09 10:23:36 -07:00
David Calavera f1f6738d97 Fix typo in Overlay documentation.
Signed-off-by: David Calavera <david.calavera@gmail.com>
2015-08-08 15:11:09 -07:00
Veres Lajos 5146232723 typofix - https://github.com/vlajos/misspell_fixer
Signed-off-by: Veres Lajos <vlajos@gmail.com>
2015-08-07 23:25:49 +01:00
Srini Brahmaroutu de3944219f daemon/graphdriver/overlay/ fix lint errors/warnings
Addresses #14756
Signed-off-by: Srini Brahmaroutu <srbrahma@us.ibm.com>
2015-08-07 18:34:59 +00:00
Tibor Vass 8534090476 Merge pull request #15252 from coolljt0725/14765_enable_golint_3
Enable golint in pkg/archive
2015-08-05 19:27:48 -04:00
Arnaud Porterie 2c3cd949c8 Merge pull request #15269 from brahmaroutu/lint_daemon_graphdriver_zfs
daemon/graphdriver/zfs fix lint errrors/warnings
2015-08-04 18:41:02 -07:00
Srini Brahmaroutu e27c904b99 daemon/graphdriver/zfs fix lint errrors/warnings
Addresses #14756
Signed-off-by: Srini Brahmaroutu <srbrahma@us.ibm.com>
2015-08-04 18:30:55 +00:00
Lei Jitang ba332b7d12 Enable golint in pkg/arcive
Signed-off-by: Lei Jitang <leijitang@huawei.com>
2015-08-04 09:52:54 +08:00
Srini Brahmaroutu 3e7f9c636a daemon/graphdriver/vfs fix lint errors/warnings
Addresses #14756
Signed-off-by: Srini Brahmaroutu <srbrahma@us.ibm.com>
2015-08-03 17:46:42 +00:00
Tibor Vass 8d2739df98 Merge pull request #15146 from kolyshkin/mkdirall
Simplify and fix MkdirAll usage
2015-07-30 22:40:57 -04:00
Srini Brahmaroutu 22873eae31 fix unit test breakage due to lint changes
Addresses #14756

Signed-off-by: Srini Brahmaroutu <srbrahma@us.ibm.com>
2015-07-31 00:22:28 +00:00
Kir Kolyshkin a83a769347 Simplify and fix os.MkdirAll() usage
TL;DR: check for IsExist(err) after a failed MkdirAll() is both
redundant and wrong -- so two reasons to remove it.

Quoting MkdirAll documentation:

> MkdirAll creates a directory named path, along with any necessary
> parents, and returns nil, or else returns an error. If path
> is already a directory, MkdirAll does nothing and returns nil.

This means two things:

1. If a directory to be created already exists, no error is returned.

2. If the error returned is IsExist (EEXIST), it means there exists
a non-directory with the same name as MkdirAll need to use for
directory. Example: we want to MkdirAll("a/b"), but file "a"
(or "a/b") already exists, so MkdirAll fails.

The above is a theory, based on quoted documentation and my UNIX
knowledge.

3. In practice, though, current MkdirAll implementation [1] returns
ENOTDIR in most of cases described in #2, with the exception when
there is a race between MkdirAll and someone else creating the
last component of MkdirAll argument as a file. In this very case
MkdirAll() will indeed return EEXIST.

Because of #1, IsExist check after MkdirAll is not needed.

Because of #2 and #3, ignoring IsExist error is just plain wrong,
as directory we require is not created. It's cleaner to report
the error now.

Note this error is all over the tree, I guess due to copy-paste,
or trying to follow the same usage pattern as for Mkdir(),
or some not quite correct examples on the Internet.

[v2: a separate aufs commit is merged into this one]

[1] https://github.com/golang/go/blob/f9ed2f75/src/os/path.go

Signed-off-by: Kir Kolyshkin <kir@openvz.org>
2015-07-30 11:48:08 -07:00
Arnaud Porterie 8724e8953d Merge pull request #15040 from vbatts/vbatts-double-decompress-fix
archive, graphdriver: double decompress fix
2015-07-29 16:48:02 -07:00
Jessie Frazelle 75f8bdd970 Merge pull request #13542 from kvasdopil/freebsd-work
Make docker build on FreeBSD
2015-07-29 15:15:37 -07:00
Tibor Vass c09c497c71 Merge pull request #15111 from Microsoft/fix-archive-copy-log
Fix log to logrus
2015-07-29 14:48:19 -04:00
Alexey Guskov 26c03d561a make docker compile on freebsd
Signed-off-by: Alexey Guskov <lexag@mail.ru>
2015-07-29 21:25:56 +03:00
John Howard 2bf73c4b1a Fix log to logrus
Signed-off-by: John Howard <jhoward@microsoft.com>
2015-07-29 09:58:07 -07:00
Srini Brahmaroutu 972a94b449 daemon/graphdriver/devmapper/ fix lint errors/warnings
Addresses #14756
Signed-off-by: Srini Brahmaroutu <srbrahma@us.ibm.com>
2015-07-29 01:43:34 +00:00
Vincent Batts 273f50c741 graphdriver/*: expect uncompressed tar for ApplyDiff
The `ApplyDiff` function takes a tar archive stream that is
automagically decompressed later. This was causing a double
decompression, and when the layer was empty, that causes an early EOF.

Signed-off-by: Vincent Batts <vbatts@redhat.com>
2015-07-28 16:36:39 -04:00
Jessie Frazelle e06df594f5 Merge pull request #14863 from brahmaroutu/lint_daemon_graphdriver_aufs
daemon/graphdriver/aufs fix lint errors/warnings
2015-07-28 11:46:40 -07:00
Srini Brahmaroutu 55885daa56 daemon/graphdriver/aufs fix lint errors/warnings
Addresses #14756
Signed-off-by: Srini Brahmaroutu <srbrahma@us.ibm.com>
2015-07-28 06:17:05 +00:00
Srini Brahmaroutu 12460f41a4 daemon/graphdriver/graphtest/ fix lint errors/warnings
Addresses #14756

Signed-off-by: Srini Brahmaroutu <srbrahma@us.ibm.com>
2015-07-28 06:01:00 +00:00
David Calavera 5bac5302e5 Fix typo in overlay's create godoc.
Signed-off-by: David Calavera <david.calavera@gmail.com>
2015-07-27 15:37:26 -07:00
Jessie Frazelle 303345dc6a Merge pull request #14875 from brahmaroutu/lint_daemon_graphdriver_btrfs
daemon/graphdriver/btrfs fix lint errors/warnings
2015-07-27 10:28:50 -07:00
Lei e1c5e9b561 Add VxFS magic number, fixes #14847
Signed-off-by: Lei Jitang <leijitang@huawei.com>
2015-07-24 16:37:56 +08:00
Srini Brahmaroutu 17c19f395f daemon/graphdriver/btrfs fix lint errors/warnings
Addresses #14756
Signed-off-by: Srini Brahmaroutu <srbrahma@us.ibm.com>
2015-07-23 22:48:45 +00:00
Flavio Castelli f95b3a6b6a ZFS driver: raise better errors during init
The ZFS driver should raise proper errors when the ZFS utility is
missing or when there's no zfs partition active on the system. Raising the
proper errors make possible to silently ignore the ZFS storage
driver when no default storage driver is specified.

Previous to this commit it was no longer possible to start the
docker daemon in that way:

  docker -d --storage-opt dm.loopdatasize=2GB

The above command resulted in an exit error because the ZFS driver
tried to use the storage options.

Signed-off-by: Flavio Castelli <fcastelli@suse.com>
2015-07-20 15:24:48 +02:00
Arnaud Porterie 98ed9a55f4 Merge pull request #14693 from LK4D4/update_libcontainer
Update libcontainer
2015-07-17 13:02:04 -07:00
Vivek Goyal 424d5e55a2 devicemapper: Change default basesize to 100G
Current default basesize is 10G. Change it to 100G. Reason being that for
some people 10G is turning out to be too small and we don't have capabilities
to grow it dyamically.

This is just overcommitting and no real space is allocated till container
actually writes data. And this is no different then fs based graphdrivers
where virtual size of a container root is unlimited.

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
2015-07-17 11:10:23 -04:00
Alexander Morozov c86189d554 Update libcontainer
Replaced github.com/docker/libcontainer with
github.com/opencontainers/runc/libcontaier.
Also I moved AppArmor profile generation to docker.

Main idea of this update is to fix mounting cgroups inside containers.
After updating docker on CI we can even remove dind.

Signed-off-by: Alexander Morozov <lk4d4@docker.com>
2015-07-16 16:02:26 -07:00
Alexander Morozov a5142f6ac3 Fix some formatting calls
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
2015-07-15 12:25:50 -07:00
Vincent Batts 5ca3e7c54c Merge pull request #14021 from rhvgoyal/detect-pool-loopback-devices
devicemapper: Check loop devices of existing pool
2015-07-13 21:15:23 -04:00
John Howard 52f4d09ffb Windows: Graph driver implementation
Signed-off-by: John Howard <jhoward@microsoft.com>
2015-07-10 14:33:11 -07:00
Arnaud Porterie f089899023 Fix panic on devicemapper initialization
The ability to save and verify base device UUID (#13896) introduced a
situation where the initialization would panic when removing the device
returns EBUSY.

Functions `verifyBaseDeviceUUID` and `saveBaseDeviceUUID` now take the
lock on the `DeviceSet`, which solves the problem as `removeDevice`
assumes it owns the lock.

Signed-off-by: Arnaud Porterie <arnaud.porterie@docker.com>
2015-07-08 09:10:20 -07:00
Jessie Frazelle 2fe51ecee7 Merge pull request #14100 from rhvgoyal/overlay-docker-inspect
overlay: Export metadata for container and image in docker-inspect
2015-07-07 18:11:27 -07:00
David Calavera e27131519f Warn when udev_sync is not supported.
Signed-off-by: David Calavera <david.calavera@gmail.com>
2015-07-07 15:10:24 -06:00
David Calavera 9af7afb9eb Revert "Fix implicit DeviceMapper selection"
This reverts commit 0a376291b2.

Signed-off-by: David Calavera <david.calavera@gmail.com>
2015-07-07 12:27:19 -07:00
Vivek Goyal bebf534439 devicemapper: Check loop devices of existing pool
Often it happens that docker is not able to shutdown/remove the thin
pool it created because some device has leaked into some mount name
space. That means device is in use and that means pool can't be removed.

Docker will leave pool as it is and exit. Later when user starts the
docker, it finds pool is already there and docker uses it. But docker
does not know it is same pool which is using the loop devices. Now
docker thinks loop devices are not being used. That means it does not
display the data correctly in "docker info", giving user wrong information.

This patch tries to detect if loop devices as created by docker are
being used for pool and fills in the right details in "docker info".

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
2015-07-07 14:13:29 -04:00
David Calavera 0a376291b2 Fix implicit DeviceMapper selection
DeviceMapper must be explicitly selected because the Docker binary might not be linked to the right devmapper library.

With this change, Docker fails fast if the driver detection finds the devicemapper directory but the driver is not the default option.
The option `override_udev_sync_check` doesn't make sense anymore, since the user must be explicit to select devicemapper, so it's being removed.
Docker fails to use devicemapper only if Docker has been built statically unless the option was explicit.

Signed-off-by: David Calavera <david.calavera@gmail.com>
2015-07-02 09:21:27 -07:00
Mary Anthony 078b23a37d Closes #13323 and carries
Entering comments

Signed-off-by: Mary Anthony <mary@docker.com>
2015-06-29 06:18:41 -07:00
Vivek Goyal 67473c6d06 overlay: Export metadata for container and image in docker-inspect
Export metadata for container and image in docker-inspect when overlay
graphdriver is in use. Right now it is done only for devicemapper graph
driver.

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
2015-06-25 17:33:20 -04:00
David Calavera 85d3b75dfd Merge pull request #13896 from rhvgoyal/verify-base-uuid
devicemapper: Compare uuid of base device on startup
2015-06-17 11:30:04 -07:00
Vivek Goyal c06b05b11e devicemapper: Compare uuid of base device on startup
It is easy for one to use docker for a while, shut it down and restart
docker with different set of storage options for device mapper driver
which will effectively change the thin pool. That means any of the
metadata stored in /var/lib/docker/devicemapper/metadata/ is not valid
for the new pool and user will run into various kind of issues like
container not found in the pool etc.

Users think that their images or containers are lost but it might just
be the case of configuration issue. People might use wrong metadata
with wrong pool.

To detect such situations, save UUID of base image and once docker
starts later, query and compare the UUID of base image with the
stored one. If they don't match, fail the initialization with the
error that UUID failed to match.

That way user will be forced to cleanup /var/lib/docker/ directory
and start docker again.

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
2015-06-16 21:12:27 -04:00
Vincent Batts e69df2589c Merge pull request #13198 from rhvgoyal/extend-docker-inspect
docker-inspect: Extend docker inspect to export image metadata related to graph driver
2015-06-16 15:03:14 -05:00
Brian Goff 49834e8d59 Fix circular import for windows vfs graphdriver
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2015-06-16 09:08:10 -04:00
John Howard 59cfc08982 Windows - Really fix VFS this time
Signed-off-by: John Howard <jhoward@microsoft.com>
2015-06-15 15:09:48 -07:00
Vivek Goyal 407a626be6 docker-inspect: Extend docker inspect to export image/container metadata related to graph driver
Export image/container metadata stored in graph driver. Right now 3 fields
DeviceId, DeviceSize and DeviceName are being exported from devicemapper.
Other graph drivers can export fields as they see fit.

This data can be used to mount the thin device outside of docker and tools
can look into image/container and do some kind of inspection.

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
2015-06-15 14:05:10 -04:00
Mary Anthony cd44018856 Carry of PR #13520
Removinig files

Signed-off-by: Mary Anthony <mary@docker.com>
2015-06-13 09:27:30 -07:00
David Calavera e7533d7f81 Merge pull request #13494 from Microsoft/10662-vfsdriveroption
Windows: Allow VFS
2015-06-12 11:12:32 -07:00
John Howard e89f837bc6 Windows: Allow VFS
Signed-off-by: John Howard <jhoward@microsoft.com>
2015-06-12 09:21:17 -07:00
Jörg Thalheim 19c31a703f zfs: correctly apply selinux context
fixes #13858

Signed-off-by: Jörg Thalheim <joerg@higgsboson.tk>
2015-06-11 11:11:37 +02:00
Tibor Vass add64dc297 Merge pull request #13832 from mapk0y/fix-error-msg
fix typo.
2015-06-10 16:38:07 -04:00
mapk0y 416e855e9b fix typo.
Signed-off-by: mapk0y <mapk0y@gmail.com>
2015-06-10 06:26:07 +09:00
John Howard 9a9dc5ba96 Windows: Don't build Linux graph drivers
Signed-off-by: John Howard <jhoward@microsoft.com>
2015-06-08 15:09:33 -07:00
Alexey Guskov 112b7e6546 avoid 88-chars mountpoint length limit on freebsd
Signed-off-by: Alexey Guskov <lexag@mail.ru>
2015-05-29 16:33:04 +03:00
Alexey Guskov 36bf6e4440 zfs magicnumber check on freebsd is fixed
Signed-off-by: Alexey Guskov <lexag@mail.ru>
2015-05-28 18:38:08 +03:00
John Howard 4e8b2509fd Windows: graphdriver refactor
Signed-off-by: John Howard <jhoward@microsoft.com>
2015-05-20 08:51:27 -07:00
Alexander Morozov 3916561619 Fix Put without Get in overlay
It is called for example on daemon start after crash

Signed-off-by: Alexander Morozov <lk4d4@docker.com>
2015-05-19 09:32:23 -07:00
Michael Crosby 70e9e4bf9f Merge pull request #13249 from coolljt0725/fix_incorrect_document_of_storage_opt
Fix incorrect document storage-opt
2015-05-18 11:35:01 -07:00
Lei Jitang 0204ecdf70 Fix incorrect document storage-opt
Signed-off-by: Lei Jitang <leijitang@huawei.com>
2015-05-15 19:13:07 +08:00
John Howard 35cdcbb323 Windows: No ZFS graphdriver
Signed-off-by: John Howard <jhoward@microsoft.com>
2015-05-14 15:57:45 -07:00
Jörg Thalheim 2cb23527e4 zfs: update filesystem cache on filesystem creation/deletion
Previously the cache was only updated once on startup, because the graph
code only check for filesystems on startup. However this breaks the API as it
was supposed and so unit tests.

Fixes #13142

Signed-off-by: Jörg Thalheim <joerg@higgsboson.tk>
2015-05-12 13:06:41 +02:00
Jörg Thalheim bad25ccf97 zfs: retrieve all filesystems on startup at once
The docker graph call driver.Exists() on initialisation for each filesystem in
the graph. This results will results in a lot `zfs get all` commands. To reduce
this, retrieve all descend filesystem at startup and cache it for later checks

Signed-off-by: Jörg Thalheim <joerg@higgsboson.tk>
2015-05-08 17:49:39 +02:00
Jörg Thalheim 11e9167a6b zfs: improve performance by using legacy mounts
instead of let zfs automaticly mount datasets, mount them on demand using mount(2).
This speed up this graph driver in 2 ways:
- less zfs processes needed to start a container
- /proc/mounts get smaller, so zfs userspace tools has less to read (which can
  a significant amount of data as the number of layer grows)

This ways it can be also ensured that the correct mountpoint is always used.

Signed-off-by: Jörg Thalheim <joerg@higgsboson.tk>
2015-05-08 17:49:39 +02:00
Jörg Thalheim ee00f07ea6 zfs: replace c for /proc/mounts parsing with go
Signed-off-by: Jörg Thalheim <joerg@higgsboson.tk>
2015-05-08 17:49:39 +02:00
Jörg Thalheim dd614b5e34 zfs: refactor error handling
thanks to @calavera

Signed-off-by: Jörg Thalheim <joerg@higgsboson.tk>
2015-05-08 17:48:20 +02:00
Jörg Thalheim a25195d86c zfs: add myself to MAINTAINERS
Signed-off-by: Jörg Thalheim <joerg@higgsboson.tk>
2015-05-08 17:48:20 +02:00
Jörg Thalheim bacecabf3b zfs: revert to NaiveGraphDriver for the moment
Signed-off-by: Jörg Thalheim <joerg@higgsboson.tk>
2015-05-08 17:48:20 +02:00
Jörg Thalheim d5151ca8ab Implement Docker on ZFS
Signed-off-by: Arthur Gautier <baloo@gandi.net>
Signed-off-by: Jörg Thalheim <joerg@higgsboson.tk>
2015-05-08 17:48:20 +02:00
Alexander Morozov 93536cfa76 Merge pull request #13067 from burke/faster-commit-for-overlay
overlay: skip superfluous metadata sets on commit
2015-05-07 13:58:46 -07:00
Burke Libbey bc5503f46b
overlay: skip superfluous metadata sets on commit
Signed-off-by: Burke Libbey <burke.libbey@shopify.com>
2015-05-07 16:15:54 -04:00
Vincent Batts 56c9917815 Merge pull request #12190 from rhvgoyal/deferred-removal
Devicemapper: Provide deferred device removal capability
2015-05-04 14:22:52 -07:00
unclejack 2e49281bd0 Merge pull request #12903 from rhvgoyal/disable-discards
devmapper: Disable mount option "discard" by default
2015-05-04 18:35:25 +03:00
Vivek Goyal 04adaaf1ee devmapper: Disable mount option "discard" by default
Right now devicemapper mounts thin device using online discards by default
and passes mount option "discard". Generally people discourage usage of
online discards as they can be a drain on performance. Instead it is 
recommended to use fstrim once in a while to reclaim the space.

In case of  containers, we recommend to keep data volumes separate. So
there might not be lot of rm, unlink operations going on and there might
not be lot of space being freed by containers. So it might not matter
much if we don't reclaim that free space in pool.

User can still pass mount option explicitly using dm.mountopt=discard to
enable discards if they would like to.

So this is more like setting the containers by default for better performance
instead of better space efficiency in pool. And user can change the behavior
if they don't like default behavior.

Reported-by: Mike Snitzer <snitzer@redhat.com>
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
2015-05-01 09:16:31 -04:00
Antonio Murdaca 844538142d Small if err cleaning
Signed-off-by: Antonio Murdaca <me@runcom.ninja>
2015-04-27 21:50:33 +02:00
Vivek Goyal ddc8acebec devmapper: Cancel deferred deactivation if device is reactivated
If device is being reactivated before it could go away and deferred 
deactivation is scheduled on it, cancel it.

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
2015-04-21 18:14:59 -04:00
Vivek Goyal 66a53819ae devmapper: Export deferred removal status in status
This will help with debugging as one could just do "docker info" and figure
out of deferred removal is enabled or not.

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
2015-04-21 18:14:59 -04:00
Vivek Goyal e37c7203bb devmapper: Use deferred removal
Make use of deferred removal of devices.

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
2015-04-21 18:14:59 -04:00
Vivek Goyal 15c158b207 devmapper: Provide a new parameter dm.deferred_device_removal
Provide a new command line knob dm.deferred_device_removal which will enable
deferred device deactivation if driver and library support it.

This patch also checks for library support and driver version.

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
2015-04-21 18:14:59 -04:00
Megan Kostick cdc63ce5d0 Updated message severity in graphdriver
Signed-off-by: Megan Kostick <mkostick@us.ibm.com>
2015-04-17 10:56:12 -07:00
Vincent Batts b68e161e5b graphdriver: prefer prior driver state
Before this, a storage driver would be defaulted to based on the
priority list, and only print a warning if there is state from other
drivers.

This meant a reordering of priority list would "break" users in an
upgrade of docker, such that there images in the prior driver's state
were now invisible.

With this change, prior state is scanned, and if present that driver is
preferred.

As such, we can reorder the priority list, and after an upgrade,
existing installs with prior drivers can have a contiguous experience,
while fresh installs may default to a driver in the new priority list.

Ref: https://github.com/docker/docker/pull/11962#issuecomment-88274858

Signed-off-by: Vincent Batts <vbatts@redhat.com>
2015-04-13 21:37:55 -04:00
Megan Kostick 7e2d05b493 Add detection for F2Fs and JFS
Signed-off-by: Megan Kostick <mkostick@us.ibm.com>

Alphabetize FSMagic list to make more human-readable.

Signed-off-by: Megan Kostick <mkostick@us.ibm.com>
2015-04-13 17:01:43 -07:00
Tibor Vass 23c12dabbd Merge pull request #11412 from vbatts/vbatts-dm_sync_is_required
devmapper: udev sync is a requirement
2015-04-10 16:52:38 -04:00
Vincent Batts 0e21782de5 devmapper: storage-opt override for udev sync
This provides an override for forcing the daemon to still attempt
running the devicemapper driver even when udev sync is not supported.

Intended to be a very clear impairment for those choosing to use it. If
udev sync is false, there will still be an error in the daemon logs,
even when the override is in place. The docs have an explicit WARNING.

Including link to the docs for users that encounter this daemon error
during an upgrade.

Signed-off-by: Vincent Batts <vbatts@redhat.com>
2015-04-10 15:43:16 -04:00
Vincent Batts ca628c6216 devmapper: udev sync is a requirement
closes #10664
closes #4036

Signed-off-by: Vincent Batts <vbatts@redhat.com>
2015-04-06 15:17:57 -04:00
Vivek Goyal c737800b7f devmapper: Retry device removal after 100ms instead of 10ms
Right now we try device removal at the interval of 10ms and keep on trying
till either device is removed or 10 seconds are over. That means if device
is busy, we will try 1000 times in those 10 seconds.

Sounds too high a frequency of deivce removal retrial. All the logs are
filled easily. I think it is a good idea to slow down a bit and retry at
the interval of 100ms instead of 10ms.

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
2015-04-02 16:47:14 -04:00
Vivek Goyal f74d12012c devmapper: Remove call to waitClose()
During device removal, we are first waiting for device to close() in a tight
loop for 10 seconds. I am not sure why do we need it. First of all we come
here once the umount() is successful so device should be free. For some reason
of device is temporarily busy, then removeDevice() logic retries device removal
logic in a loop for 10 seconds and that should cover it. Can't see why one
more 10 seoncds loop is required before attempting device removal.

One loop should be able to cover all the temporary device busy conditions and
if condition is not temporary then 10 seconds loop is not going to help anyway.

So instead of two loops of 10 seconds each, I am converting it to a single
loop of 20 seconds. May be 10 second loop is good enough but for now I am
keeping it 20 seconds to avoid any regressions.

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
2015-04-02 16:47:14 -04:00
Vivek Goyal dbf04ec4e2 devmapper: Remove extra wait after device removal
Currently in device removal path (device deactivation), we wait
for 10 seconds for devive to actually go away. waitRemove().

In current code this is not required. If dm removal task has completed
and one has done the wait on udev cookie, then device is gone and there
is no need to write another loop to wait for device removal.

This patch removes the waitRemove() which waits for 10 seconds after
device removal. This seems unnecessary.

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
2015-04-02 16:47:14 -04:00
Vivek Goyal cb7c893275 devicemapper: Remove debug messages from RemoveDevice()
devmapper graph driver retries device removal 1000 times in case of failure
and if this fills up console with 1000 messages (when daemon is running in
debug mode). So remove these debug messages.
 
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
2015-04-02 16:47:14 -04:00
Vivek Goyal e07d3cd9ac devmapper: Fix libdm logging
There are issues with libdm logging. Right now if docker daemon is run
in debug mode, logging by libdm is too verbose. And if a device can't 
be removed, thousands of messages fill the console and one can not see
what's going on.

This patch removes devicemapper.LogInitVerbose() call as that call will
only work if docker was not registering its own log handler with libdm.
For some reason docker registers one with libdm and libdm hands over
all the messages to docker (including debug ones). And now it is up to
devmapper backend to figure out which ones should go to console and
which ones should not.

So by default log only fatal messages from libdm. One can easily modify
the code to change it for debugging purposes.

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
2015-04-02 16:47:14 -04:00
Vincent Batts 2c72ff1dbf graphdriver: promote overlay above vfs
It's about time to let folks not hit 'vfs', when 'overlay' is supported
on their kernel. Especially now that v3.18.y is a long-term kernel.

Signed-off-by: Vincent Batts <vbatts@redhat.com>
2015-03-31 17:18:49 -04:00
Michael Crosby d1306e63e4 Fix AUFS logrus build errors on import
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2015-03-30 11:46:44 -07:00
Michael Crosby 14fed352cb Merge pull request #11799 from dqminh/aufs-dirperm1
aufs: apply dirperm1 by default if supported
2015-03-30 11:36:02 -07:00
Brian Goff da5c863d20 Merge pull request #11788 from reikani/pchoi
Changed snake case naming to camelCase.
2015-03-26 23:55:50 -07:00
Antonio Murdaca 6f4d847046 Replace aliased imports of logrus, fixes #11762
Signed-off-by: Antonio Murdaca <me@runcom.ninja>
2015-03-26 23:22:04 +01:00
Peter Choi ae907e7af1 Changed snake case naming to camelCase
Signed-off-by: Peter Choi <phkchoi89@gmail.com>
2015-03-26 15:05:45 -06:00
Daniel, Dao Quang Minh d68d5f2e4b print dirperm1 supported status in docker info
It's easier for users to check if their systems support dirperm1 just by using
docker info

Signed-off-by: Daniel, Dao Quang Minh <dqminh89@gmail.com>
2015-03-26 17:58:49 +00:00
Alexander Morozov dd492dc15d Merge pull request #11702 from jimmypuckett/feature/bitflag_checking_style_11668
Feature/bitflag checking style 11668.  Fixes #11668
2015-03-26 08:34:58 -07:00
Daniel, Dao Quang Minh 281abd2c8a aufs: apply dirperm1 by default if supported
Automatically detect support for aufs `dirperm1` option and apply it.
`dirperm1` tells aufs to check the permission bits of the directory on the
topmost branch and ignore the permission bits on all lower branches.
It can be used to fix aufs' permission bug (i.e., upper layer having
broader mask than the lower layer).

More information about the bug can be found at https://github.com/docker/docker/issues/783
`dirperm1` man page is at: http://aufs.sourceforge.net/aufs3/man.html

Signed-off-by: Daniel, Dao Quang Minh <dqminh89@gmail.com>
2015-03-26 07:25:42 +00:00
Anes Hasicic 3d7b9e8f30 Fixed redundant else
Signed-off-by: Anes Hasicic <anes.hasicic@gmail.com>
2015-03-25 23:44:32 +01:00
Vincent Batts b76e300b4c btrfs: #ifdef for build version
We removed it, because upstream removed it. But now it will be coming
back, so work with it either way.

Signed-off-by: Vincent Batts <vbatts@redhat.com>
2015-03-25 13:56:51 -04:00
Anes Hasicic e479e1c9f7 Fixed redundant else in GetDeviceStatus
Signed-off-by: Anes Hasicic <anes.hasicic@gmail.com>
2015-03-25 09:53:04 +01:00
Jimmy Puckett ec5e22efe3 Changing bitflag checking style to preferred style. Fixes #11668
Signed-off-by: Jimmy Puckett <jimmy.puckett@spinen.com>
2015-03-24 21:09:25 -04:00
Vincent Batts 8fc9e40086 Merge pull request #11417 from rhatdan/btrfs
Btrfs has eliminated the BTRFS_BUILD_VERSION in latest version
2015-03-24 16:00:15 -04:00
Dan Walsh 25c4a92342 Merge branch 'master' of github.com:docker/docker into btrfs
Docker-DCO-1.1-Signed-off-by: Dan Walsh <dwalsh@redhat.com> (github: rhatdan)
2015-03-24 14:14:54 -04:00
Antonio Murdaca b80fae7356 Refactor pkg/common, Fixes #11599
Signed-off-by: Antonio Murdaca <me@runcom.ninja>
2015-03-24 18:19:59 +01:00
Dan Walsh 3c136333af Btrfs has eliminated the BTRFS_BUILD_VERSION in latest version
They say we should only use the BTRFS_LIB_VERSION

They will no longer support this since it had to be managed manually

Docker-DCO-1.1-Signed-off-by: Dan Walsh <dwalsh@redhat.com> (github: rhatdan)
2015-03-20 11:43:53 -04:00
Michal Minar 210ab030bc Format error by value
- Use `%v` verb to format errors.
- Give `param` constant in portallocator some better name.

Signed-off-by: Michal Minar <miminar@redhat.com>
2015-03-16 12:05:53 +01:00
Arnaud Porterie 82f390e139 Merge pull request #11076 from hqhq/hq_use_warning_in_sysinfo
use type WARN for warning
2015-03-15 21:13:23 -07:00
Brian Goff c0f7819905 Make errors from graphdriver init friendlier
In several cases graphdriver were just returning the low-level syscall
error and that was making it all the way up to the daemon logs and in
many cases was difficult to tell it was even coming from the graphdriver
at all.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2015-03-12 08:13:00 -07:00
Deng Guangxing ce1a0b612d Fix minor typo
Signed-off-by: Deng Guangxing <dengguangxing@huawei.com>
2015-03-11 11:13:39 +08:00
Qiang Huang bffe04b582 fix warning messages
Use log.Warnf instead of log.Infof, and remove redundant `WARNING` prefix.

Signed-off-by: Qiang Huang <h.huangqiang@huawei.com>
2015-03-11 08:47:45 +08:00
Arnaud Porterie 89bdaa35e0 Remove subdirectories MAINTAINERS files
Signed-off-by: Arnaud Porterie <arnaud.porterie@docker.com>
2015-03-06 18:21:51 -08:00
Michael Crosby 92e632c84e Merge pull request #11064 from delftswa2014/10970-treesize
Move directory size calculation to pkg/ (fixes #10970)
2015-03-05 13:06:02 -08:00
Dan Walsh 4eb2fd169f Two SELinux Changes.
daemon/volumes.go

  This SetFileCon call made no sense, it was changing the labels of any
directory mounted into the containers SELinux label.  If it came from me,
then I apologize since it is a huge bug.

The Volumes Mount code should optionally do this, but it should not always
happen, and should never happen on a --privileged container.

The change to

daemon/graphdriver/vfs/driver.go, is a simplification since this it not
a relabel, it is only a setting of the shared label for docker volumes.

Docker-DCO-1.1-Signed-off-by: Dan Walsh <dwalsh@redhat.com> (github: rhatdan)
2015-03-05 13:55:44 -05:00
Martijn Dwars e2b8933d21 Move directory size calculation to pkg/ (fixes #10970)
Signed-off-by: Martijn Dwars <ikben@martijndwars.nl>
2015-03-04 21:16:31 +01:00
Lei Jitang dd56fa1906 Add xfs fs magic to graphdriver/driver.go
Signed-off-by: Lei Jitang <leijitang@huawei.com>
2015-03-04 19:01:39 +08:00
Alexander Morozov 32ccde46dd Merge pull request #11106 from MabinGo/fix_log_info
Fix some revision about log output
2015-03-02 13:55:43 -08:00
Jessie Frazelle 82aa950f4e Merge pull request #11054 from MabinGo/checkfileoper1
Add the file close operation before function return to advoid resource leaking
2015-03-02 10:44:26 -08:00
Phil Estes 738bdec490 Merge pull request #11083 from MabinGo/minor_typo
Modify the minor typo in deviceset.go
2015-03-02 13:11:14 -05:00
Mabin adfd1ddfc6 Fix some revision about log output
Signed-off-by: Mabin <bin.ma@huawei.com>
2015-03-02 17:06:38 +08:00
Mabin f4ac86de84 fix typo of comments in daemon/graphdriver/devmapper/deviceset.go
Signed-off-by: Mabin <bin.ma@huawei.com>
2015-03-02 02:11:15 +08:00
Mabin 5c1559a754 Modify the minor typo in deviceset.go
Signed-off-by: Mabin <bin.ma@huawei.com>
2015-02-28 18:04:10 +08:00
Mabin b5eeab6e06 Add the file close operation before function return to advoid resource leaking
Signed-off-by: Mabin <bin.ma@huawei.com>
2015-02-28 00:24:41 +08:00