Commit Graph

464 Commits

Author SHA1 Message Date
Vincent Demeester 2e236d0255 Merge pull request #21828 from dmcgowan/fix-overlay-on-overlay-test
Fix overlay test running on overlay
2016-04-07 10:07:25 +02:00
Vincent Demeester 8785952282 Merge pull request #21830 from anusha-ragunathan/auplink
Be more lenient on auplink errors.
2016-04-07 09:19:38 +02:00
Derek McGowan 824c72f472 Fix overlay test running on overlay
Overlay tests were failing when /var/tmp was an overlay mount with a misleading message.
Now overlay tests will be skipped when attempting to be run on overlay.
Tests will now use the TMPDIR environment variable instead of only /var/tmp

Fixes #21686

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-04-06 21:06:42 -07:00
Anusha Ragunathan dbd9b7e121 Be more lenient on auplink errors.
On aufs, auplink is run before the Unmount. Irrespective of the
result, we proceed to issue a Unmount syscall. In which case,
demote erros on auplink to warning.

Signed-off-by: Anusha Ragunathan <anusha@docker.com>
2016-04-06 18:24:19 -07:00
Alexander Morozov 65464d11f1 Merge pull request #21809 from Microsoft/jjh/tp4removal
Windows: Remove TP4 support from main codebase
2016-04-06 14:11:33 -07:00
Tibor Vass 8c361eb5da Merge pull request #21723 from devimc/master
Fix compilation errors with btrfs-progs-4.5
2016-04-06 15:56:42 -04:00
John Howard 331c8a86d4 Windows: Remove TP4 support from main code
Signed-off-by: John Howard <jhoward@microsoft.com>
2016-04-06 12:12:20 -07:00
John Howard fc9912fd00 Merge pull request #21272 from Microsoft/jstarks/manifest_updates
Add os_version and os_features to Image
2016-04-05 16:16:25 -07:00
John Starks 194eaa5c0f Add os_version and os_features to Image
These fields are needed to specify the exact version of Windows that an
image can run on. They may be useful for other platforms in the future.

This also changes image.store.Create to validate that the loaded image is
supported on the current machine. This change affects Linux as well, since
it now validates the architecture and OS fields.

Signed-off-by: John Starks <jostarks@microsoft.com>
2016-04-04 13:14:57 -07:00
Sebastiaan van Stijn b8f38747e6 Improve udev unsupported error message
Show a different message if a dynamic binary
is running, but doesn't have udev sync support.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2016-04-01 13:31:44 -07:00
Julio Montes a038cccf88 Fix compilation errors with btrfs-progs-4.5
btrfs-progs-4.5 introduces device delete by devid
for this reason btrfs_ioctl_vol_args_v2's name was encapsulated
in a union

this patch is for setting btrfs_ioctl_vol_args_v2's name
using a C function in order to preserve compatibility
with all btrfs-progs versions

Signed-off-by: Julio Montes <imc.coder@gmail.com>
2016-04-01 08:58:29 -06:00
Vincent Demeester e6aa40a017 Merge pull request #19367 from shishir-a412ed/rootfs_size_configurable_cli
CLI flag for docker create(run) to change block device size.
2016-03-29 08:52:54 +02:00
Shishir Mahajan b16decfccf CLI flag for docker create(run) to change block device size.
Signed-off-by: Shishir Mahajan <shishir.mahajan@redhat.com>
2016-03-28 10:05:18 -04:00
Tonis Tiigi 824c24e680 Protect aufs mounts with locks
Parallel aufs mount calls produce invalid argument error.

Fixes #21545


Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2016-03-26 22:53:47 -07:00
Brian Goff 65d79e3e5e Move layer mount refcounts to mountedLayer
Instead of implementing refcounts at each graphdriver, implement this in
the layer package which is what the engine actually interacts with now.
This means interacting directly with the graphdriver is no longer
explicitly safe with regard to Get/Put calls being refcounted.

In addition, with the containerd, layers may still be mounted after
a daemon restart since we will no longer explicitly kill containers when
we shutdown or startup engine.
Because of this ref counts would need to be repopulated.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2016-03-23 14:42:52 -07:00
Anusha Ragunathan 57ca2a2101 Fix use of mounted() in overlay.
Handle error and mounted case separately.

Signed-off-by: Anusha Ragunathan <anusha@docker.com>
2016-03-23 14:42:52 -07:00
Tonis Tiigi e91de9fb9d Revert "Move layer mount refcounts to mountedLayer"
This reverts commit 563d0711f8.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2016-03-23 00:33:02 -07:00
Tõnis Tiigi 92a3ece35a Merge pull request #21107 from cpuguy83/one_ctr_to_rule_them_all
Move layer mount refcounts to mountedLayer
2016-03-22 21:19:00 -07:00
Brian Goff 563d0711f8 Move layer mount refcounts to mountedLayer
Instead of implementing refcounts at each graphdriver, implement this in
the layer package which is what the engine actually interacts with now.
This means interacting directly with the graphdriver is no longer
explicitly safe with regard to Get/Put calls being refcounted.

In addition, with the containerd, layers may still be mounted after
a daemon restart since we will no longer explicitly kill containers when
we shutdown or startup engine.
Because of this ref counts would need to be repopulated.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2016-03-22 11:36:28 -04:00
Kenfe-Mickael Laventure 8af4f89cba Remove unneeded references to execDriver
This includes:
 - updating the docs
 - removing dangling variables

Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2016-03-21 13:06:08 -07:00
Jessica Frazelle 0e025b4bb1
fix variables that werent being called
Signed-off-by: Jessica Frazelle <acidburn@docker.com>
2016-03-17 13:19:55 -07:00
Vivek Goyal 4141a00921 Fix the assignment to wrong variable
We should be assigning value to minFreeMetadata instead of minFreeData. This
is copy/paste error.

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
2016-03-17 15:19:08 +00:00
Antonio Murdaca f0d83c4cdb *: fix response body leaks
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-16 17:15:42 +01:00
Brian Goff 37a1fadae6 Merge pull request #21097 from thaJeztah/dont-run-without-udev-sync
Fail when devicemapper doesn't support udev-sync
2016-03-14 21:18:01 -04:00
Vincent Batts bfed97b688 Merge pull request #20786 from rhvgoyal/min-free-space
devmapper: Add a new option dm.min_free_space_percent
2016-03-14 20:10:43 -04:00
David Calavera ed6e33eede Make sure we call every graph init with the same root path.
Remove O(n^2) check for several prior configured drivers.

Signed-off-by: David Calavera <david.calavera@gmail.com>
2016-03-11 17:15:55 -05:00
Sebastiaan van Stijn de64171510 Fail when devicemapper doesn't support udev-sync
Now what we provide dynamic binaries for all plaforms,
we shouldn't try to run docker without udev sync support.

This change changes the previous warning to an Error,
unless the user explicitly overrides the warning, in
which case they're at their own risk.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2016-03-10 19:13:44 +01:00
Vivek Goyal 2e222f69b3 devmapper: Add a new option dm.min_free_space
Once thin pool gets full, bad things can happen. Especially in case of xfs
it is possible that xfs keeps on retrying IO infinitely (for certain kind
of IO) and container hangs. 

One way to mitigate the problem is that once thin pool is about to get full,
start failing some of the docker operations like pulling new images or
creation of new containers. That way user will get warning ahead of time
and can try to rectify it by creating more free space in thin pool. This
can be done either by deleting existing images/containers or by adding more
free space to thin pool.

This patch adds a new option dm.min_free_space to devicemapper graph
driver. Say one specifies dm.min_free_space=10%. This means atleast
10% of data and metadata blocks should be free in pool before new device
creation is allowed, otherwise operation will fail.

By default min_free_space is 10%. User can change it by specifying
dm.min_free_space=X% on command line. A value of 0% will disable the
check.

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
2016-03-07 20:27:39 +00:00
Tatsushi Inagaki e8513675a2 Aufs: reduce redundant parsing of mountinfo
Check whether or not the file system type of a mountpoint is aufs
by calling statfs() instead of parsing mountinfo. This assumes
that aufs graph driver does not allow aufs as a backing file
system.

Signed-off-by: Tatsushi Inagaki <e29253@jp.ibm.com>
2016-03-04 11:39:59 +09:00
John Starks 5649030e25 Write Windows layer diffs to tar in standard format
Previously, Windows layer diffs were written using a Windows-internal
format based on the BackupRead/BackupWrite Win32 APIs. This caused
problems with tar-split and tarsum and led to performance problems
in implementing methods such as DiffPath. It also was just an
unnecessary differentiation point between Windows and Linux.

With this change, Windows layer diffs look much more like their
Linux counterparts. They use AUFS-style whiteout files for files
that have been removed, and they encode all metadata directly in
the tar file.

This change only affects Windows post-TP4, since changes to the Windows
container storage APIs were necessary to make this possible.

Signed-off-by: John Starks <jostarks@microsoft.com>
2016-03-02 16:13:40 -08:00
David Calavera 8f109829e2 Merge pull request #20475 from Microsoft/jstarks/filegetter
graphdriver: Replace DiffPath with DiffGetter
2016-03-02 08:36:36 -08:00
John Starks 58bec40d16 graphdriver: Replace DiffPath with DiffGetter
This allows a graph driver to provide a custom FileGetter for tar-split
to use. Windows will use this to provide a more efficient implementation
in a follow-up change.

Signed-off-by: John Starks <jostarks@microsoft.com>
2016-03-01 10:25:33 -08:00
hsinko 772f5495b7 folders->directories
Signed-off-by: hsinko <21551195@zju.edu.cn>
2016-02-29 21:32:30 -08:00
Brian Goff e386dfc33f fix double-lock
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2016-02-27 09:49:21 -05:00
Brian Goff c2f7777603 Revert "Add finer-grained locking for aufs"
This reverts commit f31014197c.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2016-02-27 08:01:19 -05:00
Brian Goff f31014197c Add finer-grained locking for aufs
```
benchmark                       old ns/op       new ns/op     delta
BenchmarkConcurrentAccess-8     10269529748     26834747      -99.74%

benchmark                       old allocs     new allocs     delta
BenchmarkConcurrentAccess-8     309948         7232           -97.67%

benchmark                       old bytes     new bytes     delta
BenchmarkConcurrentAccess-8     23943576      1578441       -93.41%
```

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2016-02-25 18:06:41 -05:00
Brian Goff 55c91f2ab9 Fix some issues with concurrency in aufs.
Adds a benchmark to measure performance under concurrent actions.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2016-02-25 14:32:13 -05:00
Stefan Weil 2eee613326 Fix some typos in comments and strings
Most of them were found and fixed by codespell.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2016-02-22 20:27:15 +01:00
Kai Qiang Wu(Kennan) c33cdf9ee3 Fix the typo
Signed-off-by: Kai Qiang Wu(Kennan) <wkqwu@cn.ibm.com>
2016-02-16 07:00:01 +00:00
Phil Estes 922986b76e Add proper refcounting to zfs graphdriver
Fixes issues with layer remounting (e.g. a running container which then
has `docker cp` used to copy files in or out) by applying the same
refcounting implementation that exists in other graphdrivers like
overlay and aufs.

Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
2016-02-11 01:00:54 -05:00
Stefan J. Wernli 041a9510c6 Fixing 'docker save' on Windows.
Save was failing file integrity checksums due to bugs in both
Windows and Docker. This commit includes fixes to file time handling
in tarexport and system.chtimes that are necessary along with
the Windows platform fixes to correctly support save. With this
change, sysfile_backups for windowsfilter driver are no longer
needed, so that code is removed.

Signed-off-by: Stefan J. Wernli <swernli@microsoft.com>
2016-02-08 18:08:49 -08:00
David Calavera a93cb2b856 Merge pull request #20045 from estesp/zfs-userns-permissions-fix
Fix ZFS permissions bug with user namespaces
2016-02-05 15:32:43 -08:00
Phil Estes aef0995b02 Fix ZFS permissions bug with user namespaces
Fix root directory of the mountpoint being owned by real root. This is
unique to ZFS because of the way file mountpoints are created using the
ZFS tooling, and the remapping that happens at layer unpack doesn't
impact this root (already created) holding directory for the layer.

Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
2016-02-05 14:37:33 -05:00
Liu Bo b2e27fee53 Graphdriver/btrfs: Avoid using single d.Get()
For btrfs driver, in d.Create(), Get() of parentDir is called but not followed
by Put().

If we apply SElinux mount label, we need to mount btrfs subvolumes in d.Get(),
without a Put() would end up with a later Remove() failure on
"Device resourse is busy".

This calls the subvolume helper function directly in d.Create().

Signed-off-by: Liu Bo <bo.li.liu@oracle.com>
2016-02-04 10:25:24 -08:00
Tonis Tiigi b9a395c85d Remove case sensitive duplicate dir in vendor
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2016-02-03 13:06:32 -08:00
Kai Qiang Wu(Kennan) feda5d7684 Make btrfs call same interface as others
Most storage drivers call graphdriver.GetFSMagic(home),
it is more clean to easy to maintain. So btrfs need to
adopt such change.

Signed-off-by: Kai Qiang Wu(Kennan) <wkqwu@cn.ibm.com>
2016-02-01 07:50:21 +00:00
Phil Estes 67d86d10d8 Merge pull request #19216 from scaleoutsean/master
Add GPFS filesystem IDs
2016-01-22 21:27:03 -05:00
scaleoutsean bdc8241da9 Add GPFS
Signed-off-by: Sean Lee <seanlee@tw.ibm.com>
2016-01-22 21:12:47 +08:00
Tibor Vass d2448a0c20 Merge pull request #19524 from LK4D4/iocopy_writeto
Use bufio.Reader in io.Copy source for overlay.copyRegular
2016-01-21 15:37:12 -05:00
Alexander Morozov 3f5e1c69b3 Use pools.Copy instead of io.Copy for overlay.copyRegular
That function is pretty heavy used on container start. Autoallocating
buffer can be painful.

Signed-off-by: Alexander Morozov <lk4d4@docker.com>
2016-01-21 08:53:37 -08:00