Commit Graph

532 Commits

Author SHA1 Message Date
Derek McGowan b49dd20939 Update diff apply test to check sub directories
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
(cherry picked from commit 5c05b8447d06793530e3d860b99a6e878685275a)
Signed-off-by: Tibor Vass <tibor@docker.com>
2016-07-27 14:51:42 -07:00
Zhang Wei 861fa09131 small nit: fix unmeaningful log
Truncated dir name can't give any useful information, print whole dir
name will.

Bad debug log is like this:

```
DEBU[2449] aufs error unmounting /var/lib/doc: no such file or directory
```

Signed-off-by: Zhang Wei <zhangwei555@huawei.com>
(cherry picked from commit af8359562c9561afad0a05e66386588d17788804)
Signed-off-by: Tibor Vass <tibor@docker.com>
2016-07-25 23:18:28 -07:00
allencloud 4cd455a8d9 fix typos
Signed-off-by: allencloud <allen.sun@daocloud.io>
(cherry picked from commit 4e959ef2f7f063803d04e06166f459257eb94b5c)
Signed-off-by: Tibor Vass <tibor@docker.com>
2016-07-25 23:18:15 -07:00
Damian Smyth f370132a42 Create zfs directory if it doesn't exist at startup.
Signed-off-by: Damian Smyth <damian@dsau.co>
(cherry picked from commit 205bc6fc99872ba852fa7a158c106514d5b64179)
Signed-off-by: Tibor Vass <tibor@docker.com>
2016-07-25 23:15:41 -07:00
Derek McGowan adfeccf06f Allow option to override kernel check in overlay2
Add option to skip kernel check for older kernels which have been patched to support multiple lower directories in overlayfs.

Fixes #24023

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
(cherry picked from commit ff98da0607c4d6a94a2356d9ccaa64cc9d7f6a78)
Signed-off-by: Tibor Vass <tibor@docker.com>
2016-07-12 15:50:36 -07:00
Akihiro Suda 4d4848ab68 zfs: call mount.MakePrivate
Fix #24008

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
(cherry picked from commit 5ec8441bb55ecbf83da6407a4135ce83b1fe01c7)
2016-06-30 17:12:45 -07:00
Derek McGowan 07bd7b0128 overlay2: close read end of pipe on mount exec
Use StdinPipe to ensure pipe is properly closed after startup

Fixes #23686

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
(cherry picked from commit ad4b3e11fe41080d66ac1780b455246ae18bdc35)
2016-06-30 16:47:53 -07:00
Derek McGowan d99f4cc83d Fix overlay2 ignoring whiteout files
Currently when overlay creates a whiteout file then the overlay2 layer is archived,
the correct tar header will be created for the whiteout file, but the tar logic will then attempt to open the file causing a failure.
When tar encounters such failures the file is skipped and excluded for the archive, causing the whiteout to be ignored.
By skipping the copy of empty files, no open attempt will be made on whiteout files.

Fixes #23863

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
(cherry picked from commit bd13c53f8dc1504c9e681dfabef2afc1685ccec7)
2016-06-30 16:47:46 -07:00
Sebastiaan van Stijn b3387e96ed fix "overlay" -> "overlay2" in error message
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2d4b285a75621604f31058a43cb5ef1d4618533e)
2016-06-16 23:36:57 -07:00
Kai Qiang Wu(Kennan) 82453c84ba Add ecryptfs check for overlay2
We added docs about ecryptfs check but not in code side.
Also refactor code to make it clean.

Signed-off-by: Kai Qiang Wu(Kennan) <wkqwu@cn.ibm.com>
(cherry picked from commit 136323b04315eceffbed61d680878ed23cecc015)
2016-06-16 23:36:55 -07:00
Brian Goff 6ed921dc38 Merge pull request #23446 from tiborvass/plugins-experimental
Plugins: experimental support for new plugin management
2016-06-14 20:32:14 -04:00
Tibor Vass f37117045c plugins: experimental support for new plugin management
This patch introduces a new experimental engine-level plugin management
with a new API and command line. Plugins can be distributed via a Docker
registry, and their lifecycle is managed by the engine.
This makes plugins a first-class construct.

For more background, have a look at issue #20363.

Documentation is in a separate commit. If you want to understand how the
new plugin system works, you can start by reading the documentation.

Note: backwards compatibility with existing plugins is maintained,
albeit they won't benefit from the advantages of the new system.

Signed-off-by: Tibor Vass <tibor@docker.com>
Signed-off-by: Anusha Ragunathan <anusha@docker.com>
2016-06-14 14:20:27 -07:00
Derek McGowan e6f2e7646c Remove symlinks on layer removal for overlay2
Symlinks are currently not getting cleaned up when removing layers since only the root directory is removed.
On remove, read the link file and remove the associated link from the link directory.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-06-14 12:40:35 -07:00
Sebastiaan van Stijn 31e48e85ef Merge pull request #23501 from dmcgowan/fix-overlay-diff-test
Remove failing overlay test
2016-06-14 10:49:42 +02:00
Vincent Demeester c69614deae Merge pull request #23490 from shishir-a412ed/dm_options_modularized
Modularize dm.use_deferred_removal and dm.use_deferred_deletion logic.
2016-06-14 08:47:43 +02:00
Derek McGowan 0e74aabbb9 Remove failing overlay test
Diff apply is sometimes producing a different change list causing the tests to fail.
Overlay has a known issue calculating diffs of files which occur within the same second they were created.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-06-13 22:34:57 -07:00
Michael Crosby 8a2f9a249c Merge pull request #22126 from dmcgowan/overlay-native-diff
Overlay multiple lower directory support
2016-06-13 13:15:39 -07:00
Shishir Mahajan cac6658da0 Modularize dm.use_deferred_removal and dm.use_deferred_deletion logic.
Signed-off-by: Shishir Mahajan <shishir.mahajan@redhat.com>
2016-06-13 12:05:46 -04:00
Yong Tang a72b45dbec Fix logrus formatting
This fix tries to fix logrus formatting by removing `f` from
`logrus.[Error|Warn|Debug|Fatal|Panic|Info]f` when formatting string
is not present.

This fix fixes #23459.

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
2016-06-11 13:16:55 -07:00
Antonio Murdaca 44ccbb317c *: fix logrus.Warn[f]
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-06-11 19:42:38 +02:00
Derek McGowan 23e5c94cfb Add separate overlay2 driver
Adds a new overlay driver which uses multiple lower directories to create the union fs.
Additionally it uses symlinks and relative mount paths to allow a depth of 128 and stay within the mount page size limit.
Diffs and done directly over a single directory allowing diffs to be done efficiently and without the need fo the naive diff driver.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-06-08 00:16:01 -07:00
Derek McGowan 246e993031 Add more overlay tests and benchmarks
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-06-08 00:16:01 -07:00
Derek McGowan 8b0441d42c Expand graphtest package
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-06-08 00:16:01 -07:00
Brian Goff d85491ff4b Merge pull request #21946 from chosenken/add_disk_quota_to_zfs
Add support for setting storage size on ZFS containers
2016-06-07 22:18:25 -04:00
Sven Dowideit 98c245c9e6 Merge pull request #23193 from allencloud/fix-typos
use grep to find all a/an typos
2016-06-02 18:45:08 -07:00
Vincent Demeester 22aca92ee3 Merge pull request #23121 from unclejack/disallow_ecryptfs_aufs
aufs,overlay: disable on eCryptfs
2016-06-02 12:54:43 +02:00
allencloud c1be45fa38 fix typos
Signed-off-by: allencloud <allen.sun@daocloud.io>
2016-06-02 17:17:22 +08:00
unclejack 5e85ec82af aufs,overlay: disable on eCryptfs
Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com>
2016-06-01 21:00:35 +03:00
Antonio Murdaca b18062122d graphtest: fix cleanup logic
device Base should not exists on failure:

--- FAIL: TestDevmapperCreateBase (0.06s)
    graphtest_unix.go:122: stat
/tmp/docker-graphtest-079240530/devicemapper/mnt/Base/rootfs/a subdir:
no such file or directory
--- FAIL: TestDevmapperCreateSnap (0.00s)
    graphtest_unix.go:219: devmapper: device Base already
exists.

it should be:

--- FAIL: TestDevmapperCreateBase (0.25s)
	graphtest_unix.go:122: stat
/tmp/docker-graphtest-828994195/devicemapper/mnt/Base/rootfs/a subdir:
no such file or directory
--- FAIL: TestDevmapperCreateSnap (0.13s)
	graphtest_unix.go:122: stat
/tmp/docker-graphtest-828994195/devicemapper/mnt/Snap/rootfs/a subdir:
no such file or directory

Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-05-31 20:08:57 +02:00
John Starks f182748b9b Windows: fix Windows graph driver name
Reverting "Windows filter storage driver" to "windowsfilter".

Signed-off-by: John Starks <jostarks@microsoft.com>
2016-05-27 16:24:02 -07:00
Shishir Mahajan 09d0720e2f Fixes Issue # 22992: docker commit failing.
1) docker create / run / start: this would create a snapshot device and mounts it onto the filesystem.
So the first time GET operation is called. it will create the rootfs directory and return the path to rootfs
2) Now when I do docker commit. It will call the GET operation second time. This time the refcount will check
that the count > 1 (count=2). so the rootfs already exists, it will just return the path to rootfs.

Earlier it was just returning the mp: /var/lib/docker/devicemapper/mnt/{ID} and hence the inconsistent paths error.

Signed-off-by: Shishir Mahajan <shishir.mahajan@redhat.com>
2016-05-27 14:35:46 -04:00
Nirmal Mehta 59bb86a964 Merge branch 'master' into b 2016-05-24 21:43:45 -04:00
Nirmal Mehta 36a9151b27 fixed spelling error in windows.go
Signed-off-by: Nirmal Mehta <nirmalkmehta@gmail.com>
2016-05-24 21:40:08 -04:00
Michael Crosby 0802401d22 Merge pull request #22948 from dmcgowan/remove-unused-overlay-mounted
Remove unused mounted function in overlay
2016-05-24 13:08:06 -07:00
Vincent Demeester 86a7632d63 Merge pull request #22091 from amitkris/build_solaris
Get the Docker Engine to build clean on Solaris
2016-05-24 21:41:36 +02:00
Derek McGowan 5cc0824730 Remove unused mounted function in overlay
The mount check is now done by the FSChecker. This function is no longer needed and shouldn't be called.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-05-24 11:02:07 -07:00
Michael Crosby 8bb4d31b10 Remove mountedLayer Mount and Unmount
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-05-23 16:52:58 -07:00
Amit Krishnan 86d8758e2b Get the Docker Engine to build clean on Solaris
Signed-off-by: Amit Krishnan <krish.amit@gmail.com>
2016-05-23 16:37:12 -07:00
Michael Crosby 36a82c2032 Fix overlay use of rootdir and defer
Check for the rootDir first because the mergeDir may not exist if root
is present.

Also fix unmounting in the defer to make sure it does not have a
refcount.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-05-23 16:03:40 -07:00
Michael Crosby 4bac8bce98 Add windows graph driver ref counter
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-05-23 16:03:40 -07:00
Michael Crosby 5b6b8df0c1 Add reference counting to aufs
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-05-23 15:57:23 -07:00
Michael Crosby 1ba05cdb6a Add fast path for fsmagic supported drivers
For things that we can check if they are mounted by using their fsmagic
we should use that and for others do it the slow way.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-05-23 15:57:23 -07:00
Michael Crosby 31e903b0e1 Remove restart test
This test is not applicable anymore now that containers are not stopped
when the daemon is restored.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-05-23 15:57:23 -07:00
Michael Crosby 290be017c5 Remove overlay pathCache
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-05-23 15:57:23 -07:00
Michael Crosby 009ee16bef Restore ref count
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-05-23 15:57:23 -07:00
John Howard dd0fc2be8b Windows: Remove windowsdiff driver
Signed-off-by: John Howard <jhoward@microsoft.com>
2016-05-20 16:18:19 -07:00
Ken Herner 04b4e3e6d8 Add test to ZFS for disk quota
Signed-off-by: Ken Herner <kherner@progress.com>
2016-05-19 14:49:04 -04:00
Ken Herner 373654f43e Add error check after parseStorageOpt
Signed-off-by: Ken Herner <kherner@progress.com>
2016-05-19 14:49:04 -04:00
Ken Herner e918340431 Add support for setting storage size on zfs containers
Now supports setting a containers storage size when using zfs as the
storage engine.  By passing in `--storage-opt size=<size>`, the created
container's storage size will be limited to the given size.  Note that
the way zfs works, the given specified storage size will be given in
addition to the base container size.

Example:

The node image reports a size of `671M` from `df -h` when started.
Setting `--storage-opt size=2G` will result in a drive the size of
`671M` + `2G`, `2.7G` in total.  Available space will be `2.0G`.

The storage size is achieved by setting the zfs option `quota` to the
given size on the zfs volume.

Signed-off-by: Ken Herner <kherner@progress.com>
2016-05-19 14:49:04 -04:00
John Starks b3bc5e0fe4 Windows: reexec when importing layers
This improves reliability by doing parsing of potentially untrusted data
in a separate process. It opens the door for further security improvements
if we can lock down the reexec-ed process. It also improves import
performance by only taking the backup and restore privileges once, for the
whole process.

Signed-off-by: John Starks <jostarks@microsoft.com>
2016-05-13 18:16:11 -07:00