Commit Graph

39 Commits

Author SHA1 Message Date
Daniel J Walsh a3204cf7e8
Move to golang 1.18 and later
Github.com is reporting security issues on older versions of
golang.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2023-04-03 15:26:54 -04:00
Miloslav Trmač b8f4b5f2d3 Use os.ReadDir instead of ioutil.ReadDir
That is frequently an optimization, because it avoids
a per-item lstat().

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-09-12 16:31:38 +02:00
Miloslav Trmač cfbc77122a Use os.CreateTemp instead of ioutil.TempFile
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-09-12 16:30:48 +02:00
Miloslav Trmač 4b28197720 Use os.ReadFile instead of ioutil.ReadFile
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-09-12 16:30:43 +02:00
Daniel J Walsh 3f8c0dc0de
Wrap errors properly with fmt.Errorf
Also returned errors should not begine with a capatalized errors.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2022-07-12 13:26:10 -04:00
Sascha Grunert 3455d12729
Switch to golang native error wrapping
We now use the golang error wrapping format specifier `%w` instead of the
deprecated github.com/pkg/errors package.

Signed-off-by: Sascha Grunert <sgrunert@redhat.com>
2022-07-07 13:22:46 +02:00
Daniel J Walsh 04b2f0f741
Switch most calls to filepath.Walk to filepath.WalkDir
It is faster then Walk, when you don't need to stat every
file and directory.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2022-04-06 09:42:25 -04:00
gaohuatao 67e885a378 Fix cancel deferred remove bug
When cancel the deferred removal, if the device is already gone,
continue. According to the original logic, if the device does not exist,
an error is reported.

Signed-off-by: gaohuatao <gaohuatao@huawei.com>
2021-06-12 15:25:03 +08:00
gaohuatao 416095081c Avoid failure when unmount an unmounted mountpoint
Check if the mountpoint is mounted when unmount it to avoid failure.
If user manually run the umount command before it, the function
UnmountDevice returns an error, Although this error dose not cause the
container deletion process fail for the reason that the return value of
UnmountDevice function is not processed. However, the ERROR logs in the
log system are misleading

Signed-off-by: gaohuatao <gaohuatao@huawei.com>
2021-05-25 14:35:58 +08:00
gaohuatao 6886caeaea Expand the scope of transaction in the process of deleting device
When "docker load $image" and "docker rmi $image" commands are
repeatedly executed in the background, the dockerd daemon process is
killed. As a result, the DM device where the image resides may be
unavailable. The image can be queried, but the container fails to be
run. After function “devices.issueDiscard(info)” is executed and before
function "devices.deleteTransaction(info, syncDelete)" is executed, at
this point, dockerd daemon's withdrawal would result in dm device
discarded. Howerver, the dm device is not deleted at the same time.

Signed-off-by: gaohuatao <gaohuatao@huawei.com>
2021-05-13 00:37:02 -04:00
Nalin Dahyabhai 5ef1e9d68b Use json-iterator instead of encoding/json
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2021-05-06 12:24:24 -04:00
Kir Kolyshkin dc65c45e2d devmapper: use unix.Unmount, log failures
1. Use unix.Unmount for it wraps the error message.

2. Always log umount failures (as warnings).

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-09-15 13:13:55 -07:00
Kir Kolyshkin 5f1fa8b92d devmapper.shutdown: optimize
Move the "unmount and deactivate" code into a separate method, and
optimize it a bit:

1. Do not use filepath.Walk() as there's no requirement to recursively
   go into every directory under home/mnt; a list of directories in mnt
   is sufficient. With filepath.Walk(), in case some container will fail
   to unmount, it'll go through the whole container filesystem which is
   excessive and useless.

2. Do not use GetMounts() and do not check if a directory is mounted;
   just unmount it and ignore "not mounted" error. Note the same error
   is returned in case of wrong flags set, but as flags are hardcoded
   we can safely ignore such a case.

While at it, promote "can't unmount" log level from debug to warning.

[adopted from upstream commit f1a459229724f5e.]

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-09-15 13:03:17 -07:00
Aleksa Sarai 278bfc55ca devicemapper: remove container rootfs mountPath after umount
libdm currently has a fairly substantial DoS bug that makes certain
operations fail on a libdm device if the device has active references
through mountpoints. This is a significant problem with the advent of
mount namespaces and MS_PRIVATE, and can cause certain --volume mounts
to cause libdm to no longer be able to remove containers:

  % docker run -d --name testA busybox top
  % docker run -d --name testB -v /var/lib/docker:/docker busybox top
  % docker rm -f testA
  [fails on libdm with dm_task_run errors.]

This also solves the problem of unprivileged users being able to DoS
docker by using unprivileged mount namespaces to preseve mounts that
Docker has dropped.

Signed-off-by: Aleksa Sarai <asarai@suse.de>

[picked from upstream commit: 92e45b81e0a]
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-09-15 12:24:57 -07:00
Reinhard Tartler 086943ac54
unbreak build on mipsen harder
Stat_t on MIPS uses 32-bit fields for Dev and Rdev, so we need to cast these to uint64

Turns out the change in 07171909b0
is incomplete and an additional line needs a cast.

Inpiration from moby/moby#37490
Example build failure log: https://buildd.debian.org/status/fetch.php?pkg=golang-github-containers-storage&arch=mips64el&ver=1.20.2-1&stamp=1594477648&raw=0Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2020-07-19 15:23:53 -04:00
Daniel J Walsh f76d8a7277
Store the pvcreate --metadatasize option in storage.conf
As the number of devices increase the size of the metadata
needs to be modified.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-07-01 05:27:31 -04:00
Reinhard Tartler 07171909b0 unbreak build on mipsen
Stat_t on MIPS uses 32-bit fields for Dev and Rdev, so we need to cast these to uint64

Inpiration from https://github.com/moby/moby/pull/37490
Example build failure log: https://buildd.debian.org/status/fetch.php?pkg=golang-github-containers-storage&arch=mips64el&ver=1.15.8%2Bdfsg1-1&stamp=1580872314&raw=0

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2020-05-26 07:32:59 -04:00
Kir Kolyshkin 5124860f40 Fix MkdirAll usage
This subtle bug keeps lurking in because error checking for `Mkdir()`
and `MkdirAll()` is slightly different wrt `EEXIST`/`IsExist`:

 - for `Mkdir()`, `IsExist` error should (usually) be ignored
   (unless you want to make sure directory was not there before)
   as it means "the destination directory was already there";

 - for `MkdirAll()`, `IsExist` error should NEVER be ignored.

This commit removes ignoring the IsExist error, as it should not
be ignored.

For more details, a quote from my opencontainers/runc#162 (July 2015):

-quote-

TL;DR: check for IsExist(err) after a failed MkdirAll() is both
redundant and wrong -- so two reasons to remove it.

Quoting MkdirAll documentation:

MkdirAll creates a directory named path, along with any necessary
parents, and returns nil, or else returns an error. If path
is already a directory, MkdirAll does nothing and returns nil.

This means two things:

If a directory to be created already exists, no error is
returned.

If the error returned is IsExist (EEXIST), it means there exists
a non-directory with the same name as MkdirAll need to use for
directory. Example: we want to MkdirAll("a/b"), but file "a"
(or "a/b") already exists, so MkdirAll fails.

The above is a theory, based on quoted documentation and my UNIX
knowledge.

In practice, though, current MkdirAll implementation [1] returns
ENOTDIR in most of cases described in #2, with the exception when
there is a race between MkdirAll and someone else creating the
last component of MkdirAll argument as a file. In this very case
MkdirAll() will indeed return EEXIST.
Because of #1, IsExist check after MkdirAll is not needed.

Because of #2 and #3, ignoring IsExist error is just plain wrong,
as directory we require is not created. It's cleaner to report
the error now.

Note this error is all over the tree, I guess due to copy-paste,
or trying to follow the same usage pattern as for Mkdir(),
or some not quite correct examples on the Internet.

[1] https://github.com/golang/go/blob/f9ed2f75/src/os/path.go

-end-quote-

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-05-18 07:18:00 -07:00
Kir Kolyshkin d778a0a2ec devmapper: no need to wrap Mount errors
With the previous patch, Mount error is now verbose enough
so we don't have to supply all the gory details.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-03-10 06:41:39 -07:00
Sascha Grunert d88ef6dc08
Enable goimports linter and fix lints
Signed-off-by: Sascha Grunert <sgrunert@suse.com>
2020-01-28 15:59:15 +01:00
Sascha Grunert b66d5a5c12
Enable goconst linter and fix lints
Signed-off-by: Sascha Grunert <sgrunert@suse.com>
2020-01-28 15:59:15 +01:00
Nalin Dahyabhai ba598e19f2 Disable cgo-requiring bits when cgo is not enabled
Adjust build tags in drivers and pkg so that builds with CGO_ENABLED=0
won't fail outright.  This ends up disabling btrfs (which uses kernel
headers), ostree (which uses libostree), overlayfs (which uses C headers
to define fs_disk_quota_t), and devicemapper (which uses libdevmapper
and loopback) by default.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2019-08-05 13:42:50 -04:00
Daniel J Walsh 41f2d1cd87 Fix broken sprintf
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2018-12-04 18:27:22 -05:00
Daniel J Walsh ab6be50561
Allow the passing in and retrieval of mount options
We want to allow tools like podman/buildah to override default storage
container mount options on a container by container basis.

For example if the default mount options for containers/storage include
nodev or nosuid, we want to allow podman to turn these off if the user
specifies --privileged.

We also might want to turn off certain user namespace flags that will cause
buildah and podman build to work slower when creating container images.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2018-11-19 19:14:21 -05:00
Daniel J Walsh 8b1a0f8d68
Add default mount options to pass to drivers
I believe we should be running container images mounted with nodev by default.
This would eliminate the disk of a device sneaking into the container without
being on the approved list.  This would give us the same or potentially additional
security over the device cgroup.

It would be nice if this could be passed in on an image by image basis.  So users
could also specify if they want nosuid images.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2018-07-18 10:20:27 -04:00
Daniel J Walsh 00d69971c2 Cleanup leaked devices when the container/storage is initialized
Currently the code waits 30 seconds before cleaning up deferred deleted
thinpools.  This change will cause the cleanup to happen right away and
then will fire every 30 seconds.  Note this could cause some slowdown
during startup, if and only iff their are Leaked devices.

Since this library could be used with non long running daemons like podman
and buildah, it is better to cleanup the devices right away.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2018-05-18 11:17:05 -04:00
Daniel J Walsh da1e7e5d28 Force the user to specify a device when setting up devicemapper
We do not want to fall back on to loop back devices when setting up
devicemapper. We have decided that this causes too many issues, and
we have an excellent default in Overlay to handle first setup.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2018-05-18 11:17:05 -04:00
Daniel J Walsh 98c480e157 Backport moby devmapper driver changes to containers/storage
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2017-09-26 20:45:15 +00:00
Mrunal Patel 59da502256 Update deps to github.com/sirupsen/logrus v1.0.0
Also updates github.com/Microsoft/hcsshim
and github.com/docker/go-connections so they work
with the newer version of logrus.

Signed-off-by: Mrunal Patel <mrunalp@gmail.com>
2017-08-01 07:25:56 -07:00
Nalin Dahyabhai 4027283f30 Refuse to start loopback devicemapper on AUFS
An AUFS filesystem doesn't give us the "real" device and inode numbers
of an underlying file when we stat() it, so we'll hit errors trying to
resume use of a pool when that pool is built on loopback devices using
files that live on an AUFS filesystem.  Refuse to let ourselves be put
into that situation.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2017-07-18 10:43:23 -04:00
Nalin Dahyabhai 5375906c2e Wrap some driver-level errors
Wrap graphdriver.{ErrNotSupported,ErrPrerequisites,ErrIncompatibleFS}
errors in contexts using github.com/pkg/errors, and dig them out for
comparison using errors.Cause().

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2017-07-10 14:48:46 -04:00
Nalin Dahyabhai 230b9ab91a Replace the build system
Replace the bundle-targeted build system with simpler Makefile targets
and fewer helper scripts.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2017-05-25 15:50:33 -04:00
Daniel J Walsh f5ebf4aeb9 Fix lint errors
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2017-03-22 16:08:29 -04:00
Daniel J Walsh 79a5d04c2f Switch to using opencontainers/selinux
We have moved runc/libcontainers selinux support out of libcontainer
into opencontainers/selinux.  Switching containers/storage to use
new interfaces.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2017-03-22 14:12:48 -04:00
Daniel J Walsh 850e2bcf3b Remove "docker" strings from container storage
We want to make sure that content created out of container storage
does not refer to docker.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2017-01-26 09:47:23 +01:00
Nalin Dahyabhai 2bce049286 devmapper: exclude the root of a file tree walk
When we walk the directory of mount information, don't try to load
information from the root of the walk, which produces a confusing error
message.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2016-10-25 16:17:35 -04:00
Nalin Dahyabhai d855327b5e Clean up the build machinery
Most of the build machinery was still present, but no longer
functioning.  Tear most of it down.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2016-07-18 17:35:50 -04:00
Nalin Dahyabhai a05bdbb291 Fix several golint errors and a vet error
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2016-07-18 17:35:50 -04:00
Nalin Dahyabhai 60ff079996 Rename: cow/cowman -> storage/oci-storage
Rename the library module and CLI wrapper.
Rename daemon/graphdriver to drivers.
Catch up vendoring to match modules we've pruned.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2016-07-18 17:35:50 -04:00