fuse-overlayfs snapshotter plugin for containerd

Unlike overlayfs, fuse-overlayfs can be used as a non-root user on almost all recent distros.

You do NOT need this fuse-overlayfs plugin on the following environments, because they support the real overlayfs for non-root users:

• kernel >= 5.11
• Ubuntu kernel, since circa 2015
• Debian 10 kernel
  • Debian 10 needs sudo modprobe overlay permit_mounts_in_userns=1. Future release of Debian with kernel >= 5.11 will not need this modprobe hack.

fuse-overlayfs-snapshotter is a non-core sub-project of containerd.

Requirements

• kernel >= 4.18
• containerd >= 1.4
• fuse-overlayfs >= 0.7.0

Setup

Two installation options are supported:

1. Embed fuse-overlayfs plugin into the containerd binary
2. Execute fuse-overlayfs plugin as a separate binary

Choose 1 if you don't mind recompiling containerd, otherwise choose 2.

Option 1: Embed fuse-overlayfs plugin into the containerd binary

Create builtins_fuseoverlayfs_linux.go under $GOPATH/src/github.com/containerd/containerd/cmd/containerd with the following content, and recompile the containerd binary:

/*
   Copyright The containerd Authors.

   Licensed under the Apache License, Version 2.0 (the "License");
   you may not use this file except in compliance with the License.
   You may obtain a copy of the License at

       http://www.apache.org/licenses/LICENSE-2.0

   Unless required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   See the License for the specific language governing permissions and
   limitations under the License.
*/

package main

import _ "github.com/containerd/fuse-overlayfs-snapshotter/plugin"

No extra configuration is needed.

See https://github.com/containerd/containerd/blob/master/docs/rootless.md for how to run containerd as a non-root user.

Option 2: Execute fuse-overlayfs plugin as a separate binary

"Easy way"

The easiest way is to use containerd-rootless-setuptool.sh included in nerdctl.

$ containerd-rootless-setuptool.sh install
$ containerd-rootless-setuptool.sh install-fuse-overlayfs
[INFO] Creating "/home/exampleuser/.config/systemd/user/containerd-fuse-overlayfs.service"
...
[INFO] Installed "containerd-fuse-overlayfs.service" successfully.
[INFO] To control "containerd-fuse-overlayfs.service", run: `systemctl --user (start|stop|restart) containerd-fuse-overlayfs.service`
[INFO] Add the following lines to "/home/exampleuser/.config/containerd/config.toml" manually:
### BEGIN ###
[proxy_plugins]
  [proxy_plugins."fuse-overlayfs"]
    type = "snapshot"
    address = "/run/user/1000/containerd-fuse-overlayfs.sock"
###  END  ###
[INFO] Set `export CONTAINERD_SNAPSHOTTER="fuse-overlayfs"` to use the fuse-overlayfs snapshotter.

Add the [proxy_plugins."fuse-overlayfs"] configuration shown above to ~/.config/containerd/config.toml. "1000" needs to be replaced with your actual UID.

"Hard way"

Click here to show the "hard way"

• Install containerd-fuse-overlayfs-grpc binary. The binary will be installed under $DESTDIR/bin.

$ make && DESTDIR=$HOME make install

• Create the following configuration in ~/.config/containerd/config.toml:

version = 2
# substitute "/home/suda" with your own $HOME
root = "/home/suda/.local/share/containerd"
# substitute "/run/user/1001" with your own $XDG_RUNTIME_DIR
state = "/run/user/1001/containerd"

[grpc]
  address = "/run/user/1001/containerd/containerd.sock"

[proxy_plugins]
  [proxy_plugins."fuse-overlayfs"]
    type = "snapshot"
    address = "/run/user/1001/containerd/fuse-overlayfs.sock"

• Start RootlessKit with sleep infinity (or any kind of "pause" command):

$ rootlesskit \
  --net=slirp4netns --disable-host-loopback \
  --copy-up=/etc --copy-up=/run \
  --state-dir=$XDG_RUNTIME_DIR/rootlesskit-containerd \
  sh -c "rm -rf /run/containerd ; sleep infinity"

(Note: rm -rf /run/containerd is a workaround for containerd/containerd#2767)

• Enter the RootlessKit namespaces and run containerd-fuse-overlayfs-grpc:

$ nsenter -U --preserve-credentials -m -n -t $(cat $XDG_RUNTIME_DIR/rootlesskit-containerd/child_pid) \
  containerd-fuse-overlayfs-grpc $XDG_RUNTIME_DIR/containerd/fuse-overlayfs.sock $HOME/.local/share/containerd-fuse-overlayfs

• Enter the same namespaces and run containerd:

$ nsenter -U --preserve-credentials -m -n -t $(cat $XDG_RUNTIME_DIR/rootlesskit-containerd/child_pid) \
  containerd -c $HOME/.config/containerd/config.toml

Usage

$ export CONTAINERD_SNAPSHOTTER=fuse-overlayfs
$ nerdctl run ...

How to test

To run the test as a non-root user, RootlessKit needs to be installed.

$ go test -exec rootlesskit -test.v -test.root

Project details

fuse-overlayfs-snapshotter is a containerd non-core sub-project, licensed under the Apache 2.0 license. As a containerd non-core sub-project, you will find the:

• Project governance,
• Maintainers,
• and Contributing guidelines

information in our containerd/project repository.