While resolving `workdir` we mostly create a `workdir` when `stat`
fails with `ENOENT` or `ErrNotExist` however following cases are not
true when user explicitly specifies a `workdir` while `running` using
`--workdir` which tells `podman` to only use workdir if its exists on
the container. Following configuration is implicity set with other
`run` mechanism like `podman play kube`
Problem with explicit `--workdir` or similar implicit config in `podman play
kube` is that currently podman ignores the fact that workdir can also be
a `symlink` and actual `link` could be valid.
Hence following commit ensures that in such scenarios when a `workdir`
is not found and we cannot create a `workdir` podman must perform a
check to ensure that if `workdir` is a `symlink` and `link` is resolved
successfully and resolved link is present on the container then we
return as it is.
Docker performs a similar behviour.
Signed-off-by: Aditya R <arajan@redhat.com>
podman container clone takes the id of an existing continer and creates a specgen from the given container's config
recreating all proper namespaces and overriding spec options like resource limits and the container name if given in the cli options
this command utilizes the common function DefineCreateFlags meaning that we can funnel as many create options as we want
into clone over time allowing the user to clone with as much or as little of the original config as they want.
container clone takes a second argument which is a new name and a third argument which is an image name to use instead of the original container's
the current supported flags are:
--destroy (remove the original container)
--name (new ctr name)
--cpus (sets cpu period and quota)
--cpuset-cpus
--cpu-period
--cpu-rt-period
--cpu-rt-runtime
--cpu-shares
--cpuset-mems
--memory
--run
resolves#10875
Signed-off-by: cdoern <cdoern@redhat.com>
Signed-off-by: cdoern <cbdoer23@g.holycross.edu>
Signed-off-by: cdoern <cdoern@redhat.com>
The `podman network connect` and `podman network disconnect`
commands give containers access to different networks than the
ones they were created with; these networks can also have DNS
servers associated with them. Until now, however, we did not
modify resolv.conf as network membership changed.
With this PR, `podman network connect` will add any new
nameservers supported by the new network to the container's
/etc/resolv.conf, and `podman network disconnect` command will do
the opposite, removing the network's nameservers from
`/etc/resolv.conf`.
Fixes#9603
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
Append the podman dns seach domain to the host search domains when we
use the dnsname/aardvark server. Previously it would only use podman
seach domains and discard the host domains.
Fixes#13103
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
these mount flags are already used for the /dev/shm mount on the host,
but they are not set for the bind mount itself.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
Often users want their overlayed volumes to be `non-volatile` in nature
that means that same `upper` dir can be re-used by one or more
containers but overall of nature of volumes still have to be `overlay`
so work done is still on a overlay not on the actual volume.
Following PR adds support for more advanced options i.e custom `workdir`
and `upperdir` for overlayed volumes. So that users can re-use `workdir`
and `upperdir` across new containers as well.
Usage
```console
$ podman run -it -v myvol:/data:O,upperdir=/path/persistant/upper,workdir=/path/persistant/work alpine sh
```
Signed-off-by: Aditya R <arajan@redhat.com>
This change updates the CDI API to commit 46367ec063fda9da931d050b308ccd768e824364
which addresses some inconistencies in the previous implementation.
Signed-off-by: Evan Lezar <elezar@nvidia.com>
The libpod/network packages were moved to c/common so that buildah can
use it as well. To prevent duplication use it in podman as well and
remove it from here.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
For ip/macvlan networks we cannot use the gateway as address for this
hostname. In this case the gateway is normally not on the host so we
just try to use a local ip instead.
[NO NEW TESTS NEEDED] We cannot run macvlan networks in CI.
Fixes#11351
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Some containers require certain user account(s) to exist within the
container when they are run. This option will allow callers to add a
bunch of passwd entries from the host to the container even if the
entries are not in the local /etc/passwd file on the host.
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1935831
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
When Podman is running a container in private IPC mode (default), it
creates a bind mount for /dev/shm that is then attached to a tmpfs
folder on the host file system. However, checkpointing a container has
the side-effect of stopping that container and unmount the tmpfs used
for /dev/shm. As a result, after checkpoint all files stored in the
container's /dev/shm would be lost and the container might fail to
restore from checkpoint.
To address this problem, this patch creates a tar file with the
content of /dev/shm that is included in the container checkpoint and
used to restore the container.
Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
This ensures that existing containers will still manage
`/etc/passwd` by default, as they have been doing until now. New
containers that explicitly set `false` will still have passwd
management disabled, but otherwise the code will run.
[NO NEW TESTS NEEDED] This will only be caught on upgrade and I
don't really know how to write update tests - and Ed is on PTO.
Signed-off-by: Matthew Heon <mheon@redhat.com>
added support for a new flag --passwd which, when false prohibits podman from creating entries in
/etc/passwd and /etc/groups allowing users to modify those files in the container entrypoint
resolves#11805
Signed-off-by: cdoern <cdoern@redhat.com>
Add first non localhost ipv4 of all host interfaces as destination
for host.contaners.internal for rootless containers.
Fixes: https://github.com/containers/podman/issues/12000
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
This adds the following information to the output of 'podman inspect':
* CheckpointedAt - time the container was checkpointed
Only set if the container has been checkpointed
* RestoredAt - time the container was restored
Only set if the container has been restored
* CheckpointLog - path to the checkpoint log file (CRIU's dump.log)
Only set if the log file exists (--keep)
* RestoreLog - path to the restore log file (CRIU's restore.log)
Only set if the log file exists (--keep)
* CheckpointPath - path to the actual (CRIU) checkpoint files
Only set if the checkpoint files exists (--keep)
* Restored - set to true if the container has been restored
Only set if the container has been restored
Signed-off-by: Adrian Reber <areber@redhat.com>
The new network db structure stores everything in the networks bucket.
Previously some network settings were not written the the network bucket
and only stored in the container config.
Instead of the old format which used the container ID as value in the
networks buckets we now use the PerNetworkoptions struct there.
To migrate existing users we use the state.GetNetworks() function. If it
fails to read the new format it will automatically migrate the old
config format to the new one. This is allows a flawless migration path.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
While trying to match permissions of target directory podman adds
extra `0111` which should not be needed if target path does not have
execute permission.
Signed-off-by: Aditya Rajan <arajan@redhat.com>
improve the heuristic to detect the scope that was created for the container.
This is necessary with systemd running as PID 1, since it moves itself
to a different sub-cgroup, thus stats would not account for other
processes in the same container.
Closes: https://github.com/containers/podman/issues/12400
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
Some field names are confusing. Change them so that they make more sense
to the reader.
Since these fields are only in the main branch we can safely rename them
without worrying about backwards compatibility.
Note we have to change the field names in netavark too.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Podman adds a few environment variables by default, and
currently there is no way to get rid of them from your container.
This option will allow you to specify which defaults you don't
want.
--unsetenv-all will remove all default environment variables.
Default environment variables can come from podman builtin,
containers.conf or from the container image.
Fixes: https://github.com/containers/podman/issues/11836
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Honor custom `target` if specified while running or creating containers
with secret `type=mount`.
Example:
`podman run -it --secret token,type=mount,target=TOKEN ubi8/ubi:latest
bash`
Signed-off-by: Aditya Rajan <arajan@redhat.com>
This adds the parameter '--print-stats' to 'podman container restore'.
With '--print-stats' Podman will measure how long Podman itself, the OCI
runtime and CRIU requires to restore a checkpoint and print out these
information. CRIU already creates process restore statistics which are
just read in addition to the added measurements. In contrast to just
printing out the ID of the restored container, Podman will now print
out JSON:
# podman container restore --latest --print-stats
{
"podman_restore_duration": 305871,
"container_statistics": [
{
"Id": "47b02e1d474b5d5fe917825e91ac653efa757c91e5a81a368d771a78f6b5ed20",
"runtime_restore_duration": 140614,
"criu_statistics": {
"forking_time": 5,
"restore_time": 67672,
"pages_restored": 14
}
}
]
}
The output contains 'podman_restore_duration' which contains the
number of microseconds Podman required to restore the checkpoint. The
output also includes 'runtime_restore_duration' which is the time
the runtime needed to restore that specific container. Each container
also includes 'criu_statistics' which displays the timing information
collected by CRIU.
Signed-off-by: Adrian Reber <areber@redhat.com>
This adds the parameter '--print-stats' to 'podman container checkpoint'.
With '--print-stats' Podman will measure how long Podman itself, the OCI
runtime and CRIU requires to create a checkpoint and print out these
information. CRIU already creates checkpointing statistics which are
just read in addition to the added measurements. In contrast to just
printing out the ID of the checkpointed container, Podman will now print
out JSON:
# podman container checkpoint --latest --print-stats
{
"podman_checkpoint_duration": 360749,
"container_statistics": [
{
"Id": "25244244bf2efbef30fb6857ddea8cb2e5489f07eb6659e20dda117f0c466808",
"runtime_checkpoint_duration": 177222,
"criu_statistics": {
"freezing_time": 100657,
"frozen_time": 60700,
"memdump_time": 8162,
"memwrite_time": 4224,
"pages_scanned": 20561,
"pages_written": 2129
}
}
]
}
The output contains 'podman_checkpoint_duration' which contains the
number of microseconds Podman required to create the checkpoint. The
output also includes 'runtime_checkpoint_duration' which is the time
the runtime needed to checkpoint that specific container. Each container
also includes 'criu_statistics' which displays the timing information
collected by CRIU.
Signed-off-by: Adrian Reber <areber@redhat.com>
A rootless container created with a custom userns and forwarded ports
did not work. I refactored the network setup to make the setup logic
more clear.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
There was the question about how long it takes to create a checkpoint.
CRIU already provides some statistics about how long it takes to create
a checkpoint and similar.
With this change the file 'stats-dump' is included in the checkpoint
archive and the tool checkpointctl can be used to display these
statistics:
./checkpointctl show -t /tmp/cp.tar --print-stats
Displaying container checkpoint data from /tmp/dump.tar
[...]
CRIU dump statistics
+---------------+-------------+--------------+---------------+---------------+---------------+
| FREEZING TIME | FROZEN TIME | MEMDUMP TIME | MEMWRITE TIME | PAGES SCANNED | PAGES WRITTEN |
+---------------+-------------+--------------+---------------+---------------+---------------+
| 105405 us | 1376964 us | 504399 us | 446571 us | 492153 | 88689 |
+---------------+-------------+--------------+---------------+---------------+---------------+
Signed-off-by: Adrian Reber <areber@redhat.com>
A restored container still had the state set to 'Checkpointed: true'
which seems wrong if it running again.
[NO NEW TESTS NEEDED]
Signed-off-by: Adrian Reber <areber@redhat.com>
This will change mount of /dev within container to noexec, making
containers slightly more secure.
[NO NEW TESTS NEEDED]
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Checkpoint is blowing up when you use --log-driver=none
[NO NEW TESTS NEEDED] No way currently to test checkpoint restore.
Fixes: https://github.com/containers/podman/issues/11974
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
This matches what docker does. Also make sure the net aliases are also
shown when the container is stopped.
docker-compose uses this special alias entry to check if it is already
correctly connected to the network. [1]
Because we do not support static ips on network connect at the moment
calling disconnect && connect will loose the static ip.
Fixes#11748
[1] 0bea52b18d/compose/service.py (L663-L667)
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Following PR allows containers to create and mount overlays on top of
named volumes instead of mounting actual volumes via already documented `:O`.
Signed-off-by: Aditya Rajan <arajan@redhat.com>
Let the gvproxy dns server handle the host.containers.internal entry.
Support for this is already added to gvproxy. [1]
To make sure the container uses the dns response from gvproxy we should
not add host.containers.internal to /etc/hosts in this case.
[NO TESTS NEEDED] podman machine has no tests :/
Fixes#11642
[1] 1108ea4516
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
The check for net=none was wrong. It just assumed when we do not create
the netns but have one set that we use the none mode. This however also
applies to a container which joins the pod netns.
To correctly check for the none mode use `config.NetMode.IsNone()`.
Fixes#11596
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Make use of the new network interface in libpod.
This commit contains several breaking changes:
- podman network create only outputs the new network name and not file
path.
- podman network ls shows the network driver instead of the cni version
and plugins.
- podman network inspect outputs the new network struct and not the cni
conflist.
- The bindings and libpod api endpoints have been changed to use the new
network structure.
The container network status is stored in a new field in the state. The
status should be received with the new `c.getNetworkStatus`. This will
migrate the old status to the new format. Therefore old containers should
contine to work correctly in all cases even when network connect/
disconnect is used.
New features:
- podman network reload keeps the ip and mac for more than one network.
- podman container restore keeps the ip and mac for more than one
network.
- The network create compat endpoint can now use more than one ipam
config.
The man pages and the swagger doc are updated to reflect the latest
changes.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
When inspecting a container, we now report whether the container
was stopped by a `podman checkpoint` operation via a new bool in
the State portion of inspected, `Checkpointed`.
Signed-off-by: Matthew Heon <mheon@redhat.com>
Make sure that Podman passes the LISTEN_* environment into containers.
Similar to runc, LISTEN_PID is set to 1.
Also remove conditionally passing the LISTEN_FDS as extra files.
The condition was wrong (inverted) and introduced to fix#3572 which
related to running under varlink which has been dropped entirely
with Podman 3.0. Note that the NOTIFY_SOCKET and LISTEN_* variables
are cleared when running `system service`.
Fixes: #10443
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
There are use-cases where users would want to use overlay-mounts as
workdir. For such cases workdir should be resolved after all the mounts
are completed during the container init process.
Signed-off-by: Aditya Rajan <arajan@redhat.com>
This leverages conmon's ability to proxy the SD-NOTIFY socket.
This prevents locking caused by OCI runtime blocking, waiting for
SD-NOTIFY messages, and instead passes the messages directly up
to the host.
NOTE: Also re-enable the auto-update tests which has been disabled due
to flakiness. With this change, Podman properly integrates into
systemd.
Fixes: #7316
Signed-off-by: Joseph Gooch <mrwizard@dok.org>
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Add the --userns flag to podman pod create and keep
track of the userns setting that pod was created with
so that all containers created within the pod will inherit
that userns setting.
Specifically we need to be able to launch a pod with
--userns=keep-id
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
To match Docker's behavior, in the `--net=host` case, we need to
use the host's `/etc/hosts` file, unmodified (without adding an
entry for the container). We will still respect hosts from
`--add-host` but will not make any automatic changes.
Fortuntely, this is strictly a matter of removal and refactoring
as we already base our `/etc/hosts` on the host's version - just
need to remove the code that added entries when net=host was set.
Fixes#10319
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
This adds support to checkpoint containers out of pods and restore
container into pods.
It is only possible to restore a container into a pod if it has been
checkpointed out of pod. It is also not possible to restore a non pod
container into a pod.
The main reason this does not work is the PID namespace. If a non pod
container is being restored in a pod with a shared PID namespace, at
least one process in the restored container uses PID 1 which is already
in use by the infrastructure container. If someone tries to restore
container from a pod with a shared PID namespace without a shared PID
namespace it will also fail because the resulting PID namespace will not
have a PID 1.
Signed-off-by: Adrian Reber <areber@redhat.com>
The upcoming commit to support checkpointing out of Pods requires CRIU
3.16. This changes the CRIU version check to support checking for
different versions.
Signed-off-by: Adrian Reber <areber@redhat.com>
If mounting to existing directory the uid/gid should be preserved.
Primary uid/gid of container shouldn't be used.
Signed-off-by: Matej Vasek <mvasek@redhat.com>
First, make podman diff accept optionally a second argument. This allows
the user to specify a second image/container to compare the first with.
If it is not set the parent layer will be used as before.
Second, podman container diff should only use containers and podman
image diff should only use images. Previously, podman container diff
would use the image when both an image and container with this name
exists.
To make this work two new parameters have been added to the api. If they
are not used the previous behaviour is used. The same applies to the
bindings.
Fixes#10649
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Previously podman failed when run in an environment where 127.0.0.53 is
the only nameserver but systemd-resolved is not used directly.
In practice this happened when podman was run within an alpine container
that used the host's network and the host was running systemd-resolved.
This fix makes podman ignore a file not found error when reading /run/systemd/resolve/resolv.conf.
Closes#10733
[NO TESTS NEEDED]
Signed-off-by: Max Goltzsche <max.goltzsche@gmail.com>
The container name should have the slirp interface ip set in /etc/hosts
and not the gateway ip. Commit c8dfcce6db introduced this regression.
Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1972073
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Unfortunately --pre-checkpointing never worked as intended and recent
changes to runc have shown that it is broken.
To create a pre-checkpoint CRIU expects the paths between the
pre-checkpoints to be a relative path. If having a previous checkpoint
it needs the be referenced like this: --prev-images-dir ../parent
Unfortunately Podman was giving runc (and CRIU) an absolute path.
Unfortunately, again, until March 2021 CRIU silently ignored if
the path was not relative and switch back to normal checkpointing.
This has been now fixed in CRIU and runc and running pre-checkpoint
with the latest runc fails, because runc already sees that the path is
absolute and returns an error.
This commit fixes this by giving runc a relative path.
This commit also fixes a second pre-checkpointing error which was just
recently introduced.
So summarizing: pre-checkpointing never worked correctly because CRIU
ignored wrong parameters and recent changes broke it even more.
Now both errors should be fixed.
[NO TESTS NEEDED]
Signed-off-by: Adrian Reber <areber@redhat.com>
Signed-off-by: Adrian Reber <adrian@lisas.de>
When 127.0.0.53 is the only nameserver in /etc/resolv.conf assume
systemd-resolved is used. This is better because /etc/resolv.conf does
not have to be symlinked to /run/systemd/resolve/stub-resolv.conf in
order to use systemd-resolved.
[NO TESTS NEEDED]
Fixes: #10570
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
The checkpoint archive compression was hardcoded to `archive.Gzip`.
There have been requests to make the used compression algorithm
selectable. There was especially the request to not compress the
checkpoint archive to be able to create faster checkpoints when not
compressing it.
This also changes the default from `gzip` to `zstd`. This change should
not break anything as the restore code path automatically handles
whatever compression the user provides during restore.
Signed-off-by: Adrian Reber <areber@redhat.com>
The containers /etc/resolv.conf allways preserved the ipv6 nameserves
from the host even when the container did not supported ipv6. Check
if the cni result contains an ipv6 address or slirp4netns has ipv6
support enabled and only add the ipv6 nameservers when this is the case.
The test needs to have an ipv6 nameserver in the hosts /etc/hosts but we
should never mess with this file on the host. Therefore the test is
skipped when no ipv6 is detected.
Fixes#10158
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Support UID, GID, Mode options for mount type secrets. Also, change
default secret permissions to 444 so all users can read secret.
Signed-off-by: Ashley Cui <acui@redhat.com>
This change adds the entry `host.containers.internal` to the `/etc/hosts`
file within a new containers filesystem. The ip address is determined by
the containers networking configuration and points to the gateway address
for the containers networking namespace.
Closes#5651
Signed-off-by: Baron Lenardson <lenardson.baron@gmail.com>
Docker allows relabeling of any volume passed in via -v, even
including named volumes. This normally isn't an issue at all,
given named volumes get the right label for container access
automatically, but this becomes an issue when volume plugins are
involved - these aren't managed by Podman, and may well be
unaware of SELinux labelling. We could automatically relabel
these volumes on creation, but I'm still reluctant to do that
(feels like it could break things). Instead, let's allow :z and
:Z to be used with named volumes, so users can explicitly request
relabel of a volume plugin-backed volume.
We also get :U at the same time. I don't see any real need for it
but it also doesn't seem to hurt, so I didn't bother disabling
it.
Fixes#10273
Signed-off-by: Matthew Heon <mheon@redhat.com>
Revert : https://github.com/containers/podman/pull/9895
Turns out that if Docker is in --selinux-enabeled, it still relabels if
the user tells the system to, even if running a --privileged container
or if the selinux separation is disabled --security-opt label=disable.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Env var secrets are env vars that are set inside the container but not
commited to and image. Also support reading from env var when creating a
secret.
Signed-off-by: Ashley Cui <acui@redhat.com>
extend to pods the existing check whether the cgroup is usable when
running as rootless with cgroupfs.
commit 17ce567c68 introduced the
regression.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
Migrate the Podman code base over to `common/libimage` which replaces
`libpod/image` and a lot of glue code entirely.
Note that I tried to leave bread crumbs for changed tests.
Miscellaneous changes:
* Some errors yield different messages which required to alter some
tests.
* I fixed some pre-existing issues in the code. Others were marked as
`//TODO`s to prevent the PR from exploding.
* The `NamesHistory` of an image is returned as is from the storage.
Previously, we did some filtering which I think is undesirable.
Instead we should return the data as stored in the storage.
* Touched handlers use the ABI interfaces where possible.
* Local image resolution: previously Podman would match "foo" on
"myfoo". This behaviour has been changed and Podman will now
only match on repository boundaries such that "foo" would match
"my/foo" but not "myfoo". I consider the old behaviour to be a
bug, at the very least an exotic corner case.
* Futhermore, "foo:none" does *not* resolve to a local image "foo"
without tag anymore. It's a hill I am (almost) willing to die on.
* `image prune` prints the IDs of pruned images. Previously, in some
cases, the names were printed instead. The API clearly states ID,
so we should stick to it.
* Compat endpoint image removal with _force_ deletes the entire not
only the specified tag.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
if --cgroup-parent is specified, always honor it without doing any
detection whether cgroups are supported or not.
Closes: https://github.com/containers/podman/issues/10173
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
- Persist CDIDevices in container config
- Add e2e test
- Log HasDevice error and add additional condition for safety
Signed-off-by: Sebastian Jug <seb@stianj.ug>
do not set the cgroup parent when running as rootless with cgroupfs,
even if cgroup v2 is used.
Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1947999
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
Docker does not relabel this content, and openstack is running
containers in this manner. There is a penalty for doing this
on each container, that is not worth taking on a disable SELinux
container.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Instead of creating an extra container create a network and mount
namespace inside the podman user namespace. This ns is used to
for rootless cni operations.
This helps to align the rootless and rootful network code path.
If we run as rootless we just have to set up a extra net ns and
initialize slirp4netns in it. The ocicni lib will be called in
that net ns.
This design allows allows easier maintenance, no extra container
with pause processes, support for rootless cni with --uidmap
and possibly more.
The biggest problem is backwards compatibility. I don't think
live migration can be possible. If the user reboots or restart
all cni containers everything should work as expected again.
The user is left with the rootless-cni-infa container and image
but this can safely be removed.
To make the existing cni configs work we need execute the cni plugins
in a extra mount namespace. This ensures that we can safely mount over
/run and /var which have to be writeable for the cni plugins without
removing access to these files by the main podman process. One caveat
is that we need to keep the netns files at `XDG_RUNTIME_DIR/netns`
accessible.
`XDG_RUNTIME_DIR/rootless-cni/{run,var}` will be mounted to `/{run,var}`.
To ensure that we keep the netns directory we bind mount this relative
to the new root location, e.g. XDG_RUNTIME_DIR/rootless-cni/run/user/1000/netns
before we mount the run directory. The run directory is mounted recursive,
this makes the netns directory at the same path accessible as before.
This also allows iptables-legacy to work because /run/xtables.lock is
now writeable.
Signed-off-by: Paul Holzinger <paul.holzinger@web.de>
Erik Sjolund reported an issue where a badly formated file
could be passed into the `--tz` option and then the date in the container
would be badly messed up:
```
erik@laptop:~$ echo Hello > file.txt
erik@laptop:~$ podman run --tz=../../../home/erik/file.txt --rm -ti
docker.io/library/alpine cat /etc/localtime
Hello
erik@laptop:~$ podman --version
podman version 3.0.0-rc1
erik@laptop:~$
```
This fix checks to make sure the TZ passed in is a valid
value and then proceeds with the rest of the processing.
This was first reported as a potential security issue, but it
was thought not to be. However, I thought closing the hole
sooner rather than later would be good.
Signed-off-by: TomSweeneyRedHat <tsweeney@redhat.com>
The --trace has helped in early stages analyze Podman code. However,
it's contributing to dependency and binary bloat. The standard go
tooling can also help in profiling, so let's turn `--trace` into a NOP.
[NO TESTS NEEDED]
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
To be able to reuse common checkpoint/restore functions this commit
moves code to pkg/checkpoint/crutils.
This commit has not functional changes. It only moves code around.
[NO TESTS NEEDED] - only moving code around
Signed-off-by: Adrian Reber <areber@redhat.com>
We missed bumping the go module, so let's do it now :)
* Automated go code with github.com/sirkon/go-imports-rename
* Manually via `vgrep podman/v2` the rest
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Currently if the host shares container storage with a container
running podman, the podman inside of the container resets the
storage on the host. This can cause issues on the host, as
well as causes the podman command running the container, to
fail to unmount /dev/shm.
podman run -ti --rm --privileged -v /var/lib/containers:/var/lib/containers quay.io/podman/stable podman run alpine echo hello
* unlinkat /var/lib/containers/storage/overlay-containers/a7f3c9deb0656f8de1d107e7ddff2d3c3c279c11c1635f233a0bffb16051fb2c/userdata/shm: device or resource busy
* unlinkat /var/lib/containers/storage/overlay-containers/a7f3c9deb0656f8de1d107e7ddff2d3c3c279c11c1635f233a0bffb16051fb2c/userdata/shm: device or resource busy
Since podman is volume mounting in the graphroot, it will add a flag to
/run/.containerenv to tell podman inside of container whether to reset storage or not.
Since the inner podman is running inside of the container, no reason to assume this is a fresh reboot, so if "container" environment variable is set then skip
reset of storage.
Also added tests to make sure /run/.containerenv is runnig correctly.
Fixes: https://github.com/containers/podman/issues/9191
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Make sure to not set an empty $HOME for containers and let it default to
"/".
https://github.com/containers/crun/pull/599 is required to fully
address #9378.
Partially-Fixes: #9378
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Currently podman is always chowning the WORKDIR to root:root
This PR will return if the WORKDIR already exists.
Fixes: https://github.com/containers/podman/issues/9387
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
The old copy-up implementation was very unhappy with symlinks,
which could cause containers to fail to start for unclear reasons
when a directory we wanted to copy-up contained one. Rewrite to
use the Buildah Copier, which is more recent and should be both
safer and less likely to blow up over links.
At the same time, fix a deadlock in copy-up for volumes requiring
mounting - the Mountpoint() function tried to take the
already-acquired volume lock.
Fixes#6003
Signed-off-by: Matthew Heon <mheon@redhat.com>
Implement podman secret create, inspect, ls, rm
Implement podman run/create --secret
Secrets are blobs of data that are sensitive.
Currently, the only secret driver supported is filedriver, which means creating a secret stores it in base64 unencrypted in a file.
After creating a secret, a user can use the --secret flag to expose the secret inside the container at /run/secrets/[secretname]
This secret will not be commited to an image on a podman commit
Signed-off-by: Ashley Cui <acui@redhat.com>
When resolving the workdir of a container, we may need to create unless
the user set it explicitly on the command line. Otherwise, we just do a
presence check. Unfortunately, there was a missing return that lead us
to fall through into attempting to create and chown the workdir. That
caused a regression when running on a read-only root fs.
Fixes: #9230
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
A container's workdir can be specified via the CLI via `--workdir` and
via an image config with the CLI having precedence.
Since images have a tendency to specify workdirs without necessarily
shipping the paths with the root FS, make sure that Podman creates the
workdir. When specified via the CLI, do not create the path, but check
for its existence and return a human-friendly error.
NOTE: `crun` is performing a similar check that would yield exit code
127. With this change, however, Podman performs the check and yields
exit code 126. Since this is specific to `crun`, I do not consider it
to be a breaking change of Podman.
Fixes: #9040
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
This implements support for mounting and unmounting volumes
backed by volume plugins. Support for actually retrieving
plugins requires a pull request to land in containers.conf and
then that to be vendored, and as such is not yet ready. Given
this, this code is only compile tested. However, the code for
everything past retrieving the plugin has been written - there is
support for creating, removing, mounting, and unmounting volumes,
which should allow full functionality once the c/common PR is
merged.
A major change is the signature of the MountPoint function for
volumes, which now, by necessity, returns an error. Named volumes
managed by a plugin do not have a mountpoint we control; instead,
it is managed entirely by the plugin. As such, we need to cache
the path in the DB, and calls to retrieve it now need to access
the DB (and may fail as such).
Notably absent is support for SELinux relabelling and chowning
these volumes. Given that we don't manage the mountpoint for
these volumes, I am extremely reluctant to try and modify it - we
could easily break the plugin trying to chown or relabel it.
Also, we had no less than *5* separate implementations of
inspecting a volume floating around in pkg/infra/abi and
pkg/api/handlers/libpod. And none of them used volume.Inspect(),
the only correct way of inspecting volumes. Remove them all and
consolidate to using the correct way. Compat API is likely still
doing things the wrong way, but that is an issue for another day.
Fixes#4304
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
`staticcheck` is a golang code analysis tool. https://staticcheck.io/
This commit fixes a lot of problems found in our code. Common problems are:
- unnecessary use of fmt.Sprintf
- duplicated imports with different names
- unnecessary check that a key exists before a delete call
There are still a lot of reported problems in the test files but I have
not looked at those.
Signed-off-by: Paul Holzinger <paul.holzinger@web.de>
writing to the id map fails when an extent overlaps multiple mappings
in the parent user namespace:
$ cat /proc/self/uid_map
0 1000 1
1 100000 65536
$ unshare -U sleep 100 &
[1] 1029703
$ printf "0 0 100\n" | tee /proc/$!/uid_map
0 0 100
tee: /proc/1029703/uid_map: Operation not permitted
This limitation is particularly annoying when working with rootless
containers as each container runs in the rootless user namespace, so a
command like:
$ podman run --uidmap 0:0:2 --rm fedora echo hi
Error: writing file `/proc/664087/gid_map`: Operation not permitted: OCI permission denied
would fail since the specified mapping overlaps the first
mapping (where the user id is mapped to root) and the second extent
with the additional IDs available.
Detect such cases and automatically split the specified mapping with
the equivalent of:
$ podman run --uidmap 0:0:1 --uidmap 1:1:1 --rm fedora echo hi
hi
A fix has already been proposed for the kernel[1], but even if it
accepted it will take time until it is available in a released kernel,
so fix it also in pkg/rootless.
[1] https://lkml.kernel.org/lkml/20201203150252.1229077-1-gscrivan@redhat.com/
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
When migrating a container with associated volumes, the content of
these volumes should be made available on the destination machine.
This patch enables container checkpoint/restore with named volumes
by including the content of volumes in checkpoint file. On restore,
volumes associated with container are created and their content is
restored.
The --ignore-volumes option is introduced to disable this feature.
Example:
# podman container checkpoint --export checkpoint.tar.gz <container>
The content of all volumes associated with the container are included
in `checkpoint.tar.gz`
# podman container checkpoint --export checkpoint.tar.gz --ignore-volumes <container>
The content of volumes is not included in `checkpoint.tar.gz`. This is
useful, for example, when the checkpoint/restore is performed on the
same machine.
# podman container restore --import checkpoint.tar.gz
The associated volumes will be created and their content will be
restored. Podman will exit with an error if volumes with the same
name already exist on the system or the content of volumes is not
included in checkpoint.tar.gz
# podman container restore --ignore-volumes --import checkpoint.tar.gz
Volumes associated with container must already exist. Podman will not
create them or restore their content.
Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
Instead of individual values from ContainerCheckpointOptions,
provide the options object.
This is a preparation for the next patch where one more value
of the options object is required in exportCheckpoint().
Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
When adding the HOSTNAME environment variable, only do so if it
is not already present in the spec. If it is already present, it
was likely added by the user, and we should honor their requested
value.
Fixes#8886
Signed-off-by: Matthew Heon <mheon@redhat.com>
do not check whether the specified ID is valid in the user namespace.
crun handles this case[1], so the check in Podman prevents to get to
the OCI runtime at all.
$ podman run --user 10:0 --uidmap 0:0:1 --rm -ti fedora:33 sh -c 'id; cat /proc/self/uid_map'
uid=10(10) gid=0(root) groups=0(root),65534(nobody)
10 0 1
[1] https://github.com/containers/crun/pull/556
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
This adds a new command, 'podman network reload', to reload the
networks of existing containers, forcing recreation of firewall
rules after e.g. `firewall-cmd --reload` wipes them out.
Under the hood, this works by calling CNI to tear down the
existing network, then recreate it using identical settings. We
request that CNI preserve the old IP and MAC address in most
cases (where the container only had 1 IP/MAC), but there will be
some downtime inherent to the teardown/bring-up approach. The
architecture of CNI doesn't really make doing this without
downtime easy (or maybe even possible...).
At present, this only works for root Podman, and only locally.
I don't think there is much of a point to adding remote support
(this is very much a local debugging command), but I think adding
rootless support (to kill/recreate slirp4netns) could be
valuable.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
Signed-off-by: Paul Holzinger <paul.holzinger@web.de>
We have been asked to leak some information into the container
to indicate:
* The name and id of the container
* The version of podman used to launch the container
* The image name and ID the container is based on.
* Whether the container engine is running in rootless mode.
Fixes: https://github.com/containers/podman/issues/6192
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Our users are missing certain warning messages that would
make debugging issues with Podman easier.
For example if you do a podman build with a Containerfile
that contains the SHELL directive, the Derective is silently
ignored.
If you run with the log-level warn you get a warning message explainging
what happened.
$ podman build --no-cache -f /tmp/Containerfile1 /tmp/
STEP 1: FROM ubi8
STEP 2: SHELL ["/bin/bash", "-c"]
STEP 3: COMMIT
--> 7a207be102a
7a207be102aa8993eceb32802e6ceb9d2603ceed9dee0fee341df63e6300882e
$ podman --log-level=warn build --no-cache -f /tmp/Containerfile1 /tmp/
STEP 1: FROM ubi8
STEP 2: SHELL ["/bin/bash", "-c"]
STEP 3: COMMIT
WARN[0000] SHELL is not supported for OCI image format, [/bin/bash -c] will be ignored. Must use `docker` format
--> 7bd96fd25b9
7bd96fd25b9f755d8a045e31187e406cf889dcf3799357ec906e90767613e95f
These messages will no longer be lost, when we default to WARNing level.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
The buildah/pkg/secrts package was move to
containers/common/pkg/subscriptions.
Switch to using this by default.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
When looking up a container's cgroup path, parse /proc/[PID]/cgroup.
This will work across all cgroup managers and configurations and is
supported on cgroups v1 and v2.
Fixes: #8265
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
The --hostname and containername should always be added to containers.
Added some tests to make sure you can always ping the hostname and container
name from within the container.
Fixes: https://github.com/containers/podman/issues/8095
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Allow users to specify unbindable on volume command line
Switch internal mounts to rprivate to help prevent leaks.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Most of the builtin golang functions like os.Stat and
os.Open report errors including the file system object
path. We should not wrap these errors and put the file path
in a second time, causing stuttering of errors when they
get presented to the user.
This patch tries to cleanup a bunch of these errors.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Add a new "image" mount type to `--mount`. The source of the mount is
the name or ID of an image. The destination is the path inside the
container. Image mounts further support an optional `rw,readwrite`
parameter which if set to "true" will yield the mount writable inside
the container. Note that no changes are propagated to the image mount
on the host (which in any case is read only).
Mounts are overlay mounts. To support read-only overlay mounts, vendor
a non-release version of Buildah.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
If the resolv.conf file is empty we provide default dns servers.
If the file does not exists we error and don't create the
container. We should also provide the default entries in this
case. This is also what docker does.
Fixes#8089
Signed-off-by: Paul Holzinger <paul.holzinger@web.de>
This does not match Docker, which does not add hostname in this
case, but it seems harmless enough.
Fixes#8095
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
When a container uses --net=host the default hostname is set to
the host's hostname. However, we were not creating any entries
in `/etc/hosts` despite having a hostname, which is incorrect.
This hostname, for Docker compat, will always be the hostname of
the host system, not the container, and will be assigned to IP
127.0.1.1 (not the standard localhost address).
Also, when `--hostname` and `--net=host` are both passed, still
use the hostname from `--hostname`, not the host's hostname (we
still use the host's hostname by default in this case if the
`--hostname` flag is not passed).
Fixes#8054
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
We need to do a length check before we can access the
networkStatus slice by index to prevent a runtime panic.
Fixes#8026
Signed-off-by: Paul Holzinger <paul.holzinger@web.de>
Currently the HOME environment is set to /root if
the user does not override it.
Also walk the parent directories of users homedir
to see if it is volume mounted into the container,
if yes, then set it correctly.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
When we create a container, we assign a cgroup parent based on
the current cgroup manager in use. This parent is only usable
with the cgroup manager the container is created with, so if the
default cgroup manager is later changed or overridden, the
container will not be able to start.
To solve this, store the cgroup manager that created the
container in container configuration, so we can guarantee a
container with a systemd cgroup parent will always be started
with systemd cgroups.
Unfortunately, this is very difficult to test in CI, due to the
fact that we hard-code cgroup manager on all invocations of
Podman in CI.
Fixes#7830
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
We do not populate the hostname field with the IP Address
when running within a user namespace.
Fixes https://github.com/containers/podman/issues/7490
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
This commit is courtesy of
```
for f in $(git ls-files *.go | grep -v ^vendor/); do \
sed -i 's/\(errors\..*\)"Error /\1"error /' $f;
done
for f in $(git ls-files *.go | grep -v ^vendor/); do \
sed -i 's/\(errors\..*\)"Failed to /\1"failed to /' $f;
done
```
etc.
Self-reviewed using `git diff --word-diff`, found no issues.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
In case os.Open[File], os.Mkdir[All], ioutil.ReadFile and the like
fails, the error message already contains the file name and the
operation that fails, so there is no need to wrap the error with
something like "open %s failed".
While at it
- replace a few places with os.Open, ioutil.ReadAll with
ioutil.ReadFile.
- replace errors.Wrapf with errors.Wrap for cases where there
are no %-style arguments.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
check there are enough gids in the user namespace before adding
supplementary gids from /etc/group.
Follow-up for baede7cd27
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
There is a risk here, that if the GID does not exists
within the User Namespace the container will fail to start.
This is only likely to happen in HPC Envioronments, and I think
we should add a field to disable it for this environment,
Added a FIXME for this issue.
We currently have this problem with running a rootfull container within
a user namespace, it will fail if the GID is not available.
I looked at potentially checking the usernamespace that you are assigned
to, but I believe this will be very difficult to code up and to figure out.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
The kernel will not allow you to modify existing mount flags on a volume
when bind mounting it to another place. Since /sys/fs/cgroup/systemd is
mounted noexec on the host, it needs to be mounted with the same flags
in the rootless container.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
To ensure that the user running in the container ahs a valid
entry in /etc/passwd so lookup functions for the current user
will not error, Podman previously began adding entries to the
passwd file. We did not, however, add entries to the group file,
and this created problems - our passwd entries included the group
the user is in, but said group might not exist. The solution is
to mirror our logic for /etc/passwd modifications to also edit
/etc/group in the container.
Unfortunately, this is not a catch-all solution. Our logic here
is only advanced enough to *add* to the group file - so if the
group already exists but we add a user not a part of it, we will
not modify that existing entry, and things remain inconsistent.
We can look into adding this later if we absolutely need to, but
it would involve adding significant complexity to this already
massively complicated function.
While we're here, address an edge case where Podman could add a
user or group whose UID overlapped with an existing user or
group.
Also, let's make users able to log into users we added. Instead
of generating user entries with an 'x' in the password field,
indicating they have an entry in /etc/shadow, generate a '*'
indicating the user has no password but can be logged into by
other means e.g. ssh key, su.
Fixes#7503Fixes#7389Fixes#7499
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
Usage:
```
$ podman network create foo
$ podman run -d --name web --hostname web --network foo nginx:alpine
$ podman run --rm --network foo alpine wget -O - http://web.dns.podman
Connecting to web.dns.podman (10.88.4.6:80)
...
<h1>Welcome to nginx!</h1>
...
```
See contrib/rootless-cni-infra for the design.
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
The system defaults /run to "exec" mode, and we default --read-only
mounts on /run to "exec", so --systemd should follow suit.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
We want to modify /etc/passwd to add an entry for the user in
question, but at the same time we don't want to require the
container provide a /etc/passwd (a container with a single,
statically linked binary and nothing else is perfectly fine and
should be allowed, for example). We could create the passwd file
if it does not exist, but if the container doesn't provide one,
it's probably better not to make one at all. Gate changes to
/etc/passwd behind a stat() of the file in the container
returning cleanly.
Fixes#7515
Signed-off-by: Matthew Heon <mheon@redhat.com>
We had a customer incident where they ran out of space on /run.
If you don't specify size, it will be still limited to 50% or memory
available in the cgroup the container is running in. If the cgroup is
unlimited then the /run will be limited to 50% of the total memory
on the system.
Also /run is mounted on the host as exec, so no reason for us to mount
it noexec.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Aditya R
cdoern
Matthew Heon
Paul Holzinger
Giuseppe Scrivano
Valentin Rothberg
Daniel J Walsh
Evan Lezar
OpenShift Merge Robot
Radostin Stoyanov
Matthew Heon
Adrian Reber
Parham Alvani
flouthoc
Matej Vasek
Max Goltzsche
Ashley Cui
Baron Lenardson
Sebastian Jug
Paul Holzinger
なつき
TomSweeneyRedHat
Eduardo Vega
zhangguanzhang
unknown
Josh Soref
Andy Librian
Kir Kolyshkin
Akihiro Suda