Commit Graph

3018 Commits

Author SHA1 Message Date
OpenShift Merge Robot b203e6d0f0
Merge pull request #12365 from mtrmac/random
Don't use a global RNG, and avoid conflicts, when generating NodePorts
2021-12-02 20:00:48 +01:00
OpenShift Merge Robot b41026aec7
Merge pull request #12469 from Luap99/ns-teardown-flake
Fix possible rootless netns cleanup race
2021-12-02 14:40:48 +01:00
OpenShift Merge Robot ecc663097a
Merge pull request #12464 from giuseppe/fix-race-reading-cgroup-file
container, cgroup: detect pid termination
2021-12-01 20:59:10 +01:00
Paul Holzinger 3ff47748de
Fix possible rootless netns cleanup race
rootlessNetNS.Cleanup() has an issue with how it detects if cleanup
is needed, reading the container state is not good ebough because
containers are first stopped and than cleanup will be called. So at one
time two containers could wait for cleanup but the second one will fail
because the first one triggered already the cleanup thus making rootless
netns unavailable for the second container resulting in an teardown
error. Instead of checking the container state we need to check the
netns state.

Secondly, podman unshare --rootless-netns should not do the cleanup.
This causes more issues than it is worth fixing. Users also might want
to use this to setup the namespace in a special way. If unshare also
cleans this up right away we cannot do this.

[NO NEW TESTS NEEDED]

Fixes #12459

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-12-01 19:19:44 +01:00
Giuseppe Scrivano 0afaf78378
container, cgroup: detect pid termination
If the /proc/$PID/cgroup file doesn't exist, then it is likely the
container was terminated in the meanwhile so report ErrCtrStopped that
is already handled instead of ENOENT.

commit a66f40b4df introduced the regression.

Closes: https://github.com/containers/podman/issues/12457

[NO NEW TESTS NEEDED] it solves a race in the CI that is difficult to reproduce.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-12-01 13:42:59 +01:00
Valentin Rothberg e2b344728c top: parse ps(1) args correctly
The arguments of ps(1) should be shlexed.

Fixes: #12452
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-12-01 11:42:37 +01:00
Miloslav Trmač f415b30552 Ensure the generated NodePort values are unique
... at least within a single service.

[NO NEW TESTS NEEDED]
because testing RNGs is problematic. (We _could_
probably inject a mock RNG implementation that always
returns the same value, or something like that.)

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2021-11-30 22:13:52 +01:00
Miloslav Trmač a948ecbb9c Allow containerPortsToServicePorts to fail
Add an error return to it and affected callers.

Should not affect behavior, the function can't currently fail.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2021-11-30 22:13:52 +01:00
Miloslav Trmač ec686d68dd Don't use the global math/rand RNG for service ports
Use a private RNG with the desired seed, don't interfere
with the other uses.

Introducing the servicePortState type is rather overkill
for the single member, but we'll add another one immediately.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2021-11-30 22:13:52 +01:00
Miloslav Trmač d48c00418a Move a comment to the relevant place
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2021-11-30 22:13:52 +01:00
OpenShift Merge Robot 7324d94648
Merge pull request #12406 from Luap99/xdg-checkpoint
checkpoint do not modify XDG_RUNTIME_DIR
2021-11-25 12:02:10 +01:00
OpenShift Merge Robot 12f73d5f88
Merge pull request #12403 from giuseppe/improve-cgroup-detection
libpod: improve heuristic to detect cgroup
2021-11-25 11:59:09 +01:00
Paul Holzinger bc3c5be2fb
checkpoint do not modify XDG_RUNTIME_DIR
We should not modify the XDG_RUNTIME_DIR env value during runtime of
libpod, this can cause hard to find bugs. Only set it for the OCI
runtime, this matches the other commands such as start, stop, kill...

[NO NEW TESTS NEEDED]

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-24 15:02:11 +01:00
Giuseppe Scrivano e648122b29
libpod: improve heuristic to detect cgroup
improve the heuristic to detect the scope that was created for the container.
This is necessary with systemd running as PID 1, since it moves itself
to a different sub-cgroup, thus stats would not account for other
processes in the same container.

Closes: https://github.com/containers/podman/issues/12400

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-11-24 14:50:12 +01:00
Giuseppe Scrivano a66f40b4df
libpod, inspect: export cgroup path
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-11-24 14:50:12 +01:00
Giuseppe Scrivano b25b330306
stats: get the memory limit from the spec
OCI runtimes may set the memory limits in different ways, e.g., crun
creates a sub-cgroup where the limits are applied, while runc applies
them directly on the created cgroup.  Since there is standardization
on the cgroup path to use, just use the limit specified in the spec
file.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-11-24 14:50:12 +01:00
Giuseppe Scrivano c7ebaeee0e
libpod: leave thread locked on errors
if the SELinux label could not be restored correctly, leave the OS
thread locked so that it is terminated once it returns to the threads
pool.

[NO NEW TESTS NEEDED] the failure is hard to reproduce

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-11-24 11:08:23 +01:00
Daniel J Walsh df6aa67302
Unset SocketLabel after system finishes checkpointing
This should fix the SELinux issue we are seeing with talking to
/run/systemd/private.

Fixes: https://github.com/containers/podman/issues/12362

Also unset the XDG_RUNTIME_DIR if set, since we don't know when running
as a service if this will cause issue.s

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-11-23 16:31:54 -05:00
Matthew Heon 5cf2683bfe Warn on failing to update container status
failed to send a signal to the container's PID1, but ignored the
results of that update. That's generally bad practice, since even
if we can't directly take action on an error, we should still
make an effort to report it for debugging purposes. I used Infof
instead of something more serious to avoid duplicate reporting to
the user if something has gone seriously wrong.

[NO NEW TESTS NEEDED] this is just adding additional error reporting.

Signed-off-by: Matthew Heon <mheon@redhat.com>
2021-11-23 09:50:12 -05:00
Aditya Rajan a4e4b8d926
oci: ack crun output when container is not there
`crun status ctrid` outputs `No such file or directory` when container
is not there so podman much ack it.

[NO NEW TESTS NEEDED]

Signed-off-by: Aditya Rajan <arajan@redhat.com>
2021-11-23 16:41:28 +05:30
Aditya Rajan 08558b27ff
oci: exit gracefully if container is already dead
While trying to kill a container with a `signal` we cant do anything if
container is already dead so `exit` gracefully instead of trying to
delete container again. Get container status from runtime.

[ NO NEW TESTS NEEDED ]

Signed-off-by: Aditya Rajan <arajan@redhat.com>
2021-11-23 16:40:53 +05:30
Daniel J Walsh 992fafa9ab
Merge pull request #12386 from baude/playkuberename
Rename pod on generate of container
2021-11-22 16:19:58 -05:00
Brent Baude 9c8fb5cc0c Rename pod on generate of container
When generating kube of a container, the podname and container name in
the yaml are identical.  This offends rules in podman where pods and
containers cannot have the same name.  We now append _pod to the
podname to avoid that collision.

Signed-off-by: Brent Baude <bbaude@redhat.com>
2021-11-22 11:50:08 -06:00
Paul Holzinger 044edbb9c9
Fix netavark error handling and teardown issue
The return error was not returned by podman , instead a different error
was created. Also make sure to free assigned ips on an error to not leak
them.

Lastly podman container cleanup uses the default network backend instead
of the provided one, we need to add `--network-backend` to the exit
command.

[NO NEW TESTS NEEDED]

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-22 17:49:55 +01:00
Paul Holzinger 8198e96f31
Force iptables driver for netavark tests
Firewalld cannot be used because it can connect to the dbus api but
talks to firewalld in the host namespace. This will affact your host
badly and also causes tests to fail.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-19 18:23:26 +01:00
Paul Holzinger 501643c8bd
Make sure netavark output is logged to the syslog
Create a custom writer which logs the netavark output to logrus. This
will log to the syslog when it is enabled.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-19 18:15:53 +01:00
OpenShift Merge Robot 5432bb95f1
Merge pull request #12174 from fgimenez/fix-docker-networksettings-type-discrepancy
Introduces Address type to be used in secondary IPv4 and IPv6 inspect data structure
2021-11-19 13:57:13 +01:00
OpenShift Merge Robot 319d3fba6d
Merge pull request #12354 from Luap99/exit-command
Do not store the exit command in container config
2021-11-18 23:51:12 +01:00
OpenShift Merge Robot 348aafeb1b
Merge pull request #12348 from Luap99/rootless-netns
rootless netns, one netns per libpod tmp dir
2021-11-18 21:59:13 +01:00
Paul Holzinger 0dae50f1d3
Do not store the exit command in container config
There is a problem with creating and storing the exit command when the
container was created. It only contains the options the container was
created with but NOT the options the container is started with. One
example would be a CNI network config. If I start a container once, then
change the cni config dir with `--cni-config-dir` ans start it a second
time it will start successfully. However the exit command still contains
the wrong `--cni-config-dir` because it was not updated.

To fix this we do not want to store the exit command at all. Instead we
create it every time the conmon process for the container is startet.
This guarantees us that the container cleanup process is startet with
the correct settings.

[NO NEW TESTS NEEDED]

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-18 20:28:03 +01:00
Radostin Stoyanov 6d23ea60d2
Add --file-locks checkpoint/restore option
CRIU supports checkpoint/restore of file locks. This feature is
required to checkpoint/restore containers running applications
such as MySQL.

Signed-off-by: Radostin Stoyanov <radostin@redhat.com>
2021-11-18 19:23:25 +00:00
Paul Holzinger 62d6b6bf74
rootless netns, one netns per libpod tmp dir
The netns cleanup code is checking if there are running containers, this
can fail if you run several libpod instances with diffrent root/runroot.
To fix it we use one netns for each libpod instances. To prevent name
conflicts we use a hash from the static dir as part of the name.

Previously this worked because we would use the CNI files to check if
the netns was still in use. but this is no longer possible with netavark.

[NO NEW TESTS NEEDED]

Fixes #12306

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-18 17:34:06 +01:00
Federico Gimenez 2e5d3e8fb3 Introduce Address type to be used in secondary IPv4 and IPv6 inspect data
structure.

Resolves a discrepancy between the types used in inspect for docker and podman.
This causes a panic when using the docker client against podman when the
secondary IP fields in the `NetworkSettings` inspect field are populated.

Fixes containers#12165

Signed-off-by: Federico Gimenez <fgimenez@redhat.com>
2021-11-18 17:04:49 +01:00
Paul Holzinger 97c6403a1b
rename libpod nettypes fields
Some field names are confusing. Change them so that they make more sense
to the reader.
Since these fields are only in the main branch we can safely rename them
without worrying about backwards compatibility.
Note we have to change the field names in netavark too.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-16 19:19:04 +01:00
OpenShift Merge Robot f031bd23c6
Merge pull request #12100 from rhatdan/env
Add option --unsetenv to remove default environment variables
2021-11-16 16:27:34 +01:00
OpenShift Merge Robot 8430ffc72e
Merge pull request #12283 from Luap99/machine-ports
podman machine improve port forwarding
2021-11-16 14:53:40 +01:00
OpenShift Merge Robot be681ab518
Merge pull request #12294 from flouthoc/secret-mount-target
secret: honor custom `target=` for secrets with `type=mount` for ctr.
2021-11-16 01:45:27 +01:00
OpenShift Merge Robot 45d28c2219
Merge pull request #12285 from nalind/journal-follow-not-early
journald logs: keep reading until the journal's end
2021-11-15 22:09:29 +01:00
Daniel J Walsh 44d1618dd7
Add --unsetenv & --unsetenv-all to remove def environment variables
Podman adds a few environment variables by default, and
currently there is no way to get rid of them from your container.
This option will allow  you to specify which defaults you don't
want.

--unsetenv-all will remove all default environment variables.

Default environment variables can come from podman builtin,
containers.conf or from the container image.

Fixes: https://github.com/containers/podman/issues/11836

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-11-15 15:10:12 -05:00
OpenShift Merge Robot 230f0b622e
Merge pull request #12130 from rhatdan/journal
Error logs --follow if events-backend != journald, event-logger=journald
2021-11-15 20:55:28 +01:00
Nalin Dahyabhai 63ef7135d9 journald logs: keep reading until the journal's end
When reading logs from the journal, keep going after the container
exits, in case it gets restarted.

Events logged to the journal via the normal paths don't include
CONTAINER_ID_FULL, so don't bother adding it to the "history" event we
use to force at least one entry for the container to show up in the log.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2021-11-15 13:38:36 -05:00
Aditya Rajan 014cc4b9d9
secret: honor custom target for secrets with run
Honor custom `target` if specified while running or creating containers
with secret `type=mount`.

Example:
`podman run -it --secret token,type=mount,target=TOKEN ubi8/ubi:latest
bash`

Signed-off-by: Aditya Rajan <arajan@redhat.com>
2021-11-15 23:19:27 +05:30
Paul Holzinger 295d87bb0b
podman machine improve port forwarding
This commits adds port forwarding logic directly into podman. The
podman-machine cni plugin is no longer needed.

The following new features are supported:
 - works with cni, netavark and slirp4netns
 - ports can use the hostIP to bind instead of hard coding 0.0.0.0
 - gvproxy no longer listens on 0.0.0.0:7777 (requires a new gvproxy
   version)
 - support the udp protocol

With this we no longer need podman-machine-cni and should remove it from
the packaging. There is also a change to make sure we are backwards
compatible with old config which include this plugin.

Fixes #11528
Fixes #11728

[NO NEW TESTS NEEDED] We have no podman machine test at the moment.
Please test this manually on your system.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-15 15:20:47 +01:00
Adrian Reber 80e56fa12b
Added optional container restore statistics
This adds the parameter '--print-stats' to 'podman container restore'.
With '--print-stats' Podman will measure how long Podman itself, the OCI
runtime and CRIU requires to restore a checkpoint and print out these
information. CRIU already creates process restore statistics which are
just read in addition to the added measurements. In contrast to just
printing out the ID of the restored container, Podman will now print
out JSON:

 # podman container restore --latest --print-stats
 {
     "podman_restore_duration": 305871,
     "container_statistics": [
         {
             "Id": "47b02e1d474b5d5fe917825e91ac653efa757c91e5a81a368d771a78f6b5ed20",
             "runtime_restore_duration": 140614,
             "criu_statistics": {
                 "forking_time": 5,
                 "restore_time": 67672,
                 "pages_restored": 14
             }
         }
     ]
 }

The output contains 'podman_restore_duration' which contains the
number of microseconds Podman required to restore the checkpoint. The
output also includes 'runtime_restore_duration' which is the time
the runtime needed to restore that specific container. Each container
also includes 'criu_statistics' which displays the timing information
collected by CRIU.

Signed-off-by: Adrian Reber <areber@redhat.com>
2021-11-15 11:50:25 +00:00
Adrian Reber 6202e8102b
Added optional container checkpointing statistics
This adds the parameter '--print-stats' to 'podman container checkpoint'.
With '--print-stats' Podman will measure how long Podman itself, the OCI
runtime and CRIU requires to create a checkpoint and print out these
information. CRIU already creates checkpointing statistics which are
just read in addition to the added measurements. In contrast to just
printing out the ID of the checkpointed container, Podman will now print
out JSON:

 # podman container checkpoint --latest --print-stats
 {
     "podman_checkpoint_duration": 360749,
     "container_statistics": [
         {
             "Id": "25244244bf2efbef30fb6857ddea8cb2e5489f07eb6659e20dda117f0c466808",
             "runtime_checkpoint_duration": 177222,
             "criu_statistics": {
                 "freezing_time": 100657,
                 "frozen_time": 60700,
                 "memdump_time": 8162,
                 "memwrite_time": 4224,
                 "pages_scanned": 20561,
                 "pages_written": 2129
             }
         }
     ]
 }

The output contains 'podman_checkpoint_duration' which contains the
number of microseconds Podman required to create the checkpoint. The
output also includes 'runtime_checkpoint_duration' which is the time
the runtime needed to checkpoint that specific container. Each container
also includes 'criu_statistics' which displays the timing information
collected by CRIU.

Signed-off-by: Adrian Reber <areber@redhat.com>
2021-11-15 11:50:24 +00:00
Daniel J Walsh 062c887718
Error logs --follow if events-backend != journald, event-logger=journald
Fixes: https://github.com/containers/podman/issues/11255

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-11-13 07:11:09 -05:00
OpenShift Merge Robot d6d89fa79f
Merge pull request #12267 from giuseppe/safely-create-etc-mtab
libpod: create /etc/mtab safely
2021-11-11 20:47:42 +01:00
Paul Holzinger 3af19917a1
Add failing run test for netavark
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-11 17:50:10 +01:00
Paul Holzinger fe90a45e0d
Add flag to overwrite network backend from config
To make testing easier we can overwrite the network backend with the
global `--network-backend` option.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-11 17:30:27 +01:00
Giuseppe Scrivano 9f4d63f91b
libpod: create /etc/mtab safely
make sure the /etc/mtab symlink is created inside the rootfs when /etc
is a symlink.

Closes: https://github.com/containers/podman/issues/12189

[NO NEW TESTS NEEDED] there is already a test case

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-11-11 17:00:53 +01:00