There are certain messages logged by OCI runtimes when killing a
container that has already stopped that we really do not care
about when stopping a container. Due to our architecture, there
are inherent races around stopping containers, and so we cannot
guarantee that *we* are the people to kill it - but that doesn't
matter because Podman only cares that the container has stopped,
not who delivered the fatal signal.
Unfortunately, the OCI runtimes don't understand this, and log
various warning messages when the `kill` command is invoked on a
container that was already dead. These cause our tests to fail,
as we now check for clean STDERR when running Podman. To work
around this, capture STDERR for the OCI runtime in a buffer only
for stopping containers, and go through and discard any of the
warnings we identified as spurious.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
We use shared-memory pthread mutexes to handle mutual exclusion
in Libpod. It turns out that these have configurable options for
how to handle a recursive lock (IE, a thread trying to lock a
lock that the same thread had previously locked). The mutex can
either deadlock, or allow the duplicate lock without deadlocking.
Default behavior is, helpfully, unspecified, so if not explicitly
set there is no clear indication of which of these behaviors will
be seen. Unfortunately, today is the first I learned of this, so
our initial implementation did *not* explicitly set our preferred
behavior.
This turns out to be a major problem with a language like Golang,
where multiple goroutines can (and often do) use the same OS
thread. So we can have two goroutines trying to stop the same
container, and if the no-deadlock mutex behavior is in use, both
threads will successfully acquire the lock because the C library,
not knowing about Go's lightweight threads, sees the same PID
trying to lock a mutex twice, and allows it without question.
It appears that, at least on Fedora/RHEL/Debian libc, the default
(unspecified) behavior of the locks is the non-deadlocking
version - so, effectively, our locks have been of questionable
utility within the same Podman process for the last four years.
This is somewhat concerning.
What's even more concerning is that the Golang-native sync.Mutex
that was also in use did nothing to prevent the duplicate locking
(I don't know if I like the implications of this).
Anyways, this resolves the major issue of our locks not working
correctly by explicitly setting the correct pthread mutex
behavior.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
If the first container to get the pod lock is the infra container
it's going to want to remove the entire pod, which will also
remove every other container in the pod. Subsequent containers
will get the pod lock and try to access the pod, only to realize
it no longer exists - and that, actually, the container being
removed also no longer exists.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
This was causing some CI flakes. I'm pretty sure that the pods
being removed already isn't a bug, but just the result of another
container in the pod removing it first - so no reason not to
ignore the errors.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
This fixes a lint issue, but I'm keeping it in its own commit so
it can be reverted independently if necessary; I don't know what
side effects this may have. I don't *think* there are any
issues, but I'm not sure why it wasn't a pointer in the first
place, so there may have been a reason.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
We had something like 6 different boolean options (removing a
container turns out to be rather complicated, because there are a
million-odd things that want to do it), and the function
signature was getting unreasonably large. Change to a struct to
clean things up.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
This reverts commit 9bd833bcfd.
With the fix for `podman rm -fa` merged, we no longer require
this patch.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
The infra container would try to remove the pod, despite the pod
already being in the process of being removed - oops. Add a check
to ensure we don't try and remove the pod when called by the
`podman pod rm` command.
Also, wire up noLockPod - it wasn't previously wired in, which is
concerning, and could be related?
Finally, make a few minor fixes to un-break lint.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
This probably should have been in the API since the beginning,
but it's not too late to start now.
The extra information is returned (both via the REST API, and to
the CLI handler for `podman rm`) but is not yet printed - it
feels like adding it to the output could be a breaking change?
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
This allows for accurate reporting of dependency removal, but the
work is still incomplete: pods can be removed, but do not report
the containers they removed as part of said removal. Will add
this in a subsequent commit.
Major note: I made ignoring no-such-container errors automatic
once it has been determined that a container did exist in the
first place. I can't think of any case where this would not be a
TOCTOU - IE, no reason not to ignore them. The `--ignore` option
to `podman rm` should still retain meaning as it will ignore
errors from containers that didn't exist in the first place.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
This is the initial stage of implementation. The current API
functions but does not report the additional containers and pods
removed. This is necessary to properly display results to the
user after `podman rm --all`.
The existing remove-dependencies code has been removed in favor
of this more native solution.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
We _usually_ have only one image in store, $IMAGE, but it's
perfectly fine to also have $SYSTEMD_IMAGE also. Fix a few
tests so they can handle that condition.
And, cleanup:
- remove a no-longer-useful test ("podman load NEWNAME",
functionality that was removed 2+ years ago in #8877)
- reorder some tests in the image-mount test, to make
them safer and easier to understand
- use no-such-image, not no-such-container, in image-mount test.
Computer don't care, but this human felt confused for a sec.
Signed-off-by: Ed Santiago <santiago@redhat.com>
The current way of bind mounting the host timezone file has problems.
Because /etc/localtime in the image may exist and is a symlink under
/usr/share/zoneinfo it will overwrite the targetfile. That confuses
timezone parses especially java where this approach does not work at
all. So we end up with an link which does not reflect the actual truth.
The better way is to just change the symlink in the image like it is
done on the host. However because not all images ship tzdata we cannot
rely on that either. So now we do both, when tzdata is installed then
use the symlink and if not we keep the current way of copying the host
timezone file in the container to /etc/localtime.
Also note that we need to rebuild the systemd image to include tzdata in
order to test this as our images do not contain the tzdata by default.
Fixes https://bugzilla.redhat.com/show_bug.cgi?id=2149876
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Systemd doesn't support `never` and logs a warning, systemd uses no as
default so we do not have to specify it at all.
Check systemd.service(5) for the systemd docs.
Fixes#18743
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Reduce sleep-loop time in logs test, from 1s to 0.1s,
to make 'podman stop' take effect more quickly. With 1s,
and testing with 1s resolution, we get flakes.
Fixes: #17826
Signed-off-by: Ed Santiago <santiago@redhat.com>
The new exit-code propagation test is racy: 'podman wait' can
fail if the service container has already been cleaned up by
systemd.
Solution: run the inspect and wait tests opportunistically, i.e.,
only if those commands succeed. If they fail, confirm that they
fail with ENOSUCHCONTAINER. This may silently lose us some
coverage ... but none of it is important. The important
test, systemctl final status, remains.
Also, as drive-bys:
- add a FIXME comment documenting another race condition
that I'm not bothering to fix right now
- give distinct names to unit files, for readability in
test failures
Fixes: #18732
Signed-off-by: Ed Santiago <santiago@redhat.com>
"image rm concurrent" test is still failing, even after #18664:
Error: no contents in "/tmp/podman_test967723851/Dockerfile"
Probable cause: the images are built in parallel, and p.BuildImage()
writes one single Dockerfile. (This almost certainly renders the
test less effective than intended, since the generated images
might end up being identical).
Solution: write and use a uniquely-named Dockerfile
Signed-off-by: Ed Santiago <santiago@redhat.com>
When we do path completion in images a user could try to complete a
simple relative path, e.g. podman run $IMAGE e... should complete to etc
if this path exists in the image. Right now we panic in this case as the
current check didn't account for an empty string in simplePathJoinUnix().
In such a case return the path directly because we can not alter what
the user typed on the cli and must return a path without slash as well
in order for the shell to suggest the completion.
Fixes https://bugzilla.redhat.com/show_bug.cgi?id=2209809
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Matthew Heon
OpenShift Merge Robot
Ed Santiago
Paul Holzinger
renovate[bot]
Jan Hendrik Farr
Boaz Shuster
Tom Deseyn