Forcing users to set --rm when setting --rmi is just bad UI.
If I want the image to be removed, it implies that I want the
container removed that I am creating.
Fixes: https://github.com/containers/podman/issues/15640
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
The original SELinux support in Docker and Podman does not follow the
default SELinux rules for how label transitions are supposed to be
handled. Containers always switch their user and role to
system_u:system_r, rather then maintain the collers user and role.
For example
unconfined_u:unconfined_r:container_t:s0:c1,c2
Advanced SELinux administrators want to confine users but still allow
them to create containers from their role, but not allow them to launch
a privileged container like spc_t.
This means if a user running as
container_user_u:container_user_r:container_user_t:s0
Ran a container they would get
container_user_u:container_user_r:container_t:s0:c1,c2
If they run a privileged container they would run it with:
container_user_u:container_user_r:container_user_t:s0
If they want to force the label they would get an error
podman run --security-opt label=type:spc_t ...
Should fail. Because the container_user_r can not run with the spc_t.
SELinux rules would also prevent the user from forcing system_u user and
the sytem_r role.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Signed-off-by: Chris Evich <cevich@redhat.com>
We do not use any special netns path for the netns=none case, however
callers that inspect that may still wish to join the netns path directly
without extra work to figure out /proc/$pid/ns/net.
Fixes#16716
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
And lo, a miracle occurred. Containerized checkpoint tests are
no longer hanging. Reenable them.
(Followup miracle: tests are still passing, after a year of not
running!)
Closes: #15015
Signed-off-by: Ed Santiago <santiago@redhat.com>
The intention of --read-only-tmpfs=fals when in --read-only mode was to
not allow any processes inside of the container to write content
anywhere, unless the caller also specified a volume or a tmpfs. Having
/dev and /dev/shm writable breaks this assumption.
Fixes: https://github.com/containers/podman/issues/12937
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
In certain cases REGISTRY_AUTH_FILE is set but the auth file
does not exists yet, do not throw error unless user specified
a file directly using --authfile.
Fixes: https://github.com/containers/podman/issues/18405
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
HPC Community asked for this support specifically for using GPUs
within containers. Nvidia requires the correct shared library to
to be present in the directory that matches the device mounted
into the container. These libraries have random suffixes based
on versions of the installed libraries on the host.
podman run --mount type=glob:src=/usr/lib64/nvidia\*:ro=true. This helps
quadlets be more portable for this use case.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Use "--wait" flag in "systemd start" for a one-shot container.
Should fix a CI failure I've been seeing sporadically, in which
the --==VALUE==-- string is not seen in journal.
Signed-off-by: Ed Santiago <santiago@redhat.com>
If a user puts a quadlet file in his homedirectory with
the same name as one in /etc/containers/systemd/user or
/etc/containers/systemd/user/$UID, then only use the one in
homedir and ignore the others.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
BATS 1.8.0 introduces tags: metadata that can be applied to
a single test or one entire file, then used for filtering
in a test run.
Issue #19299 introduces the possibility of using OpenQA
for podman reverse dependency testing: continuous CI on
all packages that can affect podman, so we don't go two
months with no bodhi builds then get caught by surprise
when systemd or kernel or crun change in ways that break us.
This PR introduces one bats tag, "distro-integration".
The intention is for OpenQA (or other) tests to install
the podman-tests package and run:
bats --filter-tags distro-integration /usr/share/podman/test/system
Goal is to keep the test list short and sweet: we do not
need to test command-line option parsing. We *DO* need to
test interactions with systemd, kernel, nethack, and other
critical components.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Quick followup to #19348:
- refactor into table form, for legibility
- add tests for 'podman kube play' and 'podman run'
- slightly cleaner message on failure
Signed-off-by: Ed Santiago <santiago@redhat.com>
Add a new "healthy" sdnotify policy that instructs Podman to send the
READY message once the container has turned healthy.
Fixes: #6160
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
**podman compose** is a thin wrapper around an external compose provider
such as docker-compose or podman-compose. This means that `podman
compose` is executing another tool that implements the compose
functionality but sets up the environment in a way to let the compose
provider communicate transparently with the local Podman socket. The
specified options as well the command and argument are passed directly
to the compose provider.
The default compose providers are `docker-compose` and `podman-compose`.
If installed, `docker-compose` takes precedence since it is the original
implementation of the Compose specification and is widely used on the
supported platforms (i.e., Linux, Mac OS, Windows).
If you want to change the default behavior or have a custom installation
path for your provider of choice, please change the `compose_provider`
field in `containers.conf(5)`. You may also set the
`PODMAN_COMPOSE_PROVIDER` environment variable.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
I am working on running android auto in a quadlet.
[Container]
AddDevice=/dev/dri/renderD128
AddDevice=/dev/kvm
DropCapability=all
Environment=PULSE_SERVER=$XDG_RUNTIME_DIR/pulse/native
Environment=WAYLAND_DISPLAY=wayland-0
Environment=XDG_RUNTIME_DIR
Image=quay.io/slopezpa/qemu-aaos
ContainerName=Android
PodmanArgs=--shm-size=5g
SecurityLabelDisable=true
Volume=$XDG_RUNTIME_DIR:$XDG_RUNTIME_DIR
And I need to be able to set the --shm-size option.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
`podman system service` + TCP is not a configuration we should be
recommending. There was already language about this in the
manpages, but it was not sufficient in explaining how bad of an
idea this is. Expand the manpage warnings, add a dedicated
heading so people notice, and add a warning every time the
service starts with a TCP URL that directs people to the manpage
to see that explanation.
Signed-off-by: Matt Heon <mheon@redhat.com>
Some quadlet tests are failing on RHEL8: test code was
using journalctl to check output from containers. This
fails on RHEL8, where default log driver is k8s-file.
Solution: use 'podman logs' instead. To do so, we need to
keep the containers alive (otherwise, quadlet seems to
delete them on exit). Do so by running 'top -b' (batch);
the currently-used 'top' was failing because not-a-tty.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Add key for Quadlet to set WorkingDirectory to the directory of the YAML or Unit file
Add Doc
Add E2E tests
Add System test
Signed-off-by: Ygal Blum <ygal.blum@gmail.com>
quadlet volume-path system test was making invalid assumptions
about $TMPDIR, causing test to fail when TMPDIR=/var/tmp or /dev/shm
Much more complicated than it should be, because we need to
find out the systemd value of %T.
Minor cleanup too.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Adds support for `since` as a valid filter option for `podman volume ls`
and `podman volume prune`.
Implements: #19228
Initially suggested from: #19119
Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
Fixes a bug where `podman volume ls` with multiple `label` filters would
return volumes that matched *any* of the filters, not *all* of them.
Adapts generating volume filter functions to be more in
line with how it is done for containers and pods.
Fixes: #19219
Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
Adds any required "wiring" to ensure the reserved annotations are supported by
`podman kube play`.
Addtionally fixes a bug where, when inspected, containers created using
the `--publish-all` flag had a field `.HostConfig.PublishAllPorts` whose
value was only evaluated as `false`.
Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
- the "podman {run,exec} /etc" test: runc now spits out
"is a directory" instead of "permission denied". And,
on exec, exits 255 instead of 126. Deal with it.
- workaround for https://github.com/containers/skopeo/issues/823
(skopeo XDG bug): always make sure XDG is defined for skopeo
Signed-off-by: Ed Santiago <santiago@redhat.com>
Adds an `--podman-only` flag to `podman generate kube` to allow for
reserved annotations to be included in the generated YAML file.
Associated with: #19102
Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
To silence my find-obsolete-skips script, remove the '#'
from the following issues in skip messages:
#11784#15013#15025#17433#17436#17456
Also update the messages to reflect the fact that the issues
will never be fixed.
Also remove ubuntu skips: we no longer test ubuntu.
Also remove one buildah skip that is no longer applicable:
Fixes: #17520
Signed-off-by: Ed Santiago <santiago@redhat.com>
When I reworked pod removal to provide more detailed errors
(including per-container errors, not just a single multierror
with all errors squashed), I made it part of the struct returned
by the REST API and assumed that would be enough to get errors
through to clients. Unfortunately, in case of an overarching
error removing the pod (as any error with any container would
cause), we don't send the response struct that would include the
container errors - we just send a standardized REST error. We
could work around this with custom, potentially backwards
incompatible error handling for the REST pod delete endpoint, or
we could just do what was done before, and package up all the
errors in a multierror to send to the other side. Of those
options, the multierror seems far simpler.
Fixes#19159
Signed-off-by: Matt Heon <mheon@redhat.com>
The change to use the custom dns server in aardvark-dns caused a
regression here because macvlan networks never returned the nameservers
in netavark and it also does not make sense to do so.
Instead check here if we got any network nameservers, if not we then use
the ones from the config if set otherwise fallback to host servers.
Fixes#19169
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
We use the name as alias but using the hostname makes also sense and
this is what docker does. We have to keep the short id as well for
docker compat.
While adding some tests I removed some duplicated tests that were
executed twice for nv for no reason.
Fixes#17370
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
When fetching the user name, you need to use User.Username
instead of User.Name, as with other tests.
Signed-off-by: Toshiki Sonoda <sonoda.toshiki@fujitsu.com>
Adds a `--no-trunc` flag to `podman kube generate` preventing the
annotations from being trimmed at 63 characters. However, due to
the fact the annotations will not be trimmed, any annotation that is
longer than 63 characters means this YAML will no longer be Kubernetes
compatible. However, these YAML files can still be used with `podman
kube play` due to the addition of the new flag below.
Adds a `--no-trunc` flag to `podman kube play` supporting YAML files with
annotations that were not truncated to the Kubernetes maximum length of
63 characters.
Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
This commit extends `Volume` and `Network` unit definitions with two
additional parameters, `VolumeName` and `NetworkName`, which will,
respectively, set a user-defined name for the corresponding volume and
network. This is similar to how the `ContainerName` directive currently
works, and should allow for smoother transitions to Quadlet-managed
resources.
Closes: #19003
Signed-off-by: Alex Palaistras <alex@deuill.org>
...from the test name. Eliminates scary duplication.
Followup to #19053: instead of cross-checking pasta test args
against test name, eliminate the args entirely. Determine
them all from the @test name itself.
Example:
"TCP translated port range forwarding, IPv4, loopback"
| | | | | | +-- iftype=loopback
| | | | | +-------- ip_ver=4
| | | | +-------------------- bytes=1
| | | +-------------------------- range=3
| | +------------------------------- (ignored)
| +------------------------------------------ delta=1
+--------------------------------------------- proto=tcp
Signed-off-by: Ed Santiago <santiago@redhat.com>
Do not use podman info/version as they are expensive and clutter the log
for no reason. Just checking if we can connect to the socket should be
good enough and much faster.
Fix the non existing error checking, so that we actually see an useful
error when this does not work.
Also change the interval, why wait 2s for a retry lets take 100ms steps
instead.
Fixes#19010
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Previous tests have worked by pure chance since the client and server
ran on the same host; the server picked up the credentials created by
the client login.
Extend the gating tests and add a new integration test which is further
capable of exercising the remote code.
Note that fixing authentication support requires adding a new
`--authfile` CLi flag to `manifest inspect`. This will at least allow
for passing an authfile to be bindings. Username and password are not
yet supported.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
This ended up more complicated then expected. Lets start first with the
problem to show why I am doing this:
Currently we simply execute ps(1) in the container. This has some
drawbacks. First, obviously you need to have ps(1) in the container
image. That is no always the case especially in small images. Second,
even if you do it will often be only busybox's ps which supports far
less options.
Now we also have psgo which is used by default but that only supports a
small subset of ps(1) options. Implementing all options there is way to
much work.
Docker on the other hand executes ps(1) directly on the host and tries
to filter pids with `-q` an option which is not supported by busybox's
ps and conflicts with other ps(1) arguments. That means they fall back
to full ps(1) on the host and then filter based on the pid in the
output. This is kinda ugly and fails short because users can modify the
ps output and it may not even include the pid in the output which causes
an error.
So every solution has a different drawback, but what if we can combine
them somehow?! This commit tries exactly that.
We use ps(1) from the host and execute that in the container's pid
namespace.
There are some security concerns that must be addressed:
- mount the executable paths for ps and podman itself readonly to
prevent the container from overwriting it via /proc/self/exe.
- set NO_NEW_PRIVS, SET_DUMPABLE and PDEATHSIG
- close all non std fds to prevent leaking files in that the caller had
open
- unset all environment variables to not leak any into the contianer
Technically this could be a breaking change if somebody does not
have ps on the host and only in the container but I find that very
unlikely, we still have the exec in container fallback.
Because this can be insecure when the contianer has CAP_SYS_PTRACE we
still only use the podman exec version in that case.
This updates the docs accordingly, note that podman pod top never falls
back to executing ps in the container as this makes no sense with
multiple containers so I fixed the docs there as well.
Fixes#19001
Fixes https://bugzilla.redhat.com/show_bug.cgi?id=2215572
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Users may want to replace the secret used within containers, without
destroying the secret and recreating it.
Partial fix for https://github.com/containers/podman/issues/18667
Make sure podman --remote secret inspect and podman secret inspect
return the same error message.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
1. toolbox UID/GID allocation: pick numbers < 1500. Otherwise
we run the risk of colliding with the Cirrus rootless user.
2. WaitContainerReady(): check the results of the last "podman logs"
before timing out. Otherwise, the user will see "READY" followed
immediately by "Container is not ready".
(global bug, not just toolbox, but that's where I discovered it).
Signed-off-by: Ed Santiago <santiago@redhat.com>
Using GinkgoT().TempDir() will automatically result in the directy to be
cleaned up when the test is done. This should help to prevent leaking
files and we do not need to error check every time.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Like LockTmpDir use a random tmpdir for this directory. Make sure it is
set for all parallel ginkgo processes.
Also GinkgoT().TempDir() will automatcially remove the directory at the
end so we do not need to worry about cleanup.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
AFAIK the latest podman will not even run on RHEL 7 anymore, in any case
we do not need these tests to run there.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Using the OS tempdir here is not good. This defaults to /tmp which means
the inital podman test setup uses these paths:
`--root /tmp/root --runroot /tmp/runroot and --tmpdir /tmp`
Thus we create many files directly under /tmp. Also they were never
removed thus leaked out. When running as root and then later as rooltess
this would fail to permission problems.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Ginkgo currently logs a warning because the cli version (defnied in
test/tools/go.mod) does not match the library version (defnied in
go.mod).
Simply fix this by updating ginkgo to the latest version.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
There is no need to buffer them all into an array then write them once
at the end. Just write directly to the file.
Fixes#19104
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
The --authfile flag has been ignored. Fix that and add a test to make
sure we won't regress another time. Requires a new --tls-verify flag
to actually test the code.
Also bump c/common since common/pull/1538 is required to correctly check
for updates. Note that I had to use the go-mod-edit-replace trick on
c/common as c/buildah would otherwise be moved back to 1.30.
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=2218315
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
The podman-login tests have accumulated much cruft over the
years, because that's the only place where we run a local
registry, and the process was crufty: we actually start/stopped
the registry as the first & last tests of the file. Meaning,
you couldn't do 'hack/bats 150:just-one-test' because that
would skip the registry start. And just now, a completely
unrelated test has had to be shoved into the login file.
This PR revamps the whole thing, by adding a new registry helper
module that can be used anywhere. And, once the registry is
started, it just stays running until the end of tests. (This
requires BATS 1.7 or greater).
Signed-off-by: Ed Santiago <santiago@redhat.com>
For pods with bridged and slirp4netns networking we create /etc/hosts
entries to make it more convenient for the containers to address each
other. We omitted to do this for pasta networking, however. Add the
necessary code to do this.
Closes: https://github.com/containers/podman/issues/17922
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Confirm that pasta test name agrees with the test being run.
This is a development-time-only check, it must never fail
beyond CI. The idea is to prevent something like
@test "... Single TCP ... IPv4" {
pasta_test_to 6 ... udp ...
}
Signed-off-by: Ed Santiago <santiago@redhat.com>
- typo fix, a misspelled variable resulting in test being NOP
- remove unnecessary variable (followup to #19044)
- add opportunistic CONTAINERS_CONF test (followup to #19032)
Signed-off-by: Ed Santiago <santiago@redhat.com>
Make sure we use the config field to know if we should use pasta or
slirp4netns as default.
While at it fix broken code which sets the default at two different
places, also do not set in Validate() as this should not modify the
specgen IMO, so set it directly before that.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
This reverts commit c2a24abc0d, which
itself reverted 1c08f2edac, which
reverted e33f4e0bc7.
The original e33f4e0bc7 "pasta: Use two connections instead of three
in TCP range forward tests" was a workaround to avoid intermittent
errors in CI where the pasta networking port range forwarding tests
would fail. It was reverted and unreverted when we thought we'd fixed
the problem, but that turned out not to be the case.
We're now much more confident that we've genuinely found and fixed (or
at least, worked around) the underlying problem, so we revert it again.
Link: https://github.com/containers/podman/issues/17287
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
With a number of the port range forwarding tests, we've seen occasional
failures where the sending socat fails with an EINTR on connect(). This
was mitigated by e33f4e0bc7 "pasta: Use two connections instead of three
in TCP range forward tests" (which has been reverted and un-reverted
several times). However, this did not eliminate the problem, for example
see [0].
For the failing tests we are using the socat address "EXEC:printf x" to
make socat invoke printf(1) to generate a single byte of data to transfer.
Closer analysis shows that the SIGCHLD as the printf process ends is
occasionally intersecting with the connect() call causing this failure.
This is arguably a bug in socat, to not handle this race one way or
another. However, we can easily workaround the problem by using a
temporary file with the data to transfer, rather than invoking printf every
time. Do this, to avoid the flakiness of these tests.
[0]
https://github.com/containers/podman/issues/17287#issuecomment-1611855165
Closes: https://github.com/containers/podman/issues/17287
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
https://github.com/containers/podman/pull/19021 fixed bugs with the pasta
networking tests not working on hosts with multiple interfaces. Alas, the
patch left in some stale code that generates spurious error messages for
the IPv6 case. This is sort of harmless - later code overrides what's done
here and the tests can pass anyway. However if a test fails for some other
reason it means we get a misleading irrelevant error message.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
It is pretty complicated to display the secret on the host, but is
not really secured. This patch makes it easier to examine the secret.
Partial fix for https://github.com/containers/podman/issues/18667
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
[October 2022] While mucking around in this test, I noticed that
a test was being unnecessarily skipped in rootless. Reason was,
test was creating a /etc/systemd file, which it really shouldn't
have been doing anyway.
[Flash forward to June 2023] Ugh. This got complicated, so I
abandoned it. But it's flaking in CI, so one focus-push later,
here's everything that this PR fixes:
- create systemd unit file in proper (safe) path.
- create it *with proper podman options!!!* As in, the
whole --this --that --root --tmpdir options! Sheesh!
- use a pseudorandom service name, not just "redis"
- invoke systemctl/journalctl with --system or --user
as appropriate.
- remove unnecessary "bash -c"
- remove SkipIfRootless, but add SkipIfRemote
Signed-off-by: Ed Santiago <santiago@redhat.com>
At various points the pasta bats tests need to know the name of the
interface that pasta will use by default, and the host addresses it will
use by default. Currently we use the pre-existing helper functions
ether_get_name and ipv[46]_get_addr_global to retreive that.
However, those just pick the first non-loopback interface or address, which
may not be the one that pasta uses if there are multiple connected host
interfaces.
Replace those helpers with local ones which examine the routing table to
more closely match pasta's internal logic about which interface to select.
This allows the tests to run successfully on a host with multiple
interfaces.
Closes: https://github.com/containers/podman/issues/19007
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
There was a huge cut and paste of mount options which were not constent
in parsing tmpfs, bind and volume mounts. Consolidated into a single
function to guarantee all parse the same.
Fixes: https://github.com/containers/podman/issues/18995
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
This commit was automatically cherry-picked
by buildah-vendor-treadmill v0.3
from the buildah vendor treadmill PR, #13808
Changes since 2023-05-01:
- skip a new test, it fails in remote
- skip encrypted-FROM test, broken by buildah PR 4746
Signed-off-by: Ed Santiago <santiago@redhat.com>
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
After[1] c/image no longer prints "Storing signatures" so we should
not check for it.
[1] https://github.com/containers/image/pull/2001
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Sometimes this tests flakes but in the CI log I see all expected lines
printed but still for some reason the matcher fails.
Right now it will truncate the array so it is not possible to verify
what the matcher sees. Change this be removing the truncate limit for
this specific test only.
see #18501
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
we were silently ignoring --device-cgroup-rule in rootless mode. Make
sure an error is returned if the user tries to use it.
Closes: https://github.com/containers/podman/issues/18698
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
Handle more TOCTOUs operating on listed images. Also pull in
containers/common/pull/1520 and containers/common/pull/1522 which do the
same on the internal layer tree.
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=2216700
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Add new _prefetch helper for fetching and caching images.
Use it in a few places, most importantly 120-load.bats
where our teardown() now runs 'rmi -af'.
Reason: in #17911 we discovered that podman save + load do
not actually preserve the image: annotations and other metadata
are lost. This means that a test which runs after 120-load.bats
is operating on a different $IMAGE than a test which runs before.
This is not a problem except in very obscure corner cases, like
one fixed in #18542, but it seems irresponsible to just handwave
that issue away
The _prefetch function uses skopeo for fetching and saving
images, because skopeo preserves digests and metadata.
[Side note for posterity: I tried amending basic_setup() to
always rmi -a + prefetch, instead of the current images -a +
rmi unwanted ones. That slowed down system tests by 10 minutes,
presumably because loads are much slower than queries. I reverted
that change and am documenting it as a reminder of why we do things
the way we do.]
Signed-off-by: Ed Santiago <santiago@redhat.com>
Wait before sending status code 200 for the first top call and if that
fails return a proper error code.
This was leading to some confusion in [1] because podman just reported
200 but did not wirte anything back.
[1] https://bugzilla.redhat.com/show_bug.cgi?id=2215572
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Support two new wait conditions, "healthy" and "unhealthy". This
further paves the way for integrating sdnotify with health checks which
is currently being tracked in #6160.
Fixes: #13627
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Replace /dev/foodevdir with unique paths, to avoid one
test's RemoveAll() from stepping on another test.
Closes: #18958
Signed-off-by: Ed Santiago <santiago@redhat.com>
for #18514: if we get a timeout in teardown(), run and show
the output of podman system locks
for #18831: if we hit unmount/EINVAL, nothing will ever work
again, so signal all future tests to skip.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Make sure that the create endpoint does not always return 200 even in
case of a failure. Some of the code had to be massaged since encoding a
report implies sending a 200.
Fixes: #15828
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Make sure that the push endpoint does not always return 200 even in case
of a push failure. Some of the code had to be massaged since encoding a
report implies sending a 200.
Fixes: #18751
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Add header comment suggesting podman network create instead.
Stop using it in checkpoint tests. Turned out to be much more
complicated than expected.
Also, fix two issues caught while scanning the code:
- remove obsolete f28-and-earlier code.
- remove seccomp workaround needed for RHEL7
Signed-off-by: Ed Santiago <santiago@redhat.com>
A few tests were doing "podman run -d" + "podman logs".
This is racy. Remove the unnecessary "-d".
And, as long as we're mucking around in here:
- remove the "-t" from the 800-lines test, so we get
clean output without ^Ms
- remove unnecessary "sh", "-c" from simple echo commands
- add actual error-message checks to two places that
were only checking exit status
Resolves one (not all) of the flakes tracked in #18501
Signed-off-by: Ed Santiago <santiago@redhat.com>
The platform parameter has been ignored such that images have been
looked up by name only.
Fixes: #18951
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Which revealed that absent --authfile's are ignored but shouldn't.
The issue is now being tracked in #18938.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Commit f131eaa74a changed restart to a stop+start motivated by
comments in the systemd man pages that restart behaves different than
stop+start, for instance, that it keeps certain resources open and
treats timers differently. Yet, the actually fix for #17607 in the very
same commit was dealing with an ENOENT of the CID file on container
removal.
As it turns out in in #18926, changing to stop+start regressed on
restarting dependencies when auto updating a systemd unit. Hence, move
back to using restart to make sure that dependent systemd units are
restarted as well.
An alternative could be recommending to use `BindsTo=` in Quadlet files
but this seems less common than `Requires=` and hence more risky to
cause issues on user sites.
Fixes: #18926
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Add support for `--imagestore` in podman which allows users to split the filesystem of containers vs image store, imagestore if configured will pull images in image storage instead of the graphRoot while keeping the other parts still in the originally configured graphRoot.
This is an implementation of
https://github.com/containers/storage/pull/1549 in podman.
Signed-off-by: Aditya R <arajan@redhat.com>
The backend allows for any start/end ip in the subnet. There is no
reason to limit the cli to only CIDR subnets. This allows for much more
flexibility.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
The "removed" condition mapped to an undefined state which ultimately
rendered the wait endpoint to return an incorrect exit code. Instead,
map "removed" to "exited" to make sure Podman returns the expected
exit code.
Fixes: #18889
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
For tests that use '--ip XX', random IP allocation is not
working well. Switch instead to a deterministic algorithm
with CPU affinity and a fudge factor for CNI.
Signed-off-by: Ed Santiago <santiago@redhat.com>
The code was moved to c/common so use that instead. Also add tests for
the new pasta_options config field. However there is one outstanding
problem[1]: pasta rejects most options when set more than once. Thus it is
impossible to overwrite most of them on the cli. If we cannot fix this
in pasta I need to make further changes in c/common to dedup the
options.
[1] https://archives.passt.top/passt-dev/895dae7d-3e61-4ef7-829a-87966ab0bb3a@redhat.com/
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
This commit creates a new command `podmansh` command which can be used by
administrators to provide a confined shell to their users.
The user will only have access to the volumes and capabilities for that
user.
Co-authored-by: Paul Holzinger <pholzing@redhat.com>
Co-authored-by: Daniel Walsh <dwalsh@redhat.com>
Co-authored-by: Petr Lautrbach <lautrbach@redhat.com>
Co-authored-by: Ed Santiago <santiago@redhat.com>
Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
We no longer allow to match ids in the middle, this makes no realy
sense. ID matches should always be by prefix.
https://github.com/containers/podman/issues/18471
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
podman info prints the network information about binary path,
package version, program version and DNS information.
Fixes: #18443
Signed-off-by: Toshiki Sonoda <sonoda.toshiki@fujitsu.com>
There is weird issue #18856 which causes the version check to fail.
Return the underlying error in these cases so we can see it and debug
it.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Remove an outdated comment on the absence of exit-code propagation when
running K8s workloads in systemd. The `podman-kube@` systemd template
is using default restart policy of the system. The exit-code
propagation is tested in other tests, so we can keep the logic as is.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
One feature needed for podmansh is the ability to set the default
homedir to be the workingdir when you login.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
add routes using the --route flag.
the no_default_route option in --opt prevents a default route from
getting added automatically.
Signed-off-by: Jan Hendrik Farr <github@jfarr.cc>
A c/storage PR[1] chnage the behavior to correctly report umount errors.
This is causing problem in the updgrade tests. The problem is that a
cotnainer is mounted inside another container and then unmounted on the
host. Therefore both operations happen in different mount namespaces.
this is expcted but we want to share the mounts between them. This is
the default but c/stroage make the root private by default thus the
mounts were not shared. To fix this use the `skip_mount_home` storage
option so the mount is kept shared.
[1] https://github.com/containers/storage/pull/1607
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
The Sysctl=name=value entry can be used to set --sysctl=name=value
directly without the need to use PodmanArgs=--sysctl=name=value.
Signed-off-by: Laurenz Kruty <git@laurenzkruty.de>
First: fix podman-registry script so it preserves the initial $PODMAN,
so all subsequent invocations of ps, logs, and stop will use the
same binary and arguments. Until now we've handled this by requiring
that our caller manage $PODMAN (and keep it the same), but that's
just wrong.
Next, simplify the golang interface: move the $PODMAN setting into
registry.go, instead of requiring e2e callers to set it. (This
could use some work: the local/remote conditional is icky).
IMPORTANT: To prevent registry.go from using the wrong podman binary,
the Start() call is gone. Only StartWithOptions() is valid now.
And, minor cleanup: comments, and add an actual error-message check
Reason for this PR is a recurring flake, #18355, whose multiple
failure modes I truly can't understand. I don't think this PR
is going to fix it, but this is still necessary work.
Signed-off-by: Ed Santiago <santiago@redhat.com>
- trust_test: adding 'Ordered' seems to resolve a very common
flake. I've tested this for dozens of CI runs, and haven't
seen the flake recur (normally it fails every few runs).
- exec and search tests: add FlakeAttempts(3). This is a NOP
under our current CI setup, in which we run ginkgo with
a global --flake-attempts=3. I am submitting this as an
optimistic step toward a no-flake-attempts world (#17967)
Fixes: #18358
Signed-off-by: Ed Santiago <santiago@redhat.com>
For filter=id=XXX (containers, pods) and =ctr-ids=XXX (pods):
if XXX is only hex characters, treat it as a PREFIX
otherwise, treat it as a REGEX
Add tests. Update documentation. And fix an incorrect help message.
Fixes: #18471
Signed-off-by: Ed Santiago <santiago@redhat.com>
Ginkgo test names can have more than two levels: there can be
a nested series of Describes() before the final It(). (e.g.,
quadlet_test.go). Handle that.
Before: we just assumed that the third-or-maybe-fourth line
after a "-----" divider was the test name.
Now: examine every line after the "-----" divider, until the
first empty line. Lines with /path/to/source/file are ignored,
lines with text strings are assembled together to make anchors.
This is still imperfect but it's much better than before.
SPECIAL NOTE: in order to allow linking to timing results
in the AfterSuite, I've changed the test name from Leaf to Full.
This will now be a much longer string, and hence much less
readable, but I'm inclined to think it's more correct. Please
review carefully and lmk if I should revert.
Finally, as an unrelated add-on, add links (at top) to original
log, journal, and (if applicable) podman-remote server logs.
Signed-off-by: Ed Santiago <santiago@redhat.com>
To debug a deadlock, we really want to know what lock is actually
locked, so we can figure out what is using that lock. This PR
adds support for this, using trylock to check if every lock on
the system is free or in use. Will really need to be run a few
times in quick succession to verify that it's not a transient
lock and it's actually stuck, but that's not really a big deal.
Signed-off-by: Matt Heon <mheon@redhat.com>
This is a nice quality-of-life change that should help to debug
situations where someone runs out of locks (usually when a bunch
of unused volumes accumulate).
Signed-off-by: Matt Heon <mheon@redhat.com>
Test that pasta generates a sensible error message if asked to forward a
protocol it doesn't understand.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Add a test for generate kube to verify that the ulimit
annotation is not set for the default case when the user
doesn't set any ulimits.
Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
Intented to fix an obscure, unlikely race condition in which (I
think) two parallel jobs called GetPort() and were assigned the
same port.
Also, add actual proper testing to two HTTP-registry tests, and
Skip a third that's a waste of cycles (filed #18768)
Signed-off-by: Ed Santiago <santiago@redhat.com>
This reverts commit 9bd833bcfd.
With the fix for `podman rm -fa` merged, we no longer require
this patch.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
We _usually_ have only one image in store, $IMAGE, but it's
perfectly fine to also have $SYSTEMD_IMAGE also. Fix a few
tests so they can handle that condition.
And, cleanup:
- remove a no-longer-useful test ("podman load NEWNAME",
functionality that was removed 2+ years ago in #8877)
- reorder some tests in the image-mount test, to make
them safer and easier to understand
- use no-such-image, not no-such-container, in image-mount test.
Computer don't care, but this human felt confused for a sec.
Signed-off-by: Ed Santiago <santiago@redhat.com>
The current way of bind mounting the host timezone file has problems.
Because /etc/localtime in the image may exist and is a symlink under
/usr/share/zoneinfo it will overwrite the targetfile. That confuses
timezone parses especially java where this approach does not work at
all. So we end up with an link which does not reflect the actual truth.
The better way is to just change the symlink in the image like it is
done on the host. However because not all images ship tzdata we cannot
rely on that either. So now we do both, when tzdata is installed then
use the symlink and if not we keep the current way of copying the host
timezone file in the container to /etc/localtime.
Also note that we need to rebuild the systemd image to include tzdata in
order to test this as our images do not contain the tzdata by default.
Fixes https://bugzilla.redhat.com/show_bug.cgi?id=2149876
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Systemd doesn't support `never` and logs a warning, systemd uses no as
default so we do not have to specify it at all.
Check systemd.service(5) for the systemd docs.
Fixes#18743
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Reduce sleep-loop time in logs test, from 1s to 0.1s,
to make 'podman stop' take effect more quickly. With 1s,
and testing with 1s resolution, we get flakes.
Fixes: #17826
Signed-off-by: Ed Santiago <santiago@redhat.com>
The new exit-code propagation test is racy: 'podman wait' can
fail if the service container has already been cleaned up by
systemd.
Solution: run the inspect and wait tests opportunistically, i.e.,
only if those commands succeed. If they fail, confirm that they
fail with ENOSUCHCONTAINER. This may silently lose us some
coverage ... but none of it is important. The important
test, systemctl final status, remains.
Also, as drive-bys:
- add a FIXME comment documenting another race condition
that I'm not bothering to fix right now
- give distinct names to unit files, for readability in
test failures
Fixes: #18732
Signed-off-by: Ed Santiago <santiago@redhat.com>
"image rm concurrent" test is still failing, even after #18664:
Error: no contents in "/tmp/podman_test967723851/Dockerfile"
Probable cause: the images are built in parallel, and p.BuildImage()
writes one single Dockerfile. (This almost certainly renders the
test less effective than intended, since the generated images
might end up being identical).
Solution: write and use a uniquely-named Dockerfile
Signed-off-by: Ed Santiago <santiago@redhat.com>
When we do path completion in images a user could try to complete a
simple relative path, e.g. podman run $IMAGE e... should complete to etc
if this path exists in the image. Right now we panic in this case as the
current check didn't account for an empty string in simplePathJoinUnix().
In such a case return the path directly because we can not alter what
the user typed on the cli and must return a path without slash as well
in order for the shell to suggest the completion.
Fixes https://bugzilla.redhat.com/show_bug.cgi?id=2209809
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
There are quite a lot of places in podman were we have some signal
handlers, most notably libpod/shutdown/handler.go.
However when we rexec we do not want any of that and just send all
signals we get down to the child obviously. So before we install our
signal handler we must first reset all others with signal.Reset().
Also while at it fix a problem were the joinUserAndMountNS() code path
would not forward signals at all. This code path is used when you have
running containers but the pause process was killed.
Fixes#16091
Given that signal handlers run in different goroutines parallel it would
explain why it flakes sometimes in CI. However to my understanding this
flake can only happen when the pause process is dead before we run the
podman command. So the question still is what kills the pause process?
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Use ExecStopPost instead of ExecStop to make sure containers, pods, etc.
are all cleaned up even in case of an error.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Add a new field `ExitCodePropagation` field to allow for configuring the
newly added functionality of controlling how the main PID of a kube
service exits.
Jira: issues.redhat.com/browse/RUN-1776
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Implement means for reflecting failed containers (i.e., those having
exited non-zero) to better integrate `kube play` with systemd. The
idea is to have the main PID of `kube play` exit non-zero in a
configurable way such that systemd's restart policies can kick in.
When using the default sdnotify-notify policy, the service container
acts as the main PID to further reduce the resource footprint. In that
case, before stopping the service container, Podman will lookup the exit
codes of all non-infra containers. The service will then behave
according to the following three exit-code policies:
- `none`: exit 0 and ignore containers (default)
- `any`: exit non-zero if _any_ container did
- `all`: exit non-zero if _all_ containers did
The upper values can be passed via a hidden `kube play
--service-exit-code-propagation` flag which can be used by tests and
later on by Quadlet.
In case Podman acts as the main PID (i.e., when at least one container
runs with an sdnotify-policy other than "ignore"), Podman will continue
to wait for the service container to exit and reflect its exit code.
Note that this commit also fixes a long-standing annoyance of the
service container exiting non-zero. The underlying issue was that the
service container had been stopped with SIGKILL instead of SIGTERM and
hence exited non-zero. Fixing that was a prerequisite for the exit-code
propagation to work but also improves the integration of `kube play`
with systemd and hence Quadlet with systemd.
Jira: issues.redhat.com/browse/RUN-1776
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
RHEL gating tests failing, because (sigh) journalctl doesn't
work rootless on RHEL.
I think the flake is fixed anyway, so we don't need this.
This reverts commit ba141adce4.
Signed-off-by: Ed Santiago <santiago@redhat.com>
This test is intended to test concurrent removals, so don't
risk a removal breaking a build.
Fixes#18659 .
(The sitaution that removals can break a build WIP is a real
problem that should be fixed, but that's not a target of this test.)
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Instrument system tests in hopes of tracking down #17216,
the unlinkat-ebusy-hosed flake.
Oh, also, timestamp.awk: timestamps have always been UTC, but
add a 'Z' to make it unambiguous.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Fixes: https://github.com/containers/podman/issues/18239
[NO NEW TESTS NEEDED]
@test "podman build -f test" in test/system/070-build.bats
Will test this. This was passing when run on a local system since
the remote end was using the clients path to read the Containerfile
The issue is it would not work in a podman machine since the
Containerfile would/should be a different path.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Make sure to look for the container's exit code when it's in stopped
state. With `--restart=always`, the container seems to stay in the
stopped state which led the wait logic to loop until the 20 seconds
timeout for the cleanup process to have finished kicks in.
Also defensively make sure to loop when the container is in stopped
state but no exit code has been written yet.
Add a regression test to make sure Podman doesn't wait more than 20
seconds. Even on a CI machine under high load I expect it to take much
much much less than that, so I do not expect this test to flake in the
future.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
The examples show that --dns-add 8.8.8.8,1.1.1.1 is valid but it fails,
fix this by using StringSliceVar which splits at commas.
Added tests to ensure it is working.
Fixes#18632
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
b25b330306 introduced this behaviour.
It was fine at the time because we didn't support "container update",
so the limit could not be changed at runtime. Since it is not
possible to change the memory limit at runtime, read the limit as
reported from the cgroup.
https://github.com/containers/crun/pull/1217 is required for crun.
Closes: https://github.com/containers/podman/issues/18621
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
We should not change selinux, in a parallel context this can change the
behavior of other tests and we should never disable selinux anyway.
Lets see if this passes CI or not.
Fixes#18564
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
the combination --pod and --userns is already blocked. Ignore the
PODMAN_USERNS variable when a pod is used, since it would cause to
create a new user namespace for the container.
Ideally a container should be able to do that, but its user namespace
must be a child of the pod user namespace, not a sibling. Since
nested user namespaces are not allowed in the OCI runtime specs,
disallow this case, since the end result is just confusing for the
user.
Closes: https://github.com/containers/podman/issues/18580
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
Enable the --configmap flag for the remote case of podman
kube play. Users can pass in the paths to the configmap files
for kube play to use when creating the pods and containers from
a kube yaml file. The configmap file is read and the contents are
appended to the contents of the main yaml file before passed to the
remote client.
Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
Accept a tag in the compat api endpoint. For the fromImage param we
already parse it but for fromSrc we did not.
Fixes#18597
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Init containers are currently not properly supported in
generate-systemd and there are no plans to do so since
all focus lies on Quadlet going forward.
Hence, generate systemd should through an error.
Closes: #18585
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
In run_podman(), display a nanosecond-level timestamp next to
each command and its output.
Because this clutters the results, teach logformatter to grok
these new timestamps, strip them, and display a more human-readable
time delta in the left-hand timestamp column. logformatter started off
as a mess and is now, well, 🤮. I'm sorry. I just hope its results
make it worthwhile.
Signed-off-by: Ed Santiago <santiago@redhat.com>
When running ginkgo tests locally we often only want to test a small
subset. I think most people just add the `FIt` block but then you need
to remember to undo that before pushing the changes.
With this change you can just run:
```
make localintegration FOCUS="test name here"
make localintegration FOCUS_FILE="some_test.go"
```
I updated the test Readme to use this new syntax.
The options just map to the ginkgo options, see the upstream docs
linked in the readme for more information about syntax.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Followup to #18578: move Serial to Describe(), in case new
tests get added to this module. And, explain the reasoning.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Reason: gpg tests all run with a different GNUPGHOME, and gpg-agent
does not like that, and there's no longer any way to run gpg
without the agent. So, do not run these tests in parallel, and
clean up agent after each test.
Fixes: #17966 (I hope)
May also fix#18358 but it will take some time to be sure.
Signed-off-by: Ed Santiago <santiago@redhat.com>
...in three kube tests. And, missing error-message checks.
And, reverse the sense of a confusing Expect(), plus add
a description to the test failure. And, set never-restart,
otherwise our "podman wait" will spin for an indeterminate
time.
Signed-off-by: Ed Santiago <santiago@redhat.com>
There is no reason to define the same code every time in each file, just
use global nodes. This diff should speak for itself.
CleanupSecrets()/Volume() no longer call Cleanup() directly, as the
global AfterEach node will always call Cleanup() this is no longer
necessary. If one AfterEach() node fails it will still run the others.
Also always unset the CONTAINERS_CONF env vars. This prevents people
from forgetting to unset it. And fix the special CONTAINERS_CONF logic
in the system connection tests, we do not want to preserve
CONTAINERS_CONF anyway so just remove this logic.
Ginkgo orders the BeforeEach and AfterEach nodes. They will be executed
from the outer-most defined to inner-most. This means our global
BeforeEach is always first. Only then the inner one (in the Describe()
function in each file). For AfterEach it is inverted, from the inner to
the outer.
Also see https://onsi.github.io/ginkgo/#organizing-specs-with-container-nodes
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Make sure that the directory formats are not just substituted with their
archive counterparts but actually tar'ed up directories. Also make sure
that the clients don't get chown errors by setting rootless user and
group ID instead of O when running in the user namespace.
Fixes: #15897
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
These annotations can have security implications - crun, for
example, allows rootless containers to preserve the user's groups
through an annotation. We absolutely should not include
annotations from an untrusted image off the internet by default.
We may consider whitelisting some annotations (e.g. the legacy
WASM annotations), but given that there is now a more explicit
way of specifying an image uses the WASM runtime in the OCI image
spec, I'm just tearing this out entirely for now.
Signed-off-by: Matt Heon <mheon@redhat.com>
Read the entire YAML file in case of a multi-doc file
Adjust the unit test
Add a system test
Add comment in the man page
Signed-off-by: Ygal Blum <ygal.blum@gmail.com>
Several tweaks to see if we can track down #17216, the unlinkat-ebusy
flake:
- teardown(): if a cleanup command fails, display it and its
output to the debug channel. This should never happen, but
it can and does (see #18180, dependent containers). We
need to know about it.
- selinux tests: use unique pod names. This should help when
scanning journal logs.
- many tests: add "-f -t0" to "pod rm"
And, several unrelated changes caught by accident:
- images-commit-with-comment test: was leaving a stray image
behind. Clean it up, and make a few more readability tweaks
- podman-remote-group-add test: add an explicit skip()
when not remote. (Otherwise, test passes cleanly on
podman local, which is misleading)
- lots of container cleanup and/or adding "--rm" to run commands,
to avoid leaving stray containers
Signed-off-by: Ed Santiago <santiago@redhat.com>
Yet another case of missing podman-wait. In these two, I see
no reason to run containers detached, so I just removed "-d"
Signed-off-by: Ed Santiago <santiago@redhat.com>
Another low-hanging fruit: test flake because podman-remote
trying to contact a server that hadn't come up.
Fixes: #17940
Signed-off-by: Ed Santiago <santiago@redhat.com>
Run $QUADLET and all systemctl/journalctl commands using 'timeout'.
Nothing should ever, ever take more than the default 2 minutes.
Followup to #18514, in which quadlet tests are found to be
taking 9-10 minutes.
Signed-off-by: Ed Santiago <santiago@redhat.com>
- document env vars that can be used
- list up to date dependencies
- remove unnecessary GOPATH mention, no longer needed with gomodules
- use make targets to tests everything (much faster due `-p` option)
- remove tests in container section as make shell is not a valid target
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
In rootFsSize(), instead of calculating the size of the diff for every
layer of the container's base image, ask the storage library for the sum
of the values it recorded when it first wrote those layers.
In a similar fashion, teach rwSize() to use the library's
ContainerSize() method instead of trying to roll its own.
Replace calls to pkg/util.SizeOfPath() with calls to
github.com/containers/storage/pkg/directory.Size(), which does the same
thing.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Yet another case where tests expect play-kube to be synchronous.
There are probably dozens more of these.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Fixes: https://github.com/containers/podman/issues/16354
Currently we check on the server side, which ends up generating a bad
error message.
$ podman --remote build foo/
ERRO[0000] While reading directory /home/dwalsh/go/src/github.com/containers/podman/foo: EOF
Error: stat /var/tmp/libpod_builder1249622306/build/Dockerfile: no such file or directory
With this change you will get
./bin/podman --remote build foo/
Error: Containerfile not specified and no Containerfile or Dockerfile found in context directory, /home/dwalsh/podman/foo
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
- treadmill script: run root & rootless in parallel, not
sequentially. It's only four jobs, and it seems dumb
to fix root tests, repush, then discover a rootless failure.
- apply-podman-deltas: implement skip_if_rootless(), and
use it to skip a nasty longstanding flake
- bud-tests-in-podman diffs: ugly code to fix a rootless hang.
background: rootless remote tests hang
cause: stray podman server process
root cause: no idea. No clue at all. I just gave up
workaround: seek out and kill stray server processes
Rootless buildah-bud tests are not run in regular CI,
only in the buildah treadmill.
Signed-off-by: Ed Santiago <santiago@redhat.com>
If the container was already cleaned up we should not try to do it
again. Podman stop will always try to call Cleanup() if you look at the
podman event log and just keep calling podman stop --all you see a
cleanup event every time. This is not wanted. Also in case of the host
pidns we report a error every single time, see the linked issue.
Fixes#18460
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
The logic which checks for duplicated volumes here did not work
correctly because it used filepath.Clean(). However the writes to the
volDestinations map did not thus the string no longer matched when you
included a final slash for example.
So we can either call Clean() on all or no paths. I decided to call it
on no path because this is what we do right now. Just the check did it.
Fixed#18454
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
initContainers in kubernetes deployments had no call to CompleteSpec in the
generation, which means that the default environment is not configured for
these. This causes issues with missing default environment variables like $HOME
or $PÄTH.
Also, switch to using logrus.Warn() instead of fmt.Fprintf(os.Stderr)
This fixes https://github.com/containers/podman/issues/18384
Co-authored-by: Ed Santiago <santiago@redhat.com>
Signed-off-by: Dan Čermák <dcermak@suse.com>
There are days when I really, really, really hate GNU. Remember
when someone decided that 'head -1' would no longer work, and
that it was OK to break an infinite number of legacy production
scripts? Someone now decided that egrep/fgrep are deprecated,
and our CI logs (especially pr-should-include-tests) are now
filled with hundreds of warning lines, making it difficult
to find actual errors.
I expect that those warnings will be removed quickly after
furious community backlash, just like the 'head -1' fiasco
was quietly reverted, but ITM the warnings are annoying
so I capitulate.
Signed-off-by: Ed Santiago <santiago@redhat.com>
This one got complicated, and deserves its own commit.
Problem: ginkgo logs have a lot of NUL characters, making them
difficult for logformatter to process and for humans to read.
Cause: Paul tracked it down to "podman volume export" without "-o"
(hence spitting out tar data to stdout).
Solution: add "-o tmpfile" to named podman-volume-export. In
the process, fix all sorts of other problems with that test.
And, since the e2e test no longer tests "volume export" by
itself, add a system test that does.
It is possible that there are other places that emit NULs.
One step at a time.
Signed-off-by: Ed Santiago <santiago@redhat.com>
- fix a typo that was resulting in a test being a NOP, and
add actual testing to it.
- fix two Expects() with incorrectly-ordered actual/expects
- remove leading whitespace from an It() test name
- To(BeTrue()) is evil. Wherever possible, replace it with
useful string or field checks. When not possible, use
the annotation field to indicate what failed. I got
carried away here, #sorrynotsorry
- remove unused system-test code
Signed-off-by: Ed Santiago <santiago@redhat.com>
Podman kube generate now uses the pod's restart policy
when generating the kube yaml. If generating from containers
only, use the restart policy of the first non-init container.
Podman kube play applies the pod restart policy from the yaml
file to the pod. The containers within a pod inherit this restart
policy.
Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
Add --restart flag to pod create to allow users to set the
restart policy for the pod, which applies to all the containers
in the pod. This reuses the restart policy already there for
containers and has the same restart policy options.
Add "never" to the restart policy options to match k8s syntax.
It is a synonym for "no" and does the exact same thing where the
containers are not restarted once exited.
Only the containers that have exited will be restarted based on the
restart policy, running containers will not be restarted when an exited
container is restarted in the same pod (same as is done in k8s).
Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
Requires vendoring fixes from c/common and to update the transformation
code. Also add a test to avoid future regressions.
Fixes: #17763
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Podman and Docker clients split the filter map slightly different, so
account for that when parsing the filters in the image-listing endpoint.
Fixes: #18092
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Add a workaround for #18180 so the ginkgo work can be merged without
being blocked by the issue. Please revert this commit when the issue
is fixed.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
This never worked when ginkgo runs with more than one thread, we use 3
in CI. The problem is that the SynchronizedAfterSuite() function accepts
two functions. The first one is run for each ginkgo node while the
second one is only run once for the whole suite.
Because the timings are stored as slice thus in memory we loose all
timings from the other nodes as they were only reported on node 1.
Moving the printing in the first function solves this but causes the
problem that the result is now no longer sorted. To fix this we let
each node write the result to a tmp file and only then let the final
after suite function collect the timings from all these files, then
sort them and print the output like we did before.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
podman unshare --rootless-netns leaks the namespace and slirp4netns by
design as there is no safe way to remove it without any races.
To trigger a cleanup we can spin up a container and it will
automaticallt teardown the netns for us.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
USe the new ginkgo `Serial` decorator to make sure system reset is
never executed in parallel. system reset stops teh rootless pause
process which causes major issues when other process in parallel still
use this old namesapce.
Fixes#17903
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Just like Cleanup() they should check the error codes.
While doing this it was clear that some volume tests were calling
Cleanup() twice so remove this.
Instead make sure they call Cleanup() themselves so callers only need to
do one call. This is required because we cannot use Expect().To() before
doing all the cleanup. An error causes panic does results in an early
return thus missing potentially important cleanup.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
It looks like AfterEach() is now executed even after Skip(), this is a
good idea because the fact that it did't before caused us to leak tmp
directories. However in case Skip() is called before the podmanTest is
initialized it will no result in a panic. To fix it simply prevent such
panic by checking the pointer against nil and do nothing in such case.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
The Cleanup() function tries to stop all containers, a paused contianer
cannot be stopped. The tests should make sure it works.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Porting them over to v2 requires a full rewrite.
IT is not clear who actually uses these benchmarks, Valentin who wrote
them originally is in favor of removing them. He recommends to use
script from hack/perf instead.
This commit also drop the CI integration, it is not clear who actually
uses this data. If it is needed for something please speak up.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Directly writing to stdout/err is not safe when run in parallel.
Ginkgo v2 fixed this buffering the output and syncing the output so it
is not mangled between tests.
This means we should use the GinkgoWriter everywhere to make sure the
output stays in sync.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
If we do not unset CONTAINERS_CONF before tests that create a invlid
config will cause the Cleanup to fail.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Only check exit codes last, othwerwise in case of errors it will return
early and miss other commands.
Also explicitly stop before rm, rm is not working in all cases (#18180).
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
The test will leak processes because the rm -fa in the cleanup failed.
This happens because podman tried to remove the contianers in the wrong
order and thus ppodman failed with:
`contianer XXX has dependent containers which must be removed before it`
For now I patch the test but it should be much better if we can fix it
in podman to remove in the correct order. `--all` should mean all I do
not care if there is a dependent container, just get rid of it.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
We blindy trust these commands to work but as it turns out they do not
under certain circumstances.
The "podman run ipcns ipcmk container test" can be used to fail this
reliably, if a container has dependencies the order of rm --all may
cause it to fail because the contianers are deleted in the wrong order.
This is th eonly one I found so far, adding this will uncover many more
of such problems without proper cleanup we leak processes and ginkgo v2
will block because of them.
Of course this cannot be merged without fixing these issues.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
It is not clear why but without the wait is seems like the podman
process just hangs forever which now causes ginkgo to block until it
exits.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
This is not safe at all when run in parallel, CNI needs that directory
to detect duplicated ips and also stores other important network info in
it. Removing it while container network is setup is not safe at all and
could cause a lot of weird flakes.
This "hack" was added in commit 55508c11 but provides zero context what
this was supposed to fix so I don't know what the actual issue is or was.
Fixes#18399
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Set REGISTRY_AUTH_FILE to unique path for each subtest. This
should eliminate collisions where one test runs "podman logout"
just after another does "podman login".
Also, add a test to confirm that the authfile gets written
as expected.
Also, add actual tests for expected error messages, instead
of just ExitWithError()
Fixes: #18397
Signed-off-by: Ed Santiago <santiago@redhat.com>
Podman's container-name generation depends on the global RNG state being
properly initialized (seeded). Should this not happen for some reason
(or it's seeded with a static value), podman will generate the exact
same repeating sequence of container names (assuming no clashes with
existing containers). Add a test to confirm this is always the case.
Signed-off-by: Chris Evich <cevich@redhat.com>
Ref: https://pkg.go.dev/math/rand@go1.20#Seed
Note: For `runtime_test.go`, this test-case was never actually doing
what appears as it's intent . Fixing it to work as intended would be
require incredibly libpod-invasive changes. Do the least-worse thing and
simply confirm that consecutive generated names are different.
Signed-off-by: Chris Evich <cevich@redhat.com>
Systemd supports unit files with a prefix '-' which
tells the system to check if the content exists before
using it. This would allow the QM project to specify
AddDevice=-/dev/kvm, which would add the /dev/kvm device
to the container iff it exists on the host.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Add actual tests (for expected errors, not just exit-status) to
the "push to local registry with authorization" test. As it is
now, if the registry is unreachable, the test passes a number
of steps and only fails later, with a misleading diagnostic.
Followup to, but does not fix, #18286
Signed-off-by: Ed Santiago <santiago@redhat.com>
...at least as many as possible. "run/exec -it" make no sense
in a CI environment; I believe the vast majority of these are
the result of fingers typing on autopilot, then copy/pasting
cascades from those. This PR gets rid of as many -it/-ti as
possible. Some are still needed for testing purposes.
Y'all have no idea how much I hate #10927 (the "no logs from conmon"
flake). This does not fix the underlying problem, nor does it even
eliminate the flake (The "exec terminal doesn't hang" test needs
to keep the -ti flag, and that's one of the most popular flakers).
But this at least reduces the scope of the problem. It also removes
a ton of nasty orange "input device is not a TTY" warnings from logs.
Signed-off-by: Ed Santiago <santiago@redhat.com>
The long term goal was to provide the customer a way to turn on the
preexec_hooks processing of script by having some kind of configuration
that could be read. I had tried putting it into containers.conf to
start, but that turned out to be unyieldly quickly and time is of
the essence for this fix. That is mostly due to the fact that this
code is preexecution and in C, the conatiners.conf file is read in
Go much further down the stack.
After first trying this process using an ENVVAR, I have
thought it over and chatted with others and will now look for a
/etc/containers/podman_preexec_hooks.txt file to exist. If the admin
had put one in there, we will then process the files in the
directories `/usr/libexec/podman/pre-exec-hooks`
and `/etc/containers/pre-exec-hooks`.
Thoughts/suggestions gratefully accepted. This will be a 8.8/9.2 ZeroDay
fix and will need to be backported to the v4.4.1-rhel branch.
Signed-off-by: tomsweeneyredhat <tsweeney@redhat.com>
Nasty test flake, "bad address nc-server.dns.podman"
Cause: "There is absolutely no guarantee that aardvark-dns
is ready before the container is started." (source: Paul).
Workaround (not a real solution): wait before doing a host lookup.
Also: remove a 99%-duplicate test.
Closes: #16272 (I hope)
Signed-off-by: Ed Santiago <santiago@redhat.com>
The problem right now is that --ns contianer: syntax causes use to add
the namespace path to the spec which means the runtime will try to call
setns on that. This works fine for private namespaces but when the host
namspace is used by the container a rootless user is not allowed to
join that namespace so the setns call will return with permission
denied.
The fix is to effectively switch the container to the `host` mode
instead of `container:` when the mention container used the host ns. I
tried to fix this deep into the libpod call when we assign these
namespaces but the problem is that this does not work correctly because
these namespace require much more setup. Mainly different kind of mount
points to work correctly.
We already have similar work-arounds in place for pods because they also
need this.
For some reason this does not work with the user namespace, I don't know
why and I don't think it is really needed so I left this out just to get
at least the rest working. The original issue only reported this for the
network namespace.
Fixes#18027
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Our friend #7096 is still not fixed: it continues to flake,
singletons only, and only in the "create" test (not "run").
My guess: maybe there's a race somewhere in IP assignment,
such that container1 can have an IP, but not yet be running,
and a container2 can sneak in and start with that IP, and
container1 is the one that fails?
Solution: tighten the logic so we wait for container1 to
truly be running before we start container2. And, when we
start container2, do so with -a so we get to see stdout.
(Am not expecting it to be helpful, but who knows).
Also very minor cleanup
Signed-off-by: Ed Santiago <santiago@redhat.com>
Having a container spin-wait on a /stop file, then exit, is
unsafe: 'podman exec $ctr touch /stop' can get sucked into
container cleanup before the exec terminates, resulting in
the podman-exec failing and hence the test failing.
Most existing instances of this pattern are unnecessary.
Replace those with just 'podman rm -f'.
When necessary, use a variety of safer alternatives.
Re-Closes: #10825 (already closed; this addresses remaining cases)
Signed-off-by: Ed Santiago <santiago@redhat.com>
We should return the raw state string without any extra formatting in
this case.
`{{.Status}}` returns the nicely formatted string used in the default ps
output, e.g. `Up 2 seconds ago`, while `{{.State}}` returns the state as
string, e.g. `running`.
This matches the docker output and allows better use in scripts.
Fixes#18244
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
The `exec` session somestimes exits with 137 as the exec session races
with the cleanup process of the exiting container. Fix the flake by
running a detached exec session.
Fixes: #10825
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Wire in support for writing the digest of the pushed image to a
user-specified file. Requires some massaging of _internal_ APIs
and the extension of the push endpoint to integrate the raw manifest
(i.e., in bytes) in the stream.
Closes: #18216
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
When running the remote integration test I have over 1000 zombies
because each test creates a single service process. Only after ginkgo
exists they get finally reaped by the init process. This only effected
the rootless runs.
For some reason the test use different logic between root and rootless.
This doesn't make much sense. I also see no reason to manually kill
child processes.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
This is a rather big deal. All system services shared the same tmpdir
which causes big issues for the rootless netns setup.
Also use --events-backend file like the local ones. This is important
otherwise reading events and takes ages as the jounal is shared for all
tests.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Creating a new diretory results in the test leaking it when it is not
removed via a defer call. All tests have already access to
`podmanTest.TempDir` which will be automatically removed in the
`AfterEach()` block.
While some test were fine other forgot the defer call. To keep the test
consitent and prevent other from making the same mistake convert all
users to `podmanTest.TempDir`. `CreateTempDirInTempDir()` is only used
for the `podmanTest.Setup()` call.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Some network test use the same subnet as others, because the network
config direcory is shared we must ensure subnets do not conflict as
tests are run in parallel. I see this locally when running with 12
threads.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
If a unit is not active the exit code from systemctl is 3. Thus this
test always failed because it checked the error.
Fix this by checking the exit code and remove the unnecessary output
parsing.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Running these locally always created a popup to ask me for my password
as I am in the wheel group.
I would also argue that such a test should not be run on any local
system ever even as root. First docker could be a symlink to podman so
the check if the image is there would fail. Second starting the docker
deamon in a podman test suite just feels very unexpected.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Because the test mounts the current dir it does not need to create a new
file in it. Just check if the current test file is there should fulfill
the same purpose.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Adds two custom config dirs to tests that were missed in
commit dc9a65e348.
Fixes#17946 (hopefully finally)
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
If user specifies commit --format, we were not setting it before
commit, this caused warning messages that made no sense to be
printed that made no sense.
Fixes: https://github.com/containers/podman/issues/17773
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
A cgroup could have been deleted by the time WalkDir is trying to
access it. Ignore the error and continue.
Closes: https://github.com/containers/podman/issues/17989
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
PR #15093 implemented support for NoPrune in the ImageRemoveOptions,
this PR simply brings that also to the compat API along with
regression tests.
Signed-off-by: Andreas Kohn <andreas.kohn@gmail.com>
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
uber/jaeger-client-go library is deprecated. Remove it.
Only place it's used is in one e2e test, a test that is flaking
in a way that suggests that the HostIP() weighting heuristic from
that module was not actually getting the best outgoing IP address.
So, switch to using what seems to be the current best practice.
No need to make it reusable, since it's only used in one place.
Oh, also remove undesired "-dt" from two "podman run"s. In one
it's harmless, in the other it would cause a test failure under
some circumstances.
Closes: #18269 (optimistic, aren't I?)
Signed-off-by: Ed Santiago <santiago@redhat.com>
Following @edsantiago guidance,
* Additional explanations for each step of the test
* Timezone for tests normalized to UTC
* Smarter choice of separator and use of shell substring extraction
Signed-off-by: rbagd <mail@rbagd.eu>
Adapts to pass the test even if
podman binary path is not `/usr/local/bin/podman`.
[NO NEW TESTS NEEDED]
Signed-off-by: Toshiki Sonoda <sonoda.toshiki@fujitsu.com>
Closes#17767Closes#17768
System test for image list and history dates
* Changed field separator in the test to `;` for easier parsing
* Converted date output from image history and image list to be comparable
Signed-off-by: rbagd <mail@rbagd.eu>
Fix a number of bugs wrt. filtering remote containers and how to
process specified names or IDs. I _really_ do not like the duplication
between remote and local Podman but want to focus on fixing #18153
for now.
What I desire in the future is to consolidate all functionality of
looking up containers (all, latest, filters, specified names/IDs, etc.)
and for remote clients to just call containers/list etc.
Fixes: #18153
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Podman is attempting to split the headers returned by the ps
command into a list of headers. Problem is that some headers
are multi-word, and headers are not guaranteed to be split via
a tab. This PR splits the headers bases on white space, and for
the select group of CAPS headers which are multi-word, combines
them back together.
Fixes: https://github.com/containers/podman/issues/17524
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Update e2e tests not to expect the flag
System tests - explicitly set the log driver to be able to parse the output
Signed-off-by: Ygal Blum <ygal.blum@gmail.com>
Possible cause: on Debian, maybe because of fuse-overlayfs(??),
we sometimes see unexpected inode numbers.
This PR tightens the test logic, so it runs one 'stat' command
in only one podman invocation, then cross-checks multiple lines
of output. I don't know if this will really fix the flake, but
even if it doesn't, it will at least give us much more useful
diagnostic output than before.
And, as long as I'm in here, clean up test, remove duplication,
make error messages distinct (hence more useful), and comment.
Fixes: #17979
Signed-off-by: Ed Santiago <santiago@redhat.com>
Use --restart=no on all created containers. Without this, all
containers spin forever and it's impossible to get a
reliable exit status.
As a side effort, clean up tests, make more robust and maintainable.
Fixes: #18047
Signed-off-by: Ed Santiago <santiago@redhat.com>
In setup, write a containers.conf.d file with db_backend
as specified in .cirrus.yml.
This is actually much scarier and more achy-breaky than
merely "sqlite system tests": it enables sqlite in e2e
tests. ("But wait, we already do that!" -- no, not really.
sqlite in e2e is being done via --db-backend option, and
some podman commands in e2e do not use the standard options.
See #17904.
This is unlikely to get merged any time soon (March, maybe
even April) because sqlite is still too fragile; this will
trigger more flakes than are currently acceptable. Also,
the nasty auto-update flake seems to trigger much more
reliably with sqlite. We need that one fixed.
Signed-off-by: Ed Santiago <santiago@redhat.com>
While debugging #17904 we found the test to be missing the common podman
flags. Add them to the podman invocations and remove some clutter.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
By pulling in the fixes from c/common. Add regression tests to make
sure it's not happening another time. The error messages are not
ideal and should probably be optimized in the `/auth` endpoints directly
but it's already an improvement over a nil deref.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Commands like podman-create(1), podman-run(1), podman-inspect(1),
podman-ps(1) will emit formatted output upon success. This allows
the output from commands to be emitted directly to a file and
can supersede the --noout parameter by using /dev/null. An issue
with --noout was also remedied.
This closes issue #18120.
Signed-off-by: Ali Rizvi-Santiago <arizvisa@gmail.com>
This test was added twice once for CNI and netavark, just write it once
there is no need to do this weird skip thing.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Allow users to customize the network_interface option is the network
config. For bridge this allows users to change the bridge name and for
maclvan it will be the same as `--opt parent=...`.
However the main reason for this option is to allow netavark plugins to
make use of it. I demoed the host-device plugin which makes use of this
as an example. While we could let users set them via --opt it is more
natural to just use the field which is designed for that purpose.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Wait for the expected logs to appear in the journal before using
`journalctl`. #18132 is likely flaking because `journalctl` does
not yet see the container's logs.
Also force the test to use the `passthrough` log driver to make sure
`podman logs` continues being tests.
Fixes: #18132
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Use the kube_generate_type from the containers.conf as
the default value for the --type flag for kube generate.
Override the default when userexplicitly sets the --type
flag.
Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
If resource limits is not set, do not display the following warning message:
`Resource limits are not supported and ignored on cgroups V1 rootless systems`
Ref: #17582
Signed-off-by: Toshiki Sonoda <sonoda.toshiki@fujitsu.com>
Currently --tmpdir changes the location of the pause.pid file. this
causes issues because the c code in pkg/rootless does not know about
that. I tried to fix this[1] by fixing the c code to not use the
shortcut. While this fix worked it will result in many pause processes
leaking in the integrration tests.
Commit ab88632 added this behavior but following the disccusion it was
never the intention that we end up having more than one pause process.
The issues that was trying to fix was caused by somthing else AFAICT,
the main problem seems to be that the pause.pid file parent directory
may not be created when we try to create the pid file so it failed with
ENOENT. This patch fixes it by creating this directory always and revert
the change to no longer depend on the tmpdir value.
With this commit we now always use XDG_RUNTIME_DIR/libpod/tmp/pause.pid
for all podman processes. This allows the c shortcut to work reliably
and should therefore improve perfomance over my other approach.
A system test is added to ensure we see the right behavior and that
podman system migrate actually stops the pause process. Thanks to Ed
Santiago for the improved test to make it work for both `catatonit` and
`podman pause`.
This should fix the issues with namespace missmatches that we can see in
CI as flakes.
[1] https://github.com/containers/podman/pull/18057Fixes#18057
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Bump containers/(storage, common, buildah and image)
Changes since 2023-01-01:
- skip mount-cache-selinux-long-name test under remote, with
a FIXME requesting that someone see if it can be made to work.
- skip six tests that fail under rootless-remote
- add new --build-arg-file option:
- update man page
Squash of:
* cf56eb1865
* 561f082772
Signed-off-by: Ed Santiago <santiago@redhat.com>
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Signed-off-by: Aditya R <arajan@redhat.com>
The `UserNS` key will replace the `RemapGid`, `RemapUid`, `RemapUidSize`
and `RemapUsers` options which are therefore marked as deprecated by
this commit.
Closes#17984
Signed-off-by: Cedric Staniewski <cedric@gmx.ca>
The "podman pull by digest and list --all" e2e test pulls an image using
a tagged reference when an image with the same ID is already present in
a read-only additional image store.
This causes a new image record to be created in read-write storage.
The test then removes this entry, pulls the image again using a digested
reference, and then expects the image to not have any tagged names in it
when it goes to look at it again.
Newer containers/storage will ensure that at the point when the
read-write image record is created, that it includes all of the data
items and naming information from the read-only copy of the image, so
that this information doesn't appear to be lost.
Change the test to use "untag" instead of "rmi", which should pass with
either the older or newer containers/storage.
The test is checking that `podman images` doesn't choke when it
encounters a digested name attached to an image, so the difference in
behavior between containers/storage versions is irrelevant.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Since commit f250560a80 the play kube command uses its own network.
this is racy be design because we create the network followed by
creating/running pod/containers. This means in the meantime another
prune or reset process could wipe out the network config because we have
to share the network config directory by design in the test.
The problem is we only have one host netns which is shared between
tests. If the network config dir is not shared we cannot make conflict
checks for interface names and ip address. This results in different
tests trying to use the same interface and/or ip address which will
cause runtime failures in CNI and netavark.
The only solution I see is to make sure only the reset/prune tests are
using a custom network dir. This makes sure they do not wipe configs
that are otherwise required by other parallel running tests.
Fixes#17946
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
At the time of making this commit, the package `github.com/ghodss/yaml`
is no longer actively maintained.
`sigs.k8s.io/yaml` is a permanent fork of `ghodss/yaml` and is actively
maintained by Kubernetes SIG.
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
Kube generate on pods was not checking for any underscores
in the pod name so was creating a kube yaml with an invalid
pod name when there were underscores present.
The hostname for the pod is set to the podname by default. There
is no need to set that to the container's name or the pod name
again in the generated yaml. So removed that field unless a hostname
was set for the container by the user.
Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
...mostly just test code that wasn't doing the required waits.
My first approach in the kube-play test was to add "--wait".
Bit mistake! The --wait flag, counterintuitively and counter to
documentation, actually destroys all pods+containers+everything
on exit. (Or tries -- see #17803). Since this violates POLA
and is undocumented, I include here a fix to the man page.
Despite my best intentions, I can't reasonably check every single
test for missing waits, especially in kube-play where failing
containers will get retried forever so we can't wait. We'll
just have to fix flakes as we see them.
Fixes: #17958Fixes: #18071
Signed-off-by: Ed Santiago <santiago@redhat.com>
Test has been flaking. Reason: container was run with -d, so
there's a small window in which podman-healthcheck ran on
a running container. Solution: remove -d
Signed-off-by: Ed Santiago <santiago@redhat.com>
Check the DELETE reports for both deletes. #18041 indicates that the
pod hasn't been removed which made me suspicious about the 1st delete.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
I recently wanted to check which version of Quadlet I was using and
found a `-version` flag to be missing. Since Quadlet and Podman are
bundled together, it seems reasonable to me for them to share the same
version.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
This commit adds an quadlet option `Tmpfs` which can be used to mount a
tmpfs in the container.
Closes#17907
Signed-off-by: Cedric Staniewski <cedric@gmx.ca>
One of our oldest most frustrating flakes is #16091, "Timed
out waiting for BYE".
In #17489 we added some debug output to see if the problem
was a container hang of some sort. It does not seem to be
(see #17675), and the debug output makes it hard to read
failure logs, so let's remove it.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Add the command along with the abi and tunnel support
Add e2e tests
Add man page
Add apiv2 test to ensure return codes
Signed-off-by: Ygal Blum <ygal.blum@gmail.com>
add a function to securely mount a subpath inside a volume. We cannot
trust that the subpath is safe since it is beneath a volume that could
be controlled by a separate container. To avoid TOCTOU races between
when we check the subpath and when the OCI runtime mounts it, we open
the subpath, validate it, bind mount to a temporary directory and use
it instead of the original path.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
The podman kube generate command can now generate a
Deployment kind when the --ype flag is set to deployment.
By default, a Pod spec will be generated if --type flag is
not set.
Add --replicas flag to kube generate to allow users to set
the value of replicas in the generated yaml when generating a
Deployment kind.
Add e2e and minikube tests for this feature.
Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
If the kube yaml volumes has secret.items set, then use
the values from that to set up the paths inside the container
similar to what we do for configMap.
Add tests for this as well.
Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
As found in #17828, image listing does not scale well with a growing
number of local images. Make use of recent improvements in libimage
that allow for computing the dangling and parent data with _one_ layer
tree. Prior, the layer tree had to be recomputed _twice_ for each
image.
[NO NEW TESTS NEEDED] as it's a non-functional performance change.
Fixes: #17828
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
By default go will not keep the stdout/err attach when executing
commands via exec.Command(). It is required to explicitly pass the
current stdout/err fds down to the child so we can see the error output
in the logs to debug #17966.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Add a debug line to the wait to test to see which container
is being left behind after the cleaup where the race is happening.
Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
When a userns is set we setup the network after the bind mounts, at the
point where resolv.conf is generated we do not yet know the subnet.
Just like the other dns servers for bridge networks we need to add the
ip later in completeNetworkSetup()
Fixes https://bugzilla.redhat.com/show_bug.cgi?id=2182052
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Some key are available only for user scope while there are no keys that
are supported only for system. So, better to run in user scope
Signed-off-by: Ygal Blum <ygal.blum@gmail.com>
If a path (Yaml, ConfigMap, EnvFile) starts with a systemd path
specifier, treat the path as absolute
Add tests - unit, e2e and bats
Signed-off-by: Ygal Blum <ygal.blum@gmail.com>
...not CONTAINERS_CONF. At least for most tests.
Nearly every system test currently using CONTAINERS_CONF=tmpfile
should be using CONTAINERS_CONF_OVERRIDE.
Simple reason: runtime (crun/runc), database_backend (bolt/sqlite),
logger, and other important settings from /etc/c.conf are not
usually written into the tmpfile. Those tests, therefore, are
not running podman as configured on the system.
Much more discussion: #15413
This PR is a prerequisite for enabling sqlite system tests. For
the sake of simplicity and sanity, I choose to submit the sqlite
switch as a separate PR once this passes and merges.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Race introduced in #16709, which changed 'top' to 'true', so
there was only a narrow window in which '.State.ConmonPod'
would be valid. Remove the race.
Fixes: #17882
Signed-off-by: Ed Santiago <santiago@redhat.com>
If a container with an ID starting with "db1" exists, and a
container named "db1" also exists, and they are different
containers - if I run `podman inspect db1` the container named
"db1" should be inspected, and there should not be an error that
multiple containers matched the name or id "db1". This was
already handled by BoltDB, and now is properly managed by SQLite.
Fixes#17905
Signed-off-by: Matt Heon <mheon@redhat.com>
Add yet another environment variable for loading containers.conf.
When CONTAINERS_CONF_OVERRIDE is set, the specified config file
will be loaded last - even when CONTAINERS_CONF is set.
This mechanism is needed to preserve system settings and other
environment variables. Setting CONTAINERS_CONF will load only
the specified config file and ignore all system and user paths.
That makes testing hard as many Podman tests use CONTAINERS_CONF
for testing.
The intended use of CONTAINERS_CONF_OVERRIDE is to set it during tests
and point it to a specific configuration of Podman (e.g., netavark with
sqlite backend).
Similar needs have popped up talking to users in the automotive and
high-performance computing space. In a way, such a setting allows for
specifying a specific "flavor" of Podman while preserving all existing
settings on the system.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
We use the same endpoint for libpod and docker compat API. However as
reported docker returns "id" lowercase. Because we cannot break the
libpod API right now keep the output for the libpod endpoint and only
change the docker one.
To do so simply use two types that we can cast with different JSON tags.
Fixes#17869
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
skip in setup() if journald unavailable.
To be pedantic, this is overkill: some quadlet tests pass
because they don't run journald. Too bad.
Also skip a play-kube test that requires journal
Signed-off-by: Ed Santiago <santiago@redhat.com>
...and add a comment explaining why. The minimum, determined via
binary search, is actually 27! Anything under that will barf:
$ bin/podman run --ulimit nofile=26:26 --rm quay.io/libpod/testimage:20221018 true
Error: OCI runtime error: crun: openat2 `proc/sysrq-trigger`: Too many open files
Play it safe, go with 30.
(Does this seem alarming to anyone else, or am I the only one??)
Fixes: #17860
Signed-off-by: Ed Santiago <santiago@redhat.com>
If the volume source starts with . resolve the path relative to the
location of the unit file
Update the test code to allow verification of regex for the value in key
value arguments
Add the usage of relative paths to the volume and mount test cases
Update the man page
Signed-off-by: Ygal Blum <ygal.blum@gmail.com>
Since we can't guarantee when the worker queue will come
and clean up the service container in the remote case when
podman kube play --wait is called, cleanup the service container
at the end of PlayKubeDown() to ensure that it is removed right
after all the containers, pods, volumes, etc are removed.
[NO NEW TESTS NEEDED]
Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
As described in #17777, the `restart` on-failure action did not behave
correctly when the health check is being run by a transient systemd
unit. It ran just fine when being executed outside such a unit, for
instance, manually or, as done in the system tests, in a scripted
fashion.
There were two issue causing the `restart` on-failure action to
misbehave:
1) The transient systemd units used the default `KillMode=cgroup` which
will nuke all processes in the specific cgroup including the recently
restarted container/conmon once the main `podman healthcheck run`
process exits.
2) Podman attempted to remove the transient systemd unit and timer
during restart. That is perfectly fine when manually restarting the
container but not when the restart itself is being executed inside
such a transient unit. Ultimately, Podman tried to shoot itself in
the foot.
Fix both issues by moving the restart logic in the cleanup process.
Instead of restarting the container, the `healthcheck run` will just
stop the container and the cleanup process will restart the container
once it has turned unhealthy.
Fixes: #17777
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
The default_ulimits field is currently ignored in podman run commands.
This PR fixes this.
Fixes: https://github.com/containers/podman/issues/17396
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Just like we do with RUNTIME and NETWORK. Skipped for now in
system tests because there's no way yet to actually set the
database backend.
Also, in system test oneliner (first test), include DB
Signed-off-by: Ed Santiago <santiago@redhat.com>
Add debug logs from systemctl and journalctl in hope to get more data on
the Debian flakes tracked in #17796.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
This reverts commit 1c08f2edac3f9ecf128cf8da91276e963e6ad14c: the
original failure reported in #17287 persists:
[+1306s] not ok 453 podman networking with pasta(1) - TCP translated port range forwarding, IPv4, loopback
...
[+1306s] # 2023/03/15 14:33:33 socat[119870] E connect(8, AF=2 127.0.0.1:5127, 16): Interrupted system call
[+1306s] # xx
[+1306s] # #/vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
[+1306s] # #| FAIL: Mismatch between data sent and received
[+1306s] # #| expected: = xxx
[+1306s] # #| actual: xx
[+1306s] # #\^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
so keep two connections instead of three as long as I'm too dumb to
figure this out.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Transient mode means the DB should not persist, so instead of
using the GraphRoot we should use the RunRoot instead.
Signed-off-by: Matt Heon <mheon@redhat.com>
Two main changes:
- The transient state tests relied on BoltDB paths, change to
make them agnostic
- The volume code in SQLite wasn't retrieving and setting the
volume plugin for volumes that used one.
Signed-off-by: Matt Heon <mheon@redhat.com>
When streaming events, prevent returning duplicates after a log rotation
by marking a beginning and an end for rotated events. Before starting to
stream, get a timestamp while holding the event lock. The timestamp
allows for detecting whether a rotation event happened while reading the
log file and to skip all events between the begin and end rotation
event.
In an ideal scenario, we could detect rotated events by enforcing a
chronological order when reading and skip those detected to not be more
recent than the last read event. However, events are not always
_written_ in chronological order. While this can be changed, existing
event files could not be read correctly anymore.
Fixes: #17665
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
When we searching any image at a container registry,
--cert-dir and --creds could be required
as well as push, pull, etc.
Signed-off-by: Toshiki Sonoda <sonoda.toshiki@fujitsu.com>
When the service is running with trace log level it wraps the
`http.ResponseWriter` to log extra information. The problem is that the
new type does not keep all the functions from the embedded type.
Instead we have to implement them ourselves, however only Write() was
implemented. Thus `Hijack()`could not be called on the writer. To
prevent these issues we would implement all the interfaces that the
inner type supports (Header, WriteHeader, Flush, Hijack).
Fixes#17749
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
The goal of the wait_for_port() function is to return when the port is
bound. This is to make sure we wait for application startup time.
This can be seen in some comments of the callers.
Commit 7e3d04fb caused this regression while reworking the logic to read
ports from /proc. I doesn't seem to cause problems in CI, properly
because the function returns before the port is bound.
I have not seen any flakes related to this but I only see the ones on
PRs where I rerun tests so it is best to wait for Ed to take a look.
Also fixes the broken ipv4_to_procfs() which only passes one argument to
__ipv4_to_procfs(), this results in the ipv4 not beeing inverted.
Therefore all bind checks against a direct ipv4 did not work.
This function accepts only an ipv4 but one caller passes localhost
which is invalid.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
On cgroup v1 we need to mount only the systemd named hierarchy as
writeable, so we configure the OCI runtime to mount /sys/fs/cgroup as
read-only and on top of that bind mount /sys/fs/cgroup/systemd.
But when we use a private cgroupns, we cannot do that since we don't
know the final cgroup path.
Also, do not override the mount if there is already one for
/sys/fs/cgroup/systemd.
Closes: https://github.com/containers/podman/issues/17727
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
Wait for all generated services to be ready to be sure we can iron out
race conditions. Also disable rollbacks to make sure we can analyze
the error if restarting a service fails. This information may be
crucial to understand the flakes on Debian as tracked in #17607.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
To help debug #17607, turn off rollbacks for tests that do not require
rollbacks. Error when restarting the systemd units are then not
suppressed but returned which should give us more information about what
is going on the Debian systems.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
This reverts commit e33f4e0bc7, going
back to three connections (not two) for each range in TCP tests. I'm
not sure yet what caused the original issue, but it might be fixed
now. If it does, this fixes#17287.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Currently Podman prevents SELinux container separation,
when running within a container. This PR adds a new
--security-opt label=nested
When setting this option, Podman unmasks and mountsi
/sys/fs/selinux into the containers making /sys/fs/selinux
fully exposed. Secondly Podman sets the attribute
run.oci.mount_context_type=rootcontext
This attribute tells crun to mount volumes with rootcontext=MOUNTLABEL
as opposed to context=MOUNTLABEL.
With these two settings Podman inside the container is allowed to set
its own SELinux labels on tmpfs file systems mounted into its parents
container, while still being confined by SELinux. Thus you can have
nested SELinux labeling inside of a container.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Another followup to #17608. Nightly tests were hanging,
because /run/podman/podman.sock was hardcoded (bad idea
for rootless). Poor testing on my part.
Signed-off-by: Ed Santiago <santiago@redhat.com>
This case is fixed by passt commit bad252687271 ("conf, udp: Allow
any loopback address to be used as resolver") and the fix is now
available in packages included by the CI images.
Note that, depending on the resolver on the host, we might get
1.0.0.127.in-addr.arpa resolved to localhost, or simply NXDOMAIN for
it: accept a failure on the nslookup command, as long as we have a
response for 1.0.0.127.in-addr.arpa in the output. If we have any
response, that means we could talk to the resolver.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Add a hidden flag to set the database backend and plumb it into
podman-info. Further add a system test to make sure the flag and the
info output are working properly.
Note that the test may need to be changed once we settled on how
to test the sqlite backend in CI.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
The symptoms in #17607 point to some race since it does not always flake
on Debian (and Debian only). Hence, wait for the service to be ready
before building the image to make sure that the service is started with
the old image and that everything's in order.
Fixes: #17607
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Daniel J Walsh
OpenShift Merge Robot
danishprakash
Chris Evich
Ed Santiago
Paul Holzinger
Romain Geissler
Valentin Rothberg
Boaz Shuster
Ygal Blum
Hari Kannan
Matt Heon
Jake Correnti
Lokesh Mandvekar
Chetan Giradkar
Paul Holzinger
Erik Sjölund
Toshiki Sonoda
Alex Palaistras
renovate[bot]
Black-Hole1
David Gibson
Giuseppe Scrivano
Aditya R
Jan Hendrik Farr
Laurenz Kruty
Matthew Heon
Stefano Brivio
Urvashi Mohnani
Luca Di Maio
binghongtao
Miloslav Trmač
Gunjan Vyas
Tony Duan
Dawid Kulikowski
Nalin Dahyabhai
Dan Čermák
tomsweeneyredhat
openshift-ci[bot]
Andreas Kohn
rbagd
Jakob Ahrer
Matej Vasek
Ali Rizvi-Santiago
Sohan Kunkerkar
dependabot[bot]
Cedric Staniewski
Eng Zer Jun
Alex Jia
restitux