Commit Graph

1157 Commits

Author SHA1 Message Date
OpenShift Merge Robot 0a8113be7b
Merge pull request #18323 from edsantiago/container_exit_signaling
system tests: safer container-stop signaling
2023-04-25 05:07:29 -04:00
openshift-ci[bot] 3ecb174eee
Merge pull request #18320 from Luap99/ps-state
ps: --format {{.State}} match docker output
2023-04-24 19:56:32 +00:00
Ed Santiago faeed14f61 system tests: safer container-stop signaling
Having a container spin-wait on a /stop file, then exit, is
unsafe: 'podman exec $ctr touch /stop' can get sucked into
container cleanup before the exec terminates, resulting in
the podman-exec failing and hence the test failing.

Most existing instances of this pattern are unnecessary.
Replace those with just 'podman rm -f'.

When necessary, use a variety of safer alternatives.

Re-Closes: #10825 (already closed; this addresses remaining cases)

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-04-24 11:35:49 -06:00
Paul Holzinger c5a928c5b7
ps: --format {{.State}} match docker output
We should return the raw state string without any extra formatting in
this case.
`{{.Status}}` returns the nicely formatted string used in the default ps
output, e.g. `Up 2 seconds ago`, while `{{.State}}` returns the state as
string, e.g. `running`.

This matches the docker output and allows better use in scripts.

Fixes #18244

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2023-04-24 14:18:26 +02:00
Valentin Rothberg b4eecd908a test/system/260-sdnotify.bats: fix test flake
The `exec` session somestimes exits with 137 as the exec session races
with the cleanup process of the exiting container.  Fix the flake by
running a detached exec session.

Fixes: #10825
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2023-04-24 14:09:09 +02:00
openshift-ci[bot] 69ec2d6ae6
Merge pull request #18213 from rbagd/main
Fixes format inconsistencies with docker for certain history fields
2023-04-20 23:13:21 +00:00
Daniel J Walsh 81621ce8af
Specify format to buildah before commit
If user specifies commit --format, we were not setting it before
commit, this caused warning messages that made no sense to be
printed that made no sense.

Fixes: https://github.com/containers/podman/issues/17773

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2023-04-20 06:24:35 -04:00
rbagd bce38c1afb Updated system test to be easier to read
Following @edsantiago guidance,

* Additional explanations for each step of the test
* Timezone for tests normalized to UTC
* Smarter choice of separator and use of shell substring extraction

Signed-off-by: rbagd <mail@rbagd.eu>
2023-04-19 21:10:40 +02:00
rbagd 6e0cf93447 Fixes format inconsistencies with docker for certain history fields
Closes #17767
Closes #17768

System test for image list and history dates

* Changed field separator in the test to `;` for easier parsing
* Converted date output from image history and image list to be comparable

Signed-off-by: rbagd <mail@rbagd.eu>
2023-04-19 00:20:47 +02:00
Valentin Rothberg 41d5164125 fix remote start --filter
Fix a number of bugs wrt. filtering remote containers and how to
process specified names or IDs.  I _really_ do not like the duplication
between remote and local Podman but want to focus on fixing #18153
for now.

What I desire in the future is to consolidate all functionality of
looking up containers (all, latest, filters, specified names/IDs, etc.)
and for remote clients to just call containers/list etc.

Fixes: #18153
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2023-04-17 15:11:40 +02:00
OpenShift Merge Robot 2d16f43d4a
Merge pull request #18169 from edsantiago/systest_kubero_flake
system tests: fix race in kube-play read-only
2023-04-13 09:36:02 -04:00
OpenShift Merge Robot 5e6c0647c0
Merge pull request #18171 from edsantiago/instrument_copyhardlinks
system tests: address COPY-hardlink flake
2023-04-13 05:10:13 -04:00
Ygal Blum 4aced7e3eb Quadlet - do not set log-driver by default
Update e2e tests not to expect the flag
System tests - explicitly set the log driver to be able to parse the output

Signed-off-by: Ygal Blum <ygal.blum@gmail.com>
2023-04-13 09:14:40 +03:00
Ed Santiago 51b582d0ea system tests: address COPY-hardlink flake
Possible cause: on Debian, maybe because of fuse-overlayfs(??),
we sometimes see unexpected inode numbers.

This PR tightens the test logic, so it runs one 'stat' command
in only one podman invocation, then cross-checks multiple lines
of output. I don't know if this will really fix the flake, but
even if it doesn't, it will at least give us much more useful
diagnostic output than before.

And, as long as I'm in here, clean up test, remove duplication,
make error messages distinct (hence more useful), and comment.

Fixes: #17979

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-04-12 14:58:20 -06:00
Ed Santiago 31420112ce system tests: fix race in kube-play read-only
Use --restart=no on all created containers. Without this, all
containers spin forever and it's impossible to get a
reliable exit status.

As a side effort, clean up tests, make more robust and maintainable.

Fixes: #18047

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-04-12 13:56:54 -06:00
Ed Santiago f95276bfaf CI: enable sqlite system tests
In setup, write a containers.conf.d file with db_backend
as specified in .cirrus.yml.

This is actually much scarier and more achy-breaky than
merely "sqlite system tests": it enables sqlite in e2e
tests. ("But wait, we already do that!" -- no, not really.
sqlite in e2e is being done via --db-backend option, and
some podman commands in e2e do not use the standard options.
See #17904.

This is unlikely to get merged any time soon (March, maybe
even April) because sqlite is still too fragile; this will
trigger more flakes than are currently acceptable. Also,
the nasty auto-update flake seems to trigger much more
reliably with sqlite. We need that one fixed.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-04-12 07:16:57 -06:00
OpenShift Merge Robot ab3025568f
Merge pull request #18052 from sstosh/resource-rootless
Do not display the resource limits warning message
2023-04-12 06:40:08 -04:00
OpenShift Merge Robot 8a9386d898
Merge pull request #18145 from sohankunkerkar/registry-migration
*: migrate image registry to registry.k8s.io
2023-04-12 06:37:28 -04:00
OpenShift Merge Robot cf3374e2a8
Merge pull request #18140 from umohnani8/deployments
Read kube_generate_type from containers.conf
2023-04-11 12:57:01 -04:00
OpenShift Merge Robot 8c4838f6b4
Merge pull request #18083 from Luap99/pause-single-process
rootless: make sure we only use a single pause process
2023-04-11 11:05:39 -04:00
Sohan Kunkerkar 64ea5971a1 *: migrate image registry to registry.k8s.io
This change is a part of the bigger umbrella issue: https://github.com/kubernetes/k8s.io/issues/4780
Currently, we need this change to get in https://github.com/cri-o/cri-o/pull/6742,
which is failing in CI due to inconsistent vendoring.

Signed-off-by: Sohan Kunkerkar <sohank2602@gmail.com>
2023-04-11 10:30:43 -04:00
Valentin Rothberg 676486a856 test/system/252-quadlet.bats: fix flake
Wait for the expected logs to appear in the journal before using
`journalctl`.  #18132 is likely flaking because `journalctl` does
not yet see the container's logs.

Also force the test to use the `passthrough` log driver to make sure
`podman logs` continues being tests.

Fixes: #18132
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2023-04-11 15:31:40 +02:00
Urvashi Mohnani 1bf5bdc689 Read kube_generate_type from containers.conf
Use the kube_generate_type from the containers.conf as
the default value for the --type flag for kube generate.
Override the default when userexplicitly sets the --type
flag.

Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
2023-04-11 09:15:05 -04:00
Toshiki Sonoda 4f5f89cf88 Do not display the resource limits warning message
If resource limits is not set, do not display the following warning message:
`Resource limits are not supported and ignored on cgroups V1 rootless systems`

Ref: #17582

Signed-off-by: Toshiki Sonoda <sonoda.toshiki@fujitsu.com>
2023-04-11 19:30:59 +09:00
Paul Holzinger bab95de9a2
rootless: make sure we only use a single pause process
Currently --tmpdir changes the location of the pause.pid file. this
causes issues because the c code in pkg/rootless does not know about
that. I tried to fix this[1] by fixing the c code to not use the
shortcut. While this fix worked it will result in many pause processes
leaking in the integrration tests.

Commit ab88632 added this behavior but following the disccusion it was
never the intention that we end up having more than one pause process.
The issues that was trying to fix was caused by somthing else AFAICT,
the main problem seems to be that the pause.pid file parent directory
may not be created when we try to create the pid file so it failed with
ENOENT. This patch fixes it by creating this directory always and revert
the change to no longer depend on the tmpdir value.

With this commit we now always use XDG_RUNTIME_DIR/libpod/tmp/pause.pid
for all podman processes. This allows the c shortcut to work reliably
and should therefore improve perfomance over my other approach.

A system test is added to ensure we see the right behavior and that
podman system migrate actually stops the pause process. Thanks to Ed
Santiago for the improved test to make it work for both `catatonit` and
`podman pause`.

This should fix the issues with namespace missmatches that we can see in
CI as flakes.

[1] https://github.com/containers/podman/pull/18057

Fixes #18057

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2023-04-11 10:57:46 +02:00
Daniel J Walsh 1f2775cbdc
Merge pull request #18130 from flouthoc/bump-deps
vendor: bump `containers/(storage, buildah, image, common)`
2023-04-10 13:33:44 -04:00
Aditya R b624ad7572
test/system: expect 12 char for short id
After https://github.com/containers/buildah/pull/4660 buildah
spits a 12 letter image short id instead of 13 so lets honor that.

Signed-off-by: Aditya R <arajan@redhat.com>
2023-04-10 17:30:34 +05:30
Daniel J Walsh c4e79fc169
Fix up codespell errors
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2023-04-08 00:53:38 -04:00
OpenShift Merge Robot 430603950b
Merge pull request #18098 from xduugu/quadlet-userns
quadlet: add `UserNS` option key
2023-04-07 23:31:53 -04:00
Cedric Staniewski f6a50311c5 quadlet: add `UserNS` option key
The `UserNS` key will replace the `RemapGid`, `RemapUid`, `RemapUidSize`
and `RemapUsers` options which are therefore marked as deprecated by
this commit.

Closes #17984

Signed-off-by: Cedric Staniewski <cedric@gmx.ca>
2023-04-07 17:19:21 +02:00
Urvashi Mohnani d0ffb87925 Fix invalid pod name and hostname during kube generate
Kube generate on pods was not checking for any underscores
in the pod name so was creating a kube yaml with an invalid
pod name when there were underscores present.
The hostname for the pod is set to the podname by default. There
is no need to set that to the container's name or the pod name
again in the generated yaml. So removed that field unless a hostname
was set for the container by the user.

Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
2023-04-05 17:43:02 -04:00
Cedric Staniewski 443f8d89c9 quadlet: implement `Tmpfs` option
This commit adds an quadlet option `Tmpfs` which can be used to mount a
tmpfs in the container.

Closes #17907

Signed-off-by: Cedric Staniewski <cedric@gmx.ca>
2023-04-03 20:22:02 +02:00
OpenShift Merge Robot 3978347bea
Merge pull request #18021 from edsantiago/unverbosify_sleeploop
System tests: unverbosify a flake log
2023-04-03 12:22:15 -04:00
Ed Santiago 99ace19084 System tests: unverbosify a flake log
One of our oldest most frustrating flakes is #16091, "Timed
out waiting for BYE".

In #17489 we added some debug output to see if the problem
was a container hang of some sort. It does not seem to be
(see #17675), and the debug output makes it hard to read
failure logs, so let's remove it.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-04-03 07:25:53 -06:00
Urvashi Mohnani 4f90194068 Support Deployment generation with kube generate
The podman kube generate command can now generate a
Deployment kind when the --ype flag is set to deployment.
By default, a Pod spec will be generated if --type flag is
not set.
Add --replicas flag to kube generate to allow users to set
the value of replicas in the generated yaml when generating a
Deployment kind.
Add e2e and minikube tests for this feature.

Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
2023-03-31 13:34:38 -04:00
Valentin Rothberg 7f8d5e5654 speed up image listing
As found in #17828, image listing does not scale well with a growing
number of local images.  Make use of recent improvements in libimage
that allow for computing the dangling and parent data with _one_ layer
tree.  Prior, the layer tree had to be recomputed _twice_ for each
image.

[NO NEW TESTS NEEDED] as it's a non-functional performance change.

Fixes: #17828
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2023-03-30 09:45:44 +02:00
OpenShift Merge Robot 9369a3c336
Merge pull request #17963 from Luap99/slirp-dns-userns
fix slirp4netns resolv.conf ip with a userns
2023-03-28 21:57:03 +02:00
Urvashi Mohnani c335ff241a Add debug to --wait test
Add a debug line to the wait to test to see which container
is being left behind after the cleaup where the race is happening.

Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
2023-03-28 10:01:02 -05:00
OpenShift Merge Robot 365131e0b7
Merge pull request #17930 from ygalblum/quadlet-systemd-specifiers
Quadlet - treat paths starting with systemd specifiers as absolute
2023-03-28 16:18:54 +02:00
Paul Holzinger 81e5bffc32
fix slirp4netns resolv.conf ip with a userns
When a userns is set we setup the network after the bind mounts, at the
point where resolv.conf is generated we do not yet know the subnet.
Just like the other dns servers for bridge networks we need to add the
ip later in completeNetworkSetup()

Fixes https://bugzilla.redhat.com/show_bug.cgi?id=2182052

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2023-03-28 15:52:33 +02:00
Ygal Blum da96ff6cc7 Quadlet - treat paths starting with systemd specifiers as absolute
If a path (Yaml, ConfigMap, EnvFile) starts with a systemd path
specifier, treat the path as absolute
Add tests - unit, e2e and bats

Signed-off-by: Ygal Blum <ygal.blum@gmail.com>
2023-03-28 10:30:33 +03:00
Ed Santiago d85c8d7e84 system tests: use CONTAINERS_CONF_OVERRIDE
...not CONTAINERS_CONF. At least for most tests.

Nearly every system test currently using CONTAINERS_CONF=tmpfile
should be using CONTAINERS_CONF_OVERRIDE.

Simple reason: runtime (crun/runc), database_backend (bolt/sqlite),
logger, and other important settings from /etc/c.conf are not
usually written into the tmpfile. Those tests, therefore, are
not running podman as configured on the system.

Much more discussion: #15413

This PR is a prerequisite for enabling sqlite system tests. For
the sake of simplicity and sanity, I choose to submit the sqlite
switch as a separate PR once this passes and merges.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-03-27 15:18:09 -06:00
OpenShift Merge Robot 905dc6de48
Merge pull request #17895 from vrothberg/containers_conf_extra
add CONTAINERS_CONF_OVERRIDE
2023-03-27 16:48:44 -04:00
Ed Santiago 92e0efc5d0 system tests: fix racey sdnotify test
Race introduced in #16709, which changed 'top' to 'true', so
there was only a narrow window in which '.State.ConmonPod'
would be valid. Remove the race.

Fixes: #17882

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-03-27 10:47:49 -06:00
OpenShift Merge Robot 78f1ebb80b
Merge pull request #17796 from vrothberg/au-errors
auto update: return restart error
2023-03-27 09:23:40 -04:00
Valentin Rothberg c5fc44582f add CONTAINERS_CONF_OVERRIDE
Add yet another environment variable for loading containers.conf.
When CONTAINERS_CONF_OVERRIDE is set, the specified config file
will be loaded last - even when CONTAINERS_CONF is set.

This mechanism is needed to preserve system settings and other
environment variables.  Setting CONTAINERS_CONF will load only
the specified config file and ignore all system and user paths.
That makes testing hard as many Podman tests use CONTAINERS_CONF
for testing.

The intended use of CONTAINERS_CONF_OVERRIDE is to set it during tests
and point it to a specific configuration of Podman (e.g., netavark with
sqlite backend).

Similar needs have popped up talking to users in the automotive and
high-performance computing space.  In a way, such a setting allows for
specifying a specific "flavor" of Podman while preserving all existing
settings on the system.

Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2023-03-23 15:16:59 +01:00
Alex Jia 9be4a525f4 test: podman checkpoint/restore the latest container
Signed-off-by: Alex Jia <chuanchang.jia@gmail.com>
2023-03-22 22:00:37 +08:00
Daniel J Walsh 5f274e45f2
Run make codespell
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2023-03-21 16:00:54 -04:00
OpenShift Merge Robot d9cbfdde73
Merge pull request #17861 from edsantiago/bump_min_nfiles
New ulimit test: bump up minimum nfiles
2023-03-21 13:28:28 -04:00
Ed Santiago cf5df5b805 quadlet tests: skip on RHEL8 rootless
skip in setup() if journald unavailable.

To be pedantic, this is overkill: some quadlet tests pass
because they don't run journald. Too bad.

Also skip a play-kube test that requires journal

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-03-21 07:18:14 -06:00