Commit Graph

6654 Commits

Author SHA1 Message Date
Paul Holzinger 755a06aa44
test/e2e: add netns leak check
Like we do in system tests now check for netns leaks in e2e as well. Now
because things run in parallel and this dir is shared we cannot test
after each test only once per suite. This will be a PITA to debug if
leaks happen as the netns files do not contain the container ID and are
just random bytes (maybe we should change this?)

Fixes #23715

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2024-09-18 14:05:26 +02:00
Paul Holzinger 2d469e517d
test/system: netns leak check for rootless as well
This fixes the problem where even as root we check the netns files from
root. But in order to catch any rootless bugs we must check the rootless
files from $XDG_RUNTIME_DIR/netns.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2024-09-18 12:07:11 +02:00
openshift-merge-bot[bot] 1e9464c9b4
Merge pull request #23937 from edsantiago/test-crun-17
New VMs: test crun 1.17
2024-09-17 20:28:43 +00:00
openshift-merge-bot[bot] 4dfff40840
Merge pull request #23989 from edsantiago/enable-bats-parallel
CI: system tests: enable parallel tests
2024-09-17 19:30:57 +00:00
openshift-merge-bot[bot] 75369fd283
Merge pull request #23986 from mheon/fix_23981
Match output of Compat Top API to Docker
2024-09-17 19:06:13 +00:00
Ed Santiago 8402b6535f Misc minor test fixes
...for dealing with flakes in parallel mode

Signed-off-by: Ed Santiago <santiago@redhat.com>
2024-09-17 11:19:37 -06:00
Ed Santiago 7fcf94d7b5 Add network namespace leak check
Signed-off-by: Ed Santiago <santiago@redhat.com>
2024-09-17 11:19:37 -06:00
Ed Santiago b3da5be2b1 Add workaround for buildah parallel bug
Need --layers=false in podman build, otherwise a buildah race
can trigger "layer not known" failures:

   https://github.com/containers/buildah/issues/5674

Signed-off-by: Ed Santiago <santiago@redhat.com>
2024-09-17 11:19:37 -06:00
Ed Santiago 5fc3de5583 registry: lock start attempts
When running parallel, multiple tests could be trying to start
the registry at once. Make this parallel-safe.

Also, use a safer port range for the registry. Something
outside of /proc/sys/net/ipv4/ip_local_port_range

Sorry, I'm including a FIXME section that I haven't investigated
deeply enough.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2024-09-17 11:19:37 -06:00
Ed Santiago bf6131780a Update system test template and README
Add a few best-practices examples, and add a whole section
describing the dos and donts of writing parallel-safe tests.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2024-09-17 11:19:37 -06:00
Ed Santiago 6502e30cfd bats log: differentiate parallel tests from sequential
For tests run in parallel, show file number as |nnn| (vs [nnn])

Teach logformatter to distinguish the two, adding 'p' to anchors
in parallel tests. Necessary because in this scheme we run bats
twice, thus see 'ok 1' twice, and we want to differentiate them.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2024-09-17 11:19:37 -06:00
Ed Santiago bcffa9ce30 clean_setup: create pause image
Workaround for #23292, where simultaneous 'pod create' commands
will all start a podman-build of the pause image, but only
one of them will be tagged, and the others will leak <none>
images.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2024-09-17 11:19:36 -06:00
Ed Santiago 812c7e9436 CI: make 012-manifest parallel-safe
Signed-off-by: Ed Santiago <santiago@redhat.com>
2024-09-17 10:35:01 -06:00
Matt Heon e04668c8ca Match output of Compat Top API to Docker
We were only splitting on tabs, not spaces, so we returned just a
single line most of the time, not an array of the fields in the
output of `ps`. Unfortunately, some of these fields are allowed
to contain spaces themselves, which makes things complicated, but
we got lucky in that Docker took the simplest possible solution
and just assumed that only one field would contain spaces and it
would always be the last one, which is easy enough to duplicate
on our end.

Fixes #23981

Signed-off-by: Matt Heon <mheon@redhat.com>
2024-09-17 11:34:22 -04:00
Ed Santiago d571ca6536 system test parallelization: enable two-pass approach
For the past two months we've been splitting system tests
into two categories: those that CAN be run in parallel,
and those that CANNOT. Much work has been done to replace
hardcoded names (mycontainer, mypod) with safename().
Hundreds of test runs, in CI and on Ed's laptop, have
proven this approach viable.

make {local,remote}system now runs in two steps: first
the serial ones, then the parallel ones. hack/bats will
now recognize the 'ci:parallel' tag and add --jobs (nprocs).

This requires some tweaking of leak_check, because there
can be umpteen tests running (affecting image/container/pod/etc
state) when any given test completes.

Rules for enabling parallelization in tests:

   * use unique container/pod/volume/network names (safename)
   * do not run 'podman rm -a' or 'rmi -a'
   * never use the -l (--latest) option
   * do not run 'podman ps/images' and expect precise output

Signed-off-by: Ed Santiago <santiago@redhat.com>
2024-09-17 09:25:02 -06:00
openshift-merge-bot[bot] f4a08f46b7
Merge pull request #23959 from auyer/hide-secrets-from-container-inspect
Hide secrets from container inspect command
2024-09-17 13:00:18 +00:00
Ed Santiago 8d119f0c2d New VMs: test crun 1.17
...and remove one old skip() for older debian, but leave
two others in place and mark that they're still a problem.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2024-09-17 06:51:36 -06:00
openshift-merge-bot[bot] 214e64e287
Merge pull request #23979 from edsantiago/workaround-remote-events-flake
CI: e2e: workaround for events out-of-sequence flake
2024-09-17 12:49:19 +00:00
openshift-merge-bot[bot] a4794bc9c6
Merge pull request #23977 from giuseppe/fix-permissions-copyup-volume-userns
libpod: convert owner IDs only with :idmap
2024-09-17 12:46:32 +00:00
Rafael Passos a5e9b4d126 libpod: hides env secrets from container inspect
Replaces env values supplied from podman secrets,
returns ******* instead

Fixes: #23788

Signed-off-by: Rafael Passos <rafael@rcpassos.me>
2024-09-17 09:12:39 -03:00
Ed Santiago 111a4bbe71 CI: e2e: workaround for events out-of-sequence flake
podman-remote events are not flushed, so order is not guaranteed.
This results in CI flakes. Only on Debian, for reasons unknown.

Make the network-connection events test more lenient when remote.

Closes: #23634 (but does not actually fix it)

Signed-off-by: Ed Santiago <santiago@redhat.com>
2024-09-17 06:12:26 -06:00
Giuseppe Scrivano 432325236b
libpod: convert owner IDs only with :idmap
convert the owner UID and GID into the user namespace only when
":idmap" mount is used.

This changes the behaviour of :idmap with an empty volume.  Now the
existing directory ownership is copied up as in the other case.

Closes: https://github.com/containers/podman/issues/23347

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2024-09-17 12:38:53 +02:00
renovate[bot] 1a1001d824
fix(deps): update module github.com/cpuguy83/go-md2man/v2 to v2.0.5
Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-09-16 18:26:51 +00:00
openshift-merge-bot[bot] e315697ed3
Merge pull request #23966 from containers/renovate/setuptools-75.x
chore(deps): update dependency setuptools to ~=75.1.0
2024-09-16 17:12:23 +00:00
renovate[bot] c436a9b658
chore(deps): update dependency setuptools to ~=75.1.0
Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-09-16 13:17:21 +00:00
Ed Santiago c6616004f1 CI: make 260-sdnotify parallel-safe
Use safename. Add ci:parallel tags. Do not remove pause image
nor kube network.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2024-09-16 05:04:24 -06:00
renovate[bot] 95833a6207
chore(deps): update dependency setuptools to v75
Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-09-15 16:55:11 +00:00
openshift-merge-bot[bot] d4cda112f1
Merge pull request #23921 from edsantiago/safename-710
CI: make 710-kube parallel-safe
2024-09-13 12:41:54 +00:00
openshift-merge-bot[bot] 421a80bcb7
Merge pull request #23908 from edsantiago/safename-505
CI: make 505-pasta parallel safe
2024-09-13 12:39:11 +00:00
openshift-merge-bot[bot] 29f75000dd
Merge pull request #23916 from edsantiago/safename-320
CI: mark 320-system-df *NOT* parallel safe
2024-09-13 12:33:41 +00:00
openshift-merge-bot[bot] 7764bea981
Merge pull request #23819 from l0rd/kube-play-image-type-volumes
Add `kube play` support for volumes of type image
2024-09-11 18:32:24 +00:00
openshift-merge-bot[bot] c66d46c39a
Merge pull request #23899 from edsantiago/new-vms
Bump VMs. ShellCheck is now built-in
2024-09-11 13:46:14 +00:00
Ed Santiago e61682f50e CI: make 710-kube parallel-safe
Use safename. Add ci:parallel tags. Use a random port, not
hardcoded 9999. Do not remove pause image. And especially
do not "rm -a" anything.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2024-09-10 14:49:14 -06:00
openshift-merge-bot[bot] c38c197c71
Merge pull request #23907 from edsantiago/safename-020
CI: make 020-tag parallel-safe
2024-09-10 19:09:45 +00:00
Ed Santiago 0ff89a00af CI: mark 320-system-df *NOT* parallel safe
...because it requires 100% control and knowledge of the
state of all images, containers, and volumes.

Use safename anyway, just in case we ever have a leak from here.
I'm finding safename sooooooo helpful when reading journal.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2024-09-10 08:48:44 -06:00
Mario Loriedo db12343e27 Add kube play support for image volume source
Signed-off-by: Mario Loriedo <mario.loriedo@gmail.com>
2024-09-10 12:37:06 +00:00
openshift-merge-bot[bot] ff0f6eb8c6
Merge pull request #23906 from edsantiago/safename-410
CI: make 410-selinux parallel-safe
2024-09-10 10:39:45 +00:00
renovate[bot] a83fef34a9
fix(deps): update module golang.org/x/tools to v0.25.0
Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-09-09 21:51:16 +00:00
Ed Santiago 22ec8ea06d CI: make 505-pasta parallel safe
Add ci:parallel tags; move one non-parallel-safe test to
another networking-test file; and a few drive-by fixes

Signed-off-by: Ed Santiago <santiago@redhat.com>
2024-09-09 14:02:48 -06:00
Ed Santiago 18932e0339 CI: make 020-tag parallel-safe
Use safename, with guaranteed-adjacent image names

Signed-off-by: Ed Santiago <santiago@redhat.com>
2024-09-09 13:37:57 -06:00
Ed Santiago a165289574 CI: make 410-selinux parallel-safe
Use safename for containers and pods. Add ci:parallel tags.
And reenable distro-integration tests that had been skipped
due to a container-selinux bug that is now fixed.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2024-09-09 13:09:37 -06:00
openshift-merge-bot[bot] 649730c9a1
Merge pull request #23887 from Luap99/sort-tags
podman images: sort repository with tags
2024-09-09 16:39:15 +00:00
Ed Santiago 359092e463 Bump VMs. ShellCheck is now built-in
Minor bump. Fedora VMs now include ShellCheck, so we can
remove the 'dnf install' at CI run time.

Also, FWIW, Debian *vark are now at 1.12 (from 1.9)

VMs built in https://github.com/containers/automation_images/pull/385

Signed-off-by: Ed Santiago <santiago@redhat.com>
2024-09-09 06:16:20 -06:00
Paul Holzinger a1e6603133
libpod: make use of new pasta option from c/common
pasta added a new --map-guest-addr to option that maps a to the actual
host ip. This is exactly what we need for host.containers.internal
entry. So we now make use of this option by default but still have to
keep the exclude fallback because the option is very new and some
users/distros will not have it yet.

This also fixes an issue where the --dns-forward ip were not used when
using the bridge network mode, only useful when not using aardvark-dns
as this used the proper ips there already from the rootless netns
resolv.conf file.

Fixes #19213

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2024-09-06 14:43:18 +02:00
Paul Holzinger 0abbcfa50a
podman images: sort repository with tags
When you sort by repository a user most likely also want the tags to be
sorted as well. At the very least to get a stable output as the order
could be changed pull podman tag/pull even if they keep using the same
tag name.

Fixes #23803

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2024-09-06 14:17:17 +02:00
openshift-merge-bot[bot] dde031076b
Merge pull request #23722 from umohnani8/jobs
Add support for Job to kube generate & play
2024-09-06 09:02:55 +00:00
Ed Santiago 133bbd3769 Fix CI
Collision between new spellcheck CI hook and a parallel typo

Signed-off-by: Ed Santiago <santiago@redhat.com>
2024-09-05 13:55:34 -06:00
openshift-merge-bot[bot] ae14dff812
Merge pull request #23398 from yarikoptic/enh-codespell
Add codespell support (config, workflow to detect/not fix) and make it fix few typos
2024-09-05 18:53:25 +00:00
openshift-merge-bot[bot] 094fed7dbe
Merge pull request #23846 from flouthoc/prune-cleancachemount
prune: add `--build-cache` to support clearing build cache using `CleanCacheMount`
2024-09-05 18:47:56 +00:00
Yaroslav Halchenko c203c48c70 Detect and fix typos using codespell
Signed-off-by: Yaroslav Halchenko <debian@onerussian.com>
2024-09-05 13:56:39 -04:00