podman/hack
Paul Holzinger 81c90f51c2
test/system: speed up basic_{setup,teardown}()
While these are not really slow they still take about 100-250ms if I
time this locally. Given they are run for every test this adds up
quickly. Looking at CI logs I can see the timings for skipped
tests are all in 600ms range. So I think it is safe to assume that these
functions need to get faster.

We have over 670 test cases currently so we talk about over 400s spend
in these functions in CI. This allows for big gains.

Now overall this is a tricky trade of, while all tests should cleanup
after themselves there is no guarantee for that as such errors can be
leaked into other tests making debugging much harder. To work at least a
bit against this teardown checks if the test was successful and only
skips the podman commands bases on that. Without it a single flake could
cause all following tets to fail.

As such this commit does the proper setup once one suite start then only
after a test failed.

In order for this to work at all we have to fix all leaks first, see
previous commits. And then for the future keep a very strong eye on
this during reviews.

Also add a PODMAN_BATS_LEAK_CHECK option

By default test must cleanup themselves and to speed up CI we no longer
do any cleanup in teardown by default. However there is still many cases
where we might have to debug a leak so add a new PODMAN_BATS_LEAK_CHECK
env option that can be set and should cause teardown to fail if the test
did not cleanup properly.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2024-06-18 11:06:50 +02:00
..
perf hack/perf/system-df.sh: add `df` benchmarks 2023-08-23 14:17:27 +02:00
podman-registry-go Bump Go module to v5 2024-02-08 09:35:39 -05:00
apparmor_tag.sh Use `bash` binary from env instead of /bin/bash for scripts 2020-08-17 10:42:23 +02:00
bats test/system: speed up basic_{setup,teardown}() 2024-06-18 11:06:50 +02:00
branch_commits.rb Add script for identifying commits in release branches 2021-05-17 16:40:35 -04:00
btrfs_installed_tag.sh Use CPP, CC and flags in dep check scripts 2020-11-06 10:44:27 +01:00
btrfs_tag.sh Use CPP, CC and flags in dep check scripts 2020-11-06 10:44:27 +01:00
buildah-vendor-treadmill buildah treadmill: cleaner YAML, uglier script 2023-12-07 17:55:16 -07:00
check_root.sh Cleanup: fix problems reported by shell lint 2022-09-15 20:10:34 -06:00
fork_exec_snoop.bt Add eBPF snooper that traces the entire fork/exec graph of podman 2023-04-20 11:30:47 +02:00
get_ci_vm.sh Cirrus: Support EC2 instances in hack/get_ci_vm.sh 2022-08-03 14:25:09 -04:00
golangci-lint.sh System tests: add `podman system check` tests 2024-06-04 10:00:37 -04:00
install_catatonit.sh cirrus: force-install catatonit 2021-11-15 12:53:25 +01:00
install_golangci.sh CI: retry the golangci install 2023-03-27 10:53:17 -06:00
libdm_tag.sh Use CPP, CC and flags in dep check scripts 2020-11-06 10:44:27 +01:00
libsubid_tag.sh Fix libsubid detection 2022-09-23 09:58:39 +01:00
make-and-check-size Fix size-check to display more context 2022-04-21 14:07:20 -04:00
man-page-checker add a podman-compose command 2023-07-24 19:23:04 +02:00
man-page-table-check Man pages: fix broken tables 2023-05-26 06:08:14 -06:00
markdown-preprocess make hack/markdown-preprocess parallel-safe 2023-02-02 05:32:38 -07:00
markdown-preprocess-review Man pages: refactor common options: cert-dir 2022-08-22 05:52:20 -06:00
markdown-preprocess.t markdown-preprocess: almost complete OO rewrite 2022-10-13 15:56:52 -06:00
podman-commands.sh Add shell completion with cobra 2020-11-12 11:38:31 +01:00
podman-registry system tests: refactor registry code 2023-07-04 15:26:55 -06:00
podman-socat Use url with scheme and path for the unix address 2023-09-11 07:55:12 +02:00
podmansnoop hack/podmansnoop: print arguments 2022-11-30 10:38:08 +01:00
podmanv2-retry Use `bash` binary from env instead of /bin/bash for scripts 2020-08-17 10:42:23 +02:00
swagger-check switch from "kube/play" endpoint to "play/kube" endpoint. 2022-08-02 12:09:59 +01:00
systemd_tag.sh Use CPP, CC and flags in dep check scripts 2020-11-06 10:44:27 +01:00
tree_status.sh hack/tree_status.sh: print diff at the end 2022-10-14 09:39:37 +02:00
xref-docker-options Formalize our compare-against-docker mechanism 2022-11-10 05:47:32 -07:00
xref-helpmsgs-manpages Add os, arch, and ismanifest to libpod image list 2024-04-11 08:46:37 -05:00
xref-helpmsgs-manpages.t xref-helpmsgs-manpages: cross-check Commands.rst 2024-03-18 09:09:25 -06:00
xref-quadlet-docs Quadlet: ensure all keys are documented 2024-01-18 06:17:57 -07:00