podman/test/system
OpenShift Merge Robot 2718f54a29
Merge pull request #17729 from rhatdan/selinux
Support running nested SELinux container separation
2023-03-15 12:07:03 -04:00
..
000-TEMPLATE
001-basic.bats cobra: move engine shutdown to Execute 2023-01-10 13:30:41 +01:00
005-info.bats sqlite: add a hidden --db-backend flag 2023-03-02 13:43:11 +01:00
010-images.bats Logs follow-until tests: loosen checks 2023-02-22 06:24:51 -07:00
011-image.bats
012-manifest.bats Add hidden podman manifest inspect -v option 2022-11-11 15:22:31 -05:00
015-help.bats Cleanup: fix problems reported by shell lint 2022-09-15 20:10:34 -06:00
020-tag.bats system tests: prevent leading tabs 2023-02-15 13:50:17 -07:00
030-run.bats system tests: prevent leading tabs 2023-02-15 13:50:17 -07:00
032-sig-proxy.bats install sigproxy before start/attach 2023-02-13 15:13:42 +01:00
035-logs.bats fix "podman logs --since --follow" flake 2023-02-28 13:31:28 +01:00
037-runlabel.bats system tests: runlabel: use podman-under-test 2022-10-26 05:35:10 -06:00
040-ps.bats
045-start.bats System tests: fix three races 2022-09-14 10:49:18 -06:00
050-stop.bats Output messages display rawInput 2022-08-02 18:28:37 +09:00
055-rm.bats system tests: prevent leading tabs 2023-02-15 13:50:17 -07:00
060-mount.bats Fix spelling "read only" -> "read-only" 2022-07-02 08:37:43 +02:00
065-cp.bats system tests: prevent leading tabs 2023-02-15 13:50:17 -07:00
070-build.bats Skip buildx test with VFS podman storage driver 2023-02-22 10:44:03 -05:00
075-exec.bats System tests: reenable some skipped aarch64 tests 2022-09-21 14:07:22 -06:00
080-pause.bats
090-events.bats events: no duplicates when streaming during a log rotation 2023-03-15 10:28:16 +01:00
110-history.bats Add CreatedSince & CreatedAt format fields to podman image history 2022-04-27 15:15:03 -04:00
120-load.bats test/system: Move network-related helpers to helpers.network.bash 2022-11-08 00:16:36 +01:00
125-import.bats
130-kill.bats podman wait: return 0 if container never ran 2022-07-11 13:06:40 +02:00
140-diff.bats
150-login.bats test/system: Add, use tcp_port_probe() to check for listeners rather than binds 2022-11-08 00:16:36 +01:00
160-volumes.bats Must use mountlabel when creating builtin volumes 2023-03-09 12:36:52 -05:00
170-run-userns.bats Logs follow-until tests: loosen checks 2023-02-22 06:24:51 -07:00
180-blkio.bats
190-run-ipcns.bats Correct output when inspecting containers created with --ipc 2023-01-24 12:18:39 -05:00
200-pod.bats Unify --noheading and -n to be consistent on all commands 2022-12-19 13:18:43 -05:00
220-healthcheck.bats system tests: prevent leading tabs 2023-02-15 13:50:17 -07:00
250-systemd.bats cgroupns: private cgroupns on cgroupv1 breaks --systemd 2023-03-14 12:34:52 +01:00
251-system-service.bats system service --log-level=trace: support hijack 2023-03-14 14:33:52 +01:00
252-quadlet.bats Skip rootless CGv1 quadlet tests due to issue 2023-02-22 10:31:18 -05:00
255-auto-update.bats test/system/255-auto-update.bats: multiple services 2023-03-14 10:30:32 +01:00
260-sdnotify.bats kube play: set service container as main PID when possible 2023-02-10 13:31:03 +01:00
270-socket-activation.bats testimage: add iproute2 & socat, for pasta networking 2022-10-18 11:50:48 -06:00
271-tcp-cors-server.bats test/system: Move network-related helpers to helpers.network.bash 2022-11-08 00:16:36 +01:00
272-system-connection.bats test/system: fix wait_for_port() to wait for bind 2023-03-14 14:24:04 +01:00
280-update.bats System tests for podman-update 2022-09-01 15:42:35 -06:00
300-cli-parsing.bats
320-system-df.bats Build and use a newer systemd image 2023-01-02 13:26:46 -07:00
330-corrupt-images.bats Fix stutters 2022-09-10 07:52:00 -04:00
400-unprivileged-access.bats System tests: teardown: clean up volumes 2022-10-26 09:56:58 -06:00
410-selinux.bats Support running nested SELinux container separation 2023-03-13 14:21:12 -04:00
420-cgroups.bats System tests: fix three races 2022-09-14 10:49:18 -06:00
450-interactive.bats replace golint with revive linter 2022-04-22 15:12:33 +02:00
500-networking.bats podman inspect list network when using --net=host or none 2023-03-08 17:27:08 -05:00
505-networking-pasta.bats Revert "pasta: Use two connections instead of three in TCP range forward tests" 2023-03-13 20:40:15 +01:00
520-checkpoint.bats checkpoint restore: fix --ignore-static-ip/mac 2022-12-12 18:34:49 +01:00
600-completion.bats shell completion: fix problems with container path completion 2022-06-09 13:46:55 +02:00
610-format.bats Add podman system events alias to podman events 2022-11-11 12:57:49 -05:00
620-option-conflicts.bats New system tests for conflicting options 2022-12-21 06:23:29 -05:00
700-play.bats Must use mountlabel when creating builtin volumes 2023-03-09 12:36:52 -05:00
710-kube.bats Remove ReservedAnnotations from kube generate specification 2023-01-18 08:46:24 -05:00
750-trust.bats
800-config.bats system tests: prevent leading tabs 2023-02-15 13:50:17 -07:00
900-ssh.bats ssh system test: skip until it becomes a test 2022-11-21 08:02:00 -07:00
950-preexec-hooks.bats rootless: rename auth-scripts to preexec-hooks 2023-01-17 10:58:46 +01:00
README.md
TODO.md docs: update the podman logo 2022-08-07 09:11:53 +08:00
build-systemd-image e2e tests: stop using UBI images 2023-01-06 08:21:50 -07:00
build-testimage testimage: add iproute2 & socat, for pasta networking 2022-10-18 11:50:48 -06:00
helpers.bash cgroupns: private cgroupns on cgroupv1 breaks --systemd 2023-03-14 12:34:52 +01:00
helpers.network.bash test/system: fix wait_for_port() to wait for bind 2023-03-14 14:24:04 +01:00
helpers.systemd.bash System tests: quadlet network test 2023-01-03 17:25:58 +02:00
helpers.t System tests: assert(): friendlier failure messages 2023-02-16 17:47:10 -07:00

README.md

Quick overview of podman system tests. The idea is to use BATS, but with a framework for making it easy to add new tests and to debug failures.

Quick Start

Look at 030-run.bats for a simple but packed example. This introduces the basic set of helper functions:

  • setup (implicit) - resets container storage so there's one and only one (standard) image, and no running containers.

  • parse_table - you can define tables of inputs and expected results, then read those in a while loop. This makes it easy to add new tests. Because bash is not a programming language, the caller of parse_table sometimes needs to massage the returned values; 015-run.bats offers examples of how to deal with the more typical such issues.

  • run_podman - runs command defined in $PODMAN (default: 'podman' but could also be './bin/podman' or 'podman-remote'), with a timeout. Checks its exit status.

  • is - compare actual vs expected output. Emits a useful diagnostic on failure.

  • die - output a properly-formatted message to stderr, and fail test

  • skip_if_rootless - if rootless, skip this test with a helpful message.

  • skip_if_remote - like the above, but skip if testing podman-remote

  • random_string - returns a pseudorandom alphanumeric string

Test files are of the form NNN-name.bats where NNN is a three-digit number. Please preserve this convention, it simplifies viewing the directory and understanding test order. In particular, 00x tests should be reserved for a first-pass fail-fast subset of tests:

bats test/system/00*.bats || exit 1
bats test/system

...the goal being to provide quick feedback on catastrophic failures without having to wait for the entire test suite.

Running tests

To run the tests locally in your sandbox, you can use one of these methods:

  • make;PODMAN=./bin/podman bats ./test/system/070-build.bats # runs just the specified test
  • make;PODMAN=./bin/podman bats ./test/system # runs all
  • make;PODMAN=./bin/podman NETWORK_BACKEND=netavark bats ./test/system # Assert & enable netavark testing

To test as root:

  • $ PODMAN=./bin/podman sudo --preserve-env=PODMAN bats test/system

Analyzing test failures

The top priority for this scheme is to make it easy to diagnose what went wrong. To that end, podman_run always logs all invoked commands, their output and exit codes. In a normal run you will never see this, but BATS will display it on failure. The goal here is to give you everything you need to diagnose without having to rerun tests.

The is comparison function is designed to emit useful diagnostics, in particular, the actual and expected strings. Please do not use the horrible BATS standard of [ x = y ]; that's nearly useless for tracking down failures.

If the above are not enough to help you track down a failure:

Debugging tests

Some functions have dprint statements. To see the output of these, set PODMAN_TEST_DEBUG="funcname" where funcname is the name of the function or perhaps just a substring.

Requirements

The jq tool is needed for parsing JSON output.

Further Details

TBD. For now, look in helpers.bash; each helper function has (what are intended to be) helpful header comments. For even more examples, see and/or run helpers.t; that's a regression test and provides a thorough set of examples of how the helpers work.