automation-tests/test/apiv2
Ed Santiago 7b51acd4c4 APIv2 tests: fail on syntax/logic errors
(i.e. not test failures, but actual programming bugs).

We've had a number of syntax errors creep into this test, usually
caused by a missing backslash on a test command. I've long wanted
to 'set -e' but that causes other problems. This PR introduces
error handling via 'trap', with useful diagnostics on failure.

This PR also catches and fixes two previously-unknown bugs that
were causing tests to not actually run.

And, since /events takes eons on my high-uptime laptop, add /since

Signed-off-by: Ed Santiago <santiago@redhat.com>
2021-12-14 18:28:50 -07:00
..
python Implement top streaming for containers and pods 2021-11-02 08:11:14 -07:00
00-TEMPLATE Tests for API v2 2020-01-17 09:59:22 -07:00
01-basic.at APIv2 tests: fail on syntax/logic errors 2021-12-14 18:28:50 -07:00
10-images.at APIv2 tests: fail on syntax/logic errors 2021-12-14 18:28:50 -07:00
12-imagesMore.at compat API: allow enforcing short-names resolution to Docker Hub 2021-11-30 14:22:52 +01:00
20-containers.at APIv2 tests: fail on syntax/logic errors 2021-12-14 18:28:50 -07:00
22-stop.at apiv2 tests: finally fix POST as originally intended 2021-03-10 05:24:44 -07:00
23-containersArchive.at Implement --archive flag for podman cp 2021-07-01 12:01:46 +02:00
25-containersMore.at Trim white space from /top endpoint results 2021-03-30 10:42:06 -07:00
26-containersWait.at fix: response body of containers wait endpoint 2021-05-18 20:52:09 +02:00
30-volumes.at Add until filter to volume ls filters list 2021-07-22 00:01:07 +02:00
35-networks.at apiv2 tests: refactor complicated curls 2021-12-14 12:10:19 -07:00
40-pods.at infra container: replace pause with catatonit 2021-11-15 12:53:25 +01:00
44-mounts.at System test cleanup 2021-03-15 15:27:06 -06:00
45-system.at apiv2 tests: finally fix POST as originally intended 2021-03-10 05:24:44 -07:00
50-secrets.at Add filtering functionality to http api secrets list 2021-09-03 10:29:31 +02:00
60-auth.at Fix /auth compat endpoint 2021-09-15 20:01:02 +02:00
70-short-names.at compat API: allow enforcing short-names resolution to Docker Hub 2021-11-30 14:22:52 +01:00
README.md apiv2 tests: refactor complicated curls 2021-12-14 12:10:19 -07:00
containers.conf compat API: allow enforcing short-names resolution to Docker Hub 2021-11-30 14:22:52 +01:00
test-apiv2 APIv2 tests: fail on syntax/logic errors 2021-12-14 18:28:50 -07:00

README.md

API v2 tests

This directory contains tests for the podman version 2 API (HTTP).

Tests themselves are in files of the form 'NN-NAME.at' where NN is a two-digit number, NAME is a descriptive name, and '.at' is just an extension I picked.

Running Tests

The main test runner is test-apiv2. Usage is:

$ sudo ./test-apiv2 [NAME [...]]

...where NAME is one or more optional test names, e.g. 'image' or 'pod' or both. By default, test-apiv2 will invoke all *.at tests.

test-apiv2 connects to localhost only and via TCP. There is no support here for remote hosts or for UNIX sockets. This is a framework for testing the API, not all possible protocols.

test-apiv2 will start the service if it isn't already running.

Writing Tests

The main test function is t. It runs curl against the server, with POST parameters if present, and compares return status and (optionally) string results from the server:

t GET /_ping 200 OK
  ^^^ ^^^^^^ ^^^ ^^
  |   |      |   +--- expected string result
  |   |      +------- expected return code
  |   +-------------- endpoint to access
  +------------------ method (GET, POST, DELETE, HEAD)


t POST libpod/volumes/create name=foo 201 .ID~[0-9a-f]\\{12\\}
       ^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^ ^^^ ^^^^^^^^^^^^^^^^^^^^
       |                     |        |   JSON '.ID': expect 12-char hex
       |                     |        +-- expected code
       |                     +----------- POST params
       +--------------------------------- note the missing slash

Notes:

  • If the endpoint has a leading slash (/_ping), t leaves it unchanged. If there's no leading slash, t prepends /v1.40. This is a simple convenience for simplicity of writing tests.

  • When method is POST, the argument(s) after the endpoint may be a series of POST parameters in the form 'key=value', separated by spaces: t POST myentrypoint 200 ! no params t POST myentrypoint id=$id 200 ! just one t POST myentrypoint id=$id filter='{"foo":"bar"}' 200 ! two, with json t POST myentrypoint name=$name badparam='["foo","bar"]' 500 ! etc... t will convert the param list to JSON form for passing to the server. A numeric status code terminates processing of POST parameters. ** As a special case, when one POST argument is a string ending in .tar, t will invoke curl with --data-binary @PATH and set Content-type: application/x-tar. This is useful for build endpoints. (To override Content-type, simply pass along an extra string argument matching application/*): t POST myentrypoint /mytmpdir/myfile.tar application/foo 400

  • The final arguments are one or more expected string results. If an argument starts with a dot, t will invoke jq on the output to fetch that field, and will compare it to the right-hand side of the argument. If the separator is = (equals), t will require an exact match; if ~ (tilde), t will use expr to compare.