podman/test/buildah-bud

buildah-bud tests under podman

This directory contains tools for running 'buildah bud' tests under podman. The key concept of the workflow is:

• Pull buildah @ version specified in go.mod
• Apply a small set of patches to buildah's tests directory, such that
  • BATS will use 'podman build' instead of 'buildah bud'; and
  • some not-applicable-under-podman tests are skipped

It's a teeny bit more complicated than that, but that's really most of what you need to know for most purposes. The tests run in podman CI, and for the most part are expected to just pass.

Troubleshooting

If you're reading this, it's probably because something went wrong. At the time of this writing (March 2021, initial commit) it is impossible to foresee what typical failures will look like, but my prediction is that they will fit one of two categories:

• Failure when vendoring new buildah (e.g., by dependabot)
• Other failure

Let's examine those in reverse order:

Failure when not vendoring

Aside from flakes, my only guess here is that you broke 'podman build'. If this is the case, it is very likely that you are aware of what you did; and if this is the case, your change likely falls into one of these two categories:

• "OOPS! I didn't mean to break that". Solution: fix it!
• "Uh, yeah, this is deliberate, and we choose to be incompatible with buildah". In this case, you'll need to skip or edit the failing test(s); see below.

If neither of those is the case, then I'm sorry, you're on your own. When you figure it out, please remember to update these instructions.

Failure when vendoring new buildah

This is what I predict will be the usual case; and I predict that failures will fall into one of two bins:

• failure to apply the patch
• failure because there are new buildah tests for functionality not in podman

In either case, the process for solving is the same:

• Start with a checked-out podman tree with the failing PR applied
• run ./test/buildah-bud/run-buildah-bud-tests

Presumably, something will fail here. Whatever the failure, your next step is:

• cd test-buildah-v<TAB> (this is a new directory created by the script)

If the failure was in git am, solve it (left as exercise for the reader).

If the failure was in tests run, solve it (either by adding skips to failing tests in bud.bats, or less preferably, by making other tweaks to the test code).

You now have modified files. THOSE SHOULD ONLY BE test/bud.bats or test/helpers.bash! If you changed any other file, that is a sign that something is very wrong!

Commit your changes: git commit --all --amend

Push those changes to the podman repo: ./make-new-buildah-diffs

cd back up to the podman repo

As necessary, rerun run-buildah-bud-tests. You can use --no-checkout to run tests immediately, without rerunning the git checkout.

If you're happy with the diffs, git add the modified .diff file and submit it as part of your PR.