After merging #13998 it was observed that the `docker-py` task was still
failing with the same error on `main`. The original quick-fix had
placed the full-build (`make`) call too late in the process. This
commit moves it up to right before the `make install` call which was
resulting in an error.
Again, a further future commit is planned to re-work and simplify the
entire cache setup. This is only a quick fix to make branch-builds
pass.
Signed-off-by: Chris Evich <cevich@redhat.com>
Newer versions of git (like `2.35`) fail on certain operations (like
`rebase` and `am`) without a local identity. Add a fake one from the
start, with a clearly identifiable test-value to avoid problems at
runtime.
Signed-off-by: Chris Evich <cevich@redhat.com>
Add a pair of new Cirrus test suites using Compose v2 instead of
Compose v1 (as is currently packaged in Fedora). They work
identically, and run the same tests, as the Compose v1 tests, but
with the new v2 binary instead.
[NO NEW TESTS NEEDED] This adds an entire Cirrus suite...
Signed-off-by: Matthew Heon <mheon@redhat.com>
* Add configuration to add report header for python client used in tests
* Move report headers into the individual test runners vs runner.sh
Signed-off-by: Jhon Honce <jhonce@redhat.com>
Mainly this is to confirm some changes needed for the podman-py CI setup
don't disrupt operations here. Ref:
https://github.com/containers/automation_images/pull/111
Also includes a minor steup fix WRT setting up for test-rpm build.
Signed-off-by: Chris Evich <cevich@redhat.com>
Nightly builds were failing on CI ever since the Makefile change to have
install target independent of build targets.
See: e4636ebdc8
This commit ensures everything is built before installation.
[NO NEW TESTS NEEDED]
Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
Building from source would involve separate `make` and `make install`
steps.
This removes a lot of unnecessary `-nobuild` targets which were
otherwise needed for packaging.
This commit also removes spec files for unused copr jobs.
[NO NEW TESTS NEEDED]
Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
This involves a minor code-change so the download/install can run in a
loop for the two different repositories and binaries. Given everything
is exactly the same except the URLs and names.
Signed-off-by: Chris Evich <cevich@redhat.com>
This PR adds the CI mechanisms to obtain the latest upstream netavark
binary, and set a magic env-var to indicate e2e tests should execute
podman with `--network-driver=netavark`. A future commit implement
this functionality within the e2e tests.
Due to the way the new environment is enabled, the standard task name
is too long for github to display without adding ellipsis. Force the
custom task name `Netavark Integration` to workaround this. At some
future point, when netavark is more mainstream/widely supported, this
custom task and upstream binary install can simply be removed - i.e.
netavark will simply be used by default in the normal e2e tasks.
Signed-off-by: Chris Evich <cevich@redhat.com>
viz, rootful system tests. The rootless account will be
used by image-scp tests.
Unfortunately, having ssh available means the system-connection
tests will start running, which is very bad because they will
fail, because system connection doesn't actually work (long story).
Add a few more checks to prevent this test from running.
Signed-off-by: Ed Santiago <santiago@redhat.com>
The Fedora 35 cloud images have switched to UEFI boot with a GPT
partition. Formerly, all Fedora images included support for runtime
re-partitioning. However, the requirement to test alternate storage
has since been dropped/removed. Rather than maintain a disused
feature, and supporting scripts, these Fedora VM images have reverted
to the default: Automatically resize to 100% on boot.
Signed-off-by: Chris Evich <cevich@redhat.com>
VM Images created as of this commit contain the new/required version.
Remove the `--force` install, but retain the hack script's ability to
support this in the future.
Signed-off-by: Chris Evich <cevich@redhat.com>
In F35 the hard-coded default (from
containers-common-1-32.fc35.noarch) is 'journald' despite
the upstream repository having this line commented-out.
Containerized integration tests cannot run with 'journald'
as there is no daemon/process there to receive them.
Signed-off-by: Chris Evich <cevich@redhat.com>
Don't use reexec for the rootlessport process, instead make it a
separate binary to reduce the memory usage. The problem with reexec is
that it will import all packages that podman uses and therefore loads a
lot of stuff into the heap. The rootlessport process however only needs
the rootlesskit library.
The memory usage is a concern since the rootlessport process will spawn
two process per container which has ports forwarded. The processes stay
until the container dies. On my laptop the current reexec version uses
47800 KB RSS. The new separate binary only uses 4540 KB RSS. This is
more than a 90% improvement.
The Makefile has been updated to compile the new binary and install it
to the libexec directory.
Fixes#10790
[NO TESTS NEEDED]
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Original workaround https://github.com/containers/podman/pull/11821
During VM image build, a number of packages are downloaded but not
installed, since they may interfere with some testing. Then at runtime,
where required, the packages are installed from cache and used.
However, between image build and runtime it's possible the repository
contents change, which will invalidate the package cache. Since the
`--no-download --ignore-missing` options were used, the install will
fail.
Ref: https://github.com/containers/automation_images/issues/95
Fortunately, when it comes to the docker packages, no other dependencies
are required and so `apt-get` isn't required. Switch to using a simple
dpkg install command on the necessary files. If this ever breaks due
to new dependencies, the list of files may simply be updated.
Signed-off-by: Chris Evich <cevich@redhat.com>
It looks like the containerd.io package is not present anymore in the
package cache which ultimately breaks CI since it's a requirement for
docker.
Hence, download the few packages instead of relying on the cache.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Add execution of the downstream gitlab-runner tests using
rootless podman through the magic of socket-level
docker compatibility. Include a comment suggesting how
to temporarily disable the test in case it fails beyond
podman code scope.
Signed-off-by: Chris Evich <cevich@redhat.com>
Rootless cni with ipv6 needs the `ip6_tables` module loaded, normally
the cni plugins will load this module but as rootless it does not have
the necessary permission to do so. Therefore we load it manually.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
This becomes a problem on hosts with upgraded policies. Ref:
https://github.com/containers/podman/issues/10522
Also, made a small change to compose-test setup to reduce runtime.
Signed-off-by: Chris Evich <cevich@redhat.com>
Significant bitrot results in almost immediate test failure. This
commit adds only the very basic, bare-minimum needed to get them
started.
***TESTING RESULTS ARE IGNORED***
Signed-off-by: Chris Evich <cevich@redhat.com>
Currently podman only works with --isolation chroot. This PR
fixes this by allowing the isolation mode to default to OCI and to
also allow users to pass the isolation mode into the containers.
The current tests for --isolation should cause this code to be tested.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Set of scripts to run buildah's bud.bats test using
podman build in podman CI.
podman build is not 100% compatible with buildah bud.
In particular:
* podman defaults to --layers=true; buildah to false
* podman defaults to --force-rm=true; buildah to false
* podman error exit status is 125; buildah is 2
* differences in error messages, command-line arguments
Some of the above can be dealt with programmatically,
by tweaking the buildah helpers.bash (BATS helpers).
Some need to be tweaked by patching bud.bats itself.
This PR includes a patch that will, I fear, need to
be periodically maintained over time.
There will likely be failures when vendoring in a
new buildah, possibly because new tests were added
for new features that don't exist in podman, possibly
(I hope unlikely) if existing tests are changed in
ways that make the patch file fail to apply. I've
tried to write good instructions and to write the run
script in such a way that it will offer helpful hints
on failure. My instructions and code will be imperfect;
I hope they will be good enough to merit continued use
of this test (possibly with improvements to the instructions
as we learn more about real-world failures).
Signed-off-by: Ed Santiago <santiago@redhat.com>
Initial validation of using podman-in-podman to create an
old-podman root, then use new-podman to play with the
containers created therein.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Followup to dbb9943
Despite skipping the `Smoke` check, it was observed on a *new* branch,
the `validate` task (specifically `git-validation`) will fail. This
is because:
* `$CIRRUS_LAST_GREEN_CHANGE` will be empty on a new branch.
* `$CIRRUS_BASE_SHA` is always empty for runs triggered by branch-push
* `$EPOCH_TEST_COMMIT` will be set to `YOU_FOUND_A_BUG`.
Fix this by eliminating the `Smoke` task entirely, simplifying all
the `make validate` operations into the `validate` cirrus task. Ensure
this task does not run when a new branch or tag is pushed.
Also, eliminate the `$CIRRUS_BUILD_ID` value as it's confusing and not
actually used anywhere. It was formerly used for building VM images,
but this has moved to another repo entirely.
Signed-off-by: Chris Evich <cevich@redhat.com>
The podman documentation site uses javascript to display
API documentation at:
http://docs.podman.io/en/latest/Reference.html
As input, the javascript sources from a CORS-enabled Google Cloud
Storage object. This commit ensures the storage object is present and
updated for every Cirrus-CI execution context: Tags, Branches, and PRs.
As of this commit, the documentation site only utilizes the object
uploaded by the Cirrus-CI run on the `master` branch:
`swagger-master.yaml`. The file produced and uploaded due to a PR is
intended for testing purposes: Confirm it's generation and uploading are
both functional.
Signed-off-by: Chris Evich <cevich@redhat.com>
Make sure that bindings are in sync with the code. The check is similar
to what's already being done with `make vendor`, so integrate the two.
[NO TESTS NEEDED]
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Previously automation always dropped the minor version number for
distributions. This was intended for presentation and conditional
simplicity. Bash does not support non-integer comparison natively.
With the release of version 20.10, supporting testing with it and
the LTS release (20.04) requires scripts to consider minor version
numbers for Ubuntu VMs. This is necessary because many times in
the past, some behaviors needed to be conditional on the release
version number.
With this commit, the images and embedded scripts/tooling uses an
altered format of `$UBUNTU_NAME', `$PRIOR_UBUNTU_NAME`, and (crucially)
`$OS_RELEASE_VER` and `$OS_REL_VER`. Any `.` characters appearing
in the official version (from `/etc/os-release`) are dropped, and
the result is concatenated.
For example the current Ubuntu LTS version is `20.04`. Prior to
this commit, `$OS_RELEASE_VER` would have been `20`. With this
change, `$OS_RELEASE_VER` will now show `2004`. Similarly `20.10`
is shown as `2010`.
Signed-off-by: Chris Evich <cevich@redhat.com>
Includes disk-space increase for all Fedora images to accommodate
the static-build job disk space requirements. This job substantially
leverages task-cache, which was previously failing to restore early on
in the Cirrus-CI task setup, due to disk-space limitations.
Also simplify .cirrus.yml slightly by removing an unncessary setup
and run directory change step.
Signed-off-by: Chris Evich <cevich@redhat.com>
Test VMs by design are to be single-purpose, single-use, and
readily disposable. Therefore it's unnecessary to overcomplicate
storage of runtime environment variables. This commit makes these
points clear, and reorganizes all CI-related env. vars on the system
into a single location, `/etc/ci_environment`. This file is then
automatically loaded, and variables exported, (by `lib.sh`) from
`runner.sh` prior to executing all forms of testing.
Signed-off-by: Chris Evich <cevich@redhat.com>
My patches to fix `--runtime /usr/bin/crun` being allowed to use
a different version of the crun runtime revealed a problem: we
were actually relying on that exact behavior in our E2E tests. We
specified the runtime path as `/usr/bin/runc` for the Ubuntu
tests, but that didn't exist, so Podman was actively looking for
a different, usable runc binary and using that, instead of the
path we explicitly hardcoded. Fixing the bug broke this, and thus
broke the tests.
Instead of hard-coding OCI runtime paths, swap to just using the
runtime name, `runc` or `crun`, and letting Podman figure out
where the runtime lives - it's quite good at that. This should
un-break the tests and make them more durable.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
Also removed automatic exection of setup_environment.sh since most
people using this script are podman developers (not automation/CI
folks). If executing the automation scripts is necessary, manual
attendance to required variables like `$TEST_FLAVOR` is mandatory.
Signed-off-by: Chris Evich <cevich@redhat.com>
Reimplement CI-automation to remove accumulated technical-debt and
optimize workflow. The task-dependency graph designed goal was to
shorten it's depth and increase width (i.e. more parallelism). A
reduction in redundant building (and 3rd party module download) was
also realized by caching `$GOPATH` and `$GOCACHE` early on. This
cache is then reused in favor of a fresh clone of the repository
(when possible).
Note: The system tests typically execute MUCH faster than the
integration tests. However, contrary to a fail-fast/fail-early
principal, they are executed last. This was implemented due to
debug-ability related concerns/preferences of the primary
(golang-centric) project developers.
Signed-off-by: Chris Evich <cevich@redhat.com>
All VM-building functionality has been migrated to
https://github.com/containers/automation_images
Some container-build functions are still maintained here
but are on a very-short list to also be migrated to
the repository linked above.
Signed-off-by: Chris Evich <cevich@redhat.com>
(This is an adoption of #7533 because Brent is on PTO).
Pick up new crun and crio-runc.
Also: renames from useful fedora-32 and -31 to less-useful
names; presumably this is needed by something-something in
the new VM setup.
Also: tweak two e2e tests to more properly handle a kernel
(5.8.4) with a greater set of capabilities than what we
or crun can yet handle.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Chris Evich
jason
Matthew Heon
Jhon Honce
OpenShift Merge Robot
Lokesh Mandvekar
Ed Santiago
Daniel J Walsh
Giuseppe Scrivano
Valentin Rothberg
Paul Holzinger
baude
Matthew Heon