Github-actions for large/complex tasks is hard to read and maintain.
Reimplement the multi-arch image build workflow into a set of bash
scripts that use all native contrainer-org tooling. This requires
a special VM image setup with emulation to build foreign architectures.
It also requires renaming the `helloimage` directory, because the build
script uses the directory name in the image FQIN.
Signed-off-by: Chris Evich <cevich@redhat.com>
Add a CI check to prevent unwanted bloat in binary images,
by building a baseline (pre-PR) binary then comparing file
sizes post-PR.
Part 1 (#13518) added a new script that runs multiple 'make's,
comparing image sizes against an original, and failing loudly
if growth is too big. An override mechanism is defined.
This is part 2 of 2: adding the CI rule. We couldn't do that
in part 1, because the rule would call a script that didn't
exist in the pre-PR commit.
Signed-off-by: Ed Santiago <santiago@redhat.com>
In general continuous-delivery (CD) tends to pair well with CI. More
specifically, there is a need for some reverse-dependency CI testing in
netavark/aardvark-dns. In all cases, the download URL needs to remain
consistent, without elements like `Build%20for%20fedora-35`.
The 'Total Success' task only ever executes when all dependencies are
successful. When a non `[CI:DOCS]` build is successful, gather all
binary/release artifacts in a new task which depends on 'Total Success'.
This will provide a uniform name (`artifacts`) and URL for downstream
users to use. For example:
https://api.cirrus-ci.com/v1/artifact/github/containers/podman/artifacts/binary.zip
or
https://api.cirrus-ci.com/v1/artifact/github/containers/podman/artifacts/binary/FILENAME
Where ***FILENAME*** is one of:
* `podman`
* `podman-remote`
* `rootlessport`
* `podman-release-386.tar.gz`
* `podman-release-amd64.tar.gz`
* `podman-release-arm64.tar.gz`
* `podman-release-arm.tar.gz`
* `podman-release-mips64le.tar.gz`
* `podman-release-mips64.tar.gz`
* `podman-release-mipsle.tar.gz`
* `podman-release-mips.tar.gz`
* `podman-release-ppc64le.tar.gz`
* `podman-release-s390x.tar.gz`
* `podman-remote-release-darwin_amd64.zip`
* `podman-remote-release-darwin_arm64.zip`
* `podman-remote-release-windows_amd64.zip`
* `podman-v4.0.0-dev.msi`
Signed-off-by: Chris Evich <cevich@redhat.com>
ginkgo netavark logs (and, to a lesser extent, cni logs)
are unreadable because the hide-boring-opts code did not
know about --network-backend. Now it does.
Manually filtered an existing netavark log to confirm there
are no other new options we should know about.
Signed-off-by: Ed Santiago <santiago@redhat.com>
We already link to ginkgo sources, now add links to bats.
Ugly, because we need to hardcode containers/podman (git
repo) and test/system (test file path): those can't be
determined from the log results like they can in ginkgo.
Also, great suggestion from @Luap99: in addition to the
'Annotated results' link which we append to the basic log,
include a short summary of failures. This should help a
viewer see exactly which test(s) failed, which in turn
can be helpful for diagnosing known-flake or real-problem.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Add a pair of new Cirrus test suites using Compose v2 instead of
Compose v1 (as is currently packaged in Fedora). They work
identically, and run the same tests, as the Compose v1 tests, but
with the new v2 binary instead.
[NO NEW TESTS NEEDED] This adds an entire Cirrus suite...
Signed-off-by: Matthew Heon <mheon@redhat.com>
* Add configuration to add report header for python client used in tests
* Move report headers into the individual test runners vs runner.sh
Signed-off-by: Jhon Honce <jhonce@redhat.com>
Rootless users cannot load the ip_tables module, in fedora 36 this
module is no longer loaded by default so we have to add it manually.
This is needed because rootless network setup tries to use iptables
and if iptables-legacy is used instead of iptables-nft it will fail.
To provide a better user experience we will load the module at boot.
Note that this is not needed for RHEL because iptables-legacy is not
supported on RHEL 8 and newer.
[NO NEW TESTS NEEDED]
Fixes#12661
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Since containers-common package is tied to specific versions
of Podman, add tools to build the package into the contrib directory
This should help other distributions to figure out which commont
package to ship.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Mainly this is to confirm some changes needed for the podman-py CI setup
don't disrupt operations here. Ref:
https://github.com/containers/automation_images/pull/111
Also includes a minor steup fix WRT setting up for test-rpm build.
Signed-off-by: Chris Evich <cevich@redhat.com>
We've got some python tests running in CI, and they're really hard
to troubleshoot. This PR:
1) colorizes python unittest lines (ok / skipped / fail), and
2) links to source files
The color is nice for skimming, but it's the linking that might
make it much easier to diagnose future failures.
(Context: failure today in test/python/docker/compat/test_images.py)
Signed-off-by: Ed Santiago <santiago@redhat.com>
[NO TESTS NEEDED]
Change from using a bash script to a c file
for running the image. With thanks to discussions
with @afbjorklund, the Containerfile was rigged
up to make the final image be only KB's in size.
Also add USER 1000 to make the image test/run as
non-root, and update the README.md
Signed-off-by: tomsweeneyredhat <tsweeney@redhat.com>
[NO NEW TESTS NEEDED]
This commit describes how to run the quay.io/podman/hello
image. It also contains the files necessary to build that
image localy, and a README.md explaining the image and how
to build it.
Signed-off-by: tomsweeneyredhat <tsweeney@redhat.com>
Nightly builds were failing on CI ever since the Makefile change to have
install target independent of build targets.
See: e4636ebdc8
This commit ensures everything is built before installation.
[NO NEW TESTS NEEDED]
Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
Building from source would involve separate `make` and `make install`
steps.
This removes a lot of unnecessary `-nobuild` targets which were
otherwise needed for packaging.
This commit also removes spec files for unused copr jobs.
[NO NEW TESTS NEEDED]
Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
This job is designed to be silent when Cirrus-cron executions pass.
Unless specifically instructed, the workflow itself will also remain
silent if there's an error. Fix this by catching workflow errors and
sending a notification e-mail containing a link to the failed run. This
also requires listing the recipient addresses directly in the workflow.
Otherwise (as previouslly implemented) the value would not be retrieved
if/when any previous step raised an error.
**Note**: Due to the way this workflow is implemented, there is no way
easy way to test it other than directly on the `main` repo. branch.
Signed-off-by: Chris Evich <cevich@redhat.com>
This involves a minor code-change so the download/install can run in a
loop for the two different repositories and binaries. Given everything
is exactly the same except the URLs and names.
Signed-off-by: Chris Evich <cevich@redhat.com>
This PR adds the CI mechanisms to obtain the latest upstream netavark
binary, and set a magic env-var to indicate e2e tests should execute
podman with `--network-driver=netavark`. A future commit implement
this functionality within the e2e tests.
Due to the way the new environment is enabled, the standard task name
is too long for github to display without adding ellipsis. Force the
custom task name `Netavark Integration` to workaround this. At some
future point, when netavark is more mainstream/widely supported, this
custom task and upstream binary install can simply be removed - i.e.
netavark will simply be used by default in the normal e2e tasks.
Signed-off-by: Chris Evich <cevich@redhat.com>
Since this option will also be used for netavark we should rename it to
something more generic. It is important that --cni-config-dir still
works otherwise we could break existing container cleanup commands.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
viz, rootful system tests. The rootless account will be
used by image-scp tests.
Unfortunately, having ssh available means the system-connection
tests will start running, which is very bad because they will
fail, because system connection doesn't actually work (long story).
Add a few more checks to prevent this test from running.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Podman image scp should never enter the Podman UserNS unless it needs to. This allows for
a sudo exec.Command to transfer images to and from rootful storage. If this command is run using sudo,
the simple sudo podman save/load does not work, machinectl/su is necessary here.
This modification allows for both rootful and rootless transfers, and an overall change of scp to be
more of a wrapper function for different load and save calls as well as the ssh component
Signed-off-by: cdoern <cdoern@redhat.com>
Fixes#11089 - cleanup PATH on MSI uninstall
Additionally fixes scenarios where the path can be overwritten by setx
Also removes the console flash, since the helper is built as a silent gui
Helper executable can be rerun by user to repair PATHs broken by other tools
Utilizes executable location instead of passed parameters to remove delicate escaping requirements
[NO NEW TESTS NEEDED]
Signed-off-by: Jason T. Greene <jason.greene@redhat.com>
An error was observed in another PR while downloading the swagger
binary. The error was relating to the upstream egress quota. Obviously
our downloading it every time for each CI run isn't helping. Fix this
by moving the download into the image-build process, and simply re-use
the already present binary here.
Ref: https://github.com/containers/automation_images/pull/103
Signed-off-by: Chris Evich <cevich@redhat.com>
A test name beginning with non-alpha, e.g., "--build should ...",
was not being recognized and linkified:
https://storage.googleapis.com/cirrus-ci-6707778565701632-fcae48/artifacts/containers/podman/6500723916537856/html/int-podman-fedora-34-rootless-host.log.html
Fix that. Also fix two other cases (single/double quotes) that were
resulting in weird unreliable links.
While I'm at it, add a few usability enhancements:
* Colorize [SKIPPING] and [SLOW TEST]
* Deemphasize '[It] testname' when it appears mid-test
* Replace 'Running:' with a (deemphasized) '#' or '$' prompt
Add regression tests
Signed-off-by: Ed Santiago <santiago@redhat.com>
Replace `multi-user.target` with `default.target` across the code base.
It seems like the multi-user one is not available for (rootless) users
on F35 anymore is causing issues in all kinds of ways, for instance,
enabling the podman.service or generated systemd units.
Fixes: #12438
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
I have noticed that the containers.conf file in the /home/podman
directory is owned by root and not Podman. This change fixes the
ownership.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Add a magic 'echo' to runner.sh, displaying $GIT_COMMIT in
a special syntax. The logformatter script, seeing this,
will hyperlink error messages to the failing source file.
Signed-off-by: Ed Santiago <santiago@redhat.com>
The Fedora 35 cloud images have switched to UEFI boot with a GPT
partition. Formerly, all Fedora images included support for runtime
re-partitioning. However, the requirement to test alternate storage
has since been dropped/removed. Rather than maintain a disused
feature, and supporting scripts, these Fedora VM images have reverted
to the default: Automatically resize to 100% on boot.
Signed-off-by: Chris Evich <cevich@redhat.com>
VM Images created as of this commit contain the new/required version.
Remove the `--force` install, but retain the hack script's ability to
support this in the future.
Signed-off-by: Chris Evich <cevich@redhat.com>
In F35 the hard-coded default (from
containers-common-1-32.fc35.noarch) is 'journald' despite
the upstream repository having this line commented-out.
Containerized integration tests cannot run with 'journald'
as there is no daemon/process there to receive them.
Signed-off-by: Chris Evich <cevich@redhat.com>
During initial testing of Fedora 35beta VM images in CI, the bindings
task was timing out. In order to allow time for collection of system
details (logs), execution needs to timeout earlier than the task.
Under normal conditions, the bindings test finishes in about 10-minutes.
Use the ginkgo timeout option to limit execution, so it times out after
30 minutes.
Also add the `-progress` option so the output more closely resembles how
ginkgo runs the integration tests.
Signed-off-by: Chris Evich <cevich@redhat.com>
Our fedora-minimal image on Quay bases on fedora-minimal:latest which
starting with F35 removed a number of binaries that our CI depends on.
Fix that by pulling `fedora-minimal:34` from the Fedora registry
directly.
Once the build bot on Quay has been disabled, we move the image over
there to make sure that it will not change over time.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Podman has been using catatonit for a number of years already.
Thanks to @giuseppe, catatonit is now able to run as a pause
process which allows us to replace the pause binary entirely.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
This reverts commit 9d2b8d2791 since
catatonit's new pause functionality can replace the `pause` binary
entirely.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Podman and Docker will not commit changes via RUN command
of a VOLUME directory, so we need to chown path first.
Not doing do will cause: https://bugzilla.redhat.com/show_bug.cgi?id=2009266
Signed-off-by: Jindrich Novy <jnovy@redhat.com>
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
- remove 'NO TESTS NEEDED' as a valid bypass string. Henceforth
only 'NO NEW TESTS NEEDED' will work.
- add a debugging aid for #11871, in which bodhi tests time out
in nslookup.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Future testing needs dictate rootless (in addition to root) users are
able to ssh to localhost. Add ssh-key generation commands for the
rootless user, and authorize their public key.
Minor: Also remove update of `/etc/sub{uid,gid}` files, since this is
now done automatically by `{user,group}add` commands.
Signed-off-by: Chris Evich <cevich@redhat.com>
Add the k8s pause binary to `pause/pause.c` and do the plumbing in the
Makefile to install it in $libexec/podman/pause/pause. It is intended to
replace the k8s pause image and hence the need for network connectivity
when creating pods.
[NO NEW TESTS NEEDED] since it will be tested in a following commit.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Mount a directory from /var/tmp to /tmp to make sure that /tmp is not on
an overlay mount. This should make overlay mounts possible in the
containerized tests which we're currently skipping.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Don't use reexec for the rootlessport process, instead make it a
separate binary to reduce the memory usage. The problem with reexec is
that it will import all packages that podman uses and therefore loads a
lot of stuff into the heap. The rootlessport process however only needs
the rootlesskit library.
The memory usage is a concern since the rootlessport process will spawn
two process per container which has ports forwarded. The processes stay
until the container dies. On my laptop the current reexec version uses
47800 KB RSS. The new separate binary only uses 4540 KB RSS. This is
more than a 90% improvement.
The Makefile has been updated to compile the new binary and install it
to the libexec directory.
Fixes#10790
[NO TESTS NEEDED]
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Don't hardcode /usr/bin/podman in unit files: instead, use
template files with a path replaced at install time.
Because 'make' can be invoked repeatedly, with different
PREFIX, do not leave the generated files behind in our
work directory: wipe them immediately after install.
To get this to work, fix a longstanding bug in podman.spec.in,
a PREFIX that should've been DESTDIR.
Side note: #7023 made contrib/systemd/user a symlink
to .../system but did not update paths in Makefile.
The unrelated-looking path change you see here is
a belated correction for that.
Fixes: #10787
Signed-off-by: Ed Santiago <santiago@redhat.com>
Accept both "NO TESTS NEEDED" and "NO NEW TESTS NEEDED".
That was a usability mistake I made on Day One. Fixed it
in Buildah but oops never got around to fixing it here.
Also, fix the test suite script: remove a no-longer-working
test case (changelog.txt, removed in #11467) and add a new
test for commits that include the magic string.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Original workaround https://github.com/containers/podman/pull/11821
During VM image build, a number of packages are downloaded but not
installed, since they may interfere with some testing. Then at runtime,
where required, the packages are installed from cache and used.
However, between image build and runtime it's possible the repository
contents change, which will invalidate the package cache. Since the
`--no-download --ignore-missing` options were used, the install will
fail.
Ref: https://github.com/containers/automation_images/issues/95
Fortunately, when it comes to the docker packages, no other dependencies
are required and so `apt-get` isn't required. Switch to using a simple
dpkg install command on the necessary files. If this ever breaks due
to new dependencies, the list of files may simply be updated.
Signed-off-by: Chris Evich <cevich@redhat.com>
It looks like the containerd.io package is not present anymore in the
package cache which ultimately breaks CI since it's a requirement for
docker.
Hence, download the few packages instead of relying on the cache.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Hi there! I noticed that the MSI file built for Windows was building in 32-bit mode, even though Podman is 64-bit only. I added a flag to wixl in the Makefile to tell it to build a 64-bit MSI, and I adjusted the podman.wxs file to use the right settings for 64-bit programs.
Signed-off-by: GitHub <noreply@github.com>
Add execution of the downstream gitlab-runner tests using
rootless podman through the magic of socket-level
docker compatibility. Include a comment suggesting how
to temporarily disable the test in case it fails beyond
podman code scope.
Signed-off-by: Chris Evich <cevich@redhat.com>
Add a link to the podman images readme.md to Dan's recent post
on Enable Sysadm about running containers inside of Podman
Fixes: https://github.com/containers/buildah/issues/3119
Signed-off-by: TomSweeneyRedHat <tsweeney@redhat.com>
Fixes#11417
Cross-building the podman-remote documentation requires a functional
native architecture executable. However `make` only deals with
files/timestamps, it doesn't understand if an existing binary will
function on the system or not. This makes building cross-platform
releases incredibly accident-prone and fragile.
A practical way to deal with this, is via multiple conditional (nested)
`make` calls along with careful manipulation of `$GOOS` and `$GOARCH`.
Also, when cross-building releases be kind to humans and cleanup
any non-native binaries left behind.
Update the `Alt Arch. Cross` Cirrus-CI task to build release archives
for all Linux architectures supported by golang and podman. Update
the `OSX Cross` task to additionally build for the M1 (arm64)
architecture.
Finally, update the release process documentation to reflect the
new locations (Cirrus-CI task names) for the release archives. Include
a note about additional manual work being required to produce the
signed `.dmg` file for MacOS.
Signed-off-by: Chris Evich <cevich@redhat.com>
Rootless cni with ipv6 needs the `ip6_tables` module loaded, normally
the cni plugins will load this module but as rootless it does not have
the necessary permission to do so. Therefore we load it manually.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
The changelog.txt file hasn't been kept in sync with release tags,
especially on main, so remove it.
The release notes will be featured in RELEASE_NOTES.md.
Signed-off-by: jesperpedersen <jesper.pedersen@redhat.com>
[NO TESTS NEEDED]
This becomes a problem on hosts with upgraded policies. Ref:
https://github.com/containers/podman/issues/10522
Also, made a small change to compose-test setup to reduce runtime.
Signed-off-by: Chris Evich <cevich@redhat.com>
The automation workflow was altered in recent history to build images
daily, even if the podman version didn't change. This was is necessary
so that any updates/security vulnerabilities in ancillary packages are
incorporated quickly. However, documentation was never updated to
reflect this change. This commit puts the two in sync.
Signed-off-by: Chris Evich <cevich@redhat.com>
Significant bitrot results in almost immediate test failure. This
commit adds only the very basic, bare-minimum needed to get them
started.
***TESTING RESULTS ARE IGNORED***
Signed-off-by: Chris Evich <cevich@redhat.com>
There seems to be a bug in rpm, where it fails silently if you specify
rpm --restore --quiet shadow-utils.
rpm --restore shadow-utils 2> /dev/null
Does the right thing.
[NO TESTS NEEDED] Might add tests from buildah, once we have them
working correctly.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Rather then reinstalling shadow-utils to fix permissions,
just restore the correct permissions.
[NO TESTS NEEDED] Since this does not affect Podman, just the prebuilt
images on quay.io/podman.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Require the network to be online in all (generated) systemd units to
make sure that containers and Podman run only after the network has been
fully configured.
Fixes: #10655
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
This makes sure, that the podman auto-update is not executed exactly at midnight for the same time always.
If many things do the same and many services use this keyword and react at midnight, this can cause a lot of stress to a server.
Thus, this adds a 900s/15min delay.
As [the arch wiki says](https://wiki.archlinux.org/title/Systemd/Timers#Realtime_timer):
> Special event expressions like daily and weekly refer to specific start times and thus any timers sharing such calendar events will start simultaneously. Timers sharing start events can cause poor system performance if the timers' services compete for system resources. The RandomizedDelaySec option in the [Timer] section avoids this problem by randomly staggering the start time of each timer. See systemd.timer(5).
Signed-off-by: rugk <rugk+git@posteo.de>
The rootless cni namespace needs a valid /etc/resolv.conf file. On some
distros is a symlink to somewhere under /run. Because the kernel will
follow the symlink before mounting, it is not possible to mount a file
at exactly /etc/resolv.conf. We have to ensure that the link target will
be available in the rootless cni mount ns.
Fixes#10855
Also fixed a bug in the /var/lib/cni directory lookup logic. It used
`filepath.Base` instead of `filepath.Dir` and thus looping infinitely.
Fixes#10857
[NO TESTS NEEDED]
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
The new cobra v1.2.0 release brings a number of bug fixes for shell
completion scripts. Regenerate the scripts with `make completions`
to sync them with the upstream version, currently we have some custom
ones to avoid some upstream bugs. Because the new cobra version has
all fixes we should use the upstream scripts.
Add a check to CI to ensure we always use the up to date scripts.
[NO TESTS NEEDED]
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
We no longer need to install /etc/cni/net.d/87-podman-bridge.conflist
so install.cni isn't needed either.
Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
Chris Evich