Drop a reference as to why the `rpm --setcaps...` line is needed, along
with a `TODO` reminder to check if it's still needed.
Signed-off-by: Chris Evich <cevich@redhat.com>
add support for podman-remote image scp as well as direct access via the API. This entailed
a full rework of the layering of image scp functions as well as the usual API plugging and type creation
also, implemented podman image scp tagging. which makes the syntax much more readable and allows users t tag the new image
they are loading to the local/remote machine:
allow users to pass a "new name" for the image they are transferring
`podman tag` as implemented creates a new image im `image list` when tagging, so this does the same
meaning that when transferring images with tags, podman on the remote machine/user will load two images
ex: `podman image scp computer1::alpine computer2::foobar` creates alpine:latest and localhost/foobar on the remote host
implementing tags means removal of the flexible syntax. In the currently released podman image scp, the user can either specify
`podman image scp source::img dest::` or `podman image scp dest:: source::img`. However, with tags this task becomes really hard to check
which is the image (src) and which is the new tag (dst). Removal of that streamlines the arg parsing process
Signed-off-by: Charlie Doern <cdoern@redhat.com>
One day we may use AWS for part of CI. Do you want to maintain
two separate code paths in this script for linking to artifacts
in multiple cloud providers? Can you say no? I knew you could.
Cirrus already knows the location of the artifacts and provides
a transparent mechanism for accessing them. Use it.
This PR exposed a nasty bug in our environment-variable handling:
envariables passed through to the containerized environment were
being double-space-escaped, so "FOO=a b" ended up as "FOO=a\ b"
(with a backslash), with one consequence being invalid URLs.
The solution is simple: run 'podman -e FOO', not '-e FOO=value'.
Finally, reinstate the environment-variable dump (in comments).
I had removed this in a moment of panic over leaking secrets,
but no, that doesn't happen. Exclude scary-sounding vars anyway.
Signed-off-by: Ed Santiago <santiago@redhat.com>
* Replace "setup", "lookup", "cleanup", "backup" with
"set up", "look up", "clean up", "back up"
when used as verbs. Replace also variations of those.
* Improve language in a few places.
Signed-off-by: Erik Sjölund <erik.sjolund@gmail.com>
The Containerfiles were built with sed -i, which is leading to empty
storage.conf files. This will cause Podman in a container to print
warning information about storage.driver not being set to something.
[NO NEW TESTS REQUIRED]
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Using both the 'skip' and 'only_if' features at the same time may be
hard for maintainers to decipher. Consolidate them into `only_if` since
that bypasses creation of the task all together - meaning there are
potentially fewer tasks for a developer to scroll through.
Since the `multiarch` Cirrus-Cron build no-longer depends on the direct
"build-ability" from the current repo. state, it can be further
optimized. When operating in this context, avoid running many/most
other tasks, depending instead only on `ext_svc_check`.
Finally, add a simple document describing the various runtime contexts
along with the list of expected tasks. Reference this prominently right
in front of every `only_if` so it's impossible for a maintainer to miss.
Signed-off-by: Chris Evich <cevich@redhat.com>
This was an accidental leftover from an in-development implementation.
The `sed` command further down entirely replaces the file in the image.
Strip out the unnecessary 'storage.conf' ADD instruction.
Signed-off-by: Chris Evich <cevich@redhat.com>
@cevich recently renamed all the files named Dockerfile to Containerfile
in this directory. Touching up the README.md to reflect that.
Also, as I was doing the submit, I noticed a couple of nits in the PR
request template and cleaned those up.
Signed-off-by: tomsweeneyredhat <tsweeney@redhat.com>
Fixes#14021
Substitution values built from `$(shell ...)` output can easily be empty
due to the shell's default `pipefail` behavior. This can also hide
non-zero exit codes, similarly resulting in empty values being set.
While not a perfect fix, the situation is improved by using the
`err_if_empty` function in all cases where empty values would be
unexpected. Remove the definitions for `GIT_BRANCH` and
`GIT_BRANCH_CLEAN` which don't seem to actually be used anywhere
(including in code).
Add a simple release-test to verify `podman info` outputs a non-empty
value for "GitCommit".
Signed-off-by: Chris Evich <cevich@redhat.com>
As of Fedora 36, `/etc/containers/storage.conf` with defaults is
installed under `/usr/share/containers/`. This was causing builds
to fail in the necessary `sed` command that enables fuse-overlayfs.
Fix this by using sed on the new location with an output redirect
into the `etc` location.
Also, perform a mass-cleanup of the three files to make them easier
to read/maintain. Including renaming them to `Containerfile`,
since all native build tooling is now used to produce them.
Lastly, take advantage of the `podman-next` copr repository to install
the latest/greatest podman from `main`, rather than building it from
scratch. This will greatly speed up the image build speed.
Signed-off-by: Chris Evich <cevich@redhat.com>
Most were simply deleted, the main one addressed is in the
"pre-testing" `ext_svc_check.sh` script. It will now verify
accessibility of several key test images we maintain in `quay.io`.
Signed-off-by: Chris Evich <cevich@redhat.com>
Support running `podman play kube` in systemd by exploiting the
previously added "service containers". During `play kube`, a service
container is started before all the pods and containers, and is stopped
last. The service container communicates its conmon PID via sdnotify.
Add a new systemd template to dispatch such k8s workloads. The argument
of the template is the path to the k8s file. Note that the path must be
escaped for systemd not to bark:
Let's assume we have a `top.yaml` file in the home directory:
```
$ escaped=$(systemd-escape ~/top.yaml)
$ systemctl --user start podman-play-kube@$escaped.service
```
Closes: https://issues.redhat.com/browse/RUN-1287
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Followup to #14178: I broke CI. "--fork-point" does not do
what I thought it did, so remove it.
Also, add 'set -x' to help us debug the next time something
like this happens.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Add support for new automation library version w/ `$DEBUG` fix
(ref: containers/automation_images#128) and added definitions
for commonly used Distro/version variables.
Signed-off-by: Chris Evich <cevich@redhat.com>
With the increasingly complex `.cirrus.yml` task relationships, build
cache wasn't always working as intended. Recently, non-build tasks were
observed assuming authority over `gopath_cache`. Ref.:
https://github.com/containers/podman/pull/13998#issuecomment-1108834538
Address this by an overall simplification using artifacts instead of
cache. Using artifacts allows establishing concrete
authorship/authority over cached repo. content. In this way, dependent
tasks may simply consume the artifact with `curl` instead of relying on
complex caching algorithms.
Also/Minor: Add YAML checking to the pre-commit configuration.
Signed-off-by: Chris Evich <cevich@redhat.com>
The sshd service is guaranteed to be running by the VM image build
process - it's required by the packer tool for access. Remove the
startup and check on the sshd service.
For many tests, man ssh connections to/from $ROOTLESS_USER on the
host are needed. To facilitate this, the localhost key is added to
`known_hosts` for root and `$ROOTLESS_USER`. Simplify this setup using
the `ssh-keyscan` tool.
Signed-off-by: Chris Evich <cevich@redhat.com>
There are several runtime contexts (rootless and container) where
`$DEST_BRANCH` is needed but was not supplied. A prior commit
(c486576717) removed the default value, `main` which was being
set incorrectly when CI ran on release branches. Fix this by ensuring
the variable is non-empty upon entry to `setup_environment.sh`, then
ensure it gets passed through to child environments by way of the
`/etc/ci_environment` file. This will maintain compatibility with
both CI and `hack/get_ci_vm.sh` use.
Signed-off-by: Chris Evich <cevich@redhat.com>
The use of 'C^' (parent) in 'git rebase' is counterintuitive,
at least to me: when C is a merge of multiple commits, rebase
picks each of those commits. That probably makes good sense
to a git expert, which I'm not.
Solution: forget using '^', just calculate the baseline sizes
by doing a checkout of the PR's parent.
Also: compute PR parent using git-merge-base instead of
blindly trusting $DEST_BRANCH (which may be volatile as
other PRs are merged).
Also: run git-rebase directly, not via make rule. That
indirection is too confusing here.
Signed-off-by: Ed Santiago <santiago@redhat.com>
The test-skipping optimization is failing as rootless on non-main,
because $DEST_BRANCH is not set. Solution: check for envariable,
skip test if missing. (This was part of my original PR, but was
accidentally removed in #14013)
Also: DEST_BRANCH was silently being defaulted to 'main' in lib.sh.
Remove that: per @cevich, it is no longer necessary.
Fixes: #14131
Signed-off-by: Ed Santiago <santiago@redhat.com>
Newer versions of git are much more pedantic about who owns the
repository files. When setting up to run rootless, prior to this
commit, the repo. ownership was changed from root. This causes
all subsequent git-operations as root to fail:
```
fatal: unsafe repository ('<$GOSRC>' is owned by someone else)
```
Fix this by re-ordering operations, such that the change in ownership is
done immediately before executing as a user. Also disable the
git-ownership check on the source repository assuming the CI environment
is disposable.
Signed-off-by: Chris Evich <cevich@redhat.com>
Normally installing/updating packages at test runtime is highly
discouraged for reliability and efficiency reasons. However, in this
specific case, development work of these packages is still fairly hot.
As a compromise to support podman test development, temporarily update
these two specific packages at runtime. At a future date, when updates
are less frequent, this commit can/should be safely reverted. At that
point, the versions installed at VM image build time will persist.
Signed-off-by: Chris Evich <cevich@redhat.com>
Now that netavark and aardvark are packaged and default in F36, support
CNI-based testing in F35 and Ubuntu.
* Remove the temporary/special `$TEST_ENVIRON=host-netavark` construct.
* Remove dedicated/special integration and system testing tasks.
* Update test-config setup to properly handle CNI vs netavark/aardvark
environments.
* Update package-version logging to operate based on installed packages
(along with some other minor script cleanups).
* Update global environment setup to force `$NETWORK_BACKEND=netavark`
in F36 and later. Except when `upgrade_test` task runs.
* Discontinue installing netavark and aardvark-dns binaries from
upstream build artifacts.
* Drop CGV1-vs-2 policy check. Ubuntu VMs now exclusively test CGv1,
Fedora VMs test CGv2, with F35 testing CNI and F36 testing Netavark.
Signed-off-by: Chris Evich <cevich@redhat.com>
The originally intent for skipping tests based on change-content was to
optimize the PR workflow. However, a mistake in a conditional is
causing almost all tasks running for Cron and branches to be skipped.
Fix this by checking for an empty '$CIRRUS_PR' variable. This value is
always empty when operating outside of PRs.
Signed-off-by: Chris Evich <cevich@redhat.com>
After merging #13998 it was observed that the `docker-py` task was still
failing with the same error on `main`. The original quick-fix had
placed the full-build (`make`) call too late in the process. This
commit moves it up to right before the `make install` call which was
resulting in an error.
Again, a further future commit is planned to re-work and simplify the
entire cache setup. This is only a quick fix to make branch-builds
pass.
Signed-off-by: Chris Evich <cevich@redhat.com>
A suspected race on uploading gopath cache is causing the docker-py (and
possibly other) tasks to fail unpredictably with an error from `make`
regarding missing `bin/podman`. Since this failure is affecting all
development activity, apply a quick/dirty fix to the failing task, by
simply rebuilding the binary. A more comprehensive/long-term fix will
be worked in a future PR.
Signed-off-by: Chris Evich <cevich@redhat.com>
This reverts commit cc3790f332.
We can't change rootful to rootfull because `rootful` is written into the machine config. Changing this will break json unmarshalling, which will break existing machines.
[NO NEW TESTS NEEDED]
Signed-off-by: Ashley Cui <acui@redhat.com>
The default verbosity level does not show the classes or function names.
This makes it difficult to debug problems like hangs. Also, separate
the bats and python-based tests into two sections. This allows for
easier debugging, since isolation can be done in `runner.sh` rather than
mucking with the `Makefile`. Lastly, update the logformatter script
to `autoflush stdout` (thanks @edsantiago).
Signed-off-by: Chris Evich <cevich@redhat.com>
We are inconsistent on the name, we should stick with rootfull.
[NO NEW TESTS NEEDED] Existing tests should handle this and no tests for
machines exists yet.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Newer versions of git (like `2.35`) fail on certain operations (like
`rebase` and `am`) without a local identity. Add a fake one from the
start, with a clearly identifiable test-value to avoid problems at
runtime.
Signed-off-by: Chris Evich <cevich@redhat.com>
A common pattern is to submit PRs that update only tests or docs.
When the only changes are to test/e2e, there is no point in running
test/system or test/upgrade or test/buildah-bud. Likewise, reciprocally,
and similarly for a bunch of other tests (alt, cross, apiv2, ...)
And when the only changes are under docs/ , there is no point in
running any of the above.
Exception: if $CIRRUS_<mumble> are undefined (e.g., cron), never skip
Signed-off-by: Ed Santiago <santiago@redhat.com>
This new VM image brings in two important updates to multi-arch
image build operations:
1. Future operational changes will no-longer require also updating VM
images. Updates to build-push made in `containers/automation_images`
will automatically be picked up at runtime.
ref: https://github.com/containers/automation_images/pull/123
2. On the next run, both `vX` and `vX.Y` tagged manifest-lists will be
pushed. This is now also reflected in the README.
ref: https://github.com/containers/automation_images/pull/125
Signed-off-by: Chris Evich <cevich@redhat.com>
Building multi-arch images in a standardized way is complex. Some
of the builds themselves can take a really long time to run (over
an hour). Make changes easier to test inside a PR by adding
manually-triggered image-build tasks. These mirror most of the real
cron-triggered task, without actually pushing the final images.
Signed-off-by: Chris Evich <cevich@redhat.com>
enable cgroup delegation when running as a systemd service so all the
available controllers are correctly detected.
Closes: https://github.com/containers/podman/issues/13710
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
Github-actions for large/complex tasks is hard to read and maintain.
Reimplement the multi-arch image build workflow into a set of bash
scripts that use all native contrainer-org tooling. This requires
a special VM image setup with emulation to build foreign architectures.
It also requires renaming the `helloimage` directory, because the build
script uses the directory name in the image FQIN.
Signed-off-by: Chris Evich <cevich@redhat.com>
Add a CI check to prevent unwanted bloat in binary images,
by building a baseline (pre-PR) binary then comparing file
sizes post-PR.
Part 1 (#13518) added a new script that runs multiple 'make's,
comparing image sizes against an original, and failing loudly
if growth is too big. An override mechanism is defined.
This is part 2 of 2: adding the CI rule. We couldn't do that
in part 1, because the rule would call a script that didn't
exist in the pre-PR commit.
Signed-off-by: Ed Santiago <santiago@redhat.com>
In general continuous-delivery (CD) tends to pair well with CI. More
specifically, there is a need for some reverse-dependency CI testing in
netavark/aardvark-dns. In all cases, the download URL needs to remain
consistent, without elements like `Build%20for%20fedora-35`.
The 'Total Success' task only ever executes when all dependencies are
successful. When a non `[CI:DOCS]` build is successful, gather all
binary/release artifacts in a new task which depends on 'Total Success'.
This will provide a uniform name (`artifacts`) and URL for downstream
users to use. For example:
https://api.cirrus-ci.com/v1/artifact/github/containers/podman/artifacts/binary.zip
or
https://api.cirrus-ci.com/v1/artifact/github/containers/podman/artifacts/binary/FILENAME
Where ***FILENAME*** is one of:
* `podman`
* `podman-remote`
* `rootlessport`
* `podman-release-386.tar.gz`
* `podman-release-amd64.tar.gz`
* `podman-release-arm64.tar.gz`
* `podman-release-arm.tar.gz`
* `podman-release-mips64le.tar.gz`
* `podman-release-mips64.tar.gz`
* `podman-release-mipsle.tar.gz`
* `podman-release-mips.tar.gz`
* `podman-release-ppc64le.tar.gz`
* `podman-release-s390x.tar.gz`
* `podman-remote-release-darwin_amd64.zip`
* `podman-remote-release-darwin_arm64.zip`
* `podman-remote-release-windows_amd64.zip`
* `podman-v4.0.0-dev.msi`
Signed-off-by: Chris Evich <cevich@redhat.com>
ginkgo netavark logs (and, to a lesser extent, cni logs)
are unreadable because the hide-boring-opts code did not
know about --network-backend. Now it does.
Manually filtered an existing netavark log to confirm there
are no other new options we should know about.
Signed-off-by: Ed Santiago <santiago@redhat.com>
We already link to ginkgo sources, now add links to bats.
Ugly, because we need to hardcode containers/podman (git
repo) and test/system (test file path): those can't be
determined from the log results like they can in ginkgo.
Also, great suggestion from @Luap99: in addition to the
'Annotated results' link which we append to the basic log,
include a short summary of failures. This should help a
viewer see exactly which test(s) failed, which in turn
can be helpful for diagnosing known-flake or real-problem.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Add a pair of new Cirrus test suites using Compose v2 instead of
Compose v1 (as is currently packaged in Fedora). They work
identically, and run the same tests, as the Compose v1 tests, but
with the new v2 binary instead.
[NO NEW TESTS NEEDED] This adds an entire Cirrus suite...
Signed-off-by: Matthew Heon <mheon@redhat.com>
* Add configuration to add report header for python client used in tests
* Move report headers into the individual test runners vs runner.sh
Signed-off-by: Jhon Honce <jhonce@redhat.com>
Rootless users cannot load the ip_tables module, in fedora 36 this
module is no longer loaded by default so we have to add it manually.
This is needed because rootless network setup tries to use iptables
and if iptables-legacy is used instead of iptables-nft it will fail.
To provide a better user experience we will load the module at boot.
Note that this is not needed for RHEL because iptables-legacy is not
supported on RHEL 8 and newer.
[NO NEW TESTS NEEDED]
Fixes#12661
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Since containers-common package is tied to specific versions
of Podman, add tools to build the package into the contrib directory
This should help other distributions to figure out which commont
package to ship.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Mainly this is to confirm some changes needed for the podman-py CI setup
don't disrupt operations here. Ref:
https://github.com/containers/automation_images/pull/111
Also includes a minor steup fix WRT setting up for test-rpm build.
Signed-off-by: Chris Evich <cevich@redhat.com>
We've got some python tests running in CI, and they're really hard
to troubleshoot. This PR:
1) colorizes python unittest lines (ok / skipped / fail), and
2) links to source files
The color is nice for skimming, but it's the linking that might
make it much easier to diagnose future failures.
(Context: failure today in test/python/docker/compat/test_images.py)
Signed-off-by: Ed Santiago <santiago@redhat.com>
[NO TESTS NEEDED]
Change from using a bash script to a c file
for running the image. With thanks to discussions
with @afbjorklund, the Containerfile was rigged
up to make the final image be only KB's in size.
Also add USER 1000 to make the image test/run as
non-root, and update the README.md
Signed-off-by: tomsweeneyredhat <tsweeney@redhat.com>
[NO NEW TESTS NEEDED]
This commit describes how to run the quay.io/podman/hello
image. It also contains the files necessary to build that
image localy, and a README.md explaining the image and how
to build it.
Signed-off-by: tomsweeneyredhat <tsweeney@redhat.com>
Nightly builds were failing on CI ever since the Makefile change to have
install target independent of build targets.
See: e4636ebdc8
This commit ensures everything is built before installation.
[NO NEW TESTS NEEDED]
Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
Building from source would involve separate `make` and `make install`
steps.
This removes a lot of unnecessary `-nobuild` targets which were
otherwise needed for packaging.
This commit also removes spec files for unused copr jobs.
[NO NEW TESTS NEEDED]
Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
This job is designed to be silent when Cirrus-cron executions pass.
Unless specifically instructed, the workflow itself will also remain
silent if there's an error. Fix this by catching workflow errors and
sending a notification e-mail containing a link to the failed run. This
also requires listing the recipient addresses directly in the workflow.
Otherwise (as previouslly implemented) the value would not be retrieved
if/when any previous step raised an error.
**Note**: Due to the way this workflow is implemented, there is no way
easy way to test it other than directly on the `main` repo. branch.
Signed-off-by: Chris Evich <cevich@redhat.com>
This involves a minor code-change so the download/install can run in a
loop for the two different repositories and binaries. Given everything
is exactly the same except the URLs and names.
Signed-off-by: Chris Evich <cevich@redhat.com>
This PR adds the CI mechanisms to obtain the latest upstream netavark
binary, and set a magic env-var to indicate e2e tests should execute
podman with `--network-driver=netavark`. A future commit implement
this functionality within the e2e tests.
Due to the way the new environment is enabled, the standard task name
is too long for github to display without adding ellipsis. Force the
custom task name `Netavark Integration` to workaround this. At some
future point, when netavark is more mainstream/widely supported, this
custom task and upstream binary install can simply be removed - i.e.
netavark will simply be used by default in the normal e2e tasks.
Signed-off-by: Chris Evich <cevich@redhat.com>
Since this option will also be used for netavark we should rename it to
something more generic. It is important that --cni-config-dir still
works otherwise we could break existing container cleanup commands.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
viz, rootful system tests. The rootless account will be
used by image-scp tests.
Unfortunately, having ssh available means the system-connection
tests will start running, which is very bad because they will
fail, because system connection doesn't actually work (long story).
Add a few more checks to prevent this test from running.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Podman image scp should never enter the Podman UserNS unless it needs to. This allows for
a sudo exec.Command to transfer images to and from rootful storage. If this command is run using sudo,
the simple sudo podman save/load does not work, machinectl/su is necessary here.
This modification allows for both rootful and rootless transfers, and an overall change of scp to be
more of a wrapper function for different load and save calls as well as the ssh component
Signed-off-by: cdoern <cdoern@redhat.com>
Fixes#11089 - cleanup PATH on MSI uninstall
Additionally fixes scenarios where the path can be overwritten by setx
Also removes the console flash, since the helper is built as a silent gui
Helper executable can be rerun by user to repair PATHs broken by other tools
Utilizes executable location instead of passed parameters to remove delicate escaping requirements
[NO NEW TESTS NEEDED]
Signed-off-by: Jason T. Greene <jason.greene@redhat.com>
An error was observed in another PR while downloading the swagger
binary. The error was relating to the upstream egress quota. Obviously
our downloading it every time for each CI run isn't helping. Fix this
by moving the download into the image-build process, and simply re-use
the already present binary here.
Ref: https://github.com/containers/automation_images/pull/103
Signed-off-by: Chris Evich <cevich@redhat.com>
A test name beginning with non-alpha, e.g., "--build should ...",
was not being recognized and linkified:
https://storage.googleapis.com/cirrus-ci-6707778565701632-fcae48/artifacts/containers/podman/6500723916537856/html/int-podman-fedora-34-rootless-host.log.html
Fix that. Also fix two other cases (single/double quotes) that were
resulting in weird unreliable links.
While I'm at it, add a few usability enhancements:
* Colorize [SKIPPING] and [SLOW TEST]
* Deemphasize '[It] testname' when it appears mid-test
* Replace 'Running:' with a (deemphasized) '#' or '$' prompt
Add regression tests
Signed-off-by: Ed Santiago <santiago@redhat.com>
Replace `multi-user.target` with `default.target` across the code base.
It seems like the multi-user one is not available for (rootless) users
on F35 anymore is causing issues in all kinds of ways, for instance,
enabling the podman.service or generated systemd units.
Fixes: #12438
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
I have noticed that the containers.conf file in the /home/podman
directory is owned by root and not Podman. This change fixes the
ownership.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Add a magic 'echo' to runner.sh, displaying $GIT_COMMIT in
a special syntax. The logformatter script, seeing this,
will hyperlink error messages to the failing source file.
Signed-off-by: Ed Santiago <santiago@redhat.com>
The Fedora 35 cloud images have switched to UEFI boot with a GPT
partition. Formerly, all Fedora images included support for runtime
re-partitioning. However, the requirement to test alternate storage
has since been dropped/removed. Rather than maintain a disused
feature, and supporting scripts, these Fedora VM images have reverted
to the default: Automatically resize to 100% on boot.
Signed-off-by: Chris Evich <cevich@redhat.com>
VM Images created as of this commit contain the new/required version.
Remove the `--force` install, but retain the hack script's ability to
support this in the future.
Signed-off-by: Chris Evich <cevich@redhat.com>
In F35 the hard-coded default (from
containers-common-1-32.fc35.noarch) is 'journald' despite
the upstream repository having this line commented-out.
Containerized integration tests cannot run with 'journald'
as there is no daemon/process there to receive them.
Signed-off-by: Chris Evich <cevich@redhat.com>
During initial testing of Fedora 35beta VM images in CI, the bindings
task was timing out. In order to allow time for collection of system
details (logs), execution needs to timeout earlier than the task.
Under normal conditions, the bindings test finishes in about 10-minutes.
Use the ginkgo timeout option to limit execution, so it times out after
30 minutes.
Also add the `-progress` option so the output more closely resembles how
ginkgo runs the integration tests.
Signed-off-by: Chris Evich <cevich@redhat.com>
Our fedora-minimal image on Quay bases on fedora-minimal:latest which
starting with F35 removed a number of binaries that our CI depends on.
Fix that by pulling `fedora-minimal:34` from the Fedora registry
directly.
Once the build bot on Quay has been disabled, we move the image over
there to make sure that it will not change over time.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Podman has been using catatonit for a number of years already.
Thanks to @giuseppe, catatonit is now able to run as a pause
process which allows us to replace the pause binary entirely.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
This reverts commit 9d2b8d2791 since
catatonit's new pause functionality can replace the `pause` binary
entirely.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Podman and Docker will not commit changes via RUN command
of a VOLUME directory, so we need to chown path first.
Not doing do will cause: https://bugzilla.redhat.com/show_bug.cgi?id=2009266
Signed-off-by: Jindrich Novy <jnovy@redhat.com>
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
- remove 'NO TESTS NEEDED' as a valid bypass string. Henceforth
only 'NO NEW TESTS NEEDED' will work.
- add a debugging aid for #11871, in which bodhi tests time out
in nslookup.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Future testing needs dictate rootless (in addition to root) users are
able to ssh to localhost. Add ssh-key generation commands for the
rootless user, and authorize their public key.
Minor: Also remove update of `/etc/sub{uid,gid}` files, since this is
now done automatically by `{user,group}add` commands.
Signed-off-by: Chris Evich <cevich@redhat.com>
Add the k8s pause binary to `pause/pause.c` and do the plumbing in the
Makefile to install it in $libexec/podman/pause/pause. It is intended to
replace the k8s pause image and hence the need for network connectivity
when creating pods.
[NO NEW TESTS NEEDED] since it will be tested in a following commit.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Mount a directory from /var/tmp to /tmp to make sure that /tmp is not on
an overlay mount. This should make overlay mounts possible in the
containerized tests which we're currently skipping.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Don't use reexec for the rootlessport process, instead make it a
separate binary to reduce the memory usage. The problem with reexec is
that it will import all packages that podman uses and therefore loads a
lot of stuff into the heap. The rootlessport process however only needs
the rootlesskit library.
The memory usage is a concern since the rootlessport process will spawn
two process per container which has ports forwarded. The processes stay
until the container dies. On my laptop the current reexec version uses
47800 KB RSS. The new separate binary only uses 4540 KB RSS. This is
more than a 90% improvement.
The Makefile has been updated to compile the new binary and install it
to the libexec directory.
Fixes#10790
[NO TESTS NEEDED]
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Don't hardcode /usr/bin/podman in unit files: instead, use
template files with a path replaced at install time.
Because 'make' can be invoked repeatedly, with different
PREFIX, do not leave the generated files behind in our
work directory: wipe them immediately after install.
To get this to work, fix a longstanding bug in podman.spec.in,
a PREFIX that should've been DESTDIR.
Side note: #7023 made contrib/systemd/user a symlink
to .../system but did not update paths in Makefile.
The unrelated-looking path change you see here is
a belated correction for that.
Fixes: #10787
Signed-off-by: Ed Santiago <santiago@redhat.com>
Accept both "NO TESTS NEEDED" and "NO NEW TESTS NEEDED".
That was a usability mistake I made on Day One. Fixed it
in Buildah but oops never got around to fixing it here.
Also, fix the test suite script: remove a no-longer-working
test case (changelog.txt, removed in #11467) and add a new
test for commits that include the magic string.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Original workaround https://github.com/containers/podman/pull/11821
During VM image build, a number of packages are downloaded but not
installed, since they may interfere with some testing. Then at runtime,
where required, the packages are installed from cache and used.
However, between image build and runtime it's possible the repository
contents change, which will invalidate the package cache. Since the
`--no-download --ignore-missing` options were used, the install will
fail.
Ref: https://github.com/containers/automation_images/issues/95
Fortunately, when it comes to the docker packages, no other dependencies
are required and so `apt-get` isn't required. Switch to using a simple
dpkg install command on the necessary files. If this ever breaks due
to new dependencies, the list of files may simply be updated.
Signed-off-by: Chris Evich <cevich@redhat.com>
It looks like the containerd.io package is not present anymore in the
package cache which ultimately breaks CI since it's a requirement for
docker.
Hence, download the few packages instead of relying on the cache.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Hi there! I noticed that the MSI file built for Windows was building in 32-bit mode, even though Podman is 64-bit only. I added a flag to wixl in the Makefile to tell it to build a 64-bit MSI, and I adjusted the podman.wxs file to use the right settings for 64-bit programs.
Signed-off-by: GitHub <noreply@github.com>
Add execution of the downstream gitlab-runner tests using
rootless podman through the magic of socket-level
docker compatibility. Include a comment suggesting how
to temporarily disable the test in case it fails beyond
podman code scope.
Signed-off-by: Chris Evich <cevich@redhat.com>
Add a link to the podman images readme.md to Dan's recent post
on Enable Sysadm about running containers inside of Podman
Fixes: https://github.com/containers/buildah/issues/3119
Signed-off-by: TomSweeneyRedHat <tsweeney@redhat.com>
Fixes#11417
Cross-building the podman-remote documentation requires a functional
native architecture executable. However `make` only deals with
files/timestamps, it doesn't understand if an existing binary will
function on the system or not. This makes building cross-platform
releases incredibly accident-prone and fragile.
A practical way to deal with this, is via multiple conditional (nested)
`make` calls along with careful manipulation of `$GOOS` and `$GOARCH`.
Also, when cross-building releases be kind to humans and cleanup
any non-native binaries left behind.
Update the `Alt Arch. Cross` Cirrus-CI task to build release archives
for all Linux architectures supported by golang and podman. Update
the `OSX Cross` task to additionally build for the M1 (arm64)
architecture.
Finally, update the release process documentation to reflect the
new locations (Cirrus-CI task names) for the release archives. Include
a note about additional manual work being required to produce the
signed `.dmg` file for MacOS.
Signed-off-by: Chris Evich <cevich@redhat.com>
Rootless cni with ipv6 needs the `ip6_tables` module loaded, normally
the cni plugins will load this module but as rootless it does not have
the necessary permission to do so. Therefore we load it manually.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
The changelog.txt file hasn't been kept in sync with release tags,
especially on main, so remove it.
The release notes will be featured in RELEASE_NOTES.md.
Signed-off-by: jesperpedersen <jesper.pedersen@redhat.com>
[NO TESTS NEEDED]
This becomes a problem on hosts with upgraded policies. Ref:
https://github.com/containers/podman/issues/10522
Also, made a small change to compose-test setup to reduce runtime.
Signed-off-by: Chris Evich <cevich@redhat.com>
The automation workflow was altered in recent history to build images
daily, even if the podman version didn't change. This was is necessary
so that any updates/security vulnerabilities in ancillary packages are
incorporated quickly. However, documentation was never updated to
reflect this change. This commit puts the two in sync.
Signed-off-by: Chris Evich <cevich@redhat.com>
Significant bitrot results in almost immediate test failure. This
commit adds only the very basic, bare-minimum needed to get them
started.
***TESTING RESULTS ARE IGNORED***
Signed-off-by: Chris Evich <cevich@redhat.com>
There seems to be a bug in rpm, where it fails silently if you specify
rpm --restore --quiet shadow-utils.
rpm --restore shadow-utils 2> /dev/null
Does the right thing.
[NO TESTS NEEDED] Might add tests from buildah, once we have them
working correctly.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Rather then reinstalling shadow-utils to fix permissions,
just restore the correct permissions.
[NO TESTS NEEDED] Since this does not affect Podman, just the prebuilt
images on quay.io/podman.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Require the network to be online in all (generated) systemd units to
make sure that containers and Podman run only after the network has been
fully configured.
Fixes: #10655
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
This makes sure, that the podman auto-update is not executed exactly at midnight for the same time always.
If many things do the same and many services use this keyword and react at midnight, this can cause a lot of stress to a server.
Thus, this adds a 900s/15min delay.
As [the arch wiki says](https://wiki.archlinux.org/title/Systemd/Timers#Realtime_timer):
> Special event expressions like daily and weekly refer to specific start times and thus any timers sharing such calendar events will start simultaneously. Timers sharing start events can cause poor system performance if the timers' services compete for system resources. The RandomizedDelaySec option in the [Timer] section avoids this problem by randomly staggering the start time of each timer. See systemd.timer(5).
Signed-off-by: rugk <rugk+git@posteo.de>
The rootless cni namespace needs a valid /etc/resolv.conf file. On some
distros is a symlink to somewhere under /run. Because the kernel will
follow the symlink before mounting, it is not possible to mount a file
at exactly /etc/resolv.conf. We have to ensure that the link target will
be available in the rootless cni mount ns.
Fixes#10855
Also fixed a bug in the /var/lib/cni directory lookup logic. It used
`filepath.Base` instead of `filepath.Dir` and thus looping infinitely.
Fixes#10857
[NO TESTS NEEDED]
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
The new cobra v1.2.0 release brings a number of bug fixes for shell
completion scripts. Regenerate the scripts with `make completions`
to sync them with the upstream version, currently we have some custom
ones to avoid some upstream bugs. Because the new cobra version has
all fixes we should use the upstream scripts.
Add a check to CI to ensure we always use the up to date scripts.
[NO TESTS NEEDED]
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
We no longer need to install /etc/cni/net.d/87-podman-bridge.conflist
so install.cni isn't needed either.
Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
The systemd-tmpfiles configuration is meant preserve important
paths in /tmp that are used by Podman against deletion by
systemd. However, not all paths we previously used were included.
Some older versions used the `/tmp/use-$UID/libpod` directory
instead (when `/run/user/$UID` was unavailable).
Add an entry for these old paths to ensure tmpfiles treats the
directory correctly.
Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1960948
Signed-off-by: Matthew Heon <mheon@redhat.com>
Docker VOLUMEs will inherit permissions from an existing directory at the same
path. If the path does not exist, the directory will be owned by root which
makes this image unusable in rootless mode.
Signed-off-by: Blake Burkhart <blake.burkhart@us.af.mil>
* Add podman-restart systemd unit file and add it to podman RPM package
* Fix podman start to filter all containers + unit test
Signed-off-by: Boaz Shuster <boaz.shuster.github@gmail.com>
Extend the systemd auto-update service to prune images after an update
has run. As reported by a user [1], auto updates can over time cause
the disk to run out of space. With Edge being a target use case, we
need to make sure that systems can run without much supervision, so
let's make sure to run `podman image prune` to clean up dangling images.
[1] https://twitter.com/r_isc_y/status/1388981737011793921Fixes: #10190
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Commit 7f2c27d43f added an invalid value for the log_driver in the
containers.conf file inside the podman image.
Fixes#10312
Signed-off-by: Paul Holzinger <paul.holzinger@web.de>
Update the order of image documentation to be from most to least stable.
Similarly, avoid depending on execution of upstream podman, when
building/pushing. It's easily possible for this build to function but
execution to fail due to some partially implemented feature.
Also, ensure images tagged `latest` are pushed for every matrix
item. For 'upstream' and 'testing', this replaces use of the
'master' tag.
Lastly, update workflow comments and split the 'podman' and 'containers'
FQIN steps and outputs to improve readability.
Signed-off-by: Chris Evich <cevich@redhat.com>
Also simplify `lib.sh` after supporting changes incorporated
into automation library 2.x+ (present in all VM and container images).
* No need to force-load `/etc/profile` and handle it's expectation
to **not** being in `errexit` mode.
* Slightly re-arrange loading of automation library files for
clarity.
* Update comments.
Signed-off-by: Chris Evich <cevich@redhat.com>
Distro builds on Fedora and Kubic projects use GO111MODULE=off
by default which are currently failing. This commit fixes it and
going forward, podman CI will also indicate failures in rpm builds.
The additional LDFLAGS have been removed from the spec file
which is not ideal. But, currently we only use the spec file
to check if the rpm builds fine. We can fix the LDFLAGS in a
later commit when we're working on packit integration.
conmon build has also been removed from podman.spec.in because the COPR
for which it was provided has been discontinued.
[NO TESTS NEEDED]
Fixes: #10009
Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
docker-compose test continues to flake even after #9961.
Let's try to get some useful data from the failures, by:
* adding -S (--show-error) to curl. With just -s (--silent),
curl is completely quiet. With -S, it displays errors.
(Not in TAP form, but I'm OK with that)
* oops, adding safety checks to the fix from #9961 (it
was inadvertently clobbering the curl exit status)
And, as long as I'm in this code: logformatter was not
highlighting these results, because the '1..N' TAP line
needs to be spit out at the end. Have test-compose emit
a 'TAP' header <http://testanything.org/> and make
logformatter recognize it.
Signed-off-by: Ed Santiago <santiago@redhat.com>
* Incorporate changes from abandoned #9918: Use dedicated `bin`
sub-directories for `windows` and `darwin` when building
`podman-remote`. The linux flavor remains under `bin` as before.
* Fix MacOS Documentation-generation for release-packaging.
The `install-podman-remote-%-docs` target requires local execution
of `podman-remote`, but it was assuming GOOS=linux. Fix this
by dynamically discovering the local OS/architecture type while
still permitting cross-building of MacOS binaries under Linux.
* Unify temporary directory/file behavior to use a common template.
In case of left-over temporary items left in the repository,
update the `clean` target accordingly to remove them.
* Fix broken podman-remote-static and MacOS release archive targets
mismatching the `podman-remote-%` target. Disambiguate this target
for all platforms by spelling each out in full, instead of using
a wild-card recipe.
* Fix Windows-installer target to properly recognize existing
output files and not constantly rebuild every time.
* Include the podman version number in the Windows-installer target
in case a user downloads multiple releases.
* Include a subdirectory containing the podman version number for
both `tar.gz` and `zip` targets. This prevents users clobbering
existing directories when un-archiving from releases.
Signed-off-by: Chris Evich <cevich@redhat.com>
[NO TEST NEEDED] Can not test this in CI/CD system since it needs to be
merged in order for the Dockerfiles to even work.
Modified the /etc/subuid and /etc/subgid to be able to run in rootless
containers. The Range can not be the same as on the host.
Add /home/podman/.config/containers/containers.conf to automatically
mount /proc on /proc while inside of the container. This prevents
additional permissions being required that are blocked when not in
--privileged mode.
Setup volumes for /var/lib/containers and
/home/podman/.local/share/containwers
This will prevent the errors where people are doing overlay on overlay.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Currently podman only works with --isolation chroot. This PR
fixes this by allowing the isolation mode to default to OCI and to
also allow users to pass the isolation mode into the containers.
The current tests for --isolation should cause this code to be tested.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Set of scripts to run buildah's bud.bats test using
podman build in podman CI.
podman build is not 100% compatible with buildah bud.
In particular:
* podman defaults to --layers=true; buildah to false
* podman defaults to --force-rm=true; buildah to false
* podman error exit status is 125; buildah is 2
* differences in error messages, command-line arguments
Some of the above can be dealt with programmatically,
by tweaking the buildah helpers.bash (BATS helpers).
Some need to be tweaked by patching bud.bats itself.
This PR includes a patch that will, I fear, need to
be periodically maintained over time.
There will likely be failures when vendoring in a
new buildah, possibly because new tests were added
for new features that don't exist in podman, possibly
(I hope unlikely) if existing tests are changed in
ways that make the patch file fail to apply. I've
tried to write good instructions and to write the run
script in such a way that it will offer helpful hints
on failure. My instructions and code will be imperfect;
I hope they will be good enough to merit continued use
of this test (possibly with improvements to the instructions
as we learn more about real-world failures).
Signed-off-by: Ed Santiago <santiago@redhat.com>
Initial validation of using podman-in-podman to create an
old-podman root, then use new-podman to play with the
containers created therein.
Signed-off-by: Ed Santiago <santiago@redhat.com>
git tries to recognize renamed files. This isn't always
as helpful as intended. Turn it off, so we'll always see
files as 'A'dded.
Signed-off-by: Ed Santiago <santiago@redhat.com>
This commit sets the CGO_CFLAGS variable for hardening the Fedora rpm
binaries.
The flags used are the same as those in the official Fedora rpms.
Setting the flags in upstream spec would provide early warnings for
flag adjustments or other hardening issues.
Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
This mailing-list was established to allow people to sub/unsub from
automated notifications. Add it to the list of destinations picked up
by the Github Actions workflow
`.github/workflows/check_cirrus_cron.yml`.
Signed-off-by: Chris Evich <cevich@redhat.com>
In rare cases, it's possible for one of the ginkgo processes to "hang".
When this occurs, the main output will contain this message:
``Ginkgo timed out waiting for all parallel nodes to report``
The only way to debug this was to look through concatenated printing
of the ginkgo node logs. This is a tedious and daunting task,
requiring special search knowledge, facing a "wall of text".
Simplify the situation by collecting the node logs separately, as
individual files in a cirrus-artifact. In this way, it's faster to
figure out which test "hung" by examining each log individually. The
log file which does not have a pass/fail summary at the end,
indicates the last test hung (for whatever reason), and includes it's
output (if any).
Signed-off-by: Chris Evich <cevich@redhat.com>
Prior to this commit, the "Static Build" task only produced a
`bin/podman`. Update this to also include a `bin/podman-remote`
binary.
Update the pr-should-include-tests checker to ignore the `nix`
directory, which isn't applicable.
Lastly, restore the static build task to 'required' for CI success.
Leaving the comment inplace in case it needs to be bypassed in the
future on short notice.
Signed-off-by: Chris Evich <cevich@redhat.com>
Make sure we pass the network aliases as capability args to the
cnitool in the rootless-cni-infra container. Also update the
dnsname plugin in the cni-infra container.
Fixes#8567
Signed-off-by: Paul Holzinger <paul.holzinger@web.de>
Make sure we pass the ip and mac address as CNI_ARGS to
the cnitool which is executed in the rootless-cni-infra
container.
Signed-off-by: Paul Holzinger <paul.holzinger@web.de>
Followup to dbb9943
Despite skipping the `Smoke` check, it was observed on a *new* branch,
the `validate` task (specifically `git-validation`) will fail. This
is because:
* `$CIRRUS_LAST_GREEN_CHANGE` will be empty on a new branch.
* `$CIRRUS_BASE_SHA` is always empty for runs triggered by branch-push
* `$EPOCH_TEST_COMMIT` will be set to `YOU_FOUND_A_BUG`.
Fix this by eliminating the `Smoke` task entirely, simplifying all
the `make validate` operations into the `validate` cirrus task. Ensure
this task does not run when a new branch or tag is pushed.
Also, eliminate the `$CIRRUS_BUILD_ID` value as it's confusing and not
actually used anywhere. It was formerly used for building VM images,
but this has moved to another repo entirely.
Signed-off-by: Chris Evich <cevich@redhat.com>
It looks like we always hit the caching issue in Cirrus CI described
within #8313. A solution around that is to use cachix, which has been
pre-populated from my local machine.
To push all (runtime and build) dependencies, we can leverage a
pre-populated store by:
```
> nix-store -qR --include-outputs $(nix-instantiate nix/default.nix) | cachix push podman
```
The cache can be re-used by everybody to rapidly build static Podman
binaries: https://app.cachix.org/cache/podman
[NO TESTS NEEDED]
Signed-off-by: Sascha Grunert <mail@saschagrunert.de>
The podman documentation site uses javascript to display
API documentation at:
http://docs.podman.io/en/latest/Reference.html
As input, the javascript sources from a CORS-enabled Google Cloud
Storage object. This commit ensures the storage object is present and
updated for every Cirrus-CI execution context: Tags, Branches, and PRs.
As of this commit, the documentation site only utilizes the object
uploaded by the Cirrus-CI run on the `master` branch:
`swagger-master.yaml`. The file produced and uploaded due to a PR is
intended for testing purposes: Confirm it's generation and uploading are
both functional.
Signed-off-by: Chris Evich <cevich@redhat.com>
Make sure that bindings are in sync with the code. The check is similar
to what's already being done with `make vendor`, so integrate the two.
[NO TESTS NEEDED]
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Chris Evich
cdoern
openshift-ci[bot]
Ed Santiago
Erik Sjölund
Jason T. Greene
OpenShift Merge Robot
Daniel J Walsh
tomsweeneyredhat
Andrin Brunner
Kir Kolyshkin
Valentin Rothberg
Ashley Cui
Giuseppe Scrivano
jason
Matthew Heon
Jhon Honce
Paul Holzinger
Chmouel Boudjnah
Lokesh Mandvekar
Valentin Rothberg
José Guilherme Vanz
Jindrich Novy
Easton Man
Matthew Mosesohn
Easton Pillay
jesperpedersen
rugk
Blake Burkhart
Boaz Shuster
Paul Holzinger
Matthew Heon
Sascha Grunert