Commit Graph

664 Commits

Author SHA1 Message Date
OpenShift Merge Robot dc58d4e285
Merge pull request #8241 from rhatdan/tmpfile
Use /tmp/podman-run-* for backup XDG_RUNTIME_DIR
2020-11-06 15:43:33 +00:00
Daniel J Walsh 3daef2e826
Use /tmp/podman-run-* for backup XDG_RUNTIME_DIR
We need to block systemd from cleaning up this directory
by dropping a /usr/lib/tmpfiles.d/podman.conf file in place.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-11-04 15:32:00 -05:00
Chris Evich 46498331a3
Cirrus: Use F33beta VM image
Includes disk-space increase for all Fedora images to accommodate
the static-build job disk space requirements.  This job substantially
leverages task-cache, which was previously failing to restore early on
in the Cirrus-CI task setup, due to disk-space limitations.

Also simplify .cirrus.yml slightly by removing an unncessary setup
and run directory change step.

Signed-off-by: Chris Evich <cevich@redhat.com>
2020-11-04 09:40:38 -05:00
Chris Evich 0ebee0ce8e
Cirrus: Workaround F32 BFQ Kernel bug
Signed-off-by: Chris Evich <cevich@redhat.com>
2020-10-30 09:15:56 -04:00
OpenShift Merge Robot e439aec4fa
Merge pull request #8046 from cevich/simplify_environment
Cirrus: Simplify setting/passing env. vars.
2020-10-29 18:07:29 +01:00
Chris Evich 3ba77a5618
Cirrus: Simplify setting/passing env. vars.
Test VMs by design are to be single-purpose, single-use, and
readily disposable.  Therefore it's unnecessary to overcomplicate
storage of runtime environment variables.  This commit makes these
points clear, and reorganizes all CI-related env. vars on the system
into a single location, `/etc/ci_environment`.  This file is then
automatically loaded, and variables exported, (by `lib.sh`) from
`runner.sh` prior to executing all forms of testing.

Signed-off-by: Chris Evich <cevich@redhat.com>
2020-10-29 09:02:31 -04:00
Chris Evich 53fe386da0 Cirrus: Use google mirror for docker.io
Signed-off-by: Chris Evich <cevich@redhat.com>
2020-10-28 12:25:53 -06:00
Chris Evich 5d48606dae Cirrus: Always record runc/crun versions
Signed-off-by: Chris Evich <cevich@redhat.com>
2020-10-28 12:24:06 -06:00
Matthew Heon 1b4933376f Add a system test to verify --runtime is preserved
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-10-20 09:24:32 -04:00
Matthew Heon 3858fc1d01 Use runtime names instead of paths in E2E tests
My patches to fix `--runtime /usr/bin/crun` being allowed to use
a different version of the crun runtime revealed a problem: we
were actually relying on that exact behavior in our E2E tests. We
specified the runtime path as `/usr/bin/runc` for the Ubuntu
tests, but that didn't exist, so Podman was actively looking for
a different, usable runc binary and using that, instead of the
path we explicitly hardcoded. Fixing the bug broke this, and thus
broke the tests.

Instead of hard-coding OCI runtime paths, swap to just using the
runtime name, `runc` or `crun`, and letting Podman figure out
where the runtime lives - it's quite good at that. This should
un-break the tests and make them more durable.

Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-10-20 09:20:33 -04:00
Chris Evich 23087f77ae
Cirrus: Restore APIv2 Testing
Signed-off-by: Chris Evich <cevich@redhat.com>
2020-10-13 11:42:31 -04:00
Chris Evich 5b7c64cc97
Cirrus: Fix obtaining a CI VM
Also removed automatic exection of setup_environment.sh since most
people using this script are podman developers (not automation/CI
folks).  If executing the automation scripts is necessary, manual
attendance to required variables like `$TEST_FLAVOR` is mandatory.

Signed-off-by: Chris Evich <cevich@redhat.com>
2020-10-07 15:29:43 -04:00
Chris Evich ae0f655652
Cirrus: Fix running shellcheck locally
Also, check the contents of hack/get_ci_vm.sh

Signed-off-by: Chris Evich <cevich@redhat.com>
2020-10-06 16:01:24 -04:00
Ed Santiago e1125558a2 Cirrus CI runner: refactor
While reviewing #6784 I found myself having a lot of trouble
with this script: it was a complicated mix of case statement
and helper functions, requiring a reader to jump back and
forth between the two.

This PR defines a convention such that a given TEST_FLAVOR=foo
must have a corresponding _run_foo() handler function. The
goal is to have all TEST_FLAVOR-related code in one place,
or at least less scattered (integration and system tests
still rely on other helper functions).

Signed-off-by: Ed Santiago <santiago@redhat.com>
2020-10-06 11:14:29 -06:00
Ed Santiago 9546638a09 logformatter: run on system tests & bindings
(that got accidentally dropped in the new Cirrus makeover).
Note that 'dotest' does not actually 'do tests', it's
only used for a small subset of tests.

Also, make logformatter work better in the new Cirrus setup.
Remove duplicate test/subtest, remove no-longer-used SPECIALMODE,
and make the Cirrus build/task display a little cleaner.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2020-10-05 11:31:04 -06:00
Chris Evich 0f04cf27c5
Cirrus: Fix branch-validation failure
When validating code on a branch, determining a starting commit to check
from isn't as straightforward as it would seem.  Default to using the
SHA from last time CI was green.  If for some reason that isn't
available, use an obviously wrong value to cause an intentional
failure.  Entirely skip this check on tag-push, since determining a
starting point is incredibly difficult to do automatically.

Signed-off-by: Chris Evich <cevich@redhat.com>
2020-10-05 10:13:24 -04:00
Chris Evich 2c9084e224
Cirrus: Implement podman automation 2.0
Reimplement CI-automation to remove accumulated technical-debt and
optimize workflow.  The task-dependency graph designed goal was to
shorten it's depth and increase width (i.e. more parallelism).  A
reduction in redundant building (and 3rd party module download) was
also realized by caching `$GOPATH` and `$GOCACHE` early on.  This
cache is then reused in favor of a fresh clone of the repository
(when possible).

Note: The system tests typically execute MUCH faster than the
integration tests.  However, contrary to a fail-fast/fail-early
principal, they are executed last.  This was implemented due to
debug-ability related concerns/preferences of the primary
(golang-centric) project developers.

Signed-off-by: Chris Evich <cevich@redhat.com>
2020-10-02 11:53:04 -04:00
OpenShift Merge Robot 1784f4f338
Merge pull request #7873 from edsantiago/logformatter_env
logformatter: add Synopsis at top of each page
2020-10-02 11:24:31 -04:00
Ed Santiago d380ba0394 logformatter: add Synopsis at top of each page
At the top of each generated page, add a Synopsis table with:

  PR number/name, and link to github
  Author name(s)
  Test name (fedora/ubuntu, rootless, etc)
  Cirrus build ID (usually uninteresting)
  Cirrus task ID (usu. important), with link to Cirrus
  The value of $SPECIALMODE

This is all we can get from the Cirrus environment in
which logformatter runs; we can't get things like
cgroup manager or username that the test runs under.

Note that the table is at the top, which is usually
unseen because we autoscroll to the bottom on
page load. I tentatively think that top is a more
natural place for this info than bottom, but am
willing to listen to arguments against.

Also, one minor tweak: highlight podman commands in
the BATS output. The idea is to make it easier for the eye
to spot those, then copy/paste them to find a reproducer.

And, sigh, disable the new 'podman network create'
system test. It is flaking much too much.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2020-10-01 20:33:20 -06:00
Chris Evich 7c9c159e93
Migrate container images to automation_images
Signed-off-by: Chris Evich <cevich@redhat.com>
2020-10-01 14:27:32 -04:00
Akihiro Suda eb56673b3b
rootless-cni-infra v3: fix cleaning up DNS entries
Fix "Old DNS entries are not cleaned up" by passing CNI_ARGS to `cnitool del`.

Fix #7789

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-09-30 18:59:39 +09:00
Matthew Heon 20d90320fc Bump to v2.2.0-dev
Signed-off-by: Matthew Heon <mheon@redhat.com>
2020-09-22 11:16:48 -04:00
Akihiro Suda 7bffd49ade
rootless-cni-infra: fix flakiness during bringing up lo interface
Fix #7618

This bumps up ROOTLESS_CNI_INFRA_VERSION to 2

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-09-20 00:29:23 +09:00
Ed Santiago 5095a34135 dependabot-dance: new tool for managing revendor PRs
dependabot seems to submit PRs without running 'make vendor'.
This script automates (with some safety checks) the manual
process for pulling the PR, running 'make vendor-in-container',
and force-pushing the PR.

Usage: ./contrib/dependabot-dance

It should take care of identifying your github repo, finding
all active dependabot branches, running the make, git-add,
and commit, then git-pushing.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2020-09-17 05:22:49 -06:00
Valentin Rothberg d4bea54dfe move `rootless-cni-infra` image to quay.io
Move the `rootless-cni-infra` image to `quay.io/libpod/rootless-cni-image:$tag`
where $tag has the format `$version-$architecture`.  Whenever we upload
a new image (e.g., after changing the Containerfile), we need to make
sure to increase the version number (an ordinary integer for simplicity) so we
have a notion of support.

Thanks to @AkihiroSuda for working on rootless CNI!

Fixes: #7617
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-09-15 10:14:34 +02:00
Daniel J Walsh 526f01cdf5
Fix up errors found by codespell
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-09-11 06:14:25 -04:00
OpenShift Merge Robot 2f0e803e76
Merge pull request #7460 from AkihiroSuda/allow-rootless-cni
rootless: support `podman network create` (CNI-in-slirp4netns)
2020-09-10 14:00:49 -04:00
Chris Evich ed1e87ecb5
Cirrus: Obsolete CI:IMG process & related files
All VM-building functionality has been migrated to
https://github.com/containers/automation_images

Some container-build functions are still maintained here
but are on a very-short list to also be migrated to
the repository linked above.

Signed-off-by: Chris Evich <cevich@redhat.com>
2020-09-09 13:46:52 -04:00
Akihiro Suda d34868a136
add contrib/rootless-cni-infra
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-09-09 15:45:56 +09:00
OpenShift Merge Robot 11679c27a7
Merge pull request #7538 from edsantiago/cap_test_robust
Update VM images for new crun; adapt Cap tests to work with new kernel
2020-09-08 10:40:39 -04:00
TomSweeneyRedHat 38f7084d35 [CI:DOCS] Add note on run image fuse problem - try 2
We've recently had a number of issues reported against our
pre-fabricated images on quay.io and a couple of rhel repositories
throwing a fuse error when run:
```
fuse: device not found, try 'modprobe fuse' first
```

The tip on modprobe fuse is not always seen by or displayed to
the end user.  Adding a couple of doc pointers to hopefully help.
Arises from this BZ: https://bugzilla.redhat.com/show_bug.cgi?id=1867892
and several others.

Replaces: 7453 where I was going crazy with whitespace and merge issues.

Signed-off-by: TomSweeneyRedHat <tsweeney@redhat.com>
2020-09-04 07:18:09 -04:00
Ed Santiago 36caf4ee44 WIP: update VM images
(This is an adoption of #7533 because Brent is on PTO).

Pick up new crun and crio-runc.

Also: renames from useful fedora-32 and -31 to less-useful
names; presumably this is needed by something-something in
the new VM setup.

Also: tweak two e2e tests to more properly handle a kernel
(5.8.4) with a greater set of capabilities than what we
or crun can yet handle.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2020-09-03 14:08:34 -06:00
Brent Baude 053cbd5e0c use crio runc on CICID ubuntu
when running CICD on Ubuntu where no cgroups v2, we need to use a newer runc for things like seccomp and the default ubuntu runc
 is not new enough.

Signed-off-by: Brent Baude <bbaude@redhat.com>
2020-08-31 16:40:22 -05:00
Sascha Grunert ba9f18e2b8
Use `bash` binary from env instead of /bin/bash for scripts
It's not possible to run any of the scripts on distributions which do
have `bash` not in `/bin`. This is being fixed by using `/usr/bin/env
bash` instead.

Signed-off-by: Sascha Grunert <sgrunert@suse.com>
2020-08-17 10:42:23 +02:00
OpenShift Merge Robot ca4423e94d
Merge pull request #7237 from TomSweeneyRedHat/dev/tsweeney/imagedoc
[CI:DOCS] Update podmanimages README.md
2020-08-15 12:48:06 +02:00
Valentin Rothberg 0f4e2be073 podman.service: use sdnotiy
Commit 2b6dd3fb43 set the killmode of the podman.service to the
systemd default which ultimately lead to the problem that systemd
will kill *all* processes inside the unit's cgroup and hence kill
all containers whenever the service is stopped.

Fix it by setting the type to sdnotify and the killmode to process.
`podman system service` will send the necessary notify messages
when the NOTIFY_SOCKET is set and unset it right after to prevent
the backend and container runtimes from jumping in between and send
messages as well.

Fixes: #7294
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-08-13 17:53:59 +02:00
OpenShift Merge Robot 8e4842a14d
Merge pull request #7299 from jobcespedes/patch-1
add xz as a recommended pkg
2020-08-12 13:08:54 -04:00
Brent Baude 6ff42395d0 podman-remote fixes for msi and client
correct small typo that sets the path on windows via the msi xml.

in the remote client, prompt for SSH password when no identity or alternate means of authentication are provided.

Signed-off-by: Brent Baude <bbaude@redhat.com>
2020-08-12 09:19:52 -05:00
Job Cespedes c817e5ab2b
add xz as a recommended pkg
xz package is required by buildah and podman when building a
image and ADD a tar.xz file archive is used

See https://github.com/containers/buildah/issues/2525

Signed-off-by: Job Cespedes Ortiz <jobcespedes@gmail.com>
2020-08-11 14:57:34 -06:00
OpenShift Merge Robot df0ad51075
Merge pull request #7270 from Fodoj/master
Allign container image storage configuration with Buildah
2020-08-10 19:41:30 -04:00
Kirill Shirinkin 330e3d3728
Align images with Buildah
Signed-off-by: Kirill Shirinkin <kirill@hey.com>
2020-08-10 22:15:55 +02:00
Daniel J Walsh 49fa3e7399
Remove TEST_REMOTE_CLIENT from RCLI
We know these are TEST_, hoping this makes the display in
cirrus easier for users to see true|false, since this is the
valuable information is.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-08-10 12:47:43 -04:00
OpenShift Merge Robot 162625f2e7
Merge pull request #7138 from cevich/add_python_packages
Cirrus: Add python packages to images
2020-08-10 11:52:45 -04:00
TomSweeneyRedHat cc26814f8d [CI:DOCS] Update podmanimages README.md
Updates to the README.md for the contrib/podmanimages directory.

This completes the changes to answer this Buildah issue: https://github.com/containers/buildah/issues/1693
and then also adds the quay.io/conatiners/podman images to the list of images.

Signed-off-by: TomSweeneyRedHat <tsweeney@redhat.com>
2020-08-05 16:41:50 -04:00
OpenShift Merge Robot 2e3928ee17
Merge pull request #7193 from vrothberg/fix-7190
podman.service: drop install section
2020-08-03 10:07:51 -04:00
Valentin Rothberg efcc2f5b18 Install auto-update services for users
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-08-03 13:12:07 +02:00
Peter Oliver 4384de9e6b Fix test failure regarding unpackaged files.
Signed-off-by: Peter Oliver <git@mavit.org.uk>
2020-08-03 13:12:06 +02:00
Valentin Rothberg 7dadf1b46e podman.service: drop install section
podman.service is socket activated through podman.socket. It should not
have its own [Install] section, it does not make sense to systemctl
enable podman.service.

This leads to podman.service always running on a Debian system, as
Debian's policy is to enable/start running services by default.

We don't want a daemon :^)

Fixes: #7190
Reported-by: @martinpitt
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-08-03 09:48:13 +02:00
Chris Evich ba4a1e64db
Cirrus: Install golang 1.14 on Ubuntu
This more/less reverts efd142214 + updates to 1.13
on all Ubuntus for all `containers` projects.

Signed-off-by: Chris Evich <cevich@redhat.com>
2020-07-31 10:10:24 -04:00
Chris Evich 543606c55c
Cirrus: Add python packages to images
They are needed in support of future testing additions.

Also reduce unnecessary output by not printing the downloaded package
list.  The set can be examined using other tooling if/when necessary.

Signed-off-by: Chris Evich <cevich@redhat.com>
2020-07-31 10:10:18 -04:00