This allows the containers/conmon repository to share the same VM
images produced by containers/libpod. Included are several packages
which are downloaded only since they might otherwise interfere with
testing for some repos. This allows stable versions to be at the ready
at testing runtime, avoiding any version updates surprising developers.
Also, re-enable running the VM-image check test which was not working
due to a logic problem in Cirrus-CI configuration. Update the neglected
tests so that they pass on all distros.
Signed-off-by: Chris Evich <cevich@redhat.com>
This properly prints out image-name hints when executing the hack script
without any arguments. It is required due to changes made by Ed for
test-name beatification. An identical change was made and reviewed by
Ed in the containers/storage repo.
Signed-off-by: Chris Evich <cevich@redhat.com>
Now that we're shipping containers.conf, we don't want to provide
a libpod.conf anymore. This removes libpod.conf from the repo and
as many direct uses as I can find.
There are a few more mentions in the documentation, but someone
more familiar with containers.conf should make those edits.
Signed-off-by: Matthew Heon <mheon@redhat.com>
Now that it's officially released, update to it from the beta. Also
(and significant), adjust the SELinux context of the GCP metadata
service. Add a comment to the code explaining why this is necessary.
Signed-off-by: Chris Evich <cevich@redhat.com>
Many of the packages required for CI in buildah overlap with libpod.
When building new VM images, attempt to source a package list
from the buildah repository. If found, also install the listed
packages on the VM.
Signed-off-by: Chris Evich <cevich@redhat.com>
A number of scripts relating to tooling used and the gate container
image were not exiting upon errors as intended. Coupled with
external service unavailability (i.e. downloading golangci-lint)
was observed to cause difficult to debug failures.
This change corrects the scripts inside/out of the gate container as
well as fixes many golang related path consistency problems vs other CI
jobs. After this change, all jobs use consistent path names reducing
the number of special-case overrides needed.
Lastly, I also made a documentation-pass, updating/correcting as needed,
including documenting a likely local validation-failure mode, related to
`$EPOCH_TEST_COMMIT`. This is dependent on the developers git
environment, so documentation is the only possible "fix".
Signed-off-by: Chris Evich <cevich@redhat.com>
Suspect crun might be sneaking in during VM image build via podman RPM
dependency. Add it to the removal list when building, then also force
use of runc at runtime in F30.
Also quote all true/false vars to force them as strings instead of
booleans (which will become capitalized)
Signed-off-by: Chris Evich <cevich@redhat.com>
VM Base images are used as a starting point for runtime VM images.
The in-use VM base images should never be pruned, which is an
operation that potentially occurs periodically from automation
running on the master branch of the libpod repo.
However the only place which updates timestamps (blocking pruning)
of base images, occurs during runtime VM image building. Therefor,
if images are not regularly rebuilt, it's possible their base images
go stale and are pruned.
Changes:
* Add freshly-produced base images (old ones got pruned)
* Wrap the timestamp update script to include base image names
in the update list.
Notes:
* Regularly updating base image timestamps only needs to happen
on the libpod repo's meta task, since all base images live there.
* Using a wrapper is needed to maintain compatibility with multiple
versions of the imgts container image used by other repos / branchs.
Signed-off-by: Chris Evich <cevich@redhat.com>
Cirrus-CI only sets `$CIRRUS_BASE_SHA` when testing PRs since the
destination and it's state is easy to discover. However, when
post-merge and/or tag-push testing, the previous state is not
easily discoverable (changes have already merged). The
`upload_release_archive` script incorrectly assumed this
variable was always set, causing a constant stream of post-merge
testing failures.
Tweak the `is_release()` function to properly handle an empty
`$CIRRUS_BASE_SHA` whether or not `$CIRRUS_TAG` is also set. Also
update the unit-tests to check for this. Also account for a
corner case where hack/get_ci_vm.sh is running on a VM w/o git.
Signed-off-by: Chris Evich <cevich@redhat.com>
Prior to this commit, every push to master had it's builds packaged and
uploaded to google storage. This is a waste, since potential users
are only ever concerned about tagged releases.
Unfortunately because the release process involves humans with
potentially multiple human and automation steps happening in parallel,
it's easy for automation to not detect a tagged release, or trigger on
development|pre-release tags.
Fix this in `upload_release_archive.sh` using a new unit-tested
function `is_release()`. This acts as the definitive authority
on whether or not a specific commit rage or `$CIRRUS_TAG` value
constitutes something worthy of upload.
Signed-off-by: Chris Evich <cevich@redhat.com>
When executing 'make remotesystem' testing, a varlink process is started
up but it's stdio is dumped due to the production of excessive data.
However, this also means if the process has a problem, any errors will
not be accessible.
Instead, grab only the last 100 lines and direct them into a file. Also
update automation's log collection to retrieve this file when the
`$REMOTE_CLIENT` env. var. is `true`.
Signed-off-by: Chris Evich <cevich@redhat.com>
This is needed because older versions of podman (1.5.1) do not
automatically install the new conmon package.
Also, include removal of `/usr/libexec/podman/conmon` when preparing to
install and test podman built from source.
Signed-off-by: Chris Evich <cevich@redhat.com>
Add the latest Ubuntu version into the testing matrix
and image-build workflow. This is also needed to support
other containers projects which share use of VM images
from this one.
Update package lists to include needs for contianers/storage
use of images.
Signed-off-by: Chris Evich <cevich@redhat.com>
When constructing VM cache-images, the latest/greatest podman package is
installed to ensure all necessary dependencies are met. Prior to
testing source-built binaries, most of of the packaged files are removed.
However, if the `io.podman` service or socket is enabled/running, it
could cause the packaged podman and varlink binaries to be both resident
and cached. Since this condition would cause very difficult to diagnose
behaviors, add preventative measures to ensure these services are absent
prior to removing packaged podman files.
Signed-off-by: Chris Evich <cevich@redhat.com>
Rather than spamming the podman channel with impersonal success
messages referring to PR numbers, mention the author by nick name
and include the PR title and link.
Also avoid needless logging of all bot-script interactions with
IRC when there is no error detected.
Signed-off-by: Chris Evich <cevich@redhat.com>
The initial implementation was far more complicated than necessary.
Strip out the complexities in favor of a simpler and more direct
approach.
Signed-off-by: Chris Evich <cevich@redhat.com>
Specifically pertaining to executing tests in google cloud, there are
default, pre-allocated class-a subnetworks for each region (data-center).
Each includes a gateway using a `.1` LSB and all are routable from other
regions in google cloud via these gateways.
Because the default CNI configuration also utilizes class-a subnetworks,
this creates the possibility for IPv4 address-space clashes. Since the
default regional cloud subnets are pre-defined/known, preventing clashes
can be accomplished by seeding these subnets in a dummy CNI configuration.
The default behavior of podman is to grab the highest priority CNI
configuration. Name the dummy config. appropriate so it always loads
last. Also name the bridge itself with an obvious name `do-not-use`,
such that any related testing errors should be easier to debug.
Also:
* Minor cleanup of `install_test_configs()`
* Move install_test_configs in `setup_environment.sh` to after possible
run of `remove_packaged_podman_files()` because that also strips out
`/etc/cni/net.d/87-podman-bridge.conflist`.
Signed-off-by: Chris Evich <cevich@redhat.com>
This is mainly/initially to support use of Cirrus-CI
in https://github.com/containers/buildah since that setup
re-uses the VM images from this project. However, it also
opens doors here, if libpod ever needs/wants to do things
with a dedicated storage device and/or storage-drivers.
Signed-off-by: Chris Evich <cevich@redhat.com>
It's somewhat hard to predict which units are certinly present
for any given base-image. Therefore, at image-build time, it's
distracting and unhelpful to see all the errors about units that
don't exist, on every platform. Simply ignore them and rely on
the `check_image.sh` test to confirm none are enabled.
Signed-off-by: Chris Evich <cevich@redhat.com>
For CI testing, it's important to remove as much variability from the
overall system as possible. This permits focusing just on problems
closely related to code-changes. To this end, and because VMs are very
short-lived (2 hours at most), disable all systemd services and timers
which perform periodic activities.
Signed-off-by: Chris Evich <cevich@redhat.com>
Various tasks and scripts behave differently depending on whether or not
the build is running against a PR or on a branch, post-merge. However,
a great number of them are hard-coded to the string 'master' as the
destination. Since this is not always the case (there are other
relevant branches), it makes sense to abstract the references with a
single definition.
Add a top-level `$DEST_BRANCH` variable to CI, and otherwise
default to 'master' when unset. This enables running CI builds on
additional branches without the overhead of updating all the static
references to 'master'. Simply update `$DEST_BRANCH` at the top-level
and all branch-conditional logic will function as intended.
Signed-off-by: Chris Evich <cevich@redhat.com>
It's desirable to make archives available of builds containing actual
tested content. While not official distro-releases, these will enable
third-party testing, experimentation, and development for both branches
(e.g. "master") and pull requests (e.g. "pr3106").
* Add a Makefile targets for archiving both regular podman binaries
and the remote-client. Encode release metadata within these
archives so that their exact source can be identified.
* Fix bug with cross-compiling remote clients for the Windows and Darwin
platforms.
* Add unit-testing of cross-compiles for Windows and Darwin platforms.
* A few small CI-script typo-fixes
* Add a script which operates in two modes:
1. Call Makefile targets which produce release archives.
Upload the archive to Cirrus-CI's built-in caching system
using reproducible cache keys.
2. Utilize reproduced cache keys to attempt download of cache
from each tasks. When successful, parse the file's
release metadata, using it to name the archive file. Upload
all recovered archives to a publicly accessible storage bucket
for future reference.
* Update the main testing task to call the script in mode #1 for
all primary platforms.
* Add a new `$SPECIALMODE` task to call the script in mode #1 for
Windows and Darwin targets.
* Add a new 'release' task to the CI system, dependent upon all other
tasks. This new tasks executes the script in mode #2.
* Update CI documentation
Signed-off-by: Chris Evich <cevich@redhat.com>
Building/installing dependencies from fixed source-version ensures
testing is reliable, but introduces a maintenance burden and
risks testing far outside of a real-world environment. The
sensible alternative is to install dependencies from distro-packaging
systems.
Install all development and testing dependencies at VM cache-image build
time, to help ensure testing remains stable. The existing cache-image
build workflow can be utilized at any future time to build/test
with updated packages.
***N/B***: This does not update any dockerfiles used by testing, that is
left up to future efforts.
Signed-off-by: Chris Evich <cevich@redhat.com>
The original solution using --wait does not function on F30, waiting
forever. Replace it with a simple 5-minute timeout loop.
Signed-off-by: Chris Evich <cevich@redhat.com>
Remove disused `build_cache_images` task, and
update relevant dockerfiles for F30.
Fix problem of cloud-init failing to expand root-device on boot
(/var/lib/cloud/instance left in improper state).
Fix problem of cloud-init racing with google-network-daemon.service on
boot (looking for cloudconfig metadata too early). Causing
root-device to _sometimes_ fail to expand.
Fix problem of hack/get_ci_vm.sh argument passing.
Signed-off-by: Chris Evich <cevich@redhat.com>
New base-image boots, a cache-image builds, but more work is needed for
it to be prime-time ready. This commit just adds some updates to the
scafolding necessary to build the base-image. Future work will make F30
more of a reality.
Also add log-collection scripts to test image verification task
Signed-off-by: Chris Evich <cevich@redhat.com>
Previously, it was quite difficult to affect changes to VM cache images
without lots of manual work. This commit adds a new optional testing
task which mirrors the official-image build task which only runs on
master. In contrast, the new task may be run at any time in a PR, but
including a magic phrase in the PR description.
Update documentation to describe the new task and inform on it's usage.
Signed-off-by: Chris Evich <cevich@redhat.com>
There were some build issues updating cri-o to cri-o/cri-o. Since the only thing we need cri-o for is conmon, we should just build using conmon.
Signed-off-by: Peter Hunt <pehunt@redhat.com>
Chris Evich
Matthew Heon
Dmitry Smirnov
OpenShift Merge Robot
baude
Peter Hunt