This service is meant to be used by quadlet as replacement for
network-online.target as this does not work for rootless users.
see #22197
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
The reason being that I plan to add a unit that should only be used for
the user session and otherwise there is no way to only keep a unit in
user.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Accidentally introduced in #21639.
Thanks to Paul for the Python code to prevent this from
happening again.
Signed-off-by: Ed Santiago <santiago@redhat.com>
This reverts commit 43f6173cc6.
The netavark version with nftables default is in f41 and rawhide
already so this is no longer needed. While we do not yet test f41 in CI
we have rawhide which is good enough until we update.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
The repo tar process took over 1:20 min, with zstd it takes less than
10s so we safe over a minute by doing this.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
In particular the main build task already did a make vendor and a
regeneration of the completion scripts. This means the first tre_status
would pick up both changes so the suggestion would be off. And rerunning
the same thing again here just makes thing slower than they need to be.
In particular there was the bug that make completion even rebuild podman
because generate-bindings obviously updates the timestamps of the files
as they are overwritten.
We do however must run generate-bindings as it was not run before.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
The current podman-release-%.tar.gz target does a lot more then just
checking if we can build for the given arch, in particular it first
builds a local podman-remote for the remote-docs.sh script. This makes
things slow as we compile several things and then builda and package the
docs. Given the docs are not arch specific there is realy no point in
doing all that work. All we care about is if the bianries can build on
other arches to catch compile issue for otherwise untested arches.
This should make the CI Alt Arch. tasks much faster.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
The doc generation and the validate-binaries target can be run in
parallel as they do not depend on each other and a specific ordering. As
such we pass -j $(nproc) but also --output-sync=target to ensure the
output is not intermixed between several targets which could be harder
to read in case of errors.
Hower dus the complex podman-release target we can run podman-release
and validate-binaries at the same time as the dependencies are not right
and we run podman-release first in order to get the correct binaires
build.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
In order to get better debug data for cleanup flakes. The argv is
printed with 0 bytes so replace them with spaces to make the log
readable for humans.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
For tests run in parallel, show file number as |nnn| (vs [nnn])
Teach logformatter to distinguish the two, adding 'p' to anchors
in parallel tests. Necessary because in this scheme we run bats
twice, thus see 'ok 1' twice, and we want to differentiate them.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Minor bump. Fedora VMs now include ShellCheck, so we can
remove the 'dnf install' at CI run time.
Also, FWIW, Debian *vark are now at 1.12 (from 1.9)
VMs built in https://github.com/containers/automation_images/pull/385
Signed-off-by: Ed Santiago <santiago@redhat.com>
This check has a condition on the distro name to only run once, however
the prior fedora version doesn't have to exists necessarily as we might
have to drop support there due the outdated golang version.
The current fedora version should alway exists so this seems safer.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Since commit 55ad0d6e0e we do the conditions in the cirrus.yml directly
so there is no longer any need for this.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
The renovate config is used for the renovate bot, validating this in the
prior fedora prebuild setp is just confusing and hidden.
The problem is this image is very big so it is slow to download/extract.
To speed things up given it is only a single file we check the diff if
we even changed it.
Now one could argue this should be part of the validate Makefile target
but I given the size I do not want this run by default and I am not sure
if we should do the diff check in the Makefile.
Lastly remove -it, these is meant for interactive use and throws a
warning here because we have no actual tty attached.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
This doesn't help us at all, first the list is outdated. AFAICT we no
longer connect to docker.io, registry.fedoraproject.org or
podman.cachix.org (seems to be a cache site for nix.dev?) anywhere in
our tests.
Second a simple port check is not helpful, in the most cases the
CDN's and or load balancer accept connections but return internal server
errors when the registy goes down.
This is very similar to commit 5b6de98ee8.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
In the RHEL specific branches we want to ensure that all MRs link to
at least one downstream Jira ticket. To do this we add a new test in
validate-source similar to the existing pr-should-include-tests. This
test only runs on actual pull requests.
The syntax for linking to a Jira is "Fixes " or "Fixes: ", followed by
one jira links, like so:
```
Fixes https://issues.redhat.com/browse/RHEL-50506
Fixes: https://issues.redhat.com/browse/RHEL-50506
```
Note: This is the same syntax as for a regular github issue reference.
Signed-off-by: Alexander Larsson <alexl@redhat.com>
CI will fail if quay is down, but a build-time check does not
help us in any way. It just introduces another pain point
where we have to hit the Rerun button.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Previously there were two CI tasks that ended up both testing docker-py
compatibility. Remove the duplicate from the `localapiv2-python` make
target, and symlink the identical requirements file.
Signed-off-by: Chris Evich <cevich@redhat.com>
Previously, if anyone touched these files no extra testing would
trigger. However, basically all testing depends on them. Update the
condition and test that verifies it.
Signed-off-by: Chris Evich <cevich@redhat.com>
Add generate helper function.
Also, add a troubleshooting try/catch block in case we get more flakes
during Set-LocalUser step in Windows powershell.
Resolves: https://github.com/containers/podman/issues/23468
Signed-off-by: Nicola Sella <nsella@redhat.com>
Use Schedule "afterInstallExecute" (instead of the
default "afterInstallValidate") in the Windows
installer MajorUpgrade element. That avoid
overriding eventual users changes to the podman
machine configuration file created by the
installer.
Fixes#23502
Signed-off-by: Mario Loriedo <mario.loriedo@gmail.com>
It's too difficult to keep the podman-machine image up-to-date.
And, we can't use the cache on Mac/Windows, so if quay is down
we're hosed no matter what.
Add a "nocache" mechanism to install_test_configs() and use that
in machine test setup.
Signed-off-by: Ed Santiago <santiago@redhat.com>
This contains a fix for a gvproxy crash on macos on fast connections
with heavy network load.
This should fix https://github.com/containers/podman/issues/23114
Signed-off-by: Christophe Fergeau <cfergeau@redhat.com>
Plus, I think my ampersand-quot change earlier this month
caused problems for firefox. We no longer need it (pull-option
does not need the funky double-quoted curly-brace string),
so, remove it.
Signed-off-by: Ed Santiago <santiago@redhat.com>
New tool, get-local-registry-script, intended for developers
to get a local registry running in their environment. This is
not necessary for any tests, but may be desirable for performance
reasons and/or to recreate the CI environment.
Signed-off-by: Ed Santiago <santiago@redhat.com>
This commit gets tests working under the new local-registry system:
* amend a few image names, mostly just sticking to a consistent
list of those images in our registry cache. Mostly minor
tag updates.
* trickier: pull_test: change some error messages, and remove
a test that's now a NOP. Basically, with a local (unprotected)
registry we always get "404 manifest unknown"; with a real
registry we'll get "403 I can't tell you".
* trickiest: seccomp_test: build our own images at run time,
with our desired labels. Until now we've been pulling
prebuilt images, but those will not copy to the local
cache registry. Something about v1? Anyhow, I gave up
trying to cache them, and the workaround is straightforward.
Also took the liberty of strengthening a few error-message checks
Signed-off-by: Ed Santiago <santiago@redhat.com>
As of https://github.com/containers/automation_images/pull/357
our CI VMs include a local registry preloaded with all(*)
images used in tests.
* where "all" means "most".
This commit installs a new registries.conf that redirects docker
and quay to the new local registry. The hope is that this will
reduce CI flakes.
Since tests change over time, and new tests may require new
images, this commit also adds a mechanism for pulling in
remote images at test run time. Obviously this negates
the purpose of the cache, since it introduces a flake
pain point. The idea is: DO NOT DO THIS UNLESS ABSOLUTELY
NECESSARY, and then, if we have to do this, hurry up and
spin new CI VMs that include the new image(s).
Signed-off-by: Ed Santiago <santiago@redhat.com>
Run root e2e & system tests using composefs on rawhide.
Write magic settings to storage.conf. That part is easy.
e2e tests, however, ignore storage.conf. They require everything
to be specified on the command line. And "everything", in the
case of composefs, includes a long complicated --pull-options
string which in turn requires containers-storage PR 1966
which, as of this writing, is finally vendored into podman.
Signed-off-by: Ed Santiago <santiago@redhat.com>
This test flakes frequently and its status is completely ignored in CI.
At the time of this commit, nobody has stepped up to debug or fix it.
Drop the test.
Signed-off-by: Chris Evich <cevich@redhat.com>
When we check if source code was changed also include header files.
There is only one header file currently but that can change and it may
be possible that changes in this file can break things so make sure it
is considered source code so that all tests are triggered.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Building the MSI hook on Windows
(`contrib/win-installer/podman-msihooks/check.c`)
currently requires MinGW. This commit updates the build
script so that, when MinGW is absent but the C compiler
included in Visual Studio BuildTools is installed, the
latter is used to build the MSI hook.
Other than that, `winmake.ps1` has a new `installertest`
target to run the Windows installer tests that are
currently verified by Cirrus CI.
Signed-off-by: Mario Loriedo <mario.loriedo@gmail.com>
This directory contains important tools such as ginkgo as such updates
there should run through all testing and not skip anything.
Technically we do not need to run system tests as it doesn't use any
tool from there but that
a) might change in the future and
b) would make the only_if rules much more complicated if we try to
exclude it and
c) updates in test/tools are rare and/or automated so it does not cause
inconveniences to run all anyway
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
The chocolatey tool that was fetching us wix v3 can no longer be used to
fetch wix v4+ so we had to switch to dotnet to fetch the latest wix.
This commit builds the installer with wix v5.
wix v5 is installed via the `dotnet` tool in the windows image itself
at https://github.com/containers/automation_images/pull/354.
Going forward, the `dotnet` tool will also be used to build the installer.
In the process, the wix v3 files were converted to wix v4+ using `wix
convert` followed by manual modifications along with switch to wixproj
builds with dotnet.
The GitHub Action to upload windows installer now builds the installer
using winmake.ps1.
Contributions from Mario Loriedo:
- bundle setup update to wix5
- updates to build and release process scripts
Ref: https://github.com/lsm5/podman/pull/3
- small fixes to windows installer theme
Ref: https://github.com/lsm5/podman/pull/4
- Better win-installer sidebar logo
Ref: https://github.com/lsm5/podman/pull/5
Resolves: RUN-2055
Co-authored-by: Mario Loriedo <mario.loriedo@gmail.com>
Signed-off-by: Lokesh Mandvekar <lsm5@redhat.com>
As we want to get rid of the special titles convert the existing skips
to the only_if condition, this makes it more readable as we do not need
to negate so much.
Then add similar conditions for all test tasks, this removes the need to
a special title such as CI:DOCS as the logic is smart enough to only
docs changes when no source code was changed.
Update the documentation for the new logic and no longer point
contributors to the CI:DOCS title as it is gone now.
There is a bunch of duplication in the rules as yaml doesn't allow us to
share only parts of a string. To prevent unwanted drift a test case in
contrib/cirrus/cirrus_yaml_test.py is added to ensure all conditions
follow the same base ruleset.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Try to speed up the CI tests by using tmpfs as container storage.
This is important for system tests, other tests setup their own --root
already on tmpfs so it should not effect them.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Linting code changes with golangci-lint is a very slow and resource
intensive process. However, it does not depend on compiling anything.
This means it may run in parallel with the build tasks for
a modest perceived runtime duration improvement.
Additionally, the former validation make targets that **do** require a
build execute faster than CI is able to provision a VM, simply tack them
onto the end of all build operations.
Signed-off-by: Chris Evich <cevich@redhat.com>
As of commit d9183f0587 we use the cirrus.yml skip logic to skip based
on source changes. As such the add hoc logic inside our test setup can
be removed. However as I did not yet implement the skip logic for all
tests task in cirrus.yml it must remain for the other tasks for now.
I plan to migrate the other in a week or two once we are confident that
the cirrus.yml logic works well for us.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Two enormous misunderstandings:
1) $CIRRUS_BASE_SHA is worthless. I thought it was, you know,
the BASE SHA of the current commit, but (as best I can tell)
it seems to be the SHA of the most recent commit on the
destination branch. Cirrus docs are unhelpful. Anyhow,
it's clearly not anything useful. Stop using it.
2) $EPOCH_TEST_COMMIT is closer to what we want. It is
defined in Makefile as the git merge-base. But for unknown
reasons it was being clobbered in CI scripts, and it
doesn't seem to work in all contexts, so, eliminate it
from CI setup scripts. Leave it only in Makefile.
This leaves us with no option other than defining our own
merge-base variable, PR_BASE_SHA. Do so and pass it along
to rootless jobs.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Extend Makefile and package.sh to download, sign and bundle krunkit and
its dependencies into the package.
Signed-off-by: Sergio Lopez <slp@redhat.com>
With (esp. Debian) CI VM images built by
https://github.com/containers/automation_images/ pull/338 CI no-longer
tests with runc nor cgroups v1. Add logic to fail under these
conditions. Prune back high-level YAML/script envars and logic formerly
required to support these things.
Signed-off-by: Chris Evich <cevich@redhat.com>
Now that we have source based skips there might be a case where we have
to run all tests. One option is to simply change a line in one of the
danger files but having something that can be set as title might be
easier for users.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
We do not have to test everything for each PR, we can know based on the
source if we changed (i.e. machine code) and only run the tests then.
This implements it as skip conditions, due to the nature of yaml files
we unfortunately cannot deduplicate everything, i.e. the is PR check and
danger files apply to everything but as skip is only a single yaml
string we cannot deduplicate parts of that string. If anyone knows a way
to achieve this I like to hear it.
For now I implemented this for int, system, bud and machine tests. Once
we are more comfortable with this I plan on adding it to other tests as
well.
This will replace the current _bail_if_test_can_be_skipped logic as it
covers more, marks tasks actually skipped in the github UI and works
even for the windows/macos machine tests.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
The leak check is slower (over 5mins) so we do not wnat them on PR runs
to speed system tests up. However that opens the door for someone to add
a test which forgets to do the correct cleanup themselves. This might
not cause a fatal error right away and only later when new tests would
be added. To prevent this happening the nighlty run will check leaks so
that we can fix them quickly and not notice them months/years later when
a new test is added that might trip over it.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
No idea why we need them, it passes without them so I just remove them.
Currently CI is broken as this install is failing on rawhide for some
reason. I don't know what changed there but this is working and unblocks
CI so I like to get this in.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
I'm hitting a bug with 9p when trying to transfer large files.
In RHEL at least 9p isn't supported because it's known to have a
lot of design flaws; virtiofsd is the supported and recommended
way to share files between a host and guest.
Add a new hidden `PODMAN_MACHINE_VIRTFS` environment
variable that can be set to `virtiofs` to switch to virtiofsd.
Signed-off-by: Colin Walters <walters@verbum.org>
The path mentioned above is linked in the sysadmin
article on running podman inside containers. The content
has since been moved and users are getting a 404 there now.
Add the path back with a readme pointing to the new location
of the content.
Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
podman.msi GUI has a radio-button to select WSL or Hyper-V
The checkbox in podman.msi GUI allow the user to specify if
the machine provider installation (WSL or Hyper-V) should
be part of podman installation or not.
podman-setup.exe supports 2 new variables: MachineProvider
(valid values are `wsl` and `hyperv`) and HyperVCheckbox
(valid values are `0` and `1`)
Installation creates the configuration file
`99-podman-machine-provider.conf` under folder
`%APPDATA\containers\containers.conf.d` with the selected
machine provider
Cirrus CI `win_installer_task` tests the installation with
both `hyperv` and `wsl` and verifies the configuration.
Uninstallation is tested too.
Note that podman-setup.exe GUI doesn't allow to choose the
provider yet. See https://github.com/containers/podman/issues/22492
Signed-off-by: Mario Loriedo <mario.loriedo@gmail.com>
Previously, the mac podman-machine tests installed rosetta before
executing any tests. As a best-practice (and because the Macs in CI are
shared) tests should never permanently modify the system. As of this
commit, the system setup script used for the CI Macs does the rosetta
installation. Remove the test setup code that installed rosetta and
add a CI-level confirmation that it's been pre-installed.
Signed-off-by: Chris Evich <cevich@redhat.com>
First, setup a custom TMPDIR to ensure we have no special assumptions
about hard coded paths. Second, make sure it is actually on a tmpfs so
we can catch regressions in the VM setup immediately.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Small usability improvements for our containerized validate target.
- Responds to SIGINT
- Exits if build fails, only validate if builds succeed
- Warns about potential of insufficient memory
- Document `make validatepr`
Signed-off-by: Ashley Cui <acui@redhat.com>
A long time ago, `passthrough_envars()` was defined in `lib.sh`. It has
since been moved, but the related comments were never updated. Update
the env. var. comments pointing future maintainers to the function that
relies on them. Otherwise a simple search w/in this repo. won't turn up
anything.
Signed-off-by: Chris Evich <cevich@redhat.com>
Followup to #13936 : add an exclusion to localmachine tests
so we can avoid running those on test- or doc-only PRs.
Reason: #22551, the machine-start-timeout flake, is causing
hours of wasted time.
Signed-off-by: Ed Santiago <santiago@redhat.com>
TMPDIR is typically /tmp which is typically(*) a tmpfs.
This PR ignores $TMPDIR when $CI is defined, forcing all
e2e tests to set up one central working directory in /var/tmp
instead.
Also, lots of cleanup.
(*) For many years, up to and still including the time of
this PR, /tmp on Fedora CI VMs is actually NOT tmpfs,
it is just / (root). This is nonstandard and undesirable.
Efforts are underway to remove this special case.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Both tests need the podman-registry script in $PATH, this never worked
locally as only the cirrus specific CI setup scripts configured this.
To make it work correctly locally add the hack dir to $PATH for these
Makefile targets.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
compose v1 has been deprecated for some time now, since July 2023 it no
longer receives any updates[1]. As such testing it on every PR is
pointless, it also does not provide any more coverage then compose v2.
At least I never saw only compose v1 test fails (except for flakes) so
it doesn't help us to catch regressions.
We tried to remove it before but decided against it at that time[2].
[1] https://docs.docker.com/compose/migrate/
[2] https://github.com/containers/podman/issues/18688
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
First of all this removes the need for a network connection, second
renovate can update the version as it is tracked in go.mod.
However the real important part is that the binary downloads are
broken[1]. For some reason the swagger created with them does not
include all the type information for the examples. However when building
from source the same thing works fine.
[1] https://github.com/go-swagger/go-swagger/issues/2842
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
This PR is only a first step towards being able to validate developer
code locally prior to pushing a PR and using CI. Right now, we have a
prepared image in a temporary spot (will change when done). That image
can be used to exercise various podman builds, make validate, and DCO
check.
The idea here is we have a make target that spins a podman container (or
machine) and then execute a small script to perform the actual builds.
Note, these builds are to verify code, not make production binaries so
corners are cut. As of now, we choose to not build cross-arch binaries
because most of our problems thus far have been operating system builds
and not arch.
Of course this can be expanded in the future. This is just step one to
start getting some of it in place. The rest of the work is tracked in
JIRA under two cards.
Signed-off-by: Brent Baude <bbaude@redhat.com>
make validate should work locally, this check makes no sense in a local
context as it checks for a github label.
To fix this remove this check from the validate target and only use it
as part of the CI validate run.
While at it remove old dnf install step, the issue has been closed for a
long time and it should already be part of our base images.
Fixes#22031
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
we are having second thoughts about *requiring* a policy.json on podman
machine hosts. we are concerned that we need to work out some more use
cases to be sure we do not make choices now that limit us in the near
term future. for example, should the policy files be the same for
container images and machine images? And should one live on the host
machine and the other live in the machine?
therefore, if a policy.json *is* present in the correct location, we will use and honor it; however, if it does not, we will allow the machine image to be pulled without a policy.
Signed-off-by: Brent Baude <baude@redhat.com>
Co-authored-by: Paul Holzinger <45212748+Luap99@users.noreply.github.com>
Signed-off-by: Brent Baude <bbaude@redhat.com>
For consistency with linux/osx makefile
I have added the win-gvproxy target as
an alias of win-sshproxy
[NO NEW TESTS NEEDED]
Signed-off-by: Mario Loriedo <mario.loriedo@gmail.com>
Paul Holzinger
Ed Santiago
openshift-merge-bot[bot]
Sergio Lopez
Mario Loriedo
Christophe Fergeau
Alexander Larsson
Chris Evich
Nicola Sella
Lokesh Mandvekar
Daniel J Walsh
Giuseppe Scrivano
Lokesh Mandvekar
Colin Walters
Urvashi Mohnani
Ashley Cui
Brent Baude
Brent Baude