Over several years the podman Makefile has become a
bloated complex mess. This impedes both debugging
and maintenance, besides causing general eye-strain.
Fix this by adding a simple navigation/layout guide, to help
developers quickly find what's needed. Re-organize the entire
file according to the new layout guide. Add section headers
that call out the purpose of the encompassed content, and
are easy to locate with search-tools.
Note: No recipes or definitions have been altered by this
commit, only re-arranged.
Signed-off-by: Chris Evich <cevich@redhat.com>
An in-line Python script, while flexible, is arguably
more complex and less stable than the long-lived `grep`,
`awk`, and `printf`. Make use of these simple tools
to display a column-aligned table of target and description
help output.
Also, the first target that appears in a Makefile is considered
the default (when no target is specified on the command-line).
However, despite it's name, the `default` target was not listed
first. Fix this, and redefine "default" target to "all" as
intended, instead of "help".
Lastly, add a small workaround for a vim syntax-hilighting bug.
Signed-off-by: Chris Evich <cevich@redhat.com>
Instead of shelling out frequently to resolve the current
directory, use the Makefile built-in `$(CURDIR)`. It has
the exact same meaning w/in the context of a `Makefile`.
Ref.:
https://www.gnu.org/software/make/manual/html_node/Quick-Reference.html
Signed-off-by: Chris Evich <cevich@redhat.com>
Previously this was needed for an automated release process. That
automation has long since been removed. Simplify the Makefile
by removing the target and references.
Signed-off-by: Chris Evich <cevich@redhat.com>
- Bugfix `make nixpkgs` which pin with branch `nixos-20.09`
- Code lint with `nixpkgs-fmt`
- Code sync between x86\_64 and aarch64
Signed-off-by: Wong Hoi Sing Edison <hswong3i@pantarei-design.com>
The split of install.docker and install.docker-docs makes some sense but
there should be some way to specify both for packagers.
This introduces `make install.docker-full` which installs both the
docker binary and the documentation.
Signed-off-by: Morten Linderud <morten@linderud.pw>
Commit 3908c00799 introduces a split for
installing the docker binary and the docker documentation. The
install line creating BINDIR and MANDIR was both moved to the
install.docker-docs path which makes `install.docker` fail.
Signed-off-by: Morten Linderud <morten@linderud.pw>
Add support for generating powershell completion files. This is especially
useful for people using the podman remote client on windows.
[NO TESTS NEEDED]
Signed-off-by: Paul Holzinger <paul.holzinger@web.de>
This will allow installation of the manpages without the need to rebuild
them in the installation stage of distro packaging.
[NO TESTS NEEDED]
Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
We missed bumping the go module, so let's do it now :)
* Automated go code with github.com/sirkon/go-imports-rename
* Manually via `vgrep podman/v2` the rest
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Backstory: every time you run 'make podman' or even
just 'make', you get a full recompile. This is sub-ideal.
Cause: I don't really know. It looks complicated. #5017
introduced a .PHONY for bin/podman, for reasons not
explained in the PR. Then, much later, #5880 well-
intentionedly but improperly tweaked the 'find'
command used in defining SOURCES, adding a -prune
but without the corresponding and required -print.
Let's just say, it was an unfortunate cascade of events.
This PR fixes the SOURCES definition and removes the
highly-undesired .PHONY from podman & podman-remote,
making it so you can type 'make' and, oh joy, not
build anything if it's current. The way 'make' is
supposed to work.
Why fix this now? Because my PR (#9209) was failing in CI,
in the Validate step:
Can't exec "./bin/podman": No such file or directory at hack/xref-helpmsgs-manpages line 223.
It failed even on Re-run, and only passed once I force-pushed
the PR (with no changes, just a new commit SHA). I have no idea
why bin/podman wasn't built, and I have zero interest in pursuing
that right now, but the proper solution is to add bin/podman as
a Makefile dependency for that particular test. So done.
While I'm at it, fix what is pretty clearly a typo in a .PHONY
And, finally, fix a go-md2man warning introduced in #9189
[NO TESTS NEEDED]
Signed-off-by: Ed Santiago <santiago@redhat.com>
Create a new common target, ginkgo-run, with the main ginkgo
test incantation; ginkgo and ginkgo-remote now invoke that,
with a clearer distinction between their arguments.
Reason for this: 'make remoteintegration' was not generating
ginkgo logs in CI. Reason: '-debug' option was missing from
the ginkgo incantation. Reason: impossibility of maintaining
duplicate long-complicated lines. This PR distills the common
aspects, making the differences clearer between local & remote.
IMPORTANT NOTE: '-nodes 3' was also missing from remote ginkgo.
This PR adds it. If this was intentionally omitted, please let
me know (with reasons) so I can refactor that too.
[NO TESTS NEEDED]
Signed-off-by: Ed Santiago <santiago@redhat.com>
Generate shell completion files for podman and podman-remote
and for all known shells (bash, fish, zsh). Fix Readme file
in completions dir: it suggested running 'make completion',
singular; no such target existed. Since the install target
is plural, I choose to make the new target plural also.
This is intended for use in CI some day, in a check such as:
(in contrib/cirrus/runner.sh):
make completions
SUGGESTION="run 'make completions' and commit all changes" ./hack/tree_status.sh
The goal would be to make sure that any new podman subcommands
or flags are accompanied by their corresponding shell helpers
on all commits.
IT IS NOT POSSIBLE TO ENFORCE THIS NOW. As I understand it,
Cobra is slow to incorporate community PRs, so the podman team
has chosen to sneak in completion files generated by an
unreleased and un-vendored version of Cobra. Running 'make
completions' right now would clobber those and result in
a diminished user experience.
I'm submitting this anyway as a stepping-stone toward that
future day when we can create such a CI hook.
[NO TESTS NEEDED]
Signed-off-by: Ed Santiago <santiago@redhat.com>
Make sure that bindings are in sync with the code. The check is similar
to what's already being done with `make vendor`, so integrate the two.
[NO TESTS NEEDED]
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Instead of implicitly generating the bindings, make it explicit, similar
to `make vendor`. This should prevent redundant and possibly error
prone generations. A following commit will shield CI.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
The Go gods did not shine upon us trying to understand what's going on
in #9000. The symptom is that `go generate` did not add required
imports to a generated file, ultimately breaking subsequent compilation.
While it still remains unclear *why* Go is behaving like that, the
symptom disappears when `go generate` runs in module mode; that is
without `-mod=vendor` and without `GO111MODULE=off`. This was
reproducible on two separate machines (Ubuntu and Fedora).
Also, when facing an unset GOPATH, set it to Go's default (i.e.,
$HOME/go) and make sure that GOBIN is in PATH since `goimports`
is required by `go generate`.
Fixes: #9000
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
This reverts commit de05e58168.
Running `go get -u` will change the local Go module causing CI to fail
as the local git tree is being changed. Reverting the change for now
until we have a better idea.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
I'm tired of seeing these every time I run 'make':
WARNING: go-md2man does not handle node type HTMLSpan
Cause: left-angle-brackets ( < ) in document source
Solution:
1) backquote-escape those that need to be shown, usually
ones referring to an argument or email address; or
2) Actual HTML ( <sup> and <a> ) which are meant to be
shown in generated HTML docs but can't be shown in
man pages, we filter out via a sed expression.
Signed-off-by: Ed Santiago <santiago@redhat.com>
On each PR (with a few exceptions), check the list of git-touched
files, and abort if no tests are added. Include instructions
on how to bypass the check if tests really aren't needed.
Include a hardcoded exception list for PRs that only touch a
well-known subset of "safe" files: docs, .cirrus.yml, vendor,
version, hack, contrib, or *.md. This list is likely to need
tuning over time.
Add a test suite, but not one recognized by the new script
(because it's a "*.t" file), so: [NO TESTS NEEDED]
Signed-off-by: Ed Santiago <santiago@redhat.com>
Install ginkgo on demand via `go get -u` rather than keeping a copy it's
entire source code in the vendor dirctory. The main motivation for that
is to make `golangci-lint` happy which is continuously throwing up on
the import of a program (i.e., ginkgo). The linter is broken and stupid
as it ignores flags to ignore dirs and ignores build tags (at least some
linters do) which is blocking us from updating to newer versions.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Run `go generate ./pkg/bindings/...` once for all bindings instead of
generating them separately. This should speed up bindings generation
as a given package is visited only once, and it fixes#8989 by dropping
the use of pushd and popd.
Fixes: #8989
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Set `-mod=vendor` when generating the bindings. We expect all
dependencies to be vendored already. This should slightly speed
up the bindings generation and prevent redundant network accesses.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
This involves a new test binary (a basic implementation of the
volume plugin protocol) and a new image on quay.io (Containerfile
to produce it and all sources located in this commit). The image
is used to run a containerized plugin we can test against.
Signed-off-by: Matthew Heon <mheon@redhat.com>
Add a `.generate-bindings` make target that only runs in the absence of
the `.generate-bindings` file or when a `types.go` file below
`pkg/bindings` has changed.
This will regenerate the go bindings and make sure the code is up2date.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Vendor in the latest cobra release v1.1.1
This will hurt the completion experience but is required for
proper packaging, see: #8528.
The best solution is to keep the current scripts since they
work fine with cobra v1.1.1.
Signed-off-by: Paul Holzinger <paul.holzinger@web.de>
Add a new make target (completion) to generate the shell
completion scripts. This will generate the scripts for bash,
zsh and fish for both podman and podman-remote with `podman completion`.
The scripts are put into the completions directory and can be
installed system wide with `sudo make install.completions`.
This commit replaces the current handwritten scripts for bash and zsh.
The `validate.completion` target has been adjusted to make sure nobody
edits these scripts directly.
Signed-off-by: Paul Holzinger <paul.holzinger@web.de>
Somewhere in the CIv2 migration we lost the man page vs --help
cross-checker. Add it back, by adding it into the man-page-check
Makefile target; this is part of 'make validate', which is run
in CI even on CI:DOCS PRs.
As happens when CI doesn't run, things broke. Man pages got out
of sync with --help. This PR:
1) Fixes hack/xref-helpmsgs-manpages to deal with the new
"Options" (instead of "Flags") form of podman help. #8034
did part of that, but one of my review comments was
accidentally left out.
2) Fixes hack/xref-helpmsgs-manpages to deal with the new
option syntax in man pages, post- #8292, in which each
option is preceded by four hashes so as to make them
HTML <h4> elements with named anchors.
3) Fixes man pages that #8292 accidentally missed.
4) Adds man page entries for two flags that got added
to podman but not documented (pod create --network-alias,
play kube --log-driver)
Fixes: #8296
Signed-off-by: Ed Santiago <santiago@redhat.com>
* Update reference to docker-py to docker to reflect change in library
name
* Update tests to create storage sandbox
* Enable all tests that endpoints support
* Refactor containers/{id}/rename to return 404 not 500
* Refactor tests to use quay.io vs. docker.io
Signed-off-by: Jhon Honce <jhonce@redhat.com>
We need to block systemd from cleaning up this directory
by dropping a /usr/lib/tmpfiles.d/podman.conf file in place.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
* renamed old API tests to not be discovered, they do not pass
* Updated the API tests to use a pristine storage configuration
* Skipped attach test, it needs to be re-written
Signed-off-by: Jhon Honce <jhonce@redhat.com>
Reimplement CI-automation to remove accumulated technical-debt and
optimize workflow. The task-dependency graph designed goal was to
shorten it's depth and increase width (i.e. more parallelism). A
reduction in redundant building (and 3rd party module download) was
also realized by caching `$GOPATH` and `$GOCACHE` early on. This
cache is then reused in favor of a fresh clone of the repository
(when possible).
Note: The system tests typically execute MUCH faster than the
integration tests. However, contrary to a fail-fast/fail-early
principal, they are executed last. This was implemented due to
debug-ability related concerns/preferences of the primary
(golang-centric) project developers.
Signed-off-by: Chris Evich <cevich@redhat.com>
From a fresh install of Fedora 33 Beta and a fresh clone of the repo, `make` fails with the following error when Go modules are disabled:
# Only generate the varlink code on Linux (see issue #4814).
GO111MODULE=off go generate ./pkg/varlink/...
../../vendor/github.com/varlink/go/cmd/varlink-go-interface-generator/main.go:12:2: cannot find package "github.com/varlink/go/varlink/idl" in any of:
/usr/lib/golang/src/github.com/varlink/go/varlink/idl (from $GOROOT)
/home/test/src/podman/_output/src/github.com/varlink/go/varlink/idl (from $GOPATH)
pkg/varlink/generate.go:3: running "go": exit status 1
make: *** [Makefile:646: pkg/varlink/iopodman.go] Error 1
Signed-off-by: Jordan Christiansen <xordspar0@gmail.com>
The configuration for this container has moved to the
`containers/automation_images` repository. Leverage the
image built from the new location to guarantee synchronization
with images used in CI.
Also, remove several other targets that haven't worked for a really
long time. Upon discussion, they seem to apply to a minority of
users. Remove them to clean up the `Makefile` and reduce
maintenance burden.
Signed-off-by: Chris Evich <cevich@redhat.com>
* Use the BUILDFLAGS variable for all Go builds
* Use `go install` instead of manually specifying the GOBIN path
Signed-off-by: Jordan Christiansen <xordspar0@gmail.com>
This commit adds additional targets for which we provide binary packages
on fedora and other distros.
Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
podman-remote is in better shape now. Let's see what needs
to be done to reenable remote system tests.
- logs test: skip multilog, it doesn't work remote
- diff test: use -l only when local, not with remote
- many other tests: skip_if_remote, with 'FIXME: pending #xxxx'
where xxxx is a filed issue.
Unrelated: added new helper to skip_if_remote and _if_rootless,
where we check if the source message includes "remote"/"rootless"
and insert it if missing. This is a minor usability enhancement
to make it easier to understand at-a-glance why a skip triggers.
Signed-off-by: Ed Santiago <santiago@redhat.com>
The unit tests currently require running as root. This has caused some
confusion that justifies adding a root check to `make localunit` and
error out for non-root users instead of starting the tests deemed to
fail.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Some time ago, we moved the Seccomp policy (and related setup
code) to a place where all our tools could share it [1]. We did
not, however, remove the in-repo seccomp.json file. Over the last
year or so, the in-repo seccomp policy has become progressively
more and more outdated, with no effort made to maintain it
(because what sense is there in keeping a duplicate?). Today, a
friend came to me and asked if a Podman container could access
keyctl, assuming it could not because he was reading the outdated
Seccomp policy which does not allow it. Since it's becoming clear
that this file is doing no good and actively causing confusion,
let's just drop it.
[1] https://github.com/seccomp/containers-golang
Signed-off-by: Matthew Heon <mheon@redhat.com>
We want to add this configuration file so that users can descover
how to configure the permanent connection to a remote podman instance.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Reversion of one part of #6679: my handling of 'realpath'
would not work when $PODMAN is 'podman-remote --url etc'.
Trying to handle that case got unmaintainable; so instead
let's just force 'make {local,remote}system' to invoke
with a full PODMAN path. This breaks down if someone
runs the tests with a manual 'bats' invocation, but I
think I'm the only one who ever does that.
Since podman path will now be very long in the logs,
add code to logformatter to abbreviate it like we do
for the ginkgo logs.
And, one thing that has bugged me for a long time:
in the error logs, show a different prompt ('#' vs '$')
to distinguish root vs rootless. This should make it
much easier to see at-a-glance whether a log file
is root or not. Add tests for it.
Signed-off-by: Ed Santiago <santiago@redhat.com>
podman-remote has not been tested. A principal part of the
problem was #5387 - the YAML I wrote did not have the
intended effect, it did not set TEST_REMOTE_CLIENT=true
and because of my multiple iterations I did not catch this
during testing.
Part 1 of this PR is to fix .cirrus.yml to enable remote tests.
Part 2 -- what I had first noticed and tried to fix -- is that
rootless_test.sh was never running remote because, of course,
envariables are not sent via ssh. I reworked integration_test.sh
and rootless_test.sh to use a command-line decision instead.
Part 3, sigh, is to disable one failing integration test
and *all* system tests, because so many of the latter are
failing. Addressing those failures needs to be done in
subsequent PRs. Issues #6538, #6539, #6540 are filed for
some of the problems I isolated. There will be more.
Also, minor, fixed some stale references to varlink.
Signed-off-by: Ed Santiago <santiago@redhat.com>
We experienced regression when using the latest `v1.2.0-dev` bats in
Ubuntu 20.04 (see github.com/containers/libpod/pull/6418). Using
bats v1.1.0 worked in the Ubuntu test VM.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
We need to default to building podman. If you specify no build
tags you will not build podman, not podman-remote.
Just using remote flag to indicate podman-remote and !remote for
podman.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Issue gh#6286 was already fixed in a prior commit but the Makefile still
ran some varlink steps by default.
This commit makes any varlink build steps dependent on the varlink
build tag and also makes the contrib rpm spec file independent of
varlink.
Endpoint tests will be run only if BUILDTAGS contains varlink.
Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
1) fix lost credentials.
must_pass(), added in #6375, eats the credentials
generated via 'podman run --entrypoint htpasswd'.
Run that podman instance directly, and add explicit
error check.
(The error and stdout/stderr handling here has gotten
cumbersome. There must be something I'm missing that
could make it all simpler.)
2) fix default podman path.
When setting $PODMAN, default to the locally built
one -- there may not be one in $PATH (e.g. in
Ubuntu, see #6366). This in turn requires us to:
3) run registry test in integration, not unit test
It looks like unit tests run before podman is built,
causing a chicken-egg dilemma. Try to solve that by
running the new hack/podman-registry-go test in
integration tests, not unit tests.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Handle REMOTETAGS the same way for all remote commands.
This fixes issues where remote commands are not building correctly on rhel7 and centos7 systems.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Now that we're shipping containers.conf, we don't want to provide
a libpod.conf anymore. This removes libpod.conf from the repo and
as many direct uses as I can find.
There are a few more mentions in the documentation, but someone
more familiar with containers.conf should make those edits.
Signed-off-by: Matthew Heon <mheon@redhat.com>
We’re now able to build a static podman binary based on a custom nix
derivation. This is integrated in cirrus as well, whereas a later target
would be to provide a self-contained static binary bundle which can be
installed on any Linux x64-bit system.
Fixes: https://github.com/containers/libpod/issues/1399
Signed-off-by: Sascha Grunert <sgrunert@suse.com>
Instead of being depended by docs, targets '.install.md2man' and
'docdir' should be depended by 'MANPAGES', or else the path
'docs/build/man' or 'GOMD2MAN' might not exist when it tries to
generate files in it.
This fixes a following build error:
| open docs/build/man/podman-volume-ls.1: no such file or directory
| Makefile:377: recipe for target 'docs/source/markdown/podman-volume-ls.1' failed
| make: *** [docs/source/markdown/podman-volume-ls.1] Error 1
| make: *** Waiting for unfinished jobs....
| open docs/build/man/podman-init.1: no such file or directory
| Makefile:377: recipe for target 'docs/source/markdown/podman-init.1' failed
Signed-off-by: Ming Liu <ming.liu@toradex.com>
We should not be building podman-remote with the BUILDTAGS, these
only effect server side.
CRC Group wants to use a static version of podman-remote in order
to install the same podman-remote client on any Linux box.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Distro packaging often uses non-default build and linker flags.
The current Makefile cannot be used in the package build process as it
will end up rebuilding the binaries with the default flags.
This commit introduces install targets which do not depend on the build
targets. Distro packages should prefer these if they want to use any
non-default flags.
NOTE: This commit effectively leaves prior targets unchanged, so users
won't notice any difference.
Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
There is no need to search for sources in hidden dirs. In my case
there are files from development environment that stand in the way.
Signed-off-by: Anatoli Babenia <anatoli@rainforce.org>
New script cross-references r.Handle() and r.HandleFunc()
calls against the preceding '// swagger:operation' comments,
and exits failure (with descriptive error messages) if any
comments do not match the code.
This script should not be necessary: the swagger comments
should be autogenerated from the source code.
Signed-off-by: Ed Santiago <santiago@redhat.com>
(export a=b command args) does not run (command args) with a=b,
it sets $a to b, and marks variables $a $command $args as exported,
i.e. (command args) is not run.
So we were not actually running (go mod tidy).
Fix that.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
* Added support for system service
* Enabled linting on the varlinkapi source, needed to support V2
service command
* Added support for PODMAN_SOCKET
Skip linting deprecated code
Rather than introduce bugs by correcting deprecated code, linting the
code is being skipped. Code that is being ported into V2 is being
checked.
Signed-off-by: Jhon Honce <jhonce@redhat.com>
New hack/xref-helpmsgs-manpages script, added to CI 'gate'
task, runs 'podman [subcommand] --help' and cross-references
against man pages in docs/source/markdown/podman*.1.md
See #5453 and #5460 for instances of the problems the
script has found.
The careful reader will find an alarming number of special-case
bypasses. These are a tradeoff I am making: to get perfect
coverage with no handwaving, it would be necessary to make
drastic changes to some man pages, and I believe those would
be counterproductive.
Signed-off-by: Ed Santiago <santiago@redhat.com>
API v2 has been quiet for a few days, and the test script is
actually passing. Let's take advantage of this opportunity
to get them running in CI.
Requires adding a check for cgroupsv2
Signed-off-by: Ed Santiago <santiago@redhat.com>
pre-commit is required for running certain tests by the Makefile.
Adding pre-commit to install.tools allows users to easily use
and test podman.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Closes#5130.
varlink code generation was skipped when `uname -o` did not print "GNU/Linux".
However on some Linux systems (e.g. alpine) only "Linux" is printed
which results in cmd/podman/varlink/iopodman.go not being generated.
Thus the Makefile target condition has been changed to match "Linux".
Signed-off-by: Max Goltzsche <max.goltzsche@gmail.com>
the linting task identifies gofmt issues; therefore it makes more sense to run our make gofmt first, which actually fixes the gofmt issues.
Signed-off-by: Brent Baude <bbaude@redhat.com>
Chris Evich
Wong Hoi Sing Edison
Morten Linderud
Paul Holzinger
Lokesh Mandvekar
Ming Liu
Valentin Rothberg
Matej Vasek
Ed Santiago
OpenShift Merge Robot
Matthew Heon
Josh Soref
Daniel J Walsh
Jhon Honce
Jordan Christiansen
Peter Oliver
Wong Hoi Sing Edison
Brent Baude
Sascha Grunert
Ming Liu
Anatoli Babenia
Miloslav Trmač
Max Goltzsche