Situation encountered just now after buildah #3949 but
before podman #14084: go.mod changed in such a way that
other modules were updated, not just buildah, and those
changes weren't git-added by 'make vendor'. This resulted
in the dirty-tree CI test failing.
Solution: check for untracked vendor files after 'make vendor',
and git-add them. Show a friendly message that we're doing so:
+---> Adding untracked files under containers/image, containers/storage, klauspost/compress, x/sys
In order to do this safely, we run an untracked-files check
under vendor as one of the first sanity checks. If there are
any when we start the script, fail early.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Major revamp: instead of stacking a vendor commit on top of
the treadmill changes, do it the other way around: vendor,
then apply treadmill diffs.
Reason: the build-all-new-commits test. Sigh. It fails in the
common case where our treadmill changes include a new struct
element in cmd/podman/images/build.go
Why this is good: well, superficially, it's more intuitive.
Why this is horrible: omg the rebasing games are a nightmare.
When the vendor commit is on top (HEAD), it's ultra-trivial
to drop it, rebase the treadmill changes on main, then add
a new vendor-buildah commit on top. As you can see from the
diffs in this PR, treadmill-as-HEAD introduces all sorts
of complex dance steps in which things can go catastrophically
wrong and you can lose all your treadmill patches. I try very
hard to prevent this, and to offer hints if there's a problem,
and heck in the worst case it's still git so it's still possible
to find lost commits... but it's still much riskier than the
old way.
Alternative I considered: using sed magic to disable the
build-all-new-commits test. So tempting... but that would
also disable the bloat check.
Signed-off-by: Ed Santiago <santiago@redhat.com>
More safety checks for the treadmill script:
* for --sync:
- issue warning if HEAD is not a vendor commit
- if run-buildah-bud-tests fails, leave the working dir
for user to investigate. And offer a long helpful warning.
- tweak .cirrus.yml so buildah-bud tests run early, so
we can fail early. (Remember, the top commit will never
ever ever ever be merged)
* for --pick:
- check branch merge-base (of your vendor-update branch),
compare against that of the treadmill PR. If treadmill
is newer, bail, and suggest rebasing. This would've
saved us some time in #14005.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Add more benchmarks for the most common and performance-critical image
commands. Benchmarks for `podman build` should go into a separate
section.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
This PR introduces a test suite for podman machine. It can currently be
run on developers' local machines and is not part of the official CI
testing; however, the expectation is that any work on machine should
come with an accompanying test.
At present, the test must be run on Linux. It is untested on Darwin.
There is no Makefile target for the test. It can be run like `ginkgo -v
pkg/machine/test/.`. It should be run as a unprivileged user.
Signed-off-by: Brent Baude <bbaude@redhat.com>
When going through the rebase+build loop, the repository state won't
match the exact branch or PR history. This results in the `Building:
XYZSHA` indications being entirely useless. Fix this by at least
including the title line of the commit being built. This will allow a
human to make sense of any size-check failure WRT their view of history.
Signed-off-by: Chris Evich <cevich@redhat.com>
This is the script I've been using (and tweaking) for the past
two weeks. It's ready for general review and use, with the
proviso that there are still corner cases I haven't tested.
See https://github.com/containers/podman/wiki/Buildah-Vendor-Treadmill
for an overview and instructions.
Signed-off-by: Ed Santiago <santiago@redhat.com>
This won't actually be seen except by someone who takes the
time to clickety-click into Cirrus - but that's better than
not showing it at all.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Command flags (OPTIONS) in man pages have to date been in
haphazard order. Sometimes that order is sensible, e.g.,
most-important options first, but more often they're
just in arbitrary places. This makes life hard for users.
Here, I update the man-page-check Makefile script so it
checks and enforces alphabetical order in OPTIONS sections.
Then -- the hard part -- update all existing man pages to
conform to this requirement.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Add a CI check to prevent unwanted bloat in binary images,
by building a baseline (pre-PR) binary then comparing file
sizes post-PR.
Part 1 (#13518) added a new script that runs multiple 'make's,
comparing image sizes against an original, and failing loudly
if growth is too big. An override mechanism is defined.
This is part 2 of 2: adding the CI rule. We couldn't do that
in part 1, because the rule would call a script that didn't
exist in the pre-PR commit.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Add a CI check to prevent unwanted bloat in binary images,
by building a baseline (pre-PR) binary then comparing file
sizes post-PR.
We piggyback onto the existing 'Build Each Commit' CI check
because it gives us an easy way to run 'make' against the
parent commit.
This is part 1 of 2: adding the script, not the Makefile rule.
We can't add the Makefile rule now because the script it would
invoke does not exist in the parent commit.
Signed-off-by: Ed Santiago <santiago@redhat.com>
avoid this warn:
```
golangci/golangci-lint info installed ./bin/golangci-lint
golangci/golangci-lint err this script is deprecated, please do not use it anymore. check https://github.com/goreleaser/godownloader/issues/207
```
Signed-off-by: Pascal Bourdier <pascal.bourdier@gmail.com>
Due to some recent changes in the Makefile, the setup part of the script
is now breaking with the error:
```
install: cannot stat 'bin/rootlessport': No such file or directory
make: *** [Makefile:767: install.bin] Error 1
```
The root-cause seems to be the `install` targets not
properly specifying their build dependencies. This may lead to other
problems WRT automation, but for now I'm just patching this tool to
workaround the issue.
Signed-off-by: Chris Evich <cevich@redhat.com>
Since this option will also be used for netavark we should rename it to
something more generic. It is important that --cni-config-dir still
works otherwise we could break existing container cleanup commands.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Start inching our way back to having tests for the sudo form
of podman image scp. Basically, copy an image to another user
and then back, using a pseudorandom name. Confirm that the
image makes it to the remote end, and that when we copy it
back, the original image digest is preserved.
When scp'ing as root, we identify the destination rootless
user account via the $PODMAN_ROOTLESS_USER envariable. Setting
this and creating the account is left as an exercise for the
CI framework (be it github, or Fedora/CentOS/RHEL gating, or
other).
Also: amend hack/bats to set and relay $PODMAN_ROOTLESS_USER,
so developers can test locally.
Also: remove what I'm 99% sure is a debugging printf.
Signed-off-by: Ed Santiago <santiago@redhat.com>
libsubid changes its ABI in version 4. Account for the different name
in the configure script.
Closes: https://github.com/containers/podman/issues/12654
[NO NEW TESTS NEEDED] it is a change in the build script
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
Podman logs was defined twice, once for container logs and once for pod
logs. This causes problems with the shell completion. Also podman --help
showed this command twice.
[NO NEW TESTS NEEDED]
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Some time in the last month, podman started to depend on a bunch
of external helper binaries: rootlessport, pause, catatonit.
System tests fail without these.
Update the hack/bats script to pass $CONTAINERS_HELPER_BINARIES_DIR
(set to ./bin); podman will then use locally-built helpers. (This
requires https://github.com/containers/common/pull/823 , which as
of this PR is not yet vendored into podman. There is no harm in
merging this while we wait.)
Also: if bats helper is invoked as root, run only once; i.e.,
skip the "rootless" step.
Also (piggybacked): the name of the podman pause image has
changed, from pause to podman-pause. Adjust that in our
teardown so we don't leave droppings.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Add new CI check to confirm that links and references
in SEE ALSO sections are properly formatted and that
links are valid (at least in theory: we do no actual
URL fetching to test for 404).
The check is piggybacked into existing xref-helpmsgs-manpages
script. It could conceivably be more elegant to write a
separate tool for this purpose, but I don't wish to duplicate
the logic for finding and reading markdown files.
Script identified various problems, which I fix in this PR:
. missing '**' (asterisks) around some references, or '**'
in the wrong place.
. links pointing to github.com/.../tree/ instead of /blob/
(github redirects those automatically, but I like
consistency)
. a few copy-paste errors, e.g. subgid linking to subuid.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Fix day-one sloppiness: when I first wrote this framework
it compared strings using 'expr', not '=', to be more
forgiving of extra cruft in output. This was a bad decision.
It means that warnings or additional text are ignored:
is "all is ok, NOT!" "all is ok" <-- this would pass
Solution: tighten up the 'is' check. Use '=' (direct
compare) first. If it fails, look for wild cards ('*')
or character classes ('[') in the expect string. If
so, and only then, use 'expr'. And, thanks to a clever
suggestion from Luap99, include '(using expr)' in the
error message when we do so; this could make it easier
for a developer to understand a string mismatch.
This change exposes a lot of instances in which we weren't
doing proper comparisons. Fix those. Thankfully, there
weren't as many as I'd feared.
Also, and completely unrelated, add '-T' flag to bats
helper, for showing timing results. (I will open this
as a separate PR if requested. I too find it offensive
to jumble together unrelated commits.)
Signed-off-by: Ed Santiago <santiago@redhat.com>
When building releases, the definitive canonical version of podman (or
podman-remote) is needed. Previously this was accomplished by scraping
`version/version.go`. However, due to tooling differences across
platforms, this has proven problematic, unreliable, and hard to
maintain.
Fix this by building and caching a small golang binary who's only purpose
is to print the version number to stdout. This not only provides a quick
and reliable way to determine the current version, it also acts as a check
on the version API vs tooling that relies on it.
Lastly, remove several `RELEASE_*` Makefile definitions which aren't
actually used anywhere. These were originally added a very long time
ago to serve as part of a long since retired release process. The
remaining items, were updated to make use of the new `.podmanversion`
binary on an as-required basis (i.e. not every time `make` is run).
Signed-off-by: Chris Evich <cevich@redhat.com>
The changelog.txt file hasn't been kept in sync with release tags,
especially on main, so remove it.
The release notes will be featured in RELEASE_NOTES.md.
Signed-off-by: jesperpedersen <jesper.pedersen@redhat.com>
[NO TESTS NEEDED]
This will enable remote access to /etc/subuid and /etc/subgid
information from ldap services, if shadow-utils ships with a libsubid.
[NO TESTS NEEDED] Since we have no way to test this.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Add special case for op PlayKubeDownLibpod Heuristic for guessing swagger operation id too limited for PlayKubeDownLibpod
Signed-off-by: Jhon Honce <jhonce@redhat.com>
One of the worst parts of a Podman release is writing the release
notes. It requires manually going through all merged commits
since the last release, figuring out what was actually done, and
writing a small blurb about what was fixed. The worst part of
this is the difficulty in finding the commits that were actually
included in previous releases - our extensive backports to prior
releases mean that there are usually dozens of commits that were
included in a prior release, but do not have a matching SHA (as
the original author did not do the backport, and often the commit
required massaging to cherry-pick in).
This script automates the job of finding commits in one release
branch that are not in another, with filtering to remove most
cherry-picked commits. It makes my life a lot easier during
releases, so I figured I'd include it in hack/ so anyone else
stuck with the enjoyable task of writing release notes can have a
slightly easier life.
The script is written in absolutely terrible Ruby and its
performance is absolutely terrible, but you only need to run it
once per major release and a 30-second wait to generate the list
of commits to include isn't bad.
Signed-off-by: Matthew Heon <mheon@redhat.com>
Well, new to you. It's been something I've used for years.
Simple, but it takes care of a lot of housekeeping, and
makes it ever-so-much-more pleasant to invoke bats tests.
And when it's easier to run tests, tests get run.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Commit 800a2e2d35 introduced a way to disable the conversion of `--`into
an en dash on docs.podman.io, so the ugly workaround of escaping the
dashes is no longer necessary.
Signed-off-by: Paul Holzinger <paul.holzinger@web.de>
An in-line Python script, while flexible, is arguably
more complex and less stable than the long-lived `grep`,
`awk`, and `printf`. Make use of these simple tools
to display a column-aligned table of target and description
help output.
Also, the first target that appears in a Makefile is considered
the default (when no target is specified on the command-line).
However, despite it's name, the `default` target was not listed
first. Fix this, and redefine "default" target to "all" as
intended, instead of "help".
Lastly, add a small workaround for a vim syntax-hilighting bug.
Signed-off-by: Chris Evich <cevich@redhat.com>
Until now we've only compared operations when called with the
non-default --pedantic flag, because there were way too many
exceptions.
With the merge of #9944 the rules have become much cleaner.
Still not perfect, but it's now possible to have simple
general rules with a (semi-)manageable list of exceptions.
Signed-off-by: Ed Santiago <santiago@redhat.com>
PR #9856 works around a buggy markdown processor that cleverly
converts double dashes to em-dash. The unfortunate result is
that the man page source files are unmaintainable, because
every '--foo' has to be specified as '\-\-foo'. This is
impossible for humans to remember, so let's add a helpful
diagnostic message when we detect new options added without
the escapes.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Escape the two dashes, otherwise they are combined into one long dash.
I tested that this change is safe and still renders correctly on github
and with the man pages.
This commit also contains a small change to make it build locally.
Assuming you have the dependencies installed you can do:
```
cd docs
make html
```
Preview the html files in docs/build/html with
`python -m http.server 8000 --directory build/html`.
Fixescontainers/podman.io#373
Signed-off-by: Paul Holzinger <paul.holzinger@web.de>
We missed bumping the go module, so let's do it now :)
* Automated go code with github.com/sirkon/go-imports-rename
* Manually via `vgrep podman/v2` the rest
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Followup to dbb9943
Despite skipping the `Smoke` check, it was observed on a *new* branch,
the `validate` task (specifically `git-validation`) will fail. This
is because:
* `$CIRRUS_LAST_GREEN_CHANGE` will be empty on a new branch.
* `$CIRRUS_BASE_SHA` is always empty for runs triggered by branch-push
* `$EPOCH_TEST_COMMIT` will be set to `YOU_FOUND_A_BUG`.
Fix this by eliminating the `Smoke` task entirely, simplifying all
the `make validate` operations into the `validate` cirrus task. Ensure
this task does not run when a new branch or tag is pushed.
Also, eliminate the `$CIRRUS_BUILD_ID` value as it's confusing and not
actually used anywhere. It was formerly used for building VM images,
but this has moved to another repo entirely.
Signed-off-by: Chris Evich <cevich@redhat.com>
Install golangci-lint to `./bin` instead of `$GOBIN`. The latter may be
shared with other projects who require a different version. Having a
shared version of golangci-lint is a reoccurring source of red herrings
on my work station, so I think it's time to split them.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
* verify socat and podman binaries exist
* setup a sandboxed podman service
* run podman service with socat proxy to capture API stream
* clean up sandbox leaving the log files for review
Signed-off-by: Jhon Honce <jhonce@redhat.com>
Make the order of short and long flag names in the documentation
consistent. Also adjust the man page validaten script to only allow
the `**--long**, **-s**` syntax.
Signed-off-by: Paul Holzinger <paul.holzinger@web.de>
Allow automatic generation for shell completion scripts
with the internal cobra functions (requires v1.0.0+).
This should replace the handwritten completion scripts
and even adds support for fish. With this approach it is
less likley that completions and code are out of sync.
We can now create the scripts with
- podman completion bash
- podman completion zsh
- podman completion fish
To test the completion run:
source <(podman completion bash)
The same works for podman-remote and podman --remote and
it will complete your remote containers/images with
the correct endpoints values from --url/--connection.
The completion logic is written in go and provided by the
cobra library. The completion functions lives in
`cmd/podman/completion/completion.go`.
The unit test at cmd/podman/shell_completion_test.go checks
if each command and flag has an autocompletion function set.
This prevents that commands and flags have no shell completion set.
This commit does not replace the current autocompletion scripts.
Closes#6440
Signed-off-by: Paul Holzinger <paul.holzinger@web.de>
Somewhere in the CIv2 migration we lost the man page vs --help
cross-checker. Add it back, by adding it into the man-page-check
Makefile target; this is part of 'make validate', which is run
in CI even on CI:DOCS PRs.
As happens when CI doesn't run, things broke. Man pages got out
of sync with --help. This PR:
1) Fixes hack/xref-helpmsgs-manpages to deal with the new
"Options" (instead of "Flags") form of podman help. #8034
did part of that, but one of my review comments was
accidentally left out.
2) Fixes hack/xref-helpmsgs-manpages to deal with the new
option syntax in man pages, post- #8292, in which each
option is preceded by four hashes so as to make them
HTML <h4> elements with named anchors.
3) Fixes man pages that #8292 accidentally missed.
4) Adds man page entries for two flags that got added
to podman but not documented (pod create --network-alias,
play kube --log-driver)
Fixes: #8296
Signed-off-by: Ed Santiago <santiago@redhat.com>
Allow build systems without standard cc to successfully run the
dependency checking helper scripts from the Makefile.
This supports custom compilers specified by the common CC environment
variable, preprocessors given as CPP and additional preprocessor flags
from CPPFLAGS.
Additional flags from CFLAGS and LDFLAGS are considered for compiling/linking.
Overall, this facilitates cross-compilation and similar setups.
Signed-off-by: Marcel Bargull <marcel.bargull@udo.edu>
Want to have man pages match commands, since we have lots of printed
man pages with using Options, we will change the command line to use
Options in --help.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Also removed automatic exection of setup_environment.sh since most
people using this script are podman developers (not automation/CI
folks). If executing the automation scripts is necessary, manual
attendance to required variables like `$TEST_FLAVOR` is mandatory.
Signed-off-by: Chris Evich <cevich@redhat.com>
Installing bats to /usr/local requires root privileges. Without this,
`make install.tools` fails. However, if I do `sudo make install.tools`,
then all of the other dependencies and git clones in the current
directory end up owned by root. This limits root privileges to the part
that needs it.
Signed-off-by: Jordan Christiansen <xordspar0@gmail.com>
Separate the volume endpoints into compat and libpod,
as it is done for the other endpoints.
Move the libpod image push endpoint to images.
Signed-off-by: Paul Holzinger <paul.holzinger@web.de>
It's not possible to run any of the scripts on distributions which do
have `bash` not in `/bin`. This is being fixed by using `/usr/bin/env
bash` instead.
Signed-off-by: Sascha Grunert <sgrunert@suse.com>
There are a bunch of *.rst files in docs/source, linking sometimes
to man pages and sometimes to other .rst files. These files each
have entries of the following form:
:doc:`foo <link-to-foo>` Description of foo
...for all podman sub and sub-subcommands 'foo'.
Read all .rst files and make sure that:
- all entries in a given file are in alphabetical order
- all link-to-foo targets point to existing doc files
- every subcommand known by 'podman help' has a corresponding
doc entry in a .rst file
Signed-off-by: Ed Santiago <santiago@redhat.com>
For each podman*.md file with a subcommand table (podman,
podman-container, etc), assert that the subcommand list
is sorted.
Change is bigger than it should be, because it switches from
nice clean local per-function error counting to using a nasty
global.
Signed-off-by: Ed Santiago <santiago@redhat.com>
The unit tests currently require running as root. This has caused some
confusion that justifies adding a root check to `make localunit` and
error out for non-root users instead of starting the tests deemed to
fail.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
With the advent of Podman 2.0.0 we crossed the magical barrier of go
modules. While we were able to continue importing all packages inside
of the project, the project could not be vendored anymore from the
outside.
Move the go module to new major version and change all imports to
`github.com/containers/libpod/v2`. The renaming of the imports
was done via `gomove` [1].
[1] https://github.com/KSubedi/gomove
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
New functionality in hack/man-page-checker: start cross-
referencing the man page 'Synopsis' line against the
output of 'podman foo --help'. This is part 1, flag/option
consistency. Part 2 (arg consistency) is too big and will
have to wait for later.
flag/option consistency means: if 'podman foo --help'
includes the string '[flags]' in the Usage message,
make sure the man page includes '[*options*]' in its
Synopsis line, and vice-versa. This found several
inconsistencies, which I've fixed.
While doing this I realized that Cobra automatically
includes a 'Flags:' subsection in its --help output
for all subcommands that have defined flags. This
is great - it lets us cross-check against the
usage synopsis, and make sure that '[flags]' is
present or absent as needed, without fear of
human screwups. If a flag-less subcommand ever
gets extended with flags, but the developer forgets
to add '[flags]' and remove DisableFlagsInUseLine,
we now have a test that will catch that. (This,
too, caught two instances which I fixed).
I don't actually know if the new man-page-checker
functionality will work in CI: I vaguely recall that
it might run before 'make podman' does; and also
vaguely recall that some steps were taken to remedy
that.
Signed-off-by: Ed Santiago <santiago@redhat.com>
For using the `registry:2.6` image. 2.7 and beyond dropped the
`htpasswd` binary from the rootfs which parts of our CI depends
on.
While this is not a sustainable solution (assuming `htpasswd` is gone
for ever), it unblocks the CI for now.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
We experienced regression when using the latest `v1.2.0-dev` bats in
Ubuntu 20.04 (see github.com/containers/libpod/pull/6418). Using
bats v1.1.0 worked in the Ubuntu test VM.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
We need to default to building podman. If you specify no build
tags you will not build podman, not podman-remote.
Just using remote flag to indicate podman-remote and !remote for
podman.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
1) fix lost credentials.
must_pass(), added in #6375, eats the credentials
generated via 'podman run --entrypoint htpasswd'.
Run that podman instance directly, and add explicit
error check.
(The error and stdout/stderr handling here has gotten
cumbersome. There must be something I'm missing that
could make it all simpler.)
2) fix default podman path.
When setting $PODMAN, default to the locally built
one -- there may not be one in $PATH (e.g. in
Ubuntu, see #6366). This in turn requires us to:
3) run registry test in integration, not unit test
It looks like unit tests run before podman is built,
causing a chicken-egg dilemma. Try to solve that by
running the new hack/podman-registry-go test in
integration tests, not unit tests.
Signed-off-by: Ed Santiago <santiago@redhat.com>
My initial revision of the podman-registry helper script
was written in haste, with an enormous tradeoff: no
visibility into any errors. We are now paying for this
in #6366: the script is failing on Ubuntu and we
have no way of knowing why.
This PR adds a must_pass() function used for critical
steps. This runs the action silently; if the command
fails, it displays the failing command name with
full output logs, cleans up the temporary workdir,
and exits with error status.
As a reminder, the reason this is necessary is that
our script convention is to output a series of
environment variables to stdout -- we must therefore
take pains not to emit anything else to stdout.
And, unfortunately, podman and openssl tend to be
rather verbose.
Signed-off-by: Ed Santiago <santiago@redhat.com>
In response to #6207: this is a helper script intended for
use in starting and stopping a local container registry.
It takes care of port, username, password assignments;
generates a self-signed certificate; and starts the
container in an isolated podman root/runroot to avoid
conflicting with the caller's environment.
Intended usage: invoke from shell script, using 'eval'
to get results into calling process environment. See
help message (-h) for invocation details. This will
work for shell scripts but will be difficult if
called from Go or C - if that is likely to happen,
I'd love to hear suggestions for alternate ways to
get the settings back to the caller.
Signed-off-by: Ed Santiago <santiago@redhat.com>
This properly prints out image-name hints when executing the hack script
without any arguments. It is required due to changes made by Ed for
test-name beatification. An identical change was made and reviewed by
Ed in the containers/storage repo.
Signed-off-by: Chris Evich <cevich@redhat.com>
While this commit was initially meant to fix#5847, it has turned into a
bigger refactoring which I did not manage to break into smaller pieces:
* Fix#5847 by refactoring the image-removal logic.
* Make the api handler for image-removal use the ABI code. This way,
both (i.e., ABI and Tunnel) end up using the same code. Achieving
this code share required to move some code around to prevent circular
dependencies.
* Everything in pkg/api (excluding pkg/api/types) must now only be
accessed from code using `ABISupport`.
* Avoid imports from entities on handlers to prevent circular
dependencies.
* Move `podman system service` logic into `cmd` to prevent circular
dependencies - it depends on pkg/api.
* Also remove the build header from infra/abi files. It will otherwise
confuse swagger and other tools; errors we cannot fix as go doesn't
expose a build-tag env variable.
Fixes: #5847
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
New script cross-references r.Handle() and r.HandleFunc()
calls against the preceding '// swagger:operation' comments,
and exits failure (with descriptive error messages) if any
comments do not match the code.
This script should not be necessary: the swagger comments
should be autogenerated from the source code.
Signed-off-by: Ed Santiago <santiago@redhat.com>
* Added support for system service
* Enabled linting on the varlinkapi source, needed to support V2
service command
* Added support for PODMAN_SOCKET
Skip linting deprecated code
Rather than introduce bugs by correcting deprecated code, linting the
code is being skipped. Code that is being ported into V2 is being
checked.
Signed-off-by: Jhon Honce <jhonce@redhat.com>
./hack/podmanv2-retry will first invoke $PODMAN_V2 with given
arguments. If that fails with any of the following errors:
unrecognized command
unknown flag
unknown shorthand
...it will run $PODMAN_FALLBACK with the same arguments.
Output and exit code will be those of the final podman command,
although be aware that stderr and stdout are combined.
This is a quick-hack script intended for use in v2 testing, to
test implemented commands without noise from unimplemented ones.
Signed-off-by: Ed Santiago <santiago@redhat.com>
New hack/xref-helpmsgs-manpages script, added to CI 'gate'
task, runs 'podman [subcommand] --help' and cross-references
against man pages in docs/source/markdown/podman*.1.md
See #5453 and #5460 for instances of the problems the
script has found.
The careful reader will find an alarming number of special-case
bypasses. These are a tradeoff I am making: to get perfect
coverage with no handwaving, it would be necessary to make
drastic changes to some man pages, and I believe those would
be counterproductive.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Add support to auto-update containers running in systemd units as
generated with `podman generate systemd --new`.
`podman auto-update` looks up containers with a specified
"io.containers.autoupdate" label (i.e., the auto-update policy).
If the label is present and set to "image", Podman reaches out to the
corresponding registry to check if the image has been updated. We
consider an image to be updated if the digest in the local storage is
different than the one of the remote image. If an image must be
updated, Podman pulls it down and restarts the container. Note that the
restarting sequence relies on systemd.
At container-creation time, Podman looks up the "PODMAN_SYSTEMD_UNIT"
environment variables and stores it verbatim in the container's label.
This variable is now set by all systemd units generated by
`podman-generate-systemd` and is set to `%n` (i.e., the name of systemd
unit starting the container). This data is then being used in the
auto-update sequence to instruct systemd (via DBUS) to restart the unit
and hence to restart the container.
Note that this implementation of auto-updates relies on systemd and
requires a fully-qualified image reference to be used to create the
container. This enforcement is necessary to know which image to
actually check and pull. If we used an image ID, we would not know
which image to check/pull anymore.
Fixes: #3575
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
A number of scripts relating to tooling used and the gate container
image were not exiting upon errors as intended. Coupled with
external service unavailability (i.e. downloading golangci-lint)
was observed to cause difficult to debug failures.
This change corrects the scripts inside/out of the gate container as
well as fixes many golang related path consistency problems vs other CI
jobs. After this change, all jobs use consistent path names reducing
the number of special-case overrides needed.
Lastly, I also made a documentation-pass, updating/correcting as needed,
including documenting a likely local validation-failure mode, related to
`$EPOCH_TEST_COMMIT`. This is dependent on the developers git
environment, so documentation is the only possible "fix".
Signed-off-by: Chris Evich <cevich@redhat.com>
in cases where the log file exceeds the available memory of a system, we had a bug that triggered an oom because the entire logfile was being read when the tail parameter was given. this reads in chunks and is more or less memory safe.
fixes: #5131
Signed-off-by: Brent Baude <bbaude@redhat.com>
We removed the Gitvalidation epoch in the Makefile. As such, we
don't need to adjust it anymore when we tag releases.
Signed-off-by: Matthew Heon <mheon@redhat.com>
Instead of only performing a presence check of the binary, also do a
version check and force installing the specified one if needed. This
will prevent users and the CI from using a wrong version in the future.
Move the logic into a dedicated shell script as I find built-in bash in
Makefiles hard to maintain.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Finding systemd devel packages using libsystemd does not work as
in RHEL based distro the package name is systemd-devel and for
deb/ubunutu it is libsystemd. It is also giving false result when
podman rpm is built with systemd but hack/systemd_tag.sh does not
return anything.
Install systemd-devel package in build_rpm.sh script
Moving to systemd/sd-daemon.h header files which comes from devel
packages fixes the issue.
Signed-off-by: Chandan Kumar (raukadah) <raukadah@gmail.com>
- Adopt bash strict mode
- Avoid cd errors as seen on CI vendor jobs:
hack/get_release_info.sh: line 9: cd: /go/src/github.com/containers/libpod: No such file or directory
Signed-off-by: Sorin Sbarnea <ssbarnea@redhat.com>
* Refactored code and Makefile to support new docs layout
* Removed some old code packaging code
* Add Readme.md to document what we're doing
Signed-off-by: Jhon Honce <jhonce@redhat.com>
Signed-off-by: baude <bbaude@redhat.com>
Restructuring the docs dir to make integration with sphinx easier. man
pages now exist in docs/source/man and the sphinx make files exists in
docs.
Signed-off-by: baude <bbaude@redhat.com>
Instead of unconditionally pulling the x86 binary, clone the repository
and build the binary to make it independent of the architecture.
Fixes: #2699
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Make the errors more readable, with clearer instructions on
what to look for, and which filename, and what we expect to
see, and perhaps even how to approach a fix.
Signed-off-by: Ed Santiago <santiago@redhat.com>
The initial implementation was far more complicated than necessary.
Strip out the complexities in favor of a simpler and more direct
approach.
Signed-off-by: Chris Evich <cevich@redhat.com>
* progress bar: use spinners for unknown blob sizes
* use 'containers_image_ostree' as build tag
* ostree: default is no OStree support
* Add "Env" to ImageInspectInfo
* config.go: improve debug message
* config.go: log where credentials come from
* Fix typo in docs/containers-registries.conf.5.md
* docker: delete: support all MIME types
* Try harder in storageImageDestination.TryReusingBlob
* docker: allow deleting OCI images
* ostree: improve error message
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Remove disused `build_cache_images` task, and
update relevant dockerfiles for F30.
Fix problem of cloud-init failing to expand root-device on boot
(/var/lib/cloud/instance left in improper state).
Fix problem of cloud-init racing with google-network-daemon.service on
boot (looking for cloudconfig metadata too early). Causing
root-device to _sometimes_ fail to expand.
Fix problem of hack/get_ci_vm.sh argument passing.
Signed-off-by: Chris Evich <cevich@redhat.com>
If the systemd development files are not present on the system which
builds podman, then `podman events` will error on runtime creation.
Beside this, a warning will be printed when compiling podman.
This commit mainly exists because projects which depend on libpod
would not need the podman event support and therefore do not need to
rely on the systemd headers.
Signed-off-by: Sascha Grunert <sgrunert@suse.com>
The output of this CI script leaves much to be desired: it is
output from 'diff' with little clarity on what exactly is wrong.
The proper fix is to make the output clear and readable:
podman containers --help lists a 'foo' subcommand that
is not present in docs/podman-containers.1.md
Doing this in bash would take many hours and be fragile
gibberish code. This does not seem worth the effort: the
likely case is that breakages reported by this script
will be due to a newly added subcommand, and the PR
author will find it obvious what to do. Ergo, plan B:
if the test fails, display a blurb at the end describing
how to interpret results. Three minutes' effort, plus
five for writing this commit message.
Signed-off-by: Ed Santiago <santiago@redhat.com>
podman-generate and -play had the wrong NAMEs.
podman-restart and -volume-prune the wrong SYNOPSIS.
All the rest are varying degrees of minor:
- missing a space between the NAME and description
- multi-line SYNOPSIS that could be collapsed into one
- use of UPPER CASE in synopsis instead of *asterisks*
- improper use of **double asterisks** for options
- varlink and version were transposed in podman-1
- fixed inconsistencies between the description in
the man page and that in the parent manpage. These
are too numerous for me to fix all.
Added: script that could be used in CI to prevent future
such inconsistencies. It cannot be enabled yet because
there are still 35+ inconsistencies in need of cleaning.
This will be difficult to review on github. I suggest
pulling the PR and running 'git log -1 -p | cdif | less'
'cdif' is a handy tool for colorizing individual diffs between
lines:
http://kaz-utashiro.github.io/cdif/
There are other such tools; use your favorite. Comparing
without visual highlights may be painful.
I also encourage you to run hack/man-page-checker and suggest
more fixes for the problems it's finding.
Signed-off-by: Ed Santiago <santiago@redhat.com>
* Randomize the user's UID and GID
* Simplify `setup_environment.sh`
* Support new "-r" option for `hack/get_ci_vm.sh` setting up rootless
* Connect as $ROOTLESS_USER when using "-r" with `hack/get_ci_vm.sh`
Signed-off-by: Chris Evich <cevich@redhat.com>
...caught by hack/podman-commands.sh script. Which had a little
buglet, which I fixed: add a special case for 'help', which
neither has nor needs a man page.
I believe the podman-commands.sh script is ready to be run in CI,
hint hint.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Previously, the script would bind mount the user's home directory into
the container in order to execute gcloud commands. This was done
to preserve the `.config/gcloud` directory and new ssh keys in `.ssh`.
However, it's possible the user has modified `.bash*` or `.ssh/config`
files which do not play nicely with gcloud and/or the container.
Fix this by mounting the existing temporary directory on the host, as
the user's home directory. Then bind mount in a dedicated `gcloud/ssh`
sub-directory, and the libpod repo directory on top. Pre-create the
necessary mount-points as the user, so later removal does not require
root on the host.
The gcloud tool takes minutes to setup/manage its ssh-keys, so preserving
that work between runs is a necessary optimization. Similarly, saving the
`.gcloud` directory prevents repeatedly going through the lengthy
client-auth process.
Overall, these changes make the container environment much more selective
with the host-side data it has access to use/modify. Preventing unrelated
details from getting in the way, and preserving only the bare-minimum of
details on the host, between runs.
Signed-off-by: Chris Evich <cevich@redhat.com>
Make more general-purpose: instead of hardcoding a list
of known subcommands, and duplicating sed pipelines for
each, rely on 'podman help' itself to tell us which
podman commands have subcommands; and examine each
in turn. Should there ever be new subcommands, this
will identify and test them.
A special case is needed for 'podman image trust', whose
documentation format doesn't match the others.
The change to `common.go` fixes an inconsistency: the
Usage message for commands with subcommands had an
unnecessary blank line, making it harder to parse
automatically. This simply produces consistent
Usage messages for all podman commands.
This script will not pass until #2480 is merged.
After that, the goal is to add this as a CI hook.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Quote the status output in echo to preserve the new lines.
Having the output in one line complicated debugging issues
and is not friendly to use.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
More complicated than one would think. The first problem is that,
on certain (but not all) Fedora systems, podman cannot mount
volumes read-only (issue #2312). This is baffling, and since
it's not easily reproducible it's likely that the dev team
will not spend much effort on it. Workaround: instead of bind-
mounting /tmp read-only, bind-mount a *tempdir* (subdirectory)
read-write. This is actually cleaner in some ways but it
leads to complications with the paths we use and with cleanup.
Next, allow overriding the default image and allow asking
for no sudo:
export GCLOUD_IMAGE=quay.io/edsantiago/gcloud_centos:latest
export GCLOUD_SUDO=
(yes, that's an equal-sign and EOL. Just an empty string).
The third part, unfortunately, requires a custom image because
the as_dollar_user.sh script (the one that runs gcloud in a
container) is hardwired in a cevich image and needs tweaks
in order to detect rootless and avoid sudo.
Signed-off-by: Ed Santiago <santiago@redhat.com>
* Make sure that all vendored dependencies are in sync with the code and
the vendor.conf by running `make vendor` with a follow-up status check
of the git tree.
* Vendor ginkgo and gomega to include the test dependencies.
Signed-off-by: Chris Evic <cevich@redhat.com>
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Previously it was not possible to specify keys from the ``env`` section
in the various GCE sections. Now that features is added, consolidate
all the cache image definitions into a single place, reducing
maintenance burden.
This also results in the names passing through into the VMs. This is
useful, e.g. for future tracking of image usage statistics.
Update get_ci_vm script hints for new image name definition format
Signed-off-by: Chris Evich <cevich@redhat.com>
Previously, using the ssh command directly required obtaining the
external IP of the VM and was then subject to the local configuration.
If the local configuration and/or ssh keys are incorrect, these commands
would fail, preventing automatic setup of the VM.
Fix this by using the gcloud ssh and scp wrappers. Unfortunately rsync
couldn't be made to work in this situation, so use a tarball to transfer
the local repository to the VM. Lastly, execute `setup_environment.sh`
script, then drop the caller into a bash shell sitting in the remote
`$GOSRC` directory.
Signed-off-by: Chris Evich <cevich@redhat.com>
Add support for executing an init binary as PID 1 in a container to
forward signals and reap processes. When the `--init` flag is set for
podman-create or podman-run, the init binary is bind-mounted to
`/dev/init` in the container and "/dev/init --" is prepended to the
container's command.
The default base path of the container-init binary is `/usr/libexec/podman`
while the default binary is catatonit [1]. This default can be changed
permanently via the `init_path` field in the `libpod.conf` configuration
file (which is recommended for packaging) or temporarily via the
`--init-path` flag of podman-create and podman-run.
[1] https://github.com/openSUSE/catatonitFixes: #1670
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Frequently debugging of CI-related problems requires going hands-on
within the environment. However, reproducing the environment by hand is
very tedious and error prone. This script permits authorized users to
produce VM's based on any available cache-image, and automatically remove
them upon logout.
Also: Bump up VM disk sizes to 200GB due to performance reasons
Signed-off-by: Chris Evich <cevich@redhat.com>
The docker-in-docker was script was needed to run AppArmor tests in
Travis, which is not required anymore since Travis isn't being used
for a while. Removing the script will also cure some hiccups on
some atomic testing nodes.
Signed-off-by: Valentin Rothberg <vrothberg@suse.com>
Make users of libpod more secure by adding the libpod/apparmor package
to load a pre-defined AppArmor profile. Large chunks of libpod/apparmor
come from github.com/moby/moby.
Also check if a specified AppArmor profile is actually loaded and throw
an error if necessary.
The default profile is loaded only on Linux builds with the `apparmor`
buildtag enabled.
Signed-off-by: Valentin Rothberg <vrothberg@suse.com>
Closes: #1063
Approved by: rhatdan
Ed Santiago
Valentin Rothberg
Brent Baude
Chris Evich
Daniel J Walsh
OpenShift Merge Robot
Pascal Bourdier
Paul Holzinger
Valentin Rothberg
Giuseppe Scrivano
jesperpedersen
Jhon Honce
Matthew Heon
Paul Holzinger
Tom Deseyn
Matej Vasek
Josh Soref
Marcel Bargull
Jordan Christiansen
Sascha Grunert
Matthew Heon
Chandan Kumar (raukadah)
Sorin Sbarnea
Dmitry Smirnov
Ryan Whalen
Valentin Rothberg