Commit Graph

144 Commits

Author SHA1 Message Date
Daniel J Walsh 980b1e87d4
Switch use of Flags to Options
Want to have man pages match commands, since we have lots of printed
man pages with using Options, we will change the command line to use
Options in --help.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-10-21 08:37:57 -04:00
Chris Evich 5b7c64cc97
Cirrus: Fix obtaining a CI VM
Also removed automatic exection of setup_environment.sh since most
people using this script are podman developers (not automation/CI
folks).  If executing the automation scripts is necessary, manual
attendance to required variables like `$TEST_FLAVOR` is mandatory.

Signed-off-by: Chris Evich <cevich@redhat.com>
2020-10-07 15:29:43 -04:00
OpenShift Merge Robot 0a46b9c9e6
Merge pull request #7696 from jwhonce/wip/version
Refactor version handling in cmd tree
2020-09-21 13:23:20 +00:00
Jhon Honce c4b49afad3 Refactor version handling in cmd tree
* Move from simple string to semver objects
* Change client API Version from '1' to 2.0.0

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2020-09-18 15:13:58 -07:00
Jordan Christiansen d20a37bf10 Install bats as root
Installing bats to /usr/local requires root privileges. Without this,
`make install.tools` fails. However, if I do `sudo make install.tools`,
then all of the other dependencies and git clones in the current
directory end up owned by root. This limits root privileges to the part
that needs it.

Signed-off-by: Jordan Christiansen <xordspar0@gmail.com>
2020-09-18 12:50:25 -05:00
Daniel J Walsh 526f01cdf5
Fix up errors found by codespell
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-09-11 06:14:25 -04:00
Paul Holzinger b4adc17614 [CI:DOCS] fix swagger api docs
Separate the volume endpoints into compat and libpod,
as it is done for the other endpoints.

Move the libpod image push endpoint to images.

Signed-off-by: Paul Holzinger <paul.holzinger@web.de>
2020-08-23 15:10:22 +02:00
Sascha Grunert ba9f18e2b8
Use `bash` binary from env instead of /bin/bash for scripts
It's not possible to run any of the scripts on distributions which do
have `bash` not in `/bin`. This is being fixed by using `/usr/bin/env
bash` instead.

Signed-off-by: Sascha Grunert <sgrunert@suse.com>
2020-08-17 10:42:23 +02:00
Ed Santiago 8e11a825e1 Cross-reference *.rst files too
There are a bunch of *.rst files in docs/source, linking sometimes
to man pages and sometimes to other .rst files. These files each
have entries of the following form:

   :doc:`foo <link-to-foo>` Description of foo

...for all podman sub and sub-subcommands 'foo'.

Read all .rst files and make sure that:

  - all entries in a given file are in alphabetical order
  - all link-to-foo targets point to existing doc files
  - every subcommand known by 'podman help' has a corresponding
    doc entry in a .rst file

Signed-off-by: Ed Santiago <santiago@redhat.com>
2020-08-11 08:03:37 -06:00
Ed Santiago ca893b8329 Man pages: assert that subcommands are in order
For each podman*.md file with a subcommand table (podman,
podman-container, etc), assert that the subcommand list
is sorted.

Change is bigger than it should be, because it switches from
nice clean local per-function error counting to using a nasty
global.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2020-08-11 07:49:10 -06:00
Daniel J Walsh a5e37ad280
Switch all references to github.com/containers/libpod -> podman
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-07-28 08:23:45 -04:00
Jhon Honce 964d3300c6 [WIP] Refactor podman system connection
* Add support to manage multiple connections
  * Add connection
  * Remove connection
  * Rename connection
  * Set connection as default
  * Add markdown/man pages
* Fix recursion in hack/xref-helpmsgs-manpages

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2020-07-22 15:25:44 -07:00
Valentin Rothberg d628de31d7 unit tests: root check
The unit tests currently require running as root.  This has caused some
confusion that justifies adding a root check to `make localunit` and
error out for non-root users instead of starting the tests deemed to
fail.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-07-21 13:31:43 +02:00
Valentin Rothberg 8489dc4345 move go module to v2
With the advent of Podman 2.0.0 we crossed the magical barrier of go
modules.  While we were able to continue importing all packages inside
of the project, the project could not be vendored anymore from the
outside.

Move the go module to new major version and change all imports to
`github.com/containers/libpod/v2`.  The renaming of the imports
was done via `gomove` [1].

[1] https://github.com/KSubedi/gomove

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-07-06 15:50:12 +02:00
Ed Santiago c6090c290e Docs: consistency between man / --help
New functionality in hack/man-page-checker: start cross-
referencing the man page 'Synopsis' line against the
output of 'podman foo --help'. This is part 1, flag/option
consistency. Part 2 (arg consistency) is too big and will
have to wait for later.

flag/option consistency means: if 'podman foo --help'
includes the string '[flags]' in the Usage message,
make sure the man page includes '[*options*]' in its
Synopsis line, and vice-versa. This found several
inconsistencies, which I've fixed.

While doing this I realized that Cobra automatically
includes a 'Flags:' subsection in its --help output
for all subcommands that have defined flags. This
is great - it lets us cross-check against the
usage synopsis, and make sure that '[flags]' is
present or absent as needed, without fear of
human screwups. If a flag-less subcommand ever
gets extended with flags, but the developer forgets
to add '[flags]' and remove DisableFlagsInUseLine,
we now have a test that will catch that. (This,
too, caught two instances which I fixed).

I don't actually know if the new man-page-checker
functionality will work in CI: I vaguely recall that
it might run before 'make podman' does; and also
vaguely recall that some steps were taken to remedy
that.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2020-06-24 10:39:10 -06:00
Valentin Rothberg 334d3b1ef3 CI: force registry:2.6
For using the `registry:2.6` image. 2.7 and beyond dropped the
`htpasswd` binary from the rootfs which parts of our CI depends
on.

While this is not a sustainable solution (assuming `htpasswd` is gone
for ever), it unblocks the CI for now.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-06-19 12:58:02 +02:00
Valentin Rothberg b6148b6576 force bats version to v1.1.0
We experienced regression when using the latest `v1.2.0-dev` bats in
Ubuntu 20.04 (see github.com/containers/libpod/pull/6418).  Using
bats v1.1.0 worked in the Ubuntu test VM.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-06-08 15:21:34 +02:00
Daniel J Walsh 9263ed2ab1
Remove use of ABISupport buildtag
We need to default to building podman. If you specify no build
tags you will not build podman, not podman-remote.

Just using remote flag to indicate podman-remote and !remote for
podman.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-06-03 15:11:34 -04:00
Ed Santiago b81b865b52 podman-registry: many unrelated fixes
1) fix lost credentials.

    must_pass(), added in #6375, eats the credentials
    generated via 'podman run --entrypoint htpasswd'.
    Run that podman instance directly, and add explicit
    error check.

    (The error and stdout/stderr handling here has gotten
    cumbersome. There must be something I'm missing that
    could make it all simpler.)

 2) fix default podman path.

    When setting $PODMAN, default to the locally built
    one -- there may not be one in $PATH (e.g. in
    Ubuntu, see #6366). This in turn requires us to:

 3) run registry test in integration, not unit test

    It looks like unit tests run before podman is built,
    causing a chicken-egg dilemma. Try to solve that by
    running the new hack/podman-registry-go test in
    integration tests, not unit tests.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2020-05-26 08:21:46 -06:00
Ed Santiago f75ad6d5c2 podman-registry helper script: handle errors
My initial revision of the podman-registry helper script
was written in haste, with an enormous tradeoff: no
visibility into any errors. We are now paying for this
in #6366: the script is failing on Ubuntu and we
have no way of knowing why.

This PR adds a must_pass() function used for critical
steps. This runs the action silently; if the command
fails, it displays the failing command name with
full output logs, cleans up the temporary workdir,
and exits with error status.

As a reminder, the reason this is necessary is that
our script convention is to output a series of
environment variables to stdout -- we must therefore
take pains not to emit anything else to stdout.
And, unfortunately, podman and openssl tend to be
rather verbose.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2020-05-24 08:10:54 -06:00
Daniel J Walsh 35567e706b
Attempt to turn on additional build tests
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-05-22 12:55:23 -04:00
Valentin Rothberg dfe0579a2f add go-bindings for `hack/podman-registry`
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-05-20 15:12:35 +02:00
Ed Santiago 973d624418 New tool: hack/podman-registry, manages local registry
In response to #6207: this is a helper script intended for
use in starting and stopping a local container registry.
It takes care of port, username, password assignments;
generates a self-signed certificate; and starts the
container in an isolated podman root/runroot to avoid
conflicting with the caller's environment.

Intended usage: invoke from shell script, using 'eval'
to get results into calling process environment. See
help message (-h) for invocation details. This will
work for shell scripts but will be difficult if
called from Go or C - if that is likely to happen,
I'd love to hear suggestions for alternate ways to
get the settings back to the caller.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2020-05-20 10:22:16 +02:00
Chris Evich 8d54e4855c
Cirrus: Fix image-name hints
This properly prints out image-name hints when executing the hack script
without any arguments.  It is required due to changes made by Ed for
test-name beatification.  An identical change was made and reviewed by
Ed in the containers/storage repo.

Signed-off-by: Chris Evich <cevich@redhat.com>
2020-05-14 15:00:34 -04:00
Valentin Rothberg 09dc701097 podman rmi: refactor logic
While this commit was initially meant to fix #5847, it has turned into a
bigger refactoring which I did not manage to break into smaller pieces:

 * Fix #5847 by refactoring the image-removal logic.

 * Make the api handler for image-removal use the ABI code. This way,
   both (i.e., ABI and Tunnel) end up using the same code.  Achieving
   this code share required to move some code around to prevent circular
   dependencies.

 * Everything in pkg/api (excluding pkg/api/types) must now only be
   accessed from code using `ABISupport`.

 * Avoid imports from entities on handlers to prevent circular
   dependencies.

 * Move `podman system service` logic into `cmd` to prevent circular
   dependencies - it depends on pkg/api.

 * Also remove the build header from infra/abi files.  It will otherwise
   confuse swagger and other tools; errors we cannot fix as go doesn't
   expose a build-tag env variable.

Fixes: #5847
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-04-21 08:12:25 +02:00
Ed Santiago ba26c763c5 swagger-check: new CI tool to cross-check swagger
New script cross-references r.Handle() and r.HandleFunc()
calls against the preceding '// swagger:operation' comments,
and exits failure (with descriptive error messages) if any
comments do not match the code.

This script should not be necessary: the swagger comments
should be autogenerated from the source code.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2020-04-14 11:27:44 -06:00
Jhon Honce e0847f5457 V2 podman system service
* Added support for system service
* Enabled linting on the varlinkapi source, needed to support V2
  service command
* Added support for PODMAN_SOCKET

Skip linting deprecated code

Rather than introduce bugs by correcting deprecated code, linting the
code is being skipped. Code that is being ported into V2 is being
checked.

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2020-04-07 19:22:10 -07:00
Ed Santiago 108ab380f1 podmanv2-retry - new helper for testing v2
./hack/podmanv2-retry will first invoke $PODMAN_V2 with given
arguments. If that fails with any of the following errors:

    unrecognized command
    unknown flag
    unknown shorthand

...it will run $PODMAN_FALLBACK with the same arguments.
Output and exit code will be those of the final podman command,
although be aware that stderr and stdout are combined.

This is a quick-hack script intended for use in v2 testing, to
test implemented commands without noise from unimplemented ones.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2020-04-01 14:14:00 -06:00
OpenShift Merge Robot 49a7856f2e
Merge pull request #5465 from edsantiago/man_page_option_checker
New test: man page cross-ref against --help
2020-03-19 16:53:02 +01:00
Ed Santiago 81005f463e New test: man page cross-ref against --help
New hack/xref-helpmsgs-manpages script, added to CI 'gate'
task, runs 'podman [subcommand] --help' and cross-references
against man pages in docs/source/markdown/podman*.1.md

See #5453 and #5460 for instances of the problems the
script has found.

The careful reader will find an alarming number of special-case
bypasses. These are a tradeoff I am making: to get perfect
coverage with no handwaving, it would be necessary to make
drastic changes to some man pages, and I believe those would
be counterproductive.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2020-03-18 11:03:26 -06:00
Valentin Rothberg f4e873c4e1 auto updates
Add support to auto-update containers running in systemd units as
generated with `podman generate systemd --new`.

`podman auto-update` looks up containers with a specified
"io.containers.autoupdate" label (i.e., the auto-update policy).

If the label is present and set to "image", Podman reaches out to the
corresponding registry to check if the image has been updated.  We
consider an image to be updated if the digest in the local storage is
different than the one of the remote image.  If an image must be
updated, Podman pulls it down and restarts the container.  Note that the
restarting sequence relies on systemd.

At container-creation time, Podman looks up the "PODMAN_SYSTEMD_UNIT"
environment variables and stores it verbatim in the container's label.
This variable is now set by all systemd units generated by
`podman-generate-systemd` and is set to `%n` (i.e., the name of systemd
unit starting the container).  This data is then being used in the
auto-update sequence to instruct systemd (via DBUS) to restart the unit
and hence to restart the container.

Note that this implementation of auto-updates relies on systemd and
requires a fully-qualified image reference to be used to create the
container.  This enforcement is necessary to know which image to
actually check and pull.  If we used an image ID, we would not know
which image to check/pull anymore.

Fixes: #3575
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-03-17 17:18:56 +01:00
Chris Evich d0782e7839
Cirrus: Fix gate image & false-positive exits
A number of scripts relating to tooling used and the gate container
image were not exiting upon errors as intended.  Coupled with
external service unavailability (i.e. downloading golangci-lint)
was observed to cause difficult to debug failures.

This change corrects the scripts inside/out of the gate container as
well as fixes many golang related path consistency problems vs other CI
jobs.  After this change, all jobs use consistent path names reducing
the number of special-case overrides needed.

Lastly, I also made a documentation-pass, updating/correcting as needed,
including documenting a likely local validation-failure mode, related to
`$EPOCH_TEST_COMMIT`.  This is dependent on the developers git
environment, so documentation is the only possible "fix".

Signed-off-by: Chris Evich <cevich@redhat.com>
2020-03-02 08:50:54 -05:00
Chris Evich 42f04bbdaf
Cirrus: Fix not growing Fedora root
Signed-off-by: Chris Evich <cevich@redhat.com>
2020-02-28 09:54:35 -05:00
Brent Baude e3b31a3ab8 implement reverse reader for log reads
in cases where the log file exceeds the available memory of a system, we had a bug that triggered an oom because the entire logfile was being read when the tail parameter was given.  this reads in chunks and is more or less memory safe.

fixes: #5131

Signed-off-by: Brent Baude <bbaude@redhat.com>
2020-02-20 10:22:45 -06:00
Matthew Heon 662eab1672 Fix a syntax error in hack/release.sh
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-01-31 10:55:17 -05:00
Matthew Heon 554b8d2f2f Update release script to not manage epoch
We removed the Gitvalidation epoch in the Makefile. As such, we
don't need to adjust it anymore when we tag releases.

Signed-off-by: Matthew Heon <mheon@redhat.com>
2020-01-22 07:51:09 -05:00
Valentin Rothberg d4ca3197b6 hack/install_golangci.sh: check env vars
Also make it executable and `set -eo pipefail`.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-01-16 15:26:25 +01:00
Valentin Rothberg b830d3eb07 make .install.golangci-lint: force specific version
Instead of only performing a presence check of the binary, also do a
version check and force installing the specified one if needed.  This
will prevent users and the CI from using a wrong version in the future.

Move the logic into a dedicated shell script as I find built-in bash in
Makefiles hard to maintain.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-01-15 08:55:46 +01:00
Chandan Kumar (raukadah) b54c350a07 Use systemd/sd-daemon.h headers for systemd presence
Finding systemd devel packages using libsystemd does not work as
in RHEL based distro the package name is systemd-devel and for
deb/ubunutu it is libsystemd. It is also giving false result when
podman rpm is built with systemd but hack/systemd_tag.sh does not
return anything.

Install systemd-devel package in build_rpm.sh script

Moving to systemd/sd-daemon.h header files which comes from devel
packages fixes the issue.

Signed-off-by: Chandan Kumar (raukadah) <raukadah@gmail.com>
2019-12-16 20:37:15 +05:30
Sorin Sbarnea 9db92d540a Improve hack/get_release_info.sh
- Adopt bash strict mode
- Avoid cd errors as seen on CI vendor jobs:
hack/get_release_info.sh: line 9: cd: /go/src/github.com/containers/libpod: No such file or directory

Signed-off-by: Sorin Sbarnea <ssbarnea@redhat.com>
2019-12-04 16:23:55 +00:00
Dmitry Smirnov 8d928d525f codespell: spelling corrections
Signed-off-by: Dmitry Smirnov <onlyjob@member.fsf.org>
2019-11-13 08:15:00 +11:00
Jhon Honce 486fcd4e1e Update document formatting and packaging code
* Refactored code and Makefile to support new docs layout
* Removed some old code packaging code
* Add Readme.md to document what we're doing

Signed-off-by: Jhon Honce <jhonce@redhat.com>
Signed-off-by: baude <bbaude@redhat.com>
2019-10-31 12:31:39 -05:00
baude 52b92023ed Restructure documentation dir
Restructuring the docs dir to make integration with sphinx easier.  man
pages now exist in docs/source/man and the sphinx make files exists in
docs.

Signed-off-by: baude <bbaude@redhat.com>
2019-10-31 12:31:39 -05:00
Giuseppe Scrivano 2046be6ae0
build: drop support for ostree
it is going to be removed from containers/image as well, so no longer
depend on it.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2019-10-30 11:28:10 +01:00
OpenShift Merge Robot 049aafa4c0
Merge pull request #4090 from vrothberg/build-catatonit
Build catatonit
2019-10-01 06:05:22 -07:00
Valentin Rothberg 80f6af7766 catatonit: clone and build
Instead of unconditionally pulling the x86 binary, clone the repository
and build the binary to make it independent of the architecture.

Fixes: #2699
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2019-09-25 11:05:41 +02:00
Valentin Rothberg 6f4e3beedc bump catatonit to v0.1.4
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2019-09-25 10:16:01 +02:00
Chris Evich 601052b386
Cirrus: Upload windows MSI release file
Signed-off-by: Chris Evich <cevich@redhat.com>
2019-09-24 15:13:40 -04:00
Ed Santiago 2c73633e3b hack/man_page_checker - improve diagnostics
Make the errors more readable, with clearer instructions on
what to look for, and which filename, and what we expect to
see, and perhaps even how to approach a fix.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2019-09-09 09:37:52 -06:00
Ryan Whalen d266dbea15 Exclude podman-remote
Signed-off-by: Ryan Whalen <rj.whalen@gmail.com>
2019-08-31 08:05:51 -04:00
Ryan Whalen c153895424 WIP - ignore man pages for commands besides podman
Signed-off-by: Ryan Whalen <rj.whalen@gmail.com>
2019-08-30 12:47:41 -04:00
Ryan Whalen e35a5a674d Fix formatting and enable hack/man-page-checker
Signed-off-by: Ryan Whalen <rj.whalen@gmail.com>
2019-08-30 12:00:19 -04:00
Chris Evich 370b1a887c
Cirrus: Reimplement release archive + upload
The initial implementation was far more complicated than necessary.
Strip out the complexities in favor of a simpler and more direct
approach.

Signed-off-by: Chris Evich <cevich@redhat.com>
2019-08-28 11:54:06 -04:00
Chris Evich b843804d51
Adjust get_ci_vm.sh for substitution
Signed-off-by: Chris Evich <cevich@redhat.com>
2019-08-12 12:01:55 -04:00
Valentin Rothberg c1b792c1e0 vendor github.com/containers/image@v2.0.1
* progress bar: use spinners for unknown blob sizes
* use 'containers_image_ostree' as build tag
* ostree: default is no OStree support
* Add "Env" to ImageInspectInfo
* config.go: improve debug message
* config.go: log where credentials come from
* Fix typo in docs/containers-registries.conf.5.md
* docker: delete: support all MIME types
* Try harder in storageImageDestination.TryReusingBlob
* docker: allow deleting OCI images
* ostree: improve error message

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2019-07-22 14:35:03 +02:00
Chris Evich 11484580d3
Cirrus: Add support for testing F30
Remove disused `build_cache_images` task, and
update relevant dockerfiles for F30.

Fix problem of cloud-init failing to expand root-device on boot
(/var/lib/cloud/instance left in improper state).

Fix problem of cloud-init racing with google-network-daemon.service on
boot (looking for cloudconfig metadata too early).  Causing
root-device to _sometimes_ fail to expand.

Fix problem of hack/get_ci_vm.sh argument passing.

Signed-off-by: Chris Evich <cevich@redhat.com>
2019-06-14 13:41:58 -04:00
Chris Evich 57ada47ce5
hack: support setting local region/zone
Signed-off-by: Chris Evich <cevich@redhat.com>
2019-05-29 09:30:50 -04:00
Chris Evich 1bf622a0ea
hack: ignore from all VCS files when tarballing
Signed-off-by: Chris Evich <cevich@redhat.com>
2019-05-22 13:13:40 -04:00
Chris Evich 53a2994a22
hack: shrink xfer tarball size
Signed-off-by: Chris Evich <cevich@redhat.com>
2019-05-22 12:44:38 -04:00
Chris Evich 99bd1d31c7
hack: Display IP address of VM from script
Useful for accessing it from other terminals.

Signed-off-by: Chris Evich <cevich@redhat.com>
2019-05-22 12:44:37 -04:00
Chris Evich 95141f88d4
Cirrus: fixups based on review feedback
Also remove disused distros (RHEL/CentOS/FAH) and fix get_ci_vm script

Signed-off-by: Chris Evich <cevich@redhat.com>
2019-05-21 08:44:02 -04:00
Sascha Grunert d1fc3fc702
Add `systemd` build tag
If the systemd development files are not present on the system which
builds podman, then `podman events` will error on runtime creation.
Beside this, a warning will be printed when compiling podman.

This commit mainly exists because projects which depend on libpod
would not need the podman event support and therefore do not need to
rely on the systemd headers.

Signed-off-by: Sascha Grunert <sgrunert@suse.com>
2019-05-13 14:00:27 +02:00
Chris Evich b648ca0810
Cirrus: Add check for make varlink_api_generate
Signed-off-by: Chris Evich <cevich@redhat.com>
2019-05-03 15:04:48 -04:00
Ed Santiago b57d2c7d70 CI check for --help vs man pages: usability fix
The output of this CI script leaves much to be desired: it is
output from 'diff' with little clarity on what exactly is wrong.

The proper fix is to make the output clear and readable:

    podman containers --help lists a 'foo' subcommand that
     is not present in docs/podman-containers.1.md

Doing this in bash would take many hours and be fragile
gibberish code. This does not seem worth the effort: the
likely case is that breakages reported by this script
will be due to a newly added subcommand, and the PR
author will find it obvious what to do. Ergo, plan B:
if the test fails, display a blurb at the end describing
how to interpret results. Three minutes' effort, plus
five for writing this commit message.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2019-04-08 08:17:13 -06:00
OpenShift Merge Robot b20594ea51
Merge pull request #2433 from cevich/hack_around_home
[ci skip] get_ci_vm.sh: Fix conflicting homedir files
2019-03-27 06:53:17 -07:00
Ed Santiago beb71323b1 man pages - consistency fixes
podman-generate and -play had the wrong NAMEs.

podman-restart and -volume-prune the wrong SYNOPSIS.

All the rest are varying degrees of minor:

  - missing a space between the NAME and description
  - multi-line SYNOPSIS that could be collapsed into one
  - use of UPPER CASE in synopsis instead of *asterisks*
  - improper use of **double asterisks** for options
  - varlink and version were transposed in podman-1
  - fixed inconsistencies between the description in
    the man page and that in the parent manpage. These
    are too numerous for me to fix all.

Added: script that could be used in CI to prevent future
such inconsistencies. It cannot be enabled yet because
there are still 35+ inconsistencies in need of cleaning.

This will be difficult to review on github. I suggest
pulling the PR and running 'git log -1 -p | cdif | less'

'cdif' is a handy tool for colorizing individual diffs between
lines:

   http://kaz-utashiro.github.io/cdif/

There are other such tools; use your favorite. Comparing
without visual highlights may be painful.

I also encourage you to run hack/man-page-checker and suggest
more fixes for the problems it's finding.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2019-03-20 14:37:59 -06:00
Chris Evich 37eb694bd0
Cirrus: Various fixes for rootless testing
* Randomize the user's UID and GID
* Simplify `setup_environment.sh`
* Support new "-r" option for `hack/get_ci_vm.sh` setting up rootless
* Connect as $ROOTLESS_USER when using "-r" with `hack/get_ci_vm.sh`

Signed-off-by: Chris Evich <cevich@redhat.com>
2019-03-19 13:00:18 -04:00
baude c6b205be77
Enable rootless integration tests
Signed-off-by: baude <bbaude@redhat.com>
2019-03-19 15:01:48 +01:00
Ed Santiago 91e15bd802 add podman-healthcheck(1) to podman(1)
...caught by hack/podman-commands.sh script. Which had a little
buglet, which I fixed: add a special case for 'help', which
neither has nor needs a man page.

I believe the podman-commands.sh script is ready to be run in CI,
hint hint.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2019-03-07 14:33:25 -07:00
Chris Evich 0666d25b4f
get_ci_vm.sh: Fix conflicting homedir files
Previously, the script would bind mount the user's home directory into
the container in order to execute gcloud commands.  This was done
to preserve the `.config/gcloud` directory and new ssh keys in `.ssh`.
However, it's possible the user has modified `.bash*` or `.ssh/config`
files which do not play nicely with gcloud and/or the container.

Fix this by mounting the existing temporary directory on the host, as
the user's home directory.  Then bind mount in a dedicated `gcloud/ssh`
sub-directory, and the libpod repo directory on top.  Pre-create the
necessary mount-points as the user, so later removal does not require
root on the host.

The gcloud tool takes minutes to setup/manage its ssh-keys, so preserving
that work between runs is a necessary optimization.  Similarly, saving the
`.gcloud` directory prevents repeatedly going through the lengthy
client-auth process.

Overall, these changes make the container environment much more selective
with the host-side data it has access to use/modify.  Preventing unrelated
details from getting in the way, and preserving only the bare-minimum of
details on the host, between runs.

Signed-off-by: Chris Evich <cevich@redhat.com>
2019-03-01 12:12:46 -05:00
Ed Santiago 742028e255 podman-commands script: refactor
Make more general-purpose: instead of hardcoding a list
of known subcommands, and duplicating sed pipelines for
each, rely on 'podman help' itself to tell us which
podman commands have subcommands; and examine each
in turn. Should there ever be new subcommands, this
will identify and test them.

A special case is needed for 'podman image trust', whose
documentation format doesn't match the others.

The change to `common.go` fixes an inconsistency: the
Usage message for commands with subcommands had an
unnecessary blank line, making it harder to parse
automatically. This simply produces consistent
Usage messages for all podman commands.

This script will not pass until #2480 is merged.
After that, the goal is to add this as a CI hook.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2019-02-28 13:16:24 -07:00
Daniel J Walsh b4b7b298fc
Clean up man pages to match commands
Also add podman-commands.sh to compare man pages to commands.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2019-02-27 16:37:40 -05:00
Valentin Rothberg 28ecb0f3da hack/tree_status.sh: preserve new lines
Quote the status output in echo to preserve the new lines.
Having the output in one line complicated debugging issues
and is not friendly to use.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2019-02-20 13:35:52 +01:00
Ed Santiago 17a5f39739 get_ci_vm : allow running without sudo
More complicated than one would think. The first problem is that,
on certain (but not all) Fedora systems, podman cannot mount
volumes read-only (issue #2312). This is baffling, and since
it's not easily reproducible it's likely that the dev team
will not spend much effort on it. Workaround: instead of bind-
mounting /tmp read-only, bind-mount a *tempdir* (subdirectory)
read-write. This is actually cleaner in some ways but it
leads to complications with the paths we use and with cleanup.

Next, allow overriding the default image and allow asking
for no sudo:

    export GCLOUD_IMAGE=quay.io/edsantiago/gcloud_centos:latest
    export GCLOUD_SUDO=

(yes, that's an equal-sign and EOL. Just an empty string).

The third part, unfortunately, requires a custom image because
the as_dollar_user.sh script (the one that runs gcloud in a
container) is hardwired in a cevich image and needs tweaks
in order to detect rootless and avoid sudo.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2019-02-13 05:27:56 -07:00
Valentin Rothberg 9ac0ebb079 Cirrus: add vendor_check_task
* Make sure that all vendored dependencies are in sync with the code and
  the vendor.conf by running `make vendor` with a follow-up status check
  of the git tree.

* Vendor ginkgo and gomega to include the test dependencies.

Signed-off-by: Chris Evic <cevich@redhat.com>
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2019-02-06 11:14:06 +01:00
Chris Evich d0876282a4
Cirrus: Consolidate VM image names in once place
Previously it was not possible to specify keys from the ``env`` section
in the various GCE sections.  Now that features is added, consolidate
all the cache image definitions into a single place, reducing
maintenance burden.

This also results in the names passing through into the VMs. This is
useful, e.g. for future tracking of image usage statistics.

Update get_ci_vm script hints for new image name definition format

Signed-off-by: Chris Evich <cevich@redhat.com>
2019-01-17 13:30:06 -05:00
Chris Evich 8ee7eb472d
[skip ci] Hack: Fix get_ci_vm.sh w/ gcloud ssh/scp
Previously, using the ssh command directly required obtaining the
external IP of the VM and was then subject to the local configuration.
If the local configuration and/or ssh keys are incorrect, these commands
would fail, preventing automatic setup of the VM.

Fix this by using the gcloud ssh and scp wrappers.  Unfortunately rsync
couldn't be made to work in this situation, so use a tarball to transfer
the local repository to the VM.  Lastly, execute `setup_environment.sh`
script, then drop the caller into a bash shell sitting in the remote
`$GOSRC` directory.

Signed-off-by: Chris Evich <cevich@redhat.com>
2019-01-14 10:45:12 -05:00
Valentin Rothberg 75578aad61 add container-init support
Add support for executing an init binary as PID 1 in a container to
forward signals and reap processes.  When the `--init` flag is set for
podman-create or podman-run, the init binary is bind-mounted to
`/dev/init` in the container and "/dev/init --" is prepended to the
container's command.

The default base path of the container-init binary is `/usr/libexec/podman`
while the default binary is catatonit [1].  This default can be changed
permanently via the `init_path` field in the `libpod.conf` configuration
file (which is recommended for packaging) or temporarily via the
`--init-path` flag of podman-create and podman-run.

[1] https://github.com/openSUSE/catatonit

Fixes: #1670
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2019-01-04 11:42:03 +01:00
Chris Evich 7b53e86e4f
Add script to create CI VMs for debugging
Frequently debugging of CI-related problems requires going hands-on
within the environment.  However, reproducing the environment by hand is
very tedious and error prone.  This script permits authorized users to
produce VM's based on any available cache-image, and automatically remove
them upon logout.

Also: Bump up VM disk sizes to 200GB due to performance reasons

Signed-off-by: Chris Evich <cevich@redhat.com>
2018-12-14 14:34:43 -05:00
Valentin Rothberg 08cab4ad3a remove hack/dind
The docker-in-docker was script was needed to run AppArmor tests in
Travis, which is not required anymore since Travis isn't being used
for a while.  Removing the script will also cure some hiccups on
some atomic testing nodes.

Signed-off-by: Valentin Rothberg <vrothberg@suse.com>
2018-10-10 18:05:50 +02:00
Valentin Rothberg 8569ed0305 AppArmor: runtime check if it's enabled on the host
Check at runtime if AppArmor is enabled on the host.

Signed-off-by: Valentin Rothberg <vrothberg@suse.com>

Closes: #1128
Approved by: mheon
2018-07-23 18:11:09 +00:00
Giuseppe Scrivano 1219a3cab3 build: enable ostree in containers/storage when available
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>

Closes: #1071
Approved by: rhatdan
2018-07-11 17:26:21 +00:00
Valentin Rothberg 06ab343bd7 podman/libpod: add default AppArmor profile
Make users of libpod more secure by adding the libpod/apparmor package
to load a pre-defined AppArmor profile.  Large chunks of libpod/apparmor
come from github.com/moby/moby.

Also check if a specified AppArmor profile is actually loaded and throw
an error if necessary.

The default profile is loaded only on Linux builds with the `apparmor`
buildtag enabled.

Signed-off-by: Valentin Rothberg <vrothberg@suse.com>

Closes: #1063
Approved by: rhatdan
2018-07-11 16:36:24 +00:00
W. Trevor King 7a5c376e63 Makefile: Use 'git diff' to show gofmt changes
This makes fixing errors easier.  Before this commit, errors looked
like [1]:

  $ make gofmt
  libpod/container_linux.go:1:⚠️ file is not gofmted with -s (gofmt)
  make: *** [gofmt] Error 1

But that's not very helpful when your local gofmt thinks the file is
fine.  With this commit, errors will look like:

  $ make gofmt
  find . -name '*.go' ! -path './vendor/*' -exec gofmt -s -w {} \+
  git diff --exit-code
  diff --git a/libpod/container_internal.go b/libpod/container_internal.go
  index df4de3fe..22b39870 100644
  --- a/libpod/container_internal.go
  +++ b/libpod/container_internal.go
  @@ -1,7 +1,7 @@
   package libpod

   import (
  -"bytes"
  +       "bytes"
          "context"
          "encoding/json"
          "fmt"
  make: *** [Makefile:87: gofmt] Error 1

(or whatever, I just stuffed in a formatting error for demonstration
purposes).

Also remove the helper script in favor of direct Makefile calls,
because with Git handling difference reporting and exit status, this
becomes a simpler check.  find's -exec, !, and -path arguments are
specified in POSIX [2].

[1]: https://travis-ci.org/kubernetes-incubator/cri-o/jobs/331949394#L1075
[2]: http://pubs.opengroup.org/onlinepubs/9699919799/utilities/find.html

Signed-off-by: W. Trevor King <wking@tremily.us>

Closes: #1038
Approved by: rhatdan
2018-07-03 10:39:54 +00:00
W. Trevor King 564578989b hack/ostree_tag.sh: Fill in OSTree dependencies
Copying the libraries from:

  $ git grep pkg-config vendor/github.com/containers/image/
  vendor/github.com/containers/image/ostree/ostree_dest.go:// #cgo pkg-config: glib-2.0 gobject-2.0 ostree-1 libselinux
  vendor/github.com/containers/image/ostree/ostree_src.go:// #cgo pkg-config: glib-2.0 gobject-2.0 ostree-1

We need all of those to compile the vendored Go dependency, not just
ostree-1.

Signed-off-by: W. Trevor King <wking@tremily.us>

Closes: #958
Approved by: giuseppe
2018-06-18 19:16:08 +00:00
W. Trevor King cb430d58e6 hack/release.sh: Add a guard against -dev suffixes for argv[2]
Because it's easier to recover from that if we fail early instead of
going through and creating a "Bump to v1.2.3-dev-dev" commit, etc.

Signed-off-by: W. Trevor King <wking@tremily.us>

Closes: #926
Approved by: rhatdan
2018-06-10 10:51:11 +00:00
W. Trevor King 96dc5fd0d5 hack/release.sh: Bump spec in dev_version_commit
Bump it to the next version (without a -dev suffix), based on the
precedent set by 70672652 (Bump to v0.6.1-dev, 2018-05-25, #834).
Previously I had VERSION there, which was a copy/paste error.

I've also added an explicit write_spec_version to release_commit.
That *should* be a no-op, with the spec version having already been
set by the previous release's dev_version_commit.  But better to be
safe than to cut a release with the wrong version number in the spec
file (e.g. maybe we guessed NEXT_VERSION wrong during the last
release).

Signed-off-by: W. Trevor King <wking@tremily.us>

Closes: #879
Approved by: mheon
2018-06-01 21:39:45 +00:00
W. Trevor King 500b94968b hack/release.sh: No longer need to bump setup.py
Since 727ecfea (Use Version from spec file in setup.py, 2018-05-18, #807),
setup.py has been pulling this from a PODMAN_VERSION environment
variable (which can be set in spec files), and there's no need for us
to bump it as part of our releases.

Signed-off-by: W. Trevor King <wking@tremily.us>

Closes: #879
Approved by: mheon
2018-06-01 21:39:45 +00:00
W. Trevor King 8405ff9baa hack/release.sh: Add a release script
Matthew had expressed interest in a lovely release script on IRC.
Here's my attempt to encode the changes from the v0.5.4 release
branch.  I've also added tag signing, so you may be prompted for your
passphrase during that step.

The version scheme for 0.x.y is 0.${month}.${count_that_month} [1].
We could automatically calculate those with a dozen or so lines of
shell script, but we don't think that's worth the maintenance burden
when it's easy enough for the caller to think them up on their own
[2].

The spec sed also bumps the Python package version to match, which
seems like the intended behavior until 1.0 when the Python code will
move into its own repository [3].

[1]: https://github.com/projectatomic/libpod/pull/867#issuecomment-393731907
[2]: https://github.com/projectatomic/libpod/pull/867#issuecomment-393743295
[3]: https://github.com/projectatomic/libpod/issues/786#issuecomment-390682012

Signed-off-by: W. Trevor King <wking@tremily.us>
2018-05-31 21:04:46 -07:00
W. Trevor King 5b2627dd77 Makefile: Drop find-godeps.sh for podman target
We inherited this from a031b83a (Initial checkin from CRI-O repo,
2017-11-01), but:

* The output is actually going into bin/podman, so Make will rebuild
  this target every time.  You'll never be able to save compilation
  because the target is newer than all the prerequisites.

* Make expands prerequisites immediately when loading a Makefile [1],
  and on my wimpy Chromebook SD Card, this is *slow*:

    $ time hack/find-godeps.sh ~/.local/lib/go/src/github.com/projectatomic/libpod cmd/podman github.com/projectatomic/libpod
    ...
    real    0m56.225s
    user    0m44.918s
    sys     0m21.918s

* Go is pretty good at this on its own, so having make call 'go build'
  every time will almost certainly be faster than us trying to mimic
  this in a shell script.  And by punting to Go in the recipe, Make
  invocations that do not need the podman target (e.g. 'make help')
  can skip the dependency lookup entirely.

[1]: https://www.gnu.org/software/make/manual/html_node/Reading-Makefiles.html#Rule-Definition

Signed-off-by: W. Trevor King <wking@tremily.us>

Closes: #776
Approved by: rhatdan
2018-05-16 15:42:34 +00:00
baude 8493dba23c Initial varlink implementation
Signed-off-by: baude <bbaude@redhat.com>

Closes: #627
Approved by: mheon
2018-04-23 14:29:45 +00:00
baude 61f606e192 Add fix_gofmt target
fix_gofmt will run gofmt -s -w on files that need to be
formatted.  Useful for developers prior to checking code
in.

Signed-off-by: baude <bbaude@redhat.com>

Closes: #125
Approved by: baude
2017-12-13 20:52:51 +00:00
baude 99f905243b Narrow gofmt targets
Disregard _output for gofmt'ing

Signed-off-by: baude <bbaude@redhat.com>

Closes: #77
Approved by: rhatdan
2017-11-27 17:17:11 +00:00
Matthew Heon a031b83a09 Initial checkin from CRI-O repo
Signed-off-by: Matthew Heon <matthew.heon@gmail.com>
2017-11-01 11:24:59 -04:00