Commit Graph

6355 Commits

Author SHA1 Message Date
Chris Evich 1abb217493
Cirrus: Disable most periodic services/timers
For CI testing, it's important to remove as much variability from the
overall system as possible.  This permits focusing just on problems
closely related to code-changes.  To this end, and because VMs are very
short-lived (2 hours at most), disable all systemd services and timers
which perform periodic activities.

Signed-off-by: Chris Evich <cevich@redhat.com>
2019-07-16 14:30:42 -04:00
OpenShift Merge Robot a449e9a2cf
Merge pull request #3546 from cevich/abstract_dest_branch
Cirrus: Abstract destination branch refs.
2019-07-16 19:54:51 +02:00
OpenShift Merge Robot 386ffd28d6
Merge pull request #3539 from stefanb2/topic-pr-3507-3525
Fix handling of healthcheck from image
2019-07-16 16:38:09 +02:00
OpenShift Merge Robot 7e4db4452f
Merge pull request #3513 from vrothberg/analyses
Binary bloat analyses
2019-07-16 15:52:50 +02:00
Valentin Rothberg 294ddc4720 dependency/analyses: simplify scripts
Credits to bash wizard @edsantiago for the changes.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2019-07-16 14:16:16 +02:00
Valentin Rothberg 25d63f009d dependency-tree analysis: direct and transitive
Change the script to generate two files.  One including direct
dependencies, the other including direct and transitive dependencies.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2019-07-16 14:16:16 +02:00
Valentin Rothberg 9ae3e7c1ec analyses: README: consistent code examples
Remove a blank line to make code examples more consistent and fix the
path of the 2nd example.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2019-07-16 14:16:16 +02:00
Valentin Rothberg 525c1ba897 analyses: README: fix typos
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2019-07-16 14:16:16 +02:00
Valentin Rothberg 849e2f3d4e analyses: add dependency-tree.sh
Add a new analysis script to print the dependency tree.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2019-07-16 14:16:16 +02:00
Valentin Rothberg fb31cc95a5 analyses: add README.md
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2019-07-16 14:16:16 +02:00
Valentin Rothberg 88058c3ce2 hack/analyses -> dependencies/analyses
Move the analyses scripts to the dependencies directory to avoid
scattering of the dependency management.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2019-07-16 14:16:16 +02:00
Valentin Rothberg 65a600332d hack/analyses/go-archive-analysis.sh: fix sorting
Believe it or not:
`sort -ruh` is loosing data while `sort -u | sort -rh` does not.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2019-07-16 14:16:16 +02:00
Valentin Rothberg 7ab82579bb add hack/analyses/nm-symbols-analysis.sh
The script allows for analyzing the symbols of a go binary passed as an
argument.  The tabular output looks as follows:

336             unicode/utf8.DecodeLastRune
323             unicode/utf8.DecodeLastRuneInString
518             unicode/utf8.DecodeRune
518             unicode/utf8.DecodeRuneInString
337             unicode/utf8.EncodeRune

The first column indicates the size in bytes of the symbol in the second
column.  Note that only text symbols are considered, other symbols from
the data or the bss segment are ignored to avoid information overload.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2019-07-16 14:16:16 +02:00
Valentin Rothberg aa28dbbf88 analyse package sizes
Analyse the size of all go-packages used during the build process via
the newly added `hack/analyses/go-archive-analysis.sh` script.  The
script expects the `WORK` environment variable to be set, which points
to a temporary work directory generated by `go build`.  To generate such
a work directory, set the `BUILDFLAGS="-work -a"`:

 * `-work` for creating the work directory
 * `-a` to force rebuilding all packages even when already cached

The workflow may look as follows:

```
$ BUILDFLAGS="-work -a" make podman
[...]
WORK=/tmp/go-build127001249
$ WORK=/tmp/go-build127001249 ./hack/analyses/go-archive-analysis.sh
```

The output of the script has the format `$SIZE $PACKAGE` where $SIZE is
the size of the compiled version of the go package (i.e., `.a` file) and
$PACKAGE for the corresponding package, for instance, `math/big` for a
stdlib package or vendor/...  for vendored packages.

Credits to the authors of https://github.com/jondot/goweight, which
inspired this work.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2019-07-16 14:16:16 +02:00
Hunor Csomortáni 47c1fe6a5c Completion: complete "--health-start-period" in bash
Signed-off-by: Hunor Csomortáni <csomh@redhat.com>
2019-07-16 06:15:30 +02:00
Hunor Csomortáni 3e53bfe36e Make the healthcheck flags compatible with Docker CLI
Docker CLI calls the healthcheck flags "--health-*", instead of
"--healthcheck-*".

Introduce the former, in order to keep compatibility, and alias
the later, in order to avoid breaking current usage.

Change "--healthcheck-*" to "--health-*" in the docs and tests.

Signed-off-by: Hunor Csomortáni <csomh@redhat.com>
2019-07-16 06:15:28 +02:00
Stefan Becker 5ed2de158f healthcheck: reject empty commands
An image with "HEALTHCHECK CMD ['']" is valid but as there is no command
defined the healthcheck will fail. Reject such a configuration.

Fixes #3507

Signed-off-by: Stefan Becker <chemobejk@gmail.com>
2019-07-16 07:01:43 +03:00
Stefan Becker 5caf218b11 create: ignore check if image has HEALTHCHECK NONE
If the image was built with "HEALTHCHECK NONE" then we should create a
container without healthcheck configuration. Otherwise executing the
healthcheck on the container will return "unhealthy" instead of the
correct error message that the container doesn't have a healthcheck.

We also ignore the healthcheck configuration if the command list is
empty or the command string is empty.

Fixes #3525

Signed-off-by: Stefan Becker <chemobejk@gmail.com>
2019-07-16 07:01:43 +03:00
Stefan Becker 33001a9028 create: apply defaults on image healthcheck options
If the image doesn't provide any options, e.g. interval, timeout, etc.,
then apply the Docker defaults when creating the container. Otherwise
the defaults will be left 0 and podman doesn't schedule the healtcheck
service & timer for the container or incorrectly reports unhealthy state
when the check is executed.

Fixes #3525

Signed-off-by: Stefan Becker <chemobejk@gmail.com>
2019-07-16 07:01:43 +03:00
Stefan Becker dd0ea08cef healthcheck: improve command list parser
- remove duplicate check, already called in HealthCheck()
- reject zero-length command list and empty command string as errorneous
- support all Docker command list keywords: NONE, CMD or CMD-SHELL
- use Docker default "/bin/sh -c" for CMD-SHELL

Fixes #3507

Signed-off-by: Stefan Becker <chemobejk@gmail.com>
2019-07-16 07:01:43 +03:00
Hunor Csomortáni 51c00eae8b Completion: --no-healthcheck is not an option
Signed-off-by: Hunor Csomortáni <csomh@redhat.com>
2019-07-16 05:50:52 +02:00
Chris Evich 35c174f723
Cirrus: Abstract destination branch refs.
Various tasks and scripts behave differently depending on whether or not
the build is running against a PR or on a branch, post-merge.  However,
a great number of them are hard-coded to the string 'master' as the
destination.  Since this is not always the case (there are other
relevant branches), it makes sense to abstract the references with a
single definition.

Add a top-level `$DEST_BRANCH` variable to CI, and otherwise
default to 'master' when unset.  This enables running CI builds on
additional branches without the overhead of updating all the static
references to 'master'.  Simply update `$DEST_BRANCH` at the top-level
and all branch-conditional logic will function as intended.

Signed-off-by: Chris Evich <cevich@redhat.com>
2019-07-15 16:42:41 -04:00
OpenShift Merge Robot d2291ecdd5
Merge pull request #3574 from stefanb2/topic-health-cmd-parsing
Improve parser for --healthcheck-command
2019-07-15 21:57:52 +02:00
OpenShift Merge Robot a28f8dbecd
Merge pull request #3531 from haraldh/fix_ReplySendFile
Fix the double replySendFile()
2019-07-15 21:41:00 +02:00
Chris Evich e8564dc44f
Cirrus: Print images that should be pruned
Over time unless they're removed, the project could grow quite a large
collection of VM images.  While generally cheap (less than a penny each,
per month), these will become a significant cost item if not kept
in-check.

Add a specialized container for handling image-pruning, but limit
it to only finding and printing (not actually deleting) images.

Also update the image-building workflow so that base-images used to
compose cache-images are also labeled with metadata.

N/B: As an additional safeguard, the service account which
     executes the new container in production *DOES NOT*
     have access to delete images.  This can be enabled
     by adding the GCE IAM role: CustomComputeImagePrune

Signed-off-by: Chris Evich <cevich@redhat.com>
2019-07-15 15:26:26 -04:00
OpenShift Merge Robot 547cb4e55e
Merge pull request #3532 from mheon/ensure_store_on_refresh
Ensure we have a valid store when we refresh
2019-07-15 21:26:16 +02:00
OpenShift Merge Robot 456b6ab1c2
Merge pull request #3530 from deefin/deefin-3490-logging
Improves STD output/readability in combination
2019-07-14 12:45:35 +02:00
Stefan Becker e4cba7d36a create: improve parser for --healthcheck-command
Fix Docker CLI compatibility issue: the "--healthcheck-command" option
value should not be split but instead be passed as single string to
"CMD-SHELL", i.e. "/bin/sh -c <opt>".

On the other hand implement the same extension as is already available
for "--entrypoint", i.e. allow the option value to be a JSON array of
strings. This will make life easier for tools like podman-compose.

Updated "--healthcheck-command" option values in tests accordingly.

Continuation of #3455 & #3507

Signed-off-by: Stefan Becker <chemobejk@gmail.com>
2019-07-14 10:33:01 +03:00
dom finn ee76ba5e68 Improves STD output/readability in combination
with debug output.

Added \n char to specific standard output

Signed-off-by: dom finn <dom.finn00@gmail.com>
2019-07-14 16:03:49 +10:00
OpenShift Merge Robot c59d08bc77
Merge pull request #3564 from giuseppe/cgroupsv2-no-controllers-last-path
cgroupsv2: do not enable controllers for the last component
2019-07-13 00:21:18 +02:00
OpenShift Merge Robot 9d87945005
Merge pull request #3563 from giuseppe/fix-single-mapping-rootless
spec: fix userns with less than 5 gids
2019-07-12 22:31:37 +02:00
OpenShift Merge Robot 62352b280b
Merge pull request #3537 from QiWang19/volumeabs
fix bug convert volume host path to absolute
2019-07-12 22:12:21 +02:00
OpenShift Merge Robot 686da0d60c
Merge pull request #3534 from cevich/img_sys_tst
Cirrus: Execute system-tests during image-validation
2019-07-12 16:10:37 +02:00
Harald Hoyer d52a2d1e54
Fix the double replySendFile()
Signed-off-by: Harald Hoyer <harald@redhat.com>
2019-07-12 15:36:08 +02:00
Chris Evich dc385fc8b6
Cirrus: Update to freshly built cache-images
Signed-off-by: Chris Evich <cevich@redhat.com>
2019-07-12 08:44:28 -04:00
Chris Evich 6946fe05ed
Cirrus: Execute system-tests during image-validation
Signed-off-by: Chris Evich <cevich@redhat.com>
2019-07-12 08:44:27 -04:00
Chris Evich 9043b816d4
Cirrus: Fix missing removal of packaged podman
This was originally intended, but somehow omitted from #1936

Signed-off-by: Chris Evich <cevich@redhat.com>
2019-07-12 08:44:27 -04:00
Giuseppe Scrivano 6c8ceaf21e
cgroupsv2: do not enable controllers for the last component
do not automatically enable the controllers for the last path
component.  It is necessary as once there are enabled controllers in a
cgroup, it won't possible to add processes to it.

Fix conmon being moved to the correct cgroup path when using
--cgroup-manager cgroupfs.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2019-07-12 12:11:06 +02:00
Giuseppe Scrivano d74db186a8
spec: fix userns with less than 5 gids
when the container is running in a user namespace, check if gid=5 is
available, otherwise drop the option gid=5 for /dev/pts.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2019-07-12 11:35:03 +02:00
OpenShift Merge Robot 6f3e7f7ecc
Merge pull request #3557 from rhatdan/env
Add support for --env-host
2019-07-12 10:33:18 +02:00
Daniel J Walsh 369f8b8862
Fix spelling mistakes in man pages and other docs
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2019-07-11 17:56:54 -04:00
Daniel J Walsh efe9c5b0e7
Add glob parsing for --env flag
Sometimes you want to add a few environmen variables based on the last field being a "*".

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2019-07-11 17:15:20 -04:00
OpenShift Merge Robot 20f11718de
Merge pull request #3558 from mheon/fix_pod_remove
Fix a bug where ctrs could not be removed from pods
2019-07-11 21:35:53 +02:00
OpenShift Merge Robot d614372c2f
Merge pull request #3552 from baude/golangcilint2
golangci-lint pass number 2
2019-07-11 21:35:45 +02:00
OpenShift Merge Robot 2b64f88446
Merge pull request #3491 from giuseppe/rlimit-host
podman: add --ulimit host
2019-07-11 21:35:37 +02:00
OpenShift Merge Robot 24409daa36
Merge pull request #3470 from QazerLab/docs/system-tests
Update the testing documentation with system tests.
2019-07-11 21:35:30 +02:00
OpenShift Merge Robot e065612701
Merge pull request #3554 from giuseppe/fix-cgroupfs-cleanup
cgroups: fix a leak when using cgroupfs
2019-07-11 21:23:29 +02:00
Daniel J Walsh df75fc62c8
Add support for -env-host
This flag passes the host environment into the container.  The basic idea is to
leak all environment variables from the host into the container.

Environment variables from the image, and passed in via --env and --env-file
will override the host environment.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2019-07-11 13:25:47 -04:00
Giuseppe Scrivano 7b75796a60
cgroups: fix a leak when using cgroupfs
be sure to load all the existing handlers, so that they can also be
freed in addition to the handlers we treat differently.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2019-07-11 18:07:49 +02:00
Giuseppe Scrivano 1601100b3e
cgroups: attempt a recursive rmdir
if the cgroup cannot be deleted, then attempt to delete all its
subdirectories and try again.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2019-07-11 18:07:48 +02:00