Commit 2 of 2: manual tweaks to get tests passing. Very trivial,
the vast majority of these test files worked with no changes.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Continuing work on RUN-1907: huge set of files, but not
as intimidating as it looks.
Commit 1 of 2: mindless replace of Exit(0) with ExitCleanly()
Signed-off-by: Ed Santiago <santiago@redhat.com>
There is no reason to define the same code every time in each file, just
use global nodes. This diff should speak for itself.
CleanupSecrets()/Volume() no longer call Cleanup() directly, as the
global AfterEach node will always call Cleanup() this is no longer
necessary. If one AfterEach() node fails it will still run the others.
Also always unset the CONTAINERS_CONF env vars. This prevents people
from forgetting to unset it. And fix the special CONTAINERS_CONF logic
in the system connection tests, we do not want to preserve
CONTAINERS_CONF anyway so just remove this logic.
Ginkgo orders the BeforeEach and AfterEach nodes. They will be executed
from the outer-most defined to inner-most. This means our global
BeforeEach is always first. Only then the inner one (in the Describe()
function in each file). For AfterEach it is inverted, from the inner to
the outer.
Also see https://onsi.github.io/ginkgo/#organizing-specs-with-container-nodes
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
- fix a typo that was resulting in a test being a NOP, and
add actual testing to it.
- fix two Expects() with incorrectly-ordered actual/expects
- remove leading whitespace from an It() test name
- To(BeTrue()) is evil. Wherever possible, replace it with
useful string or field checks. When not possible, use
the annotation field to indicate what failed. I got
carried away here, #sorrynotsorry
- remove unused system-test code
Signed-off-by: Ed Santiago <santiago@redhat.com>
...at least as many as possible. "run/exec -it" make no sense
in a CI environment; I believe the vast majority of these are
the result of fingers typing on autopilot, then copy/pasting
cascades from those. This PR gets rid of as many -it/-ti as
possible. Some are still needed for testing purposes.
Y'all have no idea how much I hate #10927 (the "no logs from conmon"
flake). This does not fix the underlying problem, nor does it even
eliminate the flake (The "exec terminal doesn't hang" test needs
to keep the -ti flag, and that's one of the most popular flakers).
But this at least reduces the scope of the problem. It also removes
a ton of nasty orange "input device is not a TTY" warnings from logs.
Signed-off-by: Ed Santiago <santiago@redhat.com>
I found the ginkgolinter[1] by accident, this looks for not optimal
matching and suggest how to do it better.
Overall these fixes seem to be all correct and they will give much
better error messages when something fails.
Check out the repo to see what the linter reports.
[1] https://github.com/nunnatsa/ginkgolinter
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
`--format json` should not be the same as `--format {{json .}}`, the
later should actually run through the go template and thus create one
json object per entry instead of an json array.
Includes a vendor of c/common@main since it requires a fix from there as
well.
This matches docker compat.
Fixes#16436
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
The new cobra update fixed a bug which caused some options to not be
included in --help when there was already a option with the same name
on a parent command.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
In view of https://github.com/containers/storage/pull/1337, do this:
for f in $(git grep -l stringid.GenerateNonCryptoID | grep -v '^vendor/'); do
sed -i 's/stringid.GenerateNonCryptoID/stringid.GenerateRandomID/g' $f;
done
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
The errcheck linter makes sure that errors are always check and not
ignored by accident. It spotted a lot of unchecked errors, mostly in the
tests but also some real problem in the code.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Fix a handful of instances not covered by earlier automated
replacements. Found via:
ack 'Expect\(len' test/e2e
There are still a bunch of BeNumerically(">", ...) that cannot (yet)
be handled by HaveLen(). Leave those as they are.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Write a BeValidJSON() matcher, and replace IsJSONOutputValid():
sed -i -e 's/Expect(\(.*\)\.IsJSONOutputValid()).To(BeTrue())/Expect(\1.OutputToString())\.To(BeValidJSON())/' test/e2e/*_test.go
(Plus a few manual tweaks)
Signed-off-by: Ed Santiago <santiago@redhat.com>
Many ginkgo tests have been written to use this evil form:
GrepString("foo")
Expect(that to BeTrue())
...which yields horrible useless messages on failure:
false is not true
Identify those (automatically, via script) and convert to:
Expect(output to ContainSubstring("foo"))
...which yields:
"this output" does not contain substring "foo"
There are still many BeTrue()s left. This is just a start.
This is commit 1 of 2. It includes the script I used, and
all changes to *.go are those computed by the script.
Commit 2 will apply some manual fixes.
Signed-off-by: Ed Santiago <santiago@redhat.com>
(Sorry, couldn't resist).
CI flakes have been coming down - thank you to everyone who has
been making them a priority.
This leaves a noisy subset that I've just been ignoring for months:
Running: podman ... -p 8080:something
...cannot listen on the TCP port: listen tcp4 :8080: bind: address already in use
Sometimes these are one-time errors resolved on 2nd try; sometimes
they fail three times, forcing CI user to hit Rerun. In all cases
they make noise in my flake logs, which costs me time.
My assumption is that this has to do with ginkgo running random
tests in parallel. Since many e2e tests simplemindedly use 8080,
collisions are inevitable.
Solution: simplemindedly replace 8080 with other (also arbitrarily
picked) numbers. This is imperfect -- it requires human developers
to pick a number NNNN and 'grep NNNN test/e2e/*' before adding
new tests, which I am 100% confident ain't gonna happen -- but
it's better than what we have now.
Side note: I considered writing and using a RandomAvailablePort()
helper, but that would still be racy. Plus, it would be a pain
to interpolate strings into so many places. Finally, with this
hand-tooled approach, if/when we _do_ get conflicts on port NNNN,
it should be very easy to grep for NNNN, find the offending tests
that reuse that port, and fix one of them.
Signed-off-by: Ed Santiago <santiago@redhat.com>
- Added tests to help ensure there is no future regressions
- Added WaitWithTimeout(int) rather than calling
WaitWithDefaultTimeout() multiple times
- Exposed DefaultWaitTimeout to allow test to use a multiplier
Fixes#2221
Signed-off-by: Jhon Honce <jhonce@redhat.com>
e2e test failures are rife with messages like:
Expected 1 to equal 0
These make me cry. They're anti-helpful, requiring the reader
to dive into the source code to figure out what those numbers
mean.
Solution: Go tests have a '.Should(Exit(NNN))' mechanism. I
don't know if it spits out a better diagnostic (I have no way
to run e2e tests on my laptop), but I have to fantasize that
it will, and given the state of our flakes I assume that at
least one test will fail and give me the opportunity to see
what the error message looks like.
THIS IS NOT REVIEWABLE CODE. There is no way for a human
to review it. Don't bother. Maybe look at a few random
ones for sanity. If you want to really review, here is
a reproducer of what I did:
cd test/e2e
! positive assertions. The second is the same as the first,
! with the addition of (unnecessary) parentheses because
! some invocations were written that way. The third is BeZero().
perl -pi -e 's/Expect\((\S+)\.ExitCode\(\)\)\.To\(Equal\((\d+)\)\)/Expect($1).Should(Exit($2))/' *_test.go
perl -pi -e 's/Expect\((\S+)\.ExitCode\(\)\)\.To\(\(Equal\((\d+)\)\)\)/Expect($1).Should(Exit($2))/' *_test.go
perl -pi -e 's/Expect\((\S+)\.ExitCode\(\)\)\.To\(BeZero\(\)\)/Expect($1).Should(Exit(0))/' *_test.go
! Same as above, but handles three non-numeric exit codes
! in run_exit_test.go
perl -pi -e 's/Expect\((\S+)\.ExitCode\(\)\)\.To\(Equal\((\S+)\)\)/Expect($1).Should(Exit($2))/' *_test.go
! negative assertions. Difference is the spelling of 'To(Not)',
! 'ToNot', and 'NotTo'. I assume those are all the same.
perl -pi -e 's/Expect\((\S+)\.ExitCode\(\)\)\.To\(Not\(Equal\((0)\)\)\)/Expect($1).To(ExitWithError())/' *_test.go
perl -pi -e 's/Expect\((\S+)\.ExitCode\(\)\)\.ToNot\(Equal\((0)\)\)/Expect($1).To(ExitWithError())/' *_test.go
perl -pi -e 's/Expect\((\S+)\.ExitCode\(\)\)\.NotTo\(Equal\((0)\)\)/Expect($1).To(ExitWithError())/' *_test.go
! negative, old use of BeZero()
perl -pi -e 's/Expect\((\S+)\.ExitCode\(\)\)\.ToNot\(BeZero\(\)\)/Expect($1).Should(ExitWithError())/' *_test.go
Run those on a clean copy of main branch (at the same branch
point as my PR, of course), then diff against a checked-out
copy of my PR. There should be no differences. Then all you
have to review is that my replacements above are sane.
UPDATE: nope, that's not enough, you also need to add gomega/gexec
to the files that don't have it:
perl -pi -e '$_ .= "$1/gexec\"\n" if m!^(.*/onsi/gomega)"!' $(grep -L gomega/gexec $(git log -1 --stat | awk '$1 ~ /test\/e2e\// { print $1}'))
UPDATE 2: hand-edit run_volume_test.go
UPDATE 3: sigh, add WaitWithDefaultTimeout() to a couple of places
UPDATE 4: skip a test due to bug #10935 (race condition)
Signed-off-by: Ed Santiago <santiago@redhat.com>
Currently pull policy is set incorrectly when users set --pull-never.
Also pull-policy is not being translated correctly when using
podman-remote.
Fixes: #9573
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
We missed bumping the go module, so let's do it now :)
* Automated go code with github.com/sirkon/go-imports-rename
* Manually via `vgrep podman/v2` the rest
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Podman -s crashes when the user specifies the '{{ .Size }}` format
on the podman ps command, without specifying the --size option.
This PR will stop the crash and print out a logrus.Error stating that
the caller should add the --size option.
Fixes: https://github.com/containers/podman/issues/9408
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
when printing out json format, we mistakenly changed the Created field
output to be a time.time in a different commit. This allows for
override of the Created field to be a unix ts as type int64.
Fixes: #9315
Signed-off-by: baude <bbaude@redhat.com>
- When one or more containers in the Pod reports an error on an operation
report StatusConflict and report the error(s)
- jsoniter type encoding used to marshal error as string using error.Error()
- Update test framework to allow setting any flag when creating pods
- Fix test_resize() result check
Fixes#8865
Signed-off-by: Jhon Honce <jhonce@redhat.com>
Fixup the bindings and the handling of the --external --por and --sort
flags.
The --storage option was renamed --external, make sure we use
external up and down the stack.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
`podman ps --format {{.Networks}}` will show all connected networks for
this container. For `pod ps` it will show the infra container networks.
Signed-off-by: Paul Holzinger <paul.holzinger@web.de>
Allow to filter on the network name or full id.
For pod ps it will filter on the infra container networks.
Signed-off-by: Paul Holzinger <paul.holzinger@web.de>
adds the ability to filter containers based on the filter "pod". the
value can be a pod name or its full or partial id.
Fixes: #8512
Signed-off-by: baude <bbaude@redhat.com>
This command exists in docker and is also in our documentation.
Also remove mentions of `podman ls` or `podman list`. These
commands do not exists in podman or docker.
Signed-off-by: Paul Holzinger <paul.holzinger@web.de>
All of our filters worked exclusive resulting in `--filter status=created --filter status=exited` to return nothing.
In docker filters with the same key work inclusive with the only exception being `label` which is exclusive. Filters with different keys always work exclusive.
This PR aims to match the docker behavior with podman.
Signed-off-by: Paul Holzinger <paul.holzinger@web.de>
Followon to #7965 (mirror registry). mirror.gcr.io doesn't
cache all the images we need, and I can't find a way to
add to its cache, so let's just use quay.io for those
images that it can't serve.
Tools used:
skopeo copy --all docker://docker.io/library/alpine:3.10.2 \
docker://quay.io/libpod/alpine:3.10.2
...and also:
docker.io/library/alpine:3.2
docker.io/library/busybox:latest
docker.io/library/busybox:glibc
docker.io/library/busybox:1.30.1
docker.io/library/redis:alpine
docker.io/libpod/alpine-with-bogus-seccomp:label
docker.io/libpod/alpine-with-seccomp:label
docker.io/libpod/alpine_healthcheck:latest
docker.io/libpod/badhealthcheck:latest
Since most of those were new quay.io/libpod images, they required
going in through the quay.io GUI, image, settings, Make Public.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Detached containers and detach keys are only created with the podman run, i
exec, and start commands. We do not store the detach key sequence or the
detach flags in the database, nor does Docker. The current code was ignoreing
these fields but documenting that they can be used.
Fix podman create man page and --help output to no longer indicate that
--detach and --detach-keys works.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
When defining multiple ports (same src/dst) like `-p 80:80 -p 443:443`
then podman will not show the complete output on `podman ps` (only
`0.0.0.0:80->80/tcp` in the example). This also applies to port ranges.
This patch refactors the port loop by pre-checking for ranges and
displaying them correctly to the end user.
Signed-off-by: Sascha Grunert <sgrunert@suse.com>
Now that Dan has added helpful comments to each SkipIfRemote,
let's take the next step and include those messages in the
Skip() output so someone viewing test results can easily
see if a remote test is skipped for a real reason or for
a FIXME.
This commit is the result of a simple:
perl -pi -e 's;(SkipIfRemote)\(\)(\s+//\s+(.*))?;$1("$3");' *.go
in the test/e2e directory, with a few minor (manual) changes
in wording.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Remove ones that are not needed.
Document those that should be there.
Document those that should be fixed.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Ed Santiago
Luca Di Maio
Erik Sjölund
Paul Holzinger
Daniel Lublin
Leonardo Rossetti
Kir Kolyshkin
Brent Baude
Valentin Rothberg
Jhon Honce
flouthoc
Daniel J Walsh
Paul Holzinger
Sascha Grunert
zhangguanzhang