Found by my find-obsolete-skips script. Let's see which, if any,
of these skipped tests can be reenabled.
Some Skips are "this will never work", not "this is expected to
work one day". Update the message on those to reflect that.
Some were real bugs in the test framework. Fix those.
And, joy of joys, some work today. Remove those skips.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Standardize on no-trunc through the code.
Alias notruncate where necessary.
Standardize on the man page display of no-trunc.
Fixes: https://github.com/containers/podman/issues/8941
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
e2e test failures are rife with messages like:
Expected 1 to equal 0
These make me cry. They're anti-helpful, requiring the reader
to dive into the source code to figure out what those numbers
mean.
Solution: Go tests have a '.Should(Exit(NNN))' mechanism. I
don't know if it spits out a better diagnostic (I have no way
to run e2e tests on my laptop), but I have to fantasize that
it will, and given the state of our flakes I assume that at
least one test will fail and give me the opportunity to see
what the error message looks like.
THIS IS NOT REVIEWABLE CODE. There is no way for a human
to review it. Don't bother. Maybe look at a few random
ones for sanity. If you want to really review, here is
a reproducer of what I did:
cd test/e2e
! positive assertions. The second is the same as the first,
! with the addition of (unnecessary) parentheses because
! some invocations were written that way. The third is BeZero().
perl -pi -e 's/Expect\((\S+)\.ExitCode\(\)\)\.To\(Equal\((\d+)\)\)/Expect($1).Should(Exit($2))/' *_test.go
perl -pi -e 's/Expect\((\S+)\.ExitCode\(\)\)\.To\(\(Equal\((\d+)\)\)\)/Expect($1).Should(Exit($2))/' *_test.go
perl -pi -e 's/Expect\((\S+)\.ExitCode\(\)\)\.To\(BeZero\(\)\)/Expect($1).Should(Exit(0))/' *_test.go
! Same as above, but handles three non-numeric exit codes
! in run_exit_test.go
perl -pi -e 's/Expect\((\S+)\.ExitCode\(\)\)\.To\(Equal\((\S+)\)\)/Expect($1).Should(Exit($2))/' *_test.go
! negative assertions. Difference is the spelling of 'To(Not)',
! 'ToNot', and 'NotTo'. I assume those are all the same.
perl -pi -e 's/Expect\((\S+)\.ExitCode\(\)\)\.To\(Not\(Equal\((0)\)\)\)/Expect($1).To(ExitWithError())/' *_test.go
perl -pi -e 's/Expect\((\S+)\.ExitCode\(\)\)\.ToNot\(Equal\((0)\)\)/Expect($1).To(ExitWithError())/' *_test.go
perl -pi -e 's/Expect\((\S+)\.ExitCode\(\)\)\.NotTo\(Equal\((0)\)\)/Expect($1).To(ExitWithError())/' *_test.go
! negative, old use of BeZero()
perl -pi -e 's/Expect\((\S+)\.ExitCode\(\)\)\.ToNot\(BeZero\(\)\)/Expect($1).Should(ExitWithError())/' *_test.go
Run those on a clean copy of main branch (at the same branch
point as my PR, of course), then diff against a checked-out
copy of my PR. There should be no differences. Then all you
have to review is that my replacements above are sane.
UPDATE: nope, that's not enough, you also need to add gomega/gexec
to the files that don't have it:
perl -pi -e '$_ .= "$1/gexec\"\n" if m!^(.*/onsi/gomega)"!' $(grep -L gomega/gexec $(git log -1 --stat | awk '$1 ~ /test\/e2e\// { print $1}'))
UPDATE 2: hand-edit run_volume_test.go
UPDATE 3: sigh, add WaitWithDefaultTimeout() to a couple of places
UPDATE 4: skip a test due to bug #10935 (race condition)
Signed-off-by: Ed Santiago <santiago@redhat.com>
We missed bumping the go module, so let's do it now :)
* Automated go code with github.com/sirkon/go-imports-rename
* Manually via `vgrep podman/v2` the rest
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Now that Dan has added helpful comments to each SkipIfRemote,
let's take the next step and include those messages in the
Skip() output so someone viewing test results can easily
see if a remote test is skipped for a real reason or for
a FIXME.
This commit is the result of a simple:
perl -pi -e 's;(SkipIfRemote)\(\)(\s+//\s+(.*))?;$1("$3");' *.go
in the test/e2e directory, with a few minor (manual) changes
in wording.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Remove ones that are not needed.
Document those that should be there.
Document those that should be fixed.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
With the advent of Podman 2.0.0 we crossed the magical barrier of go
modules. While we were able to continue importing all packages inside
of the project, the project could not be vendored anymore from the
outside.
Move the go module to new major version and change all imports to
`github.com/containers/libpod/v2`. The renaming of the imports
was done via `gomove` [1].
[1] https://github.com/KSubedi/gomove
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
The Expect function does not return a result of True or False
depending on the value of the first instance, but instead requires
a comparison using ".To(", so change to use ".To(ContainSubstring("
Signed-off-by: gabi beyer <gabrielle.n.beyer@intel.com>
when doing localized tests (not varlink), we can use secondary image
stores as read-only image caches. this cuts down on test time
significantly because each test does not need to restore the images from
a tarball anymore.
Signed-off-by: baude <bbaude@redhat.com>
a series of improvements to our ginkgo test framework so we can
get better ideas of whats going on when run in CI
Signed-off-by: baude <bbaude@redhat.com>
Some callers assume when SystemExec returns, the command has completed.
Other callers explicitly wait for completion (as required). However,
forgetting to do that is an incredibly easy mistake to make. Fix this
by adding an explicit parameter to the function. This requires
every caller to deliberately state whether or not a completion-check
is required.
Also address **many** resource naming / cleanup completion-races.
Signed-off-by: Chris Evich <cevich@redhat.com>
Add the ability to run the integration (ginkgo) suite using
the remote client.
Only the images_test.go file is run right now; all the rest are
isolated with a // +build !remotelinux. As more content is
developed for the remote client, we can unblock the files and
just block single tests as needed.
Signed-off-by: baude <bbaude@redhat.com>
Because our tests are getting so long, we want to be able to audit which tests are taking
the longest to complete. This may indicate a bad test, bad CI, bad code, etc and therefore
should be auditable.
Also, make speed improvements to tests by making sure we only unpack caches images that
actually get used.
Signed-off-by: baude <bbaude@redhat.com>
Closes: #1178
Approved by: mheon
When we run containers in detach mode, nothing cleans up the network stack or
the mount points. This patch will tell conmon to execute the cleanup code when
the container exits.
It can also be called to attempt to cleanup previously running containers.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Closes: #942
Approved by: mheon
Valentin Rothberg
Ed Santiago
Daniel J Walsh
gabi beyer
baude
Chris Evich
Yiqiao Pu