Commit 2 of 2: manual fixes to get tests to pass.
Mostly adding "-q", but in some cases reverting back to Exit(0)
with progress-message checks.
Plus, fix a typo in an error message
Signed-off-by: Ed Santiago <santiago@redhat.com>
Ongoing steps toward RUN-1907: replace Exit(0) with ExitCleanly()
Commit 1 of 2: simple automated string-replace, plus fixes
to includes.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Ongoing steps toward RUN-1907: replace Exit(0) with ExitCleanly()
Several manual fixes for tests that broke:
1. (the usual case): add "-q" to podman-create or -pull; or
2. Revert back to Expect(Exit(0)), and add stderr checks for
progress messages
Signed-off-by: Ed Santiago <santiago@redhat.com>
Ongoing steps toward RUN-1907: replace Exit(0) with ExitCleanly()
One manual fix: adding -q to podman commit, to avoid progress messages
Signed-off-by: Ed Santiago <santiago@redhat.com>
Ongoing steps toward RUN-1907: replace Exit(0) with ExitCleanly()
A handful of test files with trivial command-line replacement,
and no manual muckery (aside from includes).
Signed-off-by: Ed Santiago <santiago@redhat.com>
It is perfectly valid to have only scheme and path (no "authority"),
but unfortunately it doesn't work with external clients like Docker.
Signed-off-by: Anders F Björklund <anders.f.bjorklund@gmail.com>
commit 8b4a79a744 introduced
oom_score_adj clamping when the container oom_score_adj value is lower
than the current one in a rootless environment. Move the check to
init() time so it is performed every time the container starts and not
only when it is created. It is more robust if the oom_score_adj value
is changed for the current user session.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
Ongoing steps toward RUN-1907: replace Exit(0) with ExitCleanly()
Clean command-line replace, but required adding "-q" (quiet)
to all commit commands. Except one, on which I added tests
for the expected progress messages.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Ongoing steps toward RUN-1907: replace Exit(0) with ExitCleanly()
Clean command-line replace, with manual tweaks to two tests:
* clone to a pod: revert to just Exit(0), because podman issues
a namespace warning
* --destroy --force : run "top" in container, not default (shell),
to avoid the 10-second SIGKILL fallback warning
Signed-off-by: Ed Santiago <santiago@redhat.com>
Ongoing steps toward RUN-1907: replace Exit(0) with ExitCleanly()
Clean command-line replace.
Also, fix up the Containerized and Debian exceptions in matcher.
I was in a huge rush Thursday night when I added the Debian
exception. This, I hope, makes it slightly easier to understand
the cases where we don't check stderr.
Signed-off-by: Ed Santiago <santiago@redhat.com>
HC events were firing as part of the `exec` call, before it had
even been decided whether the HC succeeded or failed. As such,
the status was not going to be correct any time there was a
change (e.g. the first event after a container went healthy to
unhealthy would still read healthy). Move the event into the
actual Healthcheck function and throw it in a defer to make sure
it happens at the very end, after logs are written.
Ignores several conditions that did not log previously (container
in question does not have a healthcheck, or an internal failure
that should not really happen).
Still not a perfect solution. This relies on the HC log being
written, when instead we could just get the status straight from
the function writing the event - so if we fail to write the log,
we can still report a bad status. But if the log wasn't written,
we're in bad shape regardless - `podman ps` would disagree with
the event written, for example.
Fixes#19237
Signed-off-by: Matt Heon <mheon@redhat.com>
Shortcuts like unix:path and unix:/path do not work everywhere,
so make sure to use unix://path when quoting the url (or address)
Signed-off-by: Anders F Björklund <anders.f.bjorklund@gmail.com>
Add support to kube play to support the TerminationGracePeriodSeconds
fiels by sending the value of that to podman's stopTimeout.
Add support to kube generate to generate TerminationGracePeriodSeconds
if stopTimeout is set for a container (will ignore podman's default).
Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
If some volumes are specified in containers.conf, they are currently
added twice to the containers spec causing the container to fail:
$ head -n2 ~/.config/containers/containers.conf
[containers]
volumes = ["/tmp:/tmp"]
$ podman pod create --name foo
7ac7f97f9b74a596332483e4a13e58cb9c8d997e9c5baae46804ae0acc26cbc6
$ podman run --pod=foo alpine true
Error: "/tmp": duplicate mount destination
The fix is to ignore the setting from containers.conf when setting the
pod default configuration.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
This test checks that the pod cgroups are created and that the limits
set for a pod cgroup are enforced also after a reboot.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
Ongoing steps toward RUN-1907: replace Exit(0) with ExitCleanly()
Clean command-line replace, with one manual reversion (commented)
And -- duh! -- skip the stderr check on Debian!
Signed-off-by: Ed Santiago <santiago@redhat.com>
When the "rmi" part of "run --rmi" fails due to image being in use
by another container (or for any reason, actually), issue a warning
message, not an error.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Under some circumstances podman tries to kill a container
using signal 37, for which unix.SignalName() returns "".
Not helpful. So, when that happens, show "(signal number)".
Signed-off-by: Ed Santiago <santiago@redhat.com>
PR #19878 (checking for warnings in system tests) broke upgrade tests.
Reason: my long-ago "optimization" in which, if a PR touches only
tests in X, do not run tests in Y. Unfortunately, upgrade tests
rely on code in the system-test directory. I don't know if this
is fixable; nor if it's an acceptable tradeoff. Please discuss.
Sorry, everyone.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Also add a new `StoppedByUser` field to the container-inspect state
which can be useful during debugging and is now also used in the
regression test. Note that I moved the `false` check one test above
such that we can compare the previous Podman version which should just
be stuck in the `wait $ctr` command since it will continue restarting.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
The logic here makes little sense, basically the /tmp and /var/tmp are
always set noexec, while /run is not. I don't see a reason to set any
of the three noexec by default.
Fixes: https://github.com/containers/podman/issues/19886
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
With few exceptions, commands that exit 0 should not emit any
messages with level=warning or =error. Let's start enforcing
that in run_podman.
Allow one-off exceptions, typically when we're testing an
actual warning condition (usual case: "podman stop" where it
times out to SIGKILL). Exceptions are specified via:
run_podman 0+w subcommand...
^^^---- or, rarely, 0+e
"0" stands for "expect exit status 0", which is the default
so it's implicit anyway. The +w / +e (or even +we) is the
new part. I have added it to tests where necessary.
And, because life is what it is, add two global exceptions:
- Debian. Because runc has too many flakes.
- kube. Ditto. Kube commands emit lots of nasty error
messages (yes, level=error) that don't seem to affect
results.
Similar to #18442
Signed-off-by: Ed Santiago <santiago@redhat.com>
Small steps toward RUN-1907: replace Exit(0) with ExitCleanly()
in ginkgo tests in two test files. Also, when practical,
replace ALPINE with CITEST_IMAGE.
There are still many thousands of instances left to fix. I will
be submitting in reviewable chunks.
Signed-off-by: Ed Santiago <santiago@redhat.com>
allow the image to specify an empty list of capabilities, currently
podman chokes when the io.containers.capabilities specified in an
image does not contain at least one capability.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
Fixes infrequent but annoying flake in which system tests
call random_free_port(), get a nice-looking port, then
fail with "bind: address already in use".
Signed-off-by: Ed Santiago <santiago@redhat.com>
Primarily, build test images FROM scratch, not alpine, to
avoid image pulls and network flakes and potential pull-
the-rug-out errors if the base alpine image changes.
This was much more complicated than it should've been,
because creating unique arch-specific FROM-scratch images
triggered a weird manifest bug, filed as #19860.
Also:
- add a teardown() to clean up manifests
- remove test for skopeo (skopeo is required for sys tests)
- remove unnecessary intermediate tmpdir
- deduplicate, by looping over amd+arm
- fix indentation
- and, finally, clean up dangling images (this was the initial
reason behind my diving in here. Such a simple thing, I thought.)
Signed-off-by: Ed Santiago <santiago@redhat.com>
when running rootless, if the specified oom_score_adj for the
container process is lower than the current value, clamp it to the
current value and print a warning.
Closes: https://github.com/containers/podman/issues/19829
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
Fix unquoted string vars. Something like this:
is $output "what we expect"
...will fail with a misleading error message if $output is "".
Also fix typos in a diagnostic; this was causing unhelpful message
on failure
Signed-off-by: Ed Santiago <santiago@redhat.com>
A nearly-trivial first effort to use the new ExitCleanly().
Requires using the new CITEST_IMAGE (see prior commit)
because nginx causes the tests to fail:
[FAILED] Unexpected warnings seen on stderr: \
level=warning \
msg="HEALTHCHECK is not supported for OCI image format ...
Oh, I also took the liberty of rewriting "play kube" -> "kube play".
Signed-off-by: Ed Santiago <santiago@redhat.com>
Combined test for (exitcode == 0) && (nothing on stderr).
Returns more useful diagnostic messages than the default:
old: Expected N to equal 0
new: Command failed with exit status N
new: Unexpected warnings seen on stderr: "...."
Signed-off-by: Ed Santiago <santiago@redhat.com>
Finally, after so many years, let's start using testimage:YYYYMMDD.
Use it in place of LABELS_IMAGE, which nothing/nowhere was using.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Use the `newer` pull policy only for the "latest" tag and default to
using `missing` otherwise. This speeds up `kube play` as it'll skip
reaching out to the registry and also fixes other side-effects described
in #19801.
Fixes: #19801
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
When pulling from an OCI source, make sure to preseve the optional name.
For instance, a podman pull oci:/tmp/foo:quay.io/foo/bar:latest should
pull the image and name it quay.io/foo/bar:latest.
While at it, also fix a bug when pulling an OCI without the optional
name. Previously, we used the path to name the image which will error in
most cases due to invalid characters (e.g., capital ones). Hence, apply
the same trick as for the dir transport and generate a sha.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Use `add_compression` field from `containers.conf` if found instead and
`CLI` field `--add-compression` is not set.
Signed-off-by: Aditya R <arajan@redhat.com>
... by updating for a c/common API change.
[NO NEW TESTS NEEDED]: Only moves unchanged code,
should not change behavior.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Motivation
===========
This feature aims to make --uidmap and --gidmap easier to use, especially in rootless podman setups.
(I will focus here on the --gidmap option, although the same applies for --uidmap.)
In rootless podman, the user namespace mapping happens in two steps, through an intermediate mapping.
See https://docs.podman.io/en/latest/markdown/podman-run.1.html#uidmap-container-uid-from-uid-amount
for further detail, here is a summary:
First the user GID is mapped to 0 (root), and all subordinate GIDs (defined at /etc/subgid, and
usually >100000) are mapped starting at 1.
One way to customize the mapping is through the `--gidmap` option, that maps that intermediate mapping
to the final mapping that will be seen by the container.
As an example, let's say we have as main GID the group 1000, and we also belong to the additional GID 2000,
that we want to make accessible inside the container.
We first ask the sysadmin to subordinate the group to us, by adding "$user:2000:1" to /etc/subgid.
Then we need to use --gidmap to specify that we want to map GID 2000 into some GID inside the container.
And here is the first trouble:
Since the --gidmap option operates on the intermediate mapping, we first need to figure out where has
podman placed our GID 2000 in that intermediate mapping using:
podman unshare cat /proc/self/gid_map
Then, we may see that GID 2000 was mapped to intermediate GID 5. So our --gidmap option should include:
--gidmap 20000:5:1
This intermediate mapping may change in the future if further groups are subordinated to us (or we stop
having its subordination), so we are forced to verify the mapping with
`podman unshare cat /proc/self/gid_map` every time, and parse it if we want to script it.
**The first usability improvement** we agreed on #18333 is to be able to use:
--gidmap 20000:@2000:1
so podman does this lookup in the parent user namespace for us.
But this is only part of the problem. We must specify a **full** gidmap and not only what we want:
--gidmap 0:0:5 --gidmap 5:6:15000 --gidmap 20000:5:1
This is becoming complicated. We had to break the gidmap at 5, because the intermediate 5 had to
be mapped to another value (20000), and then we had to keep mapping all other subordinate ids... up to
close to the maximum number of subordinate ids that we have (or some reasonable value). This is hard
to explain to someone who does not understand how the mappings work internally.
To simplify this, **the second usability improvement** is to be able to use:
--gidmap "+20000:@2000:1"
where the plus flag (`+`) states that the given mapping should extend any previous/default mapping,
overriding any previous conflicting assignment.
Podman will set that mapping and fill the rest of mapped gids with all other subordinated gids, leading
to the same (or an equivalent) full gidmap that we were specifying before.
One final usability improvement related to this is the following:
By default, when podman gets a --gidmap argument but not a --uidmap argument, it copies the mapping.
This is convenient in many scenarios, since usually subordinated uids and gids are assigned in chunks
simultaneously, and the subordinated IDs in /etc/subuid and /etc/subgid for a given user match.
For scenarios with additional subordinated GIDs, this map copying is annoying, since it forces the user
to provide a --uidmap, to prevent the copy from being made. This means, that when the user wants:
--gidmap 0:0:5 --gidmap 5:6:15000 --gidmap 20000:5:1
The user has to include a uidmap as well:
--gidmap 0:0:5 --gidmap 5:6:15000 --gidmap 20000:5:1 --uidmap 0:0:65000
making everything even harder to understand without proper context.
For this reason, besides the "+" flag, we introduce the "u" and "g" flags. Those flags applied to a
mapping tell podman that the mapping should only apply to users or groups, and ignored otherwise.
Therefore we can use:
--gidmap "+g20000:@2000:1"
So the mapping only applies to groups and is ignored for uidmaps. If no "u" nor "g" flag is assigned
podman assumes the mapping applies to both users and groups as before, so we preserve backwards compatibility.
Co-authored-by: Tom Sweeney <tsweeney@redhat.com>
Signed-off-by: Sergio Oller <sergioller@gmail.com>
The docker client expects to read the OSType header from the `/_ping` response in order to determine the OS type of the server, for example, when running `docker run --device=/dev/fuse ...`
https://github.com/moby/moby/blob/master/client/ping.go#L57
Signed-off-by: chnrxn <cohawk@yahoo.com>
The usual bug that we always seem to forget about: "kube play"
needs "podman wait" before we can "podman logs". (And, reminder,
"kube play --wait" is worthless because it destroys containers).
Reference: #18074, the original PR that fixed a bunch of these flakes.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Make sure that `kube down` and `kube play --replace` do not error out
when an object does not exist (or has already been removed). Such kind
of teardown should not be treated as an ordinary `rm` but as an
`rm --ignore`. It's purpose it to make sure that all objects in a YAML
are removed; even if they existed only partially.
Fixes: #19711
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Value of `--force-compression` should be already `true` is
`--compression-format` is selected otherwise let users decide.
Signed-off-by: Aditya R <arajan@redhat.com>
Unexplained infrequent flakes in sdnotify system tests,
waiting for READY=1.
Hypothesis: race condition between the container sending
the READY string and that string making it through conmon
and socat into the log file.
Solution: don't just check once; keep trying in a loop.
Write a reusable wait_for_file_content() helper function,
and clean up a bunch more tests as long as we're at it.
Fixes: #19724
Signed-off-by: Ed Santiago <santiago@redhat.com>
Fixes: https://github.com/containers/podman/issues/19139
Service containers are defaulting to 0 seconds for Timeout rather then
the settings in containers.conf.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Currently containers created via DOCKER API without specifying
StopTimeout are defaulting to 0 seconds. This change should
default them to setting in containers.conf normally 10 seconds.
Fixes: https://github.com/containers/podman/issues/19139
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Add io.podman.annotations.infra.name annotation to kube play so
users can set the name of the infra container created.
When a pod is created with --infra-name set, the generated
kube yaml will have an infraName annotation set that will
be used when playing the generated yaml with podman.
Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
Do not close a notifyproxy more than once. Also polish the backend a
bit to reflect ealier changes from commit 4fa307f.
Fixes: #19715
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Unset the NOTIFY_SOCKET environment variable after sending the MAIN_PID
and READY message. This avoids any unintentional side-effects of other
code paths using the socket assuming they'd run in a non-server
short-lived Podman process.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
The attach API used to always return the Content-Type
`vnd.docker.raw-stream`, however docker api v1.42 added the
`vnd.docker.multiplexed-stream` type when no tty was used.
Follow suit and return the same header for docker api v1.42 and libpod
v4.7.0. This technically allows clients to make a small optimization as
they no longer need to inspect the container to see if they get a raw or
multiplexed stream.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
`exist.io` actually does exist and is not under our control. To prevent
flakes, change it to something on `podman.io`.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Adds support for --force-compression which allows end-users to force
push blobs with the selected compresison in --compression option, in
order to make sure that blobs of other compression on registry are not
reused.
Signed-off-by: Aditya R <arajan@redhat.com>
Adds support for --force-compression which allows end-users to force
push blobs with the selected compresison in --compression option, in
order to make sure that blobs of other compression on registry are not
reused.
Is equivalent to: force-compression here: https://docs.docker.com/build/exporters/#compression
Closes: https://github.com/containers/podman/issues/18660
Signed-off-by: Aditya R <arajan@redhat.com>
Kubernetes supports expanding $(FOOBAR) as environment variables within
the kube.YAML. When using podman kube play, we need to do the same, for
supporting these YAML files.
Fixes: #15983
Signed-off-by: Chee Hau Lim <ch33hau@gmail.com>
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Fixes a bug where `podman kube play` fails to set a container's Umask
to the default 0022, and sets it to 0000 instead.
Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
Users want to mount a tmpfs file system with secrets, and make
sure the secret is never saved into swap. They can do this either
by using a ramfs tmpfs mount or by passing `noswap` option to
a tmpfs mount.
Fixes: https://github.com/containers/podman/issues/19659
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
We need to actually check the output not just exit codes. While doing
this it was clear that the first test was not checking what it should
be so I had to remove the quotes from the arg.
Also this check did not work with remote testing at all, we must set the
env then restart the server as the env for conmon must be set on the
server obviously.
Also we can only match the conmon error messages on the local client.
Lastly this test requires the journald driver but we cannot use the in
container tests so skip it there.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
IPv6 test has been hard-skipped for six months.
IPv4 test is flaking in Cirrus and hard-failing in Gating.
Absent a reliable way to test in CI and gating, and absent
a strong reason to test ICMP in pasta anyway, the solution
is simple.
Closes: #19612
Signed-off-by: Ed Santiago <santiago@redhat.com>
Support a new concept in containers.conf called "modules". A "module"
is a containers.conf file located at a specific directory. More than
one module can be loaded in the specified order, following existing
override semantics.
There are three directories to load modules from:
- $CONFIG_HOME/containers/containers.conf.modules
- /etc/containers/containers.conf.modules
- /usr/share/containers/containers.conf.modules
With CONFIG_HOME pointing to $HOME/.config or, if set, $XDG_CONFIG_HOME.
Absolute paths will be loaded as is, relative paths will be resolved
relative to the three directories above allowing for admin configs
(/etc/) to override system configs (/usr/share/) and user configs
($CONFIG_HOME) to override admin configs.
Pulls in containers/common/pull/1599.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Flake suppression: going with the one-basket model of egg storage,
switch manifest_test to use an image on quay.io (was: k8s.io).
Closes: #19148
Signed-off-by: Ed Santiago <santiago@redhat.com>
This reverts commit 5b148a0a68.
Reverting to treating the `pull` query parameter as a boolean.
Because of deceiving Docker API documentation it was assumed that the
parameter is pull-policy, however that is not true. Docker does treat
`pull` as a boolean. What is interesting is that Docker indeed accepts
strings like `always` or `never` however Docekr both of these strings
treat as `true`, not as pull-policy. As matter of the fact it seems
there is no such a thing as pull-policy in Docker.
More context https://github.com/containers/podman/issues/17778#issuecomment-1673931925
Signed-off-by: Matej Vasek <mvasek@redhat.com>
...possibly because we somehow ended up with a two-line
log file for a simple 'echo hi'? Make our timestamp-getting
code safer by adding 'head -1'.
Signed-off-by: Ed Santiago <santiago@redhat.com>
We've made rather a mess of those options, due to lack of testing.
Here we have a first step toward regression tests. --env is OK,
but there are three special-case exceptions in --env-file for
three incompatibilities introduced by #19096.
To be continued, but probably in future PRs. We need this ASAP
to prevent us from making any more regressions.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Add new --farm flag to podman system connection add so that
a user can add a new connection to a farm immediately.
Update system connection remove such that when a connection is
removed, the connection is also removed from any farms that have it.
Add docs and tests for these changes.
Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
...to reduce flakes.
Reason: journald makes no guarantees. Just because a systemd job
has finished, or podman has written+flushed log entries, doesn't
mean that journald will actually know about them:
https://github.com/systemd/systemd/issues/28650
Workaround: wrap some podman-logs tests inside Eventually()
so they will be retried when log == journald
This addresses, but does not close, #18501. That's a firehose,
with many more failures than I can possibly cross-reference.
I will leave it open, then keep monitoring missing-logs flakes
over time, and pick those off as they occur.
Signed-off-by: Ed Santiago <santiago@redhat.com>
There is a problem where our tail code does not handles correctly
partial log lines. This makes podman logs --tail output possibly
incorrect lines when k8s-file is used.
This manifests as flake in CI because partial lines are only sometimes
written, basically always when the output is flushed before writing a
newline.
For our code we must not count partial lines which was already done but
the important thing we must keep reading backwards until the next full
(F) line. This is because all partial (P) lines still must be added to
the full line. See the added tests for details on how the log file looks
like.
While fixing this, I rework the tail logic a bit, there is absolutely no
reason to read the lines in a separate goroutine just to pass the lines
back via channel. We can do this in the same routine.
The logic is very simple, read the lines backwards, append lines to
result and then at the end invert the result slice as tail must return
the lines in the correct order. This more efficient then having to
allocate two different slices or to prepend the line as this would
require a new allocation for each line.
Lastly the readFromLogFile() function wrote the lines back to the log
line channel in the same routine as the log lines we read, this was bad
and causes a deadlock when the returned lines are bigger than the
channel size. There is no reason to allocate a big channel size we can
just write the log lines in a different goroutine, in this case the main
routine were read the logs anyway.
A new system test and unit tests have been added to check corner cases.
Fixes#19545
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Podman should ignore failures to find a cidfile when stoping the
container if the user specified --ignore
Fixes: https://github.com/containers/podman/issues/19546
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
An unhelpful comment doesn't give any clues why this test was originally
skipped on Ubuntu. In any case, now that CI uses Debian SID, re-enable
the test hoping that it now functions.
Signed-off-by: Chris Evich <cevich@redhat.com>
We do not allow volumes and mounts to be placed at the same
location in the container, with create-time checks to ensure this
does not happen. User-added conflicts cannot be resolved (if the
user adds two separate mounts to, say, /myapp, we can't resolve
that contradiction and error), but for many other volume sources,
we can solve the contradiction ourselves via a priority
hierarchy. Image volumes come first, and are overridden by the
`--volumes-from` flag, which are overridden by user-added mounts,
etc, etc. The problem here is that we were not properly handling
volumes-from overriding image volumes. An inherited volume from
--volumes-from would supercede an image volume, but an inherited
mount would not. Solution is fortunately simple - just clear out
the map entry for the other type when adding volumes-from
volumes.
Makes me wish for Rust sum types - conflict resolution would be a
lot simpler if we could use a sum type for volumes and bind
mounts and thus have a single map instead of two maps, one for
each type.
Fixes#19529
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
The `Exists` field of the `RemoteSocket` struct is marshaled to json with the
`omitempty` setting. This has the disadvantage that by default `podman info`
shows a `remotePath` entry (the remote path is set in
`pkg/domain/infra/abi/systems.go`: `(*ContainerEngine).Info`) but not that this
path does not exist:
```
❯ podman info --format json | jq .host.remoteSocket
{
"path": "/run/user/1000/podman/podman.sock"
}
```
By removing the `omitempty`, we ensure that the existence is always shown:
```
❯ bin/podman info --format json | jq .host.remoteSocket
{
"path": "/run/user/1000/podman/podman.sock",
"exists": false
}
```
Signed-off-by: Dan Čermák <dcermak@suse.com>
Compat api for containers/stop should take -1 value
Add support for `podman stop --time -1`
Add support for `podman restart --time -1`
Add support for `podman rm --time -1`
Add support for `podman pod stop --time -1`
Add support for `podman pod rm --time -1`
Add support for `podman volume rm --time -1`
Add support for `podman network rm --time -1`
Fixes: https://github.com/containers/podman/issues/17542
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Remove "HTTP if one registry" test. It is a NOP, has been skipped
for two months, and nobody knows what its original purpose was.
Closes: #18768
Signed-off-by: Ed Santiago <santiago@redhat.com>
First, all the defaults for TERM=xterm were removed from c/common, then accordingly the same will be added if encountered a set tty flag.
Signed-off-by: Chetan Giradkar <cgiradka@redhat.com>
Adds support for --add-compression which accepts multiple compression
formats and when used it will add all instances in a manifest list with
requested compression formats.
Signed-off-by: Aditya R <arajan@redhat.com>
Some people might expect this to work:
systemctl --wait start foo
journalctl -u foo ---> displays output from foo
Well, it does not. Not reliably, anyway:
https://github.com/systemd/systemd/issues/28650
Shrug, okay, deal with it: write value of %T to a tmpfile
instead of relying on journal. I tested with TMPDIR=<many values>
on an SELinux system and, by golly, it works fine.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Forcing users to set --rm when setting --rmi is just bad UI.
If I want the image to be removed, it implies that I want the
container removed that I am creating.
Fixes: https://github.com/containers/podman/issues/15640
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
The original SELinux support in Docker and Podman does not follow the
default SELinux rules for how label transitions are supposed to be
handled. Containers always switch their user and role to
system_u:system_r, rather then maintain the collers user and role.
For example
unconfined_u:unconfined_r:container_t:s0:c1,c2
Advanced SELinux administrators want to confine users but still allow
them to create containers from their role, but not allow them to launch
a privileged container like spc_t.
This means if a user running as
container_user_u:container_user_r:container_user_t:s0
Ran a container they would get
container_user_u:container_user_r:container_t:s0:c1,c2
If they run a privileged container they would run it with:
container_user_u:container_user_r:container_user_t:s0
If they want to force the label they would get an error
podman run --security-opt label=type:spc_t ...
Should fail. Because the container_user_r can not run with the spc_t.
SELinux rules would also prevent the user from forcing system_u user and
the sytem_r role.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Signed-off-by: Chris Evich <cevich@redhat.com>
We do not use any special netns path for the netns=none case, however
callers that inspect that may still wish to join the netns path directly
without extra work to figure out /proc/$pid/ns/net.
Fixes#16716
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
And lo, a miracle occurred. Containerized checkpoint tests are
no longer hanging. Reenable them.
(Followup miracle: tests are still passing, after a year of not
running!)
Closes: #15015
Signed-off-by: Ed Santiago <santiago@redhat.com>
The intention of --read-only-tmpfs=fals when in --read-only mode was to
not allow any processes inside of the container to write content
anywhere, unless the caller also specified a volume or a tmpfs. Having
/dev and /dev/shm writable breaks this assumption.
Fixes: https://github.com/containers/podman/issues/12937
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
In certain cases REGISTRY_AUTH_FILE is set but the auth file
does not exists yet, do not throw error unless user specified
a file directly using --authfile.
Fixes: https://github.com/containers/podman/issues/18405
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
HPC Community asked for this support specifically for using GPUs
within containers. Nvidia requires the correct shared library to
to be present in the directory that matches the device mounted
into the container. These libraries have random suffixes based
on versions of the installed libraries on the host.
podman run --mount type=glob:src=/usr/lib64/nvidia\*:ro=true. This helps
quadlets be more portable for this use case.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Use "--wait" flag in "systemd start" for a one-shot container.
Should fix a CI failure I've been seeing sporadically, in which
the --==VALUE==-- string is not seen in journal.
Signed-off-by: Ed Santiago <santiago@redhat.com>
If a user puts a quadlet file in his homedirectory with
the same name as one in /etc/containers/systemd/user or
/etc/containers/systemd/user/$UID, then only use the one in
homedir and ignore the others.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
BATS 1.8.0 introduces tags: metadata that can be applied to
a single test or one entire file, then used for filtering
in a test run.
Issue #19299 introduces the possibility of using OpenQA
for podman reverse dependency testing: continuous CI on
all packages that can affect podman, so we don't go two
months with no bodhi builds then get caught by surprise
when systemd or kernel or crun change in ways that break us.
This PR introduces one bats tag, "distro-integration".
The intention is for OpenQA (or other) tests to install
the podman-tests package and run:
bats --filter-tags distro-integration /usr/share/podman/test/system
Goal is to keep the test list short and sweet: we do not
need to test command-line option parsing. We *DO* need to
test interactions with systemd, kernel, nethack, and other
critical components.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Quick followup to #19348:
- refactor into table form, for legibility
- add tests for 'podman kube play' and 'podman run'
- slightly cleaner message on failure
Signed-off-by: Ed Santiago <santiago@redhat.com>
Add a new "healthy" sdnotify policy that instructs Podman to send the
READY message once the container has turned healthy.
Fixes: #6160
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
**podman compose** is a thin wrapper around an external compose provider
such as docker-compose or podman-compose. This means that `podman
compose` is executing another tool that implements the compose
functionality but sets up the environment in a way to let the compose
provider communicate transparently with the local Podman socket. The
specified options as well the command and argument are passed directly
to the compose provider.
The default compose providers are `docker-compose` and `podman-compose`.
If installed, `docker-compose` takes precedence since it is the original
implementation of the Compose specification and is widely used on the
supported platforms (i.e., Linux, Mac OS, Windows).
If you want to change the default behavior or have a custom installation
path for your provider of choice, please change the `compose_provider`
field in `containers.conf(5)`. You may also set the
`PODMAN_COMPOSE_PROVIDER` environment variable.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
I am working on running android auto in a quadlet.
[Container]
AddDevice=/dev/dri/renderD128
AddDevice=/dev/kvm
DropCapability=all
Environment=PULSE_SERVER=$XDG_RUNTIME_DIR/pulse/native
Environment=WAYLAND_DISPLAY=wayland-0
Environment=XDG_RUNTIME_DIR
Image=quay.io/slopezpa/qemu-aaos
ContainerName=Android
PodmanArgs=--shm-size=5g
SecurityLabelDisable=true
Volume=$XDG_RUNTIME_DIR:$XDG_RUNTIME_DIR
And I need to be able to set the --shm-size option.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
`podman system service` + TCP is not a configuration we should be
recommending. There was already language about this in the
manpages, but it was not sufficient in explaining how bad of an
idea this is. Expand the manpage warnings, add a dedicated
heading so people notice, and add a warning every time the
service starts with a TCP URL that directs people to the manpage
to see that explanation.
Signed-off-by: Matt Heon <mheon@redhat.com>
Some quadlet tests are failing on RHEL8: test code was
using journalctl to check output from containers. This
fails on RHEL8, where default log driver is k8s-file.
Solution: use 'podman logs' instead. To do so, we need to
keep the containers alive (otherwise, quadlet seems to
delete them on exit). Do so by running 'top -b' (batch);
the currently-used 'top' was failing because not-a-tty.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Add key for Quadlet to set WorkingDirectory to the directory of the YAML or Unit file
Add Doc
Add E2E tests
Add System test
Signed-off-by: Ygal Blum <ygal.blum@gmail.com>
quadlet volume-path system test was making invalid assumptions
about $TMPDIR, causing test to fail when TMPDIR=/var/tmp or /dev/shm
Much more complicated than it should be, because we need to
find out the systemd value of %T.
Minor cleanup too.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Adds support for `since` as a valid filter option for `podman volume ls`
and `podman volume prune`.
Implements: #19228
Initially suggested from: #19119
Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
Fixes a bug where `podman volume ls` with multiple `label` filters would
return volumes that matched *any* of the filters, not *all* of them.
Adapts generating volume filter functions to be more in
line with how it is done for containers and pods.
Fixes: #19219
Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
Adds any required "wiring" to ensure the reserved annotations are supported by
`podman kube play`.
Addtionally fixes a bug where, when inspected, containers created using
the `--publish-all` flag had a field `.HostConfig.PublishAllPorts` whose
value was only evaluated as `false`.
Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
- the "podman {run,exec} /etc" test: runc now spits out
"is a directory" instead of "permission denied". And,
on exec, exits 255 instead of 126. Deal with it.
- workaround for https://github.com/containers/skopeo/issues/823
(skopeo XDG bug): always make sure XDG is defined for skopeo
Signed-off-by: Ed Santiago <santiago@redhat.com>
Adds an `--podman-only` flag to `podman generate kube` to allow for
reserved annotations to be included in the generated YAML file.
Associated with: #19102
Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
To silence my find-obsolete-skips script, remove the '#'
from the following issues in skip messages:
#11784#15013#15025#17433#17436#17456
Also update the messages to reflect the fact that the issues
will never be fixed.
Also remove ubuntu skips: we no longer test ubuntu.
Also remove one buildah skip that is no longer applicable:
Fixes: #17520
Signed-off-by: Ed Santiago <santiago@redhat.com>
When I reworked pod removal to provide more detailed errors
(including per-container errors, not just a single multierror
with all errors squashed), I made it part of the struct returned
by the REST API and assumed that would be enough to get errors
through to clients. Unfortunately, in case of an overarching
error removing the pod (as any error with any container would
cause), we don't send the response struct that would include the
container errors - we just send a standardized REST error. We
could work around this with custom, potentially backwards
incompatible error handling for the REST pod delete endpoint, or
we could just do what was done before, and package up all the
errors in a multierror to send to the other side. Of those
options, the multierror seems far simpler.
Fixes#19159
Signed-off-by: Matt Heon <mheon@redhat.com>
The change to use the custom dns server in aardvark-dns caused a
regression here because macvlan networks never returned the nameservers
in netavark and it also does not make sense to do so.
Instead check here if we got any network nameservers, if not we then use
the ones from the config if set otherwise fallback to host servers.
Fixes#19169
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
We use the name as alias but using the hostname makes also sense and
this is what docker does. We have to keep the short id as well for
docker compat.
While adding some tests I removed some duplicated tests that were
executed twice for nv for no reason.
Fixes#17370
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
When fetching the user name, you need to use User.Username
instead of User.Name, as with other tests.
Signed-off-by: Toshiki Sonoda <sonoda.toshiki@fujitsu.com>
Adds a `--no-trunc` flag to `podman kube generate` preventing the
annotations from being trimmed at 63 characters. However, due to
the fact the annotations will not be trimmed, any annotation that is
longer than 63 characters means this YAML will no longer be Kubernetes
compatible. However, these YAML files can still be used with `podman
kube play` due to the addition of the new flag below.
Adds a `--no-trunc` flag to `podman kube play` supporting YAML files with
annotations that were not truncated to the Kubernetes maximum length of
63 characters.
Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
This commit extends `Volume` and `Network` unit definitions with two
additional parameters, `VolumeName` and `NetworkName`, which will,
respectively, set a user-defined name for the corresponding volume and
network. This is similar to how the `ContainerName` directive currently
works, and should allow for smoother transitions to Quadlet-managed
resources.
Closes: #19003
Signed-off-by: Alex Palaistras <alex@deuill.org>
...from the test name. Eliminates scary duplication.
Followup to #19053: instead of cross-checking pasta test args
against test name, eliminate the args entirely. Determine
them all from the @test name itself.
Example:
"TCP translated port range forwarding, IPv4, loopback"
| | | | | | +-- iftype=loopback
| | | | | +-------- ip_ver=4
| | | | +-------------------- bytes=1
| | | +-------------------------- range=3
| | +------------------------------- (ignored)
| +------------------------------------------ delta=1
+--------------------------------------------- proto=tcp
Signed-off-by: Ed Santiago <santiago@redhat.com>
Do not use podman info/version as they are expensive and clutter the log
for no reason. Just checking if we can connect to the socket should be
good enough and much faster.
Fix the non existing error checking, so that we actually see an useful
error when this does not work.
Also change the interval, why wait 2s for a retry lets take 100ms steps
instead.
Fixes#19010
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Previous tests have worked by pure chance since the client and server
ran on the same host; the server picked up the credentials created by
the client login.
Extend the gating tests and add a new integration test which is further
capable of exercising the remote code.
Note that fixing authentication support requires adding a new
`--authfile` CLi flag to `manifest inspect`. This will at least allow
for passing an authfile to be bindings. Username and password are not
yet supported.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
This ended up more complicated then expected. Lets start first with the
problem to show why I am doing this:
Currently we simply execute ps(1) in the container. This has some
drawbacks. First, obviously you need to have ps(1) in the container
image. That is no always the case especially in small images. Second,
even if you do it will often be only busybox's ps which supports far
less options.
Now we also have psgo which is used by default but that only supports a
small subset of ps(1) options. Implementing all options there is way to
much work.
Docker on the other hand executes ps(1) directly on the host and tries
to filter pids with `-q` an option which is not supported by busybox's
ps and conflicts with other ps(1) arguments. That means they fall back
to full ps(1) on the host and then filter based on the pid in the
output. This is kinda ugly and fails short because users can modify the
ps output and it may not even include the pid in the output which causes
an error.
So every solution has a different drawback, but what if we can combine
them somehow?! This commit tries exactly that.
We use ps(1) from the host and execute that in the container's pid
namespace.
There are some security concerns that must be addressed:
- mount the executable paths for ps and podman itself readonly to
prevent the container from overwriting it via /proc/self/exe.
- set NO_NEW_PRIVS, SET_DUMPABLE and PDEATHSIG
- close all non std fds to prevent leaking files in that the caller had
open
- unset all environment variables to not leak any into the contianer
Technically this could be a breaking change if somebody does not
have ps on the host and only in the container but I find that very
unlikely, we still have the exec in container fallback.
Because this can be insecure when the contianer has CAP_SYS_PTRACE we
still only use the podman exec version in that case.
This updates the docs accordingly, note that podman pod top never falls
back to executing ps in the container as this makes no sense with
multiple containers so I fixed the docs there as well.
Fixes#19001
Fixes https://bugzilla.redhat.com/show_bug.cgi?id=2215572
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Users may want to replace the secret used within containers, without
destroying the secret and recreating it.
Partial fix for https://github.com/containers/podman/issues/18667
Make sure podman --remote secret inspect and podman secret inspect
return the same error message.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
1. toolbox UID/GID allocation: pick numbers < 1500. Otherwise
we run the risk of colliding with the Cirrus rootless user.
2. WaitContainerReady(): check the results of the last "podman logs"
before timing out. Otherwise, the user will see "READY" followed
immediately by "Container is not ready".
(global bug, not just toolbox, but that's where I discovered it).
Signed-off-by: Ed Santiago <santiago@redhat.com>
Using GinkgoT().TempDir() will automatically result in the directy to be
cleaned up when the test is done. This should help to prevent leaking
files and we do not need to error check every time.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Like LockTmpDir use a random tmpdir for this directory. Make sure it is
set for all parallel ginkgo processes.
Also GinkgoT().TempDir() will automatcially remove the directory at the
end so we do not need to worry about cleanup.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
AFAIK the latest podman will not even run on RHEL 7 anymore, in any case
we do not need these tests to run there.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Using the OS tempdir here is not good. This defaults to /tmp which means
the inital podman test setup uses these paths:
`--root /tmp/root --runroot /tmp/runroot and --tmpdir /tmp`
Thus we create many files directly under /tmp. Also they were never
removed thus leaked out. When running as root and then later as rooltess
this would fail to permission problems.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Ginkgo currently logs a warning because the cli version (defnied in
test/tools/go.mod) does not match the library version (defnied in
go.mod).
Simply fix this by updating ginkgo to the latest version.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
There is no need to buffer them all into an array then write them once
at the end. Just write directly to the file.
Fixes#19104
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
The --authfile flag has been ignored. Fix that and add a test to make
sure we won't regress another time. Requires a new --tls-verify flag
to actually test the code.
Also bump c/common since common/pull/1538 is required to correctly check
for updates. Note that I had to use the go-mod-edit-replace trick on
c/common as c/buildah would otherwise be moved back to 1.30.
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=2218315
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
The podman-login tests have accumulated much cruft over the
years, because that's the only place where we run a local
registry, and the process was crufty: we actually start/stopped
the registry as the first & last tests of the file. Meaning,
you couldn't do 'hack/bats 150:just-one-test' because that
would skip the registry start. And just now, a completely
unrelated test has had to be shoved into the login file.
This PR revamps the whole thing, by adding a new registry helper
module that can be used anywhere. And, once the registry is
started, it just stays running until the end of tests. (This
requires BATS 1.7 or greater).
Signed-off-by: Ed Santiago <santiago@redhat.com>
For pods with bridged and slirp4netns networking we create /etc/hosts
entries to make it more convenient for the containers to address each
other. We omitted to do this for pasta networking, however. Add the
necessary code to do this.
Closes: https://github.com/containers/podman/issues/17922
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Confirm that pasta test name agrees with the test being run.
This is a development-time-only check, it must never fail
beyond CI. The idea is to prevent something like
@test "... Single TCP ... IPv4" {
pasta_test_to 6 ... udp ...
}
Signed-off-by: Ed Santiago <santiago@redhat.com>
- typo fix, a misspelled variable resulting in test being NOP
- remove unnecessary variable (followup to #19044)
- add opportunistic CONTAINERS_CONF test (followup to #19032)
Signed-off-by: Ed Santiago <santiago@redhat.com>
Make sure we use the config field to know if we should use pasta or
slirp4netns as default.
While at it fix broken code which sets the default at two different
places, also do not set in Validate() as this should not modify the
specgen IMO, so set it directly before that.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
This reverts commit c2a24abc0d, which
itself reverted 1c08f2edac, which
reverted e33f4e0bc7.
The original e33f4e0bc7 "pasta: Use two connections instead of three
in TCP range forward tests" was a workaround to avoid intermittent
errors in CI where the pasta networking port range forwarding tests
would fail. It was reverted and unreverted when we thought we'd fixed
the problem, but that turned out not to be the case.
We're now much more confident that we've genuinely found and fixed (or
at least, worked around) the underlying problem, so we revert it again.
Link: https://github.com/containers/podman/issues/17287
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
With a number of the port range forwarding tests, we've seen occasional
failures where the sending socat fails with an EINTR on connect(). This
was mitigated by e33f4e0bc7 "pasta: Use two connections instead of three
in TCP range forward tests" (which has been reverted and un-reverted
several times). However, this did not eliminate the problem, for example
see [0].
For the failing tests we are using the socat address "EXEC:printf x" to
make socat invoke printf(1) to generate a single byte of data to transfer.
Closer analysis shows that the SIGCHLD as the printf process ends is
occasionally intersecting with the connect() call causing this failure.
This is arguably a bug in socat, to not handle this race one way or
another. However, we can easily workaround the problem by using a
temporary file with the data to transfer, rather than invoking printf every
time. Do this, to avoid the flakiness of these tests.
[0]
https://github.com/containers/podman/issues/17287#issuecomment-1611855165
Closes: https://github.com/containers/podman/issues/17287
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
https://github.com/containers/podman/pull/19021 fixed bugs with the pasta
networking tests not working on hosts with multiple interfaces. Alas, the
patch left in some stale code that generates spurious error messages for
the IPv6 case. This is sort of harmless - later code overrides what's done
here and the tests can pass anyway. However if a test fails for some other
reason it means we get a misleading irrelevant error message.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
It is pretty complicated to display the secret on the host, but is
not really secured. This patch makes it easier to examine the secret.
Partial fix for https://github.com/containers/podman/issues/18667
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
[October 2022] While mucking around in this test, I noticed that
a test was being unnecessarily skipped in rootless. Reason was,
test was creating a /etc/systemd file, which it really shouldn't
have been doing anyway.
[Flash forward to June 2023] Ugh. This got complicated, so I
abandoned it. But it's flaking in CI, so one focus-push later,
here's everything that this PR fixes:
- create systemd unit file in proper (safe) path.
- create it *with proper podman options!!!* As in, the
whole --this --that --root --tmpdir options! Sheesh!
- use a pseudorandom service name, not just "redis"
- invoke systemctl/journalctl with --system or --user
as appropriate.
- remove unnecessary "bash -c"
- remove SkipIfRootless, but add SkipIfRemote
Signed-off-by: Ed Santiago <santiago@redhat.com>
At various points the pasta bats tests need to know the name of the
interface that pasta will use by default, and the host addresses it will
use by default. Currently we use the pre-existing helper functions
ether_get_name and ipv[46]_get_addr_global to retreive that.
However, those just pick the first non-loopback interface or address, which
may not be the one that pasta uses if there are multiple connected host
interfaces.
Replace those helpers with local ones which examine the routing table to
more closely match pasta's internal logic about which interface to select.
This allows the tests to run successfully on a host with multiple
interfaces.
Closes: https://github.com/containers/podman/issues/19007
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
There was a huge cut and paste of mount options which were not constent
in parsing tmpfs, bind and volume mounts. Consolidated into a single
function to guarantee all parse the same.
Fixes: https://github.com/containers/podman/issues/18995
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
This commit was automatically cherry-picked
by buildah-vendor-treadmill v0.3
from the buildah vendor treadmill PR, #13808
Changes since 2023-05-01:
- skip a new test, it fails in remote
- skip encrypted-FROM test, broken by buildah PR 4746
Signed-off-by: Ed Santiago <santiago@redhat.com>
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
After[1] c/image no longer prints "Storing signatures" so we should
not check for it.
[1] https://github.com/containers/image/pull/2001
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Sometimes this tests flakes but in the CI log I see all expected lines
printed but still for some reason the matcher fails.
Right now it will truncate the array so it is not possible to verify
what the matcher sees. Change this be removing the truncate limit for
this specific test only.
see #18501
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
we were silently ignoring --device-cgroup-rule in rootless mode. Make
sure an error is returned if the user tries to use it.
Closes: https://github.com/containers/podman/issues/18698
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
Handle more TOCTOUs operating on listed images. Also pull in
containers/common/pull/1520 and containers/common/pull/1522 which do the
same on the internal layer tree.
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=2216700
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Add new _prefetch helper for fetching and caching images.
Use it in a few places, most importantly 120-load.bats
where our teardown() now runs 'rmi -af'.
Reason: in #17911 we discovered that podman save + load do
not actually preserve the image: annotations and other metadata
are lost. This means that a test which runs after 120-load.bats
is operating on a different $IMAGE than a test which runs before.
This is not a problem except in very obscure corner cases, like
one fixed in #18542, but it seems irresponsible to just handwave
that issue away
The _prefetch function uses skopeo for fetching and saving
images, because skopeo preserves digests and metadata.
[Side note for posterity: I tried amending basic_setup() to
always rmi -a + prefetch, instead of the current images -a +
rmi unwanted ones. That slowed down system tests by 10 minutes,
presumably because loads are much slower than queries. I reverted
that change and am documenting it as a reminder of why we do things
the way we do.]
Signed-off-by: Ed Santiago <santiago@redhat.com>
Wait before sending status code 200 for the first top call and if that
fails return a proper error code.
This was leading to some confusion in [1] because podman just reported
200 but did not wirte anything back.
[1] https://bugzilla.redhat.com/show_bug.cgi?id=2215572
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Support two new wait conditions, "healthy" and "unhealthy". This
further paves the way for integrating sdnotify with health checks which
is currently being tracked in #6160.
Fixes: #13627
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Replace /dev/foodevdir with unique paths, to avoid one
test's RemoveAll() from stepping on another test.
Closes: #18958
Signed-off-by: Ed Santiago <santiago@redhat.com>
for #18514: if we get a timeout in teardown(), run and show
the output of podman system locks
for #18831: if we hit unmount/EINVAL, nothing will ever work
again, so signal all future tests to skip.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Make sure that the create endpoint does not always return 200 even in
case of a failure. Some of the code had to be massaged since encoding a
report implies sending a 200.
Fixes: #15828
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Make sure that the push endpoint does not always return 200 even in case
of a push failure. Some of the code had to be massaged since encoding a
report implies sending a 200.
Fixes: #18751
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Add header comment suggesting podman network create instead.
Stop using it in checkpoint tests. Turned out to be much more
complicated than expected.
Also, fix two issues caught while scanning the code:
- remove obsolete f28-and-earlier code.
- remove seccomp workaround needed for RHEL7
Signed-off-by: Ed Santiago <santiago@redhat.com>
A few tests were doing "podman run -d" + "podman logs".
This is racy. Remove the unnecessary "-d".
And, as long as we're mucking around in here:
- remove the "-t" from the 800-lines test, so we get
clean output without ^Ms
- remove unnecessary "sh", "-c" from simple echo commands
- add actual error-message checks to two places that
were only checking exit status
Resolves one (not all) of the flakes tracked in #18501
Signed-off-by: Ed Santiago <santiago@redhat.com>
The platform parameter has been ignored such that images have been
looked up by name only.
Fixes: #18951
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Which revealed that absent --authfile's are ignored but shouldn't.
The issue is now being tracked in #18938.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Commit f131eaa74a changed restart to a stop+start motivated by
comments in the systemd man pages that restart behaves different than
stop+start, for instance, that it keeps certain resources open and
treats timers differently. Yet, the actually fix for #17607 in the very
same commit was dealing with an ENOENT of the CID file on container
removal.
As it turns out in in #18926, changing to stop+start regressed on
restarting dependencies when auto updating a systemd unit. Hence, move
back to using restart to make sure that dependent systemd units are
restarted as well.
An alternative could be recommending to use `BindsTo=` in Quadlet files
but this seems less common than `Requires=` and hence more risky to
cause issues on user sites.
Fixes: #18926
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Add support for `--imagestore` in podman which allows users to split the filesystem of containers vs image store, imagestore if configured will pull images in image storage instead of the graphRoot while keeping the other parts still in the originally configured graphRoot.
This is an implementation of
https://github.com/containers/storage/pull/1549 in podman.
Signed-off-by: Aditya R <arajan@redhat.com>
The backend allows for any start/end ip in the subnet. There is no
reason to limit the cli to only CIDR subnets. This allows for much more
flexibility.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
The "removed" condition mapped to an undefined state which ultimately
rendered the wait endpoint to return an incorrect exit code. Instead,
map "removed" to "exited" to make sure Podman returns the expected
exit code.
Fixes: #18889
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
For tests that use '--ip XX', random IP allocation is not
working well. Switch instead to a deterministic algorithm
with CPU affinity and a fudge factor for CNI.
Signed-off-by: Ed Santiago <santiago@redhat.com>
The code was moved to c/common so use that instead. Also add tests for
the new pasta_options config field. However there is one outstanding
problem[1]: pasta rejects most options when set more than once. Thus it is
impossible to overwrite most of them on the cli. If we cannot fix this
in pasta I need to make further changes in c/common to dedup the
options.
[1] https://archives.passt.top/passt-dev/895dae7d-3e61-4ef7-829a-87966ab0bb3a@redhat.com/
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
This commit creates a new command `podmansh` command which can be used by
administrators to provide a confined shell to their users.
The user will only have access to the volumes and capabilities for that
user.
Co-authored-by: Paul Holzinger <pholzing@redhat.com>
Co-authored-by: Daniel Walsh <dwalsh@redhat.com>
Co-authored-by: Petr Lautrbach <lautrbach@redhat.com>
Co-authored-by: Ed Santiago <santiago@redhat.com>
Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
We no longer allow to match ids in the middle, this makes no realy
sense. ID matches should always be by prefix.
https://github.com/containers/podman/issues/18471
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
podman info prints the network information about binary path,
package version, program version and DNS information.
Fixes: #18443
Signed-off-by: Toshiki Sonoda <sonoda.toshiki@fujitsu.com>
There is weird issue #18856 which causes the version check to fail.
Return the underlying error in these cases so we can see it and debug
it.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Remove an outdated comment on the absence of exit-code propagation when
running K8s workloads in systemd. The `podman-kube@` systemd template
is using default restart policy of the system. The exit-code
propagation is tested in other tests, so we can keep the logic as is.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
One feature needed for podmansh is the ability to set the default
homedir to be the workingdir when you login.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
add routes using the --route flag.
the no_default_route option in --opt prevents a default route from
getting added automatically.
Signed-off-by: Jan Hendrik Farr <github@jfarr.cc>
A c/storage PR[1] chnage the behavior to correctly report umount errors.
This is causing problem in the updgrade tests. The problem is that a
cotnainer is mounted inside another container and then unmounted on the
host. Therefore both operations happen in different mount namespaces.
this is expcted but we want to share the mounts between them. This is
the default but c/stroage make the root private by default thus the
mounts were not shared. To fix this use the `skip_mount_home` storage
option so the mount is kept shared.
[1] https://github.com/containers/storage/pull/1607
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
The Sysctl=name=value entry can be used to set --sysctl=name=value
directly without the need to use PodmanArgs=--sysctl=name=value.
Signed-off-by: Laurenz Kruty <git@laurenzkruty.de>
First: fix podman-registry script so it preserves the initial $PODMAN,
so all subsequent invocations of ps, logs, and stop will use the
same binary and arguments. Until now we've handled this by requiring
that our caller manage $PODMAN (and keep it the same), but that's
just wrong.
Next, simplify the golang interface: move the $PODMAN setting into
registry.go, instead of requiring e2e callers to set it. (This
could use some work: the local/remote conditional is icky).
IMPORTANT: To prevent registry.go from using the wrong podman binary,
the Start() call is gone. Only StartWithOptions() is valid now.
And, minor cleanup: comments, and add an actual error-message check
Reason for this PR is a recurring flake, #18355, whose multiple
failure modes I truly can't understand. I don't think this PR
is going to fix it, but this is still necessary work.
Signed-off-by: Ed Santiago <santiago@redhat.com>
- trust_test: adding 'Ordered' seems to resolve a very common
flake. I've tested this for dozens of CI runs, and haven't
seen the flake recur (normally it fails every few runs).
- exec and search tests: add FlakeAttempts(3). This is a NOP
under our current CI setup, in which we run ginkgo with
a global --flake-attempts=3. I am submitting this as an
optimistic step toward a no-flake-attempts world (#17967)
Fixes: #18358
Signed-off-by: Ed Santiago <santiago@redhat.com>
For filter=id=XXX (containers, pods) and =ctr-ids=XXX (pods):
if XXX is only hex characters, treat it as a PREFIX
otherwise, treat it as a REGEX
Add tests. Update documentation. And fix an incorrect help message.
Fixes: #18471
Signed-off-by: Ed Santiago <santiago@redhat.com>
Ginkgo test names can have more than two levels: there can be
a nested series of Describes() before the final It(). (e.g.,
quadlet_test.go). Handle that.
Before: we just assumed that the third-or-maybe-fourth line
after a "-----" divider was the test name.
Now: examine every line after the "-----" divider, until the
first empty line. Lines with /path/to/source/file are ignored,
lines with text strings are assembled together to make anchors.
This is still imperfect but it's much better than before.
SPECIAL NOTE: in order to allow linking to timing results
in the AfterSuite, I've changed the test name from Leaf to Full.
This will now be a much longer string, and hence much less
readable, but I'm inclined to think it's more correct. Please
review carefully and lmk if I should revert.
Finally, as an unrelated add-on, add links (at top) to original
log, journal, and (if applicable) podman-remote server logs.
Signed-off-by: Ed Santiago <santiago@redhat.com>
To debug a deadlock, we really want to know what lock is actually
locked, so we can figure out what is using that lock. This PR
adds support for this, using trylock to check if every lock on
the system is free or in use. Will really need to be run a few
times in quick succession to verify that it's not a transient
lock and it's actually stuck, but that's not really a big deal.
Signed-off-by: Matt Heon <mheon@redhat.com>
This is a nice quality-of-life change that should help to debug
situations where someone runs out of locks (usually when a bunch
of unused volumes accumulate).
Signed-off-by: Matt Heon <mheon@redhat.com>
Test that pasta generates a sensible error message if asked to forward a
protocol it doesn't understand.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Add a test for generate kube to verify that the ulimit
annotation is not set for the default case when the user
doesn't set any ulimits.
Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
Intented to fix an obscure, unlikely race condition in which (I
think) two parallel jobs called GetPort() and were assigned the
same port.
Also, add actual proper testing to two HTTP-registry tests, and
Skip a third that's a waste of cycles (filed #18768)
Signed-off-by: Ed Santiago <santiago@redhat.com>
This reverts commit 9bd833bcfd.
With the fix for `podman rm -fa` merged, we no longer require
this patch.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
We _usually_ have only one image in store, $IMAGE, but it's
perfectly fine to also have $SYSTEMD_IMAGE also. Fix a few
tests so they can handle that condition.
And, cleanup:
- remove a no-longer-useful test ("podman load NEWNAME",
functionality that was removed 2+ years ago in #8877)
- reorder some tests in the image-mount test, to make
them safer and easier to understand
- use no-such-image, not no-such-container, in image-mount test.
Computer don't care, but this human felt confused for a sec.
Signed-off-by: Ed Santiago <santiago@redhat.com>
The current way of bind mounting the host timezone file has problems.
Because /etc/localtime in the image may exist and is a symlink under
/usr/share/zoneinfo it will overwrite the targetfile. That confuses
timezone parses especially java where this approach does not work at
all. So we end up with an link which does not reflect the actual truth.
The better way is to just change the symlink in the image like it is
done on the host. However because not all images ship tzdata we cannot
rely on that either. So now we do both, when tzdata is installed then
use the symlink and if not we keep the current way of copying the host
timezone file in the container to /etc/localtime.
Also note that we need to rebuild the systemd image to include tzdata in
order to test this as our images do not contain the tzdata by default.
Fixes https://bugzilla.redhat.com/show_bug.cgi?id=2149876
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Systemd doesn't support `never` and logs a warning, systemd uses no as
default so we do not have to specify it at all.
Check systemd.service(5) for the systemd docs.
Fixes#18743
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Reduce sleep-loop time in logs test, from 1s to 0.1s,
to make 'podman stop' take effect more quickly. With 1s,
and testing with 1s resolution, we get flakes.
Fixes: #17826
Signed-off-by: Ed Santiago <santiago@redhat.com>
The new exit-code propagation test is racy: 'podman wait' can
fail if the service container has already been cleaned up by
systemd.
Solution: run the inspect and wait tests opportunistically, i.e.,
only if those commands succeed. If they fail, confirm that they
fail with ENOSUCHCONTAINER. This may silently lose us some
coverage ... but none of it is important. The important
test, systemctl final status, remains.
Also, as drive-bys:
- add a FIXME comment documenting another race condition
that I'm not bothering to fix right now
- give distinct names to unit files, for readability in
test failures
Fixes: #18732
Signed-off-by: Ed Santiago <santiago@redhat.com>
"image rm concurrent" test is still failing, even after #18664:
Error: no contents in "/tmp/podman_test967723851/Dockerfile"
Probable cause: the images are built in parallel, and p.BuildImage()
writes one single Dockerfile. (This almost certainly renders the
test less effective than intended, since the generated images
might end up being identical).
Solution: write and use a uniquely-named Dockerfile
Signed-off-by: Ed Santiago <santiago@redhat.com>
When we do path completion in images a user could try to complete a
simple relative path, e.g. podman run $IMAGE e... should complete to etc
if this path exists in the image. Right now we panic in this case as the
current check didn't account for an empty string in simplePathJoinUnix().
In such a case return the path directly because we can not alter what
the user typed on the cli and must return a path without slash as well
in order for the shell to suggest the completion.
Fixes https://bugzilla.redhat.com/show_bug.cgi?id=2209809
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
There are quite a lot of places in podman were we have some signal
handlers, most notably libpod/shutdown/handler.go.
However when we rexec we do not want any of that and just send all
signals we get down to the child obviously. So before we install our
signal handler we must first reset all others with signal.Reset().
Also while at it fix a problem were the joinUserAndMountNS() code path
would not forward signals at all. This code path is used when you have
running containers but the pause process was killed.
Fixes#16091
Given that signal handlers run in different goroutines parallel it would
explain why it flakes sometimes in CI. However to my understanding this
flake can only happen when the pause process is dead before we run the
podman command. So the question still is what kills the pause process?
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Use ExecStopPost instead of ExecStop to make sure containers, pods, etc.
are all cleaned up even in case of an error.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Add a new field `ExitCodePropagation` field to allow for configuring the
newly added functionality of controlling how the main PID of a kube
service exits.
Jira: issues.redhat.com/browse/RUN-1776
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Implement means for reflecting failed containers (i.e., those having
exited non-zero) to better integrate `kube play` with systemd. The
idea is to have the main PID of `kube play` exit non-zero in a
configurable way such that systemd's restart policies can kick in.
When using the default sdnotify-notify policy, the service container
acts as the main PID to further reduce the resource footprint. In that
case, before stopping the service container, Podman will lookup the exit
codes of all non-infra containers. The service will then behave
according to the following three exit-code policies:
- `none`: exit 0 and ignore containers (default)
- `any`: exit non-zero if _any_ container did
- `all`: exit non-zero if _all_ containers did
The upper values can be passed via a hidden `kube play
--service-exit-code-propagation` flag which can be used by tests and
later on by Quadlet.
In case Podman acts as the main PID (i.e., when at least one container
runs with an sdnotify-policy other than "ignore"), Podman will continue
to wait for the service container to exit and reflect its exit code.
Note that this commit also fixes a long-standing annoyance of the
service container exiting non-zero. The underlying issue was that the
service container had been stopped with SIGKILL instead of SIGTERM and
hence exited non-zero. Fixing that was a prerequisite for the exit-code
propagation to work but also improves the integration of `kube play`
with systemd and hence Quadlet with systemd.
Jira: issues.redhat.com/browse/RUN-1776
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
RHEL gating tests failing, because (sigh) journalctl doesn't
work rootless on RHEL.
I think the flake is fixed anyway, so we don't need this.
This reverts commit ba141adce4.
Signed-off-by: Ed Santiago <santiago@redhat.com>
This test is intended to test concurrent removals, so don't
risk a removal breaking a build.
Fixes#18659 .
(The sitaution that removals can break a build WIP is a real
problem that should be fixed, but that's not a target of this test.)
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Instrument system tests in hopes of tracking down #17216,
the unlinkat-ebusy-hosed flake.
Oh, also, timestamp.awk: timestamps have always been UTC, but
add a 'Z' to make it unambiguous.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Fixes: https://github.com/containers/podman/issues/18239
[NO NEW TESTS NEEDED]
@test "podman build -f test" in test/system/070-build.bats
Will test this. This was passing when run on a local system since
the remote end was using the clients path to read the Containerfile
The issue is it would not work in a podman machine since the
Containerfile would/should be a different path.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Make sure to look for the container's exit code when it's in stopped
state. With `--restart=always`, the container seems to stay in the
stopped state which led the wait logic to loop until the 20 seconds
timeout for the cleanup process to have finished kicks in.
Also defensively make sure to loop when the container is in stopped
state but no exit code has been written yet.
Add a regression test to make sure Podman doesn't wait more than 20
seconds. Even on a CI machine under high load I expect it to take much
much much less than that, so I do not expect this test to flake in the
future.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
The examples show that --dns-add 8.8.8.8,1.1.1.1 is valid but it fails,
fix this by using StringSliceVar which splits at commas.
Added tests to ensure it is working.
Fixes#18632
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
b25b330306 introduced this behaviour.
It was fine at the time because we didn't support "container update",
so the limit could not be changed at runtime. Since it is not
possible to change the memory limit at runtime, read the limit as
reported from the cgroup.
https://github.com/containers/crun/pull/1217 is required for crun.
Closes: https://github.com/containers/podman/issues/18621
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
We should not change selinux, in a parallel context this can change the
behavior of other tests and we should never disable selinux anyway.
Lets see if this passes CI or not.
Fixes#18564
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
the combination --pod and --userns is already blocked. Ignore the
PODMAN_USERNS variable when a pod is used, since it would cause to
create a new user namespace for the container.
Ideally a container should be able to do that, but its user namespace
must be a child of the pod user namespace, not a sibling. Since
nested user namespaces are not allowed in the OCI runtime specs,
disallow this case, since the end result is just confusing for the
user.
Closes: https://github.com/containers/podman/issues/18580
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
Enable the --configmap flag for the remote case of podman
kube play. Users can pass in the paths to the configmap files
for kube play to use when creating the pods and containers from
a kube yaml file. The configmap file is read and the contents are
appended to the contents of the main yaml file before passed to the
remote client.
Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
Accept a tag in the compat api endpoint. For the fromImage param we
already parse it but for fromSrc we did not.
Fixes#18597
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Init containers are currently not properly supported in
generate-systemd and there are no plans to do so since
all focus lies on Quadlet going forward.
Hence, generate systemd should through an error.
Closes: #18585
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
In run_podman(), display a nanosecond-level timestamp next to
each command and its output.
Because this clutters the results, teach logformatter to grok
these new timestamps, strip them, and display a more human-readable
time delta in the left-hand timestamp column. logformatter started off
as a mess and is now, well, 🤮. I'm sorry. I just hope its results
make it worthwhile.
Signed-off-by: Ed Santiago <santiago@redhat.com>
When running ginkgo tests locally we often only want to test a small
subset. I think most people just add the `FIt` block but then you need
to remember to undo that before pushing the changes.
With this change you can just run:
```
make localintegration FOCUS="test name here"
make localintegration FOCUS_FILE="some_test.go"
```
I updated the test Readme to use this new syntax.
The options just map to the ginkgo options, see the upstream docs
linked in the readme for more information about syntax.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
ryanhockstad
OpenShift Merge Robot
Ryan Hockstad
Ed Santiago
Anders F Björklund
Radostin Stoyanov
danishprakash
Giuseppe Scrivano
Matt Heon
Daniel J Walsh
Urvashi Mohnani
Valentin Rothberg
renovate[bot]
Aditya R
Miloslav Trmač
Sergio Oller
chnrxn
Ygal Blum
Paul Holzinger
Jake Correnti
Peter Hunt
Matej Vasek
Chris Evich
Matthew Heon
Dan Čermák
Chetan Giradkar
Romain Geissler
Boaz Shuster
Hari Kannan
Lokesh Mandvekar
Paul Holzinger
Erik Sjölund
Toshiki Sonoda
Alex Palaistras
Black-Hole1
David Gibson
Jan Hendrik Farr
Laurenz Kruty
Stefano Brivio
Luca Di Maio
binghongtao
Gunjan Vyas