Commit Graph

186 Commits

Author SHA1 Message Date
Valentin Rothberg bd09b7aa79 bump go module to version 4
Automated for .go files via gomove [1]:
`gomove github.com/containers/podman/v3 github.com/containers/podman/v4`

Remaining files via vgrep [2]:
`vgrep github.com/containers/podman/v3`

[1] https://github.com/KSubedi/gomove
[2] https://github.com/vrothberg/vgrep

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2022-01-18 12:47:07 +01:00
Daniel J Walsh a15dfb3648
Standardize on capatalized Cgroups
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2022-01-14 12:30:20 -05:00
Daniel J Walsh 1941d45e32
add OCI Runtime name to errors
It would be easier to diagnose OCI runtime errors if the error actually
had the name of the OCI runtime that produced the error.

[NO NEW TESTS NEEDED]

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2022-01-06 16:21:02 -05:00
Daniel J Walsh 2130d18539
Update vendor or containers/common moving pkg/cgroups there
[NO NEW TESTS NEEDED] This is just moving pkg/cgroups out so
existing tests should be fine.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-12-07 06:17:11 -05:00
Paul Holzinger bc3c5be2fb
checkpoint do not modify XDG_RUNTIME_DIR
We should not modify the XDG_RUNTIME_DIR env value during runtime of
libpod, this can cause hard to find bugs. Only set it for the OCI
runtime, this matches the other commands such as start, stop, kill...

[NO NEW TESTS NEEDED]

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-24 15:02:11 +01:00
Giuseppe Scrivano c7ebaeee0e
libpod: leave thread locked on errors
if the SELinux label could not be restored correctly, leave the OS
thread locked so that it is terminated once it returns to the threads
pool.

[NO NEW TESTS NEEDED] the failure is hard to reproduce

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-11-24 11:08:23 +01:00
Daniel J Walsh df6aa67302
Unset SocketLabel after system finishes checkpointing
This should fix the SELinux issue we are seeing with talking to
/run/systemd/private.

Fixes: https://github.com/containers/podman/issues/12362

Also unset the XDG_RUNTIME_DIR if set, since we don't know when running
as a service if this will cause issue.s

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-11-23 16:31:54 -05:00
Matthew Heon 5cf2683bfe Warn on failing to update container status
failed to send a signal to the container's PID1, but ignored the
results of that update. That's generally bad practice, since even
if we can't directly take action on an error, we should still
make an effort to report it for debugging purposes. I used Infof
instead of something more serious to avoid duplicate reporting to
the user if something has gone seriously wrong.

[NO NEW TESTS NEEDED] this is just adding additional error reporting.

Signed-off-by: Matthew Heon <mheon@redhat.com>
2021-11-23 09:50:12 -05:00
Aditya Rajan a4e4b8d926
oci: ack crun output when container is not there
`crun status ctrid` outputs `No such file or directory` when container
is not there so podman much ack it.

[NO NEW TESTS NEEDED]

Signed-off-by: Aditya Rajan <arajan@redhat.com>
2021-11-23 16:41:28 +05:30
Aditya Rajan 08558b27ff
oci: exit gracefully if container is already dead
While trying to kill a container with a `signal` we cant do anything if
container is already dead so `exit` gracefully instead of trying to
delete container again. Get container status from runtime.

[ NO NEW TESTS NEEDED ]

Signed-off-by: Aditya Rajan <arajan@redhat.com>
2021-11-23 16:40:53 +05:30
OpenShift Merge Robot 319d3fba6d
Merge pull request #12354 from Luap99/exit-command
Do not store the exit command in container config
2021-11-18 23:51:12 +01:00
Paul Holzinger 0dae50f1d3
Do not store the exit command in container config
There is a problem with creating and storing the exit command when the
container was created. It only contains the options the container was
created with but NOT the options the container is started with. One
example would be a CNI network config. If I start a container once, then
change the cni config dir with `--cni-config-dir` ans start it a second
time it will start successfully. However the exit command still contains
the wrong `--cni-config-dir` because it was not updated.

To fix this we do not want to store the exit command at all. Instead we
create it every time the conmon process for the container is startet.
This guarantees us that the container cleanup process is startet with
the correct settings.

[NO NEW TESTS NEEDED]

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-18 20:28:03 +01:00
Radostin Stoyanov 6d23ea60d2
Add --file-locks checkpoint/restore option
CRIU supports checkpoint/restore of file locks. This feature is
required to checkpoint/restore containers running applications
such as MySQL.

Signed-off-by: Radostin Stoyanov <radostin@redhat.com>
2021-11-18 19:23:25 +00:00
Adrian Reber 80e56fa12b
Added optional container restore statistics
This adds the parameter '--print-stats' to 'podman container restore'.
With '--print-stats' Podman will measure how long Podman itself, the OCI
runtime and CRIU requires to restore a checkpoint and print out these
information. CRIU already creates process restore statistics which are
just read in addition to the added measurements. In contrast to just
printing out the ID of the restored container, Podman will now print
out JSON:

 # podman container restore --latest --print-stats
 {
     "podman_restore_duration": 305871,
     "container_statistics": [
         {
             "Id": "47b02e1d474b5d5fe917825e91ac653efa757c91e5a81a368d771a78f6b5ed20",
             "runtime_restore_duration": 140614,
             "criu_statistics": {
                 "forking_time": 5,
                 "restore_time": 67672,
                 "pages_restored": 14
             }
         }
     ]
 }

The output contains 'podman_restore_duration' which contains the
number of microseconds Podman required to restore the checkpoint. The
output also includes 'runtime_restore_duration' which is the time
the runtime needed to restore that specific container. Each container
also includes 'criu_statistics' which displays the timing information
collected by CRIU.

Signed-off-by: Adrian Reber <areber@redhat.com>
2021-11-15 11:50:25 +00:00
Adrian Reber 6202e8102b
Added optional container checkpointing statistics
This adds the parameter '--print-stats' to 'podman container checkpoint'.
With '--print-stats' Podman will measure how long Podman itself, the OCI
runtime and CRIU requires to create a checkpoint and print out these
information. CRIU already creates checkpointing statistics which are
just read in addition to the added measurements. In contrast to just
printing out the ID of the checkpointed container, Podman will now print
out JSON:

 # podman container checkpoint --latest --print-stats
 {
     "podman_checkpoint_duration": 360749,
     "container_statistics": [
         {
             "Id": "25244244bf2efbef30fb6857ddea8cb2e5489f07eb6659e20dda117f0c466808",
             "runtime_checkpoint_duration": 177222,
             "criu_statistics": {
                 "freezing_time": 100657,
                 "frozen_time": 60700,
                 "memdump_time": 8162,
                 "memwrite_time": 4224,
                 "pages_scanned": 20561,
                 "pages_written": 2129
             }
         }
     ]
 }

The output contains 'podman_checkpoint_duration' which contains the
number of microseconds Podman required to create the checkpoint. The
output also includes 'runtime_checkpoint_duration' which is the time
the runtime needed to checkpoint that specific container. Each container
also includes 'criu_statistics' which displays the timing information
collected by CRIU.

Signed-off-by: Adrian Reber <areber@redhat.com>
2021-11-15 11:50:24 +00:00
Giuseppe Scrivano 9e5cd32056
oci: rename sub-cgroup to runtime instead of supervisor
we are having a hard time figuring out a failure in the CI:

https://github.com/containers/podman/issues/11191

Rename the sub-cgroup created here, so we can be certain the error is
caused by this part.

[NO NEW TESTS NEEDED] we need this for the CI.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-10-28 09:18:08 +02:00
Daniel J Walsh 48d26a893e
Warn if podman stop timeout expires that sigkill was sent
Note: the Warning message will not come to podman-remote.
It would be difficult to plumb, and not really worth the effort.

Fixes: https://github.com/containers/podman/issues/11854

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-10-08 07:06:37 -04:00
OpenShift Merge Robot 470e52f272
Merge pull request #11878 from mheon/stop_stopping
Allow `podman stop` to be run on Stopping containers
2021-10-06 20:09:04 +02:00
Paul Holzinger fbce7584d7
libpod: fix race when closing STDIN
There is a race where `conn.Close()` was called before `conn.CloseWrite()`.
In this case `CloseWrite` will fail and an useless error is printed. To
fix this we move the the `CloseWrite()` call to the same goroutine to
remove the race. This ensures that `CloseWrite()` is called before
`Close()` and never afterwards.
Also fixed podman-remote run where the STDIN was never was closed.
This is causing flakes in CI testing.

[NO TESTS NEEDED]

Fixes #11856

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-10-06 17:51:07 +02:00
Matthew Heon 8bd9f58d1d Ensure `podman ps --sync` functions
The backend for `ps --sync` has been nonfunctional for a long
while now - probably since v2.0. It's questionable how useful the
flag is in modern Podman (the original case it was intended to
catch, Conmon gone via SIGKILL, should be handled now via pinging
the process with a signal to ensure it's still alive) but having
the ability to force a refresh of container state from the OCI
runtime is still useful.

Signed-off-by: Matthew Heon <mheon@redhat.com>
2021-10-06 11:19:32 -04:00
OpenShift Merge Robot b187dfef20
Merge pull request #11390 from giuseppe/logging-passthrough
logging: new mode -l passthrough
2021-09-29 15:33:06 -04:00
Valentin Rothberg 5d6ea90e75 libpod: do not call (*container).Config()
Access the container's config field directly inside of libpod instead of
calling `Config()` which in turn creates expensive JSON deep copies.

Accessing the field directly drops memory consumption of a simple
`podman run --rm busybox true` from 1245kB to 410kB.

[NO TESTS NEEDED]

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-09-28 17:18:02 +02:00
Giuseppe Scrivano 3ce98a5ec2
logging: new mode -l passthrough
it allows to pass the current std streams down to the container.

conmon support: https://github.com/containers/conmon/pull/289

[NO TESTS NEEDED] it needs a new conmon.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-09-27 12:07:01 +02:00
Daniel J Walsh 1c4e6d8624
standardize logrus messages to upper case
Remove ERROR: Error stutter from logrus messages also.

[ NO TESTS NEEDED] This is just code cleanup.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-09-22 15:29:34 -04:00
Paul Holzinger d3f0f09ad9
libpod: rootful close binded ports
For rootful users ports are forwarded via iptables. To make sure no
other process tries to use them, libpod will bind the ports and pass the
fds to conmon. There seems to be race when a container is restarted
because libpod tries to bind the port before the conmon process exited.

The problem only hapens with the podman service because it keeps the
connection open. Once we have the fd and passed it to conmon the
podman service should close the connection.

To verify run `sudo ss -tulpn` and check that only the conmon process
keeps the port open. Previously you would also see the podman server
process listed.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-09-14 11:13:28 +02:00
Paul Holzinger 7cf22279d9
Fix conmon attach socket buffer size
The conmon buffer size is 8192, however the attach socket needs two extra
bytes. The first byte of each message will be the STREAM type. The last
byte is a null byte. So when we want to read 8192 message bytes we need
to read 8193 bytes since the first one is special.
check 1ef246896b/src/ctr_stdio.c (L101-L107)

This problem can be seen in podman-remote run/exec when it prints output
with 8192 or more bytes. The output will miss the 8192 byte.

Fixes #11496

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-09-09 17:57:42 +02:00
Valentin Rothberg 10144b707e pass LISTEN_* environment into container
Make sure that Podman passes the LISTEN_* environment into containers.
Similar to runc, LISTEN_PID is set to 1.

Also remove conditionally passing the LISTEN_FDS as extra files.
The condition was wrong (inverted) and introduced to fix #3572 which
related to running under varlink which has been dropped entirely
with Podman 3.0.  Note that the NOTIFY_SOCKET and LISTEN_* variables
are cleared when running `system service`.

Fixes: #10443
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-08-31 17:23:05 +02:00
Nalin Dahyabhai 21f396de6f logs: adjust handling around partial log messages
In libpod/logs.LogLine.Write(), don't write a newline to stdout/stderr
when the log message is only part of a line.

In libpod.ConmonOCIRuntime.HTTPAttach(), don't send a newline over the
HTTP connection when the log message is only part of a line.

In pkg/api/handlers/compat.LogsFromContainer(), don't send a newline
over the HTTP connection when the log message is only part of a line,
and don't make doing so conditional on whether or not the client used
the docker or podman endpoint.

In pkg/domain/infra/tunnel.ContainerEngine.ContainerLogs(), don't add
our own newline to log messages, since they already come through from
the server when they need to.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2021-08-23 18:00:18 -04:00
Daniel J Walsh c22f3e8b4e Implement SD-NOTIFY proxy in conmon
This leverages conmon's ability to proxy the SD-NOTIFY socket.
This prevents locking caused by OCI runtime blocking, waiting for
SD-NOTIFY messages, and instead passes the messages directly up
to the host.

NOTE: Also re-enable the auto-update tests which has been disabled due
to flakiness.  With this change, Podman properly integrates into
systemd.

Fixes: #7316
Signed-off-by: Joseph Gooch <mrwizard@dok.org>
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-08-20 11:12:05 +02:00
Adrian Reber eb94467780
Support checkpoint/restore with pods
This adds support to checkpoint containers out of pods and restore
container into pods.

It is only possible to restore a container into a pod if it has been
checkpointed out of pod. It is also not possible to restore a non pod
container into a pod.

The main reason this does not work is the PID namespace. If a non pod
container is being restored in a pod with a shared PID namespace, at
least one process in the restored container uses PID 1 which is already
in use by the infrastructure container. If someone tries to restore
container from a pod with a shared PID namespace without a shared PID
namespace it will also fail because the resulting PID namespace will not
have a PID 1.

Signed-off-by: Adrian Reber <areber@redhat.com>
2021-07-27 16:10:44 +02:00
Adrian Reber 240bbc3bfa
Fix pre-checkpointing
Unfortunately --pre-checkpointing never worked as intended and recent
changes to runc have shown that it is broken.

To create a pre-checkpoint CRIU expects the paths between the
pre-checkpoints to be a relative path. If having a previous checkpoint
it needs the be referenced like this: --prev-images-dir ../parent

Unfortunately Podman was giving runc (and CRIU) an absolute path.

Unfortunately, again, until March 2021 CRIU silently ignored if
the path was not relative and switch back to normal checkpointing.

This has been now fixed in CRIU and runc and running pre-checkpoint
with the latest runc fails, because runc already sees that the path is
absolute and returns an error.

This commit fixes this by giving runc a relative path.

This commit also fixes a second pre-checkpointing error which was just
recently introduced.

So summarizing: pre-checkpointing never worked correctly because CRIU
ignored wrong parameters and recent changes broke it even more.

Now both errors should be fixed.

[NO TESTS NEEDED]

Signed-off-by: Adrian Reber <areber@redhat.com>
Signed-off-by: Adrian Reber <adrian@lisas.de>
2021-06-10 15:29:24 +02:00
Paul Holzinger 157459a638 fix container startup for empty pidfile
Commit 728b73d7c4 introduced a regression. Containers created with a
previous version do no longer start successfully. The problem is that
the PidFile in the container config is empty for those containers. If
the PidFile is empty we have to set it to the previous default.

[NO TESTS NEEDED] We should investigate why the system upgrade test did
not caught this.

Fixes #10274

Signed-off-by: Paul Holzinger <paul.holzinger@web.de>
2021-05-10 10:59:04 +02:00
Daniel J Walsh 3538815c5b
Add podman run --timeout option
This option allows users to specify the maximum amount of time to run
before conmon sends the kill signal to the container.

Fixes: https://github.com/containers/podman/issues/6412

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-04-23 11:18:05 -04:00
OpenShift Merge Robot 41677b192a
Merge pull request #8979 from haircommander/full-attach-path
Use full attach path, rather than a symlink
2021-04-21 05:25:19 -04:00
chenkang 728b73d7c4
set pidfile default value int containerconfig
Signed-off-by: chenkang <kongchen28@gmail.com>
2021-04-17 21:15:23 +08:00
Peter Hunt fdfff1dab2 oci: use `--full-path` option for conmon
and stop relying on socket path

Signed-off-by: Peter Hunt <pehunt@redhat.com>
2021-04-16 17:45:52 -04:00
wuhua.ck 8fbe06b8cb add flag "--pidfile" for podman create/run
Signed-off-by: chenkang <kongchen28@gmail.com>
2021-04-16 13:43:14 +08:00
Daniel J Walsh e29fee7329
Fix message about runtime to show only the actual runtime
Currently the debug line shows every runtime up until it finds
the correct one, confusing users on which runtime it is using.

Also move missing OCI runtime from containers/conf down to Debug level
and improved the debug message, to not report error.

[NO TESTS NEEDED] Since this is just debug.

Triggered by https://github.com/containers/podman/issues/4854

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-04-13 09:42:29 -04:00
Matthew Heon 541252afa7 Ensure that `--userns=keep-id` sets user in config
One of the side-effects of the `--userns=keep-id` command is
switching the default user of the container to the UID of the
user running Podman (though this can still be overridden by the
`--user` flag). However, it did this by setting the UID and GID
in the OCI spec, and not by informing Libpod of its intention to
switch users via the `WithUser()` option. Because of this, a lot
of the code that should have triggered when the container ran
with a non-root user was not triggering. In the case of the issue
that this fixed, the code to remove capabilities from non-root
users was not triggering. Adjust the keep-id code to properly
inform Libpod of our intention to use a non-root user to fix
this.

Also, fix an annoying race around short-running exec sessions
where Podman would always print a warning that the exec session
had already stopped.

Fixes #9919

Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2021-04-06 09:18:46 -04:00
Giuseppe Scrivano e85cf8f4a2
security: use the bounding caps with --privileged
when --privileged is used, make sure to not request more capabilities
than currently available in the current context.

[NO TESTS NEEDED] since it fixes existing tests.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-03-19 15:17:11 +01:00
Daniel J Walsh 252aec1c9a
Check for supportsKVM based on basename of the runtime
Fixes: https://github.com/containers/podman/issues/9582

This PR also adds tests to make sure SELinux labels match the runtime,
or if init is specified works with the correct label.

Add tests for selinux kvm/init labels

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-03-03 16:49:51 -05:00
Adrian Reber bf92e21113
Move checkpoint/restore code to pkg/checkpoint/crutils
To be able to reuse common checkpoint/restore functions this commit
moves code to pkg/checkpoint/crutils.

This commit has not functional changes. It only moves code around.

[NO TESTS NEEDED] - only moving code around

Signed-off-by: Adrian Reber <areber@redhat.com>
2021-03-02 17:00:06 +00:00
baude 24d9bda7ff prune remotecommand dependency
prune a dependency that was only being used for a simple struct.  Should
correct checksum issue on tarballs

[NO TESTS NEEDED]

Fixes: #9355

Signed-off-by: baude <bbaude@redhat.com>
2021-02-25 10:02:41 -06:00
Valentin Rothberg 5dded6fae7 bump go module to v3
We missed bumping the go module, so let's do it now :)

* Automated go code with github.com/sirkon/go-imports-rename
* Manually via `vgrep podman/v2` the rest

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-02-22 09:03:51 +01:00
Paul Holzinger 78c8a87362 Enable whitespace linter
Use the whitespace linter and fix the reported problems.

[NO TESTS NEEDED]

Signed-off-by: Paul Holzinger <paul.holzinger@web.de>
2021-02-11 23:01:56 +01:00
OpenShift Merge Robot db5e7ec4c4
Merge pull request #8947 from Luap99/cleanup-code
Fix problems reported by staticcheck
2021-01-12 13:15:35 -05:00
Paul Holzinger 8452b768ec Fix problems reported by staticcheck
`staticcheck` is a golang code analysis tool. https://staticcheck.io/

This commit fixes a lot of problems found in our code. Common problems are:
- unnecessary use of fmt.Sprintf
- duplicated imports with different names
- unnecessary check that a key exists before a delete call

There are still a lot of reported problems in the test files but I have
not looked at those.

Signed-off-by: Paul Holzinger <paul.holzinger@web.de>
2021-01-12 16:11:09 +01:00
OpenShift Merge Robot 5575c7be20
Merge pull request #8819 from chen-zhuohan/add-pre-checkpoint
Add pre-checkpoint and restore with previous
2021-01-12 07:57:05 -05:00
OpenShift Merge Robot 1955eee89f
Merge pull request #8933 from giuseppe/use-O_PATH-for-unix-sock
oci: use /proc/self/fd/FD to open unix socket
2021-01-12 07:26:37 -05:00
Giuseppe Scrivano fdbc278868
oci: use /proc/self/fd/FD to open unix socket
instead of opening directly the UNIX socket path, grab a reference to
it through a O_PATH file descriptor and use the fixed size string
"/proc/self/fd/%d" to open the UNIX socket.  In this way it won't hit
the 108 chars length limit.

Closes: https://github.com/containers/podman/issues/8798

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-01-12 10:38:32 +01:00
Giuseppe Scrivano ae9dab9445
oci: keep LC_ env variables to conmon
it is necessary for conmon to deal with the correct locale, otherwise
it uses C as a fallback.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1893567
Requires: https://github.com/containers/conmon/pull/215

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-01-11 13:48:04 +01:00
unknown 2aa381f2d0 add pre checkpoint
Signed-off-by: Zhuohan Chen <chen_zhuohan@163.com>
2021-01-10 21:38:28 +08:00
Daniel J Walsh db71759b1a
Handle podman exec capabilities correctly
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-01-07 05:53:50 -05:00
OpenShift Merge Robot 618c35570d
Merge pull request #8878 from mheon/no_edit_config
Ensure we do not edit container config in Exec
2021-01-04 21:11:27 -05:00
Matthew Heon 960607a4cd Ensure we do not edit container config in Exec
The existing code grabs the base container's process, and then
modifies it for use with the exec session. This could cause
errors in `podman inspect` or similar on the container, as the
definition of its OCI spec has been changed by the exec session.
The change never propagates to the DB, so it's limited to a
single process, but we should still avoid it when possible - so
deep-copy it before use.

Signed-off-by: Matthew Heon <mheon@redhat.com>
2021-01-04 14:36:41 -05:00
Giuseppe Scrivano 898f57c4c1
systemd: make rundir always accessible
so that the PIDFile can be accessed also without being in the rootless
user namespace.

Closes: https://github.com/containers/podman/issues/8506

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-01-04 14:19:58 +01:00
Giuseppe Scrivano 2a39a6195a
exec: honor --privileged
write the capabilities to the configuration passed to the OCI
runtime.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2020-12-24 22:11:14 +01:00
Giuseppe Scrivano 2a97639263
libpod: change function to accept ExecOptions
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2020-12-24 22:01:38 +01:00
OpenShift Merge Robot 9ac5ed1e08
Merge pull request #8806 from rhatdan/keyring
Pass down EnableKeyring from containers.conf to conmon
2020-12-23 21:41:25 +01:00
Josh Soref 4fa1fce930 Spelling
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
2020-12-22 13:34:31 -05:00
Daniel J Walsh b0a738ce79
Pass down EnableKeyring from containers.conf to conmon
We have a new field in containers.conf that tells whether
or not we want to generate a new keyring in a container.

This field was being ignored.  It now will be followed and
passed down to conmon.

Fixes: https://github.com/containers/podman/issues/8384

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-12-22 13:08:41 -05:00
Giuseppe Scrivano 08f76bf7a5
libpod, conmon: change log level for rootless
Change the log level when running as rootless when moving conmon to a
different cgroup.

Closes: https://github.com/containers/podman/issues/8721

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2020-12-15 18:55:51 +01:00
Daniel J Walsh f00cc25a7c
Drop default log-level from error to warn
Our users are missing certain warning messages that would
make debugging issues with Podman easier.

For example if you do a podman build with a Containerfile
that contains the SHELL directive, the Derective is silently
ignored.

If you run with the log-level warn you get a warning message explainging
what happened.

$ podman build --no-cache -f /tmp/Containerfile1 /tmp/
STEP 1: FROM ubi8
STEP 2: SHELL ["/bin/bash", "-c"]
STEP 3: COMMIT
--> 7a207be102a
7a207be102aa8993eceb32802e6ceb9d2603ceed9dee0fee341df63e6300882e

$ podman --log-level=warn build --no-cache -f /tmp/Containerfile1 /tmp/
STEP 1: FROM ubi8
STEP 2: SHELL ["/bin/bash", "-c"]
STEP 3: COMMIT
WARN[0000] SHELL is not supported for OCI image format, [/bin/bash -c] will be ignored. Must use `docker` format
--> 7bd96fd25b9
7bd96fd25b9f755d8a045e31187e406cf889dcf3799357ec906e90767613e95f

These messages will no longer be lost, when we default to WARNing level.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-12-03 06:28:09 -05:00
Daniel J Walsh 5a032acff6
Only use container/storage/pkg/homedir.Get()
We are resolving the homedir of the user in many different
places.  This Patch consolodates them to use container/storage
version.

This PR also fixes a failure mode when the homedir does not
exists, and the user sets a root path.  In this situation
podman should continue to work. Podman does not require a users
homedir to exist in order to run.

Finally the rootlessConfigHomeDirOnce and rootlessRuntimeDirOnce
were broken, because if an error ever happened, they would not be recorded
the second time, and "" would be returned as the path.

Fixes: https://github.com/containers/podman/issues/8131

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-11-04 14:47:54 -05:00
Daniel J Walsh 831d7fb0d7
Stop excessive wrapping of errors
Most of the builtin golang functions like os.Stat and
os.Open report errors including the file system object
path. We should not wrap these errors and put the file path
in a second time, causing stuttering of errors when they
get presented to the user.

This patch tries to cleanup a bunch of these errors.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-10-30 05:34:04 -04:00
Matthew Heon 4d800a5f45 Store cgroup manager on a per-container basis
When we create a container, we assign a cgroup parent based on
the current cgroup manager in use. This parent is only usable
with the cgroup manager the container is created with, so if the
default cgroup manager is later changed or overridden, the
container will not be able to start.

To solve this, store the cgroup manager that created the
container in container configuration, so we can guarantee a
container with a systemd cgroup parent will always be started
with systemd cgroups.

Unfortunately, this is very difficult to test in CI, due to the
fact that we hard-code cgroup manager on all invocations of
Podman in CI.

Fixes #7830

Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-10-08 15:25:06 -04:00
OpenShift Merge Robot 80a2317ca2
Merge pull request #7929 from kolyshkin/nits-err
Nits
2020-10-06 10:15:04 +02:00
Kir Kolyshkin 684d0079d2 Lowercase some errors
This commit is courtesy of

```
for f in $(git ls-files *.go | grep -v ^vendor/); do \
	sed -i 's/\(errors\..*\)"Error /\1"error /' $f;
done

for f in $(git ls-files *.go | grep -v ^vendor/); do \
	sed -i 's/\(errors\..*\)"Failed to /\1"failed to /' $f;
done

```

etc.

Self-reviewed using `git diff --word-diff`, found no issues.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-10-05 15:56:44 -07:00
Kir Kolyshkin 4878dff3e2 Remove excessive error wrapping
In case os.Open[File], os.Mkdir[All], ioutil.ReadFile and the like
fails, the error message already contains the file name and the
operation that fails, so there is no need to wrap the error with
something like "open %s failed".

While at it

 - replace a few places with os.Open, ioutil.ReadAll with
   ioutil.ReadFile.

 - replace errors.Wrapf with errors.Wrap for cases where there
   are no %-style arguments.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-10-05 15:30:37 -07:00
Daniel J Walsh 348f2df0c0
Support max_size logoptions
Docker supports log-opt max_size and so does conmon (ALthough poorly).
Adding support for this allows users to at least make sure their containers
logs do not become a DOS vector.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-10-05 17:51:45 -04:00
Matthew Heon 00cca405d2 HTTP Attach: Wait until both STDIN and STDOUT finish
In the old code, there was a chance that we could return when
only one of STDIN or STDOUT had finished - this could lead to us
dropping either input to the container, or output from it, in the
case that one stream terminated early.

To resolve this, use separate channels to return STDOUT and STDIN
errors, and track which ones have returned cleanly to ensure that
we need bith in order to return from the HTTP attach function and
pass control back to the HTTP handler (which would assume we
exited cleanly and close the client's attach connection).

Signed-off-by: Matthew Heon <mheon@redhat.com>
2020-09-24 14:48:26 -04:00
Matthew Heon bbbc0655b9 Fix a bug where log-driver json-file was made no logs
When we added the None log driver, it was accidentally added in
the middle of a set of Fallthrough stanzas which all should have
led to k8s-file, so that JSON file logging accidentally caused
no logging to be selected instead of k8s-file.

Signed-off-by: Matthew Heon <mheon@redhat.com>
2020-09-23 13:28:55 -04:00
OpenShift Merge Robot 881f2dfe92
Merge pull request #7403 from QiWang19/runtime-flag
Add global options --runtime-flags
2020-09-11 11:00:11 -04:00
Akihiro Suda f82abc774a
rootless: support `podman network create` (CNI-in-slirp4netns)
Usage:
```
$ podman network create foo
$ podman run -d --name web --hostname web --network foo nginx:alpine
$ podman run --rm --network foo alpine wget -O - http://web.dns.podman
Connecting to web.dns.podman (10.88.4.6:80)
...
<h1>Welcome to nginx!</h1>
...
```

See contrib/rootless-cni-infra for the design.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-09-09 15:47:38 +09:00
Qi Wang 6b0864434a Add global options --runtime-flags
Add global options --runtime-flags for setting options to container runtime.

Signed-off-by: Qi Wang <qiwan@redhat.com>
2020-09-04 15:04:36 -04:00
Matthew Heon 2ac37f10b4 Fix up some error messages
We have a lot of 'cannot stat %s' errors in our codebase. These
are terrible and confusing and utterly useless without context.
Add some context to a few of them so we actually know what part
of the code is failing.

Signed-off-by: Matthew Heon <mheon@redhat.com>
2020-08-27 15:05:12 -04:00
Matthew Heon 2ea9dac5e1 Send HTTP Hijack headers after successful attach
Our previous flow was to perform a hijack before passing a
connection into Libpod, and then Libpod would attach to the
container's attach socket and begin forwarding traffic.

A problem emerges: we write the attach header as soon as the
attach complete. As soon as we write the header, the client
assumes that all is ready, and sends a Start request. This Start
may be processed *before* we successfully finish attaching,
causing us to lose output.

The solution is to handle hijacking inside Libpod. Unfortunately,
this requires a downright extensive refactor of the Attach and
HTTP Exec StartAndAttach code. I think the result is an
improvement in some places (a lot more errors will be handled
with a proper HTTP error code, before the hijack occurs) but
other parts, like the relocation of printing container logs, are
just *bad*. Still, we need this fixed now to get CI back into
good shape...

Fixes #7195

Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-08-27 12:50:22 -04:00
Qi Wang 57967414ae fix close fds of run --preserve-fds
Test flakes mentioned in #6987 might be caused by uncorrect closing of file descriptor.
Fix the code to close file descriptors for podman run since it may close those used by other processes.

Signed-off-by: Qi Wang <qiwan@redhat.com>
2020-07-30 15:32:39 -04:00
Daniel J Walsh a5e37ad280
Switch all references to github.com/containers/libpod -> podman
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-07-28 08:23:45 -04:00
Matthew Heon 4b784b377c Remove all instances of named return "err" from Libpod
This was inspired by https://github.com/cri-o/cri-o/pull/3934 and
much of the logic for it is contained there. However, in brief,
a named return called "err" can cause lots of code confusion and
encourages using the wrong err variable in defer statements,
which can make them work incorrectly. Using a separate name which
is not used elsewhere makes it very clear what the defer should
be doing.

As part of this, remove a large number of named returns that were
not used anywhere. Most of them were once needed, but are no
longer necessary after previous refactors (but were accidentally
retained).

Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-07-09 13:54:47 -04:00
Joseph Gooch 0b1c1ef461 Implement --sdnotify cmdline option to control sd-notify behavior
--sdnotify container|conmon|ignore
With "conmon", we send the MAINPID, and clear the NOTIFY_SOCKET so the OCI
runtime doesn't pass it into the container. We also advertise "ready" when the
OCI runtime finishes to advertise the service as ready.

With "container", we send the MAINPID, and leave the NOTIFY_SOCKET so the OCI
runtime passes it into the container for initialization, and let the container advertise further metadata.
This is the default, which is closest to the behavior podman has done in the past.

The "ignore" option removes NOTIFY_SOCKET from the environment, so neither podman nor
any child processes will talk to systemd.

This removes the need for hardcoded CID and PID files in the command line, and
the PIDFile directive, as the pid is advertised directly through sd-notify.

Signed-off-by: Joseph Gooch <mrwizard@dok.org>
2020-07-06 17:47:18 +00:00
Valentin Rothberg 8489dc4345 move go module to v2
With the advent of Podman 2.0.0 we crossed the magical barrier of go
modules.  While we were able to continue importing all packages inside
of the project, the project could not be vendored anymore from the
outside.

Move the go module to new major version and change all imports to
`github.com/containers/libpod/v2`.  The renaming of the imports
was done via `gomove` [1].

[1] https://github.com/KSubedi/gomove

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-07-06 15:50:12 +02:00
Giuseppe Scrivano 6ee5f740a4
podman: add new cgroup mode split
When running under systemd there is no need to create yet another
cgroup for the container.

With conmon-delegated the current cgroup will be split in two sub
cgroups:

- supervisor
- container

The supervisor cgroup will hold conmon and the podman process, while
the container cgroup is used by the OCI runtime (using the cgroupfs
backend).

Closes: https://github.com/containers/libpod/issues/6400

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2020-06-25 17:16:12 +02:00
Qi Wang f61a7f25a8 Add --preservefds to podman run
Add --preservefds to podman run. close https://github.com/containers/libpod/issues/6458

Signed-off-by: Qi Wang <qiwan@redhat.com>
2020-06-19 09:40:13 -04:00
Matthew Heon 0e171b7b33 Do not share container log driver for exec
When the container uses journald logging, we don't want to
automatically use the same driver for its exec sessions. If we do
we will pollute the journal (particularly in the case of
healthchecks) with large amounts of undesired logs. Instead,
force exec sessions logs to file for now; we can add a log-driver
flag later (we'll probably want to add a `podman logs` command
that reads exec session logs at the same time).

As part of this, add support for the new 'none' logs driver in
Conmon. It will be the default log driver for exec sessions, and
can be optionally selected for containers.

Great thanks to Joe Gooch (mrwizard@dok.org) for adding support
to Conmon for a null log driver, and wiring it in here.

Fixes #6555

Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-06-17 11:11:46 -04:00
Matthew Heon 9d964ffb9f Ensure Conmon is alive before waiting for exit file
This came out of a conversation with Valentin about
systemd-managed Podman. He discovered that unit files did not
properly handle cases where Conmon was dead - the ExecStopPost
`podman rm --force` line was not actually removing the container,
but interestingly, adding a `podman cleanup --rm` line would
remove it. Both of these commands do the same thing (minus the
`podman cleanup --rm` command not force-removing running
containers).

Without a running Conmon instance, the container process is still
running (assuming you killed Conmon with SIGKILL and it had no
chance to kill the container it managed), but you can still kill
the container itself with `podman stop` - Conmon is not involved,
only the OCI Runtime. (`podman rm --force` and `podman stop` use
the same code to kill the container). The problem comes when we
want to get the container's exit code - we expect Conmon to make
us an exit file, which it's obviously not going to do, being
dead. The first `podman rm` would fail because of this, but
importantly, it would (after failing to retrieve the exit code
correctly) set container status to Exited, so that the second
`podman cleanup` process would succeed.

To make sure the first `podman rm --force` succeeds, we need to
catch the case where Conmon is already dead, and instead of
waiting for an exit file that will never come, immediately set
the Stopped state and remove an error that can be caught and
handled.

Signed-off-by: Matthew Heon <mheon@redhat.com>
2020-06-08 13:48:29 -04:00
Matthew Heon e7f4e98c45 Add exit commands to exec sessions
These are required for detached exec, where they will be used to
clean up and remove exec sessions when they exit.

As part of this, move all Exec related functionality for the
Conmon OCI runtime into a separate file; the existing one was
around 2000 lines.

Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-05-20 16:11:05 -04:00
Peter Hunt 92acb3676c oci conmon: tell conmon to log container name
specifying `-n=ctr-name` tells conmon to log CONTAINER_NAME=name if the log driver is journald

add this, and a test!

also, refactor the args slice creation to not append() unnecessarily.

Signed-off-by: Peter Hunt <pehunt@redhat.com>
2020-05-20 10:07:54 -04:00
Matthew Heon ab25f70dad Drop a debug line which could print very large messages
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-05-15 14:35:10 -04:00
Matthew Heon 50ed292aee Remove duplicated exec handling code
During the initial workup of HTTP exec, I duplicated most of the
existing exec handling code so I could work on it without
breaking normal exec (and compare what I was doing to the nroaml
version). Now that it's done and working, we can switch over to
the refactored version and ditch the original, removing a lot of
duplicated code.

Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-05-14 17:32:44 -04:00
Matthew Heon 2b08359faf Fix start order for APIv2 exec start endpoint
This makes the endpoint (mostly) functional.

Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-05-14 16:56:02 -04:00
Matthew Heon 50cc56bc4a Add an initial implementation of HTTP-forwarded exec
This is heavily based off the existing exec implementation, but
does not presently share code with it, to try and ensure we don't
break anything.

Still to do:
- Add code sharing with existing exec implementation
- Wire in the frontend (exec HTTP endpoint)
- Move all exec-related code in oci_conmon_linux.go into a new
  file
- Investigate code sharing between HTTP attach and HTTP exec.

Signed-off-by: Matthew Heon <mheon@redhat.com>
2020-05-14 16:51:57 -04:00
Jhon Honce b6113e2b9e WIP V2 attach bindings and test
* Add ErrLostSync to report lost of sync when de-mux'ing stream
* Add logus.SetLevel(logrus.DebugLevel) when `go test -v` given
* Add context to debugging messages

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2020-05-13 11:49:17 -07:00
Daniel J Walsh 8173e83054
Fix errors found in coverity scan
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-05-01 13:26:50 -04:00
Brent Baude ba430bfe5e podman v2 remove bloat v2
rid ourseleves of libpod references in v2 client

Signed-off-by: Brent Baude <bbaude@redhat.com>
2020-04-16 12:04:46 -05:00
Daniel J Walsh c4ca3c71ff
Add support for selecting kvm and systemd labels
In order to better support kata containers and systemd containers
container-selinux has added new types. Podman should execute the
container with an SELinux process label to match the container type.

Traditional Container process : container_t
KVM Container Process: containre_kvm_t
PID 1 Init process: container_init_t

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-04-15 16:52:16 -04:00
Matthew Heon 71f14bd792 Improve APIv2 support for Attach
A few major fixes here:
- Support for attaching to Configured containers, to match Docker
  behavior.
- Support for stream parameter has been improved (we now properly
  handle cases where it is not set).
- Initial support for logs parameter has been added.
- Setting attach streams when the container has a terminal is now
  supported.
- Errors are properly reported once the hijack has begun.

Signed-off-by: Matthew Heon <mheon@redhat.com>
2020-04-13 14:08:01 -04:00
Brent Baude e20ecc733c refactor info
the current implementation of info, while typed, is very loosely done so.  we need stronger types for our apiv2 implmentation and bindings.

Signed-off-by: Brent Baude <bbaude@redhat.com>
2020-04-06 12:45:42 -05:00
OpenShift Merge Robot 3542700d6e
Merge pull request #5707 from adrianreber/crun-checkpoint-1
Prepare for crun checkpoint support
2020-04-03 19:56:03 +02:00
Adrian Reber 001fe983df
checkpoint: handle XDG_RUNTIME_DIR
For (almost) all commands which podman passes on to a OCI runtime
XDG_RUNTIME_DIR is set to the same value. This does not happen for the
checkpoint command.

Using crun to checkpoint a container without this change will lead to
crun using XDG_RUNTIME_DIR of the currently logged in user and so it
will not find the container Podman wants to checkpoint.

This bascially just copies a few lines from on of the other commands to
handle 'checkpoint' as all the other commands.

Thanks to Giuseppe for helping me with this.

For 'restore' it is not needed as restore goes through conmon and for
calling conmon Podman already configures XDG_RUNTIME_DIR correctly.

Signed-off-by: Adrian Reber <areber@redhat.com>
2020-04-03 18:00:57 +02:00