Commit Graph

227 Commits

Author SHA1 Message Date
Daniel J Walsh 1c4e6d8624
standardize logrus messages to upper case
Remove ERROR: Error stutter from logrus messages also.

[ NO TESTS NEEDED] This is just code cleanup.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-09-22 15:29:34 -04:00
Matthew Heon 4ecbc7caae Add a backoff and retries to retrieving exited event
There's a potential race around extremely short-running
containers and events with journald. Events may not be written
for some time (small, but appreciable) after they are received,
and as such we can fail to retrieve it if there is a sufficiently
short time between us writing the event and trying to read it.

Work around this by just retrying, with a 0.25 second delay
between retries, up to 4 times.

[NO TESTS NEEDED] because I have no idea how to reproduce this
race in CI.

Fixes #11633

Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2021-09-21 15:32:07 -04:00
Valentin Rothberg 6aa666a27c container inspect: improve error handling
Improve the error handling of `container inspect` to properly handle
when the container has been removed _between_ the lookup and the
inspect.  That will yield the correct "no such object" error message in
`inspect`.

[NO TESTS NEEDED] since I do not know have a reliable and cheap
reproducer.  It's fixing a CI flake, so there's already an indicator.

Fixes: #11392
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-09-08 13:26:29 +02:00
Paul Holzinger a55f595fe1
podman stop always cleanup
When a container is configured for auto removal podman stop should still
do cleanup, there is no guarantee the the cleanup process spawned by
conmon will be successful. Also a user expects after podman stop that
the network/mounts are cleaned up. Therefore podman stop should not return
early and instead do the cleanup and ignore errors if the container was
already removed.

[NO TESTS NEEDED] I don't know how to test this.

Fixes #11384

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-09-01 12:56:38 +02:00
cdoern d28e85741f InfraContainer Rework
InfraContainer should go through the same creation process as regular containers. This change was from the cmd level
down, involving new container CLI opts and specgen creating functions. What now happens is that both container and pod
cli options are populated in cmd and used to create a podSpecgen and a containerSpecgen. The process then goes as follows

FillOutSpecGen (infra) -> MapSpec (podOpts -> infraOpts) -> PodCreate -> MakePod -> createPodOptions -> NewPod -> CompleteSpec (infra) -> MakeContainer -> NewContainer -> newContainer -> AddInfra (to pod state)

Signed-off-by: cdoern <cdoern@redhat.com>
2021-08-26 16:05:16 -04:00
Daniel J Walsh 41f94a4dc1
Fix podman unpause,pause,kill --all to work like podman stop --all
Currently if you execute podman unpause --all, podman pause --all
Podman shows attempts to unpause containers that are not paused
and prints an error.  This PR catches this error and only prints errors if
a paused container was not able to be unpaused.

Currently if you execute podman pause --all or podman kill --all, Podman
Podman shows attempts to pause or kill containers that are not running
and prints an error.  This PR catches this error and only prints errors if
a running container was not able to be paused or killed.

Also change printing of multiple errors to go to stderr and to prefix
"Error: " in front to match the output of the last error.

Fixes: https://github.com/containers/podman/issues/11098

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-08-04 12:50:00 -04:00
openshift-ci[bot] 3a922cbc99
Merge pull request #11003 from pascomnet/f_stats
stats: add a interval parameter to cli and api stats streaming
2021-08-04 09:56:57 +00:00
Adrian Reber eb94467780
Support checkpoint/restore with pods
This adds support to checkpoint containers out of pods and restore
container into pods.

It is only possible to restore a container into a pod if it has been
checkpointed out of pod. It is also not possible to restore a non pod
container into a pod.

The main reason this does not work is the PID namespace. If a non pod
container is being restored in a pod with a shared PID namespace, at
least one process in the restored container uses PID 1 which is already
in use by the infrastructure container. If someone tries to restore
container from a pod with a shared PID namespace without a shared PID
namespace it will also fail because the resulting PID namespace will not
have a PID 1.

Signed-off-by: Adrian Reber <areber@redhat.com>
2021-07-27 16:10:44 +02:00
Valentin Rothberg 7689783ae5 exec: fix cleanup
Commit 341e6a1 made sure that all exec sessions are getting cleaned up.
But it also came with a peformance penalty.  Fix that penalty by
spawning the cleanup process to really only cleanup the exec session
without attempting to remove the container.

[NO TESTS NEEDED] since we have no means to test such performance
issues in CI.

Fixes: #10701
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-07-27 12:00:15 +02:00
Thomas Weber cdbbd79155 stats: add a interval parameter to cli and api stream mode
podman stats polled by default in a 1 sec period.
This can put quite some load on a machine if you run many containers.

The default value is now 5 seconds.
You can change this interval with a new, optional, --interval, -i cli flag.
The api request got also a interval query parameter for the same purpose.

Additionally a unused const was removed.
Api and cli will fail the request if a 0 or negative value is passed in.

Signed-off-by: Thomas Weber <towe75@googlemail.com>
2021-07-27 07:20:47 +02:00
cdoern 0f708efd8b Implemented --until flag for libpod's container logs
compat containers/logs was missing actual usage of until query param.
This led me to implement the until param for libpod's container logs as well. Added e2e tests.

Signed-off-by: cdoern <cdoern@redhat.com>
2021-07-22 10:56:56 -04:00
Valentin Rothberg 9924c57d4c podman start: remove containers configured for auto removal
Make sure that containers configured for auto removal
(e.g., via `podman create --rm`) are removed in `podman start`
if starting the container failed.

Fixes: #10935
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-07-16 10:57:40 +02:00
Paul Holzinger 8f6a0243f4
podman diff accept two images or containers
First, make podman diff accept optionally a second argument. This allows
the user to specify a second image/container to compare the first with.
If it is not set the parent layer will be used as before.

Second, podman container diff should only use containers and podman
image diff should only use images. Previously, podman container diff
would use the image when both an image and container with this name
exists.

To make this work two new parameters have been added to the api. If they
are not used the previous behaviour is used. The same applies to the
bindings.

Fixes #10649

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-07-02 17:11:56 +02:00
flouthoc 2243b60203 reset: remove external containers on podman system reset
[NO TESTS NEEDED]

Signed-off-by: flouthoc <flouthoc.git@gmail.com>
2021-06-30 09:04:08 +05:30
OpenShift Merge Robot b422a4eb49
Merge pull request #10595 from boaz0/closes_10539
Add podman-restart systemd unit file
2021-06-15 16:12:13 -04:00
Boaz Shuster 302b3084eb Restart all containers with restart-policy=always on boot
* Add podman-restart systemd unit file and add it to podman RPM package
* Fix podman start to filter all containers + unit test

Signed-off-by: Boaz Shuster <boaz.shuster.github@gmail.com>
2021-06-13 15:04:34 +03:00
Matthew Heon 341e6a1628 Always spawn a cleanup process with exec
We were previously only doing this for detached exec. I don't
know why we did that, but I don't see any reason not to extend it
to all exec sessions - it guarantees that we will always clean up
exec sessions, even if the original `podman exec` process died.

[NO TESTS NEEDED] because I don't really know how to test this
one.

Signed-off-by: Matthew Heon <mheon@redhat.com>
2021-06-10 14:16:22 -04:00
Adrian Reber 8aa5340ade
Add parameter to specify checkpoint archive compression
The checkpoint archive compression was hardcoded to `archive.Gzip`.

There have been requests to make the used compression algorithm
selectable. There was especially the request to not compress the
checkpoint archive to be able to create faster checkpoints when not
compressing it.

This also changes the default from `gzip` to `zstd`. This change should
not break anything as the restore code path automatically handles
whatever compression the user provides during restore.

Signed-off-by: Adrian Reber <areber@redhat.com>
2021-06-07 08:07:15 +02:00
Daniel J Walsh 6ca721cccd
Drop container does not exist on removal to debugf
We have race conditions where a container can be removed
by two different processes when running podman --remove rm.

It can be cleaned up in the API or by the conmon executing
podman container cleanup.

When we fail to remove a container that does not exists we should
not be printing errors or warnings, we should just debug the fact.

[NO TESTS NEEDED] Since this is a race condition it is difficult to
test.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-05-21 09:37:53 -04:00
Boaz Shuster efdc7d8465 Add restart-policy to container filters & --filter to podman start
Signed-off-by: Boaz Shuster <boaz.shuster.github@gmail.com>
2021-05-06 14:35:15 +03:00
Valentin Rothberg 0f7d54b026 migrate Podman to containers/common/libimage
Migrate the Podman code base over to `common/libimage` which replaces
`libpod/image` and a lot of glue code entirely.

Note that I tried to leave bread crumbs for changed tests.

Miscellaneous changes:

 * Some errors yield different messages which required to alter some
   tests.

 * I fixed some pre-existing issues in the code.  Others were marked as
   `//TODO`s to prevent the PR from exploding.

 * The `NamesHistory` of an image is returned as is from the storage.
   Previously, we did some filtering which I think is undesirable.
   Instead we should return the data as stored in the storage.

 * Touched handlers use the ABI interfaces where possible.

 * Local image resolution: previously Podman would match "foo" on
   "myfoo".  This behaviour has been changed and Podman will now
   only match on repository boundaries such that "foo" would match
   "my/foo" but not "myfoo".  I consider the old behaviour to be a
   bug, at the very least an exotic corner case.

 * Futhermore, "foo:none" does *not* resolve to a local image "foo"
   without tag anymore.  It's a hill I am (almost) willing to die on.

 * `image prune` prints the IDs of pruned images.  Previously, in some
   cases, the names were printed instead.  The API clearly states ID,
   so we should stick to it.

 * Compat endpoint image removal with _force_ deletes the entire not
   only the specified tag.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-05-05 11:30:12 +02:00
Boaz Shuster a726a3d79c Add --all to podman start
Signed-off-by: Boaz Shuster <boaz.shuster.github@gmail.com>
Co-authored-by: Ed Santiago <santiago@redhat.com>
2021-05-03 22:17:06 +03:00
OpenShift Merge Robot d83f49ef6b
Merge pull request #9754 from mheon/add_dep
Add --requires flag to podman run/create
2021-04-06 16:16:49 -04:00
Matthew Heon 6acd265306 Add --requires flag to podman run/create
Podman has, for a long time, had an internal concept of
dependency management, used mainly to ensure that pod infra
containers are started before any other container in the pod. We
also have the ability to recursively start these dependencies,
which we use to ensure that `podman start` on a container in a
pod will not fail because the infra container is stopped. We have
not, however, exposed these via the command line until now.

Add a `--requires` flag to `podman run` and `podman create` to
allow users to manually specify dependency containers. These
containers must be running before the container will start. Also,
make recursive starting with `podman start` default so we can
start these containers and their dependencies easily.

Fixes #9250

Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2021-04-06 14:01:31 -04:00
Jakub Guzik f7d82a1f30 Containers prune endpoint should use only prune filters
Containers endpoints for HTTP compad and libpod APIs allowed usage of list HTTP
endpoint filter funcs. Documentation in case of libpod and compat API does not allow that.
This commit aligns code with the documentation.

Signed-off-by: Jakub Guzik <jakubmguzik@gmail.com>
2021-03-30 10:19:22 +02:00
Daniel J Walsh f1eb8e8162
Removing a non existing container API should return 404
Currently we were overwrapping error returned from removal
of a non existing container.

$ podman rm bogus -f
Error: failed to evict container: "": failed to find container "bogus" in state: no container with name or ID bogus found: no such container

Removal of wraps gets us to.

./bin/podman rm bogus -f
Error: no container with name or ID "bogus" found: no such container

Finally also added quotes around container name to help make it standout
when you get an error, currently it gets lost in the error.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-03-10 04:08:29 -05:00
Valentin Rothberg e5ac28f3b9 container removal: handle already removed containers
Since commit d54478d8ea, a container's lock is released before
attempting to stop it via the OCI runtime.  This opened the window
for various kinds of race conditions.  One of them led to #9479 where
the removal+cleanup sequences of a `run --rm` session overlapped with
`rm -af`.  Make both execution paths more robust by handling the case of
an already removed container.

Fixes: #9479
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-02-23 13:02:35 +01:00
Valentin Rothberg 5dded6fae7 bump go module to v3
We missed bumping the go module, so let's do it now :)

* Automated go code with github.com/sirkon/go-imports-rename
* Manually via `vgrep podman/v2` the rest

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-02-22 09:03:51 +01:00
Daniel J Walsh 958f901431
podman kill should report rawInput not container id
Docker always reports back the users input, not the full
id, we should do the same.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-02-16 09:47:38 -05:00
Daniel J Walsh 4a9bd7a18f
When stopping a container, print rawInput
When we stop a container we are printing the full id,
this does not match Docker behaviour or the start behavior.
We should be printing the users rawInput when we successfully
stop the container.

Fixes: https://github.com/containers/podman/issues/9386

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-02-16 06:26:51 -05:00
Matej Vasek fc385806df Improve ContainerEngine.ContainerWait()
Signed-off-by: Matej Vasek <mvasek@redhat.com>
2021-02-03 21:49:09 +01:00
Matej Vasek 570e1587dd Improve container libpod.Wait*() functions
Signed-off-by: Matej Vasek <mvasek@redhat.com>
2021-02-03 21:49:09 +01:00
Daniel J Walsh 073f76c132
Switch podman stop/kill/wait handlers to use abi
Change API Handlers to use the same functions that the
local podman uses.

At the same time:

 implement remote API for --all and --ignore flags for podman stop
 implement remote API for --all flags for podman stop

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-01-29 14:04:44 -05:00
Daniel J Walsh e7df73efad
Fix handling of container remove
I found several problems with container remove

podman-remote rm --all
Was not handled

podman-remote rm --ignore
Was not handled

Return better errors when attempting to remove an --external container.
Currently we return the container does not exists, as opposed to container
is an external container that is being used.

This patch also consolidates the tunnel code to use the same code for
removing the container, as the local API, removing duplication of code
and potential problems.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-01-20 07:37:13 -05:00
Matthew Heon 997de2f8e9 Initial implementation of renaming containers
Basic theory: We remove the container, but *only from the DB*.
We leave it in c/storage, we leave the lock allocated, we leave
it running (if it is). Then we create an identical container with
an altered name, and add that back to the database. Theoretically
we now have a renamed container.

The advantage of this approach is that it doesn't just apply to
rename - we can use this to make *any* configuration change to a
container that does not alter its container ID.

Potential problems are numerous. This process is *THOROUGHLY*
non-atomic at present - if you `kill -9` Podman mid-rename things
will be in a bad place, for example. Also, we can't rename
containers that can't be removed normally - IE, containers with
dependencies (pod infra containers, for example).

The largest potential improvement will be to move the majority of
the work into the DB, with a `RecreateContainer()` method - that
will add atomicity, and let us remove the container without
worrying about depencies and similar issues.

Potential problems: long-running processes that edit the DB and
may have an older version of the configuration around. Most
notable example is `podman run --rm` - the removal command needed
to be manually edited to avoid this one. This begins to get at
the heart of me not wanting to do this in the first place...

This provides CLI and API implementations for frontend, but no
tunnel implementation. It will be added in a future release (just
held back for time now - we need this in 3.0 and are running low
on time).

This is honestly kind of horrifying, but I think it will work.

Signed-off-by: Matthew Heon <mheon@redhat.com>
2021-01-14 18:29:28 -05:00
OpenShift Merge Robot db5e7ec4c4
Merge pull request #8947 from Luap99/cleanup-code
Fix problems reported by staticcheck
2021-01-12 13:15:35 -05:00
Paul Holzinger 8452b768ec Fix problems reported by staticcheck
`staticcheck` is a golang code analysis tool. https://staticcheck.io/

This commit fixes a lot of problems found in our code. Common problems are:
- unnecessary use of fmt.Sprintf
- duplicated imports with different names
- unnecessary check that a key exists before a delete call

There are still a lot of reported problems in the test files but I have
not looked at those.

Signed-off-by: Paul Holzinger <paul.holzinger@web.de>
2021-01-12 16:11:09 +01:00
unknown 2aa381f2d0 add pre checkpoint
Signed-off-by: Zhuohan Chen <chen_zhuohan@163.com>
2021-01-10 21:38:28 +08:00
Radostin Stoyanov 288ccc4c84 Include named volumes in container migration
When migrating a container with associated volumes, the content of
these volumes should be made available on the destination machine.

This patch enables container checkpoint/restore with named volumes
by including the content of volumes in checkpoint file. On restore,
volumes associated with container are created and their content is
restored.

The --ignore-volumes option is introduced to disable this feature.

Example:

 # podman container checkpoint --export checkpoint.tar.gz <container>

The content of all volumes associated with the container are included
in `checkpoint.tar.gz`

 # podman container checkpoint --export checkpoint.tar.gz --ignore-volumes <container>

The content of volumes is not included in `checkpoint.tar.gz`. This is
useful, for example, when the checkpoint/restore is performed on the
same machine.

 # podman container restore --import checkpoint.tar.gz

The associated volumes will be created and their content will be
restored. Podman will exit with an error if volumes with the same
name already exist on the system or the content of volumes is not
included in checkpoint.tar.gz

 # podman container restore --ignore-volumes --import checkpoint.tar.gz

Volumes associated with container must already exist. Podman will not
create them or restore their content.

Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2021-01-07 07:51:19 +00:00
Radostin Stoyanov 2b35876c8d Use Options as CRImportCheckpoint() argument
Instead of specifying restore option arguments individually from
RestoreOptions, provide the 'options' object to the CRImportCheckpoint
method. This change makes the code in CRImportCheckpoint easier to
extend as it doesn't require excessive number of function parameters.

Signed-off-by: Radostin Stoyanov <rstoyanov@fedoraproject.org>
2021-01-07 07:48:41 +00:00
Baron Lenardson b90f7f9095 Rework pruning to report reclaimed space
This change adds code to report the reclaimed space after a prune.
Reclaimed space from volumes, images, and containers is recorded
during the prune call in a PruneReport struct. These structs are
collected into a slice during a system prune and processed afterwards
to calculate the total reclaimed space.

Closes #8658

Signed-off-by: Baron Lenardson <lenardson.baron@gmail.com>
2020-12-30 19:57:35 -06:00
Baron Lenardson 76afb50f3a Consolidate filter logic to pkg subdirectory
Per the conversation on pull/8724 I am consolidating filter logic
and helper functions under the pkg/domain/filters dir.

Signed-off-by: Baron Lenardson <lenardson.baron@gmail.com>
2020-12-24 20:27:41 +00:00
baude c81e29525b add --cidfile to container kill
Add the ability to read container ids from one or more files for the
kill command.

Fixes: #8443

Signed-off-by: baude <bbaude@redhat.com>
2020-12-23 12:01:46 -06:00
Paul Holzinger ba545c49a2 podman logs honor stderr correctly
Make the ContainerLogsOptions support two io.Writers,
one for stdout and the other for stderr. The logline already
includes the information to which Writer it has to be written.

Signed-off-by: Paul Holzinger <paul.holzinger@web.de>
2020-12-10 22:24:43 +01:00
Daniel J Walsh dc8996ec84
Allow containers to --restart on-failure with --rm
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-11-20 13:55:19 -05:00
Paul Holzinger 4f427a89cb Align the podman ps --filter behavior with docker
All of our filters worked exclusive resulting in `--filter status=created --filter status=exited` to return nothing.

In docker filters with the same key work inclusive with the only exception being `label` which is exclusive. Filters with different keys always work exclusive.

This PR aims to match the docker behavior with podman.

Signed-off-by: Paul Holzinger <paul.holzinger@web.de>
2020-11-18 11:36:06 +01:00
Daniel J Walsh 6ca705bf1a
Add support for mounting external containers
Continue progress on use of external containers.

This PR adds the ability to mount, umount and list the
storage containers whether they are in libpod or not.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-11-04 13:52:08 -05:00
Daniel J Walsh 32af1be01a
The cidfile should be created when the container is created
Currently if you run an interactive session of podman run and
specifiy the --cidfile option, the cidfile will not get created
until the container finishes running.  If you run a detached
container, it will get created right away.  This Patch creates
the cidfile as soon as the container is created.  This could allow
other tools to use the cidefile on all running containers.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-10-26 05:53:26 -04:00
Daniel J Walsh db23e12611
Add support for external container
External containers are containers created outside of Podman.
For example Buildah and CRI-O Containers.

$ buildah from alpine
alpine-working-container
$ buildah run alpine-working-container touch /test
$ podman container exists --external alpine-working-container

$ podman container diff alpine-working-container
C /etc
A /test

Added --external flag to refer to external containers, rather then --storage.

Added --external for podman container exists and modified podman ps to use
--external rather then --storage.  It was felt that --storage would confuse
the user into thinking about changing the storage driver or options.

--storage is still supported through the use of aliases.

Finally podman contianer diff, does not require the --external flag, since it
there is little change of users making the mistake, and would just be a pain
for the user to remember the flag.

podman container exists --external is required because it could fool scripts
that rely on the existance of a Podman container, and there is a potential
for a partial deletion of a container, which could mess up existing users.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-10-15 09:51:31 -04:00
Daniel J Walsh 22c8270135
fix podman container exists and diff for storage containers
Current these commands only check if a container exists in libpod. With
this fix, the commands will also check if they are in containers/storage.

This allows users to look at differences within a buildah or CRI-O container.

Currently buildah diff does not exists, so this helps out in that situation
as well as in CRI-O since the cri does not implement a diff command.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-10-15 09:51:15 -04:00
OpenShift Merge Robot 3e2d7a992a
Merge pull request #7955 from ParkerVR/7751-alt
Container detach newlines
2020-10-13 12:31:14 -04:00
OpenShift Merge Robot fa01b838a6
Merge pull request #7891 from rhatdan/rm
This PR allows users to remove external containers directly
2020-10-09 10:55:15 -04:00
Daniel J Walsh fea78d5530
This PR allows users to remove external containers directly
Currenly if a user specifies the name or ID of an external storage
container, we report an error to them.

buildah from scratch
working-container-2
podman rm working-container-2
Error: no container with name or ID working-container-2 found: no such container

Since the user specified the correct name and the container is in storage we
force them to specify --storage to remove it. This is a bad experience for the
user.

This change will just remove the container from storage.  If the container
is known by libpod, it will remove the container from libpod as well.

The podman rm --storage option has been deprecated, and removed from docs.

Also cleaned documented options that are not available to podman-remote.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-10-09 06:06:08 -04:00
Parker Van Roy ec93468f3a newlines on all container detaches
Signed-off-by: Parker Van Roy <pvanroy@redhat.com>
2020-10-07 15:35:45 -04:00
Matthew Heon 2bb2425704 Move pod jobs to parallel execution
Make Podman pod operations that do not involve starting
containers (which needs to be done in a specific order) use the
same parallel operation code we use to make `podman stop` on
large numbers of containers fast. We were previously stopping
containers in a pod serially, which could take up to the timeout
(default 15 seconds) for each container - stopping 100 containers
that do not respond to SIGTERM would take 25 minutes.

To do this, refactor the parallel operation code a bit to remove
its dependency on libpod (damn circular import restrictions...)
and use parallel functions that just re-use the standard
container API operations - maximizes code reuse (previously each
pod handler had a separate implementation of the container
function it performed).

This is a bit of a palate cleanser after fighting CI for two
days - nice to be able to return to a land of sanity.

Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-10-07 10:00:11 -04:00
Kir Kolyshkin 684d0079d2 Lowercase some errors
This commit is courtesy of

```
for f in $(git ls-files *.go | grep -v ^vendor/); do \
	sed -i 's/\(errors\..*\)"Error /\1"error /' $f;
done

for f in $(git ls-files *.go | grep -v ^vendor/); do \
	sed -i 's/\(errors\..*\)"Failed to /\1"failed to /' $f;
done

```

etc.

Self-reviewed using `git diff --word-diff`, found no issues.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-10-05 15:56:44 -07:00
Kir Kolyshkin 4878dff3e2 Remove excessive error wrapping
In case os.Open[File], os.Mkdir[All], ioutil.ReadFile and the like
fails, the error message already contains the file name and the
operation that fails, so there is no need to wrap the error with
something like "open %s failed".

While at it

 - replace a few places with os.Open, ioutil.ReadAll with
   ioutil.ReadFile.

 - replace errors.Wrapf with errors.Wrap for cases where there
   are no %-style arguments.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-10-05 15:30:37 -07:00
Valentin Rothberg 19b955f099 stats: break out CLI options
Have a clear separation of concerns for the CLI-only options (and their
logic) from the backend.  The backend logic is now easier to understand
(e.g., `stream` instead of `noStream`).

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-09-24 14:28:10 +02:00
Valentin Rothberg 762b787fbf new endpoint: /libpod/containers/stats
Add a new endpoint for container stats allowing for batch operations on
more than one container.  The new endpoint deprecates the
single-container endpoint which will eventually be removed with the next
major release.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-09-24 14:09:55 +02:00
Valentin Rothberg 376ba349bf stats refactor
Refactor the entities' stats API to simplify using it and reduce the
risk of running into concurrency issues at the call sites.  Further
simplify the stats code by de-spaghetti-ing the logic and reducing
duplicate code.

`ContainerStats` now returns a data channel and an error.  If the error
is nil, callers can read from the channel.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-09-23 15:39:25 +02:00
Daniel J Walsh 3a7faf7cf8
Fix podman ps -l
Currently the --latest flag is ignored by podman ps command.

Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1877182

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-09-09 08:07:40 -04:00
Valentin Rothberg d3e05de156 podman stop: do not cleanup for auto-removal
Do not perform a container clean up for containers configured for
auto-removal (e.g., via `podman run --rm`).  There is a small race
window with the other process performing the removal where a clean up
during podman-stop may fail since the container has already been removed
and cleaned up.  As the removing process will clean up the container,
we don't have to do it during podman-stop.

Fixes: #7384
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-09-08 12:22:53 +02:00
Daniel J Walsh a5e37ad280
Switch all references to github.com/containers/libpod -> podman
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-07-28 08:23:45 -04:00
Valentin Rothberg 09dc77aedf log API: add context to allow for cancelling
Add a `context.Context` to the log APIs to allow for cancelling
streaming (e.g., via `podman logs -f`).  This fixes issues for
the remote API where some go routines of the server will continue
writing and produce nothing but heat and waste CPU cycles.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-07-09 15:13:07 +02:00
Valentin Rothberg 8489dc4345 move go module to v2
With the advent of Podman 2.0.0 we crossed the magical barrier of go
modules.  While we were able to continue importing all packages inside
of the project, the project could not be vendored anymore from the
outside.

Move the go module to new major version and change all imports to
`github.com/containers/libpod/v2`.  The renaming of the imports
was done via `gomove` [1].

[1] https://github.com/KSubedi/gomove

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-07-06 15:50:12 +02:00
Matthew Heon 9e4cf6ca51 Fix `system service` panic from early hangup in events
We weren't actually halting the goroutine that sent events, so it
would continue sending even when the channel closed (the most
notable cause being early hangup - e.g. Control-c on a curl
session). Use a context to cancel the events goroutine and stop
sending events.

Fixes #6805

Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-07-02 09:57:39 -04:00
Matthew Heon b20619e5b0 Allow recursive dependency start with Init()
As part of APIv2 Attach, we need to be able to attach to freshly
created containers (in ContainerStateConfigured). This isn't
something Libpod is interested in supporting, so we use Init() to
get the container into ContainerStateCreated, in which attach is
possible. Problem: Init() will fail if dependencies are not
started, so a fresh container in a fresh pod will fail. The
simplest solution is to extend the existing recursive start code
from Start() to Init(), allowing dependency containers to be
started when we initialize the container (optionally, controlled
via bool).

Also, update some comments in container_api.go to make it more
clear how some of our major API calls work.

Fixes #6646

Signed-off-by: Matthew Heon <mheon@redhat.com>
2020-06-18 09:34:04 -04:00
OpenShift Merge Robot 78ad57ebd3
Merge pull request #6636 from mheon/add_warnings
Re-add resource limit warnings to Specgen
2020-06-17 16:51:51 +02:00
Matthew Heon c51c593ff6 Re-add resource limit warnings to Specgen
These were part of Podman v1.9, but were lost in the transition
to using Specgen to create containers. Most resource limits are
checked via the sysinfo package to ensure they are safe to use
(the cgroup is mounted, kernel support is present, etc) and
removed if not safe. Further, bounds checks are performed to
ensure that values are valid.

Ensure these warnings are printed client-side when they occur.
This part is a little bit gross, as it happens in pkg/infra and
not cmd/podman, which is largely down to how we implemented
`podman run` - all the work is done in pkg/infra and it returns
only once the container has exited, and we need warnings to print
*before* the container runs. The solution here, while inelegant,
avoid the need to extensively refactor our handling of run.

Should fix blkio-limit warnings that were identified by the FCOS
test suite.

Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-06-16 17:32:01 -04:00
Matthew Heon 6589d75565 Fix podman inspect on overlapping/missing objects
This started as a small fix to `podman inspect` where a container
and image, with the same name/tag, were present, and
`podman inspect` was run on that name. `podman inspect` in 1.9
(and `docker inspect`) will give you the container; in v2.0, we
gave the image. This was an easy fix (just reorder how we check
for image/container).

Unfortunately, in the process of testing this fix, I determined
that we regressed in a different area. When you run inspect on
a number of containers, some of which do not exist,
`podman inspect` should return an array of inspect results for
the objects that exist, then print a number of errors, one for
each object that could not be found. We were bailing after the
first error, and not printing output for the containers that
succeeded. (For reference, this applied to images as well). This
required a much more substantial set of changes to properly
handle - signatures for the inspect functions in ContainerEngine
and ImageEngine, plus the implementations of these interfaces,
plus the actual inspect frontend code needed to be adjusted to
use this.

Fixes #6556

Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-06-16 14:33:33 -04:00
Daniel J Walsh 200cfa41a4
Turn on More linters
- misspell
    - prealloc
    - unparam
    - nakedret

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-06-15 07:05:56 -04:00
Daniel J Walsh 87718c4e67
Fix Id->ID where possible for lint
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-06-10 05:49:41 -04:00
Daniel J Walsh 4bb43b898d
Fixup issues found by golint
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-06-10 05:49:41 -04:00
Matthew Heon 1cc9731dfa Add parallel operation to `podman stop`
This is the other command that benefits greatly from being run in
parallel, due to the potential 15-second timeout for containers
that ignore SIGTERM.

While we're at it, also clean up how stop timeout is set. This
needs to be an optional parameter, so that the value set when the
container is created with `--stop-timeout` will be respected.

Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-06-05 16:18:53 -04:00
Matthew Heon 89a1e7db39 Add parallel execution code for container operations
This code will run container operations in parallel, up to a
given maximum number of threads. Currently, it has only been
enabled for local `podman rm` as a proof of concept.

Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-06-05 11:31:05 -04:00
Matthew Heon 61ea85bddb When stopping containers locally, ensure cleanup runs
The cleanup process was already running and ensuring that mounts
and networking configuration was cleaned up on container stop,
but this was async from the actual `podman stop` command which
breaks some expectations - the container is still mounted at the
end of `podman stop` and will be cleaned up soon, but not
immediately. Fortunately, it's a trivial change to resolve this.

Fixes #5747

Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-06-03 15:16:24 -04:00
OpenShift Merge Robot df0141dc20
Merge pull request #6473 from mheon/fix_inspect_segfault
Fix a segfault in `podman inspect -l` w/ no containers
2020-06-03 19:27:54 +02:00
Matthew Heon 4b37d4d5af Fix a segfault in `podman inspect -l` w/ no containers
We also need to rework container/image inspect to be separate,
but that can happen in another PR.

Fixes #6472

Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-06-02 17:22:00 -04:00
Matthew Heon e0d9404634 Enable detached exec for remote
The biggest obstacle here was cleanup - we needed a way to remove
detached exec sessions after they exited, but there's no way to
tell if an exec session will be attached or detached when it's
created, and that's when we must add the exit command that would
do the removal. The solution was adding a delay to the exit
command (5 minutes), which gives sufficient time for attached
exec sessions to retrieve the exit code of the session after it
exits, but still guarantees that they will be removed, even for
detached sessions. This requires Conmon 2.0.17, which has the new
`--exit-delay` flag.

As part of the exit command rework, we can drop the hack we were
using to clean up exec sessions (remove them as part of inspect).
This is a lot cleaner, and I'm a lot happier about it.

Otherwise, this is just plumbing - we need a bindings call for
detached exec, and that needed to be added to the tunnel mode
backend for entities.

Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-06-02 15:30:42 -04:00
Valentin Rothberg 3cef598fc2 container stats: fix --no-stream race
Fix a race in `podman container stats` by waiting for the client to
consume the data in the channel.  This requires a `sync.WaitGroup` (or
semaphore) in the client and to also close the channel the backend.

Fixes: #6405
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-05-27 13:51:48 +02:00
Matthew Heon 6b9e9610d8 Enable cleanup processes for detached exec
The cleanup command creation logic is made public as part of this
and wired such that we can call it both within SpecGen (to make
container exit commands) and from the ABI detached exec handler.
Exit commands are presently only used for detached exec, but
theoretically could be turned on for all exec sessions if we
wanted (I'm declining to do this because of potential overhead).

I also forgot to copy the exit command from the exec config into
the ExecOptions struct used by the OCI runtime, so it was not
being added.

There are also two significant bugfixes for exec in here. One is
for updating the status of running exec sessions - this was
always failing as I had coded it to remove the exit file *before*
reading it, instead of after (oops). The second was that removing
a running exec session would always fail because I inverted the
check to see if it was running.

Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-05-20 16:11:05 -04:00
Matthew Heon 5ec56dc790 Add ability to clean up exec sessions with cleanup
We need to be able to use cleanup processes to remove exec
sessions as part of detached exec. This PR adds that ability. A
new flag is added to `podman container cleanup`, `--exec`, to
specify an exec session to be cleaned up.

As part of this, ensure that `ExecCleanup` can clean up exec
sessions that were running, but have since exited. This ensures
that we can come back to an exec session that was running but has
since stopped, and clean it up.

Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-05-20 16:11:05 -04:00
Matthew Heon 05a034118f Add CLI frontend for detached exec
Add a new ContainerEngine method for creating a detached exec
session, and wire in the frontend code to do this. As part of
this, move Streams out of ExecOptions to the function signature
in an effort to share the struct between both methods.

Fixes #5884

Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-05-20 16:11:05 -04:00
Adrian Reber 7c7f4dbb51
Fix checkpoint --leave-running
There was typo in the variable name and in one place it was not
correctly passed to the next layer.

Signed-off-by: Adrian Reber <areber@redhat.com>
2020-05-14 14:29:04 +02:00
OpenShift Merge Robot 9db97dbab8
Merge pull request #6080 from baude/v2stats
v2 podman stats
2020-05-05 17:29:56 +02:00
baude b5a235df90 v2 podman stats
Signed-off-by: baude <bbaude@redhat.com>
2020-05-05 08:46:51 -05:00
OpenShift Merge Robot bffd3f5134
Merge pull request #6060 from sujil02/systemprune-v2
And system prune feature for v2.
2020-05-01 21:08:50 +02:00
OpenShift Merge Robot 95b9b72c0c
Merge pull request #6062 from jwhonce/wip/docs
[CI:DOC] Bring README.md up to date
2020-05-01 20:59:05 +02:00
Jhon Honce b22a39b6b1 [CI:DOCS] Bring README.md up to date
* Add notes on helper functions
* Update example

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2020-05-01 11:22:39 -07:00
Sujil02 b94862171b And system prune feature for v2.
Adds podman system prune for v2.
Refactoring for code reuse from pods containers images and volume prune.
Adds and enables testcases to support the added feature.

Signed-off-by: Sujil02 <sushah@redhat.com>
2020-05-01 13:57:16 -04:00
OpenShift Merge Robot 1230499e45
Merge pull request #6016 from giuseppe/fix-create
v2, podman: fix create and entrypoint tests
2020-05-01 15:32:00 +02:00
Giuseppe Scrivano 9d373408b5
podman, start: propagate back the raw input
this is necessary as we expect "podman start $ID_NAME" to print the
same arguments the user passed in instead of the full ID.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2020-04-30 12:23:19 +02:00
Qi Wang 75eeb40f41 testv2: enable attach test
testv2: enable attach test

Signed-off-by: Qi Wang <qiwan@redhat.com>
2020-04-29 23:47:15 -04:00
Brent Baude 517bc28360 system tests must pass
Signed-off-by: Brent Baude <bbaude@redhat.com>
2020-04-28 15:17:00 -05:00
OpenShift Merge Robot f6f7172494
Merge pull request #6000 from mheon/volume_backend_flags
Add support for volumes-from, image volumes, init
2020-04-27 21:53:37 +02:00
Matthew Heon 02671a103f Add support for volumes-from, image volumes, init
This should complete Podmanv2's support for volume-related flags.
Most code was sourced from the old pkg/spec implementation with
modifications to account for the split between frontend flags
(volume, mount, tmpfs) and the backend flags implemented here.

Also enables tests for podman run with volumes

Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2020-04-27 13:13:21 -04:00
Brent Baude a7b3310db5 enable integration tests for restart
enable integrations tests for container restart.  fixed bug where --running was not being honored.

Signed-off-by: Brent Baude <bbaude@redhat.com>
2020-04-27 09:08:27 -05:00
Giuseppe Scrivano bd7cad775a
containers, init: skip invalid state errors with --all
reintroduce the same check that exists in v1.9.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2020-04-24 17:33:27 +02:00
Jhon Honce 565f93531e V2 restore libpod.Shutdown() when exiting podman commands
Signed-off-by: Jhon Honce <jhonce@redhat.com>
2020-04-22 14:25:40 -07:00
Brent Baude ae5e7e7e78 v2podman port
add port command to podman.

Signed-off-by: Brent Baude <bbaude@redhat.com>
2020-04-21 08:33:15 -05:00
Valentin Rothberg 09dc701097 podman rmi: refactor logic
While this commit was initially meant to fix #5847, it has turned into a
bigger refactoring which I did not manage to break into smaller pieces:

 * Fix #5847 by refactoring the image-removal logic.

 * Make the api handler for image-removal use the ABI code. This way,
   both (i.e., ABI and Tunnel) end up using the same code.  Achieving
   this code share required to move some code around to prevent circular
   dependencies.

 * Everything in pkg/api (excluding pkg/api/types) must now only be
   accessed from code using `ABISupport`.

 * Avoid imports from entities on handlers to prevent circular
   dependencies.

 * Move `podman system service` logic into `cmd` to prevent circular
   dependencies - it depends on pkg/api.

 * Also remove the build header from infra/abi files.  It will otherwise
   confuse swagger and other tools; errors we cannot fix as go doesn't
   expose a build-tag env variable.

Fixes: #5847
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-04-21 08:12:25 +02:00
Brent Baude 44a8cf8676 Fixes for system tests
Various fixes to protect against regressions in system tests

Signed-off-by: Brent Baude <bbaude@redhat.com>
2020-04-17 13:22:06 -05:00
Jhon Honce 1d93d21254 V2 Enable rootless
* Enable running podman V2 rootless
* Fixed cobra.PersistentPreRunE usage in all the commands
* Leveraged cobra.PersistentPreRunE/cobra.PersistentPostRunE to manage:
  * rootless
  * trace (--trace)
  * profiling (--cpu-profile)
  * initializing the registry copies of Image/Container engines
* Help and Usage templates autoset for all sub-commands

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2020-04-16 11:08:08 -07:00
Brent Baude 30d2964ff8 v2 bloat pruning phase 2
this is second phase of removing unneeded bloat in the remote client. this is important to be able to reduce the client size as well as possible native compilation for windows/mac.

Signed-off-by: Brent Baude <bbaude@redhat.com>
2020-04-15 16:09:27 -05:00
Sujil02 ec4060aef6 Ability to prune container in api V2
Adds ability to prune containers for v2.
Adds client side prompt with force flag and filters options to prune.

Signed-off-by: Sujil02 <sushah@redhat.com>
2020-04-15 11:17:33 -04:00
Jhon Honce d517276426 Add support for the global flags and config files
Note: This PR doesn't provide full rootless support that will be
      addressed in a future PR

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2020-04-14 14:48:37 -07:00
Brent Baude d625aef0c5 podmanv2 mount and umount
add the ability to mount and unmount containers for the local client only

Signed-off-by: Brent Baude <bbaude@redhat.com>
2020-04-14 08:32:14 -05:00
Brent Baude d8d1aa49d2 v2podman add container init
add the ability to init a container both local and remote

Signed-off-by: Brent Baude <bbaude@redhat.com>
2020-04-13 19:51:20 -05:00
Brent Baude 7d0e0a7129 v2podman container cleanup
add the ability to clean up after a container has attempted to run.  this is also important for podman run --rm --rmi.

also included are fixes and tweaks to various code bits to correct regressions on output.

Signed-off-by: Brent Baude <bbaude@redhat.com>
2020-04-09 16:59:33 -05:00
Valentin Rothberg 7a3bfbf076 podmanV2: implement logs
Implement the `podman {container} logs` for the v2 client. The remote
client does not yet support it.  There's some more work needed for the
rest api; some options are missing (e.g., printing names) while others
are broken (e.g., the until http parameter).

The remote parts will be tackled in a future change.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-04-09 11:55:10 +02:00
Jhon Honce 8ae28a55ac V2 podman diff(changes) support
* Ported CLI command
* Added API endpoint
* Added bindings
* Updated swagger (TODO: n endpoints, one handler)

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2020-04-07 09:39:46 -07:00
Brent Baude 1bfb96b540 v2podman run
add the ability to run a container

Signed-off-by: Brent Baude <bbaude@redhat.com>
2020-04-06 15:23:04 -05:00
Brent Baude b22254dc8b podmanv2 ps
add the ability to list containers

Signed-off-by: Brent Baude <bbaude@redhat.com>
2020-04-06 12:12:59 -05:00
Brent Baude 5e0b449736 podmanv2 start
add the ability to start containers

Signed-off-by: Brent Baude <bbaude@redhat.com>
2020-04-06 09:26:11 -05:00
Brent Baude 4d895dcb54 v2podman attach and exec
add the ability to attach to a running container.  the tunnel side of this is not enabled yet as we have work on the endpoints and plumbing to do yet.

add the ability to exec a command in a running container.  the tunnel side is also being deferred for same reason.

Signed-off-by: Brent Baude <bbaude@redhat.com>
2020-04-05 15:54:51 -05:00
Brent Baude 6514a5c80e v2podman container create
create a container in podmanv2 using specgen approach.  this is the core implementation and still has quite a bit of code commented out specifically around volumes, devices, and namespaces.  need contributions from smes on these parts.

Signed-off-by: Brent Baude <bbaude@redhat.com>
2020-04-03 15:43:03 -05:00
Brent Baude 8a16674722 podmanv2 checkpoint and restore
add the ability to checkpoint and restore containers on v2podman

Signed-off-by: Brent Baude <bbaude@redhat.com>
2020-04-03 09:32:06 -05:00
Brent Baude bb39051616 podmanv2 export
add ability to export a container to a tarball

Signed-off-by: Brent Baude <bbaude@redhat.com>
2020-04-01 16:02:53 -05:00
Jhon Honce 46e3b2efb8 V2 podman inspect
* Expose podman container inspect
* Expose podman image inspect

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2020-04-01 08:07:43 -07:00
Brent Baude 3ff1583814 podmanv2 commit
add commit of a container to a container-image

Signed-off-by: Brent Baude <bbaude@redhat.com>
2020-03-28 10:11:50 -05:00
Valentin Rothberg 500a2d508b podmanV2: implement top
Implement the `top` command for podmanV2.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-03-27 09:14:01 +01:00
Brent Baude 2fa78938a9 podmanv2 container inspect
add ability to inspect a container

Signed-off-by: Brent Baude <bbaude@redhat.com>
2020-03-26 15:54:26 -05:00
Brent Baude ae614920bf podmanv2 volumes
add volume commands: create, inspect, ls, prune, and rm

Signed-off-by: Brent Baude <bbaude@redhat.com>
2020-03-24 16:03:49 -05:00
Brent Baude 9536560b4f podmanv2 add core container commands
add core container commands for podmanv2: kill, pause, restart, rm, stop, unpause

Signed-off-by: Brent Baude <bbaude@redhat.com>
2020-03-22 13:24:45 -05:00
Brent Baude c81e065149 podmanv2 enable remote wait
enable remote container wait with condition

Signed-off-by: Brent Baude <bbaude@redhat.com>
2020-03-20 14:33:33 -05:00
Brent Baude baf3a9b3a7 use boolreport for containerexists response
in the case of exists, use a boolreport structure so that responses can be consistent pointer and error

Signed-off-by: Brent Baude <bbaude@redhat.com>
2020-03-20 13:01:24 -05:00
Brent Baude 87293028e6 podmanv2 container exists|wait
enable container exists and wait for podmanv2

Signed-off-by: Brent Baude <bbaude@redhat.com>
2020-03-20 12:54:45 -05:00