It turns out the restart is _not_ a stop+start but keeps certain
resources open and is subject to some timeouts that may differ across
distributions' default settings.
[NO NEW TESTS NEEDED] as I have absolutely no idea how to reliably cause
the failure/flake/race.
Also ignore ENOENTS of the CID file when removing a container which has
been identified of actually fixing #17607.
Fixes: #17607
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Return the error when restarting the unit failed during an update.
The task is correctly marked to have failed but we really need to
return the error to the user.
[NO NEW TESTS NEEDED] - The flakes in #17607 will reveal errors.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Support auto updating containers running inside pods. Similar to
containers, the systemd units need to be generated via
`podman-generate-systemd --new $POD` to generate the pod's units.
Note that auto updating a container inside a pod will restart the entire
pod. Updates of multiple containers inside a pod are batched, such that
a pod is restarted at most once. That is effectively the same mechanism
for auto updating containers in a K8s YAML via the `podman-kube@`
template or via Quadlet.
Updating a single container unit without restarting the entire pod is
not possible. The reasoning behind is that pods are created with
--exit-policy=stop which will render the pod to be stopped when auto
updating the only container inside the pod. The (reverse) dependencies
between the pod and its containers unit have been carefully selected for
robustness. Changes may entail undesired side effects or backward
incompatibilities that I am not comfortable with.
Fixes: #17181
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Motivated to have a working `make lint` on Fedora 37 (beta).
Most changes come from the new `gofmt` standards.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Auto updates using the "registry" policy require container to be created
with a fully-qualified image reference. Short names are not supported
due the ambiguity of their source registry. Initially, container
creation errored out for non FQN images but it seems that Podman has
regressed.
Fixes: #15879
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Add auto-update support to `podman kube play`. Auto-update policies can
be configured for:
* the entire pod via the `io.containers.autoupdate` annotation
* a specific container via the `io.containers.autoupdate/$name` annotation
To make use of rollbacks, the `io.containers.sdnotify` policy should be
set to `container` such that the workload running _inside_ the container
can send the READY message via the NOTIFY_SOCKET once ready. For
further details on auto updates and rollbacks, please refer to the
specific article [1].
Since auto updates and rollbacks bases on Podman's systemd integration,
the k8s YAML must be executed in the `podman-kube@` systemd template.
For further details on how to run k8s YAML in systemd via Podman, please
refer to the specific article [2].
An examplary k8s YAML may look as follows:
```YAML
apiVersion: v1
kind: Pod
metadata:
annotations:
io.containers.autoupdate: "local"
io.containers.autoupdate/b: "registry"
labels:
app: test
name: test_pod
spec:
containers:
- command:
- top
image: alpine
name: a
- command:
- top
image: alpine
name: b
```
[1] https://www.redhat.com/sysadmin/podman-auto-updates-rollbacks
[2] https://www.redhat.com/sysadmin/kubernetes-workloads-podman-systemd
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Refactor the auto-update backend to allow for updating multiple
tasks/containers per unit. This commit is merely doing the plumbing.
The actual integration comes in a following commit.
[NO NEW TESTS NEEDED] as behavior should not change and existing
tests are expected to continue to pass.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
The "image" policy has been deprecated in favor of the more precise
"registry" policy. Add a code comment to leave some breadcrumbs for
future generations.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Decompose the update logic into smaller steps (update check, update,
rollback, etc.) and move the implementation into the `task` API.
This allows to transition a task from state to state, independent of its
underlying auto-update policy.
Supporting more than one container per unit is now really close.
[NO NEW TESTS NEEDED] - should not change behavior.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
As state should be kept in a single `task`. This will allow for
separating updates from rollbacks which will be needed to support
multiple containers/tasks in a single unit.
[NO NEW TESTS NEEDED] - should not change behavior.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
It is not state needed after assembling the tasks, so remove it to keep
the task struct simpler.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
To replace redundant string scattered across the code with proper
constants. The "status" will further be useful in a future change
as it can be moved into a `task`.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Will simplify the code and speed up things as we do not consult a
container's labels multiple times.
[NO NEW TESTS NEEDED] - should not change behavior.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
This will simplify the logic and pave the way for abstracting the
auto-update policies to a certain degree that allows us to better
control _when_ the updates and rollbacks happen and will ultimately
reduce redundant code.
[NO NEW TESTS NEEDED] - should not change behavior.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
If two containers use the same image and one rolled back (i.e., tagged
the old image again), make sure to repull the image for the other
container.
Once an image has caused a rollback, it may be worth marking this image
as broken and not update any other container using it but that is
outside of the scope.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Change the auto-update logic to update unit-by-unit rather by policy.
This allows for, in theory now and in practice later, to have mutliple
containers run in a single systemd unit and update them in sequence
before restarting the unit.
[NO NEW TESTS NEEDED] - should not change behavior.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
A `task` includes data and state for updating a given container image.
It will come in handy in future changes, but we are going there in
baby steps to have smaller incremental changes.
[NO NEW TESTS NEEDED] - should not change behaviour.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Introduce an `updater` type to allow for sharing state.
This will be more useful for future changes.
[NO NEW TESTS NEEDED] as it does not change behavior.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
We now use the golang error wrapping format specifier `%w` instead of
the deprecated github.com/pkg/errors package.
[NO NEW TESTS NEEDED]
Signed-off-by: Sascha Grunert <sgrunert@redhat.com>
Create an auto-update event for each invocation, independent if images
and containers are updated or not. Those events will be indicated in
the events already but users will now know why.
Fixes: #14283
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Finalizes the linked BZ to fix passing down custom authfiles during auto
updates. Also fixes the if-newer pull policy.
[NO TESTS NEEDED] for now validated manually. There's a TODO to add a
new system test that I did not find time for before PTO.
BZ: bugzilla.redhat.com/show_bug.cgi?id=2000943
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Make sure that the container's authfile label is used when pulling down
a new image.
[NO TESTS NEEDED] since it would require some larger rewrite of the
auto-update system tests that I currently have no time for. I added a
reminder to have some breadcrumbs when there is more time.
Fixes: #11171
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Add support for simple rollbacks during `podman auto-update`. Rollbacks
are enabled by default. If a systemd unit cannot be restarted after an
update, the previous image will be retagged and the unit will be
restarted a second time.
Add system tests for rollbacks. Also fix a bug in the restart sequence;
we have to use the channel to actually know whether the restart was
successful or not.
NOTE: To make rollbacks really useful, users must run their containers
with `--sdnotify=container` such that the containers send the ready
message over the (mounted) socket. This way, restarting the systemd
units during auto update will block until the message has been received
(or a timeout kicked in).
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Add a --dry-run flag to `podman auto-update` which will look for new
images but won't perform any pull or restart any service or container.
The "UPDATED" column will now indicate the availability of a newer image
via "pending".
```
$ podman auto-update --dry-run
UNIT CONTAINER IMAGE POLICY UPDATED
container-test.service 08fd34e533fd (test) localhost:5000/busybox registry false
```
Fixes: #9949
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
The rather raw and scarce output of `podman auto-update` has been a
thorn in my eyes for a longer while. So far, Podman would only print
updated systemd units, one per line, without further formatting.
Motivated by issue #9949 which is asking for some more useful
information in combination with a dry-run feature, I sat down and
reflected which information may come in handy.
Running `podman auto-update` will now look as follows:
```
$ podman auto-update
Trying to pull [...]
UNIT CONTAINER IMAGE POLICY UPDATED
container-test.service 08fd34e533fd (test) localhost:5000/busybox registry false
```
Also refactor the spaghetti code in the backend a bit to make it easier
to digest and maintain.
For easier testing and for the sake of consistency with other commands
listing output, add a `--format` flag.
The man page will get an overhaul in a follow up commit.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Pass a single context.Context through the call stack. If auto-updates
will ever be made available for REST calls, the context will help
supporting disconnected clients.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Use the `HasDifferentDigest()` method of the image to replace the
previous checks if an image needs to be updated.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Fix a bug were an authfile label in a container would mistakenly
override the authfile path for all subsequent checks.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Migrate the Podman code base over to `common/libimage` which replaces
`libpod/image` and a lot of glue code entirely.
Note that I tried to leave bread crumbs for changed tests.
Miscellaneous changes:
* Some errors yield different messages which required to alter some
tests.
* I fixed some pre-existing issues in the code. Others were marked as
`//TODO`s to prevent the PR from exploding.
* The `NamesHistory` of an image is returned as is from the storage.
Previously, we did some filtering which I think is undesirable.
Instead we should return the data as stored in the storage.
* Touched handlers use the ABI interfaces where possible.
* Local image resolution: previously Podman would match "foo" on
"myfoo". This behaviour has been changed and Podman will now
only match on repository boundaries such that "foo" would match
"my/foo" but not "myfoo". I consider the old behaviour to be a
bug, at the very least an exotic corner case.
* Futhermore, "foo:none" does *not* resolve to a local image "foo"
without tag anymore. It's a hill I am (almost) willing to die on.
* `image prune` prints the IDs of pruned images. Previously, in some
cases, the names were printed instead. The API clearly states ID,
so we should stick to it.
* Compat endpoint image removal with _force_ deletes the entire not
only the specified tag.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
changed struct to policyMapper
change "image" to "registry" in multiple locations
Updated documentation with registry alias & autoupdate local
Added relevant test
Signed-off-by: Parker Van Roy <pvanroy@redhat.com>
Digests were used to compare local image and container image
Registry alias added for Image Policy
Refactored to integrate new feature + change some naming conventions
Tested this using a modified version of the docs autoupdate instructions & it worked successfully
Signed-off-by: Parker Van Roy <pvanroy@redhat.com>
Some packages used by the remote client imported the libpod package.
This is not wanted because it adds unnecessary bloat to the client and
also causes problems with platform specific code(linux only), see #9710.
The solution is to move the used functions/variables into extra packages
which do not import libpod.
This change shrinks the remote client size more than 6MB compared to the
current master.
[NO TESTS NEEDED]
I have no idea how to test this properly but with #9710 the cross
compile should fail.
Signed-off-by: Paul Holzinger <paul.holzinger@web.de>
We missed bumping the go module, so let's do it now :)
* Automated go code with github.com/sirkon/go-imports-rename
* Manually via `vgrep podman/v2` the rest
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
With the advent of Podman 2.0.0 we crossed the magical barrier of go
modules. While we were able to continue importing all packages inside
of the project, the project could not be vendored anymore from the
outside.
Move the go module to new major version and change all imports to
`github.com/containers/libpod/v2`. The renaming of the imports
was done via `gomove` [1].
[1] https://github.com/KSubedi/gomove
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Use the architecture of the local image when looking for a new image on
a registry. It seems to be common practice on ARM to tweak the
architecture choice to pull the correct image.
Fixes: #6613
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Support using custom authfiles for auto updates by adding a new
`--authfile` flag and passing it down into the backend.
Also do some minor fixes in the help text and the man page.
Fixes: #6159
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Fix a bug in the auto-update logic causing all images to be checked and
not only the ones of containers with the specific auto-update policy.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Add support to auto-update containers running in systemd units as
generated with `podman generate systemd --new`.
`podman auto-update` looks up containers with a specified
"io.containers.autoupdate" label (i.e., the auto-update policy).
If the label is present and set to "image", Podman reaches out to the
corresponding registry to check if the image has been updated. We
consider an image to be updated if the digest in the local storage is
different than the one of the remote image. If an image must be
updated, Podman pulls it down and restarts the container. Note that the
restarting sequence relies on systemd.
At container-creation time, Podman looks up the "PODMAN_SYSTEMD_UNIT"
environment variables and stores it verbatim in the container's label.
This variable is now set by all systemd units generated by
`podman-generate-systemd` and is set to `%n` (i.e., the name of systemd
unit starting the container). This data is then being used in the
auto-update sequence to instruct systemd (via DBUS) to restart the unit
and hence to restart the container.
Note that this implementation of auto-updates relies on systemd and
requires a fully-qualified image reference to be used to create the
container. This enforcement is necessary to know which image to
actually check and pull. If we used an image ID, we would not know
which image to check/pull anymore.
Fixes: #3575
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Valentin Rothberg
Sascha Grunert
Paul Holzinger
Valentin Rothberg
Parker Van Roy
Paul Holzinger
Milivoje Legenovic
Josh Soref
Daniel J Walsh
Ondřej Kraus