To prevent the podman remote client from using libimage which causes a
lot of bloat due the c/image and c/storage dependencies add the
`!remote` tag.
This will cause a hard compile time failure if the remote client ends up
pulling in libimage.
Fixes#1702
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Move the platform code into its own directory
under libimage so it can be used by farm build in
podman without causing the binary size to increase a lot.
Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
We don't need a reference.NamedTagged now.
That also makes the namedTagged variable in the caller more local.
Should not change behavior.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Right now that's not simpler, but it will enable simplification of the caller.
Should not change behavior.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
... so that early exits are colocated.
Should not change behavior, reference.TrimNamed() updating
"name" should not change the IsShortName value.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Scary features should have scary names. Also add a comment
to make it less likely that this semantics will spread.
Should not change behavior.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
When looking up an image by digest, make sure that the entire repository
of the specified value is considered. Previously, both the repository
and the tag have been ignored and we looked for _some_ image with a
matching digest.
As outlined in #1248, Docker stopped ignoring the repository with
version v20.10.20 (Oct '22) which is a compelling reason to do the same.
To be clear, previously `something@digest` would look for any image with
`digest` while `something` is entirely ignored. With this change, both
`something` and `digest` must match the image.
This change breaks two e2e tests in Podman CI which relied on the
previous behavior. There is a risk of breaking users but there is a
strong security argument to perform this change: if the repository does
not match the (previously) returned issue, there is a fair chance of a
user error.
Fixes: containers/common#1248
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Computing the layer tree requires listing all images. Certain code
paths have all images at hand already, so let's optimize a bit to
avoid listing them redundantly.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Checking whether an image is dangling and finding a parent image
requires building a layer tree. Computing a layer tree is expensive, so
add options to `ListImages` to pre-compute the dangling and parent
information ahead of time; that requires 1 layer tree instead of N.
Context: containers/podman/issues/17828
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
`github.com/pkg/errors` is deprecated since quite some time so we now
use the native error wrapping for more idiomatic golang.
Signed-off-by: Sascha Grunert <sgrunert@redhat.com>
Use containerd's platform package for platform checks. While the OCI
image spec requires the platform values to conform with GOOS and GOARCH
definitions of Go' runtime package, the values of uname are used by
convention. Supporting these values silences annoying false-positive
warnings.
Fixes: #containers/podman/issues/14669
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Do not check the platform of an image when it was looked up by it's ID.
In that case we must assume that the user/tool knows what they are
doing.
Further make the warnings configurable via a new `PlatformPolicy` field
in the `ImageLookupOptions`. By default, the error will only be printed
on the debug logs. User can opt-in to display the error on the warning
level. Not all code paths should warn. For instance, when inspecting
an image. This way, consumers of libimage can opt-in. The policy can
later on be extended to error out instead of logging.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Check the platform when looking up images locally. When the user
requested a custom platform and a local image doesn't match, the
image will be discarded. Otherwise a warning will be emitted.
Also refactor the code to make it more maintainable in the future.
Fixes: containers/podman/issues/12682
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Warn when the platform of a pulled image does not match the
user-specified platform. The checks are only performed if the user
requested a custom platform.
Do not error out and warn only since there are many images in the wild
that claim to be of another architecture. An error would break existing
workloads; we did that once and had to revert immediately.
Fixes: containers/podman/issues/14293
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
unparam and exportloopref already work without changes.
For revive I had to silence many naming issues. I decided to silence them
instead of changing the name because I didn't want to break any code.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
if image_copy_tmp_dir is set in containers.conf it needs to be used in
the systemcontext for BigFilesTemporaryDir value.
Fixes: https://github.com/containers/podman/issues/14091
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Error checking is important. In these two cases, though, we don't have a
way to return an error, so make it explicit that we ignore the error.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Add a field to `RemoveImages` that would ingore if a specified image
does not exist and not throw an error.
The intended use case is adding a `podman rmi --ignore` flag.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
When looking up an image by a short name that prefixes another image's
ID, the one matching the short name should be returned.
This means that we need to do a final lookup in the storage with the
specified name (without normalization) to continue matching short IDs.
Since it's common that users of libimage (e.g., Buildah) internally
refer to images by full ID, let's make sure that we check for that
first. This way, we'll match full IDs on first lookup and keep the
expected performance.
Note that a name starting with `sha2556:` must be followed by a 64-byte
hex value; something we didn't check for before.
Fixes: containers/podman/issues/12761
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
When looking up an image by digest, make sure that the matching image is
also subject to further attribute checks (e.g., manifest list checks) as
specified by the user.
Fixes: containers/podman/issues/12729
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Buildah, containers and probably other container engines are normalizing
the platform parameters to support common values. For instance, "x86_64"
is normalized to the OCI conformant "amd64".
Use the same normalization when copying images and looking up local
images. Also add some debug logs to facilitate future debugging.
Fixes: containers/podman/issues/12680
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
The same filters types should be disjoint, while
each selected filter type is required. This will allow callers to pass
multiple reference filters and if an image matches it is returned.
Other filters the image has to match all filters to be returned.
Specifying "since", "after", "before", "containers", "dangling","intermediate", "readonly", "until" more then once is an error.
[NO NEW TESTS NEEDED] I will open up a validation against buildah to
make sure this passes before merging.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Make sure to enforce the "latest" tag when looking up images in the
local storage. Also make sure that digested short-names are subject
to the extended digest lookups.
Context: containers/podman/issues/11964
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
cmd/podman needs that functionality as well, so I figured it makes sense
to break it into a separate package to reduce code clones.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Support pruning images in use by external containers (e.g., build
containers). Requires extending the containers filter, adding a
callback to libpod and a new options for image removal.
Tests will be added to Podman.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Following flag allows user to configure RemoveImages in a manner so that
remove func becomes exclusive to removing manifests when needed and not
removing underlying referenced images.
Signed-off-by: Aditya Rajan <arajan@redhat.com>
If matching images resolves to a manifest list, return manifest list
instead of resolving to image instance, if manifest list is not found
try resolving image.
Signed-off-by: Aditya Rajan <flouthoc.git@gmail.com>
Fix a bug where not all removed images were actually reported as such.
A regression test will be added to Podman.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
When it was first introduced, the blob info cache's location didn't
change from the system-wide default location when we were running in
rootless mode, so we started setting its location ourselves to avoid
triggering permissions errors when updating it.
The image library has since started taking into account that it was
running in rootless mode, but its hardwired default isn't the same as
the one we were setting, so we ended up creating a second cache file.
Stop doing that.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
When writing LookupImage, I thought that it's a good idea to always
attempt to match an image against the local (or requested) platform.
The use case I had in mind is multi-arch support:
`$ podman run image` should only match `image` if it matches the local
platform. We may have previously pulled `image` for another
architecture.
The core criteria for these checks is that images set their platform
(arch/os/variant) correctly. As it turned out that is not the case.
We recently performed a number of fixes to better support multi-arch
images and this change should put the last nail in the coffin.
Hence, entirely remove the `IgnorePlatform` option and only perform
platform matches if the arch, os or variant is specified explicitly via
the LookupImageOptions or the runtime's system context (as Buildah likes
to do it).
Note that this is a breaking change, so I need to update Buildah and
Podman.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Add a method to the libimage runtime to access (a copy of) its
types.SystemContext. That can be helpful for callers which may need to
access the system context.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
We must ignore the platform of a local image when doing lookups. Some
images set an incorrect or even invalid platform (see
containers/podman/issues/10682). Doing the lookup while ignoring the
platform checks prevents redundantly downloading the same image.
Note that this has the consequence that a `--pull-never --arch=hurz` may
chose a local image of another architecture. However, I estimate the
benefit of continuing to allow potentially invalid images higher than
not running them (and breaking workloads).
The changes required to touch the corrupted checks. I used the occasion
to make the corrupted checks a bit cheaper.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Currently if the caller does not specify the BigFilesTemporaryDir,
Podman and Buildah users expect this to default TMPDIR environment
variable or /var/tmp if not set.
Moving to libimage caused a regression in this functionality.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Allow for looking up images via customizable arch, os and variant.
This prevents `podman run --arch=xxx` from redundantly pulling down the
image if needed.
Context: containers/podman/issues/10648
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Commit 7f038138c3 introduced a regression to Exists() which would
return an error if the image does not exist.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
While various execution paths in libimage already handle corrupted
images, `(*Runtime).Exists()` did not and would list an image to exist
in the storage even if it is corrupted.
Some corruptions can only be detected when accessing the individual
data. A reliable way of accessing such data is to inspect an image.
Hence, an image will only be listed to exist if a) it has been found
and b) can be inspected. If the inspection fails, the image will be
reported to not exists but without an error. That allows for users
of libimage to properly recover and repull.
Further, add a new unit tests that forces a data corruption and
gradually recovers from it.
Podman will now behave as follows:
```
$ ./bin/podman run -d --rm nginx ls
ERRO[0000] Image nginx exists in local storage but may be corrupted: layer not known
ERRO[0000] Looking up nginx in local storage: layer not known
Resolved "nginx" as an alias (/home/vrothberg/.cache/containers/short-name-aliases.conf)
Trying to pull docker.io/library/nginx:latest...
Getting image source signatures
Copying blob 596b1d696923 skipped: already exists
Copying blob 30afc0b18f67 skipped: already exists
Copying blob febe5bd23e98 skipped: already exists
Copying blob 69692152171a skipped: already exists
Copying blob 8283eee92e2f skipped: already exists
Copying blob 351ad75a6cfa done
Copying config d1a364dc54 done
Writing manifest to image destination
Storing signatures
56b65883c3c32b67277bcc173bd9f26c27cbbdbc6d3aacf6c552be796eb7a337
```
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
For the sake of Docker compatibility, support the tag@digest notation.
In that case, the tag is stripped off the reference and the digest is
the sole source of truth.
Add a number of tests to make sure we're behaving as expected.
Context: containers/podman/issues/6721
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Recent changes in the image-lookup logic will, in many cases, yield a
check whether an image is a manifest list. This had caused a regression
in Podman's test/system/330-corrupt-images.bats system tests where we're
attempting to delete a corrupted image with a missing manifest. Since
the manifest is missing, the manifest list check fails.
To make the image lookups more tolerant towards this specific error
case, we need to ignore the error but emit a warning, similar to what
we're already doing in the parent-child checks.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Commit 724e7c92b5 fixed an issue when pushing images from of a
platform different than the current machine. That required to disable
the platform matching logic when looking up the image before pushing it.
It also required some restructuring of the code such that manifest lists
are resolved and their instances looked up.
The restructuring in turn introduced a regression when looking up bare
manifest lists. To fix the regression and keep the code simple,
introduce an internal field in the LookupImageOptions that indicates
whether we're looking up a bare manifest list or not.
Now we have clearer separation of concerns between looking up images or
manfifests and whether the looked up image needs to match the current
platform or not.
Add some unit tests exercising the manifest-list code to make sure we're
not regressing again.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
When pushing an image, make sure to ignore the platform of the image to
push exactly what the user wishes to. Add a test to make sure we're not
regressing in the future.
To preserve previous behaviour with respect to attempting to push a
manifest list, move the platform check below resolving to a manifest
list.
Fixes: #containers/podman/issues/10344
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Add tests for exercising pushing images to various transports and
attempt to pull from the destinations.
Fix an error determining the storage reference and image name when
pushing to containers-storage.
Fix a bug in `RemoveImages`: leaving `names` empty and specifying no
filters should remove *all* images.
Please note that the tests are currently not exercising pushing to a
registry. That requires a local registry but since CI is currently
running inside a container, we cannot do much just yet. Once CI runs
in another environment, I will go back and extend the tests.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Paul Holzinger
Urvashi Mohnani
Miloslav Trmač
Valentin Rothberg
Karthik Elango
Sascha Grunert
OpenShift Merge Robot
Daniel J Walsh
Kir Kolyshkin
Valentin Rothberg
Aditya Rajan
flouthoc
Nalin Dahyabhai