To support artifact handling, we need to export some of the methods for
copier. The Copy and Close method have been exported. In addition, a
new constructor that is exported was created so we could introduce a
different SystemContext for artifacts.
Signed-off-by: Brent Baude <bbaude@redhat.com>
Instead of passing a nil system context when adding to a manifest
list, use a valid one, ensuring that settings like auth and TLS
verification are passed along and respected.
Fixes https://github.com/containers/podman/issues/23410
Signed-off-by: Matt Heon <mheon@redhat.com>
Add libimage.Image.ConvertToManifestList(), which will convert an image
to a manifest list if it isn't already possible to use it as one.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Both slices.Clone() and maps.Clone() return `nil` when passed `nil`, so
we don't need to check for a `nil` pointer before calling them.
Imports that used the golang.org/x/exp versions of the maps and slices
packages can now use the versions in the standard library, since we
require Go 1.21.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
When AnnotateInstance() is given a non-empty Architecture value, set the
Variant value to the one that was passed in along with it, even if that
value is empty.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
When adding a reference to a manifest list or image index, if we are
guessing that it's a reference to an image in a registry, check if we
can read something from that location instead of assuming it's correct,
and if we can't, check for a local image. This will let use use local
image IDs to refer to items that we want to add to a list or index.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
* Add a libimage.ManifestList.AddArtifact() method.
* Add IndexAnnotations and Subject to libimage.ManifestListAnnotateOptions,
for setting annotations on the index itself, and the index's subject,
respectively.
* In libimage/manifests.list.AddArtifact(), if the subject has an
artifactType in its manifest, add its value to the subject descriptor.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
When listing instances in an image index, show their artifact types and
the names of any files that they're tracking.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
To prevent the podman remote client from using libimage which causes a
lot of bloat due the c/image and c/storage dependencies add the
`!remote` tag.
This will cause a hard compile time failure if the remote client ends up
pulling in libimage.
Fixes#1702
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
manifest push API must implement and leverage `retry` logic similar to
`image push` with similar defaults.
Closes: https://github.com/containers/common/issues/1664
Signed-off-by: Aditya R <arajan@redhat.com>
These structs are (for better or worse) a part of
Podman's API, so podman-remote needs to include the subpackage that
defines them - which is all of libimage (and c/image/v5/copy) right now.
Instead, move them to libimage/define.
Alternatively, maybe Podman's API should not directly depend on (unstable)
c/common types?!
Only moves unchanged code, should not change behavior.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
When looking up an image by digest, make sure that the entire repository
of the specified value is considered. Previously, both the repository
and the tag have been ignored and we looked for _some_ image with a
matching digest.
As outlined in #1248, Docker stopped ignoring the repository with
version v20.10.20 (Oct '22) which is a compelling reason to do the same.
To be clear, previously `something@digest` would look for any image with
`digest` while `something` is entirely ignored. With this change, both
`something` and `digest` must match the image.
This change breaks two e2e tests in Podman CI which relied on the
previous behavior. There is a risk of breaking users but there is a
strong security argument to perform this change: if the repository does
not match the (previously) returned issue, there is a fair chance of a
user error.
Fixes: containers/common#1248
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
This allows using Fulcio and Rekor, without having to pass around 9 options;
and the interactivity required for OIDC authentication is handled by the caller
at some higher level (possibly only once for multiple operations).
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
ManifestInspect should contain all known formats for a valid manifest
list as of now only supported formats are `OCIv1` and `Docker` so
inspect should support fields from `OCIv1` format as well. Following
commit adds a new field to inspect i.e `Annotations` from `OCIv1`.
Example output from podman
```console
podman manifest inspect test
{
"schemaVersion": 2,
"mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
"manifests": [
{
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"size": 528,
"digest": "sha256:9b2a28eb47540823042a2ba401386845089bb7b62a9637d55816132c4c3c36eb",
"platform": {
"architecture": "amd64",
"os": "linux"
},
"annotations": {
"annotationTest1": "annotationTest2"
}
}
]
}
```
Closes: https://github.com/containers/podman/issues/15069
Signed-off-by: Aditya R <arajan@redhat.com>
`podman manifest add` uses `ManifestList.Add(` but of now `Add(` does
not locks while adding instances to the list thus causing race scenarios
where storage is not reloaded and overrided by another invocation of the
command.
Following problem is solved in two steps
* Add -> LockByInstance: Acquire a fs lock by instance ID so other
invocation waits until this invocation completes its write.
* Add -> LockByInstance -> reload: Reload instance digests from storage
just after acquiring lock to make sure we are not overriding any just
written instance.
Reproducer: https://github.com/containers/podman/issues/14667#issue-1277034660
Closes: https://github.com/containers/podman/issues/14667
[NO NEW TESTS NEEDED]
[NO TESTS NEEDED]
This needes integration tests so its hard to verify race in CI.
Signed-off-by: Aditya R <arajan@redhat.com>
`github.com/pkg/errors` is deprecated since quite some time so we now
use the native error wrapping for more idiomatic golang.
Signed-off-by: Sascha Grunert <sgrunert@redhat.com>
When writing LookupImage, I thought that it's a good idea to always
attempt to match an image against the local (or requested) platform.
The use case I had in mind is multi-arch support:
`$ podman run image` should only match `image` if it matches the local
platform. We may have previously pulled `image` for another
architecture.
The core criteria for these checks is that images set their platform
(arch/os/variant) correctly. As it turned out that is not the case.
We recently performed a number of fixes to better support multi-arch
images and this change should put the last nail in the coffin.
Hence, entirely remove the `IgnorePlatform` option and only perform
platform matches if the arch, os or variant is specified explicitly via
the LookupImageOptions or the runtime's system context (as Buildah likes
to do it).
Note that this is a breaking change, so I need to update Buildah and
Podman.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Some users rely on events being written *after* the operation ran.
Hence, defer all event writes.
Context: containers/podman/issues/10812
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Commit 724e7c92b5 fixed an issue when pushing images from of a
platform different than the current machine. That required to disable
the platform matching logic when looking up the image before pushing it.
It also required some restructuring of the code such that manifest lists
are resolved and their instances looked up.
The restructuring in turn introduced a regression when looking up bare
manifest lists. To fix the regression and keep the code simple,
introduce an internal field in the LookupImageOptions that indicates
whether we're looking up a bare manifest list or not.
Now we have clearer separation of concerns between looking up images or
manfifests and whether the looked up image needs to match the current
platform or not.
Add some unit tests exercising the manifest-list code to make sure we're
not regressing again.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Add an event system to libimage. Callers can opt-in to using events by
requesting an event channel via `(*Runtime).EventChannel()`. The
returned channel has a buffer of size 100 which should be sufficient
even under high loads. But, to be on the safe side, writing an event
will time out after 2 seconds to prevent operations from blocking.
Currently, the only user of such an event system is Podman which will
need to convert the `Event` type to what's used internally in libpod.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
The following changes were not split into smaller commits since the
entire package is still work in progress and I want to keep moving:
* Various small fixes.
* The internal image cache has been removed as it's a recipe for
inconsistencies for longer running processes. This should make
libimage easier to use for CRI-O and a Podman service.
* LookupImage now returns storage.ErrUnknownImage rather than nil.
This simplifies the callers and makes sure we have a consistent
error.
* LookupImage is now able to handle manifests lists. Unless the
platform is explicitly ignored via the options, the matching
image within the manifest list is now returned. This greatly
simplifies the spec generation in Podman; no callers should have
to worry about this kind of detail.
* LookupImage has been refactored into smaller-sized and easier to
read functions.
* RemoveImages has been changed to assemble the data of removed or
untagged images. This comes in handy for pruning images. I am
heavily against having a dedicated API for pruning since the it's
really just a combination of filtering and removing images which
RemoveImages already supports. Hence these changes to satisfy
the needs of `podman image prune`.
Furthermore, it now returns an []error slice rather than a single
error. Again to make Podman happy which needs to inspect *all*
errors for setting the appropriate exit code.
* A rather large refactoring of the removal code along with very
verbose comments. Those were largely absent in the Podman code base
but there many rules and contracts embedded that I partially could
only reconstruct by manually tests and comparing to Docker.
* Add a new `containers={true,false}` filter which allows filtering
images whether they are used by containers (=true) or if no container
is using them (=false). This filter is required for pruning images
in Podman.
* `libimage/types` has been merged into `libimage`. Podman has to do
_a lot of_ massaging for the remote client already and the types
are pretty much nailed down for the remote API. Hence, I prefer to
do some translation between `libimage` types and what Podman needs
rather than splitting `libimage` in half without an obvious reason.
This way the package is self-contained allowing for an easier
navigation and maintenance.
* `libimage.PullPolicy` has been merged into `pkg/config.PullPolicy`
to have _one_ central place to deal with pull policies. The type
system in `pkg/config` sets "always" as the default unfortunately
but I think consistency is more important at that point.
* Added `CopyOptions.DirForceCompress` to enforce layer compression
when copying to a `dir` destination.
* We now use `github.com/disiqueira/gotree` for pretty printing image
trees. That greatly simplifies the code and we don't have to worry
about the logic of printing a tree. Note that trees are now always
printed top down!
* Added a new `libimage.ManifestList` type along with an API for local
lookups and performing certain operations on it to wrap around
`libimage/manifests` as previously done in `libpod/image` and other
places in Podman.
* Correct caching of `(*Image).Inspect`.
* In addition to username, password and credentials, allow for
speciying an identity token for copying images. That's needed for
Podman's remote API.
* Make image removal more tolerant toward corrupted images.
* A new "until=timestamp" filter that can be used by all APIs
supporting filtering.
* An empty string now resolves to PullPolicyMissing.
* `(*Runtime) systemContextCopy()` returns a deep copy of the runtime's
system context. Golang's shallow copies are very dangerous for long
running processes such as Podman's system service. Hence, we need to
make sure that base data is not altered over time. That adds another
external dependency but I do not see a way around that. Long term,
I desire a `(*containers/image/types.SystemContext).Copy()` function.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Brent Baude