Commit Graph

234 Commits

Author SHA1 Message Date
Ygal Blum a525f15fc3 Image copier: make sure ReportWriter is not nil before accessing
When running in Quiet mode, the ReportWriter can be nil causing a crash

Signed-off-by: Ygal Blum <ygal.blum@gmail.com>
2023-12-04 14:34:40 +02:00
Miloslav Trmač e121cae027 Fix inspect of unsigned schema1 images
... by not incorrectly invoking the schema2 code; otherwise
it tries to unmarshal nil as JSOn, and fails.

I'm not sure whether schema1 images can contain
a comment or a health check; just not crashing is
an improvement.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2023-11-20 18:58:21 +01:00
Miloslav Trmač 82250038d7 Rename dockerManifest to dockerConfig
... just because this is incorrect and confusing.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2023-11-20 18:41:35 +01:00
Paul Holzinger 92c98a7323 libimage: add !remote tag
To prevent the podman remote client from using libimage which causes a
lot of bloat due the c/image and c/storage dependencies add the
`!remote` tag.

This will cause a hard compile time failure if the remote client ends up
pulling in libimage.

Fixes #1702

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2023-10-23 15:14:43 +02:00
Ygal Blum 2c59e6864f Copier: use StringWithinTransport instead of DockerReference
When trying an invalid image DockerReference maybe nil

Signed-off-by: Ygal Blum <ygal.blum@gmail.com>
2023-10-22 11:04:10 +03:00
Paul Holzinger ec9d00f17b libimage: Untag should error for non existent name
podman untag should error out of a name is given which does not exists
for the given image. This regression was added in commit a4a7c28aef.

There was even a test which meant to check for it but unfortunately it
did not actually check for what it should. The doNotExist check failed
early to the upper case in the repo name.
The tests have been updated to check for actual error messages to show
ensure it is failing for the right reason.
This also showed that `normalizing name` message was included twice so
I removed one case to not stutter.

Fixes a4a7c28aef ("libimage: support parallel tag/untag")

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2023-10-20 12:15:45 +02:00
Paul Holzinger a4a7c28aef libimage: support parallel tag/untag
The c/storage SetNames API is depracated because it is not race free to
first get the list of names and then append our new name then write the
full list back. Instead a better Add/RemovesNames API has been added.

Tag and Untag should use these to prevent race conditions that can be
easily reproduce using podman tag in parallel. Tests have been added to
ensure it is working correctly.

Fixes https://github.com/containers/podman/issues/17515

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2023-10-19 14:51:14 +02:00
Valentin Rothberg 9d4b1bf156 libimage: fix computing history
Computing the history did not walk the layers correctly.  Fix that and
try to improve the code to make it easier to read and maintain for
future pairs of eyes.

A regression will also be added to the Podman PR vendoring this change.

Fixes: containers/podman/issues/20375
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2023-10-17 16:18:12 +02:00
Nalin Dahyabhai 3bcb56c0be libimage: add compat interfaces for the new platform subpackage
Add compatibility interfaces for items which have been moved from the
main libimage package to its new platform subpackage.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2023-10-12 16:17:14 -04:00
Urvashi Mohnani dce8f8791d Move platform code to its own directory
Move the platform code into its own directory
under libimage so it can be used by farm build in
podman without causing the binary size to increase a lot.

Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
2023-10-12 10:38:19 -04:00
Valentin Rothberg 6923f768c2 URGENT: heal CI: fix codespell
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2023-10-03 11:15:37 +02:00
OpenShift Merge Robot 12f1c98dc5 Merge pull request #1676 from umohnani8/filters
Fix ID and Digest filters
2023-09-29 08:06:46 -04:00
Aditya R 72c6636175 libimage: add retry options for manifest push
manifest push API must implement and leverage `retry` logic similar to
`image push` with similar defaults.

Closes: https://github.com/containers/common/issues/1664

Signed-off-by: Aditya R <arajan@redhat.com>
2023-09-28 22:26:38 +05:30
Urvashi Mohnani 25bcd08753 Fix ID and Digest filters
The id and digest filters for podman images was broken
as it would only match on the full ID or Digest, so when
users would give a substring of either not results would be
given even though there were matches.
Fix to check on prefix now instead.

Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
2023-09-28 12:18:08 -04:00
Aditya R 8282fed5ba manifests_test: fix tests for compression formats
Signed-off-by: Aditya R <arajan@redhat.com>
2023-09-28 12:03:48 +05:30
guoguangwu 97991b9967 chore: import packages only once
Signed-off-by: guoguangwu <guoguangwu@magic-shield.com>
2023-09-27 15:02:02 +08:00
Aditya R e08e68ee29 libimage/layer_tree: if parent is empty and manifest list ignore
Layer tree expectes to form a relation between child and parent
instances, however it expects an instance from manifest list which is
empty, following expectation is not possible and will always resuilt in
error.

Closes: https://github.com/containers/podman/issues/19860

[NO NEW TESTS NEEDED]
Image without layer cant be built in libimage, and `podman save`
automatically malforms such image so no such external image can be
loaded.

Signed-off-by: Aditya R <arajan@redhat.com>
2023-09-18 14:30:42 +05:30
Paul Holzinger 8bb073c089 libimage: split out search filters
This allows podman to import them on the remote client without needing
the full libimage.

Based on Miloslav's work: https://github.com/containers/podman/pull/19718

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2023-09-12 13:54:20 +02:00
Valentin Rothberg de32d5a9f7 linters: enable dupword
Mostly monkey work to fix comments but there was also an error message.

Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2023-09-01 11:51:25 +02:00
Valentin Rothberg ab8db10408 linters: enable whitespace
Not changing the world but useful hygiene.

Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2023-09-01 11:14:23 +02:00
Valentin Rothberg 1fa8b65281 linters: enable forbidigo
Selectively disable cases that I cannot touch or don't bother spending
time on.  Future cases will be avoided.

Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2023-09-01 11:08:17 +02:00
OpenShift Merge Robot 7ffc273928 Merge pull request #1630 from vrothberg/oci-ref
libimage: preserve optional name when pulling from OCI transport
2023-08-30 09:59:33 +02:00
Valentin Rothberg 02a72eb33e libimage: preserve optional name when pulling from OCI transport
When pulling from an OCI source, make sure to preseve the optional name.
For instance, a `podman pull oci:/tmp/foo:quay.io/foo/bar:latest` should
pull the image and name it `quay.io/foo/bar:latest`.

While at it, also fix a bug when pulling an OCI without the optional
name.  Previously, we used the path to name the image which will error
in most cases due to invalid characters (e.g., capital ones).  Hence,
apply the same trick as for the `dir` transport and generate a sha.

Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2023-08-30 09:19:42 +02:00
OpenShift Merge Robot d24aa07e87 Merge pull request #1629 from mtrmac/ManifestListData-no-deps
Move libimage.ManifestList{Descriptor,Data} to libimage/define
2023-08-29 12:40:13 +02:00
Miloslav Trmač a6d079d218 Move libimage.ManifestList{Descriptor,Data} to libimage/define
These structs are (for better or worse) a part of
Podman's API, so podman-remote needs to include the subpackage that
defines them - which is all of libimage (and c/image/v5/copy) right now.

Instead, move them to libimage/define.

Alternatively, maybe Podman's API should not directly depend on (unstable)
c/common types?!

Only moves unchanged code, should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2023-08-28 23:17:59 +02:00
Valentin Rothberg a60d8bd633 libimage: close all `ImageCloser`s
Make sure to close all `ImageCloser` to avoid potential memory leaks.

Fixes: containers/common#1533
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2023-08-28 13:57:36 +02:00
Valentin Rothberg 91b87d2bde libimage: pull: increase timeout running under systemd
Set the `EXTEND_TIMEOUT_USEC` over DBUS when pulling an image from a
registry and when running under systemd.  This will prevent a frequent
issue when running Quadlets and exceeding the default systemd start
timeout of 90 seconds when pulling the image takes too long.

Fixes: containers/podman/issues/18353
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2023-08-23 14:38:11 +02:00
Aditya R 054a131950 manifest: add support ForceCompressionFormat
Implement https://github.com/containers/image/pull/2068 for
libimage/manifest.Push

Signed-off-by: Aditya R <arajan@redhat.com>
2023-08-11 14:27:43 +05:30
Aditya R fe8cc2aba2 libimage/copier: wire ForceCompressionFormat for image copy
Implement https://github.com/containers/image/pull/2068 for
libimage/copier.

Signed-off-by: Aditya R <arajan@redhat.com>
2023-08-11 14:22:27 +05:30
Aditya R 91e5c02d28 libimage/ManifestPushOptions: add AddCompression from #1585
Podman uses different API for pushing manifest list,
add `AddCompression` to ManifestPushOptions, which is implemented
here: https://github.com/containers/common/pull/1585

[NO NEW TESTS NEEDED]
Tests are added here: https://github.com/containers/common/pull/1585

Signed-off-by: Aditya R <arajan@redhat.com>
2023-08-01 12:45:18 +05:30
Aditya R a760be0b55 manifests,push: add support for AddCompression
`c/image` recently added support for EnsureCompressionVariantsExist,
following PR exposes the feature to `c/common` manifests so actual users
can use it.

Signed-off-by: Aditya R <arajan@redhat.com>
2023-07-27 13:51:10 +05:30
Miloslav Trmač 6c259968ca Reorganize how requiredDigest/requiredTag is determined
Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2023-07-13 22:12:37 +02:00
Miloslav Trmač f80096f3a2 Accept a reference.Named in referenceFuzzilyMatchingRepoAndTag
We don't need a reference.NamedTagged now.

That also makes the namedTagged variable in the caller more local.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2023-07-13 22:12:37 +02:00
Miloslav Trmač b5765a6100 Eliminate the isDigested variable
Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2023-07-13 22:12:37 +02:00
Miloslav Trmač 78ca348258 Pass the required tag, not just a bool, to referenceFuzzilyMatchingRepoAndTag
Right now that's not simpler, but it will enable simplification of the caller.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2023-07-13 22:12:37 +02:00
Miloslav Trmač f2ab3c0f7a Move the !IsShortName early exit a bit forward
... so that early exits are colocated.

Should not change behavior, reference.TrimNamed() updating
"name" should not change the IsShortName value.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2023-07-13 22:12:37 +02:00
Miloslav Trmač ec3c6ae88c Simplify referenceFuzzilyMatchingRepoAndTag a bit
Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2023-07-13 22:12:37 +02:00
Miloslav Trmač a8bba37a40 Rename inRepoTags to referenceFuzzilyMatchingRepoAndTag
Scary features should have scary names. Also add a comment
to make it less likely that this semantics will spread.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2023-07-13 22:12:37 +02:00
Miloslav Trmač 0d489be726 Use a digest.Digest type for the hasDigest argument
Should not change behavior, both callers now have
a value of that type.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2023-07-13 22:12:37 +02:00
Miloslav Trmač 31d9de63c6 Parse a digest in filterDigest
This causes an immediate failure on invalid values,
instead of silently not matching anything.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2023-07-13 22:12:37 +02:00
OpenShift Merge Robot 4097e2908e Merge pull request #1505 from vrothberg/bye-bye-digest-dance
libimage: harden lookup by digest
2023-07-12 22:14:00 -04:00
Valentin Rothberg ec483cf1b7 libimage: HasDifferentDigest: add InsecureSkipTLSVerify option
To fix a BZ on auto-updates, we need to improve the system tests.
To improve the system tests, we need to use the existing test
infrastructure.  To use the existing test infrastructure, we need
to be able to skip TLS verification in tests.  To be able to skip TLS
verification in the tests, we need a new --tls-verify flag for
podman-auto-update.  And for that to work, we need a way to pass that
down to libimage when checking if there's a new image on the registry.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=2218315

Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2023-07-03 08:51:27 +02:00
Valentin Rothberg 5524133c26 libimage: harden lookup by digest
When looking up an image by digest, make sure that the entire repository
of the specified value is considered.  Previously, both the repository
and the tag have been ignored and we looked for _some_ image with a
matching digest.

As outlined in #1248, Docker stopped ignoring the repository with
version v20.10.20 (Oct '22) which is a compelling reason to do the same.

To be clear, previously `something@digest` would look for any image with
`digest` while `something` is entirely ignored.  With this change, both
`something` and `digest` must match the image.

This change breaks two e2e tests in Podman CI which relied on the
previous behavior.  There is a risk of breaking users but there is a
strong security argument to perform this change:  if the repository does
not match the (previously) returned issue, there is a fair chance of a
user error.

Fixes: containers/common#1248
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2023-06-30 10:09:48 +02:00
Valentin Rothberg 816abe2da3 layer tree: handle more errors
Commit 3e9cdd9c53 took a first stab at making operating on the layer
tree more resilient by detecting unknown-image errors.  Testing in
containers/podman/pull/18980 revealed that we also need to detect
unknown-layer and -size errors as well.

Move the errors checks into a convenience function and update the
relevant call sites to facilitate future changes.  Export the
function since Podman needs the very same checks when operating
on images, for instance, when looking up image labels.

Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2023-06-26 13:11:58 +02:00
Valentin Rothberg 3e9cdd9c53 layer tree: be more resilient
Make operating on the layer tree more resilient towards parallel image
removals by checking for unknown-image errors.  This partially fixes
below linked BZ.

At the time of writing, I do not have a reliable easy (and short
running) reproducer.  Hence, I suggest to merge this change without
tests and rely on either Podman CI (which doesn't have a reliable test
yet either) and the QE team (will be informed).

BZ: bugzilla.redhat.com/show_bug.cgi?id=2216700
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2023-06-23 13:00:20 +02:00
Valentin Rothberg 5dc07587d2 make validate: silence deprecation warning
I refrain from breaking the API since Lockers are still around.

Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2023-06-16 17:08:25 +02:00
Valentin Rothberg 0c6e1393d1 make validate: fix empty block reports
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2023-06-16 17:08:25 +02:00
Valentin Rothberg 3ddaa0afa6 make validate: fix redundant if/else reports
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2023-06-16 17:08:25 +02:00
Valentin Rothberg c3c4acc667 make validate: fix unused argument reports
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2023-06-16 17:08:25 +02:00
Valentin Rothberg 53b3bac45e libimage: fix reference filters
Make sure that reference filters properly work on digests as well.
To keep things simple, try to lookup an image for the user-specified
value and compare IDs.  This will implicitly fix #containers/podman/issues/18445
and probably more (unknown) issues.

Fixes: #containers/podman/issues/18445
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2023-06-14 10:45:19 +02:00