Currently, ostree does not compile:
> ostree/ostree_dest.go:115:28: cannot use d (variable of type *ostreeImageDestination) as private.ImageDestinationInternalOnly value in argument to impl.AddCompat: *ostreeImageDestination does not implement private.ImageDestinationInternalOnly (missing method NoteOriginalOCIConfig) (typecheck)
> d.Compat = impl.AddCompat(d)
This has been broken since b941c6bf41 (Nov 18 2024, > 4 months).
Previously:
- 807381f448 (broken almost 4 months),
- 2e33bf7d27 (broken > 7 months)
Overall, it seems extremely likely that noone is using the
containers_image_ostree build tag (off by default since March 2019).
So, finally, give up, and delete the code. Most importantly, this allows us
to delete the > 3-year-old-frozen dependency, and risky CGo code.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
The ping v1 happens when the ping v2 fails, however, it causes the ping
v2 error to be skipped and not output to the user. As result, when a
registry has v1 and v2 enabled, and there are, for example, intermittent
connectivity issues making the ping v2 fail, the user is presented with
a misleading error saying "can't talk to V1 registry."
Since the only use of v1 is for the search API as a workaround for
docker.io, and new container registries setups are very unlikely to be
v1-only, there is little utility in keeping this v1 detection in the
attempt to help the user realize their setup is v1-only, hence not
compatible. On the contratry, it just presents the user with a
misleading error in certain circumstances.
Signed-off-by: Carlos Rodriguez-Fernandez <carlosrodrifernandez@gmail.com>
In particular, it is OK to use CompressionOperation: Decompress
on already-decompressed layers.
TestUpdatedMIMEType already ensures this works correctly.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
"authentication" refers to a process, not to username:password;
"authentications" is not a thing.
So, use "credential" where appropriate, and make similar related adjustments.
Leave the existing public API of pkg/docker/config, and types.DockerAuthConfig,
unchanged, though.
Should not change behavior.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
The previous PR was tested with an earlier Golang version,
so it was merged despite violating current formatting validation.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This is just the minimal update: the gofmt-created
updates have been reviewed and edited to preserve original
semantic intent, but I didn't review all
existing comments to benefit from the new syntax.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Podman's Docker-compatible REST API is in need of a mechanism to enforce
resolving to Docker Hub only. Yet there is the desire for the rest of
the stack to continue honoring the system's registries.conf.
We recently added a new field to containers.conf [1] which allows for
opting out from enforcing Docker Hub for Podman's compat API but we
still lack a way of enforcement when resolving short names; which
ultimately is *the* place to do that.
This change does the necessary plumbing. The compat REST handlers will
set the new field in the `types.SystemContext` and pass that down to
libimage and buildah.
[1] e698b8caca
Context: containers/podman/issues/12320
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Add internal/streamdigest package to stream layers to temporary files
for populating Digest and Size in BlobInfos and use in docker tarfile
and image dest. When precompute digests option is set, layers are
never uploaded that already exist on the destination registry, in
exchange for streaming layers to temporary files when digests are
unknown (ex. compressing "on the fly").
Signed-off-by: Paul Fisher <pfisher@lyft.com>
This was always sort of implied, and we now rely on that explicitly
to avoid computing the digests; it turns out the digest computation
can consume a very noticeable amount of CPU time.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
... and acknowledge that various tests are strictly speaking
invalid, to reinforce that real callers must not pass nil.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Extend the blob info cache to also cache the name of the type of
compression used on a blob that we've seen, or specific values that
indicate that we know the blob was not compressed, or that we don't
know whether or not it was compressed.
New methods for adding known blob-compression pairs and reading
candidate locations including compression information are part of a new
internal BlobInfoCache2 interface which the library's BlobInfoCache
implementors also implement.
When we copy a blob, try to record the state of compression for the
source blob, and if we applied any changes, the blob we produced.
Make sure that when TryReusingBlob successfully uses a blob from the
blob info cache, that it provides compression information in the
BlobInfo that it returns, so that manifests can be updated to describe
layers using the correct MIME types.
When attempting to write a manifest, if a manifest can't be written
because layers were compressed using an algorithm which can't be
expressed using that manifest type, continue on to trying other manifest
formats.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Add DockerLogMirrorChoice to types.SystemContext for keeping log level of the physical pull source of images.
Crio could set it log the image source if it's from a mirror.
Signed-off-by: Qi Wang <qiwan@redhat.com>
Add a new package for short-name resolution. `pkg/shortnames` is built
around the short-name aliasing in the registries.conf and introduces two
functions.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
The use of unqualified-search registries entails an ambiguity as it's
unclear from which registry a given image, referenced by a short name,
may be pulled from.
There have been reports of squatting on some registries to trap users
into pulling from a registry that may have pull precedence over the
intended registry; all depending on the relative order of the entries
in the `unqualified-search-registries` field in the `registries.conf`.
Removing the feature of unqualified-search registries is not an option,
as many users depend on it. The agreed on alternative is to introduce
aliases. Aliases make short-name resolution explicit. Similar to bash
aliases, the new `[aliases]` table (internally a `map[string]string`)
has a left-hand name and a right-hand value:
```TOML
[aliases]
name="registry.com/namespace/name"
```
Consumers of containers/image can now use the new API to resolve aliases
and add new ones.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
The new event indicates that a blob has been skipped. This information
can be used by CRI-O for more detailed image layer re-use metrics.
Signed-off-by: Sascha Grunert <sgrunert@suse.com>
Adding the support for using identitytoken in the .docker/config.json
files. Azure Container Registry is one of the case that uses this.
Signed-off-by: yihuaf <fang.yihua.eric@gmail.com>
When loading the registries.conf, allow for loading additional files
from `/etc/containers/registries.conf.d`. The files are loaded in
alpha-numerical order and specified fields will overwrite the previous
config.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
This commit adds DockerDisableDestSchema1MIMETypes to types.Context
which, if set to true, will omit DockerV2Schema1SignedMediaType and
DockerV2Schema1MediaType from
dockerImageDestination.SupportedManifestMIMETypes.
The motivation for this is that newer versions of the docker registry
(2.7.1 and later) disable schema1 by default. If a user attempts to
copy a v2schema1 image into the newer registry, the result is a
500 error. With this change, when calling copy.Image(), setting
options.DestinationCtx.DockerDisableDestSchema1MIMETypes to true
will allow copying of a schema1 image to a destination registry with
schema1 disabled. The image will be converted to schema2.
Signed-off-by: Scott Seago <sseago@redhat.com>
This has arguably been implied (OTOH, also arguably, it's a breaking change),
make it explicit.
This does not yet implement the semantics.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
The progress reader will be now created via the new `newProgressReader`
function. This way we have the possibility to trigger a new event called
`ProgressEventNewArtifact` and `ProgressEventDone` to indicate the
boundaries of the artifact download to the API consumer. During the
download, we additionally provide the `OffsetUpdate`, which can be used
to report the downloaded data during the last elapsed time interval.
Unit tests have been added as well to the progress reader, whereas the
documentation for the new types has been enhanced as well.
Signed-off-by: Sascha Grunert <sgrunert@suse.com>
Add the manifest.List interface, and implementations for OCIv1 Index and
Docker Schema2List documents.
Add an instanceDigest parameter to PutManifest(), PutSignatures(), and
LayerInfosForCopy, for symmetry with GetManifest() and GetSignatures().
Return an error if the instanceDigest is supplied to destinations which
don't support them, and add stubs that do so even to the transports
which would support it, so that we don't break compilation here.
Add a MultipleImages flag to copy.Options, and if the source for a copy
operation contains multiple images, copy all of the images if we can.
If we can't copy them all, but we were told to, return an error.
Use the generic manifest list API to select a single image to copy from
a list, so that we aren't just limited to the Docker manifest list
format for those cases.
When guessing at the type of a manifest, if the manifest contains a list
of manifests, use its declared MIME type if it included one, else assume
it's an OCI index, because an OCI index doesn't include its MIME type.
When copying, switch from using an encode-then-compare of the original
and updated versions of the list to checking if the instance list was
changed (one of the things we might have changed) or if its type has
changed due to conversion (the other change we might have made). If
neither has changed, then we don't need to change the encoded value of
the manifest.
When copying, when checking for a digest mismatch in a target image
reference, ignore a mismatch between the digest in the reference and the
digest of the main manifest if we're copying one element from a list,
and the digest in the reference matches the digest of the manifest list.
When copying, if conversion of manifests for single images is being
forced, convert manifest lists to the corresponding list types.
When copying, supply the unparsed top level to Commit() by attaching the
value to the context.Context.
Support manifest lists in the directory transport by using the instance
digest as a prefix of the filename used to store a manifest or a piece
of signature data.
Support manifest lists in the oci-layout transport by accepting indexes
as we do images, and stop guessing about Platform values to add to the
top-level index.
Support storing manifest lists to registries in the docker: transport by
using the manifest digest when we're writing one image as part of
pushing a list of them, and by using the instance digest when reading or
writing signature data, when one is specified, or the cached digest of
the non-instanced digest when one is not specified.
Add partial support for manifest lists to the storage transport: when
committing one image from a list into storage, also add a copy of the
manifest list by extracting it from the context.Context. The logic is
already in place to enable locating an image using any of multiple
manifest digests.
When writing an image that has an instanceDigest value (meaning it's a
secondary image), don't try to generate a canonical reference to add to
the image's list of names if the reference for the primary image doesn't
contain a name. That should only happen if we're writing using just an
image ID, which is unlikely, but we still need to handle it.
Avoid computing the digest of the manifest, or retrieving the
either-a-tag-or-a-digest value from the target reference, if we're given
an instanceDigest, which would override them anyway.
Move the check for non-nil instanceDigest values up into the main
PutSignatures() method instead of duplicating it in the per-strategy
helpers.
Add mention of the instanceDigest parameter and its use to various
PutManifest, PutSignatures, and LayerInfosForCopy implementations and
their declarations in interfaces.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
This change facilitates OpenShift to mount docker config/auth data captured in Secrets in locations typical of such mounts, which are not off of the pod user's home directory.
Signed-off-by: gabemontero <gmontero@redhat.com>
... so that major-version-aware Go module import
(as opposed to vX.Y.Z+incompatible, which does not allow different
packages to use different versions) works right.
Also requires adding some more GO111MODULE=on options to Makefile.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Currently, importing c/image/types for any reason drags in all the
compression implementations, which is very undesirable.
For now, we don't really want to commit to the compresion.Algorithm
API as a public interface where anyone could supply an external
implementation; so, the goal is to provide an Algorithm type
that has the same public properties (notably a Name() method),
but it is usable only within the pkg/compression namespace.
To do this, move the Algorithm struct into a pkg/compression/internal
subpackage which does not depend on the compression implementations.
This still allows defining public methods on the type, but the
subpackage can keep the access to creating values of that type and to
accessing other members restricted to pkg/compression by only using
private fields and providing a constructor and accessors that are
callable only from within pkg/compression.
Then, make the internal.Algorithm type public as an alias in a new
pkg/compression/types subpackage, without exposing anything else
from the internal subpackage.
The primary pkg/compression package continues to exist the way it used to,
only using accessors from the internal subpackage to deal with the
internal.Algorithm = compression/types.Algorithm = compression.Algorithm
type.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
When copying an image, record the compression in the BlobInfo and use
the information when updating the manifest's layer infos to set the
layers' media types correctly.
Also check for supported media types when parsing a v2s2/OCI1 manifest.
Note that consumers of the containers/image library need to update
opencontainers/image-spec to commit 775207bd45b6cb8153ce218cc59351799217451f.
Fixes: github.com/containers/libpod/issues/2013
Fixes: github.com/containers/buildah/issues/1589
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
add the possibility to choose what compression format must be used and
the compression level to use.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
Adds the "Env" field in the image schema. Allow the library user
gets the "Env" value available in the image JSON data. Close#248
Signed-off-by: José Guilherme Vanz <jguilhermevanz@suse.com>
Miloslav Trmač