When loading the registries.conf, allow for loading additional files
from `/etc/containers/registries.conf.d`. The files are loaded in
alpha-numerical order and specified fields will overwrite the previous
config.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
This commit adds DockerDisableDestSchema1MIMETypes to types.Context
which, if set to true, will omit DockerV2Schema1SignedMediaType and
DockerV2Schema1MediaType from
dockerImageDestination.SupportedManifestMIMETypes.
The motivation for this is that newer versions of the docker registry
(2.7.1 and later) disable schema1 by default. If a user attempts to
copy a v2schema1 image into the newer registry, the result is a
500 error. With this change, when calling copy.Image(), setting
options.DestinationCtx.DockerDisableDestSchema1MIMETypes to true
will allow copying of a schema1 image to a destination registry with
schema1 disabled. The image will be converted to schema2.
Signed-off-by: Scott Seago <sseago@redhat.com>
This has arguably been implied (OTOH, also arguably, it's a breaking change),
make it explicit.
This does not yet implement the semantics.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
The progress reader will be now created via the new `newProgressReader`
function. This way we have the possibility to trigger a new event called
`ProgressEventNewArtifact` and `ProgressEventDone` to indicate the
boundaries of the artifact download to the API consumer. During the
download, we additionally provide the `OffsetUpdate`, which can be used
to report the downloaded data during the last elapsed time interval.
Unit tests have been added as well to the progress reader, whereas the
documentation for the new types has been enhanced as well.
Signed-off-by: Sascha Grunert <sgrunert@suse.com>
Add the manifest.List interface, and implementations for OCIv1 Index and
Docker Schema2List documents.
Add an instanceDigest parameter to PutManifest(), PutSignatures(), and
LayerInfosForCopy, for symmetry with GetManifest() and GetSignatures().
Return an error if the instanceDigest is supplied to destinations which
don't support them, and add stubs that do so even to the transports
which would support it, so that we don't break compilation here.
Add a MultipleImages flag to copy.Options, and if the source for a copy
operation contains multiple images, copy all of the images if we can.
If we can't copy them all, but we were told to, return an error.
Use the generic manifest list API to select a single image to copy from
a list, so that we aren't just limited to the Docker manifest list
format for those cases.
When guessing at the type of a manifest, if the manifest contains a list
of manifests, use its declared MIME type if it included one, else assume
it's an OCI index, because an OCI index doesn't include its MIME type.
When copying, switch from using an encode-then-compare of the original
and updated versions of the list to checking if the instance list was
changed (one of the things we might have changed) or if its type has
changed due to conversion (the other change we might have made). If
neither has changed, then we don't need to change the encoded value of
the manifest.
When copying, when checking for a digest mismatch in a target image
reference, ignore a mismatch between the digest in the reference and the
digest of the main manifest if we're copying one element from a list,
and the digest in the reference matches the digest of the manifest list.
When copying, if conversion of manifests for single images is being
forced, convert manifest lists to the corresponding list types.
When copying, supply the unparsed top level to Commit() by attaching the
value to the context.Context.
Support manifest lists in the directory transport by using the instance
digest as a prefix of the filename used to store a manifest or a piece
of signature data.
Support manifest lists in the oci-layout transport by accepting indexes
as we do images, and stop guessing about Platform values to add to the
top-level index.
Support storing manifest lists to registries in the docker: transport by
using the manifest digest when we're writing one image as part of
pushing a list of them, and by using the instance digest when reading or
writing signature data, when one is specified, or the cached digest of
the non-instanced digest when one is not specified.
Add partial support for manifest lists to the storage transport: when
committing one image from a list into storage, also add a copy of the
manifest list by extracting it from the context.Context. The logic is
already in place to enable locating an image using any of multiple
manifest digests.
When writing an image that has an instanceDigest value (meaning it's a
secondary image), don't try to generate a canonical reference to add to
the image's list of names if the reference for the primary image doesn't
contain a name. That should only happen if we're writing using just an
image ID, which is unlikely, but we still need to handle it.
Avoid computing the digest of the manifest, or retrieving the
either-a-tag-or-a-digest value from the target reference, if we're given
an instanceDigest, which would override them anyway.
Move the check for non-nil instanceDigest values up into the main
PutSignatures() method instead of duplicating it in the per-strategy
helpers.
Add mention of the instanceDigest parameter and its use to various
PutManifest, PutSignatures, and LayerInfosForCopy implementations and
their declarations in interfaces.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
This change facilitates OpenShift to mount docker config/auth data captured in Secrets in locations typical of such mounts, which are not off of the pod user's home directory.
Signed-off-by: gabemontero <gmontero@redhat.com>
... so that major-version-aware Go module import
(as opposed to vX.Y.Z+incompatible, which does not allow different
packages to use different versions) works right.
Also requires adding some more GO111MODULE=on options to Makefile.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Currently, importing c/image/types for any reason drags in all the
compression implementations, which is very undesirable.
For now, we don't really want to commit to the compresion.Algorithm
API as a public interface where anyone could supply an external
implementation; so, the goal is to provide an Algorithm type
that has the same public properties (notably a Name() method),
but it is usable only within the pkg/compression namespace.
To do this, move the Algorithm struct into a pkg/compression/internal
subpackage which does not depend on the compression implementations.
This still allows defining public methods on the type, but the
subpackage can keep the access to creating values of that type and to
accessing other members restricted to pkg/compression by only using
private fields and providing a constructor and accessors that are
callable only from within pkg/compression.
Then, make the internal.Algorithm type public as an alias in a new
pkg/compression/types subpackage, without exposing anything else
from the internal subpackage.
The primary pkg/compression package continues to exist the way it used to,
only using accessors from the internal subpackage to deal with the
internal.Algorithm = compression/types.Algorithm = compression.Algorithm
type.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
When copying an image, record the compression in the BlobInfo and use
the information when updating the manifest's layer infos to set the
layers' media types correctly.
Also check for supported media types when parsing a v2s2/OCI1 manifest.
Note that consumers of the containers/image library need to update
opencontainers/image-spec to commit 775207bd45b6cb8153ce218cc59351799217451f.
Fixes: github.com/containers/libpod/issues/2013
Fixes: github.com/containers/buildah/issues/1589
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
add the possibility to choose what compression format must be used and
the compression level to use.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
Adds the "Env" field in the image schema. Allow the library user
gets the "Env" value available in the image JSON data. Close#248
Signed-off-by: José Guilherme Vanz <jguilhermevanz@suse.com>
Add a HasThreadSafePutBlob() method to the ImageDestination interface
and all its implementations to indicate whether the corresponding
PutBlob() method can be executed concurrently. This is a first step to
enable parallel image copying. By default, all transports are not
thread-safe and must be carefully migrated in later changes.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Add a HasThreadSafeGetBlob() method to the ImageSource interface and all
its implementations to indicate whether the corresponding GetBlob()
method can be executed concurrently. This is a first step to enable
parallel image copying. By default, all transports are not thread-safe
and must be carefully migrated in later changes.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
This will allow TryReusingBlob to substitute the required blob with an equivalent
based on cache information if possible, or not doing so if it would break signatures.
In addition, we set canSubstitute to false when signing an image, to make 100% sure
the signed blob is safe (e.g. that we are not signing a third-party-compressed version
which has been maliciously designed to attack some decompressor implementations).
Nothing implements this, so does not change behavior yet.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
For now, none of the transports actually use it, so should not change behavior.
copy.Image uses its existing cache object; config parses in image/* usually
use NoCache because they wouldn't appreciably benefit anyway.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This will, primarily, make it easier for the transport to use
alternate locations without having to somehow carry state from
HasBlob to ReapplyBlob.
Also, all instances of ReapplyBlob have been either trivial or redundant,
and there is a single primary caller of HasBlob/ReapplyBlob (+ a few
in-transport users who are even a bit cleaner with the move to
TryReusingBlob).
Should not change behavior, apart from not doing the
storageImageDestination.HasBlob check redundantly in
storageImageDestination.ReapplyBlob.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This returns a default BoltDB instance (which almost everyone
should use to maximally benefit from the cache). The default location
is chosen to exist near the containers/storage GraphRoot (i.e.
typically on the same partition, with the same SELinux context),
both for root and rootless modes of podman.
This adds no users yet, so should not immediately change behavior.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This will be used to record information about copied layers, and avoid the copies
if possible.
Does not add any users, so does not change behavior yet.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Make dockerClient sensitive to the configured registries by using the
sysregistriesv2 package. Unless specified in the SystemContext, the TLS
config's InsecureSkipVerify flag now adheres to the registries Insecure
field (if specified in the registries.conf). Notice, that the default
value remains false (i.e., TLS verification is performed).
SystemContext.DockerInsecureSkipTLSVerify is usually set by users of
this library (e.g., Skopeo and Podman). To be able to distinguish
between a default value and a user-specified value, introduce a
types.OptionalBool type that is now used for DockerInsecureSkipTLSVerify.
Signed-off-by: Valentin Rothberg <vrothberg@suse.com>
This allows a destination to opt out of updating the embedded name:tag in schema1
manifests without violating the ImageDestination.Reference / ImageReference API
expectations.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Add support to create a docker-archive with more than one RepoTag,
enabling users such as skopeo, to create or copy multitag archives.
Support for other transports can be added in the future.
Fixes: https://github.com/containers/image/issues/447
Signed-off-by: Valentin Rothberg <vrothberg@suse.com>
- Network IO paths should react to cancels now.
- File IO paths generally still won't.
- `SystemContext` objects have been renamed to `sys` to leave `ctx`
available for the stdlib context objects.
Signed-off-by: Mike Lundy <mike@fluffypenguin.org>
The whole purpose of UnparsedImage is to not parse the manifest,
so having this interface which more or less requires the manifest to be
parsed and recomputed is not desirable.
sourcedImage now uses ImageSource.LayerInfosForCopy via the private
UnparsedImage.src, which is a bit ugly, but that dependency has already
existed, and we get a more consistent public API for this price.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Add an LayerInfosForCopy() method to source images which gives them a
chance to provide updated values for the blobsums contained in the
image's manifest, if they want to. Returning `nil` implies that they
have no changes to suggest compared to what's in the manifest.
When copying an image, if we can update the manifest with those new
values during copying, do so. If we have new values, but we can't
update the manifest, copying fails.
Update storageImageSource to return its manifest and reference in
unmodified form, and supply updated blob digests via LayerInfosForCopy()
so that copying images from storage works.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
... and use them for choosing an image from a manifest list, and
verifying whether an image is acceptable in a MustMatchRuntimeOS()
destination. Propagate the types.SystemContext through the
call stack as necessary.
This adds no users and seems not all that important, but after
we re-enable fetching manifest lists, docker_transport_test.go does a
"//busybox".NewImage(), which nowadays mean copying from a manifest list,
and then we need this override to keep tests working on non-Linux platforms
for which Docker does not publish images.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Remove the .Close() methods from UnparsedImage/Image, which closed the
underlying ImageSource. Instead, just require the caller to ensure
that the ImageSource is not closed as long as the UnparsedImage/Image
are used.
This allows using several independent UnparsedImage/Image instances
for a shared ImageSource; notably independent Image objects for the
individual image instances in a manifest list. (copy.Image is already
simpler although it is only using a single instance.)
To keep ImageReference.NewImage simple and not to break all the external
callers of this, also add a simple ImageCloser wrapper which retains
the ImageSource closing functionality, and return it from image.FromSource
and ImageReference.NewImage implementations.
(It's very likely many of the NewImage callers would be surprised by how this
handles manifest lists, and it is very tempting to break this API, at least
by renaming, to force the callers to consider this; however, this would be
better done after eliminating the need of ImageReference.NewImage entirely,
by replacing the specialized types.Image extensions with something else, first.)
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
A types.UnparsedImage is now created by image.UnparsedInstance, with the
caller specifying nil or a digest.
A types.Image is (preferably) created by image.FromUnparsedImage based
on a specific instance. For continuity/compatibility, image.FromSource
continues working the old way, i.e. using the default instance, and
transparently parsing data from a chosen instance (but still returning
the manifest list in .Manifest()).
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This mirrors GetManifest, and allows / requires signatures to be per-instance.
Also add implementations in docker: and atomic:, the only transports which
support both manifest lists and signatures.
Does not change behavior yet, the only user always specifies nil
instanceDigest.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This will make the code paths more uniform for consumers of the
primary manifest and the manifest instances.
(Having an explicit support for manifest instances is necessary
for transports like docker-daemon: / oci-archive:, which
contain several images but setting up an ImageSource is very
expensive, or which don't even allow referencing images by digest.)
This is a direct replacement of GetTargetManifest, and should
not change behavior; notably the OCI implementation is still
blindly guessing the manifest type although it is probably
available in the index.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Instead, have copy.Image call .Manifest() and use manifest.MIMETypeIsMultiImage
directly.
This is not locally an improvement; but we want to move the IsMultiImage check
to work on types.UnparsedImage, and because types.Image is a superset of
types.UnparsedImage, for consistency we would have to implement a
completely useless sourcedImage.IsMultiImage which works with the cached data.
Instead, let's just drop the method from the types.Image interface and work
with the MIME type directly.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Gives the user the option to switch between the oci, v2s1, and v2s2 manifests
Add directory creation if the directory doesn't exist
Add overwrite functionality only if the directory was a container image directory
Feature added for the pulp team
Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
If supplied, this will be used instead of the `blobs` subdirectory
within OCI image layouts (which, per the OCI spec, may be empty), when
getting or putting blobs.
Signed-off-by: Jonathan Boulle <jonathanboulle@gmail.com>
According to the image spec [1], descriptors should contain
media types. Support them on the returned BlobInfo for layers.
[1] https://github.com/opencontainers/image-spec/blob/master/descriptor.md
Signed-off-by: Claudia Beresford <cberesford@pivotal.io>
Signed-off-by: William Martin <wmartin@pivotal.io>
Signed-off-by: Topher Bullock <cbullock@pivotal.io>
This package is used in authenticating a user for kpod login
and can be used for authentication in kpod push, pull etc.
Signed-off-by: umohnani8 <umohnani@redhat.com>
For the purposes of testing, it is handy to override the default configuration
file location of the system registries configuration file. This is particularilyi
true in the case of crio-o which consumes this library.
Signed-off-by: baude <bbaude@redhat.com>
The requestedManifestMIMETypes parameter was added because a destination
might not support all manifest MIME types that the the source supports,
but the original use case now passes all manifest types and lets
containers/image convert internally. In generally, internal conversion
may be more comprehensive, is more predictable, and avoids bypassing
internal checks.
Fixes: #331
Signed-off-by: Owen W. Taylor <otaylor@fishsoup.net>
According to the image spec [1], descriptors should be able to contain
annotations. Support them on the returned BlobInfo.
[1] fc936c7834/descriptor.md
Signed-off-by: Will Martin <wmartin@pivotal.io>
Valentin Rothberg