... to prevent panics if the value does not contain a :, or other unexpected
values (e.g. a path traversal).
Don't bother on paths where we computed the digest ourselves, or it is already trusted
for other reasons.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Flips dependency of internal and public manifest.List API and introduces
private manifest.List API which will be extended in future for new
features.
[NO NEW TESTS NEEDED]
[NO TESTS NEEDED]
Signed-off-by: Aditya R <arajan@redhat.com>
This matches internal/imagedestination/impl.Properties; it's not
quite worth it for the single value, but the consistency is
attractive.
Should not change behavior.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This sets up the precedent that all transports should primarily implement
the private interface; that will allow us to make future changes to the
private interface easier, because we can just change the public interface
wrappers in a single place instead of modifying transports - especially
as more stubs are added soonish.
Should not change behavior.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Mostly just name changes that should not change behavior,
apart from ioutil.ReadDir -> os.ReadDir avoiding per-item
lstat(2) in some cases.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Add the manifest.List interface, and implementations for OCIv1 Index and
Docker Schema2List documents.
Add an instanceDigest parameter to PutManifest(), PutSignatures(), and
LayerInfosForCopy, for symmetry with GetManifest() and GetSignatures().
Return an error if the instanceDigest is supplied to destinations which
don't support them, and add stubs that do so even to the transports
which would support it, so that we don't break compilation here.
Add a MultipleImages flag to copy.Options, and if the source for a copy
operation contains multiple images, copy all of the images if we can.
If we can't copy them all, but we were told to, return an error.
Use the generic manifest list API to select a single image to copy from
a list, so that we aren't just limited to the Docker manifest list
format for those cases.
When guessing at the type of a manifest, if the manifest contains a list
of manifests, use its declared MIME type if it included one, else assume
it's an OCI index, because an OCI index doesn't include its MIME type.
When copying, switch from using an encode-then-compare of the original
and updated versions of the list to checking if the instance list was
changed (one of the things we might have changed) or if its type has
changed due to conversion (the other change we might have made). If
neither has changed, then we don't need to change the encoded value of
the manifest.
When copying, when checking for a digest mismatch in a target image
reference, ignore a mismatch between the digest in the reference and the
digest of the main manifest if we're copying one element from a list,
and the digest in the reference matches the digest of the manifest list.
When copying, if conversion of manifests for single images is being
forced, convert manifest lists to the corresponding list types.
When copying, supply the unparsed top level to Commit() by attaching the
value to the context.Context.
Support manifest lists in the directory transport by using the instance
digest as a prefix of the filename used to store a manifest or a piece
of signature data.
Support manifest lists in the oci-layout transport by accepting indexes
as we do images, and stop guessing about Platform values to add to the
top-level index.
Support storing manifest lists to registries in the docker: transport by
using the manifest digest when we're writing one image as part of
pushing a list of them, and by using the instance digest when reading or
writing signature data, when one is specified, or the cached digest of
the non-instanced digest when one is not specified.
Add partial support for manifest lists to the storage transport: when
committing one image from a list into storage, also add a copy of the
manifest list by extracting it from the context.Context. The logic is
already in place to enable locating an image using any of multiple
manifest digests.
When writing an image that has an instanceDigest value (meaning it's a
secondary image), don't try to generate a canonical reference to add to
the image's list of names if the reference for the primary image doesn't
contain a name. That should only happen if we're writing using just an
image ID, which is unlikely, but we still need to handle it.
Avoid computing the digest of the manifest, or retrieving the
either-a-tag-or-a-digest value from the target reference, if we're given
an instanceDigest, which would override them anyway.
Move the check for non-nil instanceDigest values up into the main
PutSignatures() method instead of duplicating it in the per-strategy
helpers.
Add mention of the instanceDigest parameter and its use to various
PutManifest, PutSignatures, and LayerInfosForCopy implementations and
their declarations in interfaces.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
... so that major-version-aware Go module import
(as opposed to vX.Y.Z+incompatible, which does not allow different
packages to use different versions) works right.
Also requires adding some more GO111MODULE=on options to Makefile.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Add a HasThreadSafeGetBlob() method to the ImageSource interface and all
its implementations to indicate whether the corresponding GetBlob()
method can be executed concurrently. This is a first step to enable
parallel image copying. By default, all transports are not thread-safe
and must be carefully migrated in later changes.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
For now, none of the transports actually use it, so should not change behavior.
copy.Image uses its existing cache object; config parses in image/* usually
use NoCache because they wouldn't appreciably benefit anyway.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
- Network IO paths should react to cancels now.
- File IO paths generally still won't.
- `SystemContext` objects have been renamed to `sys` to leave `ctx`
available for the stdlib context objects.
Signed-off-by: Mike Lundy <mike@fluffypenguin.org>
The config file was being saved as digest.tar for the directory transport.
This is misleading as the config file is not a tar archive.
Dropped the .tar extension from all files now, including blobs.
The user can use a tool like file to determine the format of the files in the directory.
Signed-off-by: umohnani8 <umohnani@redhat.com>
Add an LayerInfosForCopy() method to source images which gives them a
chance to provide updated values for the blobsums contained in the
image's manifest, if they want to. Returning `nil` implies that they
have no changes to suggest compared to what's in the manifest.
When copying an image, if we can update the manifest with those new
values during copying, do so. If we have new values, but we can't
update the manifest, copying fails.
Update storageImageSource to return its manifest and reference in
unmodified form, and supply updated blob digests via LayerInfosForCopy()
so that copying images from storage works.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
This mirrors GetManifest, and allows / requires signatures to be per-instance.
Also add implementations in docker: and atomic:, the only transports which
support both manifest lists and signatures.
Does not change behavior yet, the only user always specifies nil
instanceDigest.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This will make the code paths more uniform for consumers of the
primary manifest and the manifest instances.
(Having an explicit support for manifest instances is necessary
for transports like docker-daemon: / oci-archive:, which
contain several images but setting up an ImageSource is very
expensive, or which don't even allow referencing images by digest.)
This is a direct replacement of GetTargetManifest, and should
not change behavior; notably the OCI implementation is still
blindly guessing the manifest type although it is probably
available in the index.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This allows us to provide in the image interfaces a method of providing
an error at close time. This is only currently used in a few situations.
Signed-off-by: Erik Hollensbe <github@hollensbe.org>
golint nowadays rejects it with
> error strings should not be capitalized or end with punctuation or a newline
(… but it accepts leading capitalization anyway, so this does not change
any of the many instances of that.)
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
https://github.com/containers/image/pull/139 has changed the semantics
to always expect a value from an ImageSource; that’s not really great
when the source has no idea, but we don’t have optionals in Golang to
distinguish between a nonsensical "" and unavailable, so this is at
least making the documentation consistent.
Hopefully ImageSources which call manifest.GuessMIMEType in GetManifest
will not proliferate…
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Use -1 because 0 is, technically, a valid size of a blob.
Also remove a FIXME about (docker save) tarballs, now we know that we
will need to make a temporary, seekable, copy.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
For lookaside signature store, and separating the read and write URLs,
we need to set up read-only and read-write states differently; having
read-write “delete” in dockerImageSource is incovenient.
In tue future, ImageSource.Delete will be a really poor fit for
docker-daemon:, where initializing the ImageSource causes the tarball
to be copied from the daemon. We could instead implement the
docker-daemon source so that it only copies the tarball on demand, but
not sharing the object is much simpler.
This leaves the Docker implementation in docker_image_src.go to make
reviewing easier.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This allows the selection to be consistent across GetManifest and
GetSignatures (which will be needed by Docker lookaside).
The API change causes lots of churn, but ultimately it just moves the
real origin of the value from image.FromSource() to transport.NewImageSource(),
both of which are static for the life of the ImageSource.
Does not change behavior.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
In principle this may return incorrect results if the underlying file is
replaced between the two calls, in practice computing and looking up the
path twice is just unnecessary overhead.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This improves on the dirReference encapsulation, and makes the methods
clearly related to dirReference instead of arbitrary strings.
Does not change behavior.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This minimizes transport-specific knowledge in image name parsing
(as in cmd/skopeo/utils.go) and allows separation of reference parsing
and their use.
Existing public NewImage... API has been removed; callers are expected
to use any of
* types.ImageTransport.ParseReference().NewImage...
(if they have a general string)
* transportpackage.ParseReference().NewImage...
(if they have a transport-specific string)
* transportpackage.NewReference().NewImage...
(if they have transport-specific raw values)
This usually adds an extra error checking step for the
ParseReference/NewReference call compared to the previous code; this is
considered not a big loss, especially because reporting “the reference
is invalid” and “the reference looks valid but connecting/using it
failed” as distinct failure modes seems quite useful for users.
The references are currently one-way (you can get a types.Image* from an
ImageReference, but not the other way around); that will be fixed soon.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This is somewhat better typed, and avoids unnecessary roundtrips using
strings when both the producer and consumer want a reference.Named value
(like in PolicyContext.requirementsForImage).
This also forces us to explicitly handle IntendedDockerReference()
returning nil, when before we could rely on it returning "", which would
then be rejected by reference.ParseNamed as invalid input; anyway,
handling that case specially just allows for better error messages.
This adds two FIXMEs about error messages which do not tell the user
which image is being rejected; that will be fixed in the future
generalized reference work.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Miloslav Trmač