... to prevent panics if the value does not contain a :, or other unexpected
values (e.g. a path traversal).
Don't bother on paths where we computed the digest ourselves, or it is already trusted
for other reasons.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This will allow imports of errors and pkg/errors to
coexist in a single package, avoid adding accidental
new uses via errors.New(), make any potential new
additions more visible in reviews.
Files that can import errors only (typically
because they only use errors.New) have been updated to
that import instead of renaming.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This means we won't save the stack, which is cheaper
(and possibly might break callers' format strings that
want to print the stack, but we never promised the stack
to be available).
Use either fmt.Errorf, or errors.New (usually as a local
edit, not carring about errors.new vs. pkg/errors.New;
that's going to be cleaned up later).
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Add the manifest.List interface, and implementations for OCIv1 Index and
Docker Schema2List documents.
Add an instanceDigest parameter to PutManifest(), PutSignatures(), and
LayerInfosForCopy, for symmetry with GetManifest() and GetSignatures().
Return an error if the instanceDigest is supplied to destinations which
don't support them, and add stubs that do so even to the transports
which would support it, so that we don't break compilation here.
Add a MultipleImages flag to copy.Options, and if the source for a copy
operation contains multiple images, copy all of the images if we can.
If we can't copy them all, but we were told to, return an error.
Use the generic manifest list API to select a single image to copy from
a list, so that we aren't just limited to the Docker manifest list
format for those cases.
When guessing at the type of a manifest, if the manifest contains a list
of manifests, use its declared MIME type if it included one, else assume
it's an OCI index, because an OCI index doesn't include its MIME type.
When copying, switch from using an encode-then-compare of the original
and updated versions of the list to checking if the instance list was
changed (one of the things we might have changed) or if its type has
changed due to conversion (the other change we might have made). If
neither has changed, then we don't need to change the encoded value of
the manifest.
When copying, when checking for a digest mismatch in a target image
reference, ignore a mismatch between the digest in the reference and the
digest of the main manifest if we're copying one element from a list,
and the digest in the reference matches the digest of the manifest list.
When copying, if conversion of manifests for single images is being
forced, convert manifest lists to the corresponding list types.
When copying, supply the unparsed top level to Commit() by attaching the
value to the context.Context.
Support manifest lists in the directory transport by using the instance
digest as a prefix of the filename used to store a manifest or a piece
of signature data.
Support manifest lists in the oci-layout transport by accepting indexes
as we do images, and stop guessing about Platform values to add to the
top-level index.
Support storing manifest lists to registries in the docker: transport by
using the manifest digest when we're writing one image as part of
pushing a list of them, and by using the instance digest when reading or
writing signature data, when one is specified, or the cached digest of
the non-instanced digest when one is not specified.
Add partial support for manifest lists to the storage transport: when
committing one image from a list into storage, also add a copy of the
manifest list by extracting it from the context.Context. The logic is
already in place to enable locating an image using any of multiple
manifest digests.
When writing an image that has an instanceDigest value (meaning it's a
secondary image), don't try to generate a canonical reference to add to
the image's list of names if the reference for the primary image doesn't
contain a name. That should only happen if we're writing using just an
image ID, which is unlikely, but we still need to handle it.
Avoid computing the digest of the manifest, or retrieving the
either-a-tag-or-a-digest value from the target reference, if we're given
an instanceDigest, which would override them anyway.
Move the check for non-nil instanceDigest values up into the main
PutSignatures() method instead of duplicating it in the per-strategy
helpers.
Add mention of the instanceDigest parameter and its use to various
PutManifest, PutSignatures, and LayerInfosForCopy implementations and
their declarations in interfaces.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
... so that major-version-aware Go module import
(as opposed to vX.Y.Z+incompatible, which does not allow different
packages to use different versions) works right.
Also requires adding some more GO111MODULE=on options to Makefile.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
- Network IO paths should react to cancels now.
- File IO paths generally still won't.
- `SystemContext` objects have been renamed to `sys` to leave `ctx`
available for the stdlib context objects.
Signed-off-by: Mike Lundy <mike@fluffypenguin.org>
The config file was being saved as digest.tar for the directory transport.
This is misleading as the config file is not a tar archive.
Dropped the .tar extension from all files now, including blobs.
The user can use a tool like file to determine the format of the files in the directory.
Signed-off-by: umohnani8 <umohnani@redhat.com>
... and use them for choosing an image from a manifest list, and
verifying whether an image is acceptable in a MustMatchRuntimeOS()
destination. Propagate the types.SystemContext through the
call stack as necessary.
This adds no users and seems not all that important, but after
we re-enable fetching manifest lists, docker_transport_test.go does a
"//busybox".NewImage(), which nowadays mean copying from a manifest list,
and then we need this override to keep tests working on non-Linux platforms
for which Docker does not publish images.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Remove the .Close() methods from UnparsedImage/Image, which closed the
underlying ImageSource. Instead, just require the caller to ensure
that the ImageSource is not closed as long as the UnparsedImage/Image
are used.
This allows using several independent UnparsedImage/Image instances
for a shared ImageSource; notably independent Image objects for the
individual image instances in a manifest list. (copy.Image is already
simpler although it is only using a single instance.)
To keep ImageReference.NewImage simple and not to break all the external
callers of this, also add a simple ImageCloser wrapper which retains
the ImageSource closing functionality, and return it from image.FromSource
and ImageReference.NewImage implementations.
(It's very likely many of the NewImage callers would be surprised by how this
handles manifest lists, and it is very tempting to break this API, at least
by renaming, to force the callers to consider this; however, this would be
better done after eliminating the need of ImageReference.NewImage entirely,
by replacing the specialized types.Image extensions with something else, first.)
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Gives the user the option to switch between the oci, v2s1, and v2s2 manifests
Add directory creation if the directory doesn't exist
Add overwrite functionality only if the directory was a container image directory
Feature added for the pulp team
Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
The requestedManifestMIMETypes parameter was added because a destination
might not support all manifest MIME types that the the source supports,
but the original use case now passes all manifest types and lets
containers/image convert internally. In generally, internal conversion
may be more comprehensive, is more predictable, and avoids bypassing
internal checks.
Fixes: #331
Signed-off-by: Owen W. Taylor <otaylor@fishsoup.net>
This means that image.FromUnparsedImage can now fail, OTOH later methods
which refer to a manifest cannot. This is an intermediate step to also
parsing the manifest at that time already.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Now that types.UnparsedImage is split from types.Image, also split the
implementations. In particular, copy.Image uses an UnparsedImage for
signature verification.
This structural separation allows us to remove the “It is essential for
signature verification”… comments all over the place in favor of a
single one in the choke point where an UnparsedImage turns into a full
genericImage.
Also, split the manifest type guessing (which involves parsing) so that
it does not happen in UnparsedImage. This needs ugly fields like
trueManifestMIMETypeSet, which will go away momentarily.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
For lookaside signature store, and separating the read and write URLs,
we need to set up read-only and read-write states differently; having
read-write “delete” in dockerImageSource is incovenient.
In tue future, ImageSource.Delete will be a really poor fit for
docker-daemon:, where initializing the ImageSource causes the tarball
to be copied from the daemon. We could instead implement the
docker-daemon source so that it only copies the tarball on demand, but
not sharing the object is much simpler.
This leaves the Docker implementation in docker_image_src.go to make
reviewing easier.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This allows the selection to be consistent across GetManifest and
GetSignatures (which will be needed by Docker lookaside).
The API change causes lots of churn, but ultimately it just moves the
real origin of the value from image.FromSource() to transport.NewImageSource(),
both of which are static for the life of the ImageSource.
Does not change behavior.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
... instead of Docker-specific certPath and tlsVerify.
Also invert the sense of tlsVerify to make the default secure.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Use filepath.Clean() to verify that the paths are in a canonical format,
to catch simple and easy to detect mistakes like double slashes etc.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This improves on the dirReference encapsulation, and makes the methods
clearly related to dirReference instead of arbitrary strings.
Does not change behavior.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This e.g. rejects relative paths for directories.
Notably docker: transport is the only one which does not do any real
validation; we could copy&paste the character ranges and lengths
directly from docker/distribution/reference, which feels fairly ugly.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Implement PolicyConfigurationIdentity and PolicyConfigurationNamespaces
for all transports, which makes them all possible to use in policy
configurations.
Then use Transport().Name instead of hard-coded "docker" in policy
transport lookup.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This makes the core policy lookup logic Docker-independent,
with the underlying Docker-specific implementation moved to
docker/policyconfiguration/naming.go.
Does not change behavior, only docker: and atomic: transports
currently implement the policy configuration naming, in a compatible
way, and the policy configuration is still not transport-aware. That
happens next.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This will consolidate Docker reference usage to all go through
types.ImageReference.
No users yet, will be migrated imminently.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This minimizes transport-specific knowledge in image name parsing
(as in cmd/skopeo/utils.go) and allows separation of reference parsing
and their use.
Existing public NewImage... API has been removed; callers are expected
to use any of
* types.ImageTransport.ParseReference().NewImage...
(if they have a general string)
* transportpackage.ParseReference().NewImage...
(if they have a transport-specific string)
* transportpackage.NewReference().NewImage...
(if they have transport-specific raw values)
This usually adds an extra error checking step for the
ParseReference/NewReference call compared to the previous code; this is
considered not a big loss, especially because reporting “the reference
is invalid” and “the reference looks valid but connecting/using it
failed” as distinct failure modes seems quite useful for users.
The references are currently one-way (you can get a types.Image* from an
ImageReference, but not the other way around); that will be fixed soon.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Miloslav Trmač