especially focus on those from client.HandleErrorResponse, which may
be very unhelpful, e.g. "unknown: Not Found".
In general, make sure the error includes an expanded Docker reference,
or a full URL, or a hostname + path.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Add an LayerInfosForCopy() method to source images which gives them a
chance to provide updated values for the blobsums contained in the
image's manifest, if they want to. Returning `nil` implies that they
have no changes to suggest compared to what's in the manifest.
When copying an image, if we can update the manifest with those new
values during copying, do so. If we have new values, but we can't
update the manifest, copying fails.
Update storageImageSource to return its manifest and reference in
unmodified form, and supply updated blob digests via LayerInfosForCopy()
so that copying images from storage works.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
This mirrors GetManifest, and allows / requires signatures to be per-instance.
Also add implementations in docker: and atomic:, the only transports which
support both manifest lists and signatures.
Does not change behavior yet, the only user always specifies nil
instanceDigest.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This will make the code paths more uniform for consumers of the
primary manifest and the manifest instances.
(Having an explicit support for manifest instances is necessary
for transports like docker-daemon: / oci-archive:, which
contain several images but setting up an ImageSource is very
expensive, or which don't even allow referencing images by digest.)
This is a direct replacement of GetTargetManifest, and should
not change behavior; notably the OCI implementation is still
blindly guessing the manifest type although it is probably
available in the index.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This package is used in authenticating a user for kpod login
and can be used for authentication in kpod push, pull etc.
Signed-off-by: umohnani8 <umohnani@redhat.com>
This has two impacts:
1. In the case where a request succeeds, break out of the loop so as not to
continue fetching unnecessary blobs.
2. Previously it was possible for a request to fail after succeeding in a
previous iteration, which would error the function, despite a successful
fetch.
Signed-off-by: Will Martin <wmartin@pivotal.io>
The requestedManifestMIMETypes parameter was added because a destination
might not support all manifest MIME types that the the source supports,
but the original use case now passes all manifest types and lets
containers/image convert internally. In generally, internal conversion
may be more comprehensive, is more predictable, and avoids bypassing
internal checks.
Fixes: containers/image#331
Signed-off-by: Owen W. Taylor <otaylor@fishsoup.net>
If we are supplied a digested reference, use it instead of possibly
fetching the manifest. This allows using signatures which are not
protected even if the manifest is, in some situations; in ordinary
operation accessing the whole image, this does not make any difference
because we are fetching the manifest either way.
To ensure consistent behavior, only use the reference digest if it uses
the digest.Canonical algorithm, which has been used so far. This could
be perhaps relaxed in the future.
Note that this does not promise that GetSignatures will never need
authentication—e.g. for X-Registry-Supports-Signatures this already
depends on the server, and for the sigstore lookaside authentication
support may be added in the future.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This is primarily the only documentation of the sigstore layout;
in addition it comments on the OpenShift API master REST API and the
OpenShift docker/distribution API extension.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This is provided by the OpenShift-integrated registry. This is
equivalent to the atomic: transport (in the “openshift”) subpackage, but
it requires less code and notably does not require an OpenShift login
context to be configured.
See https://github.com/openshift/origin/pull/12504 and
https://github.com/openshift/openshift-docs/pull/3556 for more
information on this API extension.
To preserve compatibility, we always check for a configured lookaside
sigstore first; if that is set up, we use the lookaside and ignore the
registry-native signature storage. Usually the user would not bother to
set up the lookaside, and use the native mechanism.
The code is mostly trivial; the only non-obvious aspect is the loop in
putSignaturesToAPIExtension, which is a pretty direct translation of
openshiftImageDestination.PutSignatures.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This will allow using paths starting with /extensions/v2/ in the future.
Also, we can now name the parameter “path” instead of the incorrect
“url”, and the “path” name actually matches (there is no magic
/v2/ adding involved).
Also eliminates duplication of the ping paths (in logging code only).
Does not change behavior.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This allows us to provide in the image interfaces a method of providing
an error at close time. This is only currently used in a few situations.
Signed-off-by: Erik Hollensbe <github@hollensbe.org>
This replaces the copy of github.com/docker/docker/reference in the same
place, which we have just gotten rid of, and allows using this package
even in consumers which insist on an incompatible version of
docker/distribution.
The copy has been edited to drop a reference to
github.com/docker/distribution/digestset .
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This is an intermediate step which will eventually go away.
The goal of this PR is to get rid of c/i/docker/daemon/reference and to
replace uses of it by direct calls to docker/distribution/reference.
We can't do that safely and easily, because the two have different
semantics for reference.Named.Name() and reference.Named.String(): we
return a minimized version, e.g. "busybox", upstream returns an expanded
version, e.g. "docker.io/library/busybox".
BEFORE this commit the difference is hidden by using
docker/distribution/reference.WithName, which allows using the minimized
version, and works with it correctly; but because we want to use the
upstream canonicalization code, which will change semantics, we can't
just mix and match.
To make the distinction explicit, this commmit adds an X to ALL public
names from c/i/docker/daemon/reference. E.g. a reference.XNamed type,
which has methods XName and XString.
This is pretty large, but does not change behavior at all. By
inspection it is clear to see that reference.XNamed and subtypes does
not expose any of the non-X, conflicting, method names.
Using e.g.
> git diff --word-diff-regex=.|grep -F '{+'|grep -v '^\([^{]\|{+X+}\)*{\?$'
it is possible to see that most lines in this diff only add a single X
letter, and manually inspect the few lines which don't match the regexp.
The only REALLY new code is an explicit definition of namedRef.XName()
and namedRef.XString(), and two newly added casts to namedRef in cases
where we need to use the underlying distreference.Reference within
a reference.XNamed value. Strictly speaking these changes change
behavior, in that third-party implementations of reference.XNamed are no
longer accepted; but we broke them by renaming at all.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
golint nowadays rejects it with
> error strings should not be capitalized or end with punctuation or a newline
(… but it accepts leading capitalization anyway, so this does not change
any of the many instances of that.)
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
https://github.com/containers/image/pull/139 has changed the semantics
to always expect a value from an ImageSource; that’s not really great
when the source has no idea, but we don’t have optionals in Golang to
distinguish between a nonsensical "" and unavailable, so this is at
least making the documentation consistent.
Hopefully ImageSources which call manifest.GuessMIMEType in GetManifest
will not proliferate…
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
* It's quite useful to have error types exported so that libraries
can avoid substring matchers
Signed-off-by: Claudia Beresford <cberesford@pivotal.io>
Allows configuring "sigstore" (for RW) and "sigstore-write" (write-only)
for images/repositories/namespaces/registries in
/etc/containers/repositories.d/*.yaml .
If configured, uses them to store signatures in the docker: transport.
Also includes documentation in docs/registries.d.md.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Use -1 because 0 is, technically, a valid size of a blob.
Also remove a FIXME about (docker save) tarballs, now we know that we
will need to make a temporary, seekable, copy.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
For lookaside signature store, and separating the read and write URLs,
we need to set up read-only and read-write states differently; having
read-write “delete” in dockerImageSource is incovenient.
In tue future, ImageSource.Delete will be a really poor fit for
docker-daemon:, where initializing the ImageSource causes the tarball
to be copied from the daemon. We could instead implement the
docker-daemon source so that it only copies the tarball on demand, but
not sharing the object is much simpler.
This leaves the Docker implementation in docker_image_src.go to make
reviewing easier.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This allows the selection to be consistent across GetManifest and
GetSignatures (which will be needed by Docker lookaside).
The API change causes lots of churn, but ultimately it just moves the
real origin of the value from image.FromSource() to transport.NewImageSource(),
both of which are static for the life of the ImageSource.
Does not change behavior.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
... instead of Docker-specific certPath and tlsVerify.
Also invert the sense of tlsVerify to make the default secure.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This improves the dockerReference encapsulation, and makes it more clear
that the reference.IsNameOnly case can not happen.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This minimizes transport-specific knowledge in image name parsing
(as in cmd/skopeo/utils.go) and allows separation of reference parsing
and their use.
Existing public NewImage... API has been removed; callers are expected
to use any of
* types.ImageTransport.ParseReference().NewImage...
(if they have a general string)
* transportpackage.ParseReference().NewImage...
(if they have a transport-specific string)
* transportpackage.NewReference().NewImage...
(if they have transport-specific raw values)
This usually adds an extra error checking step for the
ParseReference/NewReference call compared to the previous code; this is
considered not a big loss, especially because reporting “the reference
is invalid” and “the reference looks valid but connecting/using it
failed” as distinct failure modes seems quite useful for users.
The references are currently one-way (you can get a types.Image* from an
ImageReference, but not the other way around); that will be fixed soon.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This is somewhat better typed, and avoids unnecessary roundtrips using
strings when both the producer and consumer want a reference.Named value
(like in PolicyContext.requirementsForImage).
This also forces us to explicitly handle IntendedDockerReference()
returning nil, when before we could rely on it returning "", which would
then be rejected by reference.ParseNamed as invalid input; anyway,
handling that case specially just allows for better error messages.
This adds two FIXMEs about error messages which do not tell the user
which image is being rejected; that will be fixed in the future
generalized reference work.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
The tag is a derived value from the reference, so storing it separately
is redundant.
More importantly, this also modifies CanonicalDockerReference and
IntendedDockerReference to call the reference.Named.String() method for
formatting, so that we no longer corrupt
busybox@sha256:a59906e33509d14c036c8678d687bd4eec81ed7c4b8ce907b888c607f6a1e0e6
to busybox:sha256:a599…
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
OpenShift returns "application/json; charset=utf-8"; with greatest
respect to the strict need to care about character encoding, we don't
want to get into the business of implementing support for charset
encoding, so just drop that on the floor.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Add support to mark images for deletion from repository
Requires:
* V2 API and schema
* registry configured to allow deletes
* run registry garbage collection to free up disk space
Signed-off-by: Jhon Honce <jhonce@redhat.com>
Compute the digest ourselves, the registry is in general untrusted and
computing it ourserlves is easy enough.
The stop passing the unverifiedCanonicalDigest value around, simplifying
ImageSource.GetManifest and related code. In particular, remove
retrieveRawManifest and have internal users just call Manifest() now that
we don't need the digest.
Does not change behavior. This is a straightforward move and update of
package references, except for:
- Adding a duplicate definition of manifestSchema1 to
cmd/skopeo/copy.go. This will need to be cleaned up later, for now
preferring to make no design changes in this commit.
- Renaming parseDockerImage to NewDockerImage, to both make it public
and consistent with common golang conventions.
Miloslav Trmač
Giuseppe Scrivano
Nalin Dahyabhai
umohnani8
Tiago Scolari
Owen W. Taylor
Mrunal Patel
Michal Fojtik
Erik Hollensbe
Antonio Murdaca
George Lestaris
Crazykev
Alberto Leal
Jhon Honce