... to prevent panics if the value does not contain a :, or other unexpected
values (e.g. a path traversal).
Don't bother on paths where we computed the digest ourselves, or it is already trusted
for other reasons.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
TryReusingBlob now contains a new option `RequiredCompression` which
filters the blob by checking against a compression of the blob which is
considerd to be resued, in case `RequiredCompression` is set and `info`
of the blob being reused does not matches, no blob is returned.
Signed-off-by: Aditya R <arajan@redhat.com>
This is a subset of types.BlobInfo the transports
actually should deal with, to be much more explicit
about what the transports are responsible for.
Should not change behavior, in practice.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This is a subset of types.BlobInfo the transports
actually should deal with. Also tighten the semantics a bit,
to be much more explicit about what the transports are responsible for.
This allows us to stop tracking MIME types in storage.addedLayerInfo,
so simplify that.
Then fix the generic code so that blob annotations are not discarded
on blob reuse.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Flips dependency of internal and public manifest.List API and introduces
private manifest.List API which will be extended in future for new
features.
[NO NEW TESTS NEEDED]
[NO TESTS NEEDED]
Signed-off-by: Aditya R <arajan@redhat.com>
`github.com/pkg/errors` is deprecated since quite some time so we now
use the native error wrapping for more idiomatic golang.
Signed-off-by: Sascha Grunert <sgrunert@redhat.com>
NOTE: This makes implementation choices:
- dir, docker LOOKASIDE, ostree and storage are going to store the
generic-format blobs, and can represent any future formats
of signatures.
- docker X-R-S-S and OpenShift only accept simple signatures,
and fail when copying the other kind.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
We need something to wrap (format ID, actual data).
And a shared helper for a file/blob representation for things like
dir or c/storage.
This looks quite immature.
Tests of the non-simple signing will only be added with support for that format.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This matches internal/imagedestination/impl.Properties; it's not
quite worth it for the single value, but the consistency is
attractive.
Should not change behavior.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This sets up the precedent that all transports should primarily implement
the private interface; that will allow us to make future changes to the
private interface easier, because we can just change the public interface
wrappers in a single place instead of modifying transports - especially
as more stubs are added soonish.
Should not change behavior.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
It's very annoying that Go allows silent zero-initilization,
and we have neither mandatory constructors nor parameter labels,
i.e. a forgotten field does not cause a compile-time error. For _private_
transport implementations, relying on human review is just about acceptable.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This sets up the precedent that all transports should primarily implement
the private interface; that will allow us to make future changes to the
private interface easier, because we can just change the public interface
wrappers in a single place instead of modifying transports - especially
as more stubs are added soonish.
Should not change behavior.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This makes it clearer that dirImageDestination is only
used in a valid state.
Should not change behavior.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This will allow imports of errors and pkg/errors to
coexist in a single package, avoid adding accidental
new uses via errors.New(), make any potential new
additions more visible in reviews.
Files that can import errors only (typically
because they only use errors.New) have been updated to
that import instead of renaming.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This means we won't save the stack, which is cheaper
(and possibly might break callers' format strings that
want to print the stack, but we never promised the stack
to be available).
Use either fmt.Errorf, or errors.New (usually as a local
edit, not carring about errors.new vs. pkg/errors.New;
that's going to be cleaned up later).
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Mostly just name changes that should not change behavior,
apart from ioutil.ReadDir -> os.ReadDir avoiding per-item
lstat(2) in some cases.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Introduce internal/putblobdigest.Digester to encapsulate
the two alternatives, so that the PutBlob implementations only
need to plug it in.
Then use it throughout: let PutBlob use caller-provided digest
values, if any (and they use the right algorithm).
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Mostly remove some extra variables, instead modify stream
by wrapping it in io.TeeReader()s.
In dockerImageDestination, move more of the pipeline
construction outside of the uploadReader closure, and
use consecutiveio.TeeReader()s instead of io.MultiWriter.
Should not change (observable) behavior.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This was always sort of implied, and we now rely on that explicitly
to avoid computing the digests; it turns out the digest computation
can consume a very noticeable amount of CPU time.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Podman and other tools already add Error: to the front of returned error
message, and this ends up as a stutter.
podman pull fedora.io/fred
Trying to pull fedora.io/fred:latest...
Error: Error initializing image from source docker://fedora.io/fred:latest: invalid character '<' looking for beginning of value
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
- Use a different manifest and manifest list
- Write per-arch manifest before the top-level one
- Same for signatures
- Don't write the top-level manifest twice
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
... and acknowledge that various tests are strictly speaking
invalid, to reinforce that real callers must not pass nil.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Extend the blob info cache to also cache the name of the type of
compression used on a blob that we've seen, or specific values that
indicate that we know the blob was not compressed, or that we don't
know whether or not it was compressed.
New methods for adding known blob-compression pairs and reading
candidate locations including compression information are part of a new
internal BlobInfoCache2 interface which the library's BlobInfoCache
implementors also implement.
When we copy a blob, try to record the state of compression for the
source blob, and if we applied any changes, the blob we produced.
Make sure that when TryReusingBlob successfully uses a blob from the
blob info cache, that it provides compression information in the
BlobInfo that it returns, so that manifests can be updated to describe
layers using the correct MIME types.
When attempting to write a manifest, if a manifest can't be written
because layers were compressed using an algorithm which can't be
expressed using that manifest type, continue on to trying other manifest
formats.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Miloslav Trmač