Flush the entire input tarball to the output in the zstd:chunked
stream writer. This is needed to include any trailing zeros that
affect the uncompressed digest.
Closes: https://github.com/containers/storage/issues/1771
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
close the pipe immediately when the file is closed to avoid a possible
deadlock if the other end is waiting for EOF on the writer side.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
change the file format to store the tar-split as part of the
zstd:chunked image. This will allow clients to rebuild the entire
tarball without having to download it fully.
also store the uncompressed digest for the tarball, so that it can be
stored into the storage database.
Needs: https://github.com/containers/image/pull/1976
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
gofumpt is a superset of gofmt, enabling some more code formatting
rules.
This commit is brought to you by
gofumpt -w .
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
It is only used internally, and this avoids a warning
about a conflict with io.ByteReader.
Should not change behavior.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Go 1.18's testing package is adamant that it doesn't support the %w
specifier for wrapping errors, which makes sense because it outputs
text rather than producing an error object like fmt.Errorf() does.
Change attempts to use %w to %v.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
automatically detect holes in sparse files (the threshold is hardcoded
at 1kb for now) and add this information to the manifest file.
The receiver will create a hole (using unix.Seek and unix.Ftruncate)
instead of writing the actual zeros.
Closes: https://github.com/containers/storage/issues/1091
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
in addition to zstd:chunked, add support for the estargz format.
estargz is maintained at github.com/containerd/stargz-snapshotter
Images using estargz can be used on old clients and registries that
have no support for the zstd compression algorithm.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
containers/image/pkg/compression depends on that function, so it's
highly undesirable for it to drag in all of containers/storage.
So, move it into a separate subpackage, along with its exclusive
dependency tree.
Code that is called both from the new pkg/chunked/compressor
and the original pkg/chunked has been moved into pkg/chunked/internal ,
so that we don't expose any new public API. That move was made
purely mechanically without any concern for conceptual consistency,
to minimize the size of the dependency (and not to spend time on
that until the concept is proven to be useful).
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Giuseppe Scrivano