Commit Graph

13 Commits

Author SHA1 Message Date
Kir Kolyshkin a4d8f720a2 Format sources with gofumpt
gofumpt is a superset of gofmt, enabling some more code formatting
rules.

This commit is brought to you by

	gofumpt -w .

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2023-05-26 16:17:31 -07:00
Giuseppe Scrivano f98fa3967d
chunked: use io.github.containers
we do not own containers.io so let's use io.github.containers, since
the project is part of the containers organization under github.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2022-11-03 12:48:45 +01:00
Miloslav Trmač 5e410ef763 Misc individual warning fixes
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-14 17:17:53 +02:00
Giuseppe Scrivano 33f973c732
Revert "chunked: use valyala/gozstd"
This reverts commit 5bb6d8e65e.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2022-01-17 16:16:46 +01:00
Giuseppe Scrivano 198820877c
pkg/chunked: add support for sparse files
automatically detect holes in sparse files (the threshold is hardcoded
at 1kb for now) and add this information to the manifest file.

The receiver will create a hole (using unix.Seek and unix.Ftruncate)
instead of writing the actual zeros.

Closes: https://github.com/containers/storage/issues/1091

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2022-01-13 13:32:13 +01:00
Giuseppe Scrivano 5bb6d8e65e
chunked: use valyala/gozstd
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2022-01-10 11:28:15 +01:00
Giuseppe Scrivano 0621da79cc
chunked: improve json parsing
reduce the number of allocations done by the parser by reading into a
bytes.Buffer.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2022-01-10 11:27:42 +01:00
Giuseppe Scrivano 048f7c08ad
chunked: use json-iterator
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2022-01-10 11:27:42 +01:00
Giuseppe Scrivano 96fc5c872a
chunked: omit empty fields in json manifest
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2022-01-07 21:28:16 +01:00
Giuseppe Scrivano bfd9c8046e
chunked: chunk deduplication
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2022-01-07 21:28:15 +01:00
Giuseppe Scrivano 22ba9b01ef
chunked: add infra to dedup from partial file
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-12-24 13:28:25 +01:00
Giuseppe Scrivano 2855d17137
chunked: estargz support
in addition to zstd:chunked, add support for the estargz format.

estargz is maintained at github.com/containerd/stargz-snapshotter

Images using estargz can be used on old clients and registries that
have no support for the zstd compression algorithm.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-08-25 16:16:20 +02:00
Miloslav Trmač 92b63ead41 Split pkg/chunked.ZstdCompressor into a separate subpackage
containers/image/pkg/compression depends on that function, so it's
highly undesirable for it to drag in all of containers/storage.

So, move it into a separate subpackage, along with its exclusive
dependency tree.

Code that is called both from the new pkg/chunked/compressor
and the original pkg/chunked has been moved into pkg/chunked/internal ,
so that we don't expose any new public API. That move was made
purely mechanically without any concern for conceptual consistency,
to minimize the size of the dependency (and not to spend time on
that until the concept is proven to be useful).

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2021-07-23 21:43:52 +02:00