refactor all calls to Fchown and Fchownat into a single function, and
provide a better error messages.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
I was just reading the code and I have a mental checklist item
for "invoking open without O_CLOEXEC" that triggered here.
(See also e.g.
https://github.com/containers/composefs/pull/185#discussion_r1322925050
)
It has security-relevant properties for us, xref
CVE-2024-21626 for example.
This isn't the only missing variant of this in this codebase,
just using this targeted PR to test the waters for more PRs.
Signed-off-by: Colin Walters <walters@verbum.org>
Add validation for duplicate entries. Duplicates are ignored, unless
there is a mismatch in the values. In that case, an error is returned.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
otherwise if the root is stored as "./", it ends up adding the root
node twice causing mkcomposefs to fail.
Closes: https://github.com/containers/storage/issues/1941
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
To avoid a mismatch with the C composefs library that uses isgraph()
and isprint(), do not use the unicode package but provide the expected
definition for these functions.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
always consume the tar-split data when present to avoid blocking the
producer. Previously, the tar-split data was only read when the digest
was specified.
commit 6875c9fbcf introduced the
regression.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
Increase the threshold for auto-merging parts from 128 to 1024. This change
aims to reduce the number of parts in an HTTP multi-range request, thus
increasing the likelihood that the server will accept the request.
The previous threshold of 128 often resulted in a large number of small
ranges, which could lead to HTTP multi-range requests being rejected by
servers due to the excessive number of parts.
It partially addresses the reported issue.
Reported-by: https://github.com/containers/storage/issues/1928
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
if a digest was not specified in the TOC, ignore completely the
tar-split data.
Otherwise the clients fail to pull images created before commit
b5413c2bd6.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
it solves this error with mkcomposefs when the TOC doesn't specify the
parent directory for an entry:
Error: committing the finished image: failed to put layer using a partial pull: failed to convert json to erofs: exit status 1: mkcomposefs: Parent directory missing for /usr/share/locale/ca/LC_MESSAGES/libc.mo
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
Other TOC formats don't fill the data in.
For now, this only increases memory usage, but we will
need the data soon.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This is a microptimization, we call strings.ToLower only
once, but more importantly it will make it easier to add
more fields.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
if the layer does not have a manifest TOC, just ignore it instead of
raising a warning. There is no need to create a cache file since
there is no manifest file to parse.
Closes: https://github.com/containers/storage/issues/1909
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
it can happen for any reason, like for example using a new cache file
format, in this case the file is recreated with the last version.
This is internal only and should not be displayed by default.
Closes: https://github.com/containers/storage/issues/1905
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
if the bloom filter size is zero, the "% size" operation fails with a
divide by zero.
Closes: https://github.com/containers/storage/issues/1903
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
We have the ImageSourceChunk data type, and we already
construct these values, so scan into them directly instead
of having three separate variables for the two bits of data.
Should not change behavior.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Replace it by individual variables.
Then formally deprecate the ChecksumAnnotationTarSplit field.
Should not change behavior.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Again, decrease the size of the compression code for c/image.
We will simplify this further immediately.
Should not change behavior.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
It has no non-test users any more, so decrease the
size of this package (relevant to non-c/storage
callers of c/image).
Should not change behavior.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This code path is usually never triggered because
the annotations are present; and it was broken until recently.
Remove it to simplify the code and analysis.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
the code was copied from the composefs C version:
if (noescape_space)
hex_escape = !isprint(c);
else
hex_escape = !isgraph(c);
but unicode.IsGraphic() seems to behave differently and includes the
space:
isgraph(' ') -> 0
unicode.IsGraphic(' ') -> true
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
so that the same file path is stored only once in the cache file.
After this change, the cache file measured on the fedora:{38,39,40}
images is in average ~6% smaller.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
use a bloom filter to speed up lookup of digests in a cache file.
The biggest advantage is that it reduces page faults with the mmap'ed
cache file.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
Giuseppe Scrivano