docs: Fill in more about composefs and zstd:chunked

I've been reading more of the code and looking
at the filesystem in these modes.

Signed-off-by: Colin Walters <walters@verbum.org>

docs/containers-storage-composefs.md

@@ -30,6 +30,33 @@ convert_images = "true"
 
 This value must be a "string bool", it cannot be a native TOML boolean.
 
+## IMPLEMENTATION
+
+As is implied by the setting `use_composefs = "true"`, currently composefs
+is implemented as an "option" for the `overlay` driver. Some file formats
+remain unchanged and are inherited from the overlay driver, even when
+composefs is in use. The primary differences are enumerated below.
+
+The `diff/` directory for each layer is no longer a plain unpacking of the tarball,
+but becomes an "object hash directory", where each filename is the sha256 of its contents. This `diff/`
+directory is the backing store for a `composefs-data/composefs.blob` created for
+each layer which is the composefs "superblock" containing all the non-regular-file content (i.e. metadata) from the tarball.
+
+As with `zstd:chunked`, existing layers are scanned for matching objects, and reused
+(via hardlink or reflink as configured) if objects with a matching "full sha256" are
+found.
+
+There is currently no support for enforced integrity with composefs;
+an attempt is made to enable fsverity for the backing files and the composefs file,
+but it is not an error if unsupported. There is as of yet no defined mechanism to
+verify the fsverity digest of the composefs block before mounting; some work on that is
+ongoing.
+
+In order to mount a layer (or a full image, with all of its dependencies), any
+layer that has a composefs blob is mounted and included in the "final" overlayfs
+stack. This is optional - any layers that are not in "composefs format" but
+in the "default overlay" (unpacked) format will be reused as is.
+
 ## BUGS
 
 - https://github.com/containers/storage/issues?q=is%3Aissue+is%3Aopen+label%3Aarea%2Fcomposefs

docs/containers-storage-zstd-chunked.md

@@ -27,12 +27,25 @@ enable_partial_images = "true" | "false"
 
 Note that the value of this field must be a "string bool", it cannot be a native TOML boolean.
 
-## INTERNALS
+## IMPLEMENTATION
+
+Each layer has an associated "big data" key called `chunked-manifest-cache` that
+is a custom binary format suitable for mmap() that contains index metadata
+for each layer with the full sha256 digest of each file plus its "chunks" (as
+computed by `zstd:chunked`).
+
+When any image is pulled all existing other layers are scanned using `chunked-manifest-cache` to see if they contain a file with a matching digest. If one is found, the other file is hardlinked if `use_hardlinks = "true`",
+otherwise it is reflinked (if supported by the filesystem, or a full physical copy
+is made). There is a best-effort attempt to enable fsverity on the file if configured
+(see <https://github.com/containers/storage/issues/2017>).
+
+For more information, at the current time the file with the most information is [pkg/chunked/internal/compression.go](https://github.com/containers/storage/blob/39d469c34c96db67062e25954bc9d18f2bf6dae3/pkg/chunked/internal/compression.go).
+The above is a permanent link for stability, but be sure to check to see if there are newer changes too.
+
+## STANDARDIZATION
 
 At the current time the format is not officially standardized or documented beyond
-the comments and code in the reference implementation. At the current time
-the file with the most information is [pkg/chunked/internal/compression.go](https://github.com/containers/storage/blob/39d469c34c96db67062e25954bc9d18f2bf6dae3/pkg/chunked/internal/compression.go).
-The above is a permanent link for stability, but be sure to check to see if there are newer changes too.
+the comments and code in the reference implementation.
 
 ## BUGS