This *should* be generally safe since we got most/all downstream consumers to pin to a release (or they're building bashbrew from the same commit they're getting the script from).
Docker media types inside OCI media types is semantically fine (and Docker sure doesn't care), but technically invalid, so this increases our validation from just the manifest media type all the way down to the config and layer media types also.
In many, many, many scripts I have encoded the horrible assumption that `ArchDockerFroms` will ensure our Git repository is fetched (which is a side-effect of that function). With the implementation of `Builder: oci-import`, that side effect is not necessary for returning an accurate result in these new images. Instead of working around this in bad ways, I've decided to finally bite the bullet and add an explicit `bashbrew fetch` command that can make sure all the underlying Git commits are fully fetched into the local cache.
In the case of base images (`debian`, `alpine`, `ubuntu`, etc), using a `Dockerfile` as our method of ingestion doesn't really buy us very much. It made sense at the time it was implemented ("all `Dockerfile`, all the time"), but at this point they're all some variation on `FROM scratch \n ADD foo.tar.xz / \n CMD ["/bin/some-shell"]`, and cannot reasonably be "rebuilt" when their base image changes (which is one of the key functions of the official images) since they _are_ the base images in question.
Functionally, consuming a tarball in this way isn't _that_ much different from consuming a raw tarball that's part of, say, an OCI image layout (https://github.com/opencontainers/image-spec/blob/v1.0.2/image-layout.md) -- it's some tarball plus some metadata about what to do with it.
For less trivial images, there's a significant difference (and I'm not proposing to use this for anything beyond simple one-layer base images), but for a single layer this would be basically identical.
As a more specific use case, the Debian `rootfs.tar.xz` files are currently [100% reproducible](https://github.com/debuerreotype/debuerreotype). Unfortunately, some of that gets lost when it gets imported into Docker, and thus it takes some additional effort to get from the Docker-generated rootfs back to the original debuerreotype-generated file.
This adds the ability to consume an OCI image directly, to go even further and have a 100% fully reproducible image digest as well, which makes it easier to trace a given published image back to the reproducible source generated by the upstream tooling (especially if a given image is also pushed by the maintainer elsewhere).
Here's an example `oci-debian` file I was using for testing this:
Maintainers: Foo (@bar)
GitRepo: https://github.com/tianon/docker-debian-artifacts.git
GitFetch: refs/heads/oci-arm32v5
Architectures: arm32v5
GitCommit: d6ac440e7760b6b16e3d3da6f2b56736b9c10065
Builder: oci-import
File: index.json
Tags: bullseye, bullseye-20221114, 11.5, 11, latest
Directory: bullseye/oci
Tags: bullseye-slim, bullseye-20221114-slim, 11.5-slim, 11-slim
Directory: bullseye/slim/oci
This command will, given a remote image reference, look up the list of platforms from it and match them to supported bashbrew architectures (providing content descriptors for each).
Also, refactor registry code to be more correct: previously, this couldn't fetch from Docker without `DOCKERHUB_PUBLIC_PROXY` (see `registry-1.docker.io` change) and was ignoring content digests. Now it works correctly with or without `DOCKERHUB_PUBLIC_PROXY`, verifies the size of every object it pulls, verifies the digest, _and_ should continue working with the in-progress Moby containerd-integration (where the local image ID becomes the digest of the manifest or index instead of the digest of the config blob as it is today).
This time, they are distinct implementations because the problem they are solving is inherently different.
For listing children of a given name, we *have* to walk the entire library (since we only have tag -> FROM mappings, not the reverse, which is fundamentally the question that "children" answers).
On the flip side, listing the parents of a given name is as straightforward as looking up the FROM values and walking until we can't anymore.
In my own testing, these new implementations are significantly more correct, and handle more edge cases (including things we couldn't support before like `bashbrew children --depth=1 scratch`, `bashbrew children mcr.microsoft.com/windows/servercore`, etc).
They also more correctly handle edge cases like tags that are `FROM` a "`SharedTag`" such that they don't walk up/down both sides of the tree (for example, `orientdb:3.2` -> `FROM eclipse-temurin:8-jdk`, which is both Windows *and* Linux, even though `orientdb:3.2` is Linux-only).
This patch adds a new `Builder` entry to the RFC2822 image manifest
files, allowing individual image components to opt into buildkit
support, using docker buildx.
Additionally, docker build is changed to explicitly set
DOCKER_BUILDKIT=0 to force a non-buildkit build in preparation for
the switching default in any upcoming docker versions.
Signed-off-by: Justin Chadwell <me@jedevc.com>
I've got a very pathological branch of a certain repository where this makes a parallelized (four concurrent invocations) generation change from ~3 _minutes_ down to ~13 _seconds_.
Tianon Gravi