Commit Graph

53 Commits

Author SHA1 Message Date
Tianon Gravi fbd5182d27 Fix `build-order` calculation inside `diff-pr.sh`
I noticed this while reviewing the recent `kong` diff and seeing this odd bit:

```diff
diff --git a/_bashbrew-list-build-order b/kong_2/kong.tar.gz
similarity index 100%
copy from _bashbrew-list-build-order
copy to kong_2/kong.tar.gz
```

Knowing that `kong.tar.gz` is supposed to be an empty file, I wondered why `_bashbrew-list-build-order` was apparently empty, and discovered that it's because `.TagEntries` is only available/valid when we're actually passing _tags_ to `bashbrew`. 😅
2024-06-27 12:20:48 -07:00
Daniel Black 8cf0837e1e
diff-pr: ADD/COPY enumerate full list of options (#16945)
--from an option we just want to move on from as its in an
another layer.

Other options don't map to files.

Once we get a non-option they all are files.

ref:
* https://docs.docker.com/reference/dockerfile/#add
* https://docs.docker.com/reference/dockerfile/#copy
2024-06-11 11:37:38 -07:00
Joseph Ferguson 4c353acba7 Keep bashbrew fetch for oci-import fetching 2023-12-15 11:12:12 -08:00
Joseph Ferguson 886a25aff0 Swap back to ArchDockerFroms instead of bashbrew fetch for more reliable commit fetching 2023-12-11 14:48:48 -08:00
Joseph Ferguson 0c8a6f9b1a Improve diff-pr build order comparison
Use the first tag (aka least specific) before and after sorting by build order
2023-12-04 14:48:51 -08:00
Tianon Gravi 79a269637f Add support for "Builder: oci-import" in diff-pr
Also, update other scripts to use `bashbrew fetch` and `gitCache` effectively to remove unnecessary reimplementations of `BASHBREW_CACHE` default value calculation (new in bashbrew v0.1.8).

As a parting gift, add the raw list of total supported architectures to a file so it's more obvious when new ones are added or removed (like `ubuntu` losing `riscv64`).
2023-02-15 13:22:05 -08:00
Tianon Gravi 77fbe15d93 Update external-pins diffing to match normal diffing
When we run `diff-pr.sh` on PRs, it normally shows us a diff of either `amd64` or the first supported architecture.

However, when we implemented support for external pins, I was very silly and had it show us _all_ architectures.

This makes for very long diffs.

In this change, I've updated that external pins logic to match the prior logic -- in lieu of better diffing algorithms (targeting review instead of patching) that can do smarter things like collapsing deltas that are identical, I think this is the right thing to do.
2023-02-10 11:00:23 -08:00
Tianon Gravi cdc2c8a989 Add support for "external pins" in "diff-pr.sh"
This uses `crane` to download some of the JSON files of the remote images to give us at least *some* amount of diffing between updates to external pins.
2023-01-13 14:38:54 -08:00
Tianon Gravi 9f6ee11257 Slightly improved "diff-pr" output, especially around build-order
This should help avoid "empty cat" output we saw recently when something failed to fetch but was otherwise valid.

The current implementation is trying to do two things with one output:

1. normalized `bashbrew cat` diff
2. visualize `--build-order` changes

This change splits those into two separate things because combining them ends up making the diff harder to read than it should be ("changed tags + changed commit" leading to a full reordering of the diff because build-order also changed).
2022-12-13 15:51:30 -08:00
Tianon Gravi a3d5c1e9bb Rewrite instances of "pythonX.X", "pyX.X", etc in tarball listings
This leads to significantly smaller diffs in instances like major distro releases where the Python version moves from something like 3.9 to 3.10 (and a large portion of the diff is then just directory renames as a result).
2022-03-18 10:55:38 -07:00
Tianon Gravi f0edc7c298 Make "Munge PR" more intelligent 2021-09-09 16:00:57 -07:00
Tianon Gravi 5eaf242326 Adjust diff-pr.sh to no longer tolerate "bashbrew cat" failures (since we handle "new" and "deleted" images as special cases now) 2020-12-03 09:53:07 -08:00
Tianon Gravi 7e6ffbece8 Update diff-pr to scrape COPY from flattened Dockerfile so "COPY \\n foo \\n bar \\n baz" doesn't trip us up 2020-05-15 16:25:07 -07:00
Tianon Gravi 91d0285431 Move ~sorted "bashbrew cat" clone into a separate file for easier use externally 2020-05-11 07:36:40 -07:00
Tianon Gravi 9142251d50 Also update "diff-pr.sh" to be more friendly on "new-image" PRs 2020-05-07 21:51:20 -07:00
Tianon Gravi f27f59f5f8 Replace "_bashbrew-arches" and "_bashbrew-shared-tags" with a ~sorted "bashbrew cat" clone in diff-pr.sh 2020-05-07 21:28:27 -07:00
Tianon Gravi 563211d8c8 Fix "diff-pr" with multiple images 😅 2020-05-07 08:47:05 -07:00
Tianon Gravi 39cc4e2159 Add initial GitHub Actions CI 2020-05-05 11:38:15 -07:00
Joe Ferguson 529fe93da0 Fix diff-pr tar error on empty file
> tar: This does not look like a tar archive
2020-04-27 13:29:41 -07:00
Tianon Gravi 4863820eea Adjust "diff-pr.sh" to be more explicit about failure in certain scenarios (hopefully improves debuggability) 2019-07-31 13:57:17 -07:00
Tianon Gravi 4b4a88f075 Adjust "diff-pr.sh" to ignore "COPY --from=" (since it cannot reasonably access files from the referenced image/stage context) 2019-07-31 13:52:27 -07:00
Tianon Gravi c3aa695e2c Adjust "diff-pr" to be more specific about "Dockerfile.builder" explicitly (to cut down on false-positives) 2019-06-25 10:59:53 -07:00
Tianon Gravi e1a94ef563 Update "diff-pr.sh" to use the new DockerFroms function 2019-06-12 08:06:45 -07:00
Tianon Gravi de13dd5bb7 Replace "--ignore-all-space" with "--ignore-blank-lines", "--ignore-space-at-eol", and "--ignore-space-change"
See https://github.com/docker-library/official-images/pull/5859#issuecomment-489819055
2019-05-07 16:25:01 -07:00
Tianon Gravi 8df2fd5d54 Fix "diff-pr.sh" edge case 2019-04-19 13:21:45 -07:00
Tianon Gravi b1f25a69e4 Add ".dockerignore" to the list of things "diff-pr" optionally looks at 2019-04-17 14:51:14 -07:00
Tianon Gravi 48c82ce313 Fix diff-pr globbing on images that do "COPY ./..." (like neo4j) 2019-04-09 15:37:46 -07:00
Tianon Gravi de3335b278 Fix minor edge case in diff-pr 2019-03-21 15:55:12 -07:00
Tianon Gravi 510aeda20f Use "find" instead of "eval" for more defensive glob expansion 2019-03-07 15:54:05 -08:00
Tianon Gravi 413b9689b5 Update "diff-pr.sh" to ignore "--chown=" on COPY/ADD 2018-11-20 14:10:37 -08:00
Tianon Gravi 087cef7a9f Add support for "File:" in manifest files
See https://github.com/docker-library/go-dockerlibrary/pull/16 for format implementation details.
2018-10-22 16:06:21 -07:00
Tianon Gravi 667443bb39 Revert "breaks-rewrites" addition to diff-pr -- it causes more issues generally than it helps 2018-10-15 12:50:34 -07:00
Tianon Gravi 733d0bfaef Add explicit "--break-rewrites" value to fix overaggressive splitting 2018-10-11 10:21:15 -07:00
Tianon Gravi e21fe49dc9 Update "diff-pr" flags to be more aggressive about looking for similar files 2018-10-03 14:21:36 -07:00
Tianon Gravi dbfb3f3cd5 Add "--ignore-all-space" to "diff-pr.sh"
For the purposes of the review this script facilitates, whitespace is irrelevant.
2018-08-03 11:21:27 -07:00
Tianon Gravi f8555d46fd Update "diff-pr.sh" to strip leading "./" from tar entries 2018-04-20 09:58:15 -07:00
Tianon Gravi f1d5ef8b13 Let diff-pr be arch-aware 2018-03-20 10:59:32 -07:00
Tianon Gravi 6770ab304a Use "sort -u" in diff-pr to remove duplicates due to SharedTags 2017-11-01 12:19:08 -07:00
Tianon Gravi a92a4e8429 Add another "uninteresting directory" 2017-09-15 16:22:58 -07:00
Tianon Gravi b158337e0c Use the first arch of each entry in "diff-pr.sh"
This allows us to diff "windows-amd64"-only images, but doesn't solve the more general problem of how to cleanly show a diff across all architectures.
2017-07-13 10:35:50 -07:00
yosifkit dcfe2b8490 Merge pull request #3059 from infosiftr/diff-pr-shared-tags
Add "SharedTags" support to "diff-pr.sh"
2017-07-05 16:29:38 -07:00
Tianon Gravi f555445d48 Adjust template whitespace to match the rest of the file 2017-06-14 11:13:03 -07:00
Tianon Gravi 2ec4c7cc64 Fix "DockerFrom" failure on "windows-amd64"-only tags 2017-06-14 10:22:45 -07:00
Tianon Gravi cc73ad07d0 Add "SharedTags" support to "diff-pr.sh"
Example output of this bashbrew template (assuming `hello-world` had been converted to use `SharedTags`):

```console
$ bashbrew cat --format "$sharedTagsListTemplate" --all | grep -vE '^$'
hello-world:latest -- hello-world:linux, hello-world:nanoserver
```
2017-06-13 13:05:38 -07:00
Tianon Gravi 728e22d0d1 Add an "arches list" to "diff-pr" output
This should make it easier to see architecture changes (additions, removals, etc).
2017-06-08 15:34:50 -07:00
Tianon Gravi 764fc2406b Add some very basic initial support for BASHBREW_ARCH in "diff-pr.sh" 2017-06-06 09:33:19 -07:00
Tianon Gravi c0d1c0c69e Merge pull request #2558 from infosiftr/uninteresting-tarball-content
Remove certain paths from tarball output in "diff-pr"
2017-03-29 14:57:29 -07:00
Tianon Gravi 1ca9e62e9c Add "sort -V" to our "bashbrew list" for deterministic ordering (which should lead to simpler diffs overall) 2017-02-27 15:50:17 -08:00
Tianon Gravi 311e9d3637 Add "bashbrew list" output to "diff-pr.sh" 2017-02-24 12:48:25 -08:00
Tianon Gravi 9c8b8a0282 Remove certain paths from tarball output in "diff-pr"
These directories are simply noise in the diff (making it unnecessarily long, especially /var/lib/yum/yumdb)
2017-02-16 11:23:46 -08:00