ef26a55720
Fix "arm32" vs "arm" bug
2019-05-10 17:21:35 -07:00
d1cb5996f7
Add optional "--dry-run" flag for pre-testing "heavy" pushes
2019-05-10 16:55:53 -07:00
db8b401468
Treat 401 the same as 404 when fetching a manifest
2019-05-10 16:38:41 -07:00
af23605e63
Add new "put-multiarch" script
...
This is a replacement for `bashbrew put-shared` (without `--single-arch`) that takes some shortcuts to get a pretty decent speed gain which should allow us to go back to the naïve solution of "push everything from each arch `:xyz-tag` to the multiarch repo's `:xyz-tag` and still have reasonable performance.
This was tested by pushing all ~3700 tags we currently have to https://hub.docker.com/u/trollin and for a full no-op we were around 8.5 minutes, and for a full forced push of every tag manifest (with no necessary blob mounts) we were around 45 minutes.
2019-05-09 16:53:24 -07:00
1c24c31208
Update bashbrew readme (thanks @7AC)
2019-04-04 13:25:24 -07:00
73d98151b5
First attempt to fix Travis (force Go modules "on")
2019-03-28 21:39:21 -07:00
d5559299cf
Port bashbrew to use Go modules instead of GB
...
Something in here also fixes the "missing build output on error" bug we've been seeing on Jenkins (see https://doi-janky.infosiftr.net/job/multiarch/job/arm32v7/job/rabbitmq/184/console for example).
2019-03-28 15:45:12 -07:00
26d2eab54b
Update Bashbrew builds to Go 1.12
...
(and update `manifest-tool` to 1.0.0-rc; bombs away, @estesp!)
2019-03-14 14:52:22 -07:00
079cc45216
Fix bashism
2019-03-12 22:10:05 -07:00
ccf4f480a7
Hotfix: "bashbrew build" sorting fix
2019-01-28 17:01:16 -08:00
a93f9f538f
Improve bashbrew SharedTags handling (explicitly allowed for "bashbrew cat", etc)
2019-01-28 16:16:50 -08:00
1693d9a9e6
Update go-dockerlibrary to include https://github.com/docker-library/go-dockerlibrary/pull/19
2019-01-07 16:22:36 -08:00
2dee8376c4
Update to manifest-tool 0.9.0
2018-11-26 20:22:07 -08:00
ed5b868911
Update "gpg" invocations to use "--batch"
...
See https://bugs.debian.org/913614
2018-11-14 13:08:09 -08:00
52138c6dbe
Add "naughty-from.sh" script inspired by the report of the same name
...
This verifies that the "Architectures" field of each entry in a given pull request is a subset of its parent.
2018-10-29 22:44:56 -07:00
0335845df0
Stop testing "extra commands" on bashbrew changes (so they'll succeed more often)
2018-10-29 21:47:25 -07:00
6a1a2d2c1d
Add support for "File:" in manifest files
...
See https://github.com/docker-library/go-dockerlibrary/pull/16 for format implementation details.
2018-10-22 16:06:21 -07:00
5cbc68680a
Add new "archFilter" template function
...
Example usage:
```
{{- range $i, $e := ($archSpecific | ternary (archFilter arch $.Entries) $.Entries) -}}
...
{{- end -}}
```
2018-08-23 11:37:13 -07:00
1e37e9b879
Update bashbrew release binaries to Go 1.10
2018-08-14 12:51:30 -07:00
9911b85e11
Slightly improved error message for failed template executions
2018-08-14 12:48:04 -07:00
ae92f4d748
Add a note about pre-built bashbrew binaries
2018-07-20 13:10:57 -07:00
afbe0d6f03
Improve put-shared noop behavior and add "--force" flag for pushing operations
2018-05-03 12:21:57 -07:00
15a94e48fa
Add "gpgconf --kill all" to bashbrew build Dockerfile (partially as a test for other images)
...
This helps combat the following race condition:
+ rm -rf /tmp/tmp.FHddOB
rm: can't remove '/tmp/tmp.FHddOB/S.gpg-agent.browser': No such file or directory
2018-04-06 09:28:44 -07:00
b4cc45e64f
Adjust wording
2018-03-20 10:11:34 -07:00
f3aced2ea4
Fix grep logic
2018-03-20 10:11:34 -07:00
4e74853942
Add simple Travis test for incorrect "latest" tags
...
```console
$ TRAVIS_BRANCH=master TRAVIS_PULL_REQUEST=false ./bashbrew/travis.sh
Testing master -- BUILD ALL THE THINGS!
Incorrectly 'latest' tags detected:
geonetwork:latest-postgres
Read https://github.com/docker-library/official-images#tags-and-aliases for more details.
```
2018-03-20 10:11:34 -07:00
b84de49388
Add missing newline
2018-01-24 10:12:23 -08:00
f36ef9d30b
Update "put-shared" to allow some tags to fail per-repo (and collect errors)
2018-01-24 09:57:47 -08:00
90101d3445
Also tag SharedTags (and include them in dependency calculations, etc)
...
This will include SharedTags directly in "bashbrew list", etc, but explicitly does _not_ include them during "bashbrew push". This allows for other images to be "FROM" a SharedTags value, and should be reasonably safe since any given SharedTag should only be applicable for a given architecture at most once.
2017-10-24 10:47:17 -07:00
5d0cbc967e
Update to manifest-tool v0.7.0
...
With this release, we get to drop our custom patch for pushing to `library/xxx`, and go back to using the officially released upstream binaries (which now exist for all the same architectures we support).
2017-10-06 16:12:43 -07:00
ec5b4e484e
Add new "archNamespace" template function for parsing "--arch-namespace" parameters
...
Example usage: `{{ archNamespace "windows-amd64" }}` => `winamd64`
This is intended for use in https://github.com/docker-library/docs to help create a list of "Supported architectures" for each image which links to each supported architecture repository (and if supported by Bashbrew itself, allows us to keep https://github.com/docker-library/docs isolated from Jenkins assumptions while still allowing Jenkins to contain the single-source-of-truth for arch-to-namespace mappings).
2017-10-02 12:27:10 -07:00
50e9eb1ddd
Add a "--single-arch" flag to "put-shared" for pushing SharedTags to architecture-specific namespaces
2017-09-12 16:18:27 -07:00
0da6c3c534
Minor bashbrew de-optimization
2017-09-12 13:32:45 -07:00
dca13c025b
Build manifest-tool from source to apply a minor patch
2017-09-11 15:52:01 -07:00
9aedfc4915
Add a "--dry-run" flag to "build", "tag", "push", and "put-shared"
2017-09-07 13:53:58 -07:00
1568683373
CUE LOUD GRUMBLING
2017-09-06 13:11:15 -07:00
145add96da
Fix minor typo
2017-09-05 07:36:47 -07:00
3b0c4503cb
Use the Hub v2 API for a dramatic no-op push speedup
...
This also has the benefit of only triggering automated builds when there have been actual changes to the image.
This works by comparing the Hub's `last_updated` timestamp to the `Created` timestamp of local images (in the case of `bashbrew push`), or to the `last_updated` timestamp of the images which make up the `SharedTags` and/or multiarch images (in the case of `bashbrew put-shared`).
This is a dramatic speedup because the Hub v2 API is faster (and has no auth for public repos) than the registry API. If there are any errors in the fetching of `last_updated` from the Hub, we massage the timestamps such that the push simply always happens, as before (which will be the case for pushes to non-Hub registries, for example).
2017-08-28 09:46:47 -07:00
c4108a0534
Update go-dockerlibrary to pull in the new "getenv" function
...
See https://github.com/docker-library/go-dockerlibrary/pull/15 .
2017-07-12 16:15:18 -07:00
b9d7a8b7e7
Update manifest-tool to 0.6.0 and take advantage of new "tags:" support (for increased efficiency in pushing aliases)
2017-07-05 13:42:27 -07:00
7634587e56
Add a separate ArchDockerFrom function for better template granularity
...
Here's an example diff I plan to apply over in the docs with this: (with an example output from `python`)
```diff
diff --git a/.template-helpers/generate-dockerfile-links-partial.tmpl b/.template-helpers/generate-dockerfile-links-partial.tmpl
index 5c13047a..996b9d60 100644
--- a/.template-helpers/generate-dockerfile-links-partial.tmpl
+++ b/.template-helpers/generate-dockerfile-links-partial.tmpl
@@ -4,21 +4,13 @@ This template defines the "Supported tags and Dockerfile links" portion of an im
*/ -}}
-{{- define "dockerfilePath" -}}
- {{- $dir := .ArchDirectory arch -}}
- {{- if ne $dir "." -}}
- {{- $dir -}}
- /
- {{- end -}}
- Dockerfile
-{{- end -}}
-
{{- range $i, $e := $.Entries -}}
- {{- if $e.HasArchitecture arch -}}
+ {{- $arch := ternary (ternary arch ($e.Architectures | first) ($e.HasArchitecture arch)) arch (eq arch "amd64") -}}
+ {{- if $e.HasArchitecture $arch -}}
{{- /* force bashbrew to "fetch" the GitCommit so that it is fully resolved */ -}}
- {{- $from := $.DockerFrom $e -}}
+ {{- $from := $.ArchDockerFrom $arch $e -}}
- {{- $gitRepo := $e.ArchGitRepo arch -}}
+ {{- $gitRepo := $e.ArchGitRepo $arch -}}
{{- $repoUrlBase := $gitRepo | replace "git://" "https://" | trimSuffixes ".git" -}}
{{- $isGitHub := hasPrefix "https://github.com/ " $repoUrlBase -}}
@@ -28,9 +20,11 @@ This template defines the "Supported tags and Dockerfile links" portion of an im
- {{- "\t" -}}
{{- if $isGitHub -}} [ {{- end -}}
- ` {{- $e.Tags | join "`, `" -}} ` (* {{- template "dockerfilePath" $e -}} *)
- {{- $gitCommit := $e.ArchGitCommit arch -}}
- {{- if $isGitHub -}} ]( {{- $repoUrlBase -}} /blob/ {{- $gitCommit -}} / {{- template "dockerfilePath" $e -}} ) {{- end -}}
+ {{- $dir := .ArchDirectory $arch -}}
+ {{- $dockerfilePath := ternary "Dockerfile" (join "/" $dir "Dockerfile") (eq $dir ".") -}}
+ ` {{- $e.Tags | join "`, `" -}} ` (* {{- $dockerfilePath -}} *)
+ {{- $gitCommit := $e.ArchGitCommit $arch -}}
+ {{- if $isGitHub -}} ]( {{- $repoUrlBase -}} /blob/ {{- $gitCommit -}} / {{- $dockerfilePath -}} ) {{- end -}}
{{- /* TODO decide what to do (if anything) about non-GitHub repos with respect to URL */ -}}
{{- end -}}
{{- end -}}
diff --git a/python/README.md b/python/README.md
index c23f79f0..410b8525 100644
--- a/python/README.md
+++ b/python/README.md
@@ -22,6 +22,7 @@ WARNING:
- [`2.7.13-alpine3.6`, `2.7-alpine3.6`, `2-alpine3.6` (*2.7/alpine3.6/Dockerfile*)](e81758e60c/2.7/alpine3.6/Dockerfile1ca4a57b20/2.7/wheezy/Dockerfile7663560df7/2.7/onbuild/Dockerfiledb2d58d730/2.7/windows/windowsservercore/Dockerfile88ba87d31a/3.3/Dockerfile88ba87d31a/3.3/slim/Dockerfile88ba87d31a/3.3/alpine/Dockerfile88ba87d31a/3.5/slim/Dockerfile88ba87d31a/3.5/alpine/Dockerfile9a9021f213/3.5/onbuild/Dockerfiledb2d58d730/3.5/windows/windowsservercore/Dockerfile88ba87d31a/3.6/Dockerfile88ba87d31a/3.6/slim/Dockerfile88ba87d31a/3.6/alpine/Dockerfile88ba87d31a/3.6/alpine3.6/Dockerfile7eca63adca/3.6/onbuild/Dockerfiledb2d58d730/3.6/windows/windowsservercore/Dockerfile73d283d3dc/3.6-rc/Dockerfile73d283d3dc/3.6-rc/slim/Dockerfile73d283d3dc/3.6-rc/alpine/Dockerfile73d283d3dc/3.6-rc/alpine3.6/Dockerfile73d283d3dc/3.6-rc/onbuild/Dockerfile73d283d3dc/3.6-rc/windows/windowsservercore/Dockerfile
2017-06-20 08:36:41 -07:00
679388e80b
Add "--apply-constraints" in Travis so that "bashbrew from" doesn't fail on "windows-amd64"
2017-06-14 11:16:39 -07:00
da2823db4d
Validate (and deduplicate) Architectures values
2017-06-14 10:31:28 -07:00
1853c28093
Add an initial "manifest-tool" implementation
2017-06-13 09:33:53 -07:00
9c538e056d
Update to manifest-tool 0.5.0 and verify signatures
2017-06-08 14:35:04 -07:00
1066e05a1f
Add "arch" function for "bashbrew cat" to access the current architecture string
...
```console
$ bashbrew cat --format '{{ arch }}' ubuntu
amd64
$ bashbrew --arch=s390x cat --format '{{ arch }}' ubuntu
s390x
$ BASHBREW_ARCH=s390x bashbrew cat --format '{{ arch }}' ubuntu
s390x
```
2017-06-06 11:21:23 -07:00
e73133d8de
Add a "BASHBREW_PULL" environment variable to control the "pull policy" during build
2017-06-06 11:04:01 -07:00
cfe6c96010
Refactor bashbrew release Dockerfile to use "bashbrew arches" in the released binary names
...
```
+ file bin/bashbrew-amd64 bin/bashbrew-arm32v5 bin/bashbrew-arm32v6 bin/bashbrew-arm32v7 bin/bashbrew-arm64v8 bin/bashbrew-darwin-amd64 bin/bashbrew-i386 bin/bashbrew-ppc64le bin/bashbrew-s390x bin/bashbrew-windows-amd64.exe bin/manifest-tool-amd64 bin/manifest-tool-arm64v8 bin/manifest-tool-darwin-amd64 bin/manifest-tool-ppc64le bin/manifest-tool-s390x bin/manifest-tool-windows-amd64.exe
bin/bashbrew-amd64: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, stripped
bin/bashbrew-arm32v5: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), statically linked, stripped
bin/bashbrew-arm32v6: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), statically linked, stripped
bin/bashbrew-arm32v7: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), statically linked, stripped
bin/bashbrew-arm64v8: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, stripped
bin/bashbrew-darwin-amd64: Mach-O 64-bit x86_64 executable, flags:<NOUNDEFS>
bin/bashbrew-i386: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), statically linked, stripped
bin/bashbrew-ppc64le: ELF 64-bit LSB executable, 64-bit PowerPC or cisco 7500, version 1 (SYSV), statically linked, stripped
bin/bashbrew-s390x: ELF 64-bit MSB executable, IBM S/390, version 1 (SYSV), statically linked, stripped
bin/bashbrew-windows-amd64.exe: PE32+ executable (console) x86-64 (stripped to external PDB), for MS Windows
bin/manifest-tool-amd64: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, BuildID[sha1]=9c5441ad0c18e839627e0b61c2369292831c18a5, not stripped
bin/manifest-tool-arm64v8: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, not stripped
bin/manifest-tool-darwin-amd64: Mach-O 64-bit x86_64 executable, flags:<NOUNDEFS>
bin/manifest-tool-ppc64le: ELF 64-bit LSB executable, 64-bit PowerPC or cisco 7500, version 1 (SYSV), statically linked, not stripped
bin/manifest-tool-s390x: ELF 64-bit MSB executable, IBM S/390, version 1 (SYSV), statically linked, not stripped
bin/manifest-tool-windows-amd64.exe: PE32+ executable (console) x86-64 (stripped to external PDB), for MS Windows
---> 0acd3cdb36b1
Removing intermediate container ee37fa20846d
Successfully built 0acd3cdb36b1
```
2017-06-05 16:15:26 -07:00
51bb97f72d
Fix architecture-specific values inherited from the global entry
2017-06-05 15:40:37 -07:00
e3f0b23913
Add "--depth" flag to "bashbrew children" and "bashbrew parents"
...
```console
$ diff -u <(bashbrew children --depth 2 alpine) <(bashbrew children --depth 3 alpine)
--- /dev/fd/63 2017-06-02 07:29:56.871863245 -0700
+++ /dev/fd/62 2017-06-02 07:29:56.872863215 -0700
@@ -578,6 +578,20 @@
zookeeper:3.4.10
zookeeper:3.4
zookeeper:latest
+maven:3.5.0-jdk-7-onbuild-alpine
+maven:3.5-jdk-7-onbuild-alpine
+maven:3-jdk-7-onbuild-alpine
+maven:3.5.0-jdk-8-onbuild-alpine
+maven:3.5.0-onbuild-alpine
+maven:3.5-jdk-8-onbuild-alpine
+maven:3.5-onbuild-alpine
+maven:3-jdk-8-onbuild-alpine
+maven:3-onbuild-alpine
+maven:onbuild-alpine
+orientdb:2.2.21-spatial
+clojure:lein-2.7.1-alpine-onbuild
+clojure:lein-alpine-onbuild
+clojure:alpine-onbuild
notary:server
notary:server-0.5.0
notary:signer
$ bashbrew parents maven:onbuild-alpine
maven:3-jdk-8-alpine
openjdk:8-jdk-alpine
alpine:3.6
$ bashbrew parents --depth 1 maven:onbuild-alpine
maven:3-jdk-8-alpine
$ bashbrew parents --depth 2 maven:onbuild-alpine
maven:3-jdk-8-alpine
openjdk:8-jdk-alpine
```
2017-06-02 07:31:52 -07:00
Tianon Gravi