Commit Graph

80 Commits

Author SHA1 Message Date
Santiago Torres 032ca8d8e9
bashbrew:go:cmd-cat: remove extra v in Sprintf
The panic Sprintf for cmd-cat.go has an extra 'v' argument, which
triggers a failure when running go test. Remove the extra 'v' argument.
2020-04-19 22:05:21 -04:00
Tianon Gravi 1bc380db24 Update bashbrew bits for mips64le (still a WIP, but this lets us progress further) 2020-04-15 11:59:06 -07:00
Tianon Gravi c11e329115 Adjust "bashbrew" namespace sorting to account for more edge cases
In particular, our namespace changes made `bashbrew list --build-order wordpress php` work properly but `bashbrew --namespace amd64 list --build-order wordpress php` does not -- these changes adjust our sorting to take into account both the namespaced and the non-namespaced version of each tag in our library when checking the `FROM` values for sorting purposes.
2019-09-06 17:42:50 -07:00
Tianon Gravi 1241da9b40 Update github.com/docker-library/go-dockerlibrary for tag validation 2019-06-26 17:09:28 -07:00
Tianon Gravi 2012bf463e Update "go-dockerlibrary" for fixed "path.Clean" behavior 2019-06-18 16:22:56 -07:00
Tianon Gravi 8d9c213e61 Adjust temporary Git tags to also include namespace (and to normalize just in case something that can't be in a Git tag slips through) 2019-06-15 06:45:18 -07:00
Tianon Gravi 172a086639
Merge pull request #6054 from 7AC/master
bashbrew: account for namespaces when sorting repos
2019-06-14 20:13:49 -07:00
Tianon Gravi 8697e8b570 Add "--target-namespace" to both "push" and "put-shared" and make "tag" more intuitive 2019-06-14 16:35:27 -07:00
Tianon Gravi 5928827904 Add "DeleteTag" before "CreateTag" to avoid "ErrTagExists" 2019-06-14 14:47:58 -07:00
Tianon Gravi c8a9b29303 Make "--namespace" a global parameter that affects every command (and add "--target-namespace" to "bashbrew tag") 2019-06-13 16:55:20 -07:00
Giuseppe Valente 9d12196e27 bashbrew: account for namespaces when sorting repos 2019-06-13 16:50:11 -07:00
Tianon Gravi a0edd497d8 Update golang.org/x/crypto for proper s390x support 2019-06-13 16:37:57 -07:00
Tianon Gravi 22c68d5eca Switch most "git" operations over to https://github.com/src-d/go-git for a very dramatic speed increase (especially during dependency calculation) 2019-06-12 08:06:45 -07:00
Tianon Gravi 89697349e8 Apply gofmt 2019-06-12 08:06:45 -07:00
Tianon Gravi 1f2cab58d0 Improve Dockerfile fetching error reporting and "bashbrew children" / "bashbrew from" handling (especially in the case of no "--apply-constraints" flag) 2019-06-12 08:06:45 -07:00
Tianon Gravi 828b8ba4f7 Drop "ADD --from=" (which isn't supported by Docker) 2019-06-12 08:06:45 -07:00
Tianon Gravi 0ef77ba146 Add new "ArchLastStageFrom" function for getting the effective "base" of the given image (needed for docs generation) 2019-06-12 08:06:45 -07:00
Tianon Gravi a898c737c4 Remove "DockerFrom" function entirely to cause scripts to fail and force migration 2019-06-12 08:06:45 -07:00
Tianon Gravi 81e8f4f660 Unexpose "DockerfileMetadata" and add new exposed "DockerFroms" (since that's what's really necessary externally from that internal structure) 2019-06-12 08:06:45 -07:00
Tianon Gravi 67098be340 Add initial "multi-stage" support in bashbrew
This allows bashbrew to properly handle cross-repository and cross-tag dependencies even in the face of multiple `FROM` instructions or `COPY --from=`.

This also provides the scaffolding necessary to implement this in scripts using `bashbrew cat`.

As fallback behavior, the `*DockerFrom` functions should return the `FROM` of the last stage in the `Dockerfile` (which is essentially the `FROM` of the final image).

Also, the output of `bashbrew from` is now a space-separated list.
2019-06-12 08:06:45 -07:00
Tianon Gravi 9d30f3490d Apply 7d4d3222cd ("path.Clean" earlier so "bashbrew cat" shows the cleaned paths appropriately too) 2019-06-05 15:55:17 -07:00
Tianon Gravi 058fbda764 Port bashbrew to use Go modules instead of GB
Something in here also fixes the "missing build output on error" bug we've been seeing on Jenkins (see https://doi-janky.infosiftr.net/job/multiarch/job/arm32v7/job/rabbitmq/184/console for example).
2019-03-28 15:45:12 -07:00
Tianon Gravi 69c374aa98 Hotfix: "bashbrew build" sorting fix 2019-01-28 17:01:16 -08:00
Tianon Gravi 6930483624 Improve bashbrew SharedTags handling (explicitly allowed for "bashbrew cat", etc) 2019-01-28 16:16:50 -08:00
Tianon Gravi e692c1951d Update go-dockerlibrary to include https://github.com/docker-library/go-dockerlibrary/pull/19 2019-01-07 16:22:36 -08:00
Tianon Gravi 087cef7a9f Add support for "File:" in manifest files
See https://github.com/docker-library/go-dockerlibrary/pull/16 for format implementation details.
2018-10-22 16:06:21 -07:00
Tianon Gravi e3d787c1b5 Add new "archFilter" template function
Example usage:

```
{{- range $i, $e := ($archSpecific | ternary (archFilter arch $.Entries) $.Entries) -}}
    ...
{{- end -}}
```
2018-08-23 11:37:13 -07:00
Tianon Gravi 9232e049ed Slightly improved error message for failed template executions 2018-08-14 12:48:04 -07:00
Tianon Gravi 42c59432d2 Improve put-shared noop behavior and add "--force" flag for pushing operations 2018-05-03 12:21:57 -07:00
Tianon Gravi 891033f93a Add missing newline 2018-01-24 10:12:23 -08:00
Tianon Gravi 01c6ebec24 Update "put-shared" to allow some tags to fail per-repo (and collect errors) 2018-01-24 09:57:47 -08:00
Tianon Gravi d6ebf0c1ac Also tag SharedTags (and include them in dependency calculations, etc)
This will include SharedTags directly in "bashbrew list", etc, but explicitly does _not_ include them during "bashbrew push".  This allows for other images to be "FROM" a SharedTags value, and should be reasonably safe since any given SharedTag should only be applicable for a given architecture at most once.
2017-10-24 10:47:17 -07:00
Tianon Gravi 6e8d50e38f Add new "archNamespace" template function for parsing "--arch-namespace" parameters
Example usage: `{{ archNamespace "windows-amd64" }}` => `winamd64`

This is intended for use in https://github.com/docker-library/docs to help create a list of "Supported architectures" for each image which links to each supported architecture repository (and if supported by Bashbrew itself, allows us to keep https://github.com/docker-library/docs isolated from Jenkins assumptions while still allowing Jenkins to contain the single-source-of-truth for arch-to-namespace mappings).
2017-10-02 12:27:10 -07:00
Tianon Gravi 683425b864 Add a "--single-arch" flag to "put-shared" for pushing SharedTags to architecture-specific namespaces 2017-09-12 16:18:27 -07:00
Tianon Gravi ba9fd0d60c Minor bashbrew de-optimization 2017-09-12 13:32:45 -07:00
Tianon Gravi 9805fc1419 Add a "--dry-run" flag to "build", "tag", "push", and "put-shared" 2017-09-07 13:53:58 -07:00
Tianon Gravi 2a863261a8 CUE LOUD GRUMBLING 2017-09-06 13:11:15 -07:00
Tianon Gravi 16d5317cab Fix minor typo 2017-09-05 07:36:47 -07:00
Tianon Gravi 259302679c Use the Hub v2 API for a dramatic no-op push speedup
This also has the benefit of only triggering automated builds when there have been actual changes to the image.

This works by comparing the Hub's `last_updated` timestamp to the `Created` timestamp of local images (in the case of `bashbrew push`), or to the `last_updated` timestamp of the images which make up the `SharedTags` and/or multiarch images (in the case of `bashbrew put-shared`).

This is a dramatic speedup because the Hub v2 API is faster (and has no auth for public repos) than the registry API.  If there are any errors in the fetching of `last_updated` from the Hub, we massage the timestamps such that the push simply always happens, as before (which will be the case for pushes to non-Hub registries, for example).
2017-08-28 09:46:47 -07:00
Tianon Gravi e6ff2ce3ff Update go-dockerlibrary to pull in the new "getenv" function
See https://github.com/docker-library/go-dockerlibrary/pull/15.
2017-07-12 16:15:18 -07:00
Tianon Gravi 9d7a12f669 Update manifest-tool to 0.6.0 and take advantage of new "tags:" support (for increased efficiency in pushing aliases) 2017-07-05 13:42:27 -07:00
Tianon Gravi 89c224c724 Add a separate ArchDockerFrom function for better template granularity
Here's an example diff I plan to apply over in the docs with this: (with an example output from `python`)

```diff
diff --git a/.template-helpers/generate-dockerfile-links-partial.tmpl b/.template-helpers/generate-dockerfile-links-partial.tmpl
index 5c13047a..996b9d60 100644
--- a/.template-helpers/generate-dockerfile-links-partial.tmpl
+++ b/.template-helpers/generate-dockerfile-links-partial.tmpl
@@ -4,21 +4,13 @@ This template defines the "Supported tags and Dockerfile links" portion of an im

 */ -}}

-{{- define "dockerfilePath" -}}
-	{{- $dir := .ArchDirectory arch -}}
-	{{- if ne $dir "." -}}
-		{{- $dir -}}
-		/
-	{{- end -}}
-	Dockerfile
-{{- end -}}
-
 {{- range $i, $e := $.Entries -}}
-	{{- if $e.HasArchitecture arch -}}
+	{{- $arch := ternary (ternary arch ($e.Architectures | first) ($e.HasArchitecture arch)) arch (eq arch "amd64") -}}
+	{{- if $e.HasArchitecture $arch -}}
 		{{- /* force bashbrew to "fetch" the GitCommit so that it is fully resolved */ -}}
-		{{- $from := $.DockerFrom $e -}}
+		{{- $from := $.ArchDockerFrom $arch $e -}}

-		{{- $gitRepo := $e.ArchGitRepo arch -}}
+		{{- $gitRepo := $e.ArchGitRepo $arch -}}
 		{{- $repoUrlBase := $gitRepo | replace "git://" "https://" | trimSuffixes ".git" -}}
 		{{- $isGitHub := hasPrefix "https://github.com/" $repoUrlBase -}}

@@ -28,9 +20,11 @@ This template defines the "Supported tags and Dockerfile links" portion of an im

 		- {{- "\t" -}}
 		{{- if $isGitHub -}} [ {{- end -}}
-		` {{- $e.Tags | join "`, `" -}} ` (* {{- template "dockerfilePath" $e -}} *)
-		{{- $gitCommit := $e.ArchGitCommit arch -}}
-		{{- if $isGitHub -}} ]( {{- $repoUrlBase -}} /blob/ {{- $gitCommit -}} / {{- template "dockerfilePath" $e -}} ) {{- end -}}
+		{{- $dir := .ArchDirectory $arch -}}
+		{{- $dockerfilePath := ternary "Dockerfile" (join "/" $dir "Dockerfile") (eq $dir ".") -}}
+		` {{- $e.Tags | join "`, `" -}} ` (* {{- $dockerfilePath -}} *)
+		{{- $gitCommit := $e.ArchGitCommit $arch -}}
+		{{- if $isGitHub -}} ]( {{- $repoUrlBase -}} /blob/ {{- $gitCommit -}} / {{- $dockerfilePath -}} ) {{- end -}}
 		{{- /* TODO decide what to do (if anything) about non-GitHub repos with respect to URL */ -}}
 	{{- end -}}
 {{- end -}}
diff --git a/python/README.md b/python/README.md
index c23f79f0..410b8525 100644
--- a/python/README.md
+++ b/python/README.md
@@ -22,6 +22,7 @@ WARNING:
 -	[`2.7.13-alpine3.6`, `2.7-alpine3.6`, `2-alpine3.6` (*2.7/alpine3.6/Dockerfile*)](e81758e60c/2.7/alpine3.6/Dockerfile)
 -	[`2.7.13-wheezy`, `2.7-wheezy`, `2-wheezy` (*2.7/wheezy/Dockerfile*)](1ca4a57b20/2.7/wheezy/Dockerfile)
 -	[`2.7.13-onbuild`, `2.7-onbuild`, `2-onbuild` (*2.7/onbuild/Dockerfile*)](7663560df7/2.7/onbuild/Dockerfile)
+-	[`2.7.13-windowsservercore`, `2.7-windowsservercore`, `2-windowsservercore` (*2.7/windows/windowsservercore/Dockerfile*)](db2d58d730/2.7/windows/windowsservercore/Dockerfile)
 -	[`3.3.6`, `3.3` (*3.3/Dockerfile*)](88ba87d31a/3.3/Dockerfile)
 -	[`3.3.6-slim`, `3.3-slim` (*3.3/slim/Dockerfile*)](88ba87d31a/3.3/slim/Dockerfile)
 -	[`3.3.6-alpine`, `3.3-alpine` (*3.3/alpine/Dockerfile*)](88ba87d31a/3.3/alpine/Dockerfile)
@@ -36,16 +37,19 @@ WARNING:
 -	[`3.5.3-slim`, `3.5-slim` (*3.5/slim/Dockerfile*)](88ba87d31a/3.5/slim/Dockerfile)
 -	[`3.5.3-alpine`, `3.5-alpine` (*3.5/alpine/Dockerfile*)](88ba87d31a/3.5/alpine/Dockerfile)
 -	[`3.5.3-onbuild`, `3.5-onbuild` (*3.5/onbuild/Dockerfile*)](9a9021f213/3.5/onbuild/Dockerfile)
+-	[`3.5.3-windowsservercore`, `3.5-windowsservercore` (*3.5/windows/windowsservercore/Dockerfile*)](db2d58d730/3.5/windows/windowsservercore/Dockerfile)
 -	[`3.6.1`, `3.6`, `3`, `latest` (*3.6/Dockerfile*)](88ba87d31a/3.6/Dockerfile)
 -	[`3.6.1-slim`, `3.6-slim`, `3-slim`, `slim` (*3.6/slim/Dockerfile*)](88ba87d31a/3.6/slim/Dockerfile)
 -	[`3.6.1-alpine`, `3.6-alpine`, `3-alpine`, `alpine` (*3.6/alpine/Dockerfile*)](88ba87d31a/3.6/alpine/Dockerfile)
 -	[`3.6.1-alpine3.6`, `3.6-alpine3.6`, `3-alpine3.6`, `alpine3.6` (*3.6/alpine3.6/Dockerfile*)](88ba87d31a/3.6/alpine3.6/Dockerfile)
 -	[`3.6.1-onbuild`, `3.6-onbuild`, `3-onbuild`, `onbuild` (*3.6/onbuild/Dockerfile*)](7eca63adca/3.6/onbuild/Dockerfile)
+-	[`3.6.1-windowsservercore`, `3.6-windowsservercore`, `3-windowsservercore`, `windowsservercore` (*3.6/windows/windowsservercore/Dockerfile*)](db2d58d730/3.6/windows/windowsservercore/Dockerfile)
 -	[`3.6.2rc1`, `3.6-rc`, `rc` (*3.6-rc/Dockerfile*)](73d283d3dc/3.6-rc/Dockerfile)
 -	[`3.6.2rc1-slim`, `3.6-rc-slim`, `rc-slim` (*3.6-rc/slim/Dockerfile*)](73d283d3dc/3.6-rc/slim/Dockerfile)
 -	[`3.6.2rc1-alpine`, `3.6-rc-alpine`, `rc-alpine` (*3.6-rc/alpine/Dockerfile*)](73d283d3dc/3.6-rc/alpine/Dockerfile)
 -	[`3.6.2rc1-alpine3.6`, `3.6-rc-alpine3.6`, `rc-alpine3.6` (*3.6-rc/alpine3.6/Dockerfile*)](73d283d3dc/3.6-rc/alpine3.6/Dockerfile)
 -	[`3.6.2rc1-onbuild`, `3.6-rc-onbuild`, `rc-onbuild` (*3.6-rc/onbuild/Dockerfile*)](73d283d3dc/3.6-rc/onbuild/Dockerfile)
+-	[`3.6.2rc1-windowsservercore`, `3.6-rc-windowsservercore`, `rc-windowsservercore` (*3.6-rc/windows/windowsservercore/Dockerfile*)](73d283d3dc/3.6-rc/windows/windowsservercore/Dockerfile)

 # Quick reference

```
2017-06-20 08:36:41 -07:00
Tianon Gravi d60142b532 Validate (and deduplicate) Architectures values 2017-06-14 10:31:28 -07:00
Tianon Gravi 396ac816f8 Add an initial "manifest-tool" implementation 2017-06-13 09:33:53 -07:00
Tianon Gravi 68af563715 Add "arch" function for "bashbrew cat" to access the current architecture string
```console
$ bashbrew cat --format '{{ arch }}' ubuntu
amd64
$ bashbrew --arch=s390x cat --format '{{ arch }}' ubuntu
s390x
$ BASHBREW_ARCH=s390x bashbrew cat --format '{{ arch }}' ubuntu
s390x
```
2017-06-06 11:21:23 -07:00
Tianon Gravi 142e80b2c8 Add a "BASHBREW_PULL" environment variable to control the "pull policy" during build 2017-06-06 11:04:01 -07:00
Tianon Gravi fd7665b53d Fix architecture-specific values inherited from the global entry 2017-06-05 15:40:37 -07:00
Tianon Gravi 001e0803b6 Add "--depth" flag to "bashbrew children" and "bashbrew parents"
```console
$ diff -u <(bashbrew children --depth 2 alpine) <(bashbrew children --depth 3 alpine)
--- /dev/fd/63	2017-06-02 07:29:56.871863245 -0700
+++ /dev/fd/62	2017-06-02 07:29:56.872863215 -0700
@@ -578,6 +578,20 @@
 zookeeper:3.4.10
 zookeeper:3.4
 zookeeper:latest
+maven:3.5.0-jdk-7-onbuild-alpine
+maven:3.5-jdk-7-onbuild-alpine
+maven:3-jdk-7-onbuild-alpine
+maven:3.5.0-jdk-8-onbuild-alpine
+maven:3.5.0-onbuild-alpine
+maven:3.5-jdk-8-onbuild-alpine
+maven:3.5-onbuild-alpine
+maven:3-jdk-8-onbuild-alpine
+maven:3-onbuild-alpine
+maven:onbuild-alpine
+orientdb:2.2.21-spatial
+clojure:lein-2.7.1-alpine-onbuild
+clojure:lein-alpine-onbuild
+clojure:alpine-onbuild
 notary:server
 notary:server-0.5.0
 notary:signer

$ bashbrew parents maven:onbuild-alpine
maven:3-jdk-8-alpine
openjdk:8-jdk-alpine
alpine:3.6

$ bashbrew parents --depth 1 maven:onbuild-alpine
maven:3-jdk-8-alpine

$ bashbrew parents --depth 2 maven:onbuild-alpine
maven:3-jdk-8-alpine
openjdk:8-jdk-alpine
```
2017-06-02 07:31:52 -07:00
Tianon Gravi fc83c27756 Add initial Architectures support (and the beginnings of a "put-shared" subcommand for bringing all the SharedTags and arch-specific images together with manifest lists/indexes) 2017-06-01 16:56:04 -07:00
Tianon Gravi 79c178ab7f Add a simple bashbrew TODO 2017-03-13 10:33:54 -07:00