Commit Graph

51 Commits

Author SHA1 Message Date
Tianon Gravi b84de49388 Add missing newline 2018-01-24 10:12:23 -08:00
Tianon Gravi f36ef9d30b Update "put-shared" to allow some tags to fail per-repo (and collect errors) 2018-01-24 09:57:47 -08:00
Tianon Gravi 90101d3445 Also tag SharedTags (and include them in dependency calculations, etc)
This will include SharedTags directly in "bashbrew list", etc, but explicitly does _not_ include them during "bashbrew push".  This allows for other images to be "FROM" a SharedTags value, and should be reasonably safe since any given SharedTag should only be applicable for a given architecture at most once.
2017-10-24 10:47:17 -07:00
Tianon Gravi ec5b4e484e Add new "archNamespace" template function for parsing "--arch-namespace" parameters
Example usage: `{{ archNamespace "windows-amd64" }}` => `winamd64`

This is intended for use in https://github.com/docker-library/docs to help create a list of "Supported architectures" for each image which links to each supported architecture repository (and if supported by Bashbrew itself, allows us to keep https://github.com/docker-library/docs isolated from Jenkins assumptions while still allowing Jenkins to contain the single-source-of-truth for arch-to-namespace mappings).
2017-10-02 12:27:10 -07:00
Tianon Gravi 50e9eb1ddd Add a "--single-arch" flag to "put-shared" for pushing SharedTags to architecture-specific namespaces 2017-09-12 16:18:27 -07:00
Tianon Gravi 0da6c3c534 Minor bashbrew de-optimization 2017-09-12 13:32:45 -07:00
Tianon Gravi 9aedfc4915 Add a "--dry-run" flag to "build", "tag", "push", and "put-shared" 2017-09-07 13:53:58 -07:00
Tianon Gravi 1568683373 CUE LOUD GRUMBLING 2017-09-06 13:11:15 -07:00
Tianon Gravi 145add96da Fix minor typo 2017-09-05 07:36:47 -07:00
Tianon Gravi 3b0c4503cb Use the Hub v2 API for a dramatic no-op push speedup
This also has the benefit of only triggering automated builds when there have been actual changes to the image.

This works by comparing the Hub's `last_updated` timestamp to the `Created` timestamp of local images (in the case of `bashbrew push`), or to the `last_updated` timestamp of the images which make up the `SharedTags` and/or multiarch images (in the case of `bashbrew put-shared`).

This is a dramatic speedup because the Hub v2 API is faster (and has no auth for public repos) than the registry API.  If there are any errors in the fetching of `last_updated` from the Hub, we massage the timestamps such that the push simply always happens, as before (which will be the case for pushes to non-Hub registries, for example).
2017-08-28 09:46:47 -07:00
Tianon Gravi c4108a0534 Update go-dockerlibrary to pull in the new "getenv" function
See https://github.com/docker-library/go-dockerlibrary/pull/15.
2017-07-12 16:15:18 -07:00
Tianon Gravi b9d7a8b7e7 Update manifest-tool to 0.6.0 and take advantage of new "tags:" support (for increased efficiency in pushing aliases) 2017-07-05 13:42:27 -07:00
Tianon Gravi 7634587e56 Add a separate ArchDockerFrom function for better template granularity
Here's an example diff I plan to apply over in the docs with this: (with an example output from `python`)

```diff
diff --git a/.template-helpers/generate-dockerfile-links-partial.tmpl b/.template-helpers/generate-dockerfile-links-partial.tmpl
index 5c13047a..996b9d60 100644
--- a/.template-helpers/generate-dockerfile-links-partial.tmpl
+++ b/.template-helpers/generate-dockerfile-links-partial.tmpl
@@ -4,21 +4,13 @@ This template defines the "Supported tags and Dockerfile links" portion of an im

 */ -}}

-{{- define "dockerfilePath" -}}
-	{{- $dir := .ArchDirectory arch -}}
-	{{- if ne $dir "." -}}
-		{{- $dir -}}
-		/
-	{{- end -}}
-	Dockerfile
-{{- end -}}
-
 {{- range $i, $e := $.Entries -}}
-	{{- if $e.HasArchitecture arch -}}
+	{{- $arch := ternary (ternary arch ($e.Architectures | first) ($e.HasArchitecture arch)) arch (eq arch "amd64") -}}
+	{{- if $e.HasArchitecture $arch -}}
 		{{- /* force bashbrew to "fetch" the GitCommit so that it is fully resolved */ -}}
-		{{- $from := $.DockerFrom $e -}}
+		{{- $from := $.ArchDockerFrom $arch $e -}}

-		{{- $gitRepo := $e.ArchGitRepo arch -}}
+		{{- $gitRepo := $e.ArchGitRepo $arch -}}
 		{{- $repoUrlBase := $gitRepo | replace "git://" "https://" | trimSuffixes ".git" -}}
 		{{- $isGitHub := hasPrefix "https://github.com/" $repoUrlBase -}}

@@ -28,9 +20,11 @@ This template defines the "Supported tags and Dockerfile links" portion of an im

 		- {{- "\t" -}}
 		{{- if $isGitHub -}} [ {{- end -}}
-		` {{- $e.Tags | join "`, `" -}} ` (* {{- template "dockerfilePath" $e -}} *)
-		{{- $gitCommit := $e.ArchGitCommit arch -}}
-		{{- if $isGitHub -}} ]( {{- $repoUrlBase -}} /blob/ {{- $gitCommit -}} / {{- template "dockerfilePath" $e -}} ) {{- end -}}
+		{{- $dir := .ArchDirectory $arch -}}
+		{{- $dockerfilePath := ternary "Dockerfile" (join "/" $dir "Dockerfile") (eq $dir ".") -}}
+		` {{- $e.Tags | join "`, `" -}} ` (* {{- $dockerfilePath -}} *)
+		{{- $gitCommit := $e.ArchGitCommit $arch -}}
+		{{- if $isGitHub -}} ]( {{- $repoUrlBase -}} /blob/ {{- $gitCommit -}} / {{- $dockerfilePath -}} ) {{- end -}}
 		{{- /* TODO decide what to do (if anything) about non-GitHub repos with respect to URL */ -}}
 	{{- end -}}
 {{- end -}}
diff --git a/python/README.md b/python/README.md
index c23f79f0..410b8525 100644
--- a/python/README.md
+++ b/python/README.md
@@ -22,6 +22,7 @@ WARNING:
 -	[`2.7.13-alpine3.6`, `2.7-alpine3.6`, `2-alpine3.6` (*2.7/alpine3.6/Dockerfile*)](e81758e60c/2.7/alpine3.6/Dockerfile)
 -	[`2.7.13-wheezy`, `2.7-wheezy`, `2-wheezy` (*2.7/wheezy/Dockerfile*)](1ca4a57b20/2.7/wheezy/Dockerfile)
 -	[`2.7.13-onbuild`, `2.7-onbuild`, `2-onbuild` (*2.7/onbuild/Dockerfile*)](7663560df7/2.7/onbuild/Dockerfile)
+-	[`2.7.13-windowsservercore`, `2.7-windowsservercore`, `2-windowsservercore` (*2.7/windows/windowsservercore/Dockerfile*)](db2d58d730/2.7/windows/windowsservercore/Dockerfile)
 -	[`3.3.6`, `3.3` (*3.3/Dockerfile*)](88ba87d31a/3.3/Dockerfile)
 -	[`3.3.6-slim`, `3.3-slim` (*3.3/slim/Dockerfile*)](88ba87d31a/3.3/slim/Dockerfile)
 -	[`3.3.6-alpine`, `3.3-alpine` (*3.3/alpine/Dockerfile*)](88ba87d31a/3.3/alpine/Dockerfile)
@@ -36,16 +37,19 @@ WARNING:
 -	[`3.5.3-slim`, `3.5-slim` (*3.5/slim/Dockerfile*)](88ba87d31a/3.5/slim/Dockerfile)
 -	[`3.5.3-alpine`, `3.5-alpine` (*3.5/alpine/Dockerfile*)](88ba87d31a/3.5/alpine/Dockerfile)
 -	[`3.5.3-onbuild`, `3.5-onbuild` (*3.5/onbuild/Dockerfile*)](9a9021f213/3.5/onbuild/Dockerfile)
+-	[`3.5.3-windowsservercore`, `3.5-windowsservercore` (*3.5/windows/windowsservercore/Dockerfile*)](db2d58d730/3.5/windows/windowsservercore/Dockerfile)
 -	[`3.6.1`, `3.6`, `3`, `latest` (*3.6/Dockerfile*)](88ba87d31a/3.6/Dockerfile)
 -	[`3.6.1-slim`, `3.6-slim`, `3-slim`, `slim` (*3.6/slim/Dockerfile*)](88ba87d31a/3.6/slim/Dockerfile)
 -	[`3.6.1-alpine`, `3.6-alpine`, `3-alpine`, `alpine` (*3.6/alpine/Dockerfile*)](88ba87d31a/3.6/alpine/Dockerfile)
 -	[`3.6.1-alpine3.6`, `3.6-alpine3.6`, `3-alpine3.6`, `alpine3.6` (*3.6/alpine3.6/Dockerfile*)](88ba87d31a/3.6/alpine3.6/Dockerfile)
 -	[`3.6.1-onbuild`, `3.6-onbuild`, `3-onbuild`, `onbuild` (*3.6/onbuild/Dockerfile*)](7eca63adca/3.6/onbuild/Dockerfile)
+-	[`3.6.1-windowsservercore`, `3.6-windowsservercore`, `3-windowsservercore`, `windowsservercore` (*3.6/windows/windowsservercore/Dockerfile*)](db2d58d730/3.6/windows/windowsservercore/Dockerfile)
 -	[`3.6.2rc1`, `3.6-rc`, `rc` (*3.6-rc/Dockerfile*)](73d283d3dc/3.6-rc/Dockerfile)
 -	[`3.6.2rc1-slim`, `3.6-rc-slim`, `rc-slim` (*3.6-rc/slim/Dockerfile*)](73d283d3dc/3.6-rc/slim/Dockerfile)
 -	[`3.6.2rc1-alpine`, `3.6-rc-alpine`, `rc-alpine` (*3.6-rc/alpine/Dockerfile*)](73d283d3dc/3.6-rc/alpine/Dockerfile)
 -	[`3.6.2rc1-alpine3.6`, `3.6-rc-alpine3.6`, `rc-alpine3.6` (*3.6-rc/alpine3.6/Dockerfile*)](73d283d3dc/3.6-rc/alpine3.6/Dockerfile)
 -	[`3.6.2rc1-onbuild`, `3.6-rc-onbuild`, `rc-onbuild` (*3.6-rc/onbuild/Dockerfile*)](73d283d3dc/3.6-rc/onbuild/Dockerfile)
+-	[`3.6.2rc1-windowsservercore`, `3.6-rc-windowsservercore`, `rc-windowsservercore` (*3.6-rc/windows/windowsservercore/Dockerfile*)](73d283d3dc/3.6-rc/windows/windowsservercore/Dockerfile)

 # Quick reference

```
2017-06-20 08:36:41 -07:00
Tianon Gravi da2823db4d Validate (and deduplicate) Architectures values 2017-06-14 10:31:28 -07:00
Tianon Gravi 1853c28093 Add an initial "manifest-tool" implementation 2017-06-13 09:33:53 -07:00
Tianon Gravi 1066e05a1f Add "arch" function for "bashbrew cat" to access the current architecture string
```console
$ bashbrew cat --format '{{ arch }}' ubuntu
amd64
$ bashbrew --arch=s390x cat --format '{{ arch }}' ubuntu
s390x
$ BASHBREW_ARCH=s390x bashbrew cat --format '{{ arch }}' ubuntu
s390x
```
2017-06-06 11:21:23 -07:00
Tianon Gravi e73133d8de Add a "BASHBREW_PULL" environment variable to control the "pull policy" during build 2017-06-06 11:04:01 -07:00
Tianon Gravi 51bb97f72d Fix architecture-specific values inherited from the global entry 2017-06-05 15:40:37 -07:00
Tianon Gravi e3f0b23913 Add "--depth" flag to "bashbrew children" and "bashbrew parents"
```console
$ diff -u <(bashbrew children --depth 2 alpine) <(bashbrew children --depth 3 alpine)
--- /dev/fd/63	2017-06-02 07:29:56.871863245 -0700
+++ /dev/fd/62	2017-06-02 07:29:56.872863215 -0700
@@ -578,6 +578,20 @@
 zookeeper:3.4.10
 zookeeper:3.4
 zookeeper:latest
+maven:3.5.0-jdk-7-onbuild-alpine
+maven:3.5-jdk-7-onbuild-alpine
+maven:3-jdk-7-onbuild-alpine
+maven:3.5.0-jdk-8-onbuild-alpine
+maven:3.5.0-onbuild-alpine
+maven:3.5-jdk-8-onbuild-alpine
+maven:3.5-onbuild-alpine
+maven:3-jdk-8-onbuild-alpine
+maven:3-onbuild-alpine
+maven:onbuild-alpine
+orientdb:2.2.21-spatial
+clojure:lein-2.7.1-alpine-onbuild
+clojure:lein-alpine-onbuild
+clojure:alpine-onbuild
 notary:server
 notary:server-0.5.0
 notary:signer

$ bashbrew parents maven:onbuild-alpine
maven:3-jdk-8-alpine
openjdk:8-jdk-alpine
alpine:3.6

$ bashbrew parents --depth 1 maven:onbuild-alpine
maven:3-jdk-8-alpine

$ bashbrew parents --depth 2 maven:onbuild-alpine
maven:3-jdk-8-alpine
openjdk:8-jdk-alpine
```
2017-06-02 07:31:52 -07:00
Tianon Gravi fdf3501d49 Add initial Architectures support (and the beginnings of a "put-shared" subcommand for bringing all the SharedTags and arch-specific images together with manifest lists/indexes) 2017-06-01 16:56:04 -07:00
Tianon Gravi 24ac7970ec Add a simple bashbrew TODO 2017-03-13 10:33:54 -07:00
Tianon Gravi 35a8aa6aab Apply "gofmt" 2017-02-10 12:41:10 -08:00
Tianon Gravi a71b16ff95 Update "github.com/docker-library/go-dockerlibrary" vendoring (improved error handling, more edge cases) 2017-01-24 13:56:03 -08:00
Tianon Gravi 32e7199775 Update "pault.ag/go/debian" vendoring 2017-01-24 13:48:42 -08:00
Tianon Gravi 3086073bf4 Update bashbrew to cache remote tag fetches (speeds up "--all" operations by avoiding "git fetch" where we already have a cached answer in Git)
Example tag name: `git-/github.com/docker/notary-official-images-refs/tags/v0.5.0-FETCH_HEAD`

Referring to:
```
signer-0.5.0: git://github.com/docker/notary-official-images@v0.5.0 notary-signer
```
2016-11-17 16:51:39 -08:00
Tianon Gravi 8d1c134bfe Update bashbrew Constraints logic with several fixes
- only print a Constraints-related "skipping" warning for a given "repo:tag" once per session
- apply constraints while sorting too (so that `bashbrew build --all` on Windows doesn't try to `git fetch` Linux rootfs tarballs only to skip them during build, for example)
- add `--apply-constraints` flag to `from`, `children`, and `parents` subcommands
2016-11-17 15:53:28 -08:00
Tianon Gravi 54636545aa Move "build order" sorting functions into a common file and refactor them to use topsort properly and via a single unified function (single node per logical "thing" we're sorting, resolve tag names to canonical node names appropriately) 2016-09-22 10:18:30 -07:00
Tianon Gravi 86c6275af3 Adjust bashbrew such that env overrides config appropriately 2016-07-28 11:30:54 -07:00
Tianon Gravi 144fae7159 Add a separate "EntryIdentifier" so that build output is more informative
This changes something like:

    Using bashbrew/cache:2775145643e397ca993e842f3263a1362a49df4115ce59d59bed1452323b7af2 (buildpack-deps)

into:

    Using bashbrew/cache:2775145643e397ca993e842f3263a1362a49df4115ce59d59bed1452323b7af2 (buildpack-deps:wheezy)
2016-07-22 10:00:36 -07:00
Tianon Gravi 64a28af865 Update "github.com/docker-library/go-dockerlibrary" for parsing line-based maintainers comments 2016-07-20 11:23:22 -07:00
Joe Ferguson a712a5d847 Add DockerCacheName to be able to use it from templates
- like the following to get golang:latest and the cache name of its parent
   ```console
   $ bashbrew cat --format '{{ range .Entries }}{{ $.DockerCacheName . }}{{ range .Tags }}{{"\n"}}{{ join ":" $.RepoName . }}{{end}}{{"\n"}}{{end}}' golang:latest
   ```
2016-07-06 13:27:39 -07:00
Tianon Gravi e258fd891b Capture "command failure" while reading Dockerfiles for FROM lines 2016-06-27 11:10:14 -07:00
Tianon Gravi e559474524 Fix spurious "bad file descriptor" errors in bashbrew by using "exec.Cmd" objects more appropriately while streaming 2016-06-24 17:32:43 -07:00
Stuart Clark cae164741c Correct typo in bashbrew push --namespace message 2016-06-22 05:51:11 +01:00
Tianon Gravi b431d5eac1 Rewrite constraints logic to actually work properly 2016-06-10 11:32:23 -07:00
Tianon Gravi 45c8ce94e2 Allow only "http" and "https" URLs 2016-06-08 10:37:08 -07:00
Tianon Gravi e4353713d2 Facepalm 2016-06-08 10:24:53 -07:00
Tianon Gravi 9a6109687d First attempt to unstick Windows 2016-06-08 10:23:58 -07:00
Tianon Gravi 446e368fe1 Add support for "--pull always", "--pull missing", and "--pull never" (replacing "--pull-missing") 2016-06-07 16:40:52 -07:00
Tianon Gravi bed693c79a Auto-upgrade "git://github.com/..." to "https://github.com/..." (with a warning) 2016-06-07 16:14:30 -07:00
Tianon Gravi 60f6f6dd9e Apply constraints during "tag" and "push" (since they're only useful after "build", at least currently) 2016-06-07 12:24:01 -07:00
Tianon Gravi 19621bf245 Add extra output when we're "Using" an older dockerCacheHash, even if we didn't need to build it 2016-06-06 16:30:10 -07:00
Tianon Gravi a48a191751 Also remove "-f" from "docker tag" (since it's deprecated in 1.10+ and removed in 1.12+) 2016-06-06 11:37:54 -07:00
Tianon Gravi 41c7a5952f Explicitly note that Docker 1.10+ is required for the new Bashbrew
This is especially thanks to https://github.com/docker/docker/issues/15785 and thus c8cc4fb8d9.
2016-06-06 10:41:10 -07:00
Tianon Gravi 5b00dc4395 Add a "--repos" flag to "bashbrew list" to prefer to list only repos 2016-06-03 14:29:26 -07:00
Tianon Gravi 2068a11e02 Fix "bashbrew list --uniq --build-order alpine:latest alpine:3.3" (to properly list two tags instead of just one) 2016-06-03 14:28:40 -07:00
Tianon Gravi 3538a9d438 Rename "--verbose" to the more-apt "--debug", and adjust "docker build" to _always_ capture output (expect when debug mode is enabled and it's piped directly to the user) so that it can be included in the error response if the build fails 2016-06-03 12:15:55 -07:00
Tianon Gravi c83bafa7a7 Don't pull scratch 2016-06-03 11:01:23 -07:00
Tianon Gravi 9e49f3ed54 Use cli's "MultiError" construct to provide useful context for top-level cmd-*.go errors 2016-06-03 10:23:09 -07:00
Tianon Gravi e5d3901d84 Add a "--format-file" flag to "bashbrew cat" which overrides "--format" with the contents of a file (and add more useful functions to our template processing) 2016-06-03 10:23:09 -07:00