Commit Graph

51 Commits

Author SHA1 Message Date
Tianon Gravi 891033f93a Add missing newline 2018-01-24 10:12:23 -08:00
Tianon Gravi 01c6ebec24 Update "put-shared" to allow some tags to fail per-repo (and collect errors) 2018-01-24 09:57:47 -08:00
Tianon Gravi d6ebf0c1ac Also tag SharedTags (and include them in dependency calculations, etc)
This will include SharedTags directly in "bashbrew list", etc, but explicitly does _not_ include them during "bashbrew push".  This allows for other images to be "FROM" a SharedTags value, and should be reasonably safe since any given SharedTag should only be applicable for a given architecture at most once.
2017-10-24 10:47:17 -07:00
Tianon Gravi 6e8d50e38f Add new "archNamespace" template function for parsing "--arch-namespace" parameters
Example usage: `{{ archNamespace "windows-amd64" }}` => `winamd64`

This is intended for use in https://github.com/docker-library/docs to help create a list of "Supported architectures" for each image which links to each supported architecture repository (and if supported by Bashbrew itself, allows us to keep https://github.com/docker-library/docs isolated from Jenkins assumptions while still allowing Jenkins to contain the single-source-of-truth for arch-to-namespace mappings).
2017-10-02 12:27:10 -07:00
Tianon Gravi 683425b864 Add a "--single-arch" flag to "put-shared" for pushing SharedTags to architecture-specific namespaces 2017-09-12 16:18:27 -07:00
Tianon Gravi ba9fd0d60c Minor bashbrew de-optimization 2017-09-12 13:32:45 -07:00
Tianon Gravi 9805fc1419 Add a "--dry-run" flag to "build", "tag", "push", and "put-shared" 2017-09-07 13:53:58 -07:00
Tianon Gravi 2a863261a8 CUE LOUD GRUMBLING 2017-09-06 13:11:15 -07:00
Tianon Gravi 16d5317cab Fix minor typo 2017-09-05 07:36:47 -07:00
Tianon Gravi 259302679c Use the Hub v2 API for a dramatic no-op push speedup
This also has the benefit of only triggering automated builds when there have been actual changes to the image.

This works by comparing the Hub's `last_updated` timestamp to the `Created` timestamp of local images (in the case of `bashbrew push`), or to the `last_updated` timestamp of the images which make up the `SharedTags` and/or multiarch images (in the case of `bashbrew put-shared`).

This is a dramatic speedup because the Hub v2 API is faster (and has no auth for public repos) than the registry API.  If there are any errors in the fetching of `last_updated` from the Hub, we massage the timestamps such that the push simply always happens, as before (which will be the case for pushes to non-Hub registries, for example).
2017-08-28 09:46:47 -07:00
Tianon Gravi e6ff2ce3ff Update go-dockerlibrary to pull in the new "getenv" function
See https://github.com/docker-library/go-dockerlibrary/pull/15.
2017-07-12 16:15:18 -07:00
Tianon Gravi 9d7a12f669 Update manifest-tool to 0.6.0 and take advantage of new "tags:" support (for increased efficiency in pushing aliases) 2017-07-05 13:42:27 -07:00
Tianon Gravi 89c224c724 Add a separate ArchDockerFrom function for better template granularity
Here's an example diff I plan to apply over in the docs with this: (with an example output from `python`)

```diff
diff --git a/.template-helpers/generate-dockerfile-links-partial.tmpl b/.template-helpers/generate-dockerfile-links-partial.tmpl
index 5c13047a..996b9d60 100644
--- a/.template-helpers/generate-dockerfile-links-partial.tmpl
+++ b/.template-helpers/generate-dockerfile-links-partial.tmpl
@@ -4,21 +4,13 @@ This template defines the "Supported tags and Dockerfile links" portion of an im

 */ -}}

-{{- define "dockerfilePath" -}}
-	{{- $dir := .ArchDirectory arch -}}
-	{{- if ne $dir "." -}}
-		{{- $dir -}}
-		/
-	{{- end -}}
-	Dockerfile
-{{- end -}}
-
 {{- range $i, $e := $.Entries -}}
-	{{- if $e.HasArchitecture arch -}}
+	{{- $arch := ternary (ternary arch ($e.Architectures | first) ($e.HasArchitecture arch)) arch (eq arch "amd64") -}}
+	{{- if $e.HasArchitecture $arch -}}
 		{{- /* force bashbrew to "fetch" the GitCommit so that it is fully resolved */ -}}
-		{{- $from := $.DockerFrom $e -}}
+		{{- $from := $.ArchDockerFrom $arch $e -}}

-		{{- $gitRepo := $e.ArchGitRepo arch -}}
+		{{- $gitRepo := $e.ArchGitRepo $arch -}}
 		{{- $repoUrlBase := $gitRepo | replace "git://" "https://" | trimSuffixes ".git" -}}
 		{{- $isGitHub := hasPrefix "https://github.com/" $repoUrlBase -}}

@@ -28,9 +20,11 @@ This template defines the "Supported tags and Dockerfile links" portion of an im

 		- {{- "\t" -}}
 		{{- if $isGitHub -}} [ {{- end -}}
-		` {{- $e.Tags | join "`, `" -}} ` (* {{- template "dockerfilePath" $e -}} *)
-		{{- $gitCommit := $e.ArchGitCommit arch -}}
-		{{- if $isGitHub -}} ]( {{- $repoUrlBase -}} /blob/ {{- $gitCommit -}} / {{- template "dockerfilePath" $e -}} ) {{- end -}}
+		{{- $dir := .ArchDirectory $arch -}}
+		{{- $dockerfilePath := ternary "Dockerfile" (join "/" $dir "Dockerfile") (eq $dir ".") -}}
+		` {{- $e.Tags | join "`, `" -}} ` (* {{- $dockerfilePath -}} *)
+		{{- $gitCommit := $e.ArchGitCommit $arch -}}
+		{{- if $isGitHub -}} ]( {{- $repoUrlBase -}} /blob/ {{- $gitCommit -}} / {{- $dockerfilePath -}} ) {{- end -}}
 		{{- /* TODO decide what to do (if anything) about non-GitHub repos with respect to URL */ -}}
 	{{- end -}}
 {{- end -}}
diff --git a/python/README.md b/python/README.md
index c23f79f0..410b8525 100644
--- a/python/README.md
+++ b/python/README.md
@@ -22,6 +22,7 @@ WARNING:
 -	[`2.7.13-alpine3.6`, `2.7-alpine3.6`, `2-alpine3.6` (*2.7/alpine3.6/Dockerfile*)](e81758e60c/2.7/alpine3.6/Dockerfile)
 -	[`2.7.13-wheezy`, `2.7-wheezy`, `2-wheezy` (*2.7/wheezy/Dockerfile*)](1ca4a57b20/2.7/wheezy/Dockerfile)
 -	[`2.7.13-onbuild`, `2.7-onbuild`, `2-onbuild` (*2.7/onbuild/Dockerfile*)](7663560df7/2.7/onbuild/Dockerfile)
+-	[`2.7.13-windowsservercore`, `2.7-windowsservercore`, `2-windowsservercore` (*2.7/windows/windowsservercore/Dockerfile*)](db2d58d730/2.7/windows/windowsservercore/Dockerfile)
 -	[`3.3.6`, `3.3` (*3.3/Dockerfile*)](88ba87d31a/3.3/Dockerfile)
 -	[`3.3.6-slim`, `3.3-slim` (*3.3/slim/Dockerfile*)](88ba87d31a/3.3/slim/Dockerfile)
 -	[`3.3.6-alpine`, `3.3-alpine` (*3.3/alpine/Dockerfile*)](88ba87d31a/3.3/alpine/Dockerfile)
@@ -36,16 +37,19 @@ WARNING:
 -	[`3.5.3-slim`, `3.5-slim` (*3.5/slim/Dockerfile*)](88ba87d31a/3.5/slim/Dockerfile)
 -	[`3.5.3-alpine`, `3.5-alpine` (*3.5/alpine/Dockerfile*)](88ba87d31a/3.5/alpine/Dockerfile)
 -	[`3.5.3-onbuild`, `3.5-onbuild` (*3.5/onbuild/Dockerfile*)](9a9021f213/3.5/onbuild/Dockerfile)
+-	[`3.5.3-windowsservercore`, `3.5-windowsservercore` (*3.5/windows/windowsservercore/Dockerfile*)](db2d58d730/3.5/windows/windowsservercore/Dockerfile)
 -	[`3.6.1`, `3.6`, `3`, `latest` (*3.6/Dockerfile*)](88ba87d31a/3.6/Dockerfile)
 -	[`3.6.1-slim`, `3.6-slim`, `3-slim`, `slim` (*3.6/slim/Dockerfile*)](88ba87d31a/3.6/slim/Dockerfile)
 -	[`3.6.1-alpine`, `3.6-alpine`, `3-alpine`, `alpine` (*3.6/alpine/Dockerfile*)](88ba87d31a/3.6/alpine/Dockerfile)
 -	[`3.6.1-alpine3.6`, `3.6-alpine3.6`, `3-alpine3.6`, `alpine3.6` (*3.6/alpine3.6/Dockerfile*)](88ba87d31a/3.6/alpine3.6/Dockerfile)
 -	[`3.6.1-onbuild`, `3.6-onbuild`, `3-onbuild`, `onbuild` (*3.6/onbuild/Dockerfile*)](7eca63adca/3.6/onbuild/Dockerfile)
+-	[`3.6.1-windowsservercore`, `3.6-windowsservercore`, `3-windowsservercore`, `windowsservercore` (*3.6/windows/windowsservercore/Dockerfile*)](db2d58d730/3.6/windows/windowsservercore/Dockerfile)
 -	[`3.6.2rc1`, `3.6-rc`, `rc` (*3.6-rc/Dockerfile*)](73d283d3dc/3.6-rc/Dockerfile)
 -	[`3.6.2rc1-slim`, `3.6-rc-slim`, `rc-slim` (*3.6-rc/slim/Dockerfile*)](73d283d3dc/3.6-rc/slim/Dockerfile)
 -	[`3.6.2rc1-alpine`, `3.6-rc-alpine`, `rc-alpine` (*3.6-rc/alpine/Dockerfile*)](73d283d3dc/3.6-rc/alpine/Dockerfile)
 -	[`3.6.2rc1-alpine3.6`, `3.6-rc-alpine3.6`, `rc-alpine3.6` (*3.6-rc/alpine3.6/Dockerfile*)](73d283d3dc/3.6-rc/alpine3.6/Dockerfile)
 -	[`3.6.2rc1-onbuild`, `3.6-rc-onbuild`, `rc-onbuild` (*3.6-rc/onbuild/Dockerfile*)](73d283d3dc/3.6-rc/onbuild/Dockerfile)
+-	[`3.6.2rc1-windowsservercore`, `3.6-rc-windowsservercore`, `rc-windowsservercore` (*3.6-rc/windows/windowsservercore/Dockerfile*)](73d283d3dc/3.6-rc/windows/windowsservercore/Dockerfile)

 # Quick reference

```
2017-06-20 08:36:41 -07:00
Tianon Gravi d60142b532 Validate (and deduplicate) Architectures values 2017-06-14 10:31:28 -07:00
Tianon Gravi 396ac816f8 Add an initial "manifest-tool" implementation 2017-06-13 09:33:53 -07:00
Tianon Gravi 68af563715 Add "arch" function for "bashbrew cat" to access the current architecture string
```console
$ bashbrew cat --format '{{ arch }}' ubuntu
amd64
$ bashbrew --arch=s390x cat --format '{{ arch }}' ubuntu
s390x
$ BASHBREW_ARCH=s390x bashbrew cat --format '{{ arch }}' ubuntu
s390x
```
2017-06-06 11:21:23 -07:00
Tianon Gravi 142e80b2c8 Add a "BASHBREW_PULL" environment variable to control the "pull policy" during build 2017-06-06 11:04:01 -07:00
Tianon Gravi fd7665b53d Fix architecture-specific values inherited from the global entry 2017-06-05 15:40:37 -07:00
Tianon Gravi 001e0803b6 Add "--depth" flag to "bashbrew children" and "bashbrew parents"
```console
$ diff -u <(bashbrew children --depth 2 alpine) <(bashbrew children --depth 3 alpine)
--- /dev/fd/63	2017-06-02 07:29:56.871863245 -0700
+++ /dev/fd/62	2017-06-02 07:29:56.872863215 -0700
@@ -578,6 +578,20 @@
 zookeeper:3.4.10
 zookeeper:3.4
 zookeeper:latest
+maven:3.5.0-jdk-7-onbuild-alpine
+maven:3.5-jdk-7-onbuild-alpine
+maven:3-jdk-7-onbuild-alpine
+maven:3.5.0-jdk-8-onbuild-alpine
+maven:3.5.0-onbuild-alpine
+maven:3.5-jdk-8-onbuild-alpine
+maven:3.5-onbuild-alpine
+maven:3-jdk-8-onbuild-alpine
+maven:3-onbuild-alpine
+maven:onbuild-alpine
+orientdb:2.2.21-spatial
+clojure:lein-2.7.1-alpine-onbuild
+clojure:lein-alpine-onbuild
+clojure:alpine-onbuild
 notary:server
 notary:server-0.5.0
 notary:signer

$ bashbrew parents maven:onbuild-alpine
maven:3-jdk-8-alpine
openjdk:8-jdk-alpine
alpine:3.6

$ bashbrew parents --depth 1 maven:onbuild-alpine
maven:3-jdk-8-alpine

$ bashbrew parents --depth 2 maven:onbuild-alpine
maven:3-jdk-8-alpine
openjdk:8-jdk-alpine
```
2017-06-02 07:31:52 -07:00
Tianon Gravi fc83c27756 Add initial Architectures support (and the beginnings of a "put-shared" subcommand for bringing all the SharedTags and arch-specific images together with manifest lists/indexes) 2017-06-01 16:56:04 -07:00
Tianon Gravi 79c178ab7f Add a simple bashbrew TODO 2017-03-13 10:33:54 -07:00
Tianon Gravi 310ccec573 Apply "gofmt" 2017-02-10 12:41:10 -08:00
Tianon Gravi ae7dc214bb Update "github.com/docker-library/go-dockerlibrary" vendoring (improved error handling, more edge cases) 2017-01-24 13:56:03 -08:00
Tianon Gravi 1087b55cf9 Update "pault.ag/go/debian" vendoring 2017-01-24 13:48:42 -08:00
Tianon Gravi c034d7fdbf Update bashbrew to cache remote tag fetches (speeds up "--all" operations by avoiding "git fetch" where we already have a cached answer in Git)
Example tag name: `git-/github.com/docker/notary-official-images-refs/tags/v0.5.0-FETCH_HEAD`

Referring to:
```
signer-0.5.0: git://github.com/docker/notary-official-images@v0.5.0 notary-signer
```
2016-11-17 16:51:39 -08:00
Tianon Gravi 39f53e972c Update bashbrew Constraints logic with several fixes
- only print a Constraints-related "skipping" warning for a given "repo:tag" once per session
- apply constraints while sorting too (so that `bashbrew build --all` on Windows doesn't try to `git fetch` Linux rootfs tarballs only to skip them during build, for example)
- add `--apply-constraints` flag to `from`, `children`, and `parents` subcommands
2016-11-17 15:53:28 -08:00
Tianon Gravi 9e57342714 Move "build order" sorting functions into a common file and refactor them to use topsort properly and via a single unified function (single node per logical "thing" we're sorting, resolve tag names to canonical node names appropriately) 2016-09-22 10:18:30 -07:00
Tianon Gravi e399b2aefb Adjust bashbrew such that env overrides config appropriately 2016-07-28 11:30:54 -07:00
Tianon Gravi d5416b33ec Add a separate "EntryIdentifier" so that build output is more informative
This changes something like:

    Using bashbrew/cache:2775145643e397ca993e842f3263a1362a49df4115ce59d59bed1452323b7af2 (buildpack-deps)

into:

    Using bashbrew/cache:2775145643e397ca993e842f3263a1362a49df4115ce59d59bed1452323b7af2 (buildpack-deps:wheezy)
2016-07-22 10:00:36 -07:00
Tianon Gravi 0d0cc158db Update "github.com/docker-library/go-dockerlibrary" for parsing line-based maintainers comments 2016-07-20 11:23:22 -07:00
Joe Ferguson 965b5a788c Add DockerCacheName to be able to use it from templates
- like the following to get golang:latest and the cache name of its parent
   ```console
   $ bashbrew cat --format '{{ range .Entries }}{{ $.DockerCacheName . }}{{ range .Tags }}{{"\n"}}{{ join ":" $.RepoName . }}{{end}}{{"\n"}}{{end}}' golang:latest
   ```
2016-07-06 13:27:39 -07:00
Tianon Gravi b22c1ebbbc Capture "command failure" while reading Dockerfiles for FROM lines 2016-06-27 11:10:14 -07:00
Tianon Gravi 6e93c269d5 Fix spurious "bad file descriptor" errors in bashbrew by using "exec.Cmd" objects more appropriately while streaming 2016-06-24 17:32:43 -07:00
Stuart Clark 24f286f82e Correct typo in bashbrew push --namespace message 2016-06-22 05:51:11 +01:00
Tianon Gravi a5abea085a Rewrite constraints logic to actually work properly 2016-06-10 11:32:23 -07:00
Tianon Gravi db6c174c57 Allow only "http" and "https" URLs 2016-06-08 10:37:08 -07:00
Tianon Gravi bf00dc0332 Facepalm 2016-06-08 10:24:53 -07:00
Tianon Gravi 42f54fb607 First attempt to unstick Windows 2016-06-08 10:23:58 -07:00
Tianon Gravi 8f3101ea0a Add support for "--pull always", "--pull missing", and "--pull never" (replacing "--pull-missing") 2016-06-07 16:40:52 -07:00
Tianon Gravi b309871dd5 Auto-upgrade "git://github.com/..." to "https://github.com/..." (with a warning) 2016-06-07 16:14:30 -07:00
Tianon Gravi cb75ba5395 Apply constraints during "tag" and "push" (since they're only useful after "build", at least currently) 2016-06-07 12:24:01 -07:00
Tianon Gravi 74144f4a97 Add extra output when we're "Using" an older dockerCacheHash, even if we didn't need to build it 2016-06-06 16:30:10 -07:00
Tianon Gravi 278e72b2e9 Also remove "-f" from "docker tag" (since it's deprecated in 1.10+ and removed in 1.12+) 2016-06-06 11:37:54 -07:00
Tianon Gravi 3b661c2c61 Explicitly note that Docker 1.10+ is required for the new Bashbrew
This is especially thanks to https://github.com/docker/docker/issues/15785 and thus c8cc4fb8d9.
2016-06-06 10:41:10 -07:00
Tianon Gravi 1bf47856f0 Add a "--repos" flag to "bashbrew list" to prefer to list only repos 2016-06-03 14:29:26 -07:00
Tianon Gravi 66ff2924c2 Fix "bashbrew list --uniq --build-order alpine:latest alpine:3.3" (to properly list two tags instead of just one) 2016-06-03 14:28:40 -07:00
Tianon Gravi faf6af3ab8 Rename "--verbose" to the more-apt "--debug", and adjust "docker build" to _always_ capture output (expect when debug mode is enabled and it's piped directly to the user) so that it can be included in the error response if the build fails 2016-06-03 12:15:55 -07:00
Tianon Gravi d701c72366 Don't pull scratch 2016-06-03 11:01:23 -07:00
Tianon Gravi 7e46e0cea7 Use cli's "MultiError" construct to provide useful context for top-level cmd-*.go errors 2016-06-03 10:23:09 -07:00
Tianon Gravi b244f42327 Add a "--format-file" flag to "bashbrew cat" which overrides "--format" with the contents of a file (and add more useful functions to our template processing) 2016-06-03 10:23:09 -07:00