Commit Graph

58 Commits

Author SHA1 Message Date
Tianon Gravi 69c374aa98 Hotfix: "bashbrew build" sorting fix 2019-01-28 17:01:16 -08:00
Tianon Gravi 6930483624 Improve bashbrew SharedTags handling (explicitly allowed for "bashbrew cat", etc) 2019-01-28 16:16:50 -08:00
Tianon Gravi e692c1951d Update go-dockerlibrary to include https://github.com/docker-library/go-dockerlibrary/pull/19 2019-01-07 16:22:36 -08:00
Tianon Gravi 087cef7a9f Add support for "File:" in manifest files
See https://github.com/docker-library/go-dockerlibrary/pull/16 for format implementation details.
2018-10-22 16:06:21 -07:00
Tianon Gravi e3d787c1b5 Add new "archFilter" template function
Example usage:

```
{{- range $i, $e := ($archSpecific | ternary (archFilter arch $.Entries) $.Entries) -}}
    ...
{{- end -}}
```
2018-08-23 11:37:13 -07:00
Tianon Gravi 9232e049ed Slightly improved error message for failed template executions 2018-08-14 12:48:04 -07:00
Tianon Gravi 42c59432d2 Improve put-shared noop behavior and add "--force" flag for pushing operations 2018-05-03 12:21:57 -07:00
Tianon Gravi 891033f93a Add missing newline 2018-01-24 10:12:23 -08:00
Tianon Gravi 01c6ebec24 Update "put-shared" to allow some tags to fail per-repo (and collect errors) 2018-01-24 09:57:47 -08:00
Tianon Gravi d6ebf0c1ac Also tag SharedTags (and include them in dependency calculations, etc)
This will include SharedTags directly in "bashbrew list", etc, but explicitly does _not_ include them during "bashbrew push".  This allows for other images to be "FROM" a SharedTags value, and should be reasonably safe since any given SharedTag should only be applicable for a given architecture at most once.
2017-10-24 10:47:17 -07:00
Tianon Gravi 6e8d50e38f Add new "archNamespace" template function for parsing "--arch-namespace" parameters
Example usage: `{{ archNamespace "windows-amd64" }}` => `winamd64`

This is intended for use in https://github.com/docker-library/docs to help create a list of "Supported architectures" for each image which links to each supported architecture repository (and if supported by Bashbrew itself, allows us to keep https://github.com/docker-library/docs isolated from Jenkins assumptions while still allowing Jenkins to contain the single-source-of-truth for arch-to-namespace mappings).
2017-10-02 12:27:10 -07:00
Tianon Gravi 683425b864 Add a "--single-arch" flag to "put-shared" for pushing SharedTags to architecture-specific namespaces 2017-09-12 16:18:27 -07:00
Tianon Gravi ba9fd0d60c Minor bashbrew de-optimization 2017-09-12 13:32:45 -07:00
Tianon Gravi 9805fc1419 Add a "--dry-run" flag to "build", "tag", "push", and "put-shared" 2017-09-07 13:53:58 -07:00
Tianon Gravi 2a863261a8 CUE LOUD GRUMBLING 2017-09-06 13:11:15 -07:00
Tianon Gravi 16d5317cab Fix minor typo 2017-09-05 07:36:47 -07:00
Tianon Gravi 259302679c Use the Hub v2 API for a dramatic no-op push speedup
This also has the benefit of only triggering automated builds when there have been actual changes to the image.

This works by comparing the Hub's `last_updated` timestamp to the `Created` timestamp of local images (in the case of `bashbrew push`), or to the `last_updated` timestamp of the images which make up the `SharedTags` and/or multiarch images (in the case of `bashbrew put-shared`).

This is a dramatic speedup because the Hub v2 API is faster (and has no auth for public repos) than the registry API.  If there are any errors in the fetching of `last_updated` from the Hub, we massage the timestamps such that the push simply always happens, as before (which will be the case for pushes to non-Hub registries, for example).
2017-08-28 09:46:47 -07:00
Tianon Gravi e6ff2ce3ff Update go-dockerlibrary to pull in the new "getenv" function
See https://github.com/docker-library/go-dockerlibrary/pull/15.
2017-07-12 16:15:18 -07:00
Tianon Gravi 9d7a12f669 Update manifest-tool to 0.6.0 and take advantage of new "tags:" support (for increased efficiency in pushing aliases) 2017-07-05 13:42:27 -07:00
Tianon Gravi 89c224c724 Add a separate ArchDockerFrom function for better template granularity
Here's an example diff I plan to apply over in the docs with this: (with an example output from `python`)

```diff
diff --git a/.template-helpers/generate-dockerfile-links-partial.tmpl b/.template-helpers/generate-dockerfile-links-partial.tmpl
index 5c13047a..996b9d60 100644
--- a/.template-helpers/generate-dockerfile-links-partial.tmpl
+++ b/.template-helpers/generate-dockerfile-links-partial.tmpl
@@ -4,21 +4,13 @@ This template defines the "Supported tags and Dockerfile links" portion of an im

 */ -}}

-{{- define "dockerfilePath" -}}
-	{{- $dir := .ArchDirectory arch -}}
-	{{- if ne $dir "." -}}
-		{{- $dir -}}
-		/
-	{{- end -}}
-	Dockerfile
-{{- end -}}
-
 {{- range $i, $e := $.Entries -}}
-	{{- if $e.HasArchitecture arch -}}
+	{{- $arch := ternary (ternary arch ($e.Architectures | first) ($e.HasArchitecture arch)) arch (eq arch "amd64") -}}
+	{{- if $e.HasArchitecture $arch -}}
 		{{- /* force bashbrew to "fetch" the GitCommit so that it is fully resolved */ -}}
-		{{- $from := $.DockerFrom $e -}}
+		{{- $from := $.ArchDockerFrom $arch $e -}}

-		{{- $gitRepo := $e.ArchGitRepo arch -}}
+		{{- $gitRepo := $e.ArchGitRepo $arch -}}
 		{{- $repoUrlBase := $gitRepo | replace "git://" "https://" | trimSuffixes ".git" -}}
 		{{- $isGitHub := hasPrefix "https://github.com/" $repoUrlBase -}}

@@ -28,9 +20,11 @@ This template defines the "Supported tags and Dockerfile links" portion of an im

 		- {{- "\t" -}}
 		{{- if $isGitHub -}} [ {{- end -}}
-		` {{- $e.Tags | join "`, `" -}} ` (* {{- template "dockerfilePath" $e -}} *)
-		{{- $gitCommit := $e.ArchGitCommit arch -}}
-		{{- if $isGitHub -}} ]( {{- $repoUrlBase -}} /blob/ {{- $gitCommit -}} / {{- template "dockerfilePath" $e -}} ) {{- end -}}
+		{{- $dir := .ArchDirectory $arch -}}
+		{{- $dockerfilePath := ternary "Dockerfile" (join "/" $dir "Dockerfile") (eq $dir ".") -}}
+		` {{- $e.Tags | join "`, `" -}} ` (* {{- $dockerfilePath -}} *)
+		{{- $gitCommit := $e.ArchGitCommit $arch -}}
+		{{- if $isGitHub -}} ]( {{- $repoUrlBase -}} /blob/ {{- $gitCommit -}} / {{- $dockerfilePath -}} ) {{- end -}}
 		{{- /* TODO decide what to do (if anything) about non-GitHub repos with respect to URL */ -}}
 	{{- end -}}
 {{- end -}}
diff --git a/python/README.md b/python/README.md
index c23f79f0..410b8525 100644
--- a/python/README.md
+++ b/python/README.md
@@ -22,6 +22,7 @@ WARNING:
 -	[`2.7.13-alpine3.6`, `2.7-alpine3.6`, `2-alpine3.6` (*2.7/alpine3.6/Dockerfile*)](e81758e60c/2.7/alpine3.6/Dockerfile)
 -	[`2.7.13-wheezy`, `2.7-wheezy`, `2-wheezy` (*2.7/wheezy/Dockerfile*)](1ca4a57b20/2.7/wheezy/Dockerfile)
 -	[`2.7.13-onbuild`, `2.7-onbuild`, `2-onbuild` (*2.7/onbuild/Dockerfile*)](7663560df7/2.7/onbuild/Dockerfile)
+-	[`2.7.13-windowsservercore`, `2.7-windowsservercore`, `2-windowsservercore` (*2.7/windows/windowsservercore/Dockerfile*)](db2d58d730/2.7/windows/windowsservercore/Dockerfile)
 -	[`3.3.6`, `3.3` (*3.3/Dockerfile*)](88ba87d31a/3.3/Dockerfile)
 -	[`3.3.6-slim`, `3.3-slim` (*3.3/slim/Dockerfile*)](88ba87d31a/3.3/slim/Dockerfile)
 -	[`3.3.6-alpine`, `3.3-alpine` (*3.3/alpine/Dockerfile*)](88ba87d31a/3.3/alpine/Dockerfile)
@@ -36,16 +37,19 @@ WARNING:
 -	[`3.5.3-slim`, `3.5-slim` (*3.5/slim/Dockerfile*)](88ba87d31a/3.5/slim/Dockerfile)
 -	[`3.5.3-alpine`, `3.5-alpine` (*3.5/alpine/Dockerfile*)](88ba87d31a/3.5/alpine/Dockerfile)
 -	[`3.5.3-onbuild`, `3.5-onbuild` (*3.5/onbuild/Dockerfile*)](9a9021f213/3.5/onbuild/Dockerfile)
+-	[`3.5.3-windowsservercore`, `3.5-windowsservercore` (*3.5/windows/windowsservercore/Dockerfile*)](db2d58d730/3.5/windows/windowsservercore/Dockerfile)
 -	[`3.6.1`, `3.6`, `3`, `latest` (*3.6/Dockerfile*)](88ba87d31a/3.6/Dockerfile)
 -	[`3.6.1-slim`, `3.6-slim`, `3-slim`, `slim` (*3.6/slim/Dockerfile*)](88ba87d31a/3.6/slim/Dockerfile)
 -	[`3.6.1-alpine`, `3.6-alpine`, `3-alpine`, `alpine` (*3.6/alpine/Dockerfile*)](88ba87d31a/3.6/alpine/Dockerfile)
 -	[`3.6.1-alpine3.6`, `3.6-alpine3.6`, `3-alpine3.6`, `alpine3.6` (*3.6/alpine3.6/Dockerfile*)](88ba87d31a/3.6/alpine3.6/Dockerfile)
 -	[`3.6.1-onbuild`, `3.6-onbuild`, `3-onbuild`, `onbuild` (*3.6/onbuild/Dockerfile*)](7eca63adca/3.6/onbuild/Dockerfile)
+-	[`3.6.1-windowsservercore`, `3.6-windowsservercore`, `3-windowsservercore`, `windowsservercore` (*3.6/windows/windowsservercore/Dockerfile*)](db2d58d730/3.6/windows/windowsservercore/Dockerfile)
 -	[`3.6.2rc1`, `3.6-rc`, `rc` (*3.6-rc/Dockerfile*)](73d283d3dc/3.6-rc/Dockerfile)
 -	[`3.6.2rc1-slim`, `3.6-rc-slim`, `rc-slim` (*3.6-rc/slim/Dockerfile*)](73d283d3dc/3.6-rc/slim/Dockerfile)
 -	[`3.6.2rc1-alpine`, `3.6-rc-alpine`, `rc-alpine` (*3.6-rc/alpine/Dockerfile*)](73d283d3dc/3.6-rc/alpine/Dockerfile)
 -	[`3.6.2rc1-alpine3.6`, `3.6-rc-alpine3.6`, `rc-alpine3.6` (*3.6-rc/alpine3.6/Dockerfile*)](73d283d3dc/3.6-rc/alpine3.6/Dockerfile)
 -	[`3.6.2rc1-onbuild`, `3.6-rc-onbuild`, `rc-onbuild` (*3.6-rc/onbuild/Dockerfile*)](73d283d3dc/3.6-rc/onbuild/Dockerfile)
+-	[`3.6.2rc1-windowsservercore`, `3.6-rc-windowsservercore`, `rc-windowsservercore` (*3.6-rc/windows/windowsservercore/Dockerfile*)](73d283d3dc/3.6-rc/windows/windowsservercore/Dockerfile)

 # Quick reference

```
2017-06-20 08:36:41 -07:00
Tianon Gravi d60142b532 Validate (and deduplicate) Architectures values 2017-06-14 10:31:28 -07:00
Tianon Gravi 396ac816f8 Add an initial "manifest-tool" implementation 2017-06-13 09:33:53 -07:00
Tianon Gravi 68af563715 Add "arch" function for "bashbrew cat" to access the current architecture string
```console
$ bashbrew cat --format '{{ arch }}' ubuntu
amd64
$ bashbrew --arch=s390x cat --format '{{ arch }}' ubuntu
s390x
$ BASHBREW_ARCH=s390x bashbrew cat --format '{{ arch }}' ubuntu
s390x
```
2017-06-06 11:21:23 -07:00
Tianon Gravi 142e80b2c8 Add a "BASHBREW_PULL" environment variable to control the "pull policy" during build 2017-06-06 11:04:01 -07:00
Tianon Gravi fd7665b53d Fix architecture-specific values inherited from the global entry 2017-06-05 15:40:37 -07:00
Tianon Gravi 001e0803b6 Add "--depth" flag to "bashbrew children" and "bashbrew parents"
```console
$ diff -u <(bashbrew children --depth 2 alpine) <(bashbrew children --depth 3 alpine)
--- /dev/fd/63	2017-06-02 07:29:56.871863245 -0700
+++ /dev/fd/62	2017-06-02 07:29:56.872863215 -0700
@@ -578,6 +578,20 @@
 zookeeper:3.4.10
 zookeeper:3.4
 zookeeper:latest
+maven:3.5.0-jdk-7-onbuild-alpine
+maven:3.5-jdk-7-onbuild-alpine
+maven:3-jdk-7-onbuild-alpine
+maven:3.5.0-jdk-8-onbuild-alpine
+maven:3.5.0-onbuild-alpine
+maven:3.5-jdk-8-onbuild-alpine
+maven:3.5-onbuild-alpine
+maven:3-jdk-8-onbuild-alpine
+maven:3-onbuild-alpine
+maven:onbuild-alpine
+orientdb:2.2.21-spatial
+clojure:lein-2.7.1-alpine-onbuild
+clojure:lein-alpine-onbuild
+clojure:alpine-onbuild
 notary:server
 notary:server-0.5.0
 notary:signer

$ bashbrew parents maven:onbuild-alpine
maven:3-jdk-8-alpine
openjdk:8-jdk-alpine
alpine:3.6

$ bashbrew parents --depth 1 maven:onbuild-alpine
maven:3-jdk-8-alpine

$ bashbrew parents --depth 2 maven:onbuild-alpine
maven:3-jdk-8-alpine
openjdk:8-jdk-alpine
```
2017-06-02 07:31:52 -07:00
Tianon Gravi fc83c27756 Add initial Architectures support (and the beginnings of a "put-shared" subcommand for bringing all the SharedTags and arch-specific images together with manifest lists/indexes) 2017-06-01 16:56:04 -07:00
Tianon Gravi 79c178ab7f Add a simple bashbrew TODO 2017-03-13 10:33:54 -07:00
Tianon Gravi 310ccec573 Apply "gofmt" 2017-02-10 12:41:10 -08:00
Tianon Gravi ae7dc214bb Update "github.com/docker-library/go-dockerlibrary" vendoring (improved error handling, more edge cases) 2017-01-24 13:56:03 -08:00
Tianon Gravi 1087b55cf9 Update "pault.ag/go/debian" vendoring 2017-01-24 13:48:42 -08:00
Tianon Gravi c034d7fdbf Update bashbrew to cache remote tag fetches (speeds up "--all" operations by avoiding "git fetch" where we already have a cached answer in Git)
Example tag name: `git-/github.com/docker/notary-official-images-refs/tags/v0.5.0-FETCH_HEAD`

Referring to:
```
signer-0.5.0: git://github.com/docker/notary-official-images@v0.5.0 notary-signer
```
2016-11-17 16:51:39 -08:00
Tianon Gravi 39f53e972c Update bashbrew Constraints logic with several fixes
- only print a Constraints-related "skipping" warning for a given "repo:tag" once per session
- apply constraints while sorting too (so that `bashbrew build --all` on Windows doesn't try to `git fetch` Linux rootfs tarballs only to skip them during build, for example)
- add `--apply-constraints` flag to `from`, `children`, and `parents` subcommands
2016-11-17 15:53:28 -08:00
Tianon Gravi 9e57342714 Move "build order" sorting functions into a common file and refactor them to use topsort properly and via a single unified function (single node per logical "thing" we're sorting, resolve tag names to canonical node names appropriately) 2016-09-22 10:18:30 -07:00
Tianon Gravi e399b2aefb Adjust bashbrew such that env overrides config appropriately 2016-07-28 11:30:54 -07:00
Tianon Gravi d5416b33ec Add a separate "EntryIdentifier" so that build output is more informative
This changes something like:

    Using bashbrew/cache:2775145643e397ca993e842f3263a1362a49df4115ce59d59bed1452323b7af2 (buildpack-deps)

into:

    Using bashbrew/cache:2775145643e397ca993e842f3263a1362a49df4115ce59d59bed1452323b7af2 (buildpack-deps:wheezy)
2016-07-22 10:00:36 -07:00
Tianon Gravi 0d0cc158db Update "github.com/docker-library/go-dockerlibrary" for parsing line-based maintainers comments 2016-07-20 11:23:22 -07:00
Joe Ferguson 965b5a788c Add DockerCacheName to be able to use it from templates
- like the following to get golang:latest and the cache name of its parent
   ```console
   $ bashbrew cat --format '{{ range .Entries }}{{ $.DockerCacheName . }}{{ range .Tags }}{{"\n"}}{{ join ":" $.RepoName . }}{{end}}{{"\n"}}{{end}}' golang:latest
   ```
2016-07-06 13:27:39 -07:00
Tianon Gravi b22c1ebbbc Capture "command failure" while reading Dockerfiles for FROM lines 2016-06-27 11:10:14 -07:00
Tianon Gravi 6e93c269d5 Fix spurious "bad file descriptor" errors in bashbrew by using "exec.Cmd" objects more appropriately while streaming 2016-06-24 17:32:43 -07:00
Stuart Clark 24f286f82e Correct typo in bashbrew push --namespace message 2016-06-22 05:51:11 +01:00
Tianon Gravi a5abea085a Rewrite constraints logic to actually work properly 2016-06-10 11:32:23 -07:00
Tianon Gravi db6c174c57 Allow only "http" and "https" URLs 2016-06-08 10:37:08 -07:00
Tianon Gravi bf00dc0332 Facepalm 2016-06-08 10:24:53 -07:00
Tianon Gravi 42f54fb607 First attempt to unstick Windows 2016-06-08 10:23:58 -07:00
Tianon Gravi 8f3101ea0a Add support for "--pull always", "--pull missing", and "--pull never" (replacing "--pull-missing") 2016-06-07 16:40:52 -07:00
Tianon Gravi b309871dd5 Auto-upgrade "git://github.com/..." to "https://github.com/..." (with a warning) 2016-06-07 16:14:30 -07:00
Tianon Gravi cb75ba5395 Apply constraints during "tag" and "push" (since they're only useful after "build", at least currently) 2016-06-07 12:24:01 -07:00
Tianon Gravi 74144f4a97 Add extra output when we're "Using" an older dockerCacheHash, even if we didn't need to build it 2016-06-06 16:30:10 -07:00
Tianon Gravi 278e72b2e9 Also remove "-f" from "docker tag" (since it's deprecated in 1.10+ and removed in 1.12+) 2016-06-06 11:37:54 -07:00