Commit Graph

122 Commits

Author SHA1 Message Date
CrazyMax a03263acf8
build: handle push/load shorthands for multi exporters
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-03-05 17:05:59 +01:00
Nicolas De Loof 598bc16e5d
don't print build details when progress is rawjson
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2024-02-09 13:28:44 +01:00
CrazyMax 052f279de7
bake: fix definitions merge order
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-02-02 10:55:25 +01:00
Jonathan A. Sternberg c65b7ed24f
otel: include service instance id attribute to resource and move to metricutil package
Add the service instance id to the resource attributes to prevent
downstream OTEL processors and exporters from thinking that the CLI
invocations are a single process that keeps restarting. The unique id
can be removed through downstream aggregation to prevent cardinality
issues, but we need some way to tell OTEL that it shouldn't reset the
counters.

Move the check for the experimental flag to its own package and then use
that invocation to prevent creating exporters so metrics are disabled
completely. This makes it so we don't have to check for the experimental
flag in every place we add metrics until we decide to make metrics
stable in general.

This also moves the OTEL initialization to a `util/metricutil` package
to be more consistent with the existing util naming and to differentiate
it from the upstream `metric` name. Using both `metrics` and `metric` as
import names was confusing since `metric` was an upstream dependency and
`metrics` was a local utility. `metricutil` matches with the existing
utilities and makes clear that it isn't a spelling mistake.

The record version metric has been removed since we weren't planning on
keeping that metric anyway and most of the information is now included
in the instrumentation library name and version. That function is
included as a utility in the `otel/sdk/metric` package to retrieve the
appropriate meter from the meter provider.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2024-01-30 16:27:02 -06:00
Tonis Tiigi 147c7135b0
simplify signal handling for cobra context
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2024-01-19 17:18:42 -08:00
Laura Brehm 650a7af0ae
cobra/commands: cancel command context on signal
See https://github.com/docker/cli/pull/4599 and
https://github.com/docker/cli/pull/4769.

Since we switched to using the cobra command context instead of
`appcontext`, we need to set up the signal handling that was being
provided by `appcontext`, as well as configuring the context with
the OTEL tracing utilities also used by `appcontext`.

This commit introduces `cobrautil.ConfigureContext` which implements
the pre-existing signal handling logic from `appcontext` and cancels
the command's context when a signal is received, as well as doing
the relevant OTEL config.

Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2024-01-19 16:30:52 -08:00
Laura Brehm 4f738020fd
deps: remove `appcontext`, use `cmd.Context`
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2024-01-19 16:29:12 -08:00
Jonathan A. Sternberg 7094eb86c9
metrics: send metrics to the otel collector endpoint when active
Introduce a meter provider to the buildx cli that will send metrics to
the otel-collector included in docker desktop if enabled.

This will send usage metrics to the desktop application but also send
metrics to a user-provided otlp receiver endpoint through the standard
environment variables.

This introduces a single metric which is the cli count for build and
bake along with the command name and a few additional attributes.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2024-01-05 14:50:54 -06:00
CrazyMax ff8bca206b
Merge pull request #2100 from dvdksn/bake-empty-definition
bake: improve error message for empty definition
2023-11-06 03:17:12 -08:00
David Karlsson d0a9a81e2e bake: improve error message for empty definition
Closes #2095

Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2023-10-31 15:36:50 +01:00
Cameron Adams abfc04f621
command(bake): Specify local and remote bake files
This adds the ability to source additional local build definition files when
sourcing Bake files via a remote url.
Prefixing a file with 'cwd://' will source a bake file on the local
machine, instead of the remote location.
Local files will be read/have precedence before remote files.

Usage:
```
docker buildx bake https://github.com/example/upstream.git --file cwd://docker-bake.override.hcl --print
```
This will source a default file from the example/upstream repository,
and also source a build definition from the local machine.

Also moves remote and local files reading logic to a func

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
Signed-off-by: Cameron Adams <pnzreba@gmail.com>
Co-authored-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-24 19:37:04 +02:00
CrazyMax 23c2498dee
bake: display read definition files in build output
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-23 16:26:18 +02:00
CrazyMax fd251d2a7b
builder: use variadic options in LoadNodes func
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-17 00:45:50 +02:00
CrazyMax 6c77b76b7b
Merge pull request #2038 from crazy-max/localstate-group
build: support local state group
2023-09-30 06:01:14 +02:00
CrazyMax 3dcb03452c
build: support local state group
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-09-30 01:08:06 +02:00
Justin Chadwell e2ebab5f26 vendor: update buildkit to master@cbfd4023383d
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-09-27 11:06:18 +01:00
Justin Chadwell f5f00e68ef bake(cli): allow passing in-stream using command.Cli
ReadLocalFiles should allow passing the stdin file as an argument, which
allows us to read from dockerCli.Stdin() to be consistent with other
commands in the same package.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-09 10:50:53 +01:00
CrazyMax 0a2f35970c
build: display build details link
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-06 18:54:31 +02:00
Justin Chadwell e61a1da7fc
Merge pull request #1748 from dvdksn/docs/bake-reference
docs: move and rewrite bake reference
2023-05-11 13:00:21 +01:00
David Karlsson f8483d7243 docs: refactor bake file reference
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2023-05-11 12:54:08 +01:00
CrazyMax 621b07c799
Merge pull request #1727 from crazy-max/cmd-completion
cli: add shell completion
2023-05-09 10:59:32 +02:00
Justin Chadwell 16d5b38f2b debug: display build warnings after each build
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-24 09:58:02 +01:00
CrazyMax af011d6ca3
bake: add completion to list targets
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-11 11:34:40 +02:00
Kohei Tokunaga 8ba8659496
controller: Extract nested CommonOptions on controller API
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-04-03 21:35:04 +09:00
Justin Chadwell 87a120e8e3 bake: use build.IsRemoteURL
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-03 10:48:31 +01:00
Justin Chadwell 780531425b bake: move SOURCE_DATE_EPOCH parsing to overrides
Previously, when directly modifying the args map when reading targets,
we could end up in a scenario where bake tests that compare arg maps
would fail if SOURCE_DATE_EPOCH was set in the environment.

This patch prevents this failure by setting the SOURCE_DATE_EPOCH at the
command level (which isn't injected into tests as well), ensuring that
we test correctly even when SOURCE_DATE_EPOCH is set in the environment.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-15 10:06:53 +00:00
Justin Chadwell c2e11196dd controller: handle attestation options across api
We can perform all attestation processing, handling how the sbom and
provenance arguments interact on the client, while applying defaults on
the server.

Additionally, this allows us to start pulling fields out of CommonOpts.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-23 15:46:37 +00:00
CrazyMax 598f1f0a62
build: print instance being used
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-16 14:56:46 +01:00
Justin Chadwell a9fd128910 controller: move controllers out of commands into separate package
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-31 17:51:33 +00:00
Kohei Tokunaga a27b8395b1 Enable to run build and invoke in background
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-01-26 17:17:07 +09:00
CrazyMax 8c47277141
store: set nodegroup last activity
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-08 19:49:51 +01:00
Justin Chadwell 25aa893bad bake: add attests field
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-12-07 18:44:21 +00:00
CrazyMax e7b5ee7518
mutualize builder logic
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-06 12:13:41 +01:00
CrazyMax 63073b65c0
dockerutil pkg to manage docker api client and context
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-06 12:13:41 +01:00
CrazyMax e66410b932
Merge pull request #1313 from jedevc/bake-group-recurse
bake: recursively resolve groups
2022-11-25 11:27:09 +01:00
David Karlsson be3b41acc6 docs: refactored file and directory structure
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2022-11-04 19:57:50 +01:00
Justin Chadwell a50e89c38e progress: explicitly fail if tty requested but not available
The NewPrinter function is mostly borrowed from buildkit. However, at
some point, it seems that the implementations drifted.

This patch updates buildx to be more similar in behavior to it's
buildkit counterpart, specifically, it will explicitly fail if a TTY
output is requested using "--progress=tty", but the output is not
available.

To gracefully fallback to plain progress in this scenario,
"--progress=plain" is required.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-10-25 10:55:36 +01:00
CrazyMax c53b0b8a12
bake: do not fail printing definition if instance unavailable
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-10-04 15:38:45 +02:00
Justin Chadwell 77b33260f8 bake: recursively resolve groups
Groups that contained other groups were not recursively resolved by
ReadTargets, which prevented output from --print from being useable as a
self-contained bake file.

This patch ensures that all groups that are referenced inside the bake
file are actually defined under the groups field. This has required a
substantial refactor, as previously only a single group was returned
from ReadTargets, notably, returning a map of groups, instead of a
slice.

This does introduce a small behavior change to the behavior of --print -
while previously, passing a group name to bake would return all the
targets of that group back as the default group, now only the name of
that group will be inserted into the default group, keeping the original
group intact. The impact of this can be observed in some of the changes
to the bake_test.go file.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-09-12 13:51:34 +01:00
CrazyMax 7ef679d945
bake(compose): allow dot in target name
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-07-31 16:53:44 +02:00
Tonis Tiigi c255c04eed bake: restore consistent output for metadata
Metadata formatting should not depend on the number
of targets.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-03-08 10:06:03 -08:00
Tonis Tiigi b5bf28d722 bake: use better error in named contexts not supported
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-02-22 21:56:52 -08:00
CrazyMax 299fd19c49
build: enhance metadata json output
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-02-11 08:16:42 +01:00
Edgar Lee 2d7540fb0a Separate io.Writer from console.File for NewPrinter
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2022-02-06 09:11:47 -08:00
CrazyMax 316ca972b6
bake: fix print output
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-11-24 21:54:57 +01:00
CrazyMax a0a7db127c
cli: fix flags usage
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-11-22 10:51:59 +01:00
Tonis Tiigi 88d0775692 refactor accessing registry configs via drivers
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-11-04 10:01:23 -07:00
Tonis Tiigi aa52a5a699 build: set local sharedkey per project basename
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-10-29 21:29:04 -07:00
CrazyMax 36a37a624e
refactor: flags lexical order
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-10-26 13:11:28 +02:00
CrazyMax 9e62c9f074
Fix --driver flag usage markdown output
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-09-29 15:33:44 +02:00
Tonis Tiigi 6fc906532b bake: fix using push override with output definition
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-09-20 16:22:27 -07:00
CrazyMax 55a3ce606f
Built-in variable `BAKE_LOCAL_PLATFORM`
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-09-02 23:06:05 +02:00
CrazyMax c685e46609
bake: print default group
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-08-12 09:02:36 +02:00
Tonis Tiigi 1bb425a882 bake: allow BAKE_CMD_CONTEXT builtin var
Allows accessing the main context for bake command from bake
file that has been imported remotely.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-07-12 21:01:41 -07:00
Tonis Tiigi 9d88450118 enable opentelemetry support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-07-12 13:42:52 -07:00
CrazyMax 1253020b3d
Add metadata-file flag
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-06-30 07:41:26 +02:00
Tonis Tiigi cffcd57edb bake: support for remote files
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-12-07 22:02:51 -08:00
Ilya Dmitrichenko 40b41ac6e4
bake: ensure `--builder` is wired from root options
Signed-off-by: Ilya Dmitrichenko <errordeveloper@gmail.com>
2020-07-08 12:03:15 +01:00
Tonis Tiigi 18095ee87b bake: reset no-cache and pull if not set
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-04-30 13:01:45 -07:00
Tibor Vass c4d07f67e3 commands: check if flag is set instead of using flagutil.Tristate
Fixes --pull and --no-cache without argument

Signed-off-by: Tibor Vass <tibor@docker.com>
2020-04-30 12:25:41 -07:00
Patrick Van Stee 4121ae50b5 Modify parsing functions and config structs to accept hcl changes
Signed-off-by: Patrick Van Stee <patrick@vanstee.me>
2020-04-29 08:51:26 -04:00
Brian Goff b2ec1d331c Add builder as a global flag.
This allows all subcommands to use this flag.
Additionally reads the default value for the flag from the
`BUILDX_BUILDER` env var.

Precedence is:

CLI ARG > flag > env var > config file

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-04-27 14:37:17 -07:00
Brian Goff 213d3af3b0 Add option to build/bake to override instance
This helps prevent race conditions with concurrent build invocations.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-04-27 11:12:31 -07:00
Tibor Vass 2a257a8252 bake: allow overriding no-cache and pull per target via --set
Signed-off-by: Tibor Vass <tibor@docker.com>
2020-04-23 18:11:22 +00:00
Tibor Vass 417f52e001 bake: add --load and --push shorthands for --set
Signed-off-by: Tibor Vass <tibor@docker.com>
2020-04-23 05:54:11 +00:00
Tibor Vass 2bca8fa677 bake: allow pattern matching for target names in --set
Although bake is for running multiple targets, --set required a single
target name for overriding a property. This change allows matching
multiple targets for overrides.

Signed-off-by: Tibor Vass <tibor@docker.com>
2020-04-23 05:54:11 +00:00
Akihiro Suda 6b65b0c982 new driver: kubernetes
Tested with `kind` and GKE.

Note: "nodes" shown in `docker buildx ls` are unrelated to Kubernetes "nodes".
Probably buildx should come up with an alternative term.

Usage:

  $ kind create cluster
  $ export KUBECONFIG="$(kind get kubeconfig-path --name="kind")"

  $ docker buildx create --driver kubernetes --driver-opt replicas=3 --use
  $ docker buildx build -t foo --load .

`--load` loads the image into the local Docker.

Driver opts:

  - `image=IMAGE` - Sets the container image to be used for running buildkit.
  - `namespace=NS` - Sets the Kubernetes namespace. Defaults to the current namespace.
  - `replicas=N` - Sets the number of `Pod` replicas. Defaults to 1.
  - `rootless=(true|false)` - Run the container as a non-root user without `securityContext.privileged`. Defaults to false.
  - `loadbalance=(sticky|random)` - Load-balancing strategy. If set to "sticky", the pod is chosen using the hash of the context path. Defaults to "sticky"

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2019-11-21 10:30:39 +09:00
Tibor Vass 8b2171f78a bake: honor --no-cache and --pull
Signed-off-by: Tibor Vass <tibor@docker.com>
2019-07-30 19:39:01 -07:00
Brian Goff e23e4a6bdc Correct help output for default bake file.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2019-05-08 08:54:00 -07:00
Tonis Tiigi 94a0422a1b rename to docker/buildx
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-04-24 20:02:20 -07:00
Tonis Tiigi 8fed6d4d31 bake: rename compose file
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-04-10 15:19:41 -07:00
Tonis Tiigi a932d52e35 bake: initial implementation
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-04-09 19:26:53 -07:00