linkerd
/
linkerd2
mirror of https://github.com/linkerd/linkerd2.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Migrate CI to docker buildx and other improvements (#4765) 

* Migrate CI to docker buildx and other improvements

## Motivation
- Improve build times in forks. Specially when rerunning builds because of some flaky test.
- Start using `docker buildx` to pave the way for multiplatform builds.

## Performance improvements
These timings were taken for the `kind_integration.yml` workflow when we merged and rerun the lodash bump PR (#4762)

Before these improvements:
- when merging: `24:18`
- when rerunning after merge (docker cache warm): `19:00`
- when running the same changes in a fork (no docker cache): `32:15`

After these improvements:
- when merging: `25:38`
- when rerunning after merge (docker cache warm): `19:25`
- when running the same changes in a fork (docker cache warm): `19:25`

As explained below, non-forks and forks now use the same cache, so the important take is that forks will always start with a warm cache and we'll no longer see long build times like the `32:15` above.
The downside is a slight increase in the build times for non-forks (up to a little more than a minute, depending on the case).

## Build containers in parallel
The `docker_build` job in the `kind_integration.yml`, `cloud_integration.yml` and `release.yml` workflows relied on running `bin/docker-build` which builds all the containers in sequence. Now each container is built in parallel using a matrix strategy.

## New caching strategy
CI now uses `docker buildx` for building the container images, which allows using an external cache source for builds, a location in the filesystem in this case. That location gets cached using actions/cache, using the key `{{ runner.os }}-buildx-${{ matrix.target }}-${{ env.TAG }}` and the restore key `${{ runner.os }}-buildx-${{ matrix.target }}-`.

For example when building the `web` container, its image and all the intermediary layers get cached under the key `Linux-buildx-web-git-abc0123`. When that has been cached in the `main` branch, that cache will be available to all the child branches, including forks. If a new branch in a fork asks for a key like `Linux-buildx-web-git-def456`, the key won't be found during the first CI run, but the system falls back to the key `Linux-buildx-web-git-abc0123` from `main` and so the build will start with a warm cache (more info about how keys are matched in the [actions/cache docs](https://docs.github.com/en/actions/configuring-and-managing-workflows/caching-dependencies-to-speed-up-workflows#matching-a-cache-key)).

## Packet host no longer needed
To benefit from the warm caches both in non-forks and forks like just explained, we're required to ditch doing the builds in Packet and now everything runs in the github runners VMs.
As a result there's no longer separate logic for non-forks and forks in the workflow files; `kind_integration.yml` was greatly simplified but `cloud_integration.yml` and `release.yml` got a little bigger in order to use the actions artifacts as a repository for the images built. This bloat will be fixed when support for [composite actions](https://github.com/actions/runner/blob/users/ethanchewy/compositeADR/docs/adrs/0549-composite-run-steps.md) lands in github.

## Local builds
You still are able to run `bin/docker-build` or any of the `docker-build.*` scripts. And to make use of buildx, run those same scripts after having set the env var `DOCKER_BUILDKIT=1`. Using buildx supposes you have installed it, as instructed [here](https://github.com/docker/buildx).

## Other
- A new script `bin/docker-cache-prune` is used to remove unused images from the cache. Without that the cache grows constantly and we can rapidly hit the 5GB limit (when the limit is attained the oldest entries get evicted).
- The `go-deps` dockerfile base image was changed from `golang:1.14.2` (ubuntu based) to `golang-1:14.2-alpine` also to conserve cache space.

# Addressed separately in #4875:

Got rid of the `go-deps` image and instead added something similar on top of all the Dockerfiles dealing with `go`, as a first stage for those Dockerfiles. That continues to serve as a way to pre-populate go's build cache, which speeds up the builds in the subsequent stages. That build should in theory be rebuilt automatically only when `go.mod` or `go.sum` change, and now we don't require running `bin/update-go-deps-shas`. That script was removed along with all the logic elsewhere that used it, including the `go_dependencies` job in the `static_checks.yml` github workflow.

The list of modules preinstalled was moved from `Dockerfile-go-deps` to a new script `bin/install-deps`. I couldn't find a way to generate that list dynamically, so whenever a slow-to-compile dependency is found, we have to make sure it's included in that list.

Although this simplifies the dev workflow, note that the real motivation behind this was a limitation in buildx's `docker-container` driver that forbids us from depending on images that haven't been pushed to a registry, so we have to resort to building the dependencies as a first stage in the Dockerfiles.
This commit is contained in:
Alejandro Pedraza 2020-07-22 14:27:45 -05:00 committed by GitHub
parent 46d22f8b04
commit 5e789ba152
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
25 changed files with 237 additions and 395 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
.dockerignore
View File

@ -7,6 +7,7 @@
**/node_modules
bin
!bin/fetch-proxy
!bin/install-deps
!bin/web
**/Dockerfile*
Dockerfile*

94
.github/workflows/cloud_integration.yml vendored
View File

@ -8,35 +8,14 @@ on:
- main
env:
GH_ANNOTATION: true
DOCKER_BUILDKIT: 1
jobs:
# todo: Keep in sync with `release.yml`
docker_build:
name: Docker build
runs-on: ubuntu-18.04
steps:
- name: Checkout code
# actions/checkout@v2
uses: actions/checkout@722adc6
- name: Setup SSH config for Packet
run: |
mkdir -p ~/.ssh/
touch ~/.ssh/id && chmod 600 ~/.ssh/id
echo "${{ secrets.DOCKER_SSH_CONFIG }}" > ~/.ssh/config
echo "${{ secrets.DOCKER_PRIVATE_KEY }}" > ~/.ssh/id
echo "${{ secrets.DOCKER_KNOWN_HOSTS }}" > ~/.ssh/known_hosts
ssh linkerd-docker docker version
- name: Build docker images
env:
DOCKER_HOST: ssh://linkerd-docker
DOCKER_TRACE: 1
run: |
export PATH="`pwd`/bin:$PATH"
bin/docker-build
# todo: Keep in sync with `release.yml`
docker_push:
name: Docker push
runs-on: ubuntu-18.04
needs: [docker_build]
strategy:
matrix:
target: [proxy, controller, web, cni-plugin, debug, cli-bin, grafana]
name: Docker build (${{ matrix.target }})
steps:
- name: Checkout code
# actions/checkout@v2
@ -45,6 +24,24 @@ jobs:
run: |
. bin/_tag.sh
echo ::set-env name=TAG::$(CI_FORCE_CLEAN=1 bin/root-tag)
. bin/_docker.sh
echo ::set-env name=DOCKER_REGISTRY::$DOCKER_REGISTRY
echo ::set-env name=DOCKER_BUILDKIT_CACHE::${{ runner.temp }}/.buildx-cache
- name: Cache docker layers
# actions/cache@v2.0.0
uses: actions/cache@b820478
with:
path: ${{ env.DOCKER_BUILDKIT_CACHE }}
key: ${{ runner.os }}-buildx-${{ matrix.target }}-${{ env.TAG }}
restore-keys: |
${{ runner.os }}-buildx-${{ matrix.target }}-
- name: Build docker images
env:
DOCKER_TRACE: 1
run: |
docker buildx create --driver docker-container --use
bin/docker-build-${{ matrix.target }}
- name: Configure gcloud
# linkerd/linkerd2-action-gcloud@v1.0.1
uses: linkerd/linkerd2-action-gcloud@308c4df
@ -52,28 +49,21 @@ jobs:
cloud_sdk_service_account_key: ${{ secrets.CLOUD_SDK_SERVICE_ACCOUNT_KEY }}
gcp_project: ${{ secrets.GCP_PROJECT }}
gcp_zone: ${{ secrets.GCP_ZONE }}
- name: Docker SSH setup
run: |
mkdir -p ~/.ssh/
touch ~/.ssh/id && chmod 600 ~/.ssh/id
echo "${{ secrets.DOCKER_SSH_CONFIG }}" > ~/.ssh/config
echo "${{ secrets.DOCKER_PRIVATE_KEY }}" > ~/.ssh/id
echo "${{ secrets.DOCKER_KNOWN_HOSTS }}" > ~/.ssh/known_hosts
ssh linkerd-docker docker version
- name: Push docker images to registry
env:
DOCKER_HOST: ssh://linkerd-docker
run: |
export PATH="`pwd`/bin:$PATH"
bin/docker-push-deps
bin/docker-push $TAG
bin/docker-retag-all $TAG main
bin/docker-push main
. bin/_docker.sh
docker_push "${{ matrix.target }}" "$TAG"
docker_retag "${{ matrix.target }}" "$TAG" main
docker_push "${{ matrix.target }}" main
- name: Prune docker layers cache
# changes generate new images while the existing ones don't get removed
# so we manually do that to avoid bloating the cache
run: bin/docker-cache-prune
# todo: Keep in sync with `release.yml`
cloud_integration_tests:
name: Cloud integration tests
runs-on: ubuntu-18.04
needs: [docker_push]
needs: [docker_build]
steps:
- name: Checkout code
# actions/checkout@v2
@ -90,14 +80,14 @@ jobs:
id: install_cli
run: |
TAG="$(CI_FORCE_CLEAN=1 bin/root-tag)"
image="gcr.io/linkerd-io/cli-bin:$TAG"
id=$(bin/docker create $image)
bin/docker cp "$id:/out/linkerd-linux" "$HOME/.linkerd"
"$HOME/.linkerd" version --client
CMD="$PWD/target/release/linkerd2-cli-$TAG-linux"
bin/docker-pull-binaries $TAG
$CMD version --client
# validate CLI version matches the repo
[[ "$TAG" == "$($HOME/.linkerd version --short --client)" ]]
[[ "$TAG" == "$($CMD version --short --client)" ]]
echo "Installed Linkerd CLI version: $TAG"
echo "::set-output name=tag::$TAG"
echo "::set-env name=CMD::$CMD"
echo "::set-env name=TAG::$TAG"
- name: Create GKE cluster
# linkerd/linkerd2-action-gcloud@v1.0.1
uses: linkerd/linkerd2-action-gcloud@308c4df
@ -107,17 +97,15 @@ jobs:
gcp_zone: ${{ secrets.GCP_ZONE }}
preemptible: false
create: true
name: testing-${{ steps.install_cli.outputs.tag }}-${{ github.run_id }}
name: testing-${{ env.TAG }}-${{ github.run_id }}
num_nodes: 2
- name: Run integration tests
env:
GITCOOKIE_SH: ${{ secrets.GITCOOKIE_SH }}
run: |
export PATH="`pwd`/bin:$PATH"
echo "$GITCOOKIE_SH" | bash
version="$($HOME/.linkerd version --client --short | tr -cd '[:alnum:]-')"
bin/tests --skip-kind-create "$HOME/.linkerd"
bin/tests --skip-kind-create "$CMD"
- name: CNI tests
run: |
export TAG="$($HOME/.linkerd version --client --short)"
export TAG="$($CMD version --client --short)"
go test -cover -race -v -mod=readonly ./cni-plugin/test -integration-tests

75
.github/workflows/kind_integration.yml vendored
View File

@ -9,10 +9,14 @@ on:
- main
env:
GH_ANNOTATION: true
DOCKER_BUILDKIT: 1
jobs:
docker_build:
name: Docker build
runs-on: ubuntu-18.04
strategy:
matrix:
target: [proxy, controller, web, cni-plugin, debug, cli-bin, grafana]
name: Docker build (${{ matrix.target }})
steps:
- name: Checkout code
# actions/checkout@v2
@ -24,39 +28,35 @@ jobs:
. bin/_docker.sh
echo ::set-env name=DOCKER_REGISTRY::$DOCKER_REGISTRY
- name: Setup SSH config for Packet
if: github.event_name == 'push' || !github.event.pull_request.head.repo.fork
run: |
mkdir -p ~/.ssh/
touch ~/.ssh/id && chmod 600 ~/.ssh/id
echo "${{ secrets.DOCKER_SSH_CONFIG }}" > ~/.ssh/config
echo "${{ secrets.DOCKER_PRIVATE_KEY }}" > ~/.ssh/id
echo "${{ secrets.DOCKER_KNOWN_HOSTS }}" > ~/.ssh/known_hosts
ssh linkerd-docker docker version
echo ::set-env name=DOCKER_HOST::ssh://linkerd-docker
echo ::set-env name=DOCKER_BUILDKIT_CACHE::${{ runner.temp }}/.buildx-cache
- name: Cache docker layers
# actions/cache@v2.0.0
uses: actions/cache@b820478
with:
path: ${{ env.DOCKER_BUILDKIT_CACHE }}
key: ${{ runner.os }}-buildx-${{ matrix.target }}-${{ env.TAG }}
restore-keys: |
${{ runner.os }}-buildx-${{ matrix.target }}-
- name: Build docker images
env:
DOCKER_TRACE: 1
run: |
export PATH="`pwd`/bin:$PATH"
bin/docker-build
docker buildx create --driver docker-container --use
bin/docker-build-${{ matrix.target }}
- name: Prune docker layers cache
# changes generate new images while the existing ones don't get removed
# so we manually do that to avoid bloating the cache
run: bin/docker-cache-prune
- name: Create artifact with CLI and image archives
env:
ARCHIVES: /home/runner/archives
run: |
mkdir -p $ARCHIVES
for image in proxy controller web cni-plugin debug cli-bin grafana; do
docker save "$DOCKER_REGISTRY/$image:$TAG" > $ARCHIVES/$image.tar || tee save_fail &
done
docker save "$DOCKER_REGISTRY/${{ matrix.target }}:$TAG" > $ARCHIVES/${{ matrix.target }}.tar
# save windows cli into artifacts
cp -r ./target/cli/windows/linkerd $ARCHIVES/linkerd-windows.exe
# Wait for `docker save` background processes to complete. Exit early
# if any job failed.
wait < <(jobs -p)
test -f save_fail && exit 1 || true
if [ '${{ matrix.target }}' == 'cli-bin' ]; then
cp -r ./target/cli/windows/linkerd $ARCHIVES/linkerd-windows.exe
fi
# `with.path` values do not support environment variables yet, so an
# absolute path is used here.
#
@ -92,7 +92,6 @@ jobs:
- name: Run CLI Integration tests
run: |
go test --failfast --mod=readonly ".\test\cli" --linkerd=$PWD\image-archives\linkerd-windows.exe --cli-tests -v
# todo: Keep in sync with `release.yml`
kind_integration_tests:
strategy:
matrix:
@ -127,31 +126,12 @@ jobs:
. bin/_docker.sh
echo ::set-env name=DOCKER_REGISTRY::$DOCKER_REGISTRY
- name: Setup SSH config for Packet
if: github.event_name == 'push' || !github.event.pull_request.head.repo.fork
run: |
mkdir -p ~/.ssh/
touch ~/.ssh/id && chmod 600 ~/.ssh/id
echo "${{ secrets.DOCKER_SSH_CONFIG }}" > ~/.ssh/config
echo "${{ secrets.DOCKER_PRIVATE_KEY }}" > ~/.ssh/id
echo "${{ secrets.DOCKER_KNOWN_HOSTS }}" > ~/.ssh/known_hosts
- name: Download image archives (Forked repositories)
if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork
- name: Download image archives
# actions/download-artifact@v1
uses: actions/download-artifact@18f0f59
with:
name: image-archives
- name: Load cli-bin image into local docker images
if: github.event_name == 'push' || !github.event.pull_request.head.repo.fork
run: |
# `docker load` only accepts input from STDIN, so pipe the image
# archive into the command.
#
# In order to pipe the image archive, set `DOCKER_HOST` for a single
# command and `docker save` the CLI image from the Packet host.
DOCKER_HOST=ssh://linkerd-docker docker save "$DOCKER_REGISTRY/cli-bin:$TAG" | docker load
- name: Load cli-bin image into local docker images (Forked repositories)
if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork
run: docker load < image-archives/cli-bin.tar
- name: Install CLI
run: |
@ -162,10 +142,5 @@ jobs:
# Validate the CLI version matches the current build tag.
[[ "$TAG" == "$($HOME/.linkerd version --short --client)" ]]
- name: Run integration tests
if: github.event_name == 'push' || !github.event.pull_request.head.repo.fork
run: |
bin/tests --images --images-host ssh://linkerd-docker --name ${{ matrix.integration_test }} "$HOME/.linkerd"
- name: Run integration tests (Forked repositories)
if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork
run: |
bin/tests --images --name ${{ matrix.integration_test }} "$HOME/.linkerd"

173
.github/workflows/release.yml vendored
View File

@ -1,10 +1,6 @@
#
# This file is a mostly a concatenation of `kind_integration.yml` and
# `cloud_integration.yml`, specifically for release. Once GitHub Actions
# supports YAML anchors, we should be able to share most of the content
# between these files:
# https://github.community/t5/GitHub-Actions/Support-for-YAML-anchors/m-p/30336
#
# `cloud_integration.yml`, specifically for release.
name: Release
on:
@ -13,53 +9,14 @@ on:
- "*"
env:
GH_ANNOTATION: true
DOCKER_BUILDKIT: 1
jobs:
# todo: Keep in sync with `cloud_integration.yml`
docker_build:
name: Docker build
runs-on: ubuntu-18.04
steps:
- name: Checkout code
# actions/checkout@v2
uses: actions/checkout@722adc6
- name: Setup SSH config for Packet
run: |
mkdir -p ~/.ssh/
touch ~/.ssh/id && chmod 600 ~/.ssh/id
echo "${{ secrets.DOCKER_SSH_CONFIG }}" > ~/.ssh/config
echo "${{ secrets.DOCKER_PRIVATE_KEY }}" > ~/.ssh/id
echo "${{ secrets.DOCKER_KNOWN_HOSTS }}" > ~/.ssh/known_hosts
ssh linkerd-docker docker version
- name: Build docker images
env:
DOCKER_HOST: ssh://linkerd-docker
DOCKER_TRACE: 1
run: |
export PATH="`pwd`/bin:$PATH"
bin/docker-build
- name: Create artifact with Windows CLI binary
env:
ARCHIVES: /home/runner/archives
run: |
mkdir -p $ARCHIVES
# save windows cli into artifacts
cp -r ./target/cli/windows/linkerd $ARCHIVES/linkerd-windows.exe
# `with.path` values do not support environment variables yet, so an
# absolute path is used here.
#
# https://github.com/actions/upload-artifact/issues/8
- name: Upload artifact
# actions/upload-artifact@v1
uses: actions/upload-artifact@3446296
with:
name: image-archives
path: /home/runner/archives
# todo: Keep in sync with `cloud_integration.yml`
docker_push:
name: Docker push
runs-on: ubuntu-18.04
needs: [docker_build]
strategy:
matrix:
target: [proxy, controller, web, cni-plugin, debug, cli-bin, grafana]
name: Docker build (${{ matrix.target }})
steps:
- name: Checkout code
# actions/checkout@v2
@ -68,6 +25,48 @@ jobs:
run: |
. bin/_tag.sh
echo ::set-env name=TAG::$(CI_FORCE_CLEAN=1 bin/root-tag)
. bin/_docker.sh
echo ::set-env name=DOCKER_REGISTRY::$DOCKER_REGISTRY
echo ::set-env name=DOCKER_BUILDKIT_CACHE::${{ runner.temp }}/.buildx-cache
- name: Cache docker layers
# actions/cache@v2.0.0
uses: actions/cache@b820478
with:
path: ${{ env.DOCKER_BUILDKIT_CACHE }}
key: ${{ runner.os }}-buildx-${{ matrix.target }}-${{ env.TAG }}
restore-keys: |
${{ runner.os }}-buildx-${{ matrix.target }}-
- name: Build docker images
env:
DOCKER_TRACE: 1
run: |
docker buildx create --driver docker-container --use
bin/docker-build-${{ matrix.target }}
- name: Prune docker layers cache
# changes generate new images while the existing ones don't get removed
# so we manually do that to avoid bloating the cache
run: bin/docker-cache-prune
- name: Create artifact with CLI
# windows_static_cli_tests below needs this because it can't create linux containers
# inside windows
if: matrix.target == 'cli-bin'
env:
ARCHIVES: /home/runner/archives
run: |
mkdir -p $ARCHIVES
cp -r ./target/cli/windows/linkerd $ARCHIVES/linkerd-windows.exe
# `with.path` values do not support environment variables yet, so an
# absolute path is used here.
#
# https://github.com/actions/upload-artifact/issues/8
- name: Upload artifact
if: matrix.target == 'cli-bin'
# actions/upload-artifact@v1
uses: actions/upload-artifact@3446296
with:
name: image-archives
path: /home/runner/archives
- name: Configure gcloud
# linkerd/linkerd2-action-gcloud@v1.0.1
uses: linkerd/linkerd2-action-gcloud@308c4df
@ -75,23 +74,12 @@ jobs:
cloud_sdk_service_account_key: ${{ secrets.CLOUD_SDK_SERVICE_ACCOUNT_KEY }}
gcp_project: ${{ secrets.GCP_PROJECT }}
gcp_zone: ${{ secrets.GCP_ZONE }}
- name: Docker SSH setup
run: |
mkdir -p ~/.ssh/
touch ~/.ssh/id && chmod 600 ~/.ssh/id
echo "${{ secrets.DOCKER_SSH_CONFIG }}" > ~/.ssh/config
echo "${{ secrets.DOCKER_PRIVATE_KEY }}" > ~/.ssh/id
echo "${{ secrets.DOCKER_KNOWN_HOSTS }}" > ~/.ssh/known_hosts
ssh linkerd-docker docker version
- name: Push docker images to registry
env:
DOCKER_HOST: ssh://linkerd-docker
run: |
export PATH="`pwd`/bin:$PATH"
bin/docker-push-deps
bin/docker-push $TAG
bin/docker-retag-all $TAG main
bin/docker-push main
. bin/_docker.sh
docker_push "${{ matrix.target }}" "$TAG"
docker_retag "${{ matrix.target }}" "$TAG" main
docker_push "${{ matrix.target }}" main
# todo: Keep in sync with `kind_integration.yml`
windows_static_cli_tests:
name: Static CLI tests (windows)
@ -117,7 +105,6 @@ jobs:
- name: Run CLI Integration tests
run: |
go test --failfast --mod=readonly ".\test\cli" --linkerd=$PWD\image-archives\linkerd-windows.exe --cli-tests -v
# todo: Keep in sync with `kind_integration.yml`
kind_integration_tests:
strategy:
matrix:
@ -147,41 +134,21 @@ jobs:
${{ runner.os }}-go-
- name: Set environment variables from scripts
run: |
. bin/_tag.sh
echo ::set-env name=TAG::$(CI_FORCE_CLEAN=1 bin/root-tag)
. bin/_docker.sh
echo ::set-env name=DOCKER_REGISTRY::$DOCKER_REGISTRY
- name: Setup SSH config for Packet
run: |
mkdir -p ~/.ssh/
touch ~/.ssh/id && chmod 600 ~/.ssh/id
echo "${{ secrets.DOCKER_SSH_CONFIG }}" > ~/.ssh/config
echo "${{ secrets.DOCKER_PRIVATE_KEY }}" > ~/.ssh/id
echo "${{ secrets.DOCKER_KNOWN_HOSTS }}" > ~/.ssh/known_hosts
- name: Load cli-bin image into local docker images
run: |
# `docker load` only accepts input from STDIN, so pipe the image
# archive into the command.
#
# In order to pipe the image archive, set `DOCKER_HOST` for a single
# command and `docker save` the CLI image from the Packet host.
DOCKER_HOST=ssh://linkerd-docker docker save "$DOCKER_REGISTRY/cli-bin:$TAG" | docker load
- name: Install CLI
run: |
# Copy the CLI out of the local cli-bin container.
container_id=$(docker create "$DOCKER_REGISTRY/cli-bin:$TAG")
docker cp $container_id:/out/linkerd-linux $HOME/.linkerd
# Validate the CLI version matches the current build tag.
[[ "$TAG" == "$($HOME/.linkerd version --short --client)" ]]
TAG="$(CI_FORCE_CLEAN=1 bin/root-tag)"
CMD="$PWD/target/release/linkerd2-cli-$TAG-linux"
echo "::set-env name=CMD::$CMD"
echo "::set-env name=TAG::$TAG"
- name: Run integration tests
run: bin/tests --images --images-host ssh://linkerd-docker --name ${{ matrix.integration_test }} "$HOME/.linkerd"
run: |
bin/docker-pull-binaries $TAG
# Validate the CLI version matches the current build tag.
[[ "$TAG" == "$($CMD version --short --client)" ]]
bin/tests --images --name ${{ matrix.integration_test }} "$CMD"
# todo: Keep in sync with `cloud_integration.yml`
cloud_integration_tests:
name: Cloud integration tests
runs-on: ubuntu-18.04
needs: [docker_push]
needs: [docker_build]
steps:
- name: Checkout code
# actions/checkout@v2
@ -198,13 +165,13 @@ jobs:
id: install_cli
run: |
TAG="$(CI_FORCE_CLEAN=1 bin/root-tag)"
image="gcr.io/linkerd-io/cli-bin:$TAG"
id=$(bin/docker create $image)
bin/docker cp "$id:/out/linkerd-linux" "$HOME/.linkerd"
$HOME/.linkerd version --client
CMD="$PWD/target/release/linkerd2-cli-$TAG-linux"
bin/docker-pull-binaries $TAG
$CMD version --client
# validate CLI version matches the repo
[[ "$TAG" == "$($HOME/.linkerd version --short --client)" ]]
[[ "$TAG" == "$($CMD version --short --client)" ]]
echo "Installed Linkerd CLI version: $TAG"
echo "::set-env name=CMD::$CMD"
echo "::set-output name=tag::$TAG"
- name: Create GKE cluster
# linkerd/linkerd2-action-gcloud@v1.0.1
@ -221,13 +188,11 @@ jobs:
env:
GITCOOKIE_SH: ${{ secrets.GITCOOKIE_SH }}
run: |
export PATH="`pwd`/bin:$PATH"
echo "$GITCOOKIE_SH" | bash
version="$($HOME/.linkerd version --client --short | tr -cd '[:alnum:]-')"
bin/tests --skip-kind-create "$HOME/.linkerd"
bin/tests --skip-kind-create "$CMD"
- name: CNI tests
run: |
export TAG="$($HOME/.linkerd version --client --short)"
export TAG="$($CMD version --client --short)"
go test -cover -race -v -mod=readonly ./cni-plugin/test -integration-tests
choco_pack:
# only runs for stable tags. The conditionals are at each step level instead of the job level

23
.github/workflows/static_checks.yml vendored
View File

@ -8,29 +8,6 @@ on:
branches:
- main
jobs:
go_dependencies:
name: Go dependencies
runs-on: ubuntu-18.04
steps:
- name: Checkout code
# actions/checkout@v2
uses: actions/checkout@722adc6
- name: Dump env
run: env | sort
- name: Dump GitHub context
env:
GITHUB_CONTEXT: ${{ toJson(github) }}
run: echo "$GITHUB_CONTEXT"
- name: Dump job context
env:
JOB_CONTEXT: ${{ toJson(job) }}
run: echo "$JOB_CONTEXT"
- name: Validate go deps
run: |
. bin/_tag.sh
for f in $( grep -lR --include=Dockerfile\* go-deps: . ) ; do
validate_go_deps_tag $f
done
go_lint:
name: Go lint
runs-on: ubuntu-18.04

23
BUILD.md
View File

@ -19,7 +19,6 @@ about testing from source can be found in the [TEST.md](TEST.md) guide.
- [Rust](#rust)
- [Dependencies](#dependencies)
- [Updating protobuf dependencies](#updating-protobuf-dependencies)
- [Updating Docker dependencies](#updating-docker-dependencies)
- [Updating ServiceProfile generated
code](#updating-serviceprofile-generated-code)
- [Helm Chart](#helm-chart)
@ -365,22 +364,6 @@ DOCKER_TRACE=1 bin/docker-build-proxy
bin/protoc-go.sh
```
### Updating Docker dependencies
The Go Docker images rely on base dependency images with hard-coded SHA's:
`gcr.io/linkerd-io/go-deps` depends on
- [`go.mod`](go.mod)
- [`Dockerfile-go-deps`](Dockerfile-go-deps)
When Go dependencies change, run the following:
```bash
go mod tidy
bin/update-go-deps-shas
```
### Updating ServiceProfile generated code
The [ServiceProfile client code](./controller/gen/client) is generated by
@ -541,12 +524,6 @@ build_architecture
"workflow.yml" -> "docker-retag-all";
"workflow.yml" -> "lint";
"update-go-deps-shas" -> "_tag.sh";
"update-go-deps-shas" -> "cli/Dockerfile-bin";
"update-go-deps-shas" -> "controller/Dockerfile";
"update-go-deps-shas" -> "grafana/Dockerfile";
"update-go-deps-shas" -> "web/Dockerfile";
"web" -> "go-run";
}
build_architecture

10
Dockerfile-proxy
View File

@ -1,5 +1,13 @@
ARG RUNTIME_IMAGE=debian:buster-20200514-slim
# Precompile key slow-to-build dependencies
FROM golang:1.14.2-alpine as go-deps
WORKDIR /linkerd-build
COPY go.mod go.sum ./
COPY bin/install-deps bin/
RUN go mod download
RUN ./bin/install-deps
FROM debian:buster-20200514-slim as fetch
RUN apt-get update && apt-get install -y ca-certificates curl jq
WORKDIR /build
@ -9,7 +17,7 @@ RUN (proxy=$(bin/fetch-proxy $(cat proxy-version)) && \
mv "$proxy" linkerd2-proxy)
## compile proxy-identity agent
FROM gcr.io/linkerd-io/go-deps:61149d15 as golang
FROM go-deps as golang
WORKDIR /linkerd-build
COPY pkg/flags pkg/flags
COPY pkg/tls pkg/tls

33
bin/_docker.sh
View File

@ -14,6 +14,12 @@ export DOCKER_REGISTRY=${DOCKER_REGISTRY:-gcr.io/linkerd-io}
# When set, causes docker's build output to be emitted to stderr.
export DOCKER_TRACE=${DOCKER_TRACE:-}
# When set, use `docker buildx` and use the github actions cache to store/retrieve images
export DOCKER_BUILDKIT=${DOCKER_BUILDKIT:-}
# buildx cache directory. Needed if DOCKER_BUILDKIT is used
export DOCKER_BUILDKIT_CACHE=${DOCKER_BUILDKIT_CACHE:-}
docker_repo() {
repo=$1
@ -42,12 +48,27 @@ docker_build() {
rootdir=$( cd "$bindir"/.. && pwd )
log_debug " :; docker build $rootdir -t $repo:$tag -f $file $*"
docker build "$rootdir" \
-t "$repo:$tag" \
-f "$file" \
"$@" \
> "$output"
if [ -n "$DOCKER_BUILDKIT" ]; then
cache_params=""
if [ -n "$DOCKER_BUILDKIT_CACHE" ]; then
cache_params="--cache-from type=local,src=${DOCKER_BUILDKIT_CACHE} --cache-to type=local,dest=${DOCKER_BUILDKIT_CACHE},mode=max"
fi
log_debug " :; docker buildx $rootdir $cache_params --load -t $repo:$tag -f $file $*"
# shellcheck disable=SC2086
docker buildx build "$rootdir" $cache_params \
--load \
-t "$repo:$tag" \
-f "$file" \
"$@" \
> "$output"
else
log_debug " :; docker build $rootdir -t $repo:$tag -f $file $*"
docker build "$rootdir" \
-t "$repo:$tag" \
-f "$file" \
"$@" \
> "$output"
fi
echo "$repo:$tag"
}

36
bin/_tag.sh
View File

@ -6,12 +6,6 @@ git_sha_head() {
git rev-parse --short=8 HEAD
}
go_deps_sha() {
bindir=$( cd "${BASH_SOURCE[0]%/*}" && pwd )
rootdir=$( cd "$bindir"/.. && pwd )
cat "$rootdir/go.mod" "$rootdir/Dockerfile-go-deps" | shasum - | awk '{print $1}' |cut -c 1-8
}
clean_head() {
[ -n "${CI_FORCE_CLEAN:-}" ] || git diff-index --quiet HEAD --
}
@ -43,33 +37,3 @@ clean_head_root_tag() {
exit 3
fi
}
validate_tag() {
file=$1
shift
image=$1
shift
sha=$1
shift
dockerfile_tag=$(grep -oe "$image"':[^ ]*' "$file") || true
deps_tag="$image:$sha"
if [ -n "$dockerfile_tag" ] && [ "$dockerfile_tag" != "$deps_tag" ]; then
echo "Tag in \"$file\" does not match source tree:
$dockerfile_tag (\"$file\")
$deps_tag (source)"
return 3
fi
}
# This function should be called by any docker-build-* script that relies on Go
# dependencies. To confirm the set of scripts that should call this function,
# run:
# $ grep -ER 'docker-build-go-deps' .
validate_go_deps_tag() {
file=$1
validate_tag "$file" gcr.io/linkerd-io/go-deps "$(go_deps_sha)"
}

7
bin/docker-build-cli-bin
View File

@ -16,13 +16,6 @@ rootdir=$( cd "$bindir"/.. && pwd )
. "$bindir"/_tag.sh
dockerfile=$rootdir/cli/Dockerfile-bin
validate_go_deps_tag "$dockerfile"
(
"$bindir"/docker-build-go-deps
) >/dev/null
tag=$(head_root_tag)
docker_build cli-bin "$tag" "$dockerfile" --build-arg LINKERD_VERSION="$tag"
IMG=$(docker_repo cli-bin):$tag

3
bin/docker-build-cni-plugin
View File

@ -17,11 +17,8 @@ rootdir=$( cd "$bindir"/.. && pwd )
dockerfile=$rootdir/cni-plugin/Dockerfile
validate_go_deps_tag "$dockerfile"
(
"$bindir"/docker-build-base
"$bindir"/docker-build-go-deps
) >/dev/null
docker_build cni-plugin "$(head_root_tag)" "$dockerfile"

7
bin/docker-build-controller
View File

@ -16,12 +16,5 @@ rootdir=$( cd "$bindir"/.. && pwd )
. "$bindir"/_tag.sh
dockerfile=$rootdir/controller/Dockerfile
validate_go_deps_tag "$dockerfile"
(
"$bindir"/docker-build-go-deps
) >/dev/null
tag=$(head_root_tag)
docker_build controller "$tag" "$dockerfile" --build-arg LINKERD_VERSION="$tag"

26
bin/docker-build-go-deps
View File

@ -1,26 +0,0 @@
#!/usr/bin/env bash
# Builds (or pulls) our go-deps docker image.
set -eu
if [ $# -ne 0 ]; then
echo "no arguments allowed for ${0##*/}, given: $*" >&2
exit 64
fi
bindir=$( cd "${BASH_SOURCE[0]%/*}" && pwd )
# shellcheck source=_docker.sh
. "$bindir"/_docker.sh
# shellcheck source=_tag.sh
. "$bindir"/_tag.sh
tag=$(go_deps_sha)
if (docker_pull go-deps "$tag"); then
echo "$(docker_repo go-deps):$tag"
else
rootdir=$( cd "$bindir"/.. && pwd )
docker_build go-deps "$tag" "$rootdir/Dockerfile-go-deps"
fi

6
bin/docker-build-proxy
View File

@ -17,12 +17,6 @@ rootdir=$( cd "$bindir"/.. && pwd )
dockerfile=$rootdir/Dockerfile-proxy
validate_go_deps_tag "$dockerfile"
(
"$bindir"/docker-build-go-deps
) >/dev/null
get_extra_options() {
options=
for var in http_proxy https_proxy no_proxy; do

7
bin/docker-build-web
View File

@ -16,12 +16,5 @@ rootdir=$( cd "$bindir"/.. && pwd )
. "$bindir"/_tag.sh
dockerfile=$rootdir/web/Dockerfile
validate_go_deps_tag "$dockerfile"
(
"$bindir"/docker-build-go-deps
) >/dev/null
tag=$(head_root_tag)
docker_build web "$tag" "$dockerfile" --build-arg LINKERD_VERSION="$tag"

21
bin/docker-cache-prune Executable file
View File

@ -0,0 +1,21 @@
#!/usr/bin/env bash
# This script deletes all the files under the buildkit cache blob directory that
# are not referred to in the cache manifest file.
set -eu
export DOCKER_BUILDKIT_CACHE=${DOCKER_BUILDKIT_CACHE:-}
manifest_sha=$(jq -r .manifests[0].digest < "${DOCKER_BUILDKIT_CACHE}/index.json")
manifest=${manifest_sha#"sha256:"}
files=("$manifest")
while IFS= read -r line; do files+=("$line"); done <<< "$(jq -r '.manifests[].digest | sub("^sha256:"; "")' < "${DOCKER_BUILDKIT_CACHE}/blobs/sha256/$manifest")"
for file in "${DOCKER_BUILDKIT_CACHE}"/blobs/sha256/*; do
name=$(basename "$file")
# shellcheck disable=SC2199
if [[ ! "${files[@]}" =~ ${name} ]]; then
printf 'pruned from cache: %s\n' "$file"
rm -f "$file"
fi
done

2
bin/docker-images
View File

@ -21,5 +21,3 @@ tag=$(head_root_tag)
for img in cli-bin cni-plugin controller debug grafana proxy web ; do
docker_image $img "$tag"
done
docker_image go-deps "$(go_deps_sha)"

3
bin/docker-pull-deps → bin/docker-pull-base
View File

@ -6,8 +6,5 @@ bindir=$( cd "${BASH_SOURCE[0]%/*}" && pwd )
# shellcheck source=_docker.sh
. "$bindir"/_docker.sh
# shellcheck source=_tag.sh
. "$bindir"/_tag.sh
docker_pull base 2020-06-08.01 || true
docker_pull go-deps "$(go_deps_sha)" || true

3
bin/docker-push-deps → bin/docker-push-base
View File

@ -6,8 +6,5 @@ bindir=$( cd "${BASH_SOURCE[0]%/*}" && pwd )
# shellcheck source=_docker.sh
. "$bindir"/_docker.sh
# shellcheck source=_tag.sh
. "$bindir"/_tag.sh
docker_push base 2020-06-08.01
docker_push go-deps "$(go_deps_sha)"

26
Dockerfile-go-deps → bin/install-deps Normal file → Executable file
View File

@ -1,25 +1,15 @@
# Go dependencies
#
# Fetches all required Go dependencies. All Linkerd sources are omitted from the
# resulting image so that artifacts may be built from source over this image.
#
# When this file is changed, run `bin/update-go-deps-shas`.
#!/usr/bin/env sh
FROM golang:1.14.2
# This script is used in the multiple Dockerfiles for caching
# some of the slow-to-build go dependencies.
WORKDIR /linkerd-build
set -eu
COPY go.mod go.sum ./
bindir=$( cd "${0%/*}" && pwd )
rootdir=$( cd "$bindir"/.. && pwd )
cd "$rootdir"
RUN go mod download
# Precompile key slow-to-build dependencies. This list doesn't need to be
# complete for the build to work correctly; the completeness of this list
# only affects the speed of incremental rebuilds of Dependent Dockerfiles.
#
# This list was derived from the output of `find /go/pkg -type f`
# after building the controller.
RUN CGO_ENABLED=0 GOOS=linux go install -mod=readonly \
CGO_ENABLED=0 GOOS=linux go install -mod=readonly \
github.com/golang/protobuf/jsonpb \
github.com/grpc-ecosystem/go-grpc-prometheus \
github.com/prometheus/client_golang/api \

13
bin/update-go-deps-shas
View File

@ -1,13 +0,0 @@
#!/usr/bin/env bash
set -eu
# Updates the tag for `linkerd-io/go-deps` across all Dockerfiles in this repository.
# shellcheck source=_tag.sh
sha=$(. "${0%/*}"/_tag.sh ; go_deps_sha)
for f in $( grep -lR --include=Dockerfile\* go-deps: "$(cd "${0%/*}"/.. && pwd)" | xargs ) ; do
sed -E -i.bak -e "s|linkerd-io/go-deps:[^ ]+|linkerd-io/go-deps:$sha|" "$f"
rm "$f".bak
done

10
cli/Dockerfile-bin
View File

@ -1,5 +1,13 @@
# Precompile key slow-to-build dependencies
FROM golang:1.14.2-alpine as go-deps
WORKDIR /linkerd-build
COPY go.mod go.sum ./
COPY bin/install-deps bin/
RUN go mod download
RUN ./bin/install-deps
## compile binaries
FROM gcr.io/linkerd-io/go-deps:61149d15 as golang
FROM go-deps as golang
WORKDIR /linkerd-build
COPY cli cli
COPY charts charts

10
cni-plugin/Dockerfile
View File

@ -1,5 +1,13 @@
# Precompile key slow-to-build dependencies
FROM golang:1.14.2-alpine as go-deps
WORKDIR /linkerd-build
COPY go.mod go.sum ./
COPY bin/install-deps bin/
RUN go mod download
RUN ./bin/install-deps
## compile cni-plugin utility
FROM gcr.io/linkerd-io/go-deps:61149d15 as golang
FROM go-deps as golang
WORKDIR /linkerd-build
COPY pkg pkg
COPY controller controller

10
controller/Dockerfile
View File

@ -1,5 +1,13 @@
# Precompile key slow-to-build dependencies
FROM golang:1.14.2-alpine as go-deps
WORKDIR /linkerd-build
COPY go.mod go.sum ./
COPY bin/install-deps bin/
RUN go mod download
RUN ./bin/install-deps
## compile controller service
FROM gcr.io/linkerd-io/go-deps:61149d15 as golang
FROM go-deps as golang
WORKDIR /linkerd-build
COPY controller/gen controller/gen
COPY pkg pkg

10
web/Dockerfile
View File

@ -1,3 +1,11 @@
# Precompile key slow-to-build dependencies
FROM golang:1.14.2-alpine as go-deps
WORKDIR /linkerd-build
COPY go.mod go.sum ./
COPY bin/install-deps bin/
RUN go mod download
RUN ./bin/install-deps
## bundle web assets
FROM node:14-buster as webpack-bundle
RUN curl -o- -L https://yarnpkg.com/install.sh | bash -s -- --version 1.21.1 --network-concurrency 1
@ -21,7 +29,7 @@ COPY web/app ./web/app
RUN ./bin/web build
## compile go server
FROM gcr.io/linkerd-io/go-deps:61149d15 as golang
FROM go-deps as golang
WORKDIR /linkerd-build
RUN mkdir -p web
COPY web/main.go web