* Allow CI to run concurrent builds in master
Fixes#3911
Refactors the `cloud_integration` test to run in separate GKE clusters
that are created and torn down on the fly.
It leverages a new "gcloud" github action that is also used to set up
gcloud in other build steps (`docker_deploy` and `chart_deploy`).
The action also generates unique names for those clusters, based on the
git commit SHA and `run_id`, a recently introduced variable that is
unique per CI run and available to all the jobs.
This fixes part of #3635 in that CI runs on the same SHA don't interfere
with one another (in the `cloud_integration` test; still to do for
`kind_integration`).
The "gcloud" GH action is supported by `.github/actions/gcloud/index.js`
that has a couple of dependencies. To avoid having to commit
`node_modules`, after every change to that file one must run
```bash
# only needed the first time
npm i -g @zeit/ncc
cd .github/actions/gcloud
ncc build index.js
```
which generates the self-contained file
`.github/actions/gcloud/dist/index.js`.
(This last part might get easier in the future after other refactorings
outside this PR).
* Run integration tests for forked repos
Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
* Address reviews
Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
* Address more reviews
Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
* Move some conditionals to jobs
Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
* Change job name
Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
* Move more conditionals to job level
Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
* Added more flags to 'gcloud container clusters create' and consolidated
'create' and 'destroy' into ' action'
* Run kind cleanup only for non-forked PRs
Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
* Got rid of cloud_cleanup by using a post hook in the gcloud action
* Removed cluster naming responsibility from the gcloud action
* Consolidate .gitignore statements
* Removed bin/_gcp.sh
* Change name of Kind int. test job
Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
* Ensure `kind_cleanup` still runs on cancelled host CI runs
Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
* Add reviews
Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
* Update workflow comment
Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
* Split index.js into setup.js and destroy.js
* trigger build
* Moved the gcloud action into its own repo
* Full version for the gcloud GH action
* Rebase back to master
Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
* Remvoe additional changes
Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
* Remove additional changes
Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
* Trigger CI
Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
Co-authored-by: Alejandro Pedraza <alejandro.pedraza@gmail.com>
In light of the breaking changes we are introducing to the Helm chart and the convoluted upgrade process (see linkerd/website#647) an integration test can be quite helpful. This simply installs latest stable through helm install and then upgrades to the current head of the branch.
Signed-off-by: Zahari Dichev zaharidichev@gmail.com
## edge-20.1.3
* CLI
* Introduced `linkerd check --pre --linkerd-cni-enabled`, used when the CNI
plugin is used, to check it has been properly installed before proceeding
with the control plane installation
* Added support for the `--as-group` flag so that users can impersonate
groups for Kubernetes operations (thanks @mayankshah160!)
* Controller
* Fixed an issue where an override of the Docker registry was not being
applied to debug containers (thanks @javaducky!)
* Added check for the Subject Alternate Name attributes to the API server
when access restrictions have been enabled (thanks @javaducky!)
* Added support for arbitrary pod labels so that users can leverage the
Linkerd provided Prometheus instance to scrape for their own labels
(thanks @daxmc99!)
* Fixed an issue with CNI config parsing
Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
* Revert "Replace actions/checkout with actions/upload/download (#3602)"
This reverts commit 397970e917.
* Upgraded actions/checkout to @v2
Reverts #3602 and Fixes#3881
* Use `kind export kubeconfig` instead of `scp`
Followup to #3864
[comment](https://github.com/linkerd/linkerd2/pull/3864#discussion_r360976473)
Stop moving the kubeconfig file between the Github Action env and the
build server with `scp` and use instead `kind export kubeconfig`.
* Replaced deprecated '--loglevel debug' flag with '--verbosity 3'
* Upgrade `kind` to v0.6.1
Fixes#3852
Upgraded `/bin/kind` to pull v0.6.1.
Also have `workflow.yml` use `KUBECONFIG` explicitly for setting the
location of the config file, now that `kind get kubeconfig-path` has
been deprecated (check
https://github.com/kubernetes-sigs/kind/releases/tag/v0.6.0 for detailed
info).
Note that in the build server the kind binary for this version is
`kind-0.6.1`, leaving the `kind` binary still pointing to v0.5.1 while
this gets merged and all the PR branches get this.
* Enable mixed configuration of skip-[inbound|outbound]-ports using port numbers and ranges (#3752)
* included tests for generated output given proxy-ignore configuration options
* renamed "validate" method to "parseAndValidate" given mutation
* updated documentation to denote inclusiveness of ranges
* Updates for expansion of ignored inbound and outbound port ranges to be handled by the proxy-init rather than CLI (#3766)
This change maintains the configured ports and ranges as strings rather than unsigned integers, while still providing validation at the command layer.
* Bump versions for proxy-init to v1.3.0
Signed-off-by: Paul Balogh <javaducky@gmail.com>
This PR addresses recent JS unit test failures on CI by:
* Upgrading yarn from 1.7.0 to 1.21.1 (current stable version) in the Dockerfile
and Github Actions workflow
* Wrapping the yarn installation with the --network-concurrency 1 flag, setting the
maximum number of concurrent network requests to 1, suggested as a fix here:
https://github.com/yarnpkg/yarn/issues/2629
Follow up from #3730, GitHub Actions was failing in the following ways:
- booting 5 parallel kind clusters in the matrix job was overloading the
Docker host
- `~/.ssh/known_hosts` file was empty following gcloud sdk installation
Four changes to mitigate these issues:
- generate `known_hosts` from a secret, rather than an `ssh-keyscan`
command
- generate `~/.ssh/config` from a secret
- set `max-parallel: 3` on the matrix job
- install gcloud sdk prior to Docker ssh setup
TODO:
- remove `DOCKER_ADDRESS`, `DOCKER_HOST_PRIVATE_KEY`
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
Fixes#3566
As explained in #3566, as of go 1.13 there's a strict check that ensures a dependency's timestamp matches it's sha (as declared in go.mod). Our smi-sdk dependency has a problem with that that got resolved later on, but more work would be required to upgrade that dependency. In the meantime a quick pair of replace statements at the bottom of go.mod fix the issue.
* Keep old releases in Helm repo index
When building the Helm repo index file, keep the references to the old
releases. Also rename and keep the old index file in case
something goes wrong when generating the new one.
Fixes#3561
CI currently enforcing formatting rules by using the fmt linter of golang-ci-lint which is invoked from the bin/lint script. However it doesn't seem possible to use golang-ci-lint as a formatter, only as a linter which checks formatting. This means any formatter used by your IDE or invoked manually may or may not use the same formatting rules as golang-ci-lint depending on which formatter you use and which specific revision of that formatter you use.
In this change we stop using golang-ci-lint for format checking. We introduce `tools.go` and add goimports to the `go.mod` and `go.sum` files. This allows everyone to easily get the same revision of goimports by running `go install -mod=readonly golang.org/x/tools/cmd/goimports` from inside of the project. We add a step in the CI workflow that uses goimports via the `bin/fmt` script to check formatting.
Some shell gymnastics were required in the `bin/fmt` script to work around some limitations of `goimports`:
* goimports does not have a built-in mechanism for excluding directories, and we need to exclude the vendor director as well as the generated Go sources
* goimports returns a 0 exit code, even when formatting errors are detected
Signed-off-by: Alex Leong <alex@buoyant.io>
* Have CI push the Helm artifacts into GCS
- Added missing OWNERS and README files
- Added maintainers section to Chart.yaml
- Changed NOTES.txt so it points to the installation of the CLI
- Set the proxy-init version to v1.1.0 in values.yaml
Ref #3256
Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
The repo depended on an old version of client-go. It also depended on
stern, which itself depended on an old version of client-go, making
client-go upgrade non-trivial.
Update the repo to client-go v12.0.0, and also replace stern with a
fork.
This fork of stern includes the following changes:
- updated to use Go Modules
- updated to use client-go v12.0.0
- fixed log line interleaving:
- https://github.com/wercker/stern/issues/96
- based on:
- 8723308e46Fixes#3382
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
The kind clusters booted by the integration tests each had to pull
Prometheus and proxy-init images from the internet during linkerd
install.
Preemptively pull the images from the internet once, then execute `kind
load` commands for each of the clusters prior to starting integration
tests.
Depends on #3397
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
GitHub Actions has been running unit and integration tests, in parallel
with Travis running those same tests, and also handling master merges
and tags.
This change completes the transtion to GitHub Actions, removing all
references to Travis. Similar to Travis, GitHub Actions now acts on
master merges and tag pushes by pushing Docker images to gcr.io, and
running integration tests against a GKE cluster.
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
GitHub Action secrets are intentionally not available to forked PRs.
This causes the integration tests that require those secrets to fail.
Modify GitHub Actions such that they only run for non-forked PRs.
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
The existing Travis CI setup requires additional integrations and
permissions with Github, and also lacks some flexibility around job
dependency management.
Introduce a new CI workflow based on Github Actions. This initial
workflow performs the same CI work that Travis does, and will iniitially
run in parallel:
- Go unit tests
- JS unit tests
- Go lint
- Validate Go deps
- Integration tests (deep, upgrade, helm)
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
GitHub's community guidelines recommend a pull request template, the repo was
lacking one.
Introduce a `PULL_REQUEST_TEMPLATE.md` file.
Once merged, the
[Community profile checklist](https://github.com/linkerd/linkerd2/community)
should indicate the repo now provides a pull request template.
Fixes#3321
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
The `area/test` issue label contains a lot of tickets decribing areas of
the codebase that need more tests, or flaky test behaviours. These
issues are often fairly low-priority, so they usually aren't closed
right away, but rarely become invalid. In my opinion, it's good for
these issues to remain open so that those test deficiencies are clearly
documented.
This branch adds the `area/test` label to the list of labels ignored by
stalebot, so that they aren't closed after being left inactive.
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
Kevin Leimkuhler
Zahari Dichev
Alejandro Pedraza
Paul Balogh
Carol A. Scott
Andrew Seigner
Alex Leong
Andrew Seigner
Eliza Weisman
Kevin Lingerfelt
Thomas Rampelberg
Adam Christian
Oliver Gould