* feat: add log format annotation and helm value
Json log formatting has been added via https://github.com/linkerd/linkerd2-proxy/pull/500
but wiring the option through as an annotation/helm value is still
necessary.
This PR adds the annotation and helm value to configure log format.
Closes#2491
Signed-off-by: Naseem <naseem@transit.app>
Currently linkerd check appears to hang on HA installations where there are pods that are unscheduable. In reality it is just wating on a condition that might never become true without showing any useful information (i.e. which pods are not scheduled). This change adds sets the `surfaceErrorOnRetry: true` so the user gets feedback wrt to what conditions are not met yet instead of simply being shown waiting for check to complete.
Fix#4680
Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
Currently commands that need access to the public api are executing the `LinkerdControlPlaneExistenceChecks` This set of checks includes one that specifically checks that there is no unscheduable pods. In fact in order to run commands like stat and edge we do not need to meet that requirement.
This change relaxes all this by makind the no unschedulable pods a warning only check. Fixes#3940
Signed-off-by: Zahari Dichev zaharidichev@gmail.com
* Refactor install test helpers
- Move testResourcesPostInstall to testutil.TestResourcesPostInstall
- Move exerciseTestAppEndpoint to testutil.ExerciseTestAppEndpoint
Signed-off-by: Mayank Shah <mayankshah1614@gmail.com>
* Trigger CI
Signed-off-by: Mayank Shah <mayankshah1614@gmail.com>
Data disappears upon prometheus restarts due to it being all in-memory.
Adding an option to enabled persistence by means of a PVC would be the right approach. It is commonly seen in a wide array of helm charts.
Fixes#4576
Signed-off-by: Naseem <naseem@transit.app>
- match messaging w/website
- replace specific K8s versions with "modern" (future-proofing)
- Copyright 2019 -> 2020
- Minor tweaks
Signed-off-by: William Morgan <william@buoyant.io>
Regenerated protobuf files, using version 1.4.2 that was upgraded from
1.3.2 with the proxy-api update in #4614.
As of v1.4 protobuf messages are disallowed to be copied (because they
hold a mutex), so whenever a message is passed to or returned from a
function we need to use a pointer.
This affects _mostly_ test files.
This is required to unblock #4620 which is adding a field to the config
protobuf.
This edge release moves the proxy onto a new version of the Tokio runtime. This
allows us to more easily integrate with the ecosystem and may yield performance
benefits as well.
* Upgraded the proxy's underlying Tokio runtime and its related libraries
* Added support for PKCS8 formatted ECDSA private keys
* Added support for Helm configuration of per-component proxy resources requests
and limits (thanks @cypherfox!)
* Updated the `linkerd inject` command to throw an error while injecting
non-compliant pods (thanks @mayankshah1607)
Signed-off-by: Alex Leong <alex@buoyant.io>
This release fixes a regression that could cause service profile lookups
to be retried indefinitely, despite the server returning an
`InvalidArgument` response (which indicates the proxy should not retry).
---
* fix InvalidProfileAddr not converting into DiscoveryRejected (linkerd/linkerd2-proxy#581)
## Description
As discussed [here](https://github.com/linkerd/linkerd2/pull/4653#discussion_r445543061), the `kind_integration` job of the release workflow was not kept in sync with the changes made in #4593.
Until GitHub actions can reuse yaml for separate workflows, these sections are supposed to be kept in sync.
This would be an issue if we had tried doing a release since #4593 merged, but that has not happened yet.
## Changes
This updates the release workflow `kind_integration` job to use the use new test interface, mainly removing cluster creation and image loading as necessary prerequisites.
Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
## Summary
Change the default behavior of integration tests to be isolated by cluster.
Additionally, make running one or all tests easier than the current process.
These changes are explained more in the [Testing
RFC](https://github.com/linkerd/rfc/blob/master/design/0004-isolated-integration-tests.md)
## Changes
This is a script used only by Linkerd developers, but there is a lot of useful
usage examples and explanations in `bin/tests --help` output:
```
Run Linkerd integration tests.
Optionally specify one of the following tests: [upgrade helm helm-upgrade uninstall deep external-issuer]
Usage:
tests [--images] [--images-host ssh://linkerd-docker] [--name test-name] [--skip-kind-create] /path/to/linkerd
Examples:
# Run all tests in isolated clusters
tests /path/to/linkerd
# Run single test in isolated clusters
tests --name test-name /path/to/linkerd
# Skip KinD cluster creation and run all tests in default cluster context
tests --skip-kind-create /path/to/linkerd
# Load images from tar files located under the 'image-archives' directory
# Note: This is primarly for CI
tests --images /path/to/linkerd
# Retrieve images from a remote docker instance and then load them into KinD
# Note: This is primarly for CI
tests --images --images-host ssh://linkerd-docker /path/to/linkerd
Available Commands:
--name: the argument to this option is the specific test to run
--skip-kind-create: skip KinD cluster creation step and run tests in an existing cluster.
--images: (Primarily for CI) use 'kind load image-archive' to load the images from local .tar files in the current directory.
--images-host: (Primarily for CI) the argument to this option is used as the remote docker instance from which images are first retrieved (using 'docker save') to be then loaded into KinD. This command requires --images.
```
### Run all tests
Old:
```bash
bin/test-run $PWD/bin/linkerd
```
New:
```bash
bin/tests $PWD/bin/linkerd
```
### Run single test (upgrade for example):
Current:
```bash
. bin/_test-run.sh
init_test_run $PWD/bin/linkerd
upgrade_integration_tests
```
New:
```bash
bin/tests --name upgrade $PWD/bin/linkerd
```
### Run tests in isolated KinD clusters
Current: Not possible without running single tests in newly created clusters
manually
New:
```bash
bin/tests $PWD/bin/linkerd
```
### Run tests in isolated namespaces on an existing cluster
Old:
```bash
bin/test-run $PWD/bin/linkerd
```
New:
```bash
bin/tests --skip-kind-create $PWD/bin/linkerd
```
## CI
`kind_integration` has been updated so that it does not create a KinD cluster as
part of its test setup.
`cloud_integration` passes the `--skip-kind-create` flag so that the tests are
run serially in a non-KinD cluster.
Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
* Update inject to error out on failure
Update injection process to throw an error when the reason for failure is due to sidecar, udp, automountServiceAccountToken or hostNetwork
Signed-off-by: Mayank Shah <mayankshah1614@gmail.com>
I should review all changes to the top-level project documents.
CODEOWNERS is misconfigured, however, so that I am required to review
changes to all files named README.md, which isn't intended.
This change ensures that my review is only required on these files in
the root of the repository.
This PR adds multicluster components to the integration tests.
The existing tests have been modified to pass the `--multicluster` flag so that the entire integration test suite runs with multicluster components.
Currently, the upgrade tests do not have multicluster components installed, but this will be done in a follow-up PR.
Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
Tools like cert-manager might encode private keys in PKCS8 format instead of PKCS1
in which case linkerd would fail as it cannot parse PKCS8 encoded private keys.
With this commit support for parsing PKCS8 encoded private keys is added to linkerd,
allowing it to read ECDSA and RSA keys encoded in PKCS8.
Unit tests have been added to test the private key parsing.
This commit addresses https://github.com/jetstack/cert-manager/issues/2942.
Signed-off-by: Alexander Berger <alex.berger@nexxiot.com>
Signed-off-by: alex.berger@nexiot.ch <alex.berger@nexiot.ch>
Co-authored-by: alex.berger@nexiot.ch <alex.berger@nexiot.ch>
The `bin/go-run` script generates a temporary binary, stored in the root
of the repository.
This change moves it into `target/` so that is included in the
.dockerignore, and so that the repo can be cleaned easily by removing
the `target/` directory.
The /cni-plugin directory has additional review requirements; however,
its Dockerfile changes each time `go.mod` is updated. It was not
intended to require this extra review on these routine changes.
This change updates CODEOWNERS to make all maintainers owners of
`cni-plugin/Dockerfile`.
* Release notes for stable-2.8.1
This release fixes multicluster gateways support on EKS.
* The multicluster service-mirror has been extended to resolve DNS names for
target clusters when an IP address is not known.
* Linkerd checks could fail when run from the dashboard. Thanks to @alex-berger
for providing a fix!
* Have the service mirror controller check in `linkerd check` retry on failures.
* As of this version we're including a Chocolatey package (Windows) next to the
other binaries in the release assets in GitHub.
* Base images have been updated:
* debian:buster-20200514-slim
* grafana/grafana:7.0.3
* The shell scripts under `bin` continued to be improved, thanks to @joakimr-axis!
This PR makes the service mirror controller is running retry on failure. This brings the check in line with the rest of the checks that verify that certain Linkerd components are running. It is especially useful in integration tests when we want to wait for the service mirror component to be initialized for a certain amount of time before we simply fail the linkerd check command
Fix#4642
Signed-off-by: Zahari Dichev zaharidichev@gmail.com
* Post-2.8.0 integration test cleanup
We had some code for testing upgrades from pre-2.8.0 stables that took
care of creating the non-existent `linkerd-smi-metrics` SA, which is no
longer necessary.
I also had missed many spots in test/install_test.go from #4623
Using `/bin/env` increases portability for the shell scripts (and often using `/bin/env` is requested by e.g. Mac users). This would also facilitate testing scripts with different Bash versions via the Bash containers, as they have bash in `/usr/local` and not `/bin`. Using `/bin/env`, there is no need to change the script when testing. (I assume the latter was behind c301ea214b (diff-ecec5e3a811f60bc2739019004fa35b0), which would not happen using `/bin/env`.)
Signed-off-by: Joakim Roubert <joakimr@axis.com>
In #4595 we stopped failing integration tests whenever a pod restarted
just once, which is being caused by containerd/containerd#4068.
But we forgot to remove the warning event corresponding to that
containerd failure, and such unexpected event continues to fail the
tests. So this change adds that event to the list of expected ones.
The `choco_pack` job only runs for stable tags. In order for jobs to
depend on it to run on non-stable tags, we need to move this tag check from the
`choco_pack` job level down into its steps.
## edge-20.6.3
This edge release is a release candidate for stable-2.8.1. It includes a fix
to support multicluster gateways on EKS.
* The `config.linkerd.io/proxy-destination-get-networks` annotation configures
the networks for which a proxy can discover metadata. This is an advanced
configuration option that has security implications.
* The multicluster service-mirror has been extended to resolve DNS names for
target clusters when an IP address it not known.
* Linkerd checks could fail when run from the dashboard. Thanks to @alex-berger
for providing a fix!
* The CLI will be published for Chocolatey (Windows) on future stable releases.
* Base images have been updated:
* debian:buster-20200514-slim
* grafana/grafana:7.0.3
Signed-off-by: Zahari Dichev zaharidichev@gmail.com
Co-authored-by: Oliver Gould <ver@buoyant.io>
In #4585 we are observing an issue where a loop is encountered when using nginx ingress. The problem is that the outbound proxy does a dst lookup on the IP address which happens to be the very same address the ingress is listening on.
In order to avoid situations like that this PR introduces a way to modify the set of networks for which the proxy shall do IP based discovery. The change introduces a helm flag `.Values.global.proxy.destinationGetNetworks` that can be used to modify this value. There are two ways a user can affect the this setting:
- setting the `destinationGetNetworks` field in values during a Helm install, which changes the default on all injected pods
- using an annotation ` config.linkerd.io/proxy-destination-get-networks` for injected workloads to override this value
Note that this setting cannot be tweaked through the `install` or `inject` command
Fix: #4585
Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
Explicitly shebang `bin/update-go-deps-shas` with `#!/bin/bash` instead
of `#!/bin/sh` because the latter points to `dash` in most Ubuntu-based
distros, and the script's `bin/_tag.sh` dependency requires bash.
## Problem
#4557 changed the name of the function that `helm_upgrade_integration_tests`
uses.
`install_stable()` was renamed to `latest_release_channel()` and now takes an
argument for specifying either `edge` or `stable`.
`run_helm_upgrade_test` is a function used by the helm upgrade integration test
and was not properly updated to use `latest_release_channel()`.
This silently passed integration tests because `run_helm_upgrade_test` started
passing an empty string for the version to upgrade from, which results in the
default behavior of `install_test.go`--and therefore still passes.
## Solution
`run_helm_upgrade_test` now uses `latest_release_channel()` and passes the
proper argument.
Additionally, it checks that the version returned from
`latest_release_channel()` is not empty. If it is empty, it exits the test. This
ensures something like this does happen in the future.
Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
Adds parameters like kubernetesHelper, k8scontext, etc to the NewGenericTestHelper func allowing it to be more general, and to be able to be usable through linkerd2-conformance
* Integration tests: Warn (instead of erroring) upon pod restarts
Fixes#4595
Don't have integration tests fail whenever a pod is detected to have
restarted just once. For now we'll be just logging this out and creating
a warning annotation for it.
* Fix install-pr script
* Add image-archives path to commands to use the files
Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
Signed-off-by: Charles Pretzer <charles@buoyant.io>
Co-authored-by: Charles Pretzer <charles@buoyant.io>
Fixes#4606
This has not worked as far back as stable-2.6.0.
## Solution
The recommended upgrade process is to include `--prune` as part of `kubectl
apply ..`:
```bash
$ linkerd upgrade | kubectl apply --prune -l linkerd.io/control-plane-ns=linkerd -f -
```
This is an issue for multi-stage upgrade because `linkerd upgrade config` does
not include the `linkerd-config` ConfigMap in it's output.
`kubectl apply --prune ..` will then prune this resource because it matches the
label selector *and* is not in the above output.
The issue occurs when `linkerd upgrade control-plane` is run and expects to find
the ConfigMap that was just pruned.
This can be fixed by not suggesting to prune resources as part of the
multi-stage upgrade.
## Considered
Including `templates/config.yaml` in the install output regardless of the stage.
Instead of it being a template only used in `control-plane` stage in
[render](4aa3ca7f87/cli/cmd/install.go (L873-L886)), it could always be rendered.
This just exposes other things that are pruned in the process:
```bash
❯ bin/linkerd upgrade control-plane |kubectl apply --prune -l linkerd.io/control-plane-ns=linkerd -f -
× Failed to build upgrade configuration: secrets "linkerd-identity-issuer" not found
For troubleshooting help, visit: https://linkerd.io/upgrade/#troubleshooting
error: no objects passed to apply
```
Ultimately, resources part of the `control-plane` stage need to remain and that
will not happen if we prune all resources not in the `config` stage output
Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
This adds an integration test for upgrading from the latest edge to the current
build.
Closes#4471
Signed-off-by: Kevin Leimkuhler kevin@kleimkuhler.com
* CI steps for Chocolatey package - take 2
Followup to #4205, supersedes #4205
This adds:
- A new job psscript-analyzer into the `statics_checks.yml`
workflow for linting the Chocolatey Powershell script.
- A new `choco_pack` job in the `release.yml` workflow for
updating the Chocolatey spec file and generating the
package. This is only triggered for stable releases. It requires
a windows runner in order to run the choco tooling (in theory
it should have worked on a linux runner but in practice it
didn't).
- The `Create release` step was updated to upload the generated package,
if present.
- The source file path in `bin/win/linkerd.nuspec` was updated
to make this work.
* Name nupkg file accordingly to the other release assets
My experience of our CODEOWNERS setup is that it frequently causes us to
require additional pro-forma reviews, but I think we can do a decent job
of getting the proper reviews informally without enforcing ownership.
I'd like to simplify this by relaxing the CODEOWNERS to add
@linkerd/maintainers by default. The project infrastructure docs should
remain locked-down, requiring a review from me; and I've updated the
CHANGES review requirement to be @adleong and I (practically, I'll
review most of the CHANGES, but Alex is a suitable fallback in most
cases).
Then, we leave the CNI ownership as-is (unless others want to volunteer
for those reviews ;).
Tarun Pothulapati
Desmond Ho
Naseem
Zahari Dichev
Arthur Silva Sens
Mayank Shah
William Morgan
Alejandro Pedraza
Alex Leong
Oliver Gould
Kevin Leimkuhler
Lutz Behnke
Alexander Berger
Ivan Sim
Joakim Roubert
cpretzer