Fixes#4511
Add the `linkerd.io/control-plane-component: gateway` label to the multicluster gateway. Change the value of `linkerd.io/control-plane-component` from `linkerd-service-mirror` to `service-mirror` for the service mirror controller.
These changes are for consistency and should not result in any change in functionality.
Signed-off-by: Alex Leong <alex@buoyant.io>
This edge adds multi-arch support to Linkerd! Our docker images and CLI now
support the amd64, arm64, and arm architectures.
* Multicluster
* Added a multicluster unlink command for removing multicluster links
* Improved multicluster checks to be more informative when the remote API is
not reachable
* Proxy
* Enabled a multi-threaded runtime to substantially improve latency especially
when the proxy is serving requests for many concurrent connections
* Other
* Fixed an issue where the debug sidecar image was missing during upgrades
(thanks @javaducky!)
* Updated all control plane plane and proxy container images to be multi-arch
to support amd64, arm64, and arm (thanks @aliariff!)
* Fixed an issue where check was failing when DisableHeartBeat was set to true
(thanks @mvaal!)
Supersedes #4846
Bump proxy-init to v1.3.6, containing CNI fixes and support for
multi-arch builds.
#4846 included this in v1.3.5 but proxy.golang.org refused to update the
modified SHA
The upgrade tests were failing due to hardcoded certificates which had expired. Additionally, these tests contained large swaths of yaml that made it very difficult to understand the semantics of each test case and even more difficult to maintain.
We greatly improve the readability and maintainability of these tests by using a slightly different approach. Each test follows this basic structure:
* Render an install manifest
* Initialize a fake k8s client with the install manifest (and sometimes additional manifests)
* Render an upgrade manifest
* Parse the manifests as yaml tree structures
* Perform a structured diff on the yaml tree structured and look for expected and unexpected differences
The install manifests are generated dynamically using the regular install flow. This means that we no longer need large sections of hardcoded yaml in the tests themselves. Additionally, we now asses the output by doing a structured diff against the install manifest. This means that we no longer need golden files with explicit expected output.
All test cases were preserved except for the following:
* Any test cases related to multiphase install (config/control plane) were not replicated. This flow doesn't follow the same pattern as the tests above because the install and upgrade manifests are not expected to be the same or similar. I also felt that these tests were lower priority because the multiphase install/upgrade feature does not seem to be very popular and is a potential candidate for deprecation.
* Any tests involving upgrading from a very old config were not replicated. The code to generate these old style configs is no longer present in the codebase so in order to test this case, we would need to resort to hardcoded install manifests. These tests also seemed low priority to me because Linkerd versions that used the old config are now over 1 year old so it may no longer be critical that we support upgrading from them. We generally recommend that users upgrading from an old version of Linkerd do so by upgrading through each major version rather than directly to the latest.
Signed-off-by: Alex Leong <alex@buoyant.io>
* When releasing, build and upload the amd64, arm64 and arm architectures builds for the CLI
* Refactored `Dockerfile-bin` so it has separate stages for single and multi arch builds. The latter stage is only used for releases.
Signed-off-by: Ali Ariff <ali.ariff12@gmail.com>
This PR moves default values into add-on specific values.yaml thus
allowing us to update default values as they would not be present in
linkerd-config-addons cm.
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
This release enables a multi-threaded runtime. Previously, the proxy
would only ever use a single thread for data plane processing; now, when
the proxy is allocated more than 1 CPU share, the proxy allocates a
thread per available CPU. This has shown substantial latency
improvements in benchmarks, especially when the proxy is serving
requests for many concurrent connections.
---
* Add a `multicore` feature flag (linkerd/linkerd2-proxy#611)
* Add `multicore` to default features (linkerd/linkerd2-proxy#612)
* admin: add an endpoint to dump spawned Tokio tasks (linkerd/linkerd2-proxy#595)
* trace: roll `tracing` and `tracing-subscriber` dependencies (linkerd/linkerd2-proxy#615)
* stack: Add NewService::into_make_service (linkerd/linkerd2-proxy#618)
* trace: tweak tracing & test support for the multithreaded runtime (linkerd/linkerd2-proxy#616)
* Make FailFast cloneable (linkerd/linkerd2-proxy#617)
* Move HTTP detection & server into linkerd2_proxy_http (linkerd/linkerd2-proxy#619)
* Mark tap integration tests as flakey (linkerd/linkerd2-proxy#621)
* Introduce a SkipDetect layer to preempt detection (linkerd/linkerd2-proxy#620)
Fixes#4774
When a service mirror controller is unable to connect to the target cluster's API, the service mirror controller crashes with the error that it has failed to sync caches. This error lacks the necessary detail to debug the situation. Unfortunately, client-go does not surface more useful information about why the caches failed to sync.
To make this more debuggable we do a couple things:
1. When creating the target cluster api client, we eagerly issue a server version check to test the connection. If the connection fails, the service-mirror-controller logs now look like this:
```
time="2020-07-30T23:53:31Z" level=info msg="Got updated link broken: {Name:broken Namespace:linkerd-multicluster TargetClusterName:broken TargetClusterDomain:cluster.local TargetClusterLinkerdNamespace:linkerd ClusterCredentialsSecret:cluster-credentials-broken GatewayAddress:35.230.81.215 GatewayPort:4143 GatewayIdentity:linkerd-gateway.linkerd-multicluster.serviceaccount.identity.linkerd.cluster.local ProbeSpec:ProbeSpec: {path: /health, port: 4181, period: 3s} Selector:{MatchLabels:map[] MatchExpressions:[{Key:mirror.linkerd.io/exported Operator:Exists Values:[]}]}}"
time="2020-07-30T23:54:01Z" level=error msg="Unable to create cluster watcher: cannot connect to api for target cluster remote: Get \"https://36.199.152.138/version?timeout=32s\": dial tcp 36.199.152.138:443: i/o timeout"
```
This error also no longer causes the service mirror controller to crash. Updating the Link resource will cause the service mirror controller to reload the credentials and try again.
2. We rearrange the checks in `linkerd check --multicluster` to perform the target API connectivity checks before the service mirror controller checks. This means that we can validate the target cluster API connection even if the service mirror controller is not healthy. We also add a server version check here to quickly determine if the connection is healthy. Sample check output:
```
linkerd-multicluster
--------------------
√ Link CRD exists
√ Link resources are valid
* broken
W0730 16:52:05.620806 36735 transport.go:243] Unable to cancel request for promhttp.RoundTripperFunc
× remote cluster access credentials are valid
* failed to connect to API for cluster: [broken]: Get "https://36.199.152.138/version?timeout=30s": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
see https://linkerd.io/checks/#l5d-smc-target-clusters-access for hints
W0730 16:52:35.645499 36735 transport.go:243] Unable to cancel request for promhttp.RoundTripperFunc
× clusters share trust anchors
Problematic clusters:
* broken: unable to fetch anchors: Get "https://36.199.152.138/api/v1/namespaces/linkerd/configmaps/linkerd-config?timeout=30s": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
see https://linkerd.io/checks/#l5d-multicluster-clusters-share-anchors for hints
√ service mirror controller has required permissions
* broken
√ service mirror controllers are running
* broken
× all gateway mirrors are healthy
wrong number of (0) gateway metrics entries for probe-gateway-broken.linkerd-multicluster
see https://linkerd.io/checks/#l5d-multicluster-gateways-endpoints for hints
√ all mirror services have endpoints
‼ all mirror services are part of a Link
mirror service voting-svc-gke.emojivoto is not part of any Link
see https://linkerd.io/checks/#l5d-multicluster-orphaned-services for hints
```
Some logs from the underlying go network libraries sneak into the output which is kinda gross but I don't think it interferes too much with being able to understand what's going on.
Signed-off-by: Alex Leong <alex@buoyant.io>
Some installations upgrading from versions prior to 2.7.x may have missing debug image name and version. This fix ensures that the default values are in place for this scenario and additionally upgrades the version of debug image with the control plane version.
Signed-off-by: Paul Balogh <javaducky@gmail.com>
Build ARM docker images in the release workflow.
# Changes:
- Add a new env key `DOCKER_MULTIARCH` and `DOCKER_PUSH`. When set, it will build multi-arch images and push them to the registry. See https://github.com/docker/buildx/issues/59 for why it must be pushed to the registry.
- Usage of `crazy-max/ghaction-docker-buildx ` is necessary as it already configured with the ability to perform cross-compilation (using QEMU) so we can just use it, instead of manually set up it.
- Usage of `buildx` now make default global arguments. (See: https://docs.docker.com/engine/reference/builder/#automatic-platform-args-in-the-global-scope)
# Follow-up:
- Releasing the CLI binary file in ARM architecture. The docker images resulting from these changes already build in the ARM arch. Still, we need to make another adjustment like how to retrieve those binaries and to name it correctly as part of Github Release artifacts.
Signed-off-by: Ali Ariff <ali.ariff12@gmail.com>
The job started failing consistently today with:
```
##[error]devblackops/github-action-psscriptanalyzer/v2/action.yml (Line:
30, Col: 9): Unexpected value ''
##[error]devblackops/github-action-psscriptanalyzer/v2/action.yml (Line:
30, Col: 9): Unexpected value ''
##[error]System.ArgumentException: Unexpected type 'NullToken'
encountered while reading 'outputs'. The type 'MappingToken' was
expected.
```
It seems it's something in Github that changed today that is clashing
with the `devblackops/github-action-psscriptanalyzer` action.
I've raised devblackops/github-action-psscriptanalyzer#12
Fixes#4707
In order to remove a multicluster link, we add a `linkerd multicluster unlink` command which produces the yaml necessary to delete all of the resources associated with a `linkerd multicluster link`. These are:
* the link resource
* the service mirror controller deployment
* the service mirror controller's RBAC
* the probe gateway mirror for this link
* all mirror services for this link
This command follows the same pattern as the `linkerd uninstall` command in that its output is expected to be piped to `kubectl delete`. The typical usage of this command is:
```
linkerd --context=source multicluster unlink --cluster-name=foo | kubectl --context=source delete -f -
```
This change also fixes the shutdown lifecycle of the service mirror controller by properly having it listen for the shutdown signal and exit its main loop.
A few alternative designs were considered:
I investigated using owner references as suggested [here](https://github.com/linkerd/linkerd2/issues/4707#issuecomment-653494591) but it turns out that owner references must refer to resources in the same namespace (or to cluster scoped resources). This was not feasible here because a service mirror controller can create mirror services in many different namespaces.
I also considered having the service mirror controller delete the mirror services that it created during its own shutdown. However, this could lead to scenarios where the controller is killed before it finishes deleting the services that it created. It seemed more reliable to have all the deletions happen from `kubectl delete`. Since this is the case, we avoid having the service mirror controller delete mirror services, even when the link is deleted, to avoid the race condition where the controller and CLI both attempt to delete the same mirror services and one of them fails with a potentially alarming error message.
Signed-off-by: Alex Leong <alex@buoyant.io>
#3467 removed `NamespaceLanding` which was the only component that imported
`Accordion`, so we can delete the file.
Signed-off-by: Carol Scott carol@buoyant.io
* bump prometheus to the latest v2.19.3
latest prometheus version shows a lot of decrease in the memory usage
and other benefits
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
This PR adds the LinguiJS project to the Linkerd dashboard for i18n and
translation. It is a precursor to adding translations to the dashboard. Only
two components have been translated in this PR, to allow reviewers to evaluate
the ease of use; A second PR will add translations for the remaining components.
* CNI add support for priorityClassName
As requested in #2981 one should be able to optionally define a priorityClassName for the linkerd2 pods.
With this commit support for priorityClassName is added to the CNI plugin helm chart as well as to the
cli command for installing the CNI plugin.
Also added an `installNamespace` Helm option for the CNI installation.
Implements part of #2981.
Signed-off-by: alex.berger@nexiot.ch <alex.berger@nexiot.ch>
This pr adds `globa.prometheusUrl` field which will be used to configure publlic-api, hearbeat, grafana, etc (i,e query path) to use a external Prometheus.
* support overriding inbound and outbound connect timeouts.
* add validation on user provided TCP connect timeouts
* convert valid time values into ms
Signed-off-by: Matt Miller <mamiller@rosettastone.com>
* Introduce multicluster gateway api handler in web api server
* Added MetricsUtil for Gateway metrics
* Added gateway api helper
* Added Gateway Component
Updated metricsTable component to support gateway metrics
Added handler for gateway
Fixes#4601
Signed-off-by: Tharun <rajendrantharun@live.com>
* Add sidecar container support for linkerd-prometheus
Adds a new setting to the Prometheus' Helm config, allowing adding any kind of sidecar containers to the main container.
The specific use case that inspired this was for exporting data from Prometheus to external systems (e.g. cloudwatch, stackdriver, datadog) using a process that watches the prometheus write-ahead log (WAL).
Signed-off-by: Nathan J. Mehl <n@oden.io>
* Fixed `linkerd check` not finding Prometheus
## The Problem
`linkerd check` run right after install is failing because it can't find the Prometheus Pod.
## The Cause
The "control plane pods are ready" check used to verify the existence of all the control plane pods, blocking until all the pods were ready.
Since #4724, Prometheus is no longer included in that check because it's checked separately as an add-on. An unintended consequence is that when the ensuing "control plane self-check" is triggered, Prometheus might not be ready yet and the check fails because it doesn't do retries.
## The Fix
The "control plane self-check" uses a gRPC call (it's the only check that does that) and those weren't designed with retries in mind.
This PR adds retry functionality to the `runCheckRPC()` function, making sure the final output remains the same
It also temporarily disables the `upgrade-edge` integration test because after installing edge-20.7.4 `linkerd check` will fail because of this.
Add a new structure on the destination controller side to keep track of contextual information.
The token format has been changed from ns:<namespace> to a JSON format so that more variables can be
encdoed in the token. As part of this PR, a new field 'nodeName' has been added to help with service
topologies.
Fixes#4498
Signed-off-by: Matei David <matei.david.35@gmail.com>
Removed the dependency on the base image, and instead install the needed packages in the Dockerfiles for debug and CNI.
Also removed some obsolete info from BUILD.md
Signed-off-by: Ali Ariff <ali.ariff12@gmail.com>
This PR removes the service mirror controller from `linkerd mc install` to `linkerd mc link`, as described in https://github.com/linkerd/rfc/pull/31. For fuller context, please see that RFC.
Basic multicluster functionality works here including:
* `linkerd mc install` installs the Link CRD but not any service mirror controllers
* `linkerd mc link` creates a Link resource and installs a service mirror controller which uses that Link
* The service mirror controller creates and manages mirror services, a gateway mirror, and their endpoints.
* The `linkerd mc gateways` command lists all linked target clusters, their liveliness, and probe latences.
* The `linkerd check` multicluster checks have been updated for the new architecture. Several checks have been rendered obsolete by the new architecture and have been removed.
The following are known issues requiring further work:
* the service mirror controller uses the existing `mirror.linkerd.io/gateway-name` and `mirror.linkerd.io/gateway-ns` annotations to select which services to mirror. it does not yet support configuring a label selector.
* an unlink command is needed for removing multicluster links: see https://github.com/linkerd/linkerd2/issues/4707
* an mc uninstall command is needed for uninstalling the multicluster addon: see https://github.com/linkerd/linkerd2/issues/4708
Signed-off-by: Alex Leong <alex@buoyant.io>
## edge-20.7.4
This edge release adds support for the new Kubernetes [EndpointSlice] resource
to the Destination controller. Using the EndpointSlice API is more efficient
for the Kubernetes control plane than using the Endpoints API. If the cluster
supports EndpointSlices (a beta feature in Kubernetes 1.17), Linkerd can be
installed with `--enable-endpoint-slices` flag to use this resource rather
than the Endpoints API.
* Added fish shell completions to the `linkerd` command (thanks @WLun001!)
* Enabled the support for EndpointSlices (thanks @Matei207!)
* Separated prometheus checks and made them runnable only when the add-on
is enabled
Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
As linkerd-prometheus is optional now, the checks are also separated
and should only work when the prometheus add-on is installed.
This is done by re-using the add-on check code.
* Migrate CI to docker buildx and other improvements
## Motivation
- Improve build times in forks. Specially when rerunning builds because of some flaky test.
- Start using `docker buildx` to pave the way for multiplatform builds.
## Performance improvements
These timings were taken for the `kind_integration.yml` workflow when we merged and rerun the lodash bump PR (#4762)
Before these improvements:
- when merging: `24:18`
- when rerunning after merge (docker cache warm): `19:00`
- when running the same changes in a fork (no docker cache): `32:15`
After these improvements:
- when merging: `25:38`
- when rerunning after merge (docker cache warm): `19:25`
- when running the same changes in a fork (docker cache warm): `19:25`
As explained below, non-forks and forks now use the same cache, so the important take is that forks will always start with a warm cache and we'll no longer see long build times like the `32:15` above.
The downside is a slight increase in the build times for non-forks (up to a little more than a minute, depending on the case).
## Build containers in parallel
The `docker_build` job in the `kind_integration.yml`, `cloud_integration.yml` and `release.yml` workflows relied on running `bin/docker-build` which builds all the containers in sequence. Now each container is built in parallel using a matrix strategy.
## New caching strategy
CI now uses `docker buildx` for building the container images, which allows using an external cache source for builds, a location in the filesystem in this case. That location gets cached using actions/cache, using the key `{{ runner.os }}-buildx-${{ matrix.target }}-${{ env.TAG }}` and the restore key `${{ runner.os }}-buildx-${{ matrix.target }}-`.
For example when building the `web` container, its image and all the intermediary layers get cached under the key `Linux-buildx-web-git-abc0123`. When that has been cached in the `main` branch, that cache will be available to all the child branches, including forks. If a new branch in a fork asks for a key like `Linux-buildx-web-git-def456`, the key won't be found during the first CI run, but the system falls back to the key `Linux-buildx-web-git-abc0123` from `main` and so the build will start with a warm cache (more info about how keys are matched in the [actions/cache docs](https://docs.github.com/en/actions/configuring-and-managing-workflows/caching-dependencies-to-speed-up-workflows#matching-a-cache-key)).
## Packet host no longer needed
To benefit from the warm caches both in non-forks and forks like just explained, we're required to ditch doing the builds in Packet and now everything runs in the github runners VMs.
As a result there's no longer separate logic for non-forks and forks in the workflow files; `kind_integration.yml` was greatly simplified but `cloud_integration.yml` and `release.yml` got a little bigger in order to use the actions artifacts as a repository for the images built. This bloat will be fixed when support for [composite actions](https://github.com/actions/runner/blob/users/ethanchewy/compositeADR/docs/adrs/0549-composite-run-steps.md) lands in github.
## Local builds
You still are able to run `bin/docker-build` or any of the `docker-build.*` scripts. And to make use of buildx, run those same scripts after having set the env var `DOCKER_BUILDKIT=1`. Using buildx supposes you have installed it, as instructed [here](https://github.com/docker/buildx).
## Other
- A new script `bin/docker-cache-prune` is used to remove unused images from the cache. Without that the cache grows constantly and we can rapidly hit the 5GB limit (when the limit is attained the oldest entries get evicted).
- The `go-deps` dockerfile base image was changed from `golang:1.14.2` (ubuntu based) to `golang-1:14.2-alpine` also to conserve cache space.
# Addressed separately in #4875:
Got rid of the `go-deps` image and instead added something similar on top of all the Dockerfiles dealing with `go`, as a first stage for those Dockerfiles. That continues to serve as a way to pre-populate go's build cache, which speeds up the builds in the subsequent stages. That build should in theory be rebuilt automatically only when `go.mod` or `go.sum` change, and now we don't require running `bin/update-go-deps-shas`. That script was removed along with all the logic elsewhere that used it, including the `go_dependencies` job in the `static_checks.yml` github workflow.
The list of modules preinstalled was moved from `Dockerfile-go-deps` to a new script `bin/install-deps`. I couldn't find a way to generate that list dynamically, so whenever a slow-to-compile dependency is found, we have to make sure it's included in that list.
Although this simplifies the dev workflow, note that the real motivation behind this was a limitation in buildx's `docker-container` driver that forbids us from depending on images that haven't been pushed to a registry, so we have to resort to building the dependencies as a first stage in the Dockerfiles.
The GitHub Actions build status badge was referencing an old workflow
named `CI`, which always shows red, and is no longer used.
Fix the build status badge to reference the `Release` workload.
Also slightly reformat the matrix build yaml, as the list has grown a
bit.
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
* Small PR that uncomments the `EndpointSliceAcess` method and cleans up left over todos in the destination service.
* Based on the past three PRs related to `EndpointSlices` (#4663#4696#4740); they should now be functional (albeit prone to bugs) and ready to use.
Signed-off-by: Matei David <matei.david.35@gmail.com>
* Removes/Relaxes prometheus related checks
Now that prometheus is an add-on, There can be cases where prometheus is
disabled at which the check should show a warning but not fail. This
decouples the tight depedency.
This changes the following checks:
- Removes serviceAccount and pod checks in the CLI.
- Relaxes `linkerd-api` checks to only check for prometheus access when
the URL is not empty. This should work seamlessly with external
prometheus as that URL will be passed and it performs the same
check.
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
## edge-20.7.3
This edge release introduces an install flag for EndpointSlices. With this flag,
endpoint slices can be used as a resource in the destination service instead of
the endpoints resource.
* Introduce CLI and Helm install flag for EndpointSlices (thanks @Matei207!)
* Internal improvements to the CI process for testing Helm installations
Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
Alex Leong
Alejandro Pedraza
Ali Ariff
Marcus Vaal
Olukayode Bankole
Tarun Pothulapati
Herrmann Hinz
Oliver Gould
cpretzer
Paul Balogh
Rajat Jindal
Carol A. Scott
dependabot[bot]
Alexander Berger
Naseem
David Tyler
Mayank Shah
Matt Miller
Tharun Rajendran
memory
Matei David
Zahari Dichev
ZouYu
Andrew Seigner
Wei Lun
Kevin Leimkuhler