* Do not run cloud integration tests in CI
Closes#4963
Removed the `./.github/workflows/cloud_integration.yml` workflow, and
removed the `cloud_integration_tests` job from the ``./.github/workflows/release.yml` workflow.
This adds a `replace` statement to `go.mod` to force the newer version `1.14.x` of `github.com/grpc-ecosystem/grpc-gateway` to avoid the following warning in all the controller container logs:
```
WARNING: Package "github.com/golang/protobuf/protoc-gen-go/generator" is deprecated.
A future release of golang/protobuf will delete this package,
which has long been excluded from the compatibility promise.
```
More info [here](https://github.com/golang/protobuf/issues/1104)
Currently, This field has to be configured to make CNI work in
GKE clusters as thats where the binaries have to be stored. This
was configurable through Helm, but the same can be allowed through
the CLI too
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
* tracing: Move default values into chart
This branch updates the tracing add-on's values into their own chart's values.yaml
(just like grafana and prometheus). This prevents them from being saved into
`linkerd-config-addons` where only the overridden values are stored. Thus allowing
us to change the defaults.
This also
- Updates the check command to fall back to default values, if there are no
overridden name fields.
- Updates jaeger to `1.19.2`
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
* Push docker images to ghcr.io instead of gcr.io
The `cloud_integration.yml` and `release.yml` workflows were modified to
log into ghcr.io, and remove the `Configure gcloud` step which is no
longer necessary.
Note that besides the changes to cloud_integration.yml and release.yml, there was a change to the upgrade-stable integration test so that we do linkerd upgrade --addon-overwrite to reset the addons settings because in stable-2.8.1 the Grafana image was pegged to gcr.io/linkerd-io/grafana in linkerd-config-addons. This will need to be mentioned in the 2.9 upgrade notes.
Also the egress integration test has a debug container that now is pegged to the edge-20.9.2 tag.
Besides that, the other changes are just a global search and replace (s/gcr.io\/linkerd-io/ghcr.io\/linkerd/).
This release includes several major changes to the proxy's behavior:
- Service profile lookups are now necessary and fundamental to outbound
discovery for HTTP traffic. That is, if a service profile lookup is
rejected, endpoint discovery will not be performed; and endpoint
discovery must succeed for all destinations that are permitted by
service profiles. This simplifies caching and buffering to reduce
latency (especially under concurrency).
- Service discovery is now performed for all TCP traffic, and
connections are balanced over endpoints according to connection
latency.
- This enables mTLS for **all** meshed connections; not just HTTP.
- Outbound TCP metrics are now hydrated with endpoint-specific labels.
---
* outbound: Cache balancers within profile stack (linkerd/linkerd2-proxy#641)
* outbound: Remove unused error type (linkerd/linkerd2-proxy#648)
* Eliminate the ConnectAddr trait (linkerd/linkerd2-proxy#649)
* profiles: Do not rely on tuples as stack targets (linkerd/linkerd2-proxy#650)
* proxy-http: Remove unneeded boilerplate (linkerd/linkerd2-proxy#651)
* outbound: Clarify Http target types (linkerd/linkerd2-proxy#653)
* outbound: TCP discovery and load balancing (linkerd/linkerd2-proxy#652)
* metrics: Add endpoint labels to outbound TCP metrics (linkerd/linkerd2-proxy#654)
The proxy performs endpoint discovery for unnamed services, but not
service profiles.
The destination controller and proxy have been updated to support
lookups for unnamed services in linkerd/linkerd2#4727 and
linkerd/linkerd2-proxy#626, respectively.
This change modifies the injection template so that the
`proxy.destinationGetNetworks` configuration enables profile
discovery for all networks on which endpoint discovery is permitted.
`./bin/test-cleanup` was trying to remove the
resources with the label `linkerd.io/is-test-helm` which we're not
using. Instead, we simply call `helm delete` on the appropriate helm
releases.
This is required for a clean cleanup after the ARM integration test, whose
cluster is just cleaned by this script at the end and is not torn down.
## edge-20.9.1
This edge release contains an important proxy update that allows linkerd to
continue to operate normally in HA during node outages. We're also adding full
Kubernetes 1.19 support!
* Improved the proxy's error handling for DNS errors encountered when
discovering control plane addresses, which can be common during installation,
before all components have been started
* The destination and identity services had to be made headless in order to
support that new controller discovery (which now can leverage SRV records)
* Use SAN fields when generating the linkerd webhook configs; this completes the
Kubernetes 1.19 support which enforces them
* Fixed `linkerd check` for multicluster that was spuriously claiming the
absence of some resources
* Improved the injection test cleanup (thanks @zhouhao3!)
* Added ability to run the integration test suite using a cluster in an ARM
architecture (thanks @aliariff!)
Updating only the go 1.15 version, makes the upgrades fail from older versions,
as the identity certs do not have that setting and go 1.15 expects them.
This PR upgrades the cert generation code to have that field,
allowing us to move to go 1.15 in later versions of Linkerd.
* Create webhook certs with SANs, along with legacy Common Name
Fixes#4918
In Kubernetes 1.18, Go version has been updated to 1.15 which updated
how certificates are verified. They moved away from legacy Common Name
field to SANs.
This PR replaces the internal Helm cert generation functions to
`genSelfSignedCert` as they allow alternate DNS names to be specified.
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
The multicluster checks make sure that the correct resources exist for each service mirror controller. When looking up these resources, it uses the `linkerd.io/control-plane-component=linkerd-service-mirror` label selector. However, these resources have the label `linkerd.io/control-plane-component=service-mirror`. This causes the resource lookup to fail to find the resource and the check spuriously fails.
```
× service mirror controller has required permissions
missing ServiceAccounts: linkerd-service-mirror-self
missing ClusterRoles: linkerd-service-mirror-access-local-resources-self
missing ClusterRoleBindings: linkerd-service-mirror-access-local-resources-self
missing Roles: linkerd-service-mirror-read-remote-creds-self
missing RoleBindings: linkerd-service-mirror-read-remote-creds-self
see https://linkerd.io/checks/#l5d-multicluster-source-rbac-correct for hints
| * no service mirror controller deployment for Link self
```
Instead, use the correct label selector when looking up these resources.
Signed-off-by: Alex Leong <alex@buoyant.io>
All of the code for the service mirror controller lives in the `linkerd/linkerd2/controller/cmd` package. It is typical for control plane components to only have a `main.go` entrypoint in the cmd package. This can sometimes make it hard to find the service mirror code since I wouldn't expect it to be in the cmd package.
We move the majority of the code to a dedicated controller package, leaving only main.go in the cmd package. This is purely organizational; no behavior change is expected.
Signed-off-by: Alex Leong <alex@buoyant.io>
## edge-20.8.4
* Fixed a problem causing the `enable-endpoint-slices` flag to not be persisted
when set via `linkerd upgrade` (thanks @Matei207!)
* Removed SMI-Metrics templates and experimental sub-commands
* Use `--frozen-lockfile` to avoid accidental update of dashboard JS
dependencies in CI (thanks @tharun208!)
Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
Fixes#4790
This PR removes both the SMI-Metrics templates along with the
experimental sub-commands. This also removes pkg `smi-metrics`
as there is no direct use of it without the commands.
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
## What/How
@adleong pointed out in #4780 that when enabling slices during an upgrade, the new value does not persist in the `linkerd-config` ConfigMap. I took a closer look and it seems that we were never overwriting the values in case they were different.
* To fix this, I added an if block when validating and building the upgrade options -- if the current flag value differs from what we have in the ConfigMap, then change the ConfigMap value.
* When doing so, I made sure to check that if the cluster does not support `EndpointSlices` yet the flag is set to true, we will error out. This is done similarly (copy&paste similarily) to what's in the install part.
* Additionally, I have noticed that the helm ConfigMap template stored the flag value under `enableEndpointSlices` field name. I assume this was not changed in the initial PR to reflect the changes made in the protocol buffer. The API (and thus the CLI) uses the field name `endpointSliceEnabled` instead. I have changed the config template so that helm installations will use the same field, which can then be used in the destination service or other components that may implement slice support in the future.
Signed-off-by: Matei David <matei.david.35@gmail.com>
unit tests in ci are runned using yarn install.
So, there will be some update to the dependencies.
This is fixed by passing --frozen-lockfile in ci workflow
Fixes#3838
Signed-off-by: Tharun <rajendrantharun@live.com>
This edge release adds support for [topology-aware service routing][1]
to the Destination controller. When providing service discovery updates
to proxies, the Destination controller will now filter endpoints based
on the service's topology preferences. Additionally, this release
includes bug fixes for the `linkerd check` CLI command and web
dashboard.
* CLI
* `linkerd check` will no longer warn about a looser webhook failure
policy in HA mode
* Controller
* Added support for [topology-aware service routing][1] to the
Destination controller (thanks @Matei207)
* Changed the Destination controller to always return destination
overrides for service profiles when no traffic split is present
* Web UI
* Fixed Tap `Authority` dropdown not being populated (thanks to
@tharun208!)
[1]: https://kubernetes.io/docs/concepts/services-networking/service-topology/
Tap component is calling fetch metrics with skip_stats and authority
service type is not sent.So, authority dropdown is not getting populated.
Added a seperate call to get metrics for authority
Fixes#4697
Signed-off-by: Tharun <rajendrantharun@live.com>
## Motivation
#4879
## Solution
When no traffic split exists for services, return a single destination override
with a weight of 100%.
Using the destination client on a new linkerd installation, this results in the
following output for `linkerd-identity` service:
```
❯ go run controller/script/destination-client/main.go -method getProfile -path linkerd-identity.linkerd.svc.cluster.local:8080
INFO[0000] retry_budget:{retry_ratio:0.2 min_retries_per_second:10 ttl:{seconds:10}} dst_overrides:{authority:"linkerd-identity.linkerd.svc.cluster.local.:8080" weight:100000}
INFO[0000]
```
Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
[Link to RFC](https://github.com/linkerd/rfc/pull/23)
### What
---
* PR that puts together all past pieces of the puzzle to deliver topology-aware service routing, as specified in the [Kubernetes docs](https://kubernetes.io/docs/concepts/services-networking/service-topology/) but with a much better load balancing algorithm and all the coolness of linkerd :)
* The first piece of this PR is focused on adding topology metadata: topology preference for services and topology `<k,v>` pairs for endpoints.
* The second piece of this PR puts together the new context format and fetching the source node topology metadata in order to allow for endpoints filtering.
* The final part is doing the filtering -- passing all of the metadata to the listener and on every `Add` filtering endpoints based on the topology preference of the service, topology `<k,v>` pairs of endpoints and topology of the source (again `<k,v>` pairs).
### How
---
* **Collecting metadata**:
- Services do not have values for topology keys -- the topological keys defined in a service's spec are only there to dictate locality preference for routing; as such, I decided to store them in an array, they will be taken exactly as they are found in the service spec, this ensures we respect the preference order.
- For EndpointSlices, we are using a map -- an EndpointSlice has locality information in the form of `<k,v>` pair, where the key is a topological key (similar to what's listed in the service) and the value is the locality information -- e.g `hostname: minikube`. For each address we now have a map of topology values which gets populated when we translate the endpoints to an address set. Because normal Endpoints do not have any topology information, we create each address with an empty map which is subsequently populated ONLY for slices in the `endpointSliceToAddressSet` function.
* **Filtering endpoints**:
- This was a tricky part and filled me with doubts. I think there are a few ways to do this, but this is how I "envisioned" it. First, the `endpoint_translator.go` should be the one to do the filtering; this means that on subscription, we need to feed all of the relevant metadata to the listener. To do this, I created a new function `AddTopologyFilter` as part of the listener interface.
- To complement the `AddTopologyFilter` function, I created a new `TopologyFilter` struct in `endpoints_watcher.go`. I then embedded this structure in all listeners that implement the interface. The structure holds the source topology (source node), a boolean to tell if slices are activated in case we need to double check (or write tests for the function) and the service preference. We create the filter on Subscription -- we have access to the k8s client here as well as the service, so it's the best point to collect all of this data together. Addresses all have their own topology added to them so they do not have to be collected by the filter.
- When we add a new set of addresses, we check to see if slices are enabled -- chances are if slices are enabled, service topology might be too. This lets us skip this step if the latest version is not adopted. Prior to sending an `Add` we filter the endpoints -- if the preference is registered by the filter we strictly enforce it, otherwise nothing changes.
And that's pretty much it.
Signed-off-by: Matei David <matei.david.35@gmail.com>
Add a static check that ensures the generated files from the proto definitions have not changed.
Fix#4669
Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
Removed usage of `GITCOOKIE_SH`, which was a script stored in a secret
to authenticate requests against googlesource.com, to avoid hitting
rate limits when pulling go dependencies from that source. Now that we
use go modules, deps are pulled from http://proxy.golang.org/ and this
is no longer needed.
This PR changes the HA check that verifies that the `config.linkerd.io/admission-webhooks=disabled` is present on kube-system to be enabled only when the failure policy for the proxy injector webhook is set to `Fail`. This allows users to skip this check in cases when the label is removed because the namespace is managed by the cloud provider like in the case described in #4754Fix#4754
Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
## Motivation
These changes came up when testing mock identity. I found it useful for the
destination client to print the identity of endpoints.
```
❯ go run controller/script/destination-client/main.go -method get -path h1.test.example.com:8080
INFO[0000] Add:
INFO[0000] labels: map[concrete:h1.test.example.com:8080]
INFO[0000] - 127.0.0.1:4143
INFO[0000] - labels: map[addr:127.0.0.1:4143 h2:false]
INFO[0000] - protocol hint: UNKNOWN
INFO[0000] - identity: dns_like_identity:{name:"foo.ns1.serviceaccount.identity.linkerd.cluster.local"}
INFO[0000]
```
I also fixed a log line in the proxy-identity where used the wrong value for the
CSR path
Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
This PR corrects misspellings identified by the [check-spelling action](https://github.com/marketplace/actions/check-spelling).
The misspellings have been reported at aaf440489e (commitcomment-41423663)
The action reports that the changes in this PR would make it happy: 5b82c6c5ca
Note: this PR does not include the action. If you're interested in running a spell check on every PR and push, that can be offered separately.
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
Add PriceKinetics, part of GVC Australia to the adopters list.
Signed-off-by: Steve Gray <steve.gray@ladbrokes.com.au>
Co-authored-by: Steve Gray <steve.gray@ladbrokes.com.au>
Fixes#4708
Adds a `linkerd multicluster uninstall` command which outputs the manifests required to uninstall the mutlicluster components. This command first checks that no links exist and advises that any links must be removed with `linkerd multicluster unlink` before proceeding. Typical usage is:
```
linkerd multicluster uninstall | kubectl delete -f -
```
Signed-off-by: Alex Leong <alex@buoyant.io>
When the Link CRD does not exist, multicluster checks in `linkerd check` will be skipped. The `--multicluster` flag is intended to force these checks on, but was being ignored.
We update the options to force the multicluster checks on when the `--multicluster` flag is used, as intended.
Now when `linkerd check --multicluster` is run on a cluster without the multicluster support installed, it gives the following output:
```
linkerd-multicluster
--------------------
× Link CRD exists
multicluster.linkerd.io/Link CRD is missing: the server could not find the requested resource
see https://linkerd.io/checks/#l5d-multicluster-link-crd-exists for hints
Status check results are ×
```
Signed-off-by: Alex Leong <alex@buoyant.io>
Alejandro Pedraza
Tarun Pothulapati
cpretzer
Oliver Gould
Zahari Dichev
dependabot[bot]
Jeffrey N. Davis
Zhou Hao
Alex Leong
tbsoares
Hu Shuai
Ali Ariff
Matei David
Tharun Rajendran
Eliza Weisman
Kevin Leimkuhler
Carol A. Scott
Josh Soref
MaT1g3R
Steve Gray