Commit Graph

268 Commits

Author SHA1 Message Date
Alejandro Pedraza 61f443ad05
Schedule heartbeat 10 mins after install (#5973)
* Schedule heartbeat 10 mins after install

... for the Helm installation method, thus aligning it with the CLI
installation method, to reduce the midnight peak on the receiving end.

The logic added into the chart is now reused by the CLI as well.

Also, set `concurrencyPolicy=Replace` so that when a job fails and it's
retried, the retries get canceled when the next scheduled job is triggered.

Finally, the go client only failed when the connection failed;
successful connections with a non 200 response status were considered
successful and thus the job wasn't retried. Fixed that as well.
2021-03-31 07:49:36 -05:00
Kevin Leimkuhler 3b617a49e1
Fix parsing of ignoreInboundPorts field (#5876)
The `ignoreInboundPorts` field is not parsed correctly when passed through the
`--set` Helm flag. This was discovered in https://github.com/linkerd/linkerd2/pull/5874#pullrequestreview-606779599.

This is happening because the value is not parsed into a string before using it
in the templating.

Before:

```
linkerd install --set proxyInit.ignoreInboundPorts="12345" |grep 12345
...
- "4190,4191,%!s(int64=12345)"
...
```

After:

```
linkerd install --set proxyInit.ignoreInboundPorts="12345" |grep 12345
...
- "4190,4191,12345"
...
```

Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
2021-03-09 15:32:20 -05:00
Tarun Pothulapati 36084c6958
helm: add NOTES.txt for extension charts (#5870)
Currently, There is no `Notes` that get printed out after installatio
is performed through helm for extensions, like we do for the core
chart. This updates the viz and jaeger charts to include that
along with instructions to view the dashbaord.

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2021-03-09 15:31:18 -05:00
Kevin Leimkuhler 02afb00760
Add back ignore ports fields in values.yaml (#5874)
Add back configurations to `values.yaml` files after being removed in #5810

Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
2021-03-08 18:41:18 -05:00
Alejandro Pedraza e236c1d113
Fix Helm chart not setting the proxy-version annotation (#5840)
In #5694 we set many Helm values to empty (like version tags), and then the
templates became responsible to filling out to proper default values. We missed
doing that for the `linkerd.io/proxy-version` annotation built in the
`_metadata.tpl` template. This fixes that, and also stops setting
`proxy.image.version` in `helm install|upgrade` in the Helm integration tests,
which is what avoided catching this error .
2021-03-01 09:11:56 -05:00
Tarun Pothulapati 5c1a375a51
destination: pass opaque-ports through cmd flag (#5829)
* destination: pass opaque-ports through cmd flag

Fixes #5817

Currently, Default opaque ports are stored at two places i.e
`Values.yaml` and also at `opaqueports/defaults.go`. As these
ports are used only in destination, We can instead pass these
values as a cmd flag for destination component from Values.yaml
and remove defaultPorts in `defaults.go`.

This means that users if they override `Values.yaml`'s opauePorts
field, That change is propogated both for injection and also
discovery like expected.

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2021-03-01 16:00:20 +05:30
Oliver Gould 9e9b40d5a2
Add the linkerd-await helper to all Linkerd containers (#5821)
When a container starts up, we generally want to wait for the proxy to
initialize before starting the controller (which may initiate outbound
connections, especially to the Kubernetes API). This is true for all
pods except the identity controller, which must start before its proxy.

This change adds the linkerd-await helper to all of our container
images. Its use is explicitly disabled in the identity controller, due
to startup ordering constraints, and the heartbeat controller, because
it does not run a proxy currently.

Fixes #5819
2021-02-25 10:35:04 -08:00
Dennis Adjei-Baah 15d1809bd0
Remove linkerd prefix from extension resources (#5803)
* Remove linkerd prefix from extension resources

This change removes the `linkerd-` prefix on all non-cluster resources
in the jaeger and viz linkerd extensions. Removing the prefix makes all
linkerd extensions consistent in their naming.

Signed-off-by: Dennis Adjei-Baah <dennis@buoyant.io>
2021-02-25 11:01:31 -05:00
Kevin Leimkuhler 07d5071cc4
Remove default skip ports and add to opaque ports (#5810)
This change removes the default ignored inbound and outbound ports from the
proxy init configuration.

These ports have been moved to the the `proxy.opaquePorts` configuration so that
by default, installations will proxy all traffic on these ports opaquely.

Closes #5571 
Closes #5595 

Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
2021-02-24 16:22:09 -05:00
Max Goltzsche a8e5bff21f
Provide CA cert as env var to identity controller. (#5690)
Currently the identity controller is the only component that receives the CA certificate / trust anchors as option `-identity-trust-anchors-pem` instead of an env var.
This stops one from letting it read the trust anchors from a Secret that is managed by e.g. cert-manager.

This PR uses an env var instead of the option to provide the trust anchors. For most helm chart users this doesn't change anything. However using kustomize the helm output manifest can now be adjusted (again) so that the certificate is loaded from a ConfigMap or Secret like in [this example](https://github.com/mgoltzsche/khelm/tree/master/example/kpt/linkerd) which aims to produce a static manifest to make the installation/update more declarative and support GitOps workflows.

This PR does not provide chart options/values to specify Secrets upfront - it would introduce dependencies to other operators.

Relates to #3843, see https://github.com/linkerd/linkerd2/issues/3843#issuecomment-775516217

Fixes #3321

Signed-off-by: Max Goltzsche <max.goltzsche@gmail.com>
2021-02-22 10:30:43 -05:00
Dennis Adjei-Baah 57db567204
send serviceprofile counts in heartbeat URL (#5775)
This change counts the number of service profiles installed in a cluster
and adds that info to the heartbeat HTTP request.

Fixes #5474

Signed-off-by: Dennis Adjei-Baah <dennis@buoyant.io>
2021-02-19 10:11:26 -05:00
Alejandro Pedraza b53dc3b400
Removed "do-not-edit" entries from values.yaml files (#5758)
Fixes #5574 and supersedes #5660

- Removed from all the `values.yaml` files all those "do not edit" entries for annotation/label names, hard-coding them in the templates instead.
- The `values.go` files got simplified as a result.
- The `created-by` annotation was also refactored into a reusable partial. This means we had to add a `partials` dependency to multicluster.
2021-02-19 09:17:45 -05:00
Dennis Adjei-Baah 89ec6056c0
Report extension usage (#5761)
This change modifies `linkerd-heartbeat` to report on all linkerd
extensions being used in a cluster.

The heartbeat URL now includes extension names and a value of 
`"1"` if an extension is installed. 
```
https://versioncheck.linkerd.io/version.json?install-time=1613518301&k8s-version=v1.20.2&linkerd-jaeger=1&linkerd-multicluster=1&linkerd-viz=1
```

Fixes #5516

Signed-off-by: Dennis Adjei-Baah <dennis@buoyant.io>
2021-02-18 07:33:35 -08:00
Kevin Leimkuhler edd3812f30
Add services to proxy injector for opaque ports annotation (#5766)
This adds namespace inheritance of the opaque ports annotation to services. 

This means that the proxy injector now watches services creation in a cluster.
When a new service is created, the webhook receives an admission request for
that service and determines whether a patch needs to be created.

A patch is created if the service does not have the annotation, but the
namespace does. This means the service inherits the annotation from the
namespace.

A patch is not created if the service and the namespace do not have the
annotation, or the service has the annotation. In the case of the service having
the annotation, we don't even need to check the namespace since it would not
inherit it anyways.

If a namespace has the annotation value changed, this will not be reflected on
the service. The service would need to be recreated so that it goes through
another admission request.

None of this applies to the `inject` command which still skips service
injection. We rely on being able to check the namespace annotations, and this is
only possible in the proxy injector webhook when we can query the k8s API.

Closes #5737

Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
2021-02-17 20:58:18 -05:00
Oliver Gould 6dc7efd704
docker: Access container images via cr.l5d.io (#5756)
We've created a custom domain, `cr.l5d.io`, that redirects to `ghcr.io`
(using `scarf.sh`). This custom domain allows us to swap the underlying
container registry without impacting users. It also provides us with
important metrics about container usage, without collecting PII like IP
addresses.

This change updates our Helm charts and CLIs to reference this custom
domain. The integration test workflow now refers to the new domain,
while the release workflow continues to use the `ghcr.io/linkerd` registry
for the purpose of publishing images.
2021-02-17 14:31:54 -08:00
Tarun Pothulapati 4a6aa59673
viz: support default* fields for global overrides (#5694)
Fixes #5685

Currently, YAML anchors are not supported through Helm Values
fields. These are required for the default* flags which are
used to override flags across components.

For them to work, YAML anchors had to be removed and rely on
default function directly in the tempalte files.

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2021-02-17 08:00:33 -08:00
Alejandro Pedraza 826d579924
Upgrade proxy-init to v1.3.9 (#5759)
Fixes #5755 follow-up to #5750 and #5751

- Unifies the Go version across Docker and CI to be 1.14.15;
- Updates the GitHub Actions base image from ubuntu-18.04 to ubuntu-20.04; and
- Updates the runtime base image from debian:buster-20201117-slim to debian:buster-20210208-slim.
2021-02-16 17:59:28 -05:00
Jimil Desai 4e5bbb9433
templates: add 443 to skip ports if not present (#5638)
Fixes #5616  

Changed control-plane templates i.e controller, identity, destination, proxy-injector and sp-validator to add 443 to skip ports if not present. This prevents failing of linkerd installation when 443 is not present in ignoreOutboundPorts.

Signed-off-by: Jimil Desai <jimildesai42@gmail.com>
2021-02-16 17:11:14 -05:00
Alejandro Pedraza cbdd1cab03
Increase min k8s version to 1.16 (#5741)
... in order to support the bumped CRD and webhook config versions made
in #5603. In #5688 we asked if there were any concerns. None so far.
2021-02-15 13:03:14 -05:00
Tarun Pothulapati cb6c1fce03
viz: make prom checks dynamic by using annotations (#5680)
Fixes #5652 

This PR adds new annotation that is added when a
external Prometheus is used. Based on that
annotations, The CLI can get to know if an external instance
is being used and if the annotation is absent, that the
the default instance is present.

This updates the viz Checks to skip some checkers if the default
 Prometheus instances are absent.

This PR also removes the grafana checks as they are not useful
and add unnecessary complexity.

This also cleans up some `grafanaUrl` stuff from the core
control-plane chart.

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2021-02-12 21:25:42 +05:30
Tarun Pothulapati a393c42536
values: removal of .global field (#5699)
* values: removal of .global field

Fixes #5425

With the new extension model, We no longer need `Global` field
as we don't rely on chart dependencies anymore. This helps us
further cleanup Values, and make configuration more simpler.

To make upgrades and the usage of new CLI with older config work,
We add a new method called `config.RemoveGlobalFieldIfPresent` that
is used in the upgrade and `FetchCurrentConfiguration` paths to remove
global field and attach its child nodes if global is present. This is verified
by the `TestFetchCurrentConfiguration`'s older test that has the global
field.

We also don't yet remove .global in some helm stable-upgrade tests for
the initial install to work.

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2021-02-11 23:38:34 +05:30
Mayank Shah 96e078421c
CLI: Remove the `--disable-tap` flag from inject (#5671)
Fixes https://github.com/linkerd/linkerd2/issues/5664

- Remove `--disable-flag` from `inject`
-  Move `config.linkerd.io/disable-tap` to `viz.linkerd.io/disable-tap`

Signed-off-by: Mayank Shah <mayankshah1614@gmail.com>
2021-02-11 10:01:53 -05:00
Alejandro Pedraza e887cc79ea
Replace Addons section with Extensions in main chart README (#5714)
Fixes #5704
2021-02-11 09:20:27 -05:00
Oliver Gould 8f2d01c5c0
Use fully-qualified DNS names in proxy configuration (#5707)
Pods with unusual DNS configurations may not be able to resolve the
control plane's domain names. We can avoid search path shenanigans by
adding a trailing dot to these names.
2021-02-10 08:27:35 -08:00
Kevin Leimkuhler 75fcc9d623
Move tap from core into Viz extension (#5651)
Closes #5545.

This change moves all tap and tap-injector code into the viz directory. 

The tap and tap-injector components now also use a new tap image—separating
these components from the controller image that they are currently part of. This
means the controller image has removed all its build dependencies related to
tap.

Finally, the tap Protobuf has been separated from the metrics-api and moved into
it's own `.proto` file and gen directory. This introduces a clear split between
metrics-api and tap Protobuf.

There is no change in behavior for the `viz tap` command.

### Reviewing

#### Docker images

All the bin directory scripts should be updated to build and load the tap image.
All the CI workflows should be updated to build and push the tap image.

#### Controller and pkg directories

This is primarily deletions. Most of the deleted code in this directory is now
in the tap directory of the Viz extension.

#### viz/tap

This is the location that all the tap related code now lives in. New files are
mostly moved from the controller and pkg directories. Imports have all been
updated to point at the right locations and Protobuf.

The Protobuf here is taken from metrics-api and contains all tap-related
Protobuf.

Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
2021-02-09 12:43:21 -05:00
Takumi Sue 77add64860
Remove extra three dashes from helm templates (#5628)
(Background information)
In our company we are checking the sops-encrypted Linkerd manifest into GitHub repository,
and I came across the following problem.

---

Three dashes mean the start of the YAML document (or the end of the
directive).
https://yaml.org/spec/1.2/spec.html#id2800132

If there are only comments between `---`, the document is empty.
Assume the file which include an empty document at the top of itself.

```yaml
---
# foo
---
apiVersion: v1
kind: Namespace
metadata:
  name: foo
---
# bar
---
apiVersion: v1
kind: Namespace
metadata:
  name: bar
```

When we encrypt and decrypt it with [sops](https://github.com/mozilla/sops), the empty document will be
converted to `{}`.

```yaml
{}
---
apiVersion: v1
kind: Namespace
metadata:
    name: foo
---
apiVersion: v1
kind: Namespace
metadata:
    name: bar
```

It is invalid as k8s manifest ([apiVersion not set, kind not set]).

```
error validating data: [apiVersion not set, kind not set]
```

---

I'm afraid that it's sops's problem (at least partly), but anyhow this modification is enough harmless I think.
Thank you.

Signed-off-by: Takumi Sue <u630868b@alumni.osaka-u.ac.jp>
2021-02-01 10:51:34 -05:00
Matei David 0ce9e84a94
Introduce V1 to CRDs and Mutating Hooks (#5603)
*Closes #5484*
 ### Changes
---
*Overview*:
 * Update golden files and make necessary spec changes
 * Update test files for viz
 * Add v1 to healthcheck and uninstall
 * Fix link-crd clusterDomain field validation

- To update to v1, I had to change crd schemas to be version-based (i.e each version has to declare its own schema). I noticed an error in the link-crd (`targetClusterDomain` was `targetDomainName`). Also, additionalPrinterColumns are also version-dependent as a field now.

- For `admissionregistration` resources I had to add an additional `admissionReviewVersions` field -- I included `v1` and `v1beta1`.

- In `healthcheck.go` and `resources.go` (used by `uninstall`) I had to make some changes to the client-go versions (i.e from `v1beta1` to `v1` for admissionreg and apiextension) so that we don't see any warning messages when uninstalling or when we do any install checks. 

I tested again different cli and k8s versions to have a bit more confidence in the changes (in addition to automated tests), hope the cases below will be enough, if not let me know and I can test further.

### Tests

Linkerd local build CLI + k8s 1.19+
`install/check/mc-check/mc-install/mc-link/viz-install/viz-check/uninstall/`
```
$ kubectl version
Server Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.2+k3s1", GitCommit:"1d4adb0301b9a63ceec8cabb11b309e061f43d5f", GitTreeState:"clean", BuildDate:"2021-01-14T23:52:37Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}

$ bin/linkerd version
Client version: git-b0fd2ec8
Server version: unavailable

$ bin/linkerd install | kubectl apply -f -
- no errors, no version warnings - 

$ bin/linkerd check --expected-version git-b0fd2ec8
Status check results are :tick:

# MC

$ bin/linkerd mc install | k apply -f - 
- no erros, no version warnings - 

$ bin/linkerd mc check
Status check results are :tick:

$ bin/linkerd mc link foo | k apply -f -   # test crd creation
# had a validation error here because the schema had targetDomainName instead of targetClusterDomain
# changed, rebuilt cli, re-installed mc, tried command again
secret/cluster-credentials-foo created
link.multicluster.linkerd.io/foo created
...

# VIZ
$ bin/linkerd viz install | k apply -f - 
- no errors, no version warnings - 

$ bin/linkerd viz check 
- no errors, no version warnings - 
Status check results are :tick:

$ bin/linkerd uninstall | k delete -f -
- no errors, no version warnings - 
```

Linkerd local build CLI + k8s 1.17
`check-pre/install/mc-check/mc-install/mc-link/viz-install/viz-check`
```
$ kubectl version
Server Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.17-rc1+k3s1", GitCommit:"e8c9484078bc59f2cd04f4018b095407758073f5", GitTreeState:"clean", BuildDate:"2021-01-14T06:20:56Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"linux/amd64"}

$ bin/linkerd version
Client version: git-3d2d4df1 # made changes to link-crd after prev test case
Server version: unavailable

$ bin/linkerd check --pre --expected-version git-3d2d4df1
- no errors, no version warnings -
Status check results are :tick:

$ bin/linkerd install | k apply -f -
- no errors, no version warnings -

$ bin/linkerd check --expected-version git-3d2d4df1
- no errors, no version warnings - 
Status check results are :tick:

$ bin/linkerd mc install | k apply -f -
- no errors, no version warnings - 

$ bin/linkerd mc check 
- no errors, no version warnings - 
Status check results are :tick:

$ bin/linkerd mc link --cluster-name foo | k apply -f -
bin/linkerd mc link --cluster-name foo | k apply -f -
secret/cluster-credentials-foo created
link.multicluster.linkerd.io/foo created

# VIZ

$ bin/linkerd viz install | k apply -f - 
- no errors, no version warnings - 

$ bin/linkerd viz check
- no errors, no version warnings -
- hangs up indefinitely after linkerd-viz can talk to Kubernetes
```

Linkerd edge (21.1.3) CLI + k8s 1.17 (already installed)
`check`
```
$ linkerd version
Client version: edge-21.1.3
Server version: git-3d2d4df1

$ linkerd check
- no errors -
- warnings: mismatch between cli & control plane, control plane not up to date (both expected) -
Status check results are :tick:
```

Linkerd stable (2.9.2) CLI + k8s 1.17 (already installed)
`check/uninstall`
```
$ linkerd version
Client version: stable-2.9.2
Server version: git-3d2d4df1

$ linkerd check
× control plane ClusterRoles exist
    missing ClusterRoles: linkerd-linkerd-tap
    see https://linkerd.io/checks/#l5d-existence-cr for hints

Status check results are ×

# viz wasn't installed, hence the error, installing viz didn't help since
# the res is named `viz-tap` now
# moving to uninstall

$ linkerd uninstall | k delete -f -
- no warnings, no errors - 
```

_Note_: I used `go test ./cli/cmd/... --generate` which is why there are so many changes 😨 

Signed-off-by: Matei David <matei.david.35@gmail.com>
2021-02-01 09:18:13 -05:00
Alejandro Pedraza 8ac5360041
Extract from public-api all the Prometheus dependencies, and moves things into a new viz component 'linkerd-metrics-api' (#5560)
* Protobuf changes:
- Moved `healthcheck.proto` back from viz to `proto/common` as it remains being used by the main `healthcheck.go` library (it was moved to viz by #5510).
- Extracted from `viz.proto` the IP-related types and put them in `/controller/gen/common/net` to be used by both the public and the viz APIs.

* Added chart templates for new viz linkerd-metrics-api pod

* Spin-off viz healthcheck:
- Created `viz/pkg/healthcheck/healthcheck.go` that wraps the original `pkg/healthcheck/healthcheck.go` while adding the `vizNamespace` and `vizAPIClient` fields which were removed from the core `healthcheck`. That way the core healthcheck doesn't have any dependencies on viz, and viz' healthcheck can now be used to retrieve viz api clients.
- The core and viz healthcheck libs are now abstracted out via the new `healthcheck.Runner` interface.
- Refactored the data plane checks so they don't rely on calling `ListPods`
- The checks in `viz/cmd/check.go` have been moved to `viz/pkg/healthcheck/healthcheck.go` as well, so `check.go`'s sole responsibility is dealing with command business. This command also now retrieves its viz api client through viz' healthcheck.

* Removed linkerd-controller dependency on Prometheus:
- Removed the `global.prometheusUrl` config in the core values.yml.
- Leave the Heartbeat's `-prometheus` flag hard-coded temporarily. TO-DO: have it automatically discover viz and pull Prometheus' endpoint (#5352).

* Moved observability gRPC from linkerd-controller to viz:
- Created a new gRPC server under `viz/metrics-api` moving prometheus-dependent functions out of the core gRPC server and into it (same thing for the accompaigning http server).
- Did the same for the `PublicAPIClient` (now called just `Client`) interface. The `VizAPIClient` interface disappears as it's enough to just rely on the viz `ApiClient` protobuf type.
- Moved the other files implementing the rest of the gRPC functions from `controller/api/public` to `viz/metrics-api` (`edge.go`, `stat_summary.go`, etc.).
- Also simplified some type names to avoid stuttering.

* Added linkerd-metrics-api bootstrap files. At the same time, we strip out of the public-api's `main.go` file the prometheus parameters and other no longer relevant bits.

* linkerd-web updates: it requires connecting with both the public-api and the viz api, so both addresses (and the viz namespace) are now provided as parameters to the container.

* CLI updates and other minor things:
- Changes to command files under `cli/cmd`:
  - Updated `endpoints.go` according to new API interface name.
  - Updated `version.go`, `dashboard` and `uninstall.go` to pull the viz namespace dynamically.
- Changes to command files under `viz/cmd`:
  - `edges.go`, `routes.go`, `stat.go` and `top.go`: point to dependencies that were moved from public-api to viz.
- Other changes to have tests pass:
  - Added `metrics-api` to list of docker images to build in actions workflows.
  - In `bin/fmt` exclude protobuf generated files instead of entire directories because directories could contain both generated and non-generated code (case in point: `viz/metrics-api`).

* Add retry to 'tap API service is running' check

* mc check shouldn't err when viz is not available. Also properly set the log in multicluster/cmd/root.go so that it properly displays messages when --verbose is used
2021-01-21 18:26:38 -05:00
Matei David c63fbdf0e4
Introduce OpenAPIV3 validation for CRDs (#5573)
* Introduce OpenAPIV3 validation for CRDs

* Add validation to link crd
* Add validation to sp using kube-gen
* Add openapiv3 under schema fields in specific versions
* Modify fields to rid spec of yaml errors
* Add top level validation for all three CRDs

Signed-off-by: Matei David <matei.david.35@gmail.com>
2021-01-21 11:56:28 -05:00
Kevin Leimkuhler e7f2a3fba3
viz: add tap-injector (#5540)
## What this changes

This adds a tap-injector component to the `linkerd-viz` extension which is
responsible for adding the tap service name environment variable to the Linkerd
proxy container.

If a pod does not have a Linkerd proxy, no action is taken. If tap is disabled
via annotation on the pod or the namespace, no action is taken.

This also removes the environment variable for explicitly disabling tap through
an environment variable. Tap status for a proxy is now determined only be the
presence or absence of the tap service name environment variable.

Closes #5326

## How it changes

### tap-injector

The tap-injector component determines if `LINKERD2_PROXY_TAP_SVC_NAME` should be
added to a pod's Linkerd proxy container environment. If the pod satisfies the
following, it is added:

- The pod has a Linkerd proxy container
- The pod has not already been mutated
- Tap is not disabled via annotation on the pod or the pod's namespace

### LINKERD2_PROXY_TAP_DISABLED

Now that tap is an extension of Linkerd and not a core component, it no longer
made sense to explicitly enable or disable tap through this Linkerd proxy
environment variable. The status of tap is now determined only be if the
tap-injector adds or does not add the `LINKERD2_PROXY_TAP_SVC_NAME` environment
variable.

### controller image

The tap-injector has been added to the controller image's several startup
commands which determines what it will do in the cluster.

As a follow-up, I think splitting out the `tap` and `tap-injector` commands from
the controller image into a linkerd-viz image (or something like that) makes
sense.

Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
2021-01-21 11:24:08 -05:00
Eugene Formanenko 535a36af7c
Add log-format flag to control plane components (#5537)
Fixes #5536

Signed-off-by: Eugene Formanenko <mo4islona@gmail.com>
2021-01-15 10:51:32 -05:00
Alejandro Pedraza dd9ea0aef4
Helm template helpers cleanup (#5514)
Removed Helm template files no longer used, as well as some helper
functions.
2021-01-14 09:05:31 -05:00
Alejandro Pedraza a9317af3d8
Add back support for proxy resource settings (#5517)
The last viz refactoring removed support for modifying the k8s resources
used by the proxies injected into the control plane components (values
like `tapProxyResources`, `prometheus.proxy.resources`, etc).

This adds them back, using a consistent naming: `tap.proxy.resources`,
`dashboard.proxy.resources`, etc.

Also fixes the tap helm template that was making reference to
`.Values.tapResources` instead of `.Values.tap.resources`.

Co-authored-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
2021-01-12 10:56:43 -05:00
Mitch Hulscher 462fe32ef2
fix(linkerd2-cni): execute container preStop command `kill` command as shell builtin (#5453)
The container-image `ghcr.io/linkerd/cni-plugin:stable-2.9.1` does not contain the `kill` command as an executable. Instead, it is available as a shell built-in. In its current state, Kubernetes emits error events whenever linkerd2-cni pods are terminated because the `kill` command can not be found.

Signed-off-by: Mitch Hulscher <mitch.hulscher@lib.io>
2021-01-07 10:24:24 -05:00
Raphael Taylor-Davies c9d789156c
Add PodDisruptionBudgets to control plane (#5398) (#5406)
Closes #5398

* Add PodDisruptionBudget to controller deployments
* Add .yaml to editorconfig

Signed-off-by: Raphael Taylor-Davies <r.taylordavies@googlemail.com>
2021-01-06 09:19:15 -05:00
Tarun Pothulapati 2087c95dd8
viz: move some components into linkerd-viz (#5340)
* viz: move some components into linkerd-viz

This branch moves the grafana,prometheus,web, tap components
into a new viz chart, following the same extension model that
multi-cluster and jaeger follow.

The components in viz are not injected during install time, and
will go through the injector. The `viz install` does not have any
cli flags to customize the install directly but instead follow the Helm
way of customization by using flags such as 
`set`, `set-string`, `values`, `set-files`.

**Changes Include**
- Move `grafana`, `prometheus`, `web`, `tap` templates into viz extension.
- Remove all add-on related charts, logic and tests w.r.t CLI & Helm.
- Clean up `linkerd2/values.go` & `linkerd2/values.yaml` to not contain
 fields related to viz components.
- Update `linkerd check` Healthchecks to not check for viz components.
- Create a new top level `viz` directory with CLI logic and Helm charts.
- Clean fields in the `viz/Values.yaml` to be in the `<component>.<property>`
model. Ex: `prometheus.resources`, `dashboard.image.tag`, etc so that it is
consistent everywhere.

**Testing**

```bash
# Install the Core Linkerd Installation
./bin/linkerd install | k apply -f -

# Wait for the proxy-injector to be ready
# Install the Viz Extension
./bin/linkerd cli viz install | k apply -f -

# Customized Install
./bin/linkerd cli viz install --set prometheus.enabled=false | k apply -f -
```

What is not included in this PR:
- Move of Controller from core install into the viz extension.
- Simplification and refactoring of the core chart i.e removing `.global`, etc.

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2020-12-23 20:17:31 +05:30
Alejandro Pedraza 557f3a9f60
Remove tracing from linkerd's helm chart requirements.lock (#5411)
This avoids `bin/helm-build` to return a lint error
2020-12-21 10:40:25 -05:00
Kevin Leimkuhler 7c0843a823
Add opaque ports to destination service updates (#5294)
## Summary

This changes the destination service to start indicating whether a profile is an
opaque protocol or not.

Currently, profiles returned by the destination service are built by chaining
together updates coming from watching Profile and Traffic Split updates.

With this change, we now also watch updates to Opaque Port annotations on pods
and namespaces; if an update occurs this is now included in building a profile
update and is sent to the client.

## Details

Watching updates to Profiles and Traffic Splits is straightforward--we watch
those resources and if an update occurs on one associated to a service we care
about then the update is passed through.

For Opaque Ports this is a little different because it is an annotation on pods
or namespaces. To account for this, we watch the endpoints that we should care
about.

### When host is a Pod IP

When getting the profile for a Pod IP, we check for the opaque ports annotation
on the pod and the pod's namespace. If one is found, we'll indicate if the
profile is an opaque protocol if the requested port is in the annotation.

We do not subscribe for updates to this pod IP. The only update we really care
about is if the pod is deleted and this is already handled by the proxy.

### When host is a Service

When getting the profile for a Service, we subscribe for updates to the
endpoints of that service. For any ports set in the opaque ports annotation on
any of the pods, we check if the requested port is present.

Since the endpoints for a service can be added and removed, we do subscribe for
updates to the endpoints of the service.

Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
2020-12-18 12:38:59 -05:00
rish-onesignal 146cd1301d
Add missing tolerations in linkerd-cni helm chart (#5368) (#5369)
The linkerd-cni helm chart is missing tolerations on the daemonset. This
prevents the linkerd-cni daemonset from being installed on all intended
nodes.

We use the same template partial as used in the main linkerd helm chart
to add tolerations if specified to the linkerd-cni daemonset spec.

Fixes #5368

Signed-off-by: Rishabh Jain <rishabh@onesignal.com>
2020-12-17 10:12:31 -05:00
Tarun Pothulapati c19cfd71a1
upgrades: make webhooks restart if TLS creds are updated (#5349)
* upgrades: make webhooks restart if TLS creds are updated

Fixes #5231

Currently, we do not re-use the TLS certs during upgrades, which
means that the secrets are updated while the webhooks are still
paired with the older ones, causing the webhook requests to fail.

This can be solved by making webhooks be restarted whenever there
is a change in the certs. This can be performed by storing the hash
of the `*-rbac` file, which contains the secrets, thus making the
pod templates change whenever there is an update to the certs thus
making restarts required.

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2020-12-10 11:56:53 -05:00
Simon Weald cae4add8d0
Update RBAC API versions to avoid deprecations (#5332)
When testing the `linkerd2-cni` chart with `ct`, it flags up usage
of some deprecated apiVersions.

This PR aligns the RBAC API group across all resources in the chart.

---

Signed-off-by: Simon Weald <glitchcrab-github@simonweald.com>
2020-12-09 15:56:25 -05:00
Alex Leong cdc57d1af0
Use linkerd-jaeger extension for control plane tracing (#5299)
Now that tracing has been split out of the main control plane and into the linkerd-jaeger extension, we remove references to tracing from the main control plane including:

* removing the tracing components from the main control plane chart
* removing the tracing injection logic from the main proxy injector and inject CLI (these will be added back into the new injector in the linkerd-jaeger extension)
* removing tracing related checks (these will be added back into `linkerd jaeger check`)
* removing related tests

We also update the `--control-plane-tracing` flag to configure the control plane components to send traces to the linkerd-jaeger extension.  To make sure this works even when the linkerd-jaeger extension is installed in a non-default namespace, we also add a `--control-plane-tracing-namespace` flag which can be used to change the namespace that the control plane components send traces to.

Note that for now, only the control plane components send traces; the proxies in the control plane do not.  This is because the linkerd-jaeger injector is not yet available.  However, this change adds the appropriate namespace annotations to the control plane namespace to configure the proxies to send traces to the linkerd-jaeger extension once the linkerd-jaeger injector is available.

I tested this by doing the following:

1. bin/linkerd install | kubectl apply -f -
1. bin/helm install jaeger jaeger/charts/jaeger
1. bin/linkerd upgrade --control-plane-tracing=true | kubectl apply -f -
1. kubectl -n linkerd-jaeger port-forward svc/jaeger 16686
1. open http://localhost:16686
1. see traces from the linkerd control plane

Signed-off-by: Alex Leong <alex@buoyant.io>
2020-12-08 14:34:26 -08:00
Tarun Pothulapati 72a0ca974d
extension: Separate multicluster chart and binary (#5293)
Fixes #5257

This branch movies mc charts and cli level code to a new
top level directory. None of the logic is changed.

Also, moves some common types into `/pkg` so that they
are accessible both to the main cli and extensions.

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2020-12-04 16:36:10 -08:00
Björn Wenzel 0ee18eb168
Allow Multicluster Service to be non LoadBalancer ServiceType (#5307)
Signed-off-by: Björn Wenzel <bjoern.wenzel@dbschenker.com>
2020-12-03 13:03:49 -05:00
Alejandro Pedraza 94574d4003
Add automatic readme generation for charts (#5316)
* Add automatic readme generation for charts

The current readmes for each chart is generated
manually and doesn't contain all the information available.

Utilize helm-docs to automatically fill out readme.mds
for the helm charts by pulling metadata from values.yml.

Fixes #4156

Co-authored-by: GMarkfjard <gabma047@student.liu.se>
2020-12-02 14:37:45 -05:00
Alejandro Pedraza 9cbfb08a38
Bump proxy-init to v1.3.8 (#5283) 2020-11-27 09:07:34 -05:00
Takumi Sue 53afc7dbc4
Fix an odd indent (and test data) (#5262)
Signed-off-by: Takumi Sue <u630868b@alumni.osaka-u.ac.jp>
2020-11-20 09:34:40 -05:00
Alejandro Pedraza 5a707323e6
Update proxy-init to v1.3.7 (#5221)
This upgrades both the proxy-init image itself, and the go dependency on
proxy-init as a library, which fixes CNI in k3s and any host using
binaries coming from BusyBox, where `nsenter` has an
issue parsing arguments (see rancher/k3s#1434).
2020-11-13 15:59:14 -05:00
Shai Katz 4ffb41ab44
Add MC Gateway Service Annotations (#5185)
Fixes #5182

That will allow setting the load balancer as internal.

Signed-off-by: Shai Katz <shai@soluto.com>
2020-11-12 11:19:19 -08:00
Oliver Gould 4d85b6cd65
inject: Set LINKERD2_PROXY_CORES from the cpu limit (#5170)
Per #5165, Kubernetes does not necessarily limit the proxy's access to
cores via `cgroups` when a CPU limit is set. As of #5168, the proxy now
supports a `LINKERD2_PROXY_CORES` environment configuration that
augments CPU detection from the host operating system.

This change modifies the proxy injector to ensure that this environment
is configured from the `Values.proxy.cores` Helm value, the
`config.linkerd.io/proxy-cpu-limit` annotation, and the `--proxy-cpu-limit`
install flag.
2020-11-03 10:02:31 -08:00