Commit Graph

747 Commits

Author SHA1 Message Date
Matei David a2bd230cd6
service topologies: add Kubernetes/API EndpointSlice support (#4696)
Based on the [EndpointSlice PR](https://github.com/linkerd/linkerd2/pull/4663), this is just the k8s/api support for endpointslices to shorten the first PR.

* Adds CRD
* Adds functions that check whether the cluster has EndpointSlice access
* Adds discovery & endpointslice informers to api.

Signed-off-by: Matei David <matei.david.35@gmail.com>
2020-07-06 15:28:48 -07:00
Tarun Pothulapati 7cd188dc65
Add `values.yaml` to chart tempaltes (#4682)
This change adds add-on level values.yaml into the template to use for
rendering. No changes as of rn, but will be used by add-ons later on!
2020-07-02 22:59:41 +05:30
Tarun Pothulapati fcc3eb5411
cli: support url with addon-config flag (#4666)
adds support for urls through addon-config flag
2020-07-02 22:57:22 +05:30
Tarun Pothulapati c3131cde0e
Use cniPluginVersion with Helm for linkerd2-cni (#4693)
use `cniPluginVersion` as the fall-back version with Helm
2020-07-02 20:45:13 +05:30
Naseem 361d35bb6a
feat: add log format annotation and helm value (#4620)
* feat: add log format annotation and helm value

Json log formatting has been added via https://github.com/linkerd/linkerd2-proxy/pull/500
but wiring the option through as an annotation/helm value is still
necessary.

This PR adds the annotation and helm value to configure log format.

Closes #2491

Signed-off-by: Naseem <naseem@transit.app>
2020-07-02 10:08:52 -05:00
Arthur Silva Sens 021048d576
GoDocs for completion, dashboard and diagnostics cli commands (#4518)
Signed-off-by: arthursens <arthursens2005@gmail.com>
2020-06-30 05:53:50 -05:00
Naseem 733d911677
feat: add option to persist prometheus data (#4578)
Data disappears upon prometheus restarts due to it being all in-memory.

Adding an option to enabled persistence by means of a PVC  would be the right approach. It is commonly seen in a wide array of helm charts.

Fixes #4576

Signed-off-by: Naseem <naseem@transit.app>
2020-06-29 14:26:26 -07:00
Alejandro Pedraza aea541d6f9
Upgrade generated protobuf files to v1.4.2 (#4673)
Regenerated protobuf files, using version 1.4.2 that was upgraded from
1.3.2 with the proxy-api update in #4614.

As of v1.4 protobuf messages are disallowed to be copied (because they
hold a mutex), so whenever a message is passed to or returned from a
function we need to use a pointer.

This affects _mostly_ test files.

This is required to unblock #4620 which is adding a field to the config
protobuf.
2020-06-26 09:36:48 -05:00
Mayank Shah 2b0482c821
Update `inject` to throw an error while injecting non-compliant pods (#4346)
* Update inject to error out on failure

Update injection process to throw an error when the reason for failure is due to sidecar, udp, automountServiceAccountToken or hostNetwork

Signed-off-by: Mayank Shah <mayankshah1614@gmail.com>
2020-06-24 14:07:05 -05:00
Zahari Dichev 7f3d872930
Add destination-get-networks option (#4608)
In #4585 we are observing an issue where a loop is encountered when using nginx ingress. The problem is that the outbound proxy does a dst lookup on the IP address which happens to be the very same address the ingress is listening on.

In order to avoid situations like that this PR introduces a way to modify the set of networks for which the proxy shall do IP based discovery. The change introduces a helm flag `.Values.global.proxy.destinationGetNetworks` that can be used to modify this value. There are two ways a user can affect the this setting: 


- setting the `destinationGetNetworks` field in values during a Helm install, which changes the default on all injected pods
- using an annotation ` config.linkerd.io/proxy-destination-get-networks` for injected workloads to override this value

Note that this setting cannot be tweaked through the `install` or `inject` command

Fix: #4585

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
2020-06-18 20:07:47 +03:00
cpretzer b176fbeb6d
Upgrade Grafana to 7.0.3 (#4600)
* Upgrade Grafana to 7.0.3
* use go netdns to avoid DNS resolution errors on alpine

Signed-off-by: Charles Pretzer <charles@buoyant.io>
2020-06-17 21:35:29 -07:00
Kevin Leimkuhler fe71ef04b0
Remove `--prune` from multi-stage upgrade docs (#4613)
Fixes #4606

This has not worked as far back as stable-2.6.0.

## Solution

The recommended upgrade process is to include `--prune` as part of `kubectl
apply ..`:

```bash
$ linkerd upgrade | kubectl apply --prune -l linkerd.io/control-plane-ns=linkerd -f -
```

This is an issue for multi-stage upgrade because `linkerd upgrade config` does
not include the `linkerd-config` ConfigMap in it's output.

`kubectl apply --prune ..` will then prune this resource because it matches the
label selector *and* is not in the above output.

The issue occurs when `linkerd upgrade control-plane` is run and expects to find
the ConfigMap that was just pruned.

This can be fixed by not suggesting to prune resources as part of the
multi-stage upgrade.

## Considered

Including `templates/config.yaml` in the install output regardless of the stage.
Instead of it being a template only used in `control-plane` stage in
[render](4aa3ca7f87/cli/cmd/install.go (L873-L886)), it could always be rendered.

This just exposes other things that are pruned in the process:

```bash
❯ bin/linkerd upgrade control-plane |kubectl apply --prune -l linkerd.io/control-plane-ns=linkerd -f -
× Failed to build upgrade configuration: secrets "linkerd-identity-issuer" not found
For troubleshooting help, visit: https://linkerd.io/upgrade/#troubleshooting

error: no objects passed to apply
```

Ultimately, resources part of the `control-plane` stage need to remain and that
will not happen if we prune all resources not in the `config` stage output

Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
2020-06-17 10:29:34 -04:00
Alexander Berger b509742c7d
Fix ClusterRole for web-check (#4599)
As reported in #4259 linkerd check run from linkerd's web cconsole is
broken as the underlying RBAC Role cannot access the apiregistration.k8s.io API Group.

With this commit the RBAC Role is fixed allowing read-only access to the API Group
apiregistration.k8s.io.

Fixes #4259

Signed-off-by: alex.berger@nexiot.ch <alex.berger@nexiot.ch>
2020-06-15 10:21:00 -07:00
Joakim Roubert 99a9f1c2c2
Fix missing proxy-init v1.3.2 -> v1.3.3 (#4596)
Signed-off-by: Joakim Roubert <joakim.roubert@axis.com>
2020-06-15 10:53:01 -05:00
Alejandro Pedraza 7a9527bf00
Fix yaml in linkerd-config-addons when providing grafanaUrl (#4581)
Put back space after `grafanaUrl` label in `linkerd-config-addons.yaml`
to avoid breaking the yaml parsing.

```
$ linkerd check
...
linkerd-addons
--------------
‼ 'linkerd-config-addons' config map exists
    could not unmarshal linkerd-config-addons config-map: error
    unmarshaling JSON: while decoding JSON: json: cannot unmarshal
    string into Go struct field Values.global of type linkerd2.Global
```
This was added in #4544 to avoid having the configmap being badly formatted.

So this PR fixes the yaml, but then if we don't set `grafanaUrl` the
configmap format gets messed up, but apparently that's just a cosmetic
problem:

```
apiVersion: v1
data:
  values: "global:\n  grafanaUrl: \ngrafana:\n  enabled: true\n
  image:\n    name:
      gcr.io/linkerd-io/grafana\n  name: linkerd-grafana\n  resources:\n
      cpu:\n      limit:
          240m\n    memory:\n      limit: null\ntracing:\n  enabled:
          false"
          kind: ConfigMap
```
2020-06-09 11:08:32 -07:00
Tarun Pothulapati 4219955bdb
multicluster: checks for misconfigured mirror services (#4552)
Fixes #4541 

This PR adds the following checks
-  if a mirrored service has endpoints. (This includes gateway mirrors too).
-  if an exported service is referencing a gateway that does not exist.

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
Signed-off-by: Alex Leong <alex@buoyant.io>

Co-authored-by: Alex Leong <alex@buoyant.io>
2020-06-08 15:29:34 -07:00
Tarun Pothulapati 4aa3ca7f87
remove grafana.image.version field (#4571)
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2020-06-08 17:15:30 -05:00
Matei David 654839e639
Add namespace global flag to hold default namespace name (#4469) (#4512)
* Add namespace global flag to hold default namespace name (#4469)

Signed-off-by: Matei David <matei.david.35@gmail.com>

* Change name of controlplane install namespace constant and init point for kubeNamespace

Signed-off-by: Matei David <matei.david.35@gmail.com>
2020-06-04 10:45:07 -07:00
Lutz Behnke 108b383ab8
add flag for dumping rejected results to folder for use with external diff (#4509)
Problem
When updating / writing tests with complex data, e.g the certificates, the build-in diff is not as powerful as dedicated external tool.

Solution
Dump all resource specifications created as part of failing tests to a supplied folder for external analysis.

Signed-off-by: Lutz Behnke <lutz.behnke@finleap.com>
2020-06-04 10:49:41 -04:00
Alejandro Pedraza 7d9525e316
Removed trailing spaces from entries in configmaps (#4544)
Fixes #4454

As explained
[here](https://github.com/kubernetes/kubernetes/issues/36222#issuecomment-553966166),
trailing spaces in configmap data makes it to look funky when retrieved
later on. This is currently affecting `linkerd-config-addons` and
`linkerd-gateway-config`:

```
$ k -n linkerd-multicluster get cm linkerd-gateway-config -oyaml
apiVersion: v1
data:
  nginx.conf: "events {\n}\nstream {                                                                                                                                                                                  \n
    \  server {                                                                                                                                                                                \n
    \      listen     4180;                                                                                                                                                 \n
    \      proxy_pass 127.0.0.1:4140;                                                                                                                                 \n
    \  }                                                                                                                                                                                       \n}
    \nhttp {\n  server {\n      listen     4181;\n      location /health {\n        access_log
    off;\n        return 200 \"healthy\\n\";\n      }\n  }\n  server {\n      listen
    \    8888;\n      location /health-local {\n        access_log off;\n        return
    200 \"healthy\\n\";\n      }\n  }    \n}"
kind: ConfigMap
```

AFAIK this is only cosmetic and doesn't affect functionality.
2020-06-04 09:06:37 -05:00
Alejandro Pedraza ed5482ac3b
Fixed prom route in linkerd service profile, and some extra cleanup (#4493)
* Fixes #4305

Fixed SP route for `POST /api/v1/query`:

```
$ bin/linkerd routes -n linkerd deploy/linkerd-prometheus
ROUTE                                SERVICE   SUCCESS      RPS   LATENCY_P50   LATENCY_P95   LATENCY_P99
GET /api/v1/query_range   linkerd-prometheus   100.00%   3.9rps           1ms           2ms           2ms
GET /api/v1/series        linkerd-prometheus   100.00%   1.1rps           1ms           1ms           1ms
POST /api/v1/query        linkerd-prometheus   100.00%   3.1rps           1ms          17ms          19ms
[DEFAULT]                 linkerd-prometheus         -        -             -             -             -
```

Also added one missing route for `linkerd-grafana`, realizing afterwards there are
many other ones missing, but not really worth adding them all.

I also removed the routes in `linkerd-controller` for the tap routes
given that's no longer handled in that service.

And the tap service SP was also removed alltogether since nothing was
getting reported.
2020-06-03 12:53:50 -05:00
Lutz Behnke 163107b8cb
Extend Helm chart to allow disabling secret resources for self-signed certs. (#4289)
* allow disabling secret resources for self-signed certs. Split cert and ca bundle.

Signed-off-by: Lutz Behnke <lutz.behnke@finleap.com>
2020-06-03 09:26:24 -05:00
Kevin Leimkuhler 8f6186f9ae
Change help text to use source/target terminology in multicluster CLI (#4523)
Change terminology from local/remote to source/target in `multicluster` CLI help
text.

This does not change any variable, function, struct, or field names since
testing is still improving.

Relevant issue: #4480

Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
2020-06-02 12:33:18 -04:00
Zahari Dichev 6c3922a7f1
Probe manager simplification (#4510)
There are a few notable things happening in this PR: 

- the probe manager has been decoupled from the cluster_watcher. Now its only responsibility is to watch for mirrored gateways beeing created and to probe them. This means that probes are initiated for all gateways no matter whether there are mirrored services being paired
- the number of paired services is derived from the existing services in the cluster rather than being published as a metric by the prober
- there are no events being exchanged between the cluster watcher and the probe manager

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
2020-06-01 14:41:29 -07:00
Alejandro Pedraza 571626d524
CI: properly report errors from commands (#4514)
Failures in `bin/_test-run` from commands different than `go test`
aren't currently properly reported, in part because CI's bash default is
to have `set -e` which terminates the script and just outputs
`##[error]Process completed with exit code 2.` like
[here](https://github.com/linkerd/linkerd2/pull/4496/checks?check_run_id=720720352#step:14:116)

```
linkerd-existence
-----------------
√ 'linkerd-config' config map exists
√ heartbeat ServiceAccount exist
√ control plane replica sets are ready
× no unschedulable pods
    linkerd-controller-6c77c7ffb8-w8wh5: 0/1 nodes are available: 1 node(s) had taint {node.kubernetes.io/not-ready: }, that the pod didn't tolerate.
    linkerd-destination-6767d88f7f-rcnbq: 0/1 nodes are available: 1 node(s) had taint {node.kubernetes.io/not-ready: }, that the pod didn't tolerate.
    linkerd-grafana-76c76fcfb9-pdhfb: 0/1 nodes are available: 1 node(s) had taint {node.kubernetes.io/not-ready: }, that the pod didn't tolerate.
    linkerd-identity-5bcf97d6c8-q6rll: 0/1 nodes are available: 1 node(s) had taint {node.kubernetes.io/not-ready: }, that the pod didn't tolerate.
    linkerd-prometheus-6b95c56b44-hd9m6: 0/1 nodes are available: 1 node(s) had taint {node.kubernetes.io/not-ready: }, that the pod didn't tolerate.
    linkerd-proxy-injector-58d794ff9-jf7cj: 0/1 nodes are available: 1 node(s) had taint {node.kubernetes.io/not-ready: }, that the pod didn't tolerate.
    linkerd-sp-validator-6c5f999bfb-qg252: 0/1 nodes are available: 1 node(s) had taint {node.kubernetes.io/not-ready: }, that the pod didn't tolerate.
    linkerd-tap-6fdf84fc65-6txvr: 0/1 nodes are available: 1 node(s) had taint {node.kubernetes.io/not-ready: }, that the pod didn't tolerate.
    linkerd-web-8484fbd867-nm8z2: 0/1 nodes are available: 1 node(s) had taint {node.kubernetes.io/not-ready: }, that the pod didn't tolerate.
    see https://linkerd.io/checks/#l5d-existence-unschedulable-pods for hints

Status check results are ×
[error]Process completed with exit code 2.
```

I've made the following changes to `bin/_test-run` to generate better
messages and Github annotations when an error occurs:

- Unset `set -e` so that errors don't immediately exit the script and
don't allow us to properly format the errors.
- Removed many of the `exit_on_err` calls after go test calls because
those output enough information already (they were not being used
anyways in CI because of `set -e`). And instead have `run_test` exit
upon a `go test` error.
- Added `exit_on_err` calls right after non-`go-test` commands to
properly report their failure.
- Refactored the `exit_on_err` function so that it generates a Github
error annotation upon failure.
- Removed `trap` in `install_stable`, since the OS should be able to
handle GC for stuff under `/tmp`.

Also, I've changed the exit 2 code from `linkerd check` when it fails,
to exit code 1.
2020-06-01 15:57:33 -05:00
Alex Leong 015d352f34
Fix array handling in bin/fmt (#4489)
Quoting the list of directories passed to `goimports` was causing the list to be interpreted as a single argument which was stopping `bin/fmt` from working.

Instead, use `read` to split the list of directories into an array.

Also fix up incorrect formatting that has crept in while `bin/fmt` has been broken.

Signed-off-by: Alex Leong <alex@buoyant.io>
2020-06-01 12:10:24 -07:00
Zahari Dichev 7b46682841
Add allow and link commands (#4466)
This change adds a `allow` and `link` commands, effectivelly enabling a cluster to have more than one set of credentials that allow it to be mirrored. 

Fx #4461

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>

Co-authored-by: Alex Leong <alex@buoyant.io>
2020-05-27 14:30:55 -07:00
Alejandro Pedraza 1844fd573b
Unhide multicluster command (#4486)
Unhide multicluster command
2020-05-27 14:22:23 -05:00
Kevin Leimkuhler 4879f07334
cli: rename cluster cli command to multicluster (#4484)
This is @psinghal20's changes in #4462 which is currently failing CI.

Fixes #4456

Description from the original PR:

> This pr renames the `cluster` command in CLI to `multicluster` command. It
> also adds a shorthand `mc` for easy use.
>
> Fixes #4456
>
> Signed-off-by: psinghal20 <psinghal20@gmail.com>

The CI failure doesn't seem to be related to this change, but has only been seen
on forks. Opening this from a non-fork for now to continue investigating.

Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
Co-authored-by: psinghal20 <psinghal20@gmail.com>
2020-05-27 10:39:52 +03:00
Arthur Silva Sens bfedcd5485
Added documentation for alpha cli command (#4412)
Added comments to document several methods and strucs on cmd package. Based on GoDoc guidelines. Focus on alpha cli command

Signed-off-by: arthursens <arthursens2005@gmail.com>
2020-05-26 13:59:56 -07:00
Zahari Dichev 8fb0ea608a
Skip services that are mirrors of remote ones (#4460)
Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
2020-05-22 09:24:59 +03:00
Alex Leong acacf2e023
Add --close-wait-timeout inject flag (#4409)
Depends on https://github.com/linkerd/linkerd2-proxy-init/pull/10

Fixes #4276 

We add a `--close-wait-timeout` inject flag which configures the proxy-init container to run with `privileged: true` and to set `nf_conntrack_tcp_timeout_close_wait`. 

Signed-off-by: Alex Leong <alex@buoyant.io>
2020-05-21 14:14:14 -07:00
Tarun Pothulapati 0c53760094
update golden files with new grafana.image field format (#4455) 2020-05-21 23:05:04 +05:30
Tarun Pothulapati bd60c90e5d
Add addon-overwrite flag (#4377)
provide a `addon-overwrite` flag for upgrades to skip `linkerd-config-addons` and use `--addon-overwrite` if passed or defaults
2020-05-21 21:01:41 +05:30
Tarun Pothulapati 3473db32f8
use "/" for as the FS is virtualised (#4443)
replacing `filepath.join` in the install path in the CLI, as the fs is virtualized
2020-05-21 10:25:14 +05:30
Alex Leong 9cd4557644
Properly show the meshed count for non-selector services (#4446)
When viewing the output of `linkerd stat` for services which do not have a selector (such as services created by the service-mirror, for example) the meshed count column shows the total number which exist, even though the service actually selects no pods at all.

We update the StatSummary implementation to account for services which have no selector.

Additionally, we update the logic of the `--unmeshed` flag.  When the `--unmeshed` flag is not set, we typically skip rows for unmeshed resources because those resources would have no stats.  This is not appropriate to do when the `--from` flag is also set because in this case, metrics are not collected on the target resource but are instead collected on the client-side.  This means that stats can be present, even for unmeshed resources and these resources should still be displayed, even if the `--unmeshed` flag is not set.

Signed-off-by: Alex Leong <alex@buoyant.io>
2020-05-20 10:08:27 -07:00
Tarun Pothulapati be664571c1
Separate grafana image tag in template (#4395)
Separates grafana image field into image.name, image.version and also moves controllerImageVersion to global
2020-05-20 22:27:19 +05:30
Zahari Dichev 31e33d18d3
Enable service mirroring to work in private networks (#4440)
This change creates a gateway proxy for every gateway. This enables the probe worker to leverage the destination service functionality in order to discover the identity of the gateway.

Fix #4411

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
2020-05-20 19:48:36 +03:00
Zahari Dichev 6574f124a7
Restrict Service mirror RBACs (#4426)
This PR introduces a few changes that were requested after a bit of service mirror reviewing.

- we restrict the RBACs so the service mirror controller cannot read secrets in all namespaces but only in the one that it is installed in
- we unify the namespace namings so all multicluster resources are installedi n `linkerd-multicluster` on both clusters
- fixed checks to account for changes

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
2020-05-20 17:08:01 +03:00
Tarun Pothulapati 5f37a9f7fa
Add global.grafanaUrl for linking existing grafana use-case (#4381)
adds global.grafanaUrl for Bring your own Grafana use-case, with configuration in `linkerd-config-addons`
2020-05-20 00:56:31 +05:30
Tarun Pothulapati e91dbda287
Add health checks for grafana add-on (#4321)
* Add health checks for grafana add-on

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* update testCheck command and fixes

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* fix checkContainersRunnning function

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* linting fix

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* update test golden files

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* use hc.ControlPlanePods instead of k8s API

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* use hc.controlPLanePods directly

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* remove unnecessary comments

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* proper comments

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* update pod checks to use retries

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* add values key check

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2020-05-14 23:18:43 +05:30
Zahari Dichev 01894c700f
Make export-service handle k8s lists (#4370)
Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
2020-05-12 09:11:36 +03:00
Tarun Pothulapati 45ccc24a89
Move grafana templates into a separate sub-chart as a add-on (#4320)
* adds grafana manifests as a sub-chart

- moves grafana templates into its own chart
- implement add-on interface Grafana struct
- also add relevant conditions for grafana

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* remove redundant grafana fields in Values

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* update golden files

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* fix values issue

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* remove extra grafanaImage value

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* add add-on upgrade tests

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* fix golden file tests

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* add grafana field to linkerd-config-addons

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* Don't apply nil configuration

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* update golden files

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* make checks relaxed for grafana

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* update test to not test on grafana

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* update TestServiceAccountsMatch to contain extra members

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* replace map[string]interface{} with Grafana for better readability

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* update golden files

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2020-05-11 22:22:14 +05:30
Zahari Dichev edd9b654a7
Make gateway require TLS for incoming requests (#4339)
Make gateway require TLS for incoming requests

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
2020-05-11 10:07:48 +03:00
Zahari Dichev 3008f1f87f
Add check for validating that remote clusters share the same trust an… (#4311)
Add check for validating that remote clusters share the same trust anchors

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
2020-05-11 09:59:15 +03:00
Tarun Pothulapati 2be43a5f9d
Add Jaeger links to the Linkerd dashboard (#4177)
* Add Jaeger reverse proxy
* add jaegerLink to the metrics table
* update MetricsTable tests
* Add optional jaeger link
* rename grafana_proxy to reverse_proxy

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2020-05-07 16:35:56 -05:00
Alex Leong a703b5b1dc
Hide development flags in release builds (#4310)
Certain install flags are intended to help with Linkerd development and generally are not useful (and are potentially confusing) to users.

We hide these flags in release (edge or stable) builds of the CLI but show them in all other builds.  The list of affected flags is:

* control-plane-version
* proxy-image
* proxy-version
* image-pull-policy
* init-image
* init-image-version

Signed-off-by: Alex Leong <alex@buoyant.io>
2020-05-05 09:33:10 -07:00
Tarun Pothulapati fc7456ce2a
Refactor linkerd-config-addons configmap (#4318)
* rename linkerd-values to linkerd-config-addons

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* refactor linkerd-config-addons to be more saner

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* add comments for the linkerd-config-addons

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* move _addon.tpl to partials

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* remove unnecessary checks in _addons.tpl

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* add _addon.tpl partials to TestRenderHelm

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* remove on a copy

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2020-05-05 14:11:21 +05:30
Zahari Dichev 4e82ba8878
Multicluster checks (#4279)
Multicluster checks

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
2020-05-05 10:19:38 +03:00
Matei David 6b9aaac9d6
Add Kubeconfig contex namespace to cli commands' options (#4197) (#4291)
When using cli commands that work on namespaced resources in the cluster, the default namespace used by the cli is hardcoded to the default Kubernetes namespace (i.e 'default'). This update will allow cli commands that operate on namespaced resources to automatically infer what the name of the default  namespace is, by taking the relevant default from the currently used Kubeconfig context. In short, this allows the omission of the -n flag in commands such as linkerd metrics, when working with resources that belong to a namespace that is set as default in the currently active context.

Validation was done manually by setting the default namespace of the currently used context, as well as through two integration tests that target the tap and get command respectively.

Signed-off-by: Matei David <matei.david.35@gmail.com>
2020-05-04 16:21:05 -05:00
Naseem 6aa1e76096
Allow config of prometheus alertmanagers, rules and extra args. (#4220)
This allows end user flexibility for options such as log format. Rather than bubbling up such possible config options into helm values, extra arguments provides more flexibility.

Add prometheusAlertmanagers value allows configuring a list of statically targetted alertmanager instances.
Use rule configmaps for prometheus rules. They take a list of {name,subPath,configMap} values and mounts them accordingly. Provided that subpaths end with _rules.yml or _rules.yaml they should be loaded by prometheus as per prometheus.yml's rule_files content.

Signed-off-by: Naseem <naseem@transit.app>
2020-05-04 14:06:10 -05:00
Alex Leong 40b921508f
Inject LINKERD2_PROXY_DESTINATION_GET_NETWORKS proxy variable (#4300)
Fixes #3807

By setting the LINKERD2_PROXY_DESTINATION_GET_NETWORKS environment variable, we configure the Linkerd proxy to do destination lookups for authorities which are IP addresses in the private network range.  This allows us to get destination metadata including identity for HTTP requests which target an IP address in the cluster, Prometheus metrics scrape requests, for example.

This change allowed us to update the "direct edges" test which ensures that the edges command produces correct output for traffic which is addressed directly to a pod IP.

We also re-enabled the "linkerd stat" integration tests which had been disabled while the destination service did not yet support these types of IP queries.

Signed-off-by: Alex Leong <alex@buoyant.io>
2020-04-30 11:22:24 -07:00
Zahari Dichev 00f17d2ed6
Make export-service non side-effecting (#4307)
Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
2020-04-30 17:11:55 +03:00
Tarun Pothulapati e75c6580ec
refacor TestRenderHelm to not need addOn list (#4297)
- rather than passing the list of add-ons, they can instead be built from the values

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2020-04-29 23:44:30 +05:30
Zahari Dichev 5149152ef3
Multicluster gateway and remote setup command (#4265)
Add multicluster gateway and setup command

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
2020-04-29 20:33:23 +03:00
Zahari Dichev 17dacf5548
Add gateways command, allowing the retrieval of gateway stats (#4241)
Add gateways command, allowing the retrieval of gateway stats

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
2020-04-27 13:55:01 +03:00
Zahari Dichev 09262ebd72
Add liveliness checks and metrics for multicluster gateway (#4233)
Add liveliness checks for gateway

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
2020-04-27 13:06:58 +03:00
Alejandro Pedraza dacf87e084
Added missing annotations to addon test fixtures (#4286)
Followup to #4271

Add missing annotation `linkerd.io/workload-ns: linkerd` in in the
addons test fixtures, introduced by the downward work from #4199
2020-04-23 16:15:16 -05:00
Tarun Pothulapati 60ffd1c2a2
Support Multi-stage install with Add-On's (#4271)
* Support Multi-stage install with Add-Ons
* add upgrade tests for add-ons
* add multi stage upgrade unit tests

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2020-04-23 14:40:58 -05:00
Tarun Pothulapati 2b1cbc6fc1
charts: Using downwardAPI to mount labels to the proxy container (#4199)
* use downward API to mount labels to the proxy container as a volume
* add namespace as a label to the pod
* add a trace inject test
* add downwardAPi for controlplaneTracing
* add controlPlaneTracing condition to volumeMounts
* update add-ons to have workload-ns
* add workload-ns label to control-plane components

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2020-04-22 10:33:51 -05:00
Alejandro Pedraza b00a84126d
Some `linkerd stat` test failures were being hidden (#4272)
* Some `linkerd stat` test failures were being hidden

`linkerd stat` was doing an early `os.Exit(0)` when no traffic was
found, which avoided `go test` to report any test failure that ended in
that code path.

This was hiding a mismatch in the golden files for HA after the
introduction of the rolling update strategy (#4267), and the failure of
`linkerd stat trafficsplit` not returning results unless `--unmeshed` is
used. For the latter, I added the flag to the tests in order to temporarly pass
them, but the underlying issue remains to be fixed in a separate
PR.
2020-04-21 14:52:09 -05:00
Kevin Leimkuhler 2c38f228f7
Add MeshedPodCount field to TS resource rows (#4273)
The addition of the `--unmeshed` flag changed the rendering behavior of the
`stat` command so that resources with 0 meshed pods are not displayed by
default.

Rendering is based off the row's `MeshedPodCount` field which is currently not
set by `func trafficSplitResourceQuery`. This change sets that field now so
that in rendering, the trafficsplit resource is rendered in the output.

The reason for this not showing up in testing is addressed by #4272 where the
`stat` command behavior for no traffic is changed.

The following now works without `--unmeshed` flag being passed:

```
❯ bin/linkerd stat -A ts
NAMESPACE   NAME                    APEX          LEAF          WEIGHT   SUCCESS   RPS   LATENCY_P50   LATENCY_P95   LATENCY_P99
default     backend-traffic-split   backend-svc   backend-svc     500m         -     -             -             -             -
default     backend-traffic-split   backend-svc   failing-svc        0         -     -             -             -             -
```
2020-04-21 10:23:35 -07:00
Alex Leong 5d3862c120
Use /live for liveness probe (#4270)
Fixes #3984

We use the new `/live` admin endpoint in the Linkerd proxy for liveness probes instead of the `/metrics` endpoint.  This endpoint returns a much smaller payload.

Signed-off-by: Alex Leong <alex@buoyant.io>
2020-04-17 14:53:32 -07:00
Tarun Pothulapati 8e56166774
Refactor AddOn Installation (#4247)
* refactor add-ons install code

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2020-04-16 15:24:55 -05:00
Alex Leong e962bf1968
Improve proxy version diagnostics (#4244)
It can be difficult to know which versions of the proxy are running in your cluster, especially when you have pods running at multiple different proxy versions.

We add two pieces of CLI functionality to assist with this:

The `linkerd check --proxy` command will now list all data plane pods which are not up-to-date rather than just printing the first one it encounters:

```
‼ data plane is up-to-date
    Some data plane pods are not running the current version:
	* default/books-84958fff5-95j75 (git-ca760bdd)
	* default/authors-57c6dc9b47-djldq (git-ca760bdd)
	* default/traffic-85f58ccb66-vxr49 (git-ca760bdd)
	* default/release-name-smi-metrics-899c68958-5ctpz (git-ca760bdd)
	* default/webapp-6975dc796f-2ngh4 (git-ca760bdd)
	* default/webapp-6975dc796f-z4bc4 (git-ca760bdd)
	* emojivoto/voting-54ffc5787d-wj6cp (git-ca760bdd)
	* emojivoto/vote-bot-7b54d6999b-57srw (git-ca760bdd)
	* emojivoto/emoji-5cb99f85d8-5bhvm (git-ca760bdd)
	* emojivoto/web-7988674b8b-zfvvm (git-ca760bdd)
	* default/webapp-6975dc796f-d2fbc (git-ca760bdd)
	* default/curl (git-7f6bbc73)
    see https://linkerd.io/checks/#l5d-data-plane-version for hints
```

The `linkerd version` command now supports a `--proxy` flag which will list all proxy versions running in the cluster and the number of pods running each version:

```
linkerd version --proxy
Client version: dev-7b9d475f-alex
Server version: edge-20.4.1
Proxy versions:
	edge-20.4.1 (10 pods)
	git-ca760bdd (11 pods)
	git-7f6bbc73 (1 pods)
```

Signed-off-by: Alex Leong <alex@buoyant.io>
2020-04-16 11:28:19 -07:00
Alejandro Pedraza 7d07504b5b
Upgrade crashes proxy-init when skipping ports (#4258)
Fixes #4257

This was introduced in 2.7.0. When performing an upgrade on an
installation having used `--skip-outbound-ports` or
`--skip-inbound-ports`, the upgrade picks those values from the
ConfigMap, parses them wrongly, and then when proxy-init picks them the
iptables commands fail.

I've also improved one of the upgrade unit tests to include these flags,
and confirmed it failed before this fix.
2020-04-15 07:11:15 -05:00
Kevin Leimkuhler 0d235694af
Add `unmeshed` flag to stat command (#4254)
## Motivation

Introduces an `unmeshed` flag to the `stat` command so that users can opt-in
to viewing unmeshed resources in the `stat` output.

This changes the existing behavior of the `stat` command such that unmeshed
resources no longer render by default in the output.

Before:

```
❯ bin/linkerd stat -A deploy
NAMESPACE     NAME                     MESHED   SUCCESS      RPS   LATENCY_P50   LATENCY_P95   LATENCY_P99   TCP_CONN
kube-system   coredns                     0/1         -        -             -             -             -          -
kube-system   local-path-provisioner      0/1         -        -             -             -             -          -
kube-system   metrics-server              0/1         -        -             -             -             -          -
kube-system   traefik                     0/1         -        -             -             -             -          -
linkerd       linkerd-controller          1/1   100.00%   0.3rps           1ms           2ms           2ms          2
linkerd       linkerd-destination         1/1   100.00%   0.3rps           1ms           1ms           1ms         11
...
```

After:

```
❯ bin/linkerd stat -A deploy
NAMESPACE   NAME                     MESHED   SUCCESS      RPS   LATENCY_P50   LATENCY_P95   LATENCY_P99   TCP_CONN
linkerd     linkerd-controller          1/1   100.00%   0.3rps           1ms           1ms           1ms          2
linkerd     linkerd-destination         1/1   100.00%   0.3rps           1ms           2ms           2ms         13
...
```

Closes #3871

## Solution

Using the meshed pod count in the stat response, resources with a count of `0`
are not rendered in the table.

The `-l`/`--selector` flag do not work for all resource types, so applying a
default label does not solve this problem. While it works for pods, it does
not work for deployments as the `linkerd.io/inject` is an annotation that
cannot be selected on.

I did not think a shorthand flag was necessary for this. I do not think users
will commonly pass this flag to the `stat` command, and I didn't think adding
an additional short flag such as `u` was necessary.

Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
2020-04-14 20:24:29 -07:00
Alex Leong 7b9d475ffc
Gate SMI-Metrics behind an install flag (#4240)
This change adds a `--smi-metrics` install flag which controls if the SMI-metrics controller and associated RBAC and APIService resources are installed.  The flag defaults to false and is hidden.

We plan to remove this flag or default it to true if and when the SMI-Metrics integration graduates from experimental.

Signed-off-by: Alex Leong <alex@buoyant.io>
2020-04-09 14:34:08 -07:00
Tarun Pothulapati d35a98cb2b
Fix routes wide output formatting for empty values (#4239)
* use wider template string for empty values when -o wide

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2020-04-07 14:35:55 -05:00
Alejandro Pedraza 322ba5fd2f
`linkerd uninstall` errors when attempting to delete PSP (#4234)
* Bug in `linkerd uninstall` when attempting to delete PSP

We were using a wrong apiVersion for PSP in `linkerd uninstall`'s
output, which avoids removing that resource:

```
$ linkerd uninstall | kubectl delete -f -
clusterrole.rbac.authorization.k8s.io "linkerd-linkerd-controller"
deleted
clusterrole.rbac.authorization.k8s.io "linkerd-linkerd-destination"
deleted
...
mutatingwebhookconfiguration.admissionregistration.k8s.io
"linkerd-proxy-injector-webhook-config" deleted
validatingwebhookconfiguration.admissionregistration.k8s.io
"linkerd-sp-validator-webhook-config" deleted
namespace "linkerd" deleted
error: unable to recognize "uninstall.yml": no matches for kind
"PodSecurityPolicy" in version "extensions/v1beta1"

$ kubectl get psp -oname
podsecuritypolicy.policy/linkerd-linkerd-control-plane
```

I've also replaced the uninstall integration test with a new separate
suite that performs the installation, waits for it to be ready,
uninstalls, and then confirms `linkerd check --pre` returns as expected.
2020-04-07 11:01:11 -05:00
Zahari Dichev d6460cf0fb
Update upgrade test certs (#4236)
Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
2020-04-06 20:15:06 +03:00
Matei David fee70c064b
Add uninstall cmd functionality to cli (#3622) (#4200)
Signed-off-by: Matei David <matei.david.35@gmail.com>
2020-04-02 12:35:39 -05:00
Alex Leong d8eebee4f7
Upgrade to client-go 0.17.4 and smi-sdk-go 0.3.0 (#4221)
Here we upgrade our dependencies on client-go to 0.17.4 and smi-sdk-go to 0.3.0.  Since smi-sdk-go uses client-go 0.17.4, these upgrades must be performed simultaneously.

This also requires simultaneously upgrading our dependency on linkerd/stern to a SHA which also uses client-go 0.17.4.  This keeps all of our transitive dependencies synchronized on one version of client-go.

This ALSO requires updating our codegen scripts to use the 0.17.4 version of code-generator and running it to generate 0.17.4 compatible generated code.  I took this opportunity to update our code generation script to properly use the version of code-generater from `go.mod` rather than a hardcoded SHA.

Signed-off-by: Alex Leong <alex@buoyant.io>
2020-04-01 10:07:23 -07:00
Mayank Shah 4429c1a5b1
Update inject to handle `automountServiceAccountToken: false` (#4145)
* Handle automountServiceAccountToken

Return error during inject if pod spec has `automountServiceAccountToken: false`

Signed-off-by: Mayank Shah <mayankshah1614@gmail.com>
2020-04-01 09:39:49 -05:00
Alejandro Pedraza 0a4df947e6
Add missing PSP for linkerd-smi-metrics (#4193)
The linkerd-smi-metrics ServiceAccount wasn't hooked into linkerd's PSP
resource, which resulted in the linkerd-smi-metrics ReplicaSet failing
to spawn pods:

```
Error creating: pods "linkerd-smi-metrics-574f57ffd4-" is forbidden:
unable to validate against any pod security policy: []
```
2020-03-25 14:28:35 -05:00
Alejandro Pedraza eb322dc420
Fix error when injecting Cronjobs that have no metadata (#4180)
When injecting a Cronjob with no
`spec.jobTemplate.spec.template.metadata` we were getting the following
error:

```
Error transforming resources: jsonpatch add operation does not apply:
doc is missing path:
"/spec/jobTemplate/spec/template/metadata/annotations"
```

This only happens to Cronjobs because other workloads force having at
least a label there that is used in `spec.selector` (at least as of v1
workloads).

With this fix, if no metadata is detected, then we add it in the json patch when
injecting, prior to adding the injection annotation.

I've added a couple of new unit tests, one that verifies that this
doesn't remove metadata contents in Cronjobs that do have that metadata,
and another one that tests injection in Cronjobs that don't have
metadata (which I verified it failed prior to this fix).
2020-03-23 14:49:50 -05:00
Mayank Shah 963b9b049a
Add kubectl-style label selectors (#4120)
* Update tap, routes and top commands to support label selectors

Signed-off-by: Mayank Shah <mayankshah1614@gmail.com>
2020-03-20 10:45:06 -05:00
Tarun Pothulapati 8d64f4e135
Bump Versions of Trace components (#4182)
* Bump Versions of Tracing components
- Jaeger to 1.17.1
- OpenCensus Collector to 0.1.11
* More sane defaults of jaeger resources

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2020-03-19 16:42:21 -05:00
Zahari Dichev 40a063878d
Service mirror CLI (#4070)
Multicluster CLI tools

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
2020-03-19 20:08:11 +02:00
Alex Leong 8f82f8c241
Upgrade smi-metrics to v0.2.1 (#4186)
This version contains an fix for a bug that was rejecting all requests on clusters configured with an empty list of allowed client names.  Because smi-metrics is an apiservice, this was also preventing namespaces from terminating.

Signed-off-by: Alex Leong <alex@buoyant.io>
2020-03-19 11:03:09 -07:00
Alejandro Pedraza 1cbc26a2c1
Upgrade golangci-lint to v1.23.8 (#4181)
* Upgrade golangci-lint to v1.23.8

This should help with some timeouts we're seeing in CI.

I fixed some new warnings found in `inject.go` and `uninject.go`.
Also we now have to explicitly disable linting `/controller/gen`.

The linter was also complaining that in `/pkg/k8s/fake.go` the
`spClient.Interface` and `tsclient.Interface` returned in the function
`newFakeClientSetsFromManifests()` aren't used, but I opted to ignore
that to leave them available for future tests.
2020-03-18 09:13:19 -05:00
Alejandro Pedraza 8f79e07ee2
Bump proxy-init to v1.3.2 (#4170)
* Bump proxy-init to v1.3.2

Bumped `proxy-init` version to v1.3.2, fixing an issue with `go.mod`
(linkerd/linkerd2-proxy-init#9).
This is a non-user-facing fix.
2020-03-17 14:49:25 -05:00
Alex Leong 794abfe0d4
Add alpha clients command (#4157)
We add the `linkerd alpha clients` command which displays client side metrics from each of a resource's clients.  This allows you to see who all of your clients are and see what your resource's metrics look like from your clients' point of view.  Since these metrics are measured on the client-side, they include network latency.

```
> linkerd alpha clients deploy/web -n emojivoto
FROM                TO   SUCCESS        RPS  LATENCY_P50  LATENCY_P90  LATENCY_P99
vote-bot.emojivoto  web   97.50%     2.0rps          4ms          5ms          5ms
```

Signed-off-by: Alex Leong <alex@buoyant.io>
2020-03-12 13:45:34 -07:00
Alex Leong cfae4d6432
Add -A flag to alpha stat (#4142)
Add an `--all-namespaces` flag to `linkerd alpha stat`.  This flag ignore the value of the `--namespace` flag and looks up resources across all namespaces.

Some example usage:

```
> linkerd alpha stat po -A
NAMESPACE  NAME                                    SUCCESS        RPS  LATENCY_P50  LATENCY_P90  LATENCY_P99
default    curl                                    100.00%     0.6rps          1ms          1ms          1ms
emojivoto  emoji-ffd474b7b-nq8wc                   100.00%     2.0rps          1ms          1ms          1ms
emojivoto  vote-bot-74c4867dc6-d5j4d                90.00%     2.0rps          3ms          4ms          4ms
emojivoto  voting-6b69659f5b-6hpvx                  78.95%     0.9rps          1ms          1ms          1ms
emojivoto  web-6cfccddd6b-vrq2q                     92.86%     5.6rps          1ms          3ms          4ms
linkerd    linkerd-controller-54bbb5d485-4p9w2     100.00%     0.3rps          1ms          1ms          1ms
linkerd    linkerd-destination-69fb65c4fb-7mthj    100.00%     0.3rps          1ms          1ms          1ms
linkerd    linkerd-grafana-ffc4d969-gf5cz          100.00%     0.3rps          1ms          2ms          2ms
linkerd    linkerd-identity-6456988769-tbkx9       100.00%     0.3rps          1ms          1ms          1ms
linkerd    linkerd-prometheus-5469d5d8fd-kskc6     100.00%     2.5rps          1ms          2ms          3ms
linkerd    linkerd-proxy-injector-658f8c4cd-pfgbt  100.00%     0.3rps          1ms          1ms          1ms
linkerd    linkerd-smi-metrics-86567c5ff4-dh7rn          -     0.0rps          0ms          0ms          0ms
linkerd    linkerd-sp-validator-54c8d7dcf9-wq6jv   100.00%     0.3rps          1ms          2ms          2ms
linkerd    linkerd-tap-574b74c964-cwm6l            100.00%     0.3rps          1ms          1ms          1ms
linkerd    linkerd-web-577755788d-95slx            100.00%     0.3rps          1ms          1ms          1ms
```

```
> linkerd alpha stat po/curl --to po
FROM  TO                              SUCCESS        RPS  LATENCY_P50  LATENCY_P90  LATENCY_P99
curl  web-6cfccddd6b-vrq2q.emojivoto  100.00%     0.9rps          1ms          2ms          2ms
```

Signed-off-by: Alex Leong <alex@buoyant.io>
2020-03-09 13:29:25 -07:00
Alex Leong 9408dc7fe1
Add linkerd alpha stat command (#4130)
This PR introduces the `linkerd alpha stat` command which will eventually replace the `linkerd stat` command.  This command functions in a similar way, but with slightly different arguments and is implemented using the smi-metrics API.  This means that access to metrics can be controlled with RBAC.

See the `linkerd alpha stat` help text for full details, or try one of these commands:

* `linkerd alpha stat -n emojivoto deploy/web`
* `linkerd alpha stat -n emojivoto deploy`
* `linkerd alpha stat -n emojivoto deploy/web --to deploy/emoji`

Signed-off-by: Alex Leong <alex@buoyant.io>
2020-03-05 15:23:14 -08:00
Zahari Dichev edd7fd203d
Service Mirroring Component (#4028)
This PR introduces a service mirroring component that is responsible for watching remote clusters and mirroring their services locally.

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
2020-03-02 21:16:08 +02:00
Alex Leong 71d6a00faa
Include SMI metrics as part of Linkerd install (#4109)
Adds the SMI metrics API to the Linkerd install flow.  This installs the SMI metrics controller deployment, the SMI metrics ApiService object, and supporting RBAC, and config resources.

This is the first step toward having Linkerd consume the SMI metrics API in the CLI and web dashboard.

Signed-off-by: Alex Leong <alex@buoyant.io>
2020-03-02 10:11:16 -08:00
arminbuerkle 65eae40b6a
Remove envoy, contour restrictions (#4092)
* Remove envoy, contour restrictions

Signed-off-by: Armin Buerkle <armin.buerkle@alfatraining.de>
2020-03-02 09:18:51 -05:00
Tarun Pothulapati 948dc22a34
Tracing Add-on For Linkerd (#3955)
* Moves Common templates needed to partials

As add-ons re-use the partials helm chart, all the templates needed by multiple charts should be present in partials
This commit also updates the helm tests
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* add tracing add-on helm chart

Tracing sub-chart includes open-census and jaeger components as a sub-chart which can be enabled as needed

* Updated Install path to also install add-ons

This includes new interface for add-ons to implement, with example tracing implementation

* Updates Linkerd install path to also install add-ons

Changes include:
 - Adds an optional Linkerd Values configmap which stores add-on configuration when add-ons are present.
 - Updates Linkerd install path to check for add-ons and render their sub-charts.
 - Adds a install Option called config, which is used to pass confiugration for add-ons.
 - Uses a fork of mergo, to over-write default Values with the Values struct generated from config.

* Updates the upgrade path about add-ons.

Upgrade path now checks for the linkerd-values cm, and overwrites the default values with it, if present.
It then checks the config option, for any further overwrites

* Refactor linkerd-values and re-update tests
also adds relevant nil checks
* Refactor code to fix linting issues
* Fixes an error with linkerd-config global values

Also refactors the linkerd-values cm to work the same with helm

* Fix a nil pointer issue for tests
* Updated Tracing add-on chart meta-data
Also introduced a defaultGetFiles method for add-ons

* Add add-on/charts to gitignore
* refactor gitignore for chart deps
* Moves sub-charts to /charts directly
* Refactor linkerd values cm
* Add comment in linkerd-values
* remove extra controlplanetracing flag
* Support Stages deployment for add-ons along with tests
* linting fix
* update tracing rbac
* Removes the need for add-on Interface
- Uses helm loading capabiltiies to get info about add-ons
- Uses reflection to not have to unnecessarily add checks for each add-on type

* disable tracing flag
* Remove dep on forked mergo
- Re-use merge from helm

* Re-use helm's merge
* Override the chartDir path during tests
* add error check
* Updated the dependency iteration code

Currently, the charts directory, will not have the deps in the repo. So, Code is updated to read the dependencies from requirements.yaml
and use that info to read templates from the relevant add-ons directory.

* Hard Code add-ons name
* Remove struct details for add-ons

- As we don't use fields of a add-on struct, we don't have them to be typed. Instead we can just use the `enabled` flag using reflection
- Users can just use map[string]interface{} as the add-on type.

* update unit tests
* linting fix
* Rename flag to addon-config
* Use Chart loading logic
- This code uses chart loading to read the files and keep in a vfs.
- Once we have those files read we will then use them for generation of sub-charts.

* Go fmt fix
* Update the linkerd-values cm to use second level field
* Add relevant unit tests for mergeRaw
* linting fix
* Move addon tests to a new file
* Fix golden files
* remove addon install unit test
* Refactor sub-chart load logic
* Add install tracing unit test
* golden file update for tracing install
* Update golden files to reflect another pr changes
* Move addon-config flag to recordFlagSet
* add relevant tracing enabled checks
* linting fix

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2020-02-26 10:15:04 -08:00
Supratik Das d9956f3b35
Update control-plane-namespace label (#4061)
* Update control-plane-namespace label

Upgrade command ignores changes to the namespace object

Add linkerd.io/control-plane-ns=linkerd label to the control-plane namespace

Fixes #3958

* Add controlPlaneNamespace label to namespace.yaml
* Modify tests for updated controlPlaneNamespace label
* Fix faulty values.yaml value
* Localize reference for controlPlaneNamespace label in kubernetes_helper.go

Signed-off-by: Supratik Das <rick.das08@gmail.com>
2020-02-24 12:57:28 -08:00
Christy Jacob f9b940e89d
Support for custom prometheus registry (#4041)
* feat: added prometheus Registry Option for install command
* chore: draft commit
* Draft for custom prometheus image
* Support for custom prometheus image

This PR adds support to override the default prometheus image name and use custom image names in private repositories

* Added default Prometheus Image from values.yaml

The default can be overridden by the argument given in installOptions

* chore: fixed failing check
* Fixed fialing check
* Updated the tests as per the new flag
* Air-gapped installation for prometheus-image
* Air Gapped installation for Prometheus Image
* Added regex for prometheus repository/image cli option

Signed-off-by: Christy Jacob <christyjacob4@gmail.com>
2020-02-24 09:59:29 -08:00
Saurav Tiwary 1c19e314b7
Linkerd CLI command to get control plane diagnostics (#4050)
* CLI command to fetch control plane metrics
Fixes #3116
* Add GetResonse method to return http GET response
* Implemented timeouts using waitgroups
* Refactor metrics command by extracting common code to metrics_diagnostics_util
* Refactor diagnostics to remove code duplication
* Update portforward_test for NewContainerMetricsForward function
* Lint code
* Incorporate Alex's suggestions
* Lint code
* fix minor errors
* Add unit test for getAllContainersWithPort
* Update metrics and diagnostics to store results in a buffer and print once
* Incorporate Ivan's suggestions
* consistent error handling inside diagnostics
* add coloring for the output
* spawn goroutines for each pod instead of each container
* switch back to unbuffered channel
* remove coloring in the output
* Add a long description of the command

Signed-off-by: Saurav Tiwary <srv.twry@gmail.com>
2020-02-24 09:09:54 -08:00
Supratik Das 42efc1da01
Improve kubectl apply format by removing misplaced message (#4053)
* Improve kubectl apply format by removing misplaced message

Fixes #2956

Also separate stderr messages with a new line

Signed-off-by: Supratik Das <rick.das08@gmail.com>
2020-02-20 10:36:36 -05:00
Mayank Shah 7cff974a79
cli: handle panic caused by `linkerd metrics` port-forward failure (#4007)
* cli: handle `linkerd metrics` port-forward gracefully

- add return for routine in func `Init()` in case of error
- add return from func `getMetrics()` if error from `portforward.Init()`

* Remove select block at pkg/k8s/portforward.go

- It is now the caller's responsibility to call pf.Stop()

Signed-off-by: Mayank Shah <mayankshah1614@gmail.com>
2020-02-19 21:44:37 -08:00
Mayank Shah 3c3a4a5f5d
cli: Add label selector flag for `stat` (#4040)
* Update `linkerd-namespace` shorthand to `L`
* Add --selector (-l) flag for `stat`

Signed-off-by: Mayank Shah <mayankshah1614@gmail.com>
2020-02-17 13:40:07 -05:00
Kohsheen Tiku 19806e3626
Scroll functionality for linkerd top deploy/linkerd-web (#4011)
* Table obtained from linkerd top is not scrollable.

Added scroll functionality for the table.

Fixes #2558

Signed-off-by: Kohsheen Tiku <kohsheen.t@gmail.com>
2020-02-17 11:17:43 -05:00
Zahari Dichev 3538944d03
Unify trust anchors terminology (#4047)
Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
2020-02-15 10:12:46 +02:00
Mayank Shah c1b683147a
Update identity to make certs more diagnosable (#3990)
Update identity controller to make issuer certificates diagnosable if
cert validity is causing error

    - Add expiry time in identity log message
    - Add current time in identity log message
    - Emit k8s event with appropriate message


Signed-off-by: Mayank Shah <mayankshah1614@gmail.com>
2020-02-13 11:21:41 +02:00
Zahari Dichev 20f8da0e61
Remove experimental from CNI (#4038)
Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
2020-02-11 20:10:55 +02:00
Zahari Dichev 9b29a915d3
Improve cni resources labels (#4032)
Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
2020-02-11 12:10:08 +02:00
Zahari Dichev c609564dc8
Add helm upgrade integration test (#3976)
In light of the breaking changes we are introducing to the Helm chart and the convoluted upgrade process (see linkerd/website#647) an integration test can be quite helpful. This simply installs latest stable through helm install and then upgrades to the current head of the branch.

Signed-off-by: Zahari Dichev zaharidichev@gmail.com
2020-02-04 08:27:46 +02:00
Zahari Dichev deefeeec52 Rename no init container second take (#3972)
This is a second attempt on #3956 as it got merged in the wrong branch

Fixes #3930

Signed-off-by: Zahari Dichev zaharidichev@gmail.com
2020-01-24 12:52:55 -08:00
Zahari Dichev 113c23bdf6 Fix helm list of ports not rendering correctly (#3957)
There was a problem that caused helm install to not reflect the proper list of ignored inbound and outbound ports. Namely if you supply just one port, that would not get reflected.

To reproduce do a: 

```
 helm install \
       --name=linkerd2 \
       --set-file global.identityTrustAnchorsPEM=ca.crt \
       --set-file identity.issuer.tls.crtPEM=issuer.crt \
       --set-file identity.issuer.tls.keyPEM=issuer.key \
       --set identity.issuer.crtExpiry=2021-01-14T14:21:43Z \
       --set-string global.proxyInit.ignoreInboundPorts="6666" \
       linkerd-edge/linkerd2
```


Check your config: 

```bash
 $ kubectl get configmap -n linkerd -oyaml | grep ignoreInboundPort
 "ignoreInboundPorts":[],
```
Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
2020-01-24 09:41:11 -08:00
Kevin Leimkuhler 53baecb382
Changes for edge-20.1.3 (#3966)
## edge-20.1.3

* CLI
  * Introduced `linkerd check --pre --linkerd-cni-enabled`, used when the CNI
    plugin is used, to check it has been properly installed before proceeding
    with the control plane installation
  * Added support for the `--as-group` flag so that users can impersonate
    groups for Kubernetes operations (thanks @mayankshah160!)
* Controller
  * Fixed an issue where an override of the Docker registry was not being
    applied to debug containers (thanks @javaducky!)
  * Added check for the Subject Alternate Name attributes to the API server
    when access restrictions have been enabled (thanks @javaducky!)
  * Added support for arbitrary pod labels so that users can leverage the
    Linkerd provided Prometheus instance to scrape for their own labels
    (thanks @daxmc99!)
  * Fixed an issue with CNI config parsing

Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
2020-01-23 16:55:21 -08:00
Zahari Dichev a9d38189fb Fix CNI config parsing (#3953)
This PR addreses the problem introduced after #3766.

Fixes #3941 

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
2020-01-23 09:55:04 -08:00
Mayank Shah 60ac0d5527 Add `as-group` CLI flag (#3952)
Add CLI flag --as-group that can impersonate group for k8s operations

Signed-off-by: Mayank Shah mayankshah1614@gmail.com
2020-01-22 16:38:31 +02:00
Dax McDonald 5b75a2176f Add scraping of arbitrary pod labels (#3833)
This allows for users of Linkerd to leverage the Prometheus instance
deployed by the mesh for their metric needs. With support for pod labels
outside of the Linkerd metrics users are able to scrape metrics
based upon their own labels.

Signed-off-by: Dax McDonald <dax@rancher.com>
2020-01-22 09:55:26 +02:00
Paul Balogh dabee12b93 Fix issue for debug containers when using custom Docker registry (#3873)
**Subject**
Fixes bug where override of Docker registry was not being applied to debug containers (#3851)

**Problem**
Overrides for Docker registry are not being applied to debug containers and provide no means to correct the image.

**Solution**
This update expands the `data.proxy` configuration section within the Linkerd `ConfigMap` to maintain the overridden image name for debug containers at _install_-time similar to handling of the `proxy` and `proxyInit` images.

This change also enables the further override option of the registry for debug containers at _inject_-time given utilization of the `--registry` CLI option.

**Validation**
Several new unit tests have been created to confirm functionality.  In addition, the following workflows were run through:

### Standard Workflow with Custom Registry
This workflow installs Linkerd control plane based upon a custom registry, then injecting the debug sidecar into a service.

* Start with a k8s instance having no Linkerd installation
* Build all images locally using `bin/docker-build`
* Create custom tags (using same version) for generated images, e.g. `docker tag gcr.io/linkerd-io/debug:git-a4ebecb6 javaducky.com/linkerd-io/debug:git-a4ebecb6`
* Install Linkerd with registry override `bin/linkerd install --registry=javaducky.com/linkerd-io | kubectl apply -f -`
* Once Linkerd has been fully initialized, you should be able to confirm that the `linkerd-config` ConfigMap now contains the debug image name, pull policy, and version within the `data.proxy` section
* Request injection of the debug image into an available container.  I used the Emojivoto voting service as described in https://linkerd.io/2/tasks/using-the-debug-container/ as `kubectl -n emojivoto get deploy/voting -o yaml | bin/linkerd inject --enable-debug-sidecar - | kubectl apply -f -`
* Once the deployment creates a new pod for the service, inspection should show that the container now includes the "linkerd-debug" container name based on the applicable override image seen previously within the ConfigMap
* Debugging can also be verified by viewing debug container logs as `kubectl -n emojivoto logs deploy/voting linkerd-debug -f`
* Modifying the `config.linkerd.io/enable-debug-sidecar` annotation, setting to “false”, should show that the pod will be recreated no longer running the debug container.

### Overriding the Custom Registry Override at Injection
This builds upon the “Standard Workflow with Custom Registry” by overriding the Docker registry utilized for the debug container at the time of injection.

* “Clean” the Emojivoto voting service by removing any Linkerd annotations from the deployment
* Request injection similar to before, except provide the `--registry` option as in `kubectl -n emojivoto get deploy/voting -o yaml | bin/linkerd inject --enable-debug-sidecar --registry=gcr.io/linkerd-io - | kubectl apply -f -`
* Inspection of the deployment config should now show the override annotation for `config.linkerd.io/debug-image` having the debug container from the new registry.  Viewing the running pod should show that the `linkerd-debug` container was injected and running the correct image.  Of note, the proxy and proxy-init images are still running the “original” override images.
* As before, modifying the `config.linkerd.io/enable-debug-sidecar` annotation setting to “false”, should show that the pod will be recreated no longer running the debug container.

### Standard Workflow with Default Registry
This workflow is the typical workflow which utilizes the standard Linkerd image registry.

* Uninstall the Linkerd control plane using `bin/linkerd install --ignore-cluster | kubectl delete -f -` as described at https://linkerd.io/2/tasks/uninstall/
* Clean the Emojivoto environment using `curl -sL https://run.linkerd.io/emojivoto.yml | kubectl delete -f -` then reinstall using `curl -sL https://run.linkerd.io/emojivoto.yml | kubectl apply -f -`
* Perform standard Linkerd installation as `bin/linkerd install | kubectl apply -f -`
* Once Linkerd has been fully initialized, you should be able to confirm that the `linkerd-config` ConfigMap references the default debug image of `gcr.io/linkerd-io/debug` within the `data.proxy` section
* Request injection of the debug image into an available container as `kubectl -n emojivoto get deploy/voting -o yaml | bin/linkerd inject --enable-debug-sidecar - | kubectl apply -f -`
* Debugging can also be verified by viewing debug container logs as `kubectl -n emojivoto logs deploy/voting linkerd-debug -f`
* Modifying the `config.linkerd.io/enable-debug-sidecar` annotation, setting to “false”, should show that the pod will be recreated no longer running the debug container.

### Overriding the Default Registry at Injection
This workflow builds upon the “Standard Workflow with Default Registry” by overriding the Docker registry utilized for the debug container at the time of injection.

* “Clean” the Emojivoto voting service by removing any Linkerd annotations from the deployment
* Request injection similar to before, except provide the `--registry` option as in `kubectl -n emojivoto get deploy/voting -o yaml | bin/linkerd inject --enable-debug-sidecar --registry=javaducky.com/linkerd-io - | kubectl apply -f -`
* Inspection of the deployment config should now show the override annotation for `config.linkerd.io/debug-image` having the debug container from the new registry.  Viewing the running pod should show that the `linkerd-debug` container was injected and running the correct image.  Of note, the proxy and proxy-init images are still running the “original” override images.
* As before, modifying the `config.linkerd.io/enable-debug-sidecar` annotation setting to “false”, should show that the pod will be recreated no longer running the debug container.

Fixes issue #3851 

Signed-off-by: Paul Balogh javaducky@gmail.com
2020-01-17 10:18:03 -08:00
Zahari Dichev e30b9a9c69
Add checks for CNI plugin (#3903)
As part of the effort to remove the "experimental" label from the CNI plugin, this PR introduces cni checks to `linkerd check`

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
2020-01-17 12:11:19 +02:00
Mayank Shah b94e03a8a6 Remove empty fields from generated configs (#3886)
Fixes
- https://github.com/linkerd/linkerd2/issues/2962
- https://github.com/linkerd/linkerd2/issues/2545

### Problem
Field omissions for workload objects are not respected while marshaling to JSON.

### Solution
After digging a bit into the code, I came to realize that while marshaling, workload objects have empty structs as values for various fields which would rather be omitted. As of now, the standard library`encoding/json` does not support zero values of structs with the `omitemty` tag. The relevant issue can be found [here](https://github.com/golang/go/issues/11939). To tackle this problem, the object declaration should have _pointer-to-struct_ as a field type instead of _struct_ itself. However, this approach would be out of scope as the workload object declaration is handled by the k8s library.

I was able to find a drop-in replacement for the `encoding/json` library which supports zero value of structs with the `omitempty` tag. It can be found [here](https://github.com/clarketm/json). I have made use of this library to implement a simple filter like functionality to remove empty tags once a YAML with empty tags is generated, hence leaving the previously existing methods unaffected

Signed-off-by: Mayank Shah <mayankshah1614@gmail.com>
2020-01-13 10:02:24 -08:00
Zahari Dichev d259b23e8b
Add check to ensure kube-system has the needed annotations (HA) (#3731)
Adds a check to ensure kube-system namespace has `config.linkerd.io/admission-webhooks:disabled`

FIxes #3721

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
2020-01-10 10:03:13 +02:00
Alex Leong 93a81dce97
Change default proxy log level to "warn,linkerd=info" (#3908)
Fixes #3901 

Signed-off-by: Alex Leong <alex@buoyant.io>
2020-01-09 14:22:06 -08:00
Tarun Pothulapati 03982d8837 move more values to global (#3892)
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2020-01-09 14:57:43 -05:00
Alex Leong 3b2c1eb540
Respect registry override during inject (#3879)
Fixes https://github.com/linkerd/linkerd2/issues/3878

If the `--registry` flag is provided to Linkerd without the `--proxy-image` or `--init-image` flags, the `--registry` flag is ignored and not applied to the existing values for the proxy or init images pulled from the configmap.

We now override the registry with the value from the `--registry` flag regardless of which other flags are provided.

Signed-off-by: Alex Leong <alex@buoyant.io>
2020-01-08 15:54:09 -08:00
Zahari Dichev 287900a686
Unify issuance lifetime name (#3887)
Due to wrong snake casing, lifetime setting lifetime issuance was not reflected when installing through helm. This commit solved that problem

Signed-off-by: Zahari Dichev zaharidichev@gmail.com
2020-01-08 09:58:20 +02:00
Tarun Pothulapati 42b0c0f1a1 Bump prometheus version to 2.15.2 (#3876)
* bump prometheus version to 2.15.0
* update golden files
* update helm tests
* update to prometheus 2.15.1
* update to prometheus 2.15.2

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2020-01-07 19:44:57 -08:00
Tarun Pothulapati eac06b973c Move common values to global (#3839)
* move values to global in template

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* update inject and cli

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* update unit tests

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* fix linting issues

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* remote controllerImageVersion from global

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* move identity out of global

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* update var name and comments

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* update bin and helm tests

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* update helm readme

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* fix proxy config

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* fix proxy config indentation

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* more linting issues

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* remove unnecessary lines

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2020-01-06 14:31:41 -08:00
Alejandro Pedraza f39d4c5275
Fix `linkerd-cni` Helm chart (#3866)
* The `linkerd-cni` chart should set proper annotations/labels for the namespace

When installing through Helm, the `linkerd-cni` chart will (by default)
install itself under the same namespace ("linkerd") that the `linkerd` chart will be
installed aftewards. So it needs to set up the proper annotations and labels.

* Fix Helm install when disabling init containers

To install linkerd using Helm after having installed linkerd's CNI plugin, one needs to `--set noInitContainer=true`.
But to determine whether to use init containers or not, we weren't
evaluating that, but instead `Values.proxyInit`, which is indeed null
when installing through the CLI but not when installing with Helm. So
init containers were being set despite having passed `--set
noInitContainers=true`.
2020-01-06 13:02:27 -05:00
Tarun Pothulapati 576c2bece6 Fix Helm templating bugs, left-over smaller-cases (#3869)
* update flags to smaller
* add tests for the same
* fix control plane trace flag
* add tests for controlplane tracing install

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2019-12-30 13:34:15 -05:00
Paul Balogh 2cd2ecfa30 Enable mixed configuration of skip-[inbound|outbound]-ports (#3766)
* Enable mixed configuration of skip-[inbound|outbound]-ports using port numbers and ranges (#3752)
* included tests for generated output given proxy-ignore configuration options
* renamed "validate" method to "parseAndValidate" given mutation
* updated documentation to denote inclusiveness of ranges
* Updates for expansion of ignored inbound and outbound port ranges to be handled by the proxy-init rather than CLI (#3766)

This change maintains the configured ports and ranges as strings rather than unsigned integers, while still providing validation at the command layer.

* Bump versions for proxy-init to v1.3.0

Signed-off-by: Paul Balogh <javaducky@gmail.com>
2019-12-20 09:32:13 -05:00
Sergio C. Arteaga 7886938f4f Classify some gRPC status codes as non-errors (#3736)
Signed-off-by: Sergio Castaño Arteaga <tegioz@icloud.com>
2019-12-19 15:22:43 -05:00
Alex Leong 03762cc526
Support pod ip and service cluster ip lookups in the destination service (#3595)
Fixes #3444 
Fixes #3443 

## Background and Behavior

This change adds support for the destination service to resolve Get requests which contain a service clusterIP or pod ip as the `Path` parameter.  It returns the stream of endpoints, just as if `Get` had been called with the service's authority.  This lays the groundwork for allowing the proxy to TLS TCP connections by allowing the proxy to do destination lookups for the SO_ORIG_DST of tcp connections.  When that ip address corresponds to a service cluster ip or pod ip, the destination service will return the endpoints stream, including the pod metadata required to establish identity.

Prior to this change, attempting to look up an ip address in the destination service would result in a `InvalidArgument` error.

Updating the `GetProfile` method to support ip address lookups is out of scope and attempts to look up an ip address with the `GetProfile` method will result in `InvalidArgument`.

## Implementation

We do this by creating a `IPWatcher` which wraps the `EndpointsWatcher` and supports lookups by ip.   `IPWatcher` maintains a mapping up clusterIPs to service ids and translates subscriptions to an IP address into a subscription to the service id using the underlying `EndpointsWatcher`.

Since the service name is no longer always infer-able directly from the input parameters, we restructure `EndpointTranslator` and `PodSet` so that we propagate the service name from the endpoints API response.

## Testing

This can be tested by running the destination service locally, using the current kube context to connect to a Kubernetes cluster:

```
go run controller/cmd/main.go destination -kubeconfig ~/.kube/config
```

Then lookups can be issued using the destination client:

```
go run controller/script/destination-client/main.go -path 192.168.54.78:80 -method get -addr localhost:8086
```

Service cluster ips and pod ips can be used as the `path` argument.

Signed-off-by: Alex Leong <alex@buoyant.io>
2019-12-19 09:25:12 -08:00
Andrew Seigner 537bc76f2f
Add recommended k8s labels to control-plane (#3847)
The Kubernetes docs recommend a common set of labels for resources:
https://kubernetes.io/docs/concepts/overview/working-with-objects/common-labels/#labels

Add the following 3 labels to all control-plane workloads:
```
app.kubernetes.io/name: controller # or destination, etc
app.kubernetes.io/part-of: Linkerd
app.kubernetes.io/version: edge-X.Y.Z
```

Fixes #3816

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-12-19 09:00:55 -08:00
Eugene Glotov 748da80409 Inject preStop hook into the proxy sidecar container to stop it last (#3798)
* Inject preStop hook into the proxy sidecar container to stop it last

This commit adds support for a Graceful Shutdown technique that is used
by some Kubernetes administrators while the more perspective
configuration is being discussed in
https://github.com/kubernetes/kubernetes/issues/65502

The problem is that RollingUpdate strategy does not guarantee that all
traffic will be sent to a new pod _before_ the previous pod is removed.
Kubernetes inside is an event-driven system and when a pod is being
terminating, several processes can receive the event simultaneously.
And if an Ingress Controller gets the event too late or processes it
slower than Kubernetes removes the pod from its Service, users requests
will continue flowing into the black whole.

According [to the documentation](https://kubernetes.io/docs/concepts/workloads/pods/pod/#termination-of-pods)

> 1. If one of the Pod’s containers has defined a `preStop` hook,
> it is invoked inside of the container. If the `preStop` hook is still
> running after the grace period expires, step 2 is then invoked with
> a small (2 second) extended grace period.
>
> 2. The container is sent the `TERM` signal. Note that not all
> containers in the Pod will receive the `TERM` signal at the same time
> and may each require a preStop hook if the order in which
> they shut down matters.

This commit adds support for the `preStop` hook that can be configured
in three forms:

1. As command line argument `--wait-before-exit-seconds` for
  `linkerd inject` command.

2. As `linkerd2` Helm chart value `Proxy.WaitBeforeExitSeconds`.

2. As `config.alpha.linkerd.io/wait-before-exit-seconds` annotation.

If configured, it will add the following preHook to the proxy container
definition:

```yaml
lifecycle:
  preStop:
    exec:
      command:
        - /bin/bash
        - -c
        - sleep {{.Values.Proxy.WaitBeforeExitSeconds}}
```

To achieve max benefit from the option, the main container should have
its own `preStop` hook with the `sleep` command inside which has
a smaller period than is set for the proxy sidecar. And none of them
must be bigger than `terminationGracePeriodSeconds` configured for the
entire pod.

An example of a rendered Kubernetes resource where
`.Values.Proxy.WaitBeforeExitSeconds` is equal to `40`:

```yaml
       # application container
        lifecycle:
          preStop:
            exec:
              command:
                - /bin/bash
                - -c
                - sleep 20

        # linkerd-proxy container
        lifecycle:
          preStop:
            exec:
              command:
                - /bin/bash
                - -c
                - sleep 40
    terminationGracePeriodSeconds: 160 # for entire pod
```

Fixes #3747

Signed-off-by: Eugene Glotov <kivagant@gmail.com>
2019-12-18 16:58:14 -05:00
Sergio C. Arteaga 56c8a1429f Increase the comprehensiveness of check --pre (#3701)
* Increase the comprehensiveness of check --pre

Closes #3224

Signed-off-by: Sergio Castaño Arteaga <tegioz@icloud.com>
2019-12-18 13:27:32 -05:00
Tarun Pothulapati efb1101bdb Switch to smaller-case values in linkerd2-cni (#3827)
* update linkerd2-cni templates and cli
* update readme and docs
* update helm unit tests
* update helm build script
* use smaller case linkerd version

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2019-12-16 15:09:57 -08:00
Zahari Dichev f88b55e36e Tls certs checks (#3813)
* Added checks for cert correctness
* Add warning checks for approaching expiration
* Add unit tests
* Improve unit tests
* Address comments
* Address more comments
* Prevent upgrade from breaking proxies when issuer cert is overwritten (#3821)
* Address more comments
* Add gate to upgrade cmd that checks that all proxies roots work with the identitiy issuer that we are updating to
* Address comments
* Enable use of upgarde to modify both roots and issuer at the same time

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
2019-12-16 14:49:32 -08:00
Tarun Pothulapati 2f492a77fb Switch to Smaller-Case in Linkerd2 and Partials Charts (#3823)
* update linkerd2, partials charts
* support install and inject workflow
* update helm docs
* update comments in values
* update helm tests
* update comments in test

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2019-12-13 14:48:07 -05:00
Zahari Dichev a98fe03c5e
Consolidate certificates validation logic (#3810)
* Consolidate certificates validation logic

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>

* Add test for upgrading trust anchors when using external cert manager

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>

* Add logic to ensure issuer cert is CA

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>

* Fix golden file

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
2019-12-13 10:01:55 +02:00
Sergio C. Arteaga 7f0213d534 Fix upgrade unit tests golden files (#3815)
Signed-off-by: Sergio Castaño Arteaga <tegioz@icloud.com>
2019-12-11 14:27:18 -05:00
Sergio C. Arteaga cee8e3d0ae Add CronJobs and ReplicaSets to dashboard and CLI (#3687)
This PR adds support for CronJobs and ReplicaSets to `linkerd inject`, the web
dashboard and CLI. It adds a new Grafana dashboard for each kind of resource. 

Closes #3614 
Closes #3630 
Closes #3584 
Closes #3585

Signed-off-by: Sergio Castaño Arteaga tegioz@icloud.com
Signed-off-by: Cintia Sanchez Garcia cynthiasg@icloud.com
2019-12-11 10:02:37 -08:00
Alejandro Pedraza 2d12b88145
Pods with non empty securitycontext capabilities fail to be injected (#3806)
* Pods with non empty securitycontext capabilities fail to be injected

Followup to #3744

The `_capabilities.tpl` template got its variables scope changed in
`Values.Proxy`, which caused inject to fail when security context
capabilities were detected.

Discovered when testing injecting the nginx ingress controller.
2019-12-10 14:36:14 -05:00
Alejandro Pedraza d21fda12db
Added unit test for injecting debug sidecar into CP deployment (#3786)
* Added unit test for injecting debug sidecar into CP deployment

I realized this was missing when testing #3774 (superseded by #3784).
2019-12-10 13:45:48 -05:00
Zahari Dichev 0313f10baa
Move CNI template to helm (#3581)
* Create helm chart for the CNI plugin

Signed-off-by: zaharidichev <zaharidichev@gmail.com>

* Add helm install tests for the CNI plugin

Signed-off-by: zaharidichev <zaharidichev@gmail.com>

* Add readme for the CNI helm chart

Signed-off-by: zaharidichev <zaharidichev@gmail.com>

* Fix integration tests

Signed-off-by: zaharidichev <zaharidichev@gmail.com>

* Remove old cni-plugin.yaml

Signed-off-by: zaharidichev <zaharidichev@gmail.com>

* Add trace partial template

Signed-off-by: zaharidichev <zaharidichev@gmail.com>

* Address more comments

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
2019-12-10 13:21:28 +02:00
Zahari Dichev 7e98128782 Fix upgrade unit tests golden files (#3805)
Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
2019-12-09 13:34:08 -08:00
Zahari Dichev 7cc3815d49
Add issuer file flags to upgrade command (#3771)
* Add identity-issuer-certificate-file and identity-issuer-key-file to upgrade command

Signed-off-by: zaharidichev <zaharidichev@gmail.com>

* Implement logic to use identity-trust-anchors-file flag to update the anchors

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>

* Address remarks

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
2019-12-09 21:18:03 +02:00
Alejandro Pedraza b4d27f9d82
No need for `processYAML()` in `install` (#3784)
* No need for `processYAML()` in `install`

Since `install` uses helm to do its proxy injection, there's no need to
call `processYAML`. This also fixes an issue discovered in #3687 where
we started supporting injection of cronjobs, and even though `linkerd`'s
namespace is flagged to skip automatic injection it was being injected.

This replaces #3773 as it's a much more simpler approach.
2019-12-09 09:32:14 -05:00
Zahari Dichev e5f75a8c3d
Add validation to ensure stat time window is at least 15s (#3720)
* Add stat time window minimum of 10s

Signed-off-by: zaharidichev <zaharidichev@gmail.com>

* Address comments

Signed-off-by: zaharidichev <zaharidichev@gmail.com>
2019-12-04 08:12:01 +02:00
Zahari Dichev 36609c88b8
Error on conflicting stat options (--namespace and --all-namespaces) (#3719)
Signed-off-by: zaharidichev <zaharidichev@gmail.com>
2019-11-18 13:05:44 +02:00
Zahari Dichev ef2007a933
Add helm version annotation to tap,injector and sp-validator (#3673)
Signed-off-by: zaharidichev <zaharidichev@gmail.com>
2019-11-15 20:42:19 +02:00
Zahari Dichev a6ff442789
Traffic split integration test (#3649)
* Traffic split integration test

Signed-off-by: zaharidichev <zaharidichev@gmail.com>

* Address comments

Signed-off-by: zaharidichev <zaharidichev@gmail.com>

* Display placeholder when there is no basic stats data

Signed-off-by: zaharidichev <zaharidichev@gmail.com>
2019-11-13 21:14:34 +02:00
Alejandro Pedraza 4b6254b52e
Replaced `uuid` with `uid` from linkerd-config resource (#3694)
* Replaced `uuid` with `uid` from linkerd-config resource

Fixes #3621

Removed the old `uuid` for identifying linkerd installations, and
replaced it with the `uid` property from the `linkerd-config` ConfigMap.

I tested that this `uid` remains the same by updating the config and
also upgrading linkerd, using both the CLI and Helm.

Note that this required granting `linkerd-web` RBAC access to the
`linkerd-config` Config.

I also added an integration test to verify the stability of the uid.
2019-11-13 13:56:01 -05:00
Sergio C. Arteaga eff1714a08 Add `linkerd check` to dashboard (#3656)
`linkerd check` can now be run from the dashboard in the `/controlplane` view.
Once the check results are received, they are displayed in a modal in a similar
style to the CLI output.

Closes #3613
2019-11-12 12:37:36 -08:00
Eugene Glotov 2941ddb7f5 Support Dashboard replicas (#2899) (#3633)
This PR makes possible to increase the amount of web dashboard replicas.

Follows up #2899

Signed-off-by: Eugene Glotov <kivagant@gmail.com>
2019-11-12 11:00:23 -08:00
Zahari Dichev 038900c27e Remove destination container from controller (#3661)
Signed-off-by: zaharidichev <zaharidichev@gmail.com>
2019-11-08 14:40:25 -08:00
Tarun Pothulapati f18e27b115 use appsv1 api in identity (#3682)
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2019-11-06 15:06:09 -08:00
Alejandro Pedraza 1c879ac430
Added simplified service name to list of allowed hosts for linkerd-web (#3674)
Followup to linkerd/website#573
2019-11-06 10:27:55 -05:00
Mayank Shah e91f2020db Update uninject command to handle namespaces (Fixes #3648) (#3668)
* Add support for uninject command to uninject namespace configs
* Add relevant unit tests in cli/cmd/uninject_test.go

Signed-off-by: Mayank Shah <mayankshah1614@gmail.com>
2019-11-04 22:21:47 -08:00
StupidScience 5958111533 WIP: Added annotations parsing and doc generation (#3564)
* rework annotations doc generation from godoc parsing to map[string]string and get rid of unused yaml tags
* move annotations doc function from pkg/k8s to cli/cmd

Signed-off-by: StupidScience <tonysignal@gmail.com>
2019-11-04 14:55:50 -08:00
Zahari Dichev 86854ac845
Control plane debug (#3507)
* Add cmd to inject debug sidecar for l5d components only

Signed-off-by: zaharidichev <zaharidichev@gmail.com>

* Revert "Add cmd to inject debug sidecar for l5d components only"

This reverts commit 50b8b3577e.

Signed-off-by: zaharidichev <zaharidichev@gmail.com>

* Stop uninjecting metadata from control plane components

Signed-off-by: zaharidichev <zaharidichev@gmail.com>

* Ensure inject can be run on control plane components only if --manual is present

Signed-off-by: zaharidichev <zaharidichev@gmail.com>
2019-11-04 18:56:35 +02:00