Commit Graph

2607 Commits

Author SHA1 Message Date
William Morgan c79d36d63a
add STEERING.md (#5607)
* add STEERING.md

Signed-off-by: William Morgan <william@buoyant.io>
2021-01-27 09:39:32 -06:00
Alex Leong dd8e5fc5bc
Rename extension charts to linkerd-* (#5552)
For consistency we rename the extension charts to a common naming scheme:

linkerd-viz -> linkerd-viz (unchanged)
jaeger -> linkerd-jaeger
linkerd2-multicluster -> linkerd-multicluster
linkerd2-multicluster-link -> linkerd-multicluster-link

We also make the chart files and chart readmes a bit more uniform.

Signed-off-by: Alex Leong <alex@buoyant.io>
2021-01-26 16:20:49 -08:00
Tarun Pothulapati 9756b3f8f1
extensions: make subcmds check/wait for respective extensions (#5566)
* extensions: make subcmds check/wait for respective extensions

This commit updates the extension subcmds to check and wait
for the respective extensions to be up before running them.

The same healthcheck pkg and respective extension checks
 are used to at the check/wait logic.

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2021-01-26 23:01:25 +05:30
Dennis Adjei-Baah ae2b3499b0
Tweak error message in web script (#5596)
The `bin/web run` script sets up a local environment for linkerd
dashboard development. This script port-forwards an existing linkerd
controller and a grafana instance in a local kubernetes cluster. When
running the command with just the linkerd control plane  and no
linkerd viz extension the error message is shown below.
```
'Controller is not running. Have you installed Linkerd?'
```

This error message is a little misleading because the controller is
installed when running this after `linkerd install`. The issue here is
that the script checks for a Grafana instance but the error message it
displays when it can't find a Grafana pod is that the controller isn't
install. The error message should instead notify the developer that
Linkerd Viz is not installed.

This change modifies the error message so it is more clear.

Signed-off-by: Dennis Adjei-Baah <dennis@buoyant.io>
2021-01-26 11:21:44 -06:00
Andrew Seigner 700b4c5cb5
Move @siggy to emeritus maintainer (#5597)
Emeritus (adj): having retired but allowed to retain their title as an
honor

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2021-01-23 07:26:47 -08:00
Alex Leong 964ce11559
Update generated serviceprofile code (#5580)
I ran `bin/update-codegen.sh` to update the generated code to include the opaque ports in the generated deepcopy function for service profiles.

Signed-off-by: Alex Leong <alex@buoyant.io>
2021-01-22 14:34:49 -08:00
Tarun Pothulapati 4f0601e632
jaeger: cli and check logic cleanup (#5564)
This branch cleans up some of the unnecessary logic that is not
needed and thus making the check logic similar to that of other
extensions, namely viz.

Includes the following cleanups:

- Remove `namespace` flag in jaeger CLI and make the fetching logic
dynamic and use it in check and dashboard.
- Use `hc.KubeAPIClient` instead of creating our own in jaeger check.
- Move injection checks up before we run the readiness checks

This change adds a new extension namespace exist check for
jaeger.

Also, Updates integration tests to run the check commands.

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2021-01-22 23:31:35 +05:30
cpretzer fcb71de428
Changes for edge-21.1.3 (#5590)
* Changes for edge-21.1.3
Signed-off-by: Charles Pretzer <charles@buoyant.io>
2021-01-21 17:06:10 -08:00
Alejandro Pedraza 8ac5360041
Extract from public-api all the Prometheus dependencies, and moves things into a new viz component 'linkerd-metrics-api' (#5560)
* Protobuf changes:
- Moved `healthcheck.proto` back from viz to `proto/common` as it remains being used by the main `healthcheck.go` library (it was moved to viz by #5510).
- Extracted from `viz.proto` the IP-related types and put them in `/controller/gen/common/net` to be used by both the public and the viz APIs.

* Added chart templates for new viz linkerd-metrics-api pod

* Spin-off viz healthcheck:
- Created `viz/pkg/healthcheck/healthcheck.go` that wraps the original `pkg/healthcheck/healthcheck.go` while adding the `vizNamespace` and `vizAPIClient` fields which were removed from the core `healthcheck`. That way the core healthcheck doesn't have any dependencies on viz, and viz' healthcheck can now be used to retrieve viz api clients.
- The core and viz healthcheck libs are now abstracted out via the new `healthcheck.Runner` interface.
- Refactored the data plane checks so they don't rely on calling `ListPods`
- The checks in `viz/cmd/check.go` have been moved to `viz/pkg/healthcheck/healthcheck.go` as well, so `check.go`'s sole responsibility is dealing with command business. This command also now retrieves its viz api client through viz' healthcheck.

* Removed linkerd-controller dependency on Prometheus:
- Removed the `global.prometheusUrl` config in the core values.yml.
- Leave the Heartbeat's `-prometheus` flag hard-coded temporarily. TO-DO: have it automatically discover viz and pull Prometheus' endpoint (#5352).

* Moved observability gRPC from linkerd-controller to viz:
- Created a new gRPC server under `viz/metrics-api` moving prometheus-dependent functions out of the core gRPC server and into it (same thing for the accompaigning http server).
- Did the same for the `PublicAPIClient` (now called just `Client`) interface. The `VizAPIClient` interface disappears as it's enough to just rely on the viz `ApiClient` protobuf type.
- Moved the other files implementing the rest of the gRPC functions from `controller/api/public` to `viz/metrics-api` (`edge.go`, `stat_summary.go`, etc.).
- Also simplified some type names to avoid stuttering.

* Added linkerd-metrics-api bootstrap files. At the same time, we strip out of the public-api's `main.go` file the prometheus parameters and other no longer relevant bits.

* linkerd-web updates: it requires connecting with both the public-api and the viz api, so both addresses (and the viz namespace) are now provided as parameters to the container.

* CLI updates and other minor things:
- Changes to command files under `cli/cmd`:
  - Updated `endpoints.go` according to new API interface name.
  - Updated `version.go`, `dashboard` and `uninstall.go` to pull the viz namespace dynamically.
- Changes to command files under `viz/cmd`:
  - `edges.go`, `routes.go`, `stat.go` and `top.go`: point to dependencies that were moved from public-api to viz.
- Other changes to have tests pass:
  - Added `metrics-api` to list of docker images to build in actions workflows.
  - In `bin/fmt` exclude protobuf generated files instead of entire directories because directories could contain both generated and non-generated code (case in point: `viz/metrics-api`).

* Add retry to 'tap API service is running' check

* mc check shouldn't err when viz is not available. Also properly set the log in multicluster/cmd/root.go so that it properly displays messages when --verbose is used
2021-01-21 18:26:38 -05:00
Tarun Pothulapati 288fbefe02
viz: cleanup helm values.yaml (#5546)
* viz: cleanup helm values.yaml

This branch fixes some nits around naming of default variables
i.e replace the usage of global with default.

Renames globalLogLevel to defaultLogLevel and globalUID to
defaultUID along with some chart README updates.

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2021-01-22 00:48:16 +05:30
Tarun Pothulapati a95efe2db1
tests: update newly added golden tests (#5588)
#5507 added new golden tests but missed some updates from other PRs
that got merged meanwhile.

This branch updates those golden tests with those changes

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2021-01-21 23:47:36 +05:30
Tarun Pothulapati d0d2e0ea7a
cli: add helm customization flags to core install (#5507)
* cli: add helm customization flags to core install

Fixes #5506

This branch adds helm way of customization through
 `set`, `set-string`, `values`, `set-files` flags for
`linkerd install` cmd along with unit tests.

For this to work, the helm v3 engine rendering helpers
had to be used instead of our own wrapper type.

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2021-01-21 22:49:50 +05:30
Andrew Seigner 9c80d4d2a1
Fix `linkerd metrics` resource selector (#5567)
The `linkerd metrics` command was selecting pods based on owner resource
names. If multiple owners existed with the same name (for example
`sts/web`, `deploy/web`), additional pods would be incorrectly included
in the output.

Fix the pod selector code to validate pods have owner references to the
given workload/owner.

Before:
```
$ linkerd metrics -n emojivoto deploy/web|grep POD
  # POD web-0 (1 of 3)
  # POD web-d9ffd684f-gnbcx (2 of 3)
  # POD web-fs6l7 (3 of 3)
```

After:
```
$ bin/go-run cli metrics -n emojivoto deploy/web|grep POD
  # POD web-d9ffd684f-gnbcx (1 of 1)
```

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2021-01-21 11:57:27 -05:00
Matei David c63fbdf0e4
Introduce OpenAPIV3 validation for CRDs (#5573)
* Introduce OpenAPIV3 validation for CRDs

* Add validation to link crd
* Add validation to sp using kube-gen
* Add openapiv3 under schema fields in specific versions
* Modify fields to rid spec of yaml errors
* Add top level validation for all three CRDs

Signed-off-by: Matei David <matei.david.35@gmail.com>
2021-01-21 11:56:28 -05:00
Naseem 2cc96d4ab9
fix alertmanagers casing (one word) (#5377)
fixes #5371

Signed-off-by: naseemkullah <naseem@transit.app>
2021-01-21 11:55:24 -05:00
Kevin Leimkuhler e7f2a3fba3
viz: add tap-injector (#5540)
## What this changes

This adds a tap-injector component to the `linkerd-viz` extension which is
responsible for adding the tap service name environment variable to the Linkerd
proxy container.

If a pod does not have a Linkerd proxy, no action is taken. If tap is disabled
via annotation on the pod or the namespace, no action is taken.

This also removes the environment variable for explicitly disabling tap through
an environment variable. Tap status for a proxy is now determined only be the
presence or absence of the tap service name environment variable.

Closes #5326

## How it changes

### tap-injector

The tap-injector component determines if `LINKERD2_PROXY_TAP_SVC_NAME` should be
added to a pod's Linkerd proxy container environment. If the pod satisfies the
following, it is added:

- The pod has a Linkerd proxy container
- The pod has not already been mutated
- Tap is not disabled via annotation on the pod or the pod's namespace

### LINKERD2_PROXY_TAP_DISABLED

Now that tap is an extension of Linkerd and not a core component, it no longer
made sense to explicitly enable or disable tap through this Linkerd proxy
environment variable. The status of tap is now determined only be if the
tap-injector adds or does not add the `LINKERD2_PROXY_TAP_SVC_NAME` environment
variable.

### controller image

The tap-injector has been added to the controller image's several startup
commands which determines what it will do in the cluster.

As a follow-up, I think splitting out the `tap` and `tap-injector` commands from
the controller image into a linkerd-viz image (or something like that) makes
sense.

Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
2021-01-21 11:24:08 -05:00
Oliver Gould 6f954c3823
proxy: v2.129.0 (#5581)
This release improves diagnostics about the proxy's failfast state:

* Warnings are now emitted when the failfast state is entered;
* The "max concurrency exhausted" gRPC message has been changed to
  more-clearly indicate a failfast state error; and
* Failfast recovery has been made more robust, ensuring that a service
  can recover indepenently of new requests being received.

Furthermore, metric labeling has been improved:

* TCP server metrics are now annotated with the original `target_addr`;
* The `tls` label is now set to true for inbound TLS connections that
  lack a client ID. This is mostly helpful to clarify inbound metrics on
  the `identity` controller;
* Outbound `tls` metrics could be reported incorrectly when a proxy was
  configured to not use identity. This has been corrected.

Finally, socket-level errors now include a _client_ or _server_ prefix
to indicate which side of the proxy encountered the error.

---

* stack: remove `map_response` (linkerd/linkerd2-proxy#835)
* replace `RequestFilter` with Tower's upstream impl (linkerd/linkerd2-proxy#842)
* tracing: fix incorrect field format when logging in JSON (linkerd/linkerd2-proxy#845)
* replace `FutureService` with Tower's upstream impl (linkerd/linkerd2-proxy#839)
* integration: improve tracing in tests (linkerd/linkerd2-proxy#846)
* service-profiles: Prevent Duration coercion panics (linkerd/linkerd2-proxy#844)
* inbound: Separate HTTP server logic from protocol detection (linkerd/linkerd2-proxy#843)
* Correct gRPC 'max-concurrency exhausted' error messages (linkerd/linkerd2-proxy#847)
* Update tonic to v0.4 (linkerd/linkerd2-proxy#849)
* failfast: Improve diagnostic logging (linkerd/linkerd2-proxy#848)
* Update the base docker image (linkerd/linkerd2-proxy#850)
* stack: Implement Clone for ResultService (linkerd/linkerd2-proxy#851)
* Ensure services in failfast can become ready (linkerd/linkerd2-proxy#858)
* tests: replace string matching on metrics with parsing (linkerd/linkerd2-proxy#859)
* Decouple tls::accept from TcpStream (linkerd/linkerd2-proxy#853)
* metrics: Handle NoPeerIdFromRemote properly (linkerd/linkerd2-proxy#857)
* metrics: Reorder metrics labels (linkerd/linkerd2-proxy#856)
* Rename tls::accept to tls::server (linkerd/linkerd2-proxy#854)
* Annotate socket-level errors with a scope (linkerd/linkerd2-proxy#852)
* test: reduce repetition in metrics tests (linkerd/linkerd2-proxy#860)
* tls: Disambiguate client and server identities (linkerd/linkerd2-proxy#855)
* Update to tower v0.4.4 (linkerd/linkerd2-proxy#864)
* Update cargo dependencies (linkerd/linkerd2-proxy#865)
* metrics: add `target_addr` label for accepted transport metrics (linkerd/linkerd2-proxy#861)
* outbound: Strip endpoint identity when disabled (linkerd/linkerd2-proxy#862)

---

The opaque-ports test has been updated to reflect proxy metrics changes.
2021-01-21 06:52:38 -08:00
Oliver Gould d2ae5a8117
build: Remove the DOCKER_TRACE environment variable (#5583)
Our build scripts hide docker's output by default and only pass through
output when DOCKER_TRACE is set. Practically everyone else tends to use
DOCKER_TRACE=1 persistently. And, recently, GitHub Actions stopped
working with `/dev/stderr`

This change removes the DOCKER_TRACE environment variable so that output
is always emitted as it would when invoking docker directly.
2021-01-20 22:09:47 -08:00
Hu Shuai 08439f1f6e
Add unit test for pkg/charts/charts.go (#5565)
Add tests for MergeMap

Signed-off-by: Hu Shuai <hus.fnst@cn.fujitsu.com>
2021-01-20 09:55:01 -05:00
Tarun Pothulapati 3b755e5c1d
multicluster: add helm customization flags for install (#5534)
* multicluster: add helm customization flags

This branch updates the multicluster install flow to use the
helm engine directly instead of our own chart wrapper. This
also adds the helm customization flags.

```bash
tarun in dev in on  k3d-deep (default) linkerd2 on  tarun/mc-helm-flags [$+?] via  v1.15.4
 ./bin/go-run cli mc install --set namespace=l5d-mc | grep l5d-mc
github.com/linkerd/linkerd2/multicluster/cmd
github.com/linkerd/linkerd2/cli/cmd
  name: l5d-mc
  namespace: l5d-mc
  namespace: l5d-mc
  namespace: l5d-mc
    mirror.linkerd.io/gateway-identity: linkerd-gateway.l5d-mc.serviceaccount.identity.linkerd.cluster.local
  namespace: l5d-mc
  namespace: l5d-mc
  namespace: l5d-mc
  namespace: l5d-mc
  namespace: l5d-mc
```

* add customization flags even for link cmd

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2021-01-20 11:29:42 +05:30
Hu Shuai 37472c566f
Fix typos. (#5563)
Fix spelling: accomodate
Fix spelling: conenctions

Signed-off-by: Hu Shuai <hus.fnst@cn.fujitsu.com>
2021-01-19 15:28:57 -08:00
Risha Mars 1a5a8c0cf2
Move @rmars to emeritus maintainer (#5562)
Signed-off-by: Risha Mars <mars@buoyant.io>
2021-01-19 13:54:05 -08:00
Kevin Leimkuhler eb9b264d65
Add watch to jaeger-injector (#5548)
## What this changes

This fixes an issue in the Jaeger extension's `jaeger-injector` component that
causes an injection error in situations with high pod or namespace churn.

Because it cannot watch namespaces, it relies only off of `get` and this appears
to fall behind at a certain point. This surfaces as an error.

For example, in the `inject` test about half way through it errors with the
error:

```
=== RUN   TestInjectAutoPod
    inject_test.go:430: failed to create pod/inject-pod-test-terminus in namespace linkerd-inject-pod-test for exit status 1: Error from server: error when creating "STDIN": admission webhook "jaeger-injector.linkerd.io" denied the request: namespace "linkerd-inject-pod-test" not found
--- FAIL: TestInjectAutoPod (0.22s)
FAIL
```

Looking at the `jaeger-injector` logs, most of it's messages are about the test
namespaces not being created:

```
..
time="2021-01-15T15:34:12Z" level=info msg="received admission review request b2f36a9c-3f88-4abe-bcaa-f63c61cd24c0"
time="2021-01-15T15:34:12Z" level=info msg="received admission review request 9f5b229b-1c60-4b24-a020-b66cd201171e"
time="2021-01-15T15:34:12Z" level=error msg="failed to run webhook handler. Reason: namespace \"linkerd-inj-auto-params-test\" not found"
time="2021-01-15T15:34:12Z" level=info msg="received admission review request ae00d63a-1585-46ba-9a75-1f93d40766a8"
time="2021-01-15T15:34:12Z" level=info msg="received admission review request 998721eb-5625-4be8-9166-9db834c58f10"
time="2021-01-15T15:34:12Z" level=error msg="failed to run webhook handler. Reason: namespace \"linkerd-inj-auto-params-test\" not found"
time="2021-01-15T15:34:12Z" level=info msg="received admission review request 52e4e603-89b1-492b-a69b-dc8ff67d5f26"
time="2021-01-15T15:34:12Z" level=info msg="received admission review request 27558a16-5120-4aeb-a0bd-f22a1666b2b1"
time="2021-01-15T15:34:12Z" level=error msg="failed to run webhook handler. Reason: namespace \"linkerd-inj-auto-params-test\" not found"
..
```

Adding the `watch` verb to it's cluster role fixes this and these errors no
longer occur.

Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
2021-01-19 09:42:28 -05:00
Yashvardhan Kukreja b67bbe157b
add jaeger check: to confirm whether the jaeger injector pod is in running state or not (#5528)
Currently, the linkerd jaeger check runs multiple checks but it doesn't have a check to confirm the state of the jaeger injector to be running.

This commit adds that required check to confirm the running state of the jaeger injector pod.

Fixes #5495

Signed-off-by: Yashvardhan Kukreja <yash.kukreja.98@gmail.com>
2021-01-19 08:35:16 +05:30
Oleh Ozimok c416e78261
destination: Fix crash when EndpointSlices are enabled (#5543)
The Destination controller can panic due to a nil-deref when
the EndpointSlices API is enabled.

This change updates the controller to properly initialize values
to avoid this segmentation fault.

Fixes #5521

Signed-off-by: Oleg Ozimok <oleg.ozimok@corp.kismia.com>
2021-01-15 12:52:11 -08:00
Tarun Pothulapati 0a2f1f3a26
viz: add check sub-command (#5496)
* viz: add check sub-command

This adds a new `viz check` cmd performing checks for the resources
in linkerd-viz extension. Checks include resource checks and
the health of resources, certs, etc

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2021-01-15 15:31:45 -05:00
Eugene Formanenko 535a36af7c
Add log-format flag to control plane components (#5537)
Fixes #5536

Signed-off-by: Eugene Formanenko <mo4islona@gmail.com>
2021-01-15 10:51:32 -05:00
Alejandro Pedraza d7e4f901e6
Detect default ns for metrics, identity, routes and profile subcommands (#5530)
* Detect default ns for metrics and profile subcommands

Followup to #5485, fixes remaining cases for #5524

Properly detect the default namespace given `kubeConfigPath` and
`kubeContext` for the `metrics`, `identity`, `routes` and `profile` subcommands.

Also gets rid once and for all of the `defaultNamespace` global var.
2021-01-15 08:51:26 -05:00
Alejandro Pedraza 3365e98f13
Have 'bin/test-cleanup' clean the viz helm release (#5542)
This is needed for the tests in the ARM box to pass.
2021-01-15 00:40:37 +05:30
Tarun Pothulapati 536bdf245c
Add changes for edge-21.1.2 (#5538)
## edge-21.1.2

This edge release continues the work on decoupling non-core Linkerd components.
Commands that use the viz-extension i.e, `dashboard`, `edges`, `routes`,
`stat`, `tap` and `top` are moved to the `viz` sub-command. These commands are still
available under root but are marked deprecated and will be removed in a
later stable release.

This release also features proxy's dependencies upgrade to the
Tokio v1 ecosystem.

* Moved sub-commands that use the viz-extension under `viz`
* Started ignoring pods with status.phase=Succeeded when watching IP addresses
  in destination. This is useful for re-use of IPs of terminated pods
* Support Bring your own Jaeger use-case by adding `collector.jaegerAddr` in
  the jaeger extension.
* Fixed an issue with the generation of working manifests in the
  `podAntiAffinity` use-case
* Added support for the modification of proxy resources in the viz
  extension through `values.yaml` in Helm and flags in CLI.
* Improved error reporting for port-forward logic with namespace
  and pod data, used across dashboard, checks, etc
  (thanks @piyushsingariya)
* Added support to disable the rendering of `linkerd-viz` namespace
  resource in the viz extension (thanks @nlamirault)
* Made service-profile generation work offline with `--ignore-cluster`
  flag (thanks @piyushsingariya)
* Proxy's Tap API is disabled by default and it is enabled only when
  `LINKERD2_PROXY_TAP_SVC_NAME` configuration is set. This means that
  `LINKERD2_PROXY_TAP_DISABLED` is no longer honored
* Upgraded the proxy's dependencies to Tokio v1 ecosystem

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2021-01-14 22:16:39 +05:30
Alejandro Pedraza cf143f2068
Revert "Add default ns detection to 'linkerd identity' and fixed --namespace description"
This reverts commit 5966d7c6b6.
2021-01-14 10:09:20 -05:00
Alejandro Pedraza 5966d7c6b6
Add default ns detection to 'linkerd identity' and fixed --namespace description 2021-01-14 10:07:22 -05:00
Alejandro Pedraza dd9ea0aef4
Helm template helpers cleanup (#5514)
Removed Helm template files no longer used, as well as some helper
functions.
2021-01-14 09:05:31 -05:00
Tarun Pothulapati eeaf4a5359
viz: make viz cmds available at root (#5525)
* viz: make viz cmds available at root

Fixes #5523

This branch makes viz commands that were previously available
under root to be available at both places i.e `linkerd` and
`linkerd viz`.

We also show a depreciated notice when ran under root, asking
to use them with the `viz` prefix.

This also updates all the help messages to address these cmds
as `linkerd viz xyz` instead of `linkerd xyz`

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2021-01-14 13:14:25 +05:30
Kevin Ingelman 4004125da7
Move @klingerf to emeritus maintainer (#5533)
While he is still overwhelmingly excited about the project, @klingerf
isn't participating in the day-to-day tasks outlined in the updated
GOVERNANCE.md, and therefore requests to be moved to emeritus status.

Signed-off-by: Kevin Ingelman <ki@buoyant.io>
2021-01-13 15:45:53 -08:00
Oliver Gould 6c9745c60d
proxy: v2.128.0 (#5531)
This release primarily features an upgrade to the Tokio v1 ecosystem.

Additionally, the `LINKERD2_PROXY_TAP_DISABLED` configuration is no
longer honored. Now, the proxy's Tap API is disabled by default and it
is only enabled when the `LINKERD2_PROXY_TAP_SVC_NAME` configuration is
set (when identity is not disabled for the proxy).

---

* Add a license to all cargo manifests (linkerd/linkerd2-proxy#832)
* Remove dependency on tower-request-modifier (linkerd/linkerd2-proxy#829)
* Update ipnet to v2 (linkerd/linkerd2-proxy#830)
* Remove the unused app-profiling crate (linkerd/linkerd2-proxy#827)
* update to Tokio 1.0 (linkerd/linkerd2-proxy#826)
* Use cargo-deny to audit dependencies (linkerd/linkerd2-proxy#833)
* Rename crates to reflect directory structure (linkerd/linkerd2-proxy#834)
* Represent local identity as an Option (linkerd/linkerd2-proxy#836)
* Split linkerd-tls from linkerd-proxy-transport (linkerd/linkerd2-proxy#837)
* stack: use `LayerFn` from upstream (linkerd/linkerd2-proxy#840)
* Remove tap status being explicitly set in config (linkerd/linkerd2-proxy#838)
* Split linkerd-detect from linkerd-proxy-transport (linkerd/linkerd2-proxy#841)
2021-01-13 11:58:14 -08:00
Oliver Gould 9e7c946dc0
Update Linkerd Governance (#5477)
The governance structure documented is `GOVERNANCE.md` is no longer
suitable for the project and doesn't reflect the reality of how changes
are made.

This change proposes an updated, simplified governance structure that
clearly outlines the expectations for maintainers around project
participation and decision making. It is expected that *most*
contributions will not come from maintainers; but we need a core group
of maintainers that are ultimately responsible for technical stewardship
of the project.
2021-01-13 11:51:15 -08:00
Alejandro Pedraza f3b1ebfa99
Separate observability API (#5510)
* Separate observability API

Closes #5312

This is a preliminary step towards moving all the observability API into `/viz`, by first moving its protobuf into `viz/metrics-api`. This should facilitate review as the go files are not moved yet, which will happen in a followup PR. There are no user-facing changes here.

- Moved `proto/common/healthcheck.proto` to `viz/metrics-api/proto/healthcheck.prot`
- Moved the contents of `proto/public.proto` to `viz/metrics-api/proto/viz.proto` except for the `Version` Stuff.
- Merged `proto/controller/tap.proto` into `viz/metrics-api/proto/viz.proto`
- `grpc_server.go` now temporarily exposes `PublicAPIServer` and `VizAPIServer` interfaces to separate both APIs. This will get properly split in a followup.
- The web server provides handlers for both interfaces.
- `cli/cmd/public_api.go` and `pkg/healthcheck/healthcheck.go` temporarily now have methods to access both APIs.
- Most of the CLI commands will use the Viz API, except for `version`.

The other changes in the go files are just changes in the imports to point to the new protobufs.

Other minor changes:
- Removed `git add controller/gen` from `bin/protoc-go.sh`
2021-01-13 14:34:54 -05:00
Alex Leong 5df48e9ea8
Add collector.jaegerAddr value to allow BYOJ (#5499)
Users may have an existing Jaeger deployment and want to send traces to it from Linkerd.

We add the `collector.jaegerAddr` value to the Linkerd-Jaeger chart which configures the address of the jaeger backend which the opencensus collector sends to.  If left unspecified, the collector will use the jaeger instance in the linkerd-jaeger extension.

To test:

Install Jaeger backend separately:

```
curl https://raw.githubusercontent.com/jaegertracing/jaeger-operator/master/deploy/examples/simplest.yaml | docker run -i --rm jaegertracing/jaeger-operator:master generate | kubectl apply -n jaeger-test -f -
```

Install Linkerd and Linkerd-jaeger, specifying the existing jaeger backend

```
linkerd install | kubectl apply -f -
linkerd jaeger install --set collector.jaegerAddr='http://my-jaeger-collector.jaeger-test:14268/api/traces' | kubectl apply -f -
```

Install emojivoto and configure it:

```
linkerd inject https://run.linkerd.io/emojivoto.yml  | kubectl apply -f -
kubectl -n emojivoto set env --all deploy OC_AGENT_HOST=collector.linkerd-jaeger:55678
```

View traces in your custom jaeger backend:

```
kubectl -n jaeger-test port-forward svc/my-jaeger-query 16686 &
open http://localhost:16686
```

Signed-off-by: Alex Leong <alex@buoyant.io>
2021-01-13 11:22:05 -08:00
Tarun Pothulapati 4c3d002501
viz: move sub-cmds using viz extension under viz cmd (#5485)
* viz: move sub-cmds using viz extension under viz cmd

Fixes #5327 , #5524 

This branch moves the following commands, under the `linkerd viz`
cmd as they use the viz extension to perform the job.

- dashboard
- edges
- routes
- stat
- tap
- top

This also creates a new pkg `public-api` which fecilitates
interaction and communication with public-api to be used
across extensions.

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
Co-authored-by: Alex Leong <alex@buoyant.io>
2021-01-13 12:11:25 +05:30
Yashvardhan Kukreja 06dccac35b
cleanup: utilise linkerd uninstall to concisely delete all the linkerd resources involved in the test (#5522)
The linkerd uninstall command is able to remove a lot of the test resources used in CI but it ends up leaving the test namespaces though.
Still, the test-cleanup script can be cleaned down to a good level by getting rid of the populate_array function.

Hence, this commits adds a one-liner, alongside linkerd uninstall, to deal with the deletion of all the test namespaces and the resources instead of using the big chunk of populate_array function.

Fixes #5497

Signed-off-by: Yashvardhan Kukreja <yash.kukreja.98@gmail.com>
2021-01-12 18:51:27 -05:00
Filip Petkovski 40192e258a
Ignore pods with status.phase=Succeeded when watching IP addresses (#5412)
Ignore pods with status.phase=Succeeded when watching IP addresses

When a pod terminates successfully, some CNIs will assign its IP address
to newly created pods. This can lead to duplicate pod IPs in the same
Kubernetes cluster.

Filter out pods which are in a Succeeded phase since they are not 
routable anymore.

Fixes #5394

Signed-off-by: fpetkovski <filip.petkovsky@gmail.com>
2021-01-12 12:25:37 -05:00
cpretzer 5e7586340b
update pod affinity key for tap (#5519)
Signed-off-by: Charles Pretzer <charles@buoyant.io>
2021-01-12 08:59:37 -08:00
Alejandro Pedraza a9317af3d8
Add back support for proxy resource settings (#5517)
The last viz refactoring removed support for modifying the k8s resources
used by the proxies injected into the control plane components (values
like `tapProxyResources`, `prometheus.proxy.resources`, etc).

This adds them back, using a consistent naming: `tap.proxy.resources`,
`dashboard.proxy.resources`, etc.

Also fixes the tap helm template that was making reference to
`.Values.tapResources` instead of `.Values.tap.resources`.

Co-authored-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
2021-01-12 10:56:43 -05:00
Tarun Pothulapati 836c077898
viz: add render golden tests (#5433)
* viz: add render golden tests

This branch adds golden tests for the viz install. This would be
useful to track changes in render as more changes are added.

This also moves the common code that is used across extensions
to generate diffs into `testutil` to be able to be used widely.

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2021-01-12 11:59:16 +05:30
Piyush Singariya aa20c3e88e
Report namespace and pod name when port-forward fails (#5504)
Subject
Related to issue #5457 

Problem
Linkerd only reports the local port and the remote port whenever port-forwarding fails.
Linkerd could print out namespace and port if port-forwarding fails instead of just at the error state and then force users to collate the port themselves

Solution
Linkerd needs to print the namespace and the pod name.
- [x] Add two new string variables namespace and podName in `struct PortForward`
- [x] assign the values to the variables when a new Instance is being created in `func NewPortForward()`
run() function propagates the errors that occurred while port-forwarding
- [x] Format the error being returned by `ForwardPorts()` from client-go using `fmt.Errorf()` and add `namespace` and `podName` as suffix and return error

The error is being returned by ForwardPorts() from client-go https://github.com/kubernetes/client-go/blob/master/tools/portforward/portforward.go#L188

Fixes #5457 

Signed-off-by: Piyush Singariya <piyushsingariya@gmail.com>
2021-01-11 15:49:27 -08:00
Alejandro Pedraza 898de71098
Enable upgrade integration tests for ARM (#5513)
This enables the `helm-upgrade` and `upgrade-stable` integration tests,
that were disabled because the previous versions didn't have ARM
support, but now 2.9 does.
2021-01-11 17:34:05 -05:00
Nicolas Lamirault 8eab38d15e
Allow to disable linkerd-viz namespace installation with Helm (#5509)
If namespace is manage by an external tool , it fails on install. 
Add a feature to not manage namespace by Helm.

Signed-off-by: Nicolas Lamirault <nicolas.lamirault@gmail.com>
2021-01-11 13:31:22 -05:00
Piyush Singariya b5dddf5daf
service profile generation work offline using --ignore-cluster (#5482)
Closes #5401 

* offline profile generation with --ignore-cluster
* validation added for ignoreCluster and service profile with tap data
Signed-off-by: Piyush Singariya <piyushsingariya@gmail.com>
2021-01-09 10:23:25 -08:00
Kevin Leimkuhler 308a1f3ff3
Use linkerd path in test-cleanup (#5498)
## What this fixes

When clusters are cleaned up after tests in CI, the `bin/test-cleanup` script is
responsible for clearing the cluster of all testing resources.

Right now this does not work as expected because the script uses the `linkerd`
binary instead of the Linkerd path that is passed in to the `tests` script.

There are cases where different binaries have different uninstall behavior and
the script can complete with an incomplete uninstallation.

## How it fixes

`test-cleanup` now takes a linkerd path argument. This is used to specify the
Linkerd binary that should be used when running in the `uninstall` commands.

This value is passed through from the `tests` invocation which means that in CI,
the same binary is used for running tests as well as cleaning up the cluster.

Additionally, specifying the k8s context has now moved from an argument to the
`--context` flag. This is similar to how `tests` script works because it's not
always required.

## How to use

Shown here:

``` $ bin/test-cleanup -h Cleanup Linkerd integration tests.

Usage:
    test-cleanup [--context k8s_context] /path/to/linkerd

Examples:
    # Cleanup tests in non-default context test-cleanup --context k8s_context
    /path/to/linkerd

Available Commands:
    --context: use a non-default k8s context
```

## edge-21.1.1

This edge release introduces a new "opaque transport" feature that allows the
proxy to securely transport server-speaks-first and otherwise opaque TCP
traffic. Using the `config.linkerd.io/opaque-ports` annotation on pods and
namespaces, users can configure ports that should skip the proxy's protocol
detection.

Additionally, a new `linkerd-viz` extension has been introduced that separates
the installation of the Grafana, Prometheus, web, and tap components. This
extension closely follows the Jaeger and multicluster extensions; users can
`install` and `uninstall` with the `linkerd viz ..` command as well as configure
for HA with the `--ha` flag.

The `linkerd viz install` command does not have any cli flags to customize the
install directly, but instead follows the Helm way of customization by using
flags such as `set`, `set-string`, `values`, `set-files`.

Finally, a new `/shutdown` admin endpoint that may only be accessed over the
loopback network has been added. This allows batch jobs to gracefully terminate
the proxy on completion. The `linkerd-await` utility can be used to automate
this.

* Added a new `linkerd multicluster check` command to validate that the
  `linkerd-multicluster` extension is working correctly
* Fixed description in the `linkerd edges` command (thanks @jsoref!)
* Moved the Grafana, Prometheus, web, and tap components into a new Viz chart,
  following the same extension model that multicluster and Jaeger follow
* Introduced a new "opaque transport" feature that allows the proxy to securely
  transport server-speaks-first and otherwise opaque TCP traffic
* Removed the check comparing the `ca.crt` field in the identity issuer secret
  and the trust anchors in the Linkerd config; these values being different is
  not a failure case for the `linkerd check` command (thanks @cypherfox!)
* Removed the Prometheus check from the `linkerd check` command since it now
  depends on a component that is installed with the Viz extension
* Fixed error messages thrown by the cert checks in `linkerd check` (thanks
  @pradeepnnv!)
* Added PodDisruptionBudgets to the control plane components so that they cannot
  be all terminated at the same time during disruptions (thanks @tustvold!)
* Fixed an issue that displayed the wrong `linkerd.io/proxy-version` when it is
  overridden by annotations (thanks @mateiidavid!)
* Added support for custom registries in the `linkerd-viz` helm chart (thanks
  @jimil749!)
* Renamed `proxy-mutator` to `jaeger-injector` in the `linkerd-jaeger` extension
* Added a new `/shutdown` admin endpoint that may only be accessed over the
  loopback network allowing batch jobs to gracefully terminate the proxy on
  completion
* Introduced the `linkerd identity` command, used to fetch the TLS certificates
  for injected pods (thanks @jimil749)
* Fixed an issue with the CNI plugin where it was incorrectly terminating and
  emitting error events (thanks @mhulscher!)
* Re-added support for non-LoadBalancer service types in the
  `linkerd-multicluster` extension

Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
2021-01-08 15:24:14 -05:00