Commit Graph

418 Commits

Author SHA1 Message Date
Kevin Lingerfelt 0dcd69c465
Re-add pre-install permission checks (#2451)
* Re-add pre-install permission checks
* Fix ordering in check.go

Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2019-03-05 19:17:21 -08:00
Andrew Seigner d90fa16727
Introduce NET_ADMIN cli check (#2421)
The `linkerd-init` container requires the NET_ADMIN capability to modify
iptables. The `linkerd check` command was not verifying this.

Introduce a `has NET_ADMIN capability` check, which does the following:
1) Lists all available PodSecurityPolicies, if none found, returns
success
2) For each PodSecurityPolicy, validate one exists that:
    - the user has `use` access AND
    - provides `*` or `NET_ADMIN` capability

A couple limitations to this approach:
- It is testing whether the user running `linkerd check` has NET_ADMIN,
  but during installation time it will be the `linkerd-init` pod that
  requires NET_ADMIN.
- It assumes the presense of PodSecurityPolicies in the cluster means
  the PodSecurityPolicy admission controller is installed. If the
  admission controller is not installed, but PSPs exists that restrict
  NET_ADMIN, `linkerd check` will incorrectly report the user does not
  have that capability.

This PR also fixes the `can create CustomResourceDefinitions` check to
not specify a namespace when doing a `create` check, as CRDs are
cluster-wide.

Fixes #1732

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-03-05 17:19:11 -08:00
Ivan Sim 8f9473fbd7
Recreate the MWC when the proxy injector is restarted (#2431)
This ensures that the MWC always picks up the latest config template during version upgrade.
The removed `update()` method and RBAC permissions are superseded by @2163.

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-03-05 15:53:37 -08:00
Andrew Seigner 206ff685e2
Bump Prometheus client to v0.9.2 (#2388)
We were depending on an untagged version of prometheus/client_golang
from Feb 2018.

This bumps our dependency to v0.9.2, from Dec 2018.

Also, this is a prerequisite to #1488.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-03-05 10:31:16 -08:00
Andrew Seigner 2f89a64abd
Remove GOPATH dependency from cli dev environment (#2444)
The `linkerd install` output relies on Helm templates in the `chart`
directory. In production cli builds, these templates are compiled into
the binary. In development, they are read from the file system. This
development code path relied on GOPATH to determine the location of the
`chart` directory. In anticipation of Go Modules support (#1488), we
cannot assume the repo is within the GOPATH.

This change removes the GOPATH dependency, and instead relies on
`runtime.Caller` to determine the root of the code repo. This change
only affects development (!prod) builds.

Prerequisite to #1488.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-03-05 10:25:14 -08:00
Alejandro Pedraza ddf2e729ac
Injection consolidation (#2334)
- Created the pkg/inject package to hold the new injection shared lib.
- Extracted from `/cli/cmd/inject.go` and `/cli/cmd/inject_util.go`
the core methods doing the workload parsing and injection, and moved them into
`/pkg/inject/inject.go`. The CLI files should now deal only with
strictly CLI concerns, and applying the json patch returned by the new
lib.
- Proceeded analogously with `/cli/cmd/uninject.go` and
`/pkg/inject/uninject.go`.
- The `InjectReport` struct and helping methods were moved into
`/pkg/inject/report.go`
- Refactored webhook to use the new injection lib
- Removed linkerd-proxy-injector-sidecar-config ConfigMap
- Added the ability to add pod labels and annotations without having to
specify the already existing ones

Fixes #1748, #2289

Signed-off-by: Alejandro Pedraza <alejandro.pedraza@gmail.com>
2019-03-05 08:38:56 -05:00
Ivan Sim 28fb725901
Revert "Mark HA as non-experimental (#2430)" (#2436)
This reverts commit ec51ccf317.

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-03-04 08:27:43 -08:00
Aditya Sharma ec51ccf317 Mark HA as non-experimental (#2430)
Closes #2419

Signed-off-by: Aditya Sharma <hello@adi.run>
2019-03-02 23:02:20 -08:00
Tarun Pothulapati 2184928813 Wire up stats for Jobs (#2416)
Support for Jobs in stat/tap/top cli commands

Part of #2007

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2019-03-01 17:16:54 -08:00
Andrew Seigner d08dcb0a37
Skip outbound port 443 in control-plane (#2411)
linkerd/linkerd2#2349 introduced a `SelfSubjectAccessReview` check at
startup, to determine whether each control-plane component should
establish Kubernetes watches cluster-wide or namespace-wide. If this
check occurs before the linkerd-proxy sidecar is ready, it fails, and
the control-plane component restarts.

This change configures each control-plane pod to skip outbound port 443
when injecting the proxy, allowing the control-plane to connect to
Kubernetes regardless of the `linkerd-proxy` state.

A longer-term fix should involve a more robust control-plane startup,
that is resilient to failed Kubernetes API requests. An even longer-term
fix could involve injecting `linkerd-proxy` as a Kubernetes "sidecar"
container, when that becomes available.

Workaround for #2407

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-27 15:23:19 -08:00
Kevin Lingerfelt 40076c4de2
Remove namespace from serviceprofile CRD in install config (#2409)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2019-02-27 14:29:47 -08:00
Oliver Gould ab90263461
destination: Only return TLS identities when appropriate (#2371)
As described in #2217, the controller returns TLS identities for results even
when the destination pod may not be able to participate in identity
requester: specifically, the other pod may not have the same controller
namespace or it may not be injected with identity.

This change introduces a new annotation, linkerd.io/identity-mode that is set
when injecting pods (via both CLI and webhook). This annotation is always
added.

The destination service now only returns TLS identities when this annotation
is set to optional on a pod and the destination pod uses the same controller.
These semantics are expected to change before the 2.3 release.

Fixes #2217
2019-02-27 12:18:39 -08:00
Andrew Seigner 9f748d2d2e
lint: Enable unparam (#2369)
unparam reports unused function parameters:
https://github.com/mvdan/unparam

Part of #217

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-27 10:34:02 -08:00
Andrew Seigner ec5a0ca8d9
Authorization-aware control-plane components (#2349)
The control-plane components relied on a `--single-namespace` param,
passed from `linkerd install` into each individual component, to
determine which namespaces they were authorized to access, and whether
to support ServiceProfiles. This command-line flag was redundant given
the authorization rules encoded in the parent `linkerd install` output,
via [Cluster]Role[Binding]s.

Modify the control-plane components to query Kubernetes at startup to
determine which namespaces they are authorized to access, and whether
ServiceProfile support is available. This allows removal of the
`--single-namespace` flag on the components.

Also update `bin/test-cleanup` to cleanup the ServiceProfile CRD.

TODO:
- Remove `--single-namespace` flag on `linkerd install`, part of #2164

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-26 11:54:52 -08:00
Andrew Seigner 25e462352d
lint: Enable goimports (#2366)
goimports checks import lines, adding missing ones and removing
unreferenced ones:
https://godoc.org/golang.org/x/tools/cmd/goimports

It also requires named imports for packages whose
import paths don't match their package names:
- https://github.com/golang/go/issues/28428
- https://go-review.googlesource.com/c/tools/+/145699/

Also standardized named imports of common Kubernetes packaages.

Part of #217

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-25 15:51:10 -08:00
Oliver Gould 9e67bcb1bc
inject: Refactor report-checking from inject logic (#2379)
The inject logic combines the modification of a pod spec and the
creation of a "report" detailing problems with the pod spec.

This change extracts the report-creation-and-checking logic from the
injection logic to make the contracts of each of these functions
clearer.

No functional changes are intended.
2019-02-25 14:18:22 -08:00
Andrew Seigner 35a0b652f2
lint: Enable goconst (#2365)
goconst finds repeated strings that could be replaced by a constant:
https://github.com/jgautheron/goconst

Part of #217

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-25 12:00:03 -08:00
Risha Mars 80b6e41d5d
Modify StatSummary to also return TCP stats (#2262)
Adds a flag, tcp_stats to the StatSummary request, which queries prometheus for TCP stats.
This branch returns TCP stats at /api/tps-reports when this flag is true.

TCP stats are now displayed on the Resource Detail pages.

The current queried TCP stats are:
tcp_open_connections
tcp_read_bytes_total
tcp_write_bytes_total
2019-02-25 10:37:39 -08:00
Oliver Gould f7435800da
lint: Enable scopelint (#2364)
[scopelint][scopelint] detects a nasty reference-scoping issue in loops.

[scopelint]: https://github.com/kyoh86/scopelint
2019-02-24 08:59:51 -08:00
Andrew Seigner cc3ff70f29
Enable `unused` linter (#2357)
`unused` checks Go code for unused constants, variables, functions, and
types.

Part of #217

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-23 11:05:39 -08:00
Andrew Seigner 4b6f6aeedd
Enable gosimple linter, fix issues (#2356)
gosimple is a Go linter that specializes in simplifying code

Also fix one spelling error in `cred_test.go`

Part of #217

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-22 17:19:07 -08:00
Andrew Seigner 43d29d629e
Bump base Docker images (#2241)
- `debian:jessie-slim` -> `stretch-20190204-slim`
- `golang:1.10.3` -> `1.11.5`
- `gcr.io/linkerd-io/base:2017-10-30.01` -> `2019-02-19.01`
- bump `golangci-lint` to 1.15.0
- use `GOCACHE` in travis

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-22 15:59:18 -08:00
Ivan Sim 1e2e2bf53c
Install the Linkerd global and proxy config maps (#2344)
Also, some protobuf updates:

* Rename `api_port` to match recent changes in CLI code.
* Remove the `cni` message because it won't be used.
* Remove `registry` field from proto types. This helps to avoid having to workaround edge cases like fully-qualified image name in different format, and overriding user-specified Linkerd version etc.

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-02-22 15:28:21 -08:00
TwiN 5dcb9086f5 Implement resource limit (#2307)
Add options in CLI for setting proxy CPU and memory limits

- Deprecated `proxy-cpu` and `proxy-memory` in favor of `proxy-cpu-limit` and `proxy-memory-limit`
- Updated validations and tests to reflect new options

Signed-off-by: TwinProduction <twin@twinnation.org>
2019-02-20 15:07:10 -05:00
Oliver Gould fc82e2caf5
test: Support the -update and -pretty-diff flags (#2315)
When changing templates, it's can be pretty time-intensive to
repair all test fixtures.

This change instruments CLI tests with two flags, `-update` and
`-pretty-diff` that control how test fixtures are diffed. When the
`-update` flag is set, the tests fixtures are overwritten as tests
execute. The `-pretty-diff` flag causes the full text of the fixture
to be printed on mismatch.
2019-02-19 13:06:12 -08:00
Tarun Pothulapati 42b3595f57 Ignoring resources with empty kind field. (#2292)
Update CLI inject/uninject commands to report warnings when resource `kind` is missing.

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2019-02-19 11:44:28 -08:00
Kevin Lingerfelt 98c2686c95
CLI help updates: unhide install-cni (#2321)
* Un-hide linkerd-cni command and flags
* Tweak install-cni and install-sp descriptions

Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2019-02-18 18:13:26 -08:00
Kevin Lingerfelt 5384ca8c97
Add discovery package for managing discovery API (#2317)
* Add discovery package for managing discovery API
* Fix typo in destination server comment

Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2019-02-18 16:38:04 -08:00
Kevin Lingerfelt 25e46a3e74
CLI help updates: non-experimental auto-inject; unhide install-cni (#2319)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2019-02-18 15:32:46 -08:00
Oliver Gould 4ed84f0c0a
Split install template into component-specific files (#2313)
chart/templates/base.yaml is nearly 800 lines and contains the
kubernetes configurations for the marjority of the control plane.
Furthermore, its contents are not particularly organized (for example,
the prometheus RBAC bindings are in the middle of the controller's
configuration).

The size and complexity of this file makes it especially daunting to
introduce new functionality.

In order to make the situation easier to understand and change, this
splits base.yaml into several new template files: namespace, controller,
serviceprofile, and prometheus, and grafana. The `tls.yaml` template has
been renamed `ca.yaml`, since it installs the `linkerd-ca` resources.

This change also makes the comments uniform, adding a "header" to each
logical component.

Fixes #2154
2019-02-18 15:31:17 -08:00
Thomas Rampelberg 12527169b8
Update profile to match other CLI doc outputs (#2318)
* Update profile to match other CLI doc outputs

* Fix typo
2019-02-18 13:47:41 -08:00
Oliver Gould 71ce786dd3
Rename linkerd-proxy-api to linkerd-destination (#2281)
Up until now, the proxy-api controller service has been the sole service
that the proxy communicates with, implementing the majoriry of the API
defined in the `linkerd2-proxy-api` repo. But this is about to change:
linkerd/linkerd2-proxy-api#25 introduces a new Identity service; and
this service must be served outside of the existing proxy-api service
in the linkerd-controller deployment (so that it may run under a
distinct service account).

With this change, the "proxy-api" name becomes less descriptive. It's no
longer "the service that serves the API for the proxy," it's "the
service that serves the Destination API to the proxy." Therefore, it
seems best to bite the bullet and rename this to be the "destination"
service (i.e. because it only serves the
`io.linkerd.proxy.destination.Destination` service).

Co-authored-by: Kevin Lingerfelt <kl@buoyant.io>
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2019-02-15 15:11:04 -08:00
Thomas Rampelberg f9d353ea22
Generate CLI docs for usage by the website (#2296)
* Generate CLI docs for usage by the website

* Update description to match existing commands

* Remove global
2019-02-15 13:28:31 -08:00
Tarun Pothulapati df0357505c Added ep Alias for endpoints command (#2293)
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2019-02-14 17:36:55 -08:00
Alejandro Pedraza c78f105350
Upgrade Spinner to fix race condition (#2265)
Fixes #2264

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-02-14 09:51:25 -05:00
Alejandro Pedraza 0c50749990
Add validator for proxy log-level (#2256)
Add validator for proxy log-level

Follow-up to #2249

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-02-13 15:33:22 -05:00
Andrew Seigner a9b9908908
Bump Prometheus to v2.7.1, Grafana to 5.4.3 (#2242)
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-13 11:27:15 -08:00
Andrew Seigner 2305974202
Introduce golangci-lint tooling, fixes (#2239)
`golangci-lint` performs numerous checks on Go code, including golint,
ineffassign, govet, and gofmt.

This change modifies `bin/lint` to use `golangci-lint`, and replaces
usage of golint and govet.

Also perform a one-time gofmt cleanup:
- `gofmt -s -w controller/`
- `gofmt -s -w pkg/`

Part of #217

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-13 11:16:28 -08:00
Alejandro Pedraza 4dc204ae51
Honour install's `proxy-log-level` flag when autoinjecting proxies (#2249)
Fixes #2248

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-02-11 14:23:09 -05:00
Kevin Lingerfelt 26aa771482
Fix auto-inject config when TLS is disabled (#2246)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2019-02-11 11:01:18 -08:00
Andrew Seigner 1a60237a94
Update check command hint URLs to new alias (#2245)
The existing hint URLs printing by `linkerd check` pointed to locations
that would change if the linkerd.io website was reorganized.

linkerd/website#148 introduces an alias for hint URLs at
https://linkerd.io/checks/. This is the corresponding change to update
`linkerd check` output.

Depends on linkerd/website#148, relates to linkerd/website#146.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-11 11:00:16 -08:00
Ivan Sim f6e75ec83a
Add statefulsets to the dashboard and CLI (#2234)
Fixes #1983

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-02-08 15:37:44 -08:00
Alex Leong 5b054785e5
Read service profiles from client or server namespace instead of control namespace (#2200)
Fixes #2077 

When looking up service profiles, Linkerd always looks for the service profile objects in the Linkerd control namespace.  This is limiting because service owners who wish to create service profiles may not have write access to the Linkerd control namespace.

Instead, we have the control plane look for the service profile in both the client namespace (as read from the proxy's `proxy_id` field from the GetProfiles request and from the service's namespace.  If a service profile exists in both namespaces, the client namespace takes priority.  In this way, clients may override the behavior dictated by the service.

Signed-off-by: Alex Leong <alex@buoyant.io>
2019-02-07 14:51:43 -08:00
Andrew Seigner 907f01fba6
Improve ServiceProfile validation in linkerd check (#2218)
The `linkerd check` command was doing limited validation on
ServiceProfiles.

Make ServiceProfile validation more complete, specifically validate:
- types of all fields
- presence of required fields
- presence of unknown fields
- recursive fields

Also move all validation code into a new `Validate` function in the
profiles package.

Validation of field types and required fields is handled via
`yaml.UnmarshalStrict` in the `Validate` function. This motivated
migrating from github.com/ghodss/yaml to a fork, sigs.k8s.io/yaml.

Fixes #2190
2019-02-07 14:35:47 -08:00
Andrew Seigner 72812baf99
Introduce Discovery API and endpoints command (#2195)
The Proxy API service lacked introspection of its internal state.

Introduce a new gRPC Discovery API, implemented by two servers:
1) Proxy API Server: returns a snapshot of discovery state
2) Public API Server: pass-through to the Proxy API Server

Also wire up a new `linkerd endpoints` command.

Fixes #2165

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-07 14:02:21 -08:00
Cody Vandermyn 74eac764d8 CNI: Removed unnecessary tolerations; added created-by annotation. (#2222)
Removed unnecessary tolerations; added created-by annotation.

Signed-off-by: Cody Vandermyn <cody.vandermyn@nordstrom.com>
2019-02-07 13:36:00 -08:00
Risha Mars e531655d26
Add a --tap flag to the linkerd profile command (#2139)
Adds the ability to generate a service profile by running a tap for a configurable 
amount of time, and using the route results from the routes seen during the tap.

e.g. `linkerd profile web --tap deploy/web -n emojivoto --tap-duration 2s`
2019-02-06 12:43:16 -08:00
Alejandro Pedraza 79b875a1e8
If GOPATH env var not set, use build.Default.GOPATH (#2215)
Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-02-06 13:23:48 -05:00
Kevin Lingerfelt 02f128ec5a
Reorganize install/inject config structs (#2202)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2019-02-05 12:50:42 -08:00
Alejandro Pedraza 2a7654ce78
Consolidate timeouts for `linkerd check` (#2191)
Consolidate timeouts for `linkerd check`

- Moved the creation of contexts from inside the methods targeted by the
checks into a single place in the runCheck() and runCheckRPC() methods
where the context is built using a hard-coded timeout of 30 seconds.
- k8s' client-go doesn't allow passing along contexts, but it let's us
setting the Timeout manually.
- Reworded the description for the --wait option.

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-02-05 11:38:30 -05:00
Kevin Leimkuhler 66070c26f4
Introduce go generate to embed static templates (#2189)
# Problem
In order to switch Linkerd template rendering to use `.yaml` files, static
assets must be bundled in the Go binary for use by `linkerd install`.

# Solution
The solution should not affect the local development process of building and
testing.

[vfsgen](https://github.com/shurcooL/vfsgen) generates Go code that statically
implements the provided `http.FileSystem`. Paired with `go generate` and Go
[build tags](https://golang.org/pkg/go/build/), we can continue to use the
template files on disk when developing with no change required.

In `!prod` Go builds, the `cli/static/templates.go` file provides a
`http.FileSystem` to the local templates. In `prod` Go builds, `go generate
./cli` generates `cli/static/generated_templates.gogen.go` that statically
provides the template files.

When built with `-tags prod`, the executable will be built with the staticlly
generated file instead of the local files.

# Validation
The binaries were compiled locally with `bin/docker-build`. The binaries were
then tested with `bin/test-run (pwd)/target/cli/darwin/linkerd`. All tests
passed.

No change was required to successfully run `bin/go-run cli install`. No change
was required to run `bin/linkerd install`.

Fixes #2153

Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
2019-02-04 18:09:47 -08:00
Oliver Gould 44e31f0f67
Configure proxy keepalives via the environment (#2193)
In linkerd/linkerd2-proxy#186, the proxy supports configuration of TCP
keepalive values.

This change sets `LINKERD2_PROXY_INBOUND_ACCEPT_KEEPALIVE` and
`LINKERD2_PROXY_OUTBOUND_CONNECT_KEEPALIVE` to 10s when injecting the
proxy, so that remote connections are configured with a keepalive.

This configuration is NOT yet exposed through the CLI. This may be done
in a followup, if necessary.

Fixes #1949
2019-02-04 16:16:43 -08:00
Oliver Gould 4798ad3f44
Use the proper controller identity when configuring pods with TLS (#2196)
Since 37ae423, deployments have been prefixed with linkerd-; however
the inject logic was not changed to take this into consideration when
constructing the controller's identity.

This means that the proxy's client to the control plane has been unable to
establish TLS'd communcation to the proxy-api. Previously, the proxy would
silently fall back to plaintext, but in master this behavior recently changed to
be stricter, so this bug will prevent the proxy from connecting to proxy-api
in any way.
2019-02-04 14:59:03 -08:00
Ye Ben f2ba17d366 fix some typos (#2194)
Signed-off-by: yeya24 <ben.ye@daocloud.io>
2019-02-02 23:03:54 -08:00
Kevin Lingerfelt 4c019c27c1
Add pod spec annotation to disable injection in CLI and auto-injector (#2187)
* Add pod spec annotation to disable injection in CLI and auto-injector
* Remove support for linkerd.io/auto-inject label entirely
* Update based on review feedback
* Fix issue with finding the namespace of deployments applied to the default ns

Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2019-02-01 16:57:06 -08:00
Cody Vandermyn a283dabbed Added flags to allow further configuration of destination cni bin and cni conf directories; fixed up spacing in template. (#2181)
Signed-off-by: Cody Vandermyn <cody.vandermyn@nordstrom.com>
2019-01-31 13:52:39 -08:00
Alex Leong 3bd4231cec
Add support for timeouts in service profiles (#2149)
Fixes #2042 

Adds a new field to service profile routes called `timeout`.  Any requests to that route which take longer than the given timeout will be aborted and a 504 response will be returned instead.  If the timeout field is not specified, a default timeout of 10 seconds is used.

Signed-off-by: Alex Leong <alex@buoyant.io>
2019-01-30 16:48:55 -08:00
Andrew Seigner 5651e02496
Add hint URLs for all checks (#2159)
linkerd/website#105 introduced a FAQ page, providing resolutions for all
`linkerd check` failures.

Update each check to reference its corresponding section in the FAQ.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-01-30 16:26:57 -08:00
Alejandro Pedraza fe234cade1
Use `ca.NewCA()` for generating certs and keys for the proxy injector (#2163)
Use `ca.NewCA()` for generating certs and keys for the proxy injector

- Remove from CA controller everything that dealt with the
webhook/proxy-injector
- Remove no longer needed proxy-injector volumes for 'trust-anchors' and
'webhook-secrets'
- Remove from the proxy-injector the retrieval of the trust anchor and
secrets
- tls flag during install is no longer needed for auto-inject to work

Fixes #2095 and fixes #2166

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-01-30 16:04:33 -05:00
Cody Vandermyn 906c3cbfc5 WIP: CNI Plugin (#2071)
* Export RootOptions and BuildFirewallConfiguration so that the cni-plugin can use them.
* Created the cni-plugin based on istio-cni implementation
* Create skeleton files that need to be filled out.
* Create the install scripts and finish up plugin to write iptables
* Added in an integration test around the install_cni.sh and updated the script to handle the case where it isn't the only plugin. Removed the istio kubernetes.go file in favor of pkg/k8s; initial usage of this package; found and fixed the typo in the ClusterRole and ClusterRoleBinding; found the docker-build-cni-plugin script
* Corrected an incorrect name in the docker build file for cni-plugin
* Rename linkerd2-cni to linkerd-cni
* Fixup Dockerfile and clean up code a bit as well as logging statements.
* Update Gopkg.lock after master merge.
* Update test file to remove temporary tag.
* Fixed the command to run during the test while building up the docker run.
* Added attributions to applicable files; in the test file, use a different container for each test scenario and also print the docker logs to stdout when there is an error;
* Add the --no-init-container flag to install and inject. This flag will not output the initContainer and will add an annotation assuming that the cni will be used in this case.
* Update .travis.yml to build the cni-plugin docker image before running the tests.
* Workaround golint warnings.
* Create a new command to install the linkerd-cni plugin.
* Add the --no-init-container option to linkerd inject
* Use the setup ip tables annotation during the proxy auto inject webhook prevent/allow addition of an init container; move cni-plugin tests to the integration-test section of travis
* gate the cni-plugin tests with the -integration-tests flag; remove unnecessary deployment .yaml file.
* Incorporate PR Cleanup suggestions.
* Remove the SetupIPTablesLabel annotation and use config flags and the presence of the init container to determine whether the cni-plugin writes ip tables.
* Fix a logic bug in the cni-plugin code that prevented the iptables from being written; Address PR comments; make tests pass.
* Update go deps shas
* Changed the single file install-cni plugin filename to be .conf vs .conflist; Incorporated latest PR comments around spacing with the new renderer among others.
* Fix an issue with renaming .conf to .conflist when needed.
* Renamed some of the variables to try to make it more clear what is going on.
* Address final PR comments.
* Hide cni flags for the time being.

Signed-off-by: Cody Vandermyn <cody.vandermyn@nordstrom.com>
2019-01-30 11:51:34 -08:00
Thomas Rampelberg ea61630f9d
Update client-go to 1.13.1 (#2160)
* Update client-go to 1.13.1

Fixes #2145

* Update Dockerfile-bin with new tag

* Update all the dockerfile tags

* Clean gopkg and do not apply cluster defaults

* Update for klog

* Match existing behavior with klog

* Add klog to gopkg.lock

* Update go-deps shas

* Update klog comment

* Update comment to be a non-sentence
2019-01-28 17:42:14 -08:00
Alex Leong 872e1bb026
Add --proto flag to linkerd profile command to read protobuf files (#2128)
Fixes #1425 

Signed-off-by: Alex Leong <alex@buoyant.io>
2019-01-25 11:15:20 -08:00
Kevin Leimkuhler f3325e7d81
Refactor `install`'s `render` output to be helm compatible (#2098)
# Problem
In order to refactor `install` to allow for a more flexible configuration, we
should start with the format of the YAML that it renders. Using the Helm
YAML format will make it easier add flexible configuration options in the
future. Currently, the rendered template that `install` produces does not
follow this format.

# Solution
Use the internals that Helm itself uses to render an inject template that
follows the same formatting rules. Helm's `template` cmd provides a good
outline of what is needed to make Linkerd's `install` cmd work as if it was
a Chart.

# Validation
There are no new tests, but there may not be anything to test at this stage.
This is a WIP PR towards the ultimate goal of `install` allowing a more
flexible configuration.

However, `install` now uses all the Helm `template` internals and therefore
satisfies the needed properties for Helm Charts.

Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
2019-01-25 10:53:35 -08:00
Eliza Weisman 846975a190
Remove proxy bind timeout from CLIs (#2017)
This branch removes the `--proxy-bind-timeout` flag from the 
`linkerd inject` and `linkerd install` CLI commands, and the
`LINKERD2_PROXY_BIND_TIMEOUT` environment variable from their output.
This is in preparation for removing that timeout from the proxy (as
described in #2013). 

I thought it was prudent to remove this from the CLIs before removing it
from the proxy, so we can't create a situation where the CLIs produce
output that results in broken proxy containers.

Fixes #2013

Signed-off-by: Eliza Weisman <eliza@buoyant.io>
2019-01-24 15:34:09 -08:00
zak 8c413ca38b Wire up stats commands for daemonsets (#2006) (#2086)
DaemonSet stats are not currently shown in the cli stat command, web ui
or grafana dashboard. This commit adds daemonset support for stat.

Update stat command's help message to reference daemonsets.
Update the public-api to support stats for daemonsets.
Add tests for stat summary and api.

Add daemonset get/list/watch permissions to the linkerd-controller
cluster role that's created using the install command.
Update golden expectation test files for install command
yaml manifest output.

Update web UI with daemonsets
Update navigation, overview and pages to list daemonsets and the pods
associated to them.
Add daemonset paths to server, and ui apps.

Add grafana dashboard for daemonsets; a clone of the deployment
dashboard.

Update dependencies and dockerfile hashes

Add DaemonSet support to tap and top commands

Fixes of #2006

Signed-off-by: Zak Knill <zrjknill@gmail.com>
2019-01-24 14:34:13 -08:00
Andrew Seigner 2217a6cd2b
Add cli command to install linkerd service profiles (#2099)
This introduces a `linkerd install-sp` command to install service
profiles into the linkerd control plane. It installs one service profile
for each of controller-api, proxy-api, prometheus, and grafana.

Fixes #1901

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-01-24 11:34:19 -08:00
Andrew Seigner c9ac77cd7c
Introduce version consistency checks (#2130)
Version checks were not validating that the cli version matched the
control plane or data plane versions.

Add checks via the `linkerd check` command to validate the cli is
running the same version as the control and data plane.

Also add types around `channel-version` string parsing and matching. A
consequence being that during development `version.Version` changes from
`undefined` to `dev-undefined`.

Fixes #2076

Depends on linkerd/website#101

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-01-23 16:54:43 -08:00
Alena Varkockova 28f662c9c6 Introduce resource selector and deprecate namespace field for ListPods (#2025)
* Introduce resource selector and deprecate namespace field for ListPods
* Changes from code review
* Properly deprecate the field
* Do not check for nil
* Fix the mockProm usage
* Protoc changes revert
* Changed from code review

Signed-off-by: Alena Varkockova <varkockova.a@gmail.com>
2019-01-23 10:35:55 -08:00
Andrew Seigner 93322bed6d
Fix Unicode and color output for Windows (#2114)
The default font in Windows console did not support the Unicode
characters recently added to check and inject commands. Also the color
library the linkerd cli depends on was not being used in a
cross-platform way.

Replace the existing Unicode characters used in `check` and `inject`
with characters available in most fonts, including Windows console.
Similarly replace the spinner used in `check` with one that uses
characters available in most fonts.

Modify `check` and `inject` to use `color.Output` and `color.Error`,
which wrap `os.Stdout` and `os.Stderr`, and perform special
tranformations when on Windows.

Add a `--no-color` option to `linkerd logs`. While stern uses the same
color library that `check`/`inject` use, it is not yet using the
`color.Output` API for Windows support. That issue is tracked at:
https://github.com/wercker/stern/issues/69

Relates to https://github.com/linkerd/linkerd2/pull/2087

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-01-23 09:55:44 -08:00
Alejandro Pedraza eacc09b7ba
Follow up to #2054: move magic strings into constants (#2122)
Follow up to #2054: move magic strings into constants

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-01-22 14:53:24 -05:00
Alejandro Pedraza a5628780f7
Fix dupe word in example (#2121)
Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-01-22 13:43:31 -05:00
Kevin Lingerfelt 1989e28d8f
Add newline separator above uninject reports (#2104)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2019-01-17 11:12:54 -08:00
Alejandro Pedraza bd1955c317 When injecting, perform an uninject as a first step (#2089)
* When injecting, perform an uninject as a first step

Fixes #1970

The fixture `inject_emojivoto_already_injected.input.yml` is no longer
rejected, so I created the corresponding golden file.

Note that we'll still forbid injection over resources already injected
with third party meshes (Istio, Contour), so now we have
`HasExisting3rdPartySidecars()` to detect that.

* Generalize HasExistingSidecars() to cater for both the auto-injector and
* Convert `linkerd uninject` result format to the one used in `linkerd inject`.
* More updates to the uninject reports. Revert changes to the HasExistingSidecars func.

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-01-17 10:15:23 -08:00
Dennis Adjei-Baah f9cd9366d9
Surface logs from control plane pods (#2037)
When debugging control plane issues or issues pertaining to a linkerd proxy, it can be cumbersome to get logs from affected containers quickly. 

This PR adds a new `logs` command to the Linkerd CLI to surface log lines from any container within linkerd's control plane. This feature relies heavily on [stern](https://github.com/wercker/stern), which already provides this behavior. This PR integrates this package into the Linkerd CLI to allow users to quickly retrieve logs whenever they run into issues when using Linkerd. 

Signed-off-by: Dennis Adjei-Baah <dennis@buoyant.io>
2019-01-16 19:24:42 -08:00
Alex Leong a562f8b9fd
Improve routes command to list all routes (#2066)
Fixes #1875 

This change improves the `linkerd routes` command in a number of important ways:

* The restriction on the type of the `--to` argument is lifted and any resource type can now be used.  Try `--to ns/books`, `--to po/webapp-ABCDEF`, `--to au/linkerd.io`, or even `--to svc`.
* All routes for the target will now be populated in the table, even if there are no Prometheus metrics for that route.
* [UNKNOWN] has been renamed to [DEFAULT]
* The `Service/Authority` column will now list `Service` in all cases except for when an authority target is explicitly requested.

```
$ linkerd routes deploy/traffic --to deploy/webapp
ROUTE                       SERVICE   SUCCESS      RPS   LATENCY_P50   LATENCY_P95   LATENCY_P99
GET /                        webapp   100.00%   0.5rps          50ms         180ms         196ms
GET /authors/{id}            webapp   100.00%   0.5rps         100ms         900ms         980ms
GET /books/{id}              webapp   100.00%   0.9rps          38ms          93ms          99ms
POST /authors                webapp   100.00%   0.5rps          35ms          48ms          50ms
POST /authors/{id}/delete    webapp   100.00%   0.5rps          83ms         180ms         196ms
POST /authors/{id}/edit      webapp     0.00%   0.0rps           0ms           0ms           0ms
POST /books                  webapp    45.16%   2.1rps          75ms         425ms         485ms
POST /books/{id}/delete      webapp   100.00%   0.5rps          30ms          90ms          98ms
POST /books/{id}/edit        webapp    56.00%   0.8rps          92ms         875ms         975ms
[DEFAULT]                    webapp     0.00%   0.0rps           0ms           0ms           0ms
```

This is all made possible by a shift in the way we handle the destination resource.  When we get a request with a `ToResource`, we use the k8s API to find all Services which include at least one pod belonging to that resource.  We then fetch all service profiles for those services and display the routes from those serivce profiles.  

This shift in thinking also precipitates a change in the TopRoutes API where we no longer need special cases for `ToAll` (which can be specified by `--to au`) or `ToAuthority` (which can be specified by `--to au/<authority>`) and instead can use a `ToResource` to handle all cases.

Signed-off-by: Alex Leong <alex@buoyant.io>
2019-01-16 17:15:35 -08:00
Andrew Seigner 92f2cd9b63
Update check and inject output (#2087)
The outputs of the `check` and `inject` commands did not vary much
between successful and failed executions, and were a bit verbose and
challenging to parse.

Reorganize output of `check` and `inject` commands, to provide more
output when errors occur, and less output when successful.

Specific changes:

`linkerd check`
- visually group checks by category
- introduce `hintURL`'s, to provide doc links when checks fail
- add spinners when retrying, remove additional retry lines
- colored unicode characters to indicate success/warning/failure

`linkerd inject`
- modify default output to mirror `kubectl apply`
- only output non-successful inject reports
- support `--verbose` flag to output all inject reports

Fixes #1471, #1653, #1656, #1739

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-01-16 15:14:14 -08:00
Alex Leong 771542dde2
Add support for retries (#2038) 2019-01-16 14:13:48 -08:00
Alex Leong 3398e9391a
Fix panic when swagger has no paths (#2092)
Fixes #2059

Fix a panic in linkerd profile --open-api when the swagger spec has no Paths field.

Signed-off-by: Alex Leong <alex@buoyant.io>
2019-01-16 13:39:51 -08:00
Andrew Seigner dacd8819ff
Group checkers by category (#2083)
The linkerd check command organized the various checks via loosely
coupled category IDs, category names, and checkers themselves, all with
ordering defined by consumers of this code.

This change removes category IDs in favor of category names, groups all
checkers by category, and enforces ordering at the HealthChecker
level.

Part of #1471, depends on #2078.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-01-14 18:01:32 -08:00
Andrew Seigner 04373414ef
Modify all health checks to be specified via enums (#2078)
The set of health checks to be executed were dependent on a combination
of check enums and boolean options.

This change modifies the health checks to be governed strictly by a set
of enums.

Next steps:
- tightly couple category IDs to names
- tightly couple checks to their parent categories
- programmatic control over check ordering

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-01-14 17:16:15 -08:00
Kevin Lingerfelt 6a3f4a1a99
Leave dashboard command running when open URL fails (#2081)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2019-01-14 16:46:11 -08:00
Alejandro Pedraza d358605269
Added `linkerd uninject` command (#2054)
Add `linkerd uninject` command

uninject.go iterates through the resources annotations, labels,
initContainers and Containers, removing what we know was injected by
linkerd.

The biggest part of this commit is the refactoring of inject.go, to make
it more generic and reusable by uninject.

The idea is that in a following PR this functionality will get reused by
`linkerd inject` to uninject as as preliminary step to injection, as a
solution to #1970.

This was tested successfully on emojivoto with:

```
1) inject:
kubectl get -n emojivoto deployment -o yaml | bin/linkerd inject - |
kubectl apply -f -
2) uninject:
kubectl get -n emojivoto deployment -o yaml | bin/linkerd uninject - |
kubectl apply -f -
```

Also created unit tests for uninject.go. The fixture files from the inject
tests could be reused. But as now the input files act as outputs, they
represent existing resources and required these changes (that didn't
affect inject):
  - Rearranged fields in alphabetical order.
  - Added fields that are only relevant for existing resources (e.g.
    creationTimestamp and status.replicas in StatefulSets)

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-01-14 17:39:31 -05:00
Kevin Lingerfelt ed3fbd75f3
Setup port-forwarding for linkerd dashboard command (#2052)
* Setup port-forwarding for linkerd dashboard command
* Output port-forward logs when --verbose flag is set

Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2019-01-10 10:16:08 -08:00
Alena Varkockova 172398292d Add validation to CRD for service profiles (#2024)
* Add validation to CRD for service profiles

Signed-off-by: Alena Varkockova <varkockova.a@gmail.com>

* Use properties instead of oneof

Signed-off-by: Alena Varkockova <varkockova.a@gmail.com>
2019-01-10 07:04:40 -08:00
Alejandro Pedraza 281ba37e6d
More granular control on checks made by CLI commands (#2033)
Have the CLI commands `get`, `routes`, `stat`, `tap`and `top` perform a more limited set of checks 

Fixes #1854
2019-01-10 09:13:44 -05:00
Kevin Lingerfelt a27bb2e0ce
Proxy grafana requests through web service (#2039)
* Proxy grafana requests through web service
* Fix -grafana-addr default, clarify -api-addr flag
* Fix version check in grafana dashboards
* Fix comment typo

Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2019-01-04 16:07:57 -08:00
Alejandro Pedraza 8639a8753f
Fix reporting of injected resources (#2031)
* Fix reporting of injected resources

When reporting resources provided as a list, it was picking just one
item from the list and ignoring the rest.
Also improved test for injecting list of resources.

Fixes #1676

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-01-03 15:14:11 -05:00
Andrew Seigner a91c77d0bf
Followups from lint/comment changes (#2032)
This is a followup branch from #2023:
- delete `proxy/client.go`, move code to `destination-client`
- move `RenderTapEvent` and stat functions from `util` to `cmd`

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-01-02 15:28:09 -08:00
Alena Varkockova ef02cd6828 Disallow both pre and proxy flags for check (#2026)
Signed-off-by: Alena Varkockova <varkockova.a@gmail.com>
2019-01-02 13:36:04 -08:00
Kevin Lingerfelt f1b0983f72
Add go linting to CI config (#2018)
* Add go linting to CI config
* Fix lint warnings
* Add note about bin/lint script in TEST.md

Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-12-20 15:33:09 -08:00
Thomas Rampelberg 612ebe2b81
Add rules_file loading into prometheus (#1966) 2018-12-20 11:47:13 -08:00
Radu M 07cbfe2725 Fix most golint issues that are not comment related (#1982)
Signed-off-by: Radu Matei <radu@radu-matei.com>
2018-12-20 10:37:47 -08:00
Alex Leong 0a87ebcd5d
Add --routes flag to linkerd top (#1952)
Fixes #1910 

Add a `--routes` flag to the `top` command which show requests by route instead of by path.  

This also includes an overhaul of the live table rendering code to be more flexible and better accommodate the various combinations of columns that may be visible or hidden.

Co-authored-by: Andrew Seigner <siggy@buoyant.io>
Signed-off-by: Alex Leong <alex@buoyant.io>
2018-12-18 11:25:34 -08:00
Alex Leong cb3fa1245b
Remove TLS column from routes command output (#1956)
Signed-off-by: Alex Leong <alex@buoyant.io>
2018-12-14 21:52:49 -08:00
Kevin Lingerfelt b16f4da1be
Re-add RBAC permission to access installed namespace in single ns mode (#1987)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-12-13 15:32:53 -08:00
Kevin Lingerfelt 0866bb2a41
Remove runAsGroup field from security context settings (#1986)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-12-13 15:12:13 -08:00
Kevin Lingerfelt 86e95b7ad3
Disable serivce profiles in single-namespace mode (#1980)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-12-13 14:37:18 -08:00
Cody Vandermyn d847f66ec5 Create new service accounts for linkerd-web and linkerd-grafana. Chan… (#1978)
* Create new service accounts for linkerd-web and linkerd-grafana. Change 'serviceAccount:' to 'serviceAccountName:'
* Use dynamic namespace name

Signed-off-by: Cody Vandermyn <cody.vandermyn@nordstrom.com>
2018-12-12 18:10:50 -08:00
Cody Vandermyn aa5e5f42eb Use an emptyDir for Prometheus and Grafana (#1971)
* Allow input of a volume name for prometheus and grafana
* Make Prometheus and Grafana volume names 'data' by default and disallow user editing via cli flags
* Remove volume name from options

Signed-off-by: Cody Vandermyn <cody.vandermyn@nordstrom.com>
2018-12-12 15:54:03 -08:00
Kevin Lingerfelt fd44896644
Remove namespace definition from --single-namespace installs (#1974)
* Remove namespace definition from --single-namespace installs
* DRY up code in healthcheck.go

Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-12-12 14:53:02 -08:00