Commit Graph

203 Commits

Author SHA1 Message Date
Alejandro Pedraza 54f943a725
Uninject: remove labels at the top level (#2442)
Fixes #2377

In inject's ResourceConfig, renamed objMeta to podMeta since
it really points to the pod template metadata. And created a new field
workloadMeta that really points to the main workload (e.g. Deployment) metadata.

Refactored uninject to clean up the labels at both podMeta and
workloadMeta. Also it will remove all the labels and annotations that
start with "linkerd.io" except for the "linkerd.io/inject" annotation.

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-03-06 14:35:59 -05:00
Alejandro Pedraza f155fb9a8f
Have `NewFakeClientSets()` not swallow errors when parsing YAML (#2454)
This helps catching bad YAMLs in test resources

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-03-06 13:53:04 -05:00
Andrew Seigner d90fa16727
Introduce NET_ADMIN cli check (#2421)
The `linkerd-init` container requires the NET_ADMIN capability to modify
iptables. The `linkerd check` command was not verifying this.

Introduce a `has NET_ADMIN capability` check, which does the following:
1) Lists all available PodSecurityPolicies, if none found, returns
success
2) For each PodSecurityPolicy, validate one exists that:
    - the user has `use` access AND
    - provides `*` or `NET_ADMIN` capability

A couple limitations to this approach:
- It is testing whether the user running `linkerd check` has NET_ADMIN,
  but during installation time it will be the `linkerd-init` pod that
  requires NET_ADMIN.
- It assumes the presense of PodSecurityPolicies in the cluster means
  the PodSecurityPolicy admission controller is installed. If the
  admission controller is not installed, but PSPs exists that restrict
  NET_ADMIN, `linkerd check` will incorrectly report the user does not
  have that capability.

This PR also fixes the `can create CustomResourceDefinitions` check to
not specify a namespace when doing a `create` check, as CRDs are
cluster-wide.

Fixes #1732

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-03-05 17:19:11 -08:00
Alejandro Pedraza ddf2e729ac
Injection consolidation (#2334)
- Created the pkg/inject package to hold the new injection shared lib.
- Extracted from `/cli/cmd/inject.go` and `/cli/cmd/inject_util.go`
the core methods doing the workload parsing and injection, and moved them into
`/pkg/inject/inject.go`. The CLI files should now deal only with
strictly CLI concerns, and applying the json patch returned by the new
lib.
- Proceeded analogously with `/cli/cmd/uninject.go` and
`/pkg/inject/uninject.go`.
- The `InjectReport` struct and helping methods were moved into
`/pkg/inject/report.go`
- Refactored webhook to use the new injection lib
- Removed linkerd-proxy-injector-sidecar-config ConfigMap
- Added the ability to add pod labels and annotations without having to
specify the already existing ones

Fixes #1748, #2289

Signed-off-by: Alejandro Pedraza <alejandro.pedraza@gmail.com>
2019-03-05 08:38:56 -05:00
Tarun Pothulapati 2184928813 Wire up stats for Jobs (#2416)
Support for Jobs in stat/tap/top cli commands

Part of #2007

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2019-03-01 17:16:54 -08:00
Yan 4cd1f99e89 Check kubectl version as part of checks (#2358)
Fixes #2354

Signed-off-by: Yan Babitski <yan.babitski@gmail.com>
2019-03-01 10:03:59 -08:00
Oliver Gould ab90263461
destination: Only return TLS identities when appropriate (#2371)
As described in #2217, the controller returns TLS identities for results even
when the destination pod may not be able to participate in identity
requester: specifically, the other pod may not have the same controller
namespace or it may not be injected with identity.

This change introduces a new annotation, linkerd.io/identity-mode that is set
when injecting pods (via both CLI and webhook). This annotation is always
added.

The destination service now only returns TLS identities when this annotation
is set to optional on a pod and the destination pod uses the same controller.
These semantics are expected to change before the 2.3 release.

Fixes #2217
2019-02-27 12:18:39 -08:00
Andrew Seigner 9f748d2d2e
lint: Enable unparam (#2369)
unparam reports unused function parameters:
https://github.com/mvdan/unparam

Part of #217

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-27 10:34:02 -08:00
Oliver Gould d03014aed6
Lessen klog for security (#2386)
We currently set klog to maximum verbosity when debug logging is
enabled. This causes control plane components, however, to log their
serviceaccount tokens, leaking secret information into logs.

By setting the klog level to 6, we avoid this logging.

Fixes #2383
2019-02-26 16:00:26 -08:00
Andrew Seigner ec5a0ca8d9
Authorization-aware control-plane components (#2349)
The control-plane components relied on a `--single-namespace` param,
passed from `linkerd install` into each individual component, to
determine which namespaces they were authorized to access, and whether
to support ServiceProfiles. This command-line flag was redundant given
the authorization rules encoded in the parent `linkerd install` output,
via [Cluster]Role[Binding]s.

Modify the control-plane components to query Kubernetes at startup to
determine which namespaces they are authorized to access, and whether
ServiceProfile support is available. This allows removal of the
`--single-namespace` flag on the components.

Also update `bin/test-cleanup` to cleanup the ServiceProfile CRD.

TODO:
- Remove `--single-namespace` flag on `linkerd install`, part of #2164

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-26 11:54:52 -08:00
Andrew Seigner ad0d0b72a0
lint: Enable unconvert (#2368)
unconvert removes unnecessary type conversions:
https://github.com/mdempsky/unconvert

Part of #217

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-25 16:07:42 -08:00
Andrew Seigner 25e462352d
lint: Enable goimports (#2366)
goimports checks import lines, adding missing ones and removing
unreferenced ones:
https://godoc.org/golang.org/x/tools/cmd/goimports

It also requires named imports for packages whose
import paths don't match their package names:
- https://github.com/golang/go/issues/28428
- https://go-review.googlesource.com/c/tools/+/145699/

Also standardized named imports of common Kubernetes packaages.

Part of #217

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-25 15:51:10 -08:00
Oliver Gould f7435800da
lint: Enable scopelint (#2364)
[scopelint][scopelint] detects a nasty reference-scoping issue in loops.

[scopelint]: https://github.com/kyoh86/scopelint
2019-02-24 08:59:51 -08:00
Andrew Seigner 7fa7e962cb
Fix hint URLs not display for RPC checks (#2361)
Hint URLs should display for all failed checks in `linkerd check`, but
were not displaying for RPC checks.

Fix `runCheckRPC` to pass along the hintAnchor to the check result.

Also rename the second `can query the control plane API` to
`control plane self-check`, as there were two checks with that name.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-23 13:35:32 -08:00
Andrew Seigner cc3ff70f29
Enable `unused` linter (#2357)
`unused` checks Go code for unused constants, variables, functions, and
types.

Part of #217

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-23 11:05:39 -08:00
Andrew Seigner 4b6f6aeedd
Enable gosimple linter, fix issues (#2356)
gosimple is a Go linter that specializes in simplifying code

Also fix one spelling error in `cred_test.go`

Part of #217

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-22 17:19:07 -08:00
Oliver Gould 2640943c67
pkg/tls: Make it possible to load a CA from disk (#2335)
In preparation for creating an Identity service that can chain off of an
existing CA, it's necessary to both (1) be able to create an
intermediate CA that can be used by the identity service and (2) be able
to load a CA from existing key material.

This changes the public API of the `tls` package to deal in actual key
types (rather than opaque blobs) and provides a set of helpers that can
be used to convert these credentials between common formats.
2019-02-22 15:13:50 -08:00
Kevin Lingerfelt b1a6b6ca66
Fix handling of kubeconfig server urls that include paths (#2305)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2019-02-18 17:57:27 -08:00
Andrew Seigner 1df1683b6a
Instrument k8s clients (#2243)
The control-plane's clients, specifically the Kubernetes clients, did
not provide telemetry information.

Introduce a `prometheus.ClientWithTelemetry` wrapper to instrument
arbitrary clients. Apply this wrapper to Kubernetes clients.

Fixes #2183

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-18 09:10:02 -08:00
Kevin Leimkuhler b2bbeb05ef
Issue 2276: Do not log error when timeout is blank (#2279)
# Problem

When a route does not specify a timeout, the proxy-api defaults to the default
timeout and logs an error:

```
time="2019-02-13T16:29:12Z" level=error msg="failed to parse duration for route POST /io.linkerd.proxy.destination.Destination/GetProfile: time: invalid duration"
```

# Solution

We now check if a route timeout is blank. If it is not set, it is set to
`DefaultRouteTimeout`. If it is set, we try to parse it into a `Duration`.

A request was made to improve logging to include the service profile and
namespace as well.

# Validation

With valid service profiles installed, edit the `.yaml` to include an invalid
`timeout`:

```
...
name: GET /
timeout: foo
```

We should now see the following errors:

```
proxy-api time="2019-02-13T22:27:32Z" level=error msg="failed to parse duration for route 'GET /' in service profile 'webapp.default.svc.cluster.local' in namespace 'default': time: invalid duration foo"
```

This error does not show up when `timeout` is blank.

Fixes #2276

Signed-off-by: Kevin Leimkuhler <kevinl@buoyant.io>
2019-02-14 17:09:02 -08:00
Andrew Seigner 2305974202
Introduce golangci-lint tooling, fixes (#2239)
`golangci-lint` performs numerous checks on Go code, including golint,
ineffassign, govet, and gofmt.

This change modifies `bin/lint` to use `golangci-lint`, and replaces
usage of golint and govet.

Also perform a one-time gofmt cleanup:
- `gofmt -s -w controller/`
- `gofmt -s -w pkg/`

Part of #217

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-13 11:16:28 -08:00
Andrew Seigner 1a60237a94
Update check command hint URLs to new alias (#2245)
The existing hint URLs printing by `linkerd check` pointed to locations
that would change if the linkerd.io website was reorganized.

linkerd/website#148 introduces an alias for hint URLs at
https://linkerd.io/checks/. This is the corresponding change to update
`linkerd check` output.

Depends on linkerd/website#148, relates to linkerd/website#146.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-11 11:00:16 -08:00
Alex Leong 3f333c2860
Validate service profiles in all namespaces (#2237)
Fixes #2220 

The service profile validation which is part of `linkerd check` only validates service profiles in the Linkerd namespace.  Due to a recent change, service profiles now can exist in any namespace.

Update the logic so that service profiles in all namespaces are validated.  
Additionally:
* Relax validation of service profile names to support external names

Signed-off-by: Alex Leong <alex@buoyant.io>
2019-02-11 09:52:47 -08:00
Ivan Sim f6e75ec83a
Add statefulsets to the dashboard and CLI (#2234)
Fixes #1983

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-02-08 15:37:44 -08:00
Alex Leong 5b054785e5
Read service profiles from client or server namespace instead of control namespace (#2200)
Fixes #2077 

When looking up service profiles, Linkerd always looks for the service profile objects in the Linkerd control namespace.  This is limiting because service owners who wish to create service profiles may not have write access to the Linkerd control namespace.

Instead, we have the control plane look for the service profile in both the client namespace (as read from the proxy's `proxy_id` field from the GetProfiles request and from the service's namespace.  If a service profile exists in both namespaces, the client namespace takes priority.  In this way, clients may override the behavior dictated by the service.

Signed-off-by: Alex Leong <alex@buoyant.io>
2019-02-07 14:51:43 -08:00
Andrew Seigner 907f01fba6
Improve ServiceProfile validation in linkerd check (#2218)
The `linkerd check` command was doing limited validation on
ServiceProfiles.

Make ServiceProfile validation more complete, specifically validate:
- types of all fields
- presence of required fields
- presence of unknown fields
- recursive fields

Also move all validation code into a new `Validate` function in the
profiles package.

Validation of field types and required fields is handled via
`yaml.UnmarshalStrict` in the `Validate` function. This motivated
migrating from github.com/ghodss/yaml to a fork, sigs.k8s.io/yaml.

Fixes #2190
2019-02-07 14:35:47 -08:00
Andrew Seigner 72812baf99
Introduce Discovery API and endpoints command (#2195)
The Proxy API service lacked introspection of its internal state.

Introduce a new gRPC Discovery API, implemented by two servers:
1) Proxy API Server: returns a snapshot of discovery state
2) Public API Server: pass-through to the Proxy API Server

Also wire up a new `linkerd endpoints` command.

Fixes #2165

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-07 14:02:21 -08:00
Dennis Adjei-Baah e98ba06e5f
Modify klog to be verbose when controller log-level is set to debug (#2221)
The controller logs innocuous messages when control plane proxies aren't ready to route requests during startup from each control plane component. i.e. tap, public-api and proxy-api. Setting the log level in the control plane to `INFO` would not hide these log messages and would still show up on control plane startup.

This PR modifies `klogs` initial flag set to route innocuous logs to `/dev/null` if the controller log level is set to INFO. If set to debug, we output all loglines to stderr.

Fixes #2171 #2168
Signed-off-by: Dennis Adjei-Baah <dennis@buoyant.io>
2019-02-07 13:48:42 -08:00
Risha Mars e531655d26
Add a --tap flag to the linkerd profile command (#2139)
Adds the ability to generate a service profile by running a tap for a configurable 
amount of time, and using the route results from the routes seen during the tap.

e.g. `linkerd profile web --tap deploy/web -n emojivoto --tap-duration 2s`
2019-02-06 12:43:16 -08:00
Alejandro Pedraza 50fbcc60b5
Add support for `basePath` in swagger 2.0 files (#2211)
Fixes #2175

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-02-06 13:44:35 -05:00
Alejandro Pedraza 0b42a02bb9
Build request urls in the most generic possible way (#2206)
Build request urls in the most generic possible way

Fixes #2132

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-02-06 13:22:17 -05:00
Alejandro Pedraza 2a7654ce78
Consolidate timeouts for `linkerd check` (#2191)
Consolidate timeouts for `linkerd check`

- Moved the creation of contexts from inside the methods targeted by the
checks into a single place in the runCheck() and runCheckRPC() methods
where the context is built using a hard-coded timeout of 30 seconds.
- k8s' client-go doesn't allow passing along contexts, but it let's us
setting the Timeout manually.
- Reworded the description for the --wait option.

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-02-05 11:38:30 -05:00
Oliver Gould 4798ad3f44
Use the proper controller identity when configuring pods with TLS (#2196)
Since 37ae423, deployments have been prefixed with linkerd-; however
the inject logic was not changed to take this into consideration when
constructing the controller's identity.

This means that the proxy's client to the control plane has been unable to
establish TLS'd communcation to the proxy-api. Previously, the proxy would
silently fall back to plaintext, but in master this behavior recently changed to
be stricter, so this bug will prevent the proxy from connecting to proxy-api
in any way.
2019-02-04 14:59:03 -08:00
Andrew Seigner 3a139d0202
Fix spelling on linkerd check link (#2197)
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-04 14:00:11 -08:00
Kevin Lingerfelt 4c019c27c1
Add pod spec annotation to disable injection in CLI and auto-injector (#2187)
* Add pod spec annotation to disable injection in CLI and auto-injector
* Remove support for linkerd.io/auto-inject label entirely
* Update based on review feedback
* Fix issue with finding the namespace of deployments applied to the default ns

Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2019-02-01 16:57:06 -08:00
Alex Leong 3bd4231cec
Add support for timeouts in service profiles (#2149)
Fixes #2042 

Adds a new field to service profile routes called `timeout`.  Any requests to that route which take longer than the given timeout will be aborted and a 504 response will be returned instead.  If the timeout field is not specified, a default timeout of 10 seconds is used.

Signed-off-by: Alex Leong <alex@buoyant.io>
2019-01-30 16:48:55 -08:00
Andrew Seigner 5651e02496
Add hint URLs for all checks (#2159)
linkerd/website#105 introduced a FAQ page, providing resolutions for all
`linkerd check` failures.

Update each check to reference its corresponding section in the FAQ.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-01-30 16:26:57 -08:00
Alejandro Pedraza fe234cade1
Use `ca.NewCA()` for generating certs and keys for the proxy injector (#2163)
Use `ca.NewCA()` for generating certs and keys for the proxy injector

- Remove from CA controller everything that dealt with the
webhook/proxy-injector
- Remove no longer needed proxy-injector volumes for 'trust-anchors' and
'webhook-secrets'
- Remove from the proxy-injector the retrieval of the trust anchor and
secrets
- tls flag during install is no longer needed for auto-inject to work

Fixes #2095 and fixes #2166

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-01-30 16:04:33 -05:00
Thomas Rampelberg ea61630f9d
Update client-go to 1.13.1 (#2160)
* Update client-go to 1.13.1

Fixes #2145

* Update Dockerfile-bin with new tag

* Update all the dockerfile tags

* Clean gopkg and do not apply cluster defaults

* Update for klog

* Match existing behavior with klog

* Add klog to gopkg.lock

* Update go-deps shas

* Update klog comment

* Update comment to be a non-sentence
2019-01-28 17:42:14 -08:00
Alex Leong 872e1bb026
Add --proto flag to linkerd profile command to read protobuf files (#2128)
Fixes #1425 

Signed-off-by: Alex Leong <alex@buoyant.io>
2019-01-25 11:15:20 -08:00
Alex Leong d542571b65
GetProfiles should always respond with a (possibly empty) profile immediately (#2146)
When `GetProfiles` is called for a destination that does not have a service profile, the proxy-api service does not return any messages until a service profile is created for that service.  This can be interpreted as hanging, and can make it difficult to calculate response latency metrics.

Change the behavior of the API to always return a service profile message immediately.  If the service does not have a service profile, the default service profile is returned.

Signed-off-by: Alex Leong <alex@buoyant.io>
2019-01-24 15:22:14 -08:00
zak 8c413ca38b Wire up stats commands for daemonsets (#2006) (#2086)
DaemonSet stats are not currently shown in the cli stat command, web ui
or grafana dashboard. This commit adds daemonset support for stat.

Update stat command's help message to reference daemonsets.
Update the public-api to support stats for daemonsets.
Add tests for stat summary and api.

Add daemonset get/list/watch permissions to the linkerd-controller
cluster role that's created using the install command.
Update golden expectation test files for install command
yaml manifest output.

Update web UI with daemonsets
Update navigation, overview and pages to list daemonsets and the pods
associated to them.
Add daemonset paths to server, and ui apps.

Add grafana dashboard for daemonsets; a clone of the deployment
dashboard.

Update dependencies and dockerfile hashes

Add DaemonSet support to tap and top commands

Fixes of #2006

Signed-off-by: Zak Knill <zrjknill@gmail.com>
2019-01-24 14:34:13 -08:00
Dennis Adjei-Baah 20efe14fef
Fix issue where linkerd check would panic with replicationcontroller pod name in control plane (#2140)
When running Linkerd check with a control plane namespace that may contain an additional pod with a replication controller ID for pod names instead of a replicaSet ID, the check command panics because of an "index out of bounds" error.

This PR adds a check to make sure that, when parsing pod names during the `checkControllerRunning` healthcheck, we only check for linkerd control plane pods and that the pod name results in four or more substrings when split on '-'. This prevents the check from panicking when encountering a replication controller ID pod name.

Fixes #2084 

Signed-off-by: Dennis Adjei-Baah <dennis@buoyant.io>
2019-01-24 10:26:11 -08:00
Risha Mars 950c952d14
Add a less scary message to the user when retries are still in progress (#2141)
When a failing check is being retried, we show the current err to the user. This
can sometimes be unnecessarily alarming, as in the case of the control plane
starting up.

If a failing check is in the process of being retried, wait to show the final
error message until the retries have completed.
2019-01-24 10:24:59 -08:00
Risha Mars e7556d7edc
Check RoleBindings for specified single namespace only (#2142)
Previously, we were doing the creation checks for both Roles/RoleBindings and
ClusterRoles/ClusterRoleBindings for all namespaces, but in --single-namespace
mode, we only need to check that these can be created in the control plane
namespace.
2019-01-23 18:04:15 -08:00
Andrew Seigner c9ac77cd7c
Introduce version consistency checks (#2130)
Version checks were not validating that the cli version matched the
control plane or data plane versions.

Add checks via the `linkerd check` command to validate the cli is
running the same version as the control and data plane.

Also add types around `channel-version` string parsing and matching. A
consequence being that during development `version.Version` changes from
`undefined` to `dev-undefined`.

Fixes #2076

Depends on linkerd/website#101

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-01-23 16:54:43 -08:00
Alena Varkockova 28f662c9c6 Introduce resource selector and deprecate namespace field for ListPods (#2025)
* Introduce resource selector and deprecate namespace field for ListPods
* Changes from code review
* Properly deprecate the field
* Do not check for nil
* Fix the mockProm usage
* Protoc changes revert
* Changed from code review

Signed-off-by: Alena Varkockova <varkockova.a@gmail.com>
2019-01-23 10:35:55 -08:00
Alejandro Pedraza eacc09b7ba
Follow up to #2054: move magic strings into constants (#2122)
Follow up to #2054: move magic strings into constants

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-01-22 14:53:24 -05:00
Alejandro Pedraza bd1955c317 When injecting, perform an uninject as a first step (#2089)
* When injecting, perform an uninject as a first step

Fixes #1970

The fixture `inject_emojivoto_already_injected.input.yml` is no longer
rejected, so I created the corresponding golden file.

Note that we'll still forbid injection over resources already injected
with third party meshes (Istio, Contour), so now we have
`HasExisting3rdPartySidecars()` to detect that.

* Generalize HasExistingSidecars() to cater for both the auto-injector and
* Convert `linkerd uninject` result format to the one used in `linkerd inject`.
* More updates to the uninject reports. Revert changes to the HasExistingSidecars func.

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-01-17 10:15:23 -08:00
Andrew Seigner 92f2cd9b63
Update check and inject output (#2087)
The outputs of the `check` and `inject` commands did not vary much
between successful and failed executions, and were a bit verbose and
challenging to parse.

Reorganize output of `check` and `inject` commands, to provide more
output when errors occur, and less output when successful.

Specific changes:

`linkerd check`
- visually group checks by category
- introduce `hintURL`'s, to provide doc links when checks fail
- add spinners when retrying, remove additional retry lines
- colored unicode characters to indicate success/warning/failure

`linkerd inject`
- modify default output to mirror `kubectl apply`
- only output non-successful inject reports
- support `--verbose` flag to output all inject reports

Fixes #1471, #1653, #1656, #1739

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-01-16 15:14:14 -08:00
Alex Leong 771542dde2
Add support for retries (#2038) 2019-01-16 14:13:48 -08:00
Andrew Seigner dacd8819ff
Group checkers by category (#2083)
The linkerd check command organized the various checks via loosely
coupled category IDs, category names, and checkers themselves, all with
ordering defined by consumers of this code.

This change removes category IDs in favor of category names, groups all
checkers by category, and enforces ordering at the HealthChecker
level.

Part of #1471, depends on #2078.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-01-14 18:01:32 -08:00
Andrew Seigner 04373414ef
Modify all health checks to be specified via enums (#2078)
The set of health checks to be executed were dependent on a combination
of check enums and boolean options.

This change modifies the health checks to be governed strictly by a set
of enums.

Next steps:
- tightly couple category IDs to names
- tightly couple checks to their parent categories
- programmatic control over check ordering

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-01-14 17:16:15 -08:00
Alejandro Pedraza d358605269
Added `linkerd uninject` command (#2054)
Add `linkerd uninject` command

uninject.go iterates through the resources annotations, labels,
initContainers and Containers, removing what we know was injected by
linkerd.

The biggest part of this commit is the refactoring of inject.go, to make
it more generic and reusable by uninject.

The idea is that in a following PR this functionality will get reused by
`linkerd inject` to uninject as as preliminary step to injection, as a
solution to #1970.

This was tested successfully on emojivoto with:

```
1) inject:
kubectl get -n emojivoto deployment -o yaml | bin/linkerd inject - |
kubectl apply -f -
2) uninject:
kubectl get -n emojivoto deployment -o yaml | bin/linkerd uninject - |
kubectl apply -f -
```

Also created unit tests for uninject.go. The fixture files from the inject
tests could be reused. But as now the input files act as outputs, they
represent existing resources and required these changes (that didn't
affect inject):
  - Rearranged fields in alphabetical order.
  - Added fields that are only relevant for existing resources (e.g.
    creationTimestamp and status.replicas in StatefulSets)

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-01-14 17:39:31 -05:00
Kevin Lingerfelt ed3fbd75f3
Setup port-forwarding for linkerd dashboard command (#2052)
* Setup port-forwarding for linkerd dashboard command
* Output port-forward logs when --verbose flag is set

Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2019-01-10 10:16:08 -08:00
Alejandro Pedraza 281ba37e6d
More granular control on checks made by CLI commands (#2033)
Have the CLI commands `get`, `routes`, `stat`, `tap`and `top` perform a more limited set of checks 

Fixes #1854
2019-01-10 09:13:44 -05:00
Andrew Seigner 1c302182ef
Enable lint check for comments (#2023)
Commit 1: Enable lint check for comments

Part of #217. Follow up from #1982 and #2018.

A subsequent commit will fix the ci failure.

Commit 2: Address all comment-related linter errors.

This change addresses all comment-related linter errors by doing the
following:
- Add comments to exported symbols
- Make some exported symbols private
- Recommend via TODOs that some exported symbols should should move or
  be removed

This PR does not:
- Modify, move, or remove any code
- Modify existing comments

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-01-02 14:03:59 -08:00
Kevin Lingerfelt f1b0983f72
Add go linting to CI config (#2018)
* Add go linting to CI config
* Fix lint warnings
* Add note about bin/lint script in TEST.md

Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-12-20 15:33:09 -08:00
Radu M 07cbfe2725 Fix most golint issues that are not comment related (#1982)
Signed-off-by: Radu Matei <radu@radu-matei.com>
2018-12-20 10:37:47 -08:00
Kevin Lingerfelt 86e95b7ad3
Disable serivce profiles in single-namespace mode (#1980)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-12-13 14:37:18 -08:00
Kevin Lingerfelt fd44896644
Remove namespace definition from --single-namespace installs (#1974)
* Remove namespace definition from --single-namespace installs
* DRY up code in healthcheck.go

Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-12-12 14:53:02 -08:00
Kevin Lingerfelt 0f8bcc9159
Controller: wait for caches to sync before opening listeners (#1958)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-12-07 11:15:45 -08:00
Alex Leong 04ed200e36
Rename path_regex to pathRegex (#1951)
Rename snake case fields to camel case in service profile spec.  This improves the way they are rendered when the `kubectl describe` command is used.

Signed-off-by: Alex Leong <alex@buoyant.io>
2018-12-06 11:51:33 -08:00
Alex Leong 4f3e55e937
Rename path to path_regex in ServiceProfile CRD (#1923)
We rename path to path_regex in the ServiceProfile CRD to make it clear that this field accepts a regular expression. We also take this opportunity to remove unnecessary line anchors from regular expressions now that these anchors are added in the proxy.

Signed-off-by: Alex Leong <alex@buoyant.io>
2018-12-05 10:42:47 -08:00
Kevin Lingerfelt 37ae423bb3
Add linkerd- prefix to all objects in linkerd install (#1920)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-12-04 15:41:47 -08:00
Risha Mars e8a39cd17e
Add ability to download a service profile template from the web UI (#1893)
Adds an endpoint, at /profiles/new that allows you to input a service name and
namespace, and download a service profile yaml template. 

This will enable future work, where we can add more of the yaml customization via 
a form in the dashboard, and use that data to help the user configure routes.
2018-12-03 16:48:43 -08:00
Andrew Seigner 37a5455445
Add filtering by job in stat, tap, top; fix panic (#1904)
Filtering by Kubernetes job was not supported. Also filtering by any unknown
type caused a panic.

Add filtering support by Kubernetes job, with special case mapping `job` to
`k8s_job`, to not conflict with Prometheus' job label.

Fix panic when unknown type specified as a `--from` or `--to` flag.

Fix `job` label from `linkerd-proxy` overwriting Prometheus `job` label at
collection time. This caused all metrics collected by proxy sidecars in
Kubernetes jobs to be collected into an incorrect Prometheus job, rather than
the expected `linkerd-proxy` Prometheus job.

Fix `unsupported resource type` tap error message incorrectly printing the
target resource rather than the destination.

Set `--controller-log-level debug` in `install_test.go` for easier debugging.

Expose `slow-cooker`'s metrics via a k8s service in the tap integration test, to
validate proxy requests with a job as destination.

Fixes #1872
Part of #627

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2018-12-03 15:34:49 -08:00
Andrew Seigner d121071f87
Adjust proxy, Prometheus, and Grafana probes (#1899)
* Adjust proxy, Prometheus, and Grafana probes

High `readinessProbe.initialDelaySeconds` values delayed the controller's
readiness by up to 30s, preventing cli commands from succeeding shortly after
control plane deployment.

Decrease `readinessProbe.initialDelaySeconds` in the proxy, Prometheus, and
Grafana to the default 0s. Also change `linkerd check` controller pod ordering
to: controller, prometheus, web, grafana.

Detailed probe changes:
- proxy
  - decrease `readinessProbe.initialDelaySeconds` from 10s to 0s
- prometheus
  - decrease `readinessProbe.initialDelaySeconds` from 30s to 0s
  - decrease `readinessProbe.timeoutSeconds` from 30s to 1s
  - decrease `livenessProbe.timeoutSeconds` from 30s to 1s
- grafana
  - decrease `readinessProbe.initialDelaySeconds` from 30s to 0s
  - decrease `readinessProbe.timeoutSeconds` from 30s to 1s
  - decrease `readinessProbe.failureThreshold` from 10 to 3
  - increase `livenessProbe.initialDelaySeconds` from 0s to 30s

Fixes #1804

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2018-12-03 10:41:11 -08:00
Alejandro Pedraza bbcf5a8c9f Allow stat summary to query for multiple resources (#1841)
* Refactor util.BuildResource so it can deal with multiple resources

First step to address #1487: Allow stat summary to query for multiple
resources

Signed-off-by: Alejandro Pedraza <alejandro.pedraza@gmail.com>

* Update the stat cli help text to explain the new multi resource querying ability

Propsal for #1487: Allow stat summary to query for multiple resources

Signed-off-by: Alejandro Pedraza <alejandro.pedraza@gmail.com>

* Allow stat summary to query for multiple resources

Implement this ability by issuing parallel requests to requestStatsFromAPI()

Proposal for #1487

Signed-off-by: Alejandro Pedraza <alejandro.pedraza@gmail.com>

* Update tests as part of multi-resource support in `linkerd stat` (#1487)

- Refactor stat_test.go to reuse the same logic in multiple tests, and
add cases and files for json output.
- Add a couple of cases to api_utils_test.go to test multiple resources
validation.

Signed-off-by: Alejandro Pedraza <alejandro.pedraza@gmail.com>

* `linkerd stat` called with multiple resources should keep an ordering (#1487)

Add SortedRes holding the order of resources to be followed when
querying `linkerd stat` with multiple resources

Signed-off-by: Alejandro Pedraza <alejandro.pedraza@gmail.com>

* Extra validations for `linkerd stat` with multiple resources (#1487)

Signed-off-by: Alejandro Pedraza <alejandro.pedraza@gmail.com>

* `linkerd stat` resource grouping, ordering and name prefixing (#1487)

- Group together stats per resource type.
- When more than one resource, prepend name with type.
- Make sure tables always appear in the same order.

Signed-off-by: Alejandro Pedraza <alejandro.pedraza@gmail.com>

* Allow `linkerd stat` to be called with multiple resources

A few final refactorings as per code review.

Fixes #1487

Signed-off-by: Alejandro Pedraza <alejandro.pedraza@gmail.com>
2018-11-14 10:44:04 -08:00
Kevin Lingerfelt 4547ba7f0a
Make permission checks non-fatal, add check for CRDs (#1859)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-11-14 10:29:04 -08:00
Igor Zibarev 60bcdb15f9 controller: use GetConfig from pkg/k8s package (#1857)
This commit removes duplicate logic that loads Kubernetes config and
replaces it with GetConfig from pkg/k8s. This also allows to load
config from default sources like $KUBECONFIG instead of explicitly
passing -kubeconfig option to controller components.

Signed-off-by: Igor Zibarev <zibarev.i@gmail.com>
2018-11-13 14:41:31 -08:00
Alena Varkockova fda834cf64 Allow retrying control plane API check (#1858)
Signed-off-by: Alena Varkockova <varkockova.a@gmail.com>
2018-11-13 10:52:50 -08:00
Igor Zibarev 9b3f67e3dc Fix comment typos (#1855)
Signed-off-by: Igor Zibarev <zibarev.i@gmail.com>
2018-11-12 15:27:22 -08:00
Alena Varkockova 38dfc5308f Make version checks warning (#1844)
Signed-off-by: Alena Varkockova <varkockova.a@gmail.com>
2018-11-09 09:48:14 -08:00
Alex Leong 32d556e732
Improve ergonomics of service profile spec (#1828)
We make several changes to the service profile spec to make service profiles more ergonomic and to make them more consistent with the destination profile API.

* Allow multiple fields to be simultaneously set on a RequestMatch or ResponseMatch condition.  Doing so is equivalent to combining the fields with an "all" condition.
* Rename "responses" to "response_classes"
* Change "IsSuccess" to "is_failure"

Signed-off-by: Alex Leong <alex@buoyant.io>
2018-10-31 12:00:22 -07:00
Alex Leong 82ca821e62
Use fqdn for service profile name (#1808)
Service profiles must be named in the form `"<service>.<namespace>"`.  This is inconsistent with the fully normalized domain name that the proxy sends to the controller.  It also does not permit creating service profiles for non-Kubernetes services.

We switch to requiring that service profiles must be named with the FQDN of their service.  For Kubernetes services, this is `"<service>.<namespace>.svc.cluster.local"`.

This change alone is not sufficient for allowing service profile for non-Kubernetes services because the k8s resolver will ignore any DNS names which are not Kubernetes services.  Further refactoring of the resolver will be required to allow looking up non-Kubernetes service profiles in Kuberenetes.

Signed-off-by: Alex Leong <alex@buoyant.io>
2018-10-29 14:35:42 -07:00
Alex Leong 622185a4dd
Send metric labels in profile API (#1800)
* Send metric labels in profile API

Signed-off-by: Alex Leong <alex@buoyant.io>
2018-10-29 14:28:09 -07:00
Andrew Seigner c661b00f8e
Re-implement sidebar resource selectors (#1810)
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2018-10-26 13:28:29 -07:00
Alex Leong 6cffad277b
Make service profile validation a warning instead of an error (#1807)
The existence of an invalid service profile causes `linkerd check` to fail.  This means that it is not possible to open the Linkerd dashboard with the `linkerd dashboard` command.  While service profile validation is useful, it should not lock users out.

Add the ability to designate health checks as warnings.  A failed warning health check will display a warning output in `linkerd check` but will not affect the overall success of the command.  Switch the service profile validation to be a warning.

Signed-off-by: Alex Leong <alex@buoyant.io>
2018-10-26 13:28:10 -07:00
Alex Leong f549868033
Fix integration test and docker build (#1790)
Fix broken docker build by moving Service Profile conversion and validation into `/pkg`.

Fix broken integration test by adding service profile validation output to `check`'s expected output.

Testing done:
* `gotest -v ./...`
* `bin/docker-build`
* `bin/test-run (pwd)/bin/linkerd`

Signed-off-by: Alex Leong <alex@buoyant.io>
2018-10-19 10:23:34 -07:00
Alex Leong 5210b7b44a
Add check for service profile validation (#1775)
Add a check to `linkerd check` which validates all service profile resources.  In particular it checks:
* does the service profile refer to an existent service
* is the service profile valid

Signed-off-by: Alex Leong <alex@buoyant.io>
2018-10-18 16:37:39 -07:00
Alex Leong 1fe19bf3ce
Add ServiceProfile support to k8s utilities (#1758)
Updates to the Kubernetes utility code in `/controller/k8s` to support interacting with ServiceProfiles.

This makes use of the code generated client added in #1752 

Signed-off-by: Alex Leong <alex@buoyant.io>
2018-10-12 09:35:11 -07:00
Kevin Lingerfelt 46c887ca00
Add --single-namespace install flag for restricted permissions (#1721)
* Add --single-namespace install flag for restricted permissions
* Better formatting in install template
* Mark --single-namespace and --proxy-auto-inject as experimental
* Fix wording of --single-namespace check flag
* Small healthcheck refactor

Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-10-11 10:55:57 -07:00
Ivan Sim 4fba6aca0a Proxy init and sidecar containers auto-injection (#1714)
* Support auto sidecar-injection

1. Add proxy-injector deployment spec to cli/install/template.go
2. Inject the Linkerd CA bundle into the MutatingWebhookConfiguration
during the webhook's start-up process.
3. Add a new handler to the CA controller to create a new secret for the
webhook when a new MutatingWebhookConfiguration is created.
4. Declare a config map to store the proxy and proxy-init container
specs used during the auto-inject process.
5. Ignore namespace and pods that are labeled with
linkerd.io/auto-inject: disabled or linkerd.io/auto-inject: completed
6. Add new flag to `linkerd install` to enable/disable proxy
auto-injection

Proposed implementation for #561.

* Resolve missing packages errors
* Move the auto-inject label to the pod level
* PR review items
* Move proxy-injector to its own deployment
* Ignore pods that already have proxy injected

This ensures the webhook doesn't error out due to proxy that are injected using the  command

* PR review items on creating/updating the MWC on-start
* Replace API calls to ConfigMap with file reads
* Fixed post-rebase broken tests
* Don't mutate the auto-inject label

Since we started using healhcheck.HasExistingSidecars() to ensure pods with
existing proxies aren't mutated, we don't need to use the auto-inject label as
an indicator.

This resolves a bug which happens with the kubectl run command where the deployment
is also assigned the auto-inject label. The mutation causes the pod auto-inject
label to not match the deployment label, causing kubectl run to fail.

* Tidy up unit tests
* Include proxy resource requests in sidecar config map
* Fixes to broken YAML in CLI install config

The ignore inbound and outbound ports are changed to string type to
avoid broken YAML caused by the string conversion in the uint slice.

Also, parameterized the proxy bind timeout option in template.go.

Renamed the sidecar config map to
'linkerd-proxy-injector-webhook-config'.

Signed-off-by: ihcsim <ihcsim@gmail.com>
2018-10-10 12:09:22 -07:00
Darko Radisic 6fee0f3c2b Added --context flag to specify the context to use to talk to the Kubernetes apiserver (#1743)
* Added --context flag to specify the context to use to talk to the Kubernetes apiserver
* Fix tests that are failing
* Updated context flag description

Signed-off-by: Darko Radisic <ffd2subroutine@users.noreply.github.com>
2018-10-08 12:37:35 -07:00
Alena Varkockova 5a853e8990 Use ListPods always for data plane HC (#1701)
* Use ListPods always for data plane HC
* Missing changes in grpc_server.go
* Address review comments
* Read proxy version from spec

Signed-off-by: Alena Varkockova <varkockova.a@gmail.com>
2018-10-02 11:45:01 -07:00
Alena Varkockova 8ab9b4981b Make wait flag configurable for check and dashboard (#1654)
Signed-off-by: Alena Varkockova <varkockova.a@gmail.com>
2018-09-19 10:42:29 -07:00
Kevin Lingerfelt 732570648d
Update check command to work with older versions (#1678)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-09-17 19:41:51 -07:00
Kevin Lingerfelt 12b10e27c1
Update version checks to support release channels (#1667)
* Update version checks to support release channels
* Update based on review feedback
* Fix sidebar tests
* Update CI config for edge and stable tags

Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-09-17 17:13:50 -07:00
Alex Leong e65a9617bd
Add can-i checks to linkerd check --pre (#1644)
Add checks to `linkerd check --pre` to verify that the user has permission to create:
* namespaces
* serviceaccounts
* clusterroles
* clusterrolebindings
* services
* deployments
* configmaps

Signed-off-by: Alex Leong <alex@buoyant.io>
2018-09-17 11:31:10 -07:00
Andrew Seigner c3150d2c90
`linkerd check` sends params on version check (#1642)
The `linkerd check` parameter hits
https://versioncheck.linkerd.io/version.json to check for the latest
Linkerd version. This loses information, as that endpoint is intended to
record current version, uuid, and source.

Modify `linkerd check` to set `version`, `uuid`, and `source`
parameters when performing a version check.

Part of #1604.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2018-09-14 15:39:05 -07:00
Kevin Lingerfelt f1b3827194
Bump default check retry time to 5 minutes (#1645)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-09-14 10:58:03 -07:00
Andrew Seigner 7c70531b8e
Add data plane check for metrics Prometheus (#1635)
The `linkerd check` command was not validating whether data plane
proxies were successfully reporting metrics to Prometheus.

Introduce a new check that validates data plane proxies are found in
Prometheus. This is made possible via the existing `ListPods` endpoint
in the public API, which includes an `Added` field, indicating a pod's
metrics were found in Prometheus.

Fixes #1517

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2018-09-13 13:02:05 -07:00
Kevin Lingerfelt b5ff29c8aa
Add data plane check to validate proxy version (#1574)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-09-04 15:22:38 -07:00
Kevin Lingerfelt c7a79da89c
Add data plane check to validate proxies are ready (#1570)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-08-31 15:51:57 -07:00
Risha Mars 136b9cc7c1
Add linkerd check flag to run data plane checks (#1528)
Adds a --proxy flag to the linkerd check CLI command which will run 
to-be-implemented data plane checks
2018-08-28 10:16:24 -07:00
Kevin Lingerfelt 4450a7536d
Add --wait flag for CLI check and dashboard commands (#1503)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-08-22 12:56:42 -07:00
Kevin Lingerfelt 49f6c4c770
Refactor healthcheck init and observe setup (#1502)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-08-22 12:30:45 -07:00
Kevin Lingerfelt 5fc63cde10
Add check for running pods in control plane namepsace (#1498)
* Add check for running pods in control plane namepsace
* Better pod validation logic

Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-08-21 14:36:49 -07:00
Kevin Lingerfelt 53cd3b50d5
Add --pre flag for linkerd check command (#1497)
* Add --pre flag for linkerd check command
* Small adjustments to check help text

Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-08-20 17:09:43 -07:00