Commit Graph

241 Commits

Author SHA1 Message Date
Dennis Adjei-Baah 3e5917f7e0
Add the ability to inject a debug sidecar (#2726)
Signed-off-by: Dennis Adjei-Baah <dennis@buoyant.io>
2019-04-22 16:53:12 -07:00
Alejandro Pedraza c56766a923
Add config.linkerd.io/disable-identity annotation (#2717)
Add config.linkerd.io/disable-identity annotation

First part of #2540

We'll tackle support for `--disable-identity` in `linkerd install` in a
separate commit.

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-04-19 19:04:49 -04:00
Andrew Seigner c7bd8167a9
Remove redundant k8s request in linkerd check (#2722)
PR #2510 introduced some new checks into `linkerd check`. One set of
checks was unnecessarily calling `GetPodsByNamespace` twice.

Remove the redundant k8s API call.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-04-19 12:27:34 -07:00
Gaurav Kumar 392cab80fa Add check for unschedulable pods and psp issues (#2510)
Fixes #2465 

* Add check for unschedulable pods and psp issues (#2465)
* Return error reason and message on pod or node failure

Signed-off-by: Gaurav Kumar <gaurav.kumar9825@gmail.com>
2019-04-19 11:42:22 -07:00
Ivan Sim 8d13084f94
Split the `linkerd-version` CLI flag into `control-plane-version` and `proxy-version` (#2702)
* The 'linkerd-version' CLI flag is renamed to 'control-plane-version'
* Add version field to proxy config
* Add the control plane version to the global config
* Unit test for init image version
* Use more specific control plane and proxy versions in unit tests

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-04-19 11:35:20 -07:00
Andrew Seigner 8323e104fb
Introduce upgrade --from-manifests flag (#2697)
The `linkerd upgrade` command read the control-plane's config from
Kubernetes, which required the environment to be configured to connect
to the appropriate k8s cluster.

Intrdouce a `linkerd upgrade --from-manifests` flag, allowing the user
to feed the output of `linkerd install` into the upgrade command.

Fixes #2629

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-04-17 13:32:21 -07:00
Ivan Sim 4e19827457
Allow identity to be disabled during inject on existing cluster (#2686)
Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-04-11 13:37:06 -07:00
Katerina 938d64a16f Web server updated to read the UUID from the linkerd-config ConfigMap. (#2603)
Signed-off-by: Kateryna Melnyk <kattymelnyk@gmail.com>
2019-04-08 12:56:00 -07:00
Alejandro Pedraza edb225069c
Add validation webhook for service profiles (#2623)
Add validation webhook for service profiles

Fixes #2075

Todo in a follow-up PRs: remove the SP check from the CLI check.

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-04-05 16:10:47 -05:00
Oliver Gould e0ba802f80
proxy-identity: Set a CommonName on CSRs (#2626)
Some CA's (like AWS) require a CN be set on the CSR.

This change modifies proxy-identity to set the identity name as the
CSR's CommonName.

Fixes #2622
2019-04-03 13:54:50 -07:00
Alejandro Pedraza f6fb865183
Enhance webhook unit tests by checking returned JSON patch (#2615)
Enhance webhook unit tests by checking returned JSON patch

Also have labels/annotations added during injection to be added in order

Fixes #2560

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-04-03 15:39:27 -05:00
Ivan Sim 92f15e78a9
Define proxy version override annotation (#2593)
* Define proxy version override annotation
* Don't override global linkerd version during inject

This ensures consistent usages of the config.linkerd.io/linkerd-version and
linkerd.io/proxy-version annotations. The former will only be used to track
overridden version, while the latter shows the cluster's current default
version.

* Rename proxy version config override annotation

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-04-02 14:27:12 -07:00
Andrew Seigner a3bba0e143
Fix logging regex to handle any addr (#2619)
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-04-02 12:47:38 -07:00
Oliver Gould 9051e4f12d
upgrade: Do not require an 'install' config (#2618)
Previous control plane versions do not provide an 'install' config, so
this field cannot be required.

Now, missing empty are handled more gracefully; and upgrade repairs
install configs with missing fields.
2019-04-02 12:01:48 -07:00
Ivan Sim a80335ed51
Disable external profiles by default (#2594)
* Disable external profiles by default
* Rename the --disable-external-profiles flag to --enable-external-profiles

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-04-01 15:13:50 -07:00
Oliver Gould d74ca1bab0
cli: Introduce an upgrade command (#2564)
The `install` command errors when the deploy target contains an existing
Linkerd deployment. The `upgrade` command is introduced to reinstall or
reconfigure the Linkerd control plane.

Upgrade works as follows:

1. The controller config is fetched from the Kubernetes API. The Public
   API is not used, because we need to be able to reinstall the control
   plane when the Public API is not available; and we are not concerned
   about RBAC restrictions preventing the installer from reading the
   config (as we are for inject).

2. The install configuration is read, particularly the flags used during
   the last install/upgrade. If these flags were not set again during the
   upgrade, the previous values are used as if they were passed this time.
   The configuration is updated from the combination of these values,
   including the install configuration itself.

   Note that some flags, including the linkerd-version, are omitted
   since they are stored elsewhere in the configurations and don't make
   sense to track as overrides..

3. The issuer secrets are read from the Kubernetes API so that they can
   be re-used. There is currently no way to reconfigure issuer
   certificates. We will need to create _another_ workflow for
   updating these credentials.

4. The install rendering is invoked with values and config fetched from
   the cluster, synthesized with the new configuration.
2019-04-01 13:27:41 -07:00
Oliver Gould 655632191b
config: Store install parameters with global config (#2577)
When installing Linkerd, a user may override default settings, or may
explicitly configure defaults. Consider install options like `--ha
--controller-replicas=4` -- the `--ha` flag sets a new default value for
the controller-replicas, and then we override it.

When we later upgrade this cluster, how can we know how to configure the
cluster?

We could store EnableHA and ControllerReplicas configurations in the
config, but what if, in a later upgrade, the default value changes? How
can we know whether the user specified an override or just used the
default?

To solve this, we add an `Install` message into a new config.
This message includes (at least) the CLI flags used to invoke
install.

upgrade does not specify defaults for install/proxy-options fields and,
instead, uses the persisted install flags to populate default values,
before applying overrides from the upgrade invocation.

This change breaks the protobuf compatibility by altering the
`installation_uuid` field introduced in 9c442f6885.
Because this change was not yet released (even in an edge release), we
feel that it is safe to break.

Fixes https://github.com/linkerd/linkerd2/issues/2574
2019-03-29 10:04:20 -07:00
Ivan Sim ea07dd3938
Promote the shared injection check to the CLI and webhook (#2555)
Performing this check earlier helps to separate the specialized logic to the CLI
and webhook.
Any subsequent modification of this check logic to support config override of
existing meshed workload will be confined to the relevant component.
The shared lib can then focus only on config overrides.

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-03-27 14:51:05 -07:00
Ivan Sim 9c5bb4ec0c
Convert CLI inject proxy options to annotations (#2547)
* Include the DisableExternalProfile option even if it's 'false'. The override logic depends on this option to assign different profile suffix.
* Check for proxy and init image overrides even when registry option is empty
* Append the config annotations to the pod's meta before creating the patch. This ensures that any configs provided via the CLI options are persisted as annotations before the configs override.
* Persist linkerd version CLI option

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-03-26 14:21:22 -07:00
Alejandro Pedraza 7efe385feb
Have the Webhook react to pod creation/update only (#2472)
Have the Webhook react to pod creation/update only

This was already working almost out-of-the-box, just had to:

- Change the webhook config so it watches pods instead of deployments
- Grant some extra ClusterRole permissions
- Add the piece that figures what's the OwnerReference and add the label
for it
- Manually inject service account mount paths
- Readd volumes tests

Fixes #2342 and #1751

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-03-26 11:53:56 -05:00
Ivan Sim eafd7482f5
Use nested structure to organize workload and object data in ResourceConfig (#2530)
Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-03-22 20:53:29 -07:00
Oliver Gould da0330743f
Provide peer Identities via the Destination API (#2537)
This change reintroduces identity hinting to the destination service.
The Get endpoint includes identities for pods that are injected with an
identity-mode of "default" and have the same linkerd control plane.

A `serviceaccount` label is now also added to destination response
metadata so that it's accessible in prometheus and tap.
2019-03-22 09:19:14 -07:00
Oliver Gould 34ea302a32
inject: Configure proxies to enable Identity (#2536)
This change adds a new `linkerd2-proxy-identity` binary to the `proxy`
container image as well as a `linkerd2-proxy-run` entrypoint script.

The inject process now sets environment variables on pods to support
identity, including identity names for the destination and identity
services.

As the proxy starts, the identity helper creates a key and CSR in a
tmpfs. As the proxy starts, it reads these files, as well as a
serviceaccount token, and provisions a certificate from controller.
The proxy's /ready endpoint will not succeed until a certificate has
been provisioned.

The proxy will not participate in identity with services other than the
controllers until the Destination controller is modified to provide
identities via discovery.
2019-03-21 18:39:05 -07:00
Oliver Gould f02730a90d
Check the cluster's config for install & inject (#2535)
The introduction of identity in 0626fa37 created new state in the
control plane's configuration that must be considered when re-installing
the control plane or when injecting pods.

This change alters `install` to fail if it would seem to conflict with
an existing installation. This behavior may be disabled with the
`--ignore-cluster` flag.

Furthermore, `inject` now _requires_ that it can fetch a configuration
from the control plane in order to operate. Otherwise the
`--ignore-cluster` and `--disable-identity` flags must be specified.

This change does not actually instrument pods to use identity yet---it
lays the framework for proxy identity without changing the test fixture
output (besides a change to how identity HA is configured).

Fixes #2531
2019-03-21 12:49:46 -07:00
Kevin Lingerfelt 0d4eb02835
Add identity pod to check, web, and integration tests (#2529)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2019-03-19 20:49:31 -07:00
Oliver Gould 0626fa374a
install: Introduce the Identity controller (#2526)
https://github.com/linkerd/linkerd2/pull/2521 introduces an "Identity"
controller, but there is no way to include it in linkerd installation.

This change alters the `install` flow as follows:
- An Identity service is _always_ installed;
- Issuer credentials may be specified via the CLI;
- If no Issuer credentials are provided, they are generated each time `install` is called.
- Proxies are NOT configured to use the identity service.
- It's possible to override the credential generation logic---especially
  for tests---via install options that can be configured via the CLI.
2019-03-19 17:04:11 -07:00
Oliver Gould 91c5f07650
proxy: Upgrade to identity-capable proxy (#2524)
The new proxy has changed its configuration as follows:

- `LISTENER` urls are now `LISTEN_ADDR` addresses;
- `CONTROL_URL` is now `DESTINATION_SVC_ADDR`;
- `*_NAMESPACE` vars are no longer needed;
- The `PROXY_ID` is now the `DESTINATION_CONTEXT`;
- The "metrics" port is now the "admin" port, since it serves more than
  just metrics;
- A readiness probe now checks a dedicated /ready endpoint eagerly.

Identity injection is **NOT** configured by this branch.
2019-03-19 14:20:39 -07:00
Oliver Gould 790c13b3b2
Introduce the Identity controller implementation (#2521)
This change introduces a new Identity service implementation for the
`io.linkerd.proxy.identity.Identity` gRPC service.

The `pkg/identity` contains a core, abstract implementation of the service
(generic over both the CA and (Kubernetes) Validator interfaces).

`controller/identity` includes a concrete implementation that uses the
Kubernetes TokenReview API to validate serviceaccount tokens when
issuing certificates.

This change does **NOT** alter installation or runtime to include the
identity service. This will be included in a follow-up.
2019-03-19 13:58:45 -07:00
Oliver Gould 81f645da66
Remove `--tls=optional` and `linkerd-ca` (#2515)
The proxy's TLS implementation has changed to use a new _Identity_ controller.

In preparation for this, the `--tls=optional` CLI flag has been removed
from install and inject; and the `ca` controller has been deleted. Metrics
and UI treatments for TLS have **not** been removed, as they will continue to
be valuable for the new Identity system.

With the removal of the old identity scheme, the Destination service's proxy
ID field is now set with an opaque string (e.g. `ns:emojivoto`) to enable
locality awareness.
2019-03-18 17:40:31 -07:00
Andrew Seigner 024a77ec16
Use correct resource names for authz checks (#2496)
The `linkerd check` command was using TitleCase resource names (e.g.
"ConfigMaps") for SelfSubjectAccessReview requests. These were not
valid, they were only passing because SSARs requests return `allowed`
for unknown resource types unless explicitly restricted.

Modify the `linkerd check` authorization requests to use the correct
resource names.

Steps to reproduce:
- default AKS cluster
- running inside a pod

```bash
$ kubectl proxy
```

Fails:

```bash
$ curl -k -v -XPOST -d'{"kind":"SelfSubjectAccessReview","apiVersion":"authorization.k8s.io/v1","spec":{"resourceAttributes":{"namespace":"default","verb":"create","version":"v1","resource":
"Namespace"}}}'  -H "Accept: application/json, */*" -H "Content-Type: application/json" http://127.0.0.1:8001/apis/authorization.k8s.io/v1/selfsubjectaccessreviews

...

{
  "kind": "SelfSubjectAccessReview",
  "apiVersion": "authorization.k8s.io/v1",
  "metadata": {
    "creationTimestamp": null
  },
  "spec": {
    "resourceAttributes": {
      "namespace": "default",
      "verb": "create",
      "version": "v1",
      "resource": "Namespace"
    }
  },
  "status": {
    "allowed": false
  }
}
```

Works:

```bash
curl -k -v -XPOST -d'{"kind":"SelfSubjectAccessReview","apiVersion":"authorization.k8s.io/v1","spec":{"resourceAttributes":{"namespace":"default","verb":"create","version":"v1","resource":
"namespaces"}}}'  -H "Accept: application/json, */*" -H "Content-Type: application/json" http://127.0.0.1:8001/apis/authorization.k8s.io/v1/selfsubjectaccessreviews

...

{
  "kind": "SelfSubjectAccessReview",
  "apiVersion": "authorization.k8s.io/v1",
  "metadata": {
    "creationTimestamp": null
  },
  "spec": {
    "resourceAttributes": {
      "namespace": "default",
      "verb": "create",
      "version": "v1",
      "resource": "namespaces"
    }
  },
  "status": {
    "allowed": true,
    "reason": "RBAC: allowed by ClusterRoleBinding \"docker-build\" of ClusterRole \"docker-build\" to ServiceAccount \"docker-build/docker-build\""
  }
}
```

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-03-14 10:20:09 -07:00
Ivan Sim 468ad118f2
Support Auto-Inject Configs Overrides Via Annotations (#2471)
* Defined the config annotations as new constants in labels.go
* Introduced the getOverride() functions to override configs
* Introduced new accessors to abstract with type casting

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-03-14 08:42:12 -07:00
Carol A. Scott 92b47ed29b
Specify a default port for the Linkerd dashboard (#2486)
The CLI now specifies a default port, 50750, for the Linkerd dashboard. 
If that port is not available, it resumes the original behavior of binding to a
free ephemeral port.
2019-03-12 15:58:13 -07:00
Andrew Seigner e5d2460792
Remove single namespace functionality (#2474)
linkerd/linkerd2#1721 introduced a `--single-namespace` install flag,
enabling the control-plane to function within a single namespace. With
the introduction of ServiceProfiles, and upcoming identity changes, this
single namespace mode of operation is becoming less viable.

This change removes the `--single-namespace` install flag, and all
underlying support. The control-plane must have cluster-wide access to
operate.

A few related changes:
- Remove `--single-namespace` from `linkerd check`, this motivates
  combining some check categories, as we can always assume cluster-wide
  requirements.
- Simplify the `k8s.ResourceAuthz` API, as callers no longer need to
  make a decision based on cluster-wide vs. namespace-wide access.
  Components either have access, or they error out.
- Modify the web dashboard to always assume ServiceProfiles are enabled.

Reverts #1721
Part of #2337

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-03-12 00:17:22 -07:00
Andrew Seigner a42e8db45f
Quiet inject logging (#2483)
Manual and auto injection was logging the full patch JSON at the `Info`
level.

Modify injection to log the object type and name at the `Info` level,
and the full patch at the `Debug` level.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-03-11 10:39:10 -07:00
Alejandro Pedraza 0da851842b
Public API endpoint `Config()` (#2455)
Public API endpoint `Config()`

Retrieves Global and Proxy configurations.
Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-03-07 17:37:46 -05:00
Andrew Seigner f258cf0d3a
Introduce cli command to fetch proxy metrics (#2445)
It's sometimes helpful to spotcheck proxy metrics from a specific pod,
but doing so with kubectl requires a few steps.

Introduce a new `linkerd metrics` command. When given a pod name and
namespace, returns a dump of the proxy's /metrics endpoint.

Also modify the k8s.portforward module to accept initialized k8s config
and client objects, to enable testing.

Fixes #2350.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-03-07 10:24:38 -08:00
Andrew Seigner 8da2cd3fd4
Require cluster-wide k8s API access (#2428)
linkerd/linkerd2#2349 removed the `--single-namespace` flag, in favor of
runtime detection of cluster vs. namespace access, and also
ServiceProfile availability. This maintained control-plane support for
running in these two states.

This change requires control-plane components have cluster-wide
Kubernetes API access and ServiceProfile availability, and will error
out if not. Once #2349 merges, stage 1 install will be a requirement for
a successful stage 2 install.

Part of #2337

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-03-07 10:23:18 -08:00
Aditya Sharma 3740aa238a Remove `--api-port` flag from the cli (#2429)
* Changed the protobuf definition to take out destinationApiPort entirely
* Store destinationAPIPort as a constant in pkg/inject.go

Fixes #2351

Signed-off-by: Aditya Sharma <hello@adi.run>
2019-03-06 15:54:12 -08:00
Alejandro Pedraza 54f943a725
Uninject: remove labels at the top level (#2442)
Fixes #2377

In inject's ResourceConfig, renamed objMeta to podMeta since
it really points to the pod template metadata. And created a new field
workloadMeta that really points to the main workload (e.g. Deployment) metadata.

Refactored uninject to clean up the labels at both podMeta and
workloadMeta. Also it will remove all the labels and annotations that
start with "linkerd.io" except for the "linkerd.io/inject" annotation.

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-03-06 14:35:59 -05:00
Alejandro Pedraza f155fb9a8f
Have `NewFakeClientSets()` not swallow errors when parsing YAML (#2454)
This helps catching bad YAMLs in test resources

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-03-06 13:53:04 -05:00
Andrew Seigner d90fa16727
Introduce NET_ADMIN cli check (#2421)
The `linkerd-init` container requires the NET_ADMIN capability to modify
iptables. The `linkerd check` command was not verifying this.

Introduce a `has NET_ADMIN capability` check, which does the following:
1) Lists all available PodSecurityPolicies, if none found, returns
success
2) For each PodSecurityPolicy, validate one exists that:
    - the user has `use` access AND
    - provides `*` or `NET_ADMIN` capability

A couple limitations to this approach:
- It is testing whether the user running `linkerd check` has NET_ADMIN,
  but during installation time it will be the `linkerd-init` pod that
  requires NET_ADMIN.
- It assumes the presense of PodSecurityPolicies in the cluster means
  the PodSecurityPolicy admission controller is installed. If the
  admission controller is not installed, but PSPs exists that restrict
  NET_ADMIN, `linkerd check` will incorrectly report the user does not
  have that capability.

This PR also fixes the `can create CustomResourceDefinitions` check to
not specify a namespace when doing a `create` check, as CRDs are
cluster-wide.

Fixes #1732

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-03-05 17:19:11 -08:00
Alejandro Pedraza ddf2e729ac
Injection consolidation (#2334)
- Created the pkg/inject package to hold the new injection shared lib.
- Extracted from `/cli/cmd/inject.go` and `/cli/cmd/inject_util.go`
the core methods doing the workload parsing and injection, and moved them into
`/pkg/inject/inject.go`. The CLI files should now deal only with
strictly CLI concerns, and applying the json patch returned by the new
lib.
- Proceeded analogously with `/cli/cmd/uninject.go` and
`/pkg/inject/uninject.go`.
- The `InjectReport` struct and helping methods were moved into
`/pkg/inject/report.go`
- Refactored webhook to use the new injection lib
- Removed linkerd-proxy-injector-sidecar-config ConfigMap
- Added the ability to add pod labels and annotations without having to
specify the already existing ones

Fixes #1748, #2289

Signed-off-by: Alejandro Pedraza <alejandro.pedraza@gmail.com>
2019-03-05 08:38:56 -05:00
Tarun Pothulapati 2184928813 Wire up stats for Jobs (#2416)
Support for Jobs in stat/tap/top cli commands

Part of #2007

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2019-03-01 17:16:54 -08:00
Yan 4cd1f99e89 Check kubectl version as part of checks (#2358)
Fixes #2354

Signed-off-by: Yan Babitski <yan.babitski@gmail.com>
2019-03-01 10:03:59 -08:00
Oliver Gould ab90263461
destination: Only return TLS identities when appropriate (#2371)
As described in #2217, the controller returns TLS identities for results even
when the destination pod may not be able to participate in identity
requester: specifically, the other pod may not have the same controller
namespace or it may not be injected with identity.

This change introduces a new annotation, linkerd.io/identity-mode that is set
when injecting pods (via both CLI and webhook). This annotation is always
added.

The destination service now only returns TLS identities when this annotation
is set to optional on a pod and the destination pod uses the same controller.
These semantics are expected to change before the 2.3 release.

Fixes #2217
2019-02-27 12:18:39 -08:00
Andrew Seigner 9f748d2d2e
lint: Enable unparam (#2369)
unparam reports unused function parameters:
https://github.com/mvdan/unparam

Part of #217

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-27 10:34:02 -08:00
Oliver Gould d03014aed6
Lessen klog for security (#2386)
We currently set klog to maximum verbosity when debug logging is
enabled. This causes control plane components, however, to log their
serviceaccount tokens, leaking secret information into logs.

By setting the klog level to 6, we avoid this logging.

Fixes #2383
2019-02-26 16:00:26 -08:00
Andrew Seigner ec5a0ca8d9
Authorization-aware control-plane components (#2349)
The control-plane components relied on a `--single-namespace` param,
passed from `linkerd install` into each individual component, to
determine which namespaces they were authorized to access, and whether
to support ServiceProfiles. This command-line flag was redundant given
the authorization rules encoded in the parent `linkerd install` output,
via [Cluster]Role[Binding]s.

Modify the control-plane components to query Kubernetes at startup to
determine which namespaces they are authorized to access, and whether
ServiceProfile support is available. This allows removal of the
`--single-namespace` flag on the components.

Also update `bin/test-cleanup` to cleanup the ServiceProfile CRD.

TODO:
- Remove `--single-namespace` flag on `linkerd install`, part of #2164

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-26 11:54:52 -08:00
Andrew Seigner ad0d0b72a0
lint: Enable unconvert (#2368)
unconvert removes unnecessary type conversions:
https://github.com/mdempsky/unconvert

Part of #217

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-25 16:07:42 -08:00
Andrew Seigner 25e462352d
lint: Enable goimports (#2366)
goimports checks import lines, adding missing ones and removing
unreferenced ones:
https://godoc.org/golang.org/x/tools/cmd/goimports

It also requires named imports for packages whose
import paths don't match their package names:
- https://github.com/golang/go/issues/28428
- https://go-review.googlesource.com/c/tools/+/145699/

Also standardized named imports of common Kubernetes packaages.

Part of #217

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-25 15:51:10 -08:00
Oliver Gould f7435800da
lint: Enable scopelint (#2364)
[scopelint][scopelint] detects a nasty reference-scoping issue in loops.

[scopelint]: https://github.com/kyoh86/scopelint
2019-02-24 08:59:51 -08:00
Andrew Seigner 7fa7e962cb
Fix hint URLs not display for RPC checks (#2361)
Hint URLs should display for all failed checks in `linkerd check`, but
were not displaying for RPC checks.

Fix `runCheckRPC` to pass along the hintAnchor to the check result.

Also rename the second `can query the control plane API` to
`control plane self-check`, as there were two checks with that name.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-23 13:35:32 -08:00
Andrew Seigner cc3ff70f29
Enable `unused` linter (#2357)
`unused` checks Go code for unused constants, variables, functions, and
types.

Part of #217

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-23 11:05:39 -08:00
Andrew Seigner 4b6f6aeedd
Enable gosimple linter, fix issues (#2356)
gosimple is a Go linter that specializes in simplifying code

Also fix one spelling error in `cred_test.go`

Part of #217

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-22 17:19:07 -08:00
Oliver Gould 2640943c67
pkg/tls: Make it possible to load a CA from disk (#2335)
In preparation for creating an Identity service that can chain off of an
existing CA, it's necessary to both (1) be able to create an
intermediate CA that can be used by the identity service and (2) be able
to load a CA from existing key material.

This changes the public API of the `tls` package to deal in actual key
types (rather than opaque blobs) and provides a set of helpers that can
be used to convert these credentials between common formats.
2019-02-22 15:13:50 -08:00
Kevin Lingerfelt b1a6b6ca66
Fix handling of kubeconfig server urls that include paths (#2305)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2019-02-18 17:57:27 -08:00
Andrew Seigner 1df1683b6a
Instrument k8s clients (#2243)
The control-plane's clients, specifically the Kubernetes clients, did
not provide telemetry information.

Introduce a `prometheus.ClientWithTelemetry` wrapper to instrument
arbitrary clients. Apply this wrapper to Kubernetes clients.

Fixes #2183

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-18 09:10:02 -08:00
Kevin Leimkuhler b2bbeb05ef
Issue 2276: Do not log error when timeout is blank (#2279)
# Problem

When a route does not specify a timeout, the proxy-api defaults to the default
timeout and logs an error:

```
time="2019-02-13T16:29:12Z" level=error msg="failed to parse duration for route POST /io.linkerd.proxy.destination.Destination/GetProfile: time: invalid duration"
```

# Solution

We now check if a route timeout is blank. If it is not set, it is set to
`DefaultRouteTimeout`. If it is set, we try to parse it into a `Duration`.

A request was made to improve logging to include the service profile and
namespace as well.

# Validation

With valid service profiles installed, edit the `.yaml` to include an invalid
`timeout`:

```
...
name: GET /
timeout: foo
```

We should now see the following errors:

```
proxy-api time="2019-02-13T22:27:32Z" level=error msg="failed to parse duration for route 'GET /' in service profile 'webapp.default.svc.cluster.local' in namespace 'default': time: invalid duration foo"
```

This error does not show up when `timeout` is blank.

Fixes #2276

Signed-off-by: Kevin Leimkuhler <kevinl@buoyant.io>
2019-02-14 17:09:02 -08:00
Andrew Seigner 2305974202
Introduce golangci-lint tooling, fixes (#2239)
`golangci-lint` performs numerous checks on Go code, including golint,
ineffassign, govet, and gofmt.

This change modifies `bin/lint` to use `golangci-lint`, and replaces
usage of golint and govet.

Also perform a one-time gofmt cleanup:
- `gofmt -s -w controller/`
- `gofmt -s -w pkg/`

Part of #217

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-13 11:16:28 -08:00
Andrew Seigner 1a60237a94
Update check command hint URLs to new alias (#2245)
The existing hint URLs printing by `linkerd check` pointed to locations
that would change if the linkerd.io website was reorganized.

linkerd/website#148 introduces an alias for hint URLs at
https://linkerd.io/checks/. This is the corresponding change to update
`linkerd check` output.

Depends on linkerd/website#148, relates to linkerd/website#146.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-11 11:00:16 -08:00
Alex Leong 3f333c2860
Validate service profiles in all namespaces (#2237)
Fixes #2220 

The service profile validation which is part of `linkerd check` only validates service profiles in the Linkerd namespace.  Due to a recent change, service profiles now can exist in any namespace.

Update the logic so that service profiles in all namespaces are validated.  
Additionally:
* Relax validation of service profile names to support external names

Signed-off-by: Alex Leong <alex@buoyant.io>
2019-02-11 09:52:47 -08:00
Ivan Sim f6e75ec83a
Add statefulsets to the dashboard and CLI (#2234)
Fixes #1983

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-02-08 15:37:44 -08:00
Alex Leong 5b054785e5
Read service profiles from client or server namespace instead of control namespace (#2200)
Fixes #2077 

When looking up service profiles, Linkerd always looks for the service profile objects in the Linkerd control namespace.  This is limiting because service owners who wish to create service profiles may not have write access to the Linkerd control namespace.

Instead, we have the control plane look for the service profile in both the client namespace (as read from the proxy's `proxy_id` field from the GetProfiles request and from the service's namespace.  If a service profile exists in both namespaces, the client namespace takes priority.  In this way, clients may override the behavior dictated by the service.

Signed-off-by: Alex Leong <alex@buoyant.io>
2019-02-07 14:51:43 -08:00
Andrew Seigner 907f01fba6
Improve ServiceProfile validation in linkerd check (#2218)
The `linkerd check` command was doing limited validation on
ServiceProfiles.

Make ServiceProfile validation more complete, specifically validate:
- types of all fields
- presence of required fields
- presence of unknown fields
- recursive fields

Also move all validation code into a new `Validate` function in the
profiles package.

Validation of field types and required fields is handled via
`yaml.UnmarshalStrict` in the `Validate` function. This motivated
migrating from github.com/ghodss/yaml to a fork, sigs.k8s.io/yaml.

Fixes #2190
2019-02-07 14:35:47 -08:00
Andrew Seigner 72812baf99
Introduce Discovery API and endpoints command (#2195)
The Proxy API service lacked introspection of its internal state.

Introduce a new gRPC Discovery API, implemented by two servers:
1) Proxy API Server: returns a snapshot of discovery state
2) Public API Server: pass-through to the Proxy API Server

Also wire up a new `linkerd endpoints` command.

Fixes #2165

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-07 14:02:21 -08:00
Dennis Adjei-Baah e98ba06e5f
Modify klog to be verbose when controller log-level is set to debug (#2221)
The controller logs innocuous messages when control plane proxies aren't ready to route requests during startup from each control plane component. i.e. tap, public-api and proxy-api. Setting the log level in the control plane to `INFO` would not hide these log messages and would still show up on control plane startup.

This PR modifies `klogs` initial flag set to route innocuous logs to `/dev/null` if the controller log level is set to INFO. If set to debug, we output all loglines to stderr.

Fixes #2171 #2168
Signed-off-by: Dennis Adjei-Baah <dennis@buoyant.io>
2019-02-07 13:48:42 -08:00
Risha Mars e531655d26
Add a --tap flag to the linkerd profile command (#2139)
Adds the ability to generate a service profile by running a tap for a configurable 
amount of time, and using the route results from the routes seen during the tap.

e.g. `linkerd profile web --tap deploy/web -n emojivoto --tap-duration 2s`
2019-02-06 12:43:16 -08:00
Alejandro Pedraza 50fbcc60b5
Add support for `basePath` in swagger 2.0 files (#2211)
Fixes #2175

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-02-06 13:44:35 -05:00
Alejandro Pedraza 0b42a02bb9
Build request urls in the most generic possible way (#2206)
Build request urls in the most generic possible way

Fixes #2132

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-02-06 13:22:17 -05:00
Alejandro Pedraza 2a7654ce78
Consolidate timeouts for `linkerd check` (#2191)
Consolidate timeouts for `linkerd check`

- Moved the creation of contexts from inside the methods targeted by the
checks into a single place in the runCheck() and runCheckRPC() methods
where the context is built using a hard-coded timeout of 30 seconds.
- k8s' client-go doesn't allow passing along contexts, but it let's us
setting the Timeout manually.
- Reworded the description for the --wait option.

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-02-05 11:38:30 -05:00
Oliver Gould 4798ad3f44
Use the proper controller identity when configuring pods with TLS (#2196)
Since 37ae423, deployments have been prefixed with linkerd-; however
the inject logic was not changed to take this into consideration when
constructing the controller's identity.

This means that the proxy's client to the control plane has been unable to
establish TLS'd communcation to the proxy-api. Previously, the proxy would
silently fall back to plaintext, but in master this behavior recently changed to
be stricter, so this bug will prevent the proxy from connecting to proxy-api
in any way.
2019-02-04 14:59:03 -08:00
Andrew Seigner 3a139d0202
Fix spelling on linkerd check link (#2197)
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-04 14:00:11 -08:00
Kevin Lingerfelt 4c019c27c1
Add pod spec annotation to disable injection in CLI and auto-injector (#2187)
* Add pod spec annotation to disable injection in CLI and auto-injector
* Remove support for linkerd.io/auto-inject label entirely
* Update based on review feedback
* Fix issue with finding the namespace of deployments applied to the default ns

Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2019-02-01 16:57:06 -08:00
Alex Leong 3bd4231cec
Add support for timeouts in service profiles (#2149)
Fixes #2042 

Adds a new field to service profile routes called `timeout`.  Any requests to that route which take longer than the given timeout will be aborted and a 504 response will be returned instead.  If the timeout field is not specified, a default timeout of 10 seconds is used.

Signed-off-by: Alex Leong <alex@buoyant.io>
2019-01-30 16:48:55 -08:00
Andrew Seigner 5651e02496
Add hint URLs for all checks (#2159)
linkerd/website#105 introduced a FAQ page, providing resolutions for all
`linkerd check` failures.

Update each check to reference its corresponding section in the FAQ.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-01-30 16:26:57 -08:00
Alejandro Pedraza fe234cade1
Use `ca.NewCA()` for generating certs and keys for the proxy injector (#2163)
Use `ca.NewCA()` for generating certs and keys for the proxy injector

- Remove from CA controller everything that dealt with the
webhook/proxy-injector
- Remove no longer needed proxy-injector volumes for 'trust-anchors' and
'webhook-secrets'
- Remove from the proxy-injector the retrieval of the trust anchor and
secrets
- tls flag during install is no longer needed for auto-inject to work

Fixes #2095 and fixes #2166

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-01-30 16:04:33 -05:00
Thomas Rampelberg ea61630f9d
Update client-go to 1.13.1 (#2160)
* Update client-go to 1.13.1

Fixes #2145

* Update Dockerfile-bin with new tag

* Update all the dockerfile tags

* Clean gopkg and do not apply cluster defaults

* Update for klog

* Match existing behavior with klog

* Add klog to gopkg.lock

* Update go-deps shas

* Update klog comment

* Update comment to be a non-sentence
2019-01-28 17:42:14 -08:00
Alex Leong 872e1bb026
Add --proto flag to linkerd profile command to read protobuf files (#2128)
Fixes #1425 

Signed-off-by: Alex Leong <alex@buoyant.io>
2019-01-25 11:15:20 -08:00
Alex Leong d542571b65
GetProfiles should always respond with a (possibly empty) profile immediately (#2146)
When `GetProfiles` is called for a destination that does not have a service profile, the proxy-api service does not return any messages until a service profile is created for that service.  This can be interpreted as hanging, and can make it difficult to calculate response latency metrics.

Change the behavior of the API to always return a service profile message immediately.  If the service does not have a service profile, the default service profile is returned.

Signed-off-by: Alex Leong <alex@buoyant.io>
2019-01-24 15:22:14 -08:00
zak 8c413ca38b Wire up stats commands for daemonsets (#2006) (#2086)
DaemonSet stats are not currently shown in the cli stat command, web ui
or grafana dashboard. This commit adds daemonset support for stat.

Update stat command's help message to reference daemonsets.
Update the public-api to support stats for daemonsets.
Add tests for stat summary and api.

Add daemonset get/list/watch permissions to the linkerd-controller
cluster role that's created using the install command.
Update golden expectation test files for install command
yaml manifest output.

Update web UI with daemonsets
Update navigation, overview and pages to list daemonsets and the pods
associated to them.
Add daemonset paths to server, and ui apps.

Add grafana dashboard for daemonsets; a clone of the deployment
dashboard.

Update dependencies and dockerfile hashes

Add DaemonSet support to tap and top commands

Fixes of #2006

Signed-off-by: Zak Knill <zrjknill@gmail.com>
2019-01-24 14:34:13 -08:00
Dennis Adjei-Baah 20efe14fef
Fix issue where linkerd check would panic with replicationcontroller pod name in control plane (#2140)
When running Linkerd check with a control plane namespace that may contain an additional pod with a replication controller ID for pod names instead of a replicaSet ID, the check command panics because of an "index out of bounds" error.

This PR adds a check to make sure that, when parsing pod names during the `checkControllerRunning` healthcheck, we only check for linkerd control plane pods and that the pod name results in four or more substrings when split on '-'. This prevents the check from panicking when encountering a replication controller ID pod name.

Fixes #2084 

Signed-off-by: Dennis Adjei-Baah <dennis@buoyant.io>
2019-01-24 10:26:11 -08:00
Risha Mars 950c952d14
Add a less scary message to the user when retries are still in progress (#2141)
When a failing check is being retried, we show the current err to the user. This
can sometimes be unnecessarily alarming, as in the case of the control plane
starting up.

If a failing check is in the process of being retried, wait to show the final
error message until the retries have completed.
2019-01-24 10:24:59 -08:00
Risha Mars e7556d7edc
Check RoleBindings for specified single namespace only (#2142)
Previously, we were doing the creation checks for both Roles/RoleBindings and
ClusterRoles/ClusterRoleBindings for all namespaces, but in --single-namespace
mode, we only need to check that these can be created in the control plane
namespace.
2019-01-23 18:04:15 -08:00
Andrew Seigner c9ac77cd7c
Introduce version consistency checks (#2130)
Version checks were not validating that the cli version matched the
control plane or data plane versions.

Add checks via the `linkerd check` command to validate the cli is
running the same version as the control and data plane.

Also add types around `channel-version` string parsing and matching. A
consequence being that during development `version.Version` changes from
`undefined` to `dev-undefined`.

Fixes #2076

Depends on linkerd/website#101

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-01-23 16:54:43 -08:00
Alena Varkockova 28f662c9c6 Introduce resource selector and deprecate namespace field for ListPods (#2025)
* Introduce resource selector and deprecate namespace field for ListPods
* Changes from code review
* Properly deprecate the field
* Do not check for nil
* Fix the mockProm usage
* Protoc changes revert
* Changed from code review

Signed-off-by: Alena Varkockova <varkockova.a@gmail.com>
2019-01-23 10:35:55 -08:00
Alejandro Pedraza eacc09b7ba
Follow up to #2054: move magic strings into constants (#2122)
Follow up to #2054: move magic strings into constants

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-01-22 14:53:24 -05:00
Alejandro Pedraza bd1955c317 When injecting, perform an uninject as a first step (#2089)
* When injecting, perform an uninject as a first step

Fixes #1970

The fixture `inject_emojivoto_already_injected.input.yml` is no longer
rejected, so I created the corresponding golden file.

Note that we'll still forbid injection over resources already injected
with third party meshes (Istio, Contour), so now we have
`HasExisting3rdPartySidecars()` to detect that.

* Generalize HasExistingSidecars() to cater for both the auto-injector and
* Convert `linkerd uninject` result format to the one used in `linkerd inject`.
* More updates to the uninject reports. Revert changes to the HasExistingSidecars func.

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-01-17 10:15:23 -08:00
Andrew Seigner 92f2cd9b63
Update check and inject output (#2087)
The outputs of the `check` and `inject` commands did not vary much
between successful and failed executions, and were a bit verbose and
challenging to parse.

Reorganize output of `check` and `inject` commands, to provide more
output when errors occur, and less output when successful.

Specific changes:

`linkerd check`
- visually group checks by category
- introduce `hintURL`'s, to provide doc links when checks fail
- add spinners when retrying, remove additional retry lines
- colored unicode characters to indicate success/warning/failure

`linkerd inject`
- modify default output to mirror `kubectl apply`
- only output non-successful inject reports
- support `--verbose` flag to output all inject reports

Fixes #1471, #1653, #1656, #1739

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-01-16 15:14:14 -08:00
Alex Leong 771542dde2
Add support for retries (#2038) 2019-01-16 14:13:48 -08:00
Andrew Seigner dacd8819ff
Group checkers by category (#2083)
The linkerd check command organized the various checks via loosely
coupled category IDs, category names, and checkers themselves, all with
ordering defined by consumers of this code.

This change removes category IDs in favor of category names, groups all
checkers by category, and enforces ordering at the HealthChecker
level.

Part of #1471, depends on #2078.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-01-14 18:01:32 -08:00
Andrew Seigner 04373414ef
Modify all health checks to be specified via enums (#2078)
The set of health checks to be executed were dependent on a combination
of check enums and boolean options.

This change modifies the health checks to be governed strictly by a set
of enums.

Next steps:
- tightly couple category IDs to names
- tightly couple checks to their parent categories
- programmatic control over check ordering

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-01-14 17:16:15 -08:00
Alejandro Pedraza d358605269
Added `linkerd uninject` command (#2054)
Add `linkerd uninject` command

uninject.go iterates through the resources annotations, labels,
initContainers and Containers, removing what we know was injected by
linkerd.

The biggest part of this commit is the refactoring of inject.go, to make
it more generic and reusable by uninject.

The idea is that in a following PR this functionality will get reused by
`linkerd inject` to uninject as as preliminary step to injection, as a
solution to #1970.

This was tested successfully on emojivoto with:

```
1) inject:
kubectl get -n emojivoto deployment -o yaml | bin/linkerd inject - |
kubectl apply -f -
2) uninject:
kubectl get -n emojivoto deployment -o yaml | bin/linkerd uninject - |
kubectl apply -f -
```

Also created unit tests for uninject.go. The fixture files from the inject
tests could be reused. But as now the input files act as outputs, they
represent existing resources and required these changes (that didn't
affect inject):
  - Rearranged fields in alphabetical order.
  - Added fields that are only relevant for existing resources (e.g.
    creationTimestamp and status.replicas in StatefulSets)

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-01-14 17:39:31 -05:00
Kevin Lingerfelt ed3fbd75f3
Setup port-forwarding for linkerd dashboard command (#2052)
* Setup port-forwarding for linkerd dashboard command
* Output port-forward logs when --verbose flag is set

Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2019-01-10 10:16:08 -08:00
Alejandro Pedraza 281ba37e6d
More granular control on checks made by CLI commands (#2033)
Have the CLI commands `get`, `routes`, `stat`, `tap`and `top` perform a more limited set of checks 

Fixes #1854
2019-01-10 09:13:44 -05:00
Andrew Seigner 1c302182ef
Enable lint check for comments (#2023)
Commit 1: Enable lint check for comments

Part of #217. Follow up from #1982 and #2018.

A subsequent commit will fix the ci failure.

Commit 2: Address all comment-related linter errors.

This change addresses all comment-related linter errors by doing the
following:
- Add comments to exported symbols
- Make some exported symbols private
- Recommend via TODOs that some exported symbols should should move or
  be removed

This PR does not:
- Modify, move, or remove any code
- Modify existing comments

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-01-02 14:03:59 -08:00
Kevin Lingerfelt f1b0983f72
Add go linting to CI config (#2018)
* Add go linting to CI config
* Fix lint warnings
* Add note about bin/lint script in TEST.md

Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-12-20 15:33:09 -08:00
Radu M 07cbfe2725 Fix most golint issues that are not comment related (#1982)
Signed-off-by: Radu Matei <radu@radu-matei.com>
2018-12-20 10:37:47 -08:00
Kevin Lingerfelt 86e95b7ad3
Disable serivce profiles in single-namespace mode (#1980)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-12-13 14:37:18 -08:00
Kevin Lingerfelt fd44896644
Remove namespace definition from --single-namespace installs (#1974)
* Remove namespace definition from --single-namespace installs
* DRY up code in healthcheck.go

Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-12-12 14:53:02 -08:00
Kevin Lingerfelt 0f8bcc9159
Controller: wait for caches to sync before opening listeners (#1958)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-12-07 11:15:45 -08:00