Commit Graph

257 Commits

Author SHA1 Message Date
Alena Varkockova 7973715ee4 Output check result as json (#2666)
* Add missing file, make linter happy
* Changes from code review
* Ivan's code review
* PR review feedback
* style fixes
* Last PR comments

Signed-off-by: Alena Varkockova <varkockova.a@gmail.com>
2019-05-20 09:04:28 -07:00
Dennis Adjei-Baah a0fa1dff59
Move tap service into its own pod. (#2773)
* Split tap into its own pod in the control plane

Signed-off-by: Dennis Adjei-Baah <dennis@buoyant.io>
2019-05-15 16:28:44 -05:00
mg b965d0d30e Introduce pre-install healthcheck for clock skew (#2803)
* Adding pre-install check for clock skew
* Fixing lint error - time.Since
* Update test data for clock skew check
* Incorporating code review comments
* Additional fix - clock skew test

Signed-off-by: Matej Gera <matejgera@gmail.com>
2019-05-13 10:14:38 -07:00
Alejandro Pedraza 065c221858
Support for resources opting-out of tap (#2807)
Support for resources opting out of tap

Implements the `linkerd inject --disable-tap` flag (although hidden pending #2811) and the config override annotation `config.linkerd.io/disable-tap`.
Fixes #2778

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-05-10 14:17:23 -05:00
Andrew Seigner 2e323e7e9e
Fix Service Profile CRD check to handle k8s errors (#2781)
`ServiceProfilesAccess()`, called by control plane components at
startup, would fail if it encountered an `ErrGroupDiscoveryFailed` from
a GroupVersion request. This error is mostly innocuous, as it returns an
error if any GroupVersion fails. `ServiceProfilesAccess()` only needs to
validate ServiceProfiles are available.

Modify `ServiceProfilesAccess()` to specifically request the
ServiceProfile GroupVersion. Also add Discovery object
(`APIResourceList`) support to `NewFakeClientSets`.

Fixes #2780

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-05-03 19:51:39 +02:00
Andrew Seigner ad2f92662e
Fix check/dashboard failing from one pod when HA (#2764)
The `linkerd check` and `linkerd dashboard` commands validate control
plane pods are up via the `LinkerdAPIChecks` category of checks. These
checks will fail if a single pod is not ready, even in HA mode.

Modify the underlying `validateControlPlanePods` check to return
successful if at least one pod per control plane component is ready.

Fixes #2554

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-05-03 18:42:21 +02:00
Jack Price f758a9e428 Use port-forwarding for linkerd CLIs (#2757)
Private k8s clusters, such as the private GKE clusters offered by Google
Cloud, cannot be reached through the current API proxy method.

This commit uses the port forwarding feature already developed.

Also modify dashboard command to not fall back to ephemeral port.

Signed-off-by: Jack Price <jackprice@outlook.com>
2019-05-02 14:41:26 +02:00
Andrew Seigner 66494591e0
Multi-stage check support (#2765)
Add support for `linkerd check config`. Validates the existence of the
Linkerd Namespace, ClusterRoles, ClusterRoleBindings, ServiceAccounts,
and CustomResourceDefitions.

Part of #2337

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-04-30 17:17:59 +01:00
Ivan Sim 714035fee9
Define default resource spec for proxy-init init container (#2763)
Fixes #2750 

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-04-29 11:41:05 -07:00
Andrew Seigner 0cfc8c6f1c
Introduce k8s apiextensions support (#2759)
CustomResourceDefinition parsing and retrieval is not available via
client-go's `kubernetes.Interface`, but rather via a separate
`k8s.io/apiextensions-apiserver` package.

Introduce support for CustomResourceDefintion object parsing and
retrieval. This change facilitates retrieval of CRDs from the k8s API
server, and also provides CRD resources as mock objects.

Also introduce a `NewFakeAPI` constructor, deprecating
`NewFakeClientSets`. Callers need no longer be concerned with discreet
clientsets (for k8s resources vs. CRDs vs. (eventually)
ServiceProfiles), and can instead use the unified `KubernetesAPI`.

Part of #2337, in service to multi-stage check.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-04-28 18:55:22 -07:00
Andrew Seigner dec620f818
Fix `linkerd check --proxy` with default ns param (#2754)
The `linkerd check --proxy` command checks for proxies in all
namespaces, if the `--namespace` flag is not set. PR #2747 modified the
behavior of `KubernetesAPI.NamespaceExists`. Previously it would succeed
if given an emptry string for a namespace. Now it fails with a
`resource name may not be empty` error (for k8s server `v1.10.11`), or a
not found error (for our fake test client).

Modify the data plane proxy namespace check to return success if the
namespace is not set.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-04-25 16:37:08 -07:00
Andrew Seigner ec540a882e
Consolidate k8s APIs (#2747)
Numerous codepaths have emerged that create k8s configs, k8s clients,
and make k8s api requests.

This branch consolidates k8s client creation and APIs. The primary
change migrates most codepaths to call `k8s.NewAPI` to instantiate a
`KubernetesAPI` struct from `pkg`. `KubernetesAPI` implements the
`kubernetes.Interface` (clientset) interface, and also persists a
`client-go` `rest.Config`.

Specific list of changes:
- removes manual GET requests from `k8s.KubernetesAPI`, in favor of
  clientsets
- replaces most calls to `k8s.GetConfig`+`kubernetes.NewForConfig` with
  a single `k8s.NewAPI`
- introduces a `timeout` param to `k8s.NewAPI`, currently only used by
  healthchecks
- removes `NewClientSet` in `controller/k8s/clientset.go` in favor of
  `k8s.NewAPI`
- removes `httpClient` and `clientset` from `HealthChecker`, use
  `KubernetesAPI` instead

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-04-25 11:31:38 -07:00
Ivan Sim cd37d3f0f5
Fall back to default built-in version if versions config are missing (#2745)
Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-04-24 19:49:18 -07:00
Dennis Adjei-Baah c062cf433c
Add profiling endpoint for all control-plane components (#2739)
* add golang profile endpoint to control plane components

Signed-off-by: Dennis Adjei-Baah <dennis@buoyant.io>
2019-04-24 09:59:38 -07:00
Alejandro Pedraza 62d9a80894
New `linkerd inject` default and manual modes (#2721)
Fixes #2720 and 2711 

This changes the default behavior of `linkerd inject` to not inject the
proxy but just the `linkerd.io/inject: enabled` annotation for the
auto-injector to pick it up (regardless of any namespace annotation).

A new `--manual` mode was added, which behaves as before, injecting
the proxy in the command output.

The unit tests are running with `--manual` to avoid any changes in the
fixtures.

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-04-24 09:05:27 -05:00
Andrew Seigner b2b4780430
Introduce install stages (#2719)
This change introduces two named parameters for `linkerd install`, split
by privilege:
- `linkerd install config`
  - Namespace
  - ClusterRoles
  - ClusterRoleBindings
  - CustomResourceDefinition
  - ServiceAccounts
- `linkerd install control-plane`
  - ConfigMaps
  - Secrets
  - Deployments
  - Services

Comprehensive `linkerd install` is still supported.

TODO:
- `linkerd check` support
- `linkerd upgrade` support
- integration tests

Part of #2337

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-04-23 14:52:34 -07:00
Dennis Adjei-Baah 3e5917f7e0
Add the ability to inject a debug sidecar (#2726)
Signed-off-by: Dennis Adjei-Baah <dennis@buoyant.io>
2019-04-22 16:53:12 -07:00
Alejandro Pedraza c56766a923
Add config.linkerd.io/disable-identity annotation (#2717)
Add config.linkerd.io/disable-identity annotation

First part of #2540

We'll tackle support for `--disable-identity` in `linkerd install` in a
separate commit.

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-04-19 19:04:49 -04:00
Andrew Seigner c7bd8167a9
Remove redundant k8s request in linkerd check (#2722)
PR #2510 introduced some new checks into `linkerd check`. One set of
checks was unnecessarily calling `GetPodsByNamespace` twice.

Remove the redundant k8s API call.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-04-19 12:27:34 -07:00
Gaurav Kumar 392cab80fa Add check for unschedulable pods and psp issues (#2510)
Fixes #2465 

* Add check for unschedulable pods and psp issues (#2465)
* Return error reason and message on pod or node failure

Signed-off-by: Gaurav Kumar <gaurav.kumar9825@gmail.com>
2019-04-19 11:42:22 -07:00
Ivan Sim 8d13084f94
Split the `linkerd-version` CLI flag into `control-plane-version` and `proxy-version` (#2702)
* The 'linkerd-version' CLI flag is renamed to 'control-plane-version'
* Add version field to proxy config
* Add the control plane version to the global config
* Unit test for init image version
* Use more specific control plane and proxy versions in unit tests

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-04-19 11:35:20 -07:00
Andrew Seigner 8323e104fb
Introduce upgrade --from-manifests flag (#2697)
The `linkerd upgrade` command read the control-plane's config from
Kubernetes, which required the environment to be configured to connect
to the appropriate k8s cluster.

Intrdouce a `linkerd upgrade --from-manifests` flag, allowing the user
to feed the output of `linkerd install` into the upgrade command.

Fixes #2629

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-04-17 13:32:21 -07:00
Ivan Sim 4e19827457
Allow identity to be disabled during inject on existing cluster (#2686)
Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-04-11 13:37:06 -07:00
Katerina 938d64a16f Web server updated to read the UUID from the linkerd-config ConfigMap. (#2603)
Signed-off-by: Kateryna Melnyk <kattymelnyk@gmail.com>
2019-04-08 12:56:00 -07:00
Alejandro Pedraza edb225069c
Add validation webhook for service profiles (#2623)
Add validation webhook for service profiles

Fixes #2075

Todo in a follow-up PRs: remove the SP check from the CLI check.

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-04-05 16:10:47 -05:00
Oliver Gould e0ba802f80
proxy-identity: Set a CommonName on CSRs (#2626)
Some CA's (like AWS) require a CN be set on the CSR.

This change modifies proxy-identity to set the identity name as the
CSR's CommonName.

Fixes #2622
2019-04-03 13:54:50 -07:00
Alejandro Pedraza f6fb865183
Enhance webhook unit tests by checking returned JSON patch (#2615)
Enhance webhook unit tests by checking returned JSON patch

Also have labels/annotations added during injection to be added in order

Fixes #2560

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-04-03 15:39:27 -05:00
Ivan Sim 92f15e78a9
Define proxy version override annotation (#2593)
* Define proxy version override annotation
* Don't override global linkerd version during inject

This ensures consistent usages of the config.linkerd.io/linkerd-version and
linkerd.io/proxy-version annotations. The former will only be used to track
overridden version, while the latter shows the cluster's current default
version.

* Rename proxy version config override annotation

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-04-02 14:27:12 -07:00
Andrew Seigner a3bba0e143
Fix logging regex to handle any addr (#2619)
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-04-02 12:47:38 -07:00
Oliver Gould 9051e4f12d
upgrade: Do not require an 'install' config (#2618)
Previous control plane versions do not provide an 'install' config, so
this field cannot be required.

Now, missing empty are handled more gracefully; and upgrade repairs
install configs with missing fields.
2019-04-02 12:01:48 -07:00
Ivan Sim a80335ed51
Disable external profiles by default (#2594)
* Disable external profiles by default
* Rename the --disable-external-profiles flag to --enable-external-profiles

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-04-01 15:13:50 -07:00
Oliver Gould d74ca1bab0
cli: Introduce an upgrade command (#2564)
The `install` command errors when the deploy target contains an existing
Linkerd deployment. The `upgrade` command is introduced to reinstall or
reconfigure the Linkerd control plane.

Upgrade works as follows:

1. The controller config is fetched from the Kubernetes API. The Public
   API is not used, because we need to be able to reinstall the control
   plane when the Public API is not available; and we are not concerned
   about RBAC restrictions preventing the installer from reading the
   config (as we are for inject).

2. The install configuration is read, particularly the flags used during
   the last install/upgrade. If these flags were not set again during the
   upgrade, the previous values are used as if they were passed this time.
   The configuration is updated from the combination of these values,
   including the install configuration itself.

   Note that some flags, including the linkerd-version, are omitted
   since they are stored elsewhere in the configurations and don't make
   sense to track as overrides..

3. The issuer secrets are read from the Kubernetes API so that they can
   be re-used. There is currently no way to reconfigure issuer
   certificates. We will need to create _another_ workflow for
   updating these credentials.

4. The install rendering is invoked with values and config fetched from
   the cluster, synthesized with the new configuration.
2019-04-01 13:27:41 -07:00
Oliver Gould 655632191b
config: Store install parameters with global config (#2577)
When installing Linkerd, a user may override default settings, or may
explicitly configure defaults. Consider install options like `--ha
--controller-replicas=4` -- the `--ha` flag sets a new default value for
the controller-replicas, and then we override it.

When we later upgrade this cluster, how can we know how to configure the
cluster?

We could store EnableHA and ControllerReplicas configurations in the
config, but what if, in a later upgrade, the default value changes? How
can we know whether the user specified an override or just used the
default?

To solve this, we add an `Install` message into a new config.
This message includes (at least) the CLI flags used to invoke
install.

upgrade does not specify defaults for install/proxy-options fields and,
instead, uses the persisted install flags to populate default values,
before applying overrides from the upgrade invocation.

This change breaks the protobuf compatibility by altering the
`installation_uuid` field introduced in 9c442f6885.
Because this change was not yet released (even in an edge release), we
feel that it is safe to break.

Fixes https://github.com/linkerd/linkerd2/issues/2574
2019-03-29 10:04:20 -07:00
Ivan Sim ea07dd3938
Promote the shared injection check to the CLI and webhook (#2555)
Performing this check earlier helps to separate the specialized logic to the CLI
and webhook.
Any subsequent modification of this check logic to support config override of
existing meshed workload will be confined to the relevant component.
The shared lib can then focus only on config overrides.

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-03-27 14:51:05 -07:00
Ivan Sim 9c5bb4ec0c
Convert CLI inject proxy options to annotations (#2547)
* Include the DisableExternalProfile option even if it's 'false'. The override logic depends on this option to assign different profile suffix.
* Check for proxy and init image overrides even when registry option is empty
* Append the config annotations to the pod's meta before creating the patch. This ensures that any configs provided via the CLI options are persisted as annotations before the configs override.
* Persist linkerd version CLI option

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-03-26 14:21:22 -07:00
Alejandro Pedraza 7efe385feb
Have the Webhook react to pod creation/update only (#2472)
Have the Webhook react to pod creation/update only

This was already working almost out-of-the-box, just had to:

- Change the webhook config so it watches pods instead of deployments
- Grant some extra ClusterRole permissions
- Add the piece that figures what's the OwnerReference and add the label
for it
- Manually inject service account mount paths
- Readd volumes tests

Fixes #2342 and #1751

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-03-26 11:53:56 -05:00
Ivan Sim eafd7482f5
Use nested structure to organize workload and object data in ResourceConfig (#2530)
Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-03-22 20:53:29 -07:00
Oliver Gould da0330743f
Provide peer Identities via the Destination API (#2537)
This change reintroduces identity hinting to the destination service.
The Get endpoint includes identities for pods that are injected with an
identity-mode of "default" and have the same linkerd control plane.

A `serviceaccount` label is now also added to destination response
metadata so that it's accessible in prometheus and tap.
2019-03-22 09:19:14 -07:00
Oliver Gould 34ea302a32
inject: Configure proxies to enable Identity (#2536)
This change adds a new `linkerd2-proxy-identity` binary to the `proxy`
container image as well as a `linkerd2-proxy-run` entrypoint script.

The inject process now sets environment variables on pods to support
identity, including identity names for the destination and identity
services.

As the proxy starts, the identity helper creates a key and CSR in a
tmpfs. As the proxy starts, it reads these files, as well as a
serviceaccount token, and provisions a certificate from controller.
The proxy's /ready endpoint will not succeed until a certificate has
been provisioned.

The proxy will not participate in identity with services other than the
controllers until the Destination controller is modified to provide
identities via discovery.
2019-03-21 18:39:05 -07:00
Oliver Gould f02730a90d
Check the cluster's config for install & inject (#2535)
The introduction of identity in 0626fa37 created new state in the
control plane's configuration that must be considered when re-installing
the control plane or when injecting pods.

This change alters `install` to fail if it would seem to conflict with
an existing installation. This behavior may be disabled with the
`--ignore-cluster` flag.

Furthermore, `inject` now _requires_ that it can fetch a configuration
from the control plane in order to operate. Otherwise the
`--ignore-cluster` and `--disable-identity` flags must be specified.

This change does not actually instrument pods to use identity yet---it
lays the framework for proxy identity without changing the test fixture
output (besides a change to how identity HA is configured).

Fixes #2531
2019-03-21 12:49:46 -07:00
Kevin Lingerfelt 0d4eb02835
Add identity pod to check, web, and integration tests (#2529)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2019-03-19 20:49:31 -07:00
Oliver Gould 0626fa374a
install: Introduce the Identity controller (#2526)
https://github.com/linkerd/linkerd2/pull/2521 introduces an "Identity"
controller, but there is no way to include it in linkerd installation.

This change alters the `install` flow as follows:
- An Identity service is _always_ installed;
- Issuer credentials may be specified via the CLI;
- If no Issuer credentials are provided, they are generated each time `install` is called.
- Proxies are NOT configured to use the identity service.
- It's possible to override the credential generation logic---especially
  for tests---via install options that can be configured via the CLI.
2019-03-19 17:04:11 -07:00
Oliver Gould 91c5f07650
proxy: Upgrade to identity-capable proxy (#2524)
The new proxy has changed its configuration as follows:

- `LISTENER` urls are now `LISTEN_ADDR` addresses;
- `CONTROL_URL` is now `DESTINATION_SVC_ADDR`;
- `*_NAMESPACE` vars are no longer needed;
- The `PROXY_ID` is now the `DESTINATION_CONTEXT`;
- The "metrics" port is now the "admin" port, since it serves more than
  just metrics;
- A readiness probe now checks a dedicated /ready endpoint eagerly.

Identity injection is **NOT** configured by this branch.
2019-03-19 14:20:39 -07:00
Oliver Gould 790c13b3b2
Introduce the Identity controller implementation (#2521)
This change introduces a new Identity service implementation for the
`io.linkerd.proxy.identity.Identity` gRPC service.

The `pkg/identity` contains a core, abstract implementation of the service
(generic over both the CA and (Kubernetes) Validator interfaces).

`controller/identity` includes a concrete implementation that uses the
Kubernetes TokenReview API to validate serviceaccount tokens when
issuing certificates.

This change does **NOT** alter installation or runtime to include the
identity service. This will be included in a follow-up.
2019-03-19 13:58:45 -07:00
Oliver Gould 81f645da66
Remove `--tls=optional` and `linkerd-ca` (#2515)
The proxy's TLS implementation has changed to use a new _Identity_ controller.

In preparation for this, the `--tls=optional` CLI flag has been removed
from install and inject; and the `ca` controller has been deleted. Metrics
and UI treatments for TLS have **not** been removed, as they will continue to
be valuable for the new Identity system.

With the removal of the old identity scheme, the Destination service's proxy
ID field is now set with an opaque string (e.g. `ns:emojivoto`) to enable
locality awareness.
2019-03-18 17:40:31 -07:00
Andrew Seigner 024a77ec16
Use correct resource names for authz checks (#2496)
The `linkerd check` command was using TitleCase resource names (e.g.
"ConfigMaps") for SelfSubjectAccessReview requests. These were not
valid, they were only passing because SSARs requests return `allowed`
for unknown resource types unless explicitly restricted.

Modify the `linkerd check` authorization requests to use the correct
resource names.

Steps to reproduce:
- default AKS cluster
- running inside a pod

```bash
$ kubectl proxy
```

Fails:

```bash
$ curl -k -v -XPOST -d'{"kind":"SelfSubjectAccessReview","apiVersion":"authorization.k8s.io/v1","spec":{"resourceAttributes":{"namespace":"default","verb":"create","version":"v1","resource":
"Namespace"}}}'  -H "Accept: application/json, */*" -H "Content-Type: application/json" http://127.0.0.1:8001/apis/authorization.k8s.io/v1/selfsubjectaccessreviews

...

{
  "kind": "SelfSubjectAccessReview",
  "apiVersion": "authorization.k8s.io/v1",
  "metadata": {
    "creationTimestamp": null
  },
  "spec": {
    "resourceAttributes": {
      "namespace": "default",
      "verb": "create",
      "version": "v1",
      "resource": "Namespace"
    }
  },
  "status": {
    "allowed": false
  }
}
```

Works:

```bash
curl -k -v -XPOST -d'{"kind":"SelfSubjectAccessReview","apiVersion":"authorization.k8s.io/v1","spec":{"resourceAttributes":{"namespace":"default","verb":"create","version":"v1","resource":
"namespaces"}}}'  -H "Accept: application/json, */*" -H "Content-Type: application/json" http://127.0.0.1:8001/apis/authorization.k8s.io/v1/selfsubjectaccessreviews

...

{
  "kind": "SelfSubjectAccessReview",
  "apiVersion": "authorization.k8s.io/v1",
  "metadata": {
    "creationTimestamp": null
  },
  "spec": {
    "resourceAttributes": {
      "namespace": "default",
      "verb": "create",
      "version": "v1",
      "resource": "namespaces"
    }
  },
  "status": {
    "allowed": true,
    "reason": "RBAC: allowed by ClusterRoleBinding \"docker-build\" of ClusterRole \"docker-build\" to ServiceAccount \"docker-build/docker-build\""
  }
}
```

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-03-14 10:20:09 -07:00
Ivan Sim 468ad118f2
Support Auto-Inject Configs Overrides Via Annotations (#2471)
* Defined the config annotations as new constants in labels.go
* Introduced the getOverride() functions to override configs
* Introduced new accessors to abstract with type casting

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-03-14 08:42:12 -07:00
Carol A. Scott 92b47ed29b
Specify a default port for the Linkerd dashboard (#2486)
The CLI now specifies a default port, 50750, for the Linkerd dashboard. 
If that port is not available, it resumes the original behavior of binding to a
free ephemeral port.
2019-03-12 15:58:13 -07:00
Andrew Seigner e5d2460792
Remove single namespace functionality (#2474)
linkerd/linkerd2#1721 introduced a `--single-namespace` install flag,
enabling the control-plane to function within a single namespace. With
the introduction of ServiceProfiles, and upcoming identity changes, this
single namespace mode of operation is becoming less viable.

This change removes the `--single-namespace` install flag, and all
underlying support. The control-plane must have cluster-wide access to
operate.

A few related changes:
- Remove `--single-namespace` from `linkerd check`, this motivates
  combining some check categories, as we can always assume cluster-wide
  requirements.
- Simplify the `k8s.ResourceAuthz` API, as callers no longer need to
  make a decision based on cluster-wide vs. namespace-wide access.
  Components either have access, or they error out.
- Modify the web dashboard to always assume ServiceProfiles are enabled.

Reverts #1721
Part of #2337

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-03-12 00:17:22 -07:00
Andrew Seigner a42e8db45f
Quiet inject logging (#2483)
Manual and auto injection was logging the full patch JSON at the `Info`
level.

Modify injection to log the object type and name at the `Info` level,
and the full patch at the `Debug` level.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-03-11 10:39:10 -07:00
Alejandro Pedraza 0da851842b
Public API endpoint `Config()` (#2455)
Public API endpoint `Config()`

Retrieves Global and Proxy configurations.
Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-03-07 17:37:46 -05:00
Andrew Seigner f258cf0d3a
Introduce cli command to fetch proxy metrics (#2445)
It's sometimes helpful to spotcheck proxy metrics from a specific pod,
but doing so with kubectl requires a few steps.

Introduce a new `linkerd metrics` command. When given a pod name and
namespace, returns a dump of the proxy's /metrics endpoint.

Also modify the k8s.portforward module to accept initialized k8s config
and client objects, to enable testing.

Fixes #2350.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-03-07 10:24:38 -08:00
Andrew Seigner 8da2cd3fd4
Require cluster-wide k8s API access (#2428)
linkerd/linkerd2#2349 removed the `--single-namespace` flag, in favor of
runtime detection of cluster vs. namespace access, and also
ServiceProfile availability. This maintained control-plane support for
running in these two states.

This change requires control-plane components have cluster-wide
Kubernetes API access and ServiceProfile availability, and will error
out if not. Once #2349 merges, stage 1 install will be a requirement for
a successful stage 2 install.

Part of #2337

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-03-07 10:23:18 -08:00
Aditya Sharma 3740aa238a Remove `--api-port` flag from the cli (#2429)
* Changed the protobuf definition to take out destinationApiPort entirely
* Store destinationAPIPort as a constant in pkg/inject.go

Fixes #2351

Signed-off-by: Aditya Sharma <hello@adi.run>
2019-03-06 15:54:12 -08:00
Alejandro Pedraza 54f943a725
Uninject: remove labels at the top level (#2442)
Fixes #2377

In inject's ResourceConfig, renamed objMeta to podMeta since
it really points to the pod template metadata. And created a new field
workloadMeta that really points to the main workload (e.g. Deployment) metadata.

Refactored uninject to clean up the labels at both podMeta and
workloadMeta. Also it will remove all the labels and annotations that
start with "linkerd.io" except for the "linkerd.io/inject" annotation.

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-03-06 14:35:59 -05:00
Alejandro Pedraza f155fb9a8f
Have `NewFakeClientSets()` not swallow errors when parsing YAML (#2454)
This helps catching bad YAMLs in test resources

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-03-06 13:53:04 -05:00
Andrew Seigner d90fa16727
Introduce NET_ADMIN cli check (#2421)
The `linkerd-init` container requires the NET_ADMIN capability to modify
iptables. The `linkerd check` command was not verifying this.

Introduce a `has NET_ADMIN capability` check, which does the following:
1) Lists all available PodSecurityPolicies, if none found, returns
success
2) For each PodSecurityPolicy, validate one exists that:
    - the user has `use` access AND
    - provides `*` or `NET_ADMIN` capability

A couple limitations to this approach:
- It is testing whether the user running `linkerd check` has NET_ADMIN,
  but during installation time it will be the `linkerd-init` pod that
  requires NET_ADMIN.
- It assumes the presense of PodSecurityPolicies in the cluster means
  the PodSecurityPolicy admission controller is installed. If the
  admission controller is not installed, but PSPs exists that restrict
  NET_ADMIN, `linkerd check` will incorrectly report the user does not
  have that capability.

This PR also fixes the `can create CustomResourceDefinitions` check to
not specify a namespace when doing a `create` check, as CRDs are
cluster-wide.

Fixes #1732

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-03-05 17:19:11 -08:00
Alejandro Pedraza ddf2e729ac
Injection consolidation (#2334)
- Created the pkg/inject package to hold the new injection shared lib.
- Extracted from `/cli/cmd/inject.go` and `/cli/cmd/inject_util.go`
the core methods doing the workload parsing and injection, and moved them into
`/pkg/inject/inject.go`. The CLI files should now deal only with
strictly CLI concerns, and applying the json patch returned by the new
lib.
- Proceeded analogously with `/cli/cmd/uninject.go` and
`/pkg/inject/uninject.go`.
- The `InjectReport` struct and helping methods were moved into
`/pkg/inject/report.go`
- Refactored webhook to use the new injection lib
- Removed linkerd-proxy-injector-sidecar-config ConfigMap
- Added the ability to add pod labels and annotations without having to
specify the already existing ones

Fixes #1748, #2289

Signed-off-by: Alejandro Pedraza <alejandro.pedraza@gmail.com>
2019-03-05 08:38:56 -05:00
Tarun Pothulapati 2184928813 Wire up stats for Jobs (#2416)
Support for Jobs in stat/tap/top cli commands

Part of #2007

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2019-03-01 17:16:54 -08:00
Yan 4cd1f99e89 Check kubectl version as part of checks (#2358)
Fixes #2354

Signed-off-by: Yan Babitski <yan.babitski@gmail.com>
2019-03-01 10:03:59 -08:00
Oliver Gould ab90263461
destination: Only return TLS identities when appropriate (#2371)
As described in #2217, the controller returns TLS identities for results even
when the destination pod may not be able to participate in identity
requester: specifically, the other pod may not have the same controller
namespace or it may not be injected with identity.

This change introduces a new annotation, linkerd.io/identity-mode that is set
when injecting pods (via both CLI and webhook). This annotation is always
added.

The destination service now only returns TLS identities when this annotation
is set to optional on a pod and the destination pod uses the same controller.
These semantics are expected to change before the 2.3 release.

Fixes #2217
2019-02-27 12:18:39 -08:00
Andrew Seigner 9f748d2d2e
lint: Enable unparam (#2369)
unparam reports unused function parameters:
https://github.com/mvdan/unparam

Part of #217

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-27 10:34:02 -08:00
Oliver Gould d03014aed6
Lessen klog for security (#2386)
We currently set klog to maximum verbosity when debug logging is
enabled. This causes control plane components, however, to log their
serviceaccount tokens, leaking secret information into logs.

By setting the klog level to 6, we avoid this logging.

Fixes #2383
2019-02-26 16:00:26 -08:00
Andrew Seigner ec5a0ca8d9
Authorization-aware control-plane components (#2349)
The control-plane components relied on a `--single-namespace` param,
passed from `linkerd install` into each individual component, to
determine which namespaces they were authorized to access, and whether
to support ServiceProfiles. This command-line flag was redundant given
the authorization rules encoded in the parent `linkerd install` output,
via [Cluster]Role[Binding]s.

Modify the control-plane components to query Kubernetes at startup to
determine which namespaces they are authorized to access, and whether
ServiceProfile support is available. This allows removal of the
`--single-namespace` flag on the components.

Also update `bin/test-cleanup` to cleanup the ServiceProfile CRD.

TODO:
- Remove `--single-namespace` flag on `linkerd install`, part of #2164

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-26 11:54:52 -08:00
Andrew Seigner ad0d0b72a0
lint: Enable unconvert (#2368)
unconvert removes unnecessary type conversions:
https://github.com/mdempsky/unconvert

Part of #217

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-25 16:07:42 -08:00
Andrew Seigner 25e462352d
lint: Enable goimports (#2366)
goimports checks import lines, adding missing ones and removing
unreferenced ones:
https://godoc.org/golang.org/x/tools/cmd/goimports

It also requires named imports for packages whose
import paths don't match their package names:
- https://github.com/golang/go/issues/28428
- https://go-review.googlesource.com/c/tools/+/145699/

Also standardized named imports of common Kubernetes packaages.

Part of #217

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-25 15:51:10 -08:00
Oliver Gould f7435800da
lint: Enable scopelint (#2364)
[scopelint][scopelint] detects a nasty reference-scoping issue in loops.

[scopelint]: https://github.com/kyoh86/scopelint
2019-02-24 08:59:51 -08:00
Andrew Seigner 7fa7e962cb
Fix hint URLs not display for RPC checks (#2361)
Hint URLs should display for all failed checks in `linkerd check`, but
were not displaying for RPC checks.

Fix `runCheckRPC` to pass along the hintAnchor to the check result.

Also rename the second `can query the control plane API` to
`control plane self-check`, as there were two checks with that name.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-23 13:35:32 -08:00
Andrew Seigner cc3ff70f29
Enable `unused` linter (#2357)
`unused` checks Go code for unused constants, variables, functions, and
types.

Part of #217

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-23 11:05:39 -08:00
Andrew Seigner 4b6f6aeedd
Enable gosimple linter, fix issues (#2356)
gosimple is a Go linter that specializes in simplifying code

Also fix one spelling error in `cred_test.go`

Part of #217

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-22 17:19:07 -08:00
Oliver Gould 2640943c67
pkg/tls: Make it possible to load a CA from disk (#2335)
In preparation for creating an Identity service that can chain off of an
existing CA, it's necessary to both (1) be able to create an
intermediate CA that can be used by the identity service and (2) be able
to load a CA from existing key material.

This changes the public API of the `tls` package to deal in actual key
types (rather than opaque blobs) and provides a set of helpers that can
be used to convert these credentials between common formats.
2019-02-22 15:13:50 -08:00
Kevin Lingerfelt b1a6b6ca66
Fix handling of kubeconfig server urls that include paths (#2305)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2019-02-18 17:57:27 -08:00
Andrew Seigner 1df1683b6a
Instrument k8s clients (#2243)
The control-plane's clients, specifically the Kubernetes clients, did
not provide telemetry information.

Introduce a `prometheus.ClientWithTelemetry` wrapper to instrument
arbitrary clients. Apply this wrapper to Kubernetes clients.

Fixes #2183

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-18 09:10:02 -08:00
Kevin Leimkuhler b2bbeb05ef
Issue 2276: Do not log error when timeout is blank (#2279)
# Problem

When a route does not specify a timeout, the proxy-api defaults to the default
timeout and logs an error:

```
time="2019-02-13T16:29:12Z" level=error msg="failed to parse duration for route POST /io.linkerd.proxy.destination.Destination/GetProfile: time: invalid duration"
```

# Solution

We now check if a route timeout is blank. If it is not set, it is set to
`DefaultRouteTimeout`. If it is set, we try to parse it into a `Duration`.

A request was made to improve logging to include the service profile and
namespace as well.

# Validation

With valid service profiles installed, edit the `.yaml` to include an invalid
`timeout`:

```
...
name: GET /
timeout: foo
```

We should now see the following errors:

```
proxy-api time="2019-02-13T22:27:32Z" level=error msg="failed to parse duration for route 'GET /' in service profile 'webapp.default.svc.cluster.local' in namespace 'default': time: invalid duration foo"
```

This error does not show up when `timeout` is blank.

Fixes #2276

Signed-off-by: Kevin Leimkuhler <kevinl@buoyant.io>
2019-02-14 17:09:02 -08:00
Andrew Seigner 2305974202
Introduce golangci-lint tooling, fixes (#2239)
`golangci-lint` performs numerous checks on Go code, including golint,
ineffassign, govet, and gofmt.

This change modifies `bin/lint` to use `golangci-lint`, and replaces
usage of golint and govet.

Also perform a one-time gofmt cleanup:
- `gofmt -s -w controller/`
- `gofmt -s -w pkg/`

Part of #217

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-13 11:16:28 -08:00
Andrew Seigner 1a60237a94
Update check command hint URLs to new alias (#2245)
The existing hint URLs printing by `linkerd check` pointed to locations
that would change if the linkerd.io website was reorganized.

linkerd/website#148 introduces an alias for hint URLs at
https://linkerd.io/checks/. This is the corresponding change to update
`linkerd check` output.

Depends on linkerd/website#148, relates to linkerd/website#146.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-11 11:00:16 -08:00
Alex Leong 3f333c2860
Validate service profiles in all namespaces (#2237)
Fixes #2220 

The service profile validation which is part of `linkerd check` only validates service profiles in the Linkerd namespace.  Due to a recent change, service profiles now can exist in any namespace.

Update the logic so that service profiles in all namespaces are validated.  
Additionally:
* Relax validation of service profile names to support external names

Signed-off-by: Alex Leong <alex@buoyant.io>
2019-02-11 09:52:47 -08:00
Ivan Sim f6e75ec83a
Add statefulsets to the dashboard and CLI (#2234)
Fixes #1983

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-02-08 15:37:44 -08:00
Alex Leong 5b054785e5
Read service profiles from client or server namespace instead of control namespace (#2200)
Fixes #2077 

When looking up service profiles, Linkerd always looks for the service profile objects in the Linkerd control namespace.  This is limiting because service owners who wish to create service profiles may not have write access to the Linkerd control namespace.

Instead, we have the control plane look for the service profile in both the client namespace (as read from the proxy's `proxy_id` field from the GetProfiles request and from the service's namespace.  If a service profile exists in both namespaces, the client namespace takes priority.  In this way, clients may override the behavior dictated by the service.

Signed-off-by: Alex Leong <alex@buoyant.io>
2019-02-07 14:51:43 -08:00
Andrew Seigner 907f01fba6
Improve ServiceProfile validation in linkerd check (#2218)
The `linkerd check` command was doing limited validation on
ServiceProfiles.

Make ServiceProfile validation more complete, specifically validate:
- types of all fields
- presence of required fields
- presence of unknown fields
- recursive fields

Also move all validation code into a new `Validate` function in the
profiles package.

Validation of field types and required fields is handled via
`yaml.UnmarshalStrict` in the `Validate` function. This motivated
migrating from github.com/ghodss/yaml to a fork, sigs.k8s.io/yaml.

Fixes #2190
2019-02-07 14:35:47 -08:00
Andrew Seigner 72812baf99
Introduce Discovery API and endpoints command (#2195)
The Proxy API service lacked introspection of its internal state.

Introduce a new gRPC Discovery API, implemented by two servers:
1) Proxy API Server: returns a snapshot of discovery state
2) Public API Server: pass-through to the Proxy API Server

Also wire up a new `linkerd endpoints` command.

Fixes #2165

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-07 14:02:21 -08:00
Dennis Adjei-Baah e98ba06e5f
Modify klog to be verbose when controller log-level is set to debug (#2221)
The controller logs innocuous messages when control plane proxies aren't ready to route requests during startup from each control plane component. i.e. tap, public-api and proxy-api. Setting the log level in the control plane to `INFO` would not hide these log messages and would still show up on control plane startup.

This PR modifies `klogs` initial flag set to route innocuous logs to `/dev/null` if the controller log level is set to INFO. If set to debug, we output all loglines to stderr.

Fixes #2171 #2168
Signed-off-by: Dennis Adjei-Baah <dennis@buoyant.io>
2019-02-07 13:48:42 -08:00
Risha Mars e531655d26
Add a --tap flag to the linkerd profile command (#2139)
Adds the ability to generate a service profile by running a tap for a configurable 
amount of time, and using the route results from the routes seen during the tap.

e.g. `linkerd profile web --tap deploy/web -n emojivoto --tap-duration 2s`
2019-02-06 12:43:16 -08:00
Alejandro Pedraza 50fbcc60b5
Add support for `basePath` in swagger 2.0 files (#2211)
Fixes #2175

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-02-06 13:44:35 -05:00
Alejandro Pedraza 0b42a02bb9
Build request urls in the most generic possible way (#2206)
Build request urls in the most generic possible way

Fixes #2132

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-02-06 13:22:17 -05:00
Alejandro Pedraza 2a7654ce78
Consolidate timeouts for `linkerd check` (#2191)
Consolidate timeouts for `linkerd check`

- Moved the creation of contexts from inside the methods targeted by the
checks into a single place in the runCheck() and runCheckRPC() methods
where the context is built using a hard-coded timeout of 30 seconds.
- k8s' client-go doesn't allow passing along contexts, but it let's us
setting the Timeout manually.
- Reworded the description for the --wait option.

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-02-05 11:38:30 -05:00
Oliver Gould 4798ad3f44
Use the proper controller identity when configuring pods with TLS (#2196)
Since 37ae423, deployments have been prefixed with linkerd-; however
the inject logic was not changed to take this into consideration when
constructing the controller's identity.

This means that the proxy's client to the control plane has been unable to
establish TLS'd communcation to the proxy-api. Previously, the proxy would
silently fall back to plaintext, but in master this behavior recently changed to
be stricter, so this bug will prevent the proxy from connecting to proxy-api
in any way.
2019-02-04 14:59:03 -08:00
Andrew Seigner 3a139d0202
Fix spelling on linkerd check link (#2197)
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-04 14:00:11 -08:00
Kevin Lingerfelt 4c019c27c1
Add pod spec annotation to disable injection in CLI and auto-injector (#2187)
* Add pod spec annotation to disable injection in CLI and auto-injector
* Remove support for linkerd.io/auto-inject label entirely
* Update based on review feedback
* Fix issue with finding the namespace of deployments applied to the default ns

Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2019-02-01 16:57:06 -08:00
Alex Leong 3bd4231cec
Add support for timeouts in service profiles (#2149)
Fixes #2042 

Adds a new field to service profile routes called `timeout`.  Any requests to that route which take longer than the given timeout will be aborted and a 504 response will be returned instead.  If the timeout field is not specified, a default timeout of 10 seconds is used.

Signed-off-by: Alex Leong <alex@buoyant.io>
2019-01-30 16:48:55 -08:00
Andrew Seigner 5651e02496
Add hint URLs for all checks (#2159)
linkerd/website#105 introduced a FAQ page, providing resolutions for all
`linkerd check` failures.

Update each check to reference its corresponding section in the FAQ.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-01-30 16:26:57 -08:00
Alejandro Pedraza fe234cade1
Use `ca.NewCA()` for generating certs and keys for the proxy injector (#2163)
Use `ca.NewCA()` for generating certs and keys for the proxy injector

- Remove from CA controller everything that dealt with the
webhook/proxy-injector
- Remove no longer needed proxy-injector volumes for 'trust-anchors' and
'webhook-secrets'
- Remove from the proxy-injector the retrieval of the trust anchor and
secrets
- tls flag during install is no longer needed for auto-inject to work

Fixes #2095 and fixes #2166

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-01-30 16:04:33 -05:00
Thomas Rampelberg ea61630f9d
Update client-go to 1.13.1 (#2160)
* Update client-go to 1.13.1

Fixes #2145

* Update Dockerfile-bin with new tag

* Update all the dockerfile tags

* Clean gopkg and do not apply cluster defaults

* Update for klog

* Match existing behavior with klog

* Add klog to gopkg.lock

* Update go-deps shas

* Update klog comment

* Update comment to be a non-sentence
2019-01-28 17:42:14 -08:00
Alex Leong 872e1bb026
Add --proto flag to linkerd profile command to read protobuf files (#2128)
Fixes #1425 

Signed-off-by: Alex Leong <alex@buoyant.io>
2019-01-25 11:15:20 -08:00
Alex Leong d542571b65
GetProfiles should always respond with a (possibly empty) profile immediately (#2146)
When `GetProfiles` is called for a destination that does not have a service profile, the proxy-api service does not return any messages until a service profile is created for that service.  This can be interpreted as hanging, and can make it difficult to calculate response latency metrics.

Change the behavior of the API to always return a service profile message immediately.  If the service does not have a service profile, the default service profile is returned.

Signed-off-by: Alex Leong <alex@buoyant.io>
2019-01-24 15:22:14 -08:00
zak 8c413ca38b Wire up stats commands for daemonsets (#2006) (#2086)
DaemonSet stats are not currently shown in the cli stat command, web ui
or grafana dashboard. This commit adds daemonset support for stat.

Update stat command's help message to reference daemonsets.
Update the public-api to support stats for daemonsets.
Add tests for stat summary and api.

Add daemonset get/list/watch permissions to the linkerd-controller
cluster role that's created using the install command.
Update golden expectation test files for install command
yaml manifest output.

Update web UI with daemonsets
Update navigation, overview and pages to list daemonsets and the pods
associated to them.
Add daemonset paths to server, and ui apps.

Add grafana dashboard for daemonsets; a clone of the deployment
dashboard.

Update dependencies and dockerfile hashes

Add DaemonSet support to tap and top commands

Fixes of #2006

Signed-off-by: Zak Knill <zrjknill@gmail.com>
2019-01-24 14:34:13 -08:00
Dennis Adjei-Baah 20efe14fef
Fix issue where linkerd check would panic with replicationcontroller pod name in control plane (#2140)
When running Linkerd check with a control plane namespace that may contain an additional pod with a replication controller ID for pod names instead of a replicaSet ID, the check command panics because of an "index out of bounds" error.

This PR adds a check to make sure that, when parsing pod names during the `checkControllerRunning` healthcheck, we only check for linkerd control plane pods and that the pod name results in four or more substrings when split on '-'. This prevents the check from panicking when encountering a replication controller ID pod name.

Fixes #2084 

Signed-off-by: Dennis Adjei-Baah <dennis@buoyant.io>
2019-01-24 10:26:11 -08:00
Risha Mars 950c952d14
Add a less scary message to the user when retries are still in progress (#2141)
When a failing check is being retried, we show the current err to the user. This
can sometimes be unnecessarily alarming, as in the case of the control plane
starting up.

If a failing check is in the process of being retried, wait to show the final
error message until the retries have completed.
2019-01-24 10:24:59 -08:00
Risha Mars e7556d7edc
Check RoleBindings for specified single namespace only (#2142)
Previously, we were doing the creation checks for both Roles/RoleBindings and
ClusterRoles/ClusterRoleBindings for all namespaces, but in --single-namespace
mode, we only need to check that these can be created in the control plane
namespace.
2019-01-23 18:04:15 -08:00
Andrew Seigner c9ac77cd7c
Introduce version consistency checks (#2130)
Version checks were not validating that the cli version matched the
control plane or data plane versions.

Add checks via the `linkerd check` command to validate the cli is
running the same version as the control and data plane.

Also add types around `channel-version` string parsing and matching. A
consequence being that during development `version.Version` changes from
`undefined` to `dev-undefined`.

Fixes #2076

Depends on linkerd/website#101

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-01-23 16:54:43 -08:00