Commit Graph

554 Commits

Author SHA1 Message Date
Tarun Pothulapati 0cbba0b03e Setting SuccessfulJobHistoryLimit to 0 for CronJobs (#3193)
* setting successful job history limit to 0

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2019-08-07 16:59:14 -05:00
arminbuerkle e3d68da1dc Allow setting custom cluster domain in service profiles (#3148)
Continue of #2950.

I decided to check for the `clusterDomain` in the config map in web server main for the same reasons as as pointed out here https://github.com/linkerd/linkerd2/pull/3113#discussion_r306935817

It decouples the server implementations from the config.

Signed-off-by: Armin Buerkle <armin.buerkle@alfatraining.de>
2019-08-07 09:49:54 -07:00
Andrew Seigner 0565955428
Update `linkerd profile --tap` to Tap APIService (#3187)
PR #3167 introduced a Tap APIService, and migrated linkerd tap to it.

This change migrates `linkerd profile --tap` to the new Tap APIService.

Depends on #3186
Fixes #3169

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-08-02 12:44:58 -07:00
Andrew Seigner a185cae55b
Update `linkerd top` to use Tap APIService (#3186)
PR #3167 introduced a Tap APIService, and migrated `linkerd tap` to it.

This change migrates `linkerd top` to the new Tap APIService. It also
addresses a `panic: close of closed channel` issue, where two go
routines could both call `close(done)` on exit.

Fixes #3168

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-08-02 11:34:22 -07:00
Andrew Seigner a59c1dd32d
Introduce tap APIService, update `linkerd tap` (#3167)
The Tap Service enabled tapping of any meshed pod, regardless of user
privilege.

This change introduces a new Tap APIService. Kubernetes provides
authentication and authorization of Tap requests, and then forwards
requests to a new Tap APIServer, which implements a Kubernetes
aggregated APIServer. The Tap APIServer authenticates the client TLS
from Kubernetes, and authorizes the user via a SubjectAccessReview.

This change also modifies the `linkerd tap` command to make requests
against the new APIService.

The Tap APIService implements these Kubernetes-style endpoints:
POST /apis/tap.linkerd.io/v1alpha1/watch/namespaces/:ns/tap
POST /apis/tap.linkerd.io/v1alpha1/watch/namespaces/:ns/:res/:name/tap
GET  /apis
GET  /apis/tap.linkerd.io
GET  /apis/tap.linkerd.io/v1alpha1
GET  /healthz
GET  /healthz/log
GET  /healthz/ping
GET  /metrics
GET  /openapi/v2
GET  /version

Users authorize to the new `tap.linkerd.io/v1alpha1` via RBAC. Only the
`watch` verb is supported. Access is also available via subresources
such as `deployments/tap` and `pods/tap`.

This change introduces the following resources into the default Linkerd
install:
- Global
  - APIService/v1alpha1.tap.linkerd.io
  - ClusterRoleBinding/linkerd-linkerd-tap-auth-delegator
- `linkerd` namespace:
  - Secret/linkerd-tap-tls
- `kube-system` namespace:
  - RoleBinding/linkerd-linkerd-tap-auth-reader

Tasks not covered by this PR:
- `linkerd top`
- `linkerd dashboard`
- `linkerd profile --tap`
- removal of the unauthenticated tap controller

Fixes #2725, #3162, #3172

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-08-01 14:02:45 -07:00
Andrew Seigner 9a672dd5a9
Introduce `linkerd --as` flag for impersonation (#3173)
Similar to `kubectl --as`, global flag across all linkerd subcommands
which sets a `ImpersonationConfig` in the Kubernetes API config.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-07-31 16:05:33 -07:00
Andrew Seigner a8830b2323
Set heartbeat cronjobs to not restart on failure (#3174)
The heartbeat cronjob specified `restartPolicy: OnFailure`. In cases
where failure was non-transient, such as if a cluster did not have
internet access, this would continuously restart and fail.

Change the heartbeat cronjob to `restartPolicy: Never`, as a failed job
has no user-facing impact.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-07-31 13:51:13 -07:00
Kevin Leimkuhler 8d9cfbf670
Inject Tap service name into proxy PodSpec (#3155)
### Summary

In order for Pods' tap servers to start authorizing tap clients, the tap server
must be able to check client names against the expected tap service name.

This change injects the `LINKERD2_PROXY_TAP_SVC_NAME` into proxy PodSpecs.

### Details

The tap servers on the individual resources being tapped should be able to
verify that the client is the tap service. The `LINKERD2_PROXY_TAP_SVC_NAME` is
now injected as an environment variable in the proxies so that it can check this
value against the client name of the TLS connection. Currently, this environment
will go unused. There is an open PR (linkerd2-proxy#290) to use this variable in
the proxy, but this is *not* dependent on that merging first. 

Note: The variable is not injected if tap is disabled.

### Testing

Test output has been updated with the newly injected environment variable.

Signed-off-by: Kevin Leimkuhler <kleimkuhler@icloud.com>
2019-07-29 15:05:45 -07:00
Tarun Pothulapati 2ba2dea6a6 Added Resource Limits when ha is Configured (#3092)
* increased ha resource limits
* added resource limits to proxy when HA
* update golden files in cmd/main

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2019-07-26 09:46:36 -07:00
Alejandro Pedraza 8c07223f3b
Remove unused argument (#3149)
Removed unused argument in the `GetPatch()` function in
`pkg/inject/inject.go`

Signed-off-by: Alejandro Pedraza <alejandro.pedraza@gmail.com>
2019-07-26 11:39:25 -05:00
Andrew Seigner 18b74aa8a8
Introduce Go modules support (#2481)
The repo relied on `dep` for managing Go dependencies. Go 1.11 shipped
with Go modules support. Go 1.13 will be released in August 2019 with
module support enabled by default, deprecating GOPATH.

This change replaces `dep` with Go modules for dependency management.
All scripts, including Docker builds and ci, should work without any dev
environment changes.

To execute `go` commands directly during development, do one of the
following:
1. clone this repo outside of `GOPATH`; or
2. run `export GO111MODULE=on`

Summary of changes:
- Docker build scripts and ci set `-mod=readonly`, to ensure
  dependencies defined in `go.mod` are exactly what is used for the
  builds.
- Dependency updates to `go.mod` are accomplished by running
 `go build` and `go test` directly.
- `bin/go-run`, `bin/build-cli-bin`, and `bin/test-run` set
  `GO111MODULE=on`, permitting usage inside and outside of GOPATH.
- `gcr.io/linkerd-io/go-deps` tags hashed from `go.mod`.
- `bin/update-codegen.sh` still requires running from GOPATH,
  instructions added to BUILD.md.

Fixes #1488

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-07-25 14:41:38 -07:00
Cody Vandermyn 808fa381f9 A Slightly More Restrictive PSP (#3085)
* Adds more PSP restrictions
* Update test fixtures
* Updates PSP to be conditional on initContainer

- The proxy-init container runs as root and needs the PSP to allow this
user when there is an init container.

Signed-off-by: Cody Vandermyn <cody.vandermyn@nordstrom.com>
2019-07-24 10:12:33 -07:00
Andrew Seigner 889a4a0578
Introduce -A as a shorthand for --all-namespaces (#3125)
kubectl introduced `-A` as a shorthand for `--all-namespaces` in
`v1.14.0`:
https://github.com/kubernetes/kubernetes/pull/72006

Update linkerd cli's `edges`, `get`, and `stat` commands to match this
convention.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-07-24 07:50:22 -07:00
Andrew Seigner 64ed8e4a74
Introduce Cluster Heartbeat cronjob (#3056)
`linkerd check`, the web dashboard, and Grafana all perform version
checks to validate Linkerd is up to date. It's common for users to
seldom execute these codepaths. This makes it difficult to identify what
versions of Linkerd are currently in use and what environments it is
being run in, which helps prioritize testing and backports.

Introduce a `heartbeat` CronJob to the default Linkerd install. The
cronjob executes every 24 hours, starting from 5 minutes after
`linkerd install` is run.

Example check URL:
https://versioncheck.linkerd.io/version.json?
  install-time=1562761177&
  k8s-version=v1.15.0&
  meshed-pods=8&
  rps=3&
  source=heartbeat&
  uuid=cc4bb700-3314-426a-9f0f-ec588b9df020&
  version=git-b97ee9f7

Fixes #2961

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-07-23 17:12:30 -07:00
Andrew Seigner 48a69cb88a
Bump Prometheus to 2.11.1, Grafana to 6.2.5 (#3123)
- set `disable_sanitize_html` in `grafana.ini`.
- make all text box dimensions whole integers to fix dropdown issue,
  reported in:
  https://github.com/linkerd/linkerd2/issues/2955#issuecomment-503085444
- rev all dashboards to `schemaVersion` 18 for Grafana 6.2.5
- `prometheus-benchmark.json` based on:
  https://grafana.com/grafana/dashboards/9761
- `prometheus.json` based on:
  69c93e6401/public/app/plugins/datasource/prometheus/dashboards/prometheus_2_stats.json
- `grafana.json` based on:
  85aed0276e/public/app/plugins/datasource/prometheus/dashboards/grafana_stats.json

Fixes #2955

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-07-23 13:37:56 -07:00
Alex Leong d6ef9ea460
Update ServiceProfile CRD to version v1alpha2 and remove validation (#3078)
The openAPIV3Schema validation in the ServiceProfiles CRD is very limited in what it can validate and is obviated by more sophisticated validation done by the validating admission controller.  Therefore, we would like to remove the openAPIV3Schema validation to reduce the size and complexity of the CRD object.

To do so, we must also bump the version of the ServiceProfile custom resource from v1alpha1 to v1alpha2.  This ensures that when the controller is upgraded, it will attempt to watch the v1alpha2 resource.  If it cannot (because, for example, the controller pod started before the ServiceProfile CRD was updated and therefore the v1alpha2 version does not exist) then it will go into a crash loop backoff until it can.  This essentially means that the controller will wait for the CRD to be upgraded to include v1alpha2 before it will start.  

Bumping the version is necessary because if we did not, it would be possible for the controller to start before the CRD is updated (removing the validation).  In this case, when the CRD is edited, the controller will lose its list watch on ServiceProfiles and will stop getting updates.

Signed-off-by: Alex Leong <alex@buoyant.io>
2019-07-23 11:46:31 -07:00
arminbuerkle 010efac24b Allow custom cluster domain in controller components (#2950)
* Allow custom cluster domain in destination watcher

The change relaxes the constrains of an authority requiring a
`svc.cluster.local` suffix to only require `svc` as third part.

A unit test could be added though the destination/server and endpoint
watcher already test this behaviour.

* Update proto to allow setting custom cluster domain

Update golden templates

* Allow setting custom domain in grpc, web server

* Remove cluster domain flags from web srv and public api

* Set defaultClusterDomain in validateAndBuild if none is set

Signed-off-by: Armin Buerkle <armin.buerkle@alfatraining.de>
2019-07-23 08:59:41 -07:00
Alex Leong c8b34a8cab
Add pod status to linkerd check (#3065)
When waiting for controller pods to be created or become ready, `linkerd check` doesn't offer any hints as to whether there has been an error (such as an ImagePullBackoff).

We add pod status to the output to make this more immediately obvious.

Fixes #2877 

Signed-off-by: Alex Leong <alex@buoyant.io>
2019-07-18 15:56:19 -07:00
Tarun Pothulapati fcec1cfb8a Added Anti Affinity when HA is configured (#2893)
* Added Anti Affinity when HA is configured
* Move check to validate()
* Test output with anti-affinity when ha upgrade
* Add anti-affinity to identity deployment
* made host anti-affinity default when ha
* Define affinity template in a separate file

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2019-07-18 10:03:25 -07:00
Alejandro Pedraza ba9fd70892
`linkerd upgrade config` bombs when installation had a flag (#3097)
When installing using some of the flags that persist in install, e.g
`linkerd install --ha`, and then doing `linkerd upgrade config` a nil
pointer error is thrown.

Fixes #3094

`newCmdUpgradeConfig()` was using passing `flags` as nil because
`linkerd upgrade config` doesn't expose any flags for the subcommand,
but turns out they're still needed down the call stack in
`setFlagsFromInstall` to reuse the flags persisted during install.

I also added a new unit test catching this.

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-07-18 09:09:01 -05:00
Carol A. Scott ee1a111993
Updating CLI output for `linkerd edges` (#3048)
This PR improves the CLI output for `linkerd edges` to reflect the latest API
changes. 

Source and destination namespaces for each edge are now shown by default. The
`MSG` column has been replaced with `Secured` and contains a green checkmark or
the reason for no identity. A new `-o wide` flag shows the identity of client
and server if known.
2019-07-17 12:23:34 -07:00
Jonathan Juares Beber 2dcbde08b3 Show pod status more clearly (#1967) (#2989)
During operations with `linkerd stat` sometimes it's not clear the actual
pod status.

This commit introduces a method, to the `k8s`package, getting the pod status,
based on [`kubectl` logic](33a3e325f7/pkg/printers/internalversion/printers.go (L558-L640))
to expose the `STATUS` column for pods . Also, it changes the stat command
on the` cli` package adding a column when the resource type is a Pod.

Fixes #1967

Signed-off-by: Jonathan Juares Beber <jonathanbeber@gmail.com>
2019-07-10 12:44:44 -07:00
Andrew Seigner 7756828ae6
Update install failure message to list resources (#3050)
The existing `linkerd install` error message for existing resources was
shared with `linkerd check`. Given the different contexts, the messaging
made more sense for `linkerd check` than for `linkerd install`.

Modify the error messaging for `linkerd install` to print a bare list
of existing resources, and provide instructions for proceeding.

For example:
```bash
$ linkerd install
Unable to install the Linkerd control plane. It appears that there is an existing installation:

clusterrole.rbac.authorization.k8s.io/linkerd-linkerd-controller
clusterrole.rbac.authorization.k8s.io/linkerd-linkerd-identity

If you are sure you'd like to have a fresh install, remove these resources with:

    linkerd install --ignore-cluster | kubectl delete -f -

Otherwise, you can use the --ignore-cluster flag to overwrite the existing global resources.
```

Fixes #3045

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-07-09 20:21:19 +02:00
Andrew Seigner 9e09bd5e98
Mark High Availability as non-experimental (#3049)
Fixes #2419

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-07-09 20:20:28 +02:00
Alex Leong 92ddffa3c2
Add prometheus metrics for watchers (#3022)
To give better visibility into the inner workings of the kubernetes watchers in the destination service, we add some prometheus metrics.

Signed-off-by: Alex Leong <alex@buoyant.io>
2019-07-08 11:50:26 -07:00
Andrew Seigner 94fa653cf3
Fix `linkerd check` missing uuid on version check (#3040)
PR #2603 modified the web process to read the UUID from the
`linkerd-config` ConfigMap rather than from a command line flag. The
`linkerd check` command relied on that command line flag to retrieve the
UUID as part of its version check.

Modify `linkerd check` to correctly retrieve the UUID from
`linkerd-config`. Also refactor `linkerd-config` retrieval and parsing
code to be shared between healthcheck, install, and upgrade.

Relates to #2961

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-07-05 19:39:13 +02:00
Tarun Pothulapati eb7f9866af Fix inject with path and add tests (#3038)
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2019-07-05 09:26:25 -05:00
Alejandro Pedraza 53e589890d
Have `linkerd endpoints` use `Destination.Get` (#2990)
* Have `linkerd endpoints` use `Destination.Get`

Fixes #2885

We're refactoring `linkerd endpoints` so it hits
directly the `Destination.Get` endpoint, instead of relying on the
Discovery service.

For that, I've created a new `client.go` for Destination and added it to
the `APIClient` interface.

I've also added a `destinationClient` struct that mimics `tapClient`,
and whose common logic has been moved into `stream_client.go`.

Analogously, I added a `destinationServer` struct that mimics
`tapServer`.

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-07-03 09:11:03 -05:00
Ivan Sim 7e1c14e783
Add the 'linkerd.io/control-plane-ns' label to the Traffic Split CRD (#3026)
Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-07-02 15:46:25 -07:00
Andrew Seigner 902978fe48
Rename debug annotation to enable-debug-sidecar (#3016)
Linkerd's CLI flags all match 1:1 with their `config.linkerd.io/*`
annotation counterparts, except `--enable-debug-sidecar`, which
corresponded to `config.linkerd.io/debug`. Additionally, the Linkerd
docs assume this 1:1 mapping.

Rename the `config.linkerd.io/debug` annotation to
`config.linkerd.io/enable-debug-sidecar`.

Relates to https://github.com/linkerd/website/issues/381

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-07-02 20:01:52 +02:00
Carol A. Scott a504e8c2d8
Expand and improve edges API endpoint (#3007)
Updates functionality of `linkerd edges`, including a new `--all-namespaces`
flag and returning namespace information for SRC and DST resources.
2019-06-28 15:46:04 -07:00
Alex Leong 27373a8b78
Add traffic splitting to destination profiles (#2931)
This change implements the DstOverrides feature of the destination profile API (aka traffic splitting).

We add a TrafficSplitWatcher to the destination service which watches for TrafficSplit resources and notifies subscribers about TrafficSplits for services that they are subscribed to.  A new TrafficSplitAdaptor then merges the TrafficSplit logic into the DstOverrides field of the destination profile.

Signed-off-by: Alex Leong <alex@buoyant.io>
2019-06-28 13:19:47 -07:00
Tarun Pothulapati 7db058f096 linkerd inject from remote URL (#2988)
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2019-06-28 09:47:33 -07:00
Tarun Pothulapati 5c5ec6d816 add admin port label to proxy-injector and sp-validator (#2984)
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2019-06-27 17:25:49 -05:00
Ivan Sim 866fe6fa5e
Introduce global resources checks to install and multi-stage install (#2987)
* Introduce new checks to determine existence of global resources and the
'linkerd-config' config map.
* Update pre-check to check for existence of global resources

This ensures that multiple control planes can't be installed into
different namespaces.

* Update integration test clean-up script to delete psp and crd

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-06-27 09:59:12 -07:00
Alejandro Pedraza 73740fb503
Simplify port-forwarding code (#2976)
* Simplify port-forwarding code

Simplifies the establishment of a port-forwarding by moving the common
logic into `PortForward.Init()`

Stemmed from this
[comment](https://github.com/linkerd/linkerd2/pull/2937#discussion_r295078800)

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-06-26 11:14:57 -05:00
Andrew Seigner 81790b6735 Bump Prometheus to v2.10.0 (#2979)
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-06-21 12:51:31 -07:00
Tarun Pothulapati a3ce06bd80 Add sideEffects field to Webhooks (#2963)
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2019-06-21 11:06:10 -07:00
Ivan Sim 435fe861d0
Label all Linkerd resources (#2971)
Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-06-20 09:44:30 -07:00
Ivan Sim e2e976cce9
Add `NET_RAW` capability to the proxy-init container (#2969)
Also, update control plane PSP to match linkerd/website#94

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-06-19 19:34:37 -07:00
Dennis Adjei-Baah 694ba9c2cb
Revert add namespace name to MWC (#2946)
* revert add namespace name to MWC
2019-06-14 15:26:34 -07:00
Alejandro Pedraza 7fc6c195ad
Set MWC and VWC failure policy to 'fail' in HA mode only (#2943)
Fixes #2927

Also moved `TestInstallSP` after `TestCheckPostInstall` so we're sure
the validating webhook is ready before installing a service profile.

Signed-off-by: Alejandro Pedraza Borrero <alejandro@buoyant.io>
2019-06-14 11:50:59 -05:00
Alejandro Pedraza 28025eeb56
Remove UPDATE event from the mutating webhook config (#2919)
Fixes #2889

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-06-13 15:42:47 -05:00
Alejandro Pedraza e9bf014d34
Remove MWVC RBAC from webhook configs (#2925)
Fixes #2890

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-06-13 15:42:00 -05:00
Dennis Adjei-Baah 035ba6ae87
update sp-validator MWC golden test file (#2938) 2019-06-13 13:39:24 -07:00
Dennis Adjei-Baah 8aef9280dd
add namespace name to MWC (#2905)
When installing multiple control planes, the mutatingwebhookconfiguration of the first control plane gets overwritten by any subsequent control plane install. This is caused by the fixed name given to the mutatingwebhookconfiguration manifest at install time.

This commit adds in the namespace to the manifest so that there is a unique configuration for each control plane.

Fixes #2887
2019-06-13 12:15:43 -07:00
Ivan Sim ecc4465cd1
Introduce Control Plane's PSP and RBAC resources into Helm templates (#2920)
* Add control plane and CNI PSP and RBAC resources
* Add the '--linkerd-cni-enabled' flag to the multi-stage install subcommands

This flag ensures that the NET_ADMIN capability is omitted from the control
plane's PSP during 'install config' and the proxy-init containers aren't
injected during 'install control-plane'.

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-06-12 20:18:46 -07:00
Alejandro Pedraza 8416d326c2
If HA, set the webhooks failure policy to 'Fail' (#2906)
* If HA, set the webhooks failure policy to 'Fail'

I'm adding to the linkerd namespace a new label
`linkerd.io/is-control-plane: true` that is used in the webhook configs'
selector to skip the proxy injector for this namespace. This avoids
running into the timing issues described in #2852.

Fixes #2852

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-06-11 13:11:54 -05:00
Cody Vandermyn 33de3574ee Correctly set securityContext values on injection (#2911)
The patch provided by @ihcsim applies correct values for the securityContext during injection, namely: `allowPrivilegeEscalation = false`, `readOnlyRootFilesystem = true`, and the capabilities are copied from the primary container. Additionally, the proxy-init container securityContext has been updated with appropriate values.

Signed-off-by: Cody Vandermyn <cody.vandermyn@nordstrom.com>
2019-06-11 10:34:30 -07:00
Alex Leong c698d6bca1
Add support for TrafficSplits (#2897)
Add support for querying TrafficSplit resources through the common API layer. This is done by depending on the TrafficSplit client bindings from smi-sdk-go.

Signed-off-by: Alex Leong <alex@buoyant.io>
2019-06-11 10:04:42 -07:00
Dan 24bbd7c64b Ensure Prometheus log level is lowercase (#2823) (#2870)
Signed-off-by: Daniel Baranowski <daniel.baranowski@infinityworks.com>
2019-06-07 09:57:08 -07:00
Alejandro Pedraza 66eb829e5a
Fix HA during upgrade (#2900)
* Fix HA during upgrade

If we have a Linkerd installation with HA, and then we do `linkerd
upgrade` without specifying `--ha`, the replicas will get set back to 1,
yet the resource requests will keep their HA values.

Desired behavior: `linkerd install --ha` adds the `ha` value into the
linkerd-config, so it should be used during upgrade even if `--ha` is
not passed to `linkerd upgrade`.
Note we still can do `linkerd upgrade --ha=false` to disable HA.

This is a prerequesite to address #2852

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-06-06 17:27:27 -05:00
Alex Leong 06a69f69c5
Refactor destination service (#2786)
This is a major refactor of the destination service.  The goals of this refactor are to simplify the code for improved maintainability.  In particular:

* Remove the "resolver" interfaces.  These were a holdover from when our decision tree was more complex about how to handle different kinds of authorities.  The current implementation only accepts fully qualified kubernetes service names and thus this was an unnecessary level of indirection.
* Moved the endpoints and profile watchers into their own package for a more clear separation of concerns.  These watchers deal only in Kubernetes primitives and are agnostic to how they are used.  This allows a cleaner layering when we use them from our gRPC service.
* Renamed the "listener" types to "translator" to make it more clear that the function of these structs is to translate kubernetes updates from the watcher to gRPC messages.

Signed-off-by: Alex Leong <alex@buoyant.io>
2019-06-04 15:01:16 -07:00
Alejandro Pedraza 74ca92ea25
Split proxy-init into separate repo (#2824)
Split proxy-init into separate repo

Fixes #2563

The new repo is https://github.com/linkerd/linkerd2-proxy-init, and I
tagged the latest there `v1.0.0`.

Here, I've removed the `/proxy-init` dir and pinned the injected
proxy-init version to `v1.0.0` in the injector code and tests.

`/cni-plugin` depends on proxy-init, so I updated the import paths
there, and could verify CNI is still working (there is some flakiness
but unrelated to this PR).

For consistency, I added a `--init-image-version` flag to `linkerd
inject` along with its corresponding override config annotation.

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-06-03 16:24:05 -05:00
Tarun Pothulapati 590249c66b HA for proxy-injector and sp-validator (#2874)
* Added labels to webhook configurations in charts/
* Multiple replicas of proxy-injector and sp-validator in HA
* Use ControllerComponent template variable for webhookconfigurations

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2019-05-31 14:48:30 -07:00
Carol A. Scott 031cd9d4ba
Simplify logic for linkerd edges and output JSON in consistent order (#2858)
When `linkerd edges` returns JSON, the data will now be sorted alphabetically by
SRC name, meaning edges will be returned in a consistent order. Logic in the CLI
`edges.go` has also been simplified. These changes should result in the Travis
CI builds passing consistently.
2019-05-29 11:31:58 -07:00
Ivan Sim 86d822f8ea
Generate the debug container spec in the shared library (#2854)
This commit refactors the changes introduced by #2842 where the debug
container spec is created in the 'cli' and 'pkg' packages. This change
follows the existing pattern of annotating the YAML in the CLI code,
and injecting the sidecar spec in the shared library.

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-05-28 15:26:37 -07:00
Tarun Pothulapati 1a574def1f Added labels to webhook configurations in charts/ (#2853)
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2019-05-28 16:44:09 -05:00
Carol A. Scott 8c496e3d0d
Adding unit test for CLI edges command (#2837)
Adds a unit test for the `linkerd edges` command.
2019-05-28 13:51:45 -07:00
Dennis Adjei-Baah b9cc66c6c8
fixes roleRef name in linkerd-tap rbac (#2845)
* fixes roleRef name in linkerd-tap rbac
2019-05-28 10:05:01 -07:00
Ivan Sim 0d489d3099
Add debug container annotation (#2842)
This new annotation is used by the proxy injector to determine if the 
debug container needs to be injected.

When using 'linkerd install', the 'pkg/inject' library will only inject
annotations into the workload YAML. Even though 'conf.debugSidecar' 
is set in the CLI, the 'injectPodSpec()' function is never invoked on 
the proxy injector side. Once the workload YAML got picked up by the 
proxy injector, 'conf.debugSidecar' is already nil, since it's a different, 
new 'conf' object. The new annotation ensures that the proxy injector
injects the debug container.

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-05-24 10:11:06 -07:00
Ivan Sim 5a5f8bbfe8
Install MWC and VWC During Installation (#2806)
* Update helm charts to include webhooks config and TLS secret
* Update the webhooks to read the secret cert and key
* Update webhooks to not recreate config on restart
* Ensure upgrade preserve existing secrets
* Revert the change to rename the webhook configs

The renaming change breaks upgrade, where the new webhook configs conflict with
the existing ones. The older resources  aren't deleted during upgrade because
they are dynamically created.

* Make the secret volume read-only
* Remove unnecessary exported getter functions
* Remove obsolete mwc and vwc templates

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-05-20 12:43:50 -07:00
Alena Varkockova 7973715ee4 Output check result as json (#2666)
* Add missing file, make linter happy
* Changes from code review
* Ivan's code review
* PR review feedback
* style fixes
* Last PR comments

Signed-off-by: Alena Varkockova <varkockova.a@gmail.com>
2019-05-20 09:04:28 -07:00
Alejandro Pedraza 253068e844
Unhide CLI inject's --disable-tap now that #2811 has been addressed (#2827)
Signed-off-by: Alejandro Pedraza <alejandro.pedraza@gmail.com>
2019-05-16 11:12:20 -05:00
Dennis Adjei-Baah a0fa1dff59
Move tap service into its own pod. (#2773)
* Split tap into its own pod in the control plane

Signed-off-by: Dennis Adjei-Baah <dennis@buoyant.io>
2019-05-15 16:28:44 -05:00
Carol A. Scott 042086142a
Adding an edges command to the CLI (#2808)
Adds an edges command to the CLI. `linkerd edges` displays connections between resources, and Linkerd proxy identities. Currently this feature will only display edges where both the client identity and server identity are known. The next step will be to display edges for which identity is not known and/or one-sided traffic such as Prometheus and tap requests.
2019-05-15 13:59:27 -07:00
Alejandro Pedraza 065c221858
Support for resources opting-out of tap (#2807)
Support for resources opting out of tap

Implements the `linkerd inject --disable-tap` flag (although hidden pending #2811) and the config override annotation `config.linkerd.io/disable-tap`.
Fixes #2778

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-05-10 14:17:23 -05:00
Jack Price f758a9e428 Use port-forwarding for linkerd CLIs (#2757)
Private k8s clusters, such as the private GKE clusters offered by Google
Cloud, cannot be reached through the current API proxy method.

This commit uses the port forwarding feature already developed.

Also modify dashboard command to not fall back to ephemeral port.

Signed-off-by: Jack Price <jackprice@outlook.com>
2019-05-02 14:41:26 +02:00
Andrew Seigner 266e882d79
Define multi-stage commands as subcommands (#2772)
The multi-stage args used by install, upgrade, and check were
implemented as positional arguments to their respective parent commands.
This made the help documentation unclear, and the code ambiguous as to
which flags corresponded to which stage.

Define `config` and `control-plane` stages as subcommands. The help
menus now explicitly state flags supported.

Fixes #2729

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-05-02 12:32:01 +02:00
Andrew Seigner 66494591e0
Multi-stage check support (#2765)
Add support for `linkerd check config`. Validates the existence of the
Linkerd Namespace, ClusterRoles, ClusterRoleBindings, ServiceAccounts,
and CustomResourceDefitions.

Part of #2337

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-04-30 17:17:59 +01:00
Ivan Sim 714035fee9
Define default resource spec for proxy-init init container (#2763)
Fixes #2750 

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-04-29 11:41:05 -07:00
Andrew Seigner 0cfc8c6f1c
Introduce k8s apiextensions support (#2759)
CustomResourceDefinition parsing and retrieval is not available via
client-go's `kubernetes.Interface`, but rather via a separate
`k8s.io/apiextensions-apiserver` package.

Introduce support for CustomResourceDefintion object parsing and
retrieval. This change facilitates retrieval of CRDs from the k8s API
server, and also provides CRD resources as mock objects.

Also introduce a `NewFakeAPI` constructor, deprecating
`NewFakeClientSets`. Callers need no longer be concerned with discreet
clientsets (for k8s resources vs. CRDs vs. (eventually)
ServiceProfiles), and can instead use the unified `KubernetesAPI`.

Part of #2337, in service to multi-stage check.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-04-28 18:55:22 -07:00
Andrew Seigner be60b37e93
Group Web and Grafana ServiceAccounts with RBAC (#2756)
All ServiceAccounts are intended to be grouped together with other RBAC
resources, particularly for `linkerd install config` output. Grafana and
Web ServiceAccounts were still included with their respective
Deployments.

Group Grafana and Web ServiceAccounts with other RBAC resources.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-04-25 17:33:05 -07:00
Andrew Seigner 15ffd86cf1
Introduce multi-stage upgrade (#2723)
`linkerd install` supports a 2-stage install process, `linkerd upgrade`
did not.

Add 2-stage support for `linkerd upgrade`. Also exercise multi-stage
functionality during upgrade integration tests.

Part of #2337

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-04-25 14:29:52 -07:00
Alex Leong 4ea7c62b0d
Revert " Remove validation from service profile CRD definition (#2740)" (#2752)
This reverts commit 3de16d47be.

#2740 modified the ServiceProfiles CRD which will cause issues for users upgrading from the old CRD version to the new version.  #2748 was an attempt to fix this by bumping the service profile CRD version, however, our testing infrastructure is not well set up to accommodate changes to CRDs because they are resources which are global to the cluster.  

We revert this change for now and will revisit it in the future when we can give more thought to CRD versioning, upgrade, and testing.

Signed-off-by: Alex Leong <alex@buoyant.io>
2019-04-25 13:40:20 -07:00
Andrew Seigner ec540a882e
Consolidate k8s APIs (#2747)
Numerous codepaths have emerged that create k8s configs, k8s clients,
and make k8s api requests.

This branch consolidates k8s client creation and APIs. The primary
change migrates most codepaths to call `k8s.NewAPI` to instantiate a
`KubernetesAPI` struct from `pkg`. `KubernetesAPI` implements the
`kubernetes.Interface` (clientset) interface, and also persists a
`client-go` `rest.Config`.

Specific list of changes:
- removes manual GET requests from `k8s.KubernetesAPI`, in favor of
  clientsets
- replaces most calls to `k8s.GetConfig`+`kubernetes.NewForConfig` with
  a single `k8s.NewAPI`
- introduces a `timeout` param to `k8s.NewAPI`, currently only used by
  healthchecks
- removes `NewClientSet` in `controller/k8s/clientset.go` in favor of
  `k8s.NewAPI`
- removes `httpClient` and `clientset` from `HealthChecker`, use
  `KubernetesAPI` instead

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-04-25 11:31:38 -07:00
Ivan Sim cd37d3f0f5
Fall back to default built-in version if versions config are missing (#2745)
Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-04-24 19:49:18 -07:00
Alejandro Pedraza 53bb7c47f6
Make the auto-injector required and removed proxy-auto-inject flag (#2733)
Make the auto-injector required and removed proxy-auto-inject flag

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-04-24 13:06:51 -05:00
Alejandro Pedraza 62d9a80894
New `linkerd inject` default and manual modes (#2721)
Fixes #2720 and 2711 

This changes the default behavior of `linkerd inject` to not inject the
proxy but just the `linkerd.io/inject: enabled` annotation for the
auto-injector to pick it up (regardless of any namespace annotation).

A new `--manual` mode was added, which behaves as before, injecting
the proxy in the command output.

The unit tests are running with `--manual` to avoid any changes in the
fixtures.

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-04-24 09:05:27 -05:00
Alex Leong 3de16d47be
Remove validation from service profile CRD definition (#2740)
Fixes #2736

Signed-off-by: Alex Leong <alex@buoyant.io>
2019-04-23 16:10:50 -07:00
Andrew Seigner b2b4780430
Introduce install stages (#2719)
This change introduces two named parameters for `linkerd install`, split
by privilege:
- `linkerd install config`
  - Namespace
  - ClusterRoles
  - ClusterRoleBindings
  - CustomResourceDefinition
  - ServiceAccounts
- `linkerd install control-plane`
  - ConfigMaps
  - Secrets
  - Deployments
  - Services

Comprehensive `linkerd install` is still supported.

TODO:
- `linkerd check` support
- `linkerd upgrade` support
- integration tests

Part of #2337

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-04-23 14:52:34 -07:00
Dennis Adjei-Baah 3e5917f7e0
Add the ability to inject a debug sidecar (#2726)
Signed-off-by: Dennis Adjei-Baah <dennis@buoyant.io>
2019-04-22 16:53:12 -07:00
Alejandro Pedraza c56766a923
Add config.linkerd.io/disable-identity annotation (#2717)
Add config.linkerd.io/disable-identity annotation

First part of #2540

We'll tackle support for `--disable-identity` in `linkerd install` in a
separate commit.

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-04-19 19:04:49 -04:00
Ivan Sim 8d13084f94
Split the `linkerd-version` CLI flag into `control-plane-version` and `proxy-version` (#2702)
* The 'linkerd-version' CLI flag is renamed to 'control-plane-version'
* Add version field to proxy config
* Add the control plane version to the global config
* Unit test for init image version
* Use more specific control plane and proxy versions in unit tests

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-04-19 11:35:20 -07:00
Dennis Adjei-Baah be614656bb
add service profile integration tests for service profile metrics (#2685)
* add integration tests for retryable requests

Signed-off-by: Dennis Adjei-Baah <dennis@buoyant.io>
2019-04-18 11:01:49 -07:00
Andrew Seigner 72287ae121
Don't use spinner in cli when run without a tty (#2716)
In some non-tty environments, the `linkerd check` spinner can render
unexpected control characters.

Disable the spinner when run without a tty.

Fixes #2700

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-04-18 09:31:56 -07:00
Andrew Seigner 2d9e3686e2
Split out config objects from install templates (#2714)
This is an initial change to separate out config-specific k8s objects
from the control-plane components. The eventual goal will be rendering
these configs as the first stage of a multi-stage install.

Part of #2337

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-04-18 09:31:35 -07:00
Andrew Seigner 8323e104fb
Introduce upgrade --from-manifests flag (#2697)
The `linkerd upgrade` command read the control-plane's config from
Kubernetes, which required the environment to be configured to connect
to the appropriate k8s cluster.

Intrdouce a `linkerd upgrade --from-manifests` flag, allowing the user
to feed the output of `linkerd install` into the upgrade command.

Fixes #2629

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-04-17 13:32:21 -07:00
Douglas Jordan 80634d6c8b Create proxy-injector RBAC resources before deployment (#2707)
Fixes #2694 

Signed-off-by: Douglas Jordan <dwj300@gmail.com>
2019-04-17 10:51:00 -07:00
Ivan Sim 4e19827457
Allow identity to be disabled during inject on existing cluster (#2686)
Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-04-11 13:37:06 -07:00
Andrew Seigner 43cb3f841b
upgrade: unit tests (#2672)
This change introduces some unit tests on individual methods in the
upgrade code path, along with some minor cleanup.

Part of #2637

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-04-10 14:54:13 -07:00
Oliver Gould bbe1a60358
upgrade: Generate an Identity config if missing (#2656)
When upgrading from an older cluster that has a Linkerd config but no
identity, we need to generate an identity context so that the cluster is
configured properly.

Fixes #2650
2019-04-08 16:49:12 -07:00
Katerina 938d64a16f Web server updated to read the UUID from the linkerd-config ConfigMap. (#2603)
Signed-off-by: Kateryna Melnyk <kattymelnyk@gmail.com>
2019-04-08 12:56:00 -07:00
Oliver Gould ba65bd8039
Switch UUID implementation (#2667)
The UUID implementation we use to generate install IDs is technically
not random enough for secure uses, which ours is not. To prevent
security scanners like SNYK from flagging this false-positive, let's
just switch to the other UUID implementation (Already in our
dependencies).
2019-04-08 10:58:02 -07:00
Oliver Gould d3b0d39f3b
upgrade: Fix the linkerd version in linkerd-config (#2662)
92f15e78a9 incorrectly removed the config
version override when patching a config from options, which caused
upgrade to stop updating the config version.

Fixes #2660
2019-04-08 10:57:02 -07:00
Oliver Gould 4fd1de4340
install: Don't reuse flag set (#2649)
The instalOnlyFlagSet incorrectly extends the recordableFlagSet.

I'm not sure if this has any potential for unexpected user interactions,
but it's at least confusing when reading the code.

This change makes the flag sets distinct.
2019-04-05 14:29:52 -07:00
Alejandro Pedraza edb225069c
Add validation webhook for service profiles (#2623)
Add validation webhook for service profiles

Fixes #2075

Todo in a follow-up PRs: remove the SP check from the CLI check.

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-04-05 16:10:47 -05:00
Oliver Gould 4c5378f586
install: Change --ha to set a 100m CPU request (#2644)
When the --ha flag is set, we currently set a 10m CPU request, which
corresponds to 1% of a core, which isn't actually enough to keep the
proxy responding to health checks if you have 100 processes on the box.
Let's give ourselves a little more breathing room.

Fixes #2643
2019-04-05 13:41:00 -07:00
Andrew Seigner 1c938b3f52
Introduce upgrade command unit tests (#2639)
This change introduces a basic unit test for the `linkerd upgrade`
command. Given a mock k8s client with linkerd-config and
linkerd-identity-issuer objects, it validates the rendered yaml output
against an expected file.

To enable this testing, most of the logic in the top-level upgrade
command has been moved down into a `validateAndBuild` method.

TODO:
- test individual functions around mutating options, flags, configs, and
  values
- enable reading the install information from a manifest rather than k8s

Part of #2637

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-04-05 11:55:20 -07:00
Andrew Seigner 2f80add17a
Introduce inject integration tests (#2616)
This change introduces integration tests for `linkerd inject`. The tests
perform CLI injection, with and without params, and validates the
output, including annotations.

Also add some known errors in logs to `install_test.go`.

TODO:
- deploy uninjected and injected resources to a default and
  auto-injected cluster
- test creation and update

Part of #2459

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-04-05 11:42:49 -07:00