Commit Graph

513 Commits

Author SHA1 Message Date
Ivan Sim 954a45f751
Fix broken unit and integration tests (#3303)
Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-08-21 18:52:19 -07:00
arminbuerkle e7d303e03f Add LINKERD2_PROXY_DESTINATION_GET_SUFFIXES (#3277)
* Fix missing `clusterDomain` in render RenderTapOutputProfile
* Add LINKERD2_PROXY_DESTINATION_GET_SUFFIXES env variable

Signed-off-by: Armin Buerkle <armin.buerkle@alfatraining.de>
2019-08-21 14:28:30 -07:00
Ivan Sim 183e42e4cd
Merge the CLI 'installValues' type with Helm 'Values' type (#3291)
* Rename template-values.go
* Define new constructor of charts.Values type
* Move all Helm values related code to the pkg/charts package
* Bump dependency
* Use '/' in filepath to remain compatible with VFS requirement
* Add unit test to verify Helm YAML output
* Alejandro's feedback
* Add unit test for Helm YAML validation (HA)

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-08-20 19:26:38 -07:00
Alejandro Pedraza 99ddc66461
Always use forward-slash when interacting with the VFS (#3284)
* Always use forward-slash when interacting with the VFS

Fixes #3283

Our VFS implementation relies on `net.http.FileSystem` which always
expects `/` regardless of the OS.

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-08-19 11:10:21 -05:00
cpretzer 4e92064f3b
Add a flag to install-cni command to configure iptables wait flag (#3066)
Signed-off-by: Charles Pretzer <charles@buoyant.io>
2019-08-15 12:58:18 -07:00
Kevin Leimkuhler cc3c53fa73
Remove tap from public API and associated test infrastructure (#3240)
### Summary

After the addition of the tap APIServer, all the logic related to tap in the public API no longer needs to be there. The servers and clients that are created but not used, as well as all the old testing infrastrucure related to tap can be removed.

This deprecates TapByResource and therefore required an update to the protobuf files with `bin/protoc-go.sh`. While the change to deprecate this method was extremely small, a lot of protobuf fils were updated in the process. These changes to the code and protobuf files should probably remain coupled since `TapByResource` is officially deprecated in the public API, but a majority of the additions/deletions are related to those files.

This draft passes `go test` as well as a local run of the integration tests.

Signed-off-by: Kevin Leimkuhler <kleimkuhler@icloud.com>
2019-08-14 17:27:37 -04:00
Andrew Seigner 3b55e2e87d
Add container cpu and mem to heartbeat requests (#3238)
PR #3217 re-introduced container metrics collection to
linkerd-prometheus. This enabled linkerd-heartbeat to collect mem and
cpu metrics at the container-level.

Add container cpu and mem metrics to heartbeat requests. For each of
(destination, prometheus, linkerd-proxy), collect maximum memory and p95
cpu.

Concretely, this introduces 7 new query params to heartbeat requests:
- p99-handle-us
- max-mem-linkerd-proxy
- max-mem-destination
- max-mem-prometheus
- p95-cpu-linkerd-proxy
- p95-cpu-destination
- p95-cpu-prometheus

Part of #2961

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-08-14 12:04:08 -07:00
Carol A. Scott 00437709eb
Add trafficsplit metrics to CLI (#3176)
This PR adds `trafficsplit` as a supported resource for the `linkerd stat` command. Users can type `linkerd stat ts` to see the apex and leaf services of their trafficsplits, as well as metrics for those leaf services.
2019-08-14 10:30:57 -07:00
Ivan Sim 4d01e3720e
Update install and upgrade code to use the new helm charts (#3229)
* Delete symlink to old Helm chart
* Update 'install' code to use common Helm template structs
* Remove obsolete TLS assets functions.

These are now handle by Helm functions inside the templates

* Read defaults from values.yaml and values-ha.yaml
* Ensure that webhooks TLS assets are retained during upgrade
* Fix a few bugs in the Helm templates (see bullet points):
* Merge the way the 'install' ha and non-ha options are handled into one function
* Honor the 'NoInitContainer' option in the components templates
* Control plane mTLS will not be disabled if identity context in the
config map is empty. The data plane mTLS will still be automatically disabled
if the context is nil.
* Resolve test failures from rebase with master
* Fix linter issues
* Set service account mount path read-only field
* Add TLS variables of the webhooks and tap to values.yaml

During upgrade, these secrets are preserved to ensure they remain synced
wih the CA bundle in the webhook configurations. These Helm variables are used
to override the defaults in the templates.

* Remove obsolete 'chart' folder
* Fix bugs in templates
* Handle missing webhooks and tap TLS assets during upgrade

When upgrading from an older version that don't have these secrets, fallback to let Helm
create them by creating an empty charts.TLS struct.

* Revert the selector labels of webhooks to be compatible with that in 2.4

In 2.4, the proxy injector and profile validator webhooks already have their selector labels defined.
Since these attributes are immutable, the recent change to these selectors introduced by the Helm chart work will cause upgrade to fail.

* Alejandro's feedback
* Siggy's feedback
* Removed redundant unexported custom types

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-08-13 14:16:24 -07:00
Alejandro Pedraza 1e82f62d6e
Fix uninject (#3236)
Now that we inject at the pod level by default, `linkerd uninject` should remove the `linkerd.io/inject: enabled`
annotation. Also added a test for that.

Fix #3156

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-08-13 15:06:21 -05:00
ethan b4b2a44299 cleanup: stat.go help message words correction (#3226)
Signed-off-by: Guangming Wang <guangming.wang@daocloud.io>
2019-08-12 10:05:26 -07:00
Thomas Rampelberg ca5b4fab2e
Add container metrics and grafana dashboard (#3217)
* Add container metrics and grafana dashboard

* Review cleanup

* Update templates
2019-08-12 08:03:57 -07:00
Andrew Seigner 43bc175ea9
Enable tap-admin ClusterRole privileges for `*` (#3214)
The `linkerd-linkerd-tap-admin` ClusterRole had `watch` privileges on
`*/tap` resources. This disallowed non-namespaced tap requests of the
form: `/apis/tap.linkerd.io/v1alpha1/watch/namespaces/linkerd/tap`,
because that URL structure is interpreted by the Kubernetes API as
watching a resource of type `tap` within the linkerd namespace, rather
than tapping the linkerd namespace.

Modify `linkerd-linkerd-tap-admin` to have `watch` privileges on `*`,
enabling any request of the form
`/apis/tap.linkerd.io/v1alpha1/watch/namespaces/linkerd/*` to succeed.

Fixes #3212

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-08-08 12:04:03 -07:00
Andrew Seigner 0ff39ddf8d
Introduce tap-admin ClusterRole, web privs flag (#3203)
The web dashboard will be migrating to the new Tap APIService, which
requires RBAC privileges to access.

Introduce a new ClusterRole, `linkerd-linkerd-tap-admin`, which gives
cluster-wide tap privileges. Also introduce a new ClusterRoleBinding,
`linkerd-linkerd-web-admin` which binds the `linkerd-web` service
account to the new tap ClusterRole. This ClusterRoleBinding is enabled
by default, but may be disabled via a new `linkerd install` flag
`--restrict-dashboard-privileges`.

Fixes #3177

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-08-08 10:28:35 -07:00
Alejandro Pedraza 54b2103bba
Fix bug in service profile name generation (#3209)
Followup to #3148

Wrong args order in call to `profiles.RenderOpenAPI` was generating an
invalid service profile name.

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-08-07 18:51:32 -05:00
Alejandro Pedraza 3ae653ae92
Refactor proxy injection to use Helm charts (#3200)
* Refactor proxy injection to use Helm charts

Fixes #3128

A new chart `/charts/patch` was created, that generates the JSON patch
payload that is to be returned to the k8s API when doing the injection
through the proxy injector, and it's also leveraged by the `linkerd
inject --manual` CLI.

The VFS was used by `linkerd install` to access the old chart under
`/chart`. Now the proxy injection also uses the Helm charts to generate
the JSON patch (see above) so we've moved the VFS from `cli/static` to a
new common place under `/pkg/charts/static`, and the new root for the VFS is
now `/charts`.

`linkerd install` hasn't yet migrated to use the new charts (that'll
happen in #3127), so the only change in that regard was the creation of
`/charts/chart` which is a symlink pointing to `/chart` that
`install.go` now uses, so that the VFS contains both the old and new
charts, as a temporary measure.

You can see that `/bin/Dockerfile-bin`, `/controller/Dockerfile` and
`/bin/build-cli-bin` do now `go generate` pointing to the new location
(and the `go generate` annotation was moved from `/cli/main.go` to
`pkg/charts/static/templates.go`).

The symlink trick doesn't work when building the binaries through
Docker, so `/bin/Dockerfile-bin` replaces the symlink with an actual
copy of `/chart`.

Also note that in `/controller/Dockerfile` we now need to include the
`prod` tag in `go install` like we do in `/bin/Dockerfile-bin` so that
the proxy injector does use the VFS instead of the local file system.

- The common logic to parse a chart has been moved from `install.go` to
`/pkg/charts/util.go`.
- The special ENV var in the proxy for "outbound router capacity" that
only applies to the Prometheus pod is now handled directly in the proxy
partial and all the associated go code could be removed.
- The `patch.go` lib for generating the JSON patch in go along
with its tests `patch_test.go` are no longer needed.
- Lots of functions in `/pkg/inject/inject.go` got removed/simplified
with their logic being moved into the charts themselves. As a
consequence lots of things in `inject_test.go` became irrelevant.
- Moved `template-values.go` from `/pkg/inject` to `pkg/charts` as that
contains the go structs representation of the chart variables that
will be leveraged in #3127.

Don't forget to run `/bin/helm.sh` whenever you make changes to charts
;-)

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-08-07 17:32:37 -05:00
Tarun Pothulapati 0cbba0b03e Setting SuccessfulJobHistoryLimit to 0 for CronJobs (#3193)
* setting successful job history limit to 0

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2019-08-07 16:59:14 -05:00
arminbuerkle e3d68da1dc Allow setting custom cluster domain in service profiles (#3148)
Continue of #2950.

I decided to check for the `clusterDomain` in the config map in web server main for the same reasons as as pointed out here https://github.com/linkerd/linkerd2/pull/3113#discussion_r306935817

It decouples the server implementations from the config.

Signed-off-by: Armin Buerkle <armin.buerkle@alfatraining.de>
2019-08-07 09:49:54 -07:00
Andrew Seigner 0565955428
Update `linkerd profile --tap` to Tap APIService (#3187)
PR #3167 introduced a Tap APIService, and migrated linkerd tap to it.

This change migrates `linkerd profile --tap` to the new Tap APIService.

Depends on #3186
Fixes #3169

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-08-02 12:44:58 -07:00
Andrew Seigner a185cae55b
Update `linkerd top` to use Tap APIService (#3186)
PR #3167 introduced a Tap APIService, and migrated `linkerd tap` to it.

This change migrates `linkerd top` to the new Tap APIService. It also
addresses a `panic: close of closed channel` issue, where two go
routines could both call `close(done)` on exit.

Fixes #3168

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-08-02 11:34:22 -07:00
Andrew Seigner a59c1dd32d
Introduce tap APIService, update `linkerd tap` (#3167)
The Tap Service enabled tapping of any meshed pod, regardless of user
privilege.

This change introduces a new Tap APIService. Kubernetes provides
authentication and authorization of Tap requests, and then forwards
requests to a new Tap APIServer, which implements a Kubernetes
aggregated APIServer. The Tap APIServer authenticates the client TLS
from Kubernetes, and authorizes the user via a SubjectAccessReview.

This change also modifies the `linkerd tap` command to make requests
against the new APIService.

The Tap APIService implements these Kubernetes-style endpoints:
POST /apis/tap.linkerd.io/v1alpha1/watch/namespaces/:ns/tap
POST /apis/tap.linkerd.io/v1alpha1/watch/namespaces/:ns/:res/:name/tap
GET  /apis
GET  /apis/tap.linkerd.io
GET  /apis/tap.linkerd.io/v1alpha1
GET  /healthz
GET  /healthz/log
GET  /healthz/ping
GET  /metrics
GET  /openapi/v2
GET  /version

Users authorize to the new `tap.linkerd.io/v1alpha1` via RBAC. Only the
`watch` verb is supported. Access is also available via subresources
such as `deployments/tap` and `pods/tap`.

This change introduces the following resources into the default Linkerd
install:
- Global
  - APIService/v1alpha1.tap.linkerd.io
  - ClusterRoleBinding/linkerd-linkerd-tap-auth-delegator
- `linkerd` namespace:
  - Secret/linkerd-tap-tls
- `kube-system` namespace:
  - RoleBinding/linkerd-linkerd-tap-auth-reader

Tasks not covered by this PR:
- `linkerd top`
- `linkerd dashboard`
- `linkerd profile --tap`
- removal of the unauthenticated tap controller

Fixes #2725, #3162, #3172

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-08-01 14:02:45 -07:00
Andrew Seigner 9a672dd5a9
Introduce `linkerd --as` flag for impersonation (#3173)
Similar to `kubectl --as`, global flag across all linkerd subcommands
which sets a `ImpersonationConfig` in the Kubernetes API config.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-07-31 16:05:33 -07:00
Andrew Seigner a8830b2323
Set heartbeat cronjobs to not restart on failure (#3174)
The heartbeat cronjob specified `restartPolicy: OnFailure`. In cases
where failure was non-transient, such as if a cluster did not have
internet access, this would continuously restart and fail.

Change the heartbeat cronjob to `restartPolicy: Never`, as a failed job
has no user-facing impact.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-07-31 13:51:13 -07:00
Kevin Leimkuhler 8d9cfbf670
Inject Tap service name into proxy PodSpec (#3155)
### Summary

In order for Pods' tap servers to start authorizing tap clients, the tap server
must be able to check client names against the expected tap service name.

This change injects the `LINKERD2_PROXY_TAP_SVC_NAME` into proxy PodSpecs.

### Details

The tap servers on the individual resources being tapped should be able to
verify that the client is the tap service. The `LINKERD2_PROXY_TAP_SVC_NAME` is
now injected as an environment variable in the proxies so that it can check this
value against the client name of the TLS connection. Currently, this environment
will go unused. There is an open PR (linkerd2-proxy#290) to use this variable in
the proxy, but this is *not* dependent on that merging first. 

Note: The variable is not injected if tap is disabled.

### Testing

Test output has been updated with the newly injected environment variable.

Signed-off-by: Kevin Leimkuhler <kleimkuhler@icloud.com>
2019-07-29 15:05:45 -07:00
Tarun Pothulapati 2ba2dea6a6 Added Resource Limits when ha is Configured (#3092)
* increased ha resource limits
* added resource limits to proxy when HA
* update golden files in cmd/main

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2019-07-26 09:46:36 -07:00
Alejandro Pedraza 8c07223f3b
Remove unused argument (#3149)
Removed unused argument in the `GetPatch()` function in
`pkg/inject/inject.go`

Signed-off-by: Alejandro Pedraza <alejandro.pedraza@gmail.com>
2019-07-26 11:39:25 -05:00
Cody Vandermyn 808fa381f9 A Slightly More Restrictive PSP (#3085)
* Adds more PSP restrictions
* Update test fixtures
* Updates PSP to be conditional on initContainer

- The proxy-init container runs as root and needs the PSP to allow this
user when there is an init container.

Signed-off-by: Cody Vandermyn <cody.vandermyn@nordstrom.com>
2019-07-24 10:12:33 -07:00
Andrew Seigner 889a4a0578
Introduce -A as a shorthand for --all-namespaces (#3125)
kubectl introduced `-A` as a shorthand for `--all-namespaces` in
`v1.14.0`:
https://github.com/kubernetes/kubernetes/pull/72006

Update linkerd cli's `edges`, `get`, and `stat` commands to match this
convention.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-07-24 07:50:22 -07:00
Andrew Seigner 64ed8e4a74
Introduce Cluster Heartbeat cronjob (#3056)
`linkerd check`, the web dashboard, and Grafana all perform version
checks to validate Linkerd is up to date. It's common for users to
seldom execute these codepaths. This makes it difficult to identify what
versions of Linkerd are currently in use and what environments it is
being run in, which helps prioritize testing and backports.

Introduce a `heartbeat` CronJob to the default Linkerd install. The
cronjob executes every 24 hours, starting from 5 minutes after
`linkerd install` is run.

Example check URL:
https://versioncheck.linkerd.io/version.json?
  install-time=1562761177&
  k8s-version=v1.15.0&
  meshed-pods=8&
  rps=3&
  source=heartbeat&
  uuid=cc4bb700-3314-426a-9f0f-ec588b9df020&
  version=git-b97ee9f7

Fixes #2961

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-07-23 17:12:30 -07:00
Andrew Seigner 48a69cb88a
Bump Prometheus to 2.11.1, Grafana to 6.2.5 (#3123)
- set `disable_sanitize_html` in `grafana.ini`.
- make all text box dimensions whole integers to fix dropdown issue,
  reported in:
  https://github.com/linkerd/linkerd2/issues/2955#issuecomment-503085444
- rev all dashboards to `schemaVersion` 18 for Grafana 6.2.5
- `prometheus-benchmark.json` based on:
  https://grafana.com/grafana/dashboards/9761
- `prometheus.json` based on:
  69c93e6401/public/app/plugins/datasource/prometheus/dashboards/prometheus_2_stats.json
- `grafana.json` based on:
  85aed0276e/public/app/plugins/datasource/prometheus/dashboards/grafana_stats.json

Fixes #2955

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-07-23 13:37:56 -07:00
Alex Leong d6ef9ea460
Update ServiceProfile CRD to version v1alpha2 and remove validation (#3078)
The openAPIV3Schema validation in the ServiceProfiles CRD is very limited in what it can validate and is obviated by more sophisticated validation done by the validating admission controller.  Therefore, we would like to remove the openAPIV3Schema validation to reduce the size and complexity of the CRD object.

To do so, we must also bump the version of the ServiceProfile custom resource from v1alpha1 to v1alpha2.  This ensures that when the controller is upgraded, it will attempt to watch the v1alpha2 resource.  If it cannot (because, for example, the controller pod started before the ServiceProfile CRD was updated and therefore the v1alpha2 version does not exist) then it will go into a crash loop backoff until it can.  This essentially means that the controller will wait for the CRD to be upgraded to include v1alpha2 before it will start.  

Bumping the version is necessary because if we did not, it would be possible for the controller to start before the CRD is updated (removing the validation).  In this case, when the CRD is edited, the controller will lose its list watch on ServiceProfiles and will stop getting updates.

Signed-off-by: Alex Leong <alex@buoyant.io>
2019-07-23 11:46:31 -07:00
arminbuerkle 010efac24b Allow custom cluster domain in controller components (#2950)
* Allow custom cluster domain in destination watcher

The change relaxes the constrains of an authority requiring a
`svc.cluster.local` suffix to only require `svc` as third part.

A unit test could be added though the destination/server and endpoint
watcher already test this behaviour.

* Update proto to allow setting custom cluster domain

Update golden templates

* Allow setting custom domain in grpc, web server

* Remove cluster domain flags from web srv and public api

* Set defaultClusterDomain in validateAndBuild if none is set

Signed-off-by: Armin Buerkle <armin.buerkle@alfatraining.de>
2019-07-23 08:59:41 -07:00
Alex Leong c8b34a8cab
Add pod status to linkerd check (#3065)
When waiting for controller pods to be created or become ready, `linkerd check` doesn't offer any hints as to whether there has been an error (such as an ImagePullBackoff).

We add pod status to the output to make this more immediately obvious.

Fixes #2877 

Signed-off-by: Alex Leong <alex@buoyant.io>
2019-07-18 15:56:19 -07:00
Tarun Pothulapati fcec1cfb8a Added Anti Affinity when HA is configured (#2893)
* Added Anti Affinity when HA is configured
* Move check to validate()
* Test output with anti-affinity when ha upgrade
* Add anti-affinity to identity deployment
* made host anti-affinity default when ha
* Define affinity template in a separate file

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2019-07-18 10:03:25 -07:00
Alejandro Pedraza ba9fd70892
`linkerd upgrade config` bombs when installation had a flag (#3097)
When installing using some of the flags that persist in install, e.g
`linkerd install --ha`, and then doing `linkerd upgrade config` a nil
pointer error is thrown.

Fixes #3094

`newCmdUpgradeConfig()` was using passing `flags` as nil because
`linkerd upgrade config` doesn't expose any flags for the subcommand,
but turns out they're still needed down the call stack in
`setFlagsFromInstall` to reuse the flags persisted during install.

I also added a new unit test catching this.

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-07-18 09:09:01 -05:00
Carol A. Scott ee1a111993
Updating CLI output for `linkerd edges` (#3048)
This PR improves the CLI output for `linkerd edges` to reflect the latest API
changes. 

Source and destination namespaces for each edge are now shown by default. The
`MSG` column has been replaced with `Secured` and contains a green checkmark or
the reason for no identity. A new `-o wide` flag shows the identity of client
and server if known.
2019-07-17 12:23:34 -07:00
Jonathan Juares Beber 2dcbde08b3 Show pod status more clearly (#1967) (#2989)
During operations with `linkerd stat` sometimes it's not clear the actual
pod status.

This commit introduces a method, to the `k8s`package, getting the pod status,
based on [`kubectl` logic](33a3e325f7/pkg/printers/internalversion/printers.go (L558-L640))
to expose the `STATUS` column for pods . Also, it changes the stat command
on the` cli` package adding a column when the resource type is a Pod.

Fixes #1967

Signed-off-by: Jonathan Juares Beber <jonathanbeber@gmail.com>
2019-07-10 12:44:44 -07:00
Andrew Seigner 7756828ae6
Update install failure message to list resources (#3050)
The existing `linkerd install` error message for existing resources was
shared with `linkerd check`. Given the different contexts, the messaging
made more sense for `linkerd check` than for `linkerd install`.

Modify the error messaging for `linkerd install` to print a bare list
of existing resources, and provide instructions for proceeding.

For example:
```bash
$ linkerd install
Unable to install the Linkerd control plane. It appears that there is an existing installation:

clusterrole.rbac.authorization.k8s.io/linkerd-linkerd-controller
clusterrole.rbac.authorization.k8s.io/linkerd-linkerd-identity

If you are sure you'd like to have a fresh install, remove these resources with:

    linkerd install --ignore-cluster | kubectl delete -f -

Otherwise, you can use the --ignore-cluster flag to overwrite the existing global resources.
```

Fixes #3045

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-07-09 20:21:19 +02:00
Andrew Seigner 9e09bd5e98
Mark High Availability as non-experimental (#3049)
Fixes #2419

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-07-09 20:20:28 +02:00
Andrew Seigner 94fa653cf3
Fix `linkerd check` missing uuid on version check (#3040)
PR #2603 modified the web process to read the UUID from the
`linkerd-config` ConfigMap rather than from a command line flag. The
`linkerd check` command relied on that command line flag to retrieve the
UUID as part of its version check.

Modify `linkerd check` to correctly retrieve the UUID from
`linkerd-config`. Also refactor `linkerd-config` retrieval and parsing
code to be shared between healthcheck, install, and upgrade.

Relates to #2961

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-07-05 19:39:13 +02:00
Tarun Pothulapati eb7f9866af Fix inject with path and add tests (#3038)
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2019-07-05 09:26:25 -05:00
Alejandro Pedraza 53e589890d
Have `linkerd endpoints` use `Destination.Get` (#2990)
* Have `linkerd endpoints` use `Destination.Get`

Fixes #2885

We're refactoring `linkerd endpoints` so it hits
directly the `Destination.Get` endpoint, instead of relying on the
Discovery service.

For that, I've created a new `client.go` for Destination and added it to
the `APIClient` interface.

I've also added a `destinationClient` struct that mimics `tapClient`,
and whose common logic has been moved into `stream_client.go`.

Analogously, I added a `destinationServer` struct that mimics
`tapServer`.

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-07-03 09:11:03 -05:00
Ivan Sim 7e1c14e783
Add the 'linkerd.io/control-plane-ns' label to the Traffic Split CRD (#3026)
Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-07-02 15:46:25 -07:00
Andrew Seigner 902978fe48
Rename debug annotation to enable-debug-sidecar (#3016)
Linkerd's CLI flags all match 1:1 with their `config.linkerd.io/*`
annotation counterparts, except `--enable-debug-sidecar`, which
corresponded to `config.linkerd.io/debug`. Additionally, the Linkerd
docs assume this 1:1 mapping.

Rename the `config.linkerd.io/debug` annotation to
`config.linkerd.io/enable-debug-sidecar`.

Relates to https://github.com/linkerd/website/issues/381

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-07-02 20:01:52 +02:00
Carol A. Scott a504e8c2d8
Expand and improve edges API endpoint (#3007)
Updates functionality of `linkerd edges`, including a new `--all-namespaces`
flag and returning namespace information for SRC and DST resources.
2019-06-28 15:46:04 -07:00
Alex Leong 27373a8b78
Add traffic splitting to destination profiles (#2931)
This change implements the DstOverrides feature of the destination profile API (aka traffic splitting).

We add a TrafficSplitWatcher to the destination service which watches for TrafficSplit resources and notifies subscribers about TrafficSplits for services that they are subscribed to.  A new TrafficSplitAdaptor then merges the TrafficSplit logic into the DstOverrides field of the destination profile.

Signed-off-by: Alex Leong <alex@buoyant.io>
2019-06-28 13:19:47 -07:00
Tarun Pothulapati 7db058f096 linkerd inject from remote URL (#2988)
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2019-06-28 09:47:33 -07:00
Tarun Pothulapati 5c5ec6d816 add admin port label to proxy-injector and sp-validator (#2984)
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2019-06-27 17:25:49 -05:00
Ivan Sim 866fe6fa5e
Introduce global resources checks to install and multi-stage install (#2987)
* Introduce new checks to determine existence of global resources and the
'linkerd-config' config map.
* Update pre-check to check for existence of global resources

This ensures that multiple control planes can't be installed into
different namespaces.

* Update integration test clean-up script to delete psp and crd

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-06-27 09:59:12 -07:00
Alejandro Pedraza 73740fb503
Simplify port-forwarding code (#2976)
* Simplify port-forwarding code

Simplifies the establishment of a port-forwarding by moving the common
logic into `PortForward.Init()`

Stemmed from this
[comment](https://github.com/linkerd/linkerd2/pull/2937#discussion_r295078800)

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-06-26 11:14:57 -05:00
Andrew Seigner 81790b6735 Bump Prometheus to v2.10.0 (#2979)
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-06-21 12:51:31 -07:00
Tarun Pothulapati a3ce06bd80 Add sideEffects field to Webhooks (#2963)
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2019-06-21 11:06:10 -07:00
Ivan Sim 435fe861d0
Label all Linkerd resources (#2971)
Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-06-20 09:44:30 -07:00
Ivan Sim e2e976cce9
Add `NET_RAW` capability to the proxy-init container (#2969)
Also, update control plane PSP to match linkerd/website#94

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-06-19 19:34:37 -07:00
Dennis Adjei-Baah 694ba9c2cb
Revert add namespace name to MWC (#2946)
* revert add namespace name to MWC
2019-06-14 15:26:34 -07:00
Alejandro Pedraza 7fc6c195ad
Set MWC and VWC failure policy to 'fail' in HA mode only (#2943)
Fixes #2927

Also moved `TestInstallSP` after `TestCheckPostInstall` so we're sure
the validating webhook is ready before installing a service profile.

Signed-off-by: Alejandro Pedraza Borrero <alejandro@buoyant.io>
2019-06-14 11:50:59 -05:00
Alejandro Pedraza 28025eeb56
Remove UPDATE event from the mutating webhook config (#2919)
Fixes #2889

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-06-13 15:42:47 -05:00
Alejandro Pedraza e9bf014d34
Remove MWVC RBAC from webhook configs (#2925)
Fixes #2890

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-06-13 15:42:00 -05:00
Dennis Adjei-Baah 035ba6ae87
update sp-validator MWC golden test file (#2938) 2019-06-13 13:39:24 -07:00
Dennis Adjei-Baah 8aef9280dd
add namespace name to MWC (#2905)
When installing multiple control planes, the mutatingwebhookconfiguration of the first control plane gets overwritten by any subsequent control plane install. This is caused by the fixed name given to the mutatingwebhookconfiguration manifest at install time.

This commit adds in the namespace to the manifest so that there is a unique configuration for each control plane.

Fixes #2887
2019-06-13 12:15:43 -07:00
Ivan Sim ecc4465cd1
Introduce Control Plane's PSP and RBAC resources into Helm templates (#2920)
* Add control plane and CNI PSP and RBAC resources
* Add the '--linkerd-cni-enabled' flag to the multi-stage install subcommands

This flag ensures that the NET_ADMIN capability is omitted from the control
plane's PSP during 'install config' and the proxy-init containers aren't
injected during 'install control-plane'.

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-06-12 20:18:46 -07:00
Alejandro Pedraza 8416d326c2
If HA, set the webhooks failure policy to 'Fail' (#2906)
* If HA, set the webhooks failure policy to 'Fail'

I'm adding to the linkerd namespace a new label
`linkerd.io/is-control-plane: true` that is used in the webhook configs'
selector to skip the proxy injector for this namespace. This avoids
running into the timing issues described in #2852.

Fixes #2852

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-06-11 13:11:54 -05:00
Cody Vandermyn 33de3574ee Correctly set securityContext values on injection (#2911)
The patch provided by @ihcsim applies correct values for the securityContext during injection, namely: `allowPrivilegeEscalation = false`, `readOnlyRootFilesystem = true`, and the capabilities are copied from the primary container. Additionally, the proxy-init container securityContext has been updated with appropriate values.

Signed-off-by: Cody Vandermyn <cody.vandermyn@nordstrom.com>
2019-06-11 10:34:30 -07:00
Dan 24bbd7c64b Ensure Prometheus log level is lowercase (#2823) (#2870)
Signed-off-by: Daniel Baranowski <daniel.baranowski@infinityworks.com>
2019-06-07 09:57:08 -07:00
Alejandro Pedraza 66eb829e5a
Fix HA during upgrade (#2900)
* Fix HA during upgrade

If we have a Linkerd installation with HA, and then we do `linkerd
upgrade` without specifying `--ha`, the replicas will get set back to 1,
yet the resource requests will keep their HA values.

Desired behavior: `linkerd install --ha` adds the `ha` value into the
linkerd-config, so it should be used during upgrade even if `--ha` is
not passed to `linkerd upgrade`.
Note we still can do `linkerd upgrade --ha=false` to disable HA.

This is a prerequesite to address #2852

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-06-06 17:27:27 -05:00
Alejandro Pedraza 74ca92ea25
Split proxy-init into separate repo (#2824)
Split proxy-init into separate repo

Fixes #2563

The new repo is https://github.com/linkerd/linkerd2-proxy-init, and I
tagged the latest there `v1.0.0`.

Here, I've removed the `/proxy-init` dir and pinned the injected
proxy-init version to `v1.0.0` in the injector code and tests.

`/cni-plugin` depends on proxy-init, so I updated the import paths
there, and could verify CNI is still working (there is some flakiness
but unrelated to this PR).

For consistency, I added a `--init-image-version` flag to `linkerd
inject` along with its corresponding override config annotation.

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-06-03 16:24:05 -05:00
Tarun Pothulapati 590249c66b HA for proxy-injector and sp-validator (#2874)
* Added labels to webhook configurations in charts/
* Multiple replicas of proxy-injector and sp-validator in HA
* Use ControllerComponent template variable for webhookconfigurations

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2019-05-31 14:48:30 -07:00
Carol A. Scott 031cd9d4ba
Simplify logic for linkerd edges and output JSON in consistent order (#2858)
When `linkerd edges` returns JSON, the data will now be sorted alphabetically by
SRC name, meaning edges will be returned in a consistent order. Logic in the CLI
`edges.go` has also been simplified. These changes should result in the Travis
CI builds passing consistently.
2019-05-29 11:31:58 -07:00
Ivan Sim 86d822f8ea
Generate the debug container spec in the shared library (#2854)
This commit refactors the changes introduced by #2842 where the debug
container spec is created in the 'cli' and 'pkg' packages. This change
follows the existing pattern of annotating the YAML in the CLI code,
and injecting the sidecar spec in the shared library.

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-05-28 15:26:37 -07:00
Tarun Pothulapati 1a574def1f Added labels to webhook configurations in charts/ (#2853)
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2019-05-28 16:44:09 -05:00
Carol A. Scott 8c496e3d0d
Adding unit test for CLI edges command (#2837)
Adds a unit test for the `linkerd edges` command.
2019-05-28 13:51:45 -07:00
Dennis Adjei-Baah b9cc66c6c8
fixes roleRef name in linkerd-tap rbac (#2845)
* fixes roleRef name in linkerd-tap rbac
2019-05-28 10:05:01 -07:00
Ivan Sim 0d489d3099
Add debug container annotation (#2842)
This new annotation is used by the proxy injector to determine if the 
debug container needs to be injected.

When using 'linkerd install', the 'pkg/inject' library will only inject
annotations into the workload YAML. Even though 'conf.debugSidecar' 
is set in the CLI, the 'injectPodSpec()' function is never invoked on 
the proxy injector side. Once the workload YAML got picked up by the 
proxy injector, 'conf.debugSidecar' is already nil, since it's a different, 
new 'conf' object. The new annotation ensures that the proxy injector
injects the debug container.

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-05-24 10:11:06 -07:00
Ivan Sim 5a5f8bbfe8
Install MWC and VWC During Installation (#2806)
* Update helm charts to include webhooks config and TLS secret
* Update the webhooks to read the secret cert and key
* Update webhooks to not recreate config on restart
* Ensure upgrade preserve existing secrets
* Revert the change to rename the webhook configs

The renaming change breaks upgrade, where the new webhook configs conflict with
the existing ones. The older resources  aren't deleted during upgrade because
they are dynamically created.

* Make the secret volume read-only
* Remove unnecessary exported getter functions
* Remove obsolete mwc and vwc templates

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-05-20 12:43:50 -07:00
Alena Varkockova 7973715ee4 Output check result as json (#2666)
* Add missing file, make linter happy
* Changes from code review
* Ivan's code review
* PR review feedback
* style fixes
* Last PR comments

Signed-off-by: Alena Varkockova <varkockova.a@gmail.com>
2019-05-20 09:04:28 -07:00
Alejandro Pedraza 253068e844
Unhide CLI inject's --disable-tap now that #2811 has been addressed (#2827)
Signed-off-by: Alejandro Pedraza <alejandro.pedraza@gmail.com>
2019-05-16 11:12:20 -05:00
Dennis Adjei-Baah a0fa1dff59
Move tap service into its own pod. (#2773)
* Split tap into its own pod in the control plane

Signed-off-by: Dennis Adjei-Baah <dennis@buoyant.io>
2019-05-15 16:28:44 -05:00
Carol A. Scott 042086142a
Adding an edges command to the CLI (#2808)
Adds an edges command to the CLI. `linkerd edges` displays connections between resources, and Linkerd proxy identities. Currently this feature will only display edges where both the client identity and server identity are known. The next step will be to display edges for which identity is not known and/or one-sided traffic such as Prometheus and tap requests.
2019-05-15 13:59:27 -07:00
Alejandro Pedraza 065c221858
Support for resources opting-out of tap (#2807)
Support for resources opting out of tap

Implements the `linkerd inject --disable-tap` flag (although hidden pending #2811) and the config override annotation `config.linkerd.io/disable-tap`.
Fixes #2778

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-05-10 14:17:23 -05:00
Jack Price f758a9e428 Use port-forwarding for linkerd CLIs (#2757)
Private k8s clusters, such as the private GKE clusters offered by Google
Cloud, cannot be reached through the current API proxy method.

This commit uses the port forwarding feature already developed.

Also modify dashboard command to not fall back to ephemeral port.

Signed-off-by: Jack Price <jackprice@outlook.com>
2019-05-02 14:41:26 +02:00
Andrew Seigner 266e882d79
Define multi-stage commands as subcommands (#2772)
The multi-stage args used by install, upgrade, and check were
implemented as positional arguments to their respective parent commands.
This made the help documentation unclear, and the code ambiguous as to
which flags corresponded to which stage.

Define `config` and `control-plane` stages as subcommands. The help
menus now explicitly state flags supported.

Fixes #2729

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-05-02 12:32:01 +02:00
Andrew Seigner 66494591e0
Multi-stage check support (#2765)
Add support for `linkerd check config`. Validates the existence of the
Linkerd Namespace, ClusterRoles, ClusterRoleBindings, ServiceAccounts,
and CustomResourceDefitions.

Part of #2337

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-04-30 17:17:59 +01:00
Ivan Sim 714035fee9
Define default resource spec for proxy-init init container (#2763)
Fixes #2750 

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-04-29 11:41:05 -07:00
Andrew Seigner 0cfc8c6f1c
Introduce k8s apiextensions support (#2759)
CustomResourceDefinition parsing and retrieval is not available via
client-go's `kubernetes.Interface`, but rather via a separate
`k8s.io/apiextensions-apiserver` package.

Introduce support for CustomResourceDefintion object parsing and
retrieval. This change facilitates retrieval of CRDs from the k8s API
server, and also provides CRD resources as mock objects.

Also introduce a `NewFakeAPI` constructor, deprecating
`NewFakeClientSets`. Callers need no longer be concerned with discreet
clientsets (for k8s resources vs. CRDs vs. (eventually)
ServiceProfiles), and can instead use the unified `KubernetesAPI`.

Part of #2337, in service to multi-stage check.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-04-28 18:55:22 -07:00
Andrew Seigner be60b37e93
Group Web and Grafana ServiceAccounts with RBAC (#2756)
All ServiceAccounts are intended to be grouped together with other RBAC
resources, particularly for `linkerd install config` output. Grafana and
Web ServiceAccounts were still included with their respective
Deployments.

Group Grafana and Web ServiceAccounts with other RBAC resources.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-04-25 17:33:05 -07:00
Andrew Seigner 15ffd86cf1
Introduce multi-stage upgrade (#2723)
`linkerd install` supports a 2-stage install process, `linkerd upgrade`
did not.

Add 2-stage support for `linkerd upgrade`. Also exercise multi-stage
functionality during upgrade integration tests.

Part of #2337

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-04-25 14:29:52 -07:00
Alex Leong 4ea7c62b0d
Revert " Remove validation from service profile CRD definition (#2740)" (#2752)
This reverts commit 3de16d47be.

#2740 modified the ServiceProfiles CRD which will cause issues for users upgrading from the old CRD version to the new version.  #2748 was an attempt to fix this by bumping the service profile CRD version, however, our testing infrastructure is not well set up to accommodate changes to CRDs because they are resources which are global to the cluster.  

We revert this change for now and will revisit it in the future when we can give more thought to CRD versioning, upgrade, and testing.

Signed-off-by: Alex Leong <alex@buoyant.io>
2019-04-25 13:40:20 -07:00
Andrew Seigner ec540a882e
Consolidate k8s APIs (#2747)
Numerous codepaths have emerged that create k8s configs, k8s clients,
and make k8s api requests.

This branch consolidates k8s client creation and APIs. The primary
change migrates most codepaths to call `k8s.NewAPI` to instantiate a
`KubernetesAPI` struct from `pkg`. `KubernetesAPI` implements the
`kubernetes.Interface` (clientset) interface, and also persists a
`client-go` `rest.Config`.

Specific list of changes:
- removes manual GET requests from `k8s.KubernetesAPI`, in favor of
  clientsets
- replaces most calls to `k8s.GetConfig`+`kubernetes.NewForConfig` with
  a single `k8s.NewAPI`
- introduces a `timeout` param to `k8s.NewAPI`, currently only used by
  healthchecks
- removes `NewClientSet` in `controller/k8s/clientset.go` in favor of
  `k8s.NewAPI`
- removes `httpClient` and `clientset` from `HealthChecker`, use
  `KubernetesAPI` instead

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-04-25 11:31:38 -07:00
Ivan Sim cd37d3f0f5
Fall back to default built-in version if versions config are missing (#2745)
Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-04-24 19:49:18 -07:00
Alejandro Pedraza 53bb7c47f6
Make the auto-injector required and removed proxy-auto-inject flag (#2733)
Make the auto-injector required and removed proxy-auto-inject flag

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-04-24 13:06:51 -05:00
Alejandro Pedraza 62d9a80894
New `linkerd inject` default and manual modes (#2721)
Fixes #2720 and 2711 

This changes the default behavior of `linkerd inject` to not inject the
proxy but just the `linkerd.io/inject: enabled` annotation for the
auto-injector to pick it up (regardless of any namespace annotation).

A new `--manual` mode was added, which behaves as before, injecting
the proxy in the command output.

The unit tests are running with `--manual` to avoid any changes in the
fixtures.

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-04-24 09:05:27 -05:00
Alex Leong 3de16d47be
Remove validation from service profile CRD definition (#2740)
Fixes #2736

Signed-off-by: Alex Leong <alex@buoyant.io>
2019-04-23 16:10:50 -07:00
Andrew Seigner b2b4780430
Introduce install stages (#2719)
This change introduces two named parameters for `linkerd install`, split
by privilege:
- `linkerd install config`
  - Namespace
  - ClusterRoles
  - ClusterRoleBindings
  - CustomResourceDefinition
  - ServiceAccounts
- `linkerd install control-plane`
  - ConfigMaps
  - Secrets
  - Deployments
  - Services

Comprehensive `linkerd install` is still supported.

TODO:
- `linkerd check` support
- `linkerd upgrade` support
- integration tests

Part of #2337

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-04-23 14:52:34 -07:00
Dennis Adjei-Baah 3e5917f7e0
Add the ability to inject a debug sidecar (#2726)
Signed-off-by: Dennis Adjei-Baah <dennis@buoyant.io>
2019-04-22 16:53:12 -07:00
Alejandro Pedraza c56766a923
Add config.linkerd.io/disable-identity annotation (#2717)
Add config.linkerd.io/disable-identity annotation

First part of #2540

We'll tackle support for `--disable-identity` in `linkerd install` in a
separate commit.

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-04-19 19:04:49 -04:00
Ivan Sim 8d13084f94
Split the `linkerd-version` CLI flag into `control-plane-version` and `proxy-version` (#2702)
* The 'linkerd-version' CLI flag is renamed to 'control-plane-version'
* Add version field to proxy config
* Add the control plane version to the global config
* Unit test for init image version
* Use more specific control plane and proxy versions in unit tests

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-04-19 11:35:20 -07:00
Dennis Adjei-Baah be614656bb
add service profile integration tests for service profile metrics (#2685)
* add integration tests for retryable requests

Signed-off-by: Dennis Adjei-Baah <dennis@buoyant.io>
2019-04-18 11:01:49 -07:00
Andrew Seigner 72287ae121
Don't use spinner in cli when run without a tty (#2716)
In some non-tty environments, the `linkerd check` spinner can render
unexpected control characters.

Disable the spinner when run without a tty.

Fixes #2700

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-04-18 09:31:56 -07:00
Andrew Seigner 2d9e3686e2
Split out config objects from install templates (#2714)
This is an initial change to separate out config-specific k8s objects
from the control-plane components. The eventual goal will be rendering
these configs as the first stage of a multi-stage install.

Part of #2337

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-04-18 09:31:35 -07:00
Andrew Seigner 8323e104fb
Introduce upgrade --from-manifests flag (#2697)
The `linkerd upgrade` command read the control-plane's config from
Kubernetes, which required the environment to be configured to connect
to the appropriate k8s cluster.

Intrdouce a `linkerd upgrade --from-manifests` flag, allowing the user
to feed the output of `linkerd install` into the upgrade command.

Fixes #2629

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-04-17 13:32:21 -07:00