Commit Graph

322 Commits

Author SHA1 Message Date
Kevin Lingerfelt 86e95b7ad3
Disable serivce profiles in single-namespace mode (#1980)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-12-13 14:37:18 -08:00
Cody Vandermyn d847f66ec5 Create new service accounts for linkerd-web and linkerd-grafana. Chan… (#1978)
* Create new service accounts for linkerd-web and linkerd-grafana. Change 'serviceAccount:' to 'serviceAccountName:'
* Use dynamic namespace name

Signed-off-by: Cody Vandermyn <cody.vandermyn@nordstrom.com>
2018-12-12 18:10:50 -08:00
Cody Vandermyn aa5e5f42eb Use an emptyDir for Prometheus and Grafana (#1971)
* Allow input of a volume name for prometheus and grafana
* Make Prometheus and Grafana volume names 'data' by default and disallow user editing via cli flags
* Remove volume name from options

Signed-off-by: Cody Vandermyn <cody.vandermyn@nordstrom.com>
2018-12-12 15:54:03 -08:00
Kevin Lingerfelt fd44896644
Remove namespace definition from --single-namespace installs (#1974)
* Remove namespace definition from --single-namespace installs
* DRY up code in healthcheck.go

Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-12-12 14:53:02 -08:00
Kevin Lingerfelt 8cad97cd6c
Set proxy-injector resources at container level on install (#1972)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-12-11 13:28:39 -08:00
Cody Vandermyn 8e4d9d2ef6 add securityContext with runAsUser: {{.ControllerUID}} to the various cont… (#1929)
* add securityContext with runAsUser: {{.ProxyUID}} to the various containers in the install template
* Update golden to reflect new additions
* changed to a different user id than the proxy user id
* Added a controller-uid install option
* change the port that the proxy-injector runs
* The initContainers needs to be run as the root user.
* move security contexts to container level

Signed-off-by: Cody Vandermyn <cody.vandermyn@nordstrom.com>
2018-12-11 11:51:28 -08:00
Alejandro Pedraza 8c67bfbcc6 Add parameter to stats API to skip retrieving Prometheus stats (#1871)
* Add parameter to stats API to skip retrieving Prometheus stats

Used by the dashboard to populate list of resources.

Fixes #1022

Signed-off-by: Alejandro Pedraza <alejandro.pedraza@gmail.com>

* Prometheus queries check results were being ignored
* Refactor verifyPromQueries() to also test when no prometheus queries
should be generated

* Add test for SkipStats=true

Includes adding ability to public.GenStatSummaryResponse to not generate
basicStats

* Fix previous test
2018-12-10 16:48:12 -08:00
Andrew Seigner bef9479f57
Add input validation for profile command (#1934)
Fixes #1878

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2018-12-05 15:13:10 -08:00
Alex Leong 7169eaef27
Stop routes with the same name from different services from clobbering each other (#1936)
If the `linkerd routes` command gets two routes with the same name, it will only display one of them, even if the routes are from different services.  This is particularly obvious with the default `[UNKNOWN]` route.

We now display all routes, even if they have the same name.

Signed-off-by: Alex Leong <alex@buoyant.io>
2018-12-05 15:05:19 -08:00
Alex Leong cbb196066f
Support service profiles for external authorities (#1928)
Add support for service profiles created on external (non-service) authorities.  For example, this allows you to create a service profile named `linkerd.io` which will apply to calls made to `linkerd.io`.

This is done by changing the `LINKERD2_PROXY_DESTINATION_PROFILE_SUFFIXES` to `.` so that the proxy will attempt to lookup a service profile for any authority.  We provide the `--disable-external-profiles` proxy flag to revert this behavior in case it is a problem.

We also refactor the proxy-api implementation of GetProfiles so that it does the profile lookup, regardless of if the authority looks like a Kubernetes service name or not.  To simplify this, support for multiple resolves (which was unused) was removed.

Signed-off-by: Alex Leong <alex@buoyant.io>
2018-12-05 14:32:59 -08:00
Oliver Gould 12ec5cf922
install: Add a -disable-h2-upgrade flag (#1926)
The proxy-api service _always_ suggests that two meshed pods communicate
via HTTP/2 (i.e. via transparent protocol upgrading, if necessary).
This can complicate debugging and diagnostics at times, so it's
important that we have a way to deploy linkerd without this auto-upgrade
behavior.

This change adds a `-disable-h2-upgrade` flag to the `linkerd install`
command that disables transparent upgrading for the whole cluster.
2018-12-05 12:50:47 -08:00
Alex Leong 380ec52a39
Rework routes command to accept any resource (#1921)
We rework the routes command so that it can accept any Kubernetes resource, making it act much more similarly to the stat command.

Signed-off-by: Alex Leong <alex@buoyant.io>
2018-12-05 11:11:34 -08:00
Alex Leong 4f3e55e937
Rename path to path_regex in ServiceProfile CRD (#1923)
We rename path to path_regex in the ServiceProfile CRD to make it clear that this field accepts a regular expression. We also take this opportunity to remove unnecessary line anchors from regular expressions now that these anchors are added in the proxy.

Signed-off-by: Alex Leong <alex@buoyant.io>
2018-12-05 10:42:47 -08:00
Kevin Lingerfelt 37ae423bb3
Add linkerd- prefix to all objects in linkerd install (#1920)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-12-04 15:41:47 -08:00
Andrew Seigner ad2366f208
Revert proxy readiness initialDelaySeconds change (#1912)
Reverts part of #1899 to workaround readiness failures.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2018-12-04 14:27:55 -08:00
Risha Mars e8a39cd17e
Add ability to download a service profile template from the web UI (#1893)
Adds an endpoint, at /profiles/new that allows you to input a service name and
namespace, and download a service profile yaml template. 

This will enable future work, where we can add more of the yaml customization via 
a form in the dashboard, and use that data to help the user configure routes.
2018-12-03 16:48:43 -08:00
Andrew Seigner 37a5455445
Add filtering by job in stat, tap, top; fix panic (#1904)
Filtering by Kubernetes job was not supported. Also filtering by any unknown
type caused a panic.

Add filtering support by Kubernetes job, with special case mapping `job` to
`k8s_job`, to not conflict with Prometheus' job label.

Fix panic when unknown type specified as a `--from` or `--to` flag.

Fix `job` label from `linkerd-proxy` overwriting Prometheus `job` label at
collection time. This caused all metrics collected by proxy sidecars in
Kubernetes jobs to be collected into an incorrect Prometheus job, rather than
the expected `linkerd-proxy` Prometheus job.

Fix `unsupported resource type` tap error message incorrectly printing the
target resource rather than the destination.

Set `--controller-log-level debug` in `install_test.go` for easier debugging.

Expose `slow-cooker`'s metrics via a k8s service in the tap integration test, to
validate proxy requests with a job as destination.

Fixes #1872
Part of #627

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2018-12-03 15:34:49 -08:00
Oliver Gould 926395f616
tap: Include route labels in tap events (#1902)
This change alters the controller's Tap service to include route labels
when translating tap events, modifies the public API to include route
metadata in responses, and modifies the tap CLI command to include
rt_ labels in tap output (when -o wide is used).
2018-12-03 13:52:47 -08:00
Andrew Seigner d121071f87
Adjust proxy, Prometheus, and Grafana probes (#1899)
* Adjust proxy, Prometheus, and Grafana probes

High `readinessProbe.initialDelaySeconds` values delayed the controller's
readiness by up to 30s, preventing cli commands from succeeding shortly after
control plane deployment.

Decrease `readinessProbe.initialDelaySeconds` in the proxy, Prometheus, and
Grafana to the default 0s. Also change `linkerd check` controller pod ordering
to: controller, prometheus, web, grafana.

Detailed probe changes:
- proxy
  - decrease `readinessProbe.initialDelaySeconds` from 10s to 0s
- prometheus
  - decrease `readinessProbe.initialDelaySeconds` from 30s to 0s
  - decrease `readinessProbe.timeoutSeconds` from 30s to 1s
  - decrease `livenessProbe.timeoutSeconds` from 30s to 1s
- grafana
  - decrease `readinessProbe.initialDelaySeconds` from 30s to 0s
  - decrease `readinessProbe.timeoutSeconds` from 30s to 1s
  - decrease `readinessProbe.failureThreshold` from 10 to 3
  - increase `livenessProbe.initialDelaySeconds` from 0s to 30s

Fixes #1804

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2018-12-03 10:41:11 -08:00
Alex Leong f9d66cf4de
Add --open-api option to linkerd profiles command (#1867)
The `--open-api` flag is an alternative to the `--template` flag for the `linkerd profile` command.  It reads an OpenAPI specification file (also called a swagger file) and uses it to generate a corresponding service profile.

Signed-off-by: Alex Leong <alex@buoyant.io>
2018-11-30 09:25:19 -08:00
Alex Leong 835e34b500
Left-align the routes column and sort by route name (#1879)
Signed-off-by: Alex Leong <alex@buoyant.io>
2018-11-28 09:32:59 -08:00
Ben Lambert 297cb570f2 Added a --ha flag to install CLI (#1852)
This change allows some advised production config to be applied to the install of the control plane.
Currently this runs 3x replicas of the controller and adds some pretty sane requests to each of the components + containers of the control plane.

Fixes #1101

Signed-off-by: Ben Lambert <ben@blam.sh>
2018-11-20 23:03:59 -05:00
Alex Leong 7a7f6b6ecb
Add TopRoutes method the the public api and route CLI command to consume it (#1860)
Add a routes command which displays per-route stats for services that have service profiles defined.

This change has three parts:
* A new public-api RPC called `TopRoutes` which serves per-route stat data about a service
* An implementation of TopRoutes in the public-api service.  This implementation reads per-route data from Prometheus.  This is very similar to how the StatSummaries RPC and much of the code was able to be refactored and shared.
* A new CLI command called `routes` which displays the per-route data in a tabular or json format.  This is very similar to the `stat` command and much of the code was able to be refactored and shared.

Note that as of the currently targeted proxy version, only outbound route stats are supported so the `--from` flag must be included in order to see data.  This restriction will be lifted in an upcoming change once we add support for inbound route stats as well.

Signed-off-by: Alex Leong <alex@buoyant.io>
2018-11-19 12:20:30 -08:00
Alejandro Pedraza bbcf5a8c9f Allow stat summary to query for multiple resources (#1841)
* Refactor util.BuildResource so it can deal with multiple resources

First step to address #1487: Allow stat summary to query for multiple
resources

Signed-off-by: Alejandro Pedraza <alejandro.pedraza@gmail.com>

* Update the stat cli help text to explain the new multi resource querying ability

Propsal for #1487: Allow stat summary to query for multiple resources

Signed-off-by: Alejandro Pedraza <alejandro.pedraza@gmail.com>

* Allow stat summary to query for multiple resources

Implement this ability by issuing parallel requests to requestStatsFromAPI()

Proposal for #1487

Signed-off-by: Alejandro Pedraza <alejandro.pedraza@gmail.com>

* Update tests as part of multi-resource support in `linkerd stat` (#1487)

- Refactor stat_test.go to reuse the same logic in multiple tests, and
add cases and files for json output.
- Add a couple of cases to api_utils_test.go to test multiple resources
validation.

Signed-off-by: Alejandro Pedraza <alejandro.pedraza@gmail.com>

* `linkerd stat` called with multiple resources should keep an ordering (#1487)

Add SortedRes holding the order of resources to be followed when
querying `linkerd stat` with multiple resources

Signed-off-by: Alejandro Pedraza <alejandro.pedraza@gmail.com>

* Extra validations for `linkerd stat` with multiple resources (#1487)

Signed-off-by: Alejandro Pedraza <alejandro.pedraza@gmail.com>

* `linkerd stat` resource grouping, ordering and name prefixing (#1487)

- Group together stats per resource type.
- When more than one resource, prepend name with type.
- Make sure tables always appear in the same order.

Signed-off-by: Alejandro Pedraza <alejandro.pedraza@gmail.com>

* Allow `linkerd stat` to be called with multiple resources

A few final refactorings as per code review.

Fixes #1487

Signed-off-by: Alejandro Pedraza <alejandro.pedraza@gmail.com>
2018-11-14 10:44:04 -08:00
Alex Leong 32d556e732
Improve ergonomics of service profile spec (#1828)
We make several changes to the service profile spec to make service profiles more ergonomic and to make them more consistent with the destination profile API.

* Allow multiple fields to be simultaneously set on a RequestMatch or ResponseMatch condition.  Doing so is equivalent to combining the fields with an "all" condition.
* Rename "responses" to "response_classes"
* Change "IsSuccess" to "is_failure"

Signed-off-by: Alex Leong <alex@buoyant.io>
2018-10-31 12:00:22 -07:00
Alex Leong d8b5ebaa6d
Remove the proxy-api container (#1813)
A container called `proxy-api` runs in the Linkerd2 controller pod.  This container listens on port 8086 and serves the proxy-api but does nothing other than forward gRPC requests to the destination container which listens on port 8089.

We remove the proxy-api container altogether and change the destination container to listen on port 8086 instead of 8089.  The result is that clients still use the proxy-api by connecting to `proxy-api.<ns>.svc.cluster.local:8086` but the controller has one fewer containers.  This results in a simpler system that is easier to reason about.

Signed-off-by: Alex Leong <alex@buoyant.io>
2018-10-29 16:31:43 -07:00
Alex Leong 82ca821e62
Use fqdn for service profile name (#1808)
Service profiles must be named in the form `"<service>.<namespace>"`.  This is inconsistent with the fully normalized domain name that the proxy sends to the controller.  It also does not permit creating service profiles for non-Kubernetes services.

We switch to requiring that service profiles must be named with the FQDN of their service.  For Kubernetes services, this is `"<service>.<namespace>.svc.cluster.local"`.

This change alone is not sufficient for allowing service profile for non-Kubernetes services because the k8s resolver will ignore any DNS names which are not Kubernetes services.  Further refactoring of the resolver will be required to allow looking up non-Kubernetes service profiles in Kuberenetes.

Signed-off-by: Alex Leong <alex@buoyant.io>
2018-10-29 14:35:42 -07:00
Alex Leong 622185a4dd
Send metric labels in profile API (#1800)
* Send metric labels in profile API

Signed-off-by: Alex Leong <alex@buoyant.io>
2018-10-29 14:28:09 -07:00
Alex Leong 6cffad277b
Make service profile validation a warning instead of an error (#1807)
The existence of an invalid service profile causes `linkerd check` to fail.  This means that it is not possible to open the Linkerd dashboard with the `linkerd dashboard` command.  While service profile validation is useful, it should not lock users out.

Add the ability to designate health checks as warnings.  A failed warning health check will display a warning output in `linkerd check` but will not affect the overall success of the command.  Switch the service profile validation to be a warning.

Signed-off-by: Alex Leong <alex@buoyant.io>
2018-10-26 13:28:10 -07:00
Alex Leong 652ca161ef
Add linkerd profile --template command (#1773)
Add a new CLI command: `linkerd profile --template` which outputs a sample service profile yaml.  Users can edit this sample and then `kubectl apply` it to add a service profile.  The sample serves as "documentation by example" of what service profiles may contain.

Example usage:
```bash
linkerd profile -n emojivoto --template web-svc > web-svc-profile.yaml
# edit web-svc-profile.yaml in your favorite editor
kubectl apply -f web-svc-profile.yaml
```

Signed-off-by: Alex Leong <alex@buoyant.io>
2018-10-19 13:34:54 -07:00
Alena Varkockova 87b2773930 Fix the validation of docker registry, improve the error message (#1780)
Signed-off-by: Alena Varkockova <varkockova.a@gmail.com>
2018-10-18 15:36:28 -07:00
Alex Leong 43c22fe967
Implement getProfiles method in destination service (#1759)
We implement the getProfiles method in the destination service. This method returns a stream of destination profiles for a given authority. It does this by looking up the ServiceProfile resource in the controller namespace named `<svc>.<ns>` where `<svc>` is the name of the service and `<ns>` is the namespace of the service.

This PR includes:
* Adding a ServiceProfile Custom Resource Definition to linkerd install
* A watch based implementation of the getProfiles method in the destination service, similar to the implementation of get.
* An update to the destination client script that allows querying the getProfiles method.

Signed-off-by: Alex Leong <alex@buoyant.io>
2018-10-16 15:39:12 -07:00
Kevin Lingerfelt e9874b9c3e
Improve docker layer caching for web image (#1757)
* Improve docker layer caching for web image
* Move all web files to /linkerd dir

Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-10-16 11:10:52 -07:00
Alejandro Pedraza 37bc8a69db Added support for json output in `linkerd stat` (#1749)
Added support for json output in `linkerd stat` through a new (-o|--output)=json option.

Fixes #1417

Signed-off-by: Alejandro Pedraza <alejandro.pedraza@gmail.com>
2018-10-15 14:10:48 -07:00
Alejandro Pedraza 2d6fde274c Make room for columns in `linkerd top` (#1750)
* Make room for columns in `linkerd top`

Make room for columns in `linkerd top`. Columns with data longer than some predetermined minimum length were stepping over each other.

Proposal for #1728

* Removed unneeded truncations

Signed-off-by: Alejandro Pedraza <alejandro.pedraza@gmail.com>
2018-10-11 13:08:46 -07:00
Alex Leong f1f5b49f59
Add generated Kubernetes client for ServiceProfile custom resource (#1752)
To support reading and writing of the ServiceProfile custom resource, we add a codegen'd Kubernetes client for this resource.

* Adding the ServiceProfile type and related boilerplate to /controller/gen/apis/serviceprofile. This boilerplate also contains directives that control how codegen works.
* A script in /hack which invokes codegen that generates Kubernetes client machinery for interacting with ServiceProfile resources. The majority of the generated code lives in /controller/gen/client.
* The above-mentioned generated code.

Signed-off-by: Alex Leong <alex@buoyant.io>
2018-10-11 11:43:35 -07:00
Kevin Lingerfelt 46c887ca00
Add --single-namespace install flag for restricted permissions (#1721)
* Add --single-namespace install flag for restricted permissions
* Better formatting in install template
* Mark --single-namespace and --proxy-auto-inject as experimental
* Fix wording of --single-namespace check flag
* Small healthcheck refactor

Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-10-11 10:55:57 -07:00
Ivan Sim 4fba6aca0a Proxy init and sidecar containers auto-injection (#1714)
* Support auto sidecar-injection

1. Add proxy-injector deployment spec to cli/install/template.go
2. Inject the Linkerd CA bundle into the MutatingWebhookConfiguration
during the webhook's start-up process.
3. Add a new handler to the CA controller to create a new secret for the
webhook when a new MutatingWebhookConfiguration is created.
4. Declare a config map to store the proxy and proxy-init container
specs used during the auto-inject process.
5. Ignore namespace and pods that are labeled with
linkerd.io/auto-inject: disabled or linkerd.io/auto-inject: completed
6. Add new flag to `linkerd install` to enable/disable proxy
auto-injection

Proposed implementation for #561.

* Resolve missing packages errors
* Move the auto-inject label to the pod level
* PR review items
* Move proxy-injector to its own deployment
* Ignore pods that already have proxy injected

This ensures the webhook doesn't error out due to proxy that are injected using the  command

* PR review items on creating/updating the MWC on-start
* Replace API calls to ConfigMap with file reads
* Fixed post-rebase broken tests
* Don't mutate the auto-inject label

Since we started using healhcheck.HasExistingSidecars() to ensure pods with
existing proxies aren't mutated, we don't need to use the auto-inject label as
an indicator.

This resolves a bug which happens with the kubectl run command where the deployment
is also assigned the auto-inject label. The mutation causes the pod auto-inject
label to not match the deployment label, causing kubectl run to fail.

* Tidy up unit tests
* Include proxy resource requests in sidecar config map
* Fixes to broken YAML in CLI install config

The ignore inbound and outbound ports are changed to string type to
avoid broken YAML caused by the string conversion in the uint slice.

Also, parameterized the proxy bind timeout option in template.go.

Renamed the sidecar config map to
'linkerd-proxy-injector-webhook-config'.

Signed-off-by: ihcsim <ihcsim@gmail.com>
2018-10-10 12:09:22 -07:00
Darko Radisic 6fee0f3c2b Added --context flag to specify the context to use to talk to the Kubernetes apiserver (#1743)
* Added --context flag to specify the context to use to talk to the Kubernetes apiserver
* Fix tests that are failing
* Updated context flag description

Signed-off-by: Darko Radisic <ffd2subroutine@users.noreply.github.com>
2018-10-08 12:37:35 -07:00
Ben Lambert 69cebae1a2 Added ability to configure sidecar CPU + Memory requests (#1731)
Horizontal Pod Autoscaling does not work when container definitions in pods do not all have resource requests, so here's the ability to add CPU + Memory requests to install + inject commands by proving proxy options --proxy-cpu + --proxy-memory

Fixes #1480

Signed-off-by: Ben Lambert <ben@blam.sh>
2018-10-08 10:51:29 -07:00
Oliver Gould eaec37c64f
cli: Use updated proxy config environment vars
In linkerd/linkerd2-proxy#99, several proxy configuration variables were
deprecated.

This change updates the CLI to use the updated names to avoid
deprecation warnings during startup.
2018-10-03 11:15:39 -07:00
Andrew Seigner dccccebd79
Add LICENSE files to all Docker images (#1727)
To comply with certain environments, include our LICENSE file in all
Docker images.


Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2018-10-02 16:25:52 -07:00
Risha Mars 427078844d
Add HTTP method to Top display in the CLI and Web UI (#1709)
* Add Method column to Top tables in Web UI

* Add method to CLI top table
2018-09-25 11:15:11 -07:00
Rodrigo Chacon 783bb1c3a7 cli: add support for LINKERD_NAMESPACE environment variable (#1695)
Signed-off-by: Rodrigo Chacon <rochacon@gmail.com>
2018-09-21 17:24:10 -07:00
Alena Varkockova 8ab9b4981b Make wait flag configurable for check and dashboard (#1654)
Signed-off-by: Alena Varkockova <varkockova.a@gmail.com>
2018-09-19 10:42:29 -07:00
Alex Leong e65a9617bd
Add can-i checks to linkerd check --pre (#1644)
Add checks to `linkerd check --pre` to verify that the user has permission to create:
* namespaces
* serviceaccounts
* clusterroles
* clusterrolebindings
* services
* deployments
* configmaps

Signed-off-by: Alex Leong <alex@buoyant.io>
2018-09-17 11:31:10 -07:00
Kevin Lingerfelt f1b3827194
Bump default check retry time to 5 minutes (#1645)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-09-14 10:58:03 -07:00
Alena Varkockova 169dcf4e70 Make wait=true a default option for check and dashboard (#1640)
* Remove wait option and make it a default for check
* Switch the wait default to true
* Wait by default also for dashboard

Signed-off-by: Alena Varkockova <varkockova.a@gmail.com>
2018-09-14 09:59:04 -07:00
Andrew Seigner 5d85680ec1
Introduce inject check for known sidecars (#1619)
`linkerd inject` was not checking its input for known sidecars and
initContainers.

Modify `linkerd inject` to check for existing sidecars and
initContainers, specifically, Linkerd, Istio, and Contour.

Part of #1516

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2018-09-11 15:09:19 -07:00
Andrew Seigner bae05410fd
Bump Prometheus to v2.4.0, Grafana to 5.2.4 (#1625)
Prometheus v2.3.1 -> v2.4.0
Grafana 5.1.3 -> 5.2.4

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2018-09-11 14:45:55 -07:00
Alex Leong bd15482329
Add with-source flag to top (#1614)
Fixes #1593 

Add a `--hide-sources` flag to `linkerd top`.  Setting this removes the source column from the output.

Signed-off-by: Alex Leong <alex@buoyant.io>
2018-09-11 14:21:36 -07:00
Andrew Seigner 7eec5f181d
Inject warns on UDP ports (#1617)
linkerd only routes TCP data, but `linkerd inject` does not warn when it
injects into pods with ports set to `protocol: UDP`.

Modify `linkerd inject` to warn when injected into a pod with
`protocol: UDP`. The Linkerd sidecar will still be injected, but the
stderr output will include a warning.

Also add stderr checking on all inject unit tests.

Part of #1516.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2018-09-11 10:12:45 -07:00
Andrew Seigner c5a719da47
Modify inject to warn when file is un-injectable (#1603)
If an input file is un-injectable, existing inject behavior is to simply
output a copy of the input.

Introduce a report, printed to stderr, that communicates the end state
of the inject command. Currently this includes checking for hostNetwork
and unsupported resources.

Malformed YAML documents will continue to cause no YAML output, and return
error code 1.

This change also modifies integration tests to handle stdout and stderr separately.

example outputs...

some pods injected, none with host networking:

```
hostNetwork: pods do not use host networking...............................[ok]
supported: at least one resource injected..................................[ok]

Summary: 4 of 8 YAML document(s) injected
  deploy/emoji
  deploy/voting
  deploy/web
  deploy/vote-bot
```

some pods injected, one host networking:

```
hostNetwork: pods do not use host networking...............................[warn] -- deploy/vote-bot uses "hostNetwork: true"
supported: at least one resource injected..................................[ok]

Summary: 3 of 8 YAML document(s) injected
  deploy/emoji
  deploy/voting
  deploy/web
```

no pods injected:

```
hostNetwork: pods do not use host networking...............................[warn] -- deploy/emoji, deploy/voting, deploy/web, deploy/vote-bot use "hostNetwork: true"
supported: at least one resource injected..................................[warn] -- no supported objects found

Summary: 0 of 8 YAML document(s) injected
```

TODO: check for UDP and other init containers

Part of #1516

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2018-09-10 10:34:25 -07:00
Kevin Lingerfelt f884caf56d
Upgrade protobuf to v1.2.0 (#1591)
* Upgrade protobuf to v1.2.0
* Fix Gopkg.lock
* Switch linkerd2-proxy-api dep back to stable

Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-09-06 11:36:29 -07:00
Kevin Lingerfelt b5ff29c8aa
Add data plane check to validate proxy version (#1574)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-09-04 15:22:38 -07:00
Kevin Lingerfelt c7a79da89c
Add data plane check to validate proxies are ready (#1570)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-08-31 15:51:57 -07:00
Alex Leong 0f7d684ca9
Increase default max-rps for tap and top (#1531)
The default value for the max-rps argument to the tap and top commands is an overly conservative 1rps.  This causes the data to come in very slowly and much data to be discarded.  Furthermore, because tap requests are windowed to 10 seconds, this causes long pauses between updates.

We fix this in two ways.  Firstly we reduce the window size to 1s so that updates will come in at least once per second, even when the actual RPS of the data path is extremely high.  Secondly, we increase the default max-rps parameter from 1 to 100.  This allows tap to paint an accurate picture of the data much more quickly and sidesteps some sampling bias that happens when the max-rps is low.

In general, tap events tend to happen in bursts.  For example, one request in may trigger one or more requests out.  Likewise, a single upstream event may trigger several requests to the tapped pod in quick succession.  Sampling bias will occur when the max-rps is less than the actual rps and when the tap event limit subdivides these event bursts (biasing towards the first few events in the burst).  The greater the max-rps, the less the effects of this bias.

Fixes #1525 

Signed-off-by: Alex Leong <alex@buoyant.io>
2018-08-28 14:16:39 -07:00
Risha Mars 136b9cc7c1
Add linkerd check flag to run data plane checks (#1528)
Adds a --proxy flag to the linkerd check CLI command which will run 
to-be-implemented data plane checks
2018-08-28 10:16:24 -07:00
Risha Mars fff09c5d06
Only tap pods that are meshed (#1535)
Previously, we would tap any resource's pods, regardless of whether the pods
were meshed or not. We can't actually tap non-meshed pods, so I'm adding a check
that will filter out non-meshed pods from the pods that tap watches.

Previous behaviour:
When attempting to hang a non meshed pod, it would establish
a watch on the pods, but then never return any results. In the CLI you could
just cancel it with Ctrl-C. In the web, clicking Stop would send a
WebSocket.close(1000) but wouldn't actually close the connection... 

Behaviour after change :
If no pods under the specified resource are meshed, it'll
return an error of no pods being found to tap
2018-08-28 09:59:52 -07:00
Risha Mars 27e52a6cc0
Add ReadinessProbe and LivenessProbe to injected proxy containers (#1530)
Adds basic probes to the linkerd-proxy containers injected by linkerd inject.

- Currently the Readiness and Liveness probes are configured to be the same. 
- I haven't supplied a periodSeconds, but the default is 10.
- I also set the initialDelaySeconds to 10, but that might be a bit high.

https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/
2018-08-27 11:55:17 -07:00
Kevin Lingerfelt 4450a7536d
Add --wait flag for CLI check and dashboard commands (#1503)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-08-22 12:56:42 -07:00
Kevin Lingerfelt 49f6c4c770
Refactor healthcheck init and observe setup (#1502)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-08-22 12:30:45 -07:00
Kevin Lingerfelt 53cd3b50d5
Add --pre flag for linkerd check command (#1497)
* Add --pre flag for linkerd check command
* Small adjustments to check help text

Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-08-20 17:09:43 -07:00
Kevin Lingerfelt e97be1f5da
Move all healthcheck-related code to pkg/healthcheck (#1492)
* Move all healthcheck-related code to pkg/healthcheck
* Fix failed check formatting
* Better version check wording

Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-08-20 16:50:22 -07:00
Eliza Weisman b8434d60d4
Add resource metadata to Tap CLI output (#1437)
Closes #1170.

This branch adds a `-o wide` (or `--output wide`) flag to the Tap CLI.
Passing this flag adds `src_res` and `dst_res` elements to the Tap
output, as described in #1170. These use the metadata labels in the tap
event to describe what Kubernetes resource the source and destination
peers belong to, based on what resource type is being tapped, and fall
back to pods if either peer is not a member of the specified resource
type.

In addition, when the resource type is not `namespace`, `src_ns` and
`dst_ns` elements are added, which show what namespaces the the source
and destination peers are in. For peers which are not in the Kubernetes
cluster, none of these labels are displayed.

The source metadata added in #1434 is used to populate the `src_res` and
`src_ns` fields.

Also, this branch includes some refactoring to how tap output is
formatted.

Signed-off-by: Eliza Weisman <eliza@buoyant.io>
2018-08-20 14:25:26 -07:00
Kevin Lingerfelt 7c07ba0d53
Upgrade to dep 0.5.0, go 1.10.3 (#1479)
* Upgrade to dep 0.5.0, go 1.10.3
* Remove existing dep binary if it's the wrong version
* Add version in filename of dep binary to prevent version conflicts

Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-08-17 16:04:50 -07:00
Alex Leong 094a375015
[RFC] linkerd top (#1435)
This an initial implementation of the `linkerd top` command.  This command launches an ncurses style tabular view of current requests (using data from tap).  Most of the command line arguments are the same as tap and allow selecting the resource to inspect and filtering which requests to view.  

Fixes #1283 

Signed-off-by: Alex Leong <alex@buoyant.io>
2018-08-15 18:10:23 -07:00
Kevin Lingerfelt 00a0572098
Better CLI error messages when control plane is unavailable (#1428)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-08-09 15:40:41 -07:00
Eliza Weisman 9d8f58cb16
Add additional validation for stat command-line arguments (#1415)
Closes #776.

This branch adds the following validation to the `linkerd stat` command:

* The `--to` and `--from` flags are now mutually exclusive
* The `--to-namespace` and `--from-namespace` commands are also mutually
  exclusive.
* The `namespace` resource type conflicts with the `--namespace`, 
  `--to-namespace`, and `--from-namespace` flags.

Examples:

```
$ bin/go-run cli/main.go stat deploy --to deploy/foo --from deploy/bar
Error: --to and --from flags are mutually exclusive
Usage:
  linkerd stat [flags] (RESOURCE)
  ...
```

```
$ bin/go-run cli/main.go stat deploy --to-namespace foo --from-namespace bar
Error: --to-namespace and --from-namespace flags are mutually exclusive
Usage:
  linkerd stat [flags] (RESOURCE)
  ...
```

```
$ bin/go-run cli/main.go stat namespace foo --namespace bar
Error: --namespace flag is incompatible with namespace resource type
Usage:
  linkerd stat [flags] (RESOURCE)
  ...
```

```
$ bin/go-run cli/main.go stat ns --to-namespace bar
Error: --to-namespace flag is incompatible with namespace resource type
Usage:
  linkerd stat [flags] (RESOURCE)
  ...
```

```
$ bin/go-run cli/main.go stat namespace --from-namespace bar
Error: --from-namespace flag is incompatible with namespace resource type
Usage:
  linkerd stat [flags] (RESOURCE)
  ...
```

```
$ bin/go-run cli/main.go stat ns/foo --from-namespace bar
Error: --from-namespace flag is incompatible with namespace resource type
Usage:
  linkerd stat [flags] (RESOURCE)
  ...
```

Signed-off-by: Eliza Weisman <eliza@buoyant.io>
2018-08-08 15:35:47 -07:00
Kevin Lingerfelt 82940990e9
Rename mailing lists, remove all remaining conduit references (#1416)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-08-07 17:00:55 -07:00
Kevin Lingerfelt 4845b4ec04
Restore linkerd.io/control-plane* labels (#1411)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-08-07 13:53:29 -07:00
Kevin Lingerfelt e0a01c5dd8
Remove node scrape target, kubernetes grafana dashboard (#1410)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-08-07 13:41:38 -07:00
Kevin Lingerfelt bd19e8aaff
Update prometheus to only scrape proxies in the same mesh (#1402)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-08-06 12:05:55 -07:00
Kevin Lingerfelt f70ad7de11
Use stable version for linkerd2-proxy-api dep (#1400)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-08-03 11:59:42 -07:00
Sean McArthur c035193313
add H2 protocol to destination addrs if managed by linkerd (#1380)
Signed-off-by: Sean McArthur <sean@buoyant.io>
2018-08-03 10:14:30 -07:00
Eliza Weisman 01cc30d102
Increase outbound router capacity for Prometheus pod's proxy (#1358)
Currently, when a cluster has over 100 pods injected with the Linkerd2
proxy, Prometheus metrics are not collected correctly. This is because
Prometheus appears to be making more concurrent requests than its'
proxy's outbound router cache can handle See issue #1322 for further 
details.

This branch introduces a workaround for this issue, by increasing the
outbound router cache capacity to 10000 routes for the Prometheus pod's
proxy only. The router capacity limit of 100 active routes is primarily
due to the limitation of the number of active Destination service 
lookups, so increasing the capacity for the Prometheus pod specifically
is probably okay, as the scrape requests are made to IP addresses 
directly and therefore will not cause service discovery lookups. 

This change was originally implemented and tested in @siggy's PR #1228.
I've rebased his branch onto the current `master`, and updated the code
to reflect the project name change.

Signed-off-by: Eliza Weisman <eliza@buoyant.io>
Co-authored-by: Andrew Seigner <siggy@buoyant.io>
2018-08-02 16:44:11 -07:00
Ivan Sim eb04217a12 Update inject cmd to read from folder (#1377)
This change is a simplified implementation of the Builder.Path() and
Visitor().ExpandPathsToFileVisitors() functions used by kubectl to parse files
and directories. The filepath.Walk() function is used to recursively traverse
directories. Every .yaml or .json resource file in the directory is read
into its own io.Reader. All the readers are then passed to the YAMLDecoder in the
InjectYAML() function.

Fixes #1376

Signed-off-by: ihcsim <ihcsim@gmail.com>
2018-08-01 17:12:00 -07:00
Kevin Lingerfelt 8fe9e53f67
Remove remaining conduit references in codebase (#1381)
* Remove remaining conduit references in codebase
* Shorten emojivoto config url

Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-07-31 11:19:34 -07:00
Kevin Lingerfelt c362d5e114
Update k8s.io dependencies to 1.11.1 (#1369)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-07-27 15:23:03 -07:00
Kevin Lingerfelt 51848230a0
Send glog logs to stderr by default (#1367)
* Send glog logs to stderr by default
* Factor out more shared flag parsing code

Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-07-25 12:59:24 -07:00
Risha Mars ec3c861743
Enable Tap from the Web UI (#1356)
Adds a tap endpoint in the web api that communicates with the dashboard 
via websockets.
I've moved a bunch of code from the cli tap.go into utils so that the code 
can be shared between web and CLI. I think we should consider making the 
display more suited to web, but in the short term, reusing the CLI's 
rendering of tap events works.

Adds a Tap page in the Web UI that you can use to make tap requests. 
The form currently only allows you to enter a resource and namespace, 
other filters coming in a follow-up branch.
2018-07-24 14:23:42 -04:00
Kevin Lingerfelt 4b9700933a
Update prometheus labels to match k8s resource names (#1355)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-07-23 15:45:05 -07:00
Brian Smith a98bfb1ca7
Rename `ca-bundle-distributor` to `ca`. (#1340)
`ca-bundle-distributor` described the original role of the program but
`ca` ("Certificate Authority") better describes its current role.

Signed-off-by: Brian Smith <brian@briansmith.org>
2018-07-17 14:10:40 -10:00
Brian Smith 1b38310019
Remove executable bit from non-executable files. (#1335)
These files were created with the executable bit set accidentally due
to the way my network file system setup was configured.

Signed-off-by: Brian Smith <brian@briansmith.org>
2018-07-16 13:55:52 -10:00
Brian Smith 0fcfd2bffb
Stop using `installsuffix` when building Go code. (#1327)
* Stop using `installsuffix` when building Go code.

See https://plus.google.com/117192131596509381660/posts/eNnNePihYnK.
`-installsuffix cgo` isn't necessary as of Go 1.10 (where build caching
changed substantially) and it probably wasn't necessary earlier.

Signed-off-by: Brian Smith <brian@briansmith.org>
2018-07-16 13:48:50 -10:00
Franziska von der Goltz c7ac072acc
update grafana dashboards: conduit to linkerd (#1320)
* update grafana dashboards to remove conduit reference and replace with linkerd instances
* update test install fixtures to reflect changes

Fixes: #1315

Signed-off-by: Franziska von der Goltz <franziska@vdgoltz.eu>
2018-07-16 13:05:01 -07:00
Kevin Lingerfelt e5cce1abaf
Rename CLI from conduit to linkerd (#1312)
* Rename CLI binary
* Update integration tests for new binary name
* Rename --conduit-namespace flag, change default ns
* Rename occurrences of conduit in rest of CLI
* Rename inject and install components
* Remove conduit occurrences in docker files
* Additional miscellaneous cleanup
* Move protobuf definitions to linkerd2 package
* Rename conduit.io labels to use linkerd.io
* Rename conduit-managed segment to linkerd-managed
* Fix conduit references in web project

Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-07-12 17:14:07 -07:00
Andrew Seigner e18fa48135
Name ClusterRole objects to be namespace-specific (#1295)
The control-plane's `ClusterRole` and `ClusterRoleBinding` objects are
global. Because their names did not vary across multiple control-plane
deployments, it prevented multiple control-planes from coexisting (when
RBAC is enabled).

Modify the `ClusterRole` and `ClusterRoleBinding` objects to include the
control-plane's namespace in their names. Also modify the integration
test to first install two control-planes, and then perform its full
suite of tests, to prevent regression.

Fixes #1292.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2018-07-10 16:21:20 -07:00
Oliver Gould 941cad4a9c
Migrate build infrastructure to linkerd2 (#1298)
This PR begins to migrate Conduit to Linkerd2:
* The proxy has been completely removed from this repo, and is now located at
  github.com/linkerd/linkerd2-proxy.
* A `Dockerfile-proxy` has been added to fetch the most-recently published proxy
  binary from build.l5d.io.
* Proxy-specific protobuf bindings have been moved to
  github.com/linkerd/linkerd2-proxy-api.
* All docker images now use the gcr.io/linkerd-io registry.
* `inject` now uses `LINKERD2_PROXY_` environment variables
* Go paths have been updated to reflect the new (future) repo location.
2018-07-09 15:38:38 -07:00
Kevin Lingerfelt fd1aecfa63
Unhide --tls flag in conduit CLI (#1278)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-07-05 15:49:19 -07:00
Kevin Lingerfelt 693acdbf26
Update ListPods endpoint to return all pod owner types (#1275)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-07-05 15:14:16 -07:00
Kevin Lingerfelt f0ba8f3ee8
Fix owner types in TLS identity strings (#1257)
* Fix owner types in TLS identity strings
* Update documentation on TLSIdentity struct

Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-07-03 14:20:24 -07:00
Risha Mars 83b982b25a
Change CLI and web TLS indicators from Secured to TLS (#1247)
Previously, we had "Secured" columns in the web and CLI for the percentage of traffic 
that is TLSed. Change this to "TLS"
2018-07-03 10:51:38 -07:00
Brian Smith 252a8d39d3
Generate an ephemeral CA at startup that distributes TLS credentials (#1245)
Create a ephemeral, in-memory TLS certificate authority and integrate it into the certificate distributor.

Remove the re-creation of deleted ConfigMaps; this will be added back later in #1248.

Signed-off-by: Brian Smith brian@briansmith.org
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-07-02 18:09:31 -10:00
Oliver Gould 20276b106e
tap: Support `tls` labeling (#1244)
The proxy's metrics are instrumented with a `tls` label that describes
the state of TLS for each connection and associated messges.

This same level of detail is useful to get in `tap` output as well.

This change updates Tap in the following ways:
* `TapEvent` protobuf updated:
  * Added `source_meta` field including source labels
  * `proxy_direction` enum indicates which proxy server was used.
* The proxy adds a `tls` label to both source and destination meta indicating the state of each peer's connection
* The CLI uses the `proxy_direction` field to determine which `tls` label should be rendered.
2018-07-02 17:19:20 -07:00
Kevin Lingerfelt a685dba873
Use parent name instead of pod name in identity string (#1236)
* Use parent name instead of pod name in identity string
* Update protobuf comment

Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-06-29 14:28:13 -07:00
Brian Smith f989c56127
Proxy: Skip TLS for control plane loopback connections. (#1229)
If the controller address has a loopback host then don't use TLS to connect
to it. TLS isn't needed for security in that case. In mormal configurations
the proxy isn't terminating TLS for loopback connections anyway.

Signed-off-by: Brian Smith <brian@briansmith.org>
2018-06-28 17:24:09 -10:00
Risha Mars 5ed7fc563c
Add controller component pod uptimes to the ServiceMesh page (#1205)
- Return pod uptimes from the GetPods endpoint
- Adds filtering by namespace to api.GetPods
- Adds a --namespace filter to conduit get pods
- Adds pod uptimes to the controller component toolitps on the ServiceMesh page
- Moves the ServiceMesh page back to using /api/pods
2018-06-28 15:42:00 -07:00
Risha Mars 68586fe697
Add the ability to query stats by authority (#1181)
Adds the ability to query by a new non-kubernetes resource type, "authorities",
in the StatSummary api.

This includes an extensive refactor of stat_summary.go to deal with non-kubernetes 
resource types.

- Add documentation to Resource in the public api so we can use it for authority
- Handle non-k8s resource requests in the StatSummary endpoint
- Rewrite stat summary fetching and parsing to handle non-k8s resources
- keys stat summary metric handling by Resource instead of a generated string
- Adds authority to the CLI
- Adds /authorities to the Web UI
- Adds some more stat integration and unit tests
2018-06-28 14:31:44 -07:00
Kevin Lingerfelt ef9c890505 Fix issue with injected resource name, add test (#1226)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-06-28 10:23:38 -10:00