Commit Graph

186 Commits

Author SHA1 Message Date
Andrew Seigner 81790b6735 Bump Prometheus to v2.10.0 (#2979)
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-06-21 12:51:31 -07:00
Tarun Pothulapati a3ce06bd80 Add sideEffects field to Webhooks (#2963)
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2019-06-21 11:06:10 -07:00
Ivan Sim 435fe861d0
Label all Linkerd resources (#2971)
Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-06-20 09:44:30 -07:00
Ivan Sim e2e976cce9
Add `NET_RAW` capability to the proxy-init container (#2969)
Also, update control plane PSP to match linkerd/website#94

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-06-19 19:34:37 -07:00
Dennis Adjei-Baah 694ba9c2cb
Revert add namespace name to MWC (#2946)
* revert add namespace name to MWC
2019-06-14 15:26:34 -07:00
Alejandro Pedraza 7fc6c195ad
Set MWC and VWC failure policy to 'fail' in HA mode only (#2943)
Fixes #2927

Also moved `TestInstallSP` after `TestCheckPostInstall` so we're sure
the validating webhook is ready before installing a service profile.

Signed-off-by: Alejandro Pedraza Borrero <alejandro@buoyant.io>
2019-06-14 11:50:59 -05:00
Alejandro Pedraza 28025eeb56
Remove UPDATE event from the mutating webhook config (#2919)
Fixes #2889

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-06-13 15:42:47 -05:00
Alejandro Pedraza e9bf014d34
Remove MWVC RBAC from webhook configs (#2925)
Fixes #2890

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-06-13 15:42:00 -05:00
Dennis Adjei-Baah 8aef9280dd
add namespace name to MWC (#2905)
When installing multiple control planes, the mutatingwebhookconfiguration of the first control plane gets overwritten by any subsequent control plane install. This is caused by the fixed name given to the mutatingwebhookconfiguration manifest at install time.

This commit adds in the namespace to the manifest so that there is a unique configuration for each control plane.

Fixes #2887
2019-06-13 12:15:43 -07:00
Ivan Sim ecc4465cd1
Introduce Control Plane's PSP and RBAC resources into Helm templates (#2920)
* Add control plane and CNI PSP and RBAC resources
* Add the '--linkerd-cni-enabled' flag to the multi-stage install subcommands

This flag ensures that the NET_ADMIN capability is omitted from the control
plane's PSP during 'install config' and the proxy-init containers aren't
injected during 'install control-plane'.

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-06-12 20:18:46 -07:00
Alejandro Pedraza 8416d326c2
If HA, set the webhooks failure policy to 'Fail' (#2906)
* If HA, set the webhooks failure policy to 'Fail'

I'm adding to the linkerd namespace a new label
`linkerd.io/is-control-plane: true` that is used in the webhook configs'
selector to skip the proxy injector for this namespace. This avoids
running into the timing issues described in #2852.

Fixes #2852

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-06-11 13:11:54 -05:00
Cody Vandermyn 33de3574ee Correctly set securityContext values on injection (#2911)
The patch provided by @ihcsim applies correct values for the securityContext during injection, namely: `allowPrivilegeEscalation = false`, `readOnlyRootFilesystem = true`, and the capabilities are copied from the primary container. Additionally, the proxy-init container securityContext has been updated with appropriate values.

Signed-off-by: Cody Vandermyn <cody.vandermyn@nordstrom.com>
2019-06-11 10:34:30 -07:00
Alejandro Pedraza 74ca92ea25
Split proxy-init into separate repo (#2824)
Split proxy-init into separate repo

Fixes #2563

The new repo is https://github.com/linkerd/linkerd2-proxy-init, and I
tagged the latest there `v1.0.0`.

Here, I've removed the `/proxy-init` dir and pinned the injected
proxy-init version to `v1.0.0` in the injector code and tests.

`/cni-plugin` depends on proxy-init, so I updated the import paths
there, and could verify CNI is still working (there is some flakiness
but unrelated to this PR).

For consistency, I added a `--init-image-version` flag to `linkerd
inject` along with its corresponding override config annotation.

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-06-03 16:24:05 -05:00
Tarun Pothulapati 1a574def1f Added labels to webhook configurations in charts/ (#2853)
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2019-05-28 16:44:09 -05:00
Dennis Adjei-Baah b9cc66c6c8
fixes roleRef name in linkerd-tap rbac (#2845)
* fixes roleRef name in linkerd-tap rbac
2019-05-28 10:05:01 -07:00
Ivan Sim 5a5f8bbfe8
Install MWC and VWC During Installation (#2806)
* Update helm charts to include webhooks config and TLS secret
* Update the webhooks to read the secret cert and key
* Update webhooks to not recreate config on restart
* Ensure upgrade preserve existing secrets
* Revert the change to rename the webhook configs

The renaming change breaks upgrade, where the new webhook configs conflict with
the existing ones. The older resources  aren't deleted during upgrade because
they are dynamically created.

* Make the secret volume read-only
* Remove unnecessary exported getter functions
* Remove obsolete mwc and vwc templates

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-05-20 12:43:50 -07:00
Dennis Adjei-Baah a0fa1dff59
Move tap service into its own pod. (#2773)
* Split tap into its own pod in the control plane

Signed-off-by: Dennis Adjei-Baah <dennis@buoyant.io>
2019-05-15 16:28:44 -05:00
Ivan Sim 714035fee9
Define default resource spec for proxy-init init container (#2763)
Fixes #2750 

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-04-29 11:41:05 -07:00
Andrew Seigner be60b37e93
Group Web and Grafana ServiceAccounts with RBAC (#2756)
All ServiceAccounts are intended to be grouped together with other RBAC
resources, particularly for `linkerd install config` output. Grafana and
Web ServiceAccounts were still included with their respective
Deployments.

Group Grafana and Web ServiceAccounts with other RBAC resources.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-04-25 17:33:05 -07:00
Andrew Seigner 15ffd86cf1
Introduce multi-stage upgrade (#2723)
`linkerd install` supports a 2-stage install process, `linkerd upgrade`
did not.

Add 2-stage support for `linkerd upgrade`. Also exercise multi-stage
functionality during upgrade integration tests.

Part of #2337

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-04-25 14:29:52 -07:00
Alex Leong 4ea7c62b0d
Revert " Remove validation from service profile CRD definition (#2740)" (#2752)
This reverts commit 3de16d47be.

#2740 modified the ServiceProfiles CRD which will cause issues for users upgrading from the old CRD version to the new version.  #2748 was an attempt to fix this by bumping the service profile CRD version, however, our testing infrastructure is not well set up to accommodate changes to CRDs because they are resources which are global to the cluster.  

We revert this change for now and will revisit it in the future when we can give more thought to CRD versioning, upgrade, and testing.

Signed-off-by: Alex Leong <alex@buoyant.io>
2019-04-25 13:40:20 -07:00
Alejandro Pedraza 53bb7c47f6
Make the auto-injector required and removed proxy-auto-inject flag (#2733)
Make the auto-injector required and removed proxy-auto-inject flag

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-04-24 13:06:51 -05:00
Alex Leong 3de16d47be
Remove validation from service profile CRD definition (#2740)
Fixes #2736

Signed-off-by: Alex Leong <alex@buoyant.io>
2019-04-23 16:10:50 -07:00
Andrew Seigner b2b4780430
Introduce install stages (#2719)
This change introduces two named parameters for `linkerd install`, split
by privilege:
- `linkerd install config`
  - Namespace
  - ClusterRoles
  - ClusterRoleBindings
  - CustomResourceDefinition
  - ServiceAccounts
- `linkerd install control-plane`
  - ConfigMaps
  - Secrets
  - Deployments
  - Services

Comprehensive `linkerd install` is still supported.

TODO:
- `linkerd check` support
- `linkerd upgrade` support
- integration tests

Part of #2337

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-04-23 14:52:34 -07:00
Ivan Sim 8d13084f94
Split the `linkerd-version` CLI flag into `control-plane-version` and `proxy-version` (#2702)
* The 'linkerd-version' CLI flag is renamed to 'control-plane-version'
* Add version field to proxy config
* Add the control plane version to the global config
* Unit test for init image version
* Use more specific control plane and proxy versions in unit tests

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-04-19 11:35:20 -07:00
Andrew Seigner 2d9e3686e2
Split out config objects from install templates (#2714)
This is an initial change to separate out config-specific k8s objects
from the control-plane components. The eventual goal will be rendering
these configs as the first stage of a multi-stage install.

Part of #2337

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-04-18 09:31:35 -07:00
Katerina 938d64a16f Web server updated to read the UUID from the linkerd-config ConfigMap. (#2603)
Signed-off-by: Kateryna Melnyk <kattymelnyk@gmail.com>
2019-04-08 12:56:00 -07:00
Alejandro Pedraza edb225069c
Add validation webhook for service profiles (#2623)
Add validation webhook for service profiles

Fixes #2075

Todo in a follow-up PRs: remove the SP check from the CLI check.

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-04-05 16:10:47 -05:00
Ivan Sim a80335ed51
Disable external profiles by default (#2594)
* Disable external profiles by default
* Rename the --disable-external-profiles flag to --enable-external-profiles

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-04-01 15:13:50 -07:00
Andrew Seigner e38ad7e9d1
Update Prometheus retention param (#2584)
`storage.tsdb.retention` is deprecated in favor of
`storage.tsdb.retention.time`.

Replace all occurrences.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-03-29 10:45:02 -07:00
Oliver Gould 655632191b
config: Store install parameters with global config (#2577)
When installing Linkerd, a user may override default settings, or may
explicitly configure defaults. Consider install options like `--ha
--controller-replicas=4` -- the `--ha` flag sets a new default value for
the controller-replicas, and then we override it.

When we later upgrade this cluster, how can we know how to configure the
cluster?

We could store EnableHA and ControllerReplicas configurations in the
config, but what if, in a later upgrade, the default value changes? How
can we know whether the user specified an override or just used the
default?

To solve this, we add an `Install` message into a new config.
This message includes (at least) the CLI flags used to invoke
install.

upgrade does not specify defaults for install/proxy-options fields and,
instead, uses the persisted install flags to populate default values,
before applying overrides from the upgrade invocation.

This change breaks the protobuf compatibility by altering the
`installation_uuid` field introduced in 9c442f6885.
Because this change was not yet released (even in an edge release), we
feel that it is safe to break.

Fixes https://github.com/linkerd/linkerd2/issues/2574
2019-03-29 10:04:20 -07:00
Oliver Gould 24222da13b
install: Create auto-inject configuration (#2562)
When reading a Linkerd configuration, we cannot determine whether
auto-inject should be configured.

This change adds auto-inject configuration to the global config
structure. Currently, this configuration is effectively boolean,
determined by the presence of an empty value (versus a null).
2019-03-26 15:28:54 -07:00
Oliver Gould 9c442f6885
Store install UUID in global config (#2561)
Currently, the install UUID is regenerated each time `install` is run.
When implementing cluster upgrades, it seems most appropriate to reuse
the prior UUID, rather than generate a new one.

To this end, this change stores an "Installation UUID" in the global
linkerd config.
2019-03-26 08:45:40 -07:00
Oliver Gould da0330743f
Provide peer Identities via the Destination API (#2537)
This change reintroduces identity hinting to the destination service.
The Get endpoint includes identities for pods that are injected with an
identity-mode of "default" and have the same linkerd control plane.

A `serviceaccount` label is now also added to destination response
metadata so that it's accessible in prometheus and tap.
2019-03-22 09:19:14 -07:00
Oliver Gould 34ea302a32
inject: Configure proxies to enable Identity (#2536)
This change adds a new `linkerd2-proxy-identity` binary to the `proxy`
container image as well as a `linkerd2-proxy-run` entrypoint script.

The inject process now sets environment variables on pods to support
identity, including identity names for the destination and identity
services.

As the proxy starts, the identity helper creates a key and CSR in a
tmpfs. As the proxy starts, it reads these files, as well as a
serviceaccount token, and provisions a certificate from controller.
The proxy's /ready endpoint will not succeed until a certificate has
been provisioned.

The proxy will not participate in identity with services other than the
controllers until the Destination controller is modified to provide
identities via discovery.
2019-03-21 18:39:05 -07:00
Oliver Gould 21796be354
install: Create linkerd-config before pods (#2538)
Because the linkerd-config resource is created after pods that require
it, they can be started before the files are mounted, causing the pods
to restart integration tests to fail.

If we extract the config into its own template file, it can be inserted
before pods are created.
2019-03-21 14:01:07 -07:00
Oliver Gould 0626fa374a
install: Introduce the Identity controller (#2526)
https://github.com/linkerd/linkerd2/pull/2521 introduces an "Identity"
controller, but there is no way to include it in linkerd installation.

This change alters the `install` flow as follows:
- An Identity service is _always_ installed;
- Issuer credentials may be specified via the CLI;
- If no Issuer credentials are provided, they are generated each time `install` is called.
- Proxies are NOT configured to use the identity service.
- It's possible to override the credential generation logic---especially
  for tests---via install options that can be configured via the CLI.
2019-03-19 17:04:11 -07:00
Oliver Gould 91c5f07650
proxy: Upgrade to identity-capable proxy (#2524)
The new proxy has changed its configuration as follows:

- `LISTENER` urls are now `LISTEN_ADDR` addresses;
- `CONTROL_URL` is now `DESTINATION_SVC_ADDR`;
- `*_NAMESPACE` vars are no longer needed;
- The `PROXY_ID` is now the `DESTINATION_CONTEXT`;
- The "metrics" port is now the "admin" port, since it serves more than
  just metrics;
- A readiness probe now checks a dedicated /ready endpoint eagerly.

Identity injection is **NOT** configured by this branch.
2019-03-19 14:20:39 -07:00
Oliver Gould 81f645da66
Remove `--tls=optional` and `linkerd-ca` (#2515)
The proxy's TLS implementation has changed to use a new _Identity_ controller.

In preparation for this, the `--tls=optional` CLI flag has been removed
from install and inject; and the `ca` controller has been deleted. Metrics
and UI treatments for TLS have **not** been removed, as they will continue to
be valuable for the new Identity system.

With the removal of the old identity scheme, the Destination service's proxy
ID field is now set with an opaque string (e.g. `ns:emojivoto`) to enable
locality awareness.
2019-03-18 17:40:31 -07:00
Gaurav Kumar d0bdd4ffb4 Allow configuration of Prometheus log level (#2484) (#2487)
Signed-off-by: Gaurav Kumar <gaurav.kumar9825@gmail.com>
2019-03-18 10:34:58 -07:00
Andrew Seigner e5d2460792
Remove single namespace functionality (#2474)
linkerd/linkerd2#1721 introduced a `--single-namespace` install flag,
enabling the control-plane to function within a single namespace. With
the introduction of ServiceProfiles, and upcoming identity changes, this
single namespace mode of operation is becoming less viable.

This change removes the `--single-namespace` install flag, and all
underlying support. The control-plane must have cluster-wide access to
operate.

A few related changes:
- Remove `--single-namespace` from `linkerd check`, this motivates
  combining some check categories, as we can always assume cluster-wide
  requirements.
- Simplify the `k8s.ResourceAuthz` API, as callers no longer need to
  make a decision based on cluster-wide vs. namespace-wide access.
  Components either have access, or they error out.
- Modify the web dashboard to always assume ServiceProfiles are enabled.

Reverts #1721
Part of #2337

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-03-12 00:17:22 -07:00
Alejandro Pedraza 0da851842b
Public API endpoint `Config()` (#2455)
Public API endpoint `Config()`

Retrieves Global and Proxy configurations.
Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-03-07 17:37:46 -05:00
Aditya Sharma 3740aa238a Remove `--api-port` flag from the cli (#2429)
* Changed the protobuf definition to take out destinationApiPort entirely
* Store destinationAPIPort as a constant in pkg/inject.go

Fixes #2351

Signed-off-by: Aditya Sharma <hello@adi.run>
2019-03-06 15:54:12 -08:00
Alejandro Pedraza ddf2e729ac
Injection consolidation (#2334)
- Created the pkg/inject package to hold the new injection shared lib.
- Extracted from `/cli/cmd/inject.go` and `/cli/cmd/inject_util.go`
the core methods doing the workload parsing and injection, and moved them into
`/pkg/inject/inject.go`. The CLI files should now deal only with
strictly CLI concerns, and applying the json patch returned by the new
lib.
- Proceeded analogously with `/cli/cmd/uninject.go` and
`/pkg/inject/uninject.go`.
- The `InjectReport` struct and helping methods were moved into
`/pkg/inject/report.go`
- Refactored webhook to use the new injection lib
- Removed linkerd-proxy-injector-sidecar-config ConfigMap
- Added the ability to add pod labels and annotations without having to
specify the already existing ones

Fixes #1748, #2289

Signed-off-by: Alejandro Pedraza <alejandro.pedraza@gmail.com>
2019-03-05 08:38:56 -05:00
Tarun Pothulapati 2184928813 Wire up stats for Jobs (#2416)
Support for Jobs in stat/tap/top cli commands

Part of #2007

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2019-03-01 17:16:54 -08:00
Andrew Seigner d08dcb0a37
Skip outbound port 443 in control-plane (#2411)
linkerd/linkerd2#2349 introduced a `SelfSubjectAccessReview` check at
startup, to determine whether each control-plane component should
establish Kubernetes watches cluster-wide or namespace-wide. If this
check occurs before the linkerd-proxy sidecar is ready, it fails, and
the control-plane component restarts.

This change configures each control-plane pod to skip outbound port 443
when injecting the proxy, allowing the control-plane to connect to
Kubernetes regardless of the `linkerd-proxy` state.

A longer-term fix should involve a more robust control-plane startup,
that is resilient to failed Kubernetes API requests. An even longer-term
fix could involve injecting `linkerd-proxy` as a Kubernetes "sidecar"
container, when that becomes available.

Workaround for #2407

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-27 15:23:19 -08:00
Kevin Lingerfelt 40076c4de2
Remove namespace from serviceprofile CRD in install config (#2409)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2019-02-27 14:29:47 -08:00
Oliver Gould ab90263461
destination: Only return TLS identities when appropriate (#2371)
As described in #2217, the controller returns TLS identities for results even
when the destination pod may not be able to participate in identity
requester: specifically, the other pod may not have the same controller
namespace or it may not be injected with identity.

This change introduces a new annotation, linkerd.io/identity-mode that is set
when injecting pods (via both CLI and webhook). This annotation is always
added.

The destination service now only returns TLS identities when this annotation
is set to optional on a pod and the destination pod uses the same controller.
These semantics are expected to change before the 2.3 release.

Fixes #2217
2019-02-27 12:18:39 -08:00
Andrew Seigner ec5a0ca8d9
Authorization-aware control-plane components (#2349)
The control-plane components relied on a `--single-namespace` param,
passed from `linkerd install` into each individual component, to
determine which namespaces they were authorized to access, and whether
to support ServiceProfiles. This command-line flag was redundant given
the authorization rules encoded in the parent `linkerd install` output,
via [Cluster]Role[Binding]s.

Modify the control-plane components to query Kubernetes at startup to
determine which namespaces they are authorized to access, and whether
ServiceProfile support is available. This allows removal of the
`--single-namespace` flag on the components.

Also update `bin/test-cleanup` to cleanup the ServiceProfile CRD.

TODO:
- Remove `--single-namespace` flag on `linkerd install`, part of #2164

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-26 11:54:52 -08:00
Ivan Sim 1e2e2bf53c
Install the Linkerd global and proxy config maps (#2344)
Also, some protobuf updates:

* Rename `api_port` to match recent changes in CLI code.
* Remove the `cni` message because it won't be used.
* Remove `registry` field from proto types. This helps to avoid having to workaround edge cases like fully-qualified image name in different format, and overriding user-specified Linkerd version etc.

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-02-22 15:28:21 -08:00
Oliver Gould 4ed84f0c0a
Split install template into component-specific files (#2313)
chart/templates/base.yaml is nearly 800 lines and contains the
kubernetes configurations for the marjority of the control plane.
Furthermore, its contents are not particularly organized (for example,
the prometheus RBAC bindings are in the middle of the controller's
configuration).

The size and complexity of this file makes it especially daunting to
introduce new functionality.

In order to make the situation easier to understand and change, this
splits base.yaml into several new template files: namespace, controller,
serviceprofile, and prometheus, and grafana. The `tls.yaml` template has
been renamed `ca.yaml`, since it installs the `linkerd-ca` resources.

This change also makes the comments uniform, adding a "header" to each
logical component.

Fixes #2154
2019-02-18 15:31:17 -08:00
Oliver Gould 71ce786dd3
Rename linkerd-proxy-api to linkerd-destination (#2281)
Up until now, the proxy-api controller service has been the sole service
that the proxy communicates with, implementing the majoriry of the API
defined in the `linkerd2-proxy-api` repo. But this is about to change:
linkerd/linkerd2-proxy-api#25 introduces a new Identity service; and
this service must be served outside of the existing proxy-api service
in the linkerd-controller deployment (so that it may run under a
distinct service account).

With this change, the "proxy-api" name becomes less descriptive. It's no
longer "the service that serves the API for the proxy," it's "the
service that serves the Destination API to the proxy." Therefore, it
seems best to bite the bullet and rename this to be the "destination"
service (i.e. because it only serves the
`io.linkerd.proxy.destination.Destination` service).

Co-authored-by: Kevin Lingerfelt <kl@buoyant.io>
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2019-02-15 15:11:04 -08:00
Andrew Seigner a9b9908908
Bump Prometheus to v2.7.1, Grafana to 5.4.3 (#2242)
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-02-13 11:27:15 -08:00
Ivan Sim f6e75ec83a
Add statefulsets to the dashboard and CLI (#2234)
Fixes #1983

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-02-08 15:37:44 -08:00
Alex Leong 5b054785e5
Read service profiles from client or server namespace instead of control namespace (#2200)
Fixes #2077 

When looking up service profiles, Linkerd always looks for the service profile objects in the Linkerd control namespace.  This is limiting because service owners who wish to create service profiles may not have write access to the Linkerd control namespace.

Instead, we have the control plane look for the service profile in both the client namespace (as read from the proxy's `proxy_id` field from the GetProfiles request and from the service's namespace.  If a service profile exists in both namespaces, the client namespace takes priority.  In this way, clients may override the behavior dictated by the service.

Signed-off-by: Alex Leong <alex@buoyant.io>
2019-02-07 14:51:43 -08:00
Andrew Seigner 907f01fba6
Improve ServiceProfile validation in linkerd check (#2218)
The `linkerd check` command was doing limited validation on
ServiceProfiles.

Make ServiceProfile validation more complete, specifically validate:
- types of all fields
- presence of required fields
- presence of unknown fields
- recursive fields

Also move all validation code into a new `Validate` function in the
profiles package.

Validation of field types and required fields is handled via
`yaml.UnmarshalStrict` in the `Validate` function. This motivated
migrating from github.com/ghodss/yaml to a fork, sigs.k8s.io/yaml.

Fixes #2190
2019-02-07 14:35:47 -08:00
Oliver Gould 44e31f0f67
Configure proxy keepalives via the environment (#2193)
In linkerd/linkerd2-proxy#186, the proxy supports configuration of TCP
keepalive values.

This change sets `LINKERD2_PROXY_INBOUND_ACCEPT_KEEPALIVE` and
`LINKERD2_PROXY_OUTBOUND_CONNECT_KEEPALIVE` to 10s when injecting the
proxy, so that remote connections are configured with a keepalive.

This configuration is NOT yet exposed through the CLI. This may be done
in a followup, if necessary.

Fixes #1949
2019-02-04 16:16:43 -08:00
Eliza Weisman 846975a190
Remove proxy bind timeout from CLIs (#2017)
This branch removes the `--proxy-bind-timeout` flag from the 
`linkerd inject` and `linkerd install` CLI commands, and the
`LINKERD2_PROXY_BIND_TIMEOUT` environment variable from their output.
This is in preparation for removing that timeout from the proxy (as
described in #2013). 

I thought it was prudent to remove this from the CLIs before removing it
from the proxy, so we can't create a situation where the CLIs produce
output that results in broken proxy containers.

Fixes #2013

Signed-off-by: Eliza Weisman <eliza@buoyant.io>
2019-01-24 15:34:09 -08:00
zak 8c413ca38b Wire up stats commands for daemonsets (#2006) (#2086)
DaemonSet stats are not currently shown in the cli stat command, web ui
or grafana dashboard. This commit adds daemonset support for stat.

Update stat command's help message to reference daemonsets.
Update the public-api to support stats for daemonsets.
Add tests for stat summary and api.

Add daemonset get/list/watch permissions to the linkerd-controller
cluster role that's created using the install command.
Update golden expectation test files for install command
yaml manifest output.

Update web UI with daemonsets
Update navigation, overview and pages to list daemonsets and the pods
associated to them.
Add daemonset paths to server, and ui apps.

Add grafana dashboard for daemonsets; a clone of the deployment
dashboard.

Update dependencies and dockerfile hashes

Add DaemonSet support to tap and top commands

Fixes of #2006

Signed-off-by: Zak Knill <zrjknill@gmail.com>
2019-01-24 14:34:13 -08:00
Andrew Seigner c9ac77cd7c
Introduce version consistency checks (#2130)
Version checks were not validating that the cli version matched the
control plane or data plane versions.

Add checks via the `linkerd check` command to validate the cli is
running the same version as the control and data plane.

Also add types around `channel-version` string parsing and matching. A
consequence being that during development `version.Version` changes from
`undefined` to `dev-undefined`.

Fixes #2076

Depends on linkerd/website#101

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-01-23 16:54:43 -08:00
Kevin Lingerfelt ed3fbd75f3
Setup port-forwarding for linkerd dashboard command (#2052)
* Setup port-forwarding for linkerd dashboard command
* Output port-forward logs when --verbose flag is set

Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2019-01-10 10:16:08 -08:00
Alena Varkockova 172398292d Add validation to CRD for service profiles (#2024)
* Add validation to CRD for service profiles

Signed-off-by: Alena Varkockova <varkockova.a@gmail.com>

* Use properties instead of oneof

Signed-off-by: Alena Varkockova <varkockova.a@gmail.com>
2019-01-10 07:04:40 -08:00
Kevin Lingerfelt a27bb2e0ce
Proxy grafana requests through web service (#2039)
* Proxy grafana requests through web service
* Fix -grafana-addr default, clarify -api-addr flag
* Fix version check in grafana dashboards
* Fix comment typo

Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2019-01-04 16:07:57 -08:00
Thomas Rampelberg 612ebe2b81
Add rules_file loading into prometheus (#1966) 2018-12-20 11:47:13 -08:00
Kevin Lingerfelt 0866bb2a41
Remove runAsGroup field from security context settings (#1986)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-12-13 15:12:13 -08:00
Kevin Lingerfelt 86e95b7ad3
Disable serivce profiles in single-namespace mode (#1980)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-12-13 14:37:18 -08:00
Cody Vandermyn d847f66ec5 Create new service accounts for linkerd-web and linkerd-grafana. Chan… (#1978)
* Create new service accounts for linkerd-web and linkerd-grafana. Change 'serviceAccount:' to 'serviceAccountName:'
* Use dynamic namespace name

Signed-off-by: Cody Vandermyn <cody.vandermyn@nordstrom.com>
2018-12-12 18:10:50 -08:00
Cody Vandermyn aa5e5f42eb Use an emptyDir for Prometheus and Grafana (#1971)
* Allow input of a volume name for prometheus and grafana
* Make Prometheus and Grafana volume names 'data' by default and disallow user editing via cli flags
* Remove volume name from options

Signed-off-by: Cody Vandermyn <cody.vandermyn@nordstrom.com>
2018-12-12 15:54:03 -08:00
Cody Vandermyn 8e4d9d2ef6 add securityContext with runAsUser: {{.ControllerUID}} to the various cont… (#1929)
* add securityContext with runAsUser: {{.ProxyUID}} to the various containers in the install template
* Update golden to reflect new additions
* changed to a different user id than the proxy user id
* Added a controller-uid install option
* change the port that the proxy-injector runs
* The initContainers needs to be run as the root user.
* move security contexts to container level

Signed-off-by: Cody Vandermyn <cody.vandermyn@nordstrom.com>
2018-12-11 11:51:28 -08:00
Alex Leong cbb196066f
Support service profiles for external authorities (#1928)
Add support for service profiles created on external (non-service) authorities.  For example, this allows you to create a service profile named `linkerd.io` which will apply to calls made to `linkerd.io`.

This is done by changing the `LINKERD2_PROXY_DESTINATION_PROFILE_SUFFIXES` to `.` so that the proxy will attempt to lookup a service profile for any authority.  We provide the `--disable-external-profiles` proxy flag to revert this behavior in case it is a problem.

We also refactor the proxy-api implementation of GetProfiles so that it does the profile lookup, regardless of if the authority looks like a Kubernetes service name or not.  To simplify this, support for multiple resolves (which was unused) was removed.

Signed-off-by: Alex Leong <alex@buoyant.io>
2018-12-05 14:32:59 -08:00
Oliver Gould 12ec5cf922
install: Add a -disable-h2-upgrade flag (#1926)
The proxy-api service _always_ suggests that two meshed pods communicate
via HTTP/2 (i.e. via transparent protocol upgrading, if necessary).
This can complicate debugging and diagnostics at times, so it's
important that we have a way to deploy linkerd without this auto-upgrade
behavior.

This change adds a `-disable-h2-upgrade` flag to the `linkerd install`
command that disables transparent upgrading for the whole cluster.
2018-12-05 12:50:47 -08:00
Kevin Lingerfelt 37ae423bb3
Add linkerd- prefix to all objects in linkerd install (#1920)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-12-04 15:41:47 -08:00
Andrew Seigner ad2366f208
Revert proxy readiness initialDelaySeconds change (#1912)
Reverts part of #1899 to workaround readiness failures.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2018-12-04 14:27:55 -08:00
Andrew Seigner 37a5455445
Add filtering by job in stat, tap, top; fix panic (#1904)
Filtering by Kubernetes job was not supported. Also filtering by any unknown
type caused a panic.

Add filtering support by Kubernetes job, with special case mapping `job` to
`k8s_job`, to not conflict with Prometheus' job label.

Fix panic when unknown type specified as a `--from` or `--to` flag.

Fix `job` label from `linkerd-proxy` overwriting Prometheus `job` label at
collection time. This caused all metrics collected by proxy sidecars in
Kubernetes jobs to be collected into an incorrect Prometheus job, rather than
the expected `linkerd-proxy` Prometheus job.

Fix `unsupported resource type` tap error message incorrectly printing the
target resource rather than the destination.

Set `--controller-log-level debug` in `install_test.go` for easier debugging.

Expose `slow-cooker`'s metrics via a k8s service in the tap integration test, to
validate proxy requests with a job as destination.

Fixes #1872
Part of #627

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2018-12-03 15:34:49 -08:00
Andrew Seigner d121071f87
Adjust proxy, Prometheus, and Grafana probes (#1899)
* Adjust proxy, Prometheus, and Grafana probes

High `readinessProbe.initialDelaySeconds` values delayed the controller's
readiness by up to 30s, preventing cli commands from succeeding shortly after
control plane deployment.

Decrease `readinessProbe.initialDelaySeconds` in the proxy, Prometheus, and
Grafana to the default 0s. Also change `linkerd check` controller pod ordering
to: controller, prometheus, web, grafana.

Detailed probe changes:
- proxy
  - decrease `readinessProbe.initialDelaySeconds` from 10s to 0s
- prometheus
  - decrease `readinessProbe.initialDelaySeconds` from 30s to 0s
  - decrease `readinessProbe.timeoutSeconds` from 30s to 1s
  - decrease `livenessProbe.timeoutSeconds` from 30s to 1s
- grafana
  - decrease `readinessProbe.initialDelaySeconds` from 30s to 0s
  - decrease `readinessProbe.timeoutSeconds` from 30s to 1s
  - decrease `readinessProbe.failureThreshold` from 10 to 3
  - increase `livenessProbe.initialDelaySeconds` from 0s to 30s

Fixes #1804

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2018-12-03 10:41:11 -08:00
Alex Leong d8b5ebaa6d
Remove the proxy-api container (#1813)
A container called `proxy-api` runs in the Linkerd2 controller pod.  This container listens on port 8086 and serves the proxy-api but does nothing other than forward gRPC requests to the destination container which listens on port 8089.

We remove the proxy-api container altogether and change the destination container to listen on port 8086 instead of 8089.  The result is that clients still use the proxy-api by connecting to `proxy-api.<ns>.svc.cluster.local:8086` but the controller has one fewer containers.  This results in a simpler system that is easier to reason about.

Signed-off-by: Alex Leong <alex@buoyant.io>
2018-10-29 16:31:43 -07:00
Alex Leong 43c22fe967
Implement getProfiles method in destination service (#1759)
We implement the getProfiles method in the destination service. This method returns a stream of destination profiles for a given authority. It does this by looking up the ServiceProfile resource in the controller namespace named `<svc>.<ns>` where `<svc>` is the name of the service and `<ns>` is the namespace of the service.

This PR includes:
* Adding a ServiceProfile Custom Resource Definition to linkerd install
* A watch based implementation of the getProfiles method in the destination service, similar to the implementation of get.
* An update to the destination client script that allows querying the getProfiles method.

Signed-off-by: Alex Leong <alex@buoyant.io>
2018-10-16 15:39:12 -07:00
Kevin Lingerfelt e9874b9c3e
Improve docker layer caching for web image (#1757)
* Improve docker layer caching for web image
* Move all web files to /linkerd dir

Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-10-16 11:10:52 -07:00
Kevin Lingerfelt 46c887ca00
Add --single-namespace install flag for restricted permissions (#1721)
* Add --single-namespace install flag for restricted permissions
* Better formatting in install template
* Mark --single-namespace and --proxy-auto-inject as experimental
* Fix wording of --single-namespace check flag
* Small healthcheck refactor

Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-10-11 10:55:57 -07:00
Oliver Gould eaec37c64f
cli: Use updated proxy config environment vars
In linkerd/linkerd2-proxy#99, several proxy configuration variables were
deprecated.

This change updates the CLI to use the updated names to avoid
deprecation warnings during startup.
2018-10-03 11:15:39 -07:00
Andrew Seigner bae05410fd
Bump Prometheus to v2.4.0, Grafana to 5.2.4 (#1625)
Prometheus v2.3.1 -> v2.4.0
Grafana 5.1.3 -> 5.2.4

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2018-09-11 14:45:55 -07:00
Risha Mars fff09c5d06
Only tap pods that are meshed (#1535)
Previously, we would tap any resource's pods, regardless of whether the pods
were meshed or not. We can't actually tap non-meshed pods, so I'm adding a check
that will filter out non-meshed pods from the pods that tap watches.

Previous behaviour:
When attempting to hang a non meshed pod, it would establish
a watch on the pods, but then never return any results. In the CLI you could
just cancel it with Ctrl-C. In the web, clicking Stop would send a
WebSocket.close(1000) but wouldn't actually close the connection... 

Behaviour after change :
If no pods under the specified resource are meshed, it'll
return an error of no pods being found to tap
2018-08-28 09:59:52 -07:00
Risha Mars 27e52a6cc0
Add ReadinessProbe and LivenessProbe to injected proxy containers (#1530)
Adds basic probes to the linkerd-proxy containers injected by linkerd inject.

- Currently the Readiness and Liveness probes are configured to be the same. 
- I haven't supplied a periodSeconds, but the default is 10.
- I also set the initialDelaySeconds to 10, but that might be a bit high.

https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/
2018-08-27 11:55:17 -07:00
Kevin Lingerfelt 4845b4ec04
Restore linkerd.io/control-plane* labels (#1411)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-08-07 13:53:29 -07:00
Kevin Lingerfelt e0a01c5dd8
Remove node scrape target, kubernetes grafana dashboard (#1410)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-08-07 13:41:38 -07:00
Kevin Lingerfelt bd19e8aaff
Update prometheus to only scrape proxies in the same mesh (#1402)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-08-06 12:05:55 -07:00
Eliza Weisman 01cc30d102
Increase outbound router capacity for Prometheus pod's proxy (#1358)
Currently, when a cluster has over 100 pods injected with the Linkerd2
proxy, Prometheus metrics are not collected correctly. This is because
Prometheus appears to be making more concurrent requests than its'
proxy's outbound router cache can handle See issue #1322 for further 
details.

This branch introduces a workaround for this issue, by increasing the
outbound router cache capacity to 10000 routes for the Prometheus pod's
proxy only. The router capacity limit of 100 active routes is primarily
due to the limitation of the number of active Destination service 
lookups, so increasing the capacity for the Prometheus pod specifically
is probably okay, as the scrape requests are made to IP addresses 
directly and therefore will not cause service discovery lookups. 

This change was originally implemented and tested in @siggy's PR #1228.
I've rebased his branch onto the current `master`, and updated the code
to reflect the project name change.

Signed-off-by: Eliza Weisman <eliza@buoyant.io>
Co-authored-by: Andrew Seigner <siggy@buoyant.io>
2018-08-02 16:44:11 -07:00
Kevin Lingerfelt 51848230a0
Send glog logs to stderr by default (#1367)
* Send glog logs to stderr by default
* Factor out more shared flag parsing code

Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-07-25 12:59:24 -07:00
Franziska von der Goltz c7ac072acc
update grafana dashboards: conduit to linkerd (#1320)
* update grafana dashboards to remove conduit reference and replace with linkerd instances
* update test install fixtures to reflect changes

Fixes: #1315

Signed-off-by: Franziska von der Goltz <franziska@vdgoltz.eu>
2018-07-16 13:05:01 -07:00
Kevin Lingerfelt e5cce1abaf
Rename CLI from conduit to linkerd (#1312)
* Rename CLI binary
* Update integration tests for new binary name
* Rename --conduit-namespace flag, change default ns
* Rename occurrences of conduit in rest of CLI
* Rename inject and install components
* Remove conduit occurrences in docker files
* Additional miscellaneous cleanup
* Move protobuf definitions to linkerd2 package
* Rename conduit.io labels to use linkerd.io
* Rename conduit-managed segment to linkerd-managed
* Fix conduit references in web project

Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-07-12 17:14:07 -07:00
Andrew Seigner e18fa48135
Name ClusterRole objects to be namespace-specific (#1295)
The control-plane's `ClusterRole` and `ClusterRoleBinding` objects are
global. Because their names did not vary across multiple control-plane
deployments, it prevented multiple control-planes from coexisting (when
RBAC is enabled).

Modify the `ClusterRole` and `ClusterRoleBinding` objects to include the
control-plane's namespace in their names. Also modify the integration
test to first install two control-planes, and then perform its full
suite of tests, to prevent regression.

Fixes #1292.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2018-07-10 16:21:20 -07:00
Oliver Gould 941cad4a9c
Migrate build infrastructure to linkerd2 (#1298)
This PR begins to migrate Conduit to Linkerd2:
* The proxy has been completely removed from this repo, and is now located at
  github.com/linkerd/linkerd2-proxy.
* A `Dockerfile-proxy` has been added to fetch the most-recently published proxy
  binary from build.l5d.io.
* Proxy-specific protobuf bindings have been moved to
  github.com/linkerd/linkerd2-proxy-api.
* All docker images now use the gcr.io/linkerd-io registry.
* `inject` now uses `LINKERD2_PROXY_` environment variables
* Go paths have been updated to reflect the new (future) repo location.
2018-07-09 15:38:38 -07:00
Brian Smith f989c56127
Proxy: Skip TLS for control plane loopback connections. (#1229)
If the controller address has a loopback host then don't use TLS to connect
to it. TLS isn't needed for security in that case. In mormal configurations
the proxy isn't terminating TLS for loopback connections anyway.

Signed-off-by: Brian Smith <brian@briansmith.org>
2018-06-28 17:24:09 -10:00
Kevin Lingerfelt 12f869e7fc
Add CA certificate bundle distributor to conduit install (#675)
* Add CA certificate bundle distributor to conduit install
* Update ca-distributor to use shared informers
* Only install CA distributor when --enable-tls flag is set
* Only copy CA bundle into namespaces where inject pods have the same controller
* Update API config to only watch pods and configmaps
* Address review feedback

Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-06-21 13:12:21 -07:00
Kevin Lingerfelt e80356de34
Upgrade prometheus to v2.3.1 (#1174)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-06-21 11:02:21 -07:00
Kevin Lingerfelt 682b0274b5
Add controller admin servers and readiness probes (#1168)
* Add controller admin servers and readiness probes
* Tweak readiness probes to be more sane
* Refactor based on review feedback

Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-06-20 17:32:44 -07:00
Andrew Seigner 0b9e7ff7df
Enable get for nodes/proxy for Prometheus RBAC (#1142)
The `kubernetes-nodes-cadvisor` Prometheus queries node-level data via
the Kubernetes API server. In some configurations of Kubernetes, namely
minikube and at least one baremetal kubespray cluster, this API call
requires the `get` verb on the `nodes/proxy` resource.

Enable `get` for `nodes/proxy` for the `conduit-prometheus` service
account.

Fixes #912

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2018-06-18 17:49:23 +01:00
Risha Mars e2c2f19d2c
Propagate errors in conduit containers to the api (#1117)
- It would be nice to display container errors in the UI. This PR gets the pod's container 
statuses and returns them in the public api

- Also add a terminationMessagePolicy to conduit's inject so that we can capture the 
proxy's error messages if it terminates
2018-06-14 16:22:31 -07:00
Thomas Rampelberg 516807bde6
Add readiness/liveness checks for third party components (#1121)
* Add readiness/liveness checks for third party components

Any possible issues with the third party control plane components can wedge the services.

Take the best practices for prometheus/grafana and add them to our template. See #1116

* Update test fixtures for new output
2018-06-14 13:01:13 -07:00
Kevin Lingerfelt eebc612d52
Add install flag for sending tls identity info to proxies (#1055)
Signed-off-by: Kevin Lingerfelt <kl@buoyant.io>
2018-06-04 16:55:06 -07:00