* If HA, set the webhooks failure policy to 'Fail'
I'm adding to the linkerd namespace a new label
`linkerd.io/is-control-plane: true` that is used in the webhook configs'
selector to skip the proxy injector for this namespace. This avoids
running into the timing issues described in #2852.
Fixes#2852
Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
The patch provided by @ihcsim applies correct values for the securityContext during injection, namely: `allowPrivilegeEscalation = false`, `readOnlyRootFilesystem = true`, and the capabilities are copied from the primary container. Additionally, the proxy-init container securityContext has been updated with appropriate values.
Signed-off-by: Cody Vandermyn <cody.vandermyn@nordstrom.com>
Add support for querying TrafficSplit resources through the common API layer. This is done by depending on the TrafficSplit client bindings from smi-sdk-go.
Signed-off-by: Alex Leong <alex@buoyant.io>
Updates and pins package version numbers in `package.json` to reflect the actual
versions in `yarn.lock`. Pins `react-iframe` to `1.7.16` and `jest` to `23.6.0`
- in both cases, there are later versions but they include breaking changes.
Modifies `webpack.config.js` to work with the updated `css-loader` library.
Fixes#2908.
## edge-19.6.1
* CLI
* Fixed an issue where, when Linkerd is installed with `--ha`, running
`linkerd upgrade` without `--ha` will disable the high availability
control plane
* Added a `--init-image-version` flag to `linkerd inject` to override the
injected proxy-init container version
* Controller
* Added multiple replicas for the `proxy-injector` and `sp-validator`
controllers when run in high availability mode (thanks to @Pothulapati!)
* Proxy
* Fixed a memory leak that can occur if an HTTP/2 request with a payload
ends before the entire payload is sent to the destination
* Internal
* Moved the proxy-init container to a separate `linkerd/proxy-init` Git
repository
* Fix HA during upgrade
If we have a Linkerd installation with HA, and then we do `linkerd
upgrade` without specifying `--ha`, the replicas will get set back to 1,
yet the resource requests will keep their HA values.
Desired behavior: `linkerd install --ha` adds the `ha` value into the
linkerd-config, so it should be used during upgrade even if `--ha` is
not passed to `linkerd upgrade`.
Note we still can do `linkerd upgrade --ha=false` to disable HA.
This is a prerequesite to address #2852
Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
## stable-2.3.2
This stable release fixes a memory leak in the proxy.
To install this release, run: `curl https://run.linkerd.io/install | sh`
**Full release notes**:
* Proxy
* Fixed a memory leak that can occur if an HTTP/2 request with a payload
ends before the entire payload is sent to the destination
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
commit 790a86aa9db463af479647bb91b8b55280d74d4
Author: Sean McArthur <sean@buoyant.io>
Date: Tue Jun 4 20:28:05 2019 -0700
Update h2 to v0.1.23 (#264)
- Fixes leaked DATA frames if never polled.
Signed-off-by: Sean McArthur <sean@buoyant.io>
This is a major refactor of the destination service. The goals of this refactor are to simplify the code for improved maintainability. In particular:
* Remove the "resolver" interfaces. These were a holdover from when our decision tree was more complex about how to handle different kinds of authorities. The current implementation only accepts fully qualified kubernetes service names and thus this was an unnecessary level of indirection.
* Moved the endpoints and profile watchers into their own package for a more clear separation of concerns. These watchers deal only in Kubernetes primitives and are agnostic to how they are used. This allows a cleaner layering when we use them from our gRPC service.
* Renamed the "listener" types to "translator" to make it more clear that the function of these structs is to translate kubernetes updates from the watcher to gRPC messages.
Signed-off-by: Alex Leong <alex@buoyant.io>
Split proxy-init into separate repo
Fixes#2563
The new repo is https://github.com/linkerd/linkerd2-proxy-init, and I
tagged the latest there `v1.0.0`.
Here, I've removed the `/proxy-init` dir and pinned the injected
proxy-init version to `v1.0.0` in the injector code and tests.
`/cni-plugin` depends on proxy-init, so I updated the import paths
there, and could verify CNI is still working (there is some flakiness
but unrelated to this PR).
For consistency, I added a `--init-image-version` flag to `linkerd
inject` along with its corresponding override config annotation.
Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
* Added labels to webhook configurations in charts/
* Multiple replicas of proxy-injector and sp-validator in HA
* Use ControllerComponent template variable for webhookconfigurations
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
## edge-19.5.4
* CLI
* Added a JSON option to the `linkerd edges` command so that output is
scripting friendly and can be parsed easily (thanks @alenkacz!)
* Controller
* **New** Control plane installations now generate a self-signed certificate
and private key pair for each webhook, to prepare for future work to make
the proxy injector and service profile validator HA
* Added a debug container annotation, allowing the `--enable-debug-sidecar`
flag to work when auto-injecting Linkerd proxies
* Proxy
* Changed the proxy's routing behavior so that, when the control plane does
not resolve a destination, the proxy forwards the request with minimal
additional routing logic
* Fixed a bug in the proxy's HPACK codec that could cause requests with very
large header values to hang indefinitely
* Web UI
* Removed the Authorities table and sidebar link from the dashboard to prepare
for a new, improved dashboard view communicating authority data
* Internal
* Modified the integration test for `linkerd upgrade` to test upgrading from
the latest stable release instead of the latest edge, to reflect the typical
use case
This stable release adds a number of proxy stability improvements.
To install this release, run: `curl https://run.linkerd.io/install | sh`
**Special thanks to**: @zaharidichev and @11Takanori!
**Full release notes**:
* Proxy
* Changed the proxy's routing behavior so that, when the control plane
does not resolve a destination, the proxy forwards the request with minimal
additional routing logic
* Fixed a bug in the proxy's HPACK codec that could cause requests with
very large header values to hang indefinitely
* Replaced the fixed reconnect backoff with an exponential one (thanks,
@zaharidichev!)
* Fixed an issue where requests could be held indefinitely by the load balancer
* Added a dispatch timeout that limits the amount of time a request can be
buffered in the proxy
* Removed the limit on the number of concurrently active service discovery
queries to the destination service
* Fixed an epoll notification issue that could cause excessive CPU usage
* Added the ability to disable tap by setting an env var (thanks,
@zaharidichev!)
* * Update webpack-dev-server to "^3.2.0" in package.json with generated yarn.lock file
* * Update webpack-dev-server to "^3.2.0" in package.json with generated yarn.lock file
Signed-off-by: cpretzer <charles@buoyant.io>
* * #2817 enable shorthands for lodash
Signed-off-by: cpretzer <charles@buoyant.io>
* * #2817 enable shorthands for lodash
Signed-off-by: cpretzer <charles@buoyant.io>
* Pin webpack-dev-server to version 3.3.1
Signed-off-by: Charles Pretzer <charles@buoyant.io>
* Subject:
Remove unnecessary linting configs and rules for lodash
Problem:
Work related to supporting newer versions of node allows for the removal of lodash linting
Solution:
Remove lodash lint rules
Remove eslint-plugin-lodash from package.json
Remove lodash from plugins section of .eslintrc
Validation:
Ran all build commands (setup, dev, and run) with node versions 12 and 10
Fixes#2817#2809
Signed-off-by: Charles Pretzer <charles@buoyant.io>
* Signed-off-by: Charles Pretzer <charles@buoyant.io>
Update yarn.lock after merge with master
Fixes#2103. Hides the Authorities table in the Overview and Namespace views on
the dashboard, and removes the link to Authorities in the Resources sidebar.
This change makes way for a future dashboard view incorporating traffic split
data and communicating Authority data in a more understandable way.
When `linkerd edges` returns JSON, the data will now be sorted alphabetically by
SRC name, meaning edges will be returned in a consistent order. Logic in the CLI
`edges.go` has also been simplified. These changes should result in the Travis
CI builds passing consistently.
This commit refactors the changes introduced by #2842 where the debug
container spec is created in the 'cli' and 'pkg' packages. This change
follows the existing pattern of annotating the YAML in the CLI code,
and injecting the sidecar spec in the shared library.
Signed-off-by: Ivan Sim <ivan@buoyant.io>
This new annotation is used by the proxy injector to determine if the
debug container needs to be injected.
When using 'linkerd install', the 'pkg/inject' library will only inject
annotations into the workload YAML. Even though 'conf.debugSidecar'
is set in the CLI, the 'injectPodSpec()' function is never invoked on
the proxy injector side. Once the workload YAML got picked up by the
proxy injector, 'conf.debugSidecar' is already nil, since it's a different,
new 'conf' object. The new annotation ensures that the proxy injector
injects the debug container.
Signed-off-by: Ivan Sim <ivan@buoyant.io>
In #2679 we introduced an upgrade integration test. At the time we only
supported upgrading from a recent edge. Since that PR, a stable build
was released supporting upgrade.
Modify the upgrade integration test to upgrade from the latest stable
rather than latest edge. This fulfills the original intent of #2669.
Also add some known k8s event warnings.
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
* Update helm charts to include webhooks config and TLS secret
* Update the webhooks to read the secret cert and key
* Update webhooks to not recreate config on restart
* Ensure upgrade preserve existing secrets
* Revert the change to rename the webhook configs
The renaming change breaks upgrade, where the new webhook configs conflict with
the existing ones. The older resources aren't deleted during upgrade because
they are dynamically created.
* Make the secret volume read-only
* Remove unnecessary exported getter functions
* Remove obsolete mwc and vwc templates
Signed-off-by: Ivan Sim <ivan@buoyant.io>
commit b27dfb2d21aa8ca5466ea0edce17d27094ace7c1
Author: Takanori Ishibashi <takanori.1112@gmail.com>
Date: Wed May 15 05:58:42 2019 +0900
updaes->updates (#250)
Signed-off-by: Takanori Ishibashi <takanori.1112@gmail.com>
commit 16441c25a9d423a6ab12b689b830d9ae3798fa00
Author: Eliza Weisman <eliza@buoyant.io>
Date: Tue May 14 14:40:03 2019 -0700
Pass router::Config directly to router::Layer (#253)
Currently, router `layer`s are constructed with a single argument, a
type implementing `Recognize`. Then, the entire router stack is built
with a `router::Config`. However, in #248, it became necessary to
provide the config up front when constructing the `router::layer`, as
the layer is used in a fallback layer. Rather than providing a separate
type for a preconfigured layer, @olix0r suggested we simply change all
router layers to accept the `Config` when they're constructed (see
https://github.com/linkerd/linkerd2-proxy/pull/248#discussion_r283575008).
This branch changes `router::Layer` to accept the config up front. The
`router::Stack` types `make` function now requires no arguments, and the
implementation of `Service` for `Stack` can be called with any `T` (as
the target is now ignored).
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
commit b70c68d4504a362eac6a7828039a2e5c7fcd308a
Author: Eliza Weisman <eliza@buoyant.io>
Date: Wed May 15 13:14:04 2019 -0700
Load balancers fall back to ORIG_DST when no endpoints exist (#248)
Currently, when no endpoints exist in the load balancer for a
destination, we fail the request. This is because we expect endpoints to
be discovered by both destination service queries _and_ DNS lookups, so
if there are no endpoints for a destination, it is assumed to not exist.
In linkerd/linkerd2#2661, we intend to remove the DNS lookup from the
proxy and instead fall back to routing requests for which no endpoints
exist in the destination service to their SO_ORIGINAL_DST IP address.
This means that the current approach of failing requests when the load
balancer has no endpoints will no longer work.
This branch introduces a generic `fallback` layer, which composes a
primary and secondary service builder into a new layer. The primary
service can fail requests with an error type that propages the original
request, allowing the fallback middleware to call the fallback service
with the same request. Other errors returned by the primary service are
still propagated upstream.
In contrast to the approach used in #240, this fallback middleware is
generic and not tied directly to a load balancer or a router, and can
be used for other purposes in the future. It relies on the router cache
eviction added in #247 to drain the router when it is not being used,
rather than proactively destroying the router when endpoints are
available for the lb, and re-creating it when they exist again.
A new trait, `HasEndpointStatus`, is added in order to allow the
discovery lookup to communicate the "no endpoints" state to the
balancer. In addition, we add a new `Update::NoEndpoints` variant to
`proxy::resolve::Update`, so that when the control plane sends a no
endpoints update, we switch from the balancer to the no endpoints state
_immediately_, rather than waiting for all the endpoints to be
individually removed. When the balancer has no endpoints, it fails all
requests with a fallback error, so that the fallback middleware
A subsequent PR (#248) will remove the DNS lookups from the discovery
module.
Closes#240.
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
commit 6525b0638ad18e74510f3156269e0613f237e2f5
Author: Zahari Dichev <zaharidichev@gmail.com>
Date: Wed May 15 23:35:09 2019 +0300
Allow disabling tap by setting an env var (#252)
This PR fixeslinkerd/linkerd2#2811. Now if
`LINKERD2_PROXY_TAP_DISABLED` is set, the tap is not served at all. The
approach taken is that the `ProxyParts` is changed so the
`control_listener` is now an `Option` that will be None if tap is
disabled as this control_listener seems to be exclusively used to serve
the tap. Feel free to suggest a better approach.
Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
commit 91f32db2ea6d74470fd689c713ff87dc7586222d
Author: Zahari Dichev <zaharidichev@gmail.com>
Date: Thu May 16 00:45:23 2019 +0300
Assert that outbound TLS works before identity is certified (#251)
This commit introduces TLS capabilities to the support server as well as
tests to ensure that outbound TLS works even when there is no verified
certificate for the proxy yet.
Fixeslinkerd/linkerd2#2599
Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
commit 45aadc6b1b28e6daea0c40e694a86ae518887d85
Author: Sean McArthur <sean@buoyant.io>
Date: Wed May 15 14:25:39 2019 -0700
Update h2 to v0.1.19
Includes a couple HPACK fixes
Signed-off-by: Sean McArthur <sean@buoyant.io>
commit 3e0e00c6dfbf5a9155b887cfd594f611edfc135f
Author: Oliver Gould <ver@buoyant.io>
Date: Thu May 16 08:11:06 2019 -0700
Update mio to 0.6.17 (#257)
To pick up https://github.com/tokio-rs/mio/pull/939
Adds a check to Prometheus `edges` queries to verify that data for the requested
resource type exists. Previously, if Prometheus could not find request data for the
requested resource type, it would skip that label and still return data for
other labels in the `by` clause, leading to an incorrect response.
Adds an edges command to the CLI. `linkerd edges` displays connections between resources, and Linkerd proxy identities. Currently this feature will only display edges where both the client identity and server identity are known. The next step will be to display edges for which identity is not known and/or one-sided traffic such as Prometheus and tap requests.
* CLI
* Fixed `linkerd check` and `linkerd dashboard` failing when any control plane
pod is not ready, even when multiple replicas exist (as in HA mode)
* Controller
* Fixed control plane components failing on startup when the Kubernetes API
returns an `ErrGroupDiscoveryFailed`
* Proxy
* Added a dispatch timeout that limits the amount of time a request can be
buffered in the proxy
* Removed the limit on the number of concurrently active service discovery
queries to the Destination service
Special thanks to @zaharidichev for adding end to end tests for proxies with
TLS!
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
Support for resources opting out of tap
Implements the `linkerd inject --disable-tap` flag (although hidden pending #2811) and the config override annotation `config.linkerd.io/disable-tap`.
Fixes#2778
Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
commit 5f89351081eff47a4ab8cd88e2e1a69a04f86541
Author: Oliver Gould <ver@buoyant.io>
Date: Thu May 9 16:39:24 2019 -0700
Upgrade tower dependencies (#249)
Tower must be updated in order to pickup tower-rs/tower#281
to address linkerd/linkerd2#2804.
This adopts released crates where possible.
commit 5d5eed6f8180b8db4090d995e71fdf7b0890c647
Author: Zahari Dichev <zaharidichev@gmail.com>
Date: Thu May 9 01:08:34 2019 +0300
Assert that TLS connection is refused if identity is not certified yet (#243)
This branch adds tls capability to the support cient used in tests. In addition to that it adds two tests verifying that a TLS connection is refused in case the identity is not certified yet. This attempts to fix #https://github.com/linkerd/linkerd2/issues/2598 and provide facility to write tests for https://github.com/linkerd/linkerd2/issues/2676.
As these are still some of my first lines of Rust code, it is advised to approach everything with a healthy dose of doubt :)
Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
commit 1b9bb3745e44c959d1d41d14fed2b2822c82b5ba
Author: Oliver Gould <ver@buoyant.io>
Date: Wed May 8 14:28:37 2019 -0700
Introduce dispatch timeouts around buffers (#246)
The proxy has several buffers, especially where it routes requests over
shared stacks. If any of these routes is unavailable, then a request may
remain buffered indefinitely. Previously, before service profiles were
introduced, there was a default _response_ timeout that would cause
these requests to fail; but since this response timeout is now optional
(and is only applied once the request has been routed within a proxy),
then we need a new mechanism to prevent requests from getting "stuck".
This change does the following:
- all proxied requests are annotated with a dispatch deadline;
- each time a request is bufered, a timeout is registered.
- if the timeout fires, the response exception fails, a 503 is returned,
and the request is dropped.
- if the request is processed into the inner stack, the timeout is
ignored.
The dispatch timeout limits the _time a request is buffered in a proxy_.
This is distinct from the response timeout, as the server's response may
naturally be delayed for any number of (non-proxy-related) reasons.
The `insert_target` module has been generalized to `insert` to support
setting the DispatchDeadline extension.
The `buffer` module has been augmented with generic deadline-extraction
logic.
The `svc` module now exposes its own builder type that notably adds
a `buffer_pending` helper. It's helpful to pull a builder type into the
proxy to assist debugging type errors when modifying stacks.
Fixeslinkerd/linkerd2#2779linkerd/linkerd2#2795
commit caf899557c3b041190f63544da865396231b3e30
Author: Oliver Gould <ver@buoyant.io>
Date: Fri May 3 15:55:32 2019 -0700
router: Fail requests when the route is not ready (#241)
In linkerd/linkerd2#2779, we plan to expire requests while they are
buffered. However, the router _implicitly_ buffers requests in the
executor when the inner service is not ready.
This change alters the route to wrap all inner layers in a `LoadShed`
so it can expect all services to `poll_ready()` immediately.
commit 587bad101d9e5daeacb24b6733097c350a798356
Author: Eliza Weisman <eliza@buoyant.io>
Date: Fri May 3 14:18:08 2019 -0700
Remove Destination service query concurrency limit (#244)
Currently, the proxy enforces a limit on the number of concurrent
queries (i.e., the number of gRPC streams) to the Destination service.
This limit was added based on information about the behaviour of the
Destination service that is now known to be incorrect.
This branch removes the limit on concurrent queries from the proxy's
`control::destination` module. Although it should now be possible to
simplify this code as a result of this change, I've refrained from doing
any major refactoring in this branch --- my intention is to do this
after the DNS fallback behaviour has also been removed, as together with
this change, that will result in a _significant_ simplification of the
module. Additionally, I've removed the tests for the concurrency limit,
as they are no longer relevant.
The `LINKERD2_PROXY_DESTINATION_CLIENT_CONCURRENCY_LIMIT`
environment variable was also removed; this is not a breaking change as
neither the CLI nor the proxy injector will currently set this env var.
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
commit cbdf45b44f7e4d852dc0497716062167ab9539fb
Author: Sean McArthur <sean@buoyant.io>
Date: Thu May 2 11:47:48 2019 -0700
Remove h2::Error requirement from metrics
Signed-off-by: Sean McArthur <sean@buoyant.io>
commit 3276949d4608dc4344b7bed3de2fc4b3080c2c6e
Author: Sean McArthur <sean@buoyant.io>
Date: Thu May 2 09:44:00 2019 -0700
delete unused proxy::http::metrics::class module
Signed-off-by: Sean McArthur <sean@buoyant.io>
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
Integration test for k8s events generated during install
Fixes#2713
I did make sure a scenario like the one described in #2964 is caught.
Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
This change adds an endpoint to the public API to allow us to query Prometheus for edge data, in order to display identity information for connections between Linkerd proxies. This PR only includes changes to the controller and protobuf.
The `docker-build-proxy` script builds `Dockerfile-proxy`. That
Dockerfile depends on a go-deps image, and takes a `LINKERD_VERSION`
arg. The `docker-build-proxy` script was neither ensuring go-deps had
been built, nor setting `LINKERD_VERSION`. The former resulted in the
build failing if go-deps did not exist. The latter resulted in
`dev-undefined` log messages in the `linkerd-proxy` container.
Fix `docker-build-proxy` to ensure go-deps are built, and also set the
`LINKERD_VERSION`. This brings this script more in-line with the other
`docker-build-*` scripts.
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
`ServiceProfilesAccess()`, called by control plane components at
startup, would fail if it encountered an `ErrGroupDiscoveryFailed` from
a GroupVersion request. This error is mostly innocuous, as it returns an
error if any GroupVersion fails. `ServiceProfilesAccess()` only needs to
validate ServiceProfiles are available.
Modify `ServiceProfilesAccess()` to specifically request the
ServiceProfile GroupVersion. Also add Discovery object
(`APIResourceList`) support to `NewFakeClientSets`.
Fixes#2780
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
The `linkerd check` and `linkerd dashboard` commands validate control
plane pods are up via the `LinkerdAPIChecks` category of checks. These
checks will fail if a single pod is not ready, even in HA mode.
Modify the underlying `validateControlPlanePods` check to return
successful if at least one pod per control plane component is ready.
Fixes#2554
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
With the server configured to response with a failure of 50%, the test first
checks to ensure the actual success rate is less than 100%. Then the
service profile is edited to perform retries. The test then checks to
ensure the effective success rate is at least 95%.
This is (hopefully) more reliable than changing the test to perform waits and
retries until there is a difference between effective success rate and actual
success rate and compare them.
Signed-off-by: Ivan Sim <ivan@buoyant.io>
Carol A. Scott
Ivan Sim
Alejandro Pedraza
Cody Vandermyn
Alex Leong
Dan
Eliza Weisman
Oliver Gould
Tarun Pothulapati
Dennis Adjei-Baah
cpretzer
dependabot[bot]
Andrew Seigner
Alena Varkockova
Risha Mars
mg
William Morgan