* Wait for `helm delete` to finish in integration test
Followup to #3251
In `helm_cleanup` block till the linkerd namespace has been deleted
Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
PR #3247 introduced additional helm cleanup in `bin/test-cleanup`.
During the integration tests, `bin/test-cleanup` is called prior to
`helm_cleanup` in `bin/test-run`. This causes `helm_cleanup` to fail, as
resources have already been deleted by `bin/test-cleanup`, and the
integration tests fail with `FAIL: error cleaning up Helm`.
Modify the integration tests to first call `helm_cleanup` prior to
calling `bin/test-cleanup`.
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
When helm integration tests fail, `bin/test-run` exits prior to calling
`helm_cleanup`, leaving behind a helm namespace and clusterrolebinding.
Update `bin/test-cleanup` to delete any remaining helm resources.
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
* Delete symlink to old Helm chart
* Update 'install' code to use common Helm template structs
* Remove obsolete TLS assets functions.
These are now handle by Helm functions inside the templates
* Read defaults from values.yaml and values-ha.yaml
* Ensure that webhooks TLS assets are retained during upgrade
* Fix a few bugs in the Helm templates (see bullet points):
* Merge the way the 'install' ha and non-ha options are handled into one function
* Honor the 'NoInitContainer' option in the components templates
* Control plane mTLS will not be disabled if identity context in the
config map is empty. The data plane mTLS will still be automatically disabled
if the context is nil.
* Resolve test failures from rebase with master
* Fix linter issues
* Set service account mount path read-only field
* Add TLS variables of the webhooks and tap to values.yaml
During upgrade, these secrets are preserved to ensure they remain synced
wih the CA bundle in the webhook configurations. These Helm variables are used
to override the defaults in the templates.
* Remove obsolete 'chart' folder
* Fix bugs in templates
* Handle missing webhooks and tap TLS assets during upgrade
When upgrading from an older version that don't have these secrets, fallback to let Helm
create them by creating an empty charts.TLS struct.
* Revert the selector labels of webhooks to be compatible with that in 2.4
In 2.4, the proxy injector and profile validator webhooks already have their selector labels defined.
Since these attributes are immutable, the recent change to these selectors introduced by the Helm chart work will cause upgrade to fail.
* Alejandro's feedback
* Siggy's feedback
* Removed redundant unexported custom types
Signed-off-by: Ivan Sim <ivan@buoyant.io>
* Refactor proxy injection to use Helm charts
Fixes#3128
A new chart `/charts/patch` was created, that generates the JSON patch
payload that is to be returned to the k8s API when doing the injection
through the proxy injector, and it's also leveraged by the `linkerd
inject --manual` CLI.
The VFS was used by `linkerd install` to access the old chart under
`/chart`. Now the proxy injection also uses the Helm charts to generate
the JSON patch (see above) so we've moved the VFS from `cli/static` to a
new common place under `/pkg/charts/static`, and the new root for the VFS is
now `/charts`.
`linkerd install` hasn't yet migrated to use the new charts (that'll
happen in #3127), so the only change in that regard was the creation of
`/charts/chart` which is a symlink pointing to `/chart` that
`install.go` now uses, so that the VFS contains both the old and new
charts, as a temporary measure.
You can see that `/bin/Dockerfile-bin`, `/controller/Dockerfile` and
`/bin/build-cli-bin` do now `go generate` pointing to the new location
(and the `go generate` annotation was moved from `/cli/main.go` to
`pkg/charts/static/templates.go`).
The symlink trick doesn't work when building the binaries through
Docker, so `/bin/Dockerfile-bin` replaces the symlink with an actual
copy of `/chart`.
Also note that in `/controller/Dockerfile` we now need to include the
`prod` tag in `go install` like we do in `/bin/Dockerfile-bin` so that
the proxy injector does use the VFS instead of the local file system.
- The common logic to parse a chart has been moved from `install.go` to
`/pkg/charts/util.go`.
- The special ENV var in the proxy for "outbound router capacity" that
only applies to the Prometheus pod is now handled directly in the proxy
partial and all the associated go code could be removed.
- The `patch.go` lib for generating the JSON patch in go along
with its tests `patch_test.go` are no longer needed.
- Lots of functions in `/pkg/inject/inject.go` got removed/simplified
with their logic being moved into the charts themselves. As a
consequence lots of things in `inject_test.go` became irrelevant.
- Moved `template-values.go` from `/pkg/inject` to `pkg/charts` as that
contains the go structs representation of the chart variables that
will be leveraged in #3127.
Don't forget to run `/bin/helm.sh` whenever you make changes to charts
;-)
Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
`bin/helm.sh`: you should see the following only if you have Tiller
installed in your cluster (which is installed with `helm init`):
```
Performing dry run installation
NAME: linkerd
Performing dry run installation (HA mode)
NAME: linkerd
```
`bin/test-cleanup`: when linkerd is not installed:
Before:
```bash
$ bin/test-cleanup
cleaning up control-plane namespaces in k8s-context []
cleaning up data-plane namespaces in k8s-context []
cleaning up rolebindings in kube-system namespace in k8s-context []
```
After this PR's changes:
```
$bin/test-cleanup
cleaning up control-plane namespaces in k8s-context []
no control-plane namespaces found
cleaning up data-plane namespaces in k8s-context []
no data-plane namespaces found
no clusterrolebindings found
no clusterroles found
no mutatingwebhookconfigurations found
no validatingwebhookconfigurations found
no podsecuritypolicies found
no customresourcedefinitions found
no apiservices found
cleaning up rolebindings in kube-system namespace in k8s-context []
```
Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
* Updated controller template with proxy partials
* Declare dependency in requirements.yaml
* Add partial template for proxy's metadata
* Add proxy-init partial template
* Script to lint Helm charts and update their dependencies
* Update partials chart Chart.yaml
* Add proxy-init and resource partial templates
* Replace hard coded namespace variable in proxy env var
* Ignore chart dependencies .tgz files
* Add missing fields and re-order YAML elements to match CLI output
* Reuse control plane's resource partial template in 'partials' chart
* Set the proxy's destination service address env var
* Add Grafana's template
* Update api version of controller RBAC
* Add Heartbeat template
* Remove duplicated resources partial template
* Add remainder control plane components templates
* Add template for the 'linkerd-config' config map
* Add debug container template
* Update proxy partial with 'disable-identity' and 'disable-tap' variables
Note that these are inject-only variables.
Also added the LINKERD2_PROXY_TAP_SVC_NAME env var.
* Add validation conditions to ensure identity and tap aren't disabled for
control plane components
* Add partials for service account token mount path and security context capabilities
* Change proxy and proxy-init templates to use global scope
Some of the nested variables are removed from values.yaml to ensure changes
made to root-level variables are propagated directly into the partial
templates. The previous approach of using YAML anchors in the
values.yaml to share common values can get out-of-sync when values are
changed via the Helm's `--set` option.
* Update templates and values file to match #3161
* Perform a dry run installation if there is a local Tiller
* Reorder JSON elements in linkerd-config
* Re-adjust nested partials indentation to work with inject 'patch' chart
Previously, the partials will render their content as an element in the list.
While it works for installation, the toJson function in the 'inject' patch code
ends up converting it into a JSON list, instead of the expected JSON
object.
* Trap the last fail command in the Helm shell script
* Add the identity trust anchor
* Address Thomas' feedback on handling HA
All the HA-related variables are moved to values-ha.yaml
* Convert ignore ports string to JSON list in linkerd-config
Also fixed some indentation issues.
* Add values-ha.yaml
* Include the service account token mount path only if identity is enabled
* Fixed malformed JSON in linkerd-config config map
* Rename chart to 'linkerd2'
* Add NOTES.txt
* Fix incorrect variable path in proxy template
* Remove fake TLS assets
* Add 'required' constraint to identity trust anchors variable
* Update tap templates per #3167
* Bump default version to edge-19.8.1 due to dependency on RSA support
Signed-off-by: Ivan Sim <ivan@buoyant.io>
The Tap Service enabled tapping of any meshed pod, regardless of user
privilege.
This change introduces a new Tap APIService. Kubernetes provides
authentication and authorization of Tap requests, and then forwards
requests to a new Tap APIServer, which implements a Kubernetes
aggregated APIServer. The Tap APIServer authenticates the client TLS
from Kubernetes, and authorizes the user via a SubjectAccessReview.
This change also modifies the `linkerd tap` command to make requests
against the new APIService.
The Tap APIService implements these Kubernetes-style endpoints:
POST /apis/tap.linkerd.io/v1alpha1/watch/namespaces/:ns/tap
POST /apis/tap.linkerd.io/v1alpha1/watch/namespaces/:ns/:res/:name/tap
GET /apis
GET /apis/tap.linkerd.io
GET /apis/tap.linkerd.io/v1alpha1
GET /healthz
GET /healthz/log
GET /healthz/ping
GET /metrics
GET /openapi/v2
GET /version
Users authorize to the new `tap.linkerd.io/v1alpha1` via RBAC. Only the
`watch` verb is supported. Access is also available via subresources
such as `deployments/tap` and `pods/tap`.
This change introduces the following resources into the default Linkerd
install:
- Global
- APIService/v1alpha1.tap.linkerd.io
- ClusterRoleBinding/linkerd-linkerd-tap-auth-delegator
- `linkerd` namespace:
- Secret/linkerd-tap-tls
- `kube-system` namespace:
- RoleBinding/linkerd-linkerd-tap-auth-reader
Tasks not covered by this PR:
- `linkerd top`
- `linkerd dashboard`
- `linkerd profile --tap`
- removal of the unauthenticated tap controller
Fixes#2725, #3162, #3172
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
The repo relied on `dep` for managing Go dependencies. Go 1.11 shipped
with Go modules support. Go 1.13 will be released in August 2019 with
module support enabled by default, deprecating GOPATH.
This change replaces `dep` with Go modules for dependency management.
All scripts, including Docker builds and ci, should work without any dev
environment changes.
To execute `go` commands directly during development, do one of the
following:
1. clone this repo outside of `GOPATH`; or
2. run `export GO111MODULE=on`
Summary of changes:
- Docker build scripts and ci set `-mod=readonly`, to ensure
dependencies defined in `go.mod` are exactly what is used for the
builds.
- Dependency updates to `go.mod` are accomplished by running
`go build` and `go test` directly.
- `bin/go-run`, `bin/build-cli-bin`, and `bin/test-run` set
`GO111MODULE=on`, permitting usage inside and outside of GOPATH.
- `gcr.io/linkerd-io/go-deps` tags hashed from `go.mod`.
- `bin/update-codegen.sh` still requires running from GOPATH,
instructions added to BUILD.md.
Fixes#1488
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
The openAPIV3Schema validation in the ServiceProfiles CRD is very limited in what it can validate and is obviated by more sophisticated validation done by the validating admission controller. Therefore, we would like to remove the openAPIV3Schema validation to reduce the size and complexity of the CRD object.
To do so, we must also bump the version of the ServiceProfile custom resource from v1alpha1 to v1alpha2. This ensures that when the controller is upgraded, it will attempt to watch the v1alpha2 resource. If it cannot (because, for example, the controller pod started before the ServiceProfile CRD was updated and therefore the v1alpha2 version does not exist) then it will go into a crash loop backoff until it can. This essentially means that the controller will wait for the CRD to be upgraded to include v1alpha2 before it will start.
Bumping the version is necessary because if we did not, it would be possible for the controller to start before the CRD is updated (removing the validation). In this case, when the CRD is edited, the controller will lose its list watch on ServiceProfiles and will stop getting updates.
Signed-off-by: Alex Leong <alex@buoyant.io>
* Updates for the integration test script
1. Remove existing resources prior to starting the test
2. Remove existing resources post upgrade test
3. Fail fast if 'install_test.go` fails
4. Don't perform cleanup if any of the tests fail for debugging
opportunity
* Remove pre-test cleanup from .travis.yaml
This is now done in the bin/test-run script so that it can be shared
between l5d-bot and staging.
Signed-off-by: Ivan Sim <ivan@buoyant.io>
Integration tests may fail and leave behind namespaces that following
builds aren't able to clean up because the git sha is being included in
the namespace name, and the following builds don't know about those
shas.
This modifies the `test-cleanup` script to delete based on object labels
instead of relying on the objects names, now that after 2.4 all the
control plane components are labeled. Note that this will also remove
non-testing linkerd namespaces, but we were already kinda doing that
partially because we were removing the cluster-level resources (CRDs,
webhook configs, clusterroles, clusterrolebindings, psp).
`test-cleanup` no longer receives a namespace name as an argument.
The data plane namespaces aren't labeled though, so I've added the
`linkerd.io/is-test-data-plane` label for them in
`CreateNamespaceIfNotExists()`, and making sure all tests that need a
data plaine explicitly call that method instead of creating the
namespace as a side-effect in `KubectlApply()`.
Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
The git-commt-proxy-version script attempted to link to the specifc
SHAs. GitHub doesn't actually render these links, and the information is
redundant since we link the appropriate PR.
Each time we update the proxy from the linkerd2-proxy repo, we make the
change slightly differently. The bin/git-commit-proxy-version does all the
steps needed to update the proxy version up to and including making a
commit to this repo.
The proxy version is now stored in a .proxy-version file and is
consumed directly by Dockerfile-proxy, which both simplifies the
Dockerfile and the update process.
This script formats commit messages and emits output as follows:
```
commit c05198a851f69bdc7007974a0ef1f4c01c98d0ce (HEAD -> ver/proxy-update)
Author: Oliver Gould <ver@buoyant.io>
Date: Thu Jul 11 17:23:05 2019 +0000
proxy: Update to linkerd/linkerd2-proxy#3a3ec3b
* linkerd/linkerd2-proxy#0cc58cd fallback: Clarify fallback layering (linkerd/linkerd2-proxy#288)
* linkerd/linkerd2-proxy#b71349a Replace `log` and `env-logger` with `tracing` and `tracing-fmt` (linkerd/linkerd2-proxy#277)
* linkerd/linkerd2-proxy#3a3ec3b Use a constant-time load balancer (linkerd/linkerd2-proxy#266)
diff --git a/.proxy-version b/.proxy-version
index f81f40de..d7faa12d 100644
--- a/.proxy-version
+++ b/.proxy-version
@@ -1 +1 @@
-05b012d
+3a3ec3b
```
Integration tests on master broke following the 2.4 release, caused by
the recent disabling of multi control-plane support, coupled with the
upgrade integration test (which now upgrades from 2.4 to current sha).
The integration tests do the following:
1. install the current sha
2. test the current sha
3. install the latest stable in an `upgrade` namespace
4. in the `upgrade` namespace, upgrade from stable to latest sha
5. test the upgraded installation
Step 3 breaks because `linkerd install` with stable-2.4 will fail if
existing global resources (from step 1) are present.
For now, modify the integration tests to do the following:
1. install the latest stable in an `upgrade` namespace
2. in the `upgrade` namespace, upgrade from stable to latest sha
3. test the upgraded installation
4. upon successful step 3, remove all related resources
5. install the current sha
6. test the current sha
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
Pick up the following proxy changes:
* Update httparse to v1.3.4
* canonicalize: stop resolving when the receiver is dropped
* router: Remove interval from router eviction
Signed-off-by: Alex Leong <alex@buoyant.io>
* Introduce new checks to determine existence of global resources and the
'linkerd-config' config map.
* Update pre-check to check for existence of global resources
This ensures that multiple control planes can't be installed into
different namespaces.
* Update integration test clean-up script to delete psp and crd
Signed-off-by: Ivan Sim <ivan@buoyant.io>
* Fixed the proxy rejecting HTTP2 requests that don't have an `:authority`
* Improved idle service eviction to reduce resource consumption for clients
that send requests to many services
Signed-off-by: Kevin Leimkuhler <kleimkuhler@icloud.com>
commit 790a86aa9db463af479647bb91b8b55280d74d4
Author: Sean McArthur <sean@buoyant.io>
Date: Tue Jun 4 20:28:05 2019 -0700
Update h2 to v0.1.23 (#264)
- Fixes leaked DATA frames if never polled.
Signed-off-by: Sean McArthur <sean@buoyant.io>
Split proxy-init into separate repo
Fixes#2563
The new repo is https://github.com/linkerd/linkerd2-proxy-init, and I
tagged the latest there `v1.0.0`.
Here, I've removed the `/proxy-init` dir and pinned the injected
proxy-init version to `v1.0.0` in the injector code and tests.
`/cni-plugin` depends on proxy-init, so I updated the import paths
there, and could verify CNI is still working (there is some flakiness
but unrelated to this PR).
For consistency, I added a `--init-image-version` flag to `linkerd
inject` along with its corresponding override config annotation.
Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
In #2679 we introduced an upgrade integration test. At the time we only
supported upgrading from a recent edge. Since that PR, a stable build
was released supporting upgrade.
Modify the upgrade integration test to upgrade from the latest stable
rather than latest edge. This fulfills the original intent of #2669.
Also add some known k8s event warnings.
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
commit b27dfb2d21aa8ca5466ea0edce17d27094ace7c1
Author: Takanori Ishibashi <takanori.1112@gmail.com>
Date: Wed May 15 05:58:42 2019 +0900
updaes->updates (#250)
Signed-off-by: Takanori Ishibashi <takanori.1112@gmail.com>
commit 16441c25a9d423a6ab12b689b830d9ae3798fa00
Author: Eliza Weisman <eliza@buoyant.io>
Date: Tue May 14 14:40:03 2019 -0700
Pass router::Config directly to router::Layer (#253)
Currently, router `layer`s are constructed with a single argument, a
type implementing `Recognize`. Then, the entire router stack is built
with a `router::Config`. However, in #248, it became necessary to
provide the config up front when constructing the `router::layer`, as
the layer is used in a fallback layer. Rather than providing a separate
type for a preconfigured layer, @olix0r suggested we simply change all
router layers to accept the `Config` when they're constructed (see
https://github.com/linkerd/linkerd2-proxy/pull/248#discussion_r283575008).
This branch changes `router::Layer` to accept the config up front. The
`router::Stack` types `make` function now requires no arguments, and the
implementation of `Service` for `Stack` can be called with any `T` (as
the target is now ignored).
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
commit b70c68d4504a362eac6a7828039a2e5c7fcd308a
Author: Eliza Weisman <eliza@buoyant.io>
Date: Wed May 15 13:14:04 2019 -0700
Load balancers fall back to ORIG_DST when no endpoints exist (#248)
Currently, when no endpoints exist in the load balancer for a
destination, we fail the request. This is because we expect endpoints to
be discovered by both destination service queries _and_ DNS lookups, so
if there are no endpoints for a destination, it is assumed to not exist.
In linkerd/linkerd2#2661, we intend to remove the DNS lookup from the
proxy and instead fall back to routing requests for which no endpoints
exist in the destination service to their SO_ORIGINAL_DST IP address.
This means that the current approach of failing requests when the load
balancer has no endpoints will no longer work.
This branch introduces a generic `fallback` layer, which composes a
primary and secondary service builder into a new layer. The primary
service can fail requests with an error type that propages the original
request, allowing the fallback middleware to call the fallback service
with the same request. Other errors returned by the primary service are
still propagated upstream.
In contrast to the approach used in #240, this fallback middleware is
generic and not tied directly to a load balancer or a router, and can
be used for other purposes in the future. It relies on the router cache
eviction added in #247 to drain the router when it is not being used,
rather than proactively destroying the router when endpoints are
available for the lb, and re-creating it when they exist again.
A new trait, `HasEndpointStatus`, is added in order to allow the
discovery lookup to communicate the "no endpoints" state to the
balancer. In addition, we add a new `Update::NoEndpoints` variant to
`proxy::resolve::Update`, so that when the control plane sends a no
endpoints update, we switch from the balancer to the no endpoints state
_immediately_, rather than waiting for all the endpoints to be
individually removed. When the balancer has no endpoints, it fails all
requests with a fallback error, so that the fallback middleware
A subsequent PR (#248) will remove the DNS lookups from the discovery
module.
Closes#240.
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
commit 6525b0638ad18e74510f3156269e0613f237e2f5
Author: Zahari Dichev <zaharidichev@gmail.com>
Date: Wed May 15 23:35:09 2019 +0300
Allow disabling tap by setting an env var (#252)
This PR fixeslinkerd/linkerd2#2811. Now if
`LINKERD2_PROXY_TAP_DISABLED` is set, the tap is not served at all. The
approach taken is that the `ProxyParts` is changed so the
`control_listener` is now an `Option` that will be None if tap is
disabled as this control_listener seems to be exclusively used to serve
the tap. Feel free to suggest a better approach.
Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
commit 91f32db2ea6d74470fd689c713ff87dc7586222d
Author: Zahari Dichev <zaharidichev@gmail.com>
Date: Thu May 16 00:45:23 2019 +0300
Assert that outbound TLS works before identity is certified (#251)
This commit introduces TLS capabilities to the support server as well as
tests to ensure that outbound TLS works even when there is no verified
certificate for the proxy yet.
Fixeslinkerd/linkerd2#2599
Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
commit 45aadc6b1b28e6daea0c40e694a86ae518887d85
Author: Sean McArthur <sean@buoyant.io>
Date: Wed May 15 14:25:39 2019 -0700
Update h2 to v0.1.19
Includes a couple HPACK fixes
Signed-off-by: Sean McArthur <sean@buoyant.io>
commit 3e0e00c6dfbf5a9155b887cfd594f611edfc135f
Author: Oliver Gould <ver@buoyant.io>
Date: Thu May 16 08:11:06 2019 -0700
Update mio to 0.6.17 (#257)
To pick up https://github.com/tokio-rs/mio/pull/939
commit 5f89351081eff47a4ab8cd88e2e1a69a04f86541
Author: Oliver Gould <ver@buoyant.io>
Date: Thu May 9 16:39:24 2019 -0700
Upgrade tower dependencies (#249)
Tower must be updated in order to pickup tower-rs/tower#281
to address linkerd/linkerd2#2804.
This adopts released crates where possible.
commit 5d5eed6f8180b8db4090d995e71fdf7b0890c647
Author: Zahari Dichev <zaharidichev@gmail.com>
Date: Thu May 9 01:08:34 2019 +0300
Assert that TLS connection is refused if identity is not certified yet (#243)
This branch adds tls capability to the support cient used in tests. In addition to that it adds two tests verifying that a TLS connection is refused in case the identity is not certified yet. This attempts to fix #https://github.com/linkerd/linkerd2/issues/2598 and provide facility to write tests for https://github.com/linkerd/linkerd2/issues/2676.
As these are still some of my first lines of Rust code, it is advised to approach everything with a healthy dose of doubt :)
Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
commit 1b9bb3745e44c959d1d41d14fed2b2822c82b5ba
Author: Oliver Gould <ver@buoyant.io>
Date: Wed May 8 14:28:37 2019 -0700
Introduce dispatch timeouts around buffers (#246)
The proxy has several buffers, especially where it routes requests over
shared stacks. If any of these routes is unavailable, then a request may
remain buffered indefinitely. Previously, before service profiles were
introduced, there was a default _response_ timeout that would cause
these requests to fail; but since this response timeout is now optional
(and is only applied once the request has been routed within a proxy),
then we need a new mechanism to prevent requests from getting "stuck".
This change does the following:
- all proxied requests are annotated with a dispatch deadline;
- each time a request is bufered, a timeout is registered.
- if the timeout fires, the response exception fails, a 503 is returned,
and the request is dropped.
- if the request is processed into the inner stack, the timeout is
ignored.
The dispatch timeout limits the _time a request is buffered in a proxy_.
This is distinct from the response timeout, as the server's response may
naturally be delayed for any number of (non-proxy-related) reasons.
The `insert_target` module has been generalized to `insert` to support
setting the DispatchDeadline extension.
The `buffer` module has been augmented with generic deadline-extraction
logic.
The `svc` module now exposes its own builder type that notably adds
a `buffer_pending` helper. It's helpful to pull a builder type into the
proxy to assist debugging type errors when modifying stacks.
Fixeslinkerd/linkerd2#2779linkerd/linkerd2#2795
commit caf899557c3b041190f63544da865396231b3e30
Author: Oliver Gould <ver@buoyant.io>
Date: Fri May 3 15:55:32 2019 -0700
router: Fail requests when the route is not ready (#241)
In linkerd/linkerd2#2779, we plan to expire requests while they are
buffered. However, the router _implicitly_ buffers requests in the
executor when the inner service is not ready.
This change alters the route to wrap all inner layers in a `LoadShed`
so it can expect all services to `poll_ready()` immediately.
commit 587bad101d9e5daeacb24b6733097c350a798356
Author: Eliza Weisman <eliza@buoyant.io>
Date: Fri May 3 14:18:08 2019 -0700
Remove Destination service query concurrency limit (#244)
Currently, the proxy enforces a limit on the number of concurrent
queries (i.e., the number of gRPC streams) to the Destination service.
This limit was added based on information about the behaviour of the
Destination service that is now known to be incorrect.
This branch removes the limit on concurrent queries from the proxy's
`control::destination` module. Although it should now be possible to
simplify this code as a result of this change, I've refrained from doing
any major refactoring in this branch --- my intention is to do this
after the DNS fallback behaviour has also been removed, as together with
this change, that will result in a _significant_ simplification of the
module. Additionally, I've removed the tests for the concurrency limit,
as they are no longer relevant.
The `LINKERD2_PROXY_DESTINATION_CLIENT_CONCURRENCY_LIMIT`
environment variable was also removed; this is not a breaking change as
neither the CLI nor the proxy injector will currently set this env var.
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
commit cbdf45b44f7e4d852dc0497716062167ab9539fb
Author: Sean McArthur <sean@buoyant.io>
Date: Thu May 2 11:47:48 2019 -0700
Remove h2::Error requirement from metrics
Signed-off-by: Sean McArthur <sean@buoyant.io>
commit 3276949d4608dc4344b7bed3de2fc4b3080c2c6e
Author: Sean McArthur <sean@buoyant.io>
Date: Thu May 2 09:44:00 2019 -0700
delete unused proxy::http::metrics::class module
Signed-off-by: Sean McArthur <sean@buoyant.io>
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
The `docker-build-proxy` script builds `Dockerfile-proxy`. That
Dockerfile depends on a go-deps image, and takes a `LINKERD_VERSION`
arg. The `docker-build-proxy` script was neither ensuring go-deps had
been built, nor setting `LINKERD_VERSION`. The former resulted in the
build failing if go-deps did not exist. The latter resulted in
`dev-undefined` log messages in the `linkerd-proxy` container.
Fix `docker-build-proxy` to ensure go-deps are built, and also set the
`LINKERD_VERSION`. This brings this script more in-line with the other
`docker-build-*` scripts.
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
commit 073a1beb4a7cd709c6b1eaa56a319c1829a94d11
Author: Sean McArthur <sean@buoyant.io>
Date: Mon Apr 29 17:54:01 2019 -0700
tap: remove need to clone Services (#238)
This refactors the tap system to not require intermediary channels to
register matches and taps when a request comes through. The Dispatcher
that used to exist in order to prevent tapping more requests than the
limit asked for has been removed. In its place is a shared atomic
counter to keep the count under the limit.
The resulting behavior should be the same. There should be improved
performance as tap registration doesn't need go through a second
channel, and requests don't need to be delayed waiting for the
dispatcher to be able to process its queue.
Signed-off-by: Sean McArthur <sean@buoyant.io>
commit 7a3be8c8737188e5debbc465f9a33da0d79b8b80
Author: Zahari Dichev <zaharidichev@gmail.com>
Date: Wed May 1 01:57:01 2019 +0300
Replace fixed reconnect backoff with exponential one (#237)
When reconnecting to a destination, use an exponential, jittered backoff strategy.
Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
commit 32b813aad4fe2fcf0252e8c2215d6835101d2337
Author: Oliver Gould <ver@buoyant.io>
Date: Tue Apr 30 15:58:20 2019 -0700
Support endpoint weights (#230)
This change modifies the proxy to honor weights provided by the
destination service. When the destination service replies with a
weight, this value is divided by 10,000 to produce a weight on
[0.0, ~400000.0]. This weight is used by load the load balancer
to modify load interpretation and therefore request distribution.
A weight of 0.0 will cause the endpoint's load to be effectively infinite
so that requests will only be sent to the endpoint when no other endpoints
exists or when the other endpoints that were considered had 0-weights.
commit 501802671a346250b6dbaae73f29d9be7a4c2086
Author: Sean McArthur <sean@buoyant.io>
Date: Wed May 1 13:42:38 2019 -0700
Remove buffers from endpoint stacks (#239)
Due to the `http::settings::router`, a `buffer` was needed in each
endpoint stack. This meant that the service was always ready, even if
the client were falling over (and reconnecting). In turn, this meant
that the balancer would pick one of these endpoint stacks, because it
was always ready!
This change includes a test of a failing endpoint, that the balancer no
longer assumes it is ready, and has the following functional changes:
- Removed `http::settings::router`, instead the client HTTP settings are
detected as part of the `DstAddr`. This means that each balancer only
has endpoints with the same HTTP settings.
- Removed `buffer` layer from inside the endpoint stacks.
Signed-off-by: Sean McArthur <sean@buoyant.io>
When developing on the proxy, it's convenient to build the proxy while
the linkerd2 image is building at a given tag; but because the proxy is
built last, it's difficult to build the proxy at the same tag
simultaneously.
This is made easier by building the proxy first so that the parallel
build can be initiated after this. This shouldn't impact other
development workflows.
commit 61db2e77a247f7b0235b67581f60e8a92f8543cb
Author: Sean McArthur <sean@seanmonstar.com>
Date: Tue Apr 23 17:20:43 2019 -0700
Replace linkerd2-stack with tower-layer (#236)
Signed-off-by: Sean McArthur <sean@buoyant.io>
commit 2d6c7145cadf709832f3507bcefdaee509ebde81
Author: Sean McArthur <sean@seanmonstar.com>
Date: Thu Apr 18 12:40:48 2019 -0700
Add load shedding when over max-in-flight requests. (#225)
Also adds configuration for inbound and outbound max-in-flight requests.
Signed-off-by: Sean McArthur <sean@buoyant.io>
commit f4b5cd0b4a25d7d942e018b42af1157ae2e7dbb0
Author: Oliver Gould <ver@buoyant.io>
Date: Wed Apr 17 13:53:49 2019 -0700
Upgrade tower (#232)
This avails the proxy of newer load balancer features, an updated buffer
implementation, etc.
The new buffer implementation requires that we implement TypedExecutor
for our logging executor; and more error types have been made dynamic.
In https://github.com/linkerd/linkerd2-proxy/pull/233, we fixed an issue
in the proxy where, when the proxy performed TLS discovery (on inbound
connections), detection on a slow or idle connection could block all
other connections from being accepted on the listener.
Fixes#2581#2585#2630
* proxy: Bump pinned version to f2d907b
This change picks up:
* Added configuration for overriding the connection backoff
* Added configuration for overriding the HTTP/2 stream or connection window size
* Disable potentially info-leaking header
Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
Introduce a `bin/test-scale` script to deploy Linkerd alongside sample
apps at scale.
This script deploys the following:
- Linkerd control-plane, with service profiles
- 5 namespaces x 5 replicas of each:
- Emojivoto demo app
- Books demo app, with service profiles
- Lifecycle / bb test environment
Fixes#2517
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
The integration tests were not exercising proxy auto inject.
Introduce a `--proxy-auto-inject` flag to `install_test.go`, which
now exercises install, check, and smoke test deploy for both manual and
auto injected use cases.
Part of #2569
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
Introduce a `bin/test-clouds` and cleanup script, to run integration
tests against 4 cloud providers.
Also modify the integration tests to accept a `--context` param to
specify the Kubernetes context to run the tests against.
Fixes#2516
Signed-off-by: Andrew Seigner <siggy@buoyant.io>
Adds local and cloud integration testing for the dashboard using WebdriverIO and
SauceLabs. Includes documentation on how to set up and run the Sauce Connect
proxy locally. Adds a `bin/web integration` script that takes `local` or `cloud`
arguments to run the tests.
Note: for web development, the web server launched by `bin/web run` and `bin/web
dev` is now 7777, not 8084, because the Sauce Connect proxy can only tunnel to
certain ports.
Alejandro Pedraza
Ivan Sim
Andrew Seigner
Alex Leong
Oliver Gould
Eliza Weisman
Kevin Leimkuhler
Dennis Adjei-Baah
Carol A. Scott
Kevin Leimkuhler
Andrew Seigner