This environment variable is definted by the OpenTelemetry conventions as a way of specifying additional resource attributes to be included in traces: https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/#general-sdk-configuration
Note that this is parallel to LINKERD2_PROXY_TRACE_EXTRA_ATTRIBUTES, which is meant for common attributes that should always be provided by the proxy (pod UID, contianer name, etc.). We do not expect users to override that one, as OTEL_RESOURCE_ATTRIBUTES is the conventional way of specifying custom attributes.
Signed-off-by: Scott Fleener <scott@buoyant.io>
see https://github.com/linkerd/linkerd2/issues/8733.
we are currently in the process of upgrading our hyper, http, and http-body dependencies to their 1.0 major releases.
this branch introduces additional test coverage, and further refines existing test coverage, concerning how a `ReplayBody<B>` behaves when it reaches the "end of stream" of its internal `B`-typed body.
additional assertions are added to show that bodies with trailers may be replayed an arbitrary number of times, and that capacity errors occur precisely at their expected boundary. additional assertions are added to confirm that `ReplayBody::is_capped()` reports these conditions properly.
this branch also notably outlines the unit test suite into a separate file, due to its size. as a result, reviewers are encouraged to walk through this branch on a commit-by-commit basis when reading these changes.
i noticed some relatively minor issues with `is_end_stream()` and `size_hint()` while i was reviewing this middleware, in preparation to port it to http-body 1.0. i have left `TODO` comments noting where today's behavior is slightly askew, but _intentionally avoided_ fixing them here. my goal on that front is to highlight those wrinkles so that later fixes to these edges are more easily reviewable.
---
* refactor(http/retry): outline `ReplayBody<B>` unit tests
there are more than 500 lines of unit tests. let's move them into a
submodule, for convenience.
Signed-off-by: katelyn martin <kate@buoyant.io>
* nit(http/retry): reorganize replay tests
this is a small cosmetic change reording some test helpers.
there is a common convention of affixing a banner comment above groups
of `impl T {}` blocks, which is useful when top-level blocks are folded
in an editor.
similarly, there is a convention of defining structures at the top of a
file.
this commit reorganizes the replay body tests to follow each of these
conventions.
Signed-off-by: katelyn martin <kate@buoyant.io>
* nit(http/retry): test replays trailers twice
just to be extra sure!
Signed-off-by: katelyn martin <kate@buoyant.io>
* nit(http/retry): rename `trailers_only()` test
this is part of a family of other tests called `replays_one_chunk()`,
`replays_several_chunks()`, and `replays_trailers()`. let's name this
something that lines up with this convention.
Signed-off-by: katelyn martin <kate@buoyant.io>
* feat(http/retry): add a `TestBody` type
we have a unit test, called `eos_only_when_fully_replayed` that confirms
`Body::end_of_stream()` reports the stream ending properly.
soon, we aim to introduce additional test coverage that exercises this
when a body has trailers, as well. this will be useful for assurance
related to upgrading to http-body v1.x. see linkerd/linkerd2#8733 for
more information.
unfortunately, hyper 0.14's channel-backed body does not report itself
as having reached the end of the stream. this is an unfortunate quality
that prevents us from using `Test::new()`.
this commit adds a `TestBody` type that we can use in place of
`BoxBody::from_static(..)`, which boxes a static string, but does not
send trailers.
Signed-off-by: katelyn martin <kate@buoyant.io>
* feat(http/retry): add `is_end_stream()` coverage for trailers
this commit introduces additional test coverage that exercises
`is_end_stream()` when a replay body is wrapping a body with trailers.
this will be useful for assurance related to upgrading to http-body
v1.x. see linkerd/linkerd2#8733 for more information.
Signed-off-by: katelyn martin <kate@buoyant.io>
* feat(http/retry): add `is_capped()` test coverage
Signed-off-by: katelyn martin <kate@buoyant.io>
* feat(http/retry): further refine capacity test coverage
we want to show that exceeding the capacity is the point at which
replays will fail.
this commit defines some constants to further communicate and encode
this relationship between the bytes sent, and the capacity of the replay
body.
further, it shortens the second frame sent so that we ensure precisely
when a body becomes capped.
Signed-off-by: katelyn martin <kate@buoyant.io>
* feat(http/retry): add `size_hint()` test coverage
Signed-off-by: katelyn martin <kate@buoyant.io>
* chore(http/retry): add todo comments concerning eos
for now, a replaying body that will not yield trailers must be polled to
the `None` before reporting itself as reaching the end of the stream.
this isn't hugely important, but does affect some test control flow.
leave two todo comments so that if/when upgrading to hyper 1.0, it is
clear that these are not load-bearing or otherwise expected behavior,
should this behavior be rectified.
Signed-off-by: katelyn martin <kate@buoyant.io>
---------
Signed-off-by: katelyn martin <kate@buoyant.io>
a `ReplayBody<B>` demands a particular contract of its users: one must not poll two bodies in parallel. this is currently mentioned as an implicit assumption on an internal helper function, `acquire_state<'a>`, here:
https://github.com/linkerd/linkerd2-proxy/blob/main/linkerd/http/retry/src/replay.rs#L127-L131
```rust
/// # Panics
///
/// This panics if another clone has currently acquired the state, based on
/// the assumption that a retry body will not be polled until the previous
/// request has been dropped.
fn acquire_state<'a>(
state: &'a mut Option<BodyState<B>>,
shared: &Mutex<Option<BodyState<B>>>,
) -> &'a mut BodyState<B> {
state.get_or_insert_with(|| shared.lock().take().expect("missing body state"))
}
```
this commit echoes this sentiment on the relevant public-facing parts of this middleware: its `Body` trait methods.
Signed-off-by: katelyn martin <kate@buoyant.io>
This allows us to include arbitrary values from the k8s downward API beyond just the pod labels that are included in the trace attributes file.
See https://github.com/linkerd/linkerd2/pull/13544 for the corresponding control plane change.
Signed-off-by: Scott Fleener <scott@buoyant.io>
pr's #3564 and #3567, 1eb822f2 and 32042783 respectively, replaced uses
of defunct `http_body::Body` trait methods — namely, `data()` and
`trailers()`.
this commit updates two remaining uses of `data()` that were missed
in this initial pass.
see linkerd/linkerd2#8733 for more information.
Signed-off-by: katelyn martin <kate@buoyant.io>
based on #3564. see linkerd/linkerd2#8733.
this branch upgrades the remaining parts of the `ReplayBody<B>` test
suite to poll bodies in terms of `Frame<T>`s.
1eb822f2e6
---
* refactor(http/retry): `replays_trailers()` uses `Frame<T>`
see https://github.com/linkerd/linkerd2/issues/8733.
this commit upgrades a test that uses defunct `data()` and `trailers()`
futures.
Signed-off-by: katelyn martin <kate@buoyant.io>
* refactor(http/retry): `trailers_only()` uses `Frame<T>`
see https://github.com/linkerd/linkerd2/issues/8733.
this commit upgrades a test that uses defunct `data()` and `trailers()`
futures.
Signed-off-by: katelyn martin <kate@buoyant.io>
* feat(http/retry): `ForwardCompatibleBody::is_end_stream()`
this commit adds a method that exposes the inner `B`-typed body's
`is_end_stream()` trait method, gated for use in tests.
Signed-off-by: katelyn martin <kate@buoyant.io>
* refactor(http/retry): `body_to_string()` helper uses `Frame<T>`
this is a refactoring commit, upgrading more of the replay body test to
work in terms of `Frame<T>`. this updates the `body_to_string()` helper
in particular.
Signed-off-by: katelyn martin <kate@buoyant.io>
* refactor(http/retry): `chunk()` helper uses `Frame<T>`
Signed-off-by: katelyn martin <kate@buoyant.io>
---------
Signed-off-by: katelyn martin <kate@buoyant.io>
see linkerd/linkerd2#8733.
pr #3559 introduced some compatibility facilities to allow us to write
code in terms of `http_body_util::BodyExt::frame()`, front-running the
upgrade to be performed in #3504.
some `ReplayBody` tests use the defunct `data()` and `trailers()`
interfaces. this branch ports _two_ such unit tests. other tests are
saved for a fast follow-on, as the `chunk(..)` and `read_to_string(..)`
helpers will need some slightly more involved tweaks.
dd4fbcdb6e
---
* refactor(http/retry): `replays_trailers()` uses `Frame<T>`
see https://github.com/linkerd/linkerd2/issues/8733.
this commit upgrades a test that uses defunct `data()` and `trailers()`
futures.
Signed-off-by: katelyn martin <kate@buoyant.io>
* refactor(http/retry): `trailers_only()` uses `Frame<T>`
see https://github.com/linkerd/linkerd2/issues/8733.
this commit upgrades a test that uses defunct `data()` and `trailers()`
futures.
Signed-off-by: katelyn martin <kate@buoyant.io>
---------
Signed-off-by: katelyn martin <kate@buoyant.io>
this branch contains a sequence of commits that refactor `PeekTrailersBody<B>`.
this branch is specifically focused on making this body middleware
forward-compatible with the 1.0 interface(s) of `http_body::Body` and
`http_body_util::BodyExt`.
it does this in two main steps: (1) temporarily vendoring `http_body::Frame<T>`
and providing a compatibility shim that provides a `frame()` method for a body,
and (2) modeling `PeekTrailersBody<B>` and its peeking logic in terms of this
`Frame<'a, T>` future.
---
* feat(http/retry): add `Frame<T>` compatibility facilities
this commit introduces a `compat` submodule to `linkerd-http-retry`.
this helps us frontrun the task of replacing all of the finicky control
flow in `PeekTrailersBody<B>` using the antiquated `data()` and
`trailers()` future combinators. instead, we can perform our peeking
in terms of an approximation of `http_body_util::BodyExt::frame()`.
to accomplish this, this commit vendors a copy of the `Frame<T>` type.
we can use this to preemptively model our peek body in terms of this
type, and move to the "real" version of it when we're upgrading in
pr #3504.
additionally, this commit includes a type called
`ForwardCompatibleBody<B>`, and a variant of the `Frame<'a, T>`
combinator. these are a bit boilerplate-y, admittedly, but the pleasant
part of this is that we have, in effect, migrated the trickiest body
middleware in advance of #3504. once we upgrade to http-body 1.0, all of
these types can be removed.
https://docs.rs/http-body-util/latest/http_body_util/trait.BodyExt.html#method.framehttps://docs.rs/http-body-util/0.1.2/src/http_body_util/combinators/frame.rs.html#10
Signed-off-by: katelyn martin <kate@buoyant.io>
* refactor(http/retry): `PeekTrailersBody<B>` uses `BodyExt::frame()`
this commit reworks `PeekTrailersBody<B>`.
the most important goal here is replacing the control flow of
`read_body()`, which polls a body using `BodyExt` future combinators
`data()` and `frame()` for up to two frames, with varying levels of
persistence depending on outcomes.
to quote #3556:
> the intent of this type is to only yield the asynchronous task
> responsible for reading the body once. depending on what the inner
> body yields, and when, this can result in combinations of: no data
> frames and no trailers, no data frames with trailers, one data frame
> and no trailers, one data frame with trailers, or two data frames.
> moreover, depending on which of these are yielded, the body will call
> .await some scenarios, and only poll functions once in others.
>
> migrating this to the Frame<T> and poll_frame() style of the 1.0 Body
> interface, away from the 0.4 interface that provides distinct
> poll_data() and poll_trailers() methods, is fundamentally tricky.
this means that `PeekTrailersBody<B>` is notably difficult to port
across the http-body 0.4/1.0 upgrade boundary.
this body middleware must navigate a number of edge conditions, and once
it _has_ obtained a `Frame<T>`, make use of conversion methods to
ascertain whether it is a data or trailers frame, due to the fact that
its internal enum representation is not exposed publicly. one it has
done all of that, it must do the same thing once more to examine the
second frame.
this commit uses the compatibility facilities and backported `Frame<T>`
introduced in the previous commit, and rewrites this control flow using
a form of the `BodyExt::frame()` combinator.
this means that this middleware is forward-compatible with http-body
1.0, which will dramatically simplify the remaining migration work to be
done in #3504.
see https://github.com/linkerd/linkerd2/issues/8733 for more information
and other links related to this ongoing migration work.
Signed-off-by: katelyn martin <kate@buoyant.io>
* refactor(http/retry): mock body enforces `poll_trailers()` contract
this commit addresses a `TODO` note, and tightens the enforcement of a
rule defined by the v0.4 signature of the `Body` trait.
this commit changes the mock body type, used in tests, so that it will
panic if the caller improperly polls for a trailers frame before the
final data frame has been yielded.
previously, a comment indicated that we were "fairly sure" this was
okay. while that may have been acceptable in practice, the changes in
the previous commit mean that we now properly respect these rules.
thus, a panic can be placed here, to enforce that "[is] only be called
once `poll_data()` returns `None`", per the documentation.
Signed-off-by: katelyn martin <kate@buoyant.io>
* refactor(http/retry): rename `PeekTrailersBody::Buffered`
<https://github.com/linkerd/linkerd2-proxy/pull/3559#discussion_r1928953710>
this is a nicer name than `Unknown` for this case. not to mention, we'll
want that name shortly to address the possibility of unknown frame
variants.
Signed-off-by: katelyn martin <kate@buoyant.io>
* refactor(http/retry): encapsulate `Inner<B>` enum variants
this commit makes the inner enum variants private.
https://github.com/linkerd/linkerd2-proxy/pull/3559#discussion_r1928946521
Signed-off-by: katelyn martin <kate@buoyant.io>
* refactor(http/retry): gracefully ignore unknown frames
https://github.com/linkerd/linkerd2-proxy/pull/3559#discussion_r1928963019
Signed-off-by: katelyn martin <kate@buoyant.io>
---------
Signed-off-by: katelyn martin <kate@buoyant.io>
The `authority` label is influenced by `:authority` headers and can therefore
can substantially grow in cardinality, depending on traffic patterns.
This change completely removes the authority label from inbound metrics.
Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
this commit fixes a small, subtle bug in `PeekTrailersBody<B>`.
if wrapping an empty body, the peek body will inspect the wrong
`Option<T>` in the trailers field with the following type:
```rust
/// The inner body's trailers, if it was terminated by a `TRAILERS` frame
/// after 0-1 DATA frames, or an error if polling for trailers failed.
///
/// Yes, this is a bit of a complex type, so let's break it down:
/// - the outer `Option` indicates whether any trailers were received by
/// `WithTrailers`; if it's `None`, then we don't *know* if the response
/// had trailers, as it is not yet complete.
/// - the inner `Result` and `Option` are the `Result` and `Option` returned
/// by `HttpBody::trailers` on the inner body. If this is `Ok(None)`, then
/// the body has terminated without trailers --- it is *known* to not have
/// trailers.
trailers: Option<Result<Option<http::HeaderMap>, B::Error>>,
```
for an empty body, we *know* that there are no trailers, which means
that we have `Some(Ok(None))`.
consider also, the documentation of `is_end_stream()`:
> An end of stream means that both poll_data and poll_trailers will
> return None.
>
> A return value of false does not guarantee that a value will be
> returned from poll_stream or poll_trailers.
we can guarantee in this case that `poll_trailers()` will return
`Ok(None)` since we've already called it and proven that to be the case.
we *are* holding that value, after all.
this change will not affect any behavior w.r.t. what the peek body
yields, but it will mean that it reports `is_end_stream()` correctly
when it wraps an empty body.
Signed-off-by: katelyn martin <kate@buoyant.io>
* Revert "Revert "chore(deps): Upgrade tokio-rustls to 0.26 (#3419)""
This reverts commit de25333e23.
* Expand the set of supported signature algorithms
During the rustls upgrade, it accidentally limited the set of supported signature algorithms to ECDSA256 signatures. This would cause the identity control plane proxy to reject all certify requests with BadSignature if an RSA certificate was used instead of ECDSA.
This updates the set of supported algorithms to most of the full set of what rustls+ring supports, minus a few legacy algorithms.
Tested by deploying to a local cluster and verifying the control plane comes up correctly and app-level networking works as expected.
Signed-off-by: Scott Fleener <scott@buoyant.io>
---------
Signed-off-by: Scott Fleener <scott@buoyant.io>
`PeekTrailersBody<B>` contains some subtle edge cases related to the
number of DATA frames yielded by the inner body, and how persistent it
will be about polling for TRAILERS frames.
for example, if it yields a DATA frame, it will not await trailers being
available, but it *will* do so if the inner body does not yield a DATA
frame. if a DATA frame is yielded, it will check for a TRAILERS frame,
but it must be immmediately available.
this is all subtle, and particularly subject to change with the upgrade
to http-body 1.0's frame-oriented `Body` interface.
so, this commit introduces a test suite for `PeekTrailersBody<B>`. it
includes assertions to confirm when the peek middleware can and cannot
observe the trailers.
some `TODO(kate)` comments are left where issues exist.
Signed-off-by: katelyn martin <kate@buoyant.io>
this is a squashed commit containing the following:
---
refactor(http/retry): decompose `WithPeekTrailersBody<B>` type alias
this commit breaks this large type out into two halves.
this is a purely cosmetic change.
Signed-off-by: katelyn martin <kate@buoyant.io>
refactor(http/retry): `PeekTrailersBody` is pin projected
we must pass our `Pin<T>`'edness down to the inner `B`-typed body for
`PeekTrailersBody` to itself implement `http_body::Body`.
this commit tweaks the existing code to rely on the `pin-project`
library. this generates a `project()` method to pin inner fields whose
`poll_data()` and `poll_trailers()` functions we delegate to.
this is a noöp change.
Signed-off-by: katelyn martin <kate@buoyant.io>
refactor(http/retry): defer construction of `PeekTrailersBody<B>`
this commit refactors the polling logic in
`PeekTrailersBody<B>::read_response`.
this commit makes some subtle changes with the migration to hyper 1.0 in
mind, to make this function more easily portable to the new signature of
`http_body::Body`.
see https://github.com/linkerd/linkerd2/issues/8733 for more
information.
this commit defers the `Self` construction of the `PeekTrailersBody`
body. this means that the control flow does not need to reach through to
e.g. `body.inner` to poll the inner body being peeked. additionally, it
provides us with `let` bindings for the first data frame yielded, and
the trailers frame yielded thereafter.
this is largely cosmetic, but will make it easier to deal with the
additional matching we'll need to do when there is a single polling
function that yields us `Frame<D>` objects.
Signed-off-by: katelyn martin <kate@buoyant.io>
refactor(http/retry): `PeekTrailersBody` transforms `B`
this is a small structural change to the
`PeekTrailersBody::read_response()` function to facilitate writing some
unit tests.
rather than transforming a `Response<B>` into a
`Response<PeekTrailersBody<B>>`, we hoist the `Response::into_parts()`
and `Response::from_parts()` calls up. `read_response()` is renamed to
`read_body()`.
Signed-off-by: katelyn martin <kate@buoyant.io>
this is a squashed commit containing the following:
---
docs(http/retry): document `PeekTrailersBody::inner`
Signed-off-by: katelyn martin <kate@buoyant.io>
docs(http/retry): document `PeekTrailersBody::peek_trailers()`
Signed-off-by: katelyn martin <kate@buoyant.io>
docs(http/retry): document `PeekTrailersBody::no_trailers()`
Signed-off-by: katelyn martin <kate@buoyant.io>
* refactor(http/upgrade): `Http11Upgrade::insert_half` matches on `self`
this is a noöp change, to set the stage for subsequent changes to the
internal model of `Http11Upgrade`.
this `inner` field will shortly be an option, and this will make it
easier to only follow these panicking branches when the inner lock is
`Some(_)`.
Signed-off-by: katelyn martin <kate@buoyant.io>
* refactor(http/upgrade): `Http11Upgade` stores an `Option<T>`
this commit hinges on this change to the upgrade middleware's `inner`
field.
we still retain a reference-counted copy of the `Inner` state, but now
we may store `None` here.
```
pub struct Http11Upgrade {
half: Half,
- inner: Arc<Inner>,
+ inner: Option<Arc<Inner>>,
}
```
a new branch is added to the `insert_half` method that consumes the
"sender" and inserts an upgrade future; when this is `None` it will do
nothing, rather than panicking.
Signed-off-by: katelyn martin <kate@buoyant.io>
* refactor(http/upgrade): `Half` marker is `Copy`
this type is an empty flag to indicate whether an `Http11Upgrade`
extension corresponds to the server or client half of the upgrade
future channel.
this type is made copy, to facilitate making the `Http11Upgrade`
extension safely cloneable.
Signed-off-by: katelyn martin <kate@buoyant.io>
* refactor(http/upgrade): `Http11Upgrade` is `Clone`
this commit makes `Http11Upgrade` a cloneable type.
see <https://github.com/linkerd/linkerd2/issues/8733>.
in the 1.0 interface of the `http` crate, request and response
extensions must now satisfy a `Clone` bound.
`Http11Upgrade` was written before this was the case, and is very
intentionally designed around the idea that it *not* be cloneable.
`insert_half()` in particular could cause the proxy to panic if it were
to clone a request or response's extensions. it might call
`insert_half()` a second time, and discover that the `TryLock<T>` had
already been set.
moreover, holding on to a copy of the extensions would prevent the
`Drop` method for `Inner` from being called. This would cause
connections that negotiate an HTTP/1.1 upgrade to deadlock due to the
`OnUpgrade` futures never being polled, and failing to create a `Duplex`
that acts as the connection's I/O transport.
this commit makes use of the alterations to `Http11Upgrade` made in
previous commits, and adds a *safe* implementation of `Clone`. by
only shallowly copying the extension, we tie the upgrade glue to a
*specific* request/response.
the extension can be cloned, but any generated copies will be inert.
Signed-off-by: katelyn martin <kate@buoyant.io>
* chore(http/upgrade): fix broken intradoc links
Signed-off-by: katelyn martin <kate@buoyant.io>
* chore(http/upgrade): add `thiserror` dependency
Signed-off-by: katelyn martin <kate@buoyant.io>
* refactor(proxy/http): use `.await` syntax
`FutureExt::map_ok()` won't work if we try to return an error from this
block. the `and_then()` adaptor is used to chain futures, and also won't
work given a synchronous closure.
this can be done with the equivalent `.await` syntax, and leaves a nicer
hole for us to propagate other errors here, shortly.
Signed-off-by: katelyn martin <kate@buoyant.io>
* review(http/upgrade): propagate `insert_half()` failures
https://github.com/linkerd/linkerd2-proxy/pull/3540#discussion_r1924496977
Signed-off-by: katelyn martin <kate@buoyant.io>
Co-Authored-By: Oliver Gould <ver@buoyant.io>
* docs(http/upgrade): tweak comment
Signed-off-by: katelyn martin <kate@buoyant.io>
---------
Signed-off-by: katelyn martin <kate@buoyant.io>
Co-authored-by: Oliver Gould <ver@buoyant.io>
The proxy::http::Version type is very similar to the HTTP crate's Version type,
though it is more constrained so that the proxy may exhaustively match on it.
This change renames the module to http::variant to avoid confusion with the HTTP
crate's Version type.
To disambiguate the HTTP version type, the proxy::http::Version type is renamed
to proxy::http::Variant.
dependabot[bot]
Scott Fleener
katelyn martin
Zahari Dichev
Oliver Gould