Currently, we store whether or not a port in a parent is opaque with a simple boolean. This works, but is somewhat restrictive if we want to specifiy different port protocols in the resource definition.
This converts it to a proper `AppProtocol` enum, which will allow us to use different protocol specifications for ports in the future.
Signed-off-by: Scott Fleener <scott@buoyant.io>
This PR adds a few notable changes associated with the egress functionality of Linkerd:
- `EgressNetwork` objects are indexed into the outbound index
- outbound policy lookups are classfieid as either in-cluster or egress based on the `ip:port` combination
- `TCPRoute`, `TLSRoute`, `GRPCRoute` and `HTTPRoute` attachments are reflected for both `EgressNetwork` and `Service` targets
- the default traffic policy for `EgressNetwork` is honored by returning the appropriate default (failure/success) routes for all protocols
Note that this PR depends on an unreleased version of the linkerd2-proxy-api repo.
Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
Adds support for configuring retries and timeouts as outbound policy. Http retries can be configured as annotations on HttpRoute or Service resources like
```
retry.linkerd.io/http: 5xx,gateway-error
retry.linkerd.io/limit: "2"
retry.linkerd.io/timeout: 400ms
```
If any of these retry annotations are specified on an HttpRoute resource, they will override ALL retry annotations on the parent Service resource.
Similarly, Grpc retries can be configured as annotations on GrpcRoute or Service resources like
```
retry.linkerd.io/grpc: cancelled,deadline-exceeded,internal,resource-exhausted,unavailable
retry.linkerd.io/limit: "2"
retry.linkerd.io/timeout: 400ms
```
Outbound timeouts can be configured on HttpRoute, GrpcRoute, or Service resources like
```
timeout.linkerd.io/request: 500ms
timeout.linkerd.io/response: 100ms
timeout.linkerd.io/idle: 50ms
```
If any of these timeout annotations are specified on a HttpRoute or GrpcRoute resource, they will override ALL timeout annotations on the parent Service resource.
Signed-off-by: Alex Leong <alex@buoyant.io>
Co-authored-by: Oliver Gould <ver@buoyant.io>
We add support for grpcroute in the inbound policy API. When a Server resource has the proxy protocol set to grpc, we will now serve grpc as the protocol in the inbound policy API along with any GrpcRoutes which have been defined and attached to the Server. If grpc is specified as the proxy protocol but no GrpcRoutes are attached, we return a default catch-all grpc route.
Signed-off-by: Alex Leong <alex@buoyant.io>
We update the policy-controller to support grpcroutes in the outbound policy API. When a GrpcRoute resource has a Service as a parent ref, outbound policy requests for that service may return a proxy protocol of grpc with grpc routes.
* if a service has no HttpRoute or GrpcRoutes as children, we continue to return the default http route with a proxy protocol of Detect (so that the proxy is directed to detect Http1 vs Http2)
* similarly, if a service has HttpRoute children only, we continue to return those routes with a proxy protocol of Detect
* if a service has GrpcRoute children only, we return those routes with a proxy protocol of Grpc
* if a service has both types of routes as children, we determine the proxy protocol based on which route type has the oldest created_at timestamp as described in https://gateway-api.sigs.k8s.io/geps/gep-1016/#cross-serving and only return routes of the determined type
Signed-off-by: Alex Leong <alex@buoyant.io>
## Subject
Prepare to expand the subset of [`Gateway API`](https://gateway-api.sigs.k8s.io/api-types/grpcroute/) route types `linkerd` supports driving outbound traffic with in [`linkerd-policy-controller-k8s-index`](https://github.com/linkerd/linkerd2/tree/main/policy-controller/k8s/status).
## Problem
Currently, the policy controller's `index` component is written with `HTTPRoute` support (effectively) exclusively, both in its structure/organization as well as its naming (e.g. `HttpRoute` as a primary type name, `convert_http_route` as a method name, etc...).
In order to expand the subset of route types defined by the Gateway API that `linkerd` supports for driving outbound traffic policy, the policy controller's `index` component needs to be made more generic in both respects.
## Solution
PR introduces structural and naming changes making the codebase generic with respect to the type of route being handled (e.g. `HTTPRoute` -> `Route`). Changes are largely cosmetic, with no _behavioral_ changes introduced by any functional refactoring.
Signed-off-by: Mark S <the@wondersmith.dev>
## Subject
Prepare to expand `linkerd`'s repertoire of supported [`Gateway API`](https://gateway-api.sigs.k8s.io/api-types/grpcroute/) route types in [`linkerd-policy-controller-k8s-status`](https://github.com/linkerd/linkerd2/tree/main/policy-controller/k8s/status).
## Problem
Currently, the policy controller's `status` component is written with `HTTPRoute` support (effectively) exclusively, both in its structure/organization as well as its naming (e.g. `HttpRoute` as a primary type name, `update_http_route` as a method name, etc...).
In order to expand `linkerd`'s support for the route types defined by the Gateway API, the policy controller's `status` component needs to be made more generic in both respects.
## Solution
> **NOTE:** PR was opened out of order and should only be merged _after_ #12662
PR introduces structural and naming changes making the codebase generic with respect to the type of route being handled (e.g. `HTTPRoute` -> `Route`). Changes are almost entirely cosmetic introducing only a couple of minor functional changes, most notably:
- making the `status` argument to [`make_patch`](8d6cd57b70/policy-controller/k8s/status/src/index.rs (L734)) generic
- adding a type-aware `api_version` helper method to [`NamespaceGroupKindName`](8d6cd57b70/policy-controller/k8s/status/src/resource_id.rs (L27))
- **note:** *required for proper handling of different route types in the future*
## Validation
- [X] maintainer review
- [X] tests pass
## ~~Fixes~~ *Lays Groundwork For Addressing*
- #12404
Signed-off-by: Mark S <the@wondersmith.dev>
If an HTTPRoute references a backend service that does not exist, the policy controller synthesizes a FailureInjector in the outbound policy so that requests to that backend will fail with a 500 status code. However, we do not update the policy when backend services are created or deleted, which can result in an outbound policy that synthesizes 500s for backends, even if the backend currently exists (or vice versa).
This is often papered over because when a backend service is created or deleted, this will trigger the HTTPRoute's ResolvedRef status condition to change which will cause a reindex of the HTTPRotue and a recomputation of the backends. However, depending on the specific order that these updates are processed, the outbound policy can still be left with the incorrect backend state.
In order to be able to update the backend of an outbound policy when backend services are created or deleted, we change the way these backends are represented in the index. Previously, we had represented backends which were services that did not exist as `Backend::Invalid`. However, this discards the necessary backend information necessary to recreate the backend if the service is created. Instead, we update this to represent these backends as a `Backend::Service` but with a new field `exists` set to false. This allows us to update this field as backend services are created or deleted.
Signed-off-by: Alex Leong <alex@buoyant.io>
The [xRoute Binding KEP](https://gateway-api.sigs.k8s.io/geps/gep-1426/#namespace-boundaries) states that HttpRoutes may be created in either the namespace of their parent Service (producer routes) or in the namespace of the client initiating requests to the service (consumer routes). Linkerd currently only indexes producer routes and ignores consumer routes.
We add support for consumer routes by changing the way that HttpRoutes are indexed. We now index each route by the namespace of its parent service instead of by the namespace of the HttpRoute resource. We then further subdivide the `ServiceRoutes` struct to have a watch per-client-namespace instead of just a single watch. This is because clients from different namespaces will have a different view of the routes for a service.
When an HttpRoute is updated, if it is a producer route, we apply that HttpRoute to watches for all of the client namespaces. If the route was a consumer route, then we only apply the HttpRoute to watches for that consumer namespace.
We also add API tests for consumer and producer routes.
A few noteworthy changes:
* Because the namespace of the client factors into the lookup, we had to change the discovery target to a type which includes the client namespace.
* Because a service may have routes from different namespaces, the route metadata now needs to track group, kind, name, AND namespace instead of just using the namespace of the service. This means that many uses of the `GroupKindName` type are replaced with a `GroupKindNamespaceName` type.
Signed-off-by: Alex Leong <alex@buoyant.io>
We add support for the RequestHeaderModifier and RequestRedirect HTTP filters. The policy controller reads these filters in any HttpRoute resource that it indexes (both policy.linkerd.io and gateway.networking.k8s.io) and returns them in the outbound policy API. These filters may be added at the route rule level and at the backend level.
We add outbound api tests for this behavior for both types of HttpRoute.
Incidentally we also fix a flaky test in the outbound api tests where a watch was being recreated partway through a test, leading to a race condition.
Signed-off-by: Alex Leong <alex@buoyant.io>
Updates the policy-controller to watch `httproute.gateway.networking.k8s.io` resources in addition to watching `httproute.policy.linkerd.io` resources. Routes of either or both types can be returned in policy responses and will be appropriately identified by the `group` field on their metadata. Furthermore we update the Status of these resources to correctly reflect when they are accepted.
We add the `httproute.gateway.networking.k8s.io` CRD to the Linkerd installed CRD list and add the appropriate RBAC to the policy controller so that it may watch these resources.
Signed-off-by: Alex Leong <alex@buoyant.io>
PR #10969 adds support for the GEP-1742 `timeouts` field to the
HTTPRoute CRD. This branch implements actual support for these fields in
the policy controller. The timeout fields are now read and used to set
the timeout fields added to the proxy-api in
linkerd/linkerd2-proxy-api#243.
In addition, I've added code to ensure that the timeout fields are
parsed correctly when a JSON manifest is deserialized. The current
implementation represents timeouts in the bindings as a Rust
`std::time::Duration` type. `Duration` does implement
`serde::Deserialize` and `serde::Serialize`, but its serialization
implementation attempts to (de)serialize it as a struct consisting of a
number of seconds and a number of subsecond nanoseconds. The timeout
fields are instead supposed to be represented as strings in the Go
standard library's `time.ParseDuration` format. Therefore, I've added a
newtype which wraps the Rust `std::time::Duration` and implements the
same parsing logic as Go. Eventually, I'd like to upstream the
implementation of this to `kube-rs`; see kube-rs/kube#1222 for details.
Depends on #10969
Depends on linkerd/linkerd2-proxy-api#243
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
Controller currently serves hardcoded configuration values for failure
accrual parameters. This change adds support to discover configuration
on Service objects based on annotations.
Signed-off-by: Matei David <matei@buoyant.io>
Signed-off-by: Alex Leong <alex@buoyant.io>
Co-authored-by: Alex Leong <alex@buoyant.io>
This change improves the modularity of the policy controller by splitting code related in inbound policy and outbound policy into separate modules. There are no functional changes in this PR, only moves and renames. We have moved any code which was used by both inbound and outbound into common modules so that there are no cross dependencies between inbound and outbound.
Additionally, several types that had names beginning with "Inbound" or "Outbound" have had this prefix removed since they are now distinguishable by their modules. e.g. `InboundHttpRoute` becomes `inbound::HttpRoute`.
Signed-off-by: Alex Leong <alex@buoyant.io>
When the outbound policy API was returning backends which corresponded to Service resources, those backends were missing metadata information.
We update the policy controller to populate Service metadata for those backends.
Signed-off-by: Alex Leong <alex@buoyant.io>
Proxies in ingress and gateway configurations may discover policies by
name instead of network address.
This change updates the policy controller to parse service names from
lookups.
This removes the separate naming of “status controller” from the policy
controller resources, code, and comments. There is a single controller in all of
this — the policy controller. Part of the policy controller is maintaining the
status of policy resources. We can therefore remove this separate naming that
has been used as well as reorganize some of the code to use single naming
consts.
The lease resource has been renamed from `status-controller` to
`policy-controller-write`.
The controller name value has been renamed from
`policy.linkerd.io/status-controller` to `linkerd.io/policy-controller`. This
field appears in the `status` of HTTPRoutes indicating which controller applied
the status.
Lastly, I’ve updated the comments to remove the use of “status controller” and
moved the use of the name const to the `linkerd-policy-controller-core` package
so that it can be shared.
Signed-off-by: Kevin Leimkuhler <kleimkuhler@icloud.com>
We have a `ClusterInfo` type that includes all cluster-level
configuration. The outbound indexer does not use this type and, instead,
passes around individual configuration values.
This change updates the outbound indexer to use the ClusterInfo
configuration type. It also moves the type to its own module.
This is largely a reorganization PR, though there is one notable change:
InvalidDst response no longer reference a backend. Instead, the backend
is left empty, since it's not possible to route requests to it.
Implement the outbound policy API as defined in the proxy api: https://github.com/linkerd/linkerd2-proxy-api/blob/main/proto/outbound.proto
This API is consumed by the proxy for the routing of outbound traffic. It is intended to replace the GetProfile API which is currently served by the destination controller. It has not yet been released in a proxy-api release, so we take a git dependency on it in the mean time.
This PR adds a new index to the policy controller which indexes HTTPRoutes and Services and uses this information to serve the outbound API. We also add outbound API tests to validate the behavior of this implementation.
Signed-off-by: Alex Leong <alex@buoyant.io>
Co-authored-by: Oliver Gould <ver@buoyant.io>
Currently, if no `HTTPRoute` resources reference a `Server`, the policy
controller returns an empty list of routes. This results in the proxy
404ing HTTP traffic. Instead, we should return a default empty route in
this case.
This branch changes the policy controller to return a default route for
`Server`s which are not referenced by any `HTTPRoute`s. The default
route defines a single `HttpRouteRule` with no matches and no filters,
so it will match any HTTP request. The default route does *not* define
any authorizations, as they would potentially clobber authorizations
defined by other resources that define authorization policies targeting
that `Server` --- if it is not targeted by any other resource defining
authorization policies, the `Server` itself will still get the default
authz policy.
In addition, this branch changes the various `${Resource}Ref` enums so
that their `Default` variants take a `&'static str` rather than a
`String`, to minimize string copying, since the default names are always
pre-defined. It also adds an `InboundHttpRouteRef` type which is used as
a key in the maps of `HTTPRoute`s (instead of `String`s), since we now
define default routes as well.
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
Co-authored-by: Oliver Gould <ver@buoyant.io>
When there are multiple equivalent routes (e.g., two routes with the
same match), the proxy will use the first route in the returned list. We
need to ensure that the policy controller returns routes in a
deterministic order--and the Gateway API defines such an order:
> If ties still exist across multiple Routes, matching precedence MUST
> be determined in order of the following criteria, continuing on ties:
>
> * The oldest Route based on creation timestamp.
> * The Route appearing first in alphabetical order by
> "{namespace}/{name}".
This branch updates the policy controller to return the list of
`HttpRoute`s for an inbound server with a deterministic ordering based
on these rules. This is done by tracking the creation timestamp for
indexed `HTTPRoute` resources, and sorting the list of protobuf
`HttpRoute`s when the API server constructs an `InboundServer` response.
The implementation is *somewhat* hairy, because we can't just define a
custom `Ord` implementation for the protobuf `HttpRoute` type that
includes the timestamp --- doing so would require actually storing the
creation timestamp in the protobuf type, which would be a change in
`linkerd2-proxy-api` (and would result in serializing additional
information that the proxy itself doesn't actually care about). Instead,
we use `slice::sort_by` with a closure that looks up routes by name in
the hash map stored by the indexer in order to determine their
timestamps, and implements a custom ordering that first compares the
timestamp, and falls back to comparing the route's name if the
timestamps are equal. Note that we don't include the namespace in that
comparison, because all the routes for a given `InboundServer` are
already known to be in the same namespace.
I've also added an end-to-end test that the API returns the route list
in the correct order. Unfortunately, this test has 4 seconds of `sleep`s
in it, because the minimum resolution of Kubernetes creation timestamps
is 1 second. I figured a test that takes five or six seconds to run was
probably not a huge deal in the end to end tests --- some of the policy
tests take as long as a minute to run, at least on my machine.
Closes#8946
This change updates the policy controller to admit `AuthorizationPolicy` resources
that reference `HTTPRoute` parents. These policies configure proxies to augment
server-level authorizations with per-route authorizations.
Fixes#8890
Signed-off-by: Alex Leong <alex@buoyant.io>
In various places we read port configurations from external sources
(either the Kubernetes API or gRPC clients). We have manual checks in
place to ensure that port values are never zero. We can instead assert
this with the type system by using `NonZeroU16`.
This change updates the policy controller to use `NonZeroU16` for port
values. This allows us to replace our manual port value checks with
`NonZero::try_from`, etc.
Signed-off-by: Oliver Gould <ver@buoyant.io>
linkerd2-proxy-api v0.6.0 adds support for inbound proxies to discover
route configurations based on the Gateway API HTTPRoute types. This
change updates the policy controller to index
`gateway.networking.k8s.io/v1beta` `HTTPRoute` types to discover these
policies from the Kubernetes API.
`HTTPRoute` resources may target `Server` resources (as a `parentRef`)
to attach policies to an inbound proxy. When no routes are configured,
a default route is synthesized to allow traffic; but when at least one
route attaches to a server, only requests that match a route are
permitted (other requests are failed with a 404).
Only the *core* subset of the `HTTPRoute` filters are supported:
`RequestRedirect` and `RequestHeaderModifier`. Backends may *not* be
configured on these routes (since they may only apply to inbound/server-
side proxies). No `status` updates are currently performed on these
`HTTPRoute` resources.
This change does not yet allow `AuthorizationPolicy` resources to target
`HTTPRoute` resources. This will be added in a follow-up change.
Signed-off-by: Alex Leong <alex@buoyant.io>
Co-authored-by: Oliver Gould <ver@buoyant.io>
Issue #7709 proposes new Custom Resource types to support generalized
authorization policies:
- `AuthorizationPolicy`
- `MeshTLSAuthentication`
- `NetworkAuthentication`
This change introduces these CRDs to the default linkerd installation
(via the `linkerd-crds` chart) and updates the policy controller's
to handle these resource types. The policy admission controller
validates that these resource reference only suppported types.
This new functionality is tested at multiple levels:
* `linkerd-policy-controller-k8s-index` includes unit tests for the
indexer to test how events update the index;
* `linkerd-policy-test` includes integration tests that run in-cluster
to validate that the gRPC API updates as resources are manipulated;
* `linkerd-policy-test` includes integration tests that exercise the
admission controller's resource validation; and
* `linkerd-policy-test` includes integration tests that ensure that
proxies honor authorization resources.
This change does NOT update Linkerd's control plane and extensions to
use these new authorization primitives. Furthermore, the `linkerd` CLI
does not yet support inspecting these new resource types. These
enhancements will be made in followup changes.
Signed-off-by: Oliver Gould <ver@buoyant.io>
The identity-string parsing logic is currently implemented directly in
the ServerAuthorization indexer. In preparation of this being needed in
additional modules, this change extracts identity parsing and adds some
basic tests.
Signed-off-by: Oliver Gould <ver@buoyant.io>
While looking into replacing `dashmap` in `kube-rs`
(kube-rs/kube-rs#785), I realized that we're probably better off using
[`ahash`][ahash] in our indexing code. AHash pruports to the be "the
fastest DoS-resistant hash currently available in Rust." According to
<https://lib.rs/crates/ahash>, it has substantial usage.
This change should help to minimize lock contention (i.e. by speeding up
hashing while a lock is held). In general, this looks to be a superior
default to `std::collections::HashMap`.
Signed-off-by: Oliver Gould <ver@buoyant.io>
[ahash]: e77cab8c1e/compare/readme.md
Policy controller API responses include a set of labels. These labels
are to be used in proxy m$etrics to indicate why traffic is permitted to
a pod. This permits metrics to be associated with `Server` and
ServerAuthorization` resources (i.e. for `stat`).
This change updates the response API to include a `name` label
referencing the server's name. When the policy is derived from a default
configuration (and not a `Server` instance), the name takes the form
'default:<policy>'.
This change also updates authorization labels. Defaults are encoded as
servers are, otherwise the authorization's name is set as a label. The
`tls` and `authn` labels have been removed, as they're redundant with
other labels that are already present.
Pods may be annotated with annotations like
`config.linkerd.io/opaque-ports` and
`config.linkerd.io/proxy-require-identity-inbound-ports`--these
annotations configure default behavior that should be honored when a
`Server` does not match the workload's ports. As it stands now, the
policy controller would break opaque-ports configurations that aren't
reflected in a `Server`.
This change reworks the pod indexer to create a default server watch for
each _port_ (rather than for each pod). The cache of default server
watches is now lazy, creating watches as needed for all used
combinations of default policies. These watches are never dropped, but
there are only a few possible combinations of port configurations, so
this doesn't pose any concerns re: memory usage.
While doing this, the names used to describe these default policies are
updated to be prefixed with `default:`. This generally makes these names
more descriptive and easier to understand.
We've implemented a new controller--in Rust!--that implements discovery
APIs for inbound server policies. This change imports this code from
linkerd/polixy@25af9b5e.
This policy controller watches nodes, pods, and the recently-introduced
`policy.linkerd.io` CRD resources. It indexes these resources and serves
a gRPC API that will be used by proxies to configure the inbound proxy
for policy enforcement.
This change introduces a new policy-controller container image and adds a
container to the `Linkerd-destination` pod along with a `linkerd-policy` service
to be used by proxies.
This change adds a `policyController` object to the Helm `values.yaml` that
supports configuring the policy controller at runtime.
Proxies are not currently configured to use the policy controller at runtime. This
will change in an upcoming proxy release.
Scott Fleener
Alejandro Pedraza
Zahari Dichev
Alex Leong
Mark S
Eliza Weisman
Oliver Gould
Matei David
Kevin Leimkuhler