According to the [xRoutes Mesh Binding KEP](https://gateway-api.sigs.k8s.io/geps/gep-1426/#ports), the port in a parent reference is optional:
> By default, a Service attachment applies to all ports in the service. Users may want to attach routes to only a specific port in a Service. To do so, the parentRef.port field should be used.
> If port is set, the implementation MUST associate the route only with that port. If port is not set, the implementation MUST associate the route with all ports defined in the Service.
However, we currently ignore any HttpRoutes which don't have a port specified in the parent ref.
We update the policy controller to apply HttpRoutes which do not specify a port in the parent ref to all ports of the parent service.
We do this by storing these "portless" HttpRoutes in the index and then copying these routes into every port-specific watch for that service.
Signed-off-by: Alex Leong <alex@buoyant.io>
We add support for the RequestHeaderModifier and RequestRedirect HTTP filters. The policy controller reads these filters in any HttpRoute resource that it indexes (both policy.linkerd.io and gateway.networking.k8s.io) and returns them in the outbound policy API. These filters may be added at the route rule level and at the backend level.
We add outbound api tests for this behavior for both types of HttpRoute.
Incidentally we also fix a flaky test in the outbound api tests where a watch was being recreated partway through a test, leading to a race condition.
Signed-off-by: Alex Leong <alex@buoyant.io>
Updates the policy-controller to watch `httproute.gateway.networking.k8s.io` resources in addition to watching `httproute.policy.linkerd.io` resources. Routes of either or both types can be returned in policy responses and will be appropriately identified by the `group` field on their metadata. Furthermore we update the Status of these resources to correctly reflect when they are accepted.
We add the `httproute.gateway.networking.k8s.io` CRD to the Linkerd installed CRD list and add the appropriate RBAC to the policy controller so that it may watch these resources.
Signed-off-by: Alex Leong <alex@buoyant.io>
PR #10969 adds support for the GEP-1742 `timeouts` field to the
HTTPRoute CRD. This branch implements actual support for these fields in
the policy controller. The timeout fields are now read and used to set
the timeout fields added to the proxy-api in
linkerd/linkerd2-proxy-api#243.
In addition, I've added code to ensure that the timeout fields are
parsed correctly when a JSON manifest is deserialized. The current
implementation represents timeouts in the bindings as a Rust
`std::time::Duration` type. `Duration` does implement
`serde::Deserialize` and `serde::Serialize`, but its serialization
implementation attempts to (de)serialize it as a struct consisting of a
number of seconds and a number of subsecond nanoseconds. The timeout
fields are instead supposed to be represented as strings in the Go
standard library's `time.ParseDuration` format. Therefore, I've added a
newtype which wraps the Rust `std::time::Duration` and implements the
same parsing logic as Go. Eventually, I'd like to upstream the
implementation of this to `kube-rs`; see kube-rs/kube#1222 for details.
Depends on #10969
Depends on linkerd/linkerd2-proxy-api#243
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
Add a new version to the HttpRoute CRD: `v1beta3`. This version adds a new `timeouts` struct to the http route rule. This mirrors a corresponding new field in the Gateway API, as described in [GEP-1742](https://github.com/kubernetes-sigs/gateway-api/pull/1997). This field is currently unused, but will eventually be read by the policy controller and used to configure timeouts enforced by the proxy.
The diff between v1beta2 and v1beta3 is:
```
timeouts:
description: "Timeouts defines the timeouts that can be configured
for an HTTP request. \n Support: Core \n <gateway:experimental>"
properties:
backendRequest:
description: "BackendRequest specifies a timeout for an
individual request from the gateway to a backend service.
Typically used in conjunction with automatic retries,
if supported by an implementation. Default is the value
of Request timeout. \n Support: Extended"
format: duration
type: string
request:
description: "Request specifies a timeout for responding
to client HTTP requests, disabled by default. \n For example,
the following rule will timeout if a client request is
taking longer than 10 seconds to complete: \n ``` rules:
- timeouts: request: 10s backendRefs: ... ``` \n Support:
Core"
format: duration
type: string
type: object
```
We update the `storage` version of HttpRoute to be v1beta3 but continue to serve all versions. Since this new field is optional, the Kubernetes API will be able to automatically convert between versions.
Signed-off-by: Alex Leong <alex@buoyant.io>
Fixes#10877
Linkerd reads the list of container readiness and liveness probes for a pod in order to generate authorizations which allow probes by default. However, when reading the `path` field of a probe, we interpret this field literally rather than parsing it as a URI. This means that any non-path parts of the URI (such as URI params) will attempt to match against the path of a probe request, causing these authorizations to fail.
Instead, we parse this field as a URI and only use the path part for path matching. Invalid URIs are skipped and a warning is logged.
Signed-off-by: Alex Leong <alex@buoyant.io>
When the `namespace` field of a `backend_ref` of an `HttpRoute` is set, Linkerd ignores this field and instead assumes that the backend is in the same namespace as the parent `Service`.
To properly handle the case where the backend is in a different namespace from the parent `Service`, we change the way that service metadata is stored in the policy controller outbound index. Instead of keeping a separate service metadata map per namespace, we maintain one global service metadata map which is shared between all namespaces using an RwLock. This allows us to make the two necessary changes:
1. When validating the existence of a backend service, we now look for it in the appropriate namespace instead of the Service's namespace
2. When constructing the backend authority, we use the appropriate namespace instead of the Service's namespace
We also add an API test for this situation.
Signed-off-by: Alex Leong <alex@buoyant.io>
Fixes#10782
Added the `minItems: 1` field to `spec.identities` and `spec.identitiRefs`. This is a BC change so it's not required to bump the CRD version, plus current CRs not abiding to this constraint would be broken anyway.
```bash
$ cat << EOF | k apply -f -
> apiVersion: policy.linkerd.io/v1alpha1
kind: MeshTLSAuthentication
metadata:
name: "test"
spec:
identities: []
> EOF
The MeshTLSAuthentication "test" is invalid: spec.identities: Invalid value: 0: spec.identities in body should have at least 1 items
```
Also refactored the MeshTLSAuthentication index reset loop to avoid stop processing items when one of them fails.
Controller currently serves hardcoded configuration values for failure
accrual parameters. This change adds support to discover configuration
on Service objects based on annotations.
Signed-off-by: Matei David <matei@buoyant.io>
Signed-off-by: Alex Leong <alex@buoyant.io>
Co-authored-by: Alex Leong <alex@buoyant.io>
We update the outbound policy API to ignore any HTTPRoute if it does not have an Accepted status of True. Additionally, we remove spurious logging from both the inbound and outbound index when they ignore an HTTPRoute which does not have parents which are relevant to that API. This means that, for example, an HTTPRoute that has a Service as a parent will no longer trigger log warnings from the inbound index and vice versa.
Signed-off-by: Alex Leong <alex@buoyant.io>
This change improves the modularity of the policy controller by splitting code related in inbound policy and outbound policy into separate modules. There are no functional changes in this PR, only moves and renames. We have moved any code which was used by both inbound and outbound into common modules so that there are no cross dependencies between inbound and outbound.
Additionally, several types that had names beginning with "Inbound" or "Outbound" have had this prefix removed since they are now distinguishable by their modules. e.g. `InboundHttpRoute` becomes `inbound::HttpRoute`.
Signed-off-by: Alex Leong <alex@buoyant.io>
When the outbound policy API was returning backends which corresponded to Service resources, those backends were missing metadata information.
We update the policy controller to populate Service metadata for those backends.
Signed-off-by: Alex Leong <alex@buoyant.io>
This removes the separate naming of “status controller” from the policy
controller resources, code, and comments. There is a single controller in all of
this — the policy controller. Part of the policy controller is maintaining the
status of policy resources. We can therefore remove this separate naming that
has been used as well as reorganize some of the code to use single naming
consts.
The lease resource has been renamed from `status-controller` to
`policy-controller-write`.
The controller name value has been renamed from
`policy.linkerd.io/status-controller` to `linkerd.io/policy-controller`. This
field appears in the `status` of HTTPRoutes indicating which controller applied
the status.
Lastly, I’ve updated the comments to remove the use of “status controller” and
moved the use of the name const to the `linkerd-policy-controller-core` package
so that it can be shared.
Signed-off-by: Kevin Leimkuhler <kleimkuhler@icloud.com>
When the policy controller is creating the inbound server which contains the
HTTPRoutes that a policy client should consider, we now consider the `status`
field and filter out HTTPRoutes which have not been accepted by the status
controller.
When creating the HTTPRoutes for an inbound server, the policy controller now
looks at the `status` field for each HTTPRoute. It filters out statuses which
are not from the policy status controller. For statuses which are from the
policy status controller, it ensures that it has been accepted by a Server on
the cluster.
The tests have been updated in the index; these were failing without an
`Accepted` status.
Signed-off-by: Kevin Leimkuhler <kleimkuhler@icloud.com>
We have a `ClusterInfo` type that includes all cluster-level
configuration. The outbound indexer does not use this type and, instead,
passes around individual configuration values.
This change updates the outbound indexer to use the ClusterInfo
configuration type. It also moves the type to its own module.
This is largely a reorganization PR, though there is one notable change:
InvalidDst response no longer reference a backend. Instead, the backend
is left empty, since it's not possible to route requests to it.
Implement the outbound policy API as defined in the proxy api: https://github.com/linkerd/linkerd2-proxy-api/blob/main/proto/outbound.proto
This API is consumed by the proxy for the routing of outbound traffic. It is intended to replace the GetProfile API which is currently served by the destination controller. It has not yet been released in a proxy-api release, so we take a git dependency on it in the mean time.
This PR adds a new index to the policy controller which indexes HTTPRoutes and Services and uses this information to serve the outbound API. We also add outbound API tests to validate the behavior of this implementation.
Signed-off-by: Alex Leong <alex@buoyant.io>
Co-authored-by: Oliver Gould <ver@buoyant.io>
The previous version k8s-gateway (`v0.10.0`) did not include backendRefs
for HTTP Routes, since the policy controller did not use them for any
specific task or validation. BackendRef support is currently being added
for the status controller, and will be used as more and more route
functionality is added to Linkerd.
This change bumps k8s-gateway to the most recent version and updates the
internal model of the route to include backendRefs. Additionally, fixes
any compiler issues that cropped up from adding a field to the struct.
Signed-off-by: Matei David <matei@buoyant.io>
This branch updates the dependency on `kubert` to 0.13.0.
- [Release notes](https://github.com/olix0r/kubert/releases)
- [Commits](https://github.com/olix0r/kubert/compare/release/v0.12.0...release/v0.13.0)
Since `kubert` and other Kubernetes API dependencies must be updated in
lockstep, this branch also updates `kube` to 0.78, `k8s-openapi` to
0.13, and `k8s-gateway-api` to 0.9.
`kube-runtime` now depends on a version of the `base64` crate which has
diverged significantly from the version `rustls-pemfile` depends on.
Since both `base64` deps are transitive dependencies which we have no
control over, this branch adds a `cargo deny` exception for duplicate
dependencies on `base64`.
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Eliza Weisman <eliza@buoyant.io>
Fixes#10255
When an HttpRoute resources contains a RegularExpression for a QueryParamMatch, we incorrectly convert that to an Exact match.
Convert this to a Regex, as intended.
Signed-off-by: Alex Leong <alex@buoyant.io>
Currently, the policy-controller index's `PodPortServer` struct stores
both the `Sender` and `Receiver` sides of a `tokio::sync::watch`
channel. This is unnecessary, since `Sender` has a `subscribe` method
that creates a new `Receiver`, and a `send_if_modified` method for
conditionally checking whether the value in the `watch` channel should
be updated.
This branch refactors the `PodPortServer` type to use these `Sender`
methods. This doesn't result in any functional change, but is maybe a
little bit nicer. Not a big deal though.
* Update kubert to v0.10
* Update kube-rs to v0.75 (fixes#9339)
* Update k8s-openapi to v0.16
* Update k8s-gateway-api to v0.7
Signed-off-by: Oliver Gould <ver@buoyant.io>
Closes#8945
This adds the `policyController.probeNetworks` configuration value so that users
can configure the networks from which probes are expected to be performed.
By default, we allow all networks (`0.0.0.0/0`). Additionally, this value
differs from `clusterNetworks` is that it is a list of networks, and thus we
have to join the values in the Helm templating.
Signed-off-by: Kevin Leimkuhler <kleimkuhler@icloud.com>
This changes updates the policy controller's indexer to add default, unauthenticated routes for
endpoints referenced in a Pod's readiness/liveness/startup probe configuration. These default routes
are included when:
1. the policy controller is configured with a list of networks from which probes may originate; and
2. no other routes are configured for the server.
If a user defines routes for a Server, then they must also explicitly account for probe endpoints.
An e2e test has been added which asserts the following:
1. When no Server is configured for a Pod:port, the probe routes are authorized.
2. When a Server is configured, but there are no routes, the probe routes are still authorized.
3. When a route is configured for the Server, the probe routes are no longer authorized by default.
Related to #8961#8945
Signed-off-by: Kevin Leimkuhler <kleimkuhler@icloud.com>
Co-authored-by: Oliver Gould <ver@buoyant.io>
Co-authored-by: Eliza Weisman <eliza@buoyant.io>
Currently, if no `HTTPRoute` resources reference a `Server`, the policy
controller returns an empty list of routes. This results in the proxy
404ing HTTP traffic. Instead, we should return a default empty route in
this case.
This branch changes the policy controller to return a default route for
`Server`s which are not referenced by any `HTTPRoute`s. The default
route defines a single `HttpRouteRule` with no matches and no filters,
so it will match any HTTP request. The default route does *not* define
any authorizations, as they would potentially clobber authorizations
defined by other resources that define authorization policies targeting
that `Server` --- if it is not targeted by any other resource defining
authorization policies, the `Server` itself will still get the default
authz policy.
In addition, this branch changes the various `${Resource}Ref` enums so
that their `Default` variants take a `&'static str` rather than a
`String`, to minimize string copying, since the default names are always
pre-defined. It also adds an `InboundHttpRouteRef` type which is used as
a key in the maps of `HTTPRoute`s (instead of `String`s), since we now
define default routes as well.
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
Co-authored-by: Oliver Gould <ver@buoyant.io>
This branch updates the policy controller's validating admission
controller to use the same validation functions as the indexer when
validating `HTTPRoute` resources. This way, we can ensure that any
`HTTPRoute` spec that passes validation will also convert to a valid
`InboundRouteBinding` in the indexer.
When there are multiple equivalent routes (e.g., two routes with the
same match), the proxy will use the first route in the returned list. We
need to ensure that the policy controller returns routes in a
deterministic order--and the Gateway API defines such an order:
> If ties still exist across multiple Routes, matching precedence MUST
> be determined in order of the following criteria, continuing on ties:
>
> * The oldest Route based on creation timestamp.
> * The Route appearing first in alphabetical order by
> "{namespace}/{name}".
This branch updates the policy controller to return the list of
`HttpRoute`s for an inbound server with a deterministic ordering based
on these rules. This is done by tracking the creation timestamp for
indexed `HTTPRoute` resources, and sorting the list of protobuf
`HttpRoute`s when the API server constructs an `InboundServer` response.
The implementation is *somewhat* hairy, because we can't just define a
custom `Ord` implementation for the protobuf `HttpRoute` type that
includes the timestamp --- doing so would require actually storing the
creation timestamp in the protobuf type, which would be a change in
`linkerd2-proxy-api` (and would result in serializing additional
information that the proxy itself doesn't actually care about). Instead,
we use `slice::sort_by` with a closure that looks up routes by name in
the hash map stored by the indexer in order to determine their
timestamps, and implements a custom ordering that first compares the
timestamp, and falls back to comparing the route's name if the
timestamps are equal. Note that we don't include the namespace in that
comparison, because all the routes for a given `InboundServer` are
already known to be in the same namespace.
I've also added an end-to-end test that the API returns the route list
in the correct order. Unfortunately, this test has 4 seconds of `sleep`s
in it, because the minimum resolution of Kubernetes creation timestamps
is 1 second. I figured a test that takes five or six seconds to run was
probably not a huge deal in the end to end tests --- some of the policy
tests take as long as a minute to run, at least on my machine.
Closes#8946
The proxy won't handle httproute paths (in URI rewrites or matches) when
paths are relative. The policy admission controller and indexer should
catch this case and fail to handle routes that deal in paths that do not
start in `/`.
This branch adds validation to the admission controller and indexer to
ensure that all paths in an `httproute` rule are absolute.
Part of #8945
In order for default inbound Servers to authorize probe routes, we must track
the probe ports and their expected paths when indexing Pods.
This introduces a new `_probes` field to Pod (which will be used in a follow-up
change) which maps probe ports to their expected paths.
For example, if a Pod’s container configures the following probes
```yaml
livenessProbe:
httpGet:
path: /live
port: 4191
...
readinessProbe:
httpGet:
path: /ready
port: 4191
...
```
Then we expect `_probes == {4191: {“/live”, “/ready”}}`
Signed-off-by: Kevin Leimkuhler <kleimkuhler@icloud.com>
As discussed in #8944, Linkerd's current use of the
`gateway.networking.k8s.io` `HTTPRoute` CRD is not a spec-compliant use
of the Gateway API, because we don't support some "core" features of the
Gateway API that don't make sense in Linkerd's use-case. Therefore,
we've chosen to replace the `gateway.networking.k8s.io` `HTTPRoute` CRD
with our own `HTTPRoute` CRD in the `policy.linkerd.io` API group, which
removes the unsupported features.
PR #8949 added the Linkerd versions of those CRDs, but did not remove
support for the Gateway API CRDs. This branch removes the Gateway API
CRDs from the policy controller and `linkerd install`/Helm charts.
The various helper functions for converting the Gateway API resource
binding types from `k8s-gateway-api` to the policy controller's internal
representation is kept in place, but the actual use of that code in the
indexer is disabled. This way, we can add support for the Gateway API
CRDs again easily. Similarly, I've kept the validation code for Gateway
API types in the policy admission controller, but the admission
controller no longer actually tries to validate those resources.
Depends on #8949Closes#8944
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
Our use of the `gateway.networking.k8s.io` types is not compliant with
the gateway API spec in at least a few ways:
1. We do not support the `Gateway` types. This is considered a "core"
feature of the `HTTPRoute` type.
2. We do not currently update `HTTPRoute` status fields as dictated by
the spec.
3. Our use of Linkerd-specific `parentRef` types may not work well with
the gateway project's admission controller (untested).
Issue #8944 proposes solving this by replacing our use of
`gateway.networking.k8s.io`'s `HTTPRoute` type with our own
`policy.linkerd.io` version of the same type. That issue suggests that
the new `policy.linkerd.io` types be added separately from the change
that removes support for the `gateway.networking.k8s.io` versions, so
that the migration can be done incrementally.
This branch does the following:
* Add new `HTTPRoute` CRDs. These are based on the
`gateway.networking.k8s.io` CRDs, with the following changes:
- The group is `policy.linkerd.io`,
- The API version is `v1alpha1`,
- `backendRefs` fields are removed, as Linkerd does not support them,
- filter types Linkerd does not support (`RequestMirror` and
`ExtensionRef`), are removed.
* Add Rust bindings for the new `policy.linkerd.io` versions of
`HTTPRoute` types in `linkerd-policy-controller-k8s-api`.
The Rust bindings define their own versions of the `HttpRoute`,
`HttpRouteRule`, and `HttpRouteFilter` types, because these types'
structures are changed from the Gateway API versions (due to the
removal of unsupported filter types and fields). For other types,
which are identical to the upstream Gateway API versions (such as the
various match types and filter types), we re-export the existing
bindings from the `k8s-gateway-api`crate to minimize duplication.
* Add conversions to `InboundRouteBinding` from the `policy.linkerd.io`
`HTTPRoute` types.
When possible, I tried to factor out the code that was shared between
the conversions for Linkerd's `HTTPRoute` types and the upstream
Gateway API versions.
* Implement `kubert`'s `IndexNamespacedResource` trait for
`linkerd_policy_controller_k8s_api::policy::HttpRoute`, so that the
policy controller can index both versions of the `HTTPRoute` CRD.
* Adds validation for `policy.linkerd.io` `HTTPRoute`s to the policy
controller's validating admission webhook.
* Updated the policy controller tests to test both versions of
`HTTPRoute`.
## Notes
A couple questions I had about this approach:
- Is re-using bindings from the `k8s-gateway-api` crate appropriate
here, when the type has not changed from the Gateway API version? If
not, I can change this PR to vendor those types as well, but it will
result in a lot more code duplication.
- Right now, the indexer stores all `HTTPRoute`s in the same index.
This means that applying a `policy.linkerd.io` version of `HTTPRoute`
and then applying the Gateway API version with the same ns/name will
update the same value in the index. Is this what we want? I wasn't
entirely sure...
See #8944.
This change updates the policy controller to admit `AuthorizationPolicy` resources
that reference `HTTPRoute` parents. These policies configure proxies to augment
server-level authorizations with per-route authorizations.
Fixes#8890
Signed-off-by: Alex Leong <alex@buoyant.io>
In various places we read port configurations from external sources
(either the Kubernetes API or gRPC clients). We have manual checks in
place to ensure that port values are never zero. We can instead assert
this with the type system by using `NonZeroU16`.
This change updates the policy controller to use `NonZeroU16` for port
values. This allows us to replace our manual port value checks with
`NonZero::try_from`, etc.
Signed-off-by: Oliver Gould <ver@buoyant.io>
linkerd2-proxy-api v0.6.0 adds support for inbound proxies to discover
route configurations based on the Gateway API HTTPRoute types. This
change updates the policy controller to index
`gateway.networking.k8s.io/v1beta` `HTTPRoute` types to discover these
policies from the Kubernetes API.
`HTTPRoute` resources may target `Server` resources (as a `parentRef`)
to attach policies to an inbound proxy. When no routes are configured,
a default route is synthesized to allow traffic; but when at least one
route attaches to a server, only requests that match a route are
permitted (other requests are failed with a 404).
Only the *core* subset of the `HTTPRoute` filters are supported:
`RequestRedirect` and `RequestHeaderModifier`. Backends may *not* be
configured on these routes (since they may only apply to inbound/server-
side proxies). No `status` updates are currently performed on these
`HTTPRoute` resources.
This change does not yet allow `AuthorizationPolicy` resources to target
`HTTPRoute` resources. This will be added in a follow-up change.
Signed-off-by: Alex Leong <alex@buoyant.io>
Co-authored-by: Oliver Gould <ver@buoyant.io>
In 1a0c1c31 we updated the admission controller to allow
`AuthorizationPolicy` resources with an empty
`requiredAuthenticationRefs`. But we did NOT update the indexer, so we
would allow these resources to be created but then fail to honor them in
the API.
To fix this:
1. The `AuthorizationPolicy` admission controller is updated to exercise
the indexer's validation so that it is impossible to admit resources
that will be discarded by the indexer;
2. An e2e test is added to exercise this configuration;
3. The indexer's validation is updated to accept resources with no
authentications.
Signed-off-by: Oliver Gould <ver@buoyant.io>
* Replace deprecated uses of `ResourceExt::name` with
`ResourceExt::name_unchecked`;
* Update k8s-gateway-api to v0.6;
* Update kubert to v0.9.
Signed-off-by: Oliver Gould <ver@buoyant.io>
Closes#8565.
With this change, AuthorizationPolicys can now reference ServiceAccounts for
their target authentications. This allows users to avoid the requirement of
creating a MeshTLSAuthentication resource that references a single
ServiceAccount.
The policy admission controller only allows an AuthorizationPolicy to reference
a single MeshTLSAuthentication _or_ a ServiceAccount; it cannot reference both.
Additionally, if a ServiceAccount is reference it can onl be a single
one—similar to MeshTLSAuthentications.
Signed-off-by: Kevin Leimkuhler <kleimkuhler@icloud.com>
There is no convenient way to create an `AuthorizationPolicy` that targets all
`Server` resources in a `Namespace`. A distinct policy resource must be created
for each `Server`. This is cumbersome.
This change updates the policy controller to allow `AuthorizationPolicy`
resources to target `Namespace` resources. Policies may only target the
namespace in which the policy resource exists. The admission controller rejects
resources created that targets other namespaces.
Fixes#8297
The `identity_refs` section of the `MeshTLSAuthentication` resource currently
only supports ServiceAccount resources. We add support for Namespace resources
as well. When a `MeshTLSAuthentication` has a Namespace identity_ref, this
means that all ServiceAccounts in that Namespace are authenticated.
Fixes#8298
Signed-off-by: Alex Leong <alex@buoyant.io>
Alex Leong