Currently, the policy admission controller requires that the
`AuthorizationPolicy` resources include a non-empty
`requiredAuthenticationRefs` field. This means that all authorization
policies require at least a `NetworkAuthentication` to permit traffic.
For example:
```yaml
---
apiVersion: policy.linkerd.io/v1alpha1
kind: AuthorizationPolicy
metadata:
name: ingress
spec:
targetRef:
group: policy.linkerd.io
kind: Server
name: ingress-http
requiredAuthenticationRefs:
- group: policy.linkerd.io
kind: NetworkAuthentication
name: all-nets
---
apiVersion: policy.linkerd.io/v1alpha1
kind: NetworkAuthentication
metadata:
name: ingress-all-nets
spec:
networks:
- cidr: 0.0.0.0/0
- cidr: ::/0
```
This is needlessly verbose and can more simply be expressed as:
```yaml
---
apiVersion: policy.linkerd.io/v1alpha1
kind: AuthorizationPolicy
metadata:
name: ingress
spec:
targetRef:
group: policy.linkerd.io
kind: Server
name: ingress-http
requiredAuthenticationRefs: []
```
That is: there are explicitly no required authentications for this
policy.
This change updates the admission controller to permit such a policy.
Note that the `requiredAuthenticationRefs` field is still required so
that it's harder for simple misconfigurations to result in allowing
traffic.
This change also removes `Default` implementation for resources where do
they not make sense because there are required fields.
Signed-off-by: Oliver Gould <ver@buoyant.io>
Issue #7709 proposes new Custom Resource types to support generalized
authorization policies:
- `AuthorizationPolicy`
- `MeshTLSAuthentication`
- `NetworkAuthentication`
This change introduces these CRDs to the default linkerd installation
(via the `linkerd-crds` chart) and updates the policy controller's
to handle these resource types. The policy admission controller
validates that these resource reference only suppported types.
This new functionality is tested at multiple levels:
* `linkerd-policy-controller-k8s-index` includes unit tests for the
indexer to test how events update the index;
* `linkerd-policy-test` includes integration tests that run in-cluster
to validate that the gRPC API updates as resources are manipulated;
* `linkerd-policy-test` includes integration tests that exercise the
admission controller's resource validation; and
* `linkerd-policy-test` includes integration tests that ensure that
proxies honor authorization resources.
This change does NOT update Linkerd's control plane and extensions to
use these new authorization primitives. Furthermore, the `linkerd` CLI
does not yet support inspecting these new resource types. These
enhancements will be made in followup changes.
Signed-off-by: Oliver Gould <ver@buoyant.io>
In preparation for new policy CRD resources, this change adds end-to-end
tests to validate policy enforcement for `ServerAuthorization`
resources.
In adding these tests, it became clear that the OpenAPI validation for
`ServerAuthorization` resources is too strict. Various `oneof`
constraints have been removed in favor of admission controller
validation. These changes are semantically compatible and do not
necessitate an API version change.
The end-to-end tests work by creating `curl` pods that call an `nginx`
pod. In order to test network policies, the `curl` pod may be created
before the nginx pod, in which case an init container blocks execution
until a `curl-lock` configmap is deleted from the cluster. If the
configmap is not present to begin with, no blocking occurs.
Signed-off-by: Oliver Gould <ver@buoyant.io>
The policy controller's indexing module spans several files and relies
on an unnecessarily complex double-watch. It's generally confusing and
therefore difficult to change.
This change attempts to simplify the logic somewhat:
* All of the indexing code is now in the
`linkerd_policy_controller_k8s_index::index` module. No other files
have any dependencies on the internals of this data structure. It
exposes one public API, `Index::pod_server_rx`, used by discovery
clients.
* It uses the new `kubert::index` API so that we can avoid redundant
event-handling code. We now let kubert drive event processing so that
our indexing code is solely responsible for updating per-port server
configurations.
* A single watch is maintained for each pod:port.
* Watches are constructed lazily. The policy controller no longer
requires that all ports be documented on a pod. (The proxy still
requires this, however). This sets up for more flexible port
discovery.
Signed-off-by: Oliver Gould <ver@buoyant.io>
`ServerAuthorization` resources are not validated by the admission
controller.
This change enables validation for `ServerAuthorization` resources,
based on changes to the admission controller proposed as a part of
linkerd/linkerd2#8007. This admission controller is generalized to
support arbitrary resource types. The `ServerAuthoriation` validation
currently only ensures that network blocks are valid CIDRs and that they
are coherent. We use the new _schemars_ feature of `ipnet` v2.4.0 to
support using IpNet data structures directly in the custom resource
type bindings.
This change also adds an integration test to validate that the admission
controller behaves as expected.
Signed-off-by: Oliver Gould <ver@buoyant.io>
In preparation for introducing new policy types, this change reorganizes
the policy controller to keep more of each indexing module private.
Signed-off-by: Oliver Gould <ver@buoyant.io>
The policy controller has an validating webhook for `Server` resources,
but this functionality is not really tested.
Before adding more policy resources that need validation, let's add an
integration test that exercises resource validation. The initial test is
pretty simplistic, but this is just setup.
These test also help expose two issues:
1. The change in 8760c5f--to solely use the index for validation--is
problematic, especially in CI where quick updates can pass validation
when they should not. This is fixed by going back to making API calls
when validating `Server` resources.
2. Our pod selector overlap detection is overly simplistic. This change
updates it to at least detect when a server selects _all_ pods.
There's probably more we can do here in followup changes.
Tests are added in a new `policy-test` crate that only includes these
tests and the utiltities they need. This crate is excluded when running
unit tests and is only executed when it has a Kubernetes cluster it can
execute against. A temporary namespace is created before each test is
run and deleted as the test completes.
The policy controller's CI workflow is updated to build the core control
plane, run a k3d cluster, and exercise tests. This workflow has minimal
dependencies on the existing script/CI tooling so that the dependencies
are explicit and we can avoid some of the complexity of the existing
test infrastructure.
Signed-off-by: Oliver Gould <ver@buoyant.io>
`kubert` provides a runtime utility that helps reduce boilerplate around
process lifecycle management, construction of admin and HTTPS servers,
etc.
The admission controller server preserves the certificate reloading
functionality introduced in 96131b5 and updates the utility to read both
RSA and PKSC8 keys to close#7963.
Signed-off-by: Oliver Gould <ver@buoyant.io>
Fixes#7904
Allow the `Server` CRD to have the `PodSelector` entry be an empty object, by removing the `omitempty` tag from its go type definition and the `oneof` section in the CRD. No update to the CRD version is required, as this is BC change -- The CRD overriding was tested fine.
Also added some unit tests to confirm podSelector conditions are ANDed, and some minor refactorings in the `Selector` constructors.
Co-authored-by: Oliver Gould <ver@buoyant.io>
Kubernetes v1.19 is reaching its end-of-life date on 2021-10-28. In
anticipation of this, we should explicitly update our minimum supported
version to v1.20. This allows us keep our dependencies up-to-date and
ensures that we can actually test against our minimum supported version.
Fixes#7171
Co-authored-by: Alejandro Pedraza <alejandro@buoyant.io>
Fixes#6827
We upgrade the Server and ServerAuthorization CRD versions from v1alpha1 to v1beta1. This version update does not change the schema at all and the v1alpha1 versions will continue to be served for now. We also update the CLI and control plane to use the v1beta1 versions.
Signed-off-by: Alex Leong <alex@buoyant.io>
We initially implemented a mechanism to automatically authorize
unauthenticated traffic from each pod's Kubelet's IP. Our initial method
of determining a pod's Kubelet IP--using the first IP from its node's
pod CIDRs--is not a generally usable solution. In particular, CNIs
complicate matters (and EKS doesn't even set the podCIDRs field).
This change removes the policy controller's node watch and removes the
`default:kubelet` authorization. When using a restrictive default
policy, users will have to define `serverauthorization` resources that
permit kubelet traffic. It's probably possible to programatically
generate these authorizations (i.e. by inspecting pod probe
configurations); but this is out of scope for the core control plane
functionality.
We add a validating admission controller to the policy controller which validates `Server` resources. When a `Server` admission request is received, we look at all existing `Server` resources in the cluster and ensure that no other `Server` has an identical selector and port.
Signed-off-by: Alex Leong <alex@buoyant.io>
Co-authored-by: Oliver Gould <ver@buoyant.io>
The policy controller's readiness and liveness admin endpoint is tied to
watch state: the controller only advertises liveness when all watches
have received updates; and after a watch disconnects liveness fails
until a new update is received.
However, in some environments--especially when the API server ends the
stream before the client gracefully reconnects--the watch terminess so
that liveness is not advertises even though the client resumes watching
resources. Because the watch is resumed with a `resourceVersion`, no
updates are provided despite the watch being reestablished, and liveness
checks fail until the pod is terminated (or an update is received).
To fix this, we modify readiness advertisements to fail only until the
initial state is acquired from all watches. After this, the controller
serves cached state indefinitely.
While diagnosing this, logging changes were needed, especially for the
`Watch` type. Watches now properly maintain logging contexts and state
transitions are logged in more cases. The signature and logging context
of `Index::run` has been updated as well. Additionally, node lookup
debug logs have been elaborated to help confirm that 'pending' messages
are benign.
We can't use the typical multiarch docker build with the proxy:
qemu-hosted arm64/arm builds take 45+ minutes before failing due to
missing tooling--specifically `protoc`. (While there is a `protoc`
binary available for arm64, there are no binaries available for 32-bit
arm hosts).
To fix this, this change updates the release process to cross-build the
policy-controller on an amd64 host to the target architecture. We
separate the policy-controller's dockerfiles as `amd64.dockerfile`,
`arm64.dockerfile`, and `arm.dockerfile`. Then, in CI we build and push
each of these images individually (in parallel, via a build matrix).
Once all of these are complete, we use the `docker manifest` CLI tools
to unify these images into a single multi-arch manifest.
This cross-building approach requires that we move from using
`native-tls` to `rustls`, as we cannot build against the platform-
appropriate native TLS libraries. The policy-controller is now feature-
flagged to use `rustls` by default, though it may be necessary to use
`native-tls` in local development, as `rustls` cannot validate TLS
connections that target IP addresses.
The policy-controller has also been updated to pull in `tracing-log` for
compatibility with crates that do not use `tracing` natively. This was
helpful while debugging connectivity issue with the Kubernetes cluster.
The `bin/docker-build-policy-controller` helper script now *only* builds
the amd64 variant of the policy controller. It fails when asked to build
multiarch images.
kube v0.59 depends on k8s-openapi v0.13, which includes breaking
changes.
This change updates these dependencies and modifies our code to account
for these changes.
Furthermore, we now use the k8s-openapi feature `v1_16` so that we use
an API version that is compatible with Linkerd's minimum support
kubernetes version.
Closes#6657#6658#6659
We've implemented a new controller--in Rust!--that implements discovery
APIs for inbound server policies. This change imports this code from
linkerd/polixy@25af9b5e.
This policy controller watches nodes, pods, and the recently-introduced
`policy.linkerd.io` CRD resources. It indexes these resources and serves
a gRPC API that will be used by proxies to configure the inbound proxy
for policy enforcement.
This change introduces a new policy-controller container image and adds a
container to the `Linkerd-destination` pod along with a `linkerd-policy` service
to be used by proxies.
This change adds a `policyController` object to the Helm `values.yaml` that
supports configuring the policy controller at runtime.
Proxies are not currently configured to use the policy controller at runtime. This
will change in an upcoming proxy release.
Oliver Gould