We add support for the RequestHeaderModifier and RequestRedirect HTTP filters. The policy controller reads these filters in any HttpRoute resource that it indexes (both policy.linkerd.io and gateway.networking.k8s.io) and returns them in the outbound policy API. These filters may be added at the route rule level and at the backend level.
We add outbound api tests for this behavior for both types of HttpRoute.
Incidentally we also fix a flaky test in the outbound api tests where a watch was being recreated partway through a test, leading to a race condition.
Signed-off-by: Alex Leong <alex@buoyant.io>
## edge-23.7.1
This edge release adds support for the upstream
`gateway.networking.k8s.io` HTTPRoute resource (in addition to the
`policy.linkerd.io` CRD installed by Linkerd). Furthermore, it fixes a
bug where the ingress-mode proxy would fail to fall back to
ServiceProfiles for destinations without HTTPRoutes.
* Added support for `gateway.networking.k8s.io` HTTPRoutes in the policy
controller
* Added distinguishable version information to proxy logs and metrics
* Fixed incorrect handling of `NotFound` client policies in ingress-mode
proxies
Updates the policy-controller to watch `httproute.gateway.networking.k8s.io` resources in addition to watching `httproute.policy.linkerd.io` resources. Routes of either or both types can be returned in policy responses and will be appropriately identified by the `group` field on their metadata. Furthermore we update the Status of these resources to correctly reflect when they are accepted.
We add the `httproute.gateway.networking.k8s.io` CRD to the Linkerd installed CRD list and add the appropriate RBAC to the policy controller so that it may watch these resources.
Signed-off-by: Alex Leong <alex@buoyant.io>
When using `linkerd-await` as a preStart hook, we need to explicitly pass in the proxy's admin port if it is not the default (4191). While the admin server listener can be bound to any arbitrary port using `config.linkerd.io/admin-port` as a configuration annotation, `linkerd-await`'s template is not aware of the override resulting in start-up errors.
This change adds the override to `linkerd-await` by always using an explicit `--port` argument.
---------
Signed-off-by: jclegras <11457480+jclegras@users.noreply.github.com>
* Updated release notes for edge-23.6.2
Signed-off-by: Eric Anderson <eric@buoyant.io>
* Updated helm charts and readmes
Signed-off-by: Eric Anderson <eric@buoyant.io>
* Updated release notes for edge release
Signed-off-by: Eric Anderson <eric@buoyant.io>
---------
Signed-off-by: Eric Anderson <eric@buoyant.io>
fix: supplement the HA flag
Linkerd checks are skipped for HA because the field is missing from the configmap generated during install time.
This change introduces an HA field in the helm charts that will be persisted, thereby allowing checks to run.
---------
Signed-off-by: Takumi Sue <u630868b@alumni.osaka-u.ac.jp>
This edge release changes the behavior of the CNI plugin to run exclusively in
"chained mode". Instead of creating its own configuration file, the CNI plugin
will now wait until a `conf` file exists before appending its configuration.
Additionally, this change includes a bug fix for topology aware service
routing.
* Changed CNI plugin installer to always run in 'chained' mode; the plugin will
now wait until another CNI plugin is installed before appending its
configuration
* Added a timeout value to the HttpRoute CRD. The field is currently unused but
will be used eventually to allow for configuration of per-route timeouts
* Fixed bug where topology routing would not disable while service was under
load (thanks @MarkSRobinson!)
---------
Signed-off-by: Matei David <matei@buoyant.io>
Co-authored-by: Eliza Weisman <eliza@buoyant.io>
This release stops using the "interface" mode, and instead wait till
another CNI plugin drops a proper network config and then append the
linkerd CNI config to it. This avoids having pods start before proper
networking is established in the node.
Add a new version to the HttpRoute CRD: `v1beta3`. This version adds a new `timeouts` struct to the http route rule. This mirrors a corresponding new field in the Gateway API, as described in [GEP-1742](https://github.com/kubernetes-sigs/gateway-api/pull/1997). This field is currently unused, but will eventually be read by the policy controller and used to configure timeouts enforced by the proxy.
The diff between v1beta2 and v1beta3 is:
```
timeouts:
description: "Timeouts defines the timeouts that can be configured
for an HTTP request. \n Support: Core \n <gateway:experimental>"
properties:
backendRequest:
description: "BackendRequest specifies a timeout for an
individual request from the gateway to a backend service.
Typically used in conjunction with automatic retries,
if supported by an implementation. Default is the value
of Request timeout. \n Support: Extended"
format: duration
type: string
request:
description: "Request specifies a timeout for responding
to client HTTP requests, disabled by default. \n For example,
the following rule will timeout if a client request is
taking longer than 10 seconds to complete: \n ``` rules:
- timeouts: request: 10s backendRefs: ... ``` \n Support:
Core"
format: duration
type: string
type: object
```
We update the `storage` version of HttpRoute to be v1beta3 but continue to serve all versions. Since this new field is optional, the Kubernetes API will be able to automatically convert between versions.
Signed-off-by: Alex Leong <alex@buoyant.io>
This edge release adds some minor improvements in the MeshTLSAuthentication CRD
and the extensions charts, and fixes an issue with `linkerd multicluster check`.
* Added tolerations and nodeSelector support in extensions `namespace-metadata`
Jobs (thanks @pssalman!)
* Patched the MeshTLSAuthentication CRD to force providing at least one
identity/identityRef
* Fixed the `linkerd multicluster check` command failing in the presence of lots
of mirrored services
Fixes#10782
Added the `minItems: 1` field to `spec.identities` and `spec.identitiRefs`. This is a BC change so it's not required to bump the CRD version, plus current CRs not abiding to this constraint would be broken anyway.
```bash
$ cat << EOF | k apply -f -
> apiVersion: policy.linkerd.io/v1alpha1
kind: MeshTLSAuthentication
metadata:
name: "test"
spec:
identities: []
> EOF
The MeshTLSAuthentication "test" is invalid: spec.identities: Invalid value: 0: spec.identities in body should have at least 1 items
```
Also refactored the MeshTLSAuthentication index reset loop to avoid stop processing items when one of them fails.
The proxy caches discovery results in-memory. Linkerd supports
overriding the default eviction timeout for cached discovery results
through install (i.e. helm) values. However, it is currently not
possible to configure timeouts on a workload-per-workload basis, or to
configure the values after Linkerd has been installed (or upgraded).
This change adds support for annotation based configuration. Workloads
and namespaces now support two new configuration annotations that will
override the install values when specified.
Additionally, a typo has been fixed on the internal type representation.
This is not a breaking change since the type itself is not exposed to
users and is parsed correctly in the values.yaml file (or CLI)
Signed-off-by: Matei David <matei@buoyant.io>
Co-authored-by: Eliza Weisman <eliza@buoyant.io>
The proxy caches results in-memory, both for inbound and outbound
service (and policy) discovery. While the proxy's default values are
great in most cases, certain client configurations may require
overrides. The proxy supports overriding the default values, however, it
currently does not offer an easy way for users to configure them.
This PR introduces two new values in Linkerd's control plane chart. The
values control the inbound and outbound cache discovery idle timeout --
the amount of time a result will be kept in the cache if unused. Setting
this value will change the configuration for all injected proxies, but
not for the control plane.
---------
Signed-off-by: Matei David <matei@buoyant.io>
This release of the CNI plugin changes the base runtime Docker image
from `debian:bullseye-slim` to `alpine:3.17.3`.
---
* cni: use `scratch` as the base runtime docker image (linkerd/linkerd2-proxy-init/pull/237)
* cni: change base runtime image from `scratch` to `alpine` (linkerd/linkerd2-proxy-init#238)
Fixes#10762
The Linkerd control plane chart contains a Lease resource which is used by the Policy controller to do leader election. ArgoCD considers Leases to be runtime resources and will not deploy them. This means that Linkerd will not work for users of ArgoCD.
We remove the policy-controller-write Lease resource from the Helm chart and instead have the policy controller create this resource at startup. We create it with an `Apply` patch with `resourceVersion="0"`. This ensures that the Lease resource will only be created if it does not already exist and that if there are multiple replicas of the policy controller starting up at once, only one of them will create the Lease resource.
We also set the `linkerd-destination` Deployment as the owner reference of the Lease resource. This means that when the `linkerd-destination` Deployment is deleted (for example, when Linkerd is uninstalled) then the Lease will be garbage collected by Kubernetes.
Signed-off-by: Alex Leong <alex@buoyant.io>
* edge-23.4.2
This edge release contains a number of bug fixes.
* CLI
* Fixed `linkerd uninstall` issue for HttpRoute
* The `linkerd diagnostics policy` command now displays outbound policy when
the target resource is a Service
* CNI
* Fixed incompatibility issue with AWS CNI addon in EKS, that was
forbidding pods to acquire networking after scaling up nodes.
(thanks @frimik!)
* Added --set flag to install-cni plugin (thanks @amit-62!)
* Control Plane
* Fixed an issue where the policy controller always used the default
`cluster.local` domain
* Send Opaque protocol hint for opaque ports in destination controller
* Helm
* Fixed an issue in the viz Helm chart where the namespace metadata template
would throw `unexpected argument found` errors
* Fixed Jaeger chart installation failure
* Multicluster
* Remove namespace field from cluster scoped resources to fix pruning
* Proxy
* Updated `h2` dependency to include a patch for a theoretical
denial-of-service vulnerability discovered in CVE-2023-26964
* Handle Opaque protocol hints on endpoints
* Changed the proxy's default log level to silence warnings from
`trust_dns_proto` that are generally spurious.
* Added `outbound_http_balancer_endpoints` metric
* Fixed missing route_ metrics for requests with ServiceProfiles
* Viz
* Bump prometheus image to v2.43.0
* Add the `kubelet` NetworkAuthentication back since it is used by the
`linkerd viz allow-scrapes` subcommand.
---------
Signed-off-by: David McLaughlin <david@dmclaughlin.com>
Co-authored-by: Alejandro Pedraza <alejandro@buoyant.io>
Fixed incompatibility issue with AWS CNI addon in EKS, that was
forbidding pods to acquire networking after scaling up nodes.
Credits to @frimik for providing a diagnosis and fix, and to @JonKusz for the detailed repro
This PR added support for --set flag to linkerd cni-plugin installation command.
Also made changes to test file for cni-plugin install.
Fixed a bug at pkg/chart/charts.go for resources template.
fixes#9917
* Allow supporting all flags and values
This leverages `chartutil.CoalesceValues` in order to merge the values provided through regular flags, and the ones provided via `--set` flags. As that function consumes maps, I introduced the `ToMap` method function on the cni `Values` struct (a copy of the same function from the core linkerd `Values` struct) to convert the struct backing the regular flags into a map.
And for the `RenderCNI` method to be able to deal with value maps instead of yaml, the `charts.Chart` struct now distinguishes between `Values` (a map) and `RawValues` (YAML).
This allowed removing the `parseYAMLValue` function and avoid having to deal with individual entries in `buildValues()`, and we no longer need the `valuesOverrides` field in the `cniPluginOptions` struct.
## Tests
```bash
# Testing regular flag
$ bin/go-run cli install-cni --use-wait-flag | grep use.wait.flag
"use-wait-flag": true
# Testing using --set
$ bin/go-run cli install-cni --set useWaitFlag=true | grep use.wait.flag
"use-wait-flag": true
# Testing using --set on a setting that has no regular flag
$ bin/go-run cli install-cni --set enablePSP=true | grep PodSecurityPolicy
kind: PodSecurityPolicy
```
---------
Signed-off-by: amit-62 <kramit6662@gmail.com>
Co-authored-by: Alejandro Pedraza <alejandro.pedraza@gmail.com>
Co-authored-by: Matei David <matei.david.35@gmail.com>
* add `trust_dns=error` to default proxy log level
Since upstream has yet to release a version with PR
bluejekyll/trust-dns#1881, this commit changes the proxy's default log
level to silence warnings from `trust_dns_proto` that are generally
spurious.
Closes#10123.
The Helm docs action in CI (which changes for discrepancies in Helm chart readmes) only checks the core Linkerd Helm charts, while allowing discrepancies in extension chart readmes.
Update the action to enforce Helm doc consistency in extension charts as well.
Signed-off-by: Alex Leong <alex@buoyant.io>
Now we use label to filter all resources to uninstall, but `httproutes.policy.linkerd.io` does not have label, so every uninstall would not remove it.
Signed-off-by: Loong <loong.dai@intel.com>
Fixes#10737
The cluster domain is not passed to the policy controller as a flag. This means that the policy controller always uses the default value of `cluster.local`. If the cluster's domain is different from this default, the policy controller will return incorrect authorities in it's outbound policy API.
Pass the `--cluster-domain` flag.
Signed-off-by: Alex Leong <alex@buoyant.io>
## stable-2.13.0
This release introduces client-side policy to Linkerd, including dynamic routing
and circuit breaking. [Gateway API](https://gateway-api.sigs.k8s.io/) HTTPRoutes
can now be used to configure policy for outbound (client) proxies as well as
inbound (server) proxies, by creating HTTPRoutes with Service resources as their
`parentRef`. See the Linkerd documentation for tutorials on [dynamic request
routing] and [circuit breaking]. New functionality for debugging HTTPRoute-based
policy is also included in this release, including [new proxy metrics] and the
ability to display outbound policies in the `linkerd diagnostics policy` CLI
command.
In addition, this release adds `network-validator`, a new init container to be
used when CNI is enabled. `network-validator` ensures that local iptables rules
are working as expected. It will validate this before linkerd-proxy starts.
`network-validator` replaces the `noop` container, runs as `nobody`, and drops
all capabilities before starting.
Finally, this release includes a number of bugfixes, performance improvements,
and other smaller additions.
**Upgrade notes**: Please see the [upgrade instructions][upgrade-2130].
* CRDs
* HTTPRoutes may now have Service parents, to configure outbound policy
* Updated HTTPRoute version from `v1alpha1` to `v1beta2`
* CLI
* Added a new `linkerd prune` command to the CLI (including most extensions) to
remove resources which are no longer part of Linkerd's manifests
* Added additional shortnames for Linkerd policy resources (thanks @javaducky!)
* The `linkerd diagnostics policy` command now displays outbound policy when
the target resource is a Service
* Control Plane
* The policy controller now discovers outbound policy configurations from
HTTPRoutes that target Services.
* Added OutboundPolicies API, for use by `linkerd-proxy` to route
outbound traffic
* Added Prometheus `/metrics` endpoint to the admin server, with process
metrics
* Fixed QueryParamMatch parsing for HTTPRoutes
* Added the policy status controller which writes the `status` field to
HTTPRoutes when a parent reference Server accepts or rejects it
* Added KubeAPI server ports to `ignoreOutboundPorts` of `proxy-injector`
* No longer apply `waitBeforeExitSeconds` to control plane, viz and jaeger
extension pods
* Added support for the `internalTrafficPolicy` of a service (thanks @yc185050!)
* Added block chomping to strip trailing new lines in ConfigMap (thanks @avdicl!)
* Added protection against nil dereference in resources helm template
* Added support for Pod Security Admission (Pod Security Policy resources are
still supported but disabled by default)
* Lowered non-actionable error messages in the Destination log to debug-level
entries to avoid triggering false alarms (thanks @siddharthshubhampal!)
* Fixed an issue with EndpointSlice endpoint reconciliation on slice deletion;
when using more than one slice, a `NoEndpoints` event would be sent to the
proxy regardless of the amount of endpoints that were still available
(thanks @utay!)
* Improved diagnostic log messages
* Fixed sending of spurious profile updates
* Removed unnecessary Namespaces access from the destination controller RBAC
* Added the server_port_subscribers metric to track the number of subscribers
to Server changes associated with a pod's port
* Added the service_subscribers metric to track the number of subscribers to
Service changes
* Fixed a small memory leak in the opaque ports watcher
* Proxy
* Use the new OutboundPolicies API, supporting Gateway API-style routes
in the outbound proxy
* Added support for dynamic request routing based on HTTPRoutes
* Added HTTP circuit breaking
* Added `outbound_route_backend_http_requests_total`,
`outbound_route_backend_grpc_requests_total`, and
`outbound_http_balancer_endpoints` metrics
* Changed the proxy's behavior when traffic splitting so that only services
that are not in failfast are used. This will enable the proxy to manage
failover without external coordination
* Updated tokio (async runtime) in the proxy which should reduce CPU usage,
especially for proxy's pod local (i.e in the same network namespace)
communication
* linkerd-proxy-init
* Changed `proxy-init` iptables rules to be idempotent upon init pod
restart (thanks @jim-minter!)
* Improved logging in `proxy-init` and `linkerd-cni`
* Added a `proxyInit.privileged` setting to control whether the `proxy-init`
initContainer runs as a privileged process
* CNI
* Added static and dynamic port overrides for CNI eBPF to work with socket-level
load balancing
* Added `network-validator` init container to ensure that iptables rules are
working as expected
* Added a `resources` field in the linkerd-cni chart (thanks @jcogilvie!)
* Viz
* Added `tap.ignoredHeaders` Helm value to the linkerd-viz chart. This value
allows users to specify a comma-separated list of header names which will be
ignored by Linkerd Tap (thanks @ryanhristovski!)
* Removed duplicate SecurityContext in Prometheus manifest
* Added new flag `--viz-namespace` which avoids requiring permissions for
listing all namespaces in `linkerd viz` subcommands (thanks @danibaeyens!)
* Removed the TrafficSplit page from the Linkerd viz dashboard (thanks
@h-dav!)
* Introduced new values in the `viz` chart to allow for arbitrary annotations
on the `Service` objects (thanks @sgrzemski!)
* Added an optional AuthorizationPolicy to authorize Grafana to Prometheus
in the Viz extension
* Multicluster
* Removed duplicate AuthorizationPolicy for probes from the multicluster
gateway Helm chart
* Updated wording for linkerd-multicluster cluster when it fails to probe a
remote gateway mirror
* Added multicluster gateway `nodeSelector` and `tolerations` helm parameters
* Added new configuration options for the multicluster gateway:
* `gateway.deploymentAnnotations`
* `gateway.terminationGracePeriodSeconds` (thanks @bunnybilou!)
* `gateway.loadBalancerSourceRanges` (thanks @Tyrion85!)
* Extensions
* Removed dependency on the `curlimages/curl` 3rd-party image used to initialize
extensions namespaces metadata (so they are visible by `linkerd check`),
replaced by the new `extension-init` image
* Converted `ServerAuthorization` resources to `AuthorizationPolicy` resources
in Linkerd extensions
* Removed policy resources bound to admin servers in extensions (previously
these resources were used to authorize probes but now are authorized by
default)
* Fixed the link to the Jaeger dashboard the in viz dashboard (thanks
@eugenegoncharuk!)
* Updated linkerd-jaeger's collector to expose port 4318 in order support HTTP
alongside gRPC (thanks @uralsemih!)
* Among other dependency updates, the no-longer maintained ghodss/yaml library
was replaced with sigs.k8s.io/yaml (thanks @Juneezee!)
This release includes changes from a massive list of contributors! A special
thank-you to everyone who helped make this release possible:
* Andrew Pinkham [@jambonrose](https://github.com/jambonrose)
* Arnaud Beun [@bunnybilou](https://github.com/bunnybilou)
* Carlos Tadeu Panato Junior [@cpanato](https://github.com/cpanato)
* Christian Segundo [@someone-stole-my-name](https://github.com/someone-stole-my-name)
* Dani Baeyens [@danibaeyens](https://github.com/danibaeyens)
* Duc Tran [@ductnn](https://github.com/ductnn)
* Eng Zer Jun [@Juneezee](https://github.com/Juneezee)
* Ivan Ivic [@Tyrion85](https://github.com/Tyrion85)
* Joe Bowbeer [@joebowbeer](https://github.com/joebowbeer)
* Jonathan Ogilvie [@jcogilvie](https://github.com/jcogilvie)
* Jun [@junnplus](https://github.com/junnplus)
* Loong Dai [@daixiang0](https://github.com/daixiang0)
* María Teresa Rojas [@mtrojas](https://github.com/mtrojas)
* Mo Sattler [@MoSattler](https://github.com/MoSattler)
* Oleg Vorobev [@olegy2008](https://github.com/olegy2008)
* Paul Balogh [@javaducky](https://github.com/javaducky)
* Peter Smit [@psmit](https://github.com/psmit)
* Ryan Hristovski [@ryanhristovski](https://github.com/ryanhristovski)
* Semih Ural [@uralsemih](https://github.com/uralsemih)
* Shubhodeep Mukherjee [@shubhodeep9](https://github.com/shubhodeep9)
* Siddharth S Pal [@siddharthshubhampal](https://github.com/siddharthshubhampal)
* Subhash Choudhary [@subhashchy](https://github.com/subhashchy)
* Szymon Grzemski [@sgrzemski](https://github.com/sgrzemski)
* Takumi Sue [@mikutas](https://github.com/mikutas)
* Yannick Utard [@utay](https://github.com/utay)
* Yu Cao [@yc185050](https://github.com/yc185050)
* anoxape [@anoxape](https://github.com/anoxape)
* bastienbosser [@bastienbosser](https://github.com/bastienbosser)
* bitfactory-sem-denbroeder [@bitfactory-sem-denbroeder](https://github.com/bitfactory-sem-denbroeder)
* cui fliter [@cuishuang](https://github.com/cuishuang)
* eugenegoncharuk [@eugenegoncharuk](https://github.com/eugenegoncharuk)
* h-dav @[h-dav](https://github.com/h-dav)
* martinkubrak [@martinkubra](https://github.com/martinkubra)
* verbotenj [@verbotenj](https://github.com/verbotenj)
* ziollek [@ziollek](https://github.com/ziollek)
[dynamic request routing]: https://linkerd.io/2.13/tasks/configuring-dynamic-request-routing
[circuit breaking]: https://linkerd.io/2.13/tasks/circuit-breaking
[new proxy metrics]: https://linkerd.io/2.13/reference/proxy-metrics/#outbound-xroute-metrics
[upgrade-2130]: https://linkerd.io/2/tasks/upgrade/#upgrade-notice-stable-2130
Fixes#10694
We update the `linkerd diagnostics policy` command to support outbound policy in addition to inbound policy.
* `linkerd diagnostics policy` now requires a typed resource instead of just a pod name (i.e. `po/foo` instead of `foo`)
* if a pod is specified, inbound policy for that pod is displayed
* if a service is specified, outbound policy for that service is displayed
* no other resource types are supported
* since the output in json format is extremely verbose, we add support for a `--output / -o` flag which can be `json` or `yaml` and make `yaml` the default
Signed-off-by: Alex Leong <alex@buoyant.io>
proxy-init v2.2.1:
* Sanitize `subnets-to-ignore` flag
* Dep bumps
cni-plugin v1.1.0:
* Add support for the `config.linkerd.io/skip-subnets` annotation
* Dep bumps
validator v0.1.2:
* Dep bumps
Also, `linkerd-network-validator` is now released wrapped in a tar file, so this PR also amends `Dockerfile-proxy` to account for that.
We enable kubert's metrics feature which allows us to create a prometheus metrics endpoint on the policy controller's admin server. By default, only process metrics are surfaced.
Signed-off-by: Alex Leong <alex@buoyant.io>
Co-authored-by: Oliver Gould <ver@buoyant.io>
The existing `linkerd check` command runs extension checks based on extension namespaces already on-cluster. This approach does not permit running extension checks without cluster-side components.
Introduce "CLI Checks". These extensions run as part of `linkerd check`, if they satisfy the following criteria:
1) executable in PATH
2) prefixed by `linkerd-`
3) supports an `_extension-metadata` subcommand, that outputs self-identifying
JSON, for example:
```
$ linkerd-foo _extension-metadata
{
"name": "linkerd-foo",
"checks": "always"
}
```
4) The `name` value from `_extension-metadata` must match the filename. And `checks` must equal `always`.
If a CLI Check is found that also would have run as an on-cluster extension check, it is run as a CLI Check only.
Fixes#10544
The destination and identity controllers skip ports that are used by the
kubernetes API server to avoid startup ordering issues and to ensure
that the controllers are able to communicate with the kubernetes API
without requiring the control plane for discovery.
To avoid startup issues in the proxy-injector, the same configuration
should be applied.
This removes the separate naming of “status controller” from the policy
controller resources, code, and comments. There is a single controller in all of
this — the policy controller. Part of the policy controller is maintaining the
status of policy resources. We can therefore remove this separate naming that
has been used as well as reorganize some of the code to use single naming
consts.
The lease resource has been renamed from `status-controller` to
`policy-controller-write`.
The controller name value has been renamed from
`policy.linkerd.io/status-controller` to `linkerd.io/policy-controller`. This
field appears in the `status` of HTTPRoutes indicating which controller applied
the status.
Lastly, I’ve updated the comments to remove the use of “status controller” and
moved the use of the name const to the `linkerd-policy-controller-core` package
so that it can be shared.
Signed-off-by: Kevin Leimkuhler <kleimkuhler@icloud.com>
Implement the outbound policy API as defined in the proxy api: https://github.com/linkerd/linkerd2-proxy-api/blob/main/proto/outbound.proto
This API is consumed by the proxy for the routing of outbound traffic. It is intended to replace the GetProfile API which is currently served by the destination controller. It has not yet been released in a proxy-api release, so we take a git dependency on it in the mean time.
This PR adds a new index to the policy controller which indexes HTTPRoutes and Services and uses this information to serve the outbound API. We also add outbound API tests to validate the behavior of this implementation.
Signed-off-by: Alex Leong <alex@buoyant.io>
Co-authored-by: Oliver Gould <ver@buoyant.io>
This adds lease claims to the policy status controller so that upon startup, a
status controller attempts to claim the `status-controller` lease in the
`linkerd` namespace. With this lease, we can enforce leader election and ensure
that only one status controller on a cluster is attempting to patch HTTPRoute’s
`status` field.
Upon startup, the status controller now attempts to create the
`status-controller` lease — it will handle failure if the lease is already
present on the cluster. It then spawns a task for attempting to claim this lease
and sends all claim updates to the index `Index`.
Currently, `Index.claims` is not used, but in follow-up changes we can check
against the current claim for determining if the status controller is the
current leader on the cluster. If it is, we can make decisions about sending
updates or not to the controller `Controller`.
### Testing
Currently I’ve only manually tested this, but integration tests will definitely
be helpful follow-ups. For manually testing, I’ve asserted that the
`status-controller` is claimed when one or more status controllers startup and
are running on a cluster. I’ve also asserted that when the current leader is
deleted, another status controller claims the lease. Below is the summary of how
I tested it
```shell
$ linkerd install --ha |kubectl apply -f -
…
$ kubectl get -n linkerd leases status-controller
NAME HOLDER AGE
status-controller linkerd-destination-747b456876-dcwlb 15h
$ kubectl delete -n linkerd pod linkerd-destination-747b456876-dcwlb
pod "linkerd-destination-747b456876-dcwlb" deleted
$ kubectl get -n linkerd leases status-controller
NAME HOLDER AGE
status-controller linkerd-destination-747b456876-5zpwd 15h
```
Signed-off-by: Kevin Leimkuhler <kleimkuhler@icloud.com>
Fixes: #10262
When a resource is removed from the Linkerd manifests from one version to the next, we would like that resource to be removed from the user's cluster as part of the upgrade process. Our current recommendation is to use the `linkerd upgrade` command in conjunction with the `kubectl apply` command and the `--prune` flag to remove resources which are no longer part of the manifest. However, `--prune` has many shortcomings and does not detect resources kinds which are not part of the input manifest, nor does it detect cluster scoped resources. See https://linkerd.io/2.12/tasks/upgrade/#with-the-linkerd-cli
We add a `linkerd prune` command which locates all Linkerd resources on the cluster which are not part of the Linkerd manifest and prints their metadata so that users can delete them. The recommended upgrade procedure would then be:
```
> linkerd upgrade | kubectl apply -f -
> linkerd prune | kubectl delete -f -
```
User must take special care to use the desired version of the CLI to run the prune command since running this command will print all resources on the cluster which are not included in that version.
We also add similar prune commands to each of the `viz`, `multicluster`, and `jaeger` extensions for deleting extension resources which are not in the extension manifest.
Signed-off-by: Alex Leong <alex@buoyant.io>
### Overview
This adds a policy status controller which is responsible for patching Linkerd’s
HTTPRoute resource with a `status` field. The `status` field has a list of
parent statuses — one status for each of its parent references. Each status
indicates whether or not this parent has “accepted” the HTTPRoute.
The status controller runs on its own task in the policy controller and watches
for updates to the resources that it cares about, similar to the policy
controller’s index. One of the main differences is that while the policy
controller’s index watches many resources, the status controller currently only
cares about HTTPRoutes and Servers; HTTPRoutes can still only have parent
references that are Servers so we don’t currently need to consider any other
parent reference resources.
The status controller maintains its own index of resources so that it is
completely separated from the policy controller’s index. This allows the index
to be simpler in both its structure, how it handles `apply` and `delete`, and
what information it needs to store.
### Follow-ups
There are several important follow-ups to this change. #10124 contains changes
for the policy controller index filtering out HTTPRoutes that are not accepted
by a Server. We don’t want those changes yet. Leaving those out, the status
controller does not actually have any affect on Linkerd policy in the cluster.
We can probably add additional logging several places in the status controller;
that may even take place as part of the reviews on this. Additionally, we could
try queue size for updates to be processed.
Currently if the status controller fails in any of its potential places, we do
not re-queue updates. We probably should do that so that it is more robust
against failure.
In an HA installation, there could be multiple status controllers trying to
patch the same resource. We should explore the k8s lease API so that only one
status controller can patch a resource at a time.
### Implementation
The status controller `Controller` has a k8s client for patching resources,
`index` for tracking resources, and an `updates` channel which handles
asynchronous updates to resources.
#### Index
`Index` synchronously observes changes to resources. It determines which Servers
accept each HTTPRoute and generates a status patch for that HTTPRoute. Again,
the status contains a list of parent statuses, one for each of the HTTPRoutes
parent references.
When a Server is added or deleted, the status controller needs to recalculate
the status for all HTTPRoutes. This is because an HTTPRoute can reference
Servers in other namespaces, so if a Server is added or deleted anywhere in the
cluster it could affect any of the HTTPRoutes on the cluster.
When an HTTPRoute is added, we need to determine the status only for that
HTTPRoute. When it’s deleted we just need to make sure it’s removed from the
index.
The patches that the `Index` creates are sent to the `Controller` which is
responsible only for applying those patches to HTTPRoutes.
#### Controller
`Controller` asynchronously processes updates and applies patches to HTTPRoutes.
Signed-off-by: Kevin Leimkuhler <kleimkuhler@icloud.com>
(This came out during the k8s api calls review for #9650)
In 5dc662ae9 inheritance of opaque ports annotation from namespaces to
pods was removed, but we didn't remove the associated RBAC.
wind the new linkerd-cni build through the build. refactor image, version, and pullPolicy into an Image object.
Signed-off-by: Steve Jenson <stevej@buoyant.io>
> When using ArgoCD and Azure Key Vault Plugin to manage Linkerd via Helm, the
> identityTrustAnchorsPEM value gets passed from Azure Key Vault with a trailing
> new line. This trailing new line makes its way into the config map
> linkerd-identity-trust-roots causing Linkerd control plane to crash upon
> deployment. There aren't any other alternatives when using Azure Key Vault due
> to how multi-line secrets are created. Azure forces this trailing new line.
>
> The solution is to add a block chomping indicator to strip trailing new lines in
> the config map.
>
> More on block chomping indicators: https://yaml-multiline.info/
>
> Fixes: #10012
The original PR #10059 has staled out, but it's worth getting this change in.
Signed-off-by: Alexander Di Clemente <diclemea@gmail.com>
Co-authored-by: Kevin Leimkuhler <kleimkuhler@icloud.com>
Fixes#10150
When we added PodSecurityAdmission in #9719 (and included in
edge-23.1.1), we added the entry `seccompProfile.type=RuntimeDefault` to
the containers SecurityContext.
For PSP to accept that we require to add the annotation
`seccomp.security.alpha.kubernetes.io/allowedProfileNames:
"runtime/default"` into the PSP resource, which also implies we require
to add the entry `seccompProfile.type=RuntimeDefault` to the pod's
SecurityContext as well, not just the container's.
It also turns out the `namespace-metadata` Jobs used by extensions for
the helm installation method didn't have their ServiceAccount properly
bound to the PSP resource. This resulted in the `helm install` command
failing, and although the extensions resources did get deployed, they
were not being discoverable by `linkerd check`. This change fixes that
as well, that has been broken since 2.12.0!
Fixes#9965
Adds a `path` property to the RedirectRequestFilter in all versions. This property was absent from the CRD even though it appears in the gateway API documentation and is represented in the internal types. Adding this property to the CRD will also users to specify it.
Add a new version to the HTTPRoute CRD: v1beta2. This new version includes two changes from v1beta1:
* Added `port` property to `parentRef` for use when the parentRef is a Service
* Added `backendRefs` property to HTTPRoute rules
We switch the storage version of the HTTPRoute CRD from v1alpha1 to v1beta2 so that these new fields may be persisted.
We also update the policy admission controller to allow an HTTPRoute parentRef type to be Service (in addition to Server).
Signed-off-by: Alex Leong <alex@buoyant.io>
This change expands on existing shortnames while adding others for various policy resources. This improves the user experience when issuing commands via kubectl.
Fixes#9322
Signed-off-by: Paul Balogh <javaducky@gmail.com>
* Removed dupe imports
My IDE (vim-gopls) has been complaining for a while, so I decided to take
care of it. Found via
[staticcheck](https://github.com/dominikh/go-tools)
* Add stylecheck to go-lint checks
Helm chart has `identity.externalCA` value.
CLI code sets `identity.issuer.externalCA` and fails to produce the desired configuration. This change aligns everything to `identity.externalCA`.
Signed-off-by: Dmitry Mikhaylov <anoxape@gmail.com>
Closes#9676
This adds the `pod-security.kubernetes.io/enforce` label as described in [Pod Security Admission labels for namespaces](https://kubernetes.io/docs/concepts/security/pod-security-admission/#pod-security-admission-labels-for-namespaces).
PSA gives us three different possible values (policies or modes): [privileged, baseline and restricted](https://kubernetes.io/docs/concepts/security/pod-security-standards/).
For non-CNI mode, the proxy-init container relies on granting the NET_RAW and NET_ADMIN capabilities, which places those pods under the `restricted` policy. OTOH for CNI mode we can enforce the `restricted` policy, by setting some defaults on the containers' `securityContext` as done in this PR.
Also note this change also adds the `cniEnabled` entry in the `values.yaml` file for all the extension charts, which determines what policy to use.
Final note: this includes the fix from #9717, otherwise an empty gateway UID prevents the pod to be created under the `restricted` policy.
## How to test
As this is only enforced as of k8s 1.25, here are the instructions to run 1.25 with k3d using Calico as CNI:
```bash
# launch k3d with k8s v1.25, with no flannel CI
$ k3d cluster create --image='+v1.25' --k3s-arg '--disable=local-storage,metrics-server@server:0' --no-lb --k3s-arg --write-kubeconfig-mode=644 --k3s-arg --flannel-backend=none --k3s-arg --cluster-cidr=192.168.0.0/16 --k3s-arg '--disable=servicelb,traefik@server:0'
# install Calico
$ k apply -f https://k3d.io/v5.1.0/usage/advanced/calico.yaml
# load all the images
$ bin/image-load --k3d proxy controller policy-controller web metrics-api tap cni-plugin jaeger-webhook
# install linkerd-cni
$ bin/go-run cli install-cni|k apply -f -
# install linkerd-crds
$ bin/go-run cli install --crds|k apply -f -
# install linkerd-control-plane in CNI mode
$ bin/go-run cli install --linkerd-cni-enabled|k apply -f -
# Pods should come up without issues. You can also try the viz and jaeger extensions.
# Try removing one of the securityContext entries added in this PR, and the Pod
# won't come up. You should be able to see the PodSecurity error in the associated
# ReplicaSet.
```
To test the multicluster extension using CNI, check this [gist](https://gist.github.com/alpeb/4cbbd5ad87538b9e0d39a29b4e3f02eb) with a patch to run the multicluster integration test with CNI in k8s 1.25.
When CNI plugins run in ebpf mode, they may rewrite the packet
destination when doing socket-level load balancing (i.e in the
`connect()` call). In these cases, skipping `443` on the outbound side
for control plane components becomes redundant; the packet is re-written
to target the actual Kubernetes API Server backend (which typically
listens on port `6443`, but may be overridden when the cluster is
created).
This change adds port `6443` to the list of skipped ports for control
plane components. On the linkerd-cni plugin side, the ports are
non-configurable. Whenever a pod with the control plane component label
is handled by the plugin, we look-up the `kubernetes` service in the
default namespace and append the port values (of both ClusterIP and
backend) to the list.
On the initContainer side, we make this value configurable in Helm and
provide a sensible default (`443,6443`). Users may override this value
if the ports do not correspond to what they have in their cluster. In
the CLI, if no override is given, we look-up the service in the same way
that we do for linkerd-cni; if failures are encountered we fallback to
the default list of ports from the values file.
Closes#9817
Signed-off-by: Matei David <matei@buoyant.io>
This change aims to solve two distinct issues that have cropped up in
the proxy-init configuration.
First, it decouples `allowPrivilegeEscalation` from running proxy-init
as root. At the moment, whenever the container is run as root, privilege
escalation is also allowed. In more restrictive environments, this will
prevent the pod from coming up (e.g security policies may complain about
`allowPrivilegeEscalation=true`). Worth noting that privilege escalation
is not necessary in many scenarios since the capabilities are passed to
the iptables child process at build time.
Second, it introduces a new `privileged` value that will allow users to
run the proxy-init container without any restrictions (meaning all
capabilities are inherited). This is essentially the same as mapping
root on host to root in the container. This value may solve issues in
distributions that run security enhanced linux, since iptables will be
able to load kernel modules that it may otherwise not be able to load
(privileged mode allows the container nearly the same privileges as
processes running outside of a container on a host, this further allows
the container to set configurations in AppArmor or SELinux).
Privileged mode is independent from running the container as root. This
gives users more control over the security context in proxy-init. The
value may still be used with `runAsRoot: false`.
Fixes#9718
Signed-off-by: Matei David <matei@buoyant.io>
* edge-22.11.3 change notes
Besides the notes, this corrects a small point in `RELEASE.md`, and
bumps the proxy-init image tag to `v2.1.0`. Note that the entry under
`go.mod` wasn't bumped because moving it past v2 requires changes on
`linkerd2-proxy-init`'s `go.mod` file, and we're gonna drop that
dependency soon anyways. Finally, all the charts got their patch version
bumped, except for `linkerd2-cni` that got its minor bumped because of
the tolerations default change.
## edge-22.11.3
This edge release fixes connection errors to pods using a `hostPort` different
than their `containerPort`. Also the `network-validator` init container improves
its logging, and the `linkerd-cni` DaemonSet now gets deployed in all nodes by
default.
* Fixed `destination` service to properly discover targets using a `hostPort`
different than their `containerPort`, which was causing 502 errors
* Upgraded the `network-validator` with better logging allowing users to
determine whether failures occur as a result of their environment or the tool
itself
* Added default `Exists` toleration to the `linkerd-cni` DaemonSet, allowing it
to be deployed in all nodes by default, regardless of taints
Co-authored-by: Oliver Gould <ver@buoyant.io>
When calling `linkerd upgrade, if the `linkerd-config-overrides` Secret is not found then we ask the user to run `linkerd repair`, but that has long been removed from the CLI.
Also removed code comment as the error is explicit enough.
Fix upgrade when using --from-manifests
When the `--from-manifests` flag is used to upgrade through the CLI,
the kube client used to fetch existing configuration (from the
ConfigMap) is a "fake" client. The fake client returns values from a
local source. The two clients are used interchangeably to perform the
upgrade; which one is initialized depends on whether a value has been
passed to `--from-manifests`.
Unfortunately, this breaks CLI upgrades to any stable-2.12.x version
when the flag is used. Since a fake client is used, the upgrade will
fail when checking for the existence of CRDs, even if the CRDs have been
previously installed in the cluster.
This change fixes the issue by first initializing an actual Kubernetes
client (that will be used to check for CRDs). If the values should be
read from a local source, the client is replaced with a fake one. Since
this takes place after the CRD check, the upgrade will not fail on the
CRD precondition.
Fixes#9788
Signed-off-by: Matei David <matei@buoyant.io>
When users use CNI, we want to ensure that network rewriting inside the pod is setup before allowing linkerd to start. When rewriting isn't happening, we want to exit with a clear error message and enough information in the container log for the administrator to either file a bug report with us or fix their configuration.
This change adds a validator initContainer to all injected workloads, when linkerd is installed with "cniEnabled=false". The validator replaces the noop init container, and will prevent pods from starting up if iptables is not configured.
Part of #8120
Signed-off-by: Steve Jenson <stevej@buoyant.io>
Add a "noop" init container which uses the proxy image and runs `/bin/sleep 0` to injected pods. This init container is only added when the linkerd-cni-plugin is enabled. The idea here is that by running an init container, we trigger kubernetes to update the pod status. In particular, this ensures that the pod status IP is populated, which is necessary in certain cases where other CNIs such as Calico are involved.
Therefore, this may fix https://github.com/linkerd/linkerd2/issues/9310, but I don't have a reproduction and therefore am not able to verify.
Signed-off-by: Alex Leong <alex@buoyant.io>
Add PodMonitor resources to the Helm chart
With an external Prometheus setup installed using prometheus-operator the Prometheus instance scraping can be configured using Service/PodMonitor resources.
By adding PodMonitor resource into Linkerd Helm chart we can mimic the configuration of bundled Prometheus, see https://github.com/linkerd/linkerd2/blob/main/viz/charts/linkerd-viz/templates/prometheus.yaml#L47-L151, that comes with linkerd-viz extension. The PodMonitor resources are based on https://github.com/linkerd/website/issues/853#issuecomment-913234295 which are proven to be working. The only problem we face is that bundled Grafana charts will need to look at different jobs when querying metrics.
When enabled by `podMonitor.enabled` value in the Helm chart, PodMonitor for Linkerd resources should be installed alongside the Linkerd and Linkerd metrics should be present in the Prometheus.
Fixes#6596
Signed-off-by: Martin Odstrcilik <martin.odstrcilik@gmail.com>
Fixes issue described in [this comment](https://github.com/linkerd/linkerd2/issues/9310#issuecomment-1247201646)
Rollback #7382
Should be cherry-picked back into 2.12.1
For 2.12.0, #7382 removed the env vars `_l5d_ns` and `_l5d_trustdomain` from the proxy manifest because they were no longer used anywhere. In particular, the jaeger injector used them when injecting the env var `LINKERD2_PROXY_TAP_SVC_NAME=tap.linkerd-viz.serviceaccount.identity.$(_l5d_ns).$(_l5d_trustdomain)` but then started using values.yaml entries instead of these env vars.
The problem is when upgrading the core control plane (or anything else) to 2.12.0, the 2.11 jaeger extension will still be running and will attempt to inject the old env var into the pods, making reference to `l5d_ns` and `_l5d_trustdomain` which the new proxy container won't offer anymore. This will put the pod in an error state.
This change restores back those env vars. We will be able to remove them at last in 2.13.0, when presumably the jaeger injector would already have already been upgraded to 2.12 by the user.
Replication steps:
```bash
$ curl -sL https://run.linkerd.io/install | LINKERD2_VERSION=stable-2.11.4 sh
$ linkerd install | k apply -f -
$ linkerd jaeger install | k apply -f -
$ linkerd check
$ curl -sL https://run.linkerd.io/install | LINKERD2_VERSION=stable-2.12.0 sh
$ linkerd upgrade --crds | k apply -f -
$ linkerd upgrade | k apply -f -
$ k get po -n linkerd
NAME READY STATUS RESTARTS AGE
linkerd-identity-58544dfd8-jbgkb 2/2 Running 0 2m19s
linkerd-destination-764bf6785b-v8cj6 4/4 Running 0 2m19s
linkerd-proxy-injector-6d4b8c9689-zvxv2 2/2 Running 0 2m19s
linkerd-identity-55bfbf9cd4-4xk9g 0/2 CrashLoopBackOff 1 (5s ago) 32s
linkerd-proxy-injector-5b67589678-mtklx 0/2 CrashLoopBackOff 1 (5s ago) 32s
linkerd-destination-ff9b5f67b-jw8w5 0/4 PostStartHookError 0 (8s ago) 32s
```
The identity controller requires access to read all deployments. This
isn't necessary.
When these permissions were added in #3600, we incorrectly assumed that
we must pass a whole Deployment resource as a _parent_ when recording
events. The [EventRecorder docs] say:
> 'object' is the object this event is about. Event will make a
> reference--or you may also pass a reference to the object directly.
We can confirm this by reviewing the source for [GetReference]: we can
simply construct an ObjectReference without fetching it from the API.
This change lets us drop unnecessary privileges in the identity
controller.
[EventRecorder docs]: https://pkg.go.dev/k8s.io/client-go/tools/record#EventRecorder
[GetReference]: ab826d2728/tools/reference/ref.go (L38-L45)
Signed-off-by: Oliver Gould <ver@buoyant.io>
policy/v1beta1 PodDisruptionBudget is deprecated in K8s v1.21+ and unavailable in v1.25+.
This change updates the API version to policy/v1.
Signed-off-by: Oliver Gould <ver@buoyant.io>
Signed-off-by: Kevin Leimkuhler <kleimkuhler@icloud.com>
Co-authored-by: Oliver Gould <ver@buoyant.io>
Co-authored-by: Kevin Leimkuhler <kleimkuhler@icloud.com>
In #6635 (f9f3ebe), we removed the `Namespace` resources from the
linkerd Helm charts. But this change also removed the `namespace` field
from all generated metadata, adding conditional logic to only include it
when being installed via the CLI.
This conditional logic currently causes spurious whitespace in output
YAML. This doesn't cause problems but is aesthetically
inconsistent/distracting.
This change removes the `partials.namespace` helper and instead inlines
the value in our templates. This makes our CLI- and Helm-generated
manifests slightly more consistent and removes needless indirection.
Signed-off-by: Oliver Gould <ver@buoyant.io>
Closes#9312#9118 introduced the `linkerd.io/trust-root-sha256` annotation which is
automatically added to control plane components.
This change ensures that all injected workloads also receive this annotation.
Signed-off-by: Kevin Leimkuhler <kleimkuhler@icloud.com>
Signed-off-by: William Morgan <william@buoyant.io>
<!-- Thanks for sending a pull request!
If you already have a well-structured git commit message, chances are GitHub
set the title and description of this PR to the git commit message subject and
body, respectively. If so, you may delete these instructions and submit your PR.
If this is your first time, please read our contributor guide:
https://github.com/linkerd/linkerd2/blob/main/CONTRIBUTING.md
The title and description of your Pull Request should match the git commit
subject and body, respectively. Git commit messages are structured as follows:
```
Subject
Problem
Solution
Validation
Fixes #[GitHub issue ID]
DCO Sign off
```
Example git commit message:
```
Introduce Pull Request Template
GitHub's community guidelines recommend a pull request template, the repo was
lacking one.
Introduce a `PULL_REQUEST_TEMPLATE.md` file.
Once merged, the
[Community profile checklist](https://github.com/linkerd/linkerd2/community)
should indicate the repo now provides a pull request template.
Fixes#3321
Signed-off-by: Jane Smith <jane.smith@example.com>
```
Note the git commit message subject becomes the pull request title.
For more details around git commits, see the section on Committing in our
contributor guide:
https://github.com/linkerd/linkerd2/blob/main/CONTRIBUTING.md#committing
-->
Signed-off-by: William Morgan <william@buoyant.io>
Closes#9230#9202 prepped the release candidate for `stable-2.12.0` by removing the `-edge`
suffix and adding the `-rc2` suffix.
This preps the chart versions for the stable release by removing that `-rc2`
suffix.
Signed-off-by: Kevin Leimkuhler <kleimkuhler@icloud.com>
The `linkerd authz` command help indicates that it shows server authorizations, but it shows authorization policies too.
We update the help text to indicate that all authorizations are shown.
Signed-off-by: Alex Leong <alex@buoyant.io>
<!-- Thanks for sending a pull request!
If you already have a well-structured git commit message, chances are GitHub
set the title and description of this PR to the git commit message subject and
body, respectively. If so, you may delete these instructions and submit your PR.
If this is your first time, please read our contributor guide:
https://github.com/linkerd/linkerd2/blob/main/CONTRIBUTING.md
The title and description of your Pull Request should match the git commit
subject and body, respectively. Git commit messages are structured as follows:
```
Subject
Problem
Solution
Validation
Fixes #[GitHub issue ID]
DCO Sign off
```
Example git commit message:
```
Introduce Pull Request Template
GitHub's community guidelines recommend a pull request template, the repo was
lacking one.
Introduce a `PULL_REQUEST_TEMPLATE.md` file.
Once merged, the
[Community profile checklist](https://github.com/linkerd/linkerd2/community)
should indicate the repo now provides a pull request template.
Fixes#3321
Signed-off-by: Jane Smith <jane.smith@example.com>
```
Note the git commit message subject becomes the pull request title.
For more details around git commits, see the section on Committing in our
contributor guide:
https://github.com/linkerd/linkerd2/blob/main/CONTRIBUTING.md#committing
-->
Signed-off-by: Alex Leong <alex@buoyant.io>
In 2.11.x, proxyInit.runAsRoot was true by default, which caused the
proxy-init's runAsUser field to be 0. proxyInit.runAsRoot is now
defaulted to false in 2.12.0, but runAsUser still isn't
configurable, and when following the upgrade instructions
here, helm doesn't change runAsUser and so it conflicts with the new value
for runAsRoot=false, resulting in the pods erroring with this message:
Error: container's runAsUser breaks non-root policy (pod: "linkerd-identity-bc649c5f9-ckqvg_linkerd(fb3416d2-c723-4664-acf1-80a64a734561)", container: linkerd-init)
This PR adds a new default for runAsUser to avoid this issue.
This release is the second release candidate for stable-2.12.0.
At this point the Helm charts can be retrieved from the stable repo:
```
helm repo add linkerd https://helm.linkerd.io/stable
helm repo up
helm install linkerd-crds -n linkerd --create-namespace linkerd/linkerd-crds
helm install linkerd-control-plane \
-n linkerd \
--set-file identityTrustAnchorsPEM=ca.crt \
--set-file identity.issuer.tls.crtPEM=issuer.crt \
--set-file identity.issuer.tls.keyPEM=issuer.key \
linkerd/linkerd-control-plane
```
The following lists all the changes since edge-22.8.2:
* Fixed inheritance of the `linkerd.io/inject` annotation from Namespace to
Workloads when its value is `ingress`
* Added the `config.linkerd.io/default-inbound-policy: all-authenticated`
annotation to linkerd-multicluster’s Gateway deployment so that all clients
are required to be authenticated
* Added a `ReadHeaderTimeout` of 10s to all the go `http.Server` instances, to
avoid being vulnerable to "slowrolis" attacks
* Added check in `linkerd viz check --proxy` to warn in case namespace have the
`config.linkerd.io/default-inbound-policy: deny` annotation, which would not
authorize scrapes coming from the linkerd-viz Prometheus instance
* Added validation for accepted values for the `--default-inbound-policy` flag
* Fixed invalid URL in the `linkerd install --help` output
* Added `--destination-pod` flag to `linkerd diagnostics endpoints` subcommand
* Added `proxyInit.runAsUser` in `values.yaml` defaulting to non-zero, to
complement the new default `proxyInit.runAsRoot: false` that was rencently
changed
Depends on #9195
Currently, `linkerd inject --default-inbound-policy` does not set the
`config.linkerd.io/default-inbound-policy` annotation on the injected
resource(s).
The `inject` command does _try_ to set that annotation if it's set in
the `Values` generated by `proxyFlagSet`:
14d1dbb3b7/cli/cmd/inject.go (L485-L487)
...but, the flag in the proxy `FlagSet` doesn't set
`Values.Proxy.DefaultInboundPolicy`, it sets
`Values.PolicyController.DefaultAllowPolicy`:
7c5e3aaf40/cli/cmd/options.go (L375-L379)
This is because the flag set is shared across `linkerd inject` and
`linkerd install` subcommands, and in `linkerd install`, we want to set
the default policy for the whole cluster by configuring the policy
controller. In `linkerd inject`, though, we want to add the annotation
to the injected pods only.
This branch fixes this issue by changing the flag so that it sets the
`Values.Proxy.DefaultInboundPolicy` instead of the
`Values.PolicyController.DefaultAllowPolicy` value. In `linkerd
install`, we then set `Values.PolicyController.DefaultAllowPolicy` based
on the value of `Values.Proxy.DefaultInboundPolicy`, while in `inject`,
we will now actually add the annotation.
This branch is based on PR #9195, which adds validation to reject
invalid values for `--default-inbound-policy`, rather than on `main`.
This is because the validation code added in that PR had to be moved
around a bit, since it now needs to validate the
`Values.Proxy.DefaultInboundPolicy` value rather than the
`Values.PolicyController.DefaultAllowPolicy` value. I thought using
#9195 as a base branch was better than basing this on `main` and then
having to resolve merge conflicts later. When that PR merges, this can
be rebased onto `main`.
Fixes#9168
Closes#9141
This introduces the `--destination-pod` flag to the `linkerd diagnostics
endpoints` command which allows users to target a specific destination Pod when
there are multiple running in a cluster.
This can be useful for issues like #8956, where Linkerd HA is installed and
there seem to be stale endpoints in the destination service. Being able to run
this command and identity which destination Pod (if not all) have an incorrect
view of the cluster.
Signed-off-by: Kevin Leimkuhler <kleimkuhler@icloud.com>
Closes#9148
With this change, the value of `—default-inbound-policy` is verified to be one
of the accepted values.
When the value is not an accepted value we now error
```shell $ linkerd install --default-inbound-policy=everybody Error:
--default-inbound-policy must be one of: all-authenticated, all-unauthenticated,
cluster-authenticated, cluster-unauthenticated, deny (got everybody) Usage:
linkerd install [flags]
... ```
A unit test has also been added.
Signed-off-by: Kevin Leimkuhler <kleimkuhler@icloud.com>
* Bump proxy-init to v2.0.0
New release of proxy-init.
Updated:
* Helm values to use v2.0.0 of proxy-init
* Helm docs
* Tests
Note: go dependencies have not been updated since the new version will
break API compatibility with older versions (source files have been
moved, see issue for more details).
Closes#9164
Signed-off-by: Matei David <matei@buoyant.io>
Signed-off-by: Oliver Gould <ver@buoyant.io>
Signed-off-by: Matei David <matei@buoyant.io>
Signed-off-by: Oliver Gould <ver@buoyant.io>
Co-authored-by: Oliver Gould <ver@buoyant.io>
When Linkerd is installed with the `--default-inbound-policy` flag, this value gets propagated to the `proxy.defaultInboundPolicy` value which sets the `LINKERD2_PROXY_INBOUND_DEFAULT_POLICY` proxy env var, but not to the `policyController.defaultAllowPolicy` value which sets the `--default-policy` flag on the policy-controller.
Since the policy-controller returns default servers when a server resource does not exist, this causes the `--default-inbound-policy` value to be effectively ignored. We update this to set the `PolicyController.DefaultAllowPolicy` value which is used by the proxy as the default when `proxy.defaultInboundPolicy` is not set.
Signed-off-by: Alex Leong <alex@buoyant.io>
Fixes#9022
When updating the Linkerd trust root, for example by running a command like `linkerd upgrade --identity-trust-anchors-file=./bundle.crt | kubectl apply -f -` as described in the [trust root rotation docs](https://linkerd.io/2.11/tasks/manually-rotating-control-plane-tls-credentials/#rotating-the-trust-anchor), the trust root is updated in the Linkerd config, but the identity controller does not restart and does not pick up the new root.
We add a trust root checksum annotation which causes the control plane deployments to change when the trust anchor changes, and thus causes them to restart.
Signed-off-by: Alex Leong <alex@buoyant.io>
* Allows RSA signed trust anchors on linkerd cli (#7771)
Linkerd currently forces using an ECDSA P-256
issuer certificate along with a ECDSA trust
anchor. Still, it's still cryptographically valid
to have an ECDSA P-256 issuer certificate issued
by an RSA signed CA.
CheckCertAlgoRequirements checks if CA cert uses
ECDSA or RSA 2048/4096 signing algorithm.
Fixes#7771
Signed-off-by: Baeyens, Daniel <daniel.baeyens@gmail.com>
Co-authored-by: Alejandro Pedraza <alejandro@buoyant.io>
Some hosts may not have 'nft' modules available. Currently, proxy-init
defaults to using 'iptables-nft'; if the host does not have support for
nft modules, the init container will crash, blocking all injected
workloads from starting up.
This change defaults the 'iptablesMode' value to 'legacy'.
* Update linkerd-control-plane/values file default
* Update proxy-init partial to default to 'legacy' when no mode is
specified
* Change expected values in 'pkg/charts/linkerd2/values_test.go' and in
'cli/cmd/install_test'
* Update golden files
Fixes#9053
Signed-off-by: Matei David <matei@buoyant.io>
Closes#8945
This adds the `policyController.probeNetworks` configuration value so that users
can configure the networks from which probes are expected to be performed.
By default, we allow all networks (`0.0.0.0/0`). Additionally, this value
differs from `clusterNetworks` is that it is a list of networks, and thus we
have to join the values in the Helm templating.
Signed-off-by: Kevin Leimkuhler <kleimkuhler@icloud.com>
> In some circumstances, the lifecycle.postStart hook can cause the linkerd-proxy
> container to get stuck waiting for identity verification. After the
> linkerd-await timeout, the container will be restarted and the proxy starts
> without further incident. The linkerd-control-plane helm chart currently has a
> way to disable the lifecycle hook for injected proxies, but not for proxies on
> the control plane pods.
>
> This commit adds a new value to the linkerd-control-plane chart of
> proxy.controlPlaneAwait that can be used to disable the postStart lifecycle hook
> on the destination and proxy-injector pods. This is defaulted to true to
> maintain current behavior.
>
> The linkerd-control-plane chart was templated, setting proxy.controlPlaneAwait
> to true and false, verifying that the postStart lifecycle hook was either
> present or absent depending on the proxy.controlPlaneAwait value.
>
> Fixes#8738
This continues the now stale #8739 and removes the version bumps that were
requested.
Signed-off-by: Jacob Lambert [calrisian777@gmail.com](mailto:calrisian777@gmail.com)
Co-authored-by: Kevin Leimkuhler <kleimkuhler@icloud.com>
Previously, the `linkerd authz` command would list all ServerAuthorization resources which targeted the specified resource. With the addition of AuthorizationPolicies, we update this command to also show all AuthorizationPolicies which target the specified resource. In cases where the AuthorizationPolicy targets an HTTPRoute which belongs to the resource, we also print the HTTPRoute name.
Sample output:
```
linkerd authz -n emojivoto po
ROUTE SERVER AUTHORIZATION_POLICY SERVER_AUTHORIZATION
* emoji-grpc emoji-grpc
linkerd-metrics linkerd-admin linkerd-metrics
linkerd-probes linkerd-admin linkerd-probes
* prom prom
* voting-grpc voting-grpc
* web-http web-public
```
Signed-off-by: Alex Leong <alex@buoyant.io>
* Update CRD chart version in golden file
Signed-off-by: Alex Leong <alex@buoyant.io>
* Run go tests on charts or golden changes
Signed-off-by: Alex Leong <alex@buoyant.io>
This change introduces a new value to be used at install (or upgrade)
time. The value (`proxyInit.iptablesMode=nft|legacy`) is responsible
for starting the proxy-init container in nft or legacy mode.
By default, the init container will use iptables-nft. When the mode is set to
`nft`, it will instead use iptables-nft. Most modern Linux distributions
support both, but a subset (such as RHEL based families) only support
iptables-nft and nf_tables.
Signed-off-by: Matei David <matei@buoyant.io>
Go 1.18 features a number of important chanages, notably removing client
support for defunct TLS versions: https://tip.golang.org/doc/go1.18
This change updates our Go version in CI and development.
Signed-off-by: Oliver Gould <ver@buoyant.io>
There are several undocumented Helm values that configure control
plane resource constraints. This change fixes the default values to
include this missing documentation.
Fixes#8933
Signed-off-by: Dominik Táskai <dtaskai@pm.me>
As discussed in #8944, Linkerd's current use of the
`gateway.networking.k8s.io` `HTTPRoute` CRD is not a spec-compliant use
of the Gateway API, because we don't support some "core" features of the
Gateway API that don't make sense in Linkerd's use-case. Therefore,
we've chosen to replace the `gateway.networking.k8s.io` `HTTPRoute` CRD
with our own `HTTPRoute` CRD in the `policy.linkerd.io` API group, which
removes the unsupported features.
PR #8949 added the Linkerd versions of those CRDs, but did not remove
support for the Gateway API CRDs. This branch removes the Gateway API
CRDs from the policy controller and `linkerd install`/Helm charts.
The various helper functions for converting the Gateway API resource
binding types from `k8s-gateway-api` to the policy controller's internal
representation is kept in place, but the actual use of that code in the
indexer is disabled. This way, we can add support for the Gateway API
CRDs again easily. Similarly, I've kept the validation code for Gateway
API types in the policy admission controller, but the admission
controller no longer actually tries to validate those resources.
Depends on #8949Closes#8944
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
This change bumps the proxy-init version from v1.6.1 to the latest
version, v1.6.2. As part of the new release, proxy-init now adds
net_admin and net_raw sys caps to xtables-nft-multi so that nftables
mode can be used without requiring root privileges.
* Bump go.mod
* Bump version in helm values
* Bump version in misc files
* Bump version in code
Signed-off-by: Matei David <matei@buoyant.io>
Our use of the `gateway.networking.k8s.io` types is not compliant with
the gateway API spec in at least a few ways:
1. We do not support the `Gateway` types. This is considered a "core"
feature of the `HTTPRoute` type.
2. We do not currently update `HTTPRoute` status fields as dictated by
the spec.
3. Our use of Linkerd-specific `parentRef` types may not work well with
the gateway project's admission controller (untested).
Issue #8944 proposes solving this by replacing our use of
`gateway.networking.k8s.io`'s `HTTPRoute` type with our own
`policy.linkerd.io` version of the same type. That issue suggests that
the new `policy.linkerd.io` types be added separately from the change
that removes support for the `gateway.networking.k8s.io` versions, so
that the migration can be done incrementally.
This branch does the following:
* Add new `HTTPRoute` CRDs. These are based on the
`gateway.networking.k8s.io` CRDs, with the following changes:
- The group is `policy.linkerd.io`,
- The API version is `v1alpha1`,
- `backendRefs` fields are removed, as Linkerd does not support them,
- filter types Linkerd does not support (`RequestMirror` and
`ExtensionRef`), are removed.
* Add Rust bindings for the new `policy.linkerd.io` versions of
`HTTPRoute` types in `linkerd-policy-controller-k8s-api`.
The Rust bindings define their own versions of the `HttpRoute`,
`HttpRouteRule`, and `HttpRouteFilter` types, because these types'
structures are changed from the Gateway API versions (due to the
removal of unsupported filter types and fields). For other types,
which are identical to the upstream Gateway API versions (such as the
various match types and filter types), we re-export the existing
bindings from the `k8s-gateway-api`crate to minimize duplication.
* Add conversions to `InboundRouteBinding` from the `policy.linkerd.io`
`HTTPRoute` types.
When possible, I tried to factor out the code that was shared between
the conversions for Linkerd's `HTTPRoute` types and the upstream
Gateway API versions.
* Implement `kubert`'s `IndexNamespacedResource` trait for
`linkerd_policy_controller_k8s_api::policy::HttpRoute`, so that the
policy controller can index both versions of the `HTTPRoute` CRD.
* Adds validation for `policy.linkerd.io` `HTTPRoute`s to the policy
controller's validating admission webhook.
* Updated the policy controller tests to test both versions of
`HTTPRoute`.
## Notes
A couple questions I had about this approach:
- Is re-using bindings from the `k8s-gateway-api` crate appropriate
here, when the type has not changed from the Gateway API version? If
not, I can change this PR to vendor those types as well, but it will
result in a lot more code duplication.
- Right now, the indexer stores all `HTTPRoute`s in the same index.
This means that applying a `policy.linkerd.io` version of `HTTPRoute`
and then applying the Gateway API version with the same ns/name will
update the same value in the index. Is this what we want? I wasn't
entirely sure...
See #8944.
Dependencies like `kubert` may emit INFO level logs that are useful to
see (e.g., when the serviceaccount has insufficient RBAC). This change
updates the default policy controller log level to simply be `info`.
Signed-off-by: Oliver Gould <ver@buoyant.io>
This release adds support for per-route authorization policy using the
AuthorizationPolicy and HttpRoute resources. It also adds a configurable
shutdown grace period to the proxy which can be used to ensure that proxy
graceful shutdown completes within a certain time, even if there are outstanding
open connections.
* Removed kube-system exclusions from watchers to fix service discovery for
workloads in the kube-system namespace (thanks @JacobHenner)
* Added annotations to allow Linkerd extension deployments to be evicted by the
autoscaler when necessary
* Added missing port in the Linkerd viz chart documentation (thanks @haswalt)
* Added support for per-route policy by supporting AuthorizationPolicy resources
which target HttpRoute resources
* Fixed the `linkerd check` command crashing when unexpected pods are found in
a Linkerd namespace
* Added a `config.linkerd.io/shutdown-grace-period` annotation to configure the
proxy's maximum grace period for graceful shutdown
Signed-off-by: Alex Leong <alex@buoyant.io>
PR linkerd/linkerd2-proxy#1815 added support for a
`LINKERD2_PROXY_SHUTDOWN_GRACE_PERIOD` environment variable that
configures the proxy's maximum grace period for graceful shutdown. This
is intended to ensure that if a proxy is shut down, it will eventually
terminate in a relatively timely manner, even if some stubborn
connections don't close gracefully.
This branch adds support for a `config.linkerd.io/shutdown-grace-period`
annotation that can be used to override the default grace period
duration.
Hopefully I've added this everywhere it needs to be added --- please let
me know if I've missed anything!
Release v1.6.1 of proxy-init adds support for iptables-nft. This change
bumps up the proxy-init version used in code, chart values, and golden
files.
* Update go.mod dep
* Update CNI plugin with new opts
* Update proxy-init ref in golden files and chart values
* Update policy controller CI workflow
Signed-off-by: Matei David <matei@buoyant.io>
linkerd2-proxy-api v0.6.0 adds support for inbound proxies to discover
route configurations based on the Gateway API HTTPRoute types. This
change updates the policy controller to index
`gateway.networking.k8s.io/v1beta` `HTTPRoute` types to discover these
policies from the Kubernetes API.
`HTTPRoute` resources may target `Server` resources (as a `parentRef`)
to attach policies to an inbound proxy. When no routes are configured,
a default route is synthesized to allow traffic; but when at least one
route attaches to a server, only requests that match a route are
permitted (other requests are failed with a 404).
Only the *core* subset of the `HTTPRoute` filters are supported:
`RequestRedirect` and `RequestHeaderModifier`. Backends may *not* be
configured on these routes (since they may only apply to inbound/server-
side proxies). No `status` updates are currently performed on these
`HTTPRoute` resources.
This change does not yet allow `AuthorizationPolicy` resources to target
`HTTPRoute` resources. This will be added in a follow-up change.
Signed-off-by: Alex Leong <alex@buoyant.io>
Co-authored-by: Oliver Gould <ver@buoyant.io>
Fixes#8665
We add validation for HTTPRoute resources to the policy admission controller. We validate that for any HTTPRoute which has a Server as a parent_ref, that it doesn't have unsupported filters. For the moment we do not support any HTTP filters. As we add support for HTTP filter types, we should update the validator accordingly.
Signed-off-by: Alex Leong <alex@buoyant.io>
Co-authored-by: Oliver Gould <ver@buoyant.io>
Alex Leong