Problem - Current does Linkerd CNI Helm chart templates have hostNetwork: true set which is unnecessary and less secure.
Solution - Removed hostNetwork: true from linkerd-cni Helm chart templates
PR Fixes#11141
---------
Signed-off-by: Abhijeet Gaurav <abhijeetdav24aug@gmail.com>
Co-authored-by: Alejandro Pedraza <alejandro@buoyant.io>
Move `linkerd.io/cni-resource: "true"` from an annotation to a label in linkerd-cni.
This will improve the decoupling of the linkerd cni and the linkerd control plane (proxy injector).
With this change, even if the proxy injector is broken (for whatever reason), the linkerd-cni can still come up. Currently, since the label is not present (and if you configure the MutatingWebhook with a policy `fail`) the cni pods will completely fail to come up and this might cause and outage.
See default `objectSelector` for the `MutatingWebhook`: https://github.com/linkerd/linkerd2/blob/main/charts/linkerd-control-plane/values.yaml#L370Closes#11058
Signed-off-by: Miguel Elias dos Santos <migueleliasweb@gmail.com>
This release stops using the "interface" mode, and instead wait till
another CNI plugin drops a proper network config and then append the
linkerd CNI config to it. This avoids having pods start before proper
networking is established in the node.
This release of the CNI plugin changes the base runtime Docker image
from `debian:bullseye-slim` to `alpine:3.17.3`.
---
* cni: use `scratch` as the base runtime docker image (linkerd/linkerd2-proxy-init/pull/237)
* cni: change base runtime image from `scratch` to `alpine` (linkerd/linkerd2-proxy-init#238)
Fixed incompatibility issue with AWS CNI addon in EKS, that was
forbidding pods to acquire networking after scaling up nodes.
Credits to @frimik for providing a diagnosis and fix, and to @JonKusz for the detailed repro
This PR added support for --set flag to linkerd cni-plugin installation command.
Also made changes to test file for cni-plugin install.
Fixed a bug at pkg/chart/charts.go for resources template.
fixes#9917
* Allow supporting all flags and values
This leverages `chartutil.CoalesceValues` in order to merge the values provided through regular flags, and the ones provided via `--set` flags. As that function consumes maps, I introduced the `ToMap` method function on the cni `Values` struct (a copy of the same function from the core linkerd `Values` struct) to convert the struct backing the regular flags into a map.
And for the `RenderCNI` method to be able to deal with value maps instead of yaml, the `charts.Chart` struct now distinguishes between `Values` (a map) and `RawValues` (YAML).
This allowed removing the `parseYAMLValue` function and avoid having to deal with individual entries in `buildValues()`, and we no longer need the `valuesOverrides` field in the `cniPluginOptions` struct.
## Tests
```bash
# Testing regular flag
$ bin/go-run cli install-cni --use-wait-flag | grep use.wait.flag
"use-wait-flag": true
# Testing using --set
$ bin/go-run cli install-cni --set useWaitFlag=true | grep use.wait.flag
"use-wait-flag": true
# Testing using --set on a setting that has no regular flag
$ bin/go-run cli install-cni --set enablePSP=true | grep PodSecurityPolicy
kind: PodSecurityPolicy
```
---------
Signed-off-by: amit-62 <kramit6662@gmail.com>
Co-authored-by: Alejandro Pedraza <alejandro.pedraza@gmail.com>
Co-authored-by: Matei David <matei.david.35@gmail.com>
proxy-init v2.2.1:
* Sanitize `subnets-to-ignore` flag
* Dep bumps
cni-plugin v1.1.0:
* Add support for the `config.linkerd.io/skip-subnets` annotation
* Dep bumps
validator v0.1.2:
* Dep bumps
Also, `linkerd-network-validator` is now released wrapped in a tar file, so this PR also amends `Dockerfile-proxy` to account for that.
wind the new linkerd-cni build through the build. refactor image, version, and pullPolicy into an Image object.
Signed-off-by: Steve Jenson <stevej@buoyant.io>
Fixes#10150
When we added PodSecurityAdmission in #9719 (and included in
edge-23.1.1), we added the entry `seccompProfile.type=RuntimeDefault` to
the containers SecurityContext.
For PSP to accept that we require to add the annotation
`seccomp.security.alpha.kubernetes.io/allowedProfileNames:
"runtime/default"` into the PSP resource, which also implies we require
to add the entry `seccompProfile.type=RuntimeDefault` to the pod's
SecurityContext as well, not just the container's.
It also turns out the `namespace-metadata` Jobs used by extensions for
the helm installation method didn't have their ServiceAccount properly
bound to the PSP resource. This resulted in the `helm install` command
failing, and although the extensions resources did get deployed, they
were not being discoverable by `linkerd check`. This change fixes that
as well, that has been broken since 2.12.0!
Closes#9676
This adds the `pod-security.kubernetes.io/enforce` label as described in [Pod Security Admission labels for namespaces](https://kubernetes.io/docs/concepts/security/pod-security-admission/#pod-security-admission-labels-for-namespaces).
PSA gives us three different possible values (policies or modes): [privileged, baseline and restricted](https://kubernetes.io/docs/concepts/security/pod-security-standards/).
For non-CNI mode, the proxy-init container relies on granting the NET_RAW and NET_ADMIN capabilities, which places those pods under the `restricted` policy. OTOH for CNI mode we can enforce the `restricted` policy, by setting some defaults on the containers' `securityContext` as done in this PR.
Also note this change also adds the `cniEnabled` entry in the `values.yaml` file for all the extension charts, which determines what policy to use.
Final note: this includes the fix from #9717, otherwise an empty gateway UID prevents the pod to be created under the `restricted` policy.
## How to test
As this is only enforced as of k8s 1.25, here are the instructions to run 1.25 with k3d using Calico as CNI:
```bash
# launch k3d with k8s v1.25, with no flannel CI
$ k3d cluster create --image='+v1.25' --k3s-arg '--disable=local-storage,metrics-server@server:0' --no-lb --k3s-arg --write-kubeconfig-mode=644 --k3s-arg --flannel-backend=none --k3s-arg --cluster-cidr=192.168.0.0/16 --k3s-arg '--disable=servicelb,traefik@server:0'
# install Calico
$ k apply -f https://k3d.io/v5.1.0/usage/advanced/calico.yaml
# load all the images
$ bin/image-load --k3d proxy controller policy-controller web metrics-api tap cni-plugin jaeger-webhook
# install linkerd-cni
$ bin/go-run cli install-cni|k apply -f -
# install linkerd-crds
$ bin/go-run cli install --crds|k apply -f -
# install linkerd-control-plane in CNI mode
$ bin/go-run cli install --linkerd-cni-enabled|k apply -f -
# Pods should come up without issues. You can also try the viz and jaeger extensions.
# Try removing one of the securityContext entries added in this PR, and the Pod
# won't come up. You should be able to see the PodSecurity error in the associated
# ReplicaSet.
```
To test the multicluster extension using CNI, check this [gist](https://gist.github.com/alpeb/4cbbd5ad87538b9e0d39a29b4e3f02eb) with a patch to run the multicluster integration test with CNI in k8s 1.25.
When CNI plugins run in ebpf mode, they may rewrite the packet
destination when doing socket-level load balancing (i.e in the
`connect()` call). In these cases, skipping `443` on the outbound side
for control plane components becomes redundant; the packet is re-written
to target the actual Kubernetes API Server backend (which typically
listens on port `6443`, but may be overridden when the cluster is
created).
This change adds port `6443` to the list of skipped ports for control
plane components. On the linkerd-cni plugin side, the ports are
non-configurable. Whenever a pod with the control plane component label
is handled by the plugin, we look-up the `kubernetes` service in the
default namespace and append the port values (of both ClusterIP and
backend) to the list.
On the initContainer side, we make this value configurable in Helm and
provide a sensible default (`443,6443`). Users may override this value
if the ports do not correspond to what they have in their cluster. In
the CLI, if no override is given, we look-up the service in the same way
that we do for linkerd-cni; if failures are encountered we fallback to
the default list of ports from the values file.
Closes#9817
Signed-off-by: Matei David <matei@buoyant.io>
Introduce fs watch for cni installer
Our CNI installer script is prone to race conditions, especially when a
node is rebooted, or restarted. Order of configuration should not matter
and our CNI plugin should attach to other plugins (i.e chain to them) or
run standalone when applicable. In order to be more flexible, we
introduce a filesystem watcher through inotifywait to react to changes
in the cni config directory. We react to changes based on SHAs.
Linkerd's CNI plugin should append configuration when at least one other
file exists, but if multiple files exist, the CNI plugin should not have
to make a decision on whether thats the current file to append itself
to. As a result, most of the logic in this commit revolves around the
assumption that whatever file we detect has been created should be
injected with Linkerd's config -- the rest is up to the host.
In addition, we also introduce a sleep in the cni preStop hook, changed to
using bash and introduce procps to get access to ps and pgrep.
Closes#8070
Signed-off-by: Matei David <matei@buoyant.io>
Co-authored-by: Oliver Gould <ver@buoyant.io>
Co-authored-by: Alejandro Pedraza <alejandro@buoyant.io>
* Fix HA race when installing through Helm
Fixes#7699
The problem didn't affect 2.11, only latest edges since the Helm charts
got split into `linkerd-crds` and `linkerd-control-plane` and we stopped
creating the linkerd namespace.
With the surrendering of the creation of the namespace, we can no longer
guarantee the existence of the `config.linkerd.io/admission-webhooks`
label, so this PR creates an `objectSelector` for the injector that
filters-out control-plane components, based on the existence of the
`linkerd.io/control-plane-component` label.
Given we still want the multicluster components to be injected, we had
to be rename its `linkerd.io/control-plane-component` label to
`component`, following the same convention used by the other extensions.
The corresponding Prometheus rule for scraping the service mirrors was
updated accordingly.
A similar filter was added for the linkerd-cni DaemonSet.
Also, now that the `kubernetes.io/metadata.name` is prevalent, we're
also using it to filter out the kube-system and cert-manager namespaces.
The former namespace was already mentioned in the docs; the latter is
also included to avoid having races with cert-manager-cainjector which
can be used to provision the injector's cert.
Fixes#7391
Supersedes #7527
Some environments required privileged access in order to deploy the
linkerd-cni config under `/host/etc/cni/net.d/`
Co-authored-by: Kim Christensen kimworking@gmail.com
Fixes#6584#6620#7405
# Namespace Removal
With this change, the `namespace.yaml` template is rendered only for CLI installs and not Helm, and likewise the `namespace:` entry in the namespace-level objects (using a new `partials.namespace` helper).
The `installNamespace` and `namespace` entries in `values.yaml` have been removed.
There in the templates where the namespace is required, we moved from `.Values.namespace` to `.Release.Namespace` which is filled-in automatically by Helm. For the CLI, `install.go` now explicitly defines the contents of the `Release` map alongside `Values`.
The proxy-injector has a new `linkerd-namespace` argument given the namespace is no longer persisted in the `linkerd-config` ConfigMap, so it has to be passed in. To pass it further down to `injector.Inject()` without modifying the `Handler` signature, a closure was used.
------------
Update: Merged-in #6638: Similar changes for the `linkerd-viz` chart:
Stop rendering `namespace.yaml` in the `linkerd-viz` chart.
The additional change here is the addition of the `namespace-metadata.yaml` template (and its RBAC), _not_ rendered in CLI installs, which is a Helm `post-install` hook, consisting on a Job that executes a script adding the required annotations and labels to the viz namespace using a PATCH request against kube-api. The script first checks if the namespace doesn't already have an annotations/labels entries, in which case it has to add extra ops in that patch.
---------
Update: Merged-in the approved #6643, #6665 and #6669 which address the `linkerd2-cni`, `linkerd-multicluster` and `linkerd-jaeger` charts.
Additional changes from what's already mentioned above:
- Removes the install-namespace option from `linkerd install-cni`, which isn't found in `linkerd install` nor `linkerd viz install` anyways, and it would add some complexity to support.
- Added a dependency on the `partials` chart to the `linkerd-multicluster-link` chart, so that we can tap on the `partials.namespace` helper.
- We don't have any more the restriction on having the muticluster objects live in a separate namespace than linkerd. It's still good practice, and that's the default for the CLI install, but I removed that validation.
Finally, as a side-effect, the `linkerd mc allow` subcommand was fixed; it has been broken for a while apparently:
```console
$ linkerd mc allow --service-account-name foobar
Error: template: linkerd-multicluster/templates/remote-access-service-mirror-rbac.yaml:16:7: executing "linkerd-multicluster/templates/remote-access-service-mirror-rbac.yaml" at <include "partials.annotations.created-by" $>: error calling include: template: no template "partials.annotations.created-by" associated with template "gotpl"
```
---------
Update: see helm/helm#5465 describing the current best-practice
# Core Helm Charts Split
This removes the `linkerd2` chart, and replaces it with the `linkerd-crds` and `linkerd-control-plane` charts. Note that the viz and other extension charts are not concerned by this change.
Also note the original `values.yaml` file has been split into both charts accordingly.
### UX
```console
$ helm install linkerd-crds --namespace linkerd --create-namespace linkerd/linkerd-crds
...
# certs.yaml should contain identityTrustAnchorsPEM and the identity issuer values
$ helm install linkerd-control-plane --namespace linkerd -f certs.yaml linkerd/linkerd-control-plane
```
### Upgrade
As explained in #6635, this is a breaking change. Users will have to uninstall the `linkerd2` chart and install these two, and eventually rollout the proxies (they should continue to work during the transition anyway).
### CLI
The CLI install/upgrade code was updated to be able to pick the templates from these new charts, but the CLI UX remains identical as before.
### Other changes
- The `linkerd-crds` and `linkerd-control-plane` charts now carry a version scheme independent of linkerd's own versioning, as explained in #7405.
- These charts are Helm v3, which is reflected in the `Chart.yaml` entries and in the removal of the `requirements.yaml` files.
- In the integration tests, replaced the `helm-chart` arg with `helm-charts` containing the path `./charts`, used to build the paths for both charts.
### Followups
- Now it's possible to add a `ServiceProfile` instance for Destination in the `linkerd-control-plane` chart.
In our chart values and (some) integration tests, we're using a deprecated
label for node selection. According to the warning messages we get during
installation, the label has been deprecated since k8s `v1.14`:
```
Warning: spec.template.spec.nodeSelector[beta.kubernetes.io/os]: deprecated since v1.14; use "kubernetes.io/os" instead
Warning: spec.jobTemplate.spec.template.spec.nodeSelector[beta.kubernetes.io/os]: deprecated since v1.14; use "kubernetes.io/os" instead
```
This PR changes all occurrences of `beta.kubernetes.io/node` with
`kubernetes.io/node`.
Fixes#7225
* Do not install PSP resources by default
Fixes#6549
PodSecurityPolicy is deprecated as of k8s v1.21 and will be unavailable starting k8s v1.25. This was causing warnings to be displayed in `linkerd install/upgrade` and `linkerd check`.
By default, do not include the linkerd PSP resource along with its Role and RoleBinding. If the user wants that, they can by setting `enablePSP: true`, a new config introduced for this purpose.
This was done in the linkerd, linkerd-cni, linkerd-viz, multicluster and jaeger charts.
The associated checks were also removed, including the NET_ADMIN+NET_RAW capabilities check, which rely on the PSP API.
Increase container security by making the root file system of the cni
install plugin read-only.
Change the temporary directory used in the cni install script, add a
writable EmptyDir volume and enable readOnlyFileSystem securityContext
in cni plugin helm chart.
Tested this by building the container image of the cni plugin and
installed the chart onto a cluster. Logs looked the same as before this
change.
Fixes#6468
Signed-off-by: Gerald Pape <gerald@giantswarm.io>
* Add missing ignoreInboundports to CNI config and also fixed ignoreOutboundports to support passing just one port
This change fixes an issue where tap does not work when running Linkerd
through Linkerd CNI installed via helm charts. This issue was caused by
the CNI chart's value not including tap control and admin ports in the
config. This caused tap request traffic to go to the inbound side of the
proxy as opposed to the respective tap control port.
Fixes#6224
Signed-off-by: Dennis Adjei-Baah <dennis@buoyant.io>
This change removes the default ignored inbound and outbound ports from the
proxy init configuration.
These ports have been moved to the the `proxy.opaquePorts` configuration so that
by default, installations will proxy all traffic on these ports opaquely.
Closes#5571Closes#5595
Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
The container-image `ghcr.io/linkerd/cni-plugin:stable-2.9.1` does not contain the `kill` command as an executable. Instead, it is available as a shell built-in. In its current state, Kubernetes emits error events whenever linkerd2-cni pods are terminated because the `kill` command can not be found.
Signed-off-by: Mitch Hulscher <mitch.hulscher@lib.io>
When testing the `linkerd2-cni` chart with `ct`, it flags up usage
of some deprecated apiVersions.
This PR aligns the RBAC API group across all resources in the chart.
---
Signed-off-by: Simon Weald <glitchcrab-github@simonweald.com>
* CNI add support for priorityClassName
As requested in #2981 one should be able to optionally define a priorityClassName for the linkerd2 pods.
With this commit support for priorityClassName is added to the CNI plugin helm chart as well as to the
cli command for installing the CNI plugin.
Also added an `installNamespace` Helm option for the CNI installation.
Implements part of #2981.
Signed-off-by: alex.berger@nexiot.ch <alex.berger@nexiot.ch>
Abhijeet Gaurav