Per #5165, Kubernetes does not necessarily limit the proxy's access to
cores via `cgroups` when a CPU limit is set. As of #5168, the proxy now
supports a `LINKERD2_PROXY_CORES` environment configuration that
augments CPU detection from the host operating system.
This change modifies the proxy injector to ensure that this environment
is configured from the `Values.proxy.cores` Helm value, the
`config.linkerd.io/proxy-cpu-limit` annotation, and the `--proxy-cpu-limit`
install flag.
As discussed in #5167 & #5169, Kubernetes CPU limits are not necessarily
discoverable from within the pod. This means that the control plane
processes may allocate far more threads than can actually be used by the
process given its process limits.
This change removes the default CPU limits for all control plane
components. CPU limits may still be set via Helm configuration.
Now that the proxy can use more than one core, this behavior should be
enabled by default, even in HA mode.
This change modifies the default HA helm values to unset the cpu limit
for proxy containers.
After the 2.9 multicluster refactoring, `linkerd mc install`'s only
workload installed is the nginx gateway, whose docker image is
configured through the flags `--gateway-nginx-image` and
`--gateway-nginx-image-version`. Thus there's no longer need of the
`--registry` flag, which is used OTOH by `linkerd mc link` which deploys the service mirror.
Currently, For legacy upgrades we are fetching even external certs and
using it for upgrades which contradicts the condition at
https://github.com/linkerd/linkerd2/blob/master/cli/cmd/options.go#L550
used with install and thus causing errors.
Instead we don't retrieve them with upgrades and hence they don't get
stored into the config and secrets which seems correct as we do not want
to store certs in the config and use them with upgrades when they are
created externally.
This touches only the upgrade path i.e `fetchIssuers` and would not
effect the retrievel of external certs for checks, etc.
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
* charts: Do not store .component in linkerd-config
This removes the `.component` fields from `Values.go` and also prevents them from being emitted into `linkerd-config` by attaching them into a temporary variable during injection.
This also simplies inbound and outbound Skip ports helm logic and adds quotes to them.
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
* cli: add `--ingress` flag to inject cmd
This PR adds a new inject flag called `--ingress` which when enabled
adds a new annotation i.e `linkerd.io/inject: ingress`.
This annotation is not applied in the `--manual` case and the env
variable is directly set.
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
`linkerd mc link` wasn't properly setting the `gatewayAddresses` field
when such address had a `Hostname` field instead of `Ip`, like is the
case in EKS services of type LoadBalancer.
* Use errors.Is instead of checking underlying err messages
Fixes#5132
This PR replaces the usage of `strings.hasSuffix` with `errors.Is`
wherever error messages are being checked. So, that the code is not
effected by changes in the underlying message. Also adds a string
const for http2 response body closed error
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
Fixes#5121
* cli: skip emitting warnings in Profile
Whenever the tapDuration gets completed, there is a warning occured
which we do not emit. This looks like it has been changed in the latest
versions of the dependency.
* Use context.withDeadline instead of client.timeout
The usage of `client.Timeout` is not working correctly causing `W1022
17:20:12.372780 19049 transport.go:260] Unable to cancel request for
promhttp.RoundTripperFunc` to be emitted by the Kubernetes Client.
This is fixed by using context.WithDeadline and passing that into the
http Request.
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
Fixes#5118
This PR adds a new supported value for the `linkerd.io/inject` annotation. In addition to `enabled` and `disabled`, this annotation may now be set to `ingress`. This functions identically to `enabled` but it also causes the `LINKERD2_PROXY_INGRESS_MODE="true"` environment variable to be set on the proxy. This causes the proxy to operate in ingress mode as described in #5118
With this set, ingresses are able to properly load service profiles based on the l5d-dst-override header.
Signed-off-by: Alex Leong <alex@buoyant.io>
Followup to #5100
We had both `controllerImageVersion` and `global.controllerImageVersion`
configs, but only the latter was taken into account in the chart
templates, so this change removes all of its references.
There is no longer a proxy config `DESTINATION_GET_NETWORKS`. Instead of
reflecting this implementation in our values.yaml, this changes this
variable to the more general `clusterNetworks` to emphasize its
similarity to `clusterDomain` for the purposes of discovery.
The proxy no longer honors DESTINATION_GET variables, as profile lookups
inform when endpoint resolution is performed. Also, there is no longer
a router capacity limit.
As described in #5105, it's not currently possible to set the proxy log
level to `off`. The proxy injector's template does not quote the log
level value, and so the `off` value is handled as `false`. Thanks, YAML.
This change updates the proxy template to use helm's `quote` function
throughout, replacing manually quoted values and fixing the quoting for
the log level value.
We also remove the default logFormat value, as the default is specified
in values.yaml.
Currently the tracing deployments do not start on clusters where
restricted PodSecurityPolicies are enforced.
This PR adds the subchart's ServiceAccounts to the `linkerd-psp`
RoleBinding, thereby allowing the deployments to be satisfied.
Signed-off-by: Simon Weald <glitchcrab-github@simonweald.com>
It appears that Amazon can use the `100.64.0.0/10` network, which is
technically private, for a cluster's Pod network.
Wikipedia describes the network as:
> Shared address space for communications between a service provider
> and its subscribers when using a carrier-grade NAT.
In order to avoid requiring additional configuration on EKS clusters, we
should permit discovery for this network by default.
The proxy has a default, hardcoded set of ports on which it doesn't do
protocol detection (25, 587, 3306 -- all of which are server-first
protocols). In a recent change, this default set was removed from
the outbound proxy, since there was no way to configure it to anything
other than the default set. I had thought that there was a default set
applied to proxy-init, but this appears to not be the case.
This change adds these ports to the default Helm values to restore the
prior behavior.
I have also elected to include 443 in this set, as it is generally our
recommendation to avoid proxying HTTPS traffic, since the proxy provides
very little value on these connections today.
Additionally, the memcached port 11211 is skipped by default, as clients
do not issue any sort of preamble that is immediately detectable.
These defaults may change in the future, but seem like good choices for
the 2.9 release.
The TestUpgradeOverwriteRemoveAddonKeys was not actually verifying that the fields which should be removed were actually removed. Thus it failed to catch an error with the test itself where the `addon-overwrite` flag was spelled incorrectly and not properly registered.
We update the test to verify that the field is removed and fix the test by correcting the spelling of the flag and properly registering it.
Signed-off-by: Alex Leong <alex@buoyant.io>
The `--addon-config` flag allows users to supply a yaml config file which will override the Values used to install or upgrade Linkerd. While this is useful for supplying config for addons, it can be used to configure any part of the Values struct. Thus, we rename the flag to `--config`.
Signed-off-by: Alex Leong <alex@buoyant.io>
* Handle grafana add-on config repair
Fixes#5014
In Grafana Add-On, Default fields i.e `grafana.image.name`, `grafana.name`
have been removed from `linkerd-config-addons` after `2.8.1`. Only
overriden values are stored in `linkerd-config-addons` as of now.
Hence, `grafana.image.name` has to be removed from
`linkerd-config-addons` unless they are overriden so that updates
to it can take place especially the move from `gcr` to `ghcr`.
This also removes `grafana.name` field if they are set to default, as
its removed.
This problem will not occur again even if we update default values, as
default values are not stored in `linekrd-config-addons` anymore for all
add-ons.
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
This is a major refactor of the install/upgrade code which removes the config protobuf and replaces it with a config overrides secret which stores overrides to the values struct. Further background on this change can be found here: https://github.com/linkerd/linkerd2/discussions/4966
Note: as-is this PR breaks injection. There is work to move injection onto a Values-based config which must land before this can be merged.
A summary of the high level changes:
* the install, global, and proxy fields of linkerd-config ConfigMap are no longer populated
* the CLI install flow now follows these simple steps:
* load default Values from the chart
* update the Values based on the provided CLI flags
* render the chart with these values
* also render a Secret/linkerd-config-overrides which describes the values which have been changed from their defaults
* the CLI upgrade flow now follows these simple stesp:
* load the default Values from the chart
* if Secret/linkerd-config-overrides exists, apply the overrides onto the values
* otherwise load the legacy ConfigMap/linkerd-config and use it to updates the values
* further update the values based on the provided CLI flags
* render the chart and the Secret/linkerd-config-overrides as above
* Helm install and upgrade is unchanged
Signed-off-by: Alex Leong <alex@buoyant.io>
PR https://github.com/linkerd/linkerd2/pull/5027 added `podLabels` and `podAnnotations` to `values.yaml` to allow setting labels and annotations on pods in the Helm template. However, these fields were not added to the `Values` struct in `Values.go`. This means that these fields were not serialized out to the `linkerd-config` or to the `linkerd-config-overrides`. Furthermore, in PR #5005 which moves to using the `Values` struct more authoritatively, the `podLabels` and `podAnnotations` fields would not take effect at all.
Add these fields to the `Values` struct and update all test fixtures accordingly.
Signed-off-by: Alex Leong <alex@buoyant.io>
This adds the `podAnnotations` and `podLabels` values in `values.yml` for adding custom annotations/labels to all the control plane pods.
Closes (#5025)
Signed-off-by: Raphael Taylor-Davies <r.taylordavies@googlemail.com>
This PR Updates the Injection Logic (both CLI and proxy-injector)
to use `Values` struct instead of protobuf Config, part of our move
in removing the protobuf.
This does not touch any of the flags, install related code.
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
Co-authored-by: Alex Leong <alex@buoyant.io>
* Remove dependency of linkerd-config for most control plane components
This PR removes the dependency of `linkerd-config` into control
plane components by making all that information passed through CLI
flags. As most of these components require a couple of flags, passing
them as flags could be more helpful, as updations to the flags trigger a
rollout unlike a configMap update.
This does not update the proxy-injector as it needs a lot more data
and mounting `linkerd-config` is better.
One of the challenges with using the Cobra flag library is that it is difficult to differentiate between a flag that has been set explicitly to its default value and one that has been unset (and thus remains at its default value). In particular, when processing CLI flags for the purposes of install and upgrade, this distinction is important because we want to persist the effects of flags which have been set.
Flag is an interface which describes a command line flag that affects the Helm Values used to render Helm charts. This interface allows us to iterate over flags which have been set and apply their effects to the Values. To see how this library is to be used, see https://github.com/linkerd/linkerd2/pull/5005
Signed-off-by: Alex Leong <alex@buoyant.io>
This branch updates the check functionality to read
the new `linkerd-config.values` which contains the full
Values struct showing the current state of the Linkerd
installation. (being added in #5020 )
This is done by adding a new `FetchCurrentConfiguraiton`
which first tries to get the latest, if not falls back
to the older `linkerd-config` protobuf format.`
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
Fixes#5008
We add a `values` file to the `ConfigMap/linkerd-config` resource. This file holds the full Values which were used to render the chart except that private data such as the identity issuer key are redacted. This file is currently unused but will eventually be used by CLI commands such as `check` and `inject` which need to load Linkerd's configuration (as described in #5009).
This is one step in a larger effort to eventually get rid of the other files in `ConfigMap/linkerd-config`.
Signed-off-by: Alex Leong <alex@buoyant.io>
A conflict between #4911 and #4737 caused unit test to be broken.
#4737 added a new test to `upgrade_test.go` and the changes in
#4911 updated all of these test to ignore differences in the config
overrides secret. Since these two PRs merged in parallel, the new
test was missing this update.
Update the new test to also ignore differences in the config overrides
secret as the other ones do.
Signed-off-by: Alex Leong <alex@buoyant.io>
This PR adds a new secret to the output of `linkerd install` called `linkerd-config-overrides`. This is the first step towards simplifying the configuration of the linkerd install and upgrade flow through the CLI. This secret contains the subset of the values.yaml which have been overridden. In other words, the subset of values which differ from their default values. The idea is that this will give us a simpler way to produce the `linkerd upgrade` output while still persisting options set during install. This will eventually replace the `linkerd-config` configmap entirely.
This PR only adds and populates the new secret. The secret is not yet read or used anywhere. Subsequent PRs will update individual control plane components to accept their configuration through flags and will update the `linkerd upgrade` flow to use this secret instead of the `linkerd-config` configmap.
This secret is only generated by the CLI and is not present or required when installing or upgrading with Helm.
Here are sample contents of the secret, base64 decoded. Note that identity tls context is saved as an override so that it can be persisted across updates. Since these fields contain private key material, this object must be a secret. This secret is only used for upgrades and thus only the CLI needs to be able to read it. We will not create any RBAC bindings to grant service accounts access to this secret.
```
global:
identityTrustAnchorsPEM: |
-----BEGIN CERTIFICATE-----
MIIBhDCCASmgAwIBAgIBATAKBggqhkjOPQQDAjApMScwJQYDVQQDEx5pZGVudGl0
eS5saW5rZXJkLmNsdXN0ZXIubG9jYWwwHhcNMjAwODI1MjMzMTU3WhcNMjEwODI1
MjMzMjE3WjApMScwJQYDVQQDEx5pZGVudGl0eS5saW5rZXJkLmNsdXN0ZXIubG9j
YWwwWTATBgcqhkjOPQIBBggqhkjOPQMBBwNCAAQ0e7IPBlVZ03TL8UVlODllbh8b
2pcM5mbtSGgpX9z0l3n5M70oHn715xu2szh63oBjPl2ZfOA5Bd43cJIksONQo0Iw
QDAOBgNVHQ8BAf8EBAMCAQYwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMC
MA8GA1UdEwEB/wQFMAMBAf8wCgYIKoZIzj0EAwIDSQAwRgIhAI7Sy8P+3TYCJBlK
pIJSZD4lGTUyXPD4Chl/FwWdFfvyAiEA6AgCPbNCx1dOZ8RpjsN2icMRA8vwPtTx
oSfEG/rBb68=
-----END CERTIFICATE-----
heartbeatSchedule: '42 23 * * * '
identity:
issuer:
crtExpiry: "2021-08-25T23:32:17Z"
tls:
crtPEM: |
-----BEGIN CERTIFICATE-----
MIIBhDCCASmgAwIBAgIBATAKBggqhkjOPQQDAjApMScwJQYDVQQDEx5pZGVudGl0
eS5saW5rZXJkLmNsdXN0ZXIubG9jYWwwHhcNMjAwODI1MjMzMTU3WhcNMjEwODI1
MjMzMjE3WjApMScwJQYDVQQDEx5pZGVudGl0eS5saW5rZXJkLmNsdXN0ZXIubG9j
YWwwWTATBgcqhkjOPQIBBggqhkjOPQMBBwNCAAQ0e7IPBlVZ03TL8UVlODllbh8b
2pcM5mbtSGgpX9z0l3n5M70oHn715xu2szh63oBjPl2ZfOA5Bd43cJIksONQo0Iw
QDAOBgNVHQ8BAf8EBAMCAQYwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMC
MA8GA1UdEwEB/wQFMAMBAf8wCgYIKoZIzj0EAwIDSQAwRgIhAI7Sy8P+3TYCJBlK
pIJSZD4lGTUyXPD4Chl/FwWdFfvyAiEA6AgCPbNCx1dOZ8RpjsN2icMRA8vwPtTx
oSfEG/rBb68=
-----END CERTIFICATE-----
keyPEM: |
-----BEGIN EC PRIVATE KEY-----
MHcCAQEEIJaqjoDnqkKSsTqJMGeo3/1VMfJTBsMEuMWYzdJVxIhToAoGCCqGSM49
AwEHoUQDQgAENHuyDwZVWdN0y/FFZTg5ZW4fG9qXDOZm7UhoKV/c9Jd5+TO9KB5+
9ecbtrM4et6AYz5dmXzgOQXeN3CSJLDjUA==
-----END EC PRIVATE KEY-----
```
Signed-off-by: Alex Leong <alex@buoyant.io>
Currently the secrets for the proxy-injector, sp-validator webhooks and tap API service are using the Opaque secret type and linkerd-specific field names. This makes it impossible to use cert-manager (https://github.com/jetstack/cert-manager) to provisions and rotate the secrets for these services. This change converts the secrets defined in the linkerd2 helm charts and the controller use the kubernetes.io/tls format instead. This format is used for secrets containing the generated secrets by cert-manager.
Signed-off-by: Lutz Behnke <lutz.behnke@finleap.com>
Fixes#4191#4993
This bumps Kubernetes client-go to the latest v0.19.2 (We had to switch directly to 1.19 because of this issue). Bumping to v0.19.2 required upgrading to smi-sdk-go v0.4.1. This also depends on linkerd/stern#5
This consists of the following changes:
- Fix ./bin/update-codegen.sh by adding the template path to the gen commands, as it is needed after we moved to GOMOD.
- Bump all k8s related dependencies to v0.19.2
- Generate CRD types, client code using the latest k8s.io/code-generator
- Use context.Context as the first argument, in all code paths that touch the k8s client-go interface
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
## Motivation
Closes#4950
## Solution
Add the `config.linkerd.io/opaque-ports` annotation to either a namespace or pod
spec to set the proxy `LINKERD2_PROXY_INBOUND_PORTS_DISABLE_PROTOCOL_DETECTION`
environment variable.
Currently this environment variable is not used by the proxy, but will be
addressed by #4938.
## Valid values
Ports: `config.linkerd.io/opaque-ports: 4322,3306`
Port ranges: `config.linkerd.io/opaque-ports: 4320-4325`
Mixed ports and port ranges: `config.linkerd.io/opaque-ports: 4320-4325`
If the pod has named ports such as:
```
- name: nginx
image: nginx:latest
ports:
- name: nginx-port
containerPort: 80
protocol: TCP
```
The name can also be used as a value: `config.linkerd.io/opaque-ports:
nginx-port`
Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
* Introduce support for authenticated docker registries using imagePullSecrets
Problem: Private Docker Registries are not supported for the moment as detailed in issue #4413
Solution: Every Service Account of linkerd subcomponents are Attached with imagePullSecrets,
which in turn can then pulls the docker images from authenticated private registries using them.
The imagePullSecret is configured in global.imagePullSecret parameter of values.yaml like
imagePullSecret:
- name: <name-of-private-registry-secret-resource>
Fixes#4413
Signed-off-by: Nilakhya <nilakhya@hotmail.com>
* CNI: Use skip ports configuration in CNI
This PR updates the install and `cmdAdd` workflow (which is called
for each new Pod creation) to retrieve and set the configured Skip
Ports. This also updates the `cmdAdd` workflow to check if the new
pod is a control plane Pod, and adds `443` to OutBoundSkipPort so
that 443 (used with k8s API) is skipped as it was causing errors because
a resolve lookup was happening for them which is not intended.
Currently, This field has to be configured to make CNI work in
GKE clusters as thats where the binaries have to be stored. This
was configurable through Helm, but the same can be allowed through
the CLI too
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
* tracing: Move default values into chart
This branch updates the tracing add-on's values into their own chart's values.yaml
(just like grafana and prometheus). This prevents them from being saved into
`linkerd-config-addons` where only the overridden values are stored. Thus allowing
us to change the defaults.
This also
- Updates the check command to fall back to default values, if there are no
overridden name fields.
- Updates jaeger to `1.19.2`
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
* Push docker images to ghcr.io instead of gcr.io
The `cloud_integration.yml` and `release.yml` workflows were modified to
log into ghcr.io, and remove the `Configure gcloud` step which is no
longer necessary.
Note that besides the changes to cloud_integration.yml and release.yml, there was a change to the upgrade-stable integration test so that we do linkerd upgrade --addon-overwrite to reset the addons settings because in stable-2.8.1 the Grafana image was pegged to gcr.io/linkerd-io/grafana in linkerd-config-addons. This will need to be mentioned in the 2.9 upgrade notes.
Also the egress integration test has a debug container that now is pegged to the edge-20.9.2 tag.
Besides that, the other changes are just a global search and replace (s/gcr.io\/linkerd-io/ghcr.io\/linkerd/).
The proxy performs endpoint discovery for unnamed services, but not
service profiles.
The destination controller and proxy have been updated to support
lookups for unnamed services in linkerd/linkerd2#4727 and
linkerd/linkerd2-proxy#626, respectively.
This change modifies the injection template so that the
`proxy.destinationGetNetworks` configuration enables profile
discovery for all networks on which endpoint discovery is permitted.
Updating only the go 1.15 version, makes the upgrades fail from older versions,
as the identity certs do not have that setting and go 1.15 expects them.
This PR upgrades the cert generation code to have that field,
allowing us to move to go 1.15 in later versions of Linkerd.
Oliver Gould
Alejandro Pedraza
Tarun Pothulapati
Alex Leong
Simon Weald
Raphael Taylor-Davies
Lutz Behnke
Kevin Leimkuhler
Nil
OlivierB
Zahari Dichev
Zhou Hao
Ali Ariff