In our chart values and (some) integration tests, we're using a deprecated
label for node selection. According to the warning messages we get during
installation, the label has been deprecated since k8s `v1.14`:
```
Warning: spec.template.spec.nodeSelector[beta.kubernetes.io/os]: deprecated since v1.14; use "kubernetes.io/os" instead
Warning: spec.jobTemplate.spec.template.spec.nodeSelector[beta.kubernetes.io/os]: deprecated since v1.14; use "kubernetes.io/os" instead
```
This PR changes all occurrences of `beta.kubernetes.io/node` with
`kubernetes.io/node`.
Fixes#7225
When the `mirror.linkerd.io/remote-gateway-identity` and `mirror.linkerd.io/remote-svc-fq-name` annotations are set on an EndpointSlice object, the destination controller does not return the correct identity hints for that endpoint.
We fix an incorrect assignment to fix this. We also fix some logic that can result in a nil pointer dereference instead of comparing empty strings. We add a test case to exercise these.
Signed-off-by: Alex Leong <alex@buoyant.io>
## background
In order to upgrade `client-go` and other related libraries to `v0.22.0`, we had to address the deprecation of the service's `TopologyKeys` field. This field and it's related feature have been deprecated and superseded by [Topology Aware Hints](https://github.com/kubernetes/enhancements/blob/master/keps/sig-network/2433-topology-aware-hints/README.md).
The goal of topology aware hints is to to provide a simpler way for users to prefer endpoints by basing decisions soely off the node's `topology.kubernetes.io/zone` label. If a node is in `zone-a`, then it should prefer endpoints that _should_ be consumed by clients in `zone-a`.
kube-proxy (and now the destination controller) know that an endpoint _should_ be consumed by clients in certain zones if its `Hints.ForZones` field is set with a zone value that matches that of the client.
For example, the endpoint slice controller may add the following hint to an endpoint:
```
- addresses: ["1.1.1.1"]
zone: "zone-a"
hints:
zone: "zone-b"
```
The above endpoint is an endpoint that is located in `zone-a` but should be consumed by clients in `zone-b`.
## changes
Now that topological preference is not a concept, we can remove it from the `servicePublisher` and `portPublisher` structs. The fields were only there so that it could be populated down to individual addresses.
The `Hints` field is only present on endpoints that belong to an `EndpointSlice`, so use of this field is limited to the `endpointSliceToAddresses` function.
When endpoint slices are translated to an `AddressSet` now, for each address (endpoint) we make sure to copy the `Hints.ForZones` field if it is present. This field is only present if it's set by the endpoint slice controller and it has [several safeguards](https://kubernetes.io/docs/concepts/services-networking/topology-aware-hints/#safeguards).
After `endpointSliceToAddresses` has translated an endpoint slice into an `AddressSet` and updated the endpoint translator's `availableEndpoints`, filtering takes place and is the crux of this change.
For each potential address that we have to consider in `availableEndpoints`, we make sure to only return a set of addresses who's consumption zone (zones in `forZones` field) match that of the node's zone. That way, we only communicate with endpoints that have been labeled by the endpoint slice controller for the current node we're on.
This allows us to remove the ordering/hierarchy of topological region and considering the `*` value.
## testing
I've added a unit test which creates an endpoint translator tied to a node in `west-1a` and asserts that it only handles updates for addresses that should be consumed by clients in `west-1a`.
Closes#6637
Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
Linkerd proxy-init container is currently enforced to run as root.
Removes hardcoding `runAsNonRoot: false` and `runAsUser: 0`. This way
the container inherits the user ID from the proxy-init image instead which
may allow to run as non-root.
Fixes#5505
Signed-off-by: Schlotter, Christian <christian.schlotter@daimler.com>
Fixes#3260
## Summary
Currently, Linkerd uses a service Account token to validate a pod
during the `Certify` request with identity, through which identity
is established on the proxy. This works well and good, as Kubernetes
attaches the `default` service account token of a namespace as a volume
(unless overridden with a specific service account by the user). Catch
here being that this token is aimed at the application to talk to the
kubernetes API and not specifically for Linkerd. This means that there
are [controls outside of Linkerd](https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#use-the-default-service-account-to-access-the-api-server), to manage this service token, which
users might want to use, [causing problems with Linkerd](https://github.com/linkerd/linkerd2/issues/3183)
as Linkerd might expect it to be present.
To have a more granular control over the token, and not rely on the
service token that can be managed externally, [Bound Service Tokens](https://github.com/kubernetes/enhancements/tree/master/keps/sig-auth/1205-bound-service-account-tokens)
can be used to generate tokens that are specifically for Linkerd,
that are bound to a specific pod, along with an expiry.
## Background on Bounded Service Tokens
This feature has been GA’ed in Kubernetes 1.20, and is enabled by default
in most cloud provider distributions. Using this feature, Kubernetes can
be asked to issue specific tokens for linkerd usage (through audience bound
configuration), with a specific expiry time (as the validation happens every
24 hours when establishing identity, we can follow the same), bounded to
a specific pod (meaning verification fails if the pod object isn’t available).
Because of all these bounds, and not being able to use this token for
anything else, This feels like the right thing to rely on to validate
a pod to issue a certificate.
### Pod Identity Name
We still use the same service account name as the pod identity
(used with metrics, etc) as these tokens are all generated from the
same base service account attached to the pod (could be defualt, or
the user overriden one). This can be verified by looking at the `user`
field in the `TokenReview` response.
<details>
<summary>Sample TokenReview response</summary>
Here, The new token was created for the vault audience for a pod which
had a serviceAccount token volume projection and was using the `mine`
serviceAccount in the default namespace.
```json
"kind": "TokenReview",
"apiVersion": "authentication.k8s.io/v1",
"metadata": {
"creationTimestamp": null,
"managedFields": [
{
"manager": "curl",
"operation": "Update",
"apiVersion": "authentication.k8s.io/v1",
"time": "2021-10-19T19:21:40Z",
"fieldsType": "FieldsV1",
"fieldsV1": {"f:spec":{"f:audiences":{},"f:token":{}}}
}
]
},
"spec": {
"token": "....",
"audiences": [
"vault"
]
},
"status": {
"authenticated": true,
"user": {
"username": "system:serviceaccount:default:mine",
"uid": "889a81bd-e31c-4423-b542-98ddca89bfd9",
"groups": [
"system:serviceaccounts",
"system:serviceaccounts:default",
"system:authenticated"
],
"extra": {
"authentication.kubernetes.io/pod-name": [
"nginx"
],
"authentication.kubernetes.io/pod-uid": [
"ebf36f80-40ee-48ee-a75b-96dcc21466a6"
]
}
},
"audiences": [
"vault"
]
}
```
</details>
## Changes
- Update `proxy-injector` and install scripts to include the new
projected Volume and VolumeMount.
- Update the `identity` pod to validate the token with the linkerd
audience key.
- Added `identity.serviceAccountTokenProjection` to disable this
feature.
- Updated err'ing logic with `autoMountServiceAccount: false`
to fail only when this feature is disabled.
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
The `Group` attribute of the`GroupVersionResource` is wrong for the fake clients.
This leads to tests failing as types are not registered and keyed correctly.
Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
Co-authored-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
This adds the policy CRD APIs for `Server` and `ServerAuthorization` CRDs.
The structure of each (in their respective `types.go`) is based off the `policy-crd.yaml` specs for each CRD.
Unlike service profiles, servers and server authorizations use the `oneof` extensively so I encoded that as a struct with a pointer for each possible `oneof`. For example, a server's `PodSelector` is either `MatchExpressions` or `MatchLabels`. Therefore, a `PodSelector` is defined as:
```
type PodSelector struct {
MatchExpressions *MatchExpressions
MatchLabels *MatchLabels
}
```
Closes#6970
Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
* Remove `omitWebhookSideEffects` flag/setting
This was introduced back in #2963 to support k8s with versions before 1.12 that didn't support the `sideEffects` property in webhooks. It's been a while we no longer support 1.12, so we can safely drop this.
Make destination info logs clearer by changing log level of watchers log messages 'Establishing watch', 'Starting watch' and 'Stopping watch' from info to debug (#6917)
Signed-off-by: Bart Peeters <birtpeeters@hotmail.com>
We've previously handled inbound connections on 443 as opaque, meaning
that we don't do any TLS detection.
This prevents the proxy from reporting meaningful metadata on these TLS
connections--especially the connection's SNI value.
This change also simplifies the core control plane's configuration for
skipping outbound connection on 443 to be much simpler (and
documented!).
Fixes#6733
As policy resources provide a grouping, statistics summaries should
also be allowed on these groupings which are useful to the user. Them
being port specific provide a great way to break down these metrics
further.
This PR adds support for policy resources i.e `server` and `serverauthorization`
on the `stat` command.
## Changes
This adds a new path in the `stat_summary.go` file to handle policy
objects. I tried to see if we could re-use some of the other paths
but some of the labels seems to differ and hence a different path
had to be created. We can try to refactor and merge them though.
We support both request and TCP metrics for the `server` resource
while only the former with `serverauthorization` resources
as metrics are generated in this manner.
This also adds these policy objects into the `k8s` package to
make them as known resources.
For both the policy resources, `--from` doesn't work as these
metrics are not exposed from outbound, and there is no way to
query about the client workload from the inbound metrics. `--to`
is supported to get metrics specifically for a destination workload.
(just like on a service)
## Testing
```bash
> curl -sL https://run.linkerd.io/emojivoto.yml | linkerd inject --proxy-log-level debug - | kubectl apply -f -
> kubectl apply -f 897de1a8d5/emojivoto-policy.yml
# Initial values
on ⛵ kind-kind linkerd2 on 🌱 taru [📦📝🤷] via 🐼 v1.16.7 via via ❄️ impure (shell)
➜ ./bin/go-run cli viz stat srv -A -owide ~/work/linkerd2
NAMESPACE NAME UNAUTHORIZED SUCCESS RPS LATENCY_P50 LATENCY_P95 LATENCY_P99 TCP_CONN READ_BYTES/SEC WRITE_BYTES/SEC
emojivoto emoji-grpc 0.0rps 100.00% 1.8rps 1ms 1ms 3ms 1 188.6B/s 2072.9B/s
emojivoto prom 0.0rps - - - - - - - -
emojivoto voting-grpc 0.0rps 80.70% 0.9rps 1ms 2ms 3ms 1 91.4B/s 52.7B/s
emojivoto web-http 0.0rps 90.68% 2.0rps 2ms 10ms 28ms 1 153.7B/s 4509.4B/s
# After changing the `emoji-grpc` authz
on ⛵ kind-kind linkerd2 on 🌱 taru [📦📝🤷] via 🐼 v1.16.7 via via ❄️ impure (shell) took 2s
➜ ./bin/go-run cli viz stat srv -A -owide ~/work/linkerd2
NAMESPACE NAME UNAUTHORIZED SUCCESS RPS LATENCY_P50 LATENCY_P95 LATENCY_P99 TCP_CONN READ_BYTES/SEC WRITE_BYTES/SEC
emojivoto emoji-grpc 0.3rps 100.00% 1.1rps 0ms 0ms 0ms 1 156.5B/s 1282.4B/s
emojivoto prom 0.0rps - - - - - - - -
emojivoto voting-grpc 0.0rps 87.88% 0.6rps 0ms 0ms 0ms 1 53.5B/s 31.5B/s
emojivoto web-http 0.0rps 61.18% 1.4rps 1ms 2ms 2ms 1 110.2B/s 2195.7B/s
# after changing the `web-http` authz
on ⛵ kind-kind linkerd2 on 🌱 taru [📦📝🤷] via 🐼 v1.16.7 via via ❄️ impure (shell)
➜ ./bin/go-run cli viz stat srv -A -owide ~/work/linkerd2
NAMESPACE NAME UNAUTHORIZED SUCCESS RPS LATENCY_P50 LATENCY_P95 LATENCY_P99 TCP_CONN READ_BYTES/SEC WRITE_BYTES/SEC
emojivoto emoji-grpc 0.0rps - - - - - - - -
emojivoto prom 0.0rps - - - - - - - -
emojivoto voting-grpc 0.0rps - - - - - - - -
emojivoto web-http 1.0rps - - - - - - - -
> linkerd viz stat srv/emoji-grpc -n emojivoto -owide
NAME SUCCESS RPS LATENCY_P50 LATENCY_P95 LATENCY_P99 TCP_CONN READ_BYTES/SEC WRITE_BYTES/SEC
emoji-grpc 100.00% 2.0rps 1ms 1ms 1ms 1 199.9B/s 2208.0B/s
> linkerd viz stat srv/web-http -n emojivoto -owide
NAME SUCCESS RPS LATENCY_P50 LATENCY_P95 LATENCY_P99 TCP_CONN READ_BYTES/SEC WRITE_BYTES/SEC
web-http 94.02% 1.9rps 4ms 9ms 10ms 1 152.7B/s 4505.9B/s
> linkerd viz stat srv -n emojivoto -o wide
NAME MESHED SUCCESS RPS LATENCY_P50 LATENCY_P95 LATENCY_P99 TCP_CONN READ_BYTES/SEC WRITE_BYTES/SEC
emoji-grpc - 100.00% 2.0rps 1ms 1ms 1ms 1 201.6B/s 2209.8B/s
prom - - - - - - - - -
voting-grpc - 86.21% 1.0rps 1ms 1ms 1ms 1 98.3B/s 55.9B/s
web-http - 91.67% 2.0rps 3ms 8ms 10ms 1 157.7B/s 4600.3B/s
> linkerd viz stat serverauthorization/web-public -n emojivoto
NAME MESHED SUCCESS RPS LATENCY_P50 LATENCY_P95 LATENCY_P99
web-http - 89.83% 2.0rps 3ms 9ms 10ms
> linkerd viz stat saz -n emojivoto
NAME AUTHORIZATION MESHED SUCCESS RPS LATENCY_P50 LATENCY_P95 LATENCY_P99
emoji-grpc emoji-grpc - 100.00% 2.0rps 1ms 1ms 1ms
prom prom-prometheus - - - - - -
voting-grpc voting-grpc - 89.83% 1.0rps 1ms 1ms 1ms
web-http web-public - 94.96% 2.0rps 1ms 5ms 9ms
> linkerd viz stat saz/web-public -n emojivoto
NAME AUTHORIZATION MESHED SUCCESS RPS LATENCY_P50 LATENCY_P95 LATENCY_P99
web-http web-public - 90.00% 2.0rps 1ms 5ms 9ms
```
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
Updates linkerd2-proxy-init version to v1.4.0
Major change includes removing "redirect-non-loopback-traffic" rule; previously packets with destination != 127.0.0.1 on lo originating from proxy process would be sent to the inbound proxy port (assuming application tries to talk to itself). This is no longer the case.
Signed-off-by: Matei David <matei@buoyant.io>
#6719 changed the proxy injector so that it adds the `config.linkerd.io/opaque-ports` annotation to all pods and services if they or their namespace do not already contain the annotation. The value used is the default list of opaque ports—which is `25,443,587,3306,4444,5432,6379,9300,11211` unless otherwise specified by the user during installation.
Closes#6729
The main issue with this is that if a service exposes a service port `9090` that targets `3306`, the service _should_ have `9090` set as opaque since it targets a default opaque port, but it does not. This change ensures that services with this situation have `9090` set as opaque.
Additionally, services and pods do not need an annotation for with the entire default opaque ports list if they don't expose those ports in the first place. This change will filter out ports from the default list if the service or pod does not expose them.
### tests
I've added some unit tests that demonstrate the change in behavior and explained in the original issue #6729.
Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
The proxy injector now adds the `config.linkerd.io/default-inbound-policy` annotation to all injected pods.
Closes#6720.
If the pod has the annotation before injection then that value is used. If the pod does not have the annotation but the namespace does, then it inherits that. If both the pod and the namespace do not have the annotation, then it defaults to `.Values.policyController.defaultAllowPolicy`.
Upon injecting the sidecar container into the pod, this annotation value is used to set the `LINKERD2_PROXY_INBOUND_DEFAULT_POLICY` environment variable. Additionally, `LINKERD2_PROXY_POLICY_CLUSTER_NETWORKS` is also set to the value of `.Values.clusterNetworks`.
Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
In order to discover how a workload is configured without knowing the global defaults, the `opaque-ports` annotation is now added by the proxy injector to workloads, regardless of the list being the default or user-specified.
Closes#6689
#### core
Because core control plane components do not go through the proxy injector the annotation is added to the `destination`, `identity`, and `proxy-injector` templates.
The `linkerd-destination` and `linkerd-proxy-injector` deployments both now just have the `opaque-ports: "8443"` annotation. The `linkerd-identity` deployment and service doesn't need this annotation since it doesn't expose anything in the default list.
#### non-core
All other resources go through the proxy injector; it decides whether or not services or pods (the two resources that it can add annotations to) should get the default list.
Workloads get the default list of opaque ports added if they and their namespace do not have the annotation already. So this boils down to:
1. If the workload already has the annotation, no patch is created
2. If the namespace has the annotation but the workload does not, a patch is generated
3. If the workload and namespace do not have the annotation, a patch is generated
#### tests
A unit test has been added and I performed the following manual tests:
1. Injected a pod with the annotation: a patch is generated but there is no change to opaque ports
2. Injected a pod with the namespace annotation: a patch is genereted and opaque ports are copied down to the pod
3. Injected a pod with no annotation on it or the namespace: a patch is generated and the default opaque ports are added
4. Created a pod (not injected): a patch is generated (without the proxy) that adds the annotation (this holds true for if the pod having the annotation or the namespace having the annotation)
Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
* injector: cleanup env variables in `_proxy.tpl`
This PR updates the `_proxy.tpl` file to remove the usage of `_l5d_ns`
and `l5d_trustDomain` env variables which can be rendered directly
instead. This also moves the reference variables to the top for
simplicity purposes.
These unused variables will be removed in a future release to
prevent race conditions during upgrades.
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
Variable references are only expanded to previously defined
environment variables as per https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.19/#envvar-v1-core
which means for `LINKERD2_PROXY_POLICY_WORKLOAD` to work correctly, the
`_pod_ns` `_pod_name` should be present before they are used.
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
Fixes#6688
This PR adds the new `LINKERD2_PROXY_POLICY_SVC_ADDR` and
`LINKERD2_PROXY_POLICY_SVC_NAME` env variables which are used to specify
the address and the identity (which is `linkerd-destination`) of the
policy server respectively.
This also adds the new `LINKERD2_PROXY_POLICY_WORKLOAD` in the format
of `$ns:$pod` which is used to specify the identity of the workload itself.
A new `_pod_name` env variable has been added to get the name of the pod
through the Downward API.
These variables are only set if the `proxy.component` is not
`linkerd-identity`.
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
Another attempt at fixing #6511
Even after #6524, we continued experiencing discrepancies on the
linkerd-edges integration test. The problem ended up being the external
prometheus instance not being injected. The injector logs revealed this:
```console
2021-07-29T13:57:10.2497460Z time="2021-07-29T13:54:15Z" level=info msg="caches synced"
2021-07-29T13:57:10.2498191Z time="2021-07-29T13:54:15Z" level=info msg="starting admin server on :9995"
2021-07-29T13:57:10.2498935Z time="2021-07-29T13:54:15Z" level=info msg="listening at :8443"
2021-07-29T13:57:10.2499945Z time="2021-07-29T13:54:18Z" level=info msg="received admission review request 2b7b4970-db40-4bda-895b-bb2e95e98265"
2021-07-29T13:57:10.2511751Z time="2021-07-29T13:54:18Z" level=debug msg="admission request: &AdmissionRequest{UID:2b7b4970-db40-4bda-895b-bb2e95e98265,Kind:/v1, Kind=Service,Resource:{ v1 services},SubResource:,Name:metrics-api,Namespace:linkerd-viz...
```
Usually one expects the webhook server to start first ("listening at
:8443") and then the admin server, but in this case it happened the
other way around. The admin server serves the readiness probe, so k8s
was signaled that the injector was ready before it could listen to
webhook requests, and given the WebhookFailurePolicy is Ignore by
default, sometimes this was causing for the prometheus pod creation
event to get missed, and we see in the log above that it starts by
processing the pods that are created afterwards, which are the viz ones.
In this fix we start first the webhook server, then block on the syncing
of the k8s API, which should give enough time for the webhook to be up,
and finally we start the admin server.
Fixes#6452
We add a `linkerd-identity-trust-roots` ConfigMap which contains the configured trust root bundle. The proxy template partial is modified so that core control plane components load this bundle from the configmap through the downward API.
The identity controller is updated to mount this new configmap as a volume read the trust root bundle at startup.
Similarly, the proxy-injector also mounts this new configmap. For each pod it injects, it reads the trust root bundle file and sets it on the injected pod.
Signed-off-by: Alex Leong <alex@buoyant.io>
Fixes#5589
The core control plane has a dependency on the viz package in order to use the `BuildResource` function. This "backwards" dependency means that the viz source code needs to be included in core docker-builds and is bad for code hygiene.
We move the `BuildResource` function into the viz package. In `cli/cmd/metrics.go` we replace a call to `BuildResource` with a call directly to `CanonicalResourceNameFromFriendlyName`.
Signed-off-by: Alex Leong <alex@buoyant.io>
We were getting sporadic coverage differences on `controller/k8s/test_helper.go` and `pkg/healthcheck/healthcheck_test.go` on pushes unrelated to those files.
For the former, the problem was in tests in `controller/k8s/api_test.go` that compared slices of pods and services by sorting them. The `Sort` interface was implemented through the methods in `test_helper.go`. There is indeterminism in that sorting at the go library level apparently, in that the `Swap` method is not always called, which impacted the coverage report. The fix consists on comparing those slices item by item without needing to sort beforehand.
As for `healthcheck_test.go`, `validateControlPlanePods()` in `healthcheck.go` short-circuits on the first pod having all its containers ready. The unit tests iterate over maps, an iteration we know is not deterministic, so sometimes the short-circuiting avoided to ever cover the `!container.Ready` block, thus affecting the coverage report. This is fixed by adding a new small test that makes sure that block is covered.
The problem is for parent objects that are not supported in Linkerd, we
cannot get any metrics. For example, using a Rollout will not report any
metrics higher than a pod level.
To fix, add validation for ReplicaSet owners; if it's a valid parent,
use parent Kind and Name, otherwise use ReplicaSet.
Tested using CLI/UI
Interim solution for #6429
Signed-off-by: Matei David <matei@buoyant.io>
Fixes#6354
We add Prometheus gauges which track the client-go cache size for each resource type. For example, the following metrics are added to the destination controller:
```
# HELP endpoint_cache_size Number of items in the client-go endpoint cache
# TYPE endpoint_cache_size gauge
endpoint_cache_size 21
# HELP job_cache_size Number of items in the client-go job cache
# TYPE job_cache_size gauge
job_cache_size 0
# HELP namespace_cache_size Number of items in the client-go namespace cache
# TYPE namespace_cache_size gauge
namespace_cache_size 8
# HELP node_cache_size Number of items in the client-go node cache
# TYPE node_cache_size gauge
node_cache_size 1
# HELP pod_cache_size Number of items in the client-go pod cache
# TYPE pod_cache_size gauge
pod_cache_size 23
# HELP replica_set_cache_size Number of items in the client-go replica_set cache
# TYPE replica_set_cache_size gauge
replica_set_cache_size 40
# HELP service_cache_size Number of items in the client-go service cache
# TYPE service_cache_size gauge
service_cache_size 18
# HELP service_profile_cache_size Number of items in the client-go service_profile cache
# TYPE service_profile_cache_size gauge
service_profile_cache_size 4
# HELP traffic_split_cache_size Number of items in the client-go traffic_split cache
# TYPE traffic_split_cache_size gauge
traffic_split_cache_size 0
```
Signed-off-by: Alex Leong <alex@buoyant.io>
* Set `LINKERD2_PROXY_INBOUND_PORTS` during injection
Fixes#6267
The `LINKERD2_PROXY_INBOUND_PORTS` env var will be set during injection,
containing a comma-separated list of the ports in the non-proxy containers in
the pod. For the identity, destination and injector pods, the var is set
manually in their Helm templates.
Since the proxy-injector isn't reinvoked, containers injected by a mutating
webhook after the injector has run won't be detected. As an escape hatch, the
`config.linkerd.io/pod-inbound-ports` annotation has been added to explicit
overrides.
Other changes:
- Removed
`controller/proxy-injector/fake/data/inject-sidecar-container-spec.yaml` which
is no longer used. - Fixed bad indentation in some fixture files under
`controller/proxy-injector/fake/data`.
Default Linkerd skip and opaque port configuration
Missing default ports based on docs
Addressed: Add Redis to default list of Opaque ports #6132
Once merged, the default install values will match the recommendations in Linkerd's TCP ports guide.
Fixes#6132
Signed-off-by: jasonmorgan <jmorgan@f9vs.com>
Co-authored-by: Alejandro Pedraza <alejandro.pedraza@gmail.com>
We emit a Kubernetes event from the identity controller when successfully issuing a leaf certificate. The events include the identity, expiry, and a hash of the certificate.
Signed-off-by: Alex Leong <alex@buoyant.io>
List of changes:
- Include more output in the `simulate` mode (thanks @liuerfire!")
- Log to `stdout` instead of `stderr` (thanks @mo4islona!)
Non user-facing changes:
- Added `dependabot.yml` to receive automated dependencies upgrades PRs (both for go and github actions). As a result, also upgraded a bunch of dependencies.
Fixes#6272
The opaqueports is prone to fail, with `context deadline exceeded`
as there are numerous k8s API requests being performed.
This PR updates the pre-fetching logic to instead use
`controller/k8s` which provides a wrapper around `pkg/k8s` with
caching by using shared informers underneath!
This commit includes the following changes:
- Update `checkMisconfiguredOpaquePortAnnotations` to use
`controllerk8s.KubeAPI` instead of `hc.kubeAPI`
- `kubeAPI.Sync` fn also had to be updated as it fails to check
if the sp and ts sharedinformers are nil, which might be the
case in cases like this where they are not needed.
We had to use `controllerK8s.NewAPI` for the initialization
instead of `controllerk8s.InitializeAPI` to take-in
`hc.kubeAPI` so as to support unit testing, etc as `hc.kubeAPI`
is how we pass the fake resources in unit tests.
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
Closes#6253
### What
---
When we send a profile request with a pod IP, we get back an endpoint as part of the response. This has two advantages: we avoid building a load balancer and we can treat endpoint failure differently (with more of a fail fast approach). At the moment, when we use a pod DNS as the target of the profile lookup, we don't have an endpoint returned in the
response.
Through this change, the behaviour will be consistent. Whenever we look up a pod (either through IP or DNS name) we will get an endpoint back. The change also attempts to simplify some of the logic in GetProfile.
### How
---
We already have a way to build an endpoint and return it back to the client; I sought to re-use most of the code in an effort to also simplify `GetProfile()`. I extracted most of the code that would have been duplicated into a separate method that is responsible for building the address, looking at annotations for opaque ports and for sending the response back.
In addition, to support a pod DNS fqn I've expanded on the `else` branch of the topmost if statement -- if our host is not an IP, we parse the host to get the k8s fqn. If the parsing function returns an instance ID along with the ServiceID, then we know we are dealing directly with a pod -- if we do, we fetch the pod using the core informer and then return an endpoint for it.
### Tests
---
I've tested this mostly with the destination client script. For the tests, I used the following pods:
```
❯ kgp -n emojivoto -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
voting-ff4c54b8d-zbqc4 2/2 Running 0 3m58s 10.42.0.53 k3d-west-server-0 <none> <none>
web-0 2/2 Running 0 3m58s 10.42.0.55 k3d-west-server-0 <none> <none>
vote-bot-7d89964475-tfq7j 2/2 Running 0 3m58s 10.42.0.54 k3d-west-server-0 <none> <none>
emoji-79cc56f589-57tsh 2/2 Running 0 3m58s 10.42.0.52 k3d-west-server-0 <none> <none>
# emoji pod has an opaque port set to 8080.
# web-svc is a headless service and it backs a statefulset (which is why we have web-0).
# without a headless service we can't lookup based on pod DNS.
```
**`Responses before the change`**:
```
# request on IP, this is how things work at the moment. I included this because there shouldn't be
# any diff between the response given here and the response we get with the change.
# note: this corresponds to the emoji pod which has opaque ports set to 8080.
❯ go run controller/script/destination-client/main.go -method getProfile -path 10.42.0.52:8080
INFO[0000] opaque_protocol:true retry_budget:{retry_ratio:0.2 min_retries_per_second:10 ttl:{seconds:10}} endpoint:{addr:{ip:{ipv4:170524724} port:8080} weight:10000 metric_labels:{key:"control_plane_ns" value:"linkerd"} metric_labels:{key:"deployment" value:"emoji"} metric_labels:{key:"namespace" value:"emojivoto"} metric_labels:{key:"pod" value:"emoji-79cc56f589-57tsh"} metric_labels:{key:"pod_template_hash" value:"79cc56f589"} metric_labels:{key:"serviceaccount" value:"emoji"} tls_identity:{dns_like_identity:{name:"emoji.emojivoto.serviceaccount.identity.linkerd.cluster.local"}} protocol_hint:{h2:{} opaque_transport:{inbound_port:4143}}}
INFO[0000]
# request web-0 by IP
# there shouldn't be any diff with the response we get after the change
❯ go run controller/script/destination-client/main.go -method getProfile -path 10.42.0.55:8080
INFO[0000] retry_budget:{retry_ratio:0.2 min_retries_per_second:10 ttl:{seconds:10}} endpoint:{addr:{ip:{ipv4:170524727} port:8080} weight:10000 metric_labels:{key:"control_plane_ns" value:"linkerd"} metric_labels:{key:"namespace" value:"emojivoto"} metric_labels:{key:"pod" value:"web-0"} metric_labels:{key:"serviceaccount" value:"web"} metric_labels:{key:"statefulset" value:"web"} tls_identity:{dns_like_identity:{name:"web.emojivoto.serviceaccount.identity.linkerd.cluster.local"}} protocol_hint:{h2:{}}}
INFO[0000]
# request web-0 by DNS name -- will not work.
❯ go run controller/script/destination-client/main.go -method getProfile -path web-0.web-svc.emojivoto.svc.cluster.loc
al:8080
INFO[0000] fully_qualified_name:"web-0.web-svc.emojivoto.svc.cluster.local" retry_budget:{retry_ratio:0.2 min_retries_per_second:10 ttl:{seconds:10}} dst_overrides:{authority:"web-svc.emojivoto.svc.cluster.local.:8080" weight:10000}
INFO[0000]
INFO[0000] fully_qualified_name:"web-0.web-svc.emojivoto.svc.cluster.local" retry_budget:{retry_ratio:0.2 min_retries_per_second:10 ttl:{seconds:10}} dst_overrides:{authority:"web-svc.emojivoto.svc.cluster.local.:8080" weight:10000}
INFO[0000]
# ^
# |
# --> no endpoint in the response
```
**`Responses after the change`**:
```
# request profile for emoji, we see opaque transport being set on the endpoint.
❯ go run controller/script/destination-client/main.go -method getProfile -path 10.42.0.52:8080
INFO[0000] opaque_protocol:true retry_budget:{retry_ratio:0.2 min_retries_per_second:10 ttl:{seconds:10}} endpoint:{addr:{ip:{ipv4:170524724} port:8080} weight:10000 metric_labels:{key:"control_plane_ns" value:"linkerd"} metric_labels:{key:"deployment" value:"emoji"} metric_labels:{key:"namespace" value:"emojivoto"} metric_labels:{key:"pod" value:"emoji-79cc56f589-57tsh"} metric_labels:{key:"pod_template_hash" value:"79cc56f589"} metric_labels:{key:"serviceaccount" value:"emoji"} tls_identity:{dns_like_identity:{name:"emoji.emojivoto.serviceaccount.identity.linkerd.cluster.local"}} protocol_hint:{h2:{} opaque_transport:{inbound_port:4143}}}
INFO[0000]
# request profile for web-0 with IP.
❯ go run controller/script/destination-client/main.go -method getProfile -path 10.42.0.55:8080
INFO[0000] retry_budget:{retry_ratio:0.2 min_retries_per_second:10 ttl:{seconds:10}} endpoint:{addr:{ip:{ipv4:170524727} port:8080} weight:10000 metric_labels:{key:"control_plane_ns" value:"linkerd"} metric_labels:{key:"namespace" value:"emojivoto"} metric_labels:{key:"pod" value:"web-0"} metric_labels:{key:"serviceaccount" value:"web"} metric_labels:{key:"statefulset" value:"web"} tls_identity:{dns_like_identity:{name:"web.emojivoto.serviceaccount.identity.linkerd.cluster.local"}} protocol_hint:{h2:{}}}
INFO[0000]
# request profile for web-0 with pod DNS, resp contains endpoint.
❯ go run controller/script/destination-client/main.go -method getProfile -path web-0.web-svc.emojivoto.svc.cluster.local:8080
INFO[0000] retry_budget:{retry_ratio:0.2 min_retries_per_second:10 ttl:{seconds:10}} endpoint:{addr:{ip:{ipv4:170524727} port:8080} weight:10000 metric_labels:{key:"control_plane_ns" value:"linkerd"} metric_labels:{key:"namespace" value:"emojivoto"} metric_labels:{key:"pod" value:"web-0"} metric_labels:{key:"serviceaccount" value:"web"} metric_labels:{key:"statefulset" value:"web"} tls_identity:{dns_like_identity:{name:"web.emojivoto.serviceaccount.identity.linkerd.cluster.local"}} protocol_hint:{h2:{}}}
INFO[0000]
```
Signed-off-by: Matei David <matei@buoyant.io>
## Context
Currently, Whenever a `SP` is created with `Spec.Routes` field not being set from [golang types](https://github.com/linkerd/linkerd2/blob/main/controller/gen/apis/serviceprofile/v1alpha2/types.go#L13), k8s API rejects them with the following error
```bash
ServiceProfile.linkerd.io \"backend-svc.linkerd-smi-app.svc.cluster.local\" is invalid: spec.routes: Invalid value: \"null\": spec.routes in body must be of type array: \"null\"
```
This happens because, Golang automatically renders them it as `Routes: Null` whenever it marshaled into json. This is rejected by k8s API server as it expects that field to be an array.
[This is fixed in k8s >= 1.20](https://github.com/kubernetes/kubernetes/pull/95423) as non-nullable nulls are defaulted, and hence this error happens only in `<=1.19`.
## Problem
As `1.19` is a pretty recent version of k8s, and things like [smi-adaptor](https://github.com/linkerd/linkerd-smi/pull) may not want to manage and make sure `Spec.Routes` is not null all the time.
## Fix
This can be easily be fixed by marking `Spec.Routes` as `omitempty` in its json tags which means that the field is omitted whenever it is not set while being marshaled.
This means that the k8s API won't error out, as that field isn't set to anything invalid.
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
This PR corrects misspellings identified by the [check-spelling action](https://github.com/marketplace/actions/check-spelling).
The misspellings have been reported at 0d56327e6f (commitcomment-51603624)
The action reports that the changes in this PR would make it happy: 03a9c310aa
Note: this PR does not include the action. If you're interested in running a spell check on every PR and push, that can be offered separately.
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
This updates the destination to prefer `serviceprofiles.dstOverrides`
over `trafficsplits`. This is useful as it is important for
ServiceProfile to take preference over TrafficSplits when both are
present.
This also makes integration testing the `smi-adaptor` easier.
This also adds unit tests in the `traffic_split_adaptor` to check
for the same.
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
While uncommon, if H2 upgrades are disabled it's possible for an opaque workload
to not have it's hint.OpaqueTransport field set in it's destination profile
response.
This changes the H2 upgrade enabled check to be specific for setting the
hint.Protocol while allowing hint.OpaqueTransport to be set independent of
that value.
Signed-off-by: Kevin Leimkuhler kevin@kleimkuhler.com
Co-authored-by: Oliver Gould <ver@buoyant.io>
Go 1.16.4 includes a fix for a denial-of-service in net/http: golang/go#45710
Go's error file-line formatting changed in 1.16.3, so this change
updates tests to only do suffix matching on these error strings.
Matei David
Alex Leong
Kevin Leimkuhler
Christian Schlotter
Oliver Gould
Alejandro Pedraza
Tarun Pothulapati
Zahari Dichev
Bart Peeters
Stepan Rabotkin
LiuDui
dependabot[bot]
Jason Morgan
Josh Soref
wangchenglong01