Commit Graph

32 Commits

Author SHA1 Message Date
OlivierB f599bf9b10
Helm chart - linkerd2-collector : enable jaeger receiver (#4783)
Fixes #4778

Signed-off-by: Olivier Boudet <o.boudet@gmail.com>
2020-09-21 12:17:04 -07:00
Tarun Pothulapati f75b9fe374
tracing: Move default values into addon-chart (#4951)
* tracing: Move default values into chart

This branch updates the tracing add-on's values into their own chart's values.yaml
(just like grafana and prometheus). This prevents them from being saved into
`linkerd-config-addons` where only the overridden values are stored. Thus allowing
us to change the defaults.

This also
-  Updates the check command to fall back to default values, if there are no
overridden name fields.
- Updates jaeger to `1.19.2`

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2020-09-15 15:19:25 -05:00
Alejandro Pedraza ccf027c051
Push docker images to ghcr.io instead of gcr.io (#4953)
* Push docker images to ghcr.io instead of gcr.io

The `cloud_integration.yml` and `release.yml` workflows were modified to
log into ghcr.io, and remove the `Configure gcloud` step which is no
longer necessary.

Note that besides the changes to cloud_integration.yml and release.yml, there was a change to the upgrade-stable integration test so that we do linkerd upgrade --addon-overwrite to reset the addons settings because in stable-2.8.1 the Grafana image was pegged to gcr.io/linkerd-io/grafana in linkerd-config-addons. This will need to be mentioned in the 2.9 upgrade notes.

Also the egress integration test has a debug container that now is pegged to the edge-20.9.2 tag.

Besides that, the other changes are just a global search and replace (s/gcr.io\/linkerd-io/ghcr.io\/linkerd/).
2020-09-10 15:16:24 -05:00
Oliver Gould 7ee638bb0c
inject: Configure the proxy to discover profiles for unnamed services (#4960)
The proxy performs endpoint discovery for unnamed services, but not
service profiles.

The destination controller and proxy have been updated to support
lookups for unnamed services in linkerd/linkerd2#4727 and
linkerd/linkerd2-proxy#626, respectively.

This change modifies the injection template so that the
`proxy.destinationGetNetworks` configuration enables profile
discovery for all networks on which endpoint discovery is permitted.
2020-09-10 12:44:00 -07:00
Tarun Pothulapati c4f8ba270d
Generate Identity certs with alternate domain names (#4920)
Updating only the go 1.15 version, makes the upgrades fail from older versions,
as the identity certs do not have that setting and go 1.15 expects them. 
This PR upgrades the cert generation code to have that field, 
allowing us to move to go 1.15 in later versions of Linkerd.
2020-09-03 22:33:10 +05:30
Zahari Dichev 77c88419b8
Make destination and identity services headless (#4923)
* Make destination and identity svcs headless

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
2020-09-02 14:53:38 -05:00
Tarun Pothulapati c9c5d97405
Remove SMI-Metrics charts and commands (#4843)
Fixes #4790

This PR removes both the SMI-Metrics templates along with the
experimental sub-commands. This also removes pkg `smi-metrics`
as there is no direct use of it without the commands.

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2020-08-24 14:35:33 -07:00
Matei David 7ed904f31d
Enable endpoint slices when upgrading through CLI (#4864)
## What/How
@adleong  pointed out in #4780 that when enabling slices during an upgrade, the new value does not persist in the `linkerd-config` ConfigMap. I took a closer look and it seems that we were never overwriting the values in case they were different.

* To fix this, I added an if block when validating and building the upgrade options -- if the current flag value differs from what we have in the ConfigMap, then change the ConfigMap value.
* When doing so, I made sure to check that if the cluster does not support `EndpointSlices` yet the flag is set to true, we will error out. This is done similarly (copy&paste similarily) to what's in the install part.
* Additionally, I have noticed that the helm ConfigMap template stored the flag value under `enableEndpointSlices` field name. I assume this was not changed in the initial PR to reflect the changes made in the protocol buffer. The API (and thus the CLI) uses the field name `endpointSliceEnabled` instead. I have changed the config template so that helm installations will use the same field, which can then be used in the destination service or other components that may implement slice support in the future.

Signed-off-by: Matei David <matei.david.35@gmail.com>
2020-08-24 14:34:50 -07:00
Matei David f797ab1e65
service topologies: topology-aware service routing (#4780)
[Link to RFC](https://github.com/linkerd/rfc/pull/23)

### What
---
* PR that puts together all past pieces of the puzzle to deliver topology-aware service routing, as specified in the [Kubernetes docs](https://kubernetes.io/docs/concepts/services-networking/service-topology/) but with a much better load balancing algorithm and all the coolness of linkerd :) 
* The first piece of this PR is focused on adding topology metadata: topology preference for services and topology `<k,v>` pairs for endpoints.
* The second piece of this PR puts together the new context format and fetching the source node topology metadata in order to allow for endpoints filtering.
* The final part is doing the filtering -- passing all of the metadata to the listener and on every `Add` filtering endpoints based on the topology preference of the service, topology `<k,v>` pairs of endpoints and topology of the source (again `<k,v>` pairs).

### How
---

* **Collecting metadata**:
   -  Services do not have values for topology keys -- the topological keys defined in a service's spec are only there to dictate locality preference for routing; as such, I decided to store them in an array, they will be taken exactly as they are found in the service spec, this ensures we respect the preference order.

   - For EndpointSlices, we are using a map -- an EndpointSlice has locality information in the form of `<k,v>` pair, where the key is a topological key (similar to what's listed in the service) and the value is the locality information -- e.g `hostname: minikube`. For each address we now have a map of topology values which gets populated when we translate the endpoints to an address set. Because normal Endpoints do not have any topology information, we create each address with an empty map which is subsequently populated ONLY for slices in the `endpointSliceToAddressSet` function.

* **Filtering endpoints**:
  - This was a tricky part and filled me with doubts. I think there are a few ways to do this, but this is how I "envisioned" it. First, the `endpoint_translator.go` should be the one to do the filtering; this means that on subscription, we need to feed all of the relevant metadata to the listener. To do this, I created a new function `AddTopologyFilter` as part of the listener interface.

  - To complement the `AddTopologyFilter` function, I created a new `TopologyFilter` struct in `endpoints_watcher.go`. I then embedded this structure in all listeners that implement the interface. The structure holds the source topology (source node), a boolean to tell if slices are activated in case we need to double check (or write tests for the function) and the service preference. We create the filter on Subscription -- we have access to the k8s client here as well as the service, so it's the best point to collect all of this data together. Addresses all have their own topology added to them so they do not have to be collected by the filter.

  - When we add a new set of addresses, we check to see if slices are enabled -- chances are if slices are enabled, service topology might be too. This lets us skip this step if the latest version is not adopted. Prior to sending an `Add` we filter the endpoints -- if the preference is registered by the filter we strictly enforce it, otherwise nothing changes.

And that's pretty much it. 

Signed-off-by: Matei David <matei.david.35@gmail.com>
2020-08-18 11:11:09 -07:00
Alejandro Pedraza 4876a94ed0
Update proxy-init version to v1.3.6 (#4850)
Supersedes #4846

Bump proxy-init to v1.3.6, containing CNI fixes and support for
multi-arch builds.
#4846 included this in v1.3.5 but proxy.golang.org refused to update the
modified SHA
2020-08-11 11:54:00 -05:00
Tarun Pothulapati 7e5804d1cf
grafana: move default values into values file (#4755)
This PR moves default values into add-on specific values.yaml thus
allowing us to update default values as they would not be present in
linkerd-config-addons cm.

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2020-08-06 13:57:28 -07:00
cpretzer 670caaf8ff
Update to proxy-init v1.3.4 (#4815)
Signed-off-by: Charles Pretzer <charles@buoyant.io>
2020-07-30 15:58:58 -05:00
Tarun Pothulapati 6307868f3d
bump prometheus to the latest v2.19.3 (#4811)
* bump prometheus to the latest v2.19.3

latest prometheus version shows a lot of decrease in the memory usage
and other benefits

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2020-07-30 12:06:59 -05:00
Tarun Pothulapati c68ab23ab2
Add global.prometheusUrl field for byop use-case (#4390)
This pr adds `globa.prometheusUrl` field which will be used to configure publlic-api, hearbeat, grafana, etc (i,e query path) to use a external Prometheus.
2020-07-28 12:26:34 +05:30
Matt Miller fc33b9b9aa
support overriding inbound and outbound connect timeouts. (#4759)
* support overriding inbound and outbound connect timeouts.
* add validation on user provided TCP connect timeouts
* convert valid time values into ms

Signed-off-by: Matt Miller <mamiller@rosettastone.com>
2020-07-27 13:56:21 -07:00
Matei David 1c197b14e7
Change destination context token format (#4771)
Add a new structure on the destination controller side to keep track of contextual information.
The token format has been changed from ns:<namespace> to a JSON format so that more variables can be
encdoed in the token. As part of this PR, a new field 'nodeName' has been added to help with service
topologies.

Fixes #4498

Signed-off-by: Matei David <matei.david.35@gmail.com>
2020-07-27 09:49:48 -07:00
Matei David 8b85716eb8
Introduce install flag for EndpointSlices (#4740)
EndpointSlices have been made opt-in due to their experimental nature. This PR
introduces a new install flag 'enableEndpointSlices' that will allow adopters to
specify in their cli install or helm install step whether they would like to
use endpointslices as a resource in the destination service, instead of the
endpoints k8s resource.

Signed-off-by: Matei David <matei.david.35@gmail.com>
2020-07-15 09:53:04 -07:00
Tarun Pothulapati 2a099cb496
Move Prometheus as an Add-On (#4362)
This moves Prometheus as a add-on, thus making it optional but enabled by default. The also make `linkerd-prometheus` more configurable, and allow it to have its own life-cycle for upgrades, configuration, etc.

This work will be followed by documentation that help users configure existing Prometheus to work with Linkerd.

**Changes Include:**
- moving prometheus manifests into a separate chart at `charts/add-ons/prometheus`, and adding it as a dependency to `linkerd2`
- implement the `addOn` interface to support the same with CLI.
- include configuration in `linkerd-config-addons`

**User Facing Changes:**
The default install experience does not change much but for users who have already configured Prometheus differently, would need to apply the same using the new configuration fields present in chart README
2020-07-09 23:29:03 +05:30
cpretzer d3553c59fd
Add volume and volumeMount for buster-based proxy-init (#4692)
* Add volume and volumeMount for buster-based proxy-init

Signed-off-by: Charles Pretzer <charles@buoyant.io>
2020-07-09 09:55:07 -07:00
Matei David a2bd230cd6
service topologies: add Kubernetes/API EndpointSlice support (#4696)
Based on the [EndpointSlice PR](https://github.com/linkerd/linkerd2/pull/4663), this is just the k8s/api support for endpointslices to shorten the first PR.

* Adds CRD
* Adds functions that check whether the cluster has EndpointSlice access
* Adds discovery & endpointslice informers to api.

Signed-off-by: Matei David <matei.david.35@gmail.com>
2020-07-06 15:28:48 -07:00
Naseem 361d35bb6a
feat: add log format annotation and helm value (#4620)
* feat: add log format annotation and helm value

Json log formatting has been added via https://github.com/linkerd/linkerd2-proxy/pull/500
but wiring the option through as an annotation/helm value is still
necessary.

This PR adds the annotation and helm value to configure log format.

Closes #2491

Signed-off-by: Naseem <naseem@transit.app>
2020-07-02 10:08:52 -05:00
Naseem 733d911677
feat: add option to persist prometheus data (#4578)
Data disappears upon prometheus restarts due to it being all in-memory.

Adding an option to enabled persistence by means of a PVC  would be the right approach. It is commonly seen in a wide array of helm charts.

Fixes #4576

Signed-off-by: Naseem <naseem@transit.app>
2020-06-29 14:26:26 -07:00
Zahari Dichev 7f3d872930
Add destination-get-networks option (#4608)
In #4585 we are observing an issue where a loop is encountered when using nginx ingress. The problem is that the outbound proxy does a dst lookup on the IP address which happens to be the very same address the ingress is listening on.

In order to avoid situations like that this PR introduces a way to modify the set of networks for which the proxy shall do IP based discovery. The change introduces a helm flag `.Values.global.proxy.destinationGetNetworks` that can be used to modify this value. There are two ways a user can affect the this setting: 


- setting the `destinationGetNetworks` field in values during a Helm install, which changes the default on all injected pods
- using an annotation ` config.linkerd.io/proxy-destination-get-networks` for injected workloads to override this value

Note that this setting cannot be tweaked through the `install` or `inject` command

Fix: #4585

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
2020-06-18 20:07:47 +03:00
cpretzer b176fbeb6d
Upgrade Grafana to 7.0.3 (#4600)
* Upgrade Grafana to 7.0.3
* use go netdns to avoid DNS resolution errors on alpine

Signed-off-by: Charles Pretzer <charles@buoyant.io>
2020-06-17 21:35:29 -07:00
Alexander Berger b509742c7d
Fix ClusterRole for web-check (#4599)
As reported in #4259 linkerd check run from linkerd's web cconsole is
broken as the underlying RBAC Role cannot access the apiregistration.k8s.io API Group.

With this commit the RBAC Role is fixed allowing read-only access to the API Group
apiregistration.k8s.io.

Fixes #4259

Signed-off-by: alex.berger@nexiot.ch <alex.berger@nexiot.ch>
2020-06-15 10:21:00 -07:00
Alejandro Pedraza 7a9527bf00
Fix yaml in linkerd-config-addons when providing grafanaUrl (#4581)
Put back space after `grafanaUrl` label in `linkerd-config-addons.yaml`
to avoid breaking the yaml parsing.

```
$ linkerd check
...
linkerd-addons
--------------
‼ 'linkerd-config-addons' config map exists
    could not unmarshal linkerd-config-addons config-map: error
    unmarshaling JSON: while decoding JSON: json: cannot unmarshal
    string into Go struct field Values.global of type linkerd2.Global
```
This was added in #4544 to avoid having the configmap being badly formatted.

So this PR fixes the yaml, but then if we don't set `grafanaUrl` the
configmap format gets messed up, but apparently that's just a cosmetic
problem:

```
apiVersion: v1
data:
  values: "global:\n  grafanaUrl: \ngrafana:\n  enabled: true\n
  image:\n    name:
      gcr.io/linkerd-io/grafana\n  name: linkerd-grafana\n  resources:\n
      cpu:\n      limit:
          240m\n    memory:\n      limit: null\ntracing:\n  enabled:
          false"
          kind: ConfigMap
```
2020-06-09 11:08:32 -07:00
Tarun Pothulapati 4aa3ca7f87
remove grafana.image.version field (#4571)
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2020-06-08 17:15:30 -05:00
Alejandro Pedraza 7d9525e316
Removed trailing spaces from entries in configmaps (#4544)
Fixes #4454

As explained
[here](https://github.com/kubernetes/kubernetes/issues/36222#issuecomment-553966166),
trailing spaces in configmap data makes it to look funky when retrieved
later on. This is currently affecting `linkerd-config-addons` and
`linkerd-gateway-config`:

```
$ k -n linkerd-multicluster get cm linkerd-gateway-config -oyaml
apiVersion: v1
data:
  nginx.conf: "events {\n}\nstream {                                                                                                                                                                                  \n
    \  server {                                                                                                                                                                                \n
    \      listen     4180;                                                                                                                                                 \n
    \      proxy_pass 127.0.0.1:4140;                                                                                                                                 \n
    \  }                                                                                                                                                                                       \n}
    \nhttp {\n  server {\n      listen     4181;\n      location /health {\n        access_log
    off;\n        return 200 \"healthy\\n\";\n      }\n  }\n  server {\n      listen
    \    8888;\n      location /health-local {\n        access_log off;\n        return
    200 \"healthy\\n\";\n      }\n  }    \n}"
kind: ConfigMap
```

AFAIK this is only cosmetic and doesn't affect functionality.
2020-06-04 09:06:37 -05:00
Lutz Behnke 163107b8cb
Extend Helm chart to allow disabling secret resources for self-signed certs. (#4289)
* allow disabling secret resources for self-signed certs. Split cert and ca bundle.

Signed-off-by: Lutz Behnke <lutz.behnke@finleap.com>
2020-06-03 09:26:24 -05:00
Alex Leong acacf2e023
Add --close-wait-timeout inject flag (#4409)
Depends on https://github.com/linkerd/linkerd2-proxy-init/pull/10

Fixes #4276 

We add a `--close-wait-timeout` inject flag which configures the proxy-init container to run with `privileged: true` and to set `nf_conntrack_tcp_timeout_close_wait`. 

Signed-off-by: Alex Leong <alex@buoyant.io>
2020-05-21 14:14:14 -07:00
Tarun Pothulapati 0c53760094
update golden files with new grafana.image field format (#4455) 2020-05-21 23:05:04 +05:30
Tarun Pothulapati bd60c90e5d
Add addon-overwrite flag (#4377)
provide a `addon-overwrite` flag for upgrades to skip `linkerd-config-addons` and use `--addon-overwrite` if passed or defaults
2020-05-21 21:01:41 +05:30