Commit Graph

96 Commits

Author SHA1 Message Date
Tarun Pothulapati d0caaa86c4
Bump k8s client-go to v0.19.2 (#5002)
Fixes #4191 #4993

This bumps Kubernetes client-go to the latest v0.19.2 (We had to switch directly to 1.19 because of this issue). Bumping to v0.19.2 required upgrading to smi-sdk-go v0.4.1. This also depends on linkerd/stern#5

This consists of the following changes:

- Fix ./bin/update-codegen.sh by adding the template path to the gen commands, as it is needed after we moved to GOMOD.
- Bump all k8s related dependencies to v0.19.2
- Generate CRD types, client code using the latest k8s.io/code-generator
- Use context.Context as the first argument, in all code paths that touch the k8s client-go interface

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2020-09-28 12:45:18 -05:00
Tarun Pothulapati c9c5d97405
Remove SMI-Metrics charts and commands (#4843)
Fixes #4790

This PR removes both the SMI-Metrics templates along with the
experimental sub-commands. This also removes pkg `smi-metrics`
as there is no direct use of it without the commands.

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2020-08-24 14:35:33 -07:00
Josh Soref 72aadb540f
Spelling (#4872)
This PR corrects misspellings identified by the [check-spelling action](https://github.com/marketplace/actions/check-spelling).

The misspellings have been reported at aaf440489e (commitcomment-41423663)

The action reports that the changes in this PR would make it happy: 5b82c6c5ca

Note: this PR does not include the action. If you're interested in running a spell check on every PR and push, that can be offered separately.

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
2020-08-12 21:59:50 -07:00
Alex Leong f4000afaf3
Refactor upgrade tests to remove use of golden files (#4860)
The upgrade tests were failing due to hardcoded certificates which had expired.  Additionally, these tests contained large swaths of yaml that made it very difficult to understand the semantics of each test case and even more difficult to maintain.

We greatly improve the readability and maintainability of these tests by using a slightly different approach.  Each test follows this basic structure:

* Render an install manifest
* Initialize a fake k8s client with the install manifest (and sometimes additional manifests)
* Render an upgrade manifest
* Parse the manifests as yaml tree structures
* Perform a structured diff on the yaml tree structured and look for expected and unexpected differences

The install manifests are generated dynamically using the regular install flow.  This means that we no longer need large sections of hardcoded yaml in the tests themselves.  Additionally, we now asses the output by doing a structured diff against the install manifest.  This means that we no longer need golden files with explicit expected output.

All test cases were preserved except for the following:

* Any test cases related to multiphase install (config/control plane) were not replicated.  This flow doesn't follow the same pattern as the tests above because the install and upgrade manifests are not expected to be the same or similar.  I also felt that these tests were lower priority because the multiphase install/upgrade feature does not seem to be very popular and is a potential candidate for deprecation.
* Any tests involving upgrading from a very old config were not replicated.  The code to generate these old style configs is no longer present in the codebase so in order to test this case, we would need to resort to hardcoded install manifests.  These tests also seemed low priority to me because Linkerd versions that used the old config are now over 1 year old so it may no longer be critical that we support upgrading from them.  We generally recommend that users upgrading from an old version of Linkerd do so by upgrading through each major version rather than directly to the latest.

Signed-off-by: Alex Leong <alex@buoyant.io>
2020-08-11 09:22:29 -07:00
Tarun Pothulapati 2a099cb496
Move Prometheus as an Add-On (#4362)
This moves Prometheus as a add-on, thus making it optional but enabled by default. The also make `linkerd-prometheus` more configurable, and allow it to have its own life-cycle for upgrades, configuration, etc.

This work will be followed by documentation that help users configure existing Prometheus to work with Linkerd.

**Changes Include:**
- moving prometheus manifests into a separate chart at `charts/add-ons/prometheus`, and adding it as a dependency to `linkerd2`
- implement the `addOn` interface to support the same with CLI.
- include configuration in `linkerd-config-addons`

**User Facing Changes:**
The default install experience does not change much but for users who have already configured Prometheus differently, would need to apply the same using the new configuration fields present in chart README
2020-07-09 23:29:03 +05:30
cpretzer d3553c59fd
Add volume and volumeMount for buster-based proxy-init (#4692)
* Add volume and volumeMount for buster-based proxy-init

Signed-off-by: Charles Pretzer <charles@buoyant.io>
2020-07-09 09:55:07 -07:00
Desmond Ho 93bf079640
Added custom tolerations to helm chart (#4626)
... for the control plane resources

Signed-off-by: Desmond Ho <desmond.ho@cloverhealth.com>
2020-07-07 17:37:02 -05:00
Naseem 361d35bb6a
feat: add log format annotation and helm value (#4620)
* feat: add log format annotation and helm value

Json log formatting has been added via https://github.com/linkerd/linkerd2-proxy/pull/500
but wiring the option through as an annotation/helm value is still
necessary.

This PR adds the annotation and helm value to configure log format.

Closes #2491

Signed-off-by: Naseem <naseem@transit.app>
2020-07-02 10:08:52 -05:00
Zahari Dichev 7f3d872930
Add destination-get-networks option (#4608)
In #4585 we are observing an issue where a loop is encountered when using nginx ingress. The problem is that the outbound proxy does a dst lookup on the IP address which happens to be the very same address the ingress is listening on.

In order to avoid situations like that this PR introduces a way to modify the set of networks for which the proxy shall do IP based discovery. The change introduces a helm flag `.Values.global.proxy.destinationGetNetworks` that can be used to modify this value. There are two ways a user can affect the this setting: 


- setting the `destinationGetNetworks` field in values during a Helm install, which changes the default on all injected pods
- using an annotation ` config.linkerd.io/proxy-destination-get-networks` for injected workloads to override this value

Note that this setting cannot be tweaked through the `install` or `inject` command

Fix: #4585

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
2020-06-18 20:07:47 +03:00
Lutz Behnke 163107b8cb
Extend Helm chart to allow disabling secret resources for self-signed certs. (#4289)
* allow disabling secret resources for self-signed certs. Split cert and ca bundle.

Signed-off-by: Lutz Behnke <lutz.behnke@finleap.com>
2020-06-03 09:26:24 -05:00
Tarun Pothulapati be664571c1
Separate grafana image tag in template (#4395)
Separates grafana image field into image.name, image.version and also moves controllerImageVersion to global
2020-05-20 22:27:19 +05:30
Tarun Pothulapati 45ccc24a89
Move grafana templates into a separate sub-chart as a add-on (#4320)
* adds grafana manifests as a sub-chart

- moves grafana templates into its own chart
- implement add-on interface Grafana struct
- also add relevant conditions for grafana

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* remove redundant grafana fields in Values

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* update golden files

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* fix values issue

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* remove extra grafanaImage value

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* add add-on upgrade tests

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* fix golden file tests

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* add grafana field to linkerd-config-addons

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* Don't apply nil configuration

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* update golden files

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* make checks relaxed for grafana

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* update test to not test on grafana

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* update TestServiceAccountsMatch to contain extra members

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* replace map[string]interface{} with Grafana for better readability

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* update golden files

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2020-05-11 22:22:14 +05:30
Tarun Pothulapati 60ffd1c2a2
Support Multi-stage install with Add-On's (#4271)
* Support Multi-stage install with Add-Ons
* add upgrade tests for add-ons
* add multi stage upgrade unit tests

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2020-04-23 14:40:58 -05:00
Tarun Pothulapati 2b1cbc6fc1
charts: Using downwardAPI to mount labels to the proxy container (#4199)
* use downward API to mount labels to the proxy container as a volume
* add namespace as a label to the pod
* add a trace inject test
* add downwardAPi for controlplaneTracing
* add controlPlaneTracing condition to volumeMounts
* update add-ons to have workload-ns
* add workload-ns label to control-plane components

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2020-04-22 10:33:51 -05:00
Alex Leong 71d6a00faa
Include SMI metrics as part of Linkerd install (#4109)
Adds the SMI metrics API to the Linkerd install flow.  This installs the SMI metrics controller deployment, the SMI metrics ApiService object, and supporting RBAC, and config resources.

This is the first step toward having Linkerd consume the SMI metrics API in the CLI and web dashboard.

Signed-off-by: Alex Leong <alex@buoyant.io>
2020-03-02 10:11:16 -08:00
Tarun Pothulapati 948dc22a34
Tracing Add-on For Linkerd (#3955)
* Moves Common templates needed to partials

As add-ons re-use the partials helm chart, all the templates needed by multiple charts should be present in partials
This commit also updates the helm tests
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* add tracing add-on helm chart

Tracing sub-chart includes open-census and jaeger components as a sub-chart which can be enabled as needed

* Updated Install path to also install add-ons

This includes new interface for add-ons to implement, with example tracing implementation

* Updates Linkerd install path to also install add-ons

Changes include:
 - Adds an optional Linkerd Values configmap which stores add-on configuration when add-ons are present.
 - Updates Linkerd install path to check for add-ons and render their sub-charts.
 - Adds a install Option called config, which is used to pass confiugration for add-ons.
 - Uses a fork of mergo, to over-write default Values with the Values struct generated from config.

* Updates the upgrade path about add-ons.

Upgrade path now checks for the linkerd-values cm, and overwrites the default values with it, if present.
It then checks the config option, for any further overwrites

* Refactor linkerd-values and re-update tests
also adds relevant nil checks
* Refactor code to fix linting issues
* Fixes an error with linkerd-config global values

Also refactors the linkerd-values cm to work the same with helm

* Fix a nil pointer issue for tests
* Updated Tracing add-on chart meta-data
Also introduced a defaultGetFiles method for add-ons

* Add add-on/charts to gitignore
* refactor gitignore for chart deps
* Moves sub-charts to /charts directly
* Refactor linkerd values cm
* Add comment in linkerd-values
* remove extra controlplanetracing flag
* Support Stages deployment for add-ons along with tests
* linting fix
* update tracing rbac
* Removes the need for add-on Interface
- Uses helm loading capabiltiies to get info about add-ons
- Uses reflection to not have to unnecessarily add checks for each add-on type

* disable tracing flag
* Remove dep on forked mergo
- Re-use merge from helm

* Re-use helm's merge
* Override the chartDir path during tests
* add error check
* Updated the dependency iteration code

Currently, the charts directory, will not have the deps in the repo. So, Code is updated to read the dependencies from requirements.yaml
and use that info to read templates from the relevant add-ons directory.

* Hard Code add-ons name
* Remove struct details for add-ons

- As we don't use fields of a add-on struct, we don't have them to be typed. Instead we can just use the `enabled` flag using reflection
- Users can just use map[string]interface{} as the add-on type.

* update unit tests
* linting fix
* Rename flag to addon-config
* Use Chart loading logic
- This code uses chart loading to read the files and keep in a vfs.
- Once we have those files read we will then use them for generation of sub-charts.

* Go fmt fix
* Update the linkerd-values cm to use second level field
* Add relevant unit tests for mergeRaw
* linting fix
* Move addon tests to a new file
* Fix golden files
* remove addon install unit test
* Refactor sub-chart load logic
* Add install tracing unit test
* golden file update for tracing install
* Update golden files to reflect another pr changes
* Move addon-config flag to recordFlagSet
* add relevant tracing enabled checks
* linting fix

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2020-02-26 10:15:04 -08:00
Zahari Dichev deefeeec52 Rename no init container second take (#3972)
This is a second attempt on #3956 as it got merged in the wrong branch

Fixes #3930

Signed-off-by: Zahari Dichev zaharidichev@gmail.com
2020-01-24 12:52:55 -08:00
Paul Balogh dabee12b93 Fix issue for debug containers when using custom Docker registry (#3873)
**Subject**
Fixes bug where override of Docker registry was not being applied to debug containers (#3851)

**Problem**
Overrides for Docker registry are not being applied to debug containers and provide no means to correct the image.

**Solution**
This update expands the `data.proxy` configuration section within the Linkerd `ConfigMap` to maintain the overridden image name for debug containers at _install_-time similar to handling of the `proxy` and `proxyInit` images.

This change also enables the further override option of the registry for debug containers at _inject_-time given utilization of the `--registry` CLI option.

**Validation**
Several new unit tests have been created to confirm functionality.  In addition, the following workflows were run through:

### Standard Workflow with Custom Registry
This workflow installs Linkerd control plane based upon a custom registry, then injecting the debug sidecar into a service.

* Start with a k8s instance having no Linkerd installation
* Build all images locally using `bin/docker-build`
* Create custom tags (using same version) for generated images, e.g. `docker tag gcr.io/linkerd-io/debug:git-a4ebecb6 javaducky.com/linkerd-io/debug:git-a4ebecb6`
* Install Linkerd with registry override `bin/linkerd install --registry=javaducky.com/linkerd-io | kubectl apply -f -`
* Once Linkerd has been fully initialized, you should be able to confirm that the `linkerd-config` ConfigMap now contains the debug image name, pull policy, and version within the `data.proxy` section
* Request injection of the debug image into an available container.  I used the Emojivoto voting service as described in https://linkerd.io/2/tasks/using-the-debug-container/ as `kubectl -n emojivoto get deploy/voting -o yaml | bin/linkerd inject --enable-debug-sidecar - | kubectl apply -f -`
* Once the deployment creates a new pod for the service, inspection should show that the container now includes the "linkerd-debug" container name based on the applicable override image seen previously within the ConfigMap
* Debugging can also be verified by viewing debug container logs as `kubectl -n emojivoto logs deploy/voting linkerd-debug -f`
* Modifying the `config.linkerd.io/enable-debug-sidecar` annotation, setting to “false”, should show that the pod will be recreated no longer running the debug container.

### Overriding the Custom Registry Override at Injection
This builds upon the “Standard Workflow with Custom Registry” by overriding the Docker registry utilized for the debug container at the time of injection.

* “Clean” the Emojivoto voting service by removing any Linkerd annotations from the deployment
* Request injection similar to before, except provide the `--registry` option as in `kubectl -n emojivoto get deploy/voting -o yaml | bin/linkerd inject --enable-debug-sidecar --registry=gcr.io/linkerd-io - | kubectl apply -f -`
* Inspection of the deployment config should now show the override annotation for `config.linkerd.io/debug-image` having the debug container from the new registry.  Viewing the running pod should show that the `linkerd-debug` container was injected and running the correct image.  Of note, the proxy and proxy-init images are still running the “original” override images.
* As before, modifying the `config.linkerd.io/enable-debug-sidecar` annotation setting to “false”, should show that the pod will be recreated no longer running the debug container.

### Standard Workflow with Default Registry
This workflow is the typical workflow which utilizes the standard Linkerd image registry.

* Uninstall the Linkerd control plane using `bin/linkerd install --ignore-cluster | kubectl delete -f -` as described at https://linkerd.io/2/tasks/uninstall/
* Clean the Emojivoto environment using `curl -sL https://run.linkerd.io/emojivoto.yml | kubectl delete -f -` then reinstall using `curl -sL https://run.linkerd.io/emojivoto.yml | kubectl apply -f -`
* Perform standard Linkerd installation as `bin/linkerd install | kubectl apply -f -`
* Once Linkerd has been fully initialized, you should be able to confirm that the `linkerd-config` ConfigMap references the default debug image of `gcr.io/linkerd-io/debug` within the `data.proxy` section
* Request injection of the debug image into an available container as `kubectl -n emojivoto get deploy/voting -o yaml | bin/linkerd inject --enable-debug-sidecar - | kubectl apply -f -`
* Debugging can also be verified by viewing debug container logs as `kubectl -n emojivoto logs deploy/voting linkerd-debug -f`
* Modifying the `config.linkerd.io/enable-debug-sidecar` annotation, setting to “false”, should show that the pod will be recreated no longer running the debug container.

### Overriding the Default Registry at Injection
This workflow builds upon the “Standard Workflow with Default Registry” by overriding the Docker registry utilized for the debug container at the time of injection.

* “Clean” the Emojivoto voting service by removing any Linkerd annotations from the deployment
* Request injection similar to before, except provide the `--registry` option as in `kubectl -n emojivoto get deploy/voting -o yaml | bin/linkerd inject --enable-debug-sidecar --registry=javaducky.com/linkerd-io - | kubectl apply -f -`
* Inspection of the deployment config should now show the override annotation for `config.linkerd.io/debug-image` having the debug container from the new registry.  Viewing the running pod should show that the `linkerd-debug` container was injected and running the correct image.  Of note, the proxy and proxy-init images are still running the “original” override images.
* As before, modifying the `config.linkerd.io/enable-debug-sidecar` annotation setting to “false”, should show that the pod will be recreated no longer running the debug container.

Fixes issue #3851 

Signed-off-by: Paul Balogh javaducky@gmail.com
2020-01-17 10:18:03 -08:00
Alex Leong 93a81dce97
Change default proxy log level to "warn,linkerd=info" (#3908)
Fixes #3901 

Signed-off-by: Alex Leong <alex@buoyant.io>
2020-01-09 14:22:06 -08:00
Tarun Pothulapati 03982d8837 move more values to global (#3892)
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2020-01-09 14:57:43 -05:00
Tarun Pothulapati eac06b973c Move common values to global (#3839)
* move values to global in template

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* update inject and cli

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* update unit tests

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* fix linting issues

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* remote controllerImageVersion from global

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* move identity out of global

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* update var name and comments

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* update bin and helm tests

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* update helm readme

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* fix proxy config

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* fix proxy config indentation

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* more linting issues

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* remove unnecessary lines

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2020-01-06 14:31:41 -08:00
Tarun Pothulapati 576c2bece6 Fix Helm templating bugs, left-over smaller-cases (#3869)
* update flags to smaller
* add tests for the same
* fix control plane trace flag
* add tests for controlplane tracing install

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2019-12-30 13:34:15 -05:00
Paul Balogh 2cd2ecfa30 Enable mixed configuration of skip-[inbound|outbound]-ports (#3766)
* Enable mixed configuration of skip-[inbound|outbound]-ports using port numbers and ranges (#3752)
* included tests for generated output given proxy-ignore configuration options
* renamed "validate" method to "parseAndValidate" given mutation
* updated documentation to denote inclusiveness of ranges
* Updates for expansion of ignored inbound and outbound port ranges to be handled by the proxy-init rather than CLI (#3766)

This change maintains the configured ports and ranges as strings rather than unsigned integers, while still providing validation at the command layer.

* Bump versions for proxy-init to v1.3.0

Signed-off-by: Paul Balogh <javaducky@gmail.com>
2019-12-20 09:32:13 -05:00
Zahari Dichev a98fe03c5e
Consolidate certificates validation logic (#3810)
* Consolidate certificates validation logic

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>

* Add test for upgrading trust anchors when using external cert manager

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>

* Add logic to ensure issuer cert is CA

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>

* Fix golden file

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
2019-12-13 10:01:55 +02:00
Sergio C. Arteaga cee8e3d0ae Add CronJobs and ReplicaSets to dashboard and CLI (#3687)
This PR adds support for CronJobs and ReplicaSets to `linkerd inject`, the web
dashboard and CLI. It adds a new Grafana dashboard for each kind of resource. 

Closes #3614 
Closes #3630 
Closes #3584 
Closes #3585

Signed-off-by: Sergio Castaño Arteaga tegioz@icloud.com
Signed-off-by: Cintia Sanchez Garcia cynthiasg@icloud.com
2019-12-11 10:02:37 -08:00
Zahari Dichev 0313f10baa
Move CNI template to helm (#3581)
* Create helm chart for the CNI plugin

Signed-off-by: zaharidichev <zaharidichev@gmail.com>

* Add helm install tests for the CNI plugin

Signed-off-by: zaharidichev <zaharidichev@gmail.com>

* Add readme for the CNI helm chart

Signed-off-by: zaharidichev <zaharidichev@gmail.com>

* Fix integration tests

Signed-off-by: zaharidichev <zaharidichev@gmail.com>

* Remove old cni-plugin.yaml

Signed-off-by: zaharidichev <zaharidichev@gmail.com>

* Add trace partial template

Signed-off-by: zaharidichev <zaharidichev@gmail.com>

* Address more comments

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
2019-12-10 13:21:28 +02:00
Alejandro Pedraza b4d27f9d82
No need for `processYAML()` in `install` (#3784)
* No need for `processYAML()` in `install`

Since `install` uses helm to do its proxy injection, there's no need to
call `processYAML`. This also fixes an issue discovered in #3687 where
we started supporting injection of cronjobs, and even though `linkerd`'s
namespace is flagged to skip automatic injection it was being injected.

This replaces #3773 as it's a much more simpler approach.
2019-12-09 09:32:14 -05:00
Alejandro Pedraza 4b6254b52e
Replaced `uuid` with `uid` from linkerd-config resource (#3694)
* Replaced `uuid` with `uid` from linkerd-config resource

Fixes #3621

Removed the old `uuid` for identifying linkerd installations, and
replaced it with the `uid` property from the `linkerd-config` ConfigMap.

I tested that this `uid` remains the same by updating the config and
also upgrading linkerd, using both the CLI and Helm.

Note that this required granting `linkerd-web` RBAC access to the
`linkerd-config` Config.

I also added an integration test to verify the stability of the uid.
2019-11-13 13:56:01 -05:00
Eugene Glotov 2941ddb7f5 Support Dashboard replicas (#2899) (#3633)
This PR makes possible to increase the amount of web dashboard replicas.

Follows up #2899

Signed-off-by: Eugene Glotov <kivagant@gmail.com>
2019-11-12 11:00:23 -08:00
Zahari Dichev 0017f9a60a Cert manager support (#3600)
* Add support for --identity-issuer-mode flag to install cmd
* Change flag to be a bool
* Read correct data form identity when external issuer is used
* Add ability for identity service to dynamically reload certs
* Fix failing tests
* Minor refactor
* Load trust anchors from identity issuer secret
* Make identity service actually watch for issuer certs updates
* Add some testing around cmd line identity options validation
* Add tests ensuring that identity service loads issuer
* Take into account external-issuer flag during upgrade + tests
* Fix failing upgrade test
* Address initial review feedback
* Address further review feedback on cli and helm
* Do not persist --identity-external-issuer
* Some improvements to identitiy service
* Bring back persistane of external issuer flag
* Address more feedback
* Update dockerfiles shas
* Publishing k8s events on issuer certs rotation
* Ensure --ignore-cluster+external issuer is not supported
* Update go-deps shas
* Transition to identity issuer scheme based configuration
* Use k8s consts for secret file names

Signed-off-by: zaharidichev <zaharidichev@gmail.com>
2019-10-24 13:15:14 -07:00
Alejandro Pedraza c5d68ecb16
Add missing nodeSelector in Destination deployment (#3527)
Fixes #3526
2019-10-04 12:47:55 -05:00
Alex Leong 4799baa8e2
Revert "Trace Control Plane components using OC (#3461)" (#3484)
This reverts commit edd3b1f6d4.

This is a temporary revert of #3461 while we sort out some details of how this should configured and how it should interact with configuring a trace collector on the Linkerd proxy.  We will reintroduce this change once the config plan is straightened out.

Signed-off-by: Alex Leong <alex@buoyant.io>
2019-09-26 11:56:44 -07:00
Tarun Pothulapati edd3b1f6d4 Trace Control Plane components using OC (#3461)
* add exporter config for all components

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* add cmd flags wrt tracing

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* add ochttp tracing to web server

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* add flags to the tap deployment

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* add trace flags to install and upgrade command

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* add linkerd prefix to svc names

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* add ochttp trasport to API Internal Client

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* fix goimport linting errors

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* add ochttp handler to tap http server

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* review and fix tests

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* update test values

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* use common template

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* update tests

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* use Initialize

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* fix sample flag

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* add verbose info reg flags

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2019-09-26 08:11:48 -07:00
Eugene Glotov 616131467c Allow to disable Namespace installation with Helm (#3412) (#3413)
If the namespace is controlled by an external tool or can't be installed
with Helm, disable its installation
Fixes #3412

Signed-off-by: Eugene Glotov <kivagant@gmail.com>
2019-09-17 12:25:35 -05:00
Ivan Sim 183e42e4cd
Merge the CLI 'installValues' type with Helm 'Values' type (#3291)
* Rename template-values.go
* Define new constructor of charts.Values type
* Move all Helm values related code to the pkg/charts package
* Bump dependency
* Use '/' in filepath to remain compatible with VFS requirement
* Add unit test to verify Helm YAML output
* Alejandro's feedback
* Add unit test for Helm YAML validation (HA)

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-08-20 19:26:38 -07:00
Ivan Sim 4d01e3720e
Update install and upgrade code to use the new helm charts (#3229)
* Delete symlink to old Helm chart
* Update 'install' code to use common Helm template structs
* Remove obsolete TLS assets functions.

These are now handle by Helm functions inside the templates

* Read defaults from values.yaml and values-ha.yaml
* Ensure that webhooks TLS assets are retained during upgrade
* Fix a few bugs in the Helm templates (see bullet points):
* Merge the way the 'install' ha and non-ha options are handled into one function
* Honor the 'NoInitContainer' option in the components templates
* Control plane mTLS will not be disabled if identity context in the
config map is empty. The data plane mTLS will still be automatically disabled
if the context is nil.
* Resolve test failures from rebase with master
* Fix linter issues
* Set service account mount path read-only field
* Add TLS variables of the webhooks and tap to values.yaml

During upgrade, these secrets are preserved to ensure they remain synced
wih the CA bundle in the webhook configurations. These Helm variables are used
to override the defaults in the templates.

* Remove obsolete 'chart' folder
* Fix bugs in templates
* Handle missing webhooks and tap TLS assets during upgrade

When upgrading from an older version that don't have these secrets, fallback to let Helm
create them by creating an empty charts.TLS struct.

* Revert the selector labels of webhooks to be compatible with that in 2.4

In 2.4, the proxy injector and profile validator webhooks already have their selector labels defined.
Since these attributes are immutable, the recent change to these selectors introduced by the Helm chart work will cause upgrade to fail.

* Alejandro's feedback
* Siggy's feedback
* Removed redundant unexported custom types

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-08-13 14:16:24 -07:00
Andrew Seigner 0ff39ddf8d
Introduce tap-admin ClusterRole, web privs flag (#3203)
The web dashboard will be migrating to the new Tap APIService, which
requires RBAC privileges to access.

Introduce a new ClusterRole, `linkerd-linkerd-tap-admin`, which gives
cluster-wide tap privileges. Also introduce a new ClusterRoleBinding,
`linkerd-linkerd-web-admin` which binds the `linkerd-web` service
account to the new tap ClusterRole. This ClusterRoleBinding is enabled
by default, but may be disabled via a new `linkerd install` flag
`--restrict-dashboard-privileges`.

Fixes #3177

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-08-08 10:28:35 -07:00
Andrew Seigner a59c1dd32d
Introduce tap APIService, update `linkerd tap` (#3167)
The Tap Service enabled tapping of any meshed pod, regardless of user
privilege.

This change introduces a new Tap APIService. Kubernetes provides
authentication and authorization of Tap requests, and then forwards
requests to a new Tap APIServer, which implements a Kubernetes
aggregated APIServer. The Tap APIServer authenticates the client TLS
from Kubernetes, and authorizes the user via a SubjectAccessReview.

This change also modifies the `linkerd tap` command to make requests
against the new APIService.

The Tap APIService implements these Kubernetes-style endpoints:
POST /apis/tap.linkerd.io/v1alpha1/watch/namespaces/:ns/tap
POST /apis/tap.linkerd.io/v1alpha1/watch/namespaces/:ns/:res/:name/tap
GET  /apis
GET  /apis/tap.linkerd.io
GET  /apis/tap.linkerd.io/v1alpha1
GET  /healthz
GET  /healthz/log
GET  /healthz/ping
GET  /metrics
GET  /openapi/v2
GET  /version

Users authorize to the new `tap.linkerd.io/v1alpha1` via RBAC. Only the
`watch` verb is supported. Access is also available via subresources
such as `deployments/tap` and `pods/tap`.

This change introduces the following resources into the default Linkerd
install:
- Global
  - APIService/v1alpha1.tap.linkerd.io
  - ClusterRoleBinding/linkerd-linkerd-tap-auth-delegator
- `linkerd` namespace:
  - Secret/linkerd-tap-tls
- `kube-system` namespace:
  - RoleBinding/linkerd-linkerd-tap-auth-reader

Tasks not covered by this PR:
- `linkerd top`
- `linkerd dashboard`
- `linkerd profile --tap`
- removal of the unauthenticated tap controller

Fixes #2725, #3162, #3172

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-08-01 14:02:45 -07:00
Andrew Seigner 64ed8e4a74
Introduce Cluster Heartbeat cronjob (#3056)
`linkerd check`, the web dashboard, and Grafana all perform version
checks to validate Linkerd is up to date. It's common for users to
seldom execute these codepaths. This makes it difficult to identify what
versions of Linkerd are currently in use and what environments it is
being run in, which helps prioritize testing and backports.

Introduce a `heartbeat` CronJob to the default Linkerd install. The
cronjob executes every 24 hours, starting from 5 minutes after
`linkerd install` is run.

Example check URL:
https://versioncheck.linkerd.io/version.json?
  install-time=1562761177&
  k8s-version=v1.15.0&
  meshed-pods=8&
  rps=3&
  source=heartbeat&
  uuid=cc4bb700-3314-426a-9f0f-ec588b9df020&
  version=git-b97ee9f7

Fixes #2961

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-07-23 17:12:30 -07:00
arminbuerkle 010efac24b Allow custom cluster domain in controller components (#2950)
* Allow custom cluster domain in destination watcher

The change relaxes the constrains of an authority requiring a
`svc.cluster.local` suffix to only require `svc` as third part.

A unit test could be added though the destination/server and endpoint
watcher already test this behaviour.

* Update proto to allow setting custom cluster domain

Update golden templates

* Allow setting custom domain in grpc, web server

* Remove cluster domain flags from web srv and public api

* Set defaultClusterDomain in validateAndBuild if none is set

Signed-off-by: Armin Buerkle <armin.buerkle@alfatraining.de>
2019-07-23 08:59:41 -07:00
Ivan Sim 435fe861d0
Label all Linkerd resources (#2971)
Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-06-20 09:44:30 -07:00
Alejandro Pedraza 7fc6c195ad
Set MWC and VWC failure policy to 'fail' in HA mode only (#2943)
Fixes #2927

Also moved `TestInstallSP` after `TestCheckPostInstall` so we're sure
the validating webhook is ready before installing a service profile.

Signed-off-by: Alejandro Pedraza Borrero <alejandro@buoyant.io>
2019-06-14 11:50:59 -05:00
Alejandro Pedraza 8416d326c2
If HA, set the webhooks failure policy to 'Fail' (#2906)
* If HA, set the webhooks failure policy to 'Fail'

I'm adding to the linkerd namespace a new label
`linkerd.io/is-control-plane: true` that is used in the webhook configs'
selector to skip the proxy injector for this namespace. This avoids
running into the timing issues described in #2852.

Fixes #2852

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-06-11 13:11:54 -05:00
Dan 24bbd7c64b Ensure Prometheus log level is lowercase (#2823) (#2870)
Signed-off-by: Daniel Baranowski <daniel.baranowski@infinityworks.com>
2019-06-07 09:57:08 -07:00
Ivan Sim 5a5f8bbfe8
Install MWC and VWC During Installation (#2806)
* Update helm charts to include webhooks config and TLS secret
* Update the webhooks to read the secret cert and key
* Update webhooks to not recreate config on restart
* Ensure upgrade preserve existing secrets
* Revert the change to rename the webhook configs

The renaming change breaks upgrade, where the new webhook configs conflict with
the existing ones. The older resources  aren't deleted during upgrade because
they are dynamically created.

* Make the secret volume read-only
* Remove unnecessary exported getter functions
* Remove obsolete mwc and vwc templates

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-05-20 12:43:50 -07:00
Alejandro Pedraza 53bb7c47f6
Make the auto-injector required and removed proxy-auto-inject flag (#2733)
Make the auto-injector required and removed proxy-auto-inject flag

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-04-24 13:06:51 -05:00
Andrew Seigner b2b4780430
Introduce install stages (#2719)
This change introduces two named parameters for `linkerd install`, split
by privilege:
- `linkerd install config`
  - Namespace
  - ClusterRoles
  - ClusterRoleBindings
  - CustomResourceDefinition
  - ServiceAccounts
- `linkerd install control-plane`
  - ConfigMaps
  - Secrets
  - Deployments
  - Services

Comprehensive `linkerd install` is still supported.

TODO:
- `linkerd check` support
- `linkerd upgrade` support
- integration tests

Part of #2337

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-04-23 14:52:34 -07:00
Ivan Sim 8d13084f94
Split the `linkerd-version` CLI flag into `control-plane-version` and `proxy-version` (#2702)
* The 'linkerd-version' CLI flag is renamed to 'control-plane-version'
* Add version field to proxy config
* Add the control plane version to the global config
* Unit test for init image version
* Use more specific control plane and proxy versions in unit tests

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-04-19 11:35:20 -07:00
Oliver Gould d74ca1bab0
cli: Introduce an upgrade command (#2564)
The `install` command errors when the deploy target contains an existing
Linkerd deployment. The `upgrade` command is introduced to reinstall or
reconfigure the Linkerd control plane.

Upgrade works as follows:

1. The controller config is fetched from the Kubernetes API. The Public
   API is not used, because we need to be able to reinstall the control
   plane when the Public API is not available; and we are not concerned
   about RBAC restrictions preventing the installer from reading the
   config (as we are for inject).

2. The install configuration is read, particularly the flags used during
   the last install/upgrade. If these flags were not set again during the
   upgrade, the previous values are used as if they were passed this time.
   The configuration is updated from the combination of these values,
   including the install configuration itself.

   Note that some flags, including the linkerd-version, are omitted
   since they are stored elsewhere in the configurations and don't make
   sense to track as overrides..

3. The issuer secrets are read from the Kubernetes API so that they can
   be re-used. There is currently no way to reconfigure issuer
   certificates. We will need to create _another_ workflow for
   updating these credentials.

4. The install rendering is invoked with values and config fetched from
   the cluster, synthesized with the new configuration.
2019-04-01 13:27:41 -07:00
Oliver Gould 655632191b
config: Store install parameters with global config (#2577)
When installing Linkerd, a user may override default settings, or may
explicitly configure defaults. Consider install options like `--ha
--controller-replicas=4` -- the `--ha` flag sets a new default value for
the controller-replicas, and then we override it.

When we later upgrade this cluster, how can we know how to configure the
cluster?

We could store EnableHA and ControllerReplicas configurations in the
config, but what if, in a later upgrade, the default value changes? How
can we know whether the user specified an override or just used the
default?

To solve this, we add an `Install` message into a new config.
This message includes (at least) the CLI flags used to invoke
install.

upgrade does not specify defaults for install/proxy-options fields and,
instead, uses the persisted install flags to populate default values,
before applying overrides from the upgrade invocation.

This change breaks the protobuf compatibility by altering the
`installation_uuid` field introduced in 9c442f6885.
Because this change was not yet released (even in an edge release), we
feel that it is safe to break.

Fixes https://github.com/linkerd/linkerd2/issues/2574
2019-03-29 10:04:20 -07:00