Commit Graph

52 Commits

Author SHA1 Message Date
Lutz Behnke de098cd52d
make api service secrets compatible to cert manager (#4737)
Currently the secrets for the proxy-injector, sp-validator webhooks and tap API service are using the Opaque secret type and linkerd-specific field names. This makes it impossible to use cert-manager (https://github.com/jetstack/cert-manager) to provisions and rotate the secrets for these services. This change converts the secrets defined in the linkerd2 helm charts and the controller use the kubernetes.io/tls format instead. This format is used for secrets containing the generated secrets by cert-manager.

Signed-off-by: Lutz Behnke <lutz.behnke@finleap.com>
2020-09-29 09:17:09 -05:00
Tarun Pothulapati d0caaa86c4
Bump k8s client-go to v0.19.2 (#5002)
Fixes #4191 #4993

This bumps Kubernetes client-go to the latest v0.19.2 (We had to switch directly to 1.19 because of this issue). Bumping to v0.19.2 required upgrading to smi-sdk-go v0.4.1. This also depends on linkerd/stern#5

This consists of the following changes:

- Fix ./bin/update-codegen.sh by adding the template path to the gen commands, as it is needed after we moved to GOMOD.
- Bump all k8s related dependencies to v0.19.2
- Generate CRD types, client code using the latest k8s.io/code-generator
- Use context.Context as the first argument, in all code paths that touch the k8s client-go interface

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2020-09-28 12:45:18 -05:00
Tarun Pothulapati c9c5d97405
Remove SMI-Metrics charts and commands (#4843)
Fixes #4790

This PR removes both the SMI-Metrics templates along with the
experimental sub-commands. This also removes pkg `smi-metrics`
as there is no direct use of it without the commands.

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2020-08-24 14:35:33 -07:00
Matei David 7ed904f31d
Enable endpoint slices when upgrading through CLI (#4864)
## What/How
@adleong  pointed out in #4780 that when enabling slices during an upgrade, the new value does not persist in the `linkerd-config` ConfigMap. I took a closer look and it seems that we were never overwriting the values in case they were different.

* To fix this, I added an if block when validating and building the upgrade options -- if the current flag value differs from what we have in the ConfigMap, then change the ConfigMap value.
* When doing so, I made sure to check that if the cluster does not support `EndpointSlices` yet the flag is set to true, we will error out. This is done similarly (copy&paste similarily) to what's in the install part.
* Additionally, I have noticed that the helm ConfigMap template stored the flag value under `enableEndpointSlices` field name. I assume this was not changed in the initial PR to reflect the changes made in the protocol buffer. The API (and thus the CLI) uses the field name `endpointSliceEnabled` instead. I have changed the config template so that helm installations will use the same field, which can then be used in the destination service or other components that may implement slice support in the future.

Signed-off-by: Matei David <matei.david.35@gmail.com>
2020-08-24 14:34:50 -07:00
Alex Leong f4000afaf3
Refactor upgrade tests to remove use of golden files (#4860)
The upgrade tests were failing due to hardcoded certificates which had expired.  Additionally, these tests contained large swaths of yaml that made it very difficult to understand the semantics of each test case and even more difficult to maintain.

We greatly improve the readability and maintainability of these tests by using a slightly different approach.  Each test follows this basic structure:

* Render an install manifest
* Initialize a fake k8s client with the install manifest (and sometimes additional manifests)
* Render an upgrade manifest
* Parse the manifests as yaml tree structures
* Perform a structured diff on the yaml tree structured and look for expected and unexpected differences

The install manifests are generated dynamically using the regular install flow.  This means that we no longer need large sections of hardcoded yaml in the tests themselves.  Additionally, we now asses the output by doing a structured diff against the install manifest.  This means that we no longer need golden files with explicit expected output.

All test cases were preserved except for the following:

* Any test cases related to multiphase install (config/control plane) were not replicated.  This flow doesn't follow the same pattern as the tests above because the install and upgrade manifests are not expected to be the same or similar.  I also felt that these tests were lower priority because the multiphase install/upgrade feature does not seem to be very popular and is a potential candidate for deprecation.
* Any tests involving upgrading from a very old config were not replicated.  The code to generate these old style configs is no longer present in the codebase so in order to test this case, we would need to resort to hardcoded install manifests.  These tests also seemed low priority to me because Linkerd versions that used the old config are now over 1 year old so it may no longer be critical that we support upgrading from them.  We generally recommend that users upgrading from an old version of Linkerd do so by upgrading through each major version rather than directly to the latest.

Signed-off-by: Alex Leong <alex@buoyant.io>
2020-08-11 09:22:29 -07:00
Paul Balogh 62d54838b8
Ensure and update debug image during upgrade (#4823)
Some installations upgrading from versions prior to 2.7.x may have missing debug image name and version. This fix ensures that the default values are in place for this scenario and additionally upgrades the version of debug image with the control plane version.

Signed-off-by: Paul Balogh <javaducky@gmail.com>
2020-08-05 11:39:29 -07:00
Kevin Leimkuhler fe71ef04b0
Remove `--prune` from multi-stage upgrade docs (#4613)
Fixes #4606

This has not worked as far back as stable-2.6.0.

## Solution

The recommended upgrade process is to include `--prune` as part of `kubectl
apply ..`:

```bash
$ linkerd upgrade | kubectl apply --prune -l linkerd.io/control-plane-ns=linkerd -f -
```

This is an issue for multi-stage upgrade because `linkerd upgrade config` does
not include the `linkerd-config` ConfigMap in it's output.

`kubectl apply --prune ..` will then prune this resource because it matches the
label selector *and* is not in the above output.

The issue occurs when `linkerd upgrade control-plane` is run and expects to find
the ConfigMap that was just pruned.

This can be fixed by not suggesting to prune resources as part of the
multi-stage upgrade.

## Considered

Including `templates/config.yaml` in the install output regardless of the stage.
Instead of it being a template only used in `control-plane` stage in
[render](4aa3ca7f87/cli/cmd/install.go (L873-L886)), it could always be rendered.

This just exposes other things that are pruned in the process:

```bash
❯ bin/linkerd upgrade control-plane |kubectl apply --prune -l linkerd.io/control-plane-ns=linkerd -f -
× Failed to build upgrade configuration: secrets "linkerd-identity-issuer" not found
For troubleshooting help, visit: https://linkerd.io/upgrade/#troubleshooting

error: no objects passed to apply
```

Ultimately, resources part of the `control-plane` stage need to remain and that
will not happen if we prune all resources not in the `config` stage output

Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
2020-06-17 10:29:34 -04:00
Lutz Behnke 163107b8cb
Extend Helm chart to allow disabling secret resources for self-signed certs. (#4289)
* allow disabling secret resources for self-signed certs. Split cert and ca bundle.

Signed-off-by: Lutz Behnke <lutz.behnke@finleap.com>
2020-06-03 09:26:24 -05:00
Tarun Pothulapati bd60c90e5d
Add addon-overwrite flag (#4377)
provide a `addon-overwrite` flag for upgrades to skip `linkerd-config-addons` and use `--addon-overwrite` if passed or defaults
2020-05-21 21:01:41 +05:30
Tarun Pothulapati 8e56166774
Refactor AddOn Installation (#4247)
* refactor add-ons install code

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2020-04-16 15:24:55 -05:00
Alejandro Pedraza 7d07504b5b
Upgrade crashes proxy-init when skipping ports (#4258)
Fixes #4257

This was introduced in 2.7.0. When performing an upgrade on an
installation having used `--skip-outbound-ports` or
`--skip-inbound-ports`, the upgrade picks those values from the
ConfigMap, parses them wrongly, and then when proxy-init picks them the
iptables commands fail.

I've also improved one of the upgrade unit tests to include these flags,
and confirmed it failed before this fix.
2020-04-15 07:11:15 -05:00
Alex Leong 71d6a00faa
Include SMI metrics as part of Linkerd install (#4109)
Adds the SMI metrics API to the Linkerd install flow.  This installs the SMI metrics controller deployment, the SMI metrics ApiService object, and supporting RBAC, and config resources.

This is the first step toward having Linkerd consume the SMI metrics API in the CLI and web dashboard.

Signed-off-by: Alex Leong <alex@buoyant.io>
2020-03-02 10:11:16 -08:00
Tarun Pothulapati 948dc22a34
Tracing Add-on For Linkerd (#3955)
* Moves Common templates needed to partials

As add-ons re-use the partials helm chart, all the templates needed by multiple charts should be present in partials
This commit also updates the helm tests
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* add tracing add-on helm chart

Tracing sub-chart includes open-census and jaeger components as a sub-chart which can be enabled as needed

* Updated Install path to also install add-ons

This includes new interface for add-ons to implement, with example tracing implementation

* Updates Linkerd install path to also install add-ons

Changes include:
 - Adds an optional Linkerd Values configmap which stores add-on configuration when add-ons are present.
 - Updates Linkerd install path to check for add-ons and render their sub-charts.
 - Adds a install Option called config, which is used to pass confiugration for add-ons.
 - Uses a fork of mergo, to over-write default Values with the Values struct generated from config.

* Updates the upgrade path about add-ons.

Upgrade path now checks for the linkerd-values cm, and overwrites the default values with it, if present.
It then checks the config option, for any further overwrites

* Refactor linkerd-values and re-update tests
also adds relevant nil checks
* Refactor code to fix linting issues
* Fixes an error with linkerd-config global values

Also refactors the linkerd-values cm to work the same with helm

* Fix a nil pointer issue for tests
* Updated Tracing add-on chart meta-data
Also introduced a defaultGetFiles method for add-ons

* Add add-on/charts to gitignore
* refactor gitignore for chart deps
* Moves sub-charts to /charts directly
* Refactor linkerd values cm
* Add comment in linkerd-values
* remove extra controlplanetracing flag
* Support Stages deployment for add-ons along with tests
* linting fix
* update tracing rbac
* Removes the need for add-on Interface
- Uses helm loading capabiltiies to get info about add-ons
- Uses reflection to not have to unnecessarily add checks for each add-on type

* disable tracing flag
* Remove dep on forked mergo
- Re-use merge from helm

* Re-use helm's merge
* Override the chartDir path during tests
* add error check
* Updated the dependency iteration code

Currently, the charts directory, will not have the deps in the repo. So, Code is updated to read the dependencies from requirements.yaml
and use that info to read templates from the relevant add-ons directory.

* Hard Code add-ons name
* Remove struct details for add-ons

- As we don't use fields of a add-on struct, we don't have them to be typed. Instead we can just use the `enabled` flag using reflection
- Users can just use map[string]interface{} as the add-on type.

* update unit tests
* linting fix
* Rename flag to addon-config
* Use Chart loading logic
- This code uses chart loading to read the files and keep in a vfs.
- Once we have those files read we will then use them for generation of sub-charts.

* Go fmt fix
* Update the linkerd-values cm to use second level field
* Add relevant unit tests for mergeRaw
* linting fix
* Move addon tests to a new file
* Fix golden files
* remove addon install unit test
* Refactor sub-chart load logic
* Add install tracing unit test
* golden file update for tracing install
* Update golden files to reflect another pr changes
* Move addon-config flag to recordFlagSet
* add relevant tracing enabled checks
* linting fix

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2020-02-26 10:15:04 -08:00
Supratik Das 42efc1da01
Improve kubectl apply format by removing misplaced message (#4053)
* Improve kubectl apply format by removing misplaced message

Fixes #2956

Also separate stderr messages with a new line

Signed-off-by: Supratik Das <rick.das08@gmail.com>
2020-02-20 10:36:36 -05:00
Zahari Dichev 3538944d03
Unify trust anchors terminology (#4047)
Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
2020-02-15 10:12:46 +02:00
Mayank Shah c1b683147a
Update identity to make certs more diagnosable (#3990)
Update identity controller to make issuer certificates diagnosable if
cert validity is causing error

    - Add expiry time in identity log message
    - Add current time in identity log message
    - Emit k8s event with appropriate message


Signed-off-by: Mayank Shah <mayankshah1614@gmail.com>
2020-02-13 11:21:41 +02:00
Mayank Shah 60ac0d5527 Add `as-group` CLI flag (#3952)
Add CLI flag --as-group that can impersonate group for k8s operations

Signed-off-by: Mayank Shah mayankshah1614@gmail.com
2020-01-22 16:38:31 +02:00
Paul Balogh dabee12b93 Fix issue for debug containers when using custom Docker registry (#3873)
**Subject**
Fixes bug where override of Docker registry was not being applied to debug containers (#3851)

**Problem**
Overrides for Docker registry are not being applied to debug containers and provide no means to correct the image.

**Solution**
This update expands the `data.proxy` configuration section within the Linkerd `ConfigMap` to maintain the overridden image name for debug containers at _install_-time similar to handling of the `proxy` and `proxyInit` images.

This change also enables the further override option of the registry for debug containers at _inject_-time given utilization of the `--registry` CLI option.

**Validation**
Several new unit tests have been created to confirm functionality.  In addition, the following workflows were run through:

### Standard Workflow with Custom Registry
This workflow installs Linkerd control plane based upon a custom registry, then injecting the debug sidecar into a service.

* Start with a k8s instance having no Linkerd installation
* Build all images locally using `bin/docker-build`
* Create custom tags (using same version) for generated images, e.g. `docker tag gcr.io/linkerd-io/debug:git-a4ebecb6 javaducky.com/linkerd-io/debug:git-a4ebecb6`
* Install Linkerd with registry override `bin/linkerd install --registry=javaducky.com/linkerd-io | kubectl apply -f -`
* Once Linkerd has been fully initialized, you should be able to confirm that the `linkerd-config` ConfigMap now contains the debug image name, pull policy, and version within the `data.proxy` section
* Request injection of the debug image into an available container.  I used the Emojivoto voting service as described in https://linkerd.io/2/tasks/using-the-debug-container/ as `kubectl -n emojivoto get deploy/voting -o yaml | bin/linkerd inject --enable-debug-sidecar - | kubectl apply -f -`
* Once the deployment creates a new pod for the service, inspection should show that the container now includes the "linkerd-debug" container name based on the applicable override image seen previously within the ConfigMap
* Debugging can also be verified by viewing debug container logs as `kubectl -n emojivoto logs deploy/voting linkerd-debug -f`
* Modifying the `config.linkerd.io/enable-debug-sidecar` annotation, setting to “false”, should show that the pod will be recreated no longer running the debug container.

### Overriding the Custom Registry Override at Injection
This builds upon the “Standard Workflow with Custom Registry” by overriding the Docker registry utilized for the debug container at the time of injection.

* “Clean” the Emojivoto voting service by removing any Linkerd annotations from the deployment
* Request injection similar to before, except provide the `--registry` option as in `kubectl -n emojivoto get deploy/voting -o yaml | bin/linkerd inject --enable-debug-sidecar --registry=gcr.io/linkerd-io - | kubectl apply -f -`
* Inspection of the deployment config should now show the override annotation for `config.linkerd.io/debug-image` having the debug container from the new registry.  Viewing the running pod should show that the `linkerd-debug` container was injected and running the correct image.  Of note, the proxy and proxy-init images are still running the “original” override images.
* As before, modifying the `config.linkerd.io/enable-debug-sidecar` annotation setting to “false”, should show that the pod will be recreated no longer running the debug container.

### Standard Workflow with Default Registry
This workflow is the typical workflow which utilizes the standard Linkerd image registry.

* Uninstall the Linkerd control plane using `bin/linkerd install --ignore-cluster | kubectl delete -f -` as described at https://linkerd.io/2/tasks/uninstall/
* Clean the Emojivoto environment using `curl -sL https://run.linkerd.io/emojivoto.yml | kubectl delete -f -` then reinstall using `curl -sL https://run.linkerd.io/emojivoto.yml | kubectl apply -f -`
* Perform standard Linkerd installation as `bin/linkerd install | kubectl apply -f -`
* Once Linkerd has been fully initialized, you should be able to confirm that the `linkerd-config` ConfigMap references the default debug image of `gcr.io/linkerd-io/debug` within the `data.proxy` section
* Request injection of the debug image into an available container as `kubectl -n emojivoto get deploy/voting -o yaml | bin/linkerd inject --enable-debug-sidecar - | kubectl apply -f -`
* Debugging can also be verified by viewing debug container logs as `kubectl -n emojivoto logs deploy/voting linkerd-debug -f`
* Modifying the `config.linkerd.io/enable-debug-sidecar` annotation, setting to “false”, should show that the pod will be recreated no longer running the debug container.

### Overriding the Default Registry at Injection
This workflow builds upon the “Standard Workflow with Default Registry” by overriding the Docker registry utilized for the debug container at the time of injection.

* “Clean” the Emojivoto voting service by removing any Linkerd annotations from the deployment
* Request injection similar to before, except provide the `--registry` option as in `kubectl -n emojivoto get deploy/voting -o yaml | bin/linkerd inject --enable-debug-sidecar --registry=javaducky.com/linkerd-io - | kubectl apply -f -`
* Inspection of the deployment config should now show the override annotation for `config.linkerd.io/debug-image` having the debug container from the new registry.  Viewing the running pod should show that the `linkerd-debug` container was injected and running the correct image.  Of note, the proxy and proxy-init images are still running the “original” override images.
* As before, modifying the `config.linkerd.io/enable-debug-sidecar` annotation setting to “false”, should show that the pod will be recreated no longer running the debug container.

Fixes issue #3851 

Signed-off-by: Paul Balogh javaducky@gmail.com
2020-01-17 10:18:03 -08:00
Tarun Pothulapati eac06b973c Move common values to global (#3839)
* move values to global in template

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* update inject and cli

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* update unit tests

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* fix linting issues

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* remote controllerImageVersion from global

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* move identity out of global

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* update var name and comments

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* update bin and helm tests

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* update helm readme

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* fix proxy config

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* fix proxy config indentation

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* more linting issues

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

* remove unnecessary lines

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2020-01-06 14:31:41 -08:00
Zahari Dichev f88b55e36e Tls certs checks (#3813)
* Added checks for cert correctness
* Add warning checks for approaching expiration
* Add unit tests
* Improve unit tests
* Address comments
* Address more comments
* Prevent upgrade from breaking proxies when issuer cert is overwritten (#3821)
* Address more comments
* Add gate to upgrade cmd that checks that all proxies roots work with the identitiy issuer that we are updating to
* Address comments
* Enable use of upgarde to modify both roots and issuer at the same time

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
2019-12-16 14:49:32 -08:00
Zahari Dichev a98fe03c5e
Consolidate certificates validation logic (#3810)
* Consolidate certificates validation logic

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>

* Add test for upgrading trust anchors when using external cert manager

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>

* Add logic to ensure issuer cert is CA

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>

* Fix golden file

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
2019-12-13 10:01:55 +02:00
Zahari Dichev 0313f10baa
Move CNI template to helm (#3581)
* Create helm chart for the CNI plugin

Signed-off-by: zaharidichev <zaharidichev@gmail.com>

* Add helm install tests for the CNI plugin

Signed-off-by: zaharidichev <zaharidichev@gmail.com>

* Add readme for the CNI helm chart

Signed-off-by: zaharidichev <zaharidichev@gmail.com>

* Fix integration tests

Signed-off-by: zaharidichev <zaharidichev@gmail.com>

* Remove old cni-plugin.yaml

Signed-off-by: zaharidichev <zaharidichev@gmail.com>

* Add trace partial template

Signed-off-by: zaharidichev <zaharidichev@gmail.com>

* Address more comments

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
2019-12-10 13:21:28 +02:00
Zahari Dichev 7cc3815d49
Add issuer file flags to upgrade command (#3771)
* Add identity-issuer-certificate-file and identity-issuer-key-file to upgrade command

Signed-off-by: zaharidichev <zaharidichev@gmail.com>

* Implement logic to use identity-trust-anchors-file flag to update the anchors

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>

* Address remarks

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
2019-12-09 21:18:03 +02:00
Alejandro Pedraza b4d27f9d82
No need for `processYAML()` in `install` (#3784)
* No need for `processYAML()` in `install`

Since `install` uses helm to do its proxy injection, there's no need to
call `processYAML`. This also fixes an issue discovered in #3687 where
we started supporting injection of cronjobs, and even though `linkerd`'s
namespace is flagged to skip automatic injection it was being injected.

This replaces #3773 as it's a much more simpler approach.
2019-12-09 09:32:14 -05:00
Alejandro Pedraza 4b6254b52e
Replaced `uuid` with `uid` from linkerd-config resource (#3694)
* Replaced `uuid` with `uid` from linkerd-config resource

Fixes #3621

Removed the old `uuid` for identifying linkerd installations, and
replaced it with the `uid` property from the `linkerd-config` ConfigMap.

I tested that this `uid` remains the same by updating the config and
also upgrading linkerd, using both the CLI and Helm.

Note that this required granting `linkerd-web` RBAC access to the
`linkerd-config` Config.

I also added an integration test to verify the stability of the uid.
2019-11-13 13:56:01 -05:00
Zahari Dichev 0017f9a60a Cert manager support (#3600)
* Add support for --identity-issuer-mode flag to install cmd
* Change flag to be a bool
* Read correct data form identity when external issuer is used
* Add ability for identity service to dynamically reload certs
* Fix failing tests
* Minor refactor
* Load trust anchors from identity issuer secret
* Make identity service actually watch for issuer certs updates
* Add some testing around cmd line identity options validation
* Add tests ensuring that identity service loads issuer
* Take into account external-issuer flag during upgrade + tests
* Fix failing upgrade test
* Address initial review feedback
* Address further review feedback on cli and helm
* Do not persist --identity-external-issuer
* Some improvements to identitiy service
* Bring back persistane of external issuer flag
* Address more feedback
* Update dockerfiles shas
* Publishing k8s events on issuer certs rotation
* Ensure --ignore-cluster+external issuer is not supported
* Update go-deps shas
* Transition to identity issuer scheme based configuration
* Use k8s consts for secret file names

Signed-off-by: zaharidichev <zaharidichev@gmail.com>
2019-10-24 13:15:14 -07:00
Alejandro Pedraza 6568929028
Add --disable-heartbeat flag for linkerd install|upgrade (#3439)
Fixes #278

Add `linkerd install|upgrade --disable-heartbeat` flag, and have
`linkerd check` check for the heartbeat's SA only if it's enabled.

Also added those flags into the `linkerd upgrade -h` examples.

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-09-25 15:53:36 -05:00
Ivan Sim 183e42e4cd
Merge the CLI 'installValues' type with Helm 'Values' type (#3291)
* Rename template-values.go
* Define new constructor of charts.Values type
* Move all Helm values related code to the pkg/charts package
* Bump dependency
* Use '/' in filepath to remain compatible with VFS requirement
* Add unit test to verify Helm YAML output
* Alejandro's feedback
* Add unit test for Helm YAML validation (HA)

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-08-20 19:26:38 -07:00
Ivan Sim 4d01e3720e
Update install and upgrade code to use the new helm charts (#3229)
* Delete symlink to old Helm chart
* Update 'install' code to use common Helm template structs
* Remove obsolete TLS assets functions.

These are now handle by Helm functions inside the templates

* Read defaults from values.yaml and values-ha.yaml
* Ensure that webhooks TLS assets are retained during upgrade
* Fix a few bugs in the Helm templates (see bullet points):
* Merge the way the 'install' ha and non-ha options are handled into one function
* Honor the 'NoInitContainer' option in the components templates
* Control plane mTLS will not be disabled if identity context in the
config map is empty. The data plane mTLS will still be automatically disabled
if the context is nil.
* Resolve test failures from rebase with master
* Fix linter issues
* Set service account mount path read-only field
* Add TLS variables of the webhooks and tap to values.yaml

During upgrade, these secrets are preserved to ensure they remain synced
wih the CA bundle in the webhook configurations. These Helm variables are used
to override the defaults in the templates.

* Remove obsolete 'chart' folder
* Fix bugs in templates
* Handle missing webhooks and tap TLS assets during upgrade

When upgrading from an older version that don't have these secrets, fallback to let Helm
create them by creating an empty charts.TLS struct.

* Revert the selector labels of webhooks to be compatible with that in 2.4

In 2.4, the proxy injector and profile validator webhooks already have their selector labels defined.
Since these attributes are immutable, the recent change to these selectors introduced by the Helm chart work will cause upgrade to fail.

* Alejandro's feedback
* Siggy's feedback
* Removed redundant unexported custom types

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-08-13 14:16:24 -07:00
Andrew Seigner 0ff39ddf8d
Introduce tap-admin ClusterRole, web privs flag (#3203)
The web dashboard will be migrating to the new Tap APIService, which
requires RBAC privileges to access.

Introduce a new ClusterRole, `linkerd-linkerd-tap-admin`, which gives
cluster-wide tap privileges. Also introduce a new ClusterRoleBinding,
`linkerd-linkerd-web-admin` which binds the `linkerd-web` service
account to the new tap ClusterRole. This ClusterRoleBinding is enabled
by default, but may be disabled via a new `linkerd install` flag
`--restrict-dashboard-privileges`.

Fixes #3177

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-08-08 10:28:35 -07:00
Andrew Seigner a59c1dd32d
Introduce tap APIService, update `linkerd tap` (#3167)
The Tap Service enabled tapping of any meshed pod, regardless of user
privilege.

This change introduces a new Tap APIService. Kubernetes provides
authentication and authorization of Tap requests, and then forwards
requests to a new Tap APIServer, which implements a Kubernetes
aggregated APIServer. The Tap APIServer authenticates the client TLS
from Kubernetes, and authorizes the user via a SubjectAccessReview.

This change also modifies the `linkerd tap` command to make requests
against the new APIService.

The Tap APIService implements these Kubernetes-style endpoints:
POST /apis/tap.linkerd.io/v1alpha1/watch/namespaces/:ns/tap
POST /apis/tap.linkerd.io/v1alpha1/watch/namespaces/:ns/:res/:name/tap
GET  /apis
GET  /apis/tap.linkerd.io
GET  /apis/tap.linkerd.io/v1alpha1
GET  /healthz
GET  /healthz/log
GET  /healthz/ping
GET  /metrics
GET  /openapi/v2
GET  /version

Users authorize to the new `tap.linkerd.io/v1alpha1` via RBAC. Only the
`watch` verb is supported. Access is also available via subresources
such as `deployments/tap` and `pods/tap`.

This change introduces the following resources into the default Linkerd
install:
- Global
  - APIService/v1alpha1.tap.linkerd.io
  - ClusterRoleBinding/linkerd-linkerd-tap-auth-delegator
- `linkerd` namespace:
  - Secret/linkerd-tap-tls
- `kube-system` namespace:
  - RoleBinding/linkerd-linkerd-tap-auth-reader

Tasks not covered by this PR:
- `linkerd top`
- `linkerd dashboard`
- `linkerd profile --tap`
- removal of the unauthenticated tap controller

Fixes #2725, #3162, #3172

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-08-01 14:02:45 -07:00
Andrew Seigner 9a672dd5a9
Introduce `linkerd --as` flag for impersonation (#3173)
Similar to `kubectl --as`, global flag across all linkerd subcommands
which sets a `ImpersonationConfig` in the Kubernetes API config.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-07-31 16:05:33 -07:00
Andrew Seigner 64ed8e4a74
Introduce Cluster Heartbeat cronjob (#3056)
`linkerd check`, the web dashboard, and Grafana all perform version
checks to validate Linkerd is up to date. It's common for users to
seldom execute these codepaths. This makes it difficult to identify what
versions of Linkerd are currently in use and what environments it is
being run in, which helps prioritize testing and backports.

Introduce a `heartbeat` CronJob to the default Linkerd install. The
cronjob executes every 24 hours, starting from 5 minutes after
`linkerd install` is run.

Example check URL:
https://versioncheck.linkerd.io/version.json?
  install-time=1562761177&
  k8s-version=v1.15.0&
  meshed-pods=8&
  rps=3&
  source=heartbeat&
  uuid=cc4bb700-3314-426a-9f0f-ec588b9df020&
  version=git-b97ee9f7

Fixes #2961

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-07-23 17:12:30 -07:00
arminbuerkle 010efac24b Allow custom cluster domain in controller components (#2950)
* Allow custom cluster domain in destination watcher

The change relaxes the constrains of an authority requiring a
`svc.cluster.local` suffix to only require `svc` as third part.

A unit test could be added though the destination/server and endpoint
watcher already test this behaviour.

* Update proto to allow setting custom cluster domain

Update golden templates

* Allow setting custom domain in grpc, web server

* Remove cluster domain flags from web srv and public api

* Set defaultClusterDomain in validateAndBuild if none is set

Signed-off-by: Armin Buerkle <armin.buerkle@alfatraining.de>
2019-07-23 08:59:41 -07:00
Alejandro Pedraza ba9fd70892
`linkerd upgrade config` bombs when installation had a flag (#3097)
When installing using some of the flags that persist in install, e.g
`linkerd install --ha`, and then doing `linkerd upgrade config` a nil
pointer error is thrown.

Fixes #3094

`newCmdUpgradeConfig()` was using passing `flags` as nil because
`linkerd upgrade config` doesn't expose any flags for the subcommand,
but turns out they're still needed down the call stack in
`setFlagsFromInstall` to reuse the flags persisted during install.

I also added a new unit test catching this.

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-07-18 09:09:01 -05:00
Andrew Seigner 94fa653cf3
Fix `linkerd check` missing uuid on version check (#3040)
PR #2603 modified the web process to read the UUID from the
`linkerd-config` ConfigMap rather than from a command line flag. The
`linkerd check` command relied on that command line flag to retrieve the
UUID as part of its version check.

Modify `linkerd check` to correctly retrieve the UUID from
`linkerd-config`. Also refactor `linkerd-config` retrieval and parsing
code to be shared between healthcheck, install, and upgrade.

Relates to #2961

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-07-05 19:39:13 +02:00
Tarun Pothulapati a3ce06bd80 Add sideEffects field to Webhooks (#2963)
Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2019-06-21 11:06:10 -07:00
Alejandro Pedraza 66eb829e5a
Fix HA during upgrade (#2900)
* Fix HA during upgrade

If we have a Linkerd installation with HA, and then we do `linkerd
upgrade` without specifying `--ha`, the replicas will get set back to 1,
yet the resource requests will keep their HA values.

Desired behavior: `linkerd install --ha` adds the `ha` value into the
linkerd-config, so it should be used during upgrade even if `--ha` is
not passed to `linkerd upgrade`.
Note we still can do `linkerd upgrade --ha=false` to disable HA.

This is a prerequesite to address #2852

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-06-06 17:27:27 -05:00
Ivan Sim 5a5f8bbfe8
Install MWC and VWC During Installation (#2806)
* Update helm charts to include webhooks config and TLS secret
* Update the webhooks to read the secret cert and key
* Update webhooks to not recreate config on restart
* Ensure upgrade preserve existing secrets
* Revert the change to rename the webhook configs

The renaming change breaks upgrade, where the new webhook configs conflict with
the existing ones. The older resources  aren't deleted during upgrade because
they are dynamically created.

* Make the secret volume read-only
* Remove unnecessary exported getter functions
* Remove obsolete mwc and vwc templates

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-05-20 12:43:50 -07:00
Andrew Seigner 266e882d79
Define multi-stage commands as subcommands (#2772)
The multi-stage args used by install, upgrade, and check were
implemented as positional arguments to their respective parent commands.
This made the help documentation unclear, and the code ambiguous as to
which flags corresponded to which stage.

Define `config` and `control-plane` stages as subcommands. The help
menus now explicitly state flags supported.

Fixes #2729

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-05-02 12:32:01 +02:00
Andrew Seigner 0cfc8c6f1c
Introduce k8s apiextensions support (#2759)
CustomResourceDefinition parsing and retrieval is not available via
client-go's `kubernetes.Interface`, but rather via a separate
`k8s.io/apiextensions-apiserver` package.

Introduce support for CustomResourceDefintion object parsing and
retrieval. This change facilitates retrieval of CRDs from the k8s API
server, and also provides CRD resources as mock objects.

Also introduce a `NewFakeAPI` constructor, deprecating
`NewFakeClientSets`. Callers need no longer be concerned with discreet
clientsets (for k8s resources vs. CRDs vs. (eventually)
ServiceProfiles), and can instead use the unified `KubernetesAPI`.

Part of #2337, in service to multi-stage check.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-04-28 18:55:22 -07:00
Andrew Seigner 15ffd86cf1
Introduce multi-stage upgrade (#2723)
`linkerd install` supports a 2-stage install process, `linkerd upgrade`
did not.

Add 2-stage support for `linkerd upgrade`. Also exercise multi-stage
functionality during upgrade integration tests.

Part of #2337

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-04-25 14:29:52 -07:00
Andrew Seigner ec540a882e
Consolidate k8s APIs (#2747)
Numerous codepaths have emerged that create k8s configs, k8s clients,
and make k8s api requests.

This branch consolidates k8s client creation and APIs. The primary
change migrates most codepaths to call `k8s.NewAPI` to instantiate a
`KubernetesAPI` struct from `pkg`. `KubernetesAPI` implements the
`kubernetes.Interface` (clientset) interface, and also persists a
`client-go` `rest.Config`.

Specific list of changes:
- removes manual GET requests from `k8s.KubernetesAPI`, in favor of
  clientsets
- replaces most calls to `k8s.GetConfig`+`kubernetes.NewForConfig` with
  a single `k8s.NewAPI`
- introduces a `timeout` param to `k8s.NewAPI`, currently only used by
  healthchecks
- removes `NewClientSet` in `controller/k8s/clientset.go` in favor of
  `k8s.NewAPI`
- removes `httpClient` and `clientset` from `HealthChecker`, use
  `KubernetesAPI` instead

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-04-25 11:31:38 -07:00
Alejandro Pedraza 53bb7c47f6
Make the auto-injector required and removed proxy-auto-inject flag (#2733)
Make the auto-injector required and removed proxy-auto-inject flag

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-04-24 13:06:51 -05:00
Ivan Sim 8d13084f94
Split the `linkerd-version` CLI flag into `control-plane-version` and `proxy-version` (#2702)
* The 'linkerd-version' CLI flag is renamed to 'control-plane-version'
* Add version field to proxy config
* Add the control plane version to the global config
* Unit test for init image version
* Use more specific control plane and proxy versions in unit tests

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-04-19 11:35:20 -07:00
Andrew Seigner 8323e104fb
Introduce upgrade --from-manifests flag (#2697)
The `linkerd upgrade` command read the control-plane's config from
Kubernetes, which required the environment to be configured to connect
to the appropriate k8s cluster.

Intrdouce a `linkerd upgrade --from-manifests` flag, allowing the user
to feed the output of `linkerd install` into the upgrade command.

Fixes #2629

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-04-17 13:32:21 -07:00
Andrew Seigner 43cb3f841b
upgrade: unit tests (#2672)
This change introduces some unit tests on individual methods in the
upgrade code path, along with some minor cleanup.

Part of #2637

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-04-10 14:54:13 -07:00
Oliver Gould bbe1a60358
upgrade: Generate an Identity config if missing (#2656)
When upgrading from an older cluster that has a Linkerd config but no
identity, we need to generate an identity context so that the cluster is
configured properly.

Fixes #2650
2019-04-08 16:49:12 -07:00
Andrew Seigner 1c938b3f52
Introduce upgrade command unit tests (#2639)
This change introduces a basic unit test for the `linkerd upgrade`
command. Given a mock k8s client with linkerd-config and
linkerd-identity-issuer objects, it validates the rendered yaml output
against an expected file.

To enable this testing, most of the logic in the top-level upgrade
command has been moved down into a `validateAndBuild` method.

TODO:
- test individual functions around mutating options, flags, configs, and
  values
- enable reading the install information from a manifest rather than k8s

Part of #2637

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-04-05 11:55:20 -07:00
Carol A. Scott e15ce7f6d2
Output a URL with instructions after running `linkerd upgrade` (#2627)
Adds a URL to the `linkerd upgrade` output which contains full upgrade instructions. The message and the URL anchors are different in the case of success or failure.

Fixes #2575.
2019-04-04 16:32:11 -07:00