Commit Graph

187 Commits

Author SHA1 Message Date
Tarun Pothulapati efb1101bdb Switch to smaller-case values in linkerd2-cni (#3827)
* update linkerd2-cni templates and cli
* update readme and docs
* update helm unit tests
* update helm build script
* use smaller case linkerd version

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2019-12-16 15:09:57 -08:00
Alejandro Pedraza 2a4c71760d
Enable cert rotation test to work with dynamic namespaces, take two (#3795)
* Enable cert rotation test to work with dynamic namespaces

This PR adds support for dynamic cert generation when running the cert rotation intergration tests. This allows to avoid baking in the namespace in the certificate CN, thereby allowing us to run these tests on the clouds.

The tests in #3775 were failing because the second secret holding the issuer cert replacement was a leaf cert and not a root/intermediary cert capable of signing the CSRs. This is how the replacement cert looked like:

```bash
$ k -n l5d-integration-external-issuer get secrets linkerd-identity-issuer-new -ojson | jq '.data|.["tls.crt"]' | tr -d '"' | base64 -d | step certificate inspect -
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 2 (0x2)
    Signature Algorithm: ECDSA-SHA256
        Issuer: CN=identity.l5d-integration-external-issuer.cluster.local
        Validity
            Not Before: Dec 6 19:16:08 2019 UTC
            Not After : Dec 5 19:16:28 2020 UTC
        Subject: CN=identity.l5d-integration-external-issuer.cluster.local
        Subject Public Key Info:
            Public Key Algorithm: ECDSA
                Public-Key: (256 bit)
                X:
                    93:d5:fa:f8:d1:44:4f:9a:8c:aa:0c:9e:4f:98:a3:
                    8d:28:d9:cc:f2:74:4c:5f:76:14:52:47:b9:fb:c9:
                    a3:33
                Y:
                    d2:04:74:95:2e:b4:78:28:94:8a:90:b2:fb:66:1b:
                    e7:60:e5:02:48:d2:02:0e:4d:9e:4f:6f:e9:0a:d9:
                    22:78
                Curve: P-256
        X509v3 extensions:
            X509v3 Key Usage: critical
                Digital Signature, Key Encipherment
            X509v3 Extended Key Usage:
                TLS Web Server Authentication, TLS Web Client Authentication
            X509v3 Subject Alternative Name:
                DNS:identity.l5d-integration-external-issuer.cluster.local

    Signature Algorithm: ECDSA-SHA256
         30:46:02:21:00:f6:93:2f:10:ba:eb:be:bf:77:1a:2d:68:e6:
         04:17:a4:b4:2a:05:80:f7:c5:f7:37:82:7b:b7:9c:a1:66:6a:
         e1:02:21:00:b3:65:06:37:49:06:1e:13:98:7c:cf:f9:71:ce:
         5a:55:de:f6:1b:83:85:b0:a8:88:b7:cf:21:d1:16:f2:10:f9
```
For it to be a root/intermediate cert it should have had `CA:TRUE` under the `X509v3 extensions` section.

Why did the test pass sometimes? When it did pass for me, I could see in the linkerd-identity proxy logs something like:
```
ERR! [   320.964592s] linkerd2_proxy_identity::certify Received invalid ceritficate: invalid certificate: UnknownIssuer
```
so the cert retrieved from identity still was invalid but for some reason the proxy, sometimes, keeps on going despite that. And when one would delete the linkerd-identity pod, its proxy wouldn't come up at all, also showing that error.

With the changes from this branch, we no longer see that error in the logs and after deleting the linkerd-identity pod it comes back gracefully.
2019-12-11 15:50:06 -05:00
Zahari Dichev 6faf64e49f Revert "Enable cert rotation test to work with dynamic namespaces (#3775)" (#3787)
This reverts commit 0e45b9c03d.

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
2019-12-05 15:33:22 -05:00
Zahari Dichev 0e45b9c03d
Enable cert rotation test to work with dynamic namespaces (#3775)
This PR adds support for dynamic cert generation when running the cert rotation intergration tests. This allows to avoid baking in the namespace in the certificate CN, thereby allowing us to run these tests on the clouds.

* Enable cert rotation test to work with dynamic namespaces

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>

* Address comments

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>

* Address further comments

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
2019-12-05 10:08:01 +02:00
Joakim Roubert e1b3fdb029 Fix whitespace path handling in non-docker (build) scripts (#3650)
* Fix whitespace path handling in non-docker (build) scripts

Handling of whitespace paths was not fully implemented; this patch adds
the missing pieces. Also, only use bash where bash-specific
functionality is used/needed.

Signed-off-by: Joakim Roubert <joakimr@axis.com>
2019-11-26 09:48:41 -05:00
Alex Leong 0026103362 Unit and integration test fixups (#3730)
- Added cleanup step at the end of all integration tests.
- Disable external_issuer_integration_tests in cloud_tests due to
  namespace issue. Running this via `kind` tests is sufficient for now.
- Set a flakey test to `Skip`, relates to #3332.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-11-15 03:40:42 -08:00
Zahari Dichev 2d224302de
Add integration test for external issuer and cert rotation flows (#3709)
Signed-off-by: zaharidichev <zaharidichev@gmail.com>
2019-11-14 06:58:32 +02:00
Alejandro Pedraza 3324966702
Upgrade go to 1.13.4 (#3702)
Fixes #3566

As explained in #3566, as of go 1.13 there's a strict check that ensures a dependency's timestamp matches it's sha (as declared in go.mod). Our smi-sdk dependency has a problem with that that got resolved later on, but more work would be required to upgrade that dependency. In the meantime a quick pair of replace statements at the bottom of go.mod fix the issue.
2019-11-13 12:54:36 -05:00
Zahari Dichev 7dd5dfc2ba
Check health of meshed apps before and after linkerd upgrade (#3641)
* Check stats of deployed app before and after linkerd upgrade to ensure nothing broke

Signed-off-by: zaharidichev <zaharidichev@gmail.com>

* Address naming remarks

Signed-off-by: zaharidichev <zaharidichev@gmail.com>

* Improve application health checking

Signed-off-by: zaharidichev <zaharidichev@gmail.com>
2019-11-07 20:48:12 +02:00
Zahari Dichev 1bb9d66757 Integration test for custom cluster domain (#3660)
Signed-off-by: zaharidichev <zaharidichev@gmail.com>
2019-11-04 14:49:52 -08:00
Joakim Roubert 80d644eb1d docker-build-proxy: make apt work behind proxy (#3643)
This patch sends the proxy settings to docker build if present.
Without this, the docker build will fail on apt-get update on a
system that is behind a proxy.

Change-Id: I3fcbad4d9a9c30e5f0a00f03c6d8629ed8cc97b0
Signed-off-by: Joakim Roubert <joakimr@axis.com>
2019-11-04 13:17:44 -08:00
Joakim Roubert 478145ce45 Fix whitespace path handling in docker (build) scripts (#3634)
Handling of whitespace paths was not fully implemented; this patch adds
the missing pieces. Also, only use bash where bash-specific
functionality is used/needed.

Signed-off-by: Joakim Roubert <joakimr@axis.com>
2019-10-30 15:55:38 -07:00
Joakim Roubert b5309fad04 build-cli-bin: Use case for host_platform selection (#3626)
Increase readability and extensibility.

Change-Id: I0670950e14b59da0971397d08016176650602247
Signed-off-by: Joakim Roubert <joakimr@axis.com>
2019-10-28 16:49:56 -05:00
Joakim Roubert 3411e22bdc fetch-proxy: Make POSIX compatible (#3625)
* fetch-proxy: Make POSIX compatible
* fetch-proxy: Update old comment to match current behavior

Getting the directory where the script resides can easily be done
without bash-specific functionality, and hence the script can be POSIX
compatible.

Change-Id: I30bd69dccbc950bdce3dc5da4bea279305a7b1f9
Signed-off-by: Joakim Roubert <joakimr@axis.com>
2019-10-28 16:45:23 -05:00
Joakim Roubert 0341af86e8 build-cli-bin: POSIX compatible & handle whitespace paths (#3623)
Getting the directory where the script resides can easily be done
without bash-specific functionality, and hence the script can be POSIX
compatible. Also adding the missing pieces for handling paths with
whitespaces.

Change-Id: Ie2e867929be0322e476342438d9cf4a3d36f58f1
Signed-off-by: Joakim Roubert <joakimr@axis.com>
2019-10-28 16:36:53 -05:00
Oliver Gould 87e03ae940
Update proxy update commit messages with tag info (#3594)
Each proxy release tag now includes a message.

This change updates the git-commit-proxy-version script to include this
message in the commit message in this repo.
2019-10-18 10:20:38 -07:00
Alejandro Pedraza e76c5c3d9d
Keep old releases in Helm repo index (#3589)
* Keep old releases in Helm repo index

When building the Helm repo index file, keep the references to the old
releases. Also rename and keep the old index file in case
something goes wrong when generating the new one.

Fixes #3561
2019-10-16 17:21:53 -05:00
Alex Leong 3dcff52b9f
Switch from using golangci fmt to using goimports (#3555)
CI currently enforcing formatting rules by using the fmt linter of golang-ci-lint which is invoked from the bin/lint script.  However it doesn't seem possible to use golang-ci-lint as a formatter, only as a linter which checks formatting.  This means any formatter used by your IDE or invoked manually may or may not use the same formatting rules as golang-ci-lint depending on which formatter you use and which specific revision of that formatter you use.  

In this change we stop using golang-ci-lint for format checking.  We introduce `tools.go` and add goimports to the `go.mod` and `go.sum` files.  This allows everyone to easily get the same revision of goimports by running `go install -mod=readonly golang.org/x/tools/cmd/goimports` from inside of the project.  We add a step in the CI workflow that uses goimports via the `bin/fmt` script to check formatting.

Some shell gymnastics were required in the `bin/fmt` script to work around some limitations of `goimports`:
* goimports does not have a built-in mechanism for excluding directories, and we need to exclude the vendor director as well as the generated Go sources
* goimports returns a 0 exit code, even when formatting errors are detected

Signed-off-by: Alex Leong <alex@buoyant.io>
2019-10-16 13:56:11 -07:00
Saurav Tiwary 1e44513f30 Clean username before using as docker image tag (#3572)
* Clean username before using as docker image tag
* Allow Alphanumerics instead of just alphabets in docker image tag
Incorporate Alex's suggestions

Fixes #3570 

Signed-off-by: Saurav Tiwary <srv.twry@gmail.com>
2019-10-15 16:36:48 -07:00
Alejandro Pedraza 3de35ccc58
Remove Discovery service leftovers (#3500)
Followup to #2990, which refactored `linkerd endpoints` to use the
`Destination.Get` API instead of the `Discovery.Endpoints` API, leaving
the Discovery with no implented methods. This PR removes all the Discovery
code leftovers.

Fixes #3499
2019-10-15 11:20:21 -05:00
cpretzer 8f83a56431
Revert upgrade to buster based on CNI test failure after merge (#3486) 2019-09-26 13:40:43 -07:00
cpretzer 5455a344d8
Update base docker image to debian latest stable: buster (#3438)
* Update base docker image to debian latest stable: buster

Signed-off-by: Charles Pretzer <charles@buoyant.io>

* Update all files to use buster image
2019-09-26 09:02:12 -07:00
Kevin Leimkuhler 151104ec5a
Add script to load images into kind cluster (#3458)
## Summary

[kind](https://github.com/kubernetes-sigs/kind) has been a helpful tool for running local Kubernetes clusters and
testing linkerd builds. Once images are built with `bin/docker-build`, the
images must be loaded into the kind cluster.

This script should be run after `bin/docker-build` and will load the images into
the specified kind cluster.

Example:
```
$ bin/docker-build
$ kind get clusters # show available clusters to load images on to
kleimkuhler
$ bin/kind-load kleimkuhler
$ ./target/cli/linux/linkerd install | kubectl apply -f -
```

Signed-off-by: Kevin Leimkuhler <kleimkuhler@icloud.com>
2019-09-23 14:43:31 -07:00
Oliver Gould d51f7f77a7
proxy: Update to v2.71.0 (#3433)
Update the proxy release process to fetch artifacts from tagged GitHub releases.

* Use GitHub Actions for Pull Requests (linkerd/linkerd2-proxy#343)
* ci: Run tests inside rust container (linkerd/linkerd2-proxy#344)
* update tracing crates (linkerd/linkerd2-proxy#346)
* core: Introduce the Recover trait (linkerd/linkerd2-proxy#347)
* ci: Automate releases via GitHub Actions (linkerd/linkerd2-proxy#349)
* Add opencensus exporter (linkerd/linkerd2-proxy#338)
* Add trace context crate (linkerd/linkerd2-proxy#339)
* ci: Use a readymade release action (linkerd/linkerd2-proxy#351)
* Add 587 to the list of ports to disable protocol detection (linkerd/linkerd2-proxy#350)
* Record SHA of package artifact (linkerd/linkerd2-proxy#353)
2019-09-17 15:18:24 -07:00
Alejandro Pedraza 8270ba363c
Add chart_deploy into workflow.yml (#3415)
* Have CI push the Helm artifacts into GCS

- Added missing OWNERS and README files
- Added maintainers section to  Chart.yaml
- Changed NOTES.txt so it points to the installation of the CLI
- Set the proxy-init version to v1.1.0 in values.yaml

Ref #3256

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-09-11 12:09:50 -05:00
Alejandro Pedraza bd702b99ae
Last changes before submitting to the Helm incubator (#3292)
* Last changes before submitting to the Helm incubator

- Added missing OWNERS and README files
- Added maintainers section to  Chart.yaml
- Changed NOTES.txt so it points to the installation of the CLI
- Set the proxy-init version to v1.1.0 in values.yaml
- Added missing ProfileValidator vars, and add 'do not edit' comment to the Identity.Issuer.CrtExpiryAnnotation value
- Added new self-hosted repo
- Added  option to bin/helm-build
- Added DisableHeartBeat to README

Ref #3256

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-09-10 14:24:39 -05:00
Andrew Seigner 89deacd8d6
Decrease proxy and web Docker image sizes (#3384)
The `proxy` and `web` Docker images were 161MB and 186MB, respectively.
Most of the space was tools installed into the `linkerd.io/base` image.

Decrease `proxy` and `web` Docker images to 73MB and 90MB, respectively.
Switch these images to be based off of `debian:stretch-20190812-slim`.
Also set `-ldflags "-s -w"` for `proxy-identity` and `web`. Modify
`linkerd.io/base` to also be based off of
`debian:stretch-20190812-slim`, update tag to `2019-09-04.01`.

Fixes #3383

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-09-05 11:28:33 -07:00
Alejandro Pedraza 368d16f23c
Fix auto-injecting pods and integration tests reporting (#3335)
* Fix auto-injecting pods and integration tests reporting

When creating an Event when auto-injection occurs (#3316) we try to
fetch the parent object to associate the event to it. If the parent
doesn't exist (like in the case of stand-alone pods) the event isn't
created. I had missed dealing with one part where that parent was
expected.

This also adds a new integration test that I verified fails before this
fix.

Finally, I removed from `_test-run.sh` some `|| exit_code=$?` that was
preventing the whole suite to report failure whenever one of the tests
in `/tests` failed.

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-08-28 15:04:20 -05:00
陈谭军 981f5bc85d fix-up spelling mistake (#3328)
Signed-off-by: chentanjun <2799194073@qq.com>
2019-08-27 10:24:53 -07:00
Andrew Seigner ea27e0ca0e
Introduce integration tests into all ci runs (#3293)
The integration tests under `/test` were run separately via l5d-bot,
lacking the feedback and job management provided by ci.

Enable integration tests in ci, via a docker build and kind clusters
executed on a remote DOCKER_HOST.

CI runs are now broken into two stages, run serially. Each stage is
composed of jobs run in parallel:
- Setup stage
  - Validate go deps
  - Remote docker build
  - Kind cluster setup (deep)
  - Kind cluster setup (upgrade)
  - Kind cluster setup (helm)
- Test stage
  - Go unit tests
  - Node.js unit tests
  - Kind integration tests (deep)
  - Kind integration tests (upgrade)
  - Kind integration tests (helm)

This PR also modifies `bin/test-run.sh` to always set `--failfast` for
Go tests.

Also introduce `bin/docker` and `bin/kubectl` scripts, to ensure
cacheable, pinned executables in ci.

The existing integration tests for master merges and docker pushes,
running against GKE, remain in place.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-08-26 11:41:17 -07:00
Andrew Seigner 653ec8c5b7
Refactor bin/test-run for running tests separately (#3304)
The `bin/test-run` script executed upgrade, helm, and deep integration
test in series, but was structured in a way that did not permit running
these tests individually.

Move most of the logic from `bin/test-run` to a supporting library,
`bin/test-run.sh`, which will provide the ability to execute integration
tests individually. `bin/test-run`'s behavior is unchanged, it continues
to run upgrade, helm, and deep integration tests in series.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-08-22 11:05:06 -07:00
Andrew Seigner 4e058bfea2
Introduce bin/kind, move executables to target/bin (#3289)
`bin/helm` and `bin/protoc` were downloading their binaries into
`./target`, while `bin/lint` was downloading to the root of the repo.
Also travis was caching `./target`, which could become problematic if
that part of the test script relied on `target/cli/linux/linkerd`.

Standardize helm, kind, lint, and protoc to all download into
`./target/bin`, and modify travis to strictly cache that subdirectory.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-08-21 19:49:21 -07:00
Alejandro Pedraza 879650cef9
Wait for `helm delete` to finish in integration test (#3259)
* Wait for `helm delete` to finish in integration test

Followup to #3251

In `helm_cleanup` block till the linkerd namespace has been deleted

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-08-14 19:15:34 -05:00
Ivan Sim e52afc1197
Update the Helm build script (#3248)
* Update Helm build script to pin the Helm CLI version
* Update Linkerd version in the Helm values file

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-08-14 16:04:56 -07:00
Andrew Seigner 6c0ee2475b
Cleanup helm before full test-cleanup (#3251)
PR #3247 introduced additional helm cleanup in `bin/test-cleanup`.
During the integration tests, `bin/test-cleanup` is called prior to
`helm_cleanup` in `bin/test-run`. This causes `helm_cleanup` to fail, as
resources have already been deleted by `bin/test-cleanup`, and the
integration tests fail with `FAIL: error cleaning up Helm`.

Modify the integration tests to first call `helm_cleanup` prior to
calling `bin/test-cleanup`.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-08-14 10:47:17 -07:00
Andrew Seigner 9826cbdfe0
Label and cleanup helm after integration tests (#3247)
When helm integration tests fail, `bin/test-run` exits prior to calling
`helm_cleanup`, leaving behind a helm namespace and clusterrolebinding.

Update `bin/test-cleanup` to delete any remaining helm resources.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-08-14 09:30:48 -07:00
Ivan Sim 4d01e3720e
Update install and upgrade code to use the new helm charts (#3229)
* Delete symlink to old Helm chart
* Update 'install' code to use common Helm template structs
* Remove obsolete TLS assets functions.

These are now handle by Helm functions inside the templates

* Read defaults from values.yaml and values-ha.yaml
* Ensure that webhooks TLS assets are retained during upgrade
* Fix a few bugs in the Helm templates (see bullet points):
* Merge the way the 'install' ha and non-ha options are handled into one function
* Honor the 'NoInitContainer' option in the components templates
* Control plane mTLS will not be disabled if identity context in the
config map is empty. The data plane mTLS will still be automatically disabled
if the context is nil.
* Resolve test failures from rebase with master
* Fix linter issues
* Set service account mount path read-only field
* Add TLS variables of the webhooks and tap to values.yaml

During upgrade, these secrets are preserved to ensure they remain synced
wih the CA bundle in the webhook configurations. These Helm variables are used
to override the defaults in the templates.

* Remove obsolete 'chart' folder
* Fix bugs in templates
* Handle missing webhooks and tap TLS assets during upgrade

When upgrading from an older version that don't have these secrets, fallback to let Helm
create them by creating an empty charts.TLS struct.

* Revert the selector labels of webhooks to be compatible with that in 2.4

In 2.4, the proxy injector and profile validator webhooks already have their selector labels defined.
Since these attributes are immutable, the recent change to these selectors introduced by the Helm chart work will cause upgrade to fail.

* Alejandro's feedback
* Siggy's feedback
* Removed redundant unexported custom types

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-08-13 14:16:24 -07:00
Alejandro Pedraza d64a2f3689
Add integration test for `helm install` (#3223)
Ref #3143

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-08-13 09:14:32 -05:00
Alejandro Pedraza 0410b772f9
Bash function to setup Helm (#3218)
* helm binary wrapper

Ref #3143

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-08-12 10:08:12 -05:00
Alejandro Pedraza 3ae653ae92
Refactor proxy injection to use Helm charts (#3200)
* Refactor proxy injection to use Helm charts

Fixes #3128

A new chart `/charts/patch` was created, that generates the JSON patch
payload that is to be returned to the k8s API when doing the injection
through the proxy injector, and it's also leveraged by the `linkerd
inject --manual` CLI.

The VFS was used by `linkerd install` to access the old chart under
`/chart`. Now the proxy injection also uses the Helm charts to generate
the JSON patch (see above) so we've moved the VFS from `cli/static` to a
new common place under `/pkg/charts/static`, and the new root for the VFS is
now `/charts`.

`linkerd install` hasn't yet migrated to use the new charts (that'll
happen in #3127), so the only change in that regard was the creation of
`/charts/chart` which is a symlink pointing to `/chart` that
`install.go` now uses, so that the VFS contains both the old and new
charts, as a temporary measure.

You can see that `/bin/Dockerfile-bin`, `/controller/Dockerfile` and
`/bin/build-cli-bin` do now `go generate` pointing to the new location
(and the `go generate` annotation was moved from `/cli/main.go` to
`pkg/charts/static/templates.go`).

The symlink trick doesn't work when building the binaries through
Docker, so `/bin/Dockerfile-bin` replaces the symlink with an actual
copy of `/chart`.

Also note that in `/controller/Dockerfile` we now need to include the
`prod` tag in `go install` like we do in `/bin/Dockerfile-bin` so that
the proxy injector does use the VFS instead of the local file system.

- The common logic to parse a chart has been moved from `install.go` to
`/pkg/charts/util.go`.
- The special ENV var in the proxy for "outbound router capacity" that
only applies to the Prometheus pod is now handled directly in the proxy
partial and all the associated go code could be removed.
- The `patch.go` lib for generating the JSON patch in go along
with its tests `patch_test.go` are no longer needed.
- Lots of functions in `/pkg/inject/inject.go` got removed/simplified
with their logic being moved into the charts themselves. As a
consequence lots of things in `inject_test.go` became irrelevant.
- Moved `template-values.go` from `/pkg/inject` to `pkg/charts` as that
contains the go structs representation of the chart variables that
will be leveraged in #3127.

Don't forget to run `/bin/helm.sh` whenever you make changes to charts
;-)

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-08-07 17:32:37 -05:00
Alejandro Pedraza 5ae41fe856
Fixd bash checks in `bin/helm.sh` and `bin/test-cleanup` (#3205)
`bin/helm.sh`: you should see the following only if you have Tiller
installed in your cluster (which is installed with `helm init`):
```
Performing dry run installation
NAME:   linkerd
Performing dry run installation (HA mode)
NAME:   linkerd
```

`bin/test-cleanup`: when linkerd is not installed:

Before:
```bash
$ bin/test-cleanup
cleaning up control-plane namespaces in k8s-context []
cleaning up data-plane namespaces in k8s-context []
cleaning up rolebindings in kube-system namespace in k8s-context []
```

After this PR's changes:
```
$bin/test-cleanup
cleaning up control-plane namespaces in k8s-context []
no control-plane namespaces found
cleaning up data-plane namespaces in k8s-context []
no data-plane namespaces found
no clusterrolebindings found
no clusterroles found
no mutatingwebhookconfigurations found
no validatingwebhookconfigurations found
no podsecuritypolicies found
no customresourcedefinitions found
no apiservices found
cleaning up rolebindings in kube-system namespace in k8s-context []
```

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-08-07 12:41:33 -05:00
Ivan Sim 2bbf26748f
Add Control Plane Helm Templates And Proxy Partials (#3146)
* Updated controller template with proxy partials
* Declare dependency in requirements.yaml
* Add partial template for proxy's metadata
* Add proxy-init partial template
* Script to lint Helm charts and update their dependencies
* Update partials chart Chart.yaml
* Add proxy-init and resource partial templates
* Replace hard coded namespace variable in proxy env var
* Ignore chart dependencies .tgz files
* Add missing fields and re-order YAML elements to match CLI output
* Reuse control plane's resource partial template in 'partials' chart
* Set the proxy's destination service address env var
* Add Grafana's template
* Update api version of controller RBAC
* Add Heartbeat template
* Remove duplicated resources partial template
* Add remainder control plane components templates
* Add template for the 'linkerd-config' config map
* Add debug container template
* Update proxy partial with 'disable-identity' and 'disable-tap' variables

Note that these are inject-only variables.
Also added the LINKERD2_PROXY_TAP_SVC_NAME env var.

* Add validation conditions to ensure identity and tap aren't disabled for
control plane components
* Add partials for service account token mount path and security context capabilities
* Change proxy and proxy-init templates to use global scope

Some of the nested variables are removed from values.yaml to ensure changes
made to root-level variables are propagated directly into the partial
templates. The previous approach of using YAML anchors in the
values.yaml to share common values can get out-of-sync when values are
changed via the Helm's `--set` option.

* Update templates and values file to match #3161
* Perform a dry run installation if there is a local Tiller
* Reorder JSON elements in linkerd-config
* Re-adjust nested partials indentation to work with inject 'patch' chart

Previously, the partials will render their content as an element in the list.
While it works for installation, the toJson function in the 'inject' patch code
ends up converting it into a JSON list, instead of the expected JSON
object.

* Trap the last fail command in the Helm shell script
* Add the identity trust anchor
* Address Thomas' feedback on handling HA

All the HA-related variables are moved to values-ha.yaml

* Convert ignore ports string to JSON list in linkerd-config

Also fixed some indentation issues.

* Add values-ha.yaml
* Include the service account token mount path only if identity is enabled
* Fixed malformed JSON in linkerd-config config map
* Rename chart to 'linkerd2'
* Add NOTES.txt
* Fix incorrect variable path in proxy template
* Remove fake TLS assets
* Add 'required' constraint to identity trust anchors variable
* Update tap templates per #3167
* Bump default version to edge-19.8.1 due to dependency on RSA support

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-08-06 09:18:19 -07:00
Andrew Seigner a59c1dd32d
Introduce tap APIService, update `linkerd tap` (#3167)
The Tap Service enabled tapping of any meshed pod, regardless of user
privilege.

This change introduces a new Tap APIService. Kubernetes provides
authentication and authorization of Tap requests, and then forwards
requests to a new Tap APIServer, which implements a Kubernetes
aggregated APIServer. The Tap APIServer authenticates the client TLS
from Kubernetes, and authorizes the user via a SubjectAccessReview.

This change also modifies the `linkerd tap` command to make requests
against the new APIService.

The Tap APIService implements these Kubernetes-style endpoints:
POST /apis/tap.linkerd.io/v1alpha1/watch/namespaces/:ns/tap
POST /apis/tap.linkerd.io/v1alpha1/watch/namespaces/:ns/:res/:name/tap
GET  /apis
GET  /apis/tap.linkerd.io
GET  /apis/tap.linkerd.io/v1alpha1
GET  /healthz
GET  /healthz/log
GET  /healthz/ping
GET  /metrics
GET  /openapi/v2
GET  /version

Users authorize to the new `tap.linkerd.io/v1alpha1` via RBAC. Only the
`watch` verb is supported. Access is also available via subresources
such as `deployments/tap` and `pods/tap`.

This change introduces the following resources into the default Linkerd
install:
- Global
  - APIService/v1alpha1.tap.linkerd.io
  - ClusterRoleBinding/linkerd-linkerd-tap-auth-delegator
- `linkerd` namespace:
  - Secret/linkerd-tap-tls
- `kube-system` namespace:
  - RoleBinding/linkerd-linkerd-tap-auth-reader

Tasks not covered by this PR:
- `linkerd top`
- `linkerd dashboard`
- `linkerd profile --tap`
- removal of the unauthenticated tap controller

Fixes #2725, #3162, #3172

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-08-01 14:02:45 -07:00
Andrew Seigner f0f3f8e5c5
Bump golangci-lint to 1.17.1 (#3150)
Also add `bodyclose` linter

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-07-29 10:49:03 -07:00
Andrew Seigner 18b74aa8a8
Introduce Go modules support (#2481)
The repo relied on `dep` for managing Go dependencies. Go 1.11 shipped
with Go modules support. Go 1.13 will be released in August 2019 with
module support enabled by default, deprecating GOPATH.

This change replaces `dep` with Go modules for dependency management.
All scripts, including Docker builds and ci, should work without any dev
environment changes.

To execute `go` commands directly during development, do one of the
following:
1. clone this repo outside of `GOPATH`; or
2. run `export GO111MODULE=on`

Summary of changes:
- Docker build scripts and ci set `-mod=readonly`, to ensure
  dependencies defined in `go.mod` are exactly what is used for the
  builds.
- Dependency updates to `go.mod` are accomplished by running
 `go build` and `go test` directly.
- `bin/go-run`, `bin/build-cli-bin`, and `bin/test-run` set
  `GO111MODULE=on`, permitting usage inside and outside of GOPATH.
- `gcr.io/linkerd-io/go-deps` tags hashed from `go.mod`.
- `bin/update-codegen.sh` still requires running from GOPATH,
  instructions added to BUILD.md.

Fixes #1488

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
2019-07-25 14:41:38 -07:00
Alex Leong d6ef9ea460
Update ServiceProfile CRD to version v1alpha2 and remove validation (#3078)
The openAPIV3Schema validation in the ServiceProfiles CRD is very limited in what it can validate and is obviated by more sophisticated validation done by the validating admission controller.  Therefore, we would like to remove the openAPIV3Schema validation to reduce the size and complexity of the CRD object.

To do so, we must also bump the version of the ServiceProfile custom resource from v1alpha1 to v1alpha2.  This ensures that when the controller is upgraded, it will attempt to watch the v1alpha2 resource.  If it cannot (because, for example, the controller pod started before the ServiceProfile CRD was updated and therefore the v1alpha2 version does not exist) then it will go into a crash loop backoff until it can.  This essentially means that the controller will wait for the CRD to be upgraded to include v1alpha2 before it will start.  

Bumping the version is necessary because if we did not, it would be possible for the controller to start before the CRD is updated (removing the validation).  In this case, when the CRD is edited, the controller will lose its list watch on ServiceProfiles and will stop getting updates.

Signed-off-by: Alex Leong <alex@buoyant.io>
2019-07-23 11:46:31 -07:00
Ivan Sim f535c2d3d2
Integration Test Script Pre-/Post-Test Cleanup (#3108)
* Updates for the integration test script

1. Remove existing resources prior to starting the test
2. Remove existing resources post upgrade test
3. Fail fast if 'install_test.go` fails
4. Don't perform cleanup if any of the tests fail for debugging
opportunity

* Remove pre-test cleanup from .travis.yaml

This is now done in the bin/test-run script so that it can be shared
between l5d-bot and staging.

Signed-off-by: Ivan Sim <ivan@buoyant.io>
2019-07-19 11:13:03 -07:00
Alejandro Pedraza 68f2f694e3
Improve object cleanup when integration tests fail (#3080)
Integration tests may fail and leave behind namespaces that following
builds aren't able to clean up because the git sha is being included in
the namespace name, and the following builds don't know about those
shas.

This modifies the `test-cleanup` script to delete based on object labels
instead of relying on the objects names, now that after 2.4 all the
control plane components are labeled. Note that this will also remove
non-testing linkerd namespaces, but we were already kinda doing that
partially because we were removing the cluster-level resources (CRDs,
webhook configs, clusterroles, clusterrolebindings, psp).

`test-cleanup` no longer receives a namespace name as an argument.

The data plane namespaces aren't labeled though, so I've added the
`linkerd.io/is-test-data-plane` label for them in
`CreateNamespaceIfNotExists()`, and making sure all tests that need a
data plaine explicitly call that method instead of creating the
namespace as a side-effect in `KubectlApply()`.

Signed-off-by: Alejandro Pedraza <alejandro@buoyant.io>
2019-07-12 15:01:10 -05:00
Oliver Gould c1aaaf8114
git-commit-proxy-version: Omit SHAs from commit (#3076)
The git-commt-proxy-version script attempted to link to the specifc
SHAs. GitHub doesn't actually render these links, and the information is
redundant since we link the appropriate PR.
2019-07-11 14:48:30 -07:00
Oliver Gould 7699ef256d
git-commit-proxy-version: fixup invalid git invocation (#3075) 2019-07-11 14:21:03 -07:00