Commit Graph

254 Commits

Author SHA1 Message Date
Alex Leong cdc57d1af0
Use linkerd-jaeger extension for control plane tracing (#5299)
Now that tracing has been split out of the main control plane and into the linkerd-jaeger extension, we remove references to tracing from the main control plane including:

* removing the tracing components from the main control plane chart
* removing the tracing injection logic from the main proxy injector and inject CLI (these will be added back into the new injector in the linkerd-jaeger extension)
* removing tracing related checks (these will be added back into `linkerd jaeger check`)
* removing related tests

We also update the `--control-plane-tracing` flag to configure the control plane components to send traces to the linkerd-jaeger extension.  To make sure this works even when the linkerd-jaeger extension is installed in a non-default namespace, we also add a `--control-plane-tracing-namespace` flag which can be used to change the namespace that the control plane components send traces to.

Note that for now, only the control plane components send traces; the proxies in the control plane do not.  This is because the linkerd-jaeger injector is not yet available.  However, this change adds the appropriate namespace annotations to the control plane namespace to configure the proxies to send traces to the linkerd-jaeger extension once the linkerd-jaeger injector is available.

I tested this by doing the following:

1. bin/linkerd install | kubectl apply -f -
1. bin/helm install jaeger jaeger/charts/jaeger
1. bin/linkerd upgrade --control-plane-tracing=true | kubectl apply -f -
1. kubectl -n linkerd-jaeger port-forward svc/jaeger 16686
1. open http://localhost:16686
1. see traces from the linkerd control plane

Signed-off-by: Alex Leong <alex@buoyant.io>
2020-12-08 14:34:26 -08:00
Alejandro Pedraza deca7ede08
Consolidate integration tests under k3d (#5245)
* Consolidate integration tests under k3d

Fixes #5007

Simplified integration tests by moving all to k3d. Previously things were running in Kind, except for the multicluster tests, which implied some extra complexity in the supporting scripts.

Removed the KinD config files under `test/integration/configs`, as config is now passed as flags into the `k3d` command.

Also renamed `kind_integration.yml` to `integration_tests.yml`

Test skipping logic under ARM was also simplified.
2020-11-18 14:33:16 -05:00
Alejandro Pedraza 948aa23b2a
Remove logs comparisons in integration tests (#5223)
The rare cases where these tests were useful don't make up for the burden of
maintaing them, having different k8s version change the messages and
having unexpected warnings come up that didn't affect the final
convergence of the system.

With this we also revert the indirection added back in #4538 that
fetched unmatched warnings after a test had failed.
2020-11-13 16:00:16 -05:00
Tarun Pothulapati e4c354985c
cli: Remove get cmd and relevant tests (#5202)
Fixes #5190

`linkerd get` is not used currently and works only for pods. This can be
removed instead as per the issue. This branch removes the command and
also the associated unit and integration tests.

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2020-11-12 11:19:46 -08:00
Alex Leong da194f5dc3
Warn when webhook certificates near expiry (#5155)
Fixes #5149 

Before:

```
linkerd-webhooks-and-apisvc-tls
-------------------------------
× tap API server has valid cert
    certificate will expire on 2020-10-28T20:22:32Z
    see https://linkerd.io/checks/#l5d-tap-cert-valid for hints
```

After:

```
linkerd-webhooks-and-apisvc-tls
-------------------------------
√ tap API server has valid cert
‼ tap API server cert is valid for at least 60 days
    certificate will expire on 2020-10-28T20:22:32Z
    see https://linkerd.io/checks/#l5d-webhook-cert-not-expiring-soon for hints
√ proxy-injector webhook has valid cert
‼ proxy-injector cert is valid for at least 60 days
    certificate will expire on 2020-10-29T18:17:03Z
    see https://linkerd.io/checks/#l5d-webhook-cert-not-expiring-soon for hints
√ sp-validator webhook has valid cert
‼ sp-validator cert is valid for at least 60 days
    certificate will expire on 2020-10-28T20:21:34Z
    see https://linkerd.io/checks/#l5d-webhook-cert-not-expiring-soon for hints
```

Signed-off-by: Alex Leong <alex@buoyant.io>
2020-10-30 11:48:51 -07:00
Alejandro Pedraza 177669b377
Remove code refs to controllerImageVersion (#5119)
Followup to #5100

We had both `controllerImageVersion` and `global.controllerImageVersion`
configs, but only the latter was taken into account in the chart
templates, so this change removes all of its references.
2020-10-21 13:40:25 -05:00
Oliver Gould 25e49433fd
Do not permit cluster networks to be overridden per-pod (#5111)
In #5110 the `global.proxy.destinationGetNetworks` configuration is
renamed to `global.clusterNetworks` to better reflect its purpose.

The `config.linkerd.io/proxy-destination-get-networks` annotation allows
this configuration to be overridden per-workload, but there's no real use
case for this. I don't think we want to support this value differing
between pods in a cluster. No good can come of it.

This change removes support for the `proxy-destination-get-networks`
annotation.
2020-10-21 09:34:13 -07:00
Oliver Gould f0820bdfbf
inject: Use 'quote' function in proxy template (#5107)
As described in #5105, it's not currently possible to set the proxy log
level to `off`. The proxy injector's template does not quote the log
level value, and so the `off` value is handled as `false`. Thanks, YAML.

This change updates the proxy template to use helm's `quote` function
throughout, replacing manually quoted values and fixing the quoting for
the log level value.

We also remove the default logFormat value, as the default is specified
in values.yaml.
2020-10-20 15:36:10 -07:00
Oliver Gould 4f16a234aa
Add a default set of ports to bypass the proxy (#5093)
The proxy has a default, hardcoded set of ports on which it doesn't do
protocol detection (25, 587, 3306 -- all of which are server-first
protocols). In a recent change, this default set was removed from
the outbound proxy, since there was no way to configure it to anything
other than the default set. I had thought that there was a default set
applied to proxy-init, but this appears to not be the case.

This change adds these ports to the default Helm values to restore the
prior behavior.

I have also elected to include 443 in this set, as it is generally our
recommendation to avoid proxying HTTPS traffic, since the proxy provides
very little value on these connections today.

Additionally, the memcached port 11211 is skipped by default, as clients
do not issue any sort of preamble that is immediately detectable.

These defaults may change in the future, but seem like good choices for
the 2.9 release.
2020-10-16 11:53:41 -07:00
Alex Leong 9701f1944e
Stop rendering addon config (#5078)
The linkerd-addon-config is no longer used and can be safely removed.

Signed-off-by: Alex Leong <alex@buoyant.io>
2020-10-16 11:07:51 -07:00
Oliver Gould 60a742ab56
tests: Consolidate TestHelper.LinkerdRun error handling (#5057)
Most invocations of `TestHelper.LinkerdRun` don't actually need the stderr
output except to encode it in the error message. This changes this helper
to return an error that includes the full invoked command and error message.

Invocations that need direct access to stderr must call `TestHelper.PipeToLinkerdRun`
2020-10-15 14:57:03 -07:00
Oliver Gould 222c11400b
tests: Set proxy log to linkerd=debug (#5081)
The proxy log level `linkerd2_proxy=debug` only enables logging from a
few proxy modules. We should instead use the more general
`linkerd=debug`.
2020-10-14 15:31:03 -07:00
Alex Leong 9553fbcd75
Skip SMI metrics integration test on arm (#5086)
The SMI metrics image does not yet support arm.  Thus we must skip the SMI metrics integration test when using arm.

Signed-off-by: Alex Leong <alex@buoyant.io>
2020-10-14 14:10:33 -07:00
Zahari Dichev de8855c096
More comprehensive injection integration test (#5049)
The purpose of this test is to validate that the auto injector configures the proxy and the additional containers according to the specified config.

This is done by providing a helper that can generate the desired annotations and later inspect an injected pod in order to determine that every bit of configuration has been accounted for. This test is to provide further assurance that #5036 did not introduce any regressions.

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
2020-10-14 19:31:56 +03:00
Oliver Gould 5e7e7e6477
proxy: v2.115.0 (#5076)
This release fixes several recent regressions:

1. The proxy could incorrectly emit inbound requests with absolute-form
   URIs.
2. Inbound tap metadata did not include source addresses or identities.
3. Gateway requests included the incorrect port in the
   `l5d-dst-canonical` header.
4. Gateway requests never included a `Host` header.

Furthermore, support for the
`LINKERD2_PROXY_OUTBOUND_PORTS_DISABLE_PROTOCOL_DETECTION` environment
variable has been removed in anticipation of control plane changes that
will provide this configuration via service profiles. This configuration
is never set by the proxy injector, so this change does not pose any
issues with regard to compatibility.

---

* metrics: Coerce targets to metric labels by-reference (linkerd/linkerd2-proxy#706)
* outbound: Unify TCP & HTTP target types (linkerd/linkerd2-proxy#707)
* inbound: Fix source tap annotations (linkerd/linkerd2-proxy#712)
* trace-context: Simplify implementation with async (linkerd/linkerd2-proxy#710)
* outbound: Use profile to inform protocol detection (linkerd/linkerd2-proxy#708)
* inbound: Fix URI normalization for orig-proto requests (linkerd/linkerd2-proxy#713)
* outbound: more TCP tests, test cleanup (linkerd/linkerd2-proxy#711)
* gateway: Ensure proper outbound metadata (linkerd/linkerd2-proxy#715)
2020-10-14 08:11:17 -07:00
Tarun Pothulapati 1e7bb1217d
Update Injection to use new linkerd-config.values (#5036)
This PR Updates the Injection Logic (both CLI and proxy-injector)
to use `Values` struct instead of protobuf Config, part of our move
in removing the protobuf.

This does not touch any of the flags, install related code.

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>

Co-authored-by: Alex Leong <alex@buoyant.io>
2020-10-07 09:54:34 -07:00
Alex Leong 1784f0643e
Add linkerd-config-overrides secret (#4911)
This PR adds a new secret to the output of `linkerd install` called `linkerd-config-overrides`.  This is the first step towards simplifying the configuration of the linkerd install and upgrade flow through the CLI.  This secret contains the subset of the values.yaml which have been overridden.  In other words, the subset of values which differ from their default values.  The idea is that this will give us a simpler way to produce the `linkerd upgrade` output while still persisting options set during install.  This will eventually replace the `linkerd-config` configmap entirely.

This PR only adds and populates the new secret.  The secret is not yet read or used anywhere.  Subsequent PRs will update individual control plane components to accept their configuration through flags and will update the `linkerd upgrade` flow to use this secret instead of the `linkerd-config` configmap.

This secret is only generated by the CLI and is not present or required when installing or upgrading with Helm.

Here are sample contents of the secret, base64 decoded.  Note that identity tls context is saved as an override so that it can be persisted across updates.  Since these fields contain private key material, this object must be a secret.  This secret is only used for upgrades and thus only the CLI needs to be able to read it.  We will not create any RBAC bindings to grant service accounts access to this secret.

```
global:
  identityTrustAnchorsPEM: |
    -----BEGIN CERTIFICATE-----
    MIIBhDCCASmgAwIBAgIBATAKBggqhkjOPQQDAjApMScwJQYDVQQDEx5pZGVudGl0
    eS5saW5rZXJkLmNsdXN0ZXIubG9jYWwwHhcNMjAwODI1MjMzMTU3WhcNMjEwODI1
    MjMzMjE3WjApMScwJQYDVQQDEx5pZGVudGl0eS5saW5rZXJkLmNsdXN0ZXIubG9j
    YWwwWTATBgcqhkjOPQIBBggqhkjOPQMBBwNCAAQ0e7IPBlVZ03TL8UVlODllbh8b
    2pcM5mbtSGgpX9z0l3n5M70oHn715xu2szh63oBjPl2ZfOA5Bd43cJIksONQo0Iw
    QDAOBgNVHQ8BAf8EBAMCAQYwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMC
    MA8GA1UdEwEB/wQFMAMBAf8wCgYIKoZIzj0EAwIDSQAwRgIhAI7Sy8P+3TYCJBlK
    pIJSZD4lGTUyXPD4Chl/FwWdFfvyAiEA6AgCPbNCx1dOZ8RpjsN2icMRA8vwPtTx
    oSfEG/rBb68=
    -----END CERTIFICATE-----
heartbeatSchedule: '42 23 * * * '
identity:
  issuer:
    crtExpiry: "2021-08-25T23:32:17Z"
    tls:
      crtPEM: |
        -----BEGIN CERTIFICATE-----
        MIIBhDCCASmgAwIBAgIBATAKBggqhkjOPQQDAjApMScwJQYDVQQDEx5pZGVudGl0
        eS5saW5rZXJkLmNsdXN0ZXIubG9jYWwwHhcNMjAwODI1MjMzMTU3WhcNMjEwODI1
        MjMzMjE3WjApMScwJQYDVQQDEx5pZGVudGl0eS5saW5rZXJkLmNsdXN0ZXIubG9j
        YWwwWTATBgcqhkjOPQIBBggqhkjOPQMBBwNCAAQ0e7IPBlVZ03TL8UVlODllbh8b
        2pcM5mbtSGgpX9z0l3n5M70oHn715xu2szh63oBjPl2ZfOA5Bd43cJIksONQo0Iw
        QDAOBgNVHQ8BAf8EBAMCAQYwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMC
        MA8GA1UdEwEB/wQFMAMBAf8wCgYIKoZIzj0EAwIDSQAwRgIhAI7Sy8P+3TYCJBlK
        pIJSZD4lGTUyXPD4Chl/FwWdFfvyAiEA6AgCPbNCx1dOZ8RpjsN2icMRA8vwPtTx
        oSfEG/rBb68=
        -----END CERTIFICATE-----
      keyPEM: |
        -----BEGIN EC PRIVATE KEY-----
        MHcCAQEEIJaqjoDnqkKSsTqJMGeo3/1VMfJTBsMEuMWYzdJVxIhToAoGCCqGSM49
        AwEHoUQDQgAENHuyDwZVWdN0y/FFZTg5ZW4fG9qXDOZm7UhoKV/c9Jd5+TO9KB5+
        9ecbtrM4et6AYz5dmXzgOQXeN3CSJLDjUA==
        -----END EC PRIVATE KEY-----
```

Signed-off-by: Alex Leong <alex@buoyant.io>
2020-09-29 08:01:36 -07:00
Tarun Pothulapati d0caaa86c4
Bump k8s client-go to v0.19.2 (#5002)
Fixes #4191 #4993

This bumps Kubernetes client-go to the latest v0.19.2 (We had to switch directly to 1.19 because of this issue). Bumping to v0.19.2 required upgrading to smi-sdk-go v0.4.1. This also depends on linkerd/stern#5

This consists of the following changes:

- Fix ./bin/update-codegen.sh by adding the template path to the gen commands, as it is needed after we moved to GOMOD.
- Bump all k8s related dependencies to v0.19.2
- Generate CRD types, client code using the latest k8s.io/code-generator
- Use context.Context as the first argument, in all code paths that touch the k8s client-go interface

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2020-09-28 12:45:18 -05:00
Alejandro Pedraza e8f0724a71
Multicluster integration test (#4998)
This implements the run_multicluster_test() function in bin/_test-helpers.sh.

The idea is to create two clusters (source and target) using k3d, with linkerd and multicluster support in both, plus emojivoto (without vote-bot) in target, and vote-bot in source.
We then link the clusters and make sure traffic is flowing.

Detailed sequence:

Create certficates.
Install linkerd along with multicluster support in the target cluster.
Run the target1 test: install emojivoto in the target cluster (without vote-bot).
Run linkerd mc link on the target cluster.
Install linkerd along with multicluster support in the source cluster.
Apply the link resource in the source cluster.
Run the source test: Check linkerd mc gateways returns the target cluster link, and only install emojivoto's vote-bot in the source cluster. Note vote-bot's yaml defines the web-svc service as web-svc-target.emojivoto:80
Run the target2 test: Make sure web-svc in the target cluster is receiving requests.
2020-09-26 05:26:23 -05:00
Alejandro Pedraza b50ae6290d
Add support for k3d in integration tests (#4994)
* Add support for k3d in integration tests

KinD doesn't support setting LoadBalancer services out of the box. It can be added with some additional work, but it seems the solutions are not cross-platform.

K3d on the other hand facilitates this, so we'll be using k3d clusters for the multicluster integration test.

The current change sets the ground by generalizing some of the integration tests operations that were hard-coded to KinD.

- Added `bin/k3d` to wrap the setup and running of a pinned version of `k3d`.
- Refactored `bin/_test-helpers.sh` to account for tests to be run in either KinD or k3d.
- Renamed `bin/kind-load` to `bin/image-load` and make it more generic to load images for both KinD (default) and k3d. Also got rid of the no longer used `--images-host` option.
- Added a placeholder for the new `multicluster` test in the lists in `bin/_test-helpers.sh`. It starts by setting up two k3d clusters.

* Refactor handling of the `--multicluster` flag in integration tests (#4995)

Followup to #4994, based off of that branch (`alpeb/k3d-tests`).
This is more preliminary work previous to the more complete multicluster integration test.

- Removed the `--multicluster` flag from all the tests we had in `bin/_test-helpers.sh`, so only the new "multicluster" integration test will make use of that. Also got rid of the `TestUninstallMulticluster()` test in `install_test.go` to keep the multicluster stuff around, needed for the more complete multicluster test that will be implemented in a followup PR.
- Added "multicluster" to the list of tests in the `kind_integration.yml` workflow.
- For now, this new "multicluster" test in `run_multicluster_test()` is just running the install tests (`test/integration/install_test.go`) with the `--multicluster` flag.

Co-authored-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
2020-09-25 16:33:17 -05:00
Alejandro Pedraza 0f869f2e50
Ability for int tests to use external certs generated with openssl (#4997)
Adds bin/certs-openssl, which creates self-signed root cert/key and issuer cert/key using openssl. This will be used in the two clusters set up in the multicluster integration test (followup PR), given CI already has openssl and to avoid having to install step.
Adds a new flag `--certs-path` to the integration tests, pointing to the path where those certs (ca.crt, ca.key, issuer.key and issuer.crt) will be located to be fed into linkerd install's `--identity-*` flags.
2020-09-25 11:25:29 -05:00
Tarun Pothulapati 3d900ccc19
Integration test for smi-metrics (#4844)
* Integration test for smi-metrics

This PR adds an integration test which installs SMI-Metrics and performs
queries and matches the reply with a regex query.

Currently, We store the SMI Helm pkg locally and run the test on top, so 
That our CI does not break and we will periodically update the package
based on the newer releases of SMI-Metrics

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2020-09-23 22:49:20 +05:30
Tarun Pothulapati ecce5b91f6
tests: Add Calico CNI deep integration tests (#4952)
* tests: Add new CNI deep integration tests

Fixes #3944

This PR adds a new test, called cni-calico-deep which installs the Linkerd CNI
plugin on top of a cluster with Calico and performs the current integration tests on top, thus
validating various Linkerd features when CNI is enabled. For Calico
to work, special config is required for kind which is at `cni-calico.yaml`

This is different from the CNI integration tests that we run in
cloud integration which performs the CNI level integration tests.

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2020-09-23 19:58:28 +05:30
OlivierB f599bf9b10
Helm chart - linkerd2-collector : enable jaeger receiver (#4783)
Fixes #4778

Signed-off-by: Olivier Boudet <o.boudet@gmail.com>
2020-09-21 12:17:04 -07:00
Alejandro Pedraza ccf027c051
Push docker images to ghcr.io instead of gcr.io (#4953)
* Push docker images to ghcr.io instead of gcr.io

The `cloud_integration.yml` and `release.yml` workflows were modified to
log into ghcr.io, and remove the `Configure gcloud` step which is no
longer necessary.

Note that besides the changes to cloud_integration.yml and release.yml, there was a change to the upgrade-stable integration test so that we do linkerd upgrade --addon-overwrite to reset the addons settings because in stable-2.8.1 the Grafana image was pegged to gcr.io/linkerd-io/grafana in linkerd-config-addons. This will need to be mentioned in the 2.9 upgrade notes.

Also the egress integration test has a debug container that now is pegged to the edge-20.9.2 tag.

Besides that, the other changes are just a global search and replace (s/gcr.io\/linkerd-io/ghcr.io\/linkerd/).
2020-09-10 15:16:24 -05:00
Oliver Gould 7ee638bb0c
inject: Configure the proxy to discover profiles for unnamed services (#4960)
The proxy performs endpoint discovery for unnamed services, but not
service profiles.

The destination controller and proxy have been updated to support
lookups for unnamed services in linkerd/linkerd2#4727 and
linkerd/linkerd2-proxy#626, respectively.

This change modifies the injection template so that the
`proxy.destinationGetNetworks` configuration enables profile
discovery for all networks on which endpoint discovery is permitted.
2020-09-10 12:44:00 -07:00
Zahari Dichev 084bb678c7
Perform TLS checks on injector, sp validator and tap (#4924)
* Check sp-validator,proxy-injector and tap certs

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
2020-09-10 11:21:23 -05:00
Zahari Dichev 77c88419b8
Make destination and identity services headless (#4923)
* Make destination and identity svcs headless

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
2020-09-02 14:53:38 -05:00
Ali Ariff 5186383c81
Add ARM64 Integration Test (#4897)
* Add ARM64 Integration Test

Signed-off-by: Ali Ariff <ali.ariff12@gmail.com>
2020-08-28 10:38:40 -07:00
Tarun Pothulapati c9c5d97405
Remove SMI-Metrics charts and commands (#4843)
Fixes #4790

This PR removes both the SMI-Metrics templates along with the
experimental sub-commands. This also removes pkg `smi-metrics`
as there is no direct use of it without the commands.

Signed-off-by: Tarun Pothulapati <tarunpothulapati@outlook.com>
2020-08-24 14:35:33 -07:00
Josh Soref 72aadb540f
Spelling (#4872)
This PR corrects misspellings identified by the [check-spelling action](https://github.com/marketplace/actions/check-spelling).

The misspellings have been reported at aaf440489e (commitcomment-41423663)

The action reports that the changes in this PR would make it happy: 5b82c6c5ca

Note: this PR does not include the action. If you're interested in running a spell check on every PR and push, that can be offered separately.

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
2020-08-12 21:59:50 -07:00
Mayank Shah 25fe7237ae
conformance validation: move `tap_test.go` test helpers to `testutil` (#4800)
* Refactor `tap` test helpers

Signed-off-by: Mayank Shah <mayankshah1614@gmail.com>
2020-07-28 13:12:25 -07:00
Matei David 1c197b14e7
Change destination context token format (#4771)
Add a new structure on the destination controller side to keep track of contextual information.
The token format has been changed from ns:<namespace> to a JSON format so that more variables can be
encdoed in the token. As part of this PR, a new field 'nodeName' has been added to help with service
topologies.

Fixes #4498

Signed-off-by: Matei David <matei.david.35@gmail.com>
2020-07-27 09:49:48 -07:00
Alex Leong d540e16c8b
Make service mirror controller per target cluster (#4710)
This PR removes the service mirror controller from `linkerd mc install` to `linkerd mc link`, as described in https://github.com/linkerd/rfc/pull/31.  For fuller context, please see that RFC.

Basic multicluster functionality works here including:
* `linkerd mc install` installs the Link CRD but not any service mirror controllers
* `linkerd mc link` creates a Link resource and installs a service mirror controller which uses that Link
* The service mirror controller creates and manages mirror services, a gateway mirror, and their endpoints.
* The `linkerd mc gateways` command lists all linked target clusters, their liveliness, and probe latences.
* The `linkerd check` multicluster checks have been updated for the new architecture.  Several checks have been rendered obsolete by the new architecture and have been removed.

The following are known issues requiring further work:
* the service mirror controller uses the existing `mirror.linkerd.io/gateway-name` and `mirror.linkerd.io/gateway-ns` annotations to select which services to mirror.  it does not yet support configuring a label selector.
* an unlink command is needed for removing multicluster links: see https://github.com/linkerd/linkerd2/issues/4707
* an mc uninstall command is needed for uninstalling the multicluster addon: see https://github.com/linkerd/linkerd2/issues/4708

Signed-off-by: Alex Leong <alex@buoyant.io>
2020-07-23 14:32:50 -07:00
Tarun Pothulapati 986e0d4627
prometheus: add add-on checks (#4756)
As linkerd-prometheus is optional now, the checks are also separated
and should only work when the prometheus add-on is installed.

This is done by re-using the add-on check code.
2020-07-23 18:03:24 +05:30
Alejandro Pedraza 873bd61324
Helm integration deep tests (#4728)
This creates a new integration test target that launches the deep suite,
using a linkerd instance installed through Helm.

I've added a `global.proxyInit.ignoreInboundPorts=1234,5678` override
during install and enhanced the injection test to catch problems like
what we saw in #4679.
2020-07-10 14:48:49 -05:00
Alejandro Pedraza e225c46484
Fix upgrade integration test (#4744)
An unappropriate variable reuse resulted in the failure of the test for
upgrading using manifests. This only happened when the upgrade was
retried a second time (when there's a discrepancy in the heartbeat cron
schedule, which is bening).
2020-07-09 17:05:40 -05:00
Kevin Leimkuhler 5d400f5bcd
Fix deep integration test (#4709)
This fixes the deep integration test which currently only calls `run_test` for
`edges` integration test.

This occurs because `run_test "${tests[@]}"` will pass an entire array of
filenames when `run_test` only expects *one* filename.

The solution is to loop through `tests` and call `run_test` for each file.

Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
2020-07-09 13:42:52 -07:00
Tarun Pothulapati 2a099cb496
Move Prometheus as an Add-On (#4362)
This moves Prometheus as a add-on, thus making it optional but enabled by default. The also make `linkerd-prometheus` more configurable, and allow it to have its own life-cycle for upgrades, configuration, etc.

This work will be followed by documentation that help users configure existing Prometheus to work with Linkerd.

**Changes Include:**
- moving prometheus manifests into a separate chart at `charts/add-ons/prometheus`, and adding it as a dependency to `linkerd2`
- implement the `addOn` interface to support the same with CLI.
- include configuration in `linkerd-config-addons`

**User Facing Changes:**
The default install experience does not change much but for users who have already configured Prometheus differently, would need to apply the same using the new configuration fields present in chart README
2020-07-09 23:29:03 +05:30
Suraj Deshmukh d7dbe9cbff
Fix spelling mistakes using codespell (#4700)
Using following command the wrong spelling were found and later on
fixed:

```
codespell --skip CHANGES.md,.git,go.sum,\
    controller/cmd/service-mirror/events_formatting.go,\
    controller/cmd/service-mirror/cluster_watcher_test_util.go,\
    SECURITY_AUDIT.pdf,.gcp.json.enc,web/app/img/favicon.png \
    --ignore-words-list=aks,uint,ans,files\' --check-filenames \
    --check-hidden
```

Signed-off-by: Suraj Deshmukh <surajd.service@gmail.com>
2020-07-07 17:07:22 -05:00
Tarun Pothulapati cf34a14985
Add a Windows Linkerd cli Test (#4653)
This PR adds a new cli test to see if installation yamls are correctly
generated even on windows, this is important because of all the file
path difference between windows and Linux, and if any code uses a wrong
format might cause the chart generation commands to fail on windows.

This creates a separate workflow for both release and integration.

Also, all the exisiting integration tests are moved in to
/tests/integration to separate from /test/cli as this test does not fall
under integration tests category
2020-07-02 23:13:57 +05:30
Naseem 361d35bb6a
feat: add log format annotation and helm value (#4620)
* feat: add log format annotation and helm value

Json log formatting has been added via https://github.com/linkerd/linkerd2-proxy/pull/500
but wiring the option through as an annotation/helm value is still
necessary.

This PR adds the annotation and helm value to configure log format.

Closes #2491

Signed-off-by: Naseem <naseem@transit.app>
2020-07-02 10:08:52 -05:00
Mayank Shah f00c17e52a
conformance validation: Refactor install test helpers (#4681)
* Refactor install test helpers

- Move testResourcesPostInstall to testutil.TestResourcesPostInstall
- Move exerciseTestAppEndpoint to testutil.ExerciseTestAppEndpoint

Signed-off-by: Mayank Shah <mayankshah1614@gmail.com>

* Trigger CI

Signed-off-by: Mayank Shah <mayankshah1614@gmail.com>
2020-06-29 14:29:41 -07:00
Kevin Leimkuhler 4372ed56dd
Isolate tests by cluster and make run interface simpler (#4593)
## Summary

Change the default behavior of integration tests to be isolated by cluster.
Additionally, make running one or all tests easier than the current process.

These changes are explained more in the [Testing
RFC](https://github.com/linkerd/rfc/blob/master/design/0004-isolated-integration-tests.md)

## Changes

This is a script used only by Linkerd developers, but there is a lot of useful
usage examples and explanations in `bin/tests --help` output:

```
Run Linkerd integration tests.

Optionally specify one of the following tests: [upgrade helm helm-upgrade uninstall deep external-issuer]

Usage:
    tests [--images] [--images-host ssh://linkerd-docker] [--name test-name] [--skip-kind-create] /path/to/linkerd

Examples:
    # Run all tests in isolated clusters
    tests /path/to/linkerd

    # Run single test in isolated clusters
    tests --name test-name /path/to/linkerd

    # Skip KinD cluster creation and run all tests in default cluster context
    tests --skip-kind-create /path/to/linkerd

    # Load images from tar files located under the 'image-archives' directory
    # Note: This is primarly for CI
    tests --images /path/to/linkerd

    # Retrieve images from a remote docker instance and then load them into KinD
    # Note: This is primarly for CI
    tests --images --images-host ssh://linkerd-docker /path/to/linkerd

Available Commands:
    --name: the argument to this option is the specific test to run
    --skip-kind-create: skip KinD cluster creation step and run tests in an existing cluster.
    --images: (Primarily for CI) use 'kind load image-archive' to load the images from local .tar files in the current directory.
    --images-host: (Primarily for CI) the argument to this option is used as the remote docker instance from which images are first retrieved (using 'docker save') to be then loaded into KinD. This command requires --images.
```

### Run all tests

Old:

```bash
bin/test-run $PWD/bin/linkerd
```

New:

```bash
bin/tests $PWD/bin/linkerd
```

### Run single test (upgrade for example):

Current:

```bash
. bin/_test-run.sh
init_test_run $PWD/bin/linkerd
upgrade_integration_tests
```

New:

```bash
bin/tests --name upgrade $PWD/bin/linkerd
```

### Run tests in isolated KinD clusters

Current: Not possible without running single tests in newly created clusters
manually

New:

```bash
bin/tests $PWD/bin/linkerd
```

### Run tests in isolated namespaces on an existing cluster

Old:

```bash
bin/test-run $PWD/bin/linkerd
```

New:

```bash
bin/tests --skip-kind-create $PWD/bin/linkerd
```

## CI

`kind_integration` has been updated so that it does not create a KinD cluster as
part of its test setup.

`cloud_integration` passes the `--skip-kind-create` flag so that the tests are
run serially in a non-KinD cluster.


Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
2020-06-24 17:06:29 -04:00
Zahari Dichev 904f146558
Multicluster install integration test (#4540)
This PR adds multicluster components to the integration tests.

The existing tests have been modified to pass the `--multicluster` flag so that the entire integration test suite runs with multicluster components.

Currently, the upgrade tests do not have multicluster components installed, but this will be done in a follow-up PR. 

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
2020-06-24 14:32:22 -04:00
Lutz Behnke 846d2f11d4
Add support for Helm configuration of per-component proxy resources requests and limits (#4226)
Signed-off-by: Lutz Behnke <lutz.behnke@finleap.com>
2020-06-24 12:54:27 -05:00
Mayank Shah 7f29717a64
Refactor helper functions from `inject` integration tests (#4644)
move `applyPatch` `useTestImageTag`, `validateInject``getProxyContainers` as global functions to be used!
2020-06-22 23:15:52 +05:30
Alejandro Pedraza 27b2838c52
Post-2.8.0 integration test cleanup (#4641)
* Post-2.8.0 integration test cleanup

We had some code for testing upgrades from pre-2.8.0 stables that took
care of creating the non-existent `linkerd-smi-metrics` SA, which is no
longer necessary.

I also had missed many spots in test/install_test.go from #4623
2020-06-22 09:09:04 -05:00
Alejandro Pedraza c8c5980d63
Integration tests: Warn (instead of erroring) upon pod restarts (#4623)
* Integration tests: Warn (instead of erroring) upon pod restarts

Fixes #4595

Don't have integration tests fail whenever a pod is detected to have
restarted just once. For now we'll be just logging this out and creating
a warning annotation for it.
2020-06-18 06:08:05 -05:00
Kevin Leimkuhler f6bd722e2c
Fix install-pr script (#4610)
* Fix install-pr script
* Add image-archives path to commands to use the files

Signed-off-by: Kevin Leimkuhler <kevin@kleimkuhler.com>
Signed-off-by: Charles Pretzer <charles@buoyant.io>
Co-authored-by: Charles Pretzer <charles@buoyant.io>
2020-06-17 21:32:01 -07:00