diff --git a/README.md b/README.md index bc2fce37..35931d95 100644 --- a/README.md +++ b/README.md @@ -127,12 +127,11 @@ spec: interval: 30s # external checks (optional) webhooks: - - name: integration-tests - url: http://podinfo.test:9898/echo - timeout: 1m + - name: load-test + url: http://flagger-loadtester.test/ + timeout: 5s metadata: - test: "all" - token: "16688eb5e9f289f1991c" + cmd: "hey -z 1m -q 10 -c 2 http://podinfo.test:9898/" ``` The canary analysis is using the following promql queries: @@ -211,6 +210,13 @@ kubectl apply -f ${REPO}/artifacts/canaries/deployment.yaml kubectl apply -f ${REPO}/artifacts/canaries/hpa.yaml ``` +Deploy the load testing service to generate traffic during the canary analysis: + +```bash +kubectl -n test apply -f ${REPO}/artifacts/loadtester/deployment.yaml +kubectl -n test apply -f ${REPO}/artifacts/loadtester/service.yaml +``` + Create a canary promotion custom resource (replace the Istio gateway and the internet domain with your own): ```bash @@ -239,7 +245,7 @@ Trigger a canary deployment by updating the container image: ```bash kubectl -n test set image deployment/podinfo \ -podinfod=quay.io/stefanprodan/podinfo:1.2.1 +podinfod=quay.io/stefanprodan/podinfo:1.4.0 ``` Flagger detects that the deployment revision changed and starts a new canary analysis: diff --git a/artifacts/canaries/canary.yaml b/artifacts/canaries/canary.yaml index e9678b0a..d8eb9767 100644 --- a/artifacts/canaries/canary.yaml +++ b/artifacts/canaries/canary.yaml @@ -51,9 +51,8 @@ spec: interval: 30s # external checks (optional) webhooks: - - name: integration-tests - url: https://httpbin.org/post - timeout: 1m + - name: load-test + url: http://flagger-loadtester.test/ + timeout: 5s metadata: - test: "all" - token: "16688eb5e9f289f1991c" \ No newline at end of file + cmd: "hey -z 1m -q 10 -c 2 http://podinfo.test:9898/" \ No newline at end of file diff --git a/docs/gitbook/how-it-works.md b/docs/gitbook/how-it-works.md index 00adeef4..760ae8c9 100644 --- a/docs/gitbook/how-it-works.md +++ b/docs/gitbook/how-it-works.md @@ -281,4 +281,45 @@ Response status codes: On a non-2xx response Flagger will include the response body (if any) in the failed checks log and Kubernetes events. +### Load Testing + +For workloads that are not receiving constant traffic Flagger can be configured with a webhook, +that when called, will start a load test for the target workload. +If the target workload doesn't receive any traffic during the canary analysis, +Flagger metric checks will fail with "no values found for metric istio_requests_total". + +Flagger comes with a load testing service based on [rakyll/hey](https://github.com/rakyll/hey) +that generates traffic during analysis when configured as a webhook. + +First you need to deploy the load test runner in a namespace with Istio sidecar injection enabled: + +```bash +export REPO=https://raw.githubusercontent.com/stefanprodan/flagger/master + +kubectl -n test apply -f ${REPO}/artifacts/loadtester/deployment.yaml +kubectl -n test apply -f ${REPO}/artifacts/loadtester/service.yaml +``` + +When deployed the load tester API will be available at `http://flagger-loadtester.test/`. + +Now you can add webhooks to the canary analysis spec: + +```yaml +webhooks: + - name: load-test-get + url: http://flagger-loadtester.test/ + timeout: 5s + metadata: + cmd: "hey -z 1m -q 10 -c 2 http://podinfo.test:9898/" + - name: load-test-post + url: http://flagger-loadtester.test/ + timeout: 5s + metadata: + cmd: "hey -z 1m -q 10 -c 2 -m POST -d '{test: 2}' http://podinfo.test:9898/echo" +``` + +When the canary analysis starts, Flagger will call the webhooks and the load tester will run the `hey` commands +in the background, if they are not already running. This will ensure that during the +analysis, the `podinfo.test` virtual service will receive a steady steam of GET and POST requests. + diff --git a/docs/gitbook/usage/progressive-delivery.md b/docs/gitbook/usage/progressive-delivery.md index d2805897..ab1d2fff 100644 --- a/docs/gitbook/usage/progressive-delivery.md +++ b/docs/gitbook/usage/progressive-delivery.md @@ -17,6 +17,13 @@ kubectl apply -f ${REPO}/artifacts/canaries/deployment.yaml kubectl apply -f ${REPO}/artifacts/canaries/hpa.yaml ``` +Deploy the load testing service to generate traffic during the canary analysis: + +```bash +kubectl -n test apply -f ${REPO}/artifacts/loadtester/deployment.yaml +kubectl -n test apply -f ${REPO}/artifacts/loadtester/service.yaml +``` + Create a canary custom resource \(replace example.com with your own domain\): ```yaml @@ -70,6 +77,13 @@ spec: # milliseconds threshold: 500 interval: 30s + # generate traffic during analysis + webhooks: + - name: load-test + url: http://flagger-loadtester.test/ + timeout: 5s + metadata: + cmd: "hey -z 1m -q 10 -c 2 http://podinfo.test:9898/" ``` Save the above resource as podinfo-canary.yaml and then apply it: @@ -99,7 +113,7 @@ Trigger a canary deployment by updating the container image: ```bash kubectl -n test set image deployment/podinfo \ -podinfod=quay.io/stefanprodan/podinfo:1.2.1 +podinfod=quay.io/stefanprodan/podinfo:1.4.0 ``` Flagger detects that the deployment revision changed and starts a new rollout: