Commit Graph

63 Commits

Author SHA1 Message Date
Xiaopeng Han d73302933b
add missing component in _rewrite_helm_repo (#14276) 2023-12-06 09:44:02 -08:00
SRodi dd4f01fa26
Create generic args.md with common args consumed by test scripts (#13087)
* generic-args-md

* create generic boilerplates/args.md
* include gw_api_version, full_version, previous_version
* update install/operator/test.sh to source and use new vars
* update test/util/gateway-api.sh to source and use new gw var
* remove gateway-api-version.md
* relates to #13085

* fix linting fail for args.md

* remove sed command for revision string, use param expansion
2023-04-21 08:59:23 -07:00
Frank Budinsky cd2608bf8b
Fix gtw wait in util functions (#12953) 2023-03-27 08:42:58 -07:00
Eric Van Norman abf5268f8a
Update test reference to latest istio commit (#12895)
* Update test reference to latest istio commit

* make clean g&& go mod tidy

* Use tests/utils to delete sample

* Move some test script to use snippets

* Add _wait

* Remove extraneous source
2023-03-15 14:17:18 -07:00
zirain 6c486aae1d
fix lint (#12639) 2023-02-12 14:35:06 -08:00
Eric Van Norman cc76a09425
Update istio.io test reference to 1.17.0-beta.2 (#12591)
* Update istio.io test reference to 1.17.0-beta.2

* Update URL
2023-02-07 09:23:00 -08:00
Eric Van Norman deada82e56
Is the first sllep delete affecting the later start? (#12495) 2023-01-20 13:31:13 -08:00
Frank Budinsky b4ec649054
Add gateway api instructions to external control plane doc (#12463)
* Add gateway api instructions to external control plane doc

* regen and lint

* fix test

* more test fixes
2023-01-16 08:45:30 -08:00
Frank Budinsky 69981b01b1
gateway-api: add gateway api instruction to request routing task (#12204)
* gateway-api: add gateway api instruction to request routing task

* lint

* regen

* improve gamma tip/warning

* lint

* add port

* fix samples.sh

* fixes
2022-11-10 14:32:57 -05:00
kezhenxu94 7d43f35867
Add documentations for SkyWalking integration and task (#11683)
* Add documentations for SkyWalking integration and task

* Add script to undeploy skywalking

* Clean up istio namespace

* Update index.md

* Address review comments

* Apply suggestions from code review

Co-authored-by: Eric Van Norman <ericvn@us.ibm.com>

Co-authored-by: Eric Van Norman <ericvn@us.ibm.com>
2022-11-03 09:09:26 -07:00
Frank Budinsky d9970beaa3
gateway-api: enable mutual tls test (#12156)
* gateway-api: enable mutual tls test

* verify_failure should retry

* fix
2022-10-31 13:27:43 -07:00
Frank Budinsky f31d54066f
Fix broken sni-passthrough doc test (#12109)
* Fix broken sni-passthrough doc test

* remove exit
2022-10-18 00:24:38 -07:00
Frank Budinsky dd6b1c09cc
gateway-api doc: ingress-sni-passthrough (#12087)
* gateway-api doc: ingress-sni-passthrough

* use kustomize for crds

* debug

* more debug

* use standard crd install

* try profile=none

* uninstall

* confirm install

* disable test for now

* regen
2022-10-17 10:39:10 -07:00
Eric Van Norman 20491e3165
use short_codes for gateway api version (#12042)
* use short_codes for gateway api version and tpye

* Update function name. Forcing name doesn't work for boilerplates?

* Fix lint

* Remove k8s_gateway_api_type

* Add update-gateway-version mkaefile target

* Fix version in test string

* Simplify id
2022-10-04 12:07:49 -07:00
Frank Budinsky 5e89d73d81
Add gateway-api instructions to getting-started and bookinfo docs (#12021)
* Add gateway-api instructions to getting-started and bookinfo docs

* fix lint

* fix cleanup

* regen

* fix lint

* fix setup

* fix include path

* v1alpha2 -> v1beta1

* regen

* install k8s crds

* fix lint

* fix cleanup

* lint

* workaround duplicate crd problem

* fix cmd

* fixes

* lint

* more fixes

* typo

* regen

* uncomment failing snip

* no choice in getting-started

* lint

* typo

* remove duplicate section title
2022-10-03 12:11:48 -07:00
zirain 4830422f75
fix open telemetry doc (#11732)
* fix OpenTelemetry doc

* replace istio.io/istio for test

* fix cleanup_otel_sample

* update test reference

Signed-off-by: hejianpeng <hejianpeng2@huawei.com>

* fix gen

Signed-off-by: hejianpeng <hejianpeng2@huawei.com>

* fix otel-provider

* remove extra changes in the go.*

Signed-off-by: hejianpeng <hejianpeng2@huawei.com>
2022-08-30 07:00:38 -07:00
Frank Budinsky cb70a40db3
Fix verify_like matching token prefix not ending in - (#11693)
* Fix verify_like matching token prefix not ending in -

* change test
2022-08-09 09:59:37 -07:00
Daniel Hawton 63d5724fde
Helm and gateway tests (#11276)
* add tests to gateway setup

* manually cleanup the minimal istio install

* Add canary upgrade test

* convert rewrite-repo to a helper function

* upgrade helm test

* lint fixes

* left over validatingwebhook from a prior test

* remove boilerplate check

* undo elided pod names

* gen snip

* Remove validatingwebhookconfigurations deletion

* remove webhook configs pending fix in istio

* remove webhook configs pending fix in istio

* revert some changes

* remove temp webhook removals

* remove revision labeled mutating webhooks

* revert revision-tags-middle change

* make gen
2022-06-07 08:34:59 -07:00
zirain 7a90100810
add otel telemetry doc (#10977)
* add otel telemetry doc

* fix nit

* fix gen

* update content

* remove patch meshconfig

* fix test.sh

* fix probability of failure
2022-03-15 11:45:41 -07:00
Yangmin Zhu 84a63d0e33
add test for the dry-run task (#10526)
* add test for the dry-run task

* Update content/en/docs/tasks/security/authorization/authz-dry-run/index.md

Co-authored-by: Frank Budinsky <frankb@ca.ibm.com>

Co-authored-by: Frank Budinsky <frankb@ca.ibm.com>
2021-12-08 07:50:27 -08:00
Eric Van Norman eed66e769b
Manual cherry-pick of #10352 (#10358) 2021-09-16 13:59:00 -07:00
Pengyuan Bian fb05d1584f
Revamp CNI doc. (#10090)
* wip

* rewrite install-cni page.

* Add troubleshoot guide.

* Add doc test.

* fix test.

* fix test.

* validation log message.

* update.

* update.

* fix.

* update.

* incorporate supportability review feedback.

* Update content/en/docs/ops/diagnostic-tools/cni/index.md

Co-authored-by: craigbox <craigbox@google.com>

* Update content/en/docs/ops/diagnostic-tools/cni/index.md

Co-authored-by: craigbox <craigbox@google.com>

* Apply suggestions from code review

Co-authored-by: craigbox <craigbox@google.com>

* fix.

Co-authored-by: craigbox <craigbox@google.com>
2021-08-01 13:40:03 -07:00
Eric Van Norman d9658dae3d
Restore istioctl experimental wait in _wait_for_istio() (#9426) 2021-04-05 13:02:04 -07:00
Eric Van Norman 87e8f20093
Ignore error on first kiali apply (#9048) 2021-02-23 19:25:19 -08:00
Eric Van Norman a38324b63e
Fix istio.io tests when moving to later kubectl (#9046) 2021-02-23 16:03:57 -08:00
John Howard 51650924c9
VMs: add doc test, re-org examples (#8892)
* Add documentation test for VMs

* Remove single and multinetwork docs, move up a level

* Fixes

* fix tests

* fix test
2021-02-04 21:31:44 -08:00
Eric Van Norman 5d984eb560
Update istio.io/istio ref to 1.9.0-beta.0 (#8562)
* Update istio/istio ref to latest master

* Move to latest before testing.

* Update release in Makefile

* Fix some tests

* Update to latest isti.io/istio again

* Update to latest istio.io/istio

* Update to latest istio.io/istio

* Update ref to latest master

* Fix instioctl-analyze

* Add back @howardjohn commit I inadvertantly deleted

* Fix lint

* Pick up new stio: fix empty iop read from stdin for operator

* go mod tidy without itermediate go gets

* Update to current stats-filter's

* Needed another 'make gen'

* More go.* changes for changes in ref'd istio/istio

* Update istio to 1.9.0-beta.0
2021-01-19 11:57:22 -08:00
Nathan Mittler 616f45b1d5
Several test framework improvements (#8524)
Overview of the changes:

- Adding ability to verify that expected output occurs a number of times consecutively. This is needed for https://github.com/istio/istio.io/pull/8402.

- Moving snapshot checking logic to Go code so that it can be separated out into separate test steps, which are timed and contain their own output directories/files. This makes the code cleaner and also makes the snapshot logic more transparent.

- Updating debug.sh to use newer bash syntax that allows it to dynamically select a free file descriptor. Without this, I was seeing all commands echoed to my console in goland.
2020-11-23 09:23:54 -05:00
Nathan Mittler 132c2375bc
Add wait for gateway to multicluster tests (#8346)
This required some other changes WRT verification:

- Change __cmp_like to allow for not accepting <pending> for an IP address.

- Change __verify_with_retry to use a timeout rathan than number of retries. This is a more intuitive interface and aligns with the way we do retries in istio/istio. I also got rid of exponential backoff and allow both the timeout and delay between retries to be configured.
2020-10-26 06:45:38 -07:00
Nathan Mittler b28bb174de
Fixes for multicluster tests (#8309) 2020-10-21 06:46:49 -07:00
Nathan Mittler f9a9f54eba
[Test Framework] Fixes to support multicluster (#8253)
* [Test Framework] Fixes to support multicluster

* use current context from each kubeconfig file.

* addressing comments
2020-10-02 14:05:09 -07:00
Eric Van Norman 70d361b30d
Try different timeout for tasks/security/authorization/authz-ingress (#8163)
* Try longer timeout for tasks/security/authorization/authz-ingress/test.sh

* Go back to old 5m timeout and add failure on timeout back in

* Test with individual wait_for_istio call updates

* Revert to simply changing timeout (but keep duration in output)

* Fix call

* Comment out istioctl wait call

* Add TODO remark

* Fix lint error

* Add temporary sleep until _wait_for_istio is re-enabled

* Add wait for sleep deployment to complete
2020-09-22 19:25:13 -07:00
Eric Van Norman cc1a34f99c
Update test reference - master (#8103)
* Update test reference

* Test framework changes

* Another required change

* Update Tag to 1.8

* Pick istio/istio commit that actually exists

* Disable ISTIO_META_DNS_CAPTURE

* Add --skip-confirmation to istioctl installl commands

* Increase test timeout. First pass at fixes.

* Update to later istio/istio that fixes DNS and minor fixes

* test fixes

* Pick up go.mod `replace` changes from #8118

* Fix istioctl-analayze and mirror

* Fix mtls-migration test

* Update istio to include commit to fix egress

* Re-enable verify with fix

* Update istio/istio ref for egress fix

* Fix tasks/security/authorization/authz-td-migration - remove ns

* Shorten wait timeout so tests complete in under an hr

* Let tests continue after wait timeout

* Fix --skip-confirmation to -y and use yes | in tests

* revert yes | to echo y |

* Additional echo y fix

* Code review comments

* Change verify from same to contains as k8s 1.19 has extra warning lines.
2020-09-16 11:44:20 -07:00
Eric Van Norman 3795f41e52
Add back verify - grep out CR failures (#8141) 2020-09-15 07:36:22 -07:00
Frank Budinsky 8799da5317
Disable cluster cleanup check until missing CRDs at start of profile_… (#8137)
* Disable cluster cleanup check until missing CRDs at start of profile_none problem is fixed

* lint
2020-09-14 14:13:26 -07:00
Frank Budinsky 82ceae7d73
Accessing external services task test and more egress task cleanup (#8064)
* Egress tasks cleanup and final test

* cleanup

* fix lint

* regen

* strip trailing space from output

* lint

* lint

* review comment

* regen

* fixes

* ignore space

* wait for istiod

* restart sleep

* regen

* fix sleep restart

* debug

* fix check

* fix

* more debug

* more debug

* cleanup and disable last check
2020-09-09 14:39:26 -07:00
Suchith J N 28d61ab090
Automated test - Remotely access telemetry addons (both secure and insecure ways) (#8042)
* automated test - insecure remote access to telemetry addons

* Change profile to demo and add snippets to clean up DestinationRule (besides Gateway and VirtualService)

* automated doc test = secure remote access to telemetry addons through gateway

* Automated test for secure access of common telemetry addons and factor out common snippets to test_utils.sh

* moved test_utils to tests/util/addons.sh so that it can be reused across tests

* refactor addons with a more flexible deploy/undeploy function and fix naming of functions to access addons

* fix cleanup after tests

* fix sourcing

* remove verification for istio config deployment

* don't assume the existence of load-balancer in CI environment, but use _set_ingress_environment_variables instead of snippet in the documentation page

* add verification steps back

* fix output snippets and add verifications
2020-09-01 07:34:10 -07:00
Suchith J N 4b601f607e
automated test for zipkin dashboard and modify snippet generation to take boilerplate into account (#8003)
* automated test for viewing traces from zipkin dashboard (expectation is that traces are seen correctly)

* fix shellcheck linting errors

* remove verification of traces and just check if zipkin dashboard is accessible through port-forward

* Modify snippet generation logic to take {{< boilerplate >}} into account

* snippets for boilerplate

* fix shellcheck (linting) issues

* snippets generated with new logic

* automated test - check if zipkin dashboard is accessible by port-forwarding (as we cannot verify screenshot and the trace is already verified in istio/istio integration test

* clean up ZIPKIN_URL comment

* Address code review comments
1. Simplify boilerplate snippet filenames (remove .md_snips)
2. Sourced filenames should be in double quote(") instead of single quote(')

* fix gencheck_istio flake

* do not generate or include boilerplate snippets if there is not {{<text>}} in them

* fix linting errors in snip.py

* fix bug - filter out boilerplates without snippets very early
2020-08-26 08:37:50 -07:00
Frank Budinsky 4801cd62fa
Verify test cleanup (#7889)
* Verify test cleanup

* disable istio-system checks

* more tolerant checks

* fix lint

* more lint

* limited check cleanup

* always check

* snapshot tweak
2020-08-11 08:19:21 -07:00
Mitch Connors cb788bb993
Print proxy status when debug fails for root cause (#7736) 2020-07-15 13:03:30 -07:00
Frank Budinsky 412f03105c
Kubernetes Ingress Test + fixes (#7662)
* Kubernetes Ingress Test + fixes

* cleanup

* fix circuit-breaking test

* fix lint

* remove -it
2020-07-06 07:54:35 -07:00
Frank Budinsky 67e181ce34
More test framework doc (#7635)
* More test framework doc

* Update tests/README.md

Co-authored-by: Hongyi Zhang <hongyizhang@google.com>

* Update tests/README.md

Co-authored-by: Hongyi Zhang <hongyizhang@google.com>

* tweak

* tweak

* increase default retry

* increase retries to 8

* fix test failing

* disable istioctl install

* fix lint

Co-authored-by: Hongyi Zhang <hongyizhang@google.com>
2020-06-30 11:06:42 -07:00
Frank Budinsky 01c4cce951
Add proper wait for config calls to tests (#7620)
* Add wait for config calls to tests

* two more
2020-06-24 09:48:29 -07:00
Hongyi Zhang 420a54447b
Migration to a new framework for doc testing (#7465)
* add an example task to test

* main test function: save progress

* a working example: routing request

* improve log info and error handling

* introduce makefile

* run each test as a subtest; remove common setup from test.sh

* add another test.sh: fault-injection

* improve error handling

* check test environment

* add two more test.sh files

* fix make command for istio setup

* update two test.sh files from upstream

* add comments and update README.md

* update test.sh files from upstream

* support multiple test names

* update README

* update README.md for new framework

* remove documentation of migration steps

* undo format changes

* change separation line to '# @cleanup'

* move go code and makefile from content/ to tests/

* change package name

* make for loop more readable

* change the set of auto-sourced scripts

* add docs for all functions

* approach to deal with folders with the same name

* minor fixes to ensure everything still runs

* fix make gen error

* add a TIMEOUT argument

* make sure util/debug.sh works with new framework

* make lint-go happy

* [BIG CHANGE] allow different istio setup configs

* make linters happy

* make linters happier

* changed wording and function orders

* make error return as the 2nd argument

* add TODOs

* Update content/en/docs/tasks/traffic-management/traffic-shifting/test.sh

Co-authored-by: Frank Budinsky <frankb@ca.ibm.com>

* Update tests/README.md

Co-authored-by: Frank Budinsky <frankb@ca.ibm.com>

* Update tests/README.md

Co-authored-by: Frank Budinsky <frankb@ca.ibm.com>

* Update tests/README.md

Co-authored-by: Frank Budinsky <frankb@ca.ibm.com>

* Update tests/README.md

Co-authored-by: Frank Budinsky <frankb@ca.ibm.com>

* Update tests/README.md

Co-authored-by: Frank Budinsky <frankb@ca.ibm.com>

* Update tests/README.md

Co-authored-by: Frank Budinsky <frankb@ca.ibm.com>

* only test english docs

* Update tests/README.md

Co-authored-by: Frank Budinsky <frankb@ca.ibm.com>

* allow test.sh as suffix

* move adding setup configs to tests/setup

* recommend full paths

* Update tests/README.md

Co-authored-by: Frank Budinsky <frankb@ca.ibm.com>

* require full test paths

Co-authored-by: Frank Budinsky <frankb@ca.ibm.com>
2020-06-09 14:11:53 -07:00
Frank Budinsky b3e79edd95
Rename _run_and_verify_xxx functions to _verify_xxx (#7452)
* Remove _verify_ calls

* Rename _run_and_verify functions to _verify

* Update tests/configuration/scripts/liveness_and_readiness_probes.sh

* error exit

* cleaner errexit
2020-06-03 06:24:29 -07:00
Frank Budinsky 3093ddc581
New verify function: _run_and_verify_lines (#7451)
* New verify function: _run_and_verify_lines

* fix lint
2020-06-02 11:23:43 -07:00
Justin Pettit c709c976da
Move generally useful "sample_" functions to new "helpers.sh". (#7441) 2020-06-01 13:09:30 -07:00
Frank Budinsky f129da0ee7
Verify function improvements (#7440) 2020-06-01 11:16:15 -07:00
Frank Budinsky 55a4416230
Remove temporary login kludge (#7429) 2020-05-29 09:04:07 -07:00
Frank Budinsky 77d3dc6bf1
Fix mirroring test (#7399)
* Fix mirroring test

* fix typo

* fixes

* export vars

* lint
2020-05-26 12:57:19 -07:00