Commit Graph

28 Commits

Author SHA1 Message Date
Frank Budinsky d9970beaa3
gateway-api: enable mutual tls test (#12156)
* gateway-api: enable mutual tls test

* verify_failure should retry

* fix
2022-10-31 13:27:43 -07:00
Frank Budinsky cb70a40db3
Fix verify_like matching token prefix not ending in - (#11693)
* Fix verify_like matching token prefix not ending in -

* change test
2022-08-09 09:59:37 -07:00
Yangmin Zhu 84a63d0e33
add test for the dry-run task (#10526)
* add test for the dry-run task

* Update content/en/docs/tasks/security/authorization/authz-dry-run/index.md

Co-authored-by: Frank Budinsky <frankb@ca.ibm.com>

Co-authored-by: Frank Budinsky <frankb@ca.ibm.com>
2021-12-08 07:50:27 -08:00
Nathan Mittler 616f45b1d5
Several test framework improvements (#8524)
Overview of the changes:

- Adding ability to verify that expected output occurs a number of times consecutively. This is needed for https://github.com/istio/istio.io/pull/8402.

- Moving snapshot checking logic to Go code so that it can be separated out into separate test steps, which are timed and contain their own output directories/files. This makes the code cleaner and also makes the snapshot logic more transparent.

- Updating debug.sh to use newer bash syntax that allows it to dynamically select a free file descriptor. Without this, I was seeing all commands echoed to my console in goland.
2020-11-23 09:23:54 -05:00
Nathan Mittler 132c2375bc
Add wait for gateway to multicluster tests (#8346)
This required some other changes WRT verification:

- Change __cmp_like to allow for not accepting <pending> for an IP address.

- Change __verify_with_retry to use a timeout rathan than number of retries. This is a more intuitive interface and aligns with the way we do retries in istio/istio. I also got rid of exponential backoff and allow both the timeout and delay between retries to be configured.
2020-10-26 06:45:38 -07:00
Nathan Mittler b28bb174de
Fixes for multicluster tests (#8309) 2020-10-21 06:46:49 -07:00
Nathan Mittler f9a9f54eba
[Test Framework] Fixes to support multicluster (#8253)
* [Test Framework] Fixes to support multicluster

* use current context from each kubeconfig file.

* addressing comments
2020-10-02 14:05:09 -07:00
Eric Van Norman cc1a34f99c
Update test reference - master (#8103)
* Update test reference

* Test framework changes

* Another required change

* Update Tag to 1.8

* Pick istio/istio commit that actually exists

* Disable ISTIO_META_DNS_CAPTURE

* Add --skip-confirmation to istioctl installl commands

* Increase test timeout. First pass at fixes.

* Update to later istio/istio that fixes DNS and minor fixes

* test fixes

* Pick up go.mod `replace` changes from #8118

* Fix istioctl-analayze and mirror

* Fix mtls-migration test

* Update istio to include commit to fix egress

* Re-enable verify with fix

* Update istio/istio ref for egress fix

* Fix tasks/security/authorization/authz-td-migration - remove ns

* Shorten wait timeout so tests complete in under an hr

* Let tests continue after wait timeout

* Fix --skip-confirmation to -y and use yes | in tests

* revert yes | to echo y |

* Additional echo y fix

* Code review comments

* Change verify from same to contains as k8s 1.19 has extra warning lines.
2020-09-16 11:44:20 -07:00
Eric Van Norman 3795f41e52
Add back verify - grep out CR failures (#8141) 2020-09-15 07:36:22 -07:00
Frank Budinsky 8799da5317
Disable cluster cleanup check until missing CRDs at start of profile_… (#8137)
* Disable cluster cleanup check until missing CRDs at start of profile_none problem is fixed

* lint
2020-09-14 14:13:26 -07:00
Frank Budinsky 82ceae7d73
Accessing external services task test and more egress task cleanup (#8064)
* Egress tasks cleanup and final test

* cleanup

* fix lint

* regen

* strip trailing space from output

* lint

* lint

* review comment

* regen

* fixes

* ignore space

* wait for istiod

* restart sleep

* regen

* fix sleep restart

* debug

* fix check

* fix

* more debug

* more debug

* cleanup and disable last check
2020-09-09 14:39:26 -07:00
Frank Budinsky 4801cd62fa
Verify test cleanup (#7889)
* Verify test cleanup

* disable istio-system checks

* more tolerant checks

* fix lint

* more lint

* limited check cleanup

* always check

* snapshot tweak
2020-08-11 08:19:21 -07:00
Frank Budinsky 412f03105c
Kubernetes Ingress Test + fixes (#7662)
* Kubernetes Ingress Test + fixes

* cleanup

* fix circuit-breaking test

* fix lint

* remove -it
2020-07-06 07:54:35 -07:00
Frank Budinsky 67e181ce34
More test framework doc (#7635)
* More test framework doc

* Update tests/README.md

Co-authored-by: Hongyi Zhang <hongyizhang@google.com>

* Update tests/README.md

Co-authored-by: Hongyi Zhang <hongyizhang@google.com>

* tweak

* tweak

* increase default retry

* increase retries to 8

* fix test failing

* disable istioctl install

* fix lint

Co-authored-by: Hongyi Zhang <hongyizhang@google.com>
2020-06-30 11:06:42 -07:00
Frank Budinsky b3e79edd95
Rename _run_and_verify_xxx functions to _verify_xxx (#7452)
* Remove _verify_ calls

* Rename _run_and_verify functions to _verify

* Update tests/configuration/scripts/liveness_and_readiness_probes.sh

* error exit

* cleaner errexit
2020-06-03 06:24:29 -07:00
Frank Budinsky 3093ddc581
New verify function: _run_and_verify_lines (#7451)
* New verify function: _run_and_verify_lines

* fix lint
2020-06-02 11:23:43 -07:00
Frank Budinsky f129da0ee7
Verify function improvements (#7440) 2020-06-01 11:16:15 -07:00
Frank Budinsky 77d3dc6bf1
Fix mirroring test (#7399)
* Fix mirroring test

* fix typo

* fixes

* export vars

* lint
2020-05-26 12:57:19 -07:00
Justin Pettit 0fb1c50a49
verify.sh: Don't exit on error in _run_and_verify_*() funcs. (#7359)
* verify.sh: Don't exit on error in _run_and_verify_*() funcs.

Most tests include "set -e", which causes the script to exit if a
statement returns a non-true return value.  In some cases, the command
may exit with a non-true return value, but we want to retry the command
later.  We want to temporarily disable that "errexit" behavior.

* verify.sh: Clarify __verify_with_retry() comment and variable name.
2020-05-19 18:06:21 -07:00
Justin Pettit 7782eb1d0a
Authz jwt (#7338)
* snip.py: Replace github file token with release-specific URL.

* verify.sh: Show the expected output as well as the actual output.

* snip.py: Update the githubfile regex to not include email addresses.

When generating snip scripts, pairs of "@" signs indicate a link to
GitHub repo content.  However, JWT attribute values contained pairs of
email addresses such as:

    `testing@secure.istio.io/testing@secure.istio.io`

which would be treated as an email address and mangled.  This commit
rewrites the regex to not match on email addresses.

* Add authz-jwt user guide test.
2020-05-17 13:51:45 -04:00
Frank Budinsky 90103ae557
Cleanup circuit breaking test (#7335)
* Cleanup circuit breaking test

* remove old files

* updated expected
2020-05-15 16:59:58 -07:00
Justin Pettit 8cacec48de
Authz deny (#7315)
* Add authz-tcp user guide test.

* Add run and verify functions for user doc tests.
2020-05-14 16:10:20 -07:00
Frank Budinsky 4f7622e3aa
More doc testing (#7253)
* Ingress control test

* Ingress control test

* fix lint

* fix package

* fix path

* improve _verify_like

* curl -s

* debug

* fix compare

* remove flaky verify call

* fix path

* regen
2020-05-12 05:58:07 -07:00
Frank Budinsky affe99f97a
More traffic management task tests (#7250)
* More traffic management task tests

* lint errors

* fix kubectl output

* fix arg check

* login to bookinfo

* curl --user

* temporary kludge for login not working

* fault injection test

* rename test

* fix cleanup

* timeouts tests

* regen

* make sure sleep pods are gone
2020-05-10 15:05:22 -07:00
Frank Budinsky f45bc9e1a8
Remove obsolete '# ' uses from all tests (#7233) 2020-05-06 19:17:11 -04:00
Frank Budinsky a7107678f5
Convert TCP traffic shifting test (#7131)
* Convert TCP traffic shifting test

* lint errors

* regen

* test disable spellcheck

* temporary disable gen-check

* disable inappropriate bash lint errors

* disable gen check

* one more regen

* reenable gen check

* remove -it options

* revert commands

* fix app health check outdated
2020-04-22 18:26:27 -07:00
Frank Budinsky a9541a2b42
Convert bookinfo test to new model (#7127)
* Convert bookinfo test to new model

* missing include

* Fix kubectl output

* fix elapsed time regex
2020-04-22 11:36:01 -07:00
Frank Budinsky 1e7a781bdf
Add built-in verify functions to test framework (#7110)
* Add builtin verify functions to test framework

* fix lint

* tweaks

* fix bold
2020-04-20 13:29:46 -07:00