Overview of the changes:
- Adding ability to verify that expected output occurs a number of times consecutively. This is needed for https://github.com/istio/istio.io/pull/8402.
- Moving snapshot checking logic to Go code so that it can be separated out into separate test steps, which are timed and contain their own output directories/files. This makes the code cleaner and also makes the snapshot logic more transparent.
- Updating debug.sh to use newer bash syntax that allows it to dynamically select a free file descriptor. Without this, I was seeing all commands echoed to my console in goland.
This required some other changes WRT verification:
- Change __cmp_like to allow for not accepting <pending> for an IP address.
- Change __verify_with_retry to use a timeout rathan than number of retries. This is a more intuitive interface and aligns with the way we do retries in istio/istio. I also got rid of exponential backoff and allow both the timeout and delay between retries to be configured.
This was noticed when investigating why errors are not causing the test to fail. It's not clear if this is related to the problem, but the existing logic was definitely atypical.
* test framework enhancement for multicluster doc test
* simplify multicluster test framework with setup=multicluster directive
* restore documentation comment
* Update istio version for testing
* Framework changes in base istio required changes
* Work on whitelist
* Remove whitelist and move to a new feature
* Another istio.io/istio update
* Another framework change - remove test suite name
When tests are run, shell script debugging is enabled and written to a
file called "<script_name>_debug.txt" in the same directory as the
"<script_name>_output.txt" file.
Eric Van Norman
John Howard
Frank Budinsky
Nathan Mittler
Suchith J N
Hongyi Zhang
Hongyi Zhang
Justin Pettit