* wip resource churn
* updates to scenario
* add default 'system' filters, format api_benchmark
* optional filters, remove cruft, fix list urls
* added separate VU params
* re-add old change_configmaps.js, rename new configmap script
* add potential threshold updates, resolve various feedback points
* move script to vai dir
* fix per-scenario thresholds
* final threshold fix
* modify configmap.yaml template to account for new dir structure, have load.go account for new dir structure, added TODO to load.go
* account for new k6 folder structure in k6Run()
* add .helmignore
* fix linter issues
* use sync.Once to ensure file path entries are only cached once per-command execution
* remove debug log lines
* add comments to 'FileEntries' functions
* fix unused assignment
* add relative paths to K6Run() calls
* update providers, add VPC and SG rules, reduce flakiness of k3s/rke2 modules, add toggle for public ips and tunnels
* fix install scripts
* only use prefix list if input var is not null
* reintroduce sudo -s block
* fix distro install
* remove misc cidrs from ssh SG
* remove default of private SG
* tofu/postgres: bugfix: directory already existed, use it
Signed-off-by: Silvio Moioli <silvio@moioli.net>
* tofu/postgres: bugfix: use correct sh syntax
Signed-off-by: Silvio Moioli <silvio@moioli.net>
* tofu/postgres: listen from any addresses to allow use of non-standalone kine (integrated in distros)
Signed-off-by: Silvio Moioli <silvio@moioli.net>
* tofu/postgres: do not install kine standalone
Signed-off-by: Silvio Moioli <silvio@moioli.net>
* tofu/postgres: bugfix: use correct username
Signed-off-by: Silvio Moioli <silvio@moioli.net>
* test_environment: support postgres as kine backend
Signed-off-by: Silvio Moioli <silvio@moioli.net>
* tofu/postgres: make kine password a parameter
Signed-off-by: Silvio Moioli <silvio@moioli.net>
---------
Signed-off-by: Silvio Moioli <silvio@moioli.net>
* first version from Gemini
Prompt used:
Take a look at the @k6 directory scripts, particularly @k6/tests/api_benchmark.js targeting Steve. I need a new k6 benchmark script along the same lines that tests the Watch functionality described in https://github.com/moio/apiserver/blob/watch_readme/README.md and https://github.com/moio/steve/blob/readme_watch/README.md. Specifically, I need to start watching the same resource (default: configmaps) in a configurable namespace (default: scalability-tests) from multiple Steve servers backed by the same Kubernetes API. A parallel scenario should change configmaps at a configurable rate, updating one configmaps at random via PUT to one of the configured Steve servers. There will be some delay between resource.changed events coming from the Steve servers, as they might not always have exactly the same speed. I need to measure and report the delta between the fastest and the slowest Steve (with stats: min, max, avg, p(95)), as well as the delay between a change in a configmap and the time the first Steve observes the change via Watch (with stats) and the time the last Steve observes it (with stats). mode should be the default for this test, but make it configurable to use resources.changes optionally.
Signed-off-by: Silvio Moioli <silvio@moioli.net>
* fix durations and iterations
Signed-off-by: Silvio Moioli <silvio@moioli.net>
* fix cookie passing in the websocket case - api is different there
Signed-off-by: Silvio Moioli <silvio@moioli.net>
* fixup! fix durations and iterations
Signed-off-by: Silvio Moioli <silvio@moioli.net>
* Rework delay computation logic. VUs do not share data, not even global variables, piggyback on the ConfigMap instead
Signed-off-by: Silvio Moioli <silvio@moioli.net>
* spray logging all over
Signed-off-by: Silvio Moioli <silvio@moioli.net>
* Use WebSocket async API
Gemini prompt:
you used the standard k6 websocket api documented here https://grafana.com/docs/k6/latest/using-k6/protocols/websockets/. The problem is that it is blocking, so no more than one websocket is opened at any time. Re-read @k6/tests/steve_watch_benchmark.js considering all manual changes I've made as the good starting point. Migrate it over the new experimental API which is non-blocking, see https://grafana.com/docs/k6/latest/javascript-api/k6-experimental/websockets/
Signed-off-by: Silvio Moioli <silvio@moioli.net>
* fix timeout and error handling
Signed-off-by: Silvio Moioli <silvio@moioli.net>
* add time after creation to not pollute watch measurements
Signed-off-by: Silvio Moioli <silvio@moioli.net>
* improve logging
Signed-off-by: Silvio Moioli <silvio@moioli.net>
* bugfix: ensure ints are ints
Signed-off-by: Silvio Moioli <silvio@moioli.net>
* add settle time for watchers to start watching
Signed-off-by: Silvio Moioli <silvio@moioli.net>
* hardcode watchers vus to 1. What we are interested in measuring is delay of replicas, not load here
Signed-off-by: Silvio Moioli <silvio@moioli.net>
* make change vus parametric
Signed-off-by: Silvio Moioli <silvio@moioli.net>
* add sainty thresholds
Signed-off-by: Silvio Moioli <silvio@moioli.net>
* tear down before setup, in case an earlier run is half finished
Signed-off-by: Silvio Moioli <silvio@moioli.net>
* rework logging so that verbose messages are only output with the verbose option
Signed-off-by: Silvio Moioli <silvio@moioli.net>
* fix id to be truly unique and avoid misassignment of timestamps
Signed-off-by: Silvio Moioli <silvio@moioli.net>
* hardcode resource type for now
Signed-off-by: Silvio Moioli <silvio@moioli.net>
* add the ability to target both Steve and Kubernetes APIs
Signed-off-by: Silvio Moioli <silvio@moioli.net>
* as an additional check, log the time the listener takes to run
Signed-off-by: Silvio Moioli <silvio@moioli.net>
* add kube api watching
Signed-off-by: Silvio Moioli <silvio@moioli.net>
* tune settle time
Signed-off-by: Silvio Moioli <silvio@moioli.net>
* explainer for origin url
Signed-off-by: Silvio Moioli <silvio@moioli.net>
---------
Signed-off-by: Silvio Moioli <silvio@moioli.net>
* bigger arp cache needed for many hosts in same network
* inotify limits needed, e.g. for k3k vclusters
Co-authored-by: Mario Manno <mario.manno@suse.com>
Using unique names allows multiple kubeconfigs to be specified in
KUBECONFIG or merged into single kubeconfig file.
```
KUBECONFIG="$PWD/upstream.yaml:$PWD/downstream-0-0.yaml"
kubectl config view --flatten --raw > kubeconfig-all.yaml
```
Apparently the user name has no connection to the common name in the
certificate.
Co-authored-by: Mario Manno <mario.manno@suse.com>
* Fix Harvester auto-downloaded image namespace
Signed-off-by: Chris Kim <oats87g@gmail.com>
* Disable IPv6 on Harvester VMs
Signed-off-by: Chris Kim <oats87g@gmail.com>
---------
Signed-off-by: Chris Kim <oats87g@gmail.com>
Iramis Valentin