- Stress test tests 1,000 Namespaces, CofnigMaps, & CronTabs (CR)
- Stress test is a new test suite with its own make entrypoint
- Refactor shared test code so the e2e and stress tests can both use it
- Update test client QPS to 20 (from 5)
- Using field.Error allows more errors to be wrapped as a
ValidationError, instead of interrupting validation and exiting
early.
- Add explicit validation for kind (clearer error).
- Move NestedField from testutil to object, so it can be used at
runtime.
Tests will now each have a context that is cancelled on timeout.
BeforeEach and AfterEach blocks have their own timeouts, to allow
bounded setup and cleanup.
- Test Timeout: 5m
- Before Test Timeout: 30s
- After Test Timeout: 30s
- WaitEvent can be Pending, Reconciled, Skipped, or Timeout.
Skipped, Pending, and Reconciled events are sent at task start.
Reconciled events are sent later as status updates are recieved.
Timeout events are sent for remaining events on timeout.
- Rewrite WaitTask.Start to use context.WithTimeout and a goroutine to
handle task completion (replacing setTimer and the token hack).
- Replaced Task.ClearTimeout with Task.Cancel.
- Replaced WaitTask.complete & checkCondition with Task.StatusUpdate.
- Replaced WaitTask.startAndComplete with a check in WaitTask.Start.
- Replaced WaitTask.amendTimeoutError with WaitTask.sendTimeoutEvents
to send multiple timeout events, instead of one event with a list of
TimedOutResources.
- Updated all printers to handle WaitEvent.
Event printer now includes reconcile events.
JSON printer now includes resourceReconciled eventType.
Table printer not includes reconcile column.
- Added JSON printer tests for error handling.
- Updated Formatter.FormatActionGroupEvent to collect WaitStats.
- Enable status events by default for kapply with table output
BREAKING CHANGE: WaitEvents now sent for each object
- Destroyer context added, with cancel test.
- Fix Applier cancel test not using specified timeouts.
- Added a few logs to help with debugging test failures
- GroupName can be used to identify which ActionGroupEvent the
Apply/Prune/Delete belongs to. This is useful for tracking progress
during a long apply/destroy.
- Unstructured() added to ResourceReference to aid test operations
- Test artifacts moved into artifacts_test.go
- deploymentManifest and apiserviceManifest replaced with
yaml->unstructured, for consistency with other test artifacts
- Added withReplicas, withNamespace, and withDependsOn for easier inline
mutation of test artifacts
- Added deleteUnstructuredAndWait, assertUnstructuredExists, and
assertUnstructuredDoesNotExist to validate actual cluster state,
not just events. Also reduced flakiness by waiting for delete.
- Removed error check before event validation. The event diff
provides much more context for debugging.
- Replaced typed resources with unstructured and used
testutil.NestedField to extract values.
- Updated prune error test to use full event list comparison and
validate server state.
- Add 2nd resource to continue on error test
- Don't fail early on error from event list
- Fix flakey prune error test
Add wait for pod2 after creation
Add wait for pod2 after deletion
Re-use inventory to ensure prune of pod2 not skipped
- Add optional klog support for e2e tests
- Add testutil.Equal to show comparisons with cmp.Diff output
- Add github.com/google/go-cmp for cmp.Equal and cmp.Diff
These comparison tools make debugging easier and properly handle
error comparisons, using cmpopts.EquateErrors()
- Added EventsToExpEvents, EventToExpEvent, and RemoveEqualEvents
to testutil to help with comparing event lists
- Add InitEvent to ExpEvent, to enable validating it exists
- e2e tests have all been upgraded to validate a full list of
events, rather than just a partial list with partial contents.
This should increase confidence that new changes don't break things.
Karl Isenberg