Previous sorting method was not stable, and only worked coincidentally
for the two use cases that were using it. This new method works on
more event types and only sorts contiguous events. This should make
the sort usable when we add parallel apply and watch instead of poll.
Event Changes:
- Renamed ActionGroupEvent.Type -> Status
- Renamed Event.Operation -> Status
- Renamed Status fields to use consistent prefixes and suffixes
- Combined Applied, Changed, Unchanged, and ServersideApplied into
ApplySuccessful
- Added Failed status for apply, prune, and delete events
- Replaced Unspecified with Pending
- Made enum String output more consistent
Printer Changes:
- Added FormatSummary to print summary stats at the end of the
apply/destroy, instead of after the last of each type of action
group.
- Modified printer output to match new more consistent events.
- Updated JSON printer docs with latest schema details.
BREAKING CHANGE: Event "operations" and "type" are now "status"
BREAKING CHANGE: JSON printer schema changed to match events
BREAKING CHANGE: Event status enums renamed/refactored
- Rewrite actuation filters to return an error with the reason for
skipping.
- Add explicit error types for most skip errors, to make it easier to catch
and handle them.
- Add Is method to explicit error types to allow use of errors.Is for
recursive unwrapped matching.
- Rename InventoryPolicyFilter to InventoryPolicyPruneFilter for
consistency with InventoryPolicyApplyFilter
- Update deletion prevention inventory-id removal to use errors.As instead
of matching the filter name.
- Convert error structs to use pointers to allow nil errors and avoid
copying contents.
- Update printers to handle skip errors
BREAKING CHANGE: Skipped actuation events now include an error.
BREAKING CHANGE: DeleteEvent.Reason replaced with an error.
BREAKING CHANGE: Unused InventoryNamespaceInSet error removed.
BREAKING CHANGE: InventoryOverlapError replaced with PolicyPreventedActuationError.
BREAKING CHANGE: NeedAdoptionError replaced with PolicyPreventedActuationError.
BREAKING CHANGE: NoInventoryObjError & MultipleInventoryObjError now use pointers.
- Add ValidationPolicy:
ExitEarly (default) - exit before apply/prune if any objects are invalid
SkipInvalid - apply/prune valid objects & skip invalid ones
- Add ValidationEvent to be sent once for every invalid object or set of
objects, when the SkipInvalid policy is selected. For ExitEarly,
return an error for reverse compatibility.
- Add validation.Collector to simplify aggregating validation errors
from multiple sources and extracting invalid object IDs.
- Add invalid objects to the TestContext so they can be retained in
the inventory (only if already present). This primarily applies to
invalid annotations and dependencies. Objects without name or kind
should never be added to the inventory.
- Update Solver to use validation.Collector and filter invalid objects.
- Add e2e test for invalid objects.
- Update Printers to handle ValidationEvent
- Add ExternalDependencyError & InvalidAnnotationError to make it easier
to handle and introspect validation errors.
- Make FakeRESTMapper comparable
- Add Comparer for WaitTask
- Add Asserter to allow pre-configuring comparison options for Equal/NotEqual
- Update solver tests to use Asserter for task list comparison (more actionable errors)
- Using field.Error allows more errors to be wrapped as a
ValidationError, instead of interrupting validation and exiting
early.
- Add explicit validation for kind (clearer error).
- Move NestedField from testutil to object, so it can be used at
runtime.
- Converted WaitTask to use a context for timeout/cancellation, to
improve readability and reduce error cases. Now it only sends
TaskResult events from one place, removing the need for a token to
silence subsequent complete() calls.
- Add pending object tracking to WaitTask to ensure all objects are
accounted for at least one WaitEvent.
- Upgraded applier tests to use full event comparison, for better signal
on breaking changes.
- Enhanced task tests to consume all events before validating and test
actual event output.
- Add set sorting to graph.SortObjs, to make wait event ordering more
consistent, to make testing easier.
BREAKING CHANGE: wait tasks always execute after apply/prune/delete (except dry runs)
BREAKING CHANGE: wait tasks default to waiting until cancelled (previously 1m default)
- WaitEvent can be Pending, Reconciled, Skipped, or Timeout.
Skipped, Pending, and Reconciled events are sent at task start.
Reconciled events are sent later as status updates are recieved.
Timeout events are sent for remaining events on timeout.
- Rewrite WaitTask.Start to use context.WithTimeout and a goroutine to
handle task completion (replacing setTimer and the token hack).
- Replaced Task.ClearTimeout with Task.Cancel.
- Replaced WaitTask.complete & checkCondition with Task.StatusUpdate.
- Replaced WaitTask.startAndComplete with a check in WaitTask.Start.
- Replaced WaitTask.amendTimeoutError with WaitTask.sendTimeoutEvents
to send multiple timeout events, instead of one event with a list of
TimedOutResources.
- Updated all printers to handle WaitEvent.
Event printer now includes reconcile events.
JSON printer now includes resourceReconciled eventType.
Table printer not includes reconcile column.
- Added JSON printer tests for error handling.
- Updated Formatter.FormatActionGroupEvent to collect WaitStats.
- Enable status events by default for kapply with table output
BREAKING CHANGE: WaitEvents now sent for each object
Today, when a WaitTask timeout happens, the WaitTask sends the
TimeoutError on the TaskChannel. After receiving the TimeoutError,
`baseRunner.run` terminates immediately by returning the error to its
caller (Applier.Run or Destroyer.Run). The caller then sends the error
onto the EventChannel and terminates.
With this PR, when a WaitTask timeout happens, the WaitTask sends a
WaitType Event including the TimeoutError on the EventChannel, and then
sends an empty TaskResult on the TaskChannel. An empty TaskResult
suggests that the task finished successfully, and therefore
`baseRunner.run` would continue instead of terminate.
The motivation of this change is to make sure that cli-utils only
terminates on fatal errors (such as inventory-related errors, and
ApplyOptions creation errors). A WaitTask timeout may not always mean a
fatal error (it may happen because the StatusPoller has not finished
polling everything, or some but not all the resources have not reached
the desired status), and therefore should not terminate cli-utils.
- Add test for applier.Run context cancel/timeout
- Add AssertEqual & AssertNotEqual to wrap testutil.Equal matcher
- Move common test utils to common_test.go
- Use yaml artifacts, instead of Unstructured
- Use apply.Options as test input
- Use UnstructuredSet as test input
- Extract Applier construction with fake inputs to newTestApplier
- Extract nil inventory to its own test to simplify test inputs
to TestReadAndPrepareObjects
- Add GroupedEventsByID to impliment a custom sort algorithm fo use by
tests, to simplify the comparison of otherwise unsorted
apply/prune/delete events in an action group.
- GroupName can be used to identify which ActionGroupEvent the
Apply/Prune/Delete belongs to. This is useful for tracking progress
during a long apply/destroy.
- Detect `config.kubernetes.io/apply-time-mutation` annotation
- Parse annotation string value as YAML
- Treat source resource as a dependency
- Before applying, apply specified substitutions
- Each mutation may include one or more substitution
- Substitutions may optionally replace a token in the existing
string value, or replace the whole value (e.g. non-strings)
- Source and target fields are specified with JSONPath expressions
- Using github.com/spyzhov/ajson because it supports mutation, and
not just retrieval
- ApplyTimeMutator uses an in-memory ResourceCache to reduce GETs
- Add testutil.Equal to show comparisons with cmp.Diff output
- Add github.com/google/go-cmp for cmp.Equal and cmp.Diff
These comparison tools make debugging easier and properly handle
error comparisons, using cmpopts.EquateErrors()
- Added EventsToExpEvents, EventToExpEvent, and RemoveEqualEvents
to testutil to help with comparing event lists
- Add InitEvent to ExpEvent, to enable validating it exists
- e2e tests have all been upgraded to validate a full list of
events, rather than just a partial list with partial contents.
This should increase confidence that new changes don't break things.
- Move from pkg/object to pkg/object/dependson
- Add DependencySet type to abstract some complexity
- Add Marshal/Unmarshal funcs for ObjMetadata and DependencySet
- Refactor primary API to ReadAnnotation and WriteAnnotation
- Move Write/Marshal behavior from testutil into dependson pkg
- Make constants private to encourage using functions as primary API
Stefan Prodan
Ramon Quitales
Karl Isenberg
Haiyan Meng
Morten Torkildsen
Sean Sullivan