- Add DefaultStatusWatcher that wraps DynamicClient and manages
informers for a set of resource objects.
- Supports two modes: root-scoped & namespace-scoped.
- Root-scoped mode uses root-scoped informers to efficiency and
performance.
- Namespace-scoped mode uses namespace-scoped informers to
minimize the permissions needed to run and the size of the
in-memory object cache.
- Automatic mode selects which mode to use based on whether the
objects being watched are in one or multiple namespaces.
This is the default mode, optimizing for performance.
- If CRDs are being watched, the creation/deletion of CRDs can
cause informers for those custom resources to be created/deleted.
- In namespace-scope mode, if namespaces are being watched, the
creation/deletion of namespaces can also trigger informers to
be created/deleted.
- All creates/updates/deletes to CRDs also cause RESTMapper reset.
- Allow pods to be unschedulable for 15s before reporting the
status as Failed. Any update resets the timer.
- Add BlindStatusWatcher for testing and disabling for dry-run.
- Add DynamicClusterReader that wraps DynamicClient.
This is now used to look up generated resources
(ex: Deployment > ReplicaSets > Pods).
- Add DefaultStatusReader which uses a DelegatingStatusReader to
wrap a list of conventional and specific StatusReaders.
This should make it easier to extend the list of StatusReaders.
- Move some pending WaitEvents to be optional in tests, now that
StatusWatcher can resolve their status before the WaitTask starts.
- Add a new Thousand Deployments stress test (10x kind nodes)
- Add some new logs for easier debugging
- Add internal SyncEvent so that apply/delete tasks don't start
until the StatusWatcher has finished initial synchronization.
This helps avoid missing events from actions that happen while
synchronization is incomplete.
- Filter optional pending WaitEvents when testing.
BREAKING CHANGE: Replace StatusPoller w/ StatusWatcher
BREAKING CHANGE: Remove PollInterval (obsolete with watcher)
- Pass TaskContext into TaskBuilder.Build
- Combine dependency graph for apply and prune objects.
This is required to catch dependencies that would have been deleted.
- Replace graph.SortObjs into DependencyGraph + Sort + HydrateSetList
- Replace graph.ReverseSortObjs with ReverseSetList to perform on the
combined (apply + prune) set list.
- Add planned pending applies and prune to the InventoryManager
before executing the task queue.
This allows the DependencyFilter to validate against the planned
actuation strategy of objects that haven't been applied/pruned yet.
- Add the dependency graph to the TaskContext, for the
DependencyFilter to use.
This can be removed in the future if the filters are managed by the
solver.
- Make Graph.Sort non-destructive, so the graph can be re-used by the
DependencyFilter.
- Add Graph.EdgesFrom and EdgesTo for the DependencyFilter to use.
This requires storing the reverse edge list.
- Add an e2e test for the DependencyFilter
- Add an e2e test for the LocalNamespaceFilter
Fixes https://github.com/kubernetes-sigs/cli-utils/issues/526
Fixes https://github.com/kubernetes-sigs/cli-utils/issues/528
Capture the apply status for individual resources after each apply.
If the inventory object supports status field, it is updated
after setting the inventory list at then of one apply process.
BREAKING CHANGE: Update the inventory client and inventory interfaces
to pass the apply/reconcile status.
- Add ValidationPolicy:
ExitEarly (default) - exit before apply/prune if any objects are invalid
SkipInvalid - apply/prune valid objects & skip invalid ones
- Add ValidationEvent to be sent once for every invalid object or set of
objects, when the SkipInvalid policy is selected. For ExitEarly,
return an error for reverse compatibility.
- Add validation.Collector to simplify aggregating validation errors
from multiple sources and extracting invalid object IDs.
- Add invalid objects to the TestContext so they can be retained in
the inventory (only if already present). This primarily applies to
invalid annotations and dependencies. Objects without name or kind
should never be added to the inventory.
- Update Solver to use validation.Collector and filter invalid objects.
- Add e2e test for invalid objects.
- Update Printers to handle ValidationEvent
- Add ExternalDependencyError & InvalidAnnotationError to make it easier
to handle and introspect validation errors.
- Move Validator to pkg/object/validation
- Replace ValidationError with validation.Error
- Replace MultiValidationError with generic MultiError
- Update Validator & SortObjs to use MultiError
- Add ResourceReferenceFromObjMetadata
- Rename NewResourceReference -> ResourceReferenceFromUnstructured
- Delete duplicate ResourceReference.ObjMetadata()
- Modify some error messages for consistency and clarity
- Use templating to generate some test artifacts
BREAKING CHANGE: apply-time-mutation namespace required for namespace-scoped resources
- Using field.Error allows more errors to be wrapped as a
ValidationError, instead of interrupting validation and exiting
early.
- Add explicit validation for kind (clearer error).
- Move NestedField from testutil to object, so it can be used at
runtime.
- Converted WaitTask to use a context for timeout/cancellation, to
improve readability and reduce error cases. Now it only sends
TaskResult events from one place, removing the need for a token to
silence subsequent complete() calls.
- Add pending object tracking to WaitTask to ensure all objects are
accounted for at least one WaitEvent.
- Upgraded applier tests to use full event comparison, for better signal
on breaking changes.
- Enhanced task tests to consume all events before validating and test
actual event output.
- Add set sorting to graph.SortObjs, to make wait event ordering more
consistent, to make testing easier.
BREAKING CHANGE: wait tasks always execute after apply/prune/delete (except dry runs)
BREAKING CHANGE: wait tasks default to waiting until cancelled (previously 1m default)
Historically, the objects in ApplyTask were being modified in the
Start func by BuildInfo to remove the path annotation, but this can
cause a race condition in the task runner if it tried to read the
task objects.
Now a DeepCopy is made in BuildInfo instead. This avoids the race
condition but broke mutation, which executes after BuildInfo on the
original object. So we now extract the object from the into after
BuildInfo and use that instead for mutations, filters, and events.
- Added SkippedApplies and SkippedDeletes to the TaskContext
- Modified tasks to use the new skipped tracking, replacing usage
of failure tracking, where skipped is more accurate.
- Renamed some TaskContext methods for consistency
- Added ObjMetadataSetFromMap for use by TaskContext
- Added ObjectMetadataSet.Intersection for use by InvSetTask
- Cleaned up InvSetTask to be more readable with comments explaining
intended behavior, including handling of skips.
- Added apply and prune tests for skipped, failure, and abandoned
- Refactor usages to []Unstructured to use UnstructuredSet
- Add UnstructuredSet.Equal to handle set equality comparison
- Avoid updating tests to prove reverse compatibility
(except: Graph.SortObj, which returns a list of sets)
- Refactor usages of []ObjMetadata to use ObjMetadataSet
- Move Union, Diff, Contains, Hash, Remove, and Equal into
ObjMetadataSet
- Add ToStringMap and FromStringMap for inventory serialization
- Unstructured() added to ResourceReference to aid test operations
- Test artifacts moved into artifacts_test.go
- deploymentManifest and apiserviceManifest replaced with
yaml->unstructured, for consistency with other test artifacts
- Added withReplicas, withNamespace, and withDependsOn for easier inline
mutation of test artifacts
- Added deleteUnstructuredAndWait, assertUnstructuredExists, and
assertUnstructuredDoesNotExist to validate actual cluster state,
not just events. Also reduced flakiness by waiting for delete.
- Removed error check before event validation. The event diff
provides much more context for debugging.
- Replaced typed resources with unstructured and used
testutil.NestedField to extract values.
- Updated prune error test to use full event list comparison and
validate server state.
- Add 2nd resource to continue on error test
- Don't fail early on error from event list
- Fix flakey prune error test
Add wait for pod2 after creation
Add wait for pod2 after deletion
Re-use inventory to ensure prune of pod2 not skipped
- Add optional klog support for e2e tests
- apimachinery yaml.Unmarshal fixes number types to int64 and float64
- Fix inaccurate error message in mutator
- Move YamlStringer to object pkg for reuse by other pkgs
- Fix jsonpath tests parsing y as a bool
- Fix kstatus example_test.go to actually be a test
- Detect `config.kubernetes.io/apply-time-mutation` annotation
- Parse annotation string value as YAML
- Treat source resource as a dependency
- Before applying, apply specified substitutions
- Each mutation may include one or more substitution
- Substitutions may optionally replace a token in the existing
string value, or replace the whole value (e.g. non-strings)
- Source and target fields are specified with JSONPath expressions
- Using github.com/spyzhov/ajson because it supports mutation, and
not just retrieval
- ApplyTimeMutator uses an in-memory ResourceCache to reduce GETs
- Move from pkg/object to pkg/object/dependson
- Add DependencySet type to abstract some complexity
- Add Marshal/Unmarshal funcs for ObjMetadata and DependencySet
- Refactor primary API to ReadAnnotation and WriteAnnotation
- Move Write/Marshal behavior from testutil into dependson pkg
- Make constants private to encourage using functions as primary API
Stefan Prodan
Ramon Quitales
Karl Isenberg
Jingfang Liu
Kubernetes Prow Robot
Morten Torkildsen
Mikhail Mazurskiy
Sean Sullivan