Commit Graph

129 Commits

Author SHA1 Message Date
Karl Isenberg 4e94795493 chore: refactor inventory pkg to avoid stuttering 2022-02-15 14:48:39 -08:00
Karl Isenberg 7f561ae8b8 chore: result stuttering in info pkg 2022-02-15 14:30:10 -08:00
Kubernetes Prow Robot 69f42a927d
Merge pull request #543 from karlkfi/karl-unsort
chore: Remove redundant object sorting
2022-02-15 11:14:39 -08:00
Karl Isenberg c1f572dc4c chore: Remove redundant object sorting
- The Solver Graph performs sorting.
  So the applier & destroyer don't need to.
2022-02-14 22:48:45 -08:00
Karl Isenberg fca07d568c chore: Move AdoptAll logic out of the Applier 2022-02-14 21:39:26 -08:00
Morten Torkildsen 5986969dc0 Expose ClusterReader options in the API 2022-02-01 11:37:49 -08:00
Mikhail Mazurskiy 2c815d2843
feat!: applier configuration options
BREAKING CHANGE: NewApplierBuilder() is the new way to construct an Applier
BREAKING CHANGE: apply.Options renamed into apply.ApplierOptions
2022-01-29 10:04:03 +11:00
Karl Isenberg 9acdbce963 feat: Add ValidationPolicy & ValidationEvent
- Add ValidationPolicy:
  ExitEarly (default) - exit before apply/prune if any objects are invalid
  SkipInvalid - apply/prune valid objects & skip invalid ones
- Add ValidationEvent to be sent once for every invalid object or set of
  objects, when the SkipInvalid policy is selected. For ExitEarly,
  return an error for reverse compatibility.
- Add validation.Collector to simplify aggregating validation errors
  from multiple sources and extracting invalid object IDs.
- Add invalid objects to the TestContext so they can be retained in
  the inventory (only if already present). This primarily applies to
  invalid annotations and dependencies. Objects without name or kind
  should never be added to the inventory.
- Update Solver to use validation.Collector and filter invalid objects.
- Add e2e test for invalid objects.
- Update Printers to handle ValidationEvent
- Add ExternalDependencyError & InvalidAnnotationError to make it easier
  to handle and introspect validation errors.
2022-01-25 12:36:02 -08:00
Karl Isenberg 2861fb9be4 fix: Merge task runner impls into taskStatusRunner
- Delete taskRunner & baseRunner
- Add short strings for ResourceAction enums
- Add detail to task errors
- Modify taskStatusRunner.Run to take TaskContext instead of
  eventChannel and resouceCache.
- Update runner tests to use fakePoller

BREAKING CHANGE: Changed taskStatusRunner.Run function signature
2022-01-19 13:06:06 -08:00
Karl Isenberg f67aaa87ac feat: MultiError for invalid annotations
- Move Validator to pkg/object/validation
- Replace ValidationError with validation.Error
- Replace MultiValidationError with generic MultiError
- Update Validator & SortObjs to use MultiError
- Add ResourceReferenceFromObjMetadata
- Rename NewResourceReference -> ResourceReferenceFromUnstructured
- Delete duplicate ResourceReference.ObjMetadata()
- Modify some error messages for consistency and clarity
- Use templating to generate some test artifacts

BREAKING CHANGE: apply-time-mutation namespace required for namespace-scoped resources
2022-01-12 15:24:53 -08:00
Karl Isenberg 242b6aa83e fix: Don't panic when ObjMetadata is invalid
- UnstructuredToObjMetaOrDie -> UnstructuredToObjMetadata
- UnstructuredToObjMeta -> UnstructuredToObjectMetadata
- UnstructuredsToObjMetasOrDie -> UnstructuredSetToObjMetadataSet
- UnstructuredsToObjMetas -> UnstructuredSetToObjMetadataSet
- Remove CreateObjMetadata (Validator performs validation)
2022-01-07 18:50:30 -08:00
Morten Torkildsen f33a0cdb32 Remove StatusPoller factory and support custom status readers 2021-12-16 19:38:10 +01:00
Kubernetes Prow Robot a0994fc794
Merge pull request #473 from ash2k/ash2k/missing-return
Add missing returns
2021-11-05 11:05:10 -07:00
Mikhail Mazurskiy 38f7478964
No need to lowercase namespaces
They are lowercase and that's enforced by Kubernetes.
2021-11-05 16:41:24 +11:00
Mikhail Mazurskiy 7b53399fc6
Add missing returns 2021-11-05 16:37:16 +11:00
Sean Sullivan 944b7a7d63 Exports Applier StatusPoller field 2021-11-02 11:53:52 -07:00
Karl Isenberg a5030ffd18 Clean up prune package
- Rename PruneOptions to Pruner (it does the work itself)
- Rename variables and internal methods for clarity/consistency
- Extract deleteObject method to improve readability
- Rename GetObject to getObject (only used internally and by test)
- Modify log messages for readability
- Add log support for prune tests
- Make log messages more consistent
2021-10-20 16:20:00 -07:00
Kubernetes Prow Robot a1b40b59d5
Merge pull request #422 from haiyanmeng/skip
Skip InventoryPolicyApplyFilter for the `AdoptAll` inventory policy to improve the performance
2021-10-08 14:38:24 -07:00
Karl Isenberg 604c9deb8b Add UnstructuredSet to encapsulate set functions
- Refactor usages to []Unstructured to use UnstructuredSet
- Add UnstructuredSet.Equal to handle set equality comparison
- Avoid updating tests to prove reverse compatibility
  (except: Graph.SortObj, which returns a list of sets)
2021-10-08 13:45:46 -07:00
Haiyan Meng 97b02cd107 Skip InventoryPolicyApplyFilter for the `AdoptAll` inventory policy to
improve the performance

Running InventoryPolicyApplyFilter involves getting every object in the
current inventory from the live cluster, which can be expensive if the
inventory includes lots of objects.

If the inventory policy is `AdoptAll`, the current inventory can take
ownership of any objects. Therefore, it is not necessary to run
InventoryPolicyApplyFilter.
2021-10-07 10:02:30 -07:00
Karl Isenberg d964b0397b Merge Collector and ResourceCache
- Make ResourceCache thread-safe
- Make ResourceCache store status and messagei
- Add ResourceCache to baseRunner and TaskContext
- Make Mutator compute resource status for uncached resources
- Share cache between StatusPoller and Mutator
- Move Condition and conditionMet() to its own file
- Simplify WaitTask.checkCondition
- Simplify baseRunner.amendTimeoutError
2021-09-30 12:01:19 -07:00
Karl Isenberg 02c0fb7229 Add apply-time-mutation feature
- Detect `config.kubernetes.io/apply-time-mutation` annotation
- Parse annotation string value as YAML
- Treat source resource as a dependency
- Before applying, apply specified substitutions
- Each mutation may include one or more substitution
- Substitutions may optionally replace a token in the existing
  string value, or replace the whole value (e.g. non-strings)
- Source and target fields are specified with JSONPath expressions
- Using github.com/spyzhov/ajson because it supports mutation, and
  not just retrieval
- ApplyTimeMutator uses an in-memory ResourceCache to reduce GETs
2021-09-21 11:00:37 -07:00
Sean Sullivan 45cb7e31e0 Moves inventory policy check for apply into filter 2021-08-30 10:21:38 -07:00
Kubernetes Prow Robot f7e13914ee
Merge pull request #396 from seans3/cyclic-dep-error
Expose cyclic dependency error
2021-08-23 12:08:23 -07:00
Sean Sullivan 769f5bd735 Expose cyclic dependency error 2021-08-23 11:24:05 -07:00
Kubernetes Prow Robot 24030c97fa
Merge pull request #395 from seans3/inv-annotation-fix
Move inventory annotation addition
2021-08-23 10:44:01 -07:00
Sean Sullivan aa79285f4b Move inventory annotation addition 2021-08-17 11:09:58 -07:00
Morten Torkildsen 91df668cb8 Prune/Delete should use the specified propagation policy 2021-08-17 09:48:31 -07:00
Mikhail Mazurskiy 4b8dd62c53
Remove strings.TrimSpace()
Names and labels cannot have any whitespace. If they do, it's an error that should be reported and fixed.
2021-08-08 11:29:43 +10:00
Kubernetes Prow Robot 69388f265d
Merge pull request #389 from justinsb/infohelper
Pass the RESTMapper explicitly into infoHelper
2021-07-29 13:51:18 -07:00
Justin SB 669fb3f739 Pass the RESTMapper explicitly into infoHelper
This allows for better control, for example allowing us to invalidate
the discovery cache before proceeding.
2021-07-29 18:39:11 +00:00
Morten Torkildsen dacbf97143 Avoid dryRun as state in inventory client 2021-07-14 17:34:18 -07:00
Morten Torkildsen 136d1efeeb Better validation of resources prior to actuation 2021-07-01 17:02:57 -07:00
Mikhail Mazurskiy c1c7c9ef9c
Remove Provider interface 2021-06-30 15:21:03 +10:00
Sean Sullivan 383c0042fb destroyer calculates emit status events 2021-06-28 16:56:32 -07:00
Mikhail Mazurskiy 4eb7dc1f6f
Remove Initialize() on Destroyer and PruneOptions 2021-06-18 14:58:49 +10:00
Sean Sullivan 4432f51ac3 Refactor prune/delete using ValidationFilter interface 2021-06-16 11:29:37 -07:00
Mikhail Mazurskiy 8dc334f344
Remove Applier.Initialize() 2021-06-16 08:53:30 +10:00
Sean Sullivan b77cc7ea62 Prune refactor to take objects to prune 2021-06-10 16:37:18 -07:00
Sean Sullivan af417c6fab Refactor destroyer to use tasks 2021-06-09 15:40:32 -07:00
Mikhail Mazurskiy d41f250f19
Update dependencies to Kubernetes v1.21.1 2021-05-28 12:53:51 +10:00
Sean Sullivan 42e330d4d8 Move inventory operations into tasks 2021-05-26 17:47:00 -07:00
Morten Torkildsen ae80e561e2 Improve the event hierarchy 2021-05-18 14:58:41 -07:00
Morten Torkildsen 7be21cf94e Add validation of inventory-id for name strategy if inventory object already exists 2021-04-04 15:52:33 -07:00
Sean Sullivan 185aa3d048 Better final inventory calculation during errors 2021-02-04 22:33:54 -08:00
Sean Sullivan 6db68fda8d Fix for final inventory update 2021-01-25 22:30:22 -08:00
Jingfang Liu 5184286403 enable InventoryPolicy in applier 2020-12-08 13:22:08 -08:00
Jingfang Liu bff0ab8ace add inventory policy 2020-11-30 09:47:28 -08:00
Jingfang Liu 3224d5f259 splti the provider interface 2020-11-16 08:30:49 -08:00
Jingfang Liu 7915dc5de7 change the apply API to InventoryInfo 2020-11-11 15:07:46 -08:00