Commit Graph

900 Commits

Author SHA1 Message Date
Mikhail Mazurskiy ace8272244
Remove unused ResetRESTMapperTask task 2021-10-12 19:43:52 +11:00
Mikhail Mazurskiy 7333873c91
Drop k8s.io/apiextensions-apiserver dependency 2021-10-11 21:27:48 +11:00
Kubernetes Prow Robot a1b40b59d5
Merge pull request #422 from haiyanmeng/skip
Skip InventoryPolicyApplyFilter for the `AdoptAll` inventory policy to improve the performance
2021-10-08 14:38:24 -07:00
Kubernetes Prow Robot 9cae550146
Merge pull request #426 from karlkfi/karl-unstructured-set
Add UnstructuredSet to encapsulate set functions
2021-10-08 14:18:24 -07:00
Karl Isenberg 604c9deb8b Add UnstructuredSet to encapsulate set functions
- Refactor usages to []Unstructured to use UnstructuredSet
- Add UnstructuredSet.Equal to handle set equality comparison
- Avoid updating tests to prove reverse compatibility
  (except: Graph.SortObj, which returns a list of sets)
2021-10-08 13:45:46 -07:00
Kubernetes Prow Robot 050ae461c8
Merge pull request #420 from ash2k/ash2/cleanups
Minor cleanups
2021-10-08 06:16:58 -07:00
Kubernetes Prow Robot 223d4d51b4
Merge pull request #419 from karlkfi/karl-sets
Add ObjMetadataSet to encapsulate set functions
2021-10-07 12:17:33 -07:00
Haiyan Meng 97b02cd107 Skip InventoryPolicyApplyFilter for the `AdoptAll` inventory policy to
improve the performance

Running InventoryPolicyApplyFilter involves getting every object in the
current inventory from the live cluster, which can be expensive if the
inventory includes lots of objects.

If the inventory policy is `AdoptAll`, the current inventory can take
ownership of any objects. Therefore, it is not necessary to run
InventoryPolicyApplyFilter.
2021-10-07 10:02:30 -07:00
Mikhail Mazurskiy 7ee6ca6082
Optimize allocations a bit
No need to allocate an empty slice, nil is fine in this case. Map of empty structs is good enough and uses a bit less memory.
2021-10-07 14:38:48 +11:00
Mikhail Mazurskiy a33618ceea
Don't embed mutex
Embedding it exposes the methods, which breaks encapsulation
2021-10-07 14:37:00 +11:00
Kubernetes Prow Robot 231f3f4a69
Merge pull request #417 from haiyanmeng/rest
Skip pruning an object whose Group/Kind is not found
2021-10-06 16:04:41 -07:00
Haiyan Meng f0e71ba543 Skip pruning an object whose Group/Kind is not found 2021-10-06 14:44:21 -07:00
Karl Isenberg d83ce93efd Add ObjMetadataSet to encapsulate set functions
- Refactor usages of []ObjMetadata to use ObjMetadataSet
- Move Union, Diff, Contains, Hash, Remove, and Equal into
  ObjMetadataSet
- Add ToStringMap and FromStringMap for inventory serialization
2021-10-06 14:25:12 -07:00
Morten Torkildsen 854d4fc26b Update owners file to only include active contributors 2021-10-06 11:42:46 -07:00
Kubernetes Prow Robot 1f62349d34
Merge pull request #418 from karlkfi/karl-err-event2
Add ExpErrorEvent for better e2e test failures
2021-10-06 11:24:32 -07:00
Karl Isenberg 8b1ad5067c Add ExpErrorEvent for better e2e test failures 2021-10-06 08:56:30 -07:00
Kubernetes Prow Robot 2a6b353b46
Merge pull request #416 from karlkfi/karl-fix-flaky
Fix flaky continue on error test
2021-10-05 17:46:33 -07:00
Karl Isenberg 50fb925f41 Fix flaky continue on error test
- Add GroupedEventsByID to impliment a custom sort algorithm fo use by
  tests, to simplify the comparison of otherwise unsorted
  apply/prune/delete events in an action group.
2021-10-05 17:17:19 -07:00
Karl Isenberg 853f4b0d0b Add GroupName to Apply, Prune, and Delete events
- GroupName can be used to identify which ActionGroupEvent the
  Apply/Prune/Delete belongs to. This is useful for tracking progress
  during a long apply/destroy.
2021-10-05 17:14:11 -07:00
Kubernetes Prow Robot d50496a12b
Merge pull request #413 from karlkfi/karl-collector
Merge Collector and ResourceCache
2021-09-30 16:53:22 -07:00
Karl Isenberg d964b0397b Merge Collector and ResourceCache
- Make ResourceCache thread-safe
- Make ResourceCache store status and messagei
- Add ResourceCache to baseRunner and TaskContext
- Make Mutator compute resource status for uncached resources
- Share cache between StatusPoller and Mutator
- Move Condition and conditionMet() to its own file
- Simplify WaitTask.checkCondition
- Simplify baseRunner.amendTimeoutError
2021-09-30 12:01:19 -07:00
Kubernetes Prow Robot 6214d1975c
Merge pull request #414 from karlkfi/karl-run-collect
Use runCollect in e2e tests more consistently
2021-09-30 10:04:07 -07:00
Karl Isenberg a2964373c8 Use runCollect in e2e tests more consistently 2021-09-29 16:02:04 -07:00
Kubernetes Prow Robot 3544d7d96e
Merge pull request #411 from seans3/fix-prune-wait-timeout
Fixes prune timeout value
2021-09-29 15:16:44 -07:00
Sean Sullivan bbd7cf104b Fixes prune timeout value 2021-09-27 15:22:27 -07:00
Kubernetes Prow Robot 73a24dc859
Merge pull request #408 from karlkfi/karl-test-fixes
e2e test cleanup
2021-09-27 12:13:45 -07:00
Kubernetes Prow Robot 8dad92fa3d
Merge pull request #410 from karlkfi/karl-filter-reasons
Improve clarity of filter and mutator reasons
2021-09-26 20:36:22 -07:00
Karl Isenberg 9fc0cd1995 Improve clarity of filter and mutator reasons
- Use stringer for all enums (to translate ids into names)
- Remove redundant effect from reason strings
- Use consistent pattern for variables in reason strings.
  We can't use structured json, but this is close, while still human
  readable.
- Remove copyright on generated files
2021-09-24 18:41:14 -07:00
Karl Isenberg 3d81e21acf e2e test cleanup
- Unstructured() added to ResourceReference to aid test operations
- Test artifacts moved into artifacts_test.go
- deploymentManifest and apiserviceManifest replaced with
  yaml->unstructured, for consistency with other test artifacts
- Added withReplicas, withNamespace, and withDependsOn for easier inline
  mutation of test artifacts
- Added deleteUnstructuredAndWait, assertUnstructuredExists, and
  assertUnstructuredDoesNotExist to validate actual cluster state,
  not just events. Also reduced flakiness by waiting for delete.
- Removed error check before event validation. The event diff
  provides much more context for debugging.
- Replaced typed resources with unstructured and used
  testutil.NestedField to extract values.
- Updated prune error test to use full event list comparison and
  validate server state.
- Add 2nd resource to continue on error test
- Don't fail early on error from event list
- Fix flakey prune error test
  Add wait for pod2 after creation
  Add wait for pod2 after deletion
  Re-use inventory to ensure prune of pod2 not skipped
- Add optional klog support for e2e tests
2021-09-24 17:54:17 -07:00
Kubernetes Prow Robot aed715e140
Merge pull request #409 from haiyanmeng/id
Include the resource ids in the timeout error msg
2021-09-24 13:50:51 -07:00
Haiyan Meng 9da829eb5a Include the resource ids in the timeout error msg 2021-09-24 12:26:28 -07:00
Kubernetes Prow Robot eac0e4437a
Merge pull request #406 from karlkfi/karl-jsonyaml-fixes
Marshal with k8s.io/apimachinery/pkg/util/yaml
2021-09-24 11:29:50 -07:00
Kubernetes Prow Robot f72f0008ce
Merge pull request #407 from haiyanmeng/adoptall
Add a new flag value `force-adopt` to the `--inventory-policy` flag
2021-09-24 10:59:49 -07:00
Karl Isenberg 4f4bd30c57 Change YamlStringer to only accept Unstructured
- This avoids a runtime schema dependency
2021-09-24 10:35:28 -07:00
Haiyan Meng cb132d9e84 Add a new flag value `force-adopt` to the `--inventory-policy` flag
`force-adopt` allows the current inventory take ownership of any objects.
2021-09-23 11:31:47 -07:00
Karl Isenberg a468a88337 Unmarshal with k8s.io/apimachinery/pkg/util/yaml
- apimachinery yaml.Unmarshal fixes number types to int64 and float64
- Fix inaccurate error message in mutator
- Move YamlStringer to object pkg for reuse by other pkgs
- Fix jsonpath tests parsing y as a bool
- Fix kstatus example_test.go to actually be a test
2021-09-22 16:33:06 -07:00
Kubernetes Prow Robot 61a4552508
Merge pull request #400 from karlkfi/karl-apply-time-mutation
Add apply-time-mutation feature
2021-09-21 12:10:19 -07:00
Karl Isenberg 02c0fb7229 Add apply-time-mutation feature
- Detect `config.kubernetes.io/apply-time-mutation` annotation
- Parse annotation string value as YAML
- Treat source resource as a dependency
- Before applying, apply specified substitutions
- Each mutation may include one or more substitution
- Substitutions may optionally replace a token in the existing
  string value, or replace the whole value (e.g. non-strings)
- Source and target fields are specified with JSONPath expressions
- Using github.com/spyzhov/ajson because it supports mutation, and
  not just retrieval
- ApplyTimeMutator uses an in-memory ResourceCache to reduce GETs
2021-09-21 11:00:37 -07:00
Kubernetes Prow Robot 7369085313
Merge pull request #401 from seans3/prune-retrieve-error
Ignore prune object "not found" error
2021-09-20 11:03:09 -07:00
Kubernetes Prow Robot 206bd1c217
Merge pull request #405 from karlkfi/karl-e2e-events
Add more robust event list testing in e2e tests
2021-09-16 15:37:32 -07:00
Karl Isenberg 1b9059b640 Fix apply and destroy e2e test to share inventory 2021-09-16 08:59:00 -07:00
Karl Isenberg 6992b74beb Add more robust event list testing in e2e tests
- Add testutil.Equal to show comparisons with cmp.Diff output
- Add github.com/google/go-cmp for cmp.Equal and cmp.Diff
  These comparison tools make debugging easier and properly handle
  error comparisons, using cmpopts.EquateErrors()
- Added EventsToExpEvents, EventToExpEvent, and RemoveEqualEvents
  to testutil to help with comparing event lists
- Add InitEvent to ExpEvent, to enable validating it exists
- e2e tests have all been upgraded to validate a full list of
  events, rather than just a partial list with partial contents.
  This should increase confidence that new changes don't break things.
2021-09-15 00:36:16 -07:00
Kubernetes Prow Robot 8074ebcfab
Merge pull request #402 from karlkfi/karl-dependson
Refactor depends-on code
2021-09-13 15:09:50 -07:00
Karl Isenberg 15ef662b14 Rename Marshal/Unmarshal to Format/Parse 2021-09-13 13:43:24 -07:00
Karl Isenberg 325537204d Refactor depends-on code
- Move from pkg/object to pkg/object/dependson
- Add DependencySet type to abstract some complexity
- Add Marshal/Unmarshal funcs for ObjMetadata and DependencySet
- Refactor primary API to ReadAnnotation and WriteAnnotation
- Move Write/Marshal behavior from testutil into dependson pkg
- Make constants private to encourage using functions as primary API
2021-09-13 13:20:29 -07:00
Sean Sullivan 12425d7593 Ignore prune object retrieval error 2021-09-10 16:57:26 -07:00
Kubernetes Prow Robot 2941a6243a
Merge pull request #399 from seans3/inv-policy-apply-filter
Moves inventory policy check for apply into filter
2021-08-30 15:27:13 -07:00
Sean Sullivan 45cb7e31e0 Moves inventory policy check for apply into filter 2021-08-30 10:21:38 -07:00
Kubernetes Prow Robot a4d5c55f2a
Merge pull request #398 from karlkfi/karl-license-update
Update addlicense to v1.0.0 and year 2021
2021-08-25 10:34:42 -07:00
Karl Isenberg bd0dd87de9 Fix addlicense path and enable verbose mode 2021-08-25 09:08:04 -07:00