Commit Graph

46 Commits

Author SHA1 Message Date
Karl Isenberg a42a1517d4 fix: Update ObjMetadataSet.Hash to not error 2022-01-19 11:15:08 -08:00
Karl Isenberg f67aaa87ac feat: MultiError for invalid annotations
- Move Validator to pkg/object/validation
- Replace ValidationError with validation.Error
- Replace MultiValidationError with generic MultiError
- Update Validator & SortObjs to use MultiError
- Add ResourceReferenceFromObjMetadata
- Rename NewResourceReference -> ResourceReferenceFromUnstructured
- Delete duplicate ResourceReference.ObjMetadata()
- Modify some error messages for consistency and clarity
- Use templating to generate some test artifacts

BREAKING CHANGE: apply-time-mutation namespace required for namespace-scoped resources
2022-01-12 15:24:53 -08:00
Kubernetes Prow Robot d7d63f4b62
Merge pull request #505 from karlkfi/karl-no-die
fix: Don't panic when ObjMetadata is invalid (missing kind/name)
2022-01-07 19:27:03 -08:00
Karl Isenberg 242b6aa83e fix: Don't panic when ObjMetadata is invalid
- UnstructuredToObjMetaOrDie -> UnstructuredToObjMetadata
- UnstructuredToObjMeta -> UnstructuredToObjectMetadata
- UnstructuredsToObjMetasOrDie -> UnstructuredSetToObjMetadataSet
- UnstructuredsToObjMetas -> UnstructuredSetToObjMetadataSet
- Remove CreateObjMetadata (Validator performs validation)
2022-01-07 18:50:30 -08:00
Karl Isenberg a0743e6912 fix: Clean up object.FieldPath
- Use string.Builder to be slightly more efficient
- Format invalid types with Go syntax, wrapped with brackets
- Test that invalid types don't panic
2022-01-07 18:27:57 -08:00
Karl Isenberg fb98bc7877 fix: Handle more validation errors as field errors
- Using field.Error allows more errors to be wrapped as a
  ValidationError, instead of interrupting validation and exiting
  early.
- Add explicit validation for kind (clearer error).
- Move NestedField from testutil to object, so it can be used at
  runtime.
2022-01-07 12:06:43 -08:00
Karl Isenberg c5636c0243 feat: Always wait for reconciliation
- Converted WaitTask to use a context for timeout/cancellation, to
  improve readability and reduce error cases. Now it only sends
  TaskResult events from one place, removing the need for a token to
  silence subsequent complete() calls.
- Add pending object tracking to WaitTask to ensure all objects are
  accounted for at least one WaitEvent.
- Upgraded applier tests to use full event comparison, for better signal
  on breaking changes.
- Enhanced task tests to consume all events before validating and test
  actual event output.
- Add set sorting to graph.SortObjs, to make wait event ordering more
  consistent, to make testing easier.

BREAKING CHANGE: wait tasks always execute after apply/prune/delete (except dry runs)
BREAKING CHANGE: wait tasks default to waiting until cancelled (previously 1m default)
2021-11-10 12:34:54 -08:00
Karl Isenberg b977b5960f fix: race condition in BuildInfo
Historically, the objects in ApplyTask were being modified in the
Start func by BuildInfo to remove the path annotation, but this can
cause a race condition in the task runner if it tried to read the
task objects.

Now a DeepCopy is made in BuildInfo instead. This avoids the race
condition but broke mutation, which executes after BuildInfo on the
original object. So we now extract the object from the into after
BuildInfo and use that instead for mutations, filters, and events.
2021-11-03 13:11:20 -07:00
Kubernetes Prow Robot f4b64aada8
Merge pull request #430 from ash2k/ash2k/crd-check-version
Check versions a CRD defines
2021-11-01 13:35:47 -07:00
Karl Isenberg eda3554fb1 fix: skipped deletes no longer cause waiting
- Added SkippedApplies and SkippedDeletes to the TaskContext
- Modified tasks to use the new skipped tracking, replacing usage
  of failure tracking, where skipped is more accurate.
- Renamed some TaskContext methods for consistency
- Added ObjMetadataSetFromMap for use by TaskContext
- Added ObjectMetadataSet.Intersection for use by InvSetTask
- Cleaned up InvSetTask to be more readable with comments explaining
  intended behavior, including handling of skips.
- Added apply and prune tests for skipped, failure, and abandoned
2021-10-29 10:28:06 -07:00
Morten Torkildsen 73dca2b0dc Update to latest version of kyaml and k8s libs 2021-10-19 17:41:57 -07:00
Mikhail Mazurskiy 7e47e6bd0e
Check exact version when determining scope 2021-10-15 12:36:25 +11:00
Mikhail Mazurskiy 9f6552d595
Check versions a CRD defines 2021-10-15 12:36:23 +11:00
Mikhail Mazurskiy 7333873c91
Drop k8s.io/apiextensions-apiserver dependency 2021-10-11 21:27:48 +11:00
Karl Isenberg 604c9deb8b Add UnstructuredSet to encapsulate set functions
- Refactor usages to []Unstructured to use UnstructuredSet
- Add UnstructuredSet.Equal to handle set equality comparison
- Avoid updating tests to prove reverse compatibility
  (except: Graph.SortObj, which returns a list of sets)
2021-10-08 13:45:46 -07:00
Karl Isenberg d83ce93efd Add ObjMetadataSet to encapsulate set functions
- Refactor usages of []ObjMetadata to use ObjMetadataSet
- Move Union, Diff, Contains, Hash, Remove, and Equal into
  ObjMetadataSet
- Add ToStringMap and FromStringMap for inventory serialization
2021-10-06 14:25:12 -07:00
Karl Isenberg 3d81e21acf e2e test cleanup
- Unstructured() added to ResourceReference to aid test operations
- Test artifacts moved into artifacts_test.go
- deploymentManifest and apiserviceManifest replaced with
  yaml->unstructured, for consistency with other test artifacts
- Added withReplicas, withNamespace, and withDependsOn for easier inline
  mutation of test artifacts
- Added deleteUnstructuredAndWait, assertUnstructuredExists, and
  assertUnstructuredDoesNotExist to validate actual cluster state,
  not just events. Also reduced flakiness by waiting for delete.
- Removed error check before event validation. The event diff
  provides much more context for debugging.
- Replaced typed resources with unstructured and used
  testutil.NestedField to extract values.
- Updated prune error test to use full event list comparison and
  validate server state.
- Add 2nd resource to continue on error test
- Don't fail early on error from event list
- Fix flakey prune error test
  Add wait for pod2 after creation
  Add wait for pod2 after deletion
  Re-use inventory to ensure prune of pod2 not skipped
- Add optional klog support for e2e tests
2021-09-24 17:54:17 -07:00
Karl Isenberg 4f4bd30c57 Change YamlStringer to only accept Unstructured
- This avoids a runtime schema dependency
2021-09-24 10:35:28 -07:00
Karl Isenberg a468a88337 Unmarshal with k8s.io/apimachinery/pkg/util/yaml
- apimachinery yaml.Unmarshal fixes number types to int64 and float64
- Fix inaccurate error message in mutator
- Move YamlStringer to object pkg for reuse by other pkgs
- Fix jsonpath tests parsing y as a bool
- Fix kstatus example_test.go to actually be a test
2021-09-22 16:33:06 -07:00
Karl Isenberg 02c0fb7229 Add apply-time-mutation feature
- Detect `config.kubernetes.io/apply-time-mutation` annotation
- Parse annotation string value as YAML
- Treat source resource as a dependency
- Before applying, apply specified substitutions
- Each mutation may include one or more substitution
- Substitutions may optionally replace a token in the existing
  string value, or replace the whole value (e.g. non-strings)
- Source and target fields are specified with JSONPath expressions
- Using github.com/spyzhov/ajson because it supports mutation, and
  not just retrieval
- ApplyTimeMutator uses an in-memory ResourceCache to reduce GETs
2021-09-21 11:00:37 -07:00
Karl Isenberg 15ef662b14 Rename Marshal/Unmarshal to Format/Parse 2021-09-13 13:43:24 -07:00
Karl Isenberg 325537204d Refactor depends-on code
- Move from pkg/object to pkg/object/dependson
- Add DependencySet type to abstract some complexity
- Add Marshal/Unmarshal funcs for ObjMetadata and DependencySet
- Refactor primary API to ReadAnnotation and WriteAnnotation
- Move Write/Marshal behavior from testutil into dependson pkg
- Make constants private to encourage using functions as primary API
2021-09-13 13:20:29 -07:00
Sean Sullivan 769f5bd735 Expose cyclic dependency error 2021-08-23 11:24:05 -07:00
Mikhail Mazurskiy 4b8dd62c53
Remove strings.TrimSpace()
Names and labels cannot have any whitespace. If they do, it's an error that should be reported and fixed.
2021-08-08 11:29:43 +10:00
Mikhail Mazurskiy 275f2c27e0
Drop github.com/go-errors/errors dependency 2021-07-02 10:53:03 +10:00
Morten Torkildsen 136d1efeeb Better validation of resources prior to actuation 2021-07-01 17:02:57 -07:00
Sean Sullivan bf6bc5f346 Adds object dependency sorting and tests 2021-06-25 20:01:59 -07:00
Sean Sullivan 5b31557b7e Re-organizes pkg/object, adds annotation helpers 2021-06-23 15:49:25 -07:00
Sean Sullivan fcd4cffa80 Object graph functionality 2021-06-07 20:53:50 -07:00
Mikhail Mazurskiy 3514065221
Simplify: remove unneeded meta.Accessor() usage 2021-05-28 15:03:09 +10:00
Mikhail Mazurskiy a0d1024504
Reduce allocations in set operations 2021-05-20 15:46:24 +10:00
Morten Torkildsen ae80e561e2 Improve the event hierarchy 2021-05-18 14:58:41 -07:00
Sean Sullivan 5af6137d68 Remove special-case object name validation 2021-02-02 10:31:53 -08:00
Jingfang Liu 5d8fb759a0 update events to include indentifier and error 2020-11-17 10:07:50 -08:00
Morten Torkildsen 3ac57786f6 Update the ManifestReader interface and implementations to use Unstructured 2020-10-26 12:13:44 -07:00
Morten Torkildsen 507d11f31e Use Unstructured instead of Info 2020-10-20 15:38:13 -07:00
Sean Sullivan 2de4d86359 Applies inventory namespace first if necessary 2020-09-11 16:11:50 -07:00
Sean Sullivan 9d0ec2cd71 Allow colon in the name of RBAC resources 2020-08-19 12:14:28 -07:00
Sean R. Sullivan c9e7d329da Updates prune algorithm to use one inventory object 2020-07-29 18:33:49 -07:00
Christoph Glaubitz 3757e60840 Added parsing to objmetadata_test.TestCreateObjMetadata
This commit extends TestCreateObjMetadata. The test function does not
just verify CreateObjMetadata, but also uses ParseObjMetadata to verify
back and forth between both functions. I left specific testing of
ParseObjMetadata in place, because there might be edge cases, that
should be tested on its own.

I'm aware, that the TestCreateObjMetadata now tests two things, but I
think it makes life of maintainers and new contributors far easier.
E.g. the change I introduced in #193 would have been catched by tests,
rather than someone with a mental model of all of the code.
2020-07-15 10:52:13 +02:00
Sean Sullivan 2f6a1ef017 Validates resource names before storing in inventory 2020-07-14 14:18:56 -07:00
Sean R. Sullivan aa3a3280a2 Cleanup grouping nomenclature as inventory 2020-05-05 14:59:20 -07:00
Morten Torkildsen e88ffb1904 Wait for CRDs to become established before applying CRs 2020-04-22 15:45:19 -07:00
Morten Torkildsen b494b8e33b Use resource generation in status decisions 2020-04-21 20:43:41 -07:00
Sean R. Sullivan 52db385b3c Updates prune to use UID instead of ObjMetadata 2020-04-08 10:11:34 -07:00
Morten Torkildsen c58fbb8623 Move ObjMetadata into a separate package 2020-03-09 09:31:20 -07:00