Commit Graph

1050 Commits

Author SHA1 Message Date
Kubernetes Prow Robot a5fa28845d
Merge pull request #567 from karlkfi/karl-list-limit
feat: pagninate status poller lists
2022-03-21 17:30:14 -07:00
Karl Isenberg 95d4274c2d feat: pagninate status poller lists
- Use the client-go ListPager to paginate requests made by the
  CachingClusterReader used by the StatusPoller.
- Limit requests to 500 items per response (ListPager default)
- Benefits:
  - Reduce the length of blocking etcd operations, allowing for more
    responsiveness and higher throughput
  - Reduce risk of request timeout
  - Reduce size of response for extra large objects
2022-03-17 13:14:21 -07:00
Karl Isenberg 7aee3cc6b4 feat: error on skip
- Rewrite actuation filters to return an error with the reason for
  skipping.
- Add explicit error types for most skip errors, to make it easier to catch
  and handle them.
- Add Is method to explicit error types to allow use of errors.Is for
  recursive unwrapped matching.
- Rename InventoryPolicyFilter to InventoryPolicyPruneFilter for
  consistency with InventoryPolicyApplyFilter
- Update deletion prevention inventory-id removal to use errors.As instead
  of matching the filter name.
- Convert error structs to use pointers to allow nil errors and avoid
  copying contents.
- Update printers to handle skip errors

BREAKING CHANGE: Skipped actuation events now include an error.
BREAKING CHANGE: DeleteEvent.Reason replaced with an error.
BREAKING CHANGE: Unused InventoryNamespaceInSet error removed.
BREAKING CHANGE: InventoryOverlapError replaced with PolicyPreventedActuationError.
BREAKING CHANGE: NeedAdoptionError replaced with PolicyPreventedActuationError.
BREAKING CHANGE: NoInventoryObjError & MultipleInventoryObjError now use pointers.
2022-03-16 17:40:28 -07:00
Kubernetes Prow Robot a9d2ca7961
Merge pull request #565 from karlkfi/karl-stress-test
chore: add stress test
2022-03-10 17:53:07 -08:00
Karl Isenberg 34a8a7b334 chore: add stress test
- Stress test tests 1,000 Namespaces, CofnigMaps, & CronTabs (CR)
- Stress test is a new test suite with its own make entrypoint
- Refactor shared test code so the e2e and stress tests can both use it
- Update test client QPS to 20 (from 5)
2022-03-10 17:21:26 -08:00
Kubernetes Prow Robot d84328fc3c
Merge pull request #566 from Liujingfang1/master
Use empty object status list when the StatusPolicyNone is used
2022-03-07 17:28:33 -08:00
Jingfang Liu ebbd19deca fix: Use empty object status list when the StatusPolicyNone is used
This reduces the request size when StatusPolicyNone is used
2022-03-07 16:36:36 -08:00
Kubernetes Prow Robot 010f57af33
Merge pull request #564 from karlkfi/karl-dep-uid
fix: Avoid logging UID error for skipped apply
2022-03-05 17:58:52 -08:00
Karl Isenberg 98d3504e3d fix: Avoid logging UID error for skipped apply 2022-03-05 16:12:20 -08:00
Kubernetes Prow Robot 9caf88253a
Merge pull request #563 from karlkfi/karl-disable-status
feat: Add inventory.StatusPolicy
2022-03-04 21:04:52 -08:00
Karl Isenberg be7d6e2ab4 chore: Add e2e tests for inventory status 2022-03-04 19:09:28 -08:00
Karl Isenberg 2bdc05945d feat: Add inventory.StatusPolicy
- StatusPolicyNone disables inventory status updates.
- StatusPolicyAll fully enables inventory status updates.
- This allows an opt-out feature for working around the problem
  that adding status can make the inventory larger than the max
  etcd object size, causing the applier to exit without applying
  or pruning anything. With StatusPolicyNone, the user can still
  safely prune objects to make their inventory smaller, and then
  re-enable the status with StatusPolicyAll.
- Note: the default ConfigMap does not currently support status,
  so this only affects custom inventory impls.
2022-03-04 19:09:28 -08:00
Kubernetes Prow Robot 16847b444b
Merge pull request #560 from karlkfi/karl-fix-empty-set
fix: Allow empty set of apply objects
2022-03-02 11:05:15 -08:00
Karl Isenberg c103784000 fix: Allow empty set of apply objects
- Add e2e test to ensure the inventory is still correctly created
  and destroyed when the apply object set is empty.
2022-03-02 10:36:59 -08:00
Kubernetes Prow Robot 326b8a2c92
Merge pull request #558 from karlkfi/karl-fix-dep-filter
fix: Make DependencyFilter handle DryRun
2022-03-01 16:11:55 -08:00
Karl Isenberg 0f7f95b24a fix: Make DependencyFilter handle DryRun
DryRun skips WaitEvents. So the DependencyFilter needs to skip
checking for reconciliation if DryRun is enabled.
2022-03-01 15:48:02 -08:00
Kubernetes Prow Robot 5c6134aeac
Merge pull request #555 from karlkfi/karl-dep-filter2
feat: Add dependency filter
2022-02-28 19:27:47 -08:00
Karl Isenberg 4599e4fb00 feat: Add dependency filter
- Pass TaskContext into TaskBuilder.Build
- Combine dependency graph for apply and prune objects.
  This is required to catch dependencies that would have been deleted.
- Replace graph.SortObjs into DependencyGraph + Sort + HydrateSetList
- Replace graph.ReverseSortObjs with ReverseSetList to perform on the
  combined (apply + prune) set list.
- Add planned pending applies and prune to the InventoryManager
  before executing the task queue.
  This allows the DependencyFilter to validate against the planned
  actuation strategy of objects that haven't been applied/pruned yet.
- Add the dependency graph to the TaskContext, for the
  DependencyFilter to use.
  This can be removed in the future if the filters are managed by the
  solver.
- Make Graph.Sort non-destructive, so the graph can be re-used by the
  DependencyFilter.
- Add Graph.EdgesFrom and EdgesTo for the DependencyFilter to use.
  This requires storing the reverse edge list.
- Add an e2e test for the DependencyFilter
- Add an e2e test for the LocalNamespaceFilter

Fixes https://github.com/kubernetes-sigs/cli-utils/issues/526
Fixes https://github.com/kubernetes-sigs/cli-utils/issues/528
2022-02-28 17:31:45 -08:00
Kubernetes Prow Robot bd20d622f6
Merge pull request #556 from karlkfi/karl-set-status
fix: Inventory.SetObjectStatus
2022-02-25 21:05:22 -08:00
Karl Isenberg ca8fcbec60 fix: Inventory.SetObjectStatus
Fixes https://github.com/kubernetes-sigs/cli-utils/issues/554
2022-02-25 18:10:33 -08:00
Kubernetes Prow Robot e06456e4a1
Merge pull request #549 from karlkfi/karl-solver
chore: Change solver to decide which tasks to use
2022-02-24 15:43:56 -08:00
Karl Isenberg c5b965fda1 chore: Change solver to decide which tasks to use
- This redesign of the solver moves complexity about deciding which
  tasks to use from the applier/destroyer into the solver. This makes
  the solver more of a solver and less of a "do what your told"-er.
- One major difference is that the InvAddTask is now skipped if there
  are no objects to be applied. This makes logic sense because the
  inventory doesn't need to be expanded.
- This change unblocks unification of graph dependency resolution
  using both the apploy objects and prune objects, which will help
  distinguish between external dependencies and dependencies being
  accidentally deleted.
2022-02-24 15:21:36 -08:00
Kubernetes Prow Robot f999cd1722
Merge pull request #550 from rohitagarwal003/context
Ensure that Daemonset controller has acted on the Daemonset.
2022-02-23 19:17:33 -08:00
Kubernetes Prow Robot 7b15e3b215
Merge pull request #551 from mortent/FixInvNamespaceAlreadyExists
fix: Don't error if inventory namespace already exists
2022-02-18 14:12:22 -08:00
Morten Torkildsen 7335790f63 fix: Don't error if inventory namespace already exists 2022-02-18 10:51:55 -08:00
Rohit Agarwal 4a633f66d6 Ensure that Daemonset controller has acted on the Daemonset.
In status.Compute() -> checkGenericProperties() -> checkGeneration(),
we check that metadata.generation is equal to status.observedGeneration
but don't return an InProgress status if either of them is unset.

This may be reasonable for generic resource where we are unsure if these
fields would be set, but for daemonsets, we *know* that these fields get
set by the controller and so we should ensure that.

This addresses https://github.com/kubernetes-sigs/cli-utils/issues/548
2022-02-17 21:56:39 -08:00
Kubernetes Prow Robot cb75cc801b
Merge pull request #530 from Liujingfang1/master
capture the apply/reconcile status in the inventory object
2022-02-16 14:18:45 -08:00
Jingfang Liu 6bd4b565e8 Capture the apply status for individual resources after each apply.
If the inventory object supports status field, it is updated
after setting the inventory list at then of one apply process.

BREAKING CHANGE: Update the inventory client and inventory interfaces
to pass the apply/reconcile status.
2022-02-16 12:02:02 -08:00
Kubernetes Prow Robot b450b79991
Merge pull request #547 from rohitagarwal003/context
Don't store Context inside a struct, pass it explicitly to each function that needs it.
2022-02-16 07:52:38 -08:00
Rohit Agarwal 9a50bc7031 Don't store Context inside a struct, pass it explicitly to each function that needs it.
This is from the context package documentation: https://pkg.go.dev/context

Tested by running `go test ./...` inside `pkg/kstatus`.
2022-02-16 01:03:20 -08:00
Kubernetes Prow Robot f155aaf7fd
Merge pull request #538 from karlkfi/karl-lint-upgrade
chore: Upgrade golint, replace deprecated linters, and fix new lint errors
2022-02-15 22:10:36 -08:00
Karl Isenberg 69c2289534 chore: Disable stutter linter on actuation enums 2022-02-15 15:28:54 -08:00
Karl Isenberg 4b30ea4e83 chore: Rename fake.FakeStatusReader to fake.StatusReader 2022-02-15 15:22:42 -08:00
Karl Isenberg f265615342 chore: Rename fake.FakeClusterReader to fake.ClusterReader 2022-02-15 15:17:19 -08:00
Karl Isenberg 94c0b1c6f0 chore: export TaskStatusRunner - used externally 2022-02-15 14:50:34 -08:00
Karl Isenberg 5d235016f5 chore: unexport table printer internal types 2022-02-15 14:50:34 -08:00
Karl Isenberg 026b9cc5f8 chore: return exposed types from error matchers 2022-02-15 14:50:34 -08:00
Karl Isenberg eb749e933c chore: allow stuttering for PruneEventFactory 2022-02-15 14:50:34 -08:00
Karl Isenberg 8dfc33dd8e chore: rename printers to avoid stuttering 2022-02-15 14:50:34 -08:00
Karl Isenberg 280f3e2c8f chore: export table & event printer for extension 2022-02-15 14:50:34 -08:00
Karl Isenberg 6ce2c31c3e chore: export manifestreader.FakeLoader for tests 2022-02-15 14:50:34 -08:00
Karl Isenberg 4e94795493 chore: refactor inventory pkg to avoid stuttering 2022-02-15 14:48:39 -08:00
Karl Isenberg e2f4d4cada chore: rename event.EventType to reduce stuttering
BREAKING CHANGE: renamed Event.EventType to Event.Type
2022-02-15 14:33:52 -08:00
Karl Isenberg 7f561ae8b8 chore: result stuttering in info pkg 2022-02-15 14:30:10 -08:00
Karl Isenberg 26b16473c7 chore: reduce stuttering in cmd pkg 2022-02-15 14:30:10 -08:00
Karl Isenberg 3843c7e122 chore: replace += 1 with ++ 2022-02-15 14:30:09 -08:00
Karl Isenberg 57b5483e5b chore: Upgrade linter and replace deprecated pkgs 2022-02-15 14:30:09 -08:00
Kubernetes Prow Robot 8575b8be6e
Merge pull request #546 from karlkfi/karl-inventory-discovery
fix: Always copy ResourceVersion before update
2022-02-15 13:53:18 -08:00
Kubernetes Prow Robot 0680d3cb90
Merge pull request #542 from karlkfi/karl-actuation-api
chore: Move Inventory object to pkg/apis/actuation
2022-02-15 13:17:19 -08:00
Karl Isenberg 8bc791b048 fix: Always copy ResourceVersion before update
- Fix FakeDynamicClient to return an object from deletes
- Fix FakeDynamicClient to return DeepCopy to avoid mutation
- Fix InventoryClient to return a DeepCopy of the input when
  dry-run is enabled.
2022-02-15 13:05:38 -08:00