Commit Graph

264 Commits

Author SHA1 Message Date
Stefan Prodan dde74d9ea5
Change copyright to Flux authors
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-10-27 18:49:02 +02:00
Hidde Beydals 2b861622b1 Move dedicated watchers to in-controller watches
This prevents the resources from getting annotated, and instead uses
the `handler.EnqueueRequestsFromMapFunc` to queue requests based on
changes to the source objects.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2020-10-27 13:05:34 +01:00
Stefan Prodan 8bb4f4c80b
Merge pull request #151 from ordovicia/supress-healthchecks-when-no-changes
Suppress health check events when no changes made
2020-10-22 14:54:43 +03:00
Michael Bridgen e78726f97c Factor out requestReconciliation
There were two of these methods, identical aside from the receiver --
and the requirement on the receiver was just that it was a client that
knows the Kustomization type.
2020-10-22 11:37:13 +01:00
Hidehito Yabuuchi 5ed7463552 Suppress health check events regardless of dependencies
Because if a dependency is failing, the reconciliation will never get to
the health check part.
2020-10-21 19:28:44 +09:00
Hidehito Yabuuchi 4df55dde11 Suppress health check events when no changes made 2020-10-21 19:05:04 +09:00
Hidde Beydals 3315e66586 Switch to controller-runtime utils for finalizers 2020-10-21 11:15:30 +02:00
leigh capili 7a1c06571a Implement non-caching, per-kustomization GC-client/statusPoller for cross-cluster kubeconfigs 2020-10-15 09:44:44 -06:00
Stefan Prodan 7ec444545b
Merge pull request #145 from fluxcd/fix-error-event
Set correct status on failure events
2020-10-15 16:29:52 +03:00
Stefan Prodan e1c7fb1bd0
Merge pull request #143 from fluxcd/yaml-validation
Validate manifests when generating kustomization.yaml
2020-10-15 16:23:00 +03:00
Stefan Prodan 826051ac54
Set correct status on failure events
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-10-15 16:12:58 +03:00
Stefan Prodan ec0c43d311
Validate manifests when generating kustomization.yaml
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-10-15 14:18:20 +03:00
Stefan Prodan 4fc1466443
Fix status reporting when the source is not found
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-10-15 10:12:06 +03:00
Stefan Prodan d4cef2f046
Use events and metrics from fluxcd/pkg/runtime
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-10-13 12:37:45 +03:00
Stefan Prodan deb902a13f
Record reconcile duration as Prometheus histogram
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-10-13 09:50:00 +03:00
Stefan Prodan a82352e892
Make the condition metric exporter generic
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-10-13 08:42:04 +03:00
Stefan Prodan b2d19e469f
Set ready metric to unknown when condition is missing
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-10-09 18:53:49 +03:00
Stefan Prodan ef360ebc3e
Add metrics recorder test
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-10-09 18:47:59 +03:00
Stefan Prodan 6223abdd06
Record the ready status as Prometheus metric
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-10-09 16:27:43 +03:00
stefanprodan 50104826ae Promote API to v1beta1 2020-09-30 19:10:27 +03:00
stefanprodan 7b8fef2984 Implement `fluxcd/pkg/meta/api` in APIs 2020-09-30 14:01:17 +03:00
stefanprodan 4a283d74b4 Implement reconciliation on remote clusters 2020-09-30 11:36:37 +03:00
stefanprodan 45bfe6dfe5 Record last handled reconcile at annotation 2020-09-29 21:14:19 +03:00
stefanprodan e5e2cd7dcb Implement namespace overriding 2020-09-29 18:56:24 +03:00
stefanprodan 1978f987de Update kustomize/api to v0.6.1 2020-09-19 15:16:36 +03:00
stefanprodan 0d6f715efc Add support for S3 bucket sources 2020-09-19 13:32:38 +03:00
Philip Laine 8fed231d2b Change event message 2020-09-19 11:53:17 +02:00
Philip Laine cff825d6cf Send event when reconcile succeeds with update metadata 2020-09-19 11:53:15 +02:00
stefanprodan 2b9370e658 Normalize the ready condition reasons
- use reconciliation instead of apply for condition reasons
- add the reconciliation errors to the condition message
- trim the condition message to 4000 characters
- update the API docs and readme with the new condition reasons
2020-09-17 12:23:50 +03:00
stefanprodan fcd0155550 Add the deleted object kind to GC events 2020-09-16 10:39:34 +03:00
stefanprodan 1679d9cd35 Label objects with the kustomization name/namespace 2020-09-16 10:00:14 +03:00
stefanprodan 544e7cf0eb Use manifests checksum in GC snapshot 2020-09-16 09:34:22 +03:00
Hidde Beydals e46add39ed Support dependency references to other namespaces 2020-09-15 15:18:34 +02:00
stefanprodan df1761f8f5 Set kubectl cache dir to /tmp 2020-09-15 13:28:14 +03:00
stefanprodan 9363703634 Refactor kstatus health check 2020-09-14 10:48:24 +03:00
Stefan Prodan d105ecff2f
Merge pull request #101 from phillebaba/feature/kstatus
Implement kstatus for health checks
2020-09-14 09:43:24 +03:00
Philip Laine 042f9dcbae Change poll rate 2020-09-13 15:41:20 +02:00
Hidde Beydals 9140483c8d Include PATCH rule for events in manager-role
During high custom resource count / low interval tests, I was greated
with a `cannot patch resource "events"` message. This happened due to
event compaction, where it will perform a patch instead of a create.
By giving the role the permission to do so this should no longer pose
a problem.
2020-09-10 21:19:44 +02:00
Stefan Prodan 845947c814
Merge pull request #103 from fluxcd/use-pkg-for-runtime-bits
Use annotation and predicate from fluxcd/pkg
2020-09-10 15:56:20 +03:00
Michael Bridgen 47f5d0b4b0 Use annotation and predicate from fluxcd/pkg
The reconcileAt annotation and the predicate that recognises it are
now in fluxcd/pkg/runtime. This removes (near) duplicate definitions
in favour of using those.
2020-09-10 13:43:49 +01:00
stefanprodan 087be46136 Refactor garbage collector 2020-09-10 14:26:14 +03:00
Philip Laine 2baaba29a5 Fix failing tests 2020-09-08 23:23:42 +02:00
Philip Laine 4e45c916e2 Refactor health check completed logic 2020-09-08 23:21:47 +02:00
Philip Laine ea627e3448 Change health check from WorkloadReference to CrossNamespaceObjectReference 2020-09-07 23:31:31 +02:00
Philip Laine ca80431ef8 Change kind ref in health checks to group kind 2020-09-06 20:43:00 +02:00
Philip Laine 9c54e2cb30 Implement kstatus check 2020-09-06 17:59:25 +02:00
stefanprodan 38f2ec2862 Fix GC snapshot 2020-09-05 10:35:29 +03:00
Hidde Beydals 07f13e56eb GPG decryption in contained environment 2020-09-02 15:42:02 +02:00
stefanprodan c605ccf6d2 Implement Mozilla SOPS decryption 2020-09-01 15:51:22 +03:00
stefanprodan 7e06af6d4b Refactor garbage collection 2020-09-01 13:35:05 +03:00
stefanprodan b00a841162 Refactor kustomization file generation 2020-09-01 12:34:50 +03:00
stefanprodan f9b748378b Fix GC ignore unknown resource kind 2020-08-31 12:04:34 +03:00
stefanprodan 2963ad27e0 Implement kustomize create and build
- Replace kustomize shell-out with kustomize/api
- Remove kustomize binary from Dockerfile
- Update kubectl binary to 1.19.0
2020-08-31 11:06:47 +03:00
Hidde Beydals dbbb672570
Merge pull request #82 from fluxcd/dedicated-pkg
Make use of dedicated fluxcd/pkg/testserver module
2020-08-18 12:47:55 +02:00
Hidde Beydals 3187928893 Make use of dedicated fluxcd/pkg/testserver
This includes a dependency change from github/fluxcd/source-controller
to github.com/fluxcd/source-controller/api (v0.0.9) to decrease the
amount of indirect dependencies.
2020-08-18 12:35:13 +02:00
stefanprodan 65f511a58b Allow kustomizations to load files from outside their root
Add git OS package to enable kustomize remote URLs
2020-08-18 11:34:16 +03:00
Hidde Beydals 5905f3a85f Change CRD domain to 'kustomize.toolkit.fluxcd.io'
Due to required domain changes for the helm-controller so that it
can co-exist in a cluster with the Helm Operator, other Toolkit
components are moving to a *.toolklit.fluxcd.io domain too.
2020-07-30 22:40:16 +02:00
stefanprodan 40ff1a7038 Rename apply succeeded reason 2020-07-30 14:48:29 +03:00
stefanprodan 3d7687e216 Set event reason from ready status reason 2020-07-30 14:13:46 +03:00
stefanprodan c3599135c9 Use SA token for impersonation 2020-07-25 09:45:31 +03:00
Hidde Beydals 4231b9b6c8 Take observed generation of deps into account
To prevent operations from being performed while they are "Ready",
but the latest generation of the depenceny has not been reconciled
yet.
2020-07-23 17:15:33 +02:00
Hidde Beydals 61c1dce06c Refactor garbage collector to finalizer 2020-07-16 15:06:37 +02:00
stefanprodan 2a025c49cb Implement cross-namespace source references 2020-07-16 10:28:15 +03:00
stefanprodan 0934fda436 Improve error handling and reporting
- return reconciliation error so that controller runtime metrics record failures
- change structure logging labels to match the controller runtime format
- prevent spurious info events by issuing events only when revision changes
2020-07-15 15:00:45 +03:00
stefanprodan f79ac6dd44 Rename syncAt annotation to reconcileAt
Use `fluxcd.io/reconcileAt` annotation across all controllers.
2020-07-13 17:14:57 +03:00
stefanprodan 47d239312a Update source-controller to v0.0.5 2020-07-13 17:14:13 +03:00
stefanprodan 706e85ecec Use fluxcd/pkg/untar for artifact extraction 2020-07-10 10:44:57 +03:00
stefanprodan a69af9d57c Add last attempted revision to status 2020-07-10 09:54:45 +03:00
stefanprodan a7fae2d5c7 Update source-controller to v0.0.2 2020-07-06 17:43:44 +03:00
stefanprodan 90c87646c4 Add namespace reconciliation test 2020-07-06 17:37:17 +03:00
stefanprodan a7877ad665 Init unit tests with ginkgo 2020-07-06 14:49:20 +03:00
stefanprodan 4d3a3a7bae Emit events for garbage collection 2020-07-02 11:52:03 +03:00
stefanprodan 406ce977a4 Add revision to events metadata 2020-07-02 08:01:06 +03:00
stefanprodan 2ebd5b6450 Implement event recording
- emit Kubernetes events for reconciliation actions
- forward events to notification controller
- remove the Profile API/CRD
2020-07-01 20:39:48 +03:00
stefanprodan ab15f3c185 Migrate to fluxcd/pkg 2020-06-30 17:25:03 +03:00
Hidde Beydals 3f6edc5280 Check suspend status before progressing
When a release is marked as suspended no operations should be performed
for the release except acknowledging the reconciler has noticed it has
been suspended. This means the suspend check should happen _before_ the
Kustomization is marked as progressing (which should be an indicator
reconciliation is enabled for the Kustomization).
2020-06-30 11:15:43 +02:00
stefanprodan d11e76d322 Configurable dependency requeuing
Add command argument for configuring the interval at which failing dependencies are reevaluated.
2020-06-20 10:30:37 +03:00
stefanprodan 9c18d044de Fix artifact extraction 2020-06-10 11:55:54 +03:00
stefanprodan 1f25b5852b Watch resources in the runtime namespace only 2020-06-09 15:53:20 +03:00
stefanprodan 0c47dfd549 Retry apply on missing Kind 2020-05-13 19:04:00 +03:00
stefanprodan d80ff203ee Fix assignment to entry in nil map 2020-05-11 22:29:52 +03:00
stefanprodan 7e584dcbf2 Improve health checking
Wait for a deployment to be created before running the liveness check
2020-05-09 12:18:44 +03:00
Hidde Beydals fdbf69cea5 Sort dependencies with Tarjan's SCCS algorithm 2020-05-08 14:30:07 +02:00
Hidde Beydals a8cf35aa56 controllers: GitRepositoryWatcher log improvements
* log namespaced Kustomization name
* do not log success message on annotation failure
* change log messages to better reflect reality
2020-05-06 11:37:17 +02:00
stefanprodan 8763bbec25 Update GC API spec and examples 2020-05-05 17:19:38 +03:00
stefanprodan f96f3e326f Implement pruning based on snapshots 2020-05-05 16:56:27 +03:00
stefanprodan dbe3e69508 Introduce garbage collection based on snapshots 2020-05-05 15:42:49 +03:00
stefanprodan 6b1bbb53b8 Retry sync trigger 2020-05-02 12:06:48 +03:00
stefanprodan 5a73ba7477 Implement SA impersonation 2020-05-01 22:13:54 +03:00
stefanprodan f2fe44eaba Refactor GC
- remove namespaced objects before cluster ones
2020-05-01 20:39:51 +03:00
stefanprodan af68aa0e9e build: install tar 2020-04-28 16:25:05 +03:00
stefanprodan deb9161f94 gc: remove non-namespaced objects first 2020-04-28 14:20:02 +03:00
stefanprodan 86f6e84d59 Report progressing status
Set ready condition to unknown while the reconciliation is progressing.
This allows other operators to wait for a sync to complete.
2020-04-27 00:39:48 +03:00
stefanprodan 696658dcc9 Record last applied revision
- add LastAppliedRevision field to Status
- set LastAppliedRevision after a successful sync
- add Status section spec
2020-04-24 00:44:32 +03:00
stefanprodan b88626c0cc Generate label transformers for the prune selectors 2020-04-23 12:52:35 +03:00
stefanprodan d37741e633 Implement kustomization.yaml generation 2020-04-22 20:34:40 +03:00
stefanprodan 81ff97bc8d Implement Slack and Discord alerting 2020-04-21 14:09:26 +03:00
stefanprodan 057096853e Add profile controller 2020-04-21 14:03:59 +03:00
stefanprodan fe88706613 Introduce dependencies between kustomizations
- add DependsOn field to API
- implement ordered apply of kustomizations based on dependencies and readiness status
- retry execution every half a minute for kustomizations with dependencies that do not meet the ready condition
- add interdependent kustomizations to config testdata
2020-04-19 18:27:32 +03:00
stefanprodan b10a0fee41 Remove all API resource on object deletion 2020-04-19 18:08:18 +03:00
stefanprodan 508b7cca6a Refactor controller sync
- extract sync operations to dedicated functions
- add health check failed reason
2020-04-19 11:04:05 +03:00
stefanprodan 8fd0d45916 Add timeout to API
- add timeout field to API (defaults to interval duration)
- set timeout for validation, apply and heath check operations
- set concurrent reconciles to 4 (avoid health checks from delaying other operations)
2020-04-19 09:44:32 +03:00
stefanprodan 0dae17ffd6 Introduce health assessment
- add health checks to API
- implement rollout status check
2020-04-19 01:28:58 +03:00
stefanprodan a3b8bcb310 Allow kustomize execution to be suspended 2020-04-18 18:16:20 +03:00
stefanprodan 3f50c71af4 Introduce validation
- add validation field to API
- perform dry-run apply
- add validation strategies to docs
2020-04-18 14:14:51 +03:00
stefanprodan 6d28f7c9b2 Introduce JSON logging
- add JSON log format cmd flag
- format apply output as a map of object/operation
- add apply logs to docs
2020-04-18 11:11:44 +03:00
stefanprodan 58846979bc Trigger sync on artifact creation 2020-04-17 19:50:17 +03:00
stefanprodan eee47333cf Implement garbage collection
- add GarbageCollect predicate
- remove the Kubernetes objects previously applied on the cluster, based on the prune label select
- add garbage collection spec to docs
2020-04-17 13:25:36 +03:00
stefanprodan 681349f425 Set apply timeout to interval duration 2020-04-17 11:15:39 +03:00
stefanprodan 9f7d995577 Add locking to kustomization apply 2020-04-17 10:30:40 +03:00
stefanprodan c564555eb0 Use Source interface instead of Git concrete type
- use TypedLocalObjectReference instead of LocalObjectReference for referencing a source
- update docs for GitRepository kind
- update source API to 0.0.1-alpha.1
2020-04-16 18:30:18 +03:00
stefanprodan ee74f8e329 Trigger kustomization apply on source changes
- detect revision changes in Git repositories
- index kustomizations based on source
- trigger kustomizations on revision changes
2020-04-16 10:40:28 +03:00
stefanprodan 17ec746136 Implement kustomization reconciler
- fetch artifact from source controller based on the referenced Git repository
- run kustomize build for the specified path
- apply kustomization on cluster
- prune objects based on label selector
2020-04-15 15:20:21 +03:00
stefanprodan 3df6bfebfa Init controller with kubebuilder 2020-04-15 09:58:33 +03:00