Commit Graph

179 Commits

Author SHA1 Message Date
Stefan Prodan a77d6cb96e
Allow the controller to be run locally
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-02-14 11:25:18 +02:00
Stefan Prodan f3d9c36691
Add support for variable substitutions
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-02-11 18:17:53 +02:00
Stefan Prodan e99f328200
Add source kind and name to not found error
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-02-08 15:20:13 +02:00
Stefan Prodan 5e17dd48f4
Add Healthy status condition
- record the last health assessment result in a dedicated status condition
- use the condition status when issuing events to prevent notifications spam

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-02-02 14:59:06 +02:00
Christian Hernvall d0f2dc6e4a
Support impersonation for validation
Signed-off-by: Christian Hernvall <c.hernvall@yubico.com>
2021-01-29 14:56:45 +01:00
Stefan Prodan 100d362ce7
Fix reconciliation retry scheduler
Log the reconciliation error instead of returning it, so that controller-runtime doesn't requeue immediately. Reconciliation failures should be scheduled at the specified retry interval.

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-01-25 13:17:38 +02:00
Stefan Prodan dfba88ccc1
Requeue a failed reconciliation based on retry interval
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-01-22 15:50:45 +02:00
Stefan Prodan 085588b632
Disable kyaml
Workaround for upstream bug: https://github.com/kubernetes-sigs/kustomize/issues/3446

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-01-18 14:14:23 +02:00
Stefan Prodan 7859a639ed
Update kustomize/api to v0.7.2
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-01-17 22:00:51 +02:00
Hidde Beydals c6353b4afd Upgrade runtime package to v0.6.2
To include a bug fix to the `ReconcilateAtChangedPredicate`
and renaming to `ReconcileRequestedPredicate`.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-01-14 12:34:10 +01:00
Stefan Prodan 5a0e5abd9a
Setup impersonation for GC and health checks
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-01-04 11:51:04 +02:00
Stefan Prodan 696f91d380
Refactor impersonation
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-01-04 11:50:59 +02:00
Stefan Prodan f14cd2323c
Upgrade controller-runtime to v0.7.0
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-01-04 11:50:58 +02:00
Hidehito Yabuuchi 8b132d18be Emit healthcheck event when Kustomization was not ready
Signed-off-by: Hidehito Yabuuchi <hdht.ybuc@gmail.com>
2020-12-17 17:50:05 +09:00
Stefan Prodan 8be1e169d2
Merge pull request #210 from fluxcd/gc-prune
Refactor garbage collection
2020-12-16 16:29:37 +02:00
Hidde Beydals d7a0deac97 Write KubeConfig to tmp file in working dir
Instead of using the name of the secret, as this can cause unexpected
collisions in edge case scenarios.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2020-12-16 12:59:19 +01:00
Hidde Beydals 6a4bf74cf3 Add safe guards for relative paths
This commit ensures that relative (user configurable) paths never
traverse outside their working directory.

It does _not_ provide protection against path traversal within
`kustomization.yaml` files.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2020-12-16 12:44:13 +01:00
Stefan Prodan 59845b0891
Refactor garbage collection
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-12-16 13:09:30 +02:00
Stefan Prodan d65ea71699
Refactor apply error reporting
- filter kubectl apply output and extract errors
- limit apply output to 20K charts (avoid reaching max etcd size)
- log kubectl exit code when the process is killed

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-12-14 17:34:55 +02:00
Sylvain Rabot 3f34e450ec
Set --field-manager when applying
Signed-off-by: Sylvain Rabot <s.rabot@lectra.com>
2020-11-26 13:32:26 +01:00
Stefan Prodan 5a9e122e77
Set progressing after source readiness check
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-11-26 09:29:34 +02:00
Stefan Prodan 2312d69a51
Do not mark suspended resource as not ready
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-11-26 09:00:52 +02:00
Stefan Prodan 73546e92d2
Requeue after interval on source not found errors
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-11-24 11:35:47 +02:00
Stefan Prodan 0c9170241f
Use ServiceAccountName for impersonation
Drop the ServiceAccount field in favour of ServiceAccountName to prevent privilege escalation in multi-tenancy environments.

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-11-20 10:58:24 +02:00
Aurel Canciu ac6cc88e29
Refactor to adopt k8s standardized Condition type
Updates to use metav1.Condition type and removes references for
deprecated corev1.Condition* constants and uses the new k8s api/meta
helpers in place of the old pkg/apis/meta types.

Signed-off-by: Aurel Canciu <aurelcanciu@gmail.com>
2020-11-19 19:49:04 +02:00
Hidde Beydals c53e5eeab2 Use DeletionTimestamp for prune and readiness
Signed-off-by: Hidde Beydals <hello@hidde.co>
2020-11-19 12:23:48 +01:00
Stefan Prodan 5aded37954
Patch status sub-resource
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-11-12 13:04:16 +02:00
Stefan Prodan 2ba6252d76
Allow disabling validation
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-11-12 12:18:04 +02:00
Stefan Prodan 8ec066bf37
Use latest generation when updating final status
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-11-04 13:04:01 +02:00
Stefan Prodan 46f828ff43
Omit checksum label if GC is disabled
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-11-03 15:22:25 +02:00
Hidde Beydals 26db48b9f0 Compare artifact <> Kustomizations in enqueuers
The reason for this is the `EnqueueRequestsFromMapFunc` calling the
enqueuer for _both_ the old and the new object, and we only want to act
on the ones that contain a revision different from the one that we have
recorded in the status object of the `Kustomization`.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2020-10-28 22:36:34 +01:00
Hidde Beydals 752b6b6bf1 Bundle revision change predicates into one
Signed-off-by: Hidde Beydals <hello@hidde.co>
2020-10-28 21:56:44 +01:00
Hidde Beydals 824af43beb Add Kustomization finalizers permissions
Signed-off-by: Hidde Beydals <hello@hidde.co>
2020-10-28 13:31:08 +01:00
Hidde Beydals 6bd8971cec Re-add status GET permissions
Signed-off-by: Hidde Beydals <hello@hidde.co>
2020-10-28 13:21:53 +01:00
Hidde Beydals e8d2c31b40 Log reconciliation requests by watcher
Signed-off-by: Hidde Beydals <hello@hidde.co>
2020-10-28 13:11:17 +01:00
Stefan Prodan dde74d9ea5
Change copyright to Flux authors
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-10-27 18:49:02 +02:00
Hidde Beydals 2b861622b1 Move dedicated watchers to in-controller watches
This prevents the resources from getting annotated, and instead uses
the `handler.EnqueueRequestsFromMapFunc` to queue requests based on
changes to the source objects.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2020-10-27 13:05:34 +01:00
Stefan Prodan 8bb4f4c80b
Merge pull request #151 from ordovicia/supress-healthchecks-when-no-changes
Suppress health check events when no changes made
2020-10-22 14:54:43 +03:00
Michael Bridgen e78726f97c Factor out requestReconciliation
There were two of these methods, identical aside from the receiver --
and the requirement on the receiver was just that it was a client that
knows the Kustomization type.
2020-10-22 11:37:13 +01:00
Hidehito Yabuuchi 5ed7463552 Suppress health check events regardless of dependencies
Because if a dependency is failing, the reconciliation will never get to
the health check part.
2020-10-21 19:28:44 +09:00
Hidehito Yabuuchi 4df55dde11 Suppress health check events when no changes made 2020-10-21 19:05:04 +09:00
Hidde Beydals 3315e66586 Switch to controller-runtime utils for finalizers 2020-10-21 11:15:30 +02:00
leigh capili 7a1c06571a Implement non-caching, per-kustomization GC-client/statusPoller for cross-cluster kubeconfigs 2020-10-15 09:44:44 -06:00
Stefan Prodan 7ec444545b
Merge pull request #145 from fluxcd/fix-error-event
Set correct status on failure events
2020-10-15 16:29:52 +03:00
Stefan Prodan 826051ac54
Set correct status on failure events
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-10-15 16:12:58 +03:00
Stefan Prodan 4fc1466443
Fix status reporting when the source is not found
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-10-15 10:12:06 +03:00
Stefan Prodan d4cef2f046
Use events and metrics from fluxcd/pkg/runtime
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-10-13 12:37:45 +03:00
Stefan Prodan deb902a13f
Record reconcile duration as Prometheus histogram
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-10-13 09:50:00 +03:00
Stefan Prodan a82352e892
Make the condition metric exporter generic
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-10-13 08:42:04 +03:00
Stefan Prodan b2d19e469f
Set ready metric to unknown when condition is missing
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-10-09 18:53:49 +03:00
Stefan Prodan ef360ebc3e
Add metrics recorder test
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-10-09 18:47:59 +03:00
Stefan Prodan 6223abdd06
Record the ready status as Prometheus metric
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-10-09 16:27:43 +03:00
stefanprodan 50104826ae Promote API to v1beta1 2020-09-30 19:10:27 +03:00
stefanprodan 7b8fef2984 Implement `fluxcd/pkg/meta/api` in APIs 2020-09-30 14:01:17 +03:00
stefanprodan 4a283d74b4 Implement reconciliation on remote clusters 2020-09-30 11:36:37 +03:00
stefanprodan 45bfe6dfe5 Record last handled reconcile at annotation 2020-09-29 21:14:19 +03:00
stefanprodan 1978f987de Update kustomize/api to v0.6.1 2020-09-19 15:16:36 +03:00
stefanprodan 0d6f715efc Add support for S3 bucket sources 2020-09-19 13:32:38 +03:00
Philip Laine 8fed231d2b Change event message 2020-09-19 11:53:17 +02:00
Philip Laine cff825d6cf Send event when reconcile succeeds with update metadata 2020-09-19 11:53:15 +02:00
stefanprodan 2b9370e658 Normalize the ready condition reasons
- use reconciliation instead of apply for condition reasons
- add the reconciliation errors to the condition message
- trim the condition message to 4000 characters
- update the API docs and readme with the new condition reasons
2020-09-17 12:23:50 +03:00
stefanprodan 544e7cf0eb Use manifests checksum in GC snapshot 2020-09-16 09:34:22 +03:00
Hidde Beydals e46add39ed Support dependency references to other namespaces 2020-09-15 15:18:34 +02:00
stefanprodan df1761f8f5 Set kubectl cache dir to /tmp 2020-09-15 13:28:14 +03:00
stefanprodan 9363703634 Refactor kstatus health check 2020-09-14 10:48:24 +03:00
Stefan Prodan d105ecff2f
Merge pull request #101 from phillebaba/feature/kstatus
Implement kstatus for health checks
2020-09-14 09:43:24 +03:00
Philip Laine 042f9dcbae Change poll rate 2020-09-13 15:41:20 +02:00
Hidde Beydals 9140483c8d Include PATCH rule for events in manager-role
During high custom resource count / low interval tests, I was greated
with a `cannot patch resource "events"` message. This happened due to
event compaction, where it will perform a patch instead of a create.
By giving the role the permission to do so this should no longer pose
a problem.
2020-09-10 21:19:44 +02:00
Stefan Prodan 845947c814
Merge pull request #103 from fluxcd/use-pkg-for-runtime-bits
Use annotation and predicate from fluxcd/pkg
2020-09-10 15:56:20 +03:00
Michael Bridgen 47f5d0b4b0 Use annotation and predicate from fluxcd/pkg
The reconcileAt annotation and the predicate that recognises it are
now in fluxcd/pkg/runtime. This removes (near) duplicate definitions
in favour of using those.
2020-09-10 13:43:49 +01:00
stefanprodan 087be46136 Refactor garbage collector 2020-09-10 14:26:14 +03:00
Philip Laine 4e45c916e2 Refactor health check completed logic 2020-09-08 23:21:47 +02:00
Philip Laine ea627e3448 Change health check from WorkloadReference to CrossNamespaceObjectReference 2020-09-07 23:31:31 +02:00
Philip Laine ca80431ef8 Change kind ref in health checks to group kind 2020-09-06 20:43:00 +02:00
Philip Laine 9c54e2cb30 Implement kstatus check 2020-09-06 17:59:25 +02:00
stefanprodan 38f2ec2862 Fix GC snapshot 2020-09-05 10:35:29 +03:00
Hidde Beydals 07f13e56eb GPG decryption in contained environment 2020-09-02 15:42:02 +02:00
stefanprodan c605ccf6d2 Implement Mozilla SOPS decryption 2020-09-01 15:51:22 +03:00
stefanprodan 7e06af6d4b Refactor garbage collection 2020-09-01 13:35:05 +03:00
stefanprodan b00a841162 Refactor kustomization file generation 2020-09-01 12:34:50 +03:00
stefanprodan f9b748378b Fix GC ignore unknown resource kind 2020-08-31 12:04:34 +03:00
stefanprodan 2963ad27e0 Implement kustomize create and build
- Replace kustomize shell-out with kustomize/api
- Remove kustomize binary from Dockerfile
- Update kubectl binary to 1.19.0
2020-08-31 11:06:47 +03:00
stefanprodan 65f511a58b Allow kustomizations to load files from outside their root
Add git OS package to enable kustomize remote URLs
2020-08-18 11:34:16 +03:00
Hidde Beydals 5905f3a85f Change CRD domain to 'kustomize.toolkit.fluxcd.io'
Due to required domain changes for the helm-controller so that it
can co-exist in a cluster with the Helm Operator, other Toolkit
components are moving to a *.toolklit.fluxcd.io domain too.
2020-07-30 22:40:16 +02:00
stefanprodan 40ff1a7038 Rename apply succeeded reason 2020-07-30 14:48:29 +03:00
stefanprodan 3d7687e216 Set event reason from ready status reason 2020-07-30 14:13:46 +03:00
stefanprodan c3599135c9 Use SA token for impersonation 2020-07-25 09:45:31 +03:00
Hidde Beydals 4231b9b6c8 Take observed generation of deps into account
To prevent operations from being performed while they are "Ready",
but the latest generation of the depenceny has not been reconciled
yet.
2020-07-23 17:15:33 +02:00
Hidde Beydals 61c1dce06c Refactor garbage collector to finalizer 2020-07-16 15:06:37 +02:00
stefanprodan 2a025c49cb Implement cross-namespace source references 2020-07-16 10:28:15 +03:00
stefanprodan 0934fda436 Improve error handling and reporting
- return reconciliation error so that controller runtime metrics record failures
- change structure logging labels to match the controller runtime format
- prevent spurious info events by issuing events only when revision changes
2020-07-15 15:00:45 +03:00
stefanprodan 47d239312a Update source-controller to v0.0.5 2020-07-13 17:14:13 +03:00
stefanprodan 706e85ecec Use fluxcd/pkg/untar for artifact extraction 2020-07-10 10:44:57 +03:00
stefanprodan a69af9d57c Add last attempted revision to status 2020-07-10 09:54:45 +03:00
stefanprodan 4d3a3a7bae Emit events for garbage collection 2020-07-02 11:52:03 +03:00
stefanprodan 406ce977a4 Add revision to events metadata 2020-07-02 08:01:06 +03:00
stefanprodan 2ebd5b6450 Implement event recording
- emit Kubernetes events for reconciliation actions
- forward events to notification controller
- remove the Profile API/CRD
2020-07-01 20:39:48 +03:00
stefanprodan ab15f3c185 Migrate to fluxcd/pkg 2020-06-30 17:25:03 +03:00
Hidde Beydals 3f6edc5280 Check suspend status before progressing
When a release is marked as suspended no operations should be performed
for the release except acknowledging the reconciler has noticed it has
been suspended. This means the suspend check should happen _before_ the
Kustomization is marked as progressing (which should be an indicator
reconciliation is enabled for the Kustomization).
2020-06-30 11:15:43 +02:00
stefanprodan d11e76d322 Configurable dependency requeuing
Add command argument for configuring the interval at which failing dependencies are reevaluated.
2020-06-20 10:30:37 +03:00