Commit Graph

264 Commits

Author SHA1 Message Date
Stefan Prodan a49bbf9ed3
Skip garbage collection of objects with owner references
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-08-18 12:49:28 +03:00
Somtochi Onyekwere b1071bf871 Add kustomize transformer tests
Signed-off-by: Somtochi Onyekwere <somtochionyekwere@gmail.com>
2021-08-16 15:40:14 +01:00
Somtochi Onyekwere 659e660b6d Tests for replacement transformer
Signed-off-by: Somtochi Onyekwere <somtochionyekwere@gmail.com>
2021-08-13 07:28:19 +01:00
Stefan Prodan 5d1cb91841
Make the kubeconfig secrets compatible with SOPS
Add `values.yaml` to the supported kubeconfig secret key names in order for SOPS to correctly detect the storage format based on the file extension.

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-08-04 14:43:33 +03:00
Sunny 930e7d398e
Remove old util ObjectKey
Replace old utility function ObjectKey with controller-runtime's
client.ObjectKeyFromObject.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2021-08-01 04:27:35 +05:30
Max Jonas Werner 08920d3286
fix: prevent nil pointer dereference in health checks
When checking the health status of each declared resource, kstatus
might return nil for certain resources (for whatever reason). In that
case, this information is now conveyed in the health status event.

https://github.com/fluxcd/kustomize-controller/pull/374

Signed-off-by: Max Jonas Werner <mail@makk.es>
2021-07-26 19:03:46 +02:00
Stefan Prodan f556cb29f6
Fix generation conflicts in tests
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-07-08 18:21:22 +03:00
Vincent Boutour 23b0a2d455
Adding test for materializing #375
Signed-off-by: Vincent Boutour <bob@vibioh.fr>
2021-07-07 18:01:57 +02:00
Stefan Prodan bb71e74d53
Merge pull request #374 from makkes/better-health-check-messaging
feat: make it easier to reason about health check failures
2021-07-05 10:34:15 +03:00
Max Jonas Werner bbc4208d1c
feat: make it easier to reason about health check failures
Whenever a health check times out now, the most recently collected
error for each resource will be printed as part of the error message.
This excludes errors for those resources for which no error was
reported in the last update. This is because whenever a timeout
occurs, an error is reported on ALL resources, even those that have
been seen as healthy before.

Also, this commit causes all successfully checked resources to be
omitted in the error event.

Signed-off-by: Max Jonas Werner <mail@makk.es>
2021-07-04 21:51:41 +02:00
Jodok Batlogg d7c45de5ca fixed typo
Signed-off-by: Jodok Batlogg <jodok@batlogg.com>
2021-07-03 00:08:11 +02:00
Jane Liu L d72d8f5b2d
Test kustomization depend on for improve ut
Signed-off-by: Jane Liu L <jane.l.liu@ericsson.com>
2021-06-28 20:13:44 +08:00
Jane Liu L 8ade434726
Test kustomization depend on for improve ut
Signed-off-by: Jane Liu L <jane.l.liu@ericsson.com>
2021-06-28 20:04:24 +08:00
Stefan Prodan 6672928023
Merge pull request #364 from Kissy/generic-patches
Add support for in-line generic patches to Flux Kustomization API
2021-06-14 14:28:15 +03:00
Stefan Prodan 24f78af641
Simplify staleness check
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-06-14 13:45:38 +03:00
Stefan Prodan 2d85398921
Add test for GC disabling
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-06-14 13:43:05 +03:00
Stefan Prodan f8cac4a35d
Add missing ConfigMap RBAC
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-06-14 13:41:14 +03:00
Jane Liu L 6979dbab83
Move the GC checksum from labels to annotations
Signed-off-by: Jane Liu L <jane.l.liu@ericsson.com>
2021-06-11 21:06:24 +08:00
Guillaume Le Biller b797eac907
Implement generic patches in Kustomization
Allow patching multiple resources instead of a single existing one as
StrategicMerge & JSON6902 are forced to target existing named resources.

Signed-off-by: Guillaume Le Biller <glebiller@Traveldoo.com>
2021-06-11 09:04:37 +02:00
Stefan Prodan dce166f7a6
Update controller-runtime and controller-gen
- Bump controller-runtime to v0.9.0
- Bump controller-gen to v0.5.0
- Use Environment.AddUser to generate the envtest cluster admin kubeconfig

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-06-08 20:34:16 +03:00
Stefan Prodan 0711908892
Workaround for kustomize build panic
Serialize kustomize build runs to avoid kyaml OpenAPI concurrent map read/write panic
https://github.com/kubernetes-sigs/kustomize/issues/3659

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-06-08 20:34:14 +03:00
Stefan Prodan e5ee0367f8
Upgrade to kustomize v4
- Upgrade sigs.k8s.io/kustomize/api from v0.7.4 to v0.8.9
- Upgrade sigs.k8s.io/cli-utils from v0.22.4 to v0.25.0
- Pin sigs.k8s.io/kustomize/kyaml to v0.10.17 (cli-utils compat)
- Pin k8s.io/kube-openapi v0.0.0-20201113171705-d219536bb9fd (cli-utils compat)

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-06-08 20:34:14 +03:00
Gorka Maiztegi 56739d387b Support decrypting any file format in secret generator
Signed-off-by: Gorka Maiztegi <gmaiztegi@reviewpro.com>
2021-05-31 16:06:05 +02:00
Hidehito Yabuuchi 871c2a14bf Fix validation and application timeout handling
Signed-off-by: Hidehito Yabuuchi <hdht.ybuc@gmail.com>
2021-05-18 17:47:58 +09:00
Stefan Prodan 2a03b6e922
Merge pull request #329 from bob-rohan/328_base64_encoded_sops_encrypted_secrets
SOPS: Decrypt Kubernetes secrets generated by kustomize
2021-04-29 10:32:12 +03:00
bob.rohan a77ea03ec6 Decrypt base64 encoded SOPS encrypted secrets
Signed-off-by: Bob Rohan <bob.rohan@hodge.co.uk>
2021-04-28 11:54:25 +01:00
Kazuki Suda e91129c4f1
Confirm the right string is returned
Signed-off-by: Kazuki Suda <kazuki.suda@gmail.com>
2021-04-27 19:29:03 +09:00
Kazuki Suda 178ae70f0d
Extracting validation error from apply server dry run output
Signed-off-by: Kazuki Suda <kazuki.suda@gmail.com>
2021-04-27 18:08:38 +09:00
Chanwit Kaewkasi 147df26298 replace redundant indexers code with high-order functions
Signed-off-by: Chanwit Kaewkasi <chanwit@gmail.com>
2021-04-14 22:51:35 +07:00
Stefan Prodan 23f08ba3f6
Merge pull request #317 from allenporter/log-level
Make log level info for 'Dependencies do not meet ready condition'
2021-04-08 11:54:30 +03:00
Hidde Beydals fa6565b806 Move GC test to Go test suite
Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-04-08 10:18:02 +02:00
Allen Porter 63d6c8c802 Make log level info for 'Dependencies do not meet ready condition'
Reduce the log level from error to info to match the level of the event.

Signed-off-by: Allen Porter <allen.porter@gmail.com>
Signed-off-by: Allen Porter <allen@thebends.org>
2021-04-07 23:33:08 -07:00
Hidde Beydals b7793ac1ab
Merge pull request #313 from fluxcd/override-e2e-to-go 2021-04-06 11:50:52 +02:00
Hidde Beydals 5a8d5026f6 Move Kustomize patch tests to Go test suite
Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-04-06 11:18:11 +02:00
Hidde Beydals 32363048f4 Detect and replace empty err output on apply
This should give users some guidance when `kubectl apply` itself does
not give any useful output back itself, till date only observed when
it times out waiting.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-04-01 17:05:25 +02:00
Hidde Beydals 8688fd6159 Look for `.agekey` to prevent future collisions
The previous `.txt` is very generic and could have resulted in
collisions when a new encryption format would be introduced in the
future.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-04-01 13:20:17 +02:00
Hidde Beydals 21996ec82a Test SOPS decryption in Go tests and remove e2e
Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-04-01 13:02:43 +02:00
Hidde Beydals b8bdc0c999 Support decrypting using age keys
Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-03-31 11:55:44 +02:00
Hidde Beydals c3e1252665 Only GPG import keys with `.asc` extension
Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-03-30 13:20:13 +02:00
Stefan Prodan 446545c71f
Expose suspended status as Prometheus metric
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-03-17 11:27:21 +02:00
Florian Richter 8312a2574c Fixed small typos
Signed-off-by: Florian Richter <floririchte@gmail.com>
2021-03-05 21:35:00 +01:00
Stefan Prodan 8708205edc
Do not override the artifact fetch timeout
Use the timeout set by the http client when retrying with exponential backoff

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-03-05 08:46:52 +02:00
Stefan Prodan cb72a77753
Validate the var names before substitution
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-02-27 12:09:57 +02:00
Stefan Prodan 9d48b6299d
Retry with exponential backoff when fetching artifacts
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-02-26 12:29:14 +02:00
Aurel Canciu 729dc9770e
Support recreating objects on immutable field updates
Allow passing --force to kubectl apply. Useful when dealing with
immutable field changes in resources.

Signed-off-by: Aurel Canciu <aurelcanciu@gmail.com>
2021-02-22 16:59:01 +02:00
Stefan Prodan 453b6e6e46
Merge pull request #281 from davidkarlsen/fixGpgImport
Avoid prompts on import by adding batch flag to gpg
2021-02-22 11:43:30 +02:00
David J. M. Karlsen 750a45a34b
Avoid promts on import by adding batch flag to gpg
Signed-off-by: David J. M. Karlsen <david@davidkarlsen.com>
2021-02-20 23:36:31 +01:00
Laszlo Fogas 48ab6a0205 Extracting validation error from apply dry run output
Signed-off-by: Laszlo Fogas <laszlo@laszlo.cloud>
2021-02-19 16:28:09 +01:00
Stefan Prodan 401fec6c8d
Allow disabling var substitution for certain resources
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-02-16 10:40:01 +02:00
Stefan Prodan 0ac1f9e631
Implement var substitution from ConfigMaps and Secrets
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-02-16 09:20:00 +02:00
Stefan Prodan a77d6cb96e
Allow the controller to be run locally
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-02-14 11:25:18 +02:00
Stefan Prodan f3d9c36691
Add support for variable substitutions
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-02-11 18:17:53 +02:00
Stefan Prodan e99f328200
Add source kind and name to not found error
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-02-08 15:20:13 +02:00
Stefan Prodan b2cfe6d3f8
Allow disabling of prune on certain resources
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-02-07 14:16:25 +02:00
Hidde Beydals 7c23accfeb Use Strategic Merge and JSON 6902 patches
Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-02-03 21:52:36 +01:00
Stefan Prodan 5e17dd48f4
Add Healthy status condition
- record the last health assessment result in a dedicated status condition
- use the condition status when issuing events to prevent notifications spam

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-02-02 14:59:06 +02:00
Christian Hernvall d0f2dc6e4a
Support impersonation for validation
Signed-off-by: Christian Hernvall <c.hernvall@yubico.com>
2021-01-29 14:56:45 +01:00
Brian Atkinson 2ca0edcf92 impersonation: Clear config.BearerTokenFile when setting BearerToken.
When running in a cluster, the BearerTokenFile is set to point to
`/var/run/secrets/kubernetes.io/serviceaccount/token` where the service
account's token is auto-mounted. If this value is not cleared, the
setting of the BearerToken field will have no effect. Relevant
documentation:
https://pkg.go.dev/k8s.io/client-go@v0.20.2/rest#Config.BearerTokenFile

Signed-off-by: Brian Atkinson <brian@atkinson.mn>
2021-01-25 15:14:35 -08:00
Stefan Prodan 100d362ce7
Fix reconciliation retry scheduler
Log the reconciliation error instead of returning it, so that controller-runtime doesn't requeue immediately. Reconciliation failures should be scheduled at the specified retry interval.

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-01-25 13:17:38 +02:00
Stefan Prodan e5c93965a5
Add reconciliation unit test
Verify that objects are create in cluster and labeled accordingly

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-01-22 15:52:19 +02:00
Stefan Prodan dfba88ccc1
Requeue a failed reconciliation based on retry interval
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-01-22 15:50:45 +02:00
Philip Laine 26a46c9855 Specify kubeconfig in tests
Signed-off-by: Philip Laine <philip.laine@gmail.com>
2021-01-22 12:50:04 +01:00
Stefan Prodan 62a764a28e
Use the object reference type from fluxcd/pkg/meta
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-01-21 13:24:32 +02:00
Stefan Prodan 085588b632
Disable kyaml
Workaround for upstream bug: https://github.com/kubernetes-sigs/kustomize/issues/3446

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-01-18 14:14:23 +02:00
Stefan Prodan 7859a639ed
Update kustomize/api to v0.7.2
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-01-17 22:00:51 +02:00
Hidde Beydals 59c24e7d38 Look for all accepted Kustomization filenames
Before this commit we only checked if a `kustomization.yaml` existed at
the root of the given directory, this caused problems when people for
example used `.yml` as the extension, as the generated
`kustomization.yaml` would conflict with the `.yml` file.

After this commit all recognized Kustomization filenames as listed by
Kustomize itself are accepted, including files _without_ an extension
(`Kustomization`).

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-01-15 15:31:20 +01:00
Hidde Beydals c6353b4afd Upgrade runtime package to v0.6.2
To include a bug fix to the `ReconcilateAtChangedPredicate`
and renaming to `ReconcileRequestedPredicate`.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-01-14 12:34:10 +01:00
Stefan Prodan 5a0e5abd9a
Setup impersonation for GC and health checks
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-01-04 11:51:04 +02:00
Stefan Prodan 696f91d380
Refactor impersonation
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-01-04 11:50:59 +02:00
Stefan Prodan f14cd2323c
Upgrade controller-runtime to v0.7.0
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-01-04 11:50:58 +02:00
Michał Flendrich af358709f5 No longer treat dirs as kustomization files.
Fixes https://github.com/fluxcd/kustomize-controller/issues/223

Signed-off-by: Michał Flendrich <michal@flendrich.pro>
2020-12-24 21:04:38 +01:00
Nicolas Lamirault 460eae2a2d
Add: Sops user error
Signed-off-by: Nicolas Lamirault <nicolas.lamirault@gmail.com>
2020-12-17 19:07:31 +01:00
Stefan Prodan 5efc06838a
Merge pull request #218 from fluxcd/gc-debug
Add debug logs to garbage collection
2020-12-17 11:37:03 +02:00
Hidehito Yabuuchi 8b132d18be Emit healthcheck event when Kustomization was not ready
Signed-off-by: Hidehito Yabuuchi <hdht.ybuc@gmail.com>
2020-12-17 17:50:05 +09:00
Stefan Prodan fe3cc52e37
Add debug logs to garbage collection
Log the GC query errors to help diagnose objects skipped from pruning

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-12-17 10:39:51 +02:00
Stefan Prodan 8be1e169d2
Merge pull request #210 from fluxcd/gc-prune
Refactor garbage collection
2020-12-16 16:29:37 +02:00
Stefan Prodan 88b6d40ef5
Increase termination grace period to 1m
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-12-16 14:28:42 +02:00
Hidde Beydals d7a0deac97 Write KubeConfig to tmp file in working dir
Instead of using the name of the secret, as this can cause unexpected
collisions in edge case scenarios.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2020-12-16 12:59:19 +01:00
Hidde Beydals 6a4bf74cf3 Add safe guards for relative paths
This commit ensures that relative (user configurable) paths never
traverse outside their working directory.

It does _not_ provide protection against path traversal within
`kustomization.yaml` files.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2020-12-16 12:44:13 +01:00
Stefan Prodan 59845b0891
Refactor garbage collection
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-12-16 13:09:30 +02:00
Stefan Prodan d65ea71699
Refactor apply error reporting
- filter kubectl apply output and extract errors
- limit apply output to 20K charts (avoid reaching max etcd size)
- log kubectl exit code when the process is killed

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-12-14 17:34:55 +02:00
Stefan Prodan 1438ef8735
Upgrade dependencies
- `github.com/fluxcd/pkg/apis/meta` to `v0.5.0`
- `github.com/fluxcd/pkg/runtime` to `v0.4.0`
- `github.com/fluxcd/source-controller/api` to `v0.5.1`
- `k8s.io/{api,apiextensions-apiserver,apimachinery,cli-runtime,client-go}` to `v0.19.4`
- `sigs.k8s.io/controller-runtime` to `v0.6.4`

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-12-10 19:48:41 +02:00
Somtochi Onyekwere 5e62583cec Implements image overrides
Signed-off-by: Somtochi Onyekwere <somtochionyekwere@gmail.com>
2020-12-10 13:23:55 +01:00
Sylvain Rabot 3f34e450ec
Set --field-manager when applying
Signed-off-by: Sylvain Rabot <s.rabot@lectra.com>
2020-11-26 13:32:26 +01:00
Stefan Prodan 5a9e122e77
Set progressing after source readiness check
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-11-26 09:29:34 +02:00
Stefan Prodan 2312d69a51
Do not mark suspended resource as not ready
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-11-26 09:00:52 +02:00
Stefan Prodan 73546e92d2
Requeue after interval on source not found errors
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-11-24 11:35:47 +02:00
Stefan Prodan 0c9170241f
Use ServiceAccountName for impersonation
Drop the ServiceAccount field in favour of ServiceAccountName to prevent privilege escalation in multi-tenancy environments.

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-11-20 10:58:24 +02:00
Aurel Canciu ac6cc88e29
Refactor to adopt k8s standardized Condition type
Updates to use metav1.Condition type and removes references for
deprecated corev1.Condition* constants and uses the new k8s api/meta
helpers in place of the old pkg/apis/meta types.

Signed-off-by: Aurel Canciu <aurelcanciu@gmail.com>
2020-11-19 19:49:04 +02:00
Hidde Beydals c53e5eeab2 Use DeletionTimestamp for prune and readiness
Signed-off-by: Hidde Beydals <hello@hidde.co>
2020-11-19 12:23:48 +01:00
Stefan Prodan 5aded37954
Patch status sub-resource
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-11-12 13:04:16 +02:00
Stefan Prodan 2ba6252d76
Allow disabling validation
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-11-12 12:18:04 +02:00
Stefan Prodan 8ec066bf37
Use latest generation when updating final status
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-11-04 13:04:01 +02:00
Stefan Prodan d8caeb7cec
Remove unused code
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-11-03 16:07:44 +02:00
Stefan Prodan 46f828ff43
Omit checksum label if GC is disabled
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-11-03 15:22:25 +02:00
Hidde Beydals 26db48b9f0 Compare artifact <> Kustomizations in enqueuers
The reason for this is the `EnqueueRequestsFromMapFunc` calling the
enqueuer for _both_ the old and the new object, and we only want to act
on the ones that contain a revision different from the one that we have
recorded in the status object of the `Kustomization`.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2020-10-28 22:36:34 +01:00
Hidde Beydals 752b6b6bf1 Bundle revision change predicates into one
Signed-off-by: Hidde Beydals <hello@hidde.co>
2020-10-28 21:56:44 +01:00
Hidde Beydals 824af43beb Add Kustomization finalizers permissions
Signed-off-by: Hidde Beydals <hello@hidde.co>
2020-10-28 13:31:08 +01:00
Hidde Beydals 6bd8971cec Re-add status GET permissions
Signed-off-by: Hidde Beydals <hello@hidde.co>
2020-10-28 13:21:53 +01:00
Hidde Beydals e8d2c31b40 Log reconciliation requests by watcher
Signed-off-by: Hidde Beydals <hello@hidde.co>
2020-10-28 13:11:17 +01:00