Commit Graph

83 Commits

Author SHA1 Message Date
Aurel Canciu f1b5768200
Fix values file override
`io.Read` was used incorrectly to read from the override file provided
by the user.
This is now replaced with `ioutil.ReadFile` for better handling and
error reporting.

Fixes #263

Signed-off-by: Aurel Canciu <aurelcanciu@gmail.com>
2021-01-21 13:40:12 +02:00
Aurel Canciu 778b07cdd4
Fix unreachable code issue in helmchart reconciler
Signed-off-by: Aurel Canciu <aurelcanciu@gmail.com>
2021-01-18 23:06:30 +02:00
Hidde Beydals 34c4290cda Alin HelmChart source indexer and watches
Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-01-18 17:08:25 +01:00
Hidde Beydals 566ce8aa15 Upgrade runtime package to v0.6.2
To include a bug fix to the `ReconcilateAtChangedPredicate`
and renaming to `ReconcileRequestedPredicate`.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-01-14 11:39:47 +01:00
Hidde Beydals fda481efe9 Upgrade controller-runtime to v0.7.0
This commit upgrades the `controller-runtime` dependency to `v0.7.0`,
including all changes required to make all wiring work again.

- Upgrade `runtime` to v0.6.0 to include `controller-runtime` changes.
- Loggers have been removed from the reconcilers and are now retrieved
  from the `context.Context` passed to the `Reconcile` method and
  downwards functions.
- Logger configuration flags are now bound to the flag set using
  `BindFlags` from `runtime/logger`, ensuring the same contract across
  GitOps Toolkit controllers, and the `--log-json` flag has been
  deprecated in favour of the `--log-encoding=json` default.
- The `ChangePredicate` from `runtime` has changed to a
  `ReconcilateAtChangedPredicate`, and is now chained with the
  `GenerationChangedPredicate` from `controller-runtime` using
  `predicate.Or`.
- Signatures that made use of `runtime.Object` have changed to
  `client.Object`, removing the requirement to e.g. call
  `runtime.Object#Object`.
- The `client.MatchingField` function was deprecated, and has been
  replaced with `client.MatchingFields{}`.
- The `leader-election-role` was changed, as leader election now works
  via the `coordination/v1` API.

Other notable changes:

- `util.ObjectKey` was added to easily construct a `client.ObjectKey` /
  `types.NamespacedName` from a `metav1.Object`.
- The `SourceIndexKey` constant has been split out into
  `{GitRepository,HelmRepository,Bucket}IndexKey` constants.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2020-12-18 19:17:10 +01:00
Aurel Canciu a55c502bb4
Fix HelmChart valuesFile chart path restriction
As part of the feature implementation to support helm chart
dependencies, the functionality for allowing values files overwriting
from any location scoped to the same source was altered. This should fix
the problem by allowing users to load files from any arbitrary location
as long as it's in the context of the same source from where the helm
chart itself is loaded.

Signed-off-by: Aurel Canciu <aurelcanciu@gmail.com>
2020-12-17 12:57:41 +02:00
Hidde Beydals 7cefdee6d8 Prevent HelmChart controller panic on init failure
The controller logic is a serious candidate to be refactored so that
errors can be taken into account better, and do not always result in a
requeue. But this will do for the time being.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2020-12-16 15:25:17 +01:00
Hidde Beydals 29a051c5f4 Refactor and document DependencyManager
Mostly to re-use the fields of the structure instead of copying things
around.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2020-12-15 15:37:42 +01:00
Hidde Beydals 8d0b54e431 Make proper use of errgroup context
Signed-off-by: Hidde Beydals <hello@hidde.co>
2020-12-15 09:56:40 +01:00
Hidde Beydals bc890874e1 Add safe guards for relative paths
Signed-off-by: Hidde Beydals <hello@hidde.co>
2020-12-15 09:56:40 +01:00
Hidde Beydals 5f3c014966 Validate provided name for charts from HelmRepos
Following the rules described in
https://helm.sh/docs/chart_best_practices/conventions/#chart-names.

This guards against people following the wrong guidance of Artifactory,
that supports and promotes repository indexes with e.g. '/' in the
chart names.

In a future version this should be moved to a validation webhook, but
there are still discussions ongoing around the TLS certificates for
this.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2020-12-09 15:37:10 +01:00
Hidde Beydals 254de30f6a List HelmChart for GitRepository change
Signed-off-by: Hidde Beydals <hello@hidde.co>
2020-11-26 18:03:04 +01:00
Hidde Beydals 0c904a1879 Watch chart sources for revision changes
To enqueue a new reconciliation for the HelmChart sources as soon as
the revision of their upstream source changes.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2020-11-26 13:02:55 +01:00
Stefan Prodan 5579dc8c96
Implement source suspension
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-11-26 12:58:00 +02:00
Hidde Beydals 0b4586253c Add LastHandledReconcileAt to source statuses
Signed-off-by: Hidde Beydals <hello@hidde.co>
2020-11-19 13:21:59 +01:00
Aurel Canciu b80f450b68
Switch to new pkg/apis/meta SetResourceCondition
Use SetResourceCondition as a generic method to set conditions for CRs,
implmeneting the ObjectWithStatusConditions interface used as input
type.

Signed-off-by: Aurel Canciu <aurelcanciu@gmail.com>
2020-11-19 13:08:33 +02:00
Aurel Canciu 00bb853d0e
Refactor to adopt k8s standardized Condition type
Updates to use metav1.Condition type and removes references for
deprecated corev1.Condition* constants and uses the new k8s api/meta
helpers in place of the old pkg/apis/meta types.

Signed-off-by: Aurel Canciu <aurelcanciu@gmail.com>
2020-11-19 13:08:33 +02:00
Hidde Beydals a3f177328a Move Helm dependency manager to `helm` package
Signed-off-by: Hidde Beydals <hello@hidde.co>
2020-11-19 09:50:50 +01:00
Hidde Beydals 2f50e3ff59 Make GC behavior depend on deletion timestamp
Signed-off-by: Hidde Beydals <hello@hidde.co>
2020-11-18 23:03:44 +01:00
Hidde Beydals 93d1559b50 Prevent GC from wiping artifacts in the same NS
Signed-off-by: Hidde Beydals <hello@hidde.co>
2020-11-18 22:56:08 +01:00
Stefan Prodan 81d3ecf612
Patch status sub-resource
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-11-12 17:02:39 +02:00
Philip Laine 01d0053871 Fix duration default values
Signed-off-by: Philip Laine <philip.laine@xenit.se>
2020-11-02 09:55:39 +01:00
Aurel Canciu 946d1e6ac0
Remove use of chartutil.ProcessDependencies
It looks like the use of chartutil.ProcessDependencies in the HelmChart
Controller was not correct, this method seems to be used in Helm only
during install/upgrade. The intention was to load the dependencies but
this seems to not be needed as it's already done through the loaders
(loader.Load).

The use of this method caused a regression where Chart.yaml files would
be overwritten and registered subcharts that had aliases would be
renamed using the alias name. While this is an expected behaviour of
chartutil.ProcessDependencies it is not what the controller should do
to the chart during (re)packaging.

Signed-off-by: Aurel Canciu <aurelcanciu@gmail.com>
2020-10-30 19:16:28 +02:00
Aurel Canciu f1362bd3a9
Implement DepdendencyManager for non-packaged
Non-packaged charts that don't have their dependencies present in
charts/ will now have these dependencies built using the
DependencyManager. The idea behind it is to replicate the logic
implemeneted in Helm's downloader.Manager with the support for already
existing HelmRepository resources and their chart retrieval capabilities.

Signed-off-by: Aurel Canciu <aurelcanciu@gmail.com>
2020-10-28 15:46:18 +02:00
Aurel Canciu 38317ab7c0
Extract HelmRepository secret retrieval
Method getHelmRepositorySecret on the HelmChartReconciler

Signed-off-by: Aurel Canciu <aurelcanciu@gmail.com>
2020-10-28 15:27:11 +02:00
Aurel Canciu f3370d8004
Extract chart repo url normalization logic
Move the logic to helm/utils exported as func
NormalizeChartRepositoryURL

Signed-off-by: Aurel Canciu <aurelcanciu@gmail.com>
2020-10-28 14:57:12 +02:00
Hidde Beydals 54aaef5380
Index HelmRepository resources by their URL
To facilitate an inexpensive lookup when collecting credentials and
index artifacts while working with chart dependencies.

Signed-off-by: Aurel Canciu <aurelcanciu@gmail.com>
2020-10-28 14:57:12 +02:00
Stefan Prodan 08d98ff214
Add finalizers to RBAC
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-10-28 14:39:53 +02:00
Hidde Beydals 582c2092b6 Change copyright to Flux authors
Signed-off-by: Hidde Beydals <hello@hidde.co>
2020-10-27 17:54:22 +01:00
Aurel Canciu 684624b1a0
Add support for loading packaged helm charts
The feature allows the source-controller to load packaged helm charts
for HelmChart resource artifacts from GitRepository and Bucket sources

Signed-off-by: Aurel Canciu <aurelcanciu@gmail.com>
2020-10-27 11:15:17 +02:00
Hidde Beydals 504ae25954 Switch to controller-runtime utils for finalizers 2020-10-21 11:50:13 +02:00
Stefan Prodan 03e32491bf
Implement Prometheus instrumentation
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-10-13 13:59:30 +03:00
Stefan Prodan c8c2eec3a6
Update fluxcd/pkg/runtime to v0.1.0
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-10-13 13:34:47 +03:00
stefanprodan f16e5f041a Promote API to v1beta1 2020-09-30 16:11:00 +03:00
Hidde Beydals 431ea05427 Implement `fluxcd/pkg/meta/api` in APIs 2020-09-30 10:18:43 +02:00
Hidde Beydals 8bf7d8f440 Factor out Helm repo index and chart download 2020-09-24 12:40:04 +02:00
Hidde Beydals b9576d56f1 Prevent resources getting stuck on transient err
This commit ensures that resources will only return early if they are
already in a `Ready==True` state. If not, but the status object somehow
still reports that it has an artifact, the reconciliation will continue
to ensure and/or guarantee state, and to prevent a deadlock from
happening.
2020-09-22 17:00:54 +02:00
Hidde Beydals 969a46f4d6 internal/helm: return callback on empty TLS config
...and no-op on empty valuesFile string.
2020-09-21 23:27:38 +02:00
Hidde Beydals 4da80b65e5 storage: add CopyFromPath method 2020-09-21 23:16:13 +02:00
Hidde Beydals 270b6a5c0c api: add HasRevision method to Artifact 2020-09-21 22:41:51 +02:00
Hidde Beydals 03ce9d96da Support Helm charts from Bucket sources 2020-09-21 12:40:15 +02:00
Hidde Beydals 7268c8b61d Refactor and factor out chart values replacement 2020-09-11 12:16:41 +02:00
Hidde Beydals fd36d2d4f9 Allow overwriting chart values from HelmRepository 2020-09-11 11:32:54 +02:00
Hidde Beydals 2e0b6cb6c9 Allow overwriting chart values from GitRepository 2020-09-11 11:32:45 +02:00
Hidde Beydals f40a42115b Conditionally remove artifact from source status
We only want to remove it if it is no longer in storage, otherwise we
still want to keep it around till we have a new artifact.
2020-09-11 10:10:47 +02:00
Hidde Beydals b5d869db03 Include PATCH rule for events in manager-role
During high custom resource count / low interval tests, I was greated
with a `cannot patch resource "events"` message. This happened due to
event compaction, where it will perform a patch instead of a create.
By giving the role the permission to do so this should no longer pose
a problem.
2020-09-10 21:17:10 +02:00
Hidde Beydals d03f4fa4c4 Change advertised artifact URLs on hostname change 2020-09-10 14:13:46 +02:00
Hidde Beydals 7a3a5938d3 Mark resources as progressing on spec changes 2020-09-10 14:13:41 +02:00
Hidde Beydals 42706a342b Calculate checksums during file writes 2020-09-10 13:00:08 +02:00
Hidde Beydals 1ab12869ac Make storage file writes atomic 2020-09-10 12:02:32 +02:00