Commit Graph

74 Commits

Author SHA1 Message Date
Hidde Beydals b801b23911 Properly detect chart version divergences
This is a regression bug fix due to the introduction of the default
values in the source-controller CRDs.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2020-11-18 10:06:40 +01:00
Hidde Beydals bb85850448 Patch status sub-resource
Signed-off-by: Hidde Beydals <hello@hidde.co>
2020-11-17 11:42:42 +01:00
Hidde Beydals 485b6f5fc8 Requeue on chart load and values composition err
Signed-off-by: Hidde Beydals <hello@hidde.co>
2020-11-17 11:38:55 +01:00
Hidde Beydals fa598261e7 Compare artifact <> HelmRelease in enqueuer
The reason for this is the `EnqueueRequestsFromMapFunc` calling the
enqueuer for _both_ the old and the new object, and we only want to act
on the ones that contain a revision different from the one that we have
recorded in the status object of the `HelmRelease`.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2020-10-28 22:42:29 +01:00
Hidde Beydals a35a8a9707 Do not requeue when chart is not ready
As the watcher should trigger a reconciliation once the artifact
is available.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2020-10-28 14:26:04 +01:00
Hidde Beydals b3baf39e11 Move dedicated watcher to in-controller watcher
This prevents the resources from getting annotated, and instead uses
the `handler.EnqueueRequestsFromMapFunc` to queue requests based on
changes to the source objects.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2020-10-28 13:28:11 +01:00
Hidde Beydals 7ac2a41e1a Change copyright to Flux authors
Signed-off-by: Hidde Beydals <hello@hidde.co>
2020-10-27 17:55:18 +01:00
Michael Bridgen 0928cc1e4d Use reconcile annotation helpers
Per https://github.com/fluxcd/pkg/issues/39
2020-10-22 12:26:48 +01:00
Hidde Beydals 15914beca1 Support cross-cluster Helm releases
This is an initial implementation for cross-cluster Helm release
support that relies on a KubeConfig secret, and a reference to it in
the HelmRelease resource.

If set, all actions taken by the Helm runner are executed using the
KubeConfig from the secret. The Helm storage is stored on the remote
cluster in a namespace that equals to the namespace of the HelmRelease
in the managing cluster, the release itself is made in either this
namespace, or the configured TargetNamespace. In any case, both are
expected to exist and/or created beforehand.

Other references to Kubernetes resources in the HelmRelease, like
ValuesReference resources, are expected to exist on the managing
cluster.
2020-10-20 21:30:56 +02:00
Hidde Beydals 236e78d3a1 Change chart reconciliation and readiness logic
By confirming the observed generation is up-to-date before checking
the `Ready` condition state after chart reconciliation, we guarantee
the chart we use for the release always is on par with the state of
the `HelmRelease`.

In addition, we requeue to prevent the `HelmRelease` getting stuck
in a "HelmChart is not ready" state. Previously triggered by a no-op
chart update without a revision change that caused the watcher to
not request a reconciliation for the `HelmRelease`.
2020-10-16 14:39:32 +02:00
Hidde Beydals 3fccbb943f Declare and assign result Values before composing
As `ParseInto` expects the destination to be initialized, which led
to an `unable to parse key: assignment to entry in nil map` error in
configurations where the first `ValuesReference` had a `targetPath`
defined.
2020-10-15 19:12:19 +02:00
Stefan Prodan 1819f143a9
Implement Prometheus instrumentation
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-10-13 16:24:56 +03:00
Stefan Prodan 6a04f769b2
Update fluxcd/pkg/runtime to v0.1.0
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-10-13 15:33:46 +03:00
Hidde Beydals 2fb64a1095 Prevent possible future nil pointer dereference
As the existing logic relied on other conditions that caused the
`released` to never be nil, but this may not be true for future
iterations, which makes this a case of better safe than sorry.
2020-10-05 14:03:44 +02:00
Hidde Beydals 899e8bedc7 Configure release namespace in runner constructor 2020-10-02 19:05:45 +02:00
Hidde Beydals accd4762fe Promote API to v2beta1 2020-09-30 19:37:23 +02:00
Hidde Beydals 8129c49b9c Implement `fluxcd/pkg/apis/meta` in APIs 2020-09-30 12:14:51 +02:00
Hidde Beydals d554a92f81 Record last handled reconcile at annotation
This makes it possible for e.g. the GOTK CLI to observe if the
controller has handled the resource since the manual reconciliation
request was made. It replaces the `LastObservedTime` status field,
as this was prone to time skew issues and does not offer much additional
value over the timestamps of the conditions.
2020-09-29 10:58:04 +02:00
Sean Eagan 30a8f936e2 Fix logging for next run delay
We were logging the spec interval duration, which was incorrect:

1. On failures, which use exponential backoff
2. On dependency not ready, which uses a separately defined static interval.

This changes to log result.RequeueAfter directly when set.
2020-09-28 09:36:29 -05:00
Hidde Beydals d80de0af02 Use single version of values reference on merge
This ensures that a subsequent update to a values reference during
the merging process does not result in an unexpected mixture of old(er)
and new values.
2020-09-28 14:06:56 +02:00
Hidde Beydals 68a8f62a41 Use `ObserveLastRelease` in garbage collector
Which uses `Last()` instead of `Deployed()` to ensure we garbage
collect Helm releases with statuses other than `deployed`,
e.g. `failed`.
2020-09-25 16:18:15 +02:00
Hidde Beydals 4a9a04aa4f Factor out and restructure various elements
* Move `ReleaseRevision` function to util
* Rename `release` method to `reconileRelease` to match
  `reconcileChart`
* Refactor chart artifact download to make use of a temporary file,
  which is removed as soon as the tarbal has been loaded into memory
2020-09-25 16:03:44 +02:00
Hidde Beydals 46d8776b18 Remove reconciliation lock
As the observed generation is now pushed before syncing the resource
(https://github.com/fluxcd/helm-operator/issues/437), and the
controller runtime queue guarantuees there are no consistency
issues (see: https://openkruise.io/en-us/blog/blog2.html).
2020-09-25 15:40:45 +02:00
Hidde Beydals 69ced56bb7 Split HelmChart and HelmRelease garbage collection 2020-09-25 15:40:45 +02:00
Hidde Beydals 7cf267c15c Factor out util functions 2020-09-25 15:40:45 +02:00
Hidde Beydals c526837d76 Factor out Helm actions into runner package 2020-09-25 15:40:41 +02:00
Hidde Beydals 1a88b40936 Support optional values references
This commit adds support for optional values references, as discussions
have brought to light that there are some valid use cases in which
having this option is desirable. For example to allow a user to extend
behaviour at a later date with their own repository without modifying
the HelmRelease object.

When a values reference is marked as optional, not found errors for the
value reference are ignored, but any ValuesKey, TargetPath or transient
error will still result in a reconciliation failure.
2020-09-23 10:00:52 +02:00
Hidde Beydals 8fb854a7b5 Support dependency references to other namespaces
This also removes the sorting from the `HelmChartWatcher`, as with
the current `HelmChartTemplateSpec` a chart is only used by a single
`HelmRelease`. Rendering the action obsolete.
2020-09-15 14:55:11 +02:00
Sean Eagan 0fd937eaee Refactor status API
This removes:

- Installed, Upgraded, RolledBack, and Uninstalled status conditions
  since they did not represent current state, but rather actions
  taken, which are already recorded by events.
- status.observedStateReconciled since it solved the problem of
  remembering past release (install/upgrade/test) success, but not
  past release failures, after other subsequent failures such as
  dependency failures, k8s API failures, etc.

This adds:

- Remediated status condition which records whether the release is
  currently in a remediated state. It is used to prevent release retries
  after remediation failures. We were previously not doing this for
  rollback failures.
- Released status condition which records whether the current state
  has been successfully released (install/upgrade/test). This is used to
  remember the last release attempt status, regardless of any subsequent
  other failures such as dependency failures, k8s API failures, etc.

This renames:

- Tested > TestsSuccess status condition, for forward compatibility
  with interval based helm tests.
2020-09-14 08:12:23 -05:00
Hidde Beydals 2f9d29f693 Support HelmChart ValuesFile field 2020-09-11 14:23:42 +02:00
Hidde Beydals 1388169296 Include PATCH rule for events in manager-role
During high custom resource count / low interval tests, I was greated
with a `cannot patch resource "events"` message. This happened due to
event compaction, where it will perform a patch instead of a create.
By giving the role the permission to do so this should no longer pose
a problem.
2020-09-10 21:14:29 +02:00
Michael Bridgen b164d63bff Use pkg/runtime for annotation and predicate
The predicate which filters update events for generation change or the
reconcileAt annotation has been factored out into fluxcd/pkg/runtime.
2020-09-09 18:56:35 +01:00
Hidde Beydals a7e6727d8e Support charts from GitRepository sources 2020-09-01 18:32:17 +02:00
Sean Eagan 0d64e8dc73 Update status iff new state or done reconciling
Also set status.lastObservedTime to the actual time of the status update.
2020-08-31 16:00:43 -05:00
Sean Eagan 55f603806d Record last observed time in status
This adds a .status.lastObservedTime field which records when the
HelmRelease was last observed by the controller. This allows the user
to observe whether the spec.interval and reconcileAt annotations are
triggering reconciliation attempts as desired.
2020-08-31 13:33:49 -05:00
Sean Eagan fd7d23256a Rename knownStateApplied to observedStateReconciled
This is more consistent with the existing terminology used.
2020-08-31 12:12:48 -05:00
Sean Eagan be9e22f5f5 More graceful failure recovery
- Ensure upgrade actually occurs if known state was not reached
  for any reason (other than install failure).
- After transient failures not tied to new state application, ensure
  spurious upgrades do not occur and ready state is again reached,
  by remembering that the known state was already successfully applied.
- Reset failure counts after success so they're not stale.
- Only lookup post-deployment release revision on remediation,
  since otherwise we already have it.
- Push status update after finding new state so user can observe.
2020-08-31 08:47:18 -05:00
Sean Eagan b8853ad7a5 Implement conditional remediation
This commit adds support for conditional remediation, enabling the user
to:

* configure if test failures should be ignored
* configure what action should taken when a Helm install or upgrade
  action fails (e.g. rollback, uninstall)
* configure if a failed Helm action should be retried
* configure if a failed release should be kept for debugging purposes

The previous behaviour where failed Helm tests did not mark the
`HelmRelease` as not `Ready` has changed, it now marks them as failed
by default.

Co-authored-by: Hidde Beydals <hello@hidde.co>
2020-08-25 17:35:20 +02:00
Hidde Beydals e91b9d303e api: make optional structs pointers 2020-08-22 01:09:20 +02:00
Hidde Beydals f2f8b23a4f Support targeting a path in ValuesReferences 2020-08-20 10:41:07 +02:00
Hidde Beydals 86cc4f9908 Allow overwrites of map values with flat values
After taking a closer look at the Helm code, this is the behaviour
we are actually after, given we want to mimic the merging behaviour
of the -f flag available to Helm install and/or upgrade commands.
2020-08-19 14:01:19 +02:00
Hidde Beydals 17597d1e19 Change CRD domain to 'helm.toolkit.fluxcd.io'
To allow the Helm Operator and helm-controller HelmReleases to
co-exist in the cluster, while being handled by separate controllers
during e.g. the migration period.

This is not possible without using another domain due to how Custom
Resource Definitions work, as newer API versions are seen as a
replacement of older versions, and are expected to be handled by a
single controller.
2020-07-30 22:38:11 +02:00
Hidde Beydals 6b1d28a736 Record SHA1 of values of last release attempt 2020-07-28 16:05:09 +02:00
Hidde Beydals 6dba659c3c Compose values from references 2020-07-28 15:36:38 +02:00
Hidde Beydals 03e18ee14f Take observed generation of deps into account
To prevent operations from being performed while they are "Ready",
but the latest generation of the depenceny has not been reconciled
yet.
2020-07-23 16:21:25 +02:00
Hidde Beydals 354590249e Rename conditions to describe the current state
As "condition type names should describe the current observed state of
the resource, rather than describing the current state transitions".

Described by the draft convention for update conditions in
kubernetes/community#4521.
2020-07-23 12:23:52 +02:00
stefanprodan 0efee4a41b Reevaluate ready status after transient errors 2020-07-22 14:21:11 +03:00
stefanprodan 5411834f14 Fix Helm chart update 2020-07-21 17:35:26 +03:00
Hidde Beydals 5a81f5af70 Fix: use composed release name in Helm actions 2020-07-21 14:08:48 +02:00
Hidde Beydals 355e4f2ec3 Fix: use fetched HelmChart object for update 2020-07-21 11:30:18 +02:00