Introduce the rewritten reconciler which uses v1beta2 API and the new
internal/policy and internal/source packages for performing all the
operations. The reconciliation model is written similar to the other
flux controller, with simplified events and logs, and kstatus support.
The reconciliation result computation similar to
image-reflector-controller, using the same reconciler helpers from
pkg/runtime repo.
With the new status fields ObservedPolicies and ObservedSourceRevision,
the reconciler now avoid full sync of the source which involves cloning
the source every time. If the policies and the remote source have not
changed since the last reconciliation, the reconciliation is returned
early without cloning and apply the updates. This is only applicable
when the checkout branch and the push branch are the same. For a
different push branch and refspec, full sync is always performed as
before.
Notifications are now only sent when there's something new to inform
about. If there is no change, an source up-to-date notification is sent.
When there's an update, the details about the pushed commit with any
rendered commit template is send. When there's a failure, the error is
sent in the notification. And when there's an error recovery, a success
notification is sent to rely that the automation has recovered.
Signed-off-by: Sunny <github@darkowlzz.space>
The metrics helper now accepts owned finalizers to determine if an
object is no longer managed by the controller and is being deleted, and
deletes the metrics associated with the object.
Call the metrics recording defer function in controller early to be able
to record the object in deleting state.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
Set package name in the files under internal/controller to have the
base name of the directory.
This style is recommended by Go, and certain text editors/IDEs get
confused when the names don't match.
Signed-off-by: Hidde Beydals <hidde@hhh.computer>
This deals with various breaking changes in controller-runtime, as
documented in the release notes:
https://github.com/kubernetes-sigs/controller-runtime/releases/tag/v0.15.0
In short:
- `Watches` now use a `client.Object` instead of a `source.Kind`.
- `handler.MapFunc` signature accepts a Go context, which is used to
log any errors, instead of silently ignoring them and/or panicking.
- Fake clients used in tests are now configured using
`WithStatusSubresource` to enable the correct behavior for status
updates and patches.
- Max concurrent reconciles and panic recovery is configured on the
manager, instead of configuring them per reconciler instance.
- Various manager configuration options have been moved to new
structures and/or fields.
In addition to this, all other dependencies which had updates are
updated to their latest (compatible) versions as well. Including an
update of kyaml to match Kustomize v5.
Signed-off-by: Hidde Beydals <hidde@hhh.computer>
The new `--watch-label-selector` lets operators provide a label to the
controller manager which in turn uses it to reconcile only those resources
`ImageUpdateAutomations` that match the given label.
This way operators can deploy multiple instances of IAC, each reconciling a
distinct set of resources based on their labels and effectively scale IAC
horizontally.
closes#503
Signed-off-by: Max Jonas Werner <mail@makk.es>
This uses the newly introduced helper from runtime, which also
configures the logger for `klog`.
Resulting in all logs now being properly formatted, even when logged by
internal Kubernetes elements like the leader election or a dynamic
client.
The update from `fluxcd/pkg/runtime` from `v0.29.0` to `v0.31.0` in
addition includes a change which configures the event recoder to only
log on level `trace` (as opposed to `debug`).
Signed-off-by: Hidde Beydals <hidde@hhh.computer>
ForceGoGitImplementation ignores the value set for gitImplementation
and ensures that go-git is used for all GitRepository objects.
This can be used to confirm that Flux instances won't break if/when
the libgit2 implementation was to be deprecated.
When enabled, libgit2 won't be initialized, nor will any git2go cgo
code be called.
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
The controller-runtime provides the ability to recover from
panics that have arisen from a reconciliation. This change
enables this functionality by default.
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
The new flag allow users to set the list of hostkey algorithms
to use for ssh connections, enabling them to ensure specific
are/aren't used.
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
GitManagedTransport enables the use of Managed Transport. This replaces
the previous 'EXPERIMENTAL_GIT_TRANSPORT' environment variable that was
used for the same result. This commit also enables it by default.
This is an opt-out feature, which can be disabled by starting the
controller with the argument '--feature-gates=GitManagedTransport=false'.
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
Add two new flags to enable users to configure exponential
back-off for Flux objects. The default values are now
set to 750ms for minimum retry time, and 15min for max.
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
libgit2 network operations are blocking and do not provide timeout nor context capabilities,
leading to several reports of the controllers hanging indefinitely.
By using managed transport, golang primitives such as http.Transport and net.Dial can be used
to ensure timeouts are enforced.
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
This bumps the version of the image reflector types to v1beta1. This
doesn't technically make a difference for the minute, since the
Kubernetes API server would convert between versions, but is tidier.
Signed-off-by: Michael Bridgen <michael@weave.works>
This does the following:
- copies the type definitions from v1alpha2 to v1beta1
- changes the "stored" version to v1beta1
- gives the CRD a conversion strategy of None, meaning just rewrite the version
- switches the controller to use v1beta1
- moves the generated documentation to v1beta1
This effectively rebadges the v1alpha2 version of this part of the
image API to v1beta1. The v1alpha2 version is left in place; there are
no conversion issues, as with v1alpha1->v1alpha2. The CRD specifies
that converting between v1alpha2 and v1beta1 just means changing the
version (i.e., the schema and semantics are the same).
Signed-off-by: Michael Bridgen <michael@weave.works>
The controller is now working with 4 concurrent workers by default.
This value is configurable through the `--concurrent` flag.
Signed-off-by: Max Jonas Werner <mail@makk.es>
This finishes the v1alpha2 API, and rewrites everything needed so that
the controller supports it and the tests pass. For the most part, that
is just changing the location of fields. However, there's a few
notable extras:
- check that the `sourceRef` is a git repo (that's the default), and
that a `.spec.git` is supplied;
- change a test that blindly patched an update object, so that it
first gets the object it's patching. Previously, it succeeded
because it was OK to patch everything to empty strings, but that's
no longer the case since SourceReference.Kind is an enum.
Signed-off-by: Michael Bridgen <michael@weave.works>
This adds a v1alpha2 API, without changing the controller or tests to
use it (yet). The new API has roughly the desired shape, per
https://github.com/fluxcd/flux2/discussions/1124, but supporting only
things that the controller supports now.
It's necessary to give the v1alpha1 type a `storageversion` marker so
that 1. code generation keeps working, and 2. tests still work, since
they still expect v1alphav1 types. v1alpha1 will be removed once the
controller and tests are ported to v1alpha2.
Signed-off-by: Michael Bridgen <michael@weave.works>
Using the helper from `pkg/runtime/pprof`, which follows the suggestion
from controller-runtime to use `AddMetricsExtraHandler`.
Signed-off-by: Hidde Beydals <hello@hidde.co>
This commit upgrades the `controller-runtime` dependency to `v0.7.0`,
including all changes required to make all wiring work again.
- Upgrade `runtime` to v0.6.2 to include `controller-runtime` changes.
- Logger has been removed from the reconciler, and is now retrieved
from the `context.Context` passed to the `Reconcile` method and
downwards functions.
- Logger configuration flags are now bound to the flag set using
`BindFlags` from `runtime/logger`, ensuring the same contract across
GitOps Toolkit controllers, and the `--log-json` flag has been
deprecated in favour of the `--log-encoding=json` default.
- The `ChangePredicate` from `runtime` has changed to a
`ReconcileRequestedPredicate`, and is now chained with the
`GenerationChangedPredicate` from `controller-runtime` using
`predicate.Or`.
- Signatures that made use of `runtime.Object` have changed to
`client.Object`, removing the requirement to e.g. call
`runtime.Object#Object`.
- The `leader-election-role` was changed, as leader election now works
via the `coordination/v1` API.
Other notable changes:
- Upgrade of `image-reflector-controller` API package to include
controller-runtime changes.
- Upgrade of `source-controller` API package to v0.6.1.
Signed-off-by: Hidde Beydals <hello@hidde.co>
This adds the external event recorder (a.k.a., notifications client)
to the reconciler, and expands the definition of
`<reconciler>.event(...)` so that it will send a notification whenever
an event is emitted. This is the conventional way of handling events
amongst the GitOps Toolkit controllers.
Signed-off-by: Michael Bridgen <michael@weave.works>
- make the healthz endpoint a flag and give the value to the
controller runtime
- set up probe endpoints; copy the func from elsewhere for now
- add the probe defs to the deployment
Signed-off-by: Michael Bridgen <michael@weave.works>
This is the GitOps Toolkit convention. This commit also puts the
flags in the base deployment config, in the same order as for other
GOTK controllers.
Signed-off-by: Michael Bridgen <michael@weave.works>
This gives the controller an event recorder, without using it yet, and
makes sure it is registered when setting everything up in main.go.
Signed-off-by: Michael Bridgen <michael@weave.works>
This is a convention among the GitOps toolkit controllers; if false,
the controller will watch only resources in the namespace in which
it's running.
Signed-off-by: Michael Bridgen <michael@weave.works>
This includes changes to:
* Bump the Kubernetes dependencies to v1.19.3 to align
with the other toolkit controllers.
* Update controller-runtime dependency to v0.6.3 to align
with the other toolkit controllers.
* Update the source-controller dependency to the most recent
v0.2.2 version to include the v1beta1 API.
* Add the `pkg/gittestserver` dependency for the Git tests,
as this package was removed in a newer source-controller version.
* Bump the Go version to v1.15.
Signed-off-by: Hidde Beydals <hello@hidde.co>
Matheus Pimenta
Sunny
Stefan Prodan
Sunny
Hidde Beydals
Max Jonas Werner
Sanskar Jaiswal
Paulo Gomes
Sanskar Jaiswal
Somtochi Onyekwere
Michael Bridgen
Hidde Beydals