History

Sunny 581695b4d6 gitrepo: Intro contentConfigChecksum & improvement Introduce contentConfigChecksum in the GitRepository.Status to track the configurations that affect the content of the artifact. It is used to detect a change in the configuration that requires rebuilding the whole artifact. This helps skip the reconciliation early when we find out that the remote repository has not changed. Moves fetching the included repositories in reconcileSource() to collect enough information in reconcileSource() to be able to decide if the full reconciliation can be skipped. This results in reconcileInclude() to just copy artifact to the source build directory. Introduce a gitCheckout() method to perform construction of all the git checkout options and perform the checkout operation. This helps to easily perform checkout multiple times when we need it in reconcileSource(). When we check with the remote repository if there's an update, and find out that there's no update, we check if any other configurations that affect the source content has changed, like includes, ignore rules, etc. If there's a change, we need to perform a full checkout of the remote repository in order to fetch the complete source. The git checkout no-op optimization is enabled in this method based on the presence of an artifact in the storage. The failure notification handler is modifed to handle the recovery of a no-op reconcile failure and create a notification message accordingly with the partial commit. Signed-off-by: Sunny <darkowlzz@protonmail.com>		2022-05-20 19:52:18 +05:30
..
v1alpha1	Updated docs to include GCP provider instructions	2021-10-14 14:26:33 -05:00
v1beta1	Fix make verify failures	2022-05-09 16:16:04 +01:00
v1beta2	gitrepo: Intro contentConfigChecksum & improvement	2022-05-20 19:52:18 +05:30
README.md	Document Bucket API v1beta2 spec	2022-03-11 10:04:14 +01:00

README.md

Source Controller

The main goal is to define a set of Kubernetes objects that cluster admins and various automated operators can interact with to offload the sources (e.g. Git and Helm repositories) registration, authentication, verification and resource fetching to a dedicated controller.

Motivation

Each Flux and each Helm operator mirrors the Git repositories they are using, in the same way, using the same code. But other components might benefit from access to the source mirrors, and Flux and the Helm operator could work more in sympathy with Kubernetes by factoring it out.

If "sources" (usually git repos, but also Helm charts and potentially other things) existed in their own right as Kubernetes resources, components like Flux and Helm operator could use standard Kubernetes mechanisms to build on them; and, they could be managed independently of the components using them.

API Specification

Implementation

The controller implementation will watch for source objects in a cluster and act on them. The actions performed by the source controller could be:

validate source definitions
authenticate to sources and validate authenticity
detect source changes based on update policies (semver)
fetch resources on-demand and on-a-schedule
package the fetched resources into a well known format (tar.gz, yaml)
store the artifacts locally
make the artifacts addressable by their source identifier (sha, version, ts)
make the artifacts available in-cluster to interested 3rd parties
notify interested 3rd parties of source changes and availability (status conditions, events, hooks)

Impact to Flux

Having a dedicated controller that manages Git repositories defined with Kubernetes custom resources would:

simplify Flux configuration as fluxd could subscribe to Git sources in-cluster and pull the artifacts automatically without manual intervention from users to reconfigure and redeploy FLux
improve the installation experience as users will not have to patch fluxd's deployment to inject the HTTPS basic auth credentials, change the source URL or other Git and PGP related settings
enable fluxd to compose the desired state of a cluster from multiple sources by applying all artifacts present in flux namespace
enable fluxd to apply manifests coming from other sources than Git, e.g. S3 buckets
allow fluxd to run under a non-root user as it wouldn't need to shell out to ssh-keygen, git or pgp
enable fluxd to apply manifests coming from the most recent semver tag of a Git repository
allow user to pin the cluster desired state to a specific Git commit or Git tag

Impact to Helm Operator

Having a dedicated controller that manages Helm repositories and charts defined with Kubernetes custom resources would:

simplify the Helm Operator configuration as repository and chart definitions can be re-used across HelmRelease resources (see fluxcd/helm-operator#142)
improve the user experience as repositories requiring authentication will no longer require a repositories.yaml import / file mount
simplify the architecture of the Helm Operator as it allows the operator to work with a single source type (HelmChart) and way of preparing and executing installations and/or upgrades
allow the Helm Operator to run under a non-root user as it wouldn't need to shell out to git