Commit Graph

165 Commits

Author SHA1 Message Date
Stefan Prodan 3f7d4630cc
Use the OCI artifact revision in status and events
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-09-26 13:30:28 +03:00
Stefan Prodan 082028e115
Refactor internal OCI package
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-09-20 18:34:02 +03:00
Stefan Prodan 21af88fbea
Document the Cosign verification procedure
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-09-20 14:07:11 +03:00
Stefan Prodan e1ad5a6fd3
Add `spec.insecure` to OCIRepository API
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-08-31 11:10:25 +03:00
Daniel Holbach 9669af1f38 update to new doc links structure
Signed-off-by: Daniel Holbach <daniel@weave.works>
2022-08-30 15:17:06 +02:00
Soule BA 6a8369f3b1
Add docker-registry secrets to OCI doc
A new statement is added to specify that docker-registry secrets are
accepted by OCI Helm repositories.

Signed-off-by: Soule BA <soule@weave.works>
2022-08-26 15:49:50 +02:00
Soule BA ad3eb5ca47
Enable contextual login for helm OCI
If implemented, this pr will enable user to use the auto login feature
in order to automatically login to their provider of choice's container
registry (i.e. aws, gcr, acr).

Signed-off-by: Soule BA <soule@weave.works>
2022-08-25 22:27:35 +02:00
Stefan Prodan e5cb32b0f2
Add OCI layer selector to API docs
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-08-24 12:46:04 +03:00
Paulo Gomes 106d3fcd54
Merge branch 'main' into sas-key-azure-blob 2022-08-12 15:46:03 +01:00
Somtochi Onyekwere 35268638ba Add Support for SAS keys in Azure Blob
Signed-off-by: Somtochi Onyekwere <somtochionyekwere@gmail.com>
2022-08-11 20:49:59 +01:00
Sunny c1723d4485
OCIRepo docs: auto-login setup details
Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-08-12 00:08:02 +05:30
Soule BA 0a390a0179
Document Auto-Login usage
If implemented, this provides examples on how to patch the
source-controller to enable auto-login.

Signed-off-by: Soule BA <soule@weave.works>
2022-08-11 13:25:05 +02:00
Stefan Prodan 94e98ee5ca
Add the opencontainers annotations to API docs
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-08-08 12:58:04 +03:00
Stefan Prodan 1a59935858
Add OCI failure reasons to API
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-08-05 13:24:06 +03:00
Stefan Prodan 196641147e
API docs improvements
Co-authored-by: Paulo Gomes <paulo.gomes@weave.works>
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-08-05 12:21:47 +03:00
Stefan Prodan 63c94397f7
Implement OCI auth for cloud providers
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-08-04 13:29:03 +03:00
Stefan Prodan acc95d8c50
Add upstream source and revision to logs and events
Enrich the successful reconciliation event message with the upstream opencontainers annotations

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-08-02 13:11:15 +03:00
Stefan Prodan 4b0729203b
Add OCIRepository API spec to docs
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-08-02 13:07:07 +03:00
Sanskar Jaiswal 39fd59781b docs: remove managed transport section
Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
2022-07-20 18:56:37 +05:30
Paulo Gomes 87fba45aa2
docs: Add password-protected SSH keys information
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-07-01 11:19:41 +01:00
Somtochi Onyekwere bb7aa14538 Add docs on managed identity for Azure Blob
Signed-off-by: Somtochi Onyekwere <somtochionyekwere@gmail.com>
2022-06-03 10:05:31 +01:00
Sanskar Jaiswal 04e167cd01 Release v0.25.0
Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
2022-06-01 16:27:30 +05:30
Paulo Gomes a0d0a6312d
libgit2: enable managed transport by default
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-05-24 10:46:32 +01:00
Sunny 581695b4d6
gitrepo: Intro contentConfigChecksum & improvement
Introduce contentConfigChecksum in the GitRepository.Status to track the
configurations that affect the content of the artifact. It is used to
detect a change in the configuration that requires rebuilding the whole
artifact. This helps skip the reconciliation early when we find out that
the remote repository has not changed.

Moves fetching the included repositories in reconcileSource() to collect
enough information in reconcileSource() to be able to decide if the full
reconciliation can be skipped. This results in reconcileInclude() to
just copy artifact to the source build directory.

Introduce a gitCheckout() method to perform construction of all the git
checkout options and perform the checkout operation. This helps to
easily perform checkout multiple times when we need it in
reconcileSource(). When we check with the remote repository if there's
an update, and find out that there's no update, we check if any other
configurations that affect the source content has changed, like
includes, ignore rules, etc. If there's a change, we need to perform a
full checkout of the remote repository in order to fetch the complete
source. The git checkout no-op optimization is enabled in this method
based on the presence of an artifact in the storage.

The failure notification handler is modifed to handle the recovery of a
no-op reconcile failure and create a notification message accordingly
with the partial commit.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-05-20 19:52:18 +05:30
Max Jonas Werner 841ed7ae66
[RFC 0002] Flux OCI support for Helm (#690)
* Add OCI Helm support

* users will be able to declare OCI HelmRepository by using the `.spec.type` field of the HelmRepository API. Contrary to the HTTP/S HelmRepository no index.yaml is reconciled from source, instead a simple url and credentials validation is performed.
* For backwards-compatibility, an empty `.spec.type` field leads to the HelmRepository being treated as a plain old HTTP Helm repository.
* users will be able to declare the new OCI HelmRepository type as source using the .Spec.SourceRef field of the HelmChart API. This will result in reconciling a chart from an OCI repository.
* Add registryTestServer in the test suite and OCI HelmRepository test case
* Add a new OCI chart repository type that manage tags and charts from an OCI registry.
* Adapat RemoteBuilder to accept both repository types
* discard output from OCI registry client; The client has no way to set a verbosity level and spamming the controller logs with "Login succeeded" every time the object is reconciled doesn't help much.

Signed-off-by: Soule BA <soule@weave.works>
Signed-off-by: Max Jonas Werner <mail@makk.es>
Co-authored-by: Soule BA <soule@weave.works>
2022-05-19 14:50:16 +02:00
Paulo Gomes 262efc08bc
Add git-optimized-clones feature gate
OptimizedGitClones decreases resource utilization for GitRepository
reconciliations. It supports both go-git and libgit2 implementations
when cloning repositories using branches or tags.

This is an opt-out feature, which can be disabled by starting the
controller with the argument '--feature-gates=OptimizedGitClones=false'.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-05-11 14:39:21 +01:00
Paulo Gomes 50cb97f331
Fix make verify failures
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-05-09 16:16:04 +01:00
Alexander Block 065a760752 docs: Remove all traces of "identity.pub" from docs
"identity.pub" is referenced multiple times in CRDs and docs. This secret
is however never used in any place. Instead, the public key is derived from
the "identity" private key.

This commit/PR removes all traces of "identity.pub" from v1beta2 CRDs and
docs.

Signed-off-by: Alexander Block <ablock84@gmail.com>
2022-04-29 21:17:53 +02:00
Soule BA 7ff96a8b0c
Update the docs with a section regarding the cache usage
Signed-off-by: Soule BA <soule@weave.works>
2022-04-02 18:13:59 +02:00
Sunny b869716ddf
Update docs with new conditions and events
Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-03-30 17:06:16 +05:30
Paulo Gomes ca64fac640
Add experimental transport to spec documentation.
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-03-22 19:22:19 +00:00
Sunny 6830e4e423
docs: Update docs with new condition and reasons
Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-03-14 21:20:41 +05:30
Hidde Beydals ff536e10bb doc/spec: use --field-manager in kubectl examples
To align with:
https://fluxcd.io/docs/faq/#why-are-kubectl-edits-rolled-back-by-flux

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-03-11 10:04:14 +01:00
Hidde Beydals abf29da106 docs/spec: document Bucket's Azure Blob support
Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-03-11 10:04:14 +01:00
Sunny 8d4fce1955 Add more details in helmchart spec doc
Details about the source reference, reconcile strategy and artifact
revision value based on the reconcile strategy.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-03-11 10:04:14 +01:00
Sunny 9e73bfab9e docs/spec: Update the outputs
Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-03-11 10:04:14 +01:00
Sunny f59b48c339 Document HelmChart API v1beta2 spec
Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-03-11 10:04:14 +01:00
Hidde Beydals cbffd82d8f Document GitRepository API v1beta2 spec
Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-03-11 10:04:14 +01:00
Hidde Beydals 86d1d80bf2 Document HelmRepository API v1beta2 spec
Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-03-11 10:04:14 +01:00
Hidde Beydals 5832296ef5 Document Bucket API v1beta2 spec
Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-03-11 10:04:14 +01:00
Stefan Prodan fb45c9caf6
Increase default timeout to 60s
Increase the default timeout from 20s to 60s for Git repository cloning and Bucket download

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-02-07 10:29:43 +02:00
Sunny 017207d99e
docs: Add git proxy support docs
Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-01-20 15:21:49 +05:30
Hidde Beydals d0ca107e55 docs: GitRepository commit without branch example
Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-27 17:00:11 +05:30
pa250194 7921caf056 Updated docs to include GCP provider instructions
Signed-off-by: pa250194 <pa250194@ncr.com>
2021-10-14 14:26:33 -05:00
pa250194 99c79bffaa Tests for GCP Bucket Provider
Signed-off-by: pa250194 <pa250194@ncr.com>
2021-10-14 14:19:19 -05:00
pa250194 39811ed46a Add Support for GCP storage with workload identity
Added Support for Google Cloud Storage with Workload Identity as
Source Provider. This enables the use of GCP without enabling S3
compatible access.

Signed-off-by: pa250194 <pa250194@ncr.com>
2021-10-14 14:07:11 -05:00
Dylan Arbour 5e6abae9e8 Add ReconcileStrategy to HelmChart
This commit adds a `ReconcileStrategy` field to the `HelmChart` resource, which
allows defining when a new chart should be packaged and/or published if it
originates from a `Bucket` or `GitRepository` resource.

The two available strategies are:

- `ChartVersion`: creates a new artifact when the version of the Helm chart as
  defined in the `Chart.yaml` from the Source is different from the current
  version.
- `Revision`: creates a new artifact when the revision of the Source is
  different from the current revision.

For the `Revision` strategy, the (checksum part of the) revision of the
artifact the chart originatesfrom is added as SemVer metadata.

A chart from a `GitRepository` with Artifact revision
`main/f0faacd5164a875ebdbd9e3fab778f49c5aadbbc` and a chart with e.g. SemVer
`0.1.0` will be published as `0.1.0+f0faacd5164a875ebdbd9e3fab778f49c5aadbbc`.

A chart from a `Bucket` with Artifact revision
`f0faacd5164a875ebdbd9e3fab778f49c5aadbbc` and a chart with e.g. SemVer `0.1.0`
will be published as `0.1.0+f0faacd5164a875ebdbd9e3fab778f49c5aadbbc`.

Signed-off-by: Dylan Arbour <arbourd@users.noreply.github.com>
2021-10-14 13:49:52 -05:00
pa250194 a6be9c8a21 Updated docs to include GCP provider instructions
Signed-off-by: pa250194 <pa250194@ncr.com>
2021-09-16 12:16:17 -05:00
pa250194 eeb38bdf2c Tests for GCP Bucket Provider
Signed-off-by: pa250194 <pa250194@ncr.com>
2021-09-16 09:49:56 -05:00
pa250194 7da9619b58 Feature: Add Support for Google Cloud Storage along with Workload Identity
Added Support for Google Cloud Storage with Workload Identity as Source Provider. This enables the use of GCP without enabling S3 compatible access.

Signed-off-by: pa250194 <pa250194@ncr.com>
2021-09-16 09:49:56 -05:00
Hidde Beydals 1f27410b34 Update Helm to v3.6.1
v3.6.1 is a a security update from Helm, ensuring that credentials are
always only passed to the defined repository host.

Based on Helm user reports, disabling this behavior may be required for
some Helm repository solutions like Artifactory, and may be done by
setting `PassCredentials` in the `HelmRepositorySpec`.

For more information, see:
https://github.com/helm/helm/security/advisories/GHSA-56hp-xqp3-w2jf

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-06-18 12:31:23 +02:00
Stefan Prodan add5444f16
Fix GitRepository include for nested paths
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-05-28 01:17:40 +03:00
Stefan Prodan 145a82a429
Release v0.13.0
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-05-26 14:45:14 +03:00
Philip Laine fcf7048992 Add include property to GitRepositories
Signed-off-by: Philip Laine <philip.laine@gmail.com>
Signed-off-by: Philip Laine <philip.laine@xenit.se>
2021-05-11 09:46:50 +02:00
Stefan Prodan bd0f1fdd19
Release v0.12.0
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-04-21 17:57:50 +03:00
Dylan Arbour 4a834e1d2d Add `ValuesFiles` to HelmChart spec
Signed-off-by: Dylan Arbour <arbourd@users.noreply.github.com>
2021-04-19 09:16:53 -04:00
Stefan Prodan 9a08c0cc52
Add well-known CI configs to exclusion list
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-03-31 14:39:12 +03:00
Stefan Prodan 664a568822
Add support for Git submodules with go-git
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-03-30 13:00:13 +03:00
Stefan Prodan 97d0326432
Enable git submodules cloning when using go-git
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-03-29 15:25:33 +03:00
Stefan Prodan f0016cfad1
Enable self-signed certs for go-git
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-03-29 13:23:32 +03:00
Sam Tavakoli 64aa488f4a
Add IAM role S3 bucket policy example to documentation
Quick overview for what permissions are requried by the controller.

Signed-off-by: Sam Tavakoli <sam.tavakoli@form3.tech>
2021-03-26 13:26:43 +00:00
Jonathan Innis df91005cdf
Typo fix in GitRepositories doc file
Signed-off-by: jonathan-innis <jonathan.innis.ji@gmail.com>
2021-02-22 16:24:17 -08:00
Philip Laine c063484761 Add custom certificate validation
Signed-off-by: Philip Laine <philip.laine@gmail.com>
2021-02-08 12:19:22 +01:00
Philip Laine f044ebc253 Rename git2go option to libgit2
Signed-off-by: Philip Laine <philip.laine@xenit.se>
2020-12-06 23:19:57 +01:00
Philip Laine 1b4eacc588 Refactor argument name to enable git2go
Signed-off-by: Philip Laine <philip.laine@xenit.se>
2020-12-04 10:07:17 +01:00
Philip Laine 5da36ec864 Add docs for v2 compatibility
Signed-off-by: Philip Laine <philip.laine@xenit.se>
2020-12-02 20:18:29 +01:00
Stefan Prodan 07d09e2c2e
Add suspend field to Source API
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-11-26 12:49:12 +02:00
Hidde Beydals 0b4586253c Add LastHandledReconcileAt to source statuses
Signed-off-by: Hidde Beydals <hello@hidde.co>
2020-11-19 13:21:59 +01:00
Tomas Virgl d55b37e838 docs: minor typo fix
Signed-off-by: Tomas Virgl <tomas@virgl.net>
2020-11-08 00:59:09 +01:00
Hidde Beydals b2aaf2932b Change links to SemVer range docs
Signed-off-by: Hidde Beydals <hello@hidde.co>
2020-10-28 12:43:23 +01:00
Hidde Beydals c657e68ba1 Various API doc corrections
Signed-off-by: Hidde Beydals <hello@hidde.co>
2020-10-28 01:09:21 +01:00
Stefan Prodan 4b33c816ef
Add namespace to API docs examples
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-10-13 17:05:22 +03:00
Yiannis 1707898426 Add interval to a couple of examples in docs as it is now a required
field
2020-10-06 11:06:18 +01:00
Hidde Beydals e7946f2827 Drop `gitops-system` namespace reference from spec 2020-10-02 13:19:01 +02:00
stefanprodan f16e5f041a Promote API to v1beta1 2020-09-30 16:11:00 +03:00
Hidde Beydals 431ea05427 Implement `fluxcd/pkg/meta/api` in APIs 2020-09-30 10:18:43 +02:00
Hidde Beydals 8cdb821946 docs: document Bucket support HelmChart 2020-09-21 12:40:15 +02:00
stefanprodan 133904d025 Add Bucket type to source API spec 2020-09-18 14:01:28 +03:00
Hidde Beydals 99b74da044 api/artifact: add checksum field to artifact
This includes a change to how the revision for HelmRepository sources is
recorded, as this will now equal to the generated timestamp from the index
in RFC3339Nano format.
2020-09-09 13:41:56 +02:00
Hidde Beydals ce1b80e122 spec: update HelmChart examples 2020-09-04 14:25:58 +02:00
Hidde Beydals 24f47ac183 Support Helm charts from GitRepository sources 2020-08-31 16:21:53 +02:00
stefanprodan 600006293d Add sections to Git examples 2020-08-31 13:17:50 +03:00
stefanprodan 4486ab7a5e Implement HelmRepository timeout 2020-08-12 14:10:03 +03:00
Hidde Beydals 8e1b213da5 Change CRD domain to 'source.toolkit.fluxcd.io'
Due to required domain changes for the helm-controller so that it
can co-exist in a cluster with the Helm Operator, other Toolkit
components are moving to a *.toolklit.fluxcd.io domain too.
2020-07-30 21:50:46 +02:00
stefanprodan b3e58b48dd Rename syncAt annotation to reconcileAt
Use `fluxcd.io/reconcileAt` annotation across all controllers.
2020-07-13 16:06:16 +03:00
stefanprodan b9dc2ecf64 Add ignore field to GitRepository spec 2020-07-09 14:48:02 +03:00
stefanprodan 4887764947 Move Receiver API to notification controller 2020-07-03 16:10:57 +03:00
Hidde Beydals 283c0a5275 Release v0.0.1 2020-06-24 16:17:39 +02:00
Hidde Beydals 43ea51b435 Spec: document Git artifact file exclusion 2020-06-06 19:15:57 +02:00
stefanprodan c9db552555 git: fix reference json serialization
- add omitempty to GitRepositoryRef fields
2020-04-30 09:54:34 +03:00
stefanprodan 8071dadbf0 status: record progressing
Set ready condition to unknown while the reconciliation is progressing.
This allows other operators to wait for a sync to complete.
2020-04-27 11:16:35 +03:00
Hidde Beydals 13a0f1232a spec: document GitRepositorySpec Timeout field 2020-04-24 11:59:34 +02:00
stefanprodan 8bf27c0e3b proposal: webhook receiver 2020-04-22 10:24:32 +03:00
stefanprodan fce046f440 docs: Sync specification with API
- update status examples to mach the implementation
- add interval to HelmChart examples
2020-04-20 10:50:26 +03:00
Hidde Beydals 000bf41dc2 Address comments 2020-04-16 13:12:27 +03:00
Hidde Beydals 8b4432fc5c Add impact to Helm Operator 2020-04-16 13:12:27 +03:00
stefanprodan 0084e56971 Add impact to Flux
- reword motivation by squaremo
2020-04-16 13:12:27 +03:00
stefanprodan 7323a45504 Add signature verification to Git spec 2020-04-16 13:12:27 +03:00
Hidde Beydals 51f82e72a8 Drop interval fallback for HelmChart 2020-04-16 13:12:27 +03:00
Hidde Beydals e0eb330d42 Add source interface spec 2020-04-16 13:12:27 +03:00