Commit Graph

1741 Commits

Author SHA1 Message Date
Stefan Prodan 3168bb6051
Introduce artifact max size limit of 50MiB
Add a controller flag named "--artifact-max-size=<bytes>" with the default value of 50MiB.
To disable the limit, the value can be set to "--artifact-max-size=-1".
The flag enforces a max size limit for the artifact contents produced by source-controller,
to avoid out-of-memory crashes of consumers such as kustomize-controller.

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-10-11 17:53:52 +03:00
Sunny 35ea086358
Merge pull request #932 from fluxcd/update-deps-oci-status-check
update dependencies
2022-10-11 20:20:55 +05:30
Sunny df1894b7ed
update dependencies
- replace github.com/darkowlzz/controller-check/status ->
  github.com/fluxcd/pkg/runtime/conditions/check
- github.com/fluxcd/pkg/runtime - v0.21.0
- github.com/fluxcd/pkg/oci - v0.13.0
- github.com/fluxcd/pkg/apis/meta - v0.17.0
- k8s.io/cli-runtime - v0.25.2

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-10-11 19:08:44 +05:30
Stefan Prodan 5e83ecacec
Merge pull request #926 from fluxcd/status-observed-source-config
Add explicit observed artifact content configurations in status
2022-10-11 13:46:05 +03:00
Sunny a6d7948667 Bucket: Add status.observedIgnore
Introduce status.observedIgnore in the Bucket API for consistency with
other sources with ignore.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-10-10 23:06:02 +05:30
Sunny e996848555 GitRepo: Add observed content config in status
Replace content config checksum with explicit artifact content config
observations. It makes the observations of the controller more
transparent and easier to debug.

Introduces `observedIgnore`, `observedRecurseSubmodules` and
`observedInclude` status fields.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-10-10 23:06:02 +05:30
Sunny 278a223bc6 OCIRepo: Add observed content config in status
Replace content config checksum with explicit artifact content config
observations. It makes the observations of the controller more
transparent and easier to debug.

Introduces `observedIgnore` and `observedLayerSelector` status fields.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-10-10 23:06:02 +05:30
Paulo Gomes 70d9f126f9
Merge pull request #930 from pjbgf/release-0.30.1
Release v0.30.1
2022-10-10 17:07:13 +01:00
Paulo Gomes a969244eeb
Release v0.30.1
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-10-10 16:39:19 +01:00
Paulo Gomes 34f127b67d
Merge pull request #906 from somtochiama/sas-docs
List objects when checking if bucket exists to allow use of container-level SAS token
2022-10-07 20:45:09 +01:00
Somtochi Onyekwere 874714aed1 correct spacing
Signed-off-by: Somtochi Onyekwere <somtochionyekwere@gmail.com>
2022-10-07 20:27:02 +01:00
Somtochi Onyekwere 5eeaa6455d List objects instead when checking if bucket exists in Azure
Signed-off-by: Somtochi Onyekwere <somtochionyekwere@gmail.com>
2022-10-07 20:11:04 +01:00
Stefan Prodan f4de0a4470
Merge pull request #923 from fluxcd/release-v0.30.0
Release v0.30.0
2022-09-29 18:52:00 +03:00
Paulo Gomes 7349c6eaee
Release v0.30.0
Signed-off-by: Sunny <darkowlzz@protonmail.com>
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-09-29 18:38:18 +03:00
Stefan Prodan 0598bee9e1
Merge pull request #922 from fluxcd/update-oci-0.11.0
Update pkg/oci to v0.11.0
2022-09-29 17:58:11 +03:00
Sunny 36a9e99e27 Update pkg/oci to v0.11.0
This version allows for using the OCI HelmRepo URL that points at the
root of an AWS ECR repository.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-09-29 19:48:34 +05:30
Stefan Prodan 2a2b525dbd
Merge pull request #916 from souleb/fix-915
Accept a slice of remote.Option for cosign verification
2022-09-29 16:03:21 +03:00
Soule BA f51c98ecad
Fail when verifying with insecure
If implemented we fails when trying to verify with insecure set. This
will likely change once cosign add support for insecure registries.

Signed-off-by: Soule BA <soule@weave.works>
2022-09-29 14:45:25 +02:00
Soule BA e2f4e5e7c5
Accept a slice of remote.Option for cosign verification
If implemented this enable passing a keychain, an authenticator and a
custom transport as remote.Option to the verifier. It enables contextual
login, self-signed certificates and insecure registries.

Signed-off-by: Soule BA <soule@weave.works>

refactor makeOptions

Reduce complexity by replacing the functional options with a flat out
conditional logic in makeOptions.

Signed-off-by: Soule BA <soule@weave.works>
2022-09-29 14:45:25 +02:00
Stefan Prodan 95cbf40941
Merge pull request #917 from fluxcd/improv-ocirepo-optimized-reconcile
OCIRepositoryReconciler no-op improvements
2022-09-29 10:39:24 +03:00
Sunny dcd0db406e
OCIRepo: Implement source ignore
This implements source ignore in OCIRepositoryReconcilers'
reconcileArtifact so that the ignore rules are considered when building
the artifact.

Adds tests based on the artifact checksum change when ignore rules are
applied.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-09-29 09:48:32 +03:00
Sunny f4aed8baf8
OCIRepoReconciler: no-op reconcile improvements
Introduce contentConfigChecksum in the OCIRepository status to store a
checksum of the values that affect the source artifact. It is used to
detect when to rebuild an artifact when the spec changes.

The considerations for this are similar to the GitRepository
reconciler no-op clone implementation. Both reconcileSource and
reconcileArtifact need to consider the source configuration change
when deciding if the artifact in the storage is up-to-date.

Adds tests for reconcileSource and reconcileArtifact for the noop
cases.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-09-29 09:48:27 +03:00
Paulo Gomes 5ea49229f7
Merge pull request #921 from pjbgf/bump-libgit2-image
Bump libgit2 image and disable cosign verification for CI
2022-09-29 07:21:55 +01:00
Paulo Gomes 6c06f4e222
The libgit2 libraries are downloaded and verified before
some of the make targets are executed. This assures the
provenance of such files before using them and is very
important specially for end users running such tests on
their machines.

Note that has been disabled specially due to recent issues
we experienced at CI which can be seen in:
fluxcd/source-controller#899

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-09-29 07:01:36 +01:00
Paulo Gomes 1ab76264de
Bump libgit2 image to v0.3.0
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-09-29 06:55:18 +01:00
Stefan Prodan ac80849d29
Merge pull request #920 from fluxcd/build-go1.19
Build with Go 1.19
2022-09-28 15:54:48 +03:00
Stefan Prodan ca3496e758
Build with Go 1.19
- Update Go to 1.19 in CI
- Use Go 1.19 in base image
- Update controller-gen v0.8.0 and regenerate manifests

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-09-28 15:35:26 +03:00
Stefan Prodan 2833caa346
Merge pull request #919 from fluxcd/controller-runtime-v0.13.0
Update dependencies
2022-09-28 15:06:07 +03:00
Stefan Prodan 8614543e73
Update dependencies
- k8s.io/* v0.25.2
- helm.sh/helm/v3 v3.10.0
- sigs.k8s.io/controller-runtime v0.13.0
- cloud.google.com/go/storage v1.27.0
- fluxcd/pkg/runtime v0.19.0
- sigstore/sigstore v1.4.2
- github.com/fluxcd/git2go/v33 v33.0.9-flux (use Flux own fork)

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-09-28 14:45:19 +03:00
Stefan Prodan 8bc36bcfc5
Merge pull request #913 from fluxcd/oci-copy-layer
[OCIRepository] Optimise OCI artifacts reconciliation
2022-09-26 13:51:15 +03:00
Stefan Prodan 3f7d4630cc
Use the OCI artifact revision in status and events
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-09-26 13:30:28 +03:00
Stefan Prodan aae9d917fb
Optimise OCI artifacts reconciliation
- Fetch the upstream digest before validation and pulling
- Pull artifact only if the upstream digest is different from the one in storage
- Add the image tag to the revision string `<tag>/<digest-hex>` for a better UX
- Extract the layer processing to a dedicated function

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-09-24 10:30:46 +03:00
Stefan Prodan 4ec51ca306
Add option to copy the OCI layer to storage
Add on optional field to the `OCIRepository.spec.layerSelector` called `operation` that accepts one of the following values: `extract` or `copy`. When the operation is set to `copy`, instead of extracting the compressed layer, the controller copies the compressed blob as it is to storage, thus keeping the original content unaltered.

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-09-23 19:00:30 +03:00
Stefan Prodan 9c6dc330ae
Merge pull request #904 from fluxcd/add-ca-cert
Add custom CA certificates to system certificates
2022-09-23 11:11:52 +03:00
Max Jonas Werner 7a139647a2
add custom CA certificates to system certificates
When a custom CA certificate is provided in a Secret's `caCert` field
referenced in `HelmRelease.spec.secretRef` then that CA cert is now
added to the list of system certificates instead of it replacing the
system certificates. This makes HelmRepositories work in mixed
environments where charts are pulled from both, a public repository
and a private repository (e.g. through a chart dependency).

The test that is added as part of this change will fail without the
change and passes with it.

closes #866
closes fluxcd/helm-controller#519

Signed-off-by: Max Jonas Werner <max@e13.dev>
2022-09-23 10:57:00 +03:00
Hidde Beydals 20fa94acc4
Merge pull request #911 from fluxcd/update-bucket-deps
Update Bucket related SDK dependencies
2022-09-22 16:42:47 +00:00
Hidde Beydals 588fe0d15a Update Bucket related SDK dependencies
Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-09-22 14:57:25 +00:00
Stefan Prodan ebbc9983e9
Merge pull request #907 from fluxcd/summarize-with-bipolarity
Consider bipolarity conditions in Ready condition summarization
2022-09-22 15:26:32 +03:00
Sunny 90b7cec915 ocirepo: Fix event trace type value
Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-09-22 15:42:53 +05:30
Sunny 64bd34f116 Use bipolarity option in gitrepo and ocirepo
Use the bipolarity condition options in OCIRepository and GitRepository
reconcilers.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-09-22 15:42:53 +05:30
Sunny e5d3aa3701 summarize: consider bipolarity in status condition
This introduces the consideration of bipolarity conditions in the status
condition summary for Ready condition. The summarize.HelperOptions can
now be configured with a list of bipolarity conditions which are used in
SummarizeAndPatch() to set the Ready condition to failing bipolarity
condition with the highest priority.

Bipolarity condition is not a typical status property. It is a mix of
positive and negative polarities. It's "normal-true" and
"abnormal-false". Failing bipolarity conditions are prioritized over
other conditions to show the actual reason of failure on the Ready
status.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-09-22 15:42:53 +05:30
Stefan Prodan c9a5a56cfb
Merge pull request #876 from developer-guy/feature/863
[RFC-0003] Implement OCIRepository verification using Cosign
2022-09-22 13:04:55 +03:00
Stefan Prodan 3b637a82fe
Add tests for keyless verification
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-09-22 12:25:40 +03:00
Stefan Prodan b5ffc9fc65
Update sigstore/cosign to v1.12.1
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-09-22 11:52:19 +03:00
Stefan Prodan 7c92949d17
Retry failed verifications with exponential backoff
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-09-20 21:26:34 +03:00
Stefan Prodan 082028e115
Refactor internal OCI package
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-09-20 18:34:02 +03:00
Stefan Prodan 21af88fbea
Document the Cosign verification procedure
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-09-20 14:07:11 +03:00
Furkan 03af4f6418
fix: ocirepository_controller reviews
Signed-off-by: Furkan <furkan.turkal@trendyol.com>
2022-09-20 14:07:11 +03:00
Batuhan Apaydın 2db2715988
feat: add condition tests for verification logic
Signed-off-by: Batuhan Apaydın <batuhan.apaydin@trendyol.com>
2022-09-20 14:07:11 +03:00
Stefan Prodan 07b532674c
Add omitempty to cosgin secretRef
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-09-20 14:07:11 +03:00