With static HelmRepository OCI, the interval become optional. Make
interval optional in the API. Introduce getters for interval, in the
form of GetRequeueAfter(), and timeout with internal default values.
HelmRepository will not have interval and timeout fields unless it's
explicitly set.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
Remove the HelmRepositoryOCI reconciler and make HelmRepository of type
OCI static. The existing HelmRepository OCI objects are migrated to
static object by removing their finalizers and status. New
HelmRepository OCI objects go through one time migration to remove the
status. These are not reconciled again, unless the type is changed to
default. On type switching from HelmRepository default to OCI, the
finalizer, status and artifact are removed to make the object static. On
switching from OCI to default, a complete reconciliation of
HelmRepository takes place to build artifact and add status and
finalizer.
The HelmRepository .spec.url has a new validation to check the URL
scheme. This is to add some validation to HelmRepository OCI since it's
not backed by a reconciler for full validation.
Add HelmRepositoryOCIMigrationPredicate predicate to detect and allow
reconciliation of HelmRepository OCI objects that need migration. The
other predicates that filtered the HelmRepository events based on the
type have been removed as all the HelmRepositories will now be
reconciled by a single reconciler. HelmRepositoryOCIMigrationPredicate
readily allows non-OCI objects and only checks if a migration is needed
for OCI type object.
Add controller tests for different migration scenarios.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
Modify `GetHelmClientOpts()` to only configure the TLS login option when
an authentication login option is configured. This prevents the
reconciler from trying to authenticate against public registries.
Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
Add `.spec.verify.matchOIDCIdentity` to OCIRepository and HelmChart.
It allows specifying regular expressions to match against the subject and
issuer of the certificate related to the artifact signature. Its used
only if the artifact was signed using Cosign keyless signing.
Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
Do not error out when upstream artifacts contain symlinks in the content layer, instead skip all symlinks during decompression.
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
`crane` package is the highest level of abstraction that GGCR provides,
it's easy to use, however it doesn't give user much control.
This change moves `OCIRepository` controller logic to a lower-level
`remote` package and makes handling of references more explicit with
`name.Repository`, `name.Digest` and `name.Tag`.
It also simplifies options builder, as there is no need to have separate
sets of options for cosign and crane.
Signed-off-by: Ilya Dmitrichenko <errordeveloper@gmail.com>
Introduce a new event reason for patch operation failure and update all
the returned errors from serial patcher to be a generic error so that
they are handled like any other error with an associated warning event.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
Remove deprecated Event error. Event error was used for scenarios where
an error should result in an event/notification. It was introduced as a
contextual error along with Stalling and Waiting errors but was later
replaced with Generic error which doesn't have any contextual meaning.
The Generic error provided error configuration which allowed defining
how the error should be handled. This replaced the contextual error
handling with error action handlers which behaved on the error
configuration of the errors.
The Generic error was first introduced to be used in GitRepository
reconciler and was used by new reconcilers like the OCIRepository
reconcilers. The old reconcilers bucket, helmrepository and helmchart
reconcilers were still using the deprecated Event error. This change
replaces the Event errors in these reconcilers with a Generic error.
It also fixes a bug in the Generic error constructor which configured
the error to be logged by default. This resulted in an error to be
logged by the result processor and the runtime, double logging. This
behavior has been changed to not log explicitly and allow the runtime to
log the error. Since the Generic error is based on defining the error
handling behavior in the error configuration, a generic error that needs
to be ignored (not returned to the runtime), but logged can enable the
logging behavior explicitly on the Generic error instance. This is done
in GitRepository reconciler for no-op reconciliations where an ignore
error is returned.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
This is a regression fix introduced in a302c71 which would wrongly check
for the type of the Secret specified in `.spec.secretRef` while
configuring TLS data.
Introduce `LegacyTLSClientConfigFromSecret` which does not check the
Secret type while constructing the TLS config.
Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
Check the auth secret for the `ca.crt` key for CA certificate data.
`ca.crt` takes precdence over `caFile`.
Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
Adopt Kubernetes TLS secrets API to check for TLS data in the Secret
referred to by `.spec.certSecretRef`, i.e. check for keys `tls.crt` and
`tls.key` for the certificate and private key. Use `ca.crt` for the CA
certificate.
Deprecate the usage of `caFile`, `certFile` and `keyFile` keys.
Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
Adopt Kubernetes TLS secrets API to check for TLS data in the Secret
referred to by `.spec.certSecretRef`, i.e. check for keys `tls.crt` and
`tls.key` for the certificate and private key. Use `ca.crt` for the CA
certificate.
Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
Add support for verifying tags and optionally the commit object it
points to. Modify the reconciler to trigger a full reconciliation if the
object contains a verification configuration that implies that we need
to verify one (or more) Git objects that we haven't previosuly verified.
Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
Move record suspend metrics next to readiness and duration metrics so
that it gets recorded along with others always at the end and the
metrics delete, which requires the knowledge of deleted finalizers,
applies to suspend too.
HelmRepository cache event metrics for a given helmrepo also continues
to be exported even after the object is deleted. This change deletes
the cache event metrics when the object is deleted.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
This adds a `--interval-jitter-percentage` flag to the controller to
add a +/- percentage jitter to the interval defined in resources
(defaults to 10%).
Effectively, this results in a reconcilation every 4.5 - 5.5 minutes
for a resource with an interval of 5 minutes.
Main reason to add this change is to mitigate spikes in memory and
CPU usage caused by many resources being configured with the same
interval.
Signed-off-by: Hidde Beydals <hidde@hhh.computer>
If implemented user will be able to provide their own custom start and
bypass tls verification when interacting with OCI registries over https
to pull helmCharts.
Signed-off-by: Soule BA <soule@weave.works>
In Reconcile() methods, move the object deletion above add finalizer.
Finalizers can't be set when an object is being deleted.
Introduce a cacheless client in suite_test to use for testing this
change. It ensures that the Reconcile() call always operates on the
latest version of the object which has the deletion timestamp and
existing finalizer.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
Add tests to test Cosign support for insecure registries. Furthermore,
refactor OCI test utils to be more user friendly and enable accurate
testing of HTTPS and HTTP OCI registries by circumnavigating Docker's
automatic connection downgrade for registries hosted on localhost.
Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
Add support for verifying insecure HTTP OCI repositories with cosign. If
`.spec.insecure` set to true, then cosign uses plain HTTP connections to
communicate with the registry.
Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
Add support for specifying TLS auth data via `.spec.certSecretRef` in
HelmRepository and log a deprecation warning if TLS is configured via
`.spec.secretRef`. Introduce (and refactor) Helm client builder and
auth helpers to reduce duplicated code and increase uniformity and
testability.
Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
Add `.spec.proxySecretRef.name` to the `GitRepository` API to allow
referencing a secret containing the proxy settings to be used for all
remote Git operations for the particular `GitRepository` object.
It takes precedence over any proxy configured through enviornment
variables.
Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
This is required because the test fails with Git >=v2.41.0 due to
changes to commands used by the Git test server. Causing the server to
return an error when cloning an empty repository, instead of yielding
an empty object.
Signed-off-by: Hidde Beydals <hidde@hhh.computer>
This adds a bit back which got removed in
69f567bdc7, as there are reasons for the
controller to perform a non-optimized clone.
However, we always want to attempt the optimized version first without
it being put behind a feature gate. Which was the original intent of
the referenced commit.
Signed-off-by: Hidde Beydals <hidde@hhh.computer>
Remove the `OptimizedGitClones` feature gate, making optimized Git
clones when using a branch or tag to checkout, the default behavior.
Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
This deals with various breaking changes in controller-runtime, as
documented in the release notes:
https://github.com/kubernetes-sigs/controller-runtime/releases/tag/v0.15.0
In short:
- `Watches` now use a `client.Object` instead of a `source.Kind`.
- `handler.MapFunc` signature accepts a Go context, which is used to
log any errors, instead of silently ignoring them and/or panicking.
- Fake clients used in tests are now configured using
`WithStatusSubresource` to enable the correct behavior for status
updates and patches.
- Max concurrent reconciles is configured on the manager, instead of
configuring them per reconciler instance.
- Various manager configuration options have been moved to new
structures and/or fields.
In addition to this, all other dependencies which had updates are
updated to their latest (compatible) versions as well.
Signed-off-by: Hidde Beydals <hidde@hhh.computer>
This commit properly sets `IgnoreTlog` to `true` when a public key is
provided to check the signature against, which matches the (silent)
default behavior from cosign v1.
However, during this exercise it has become apparant that this
assumption isn't necessarily true. As you can theoretically have a
custom key and a tlog entry.
Given this, we should inventarise the possible configuration options
and the potential value they have to users (e.g. defining a custom
Rekor URL seems to be valuable as well), and extend our API to
facilitate these needs.
In addition to the above, the CTLog public keys are now properly
retrieved to avoid a `none of the CTFE keys have been found` error.
Signed-off-by: Hidde Beydals <hidde@hhh.computer>
This commit ensures that files with exec permissions set continue to be
executable by the user extracting the archive.
This is not of use to any of Flux itself, but does help downstream
dependents making use of the controller to facilitate artifact
acquisitions for their (CI/CD) software suite.
Co-authored-by: Hidde Beydals <hidde@hhh.computer>
Signed-off-by: Rashed Kamal <krashed@vmware.com>
Given:
- None of the methods of the `Storage` are mutating the storage
itself.
- It must be instantiated to be usable, as there is a strict
reliance on values.
- The struct itself is light.
This seems to be more fitting.
Signed-off-by: Hidde Beydals <hidde@hhh.computer>
This commits adds verification of the digest of the artifact in storage
to all reconcilers which manage artifacts.
When the artifact does not have a digest or if it mismatches with the
file in storage, the file is removed from the storage and status of the
object.
This hardens the storage against potential tampering, in addition to
resolving an issue where users upgrading from a (much) older version of
the controller would run into an error after the checksum field was
removed from the API.
This would cause the controller to not advertise any checksum at all,
while not producing a new one until a new revision was detected.
Resulting in fetch failures for consumers while they would try to
verify the digest of the advertised artifact.
While not strictly part of this exercise, some of the tests were
altered to prepare the storage used in test cases to become isolated
by strictly using the `storage` provided via the callback. Actually
isolating this has however been left as a task at a later moment.
Signed-off-by: Hidde Beydals <hidde@hhh.computer>
Sunny
Sanskar Jaiswal
Stefan Prodan
Hidde Beydals
Ilya Dmitrichenko
Somtochi Onyekwere
Soule BA
Kevin McDermott
Rashed Kamal