If implemented user will be able to provide their own custom start and
bypass tls verification when interacting with OCI registries over https
to pull helmCharts.
Signed-off-by: Soule BA <soule@weave.works>
In Reconcile() methods, move the object deletion above add finalizer.
Finalizers can't be set when an object is being deleted.
Introduce a cacheless client in suite_test to use for testing this
change. It ensures that the Reconcile() call always operates on the
latest version of the object which has the deletion timestamp and
existing finalizer.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
Add tests to test Cosign support for insecure registries. Furthermore,
refactor OCI test utils to be more user friendly and enable accurate
testing of HTTPS and HTTP OCI registries by circumnavigating Docker's
automatic connection downgrade for registries hosted on localhost.
Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
Add support for verifying insecure HTTP OCI repositories with cosign. If
`.spec.insecure` set to true, then cosign uses plain HTTP connections to
communicate with the registry.
Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
Add support for specifying TLS auth data via `.spec.certSecretRef` in
HelmRepository and log a deprecation warning if TLS is configured via
`.spec.secretRef`. Introduce (and refactor) Helm client builder and
auth helpers to reduce duplicated code and increase uniformity and
testability.
Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
Add `.spec.proxySecretRef.name` to the `GitRepository` API to allow
referencing a secret containing the proxy settings to be used for all
remote Git operations for the particular `GitRepository` object.
It takes precedence over any proxy configured through enviornment
variables.
Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
This is required because the test fails with Git >=v2.41.0 due to
changes to commands used by the Git test server. Causing the server to
return an error when cloning an empty repository, instead of yielding
an empty object.
Signed-off-by: Hidde Beydals <hidde@hhh.computer>
This adds a bit back which got removed in
69f567bdc7, as there are reasons for the
controller to perform a non-optimized clone.
However, we always want to attempt the optimized version first without
it being put behind a feature gate. Which was the original intent of
the referenced commit.
Signed-off-by: Hidde Beydals <hidde@hhh.computer>
Remove the `OptimizedGitClones` feature gate, making optimized Git
clones when using a branch or tag to checkout, the default behavior.
Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
This deals with various breaking changes in controller-runtime, as
documented in the release notes:
https://github.com/kubernetes-sigs/controller-runtime/releases/tag/v0.15.0
In short:
- `Watches` now use a `client.Object` instead of a `source.Kind`.
- `handler.MapFunc` signature accepts a Go context, which is used to
log any errors, instead of silently ignoring them and/or panicking.
- Fake clients used in tests are now configured using
`WithStatusSubresource` to enable the correct behavior for status
updates and patches.
- Max concurrent reconciles is configured on the manager, instead of
configuring them per reconciler instance.
- Various manager configuration options have been moved to new
structures and/or fields.
In addition to this, all other dependencies which had updates are
updated to their latest (compatible) versions as well.
Signed-off-by: Hidde Beydals <hidde@hhh.computer>
This commit properly sets `IgnoreTlog` to `true` when a public key is
provided to check the signature against, which matches the (silent)
default behavior from cosign v1.
However, during this exercise it has become apparant that this
assumption isn't necessarily true. As you can theoretically have a
custom key and a tlog entry.
Given this, we should inventarise the possible configuration options
and the potential value they have to users (e.g. defining a custom
Rekor URL seems to be valuable as well), and extend our API to
facilitate these needs.
In addition to the above, the CTLog public keys are now properly
retrieved to avoid a `none of the CTFE keys have been found` error.
Signed-off-by: Hidde Beydals <hidde@hhh.computer>
This commit ensures that files with exec permissions set continue to be
executable by the user extracting the archive.
This is not of use to any of Flux itself, but does help downstream
dependents making use of the controller to facilitate artifact
acquisitions for their (CI/CD) software suite.
Co-authored-by: Hidde Beydals <hidde@hhh.computer>
Signed-off-by: Rashed Kamal <krashed@vmware.com>
Given:
- None of the methods of the `Storage` are mutating the storage
itself.
- It must be instantiated to be usable, as there is a strict
reliance on values.
- The struct itself is light.
This seems to be more fitting.
Signed-off-by: Hidde Beydals <hidde@hhh.computer>
This commits adds verification of the digest of the artifact in storage
to all reconcilers which manage artifacts.
When the artifact does not have a digest or if it mismatches with the
file in storage, the file is removed from the storage and status of the
object.
This hardens the storage against potential tampering, in addition to
resolving an issue where users upgrading from a (much) older version of
the controller would run into an error after the checksum field was
removed from the API.
This would cause the controller to not advertise any checksum at all,
while not producing a new one until a new revision was detected.
Resulting in fetch failures for consumers while they would try to
verify the digest of the advertised artifact.
While not strictly part of this exercise, some of the tests were
altered to prepare the storage used in test cases to become isolated
by strictly using the `storage` provided via the callback. Actually
isolating this has however been left as a task at a later moment.
Signed-off-by: Hidde Beydals <hidde@hhh.computer>
In #1001 bits around the Helm repository reconciliation logic were
rewritten, mostly based on the documented behavior instead of the
actual code. This resulted in the reintroduction of a YAML marshal of
the (sorted) index YAML instead of reliance of just the checksum of the
file.
This to take situations into account in which a repository would e.g.
provide a new random order on every generation. However, this approach
is (extremely) expensive as the marshal goes through a JSON -> YAML
loop, eating lots of RAM in the process.
As the further (silently) introduced behavior has not resulted in any
reported issues, I deem this approach safe and better than e.g.
encoding to just JSON which would still require a substantial amount of
memory.
Signed-off-by: Hidde Beydals <hello@hidde.co>
This includes changes to the `ChartRepository`, to allow calculating
the revision and digest and tidy things.
In addition, the responsibility of caching the `IndexFile` has been
moved to the reconcilers. As this allowed to remove a lot of
complexities within the `ChartRepository`, and prevented passing on
the cache in general.
Change `HelmRepository`'s Revision to digest
Signed-off-by: Hidde Beydals <hello@hidde.co>
This allows using the condition checker as a test helper with proper
test like assertion failure and stacktrace.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
Add chart address in the OCI chart download failure error message to make
it clear about the chart URL that was attempted to download.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
You can re-enable caching by starting the controller
with the argument '--feature-gates=CacheSecretsAndConfigMaps=true'
Signed-off-by: Mac Chaffee <machaffe@renci.org>
Replace the patch Helper with SerialPatcher which is used for
progressive status patching.
Update the tests to use progressive status reasons in tests.
Add ProgressingWithRetry Reconciling reason for failed
reconciliation result to indicate a finished failure operation.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
Update the implementations of the helm repository downloaders to return
implementation specific typed error from GetChartVersion(). This is
needed to distinguish between persistent build error and transient build
error.
In the case of OCI charts, a transient network failure shouldn't be
considered a persistent build failure of the chart and should be
retried.
Two repository errors, ErrReference and ErrExternal are introduced for
the repository downloader implementations to provide enough context
about the failure which can be used by the caller to add appropriate
context as per the needs. In case of chart builder, it adds the build
error context based on the repository error value.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
If implemented, this fix the issue were aliased chart dependencies were
detected but not included in the final packaged chart.
Signed-off-by: Soule BA <soule@weave.works>
ForceGoGitImplementation ignores the value set for gitImplementation
and ensures that go-git is used for all GitRepository objects.
This can be used to confirm that Flux instances won't break if/when
the libgit2 implementation was to be deprecated.
When enabled, libgit2 won't be initialized, nor will any git2go cgo
code be called.
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
Delete a failed verification condition at the beginning of the source
reconciliation and set `SourceVerifiedCondition` to false approprietly.
Set the `BuildOptions.Verify` to true as long as Verify is enabled in the
API fields.
Signed-off-by: Soule BA <soule@weave.works>
If implemented the oras registry loginOption will only be used internaly
with the specific ChartRepo struct.
This will permit reusing more easily feature developped with
googlecontainerregistry authn.
Signed-off-by: Soule BA <soule@weave.works>
Somtochi Onyekwere
Soule BA
Hidde Beydals
Sunny
Sanskar Jaiswal
Kevin McDermott
Stefan Prodan
Rashed Kamal
Hidde Beydals
Mac Chaffee
Paulo Gomes