This commit ensures that files with exec permissions set continue to be
executable by the user extracting the archive.
This is not of use to any of Flux itself, but does help downstream
dependents making use of the controller to facilitate artifact
acquisitions for their (CI/CD) software suite.
Co-authored-by: Hidde Beydals <hidde@hhh.computer>
Signed-off-by: Rashed Kamal <krashed@vmware.com>
Given:
- None of the methods of the `Storage` are mutating the storage
itself.
- It must be instantiated to be usable, as there is a strict
reliance on values.
- The struct itself is light.
This seems to be more fitting.
Signed-off-by: Hidde Beydals <hidde@hhh.computer>
This commits adds verification of the digest of the artifact in storage
to all reconcilers which manage artifacts.
When the artifact does not have a digest or if it mismatches with the
file in storage, the file is removed from the storage and status of the
object.
This hardens the storage against potential tampering, in addition to
resolving an issue where users upgrading from a (much) older version of
the controller would run into an error after the checksum field was
removed from the API.
This would cause the controller to not advertise any checksum at all,
while not producing a new one until a new revision was detected.
Resulting in fetch failures for consumers while they would try to
verify the digest of the advertised artifact.
While not strictly part of this exercise, some of the tests were
altered to prepare the storage used in test cases to become isolated
by strictly using the `storage` provided via the callback. Actually
isolating this has however been left as a task at a later moment.
Signed-off-by: Hidde Beydals <hidde@hhh.computer>
In #1001 bits around the Helm repository reconciliation logic were
rewritten, mostly based on the documented behavior instead of the
actual code. This resulted in the reintroduction of a YAML marshal of
the (sorted) index YAML instead of reliance of just the checksum of the
file.
This to take situations into account in which a repository would e.g.
provide a new random order on every generation. However, this approach
is (extremely) expensive as the marshal goes through a JSON -> YAML
loop, eating lots of RAM in the process.
As the further (silently) introduced behavior has not resulted in any
reported issues, I deem this approach safe and better than e.g.
encoding to just JSON which would still require a substantial amount of
memory.
Signed-off-by: Hidde Beydals <hello@hidde.co>
This includes changes to the `ChartRepository`, to allow calculating
the revision and digest and tidy things.
In addition, the responsibility of caching the `IndexFile` has been
moved to the reconcilers. As this allowed to remove a lot of
complexities within the `ChartRepository`, and prevented passing on
the cache in general.
Change `HelmRepository`'s Revision to digest
Signed-off-by: Hidde Beydals <hello@hidde.co>
This allows using the condition checker as a test helper with proper
test like assertion failure and stacktrace.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
Add chart address in the OCI chart download failure error message to make
it clear about the chart URL that was attempted to download.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
You can re-enable caching by starting the controller
with the argument '--feature-gates=CacheSecretsAndConfigMaps=true'
Signed-off-by: Mac Chaffee <machaffe@renci.org>
Replace the patch Helper with SerialPatcher which is used for
progressive status patching.
Update the tests to use progressive status reasons in tests.
Add ProgressingWithRetry Reconciling reason for failed
reconciliation result to indicate a finished failure operation.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
Update the implementations of the helm repository downloaders to return
implementation specific typed error from GetChartVersion(). This is
needed to distinguish between persistent build error and transient build
error.
In the case of OCI charts, a transient network failure shouldn't be
considered a persistent build failure of the chart and should be
retried.
Two repository errors, ErrReference and ErrExternal are introduced for
the repository downloader implementations to provide enough context
about the failure which can be used by the caller to add appropriate
context as per the needs. In case of chart builder, it adds the build
error context based on the repository error value.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
If implemented, this fix the issue were aliased chart dependencies were
detected but not included in the final packaged chart.
Signed-off-by: Soule BA <soule@weave.works>
ForceGoGitImplementation ignores the value set for gitImplementation
and ensures that go-git is used for all GitRepository objects.
This can be used to confirm that Flux instances won't break if/when
the libgit2 implementation was to be deprecated.
When enabled, libgit2 won't be initialized, nor will any git2go cgo
code be called.
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
Delete a failed verification condition at the beginning of the source
reconciliation and set `SourceVerifiedCondition` to false approprietly.
Set the `BuildOptions.Verify` to true as long as Verify is enabled in the
API fields.
Signed-off-by: Soule BA <soule@weave.works>
If implemented the oras registry loginOption will only be used internaly
with the specific ChartRepo struct.
This will permit reusing more easily feature developped with
googlecontainerregistry authn.
Signed-off-by: Soule BA <soule@weave.works>
Add setters and getters for spec.suspend and status.artifact.
This is needed for writing generic tests for any source kind.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
If implemented this enable passing a keychain, an authenticator and a
custom transport as remote.Option to the verifier. It enables contextual
login, self-signed certificates and insecure registries.
Signed-off-by: Soule BA <soule@weave.works>
refactor makeOptions
Reduce complexity by replacing the functional options with a flat out
conditional logic in makeOptions.
Signed-off-by: Soule BA <soule@weave.works>
- Update Go to 1.19 in CI
- Use Go 1.19 in base image
- Update controller-gen v0.8.0 and regenerate manifests
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
When a custom CA certificate is provided in a Secret's `caCert` field
referenced in `HelmRelease.spec.secretRef` then that CA cert is now
added to the list of system certificates instead of it replacing the
system certificates. This makes HelmRepositories work in mixed
environments where charts are pulled from both, a public repository
and a private repository (e.g. through a chart dependency).
The test that is added as part of this change will fail without the
change and passes with it.
closes#866closesfluxcd/helm-controller#519
Signed-off-by: Max Jonas Werner <max@e13.dev>
This introduces the consideration of bipolarity conditions in the status
condition summary for Ready condition. The summarize.HelperOptions can
now be configured with a list of bipolarity conditions which are used in
SummarizeAndPatch() to set the Ready condition to failing bipolarity
condition with the highest priority.
Bipolarity condition is not a typical status property. It is a mix of
positive and negative polarities. It's "normal-true" and
"abnormal-false". Failing bipolarity conditions are prioritized over
other conditions to show the actual reason of failure on the Ready
status.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
If implemented, this pr will enable user to use the auto login feature
in order to automatically login to their provider of choice's container
registry (i.e. aws, gcr, acr).
Signed-off-by: Soule BA <soule@weave.works>
This change allows for a HelmRepository to point to e.g. "ghcr.io" and
then a HelmRelease pointing to the chart "stefanprodan/charts/podinfo"
in its `.spec.chart.spec.chart` field.
Related discussion: https://github.com/fluxcd/flux2/discussions/2959
Signed-off-by: Max Jonas Werner <max@e13.dev>
Taking this shortcut has two benefits:
1. It allows charts to be fetched from AWS's public container registry
at public.ecr.aws
2. It makes reconciling a HelmChart faster by skipping one or more
potentially expensive API calls to the registry.
I adapted the unit tests to the new behavior that the
OCIChartRepository doesn't fail anymore for the case where a specific
chart version has been requested that doesn't actually exist in the
registry.
refs #845
Signed-off-by: Max Jonas Werner <max@e13.dev>
1. moved chartRepo.Unload() from reconcileSource() to the defer func in reconcileArtifact to allow caching index in memory
2. added step to init memory cache in reconcileArtifact()
3. added step to save helmrepo index into memory cache in reconcileArtifact()
Signed-off-by: York Chen <ychen@d2iq.com>
Decommission libgit2 unmanaged transport and remove the related feature
gate, making managed transport the default.
Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
If implemented, this make sure than we clear only referenced
downloaders.
It is also checked if the repository url is supported.
Signed-off-by: Soule BA <soule@weave.works>
If implemented, the source controller will be able to resolve charts
dependencies from OCI repositories.
The remote builder has been refactored as part of this work.
Signed-off-by: Soule BA <soule@weave.works>
Stefan Prodan
Rashed Kamal
Hidde Beydals
Sunny
Hidde Beydals
Somtochi Onyekwere
Mac Chaffee
Soule BA
Paulo Gomes
Max Jonas Werner
Batuhan Apaydın
Furkan
York Chen
Sanskar Jaiswal