In #1001 bits around the Helm repository reconciliation logic were
rewritten, mostly based on the documented behavior instead of the
actual code. This resulted in the reintroduction of a YAML marshal of
the (sorted) index YAML instead of reliance of just the checksum of the
file.
This to take situations into account in which a repository would e.g.
provide a new random order on every generation. However, this approach
is (extremely) expensive as the marshal goes through a JSON -> YAML
loop, eating lots of RAM in the process.
As the further (silently) introduced behavior has not resulted in any
reported issues, I deem this approach safe and better than e.g.
encoding to just JSON which would still require a substantial amount of
memory.
Signed-off-by: Hidde Beydals <hello@hidde.co>
This includes changes to the `ChartRepository`, to allow calculating
the revision and digest and tidy things.
In addition, the responsibility of caching the `IndexFile` has been
moved to the reconcilers. As this allowed to remove a lot of
complexities within the `ChartRepository`, and prevented passing on
the cache in general.
Change `HelmRepository`'s Revision to digest
Signed-off-by: Hidde Beydals <hello@hidde.co>
This allows using the condition checker as a test helper with proper
test like assertion failure and stacktrace.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
Add chart address in the OCI chart download failure error message to make
it clear about the chart URL that was attempted to download.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
You can re-enable caching by starting the controller
with the argument '--feature-gates=CacheSecretsAndConfigMaps=true'
Signed-off-by: Mac Chaffee <machaffe@renci.org>
Replace the patch Helper with SerialPatcher which is used for
progressive status patching.
Update the tests to use progressive status reasons in tests.
Add ProgressingWithRetry Reconciling reason for failed
reconciliation result to indicate a finished failure operation.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
Update the implementations of the helm repository downloaders to return
implementation specific typed error from GetChartVersion(). This is
needed to distinguish between persistent build error and transient build
error.
In the case of OCI charts, a transient network failure shouldn't be
considered a persistent build failure of the chart and should be
retried.
Two repository errors, ErrReference and ErrExternal are introduced for
the repository downloader implementations to provide enough context
about the failure which can be used by the caller to add appropriate
context as per the needs. In case of chart builder, it adds the build
error context based on the repository error value.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
If implemented, this fix the issue were aliased chart dependencies were
detected but not included in the final packaged chart.
Signed-off-by: Soule BA <soule@weave.works>
ForceGoGitImplementation ignores the value set for gitImplementation
and ensures that go-git is used for all GitRepository objects.
This can be used to confirm that Flux instances won't break if/when
the libgit2 implementation was to be deprecated.
When enabled, libgit2 won't be initialized, nor will any git2go cgo
code be called.
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
Delete a failed verification condition at the beginning of the source
reconciliation and set `SourceVerifiedCondition` to false approprietly.
Set the `BuildOptions.Verify` to true as long as Verify is enabled in the
API fields.
Signed-off-by: Soule BA <soule@weave.works>
If implemented the oras registry loginOption will only be used internaly
with the specific ChartRepo struct.
This will permit reusing more easily feature developped with
googlecontainerregistry authn.
Signed-off-by: Soule BA <soule@weave.works>
Add setters and getters for spec.suspend and status.artifact.
This is needed for writing generic tests for any source kind.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
If implemented this enable passing a keychain, an authenticator and a
custom transport as remote.Option to the verifier. It enables contextual
login, self-signed certificates and insecure registries.
Signed-off-by: Soule BA <soule@weave.works>
refactor makeOptions
Reduce complexity by replacing the functional options with a flat out
conditional logic in makeOptions.
Signed-off-by: Soule BA <soule@weave.works>
- Update Go to 1.19 in CI
- Use Go 1.19 in base image
- Update controller-gen v0.8.0 and regenerate manifests
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
When a custom CA certificate is provided in a Secret's `caCert` field
referenced in `HelmRelease.spec.secretRef` then that CA cert is now
added to the list of system certificates instead of it replacing the
system certificates. This makes HelmRepositories work in mixed
environments where charts are pulled from both, a public repository
and a private repository (e.g. through a chart dependency).
The test that is added as part of this change will fail without the
change and passes with it.
closes#866closesfluxcd/helm-controller#519
Signed-off-by: Max Jonas Werner <max@e13.dev>
This introduces the consideration of bipolarity conditions in the status
condition summary for Ready condition. The summarize.HelperOptions can
now be configured with a list of bipolarity conditions which are used in
SummarizeAndPatch() to set the Ready condition to failing bipolarity
condition with the highest priority.
Bipolarity condition is not a typical status property. It is a mix of
positive and negative polarities. It's "normal-true" and
"abnormal-false". Failing bipolarity conditions are prioritized over
other conditions to show the actual reason of failure on the Ready
status.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
If implemented, this pr will enable user to use the auto login feature
in order to automatically login to their provider of choice's container
registry (i.e. aws, gcr, acr).
Signed-off-by: Soule BA <soule@weave.works>
This change allows for a HelmRepository to point to e.g. "ghcr.io" and
then a HelmRelease pointing to the chart "stefanprodan/charts/podinfo"
in its `.spec.chart.spec.chart` field.
Related discussion: https://github.com/fluxcd/flux2/discussions/2959
Signed-off-by: Max Jonas Werner <max@e13.dev>
Taking this shortcut has two benefits:
1. It allows charts to be fetched from AWS's public container registry
at public.ecr.aws
2. It makes reconciling a HelmChart faster by skipping one or more
potentially expensive API calls to the registry.
I adapted the unit tests to the new behavior that the
OCIChartRepository doesn't fail anymore for the case where a specific
chart version has been requested that doesn't actually exist in the
registry.
refs #845
Signed-off-by: Max Jonas Werner <max@e13.dev>
1. moved chartRepo.Unload() from reconcileSource() to the defer func in reconcileArtifact to allow caching index in memory
2. added step to init memory cache in reconcileArtifact()
3. added step to save helmrepo index into memory cache in reconcileArtifact()
Signed-off-by: York Chen <ychen@d2iq.com>
Decommission libgit2 unmanaged transport and remove the related feature
gate, making managed transport the default.
Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
If implemented, this make sure than we clear only referenced
downloaders.
It is also checked if the repository url is supported.
Signed-off-by: Soule BA <soule@weave.works>
If implemented, the source controller will be able to resolve charts
dependencies from OCI repositories.
The remote builder has been refactored as part of this work.
Signed-off-by: Soule BA <soule@weave.works>
Since the transport reuse is dependent on the garbage collection, the
result is inconsistent. It fails frequently when running the tests with
the go race detector. Remove the test.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
If implemented this fix the issue where we previously did a string
ordering of matching semver versions when retrieving a list of tags from
an OCI registry.
Signed-off-by: Soule BA <soule@weave.works>
Registry login option should verify that the obtained dockerconfig
credentials are for the same host. When the helmrepo URL and the URL in
docker auth config don't match, the docker config store returns an
empty auth config, instead of failing. This results in accepting empty
username and password. The HelmRepo would appear to be ready in such
situation because the creds are empty, no login is attempted. But when
a HelmChart tries to use the login options, it'd fail.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
The observed generation must be set only when actual observation is
made. When an actual observation is made, some conditions are set on the
object. Introduce a helper function
addPatchOptionWithStatusObservedGeneration() to set the patcher option
WithStatusObservedGeneration only when there's any condition in the
status.
Updates the existing tests that depended on this behavior.
This fixes the issue where the observed generation is set by the patcher
when a reconciler does an early return for setting the finalizers only.
With this, the observed generation will be updated only when some
observations are made on the object based on the usual rules of success
result, no error, ignore error and stalled condition.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
Hidde Beydals
Somtochi Onyekwere
Sunny
Mac Chaffee
Soule BA
Paulo Gomes
Stefan Prodan
Max Jonas Werner
Batuhan Apaydın
Furkan
York Chen
Sanskar Jaiswal
Max Jonas Werner