- cloud.google.com/go/storage to v1.28.0
- github.com/distribution/distribution/v3 to v3.0.0-20221108081720-e9a25da7a47e
- github.com/docker/cli to v20.10.21+incompatible
- github.com/google/go-containerregistry to v0.12.0
- github.com/google/go-containerregistry/pkg/authn/k8schain to v0.0.0-20221103173901-353a117661ae
- github.com/minio/minio-go/v7 to v7.0.43
- github.com/onsi/gomega to v1.24.0
- github.com/prometheus/client_golang to v1.14.0
- github.com/sigstore/sigstore to v1.4.5
- google.golang.org/api to v0.102.0
- k8s.io/utils to v0.0.0-20221107191617-1a15be271d1d
- sigs.k8s.io/cli-utils to v0.34.0
- sigs.k8s.io/controller-runtime to v0.13.1
Signed-off-by: Hidde Beydals <hello@hidde.co>
Delete a failed verification condition at the beginning of the source
reconciliation and set `SourceVerifiedCondition` to false approprietly.
Set the `BuildOptions.Verify` to true as long as Verify is enabled in the
API fields.
Signed-off-by: Soule BA <soule@weave.works>
This remove test case for contextual login on
oci://123456789000.dkr.ecr.us-east-2.amazonaws.com.
This is not longer a wrong url since f7c66eb06a
and we no longer error on nil auth.
Signed-off-by: Soule BA <soule@weave.works>
If implemented the oras registry loginOption will only be used internaly
with the specific ChartRepo struct.
This will permit reusing more easily feature developped with
googlecontainerregistry authn.
Signed-off-by: Soule BA <soule@weave.works>
Reorders the object suspended check in all the reconcilers to allow
deletion of objects when they are suspended. Objects used to get stuck
on delete because the finalizers were not getting removed due to the
suspended state.
Adds a generic test for all the reconcilers to check if a suspended
source object can be delete.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
Replace content config checksum with explicit artifact content config
observations. It makes the observations of the controller more
transparent and easier to debug.
Introduces `observedIgnore`, `observedRecurseSubmodules` and
`observedInclude` status fields.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
Replace content config checksum with explicit artifact content config
observations. It makes the observations of the controller more
transparent and easier to debug.
Introduces `observedIgnore` and `observedLayerSelector` status fields.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
If implemented we fails when trying to verify with insecure set. This
will likely change once cosign add support for insecure registries.
Signed-off-by: Soule BA <soule@weave.works>
If implemented this enable passing a keychain, an authenticator and a
custom transport as remote.Option to the verifier. It enables contextual
login, self-signed certificates and insecure registries.
Signed-off-by: Soule BA <soule@weave.works>
refactor makeOptions
Reduce complexity by replacing the functional options with a flat out
conditional logic in makeOptions.
Signed-off-by: Soule BA <soule@weave.works>
This implements source ignore in OCIRepositoryReconcilers'
reconcileArtifact so that the ignore rules are considered when building
the artifact.
Adds tests based on the artifact checksum change when ignore rules are
applied.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
Introduce contentConfigChecksum in the OCIRepository status to store a
checksum of the values that affect the source artifact. It is used to
detect when to rebuild an artifact when the spec changes.
The considerations for this are similar to the GitRepository
reconciler no-op clone implementation. Both reconcileSource and
reconcileArtifact need to consider the source configuration change
when deciding if the artifact in the storage is up-to-date.
Adds tests for reconcileSource and reconcileArtifact for the noop
cases.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
- Fetch the upstream digest before validation and pulling
- Pull artifact only if the upstream digest is different from the one in storage
- Add the image tag to the revision string `<tag>/<digest-hex>` for a better UX
- Extract the layer processing to a dedicated function
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
Add on optional field to the `OCIRepository.spec.layerSelector` called `operation` that accepts one of the following values: `extract` or `copy`. When the operation is set to `copy`, instead of extracting the compressed layer, the controller copies the compressed blob as it is to storage, thus keeping the original content unaltered.
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
When a custom CA certificate is provided in a Secret's `caCert` field
referenced in `HelmRelease.spec.secretRef` then that CA cert is now
added to the list of system certificates instead of it replacing the
system certificates. This makes HelmRepositories work in mixed
environments where charts are pulled from both, a public repository
and a private repository (e.g. through a chart dependency).
The test that is added as part of this change will fail without the
change and passes with it.
closes#866closesfluxcd/helm-controller#519
Signed-off-by: Max Jonas Werner <max@e13.dev>
Context in the reconcilers were overwritten earlier after adding new
log field `reconcileID` in the logger. Since the `reconcileID` is now
set by controller-runtime, this is no longer needed. The logger in the
context already has the field set and when the context is passed to
other functions, they too have the logger with the reconcileID set.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
With the new controller-runtime, the reconcileID is automatically set
per reconciliation and need not be set explicitly.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
If implemented, this pr will enable user to use the auto login feature
in order to automatically login to their provider of choice's container
registry (i.e. aws, gcr, acr).
Signed-off-by: Soule BA <soule@weave.works>
Use of MUSL was a temporary solution to mitigate cross-platform
issues while building openssl and libssh2. Since Unmanaged transport has
been deprecated, openssl and libssh2 dependencies are no longer required
and by extension MUSL.
Enables libgit2 threadless support and provides a regression assurance
for fluxcd/image-automation-controller#339.
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
If implemented this merges
TestHelmChartReconciler_reconcileFromHelmRepository and
TestHelmChartReconciler_Reconcile
Signed-off-by: Soule BA <soule@weave.works>
1. moved chartRepo.Unload() from reconcileSource() to the defer func in reconcileArtifact to allow caching index in memory
2. added step to init memory cache in reconcileArtifact()
3. added step to save helmrepo index into memory cache in reconcileArtifact()
Signed-off-by: York Chen <ychen@d2iq.com>
Decommission libgit2 unmanaged transport and remove the related feature
gate, making managed transport the default.
Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
If implemented, the helmrepository type will be used to decide whether a
reconciliation can continue in the absence of source artifact, instead
of url.
Signed-off-by: Soule BA <soule@weave.works>
If implemented, the source controller will be able to resolve charts
dependencies from OCI repositories.
The remote builder has been refactored as part of this work.
Signed-off-by: Soule BA <soule@weave.works>
- Update the comments around artifact retention fields in Storage.
- Update the comments around reconcileStorage regarding artifact
retention and garbage collection.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
GitRepository introduced correlation ID to improve
transport level logging. This change aligns the other
reconcilers to the same approach.
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
Debugging connection issues can be extremely difficult, even more so at scale or when
concurrent connections are required to trigger specific issues.
Changes:
- Add a correlation identifier for each reconciliation, which allows for greater traceability when
going through all the reconciliation operations - including at transport level.
- Add transportType to segregate HTTP and SSH transport logging.
- SSH operations are now enriched with addr containing server address, and HTTP url.
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
Refactors libgit2 checkout tests to test managed and unmanaged
transport by making sure the tests requiring unmanaged transport are run
before, any tests that require managed transport (since disabling
managed transport isn't possible). This is done via arranging the tests
carefully in alphabetically sorted names, i.e. the tests with unmanaged
transport go in `checkout_test.go`, which forces golang to run the tests
in that file before any other tests.
Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
Remove stale condition from HelmRepo during garbage collection when a
type switch to OCI HelmRepo occurs. This ensures the OCI HelmRepo does
not have any conditions from the previous type.
Co-authored-by: Soule BA <soule@weave.works>
Signed-off-by: Sunny <darkowlzz@protonmail.com>
Refactor the OCI HelmRepo reconciler to remove extra custom status
conditions and manage Ready, Reconciling and Stalled conditions within
the reconciler, in-line.
The internal/reconcile/summarize package uses the patch helper
conditions summary before patching which results in overwriting the
Ready condition with Reconciling condition as it's a negative polarity
condition.
For OCI HelmRepository, since it's a simple reconciler, it'd be better
to not introduce extra conditions and utilize the three base conditions
to represent the status. In order to have the same consistent status
conditions, a new summarization is written within the reconciler based
on the context. It takes into consideration a lot of the details from
the internal/reconcile/summarize package and handles certain scenarios
in context specific ways. All the result and error abstractions are
removed since they are only needed when using internal/reconcile
package.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
Some scenarios could lead a goroutine to be running indefinetely within managed ssh.
Previously between the two git operations, the reconciliation
could take twice the timeout set for the Flux object.
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
Injects transport and auth options at the transport level directly to
bypass the inbuilt credentials callback because of it's several
shortcomings. Moves some of the pre-existing logic from the reconciler
to the checkout implementation.
Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
gitCheckout() should return typed errors only. This helps prevent
error type assertions by the caller to determine how to handle the
error. gitCheckout() also sets the appropriate conditions associated
with the error, if any.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
Some test cases rely on checksum to match in order to pass.
Those checksums were calculated based on file headers which
contain their file modes.
In Ubuntu, the umask is set to 002 by default, resulting in
the tests files having different permissions then when the
same files are cloned on another Linux machine with umask
set to 022.
This change ensures that the files are always set (to 0644 and
the directories to 0755) before running the aforementioned tests.
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
In TestHelmRepositoryReconciler_ReconcileTypeUpdatePredicateFilter, when
the type of HelmRepo is updated and immediately checked for the object
to be ready, if the check happens before the client cache is updated, it
results in observing the object to be ready in the previous generation.
This results in status check failure:
```
[Check-FAIL]: [Ready condition must be False when the ObservedGeneration is less than the object Generation, Ready condition must be False when any of the status condition's ObservedGeneration is less than the object Generation: [Ready ArtifactInStorage]]
```
Explicitly look for the object with the next generation to prevent such
failure.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
This new package holds all Helm OCI registry-specific code now so we
have a single location to look for such code which makes it easier to
find yourself around.
Signed-off-by: Max Jonas Werner <mail@makk.es>
When setup of one of the two controller reconciling HelmRepositories
fails, it's now possible to judge from the log which setup call failed
by regarding the "type" log field.
Signed-off-by: Max Jonas Werner <mail@makk.es>
Hidde Beydals
Paulo Gomes
Sanskar Jaiswal
Soule BA
Sunny
Stefan Prodan
Max Jonas Werner
Furkan
Batuhan Apaydın
Adrien Fillon
Somtochi Onyekwere
Santosh Kaluskar
Rashed Kamal
York Chen
Max Jonas Werner