Commit Graph

412 Commits

Author SHA1 Message Date
Furkan 697f260dba
Introduce Initial OCIRepository Source Verification
Fixes #863

Signed-off-by: Furkan <furkan.turkal@trendyol.com>
Co-authored-by: Batuhan <batuhan.apaydin@trendyol.com>
Signed-off-by: Batuhan Apaydın <batuhan.apaydin@trendyol.com>
2022-09-20 14:07:10 +03:00
Adrien Fillon 03ba63bec3
Handle nil OCI authenticator with malformed registry
Fixes #896

Signed-off-by: Adrien Fillon <adrien.fillon@manomano.com>
2022-09-13 19:53:41 +02:00
Soule BA 869c73d0ad
secretRef take precedence over provider
if secretRef is provided, we do not attempt to resolve oidc

Signed-off-by: Soule BA <soule@weave.works>
2022-09-09 14:43:35 +02:00
Sunny 59294bf582
controllers: Remove ctx overwrite
Context in the reconcilers were overwritten earlier after adding new
log field `reconcileID` in the logger. Since the `reconcileID` is now
set by controller-runtime, this is no longer needed. The logger in the
context already has the field set and when the context is passed to
other functions, they too have the logger with the reconcileID set.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-09-04 18:56:02 +05:30
Sunny 658134fe88
Remove setting reconcileID in helmrepo-oci logger
With the new controller-runtime, the reconcileID is automatically set
per reconciliation and need not be set explicitly.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-09-04 17:15:06 +05:30
Somtochi Onyekwere c38fafe128 Align controller logs to Kubernetes structured logging
Signed-off-by: Somtochi Onyekwere <somtochionyekwere@gmail.com>
2022-08-31 14:24:40 +01:00
Stefan Prodan e1ad5a6fd3
Add `spec.insecure` to OCIRepository API
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-08-31 11:10:25 +03:00
Stefan Prodan 181b2177fe
Add support for plain HTTP OCIRepositories
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-08-31 10:08:29 +03:00
Soule BA ad3eb5ca47
Enable contextual login for helm OCI
If implemented, this pr will enable user to use the auto login feature
in order to automatically login to their provider of choice's container
registry (i.e. aws, gcr, acr).

Signed-off-by: Soule BA <soule@weave.works>
2022-08-25 22:27:35 +02:00
Stefan Prodan 49dc30922d
Add tests for OCI layer selector
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-08-24 12:27:30 +03:00
Stefan Prodan 11dc0a3bc7
Select layer by OCI media type
Allow specifying the media type of the layer which should be extracted from the OCI artifact.

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-08-23 12:25:18 +03:00
Stefan Prodan f873d71ec5
Use sourceignore from fluxcd/pkg
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-08-17 18:04:48 +03:00
Santosh Kaluskar 1ad2f004ac Implementing RecoverPanic on reconcilers to ensure it recovers from panic instead of crashing the controller and Squashed commits.
Signed-off-by: Santosh Kaluskar <dtshbl@gmail.com>
2022-08-11 18:35:25 +05:30
Stefan Prodan 94e98ee5ca
Add the opencontainers annotations to API docs
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-08-08 12:58:04 +03:00
Stefan Prodan 1a59935858
Add OCI failure reasons to API
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-08-05 13:24:06 +03:00
Stefan Prodan c52576c151
Mark resource as stalled on invalid URL
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-08-04 16:52:55 +03:00
Stefan Prodan 63c94397f7
Implement OCI auth for cloud providers
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-08-04 13:29:03 +03:00
Stefan Prodan acc95d8c50
Add upstream source and revision to logs and events
Enrich the successful reconciliation event message with the upstream opencontainers annotations

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-08-02 13:11:15 +03:00
Somtochi Onyekwere 25b88256ef
Add tests for reconcile delete
Signed-off-by: Somtochi Onyekwere <somtochionyekwere@gmail.com>
2022-08-02 13:11:08 +03:00
Somtochi Onyekwere eb40efea1c
reconcile artifact
Signed-off-by: Somtochi Onyekwere <somtochionyekwere@gmail.com>
2022-08-02 13:07:08 +03:00
Somtochi Onyekwere 648beef063
Add test for reconcileArtifact
Signed-off-by: Somtochi Onyekwere <somtochionyekwere@gmail.com>
2022-08-02 13:07:08 +03:00
Somtochi Onyekwere e42e9d086c
Add tests for getArtifactURL
Signed-off-by: Somtochi Onyekwere <somtochionyekwere@gmail.com>
2022-08-02 13:07:08 +03:00
Somtochi Onyekwere b072d78874
Add tests for oci controller
Signed-off-by: Somtochi Onyekwere <somtochionyekwere@gmail.com>
2022-08-02 13:07:07 +03:00
Stefan Prodan 05f9c0ee2b
Add the OCI metadata to the internal artifact
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-08-02 13:07:07 +03:00
Stefan Prodan 4b0729203b
Add OCIRepository API spec to docs
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-08-02 13:07:07 +03:00
Stefan Prodan 9a6ff19487
Normalise error messages
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-08-02 13:07:07 +03:00
Rashed Kamal 942d92834b
OCIRepository client cert auth
Signed-off-by: Rashed Kamal <krashed@vmware.com>
2022-08-02 13:07:07 +03:00
Stefan Prodan 4506acb9d6
Use the internal pkg to handle errors
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-08-02 13:07:06 +03:00
Stefan Prodan ded0c2d78b
Add `oci://` prefix
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-08-02 13:07:06 +03:00
rashedkvm c9f5af7ddc
Implements basic auth with static credentials OCIRepository
Signed-off-by: rashedkvm <krashed@vmware.com>
2022-08-02 13:07:06 +03:00
Stefan Prodan 768adc2dd9
Implement OCIRepository ref.semver
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-08-02 13:07:06 +03:00
Stefan Prodan 07466730c0
Implement OCIRepository controller for public repos
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-08-02 13:07:06 +03:00
Paulo Gomes f51f9d3305
Remove MUSL and enable threadless libgit2 support
Use of MUSL was a temporary solution to mitigate cross-platform
issues while building openssl and libssh2. Since Unmanaged transport has
been deprecated, openssl and libssh2 dependencies are no longer required
and by extension MUSL.

Enables libgit2 threadless support and provides a regression assurance
for fluxcd/image-automation-controller#339.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-07-31 18:38:27 +01:00
Soule BA fba6477bc3
Merge HelmChart Reconcile tests
If implemented this merges
TestHelmChartReconciler_reconcileFromHelmRepository and
TestHelmChartReconciler_Reconcile

Signed-off-by: Soule BA <soule@weave.works>
2022-07-27 14:06:23 +02:00
York Chen d5a75f6b2f feat: cache helmrepo early after reconcile
1. moved chartRepo.Unload() from reconcileSource() to the defer func in reconcileArtifact to allow caching index in memory
2. added step to init memory cache in reconcileArtifact()
3. added step to save helmrepo index into memory cache in reconcileArtifact()

Signed-off-by: York Chen <ychen@d2iq.com>
2022-07-21 18:17:26 +01:00
Sanskar Jaiswal 0978a7ab61 gitrepo: update reconciler to be injected with transport initialization knowledge
Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
2022-07-20 18:56:37 +05:30
Sanskar Jaiswal f5ada743d5 libgit2: decommission unmanaged transport
Decommission libgit2 unmanaged transport and remove the related feature
gate, making managed transport the default.

Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
2022-07-20 18:56:37 +05:30
Paulo Gomes 9c21f8a7a1
tests: fix error message for invalid x509 in darwin
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-07-14 19:02:45 +01:00
Soule BA f3ab2e0d46
Fix Panic when no artifact in source
If implemented, the helmrepository type will be used to decide whether a
reconciliation can continue in the absence of source artifact, instead
of url.

Signed-off-by: Soule BA <soule@weave.works>
2022-07-14 10:57:22 +02:00
Paulo Gomes 60e46d139c
Decrease fs perms to 0o700
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-07-07 09:57:51 +01:00
Soule BA 361b975bf4
Enable remote dependencies from OCI repositories
If implemented, the source controller will be able to resolve charts
dependencies from OCI repositories.

The remote builder has been refactored as part of this work.

Signed-off-by: Soule BA <soule@weave.works>
2022-07-06 19:11:01 +02:00
Soule BA b402e546bc
Refactor repository logic
Signed-off-by: Soule BA <soule@weave.works>
2022-07-06 19:11:00 +02:00
Sunny e345e71eca
Minor comment updates
- Update the comments around artifact retention fields in Storage.
- Update the comments around reconcileStorage regarding artifact
  retention and garbage collection.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-07-04 19:58:26 +05:30
Paulo Gomes f1799dcb6b
git: fix reconcileSource_authStrategy
Co-authored-by: Sunny <darkowlzz@protonmail.com>
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-06-14 09:25:54 +01:00
Paulo Gomes 42dcb87345
Add reconcileID to all reconcilers
GitRepository introduced correlation ID to improve
transport level logging. This change aligns the other
reconcilers to the same approach.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-06-14 08:59:44 +01:00
Paulo Gomes cea9ea9142
libgit2: improve subtransport logging
Debugging connection issues can be extremely difficult, even more so at scale or when
concurrent connections are required to trigger specific issues.

Changes:
- Add a correlation identifier for each reconciliation, which allows for greater traceability when
going through all the reconciliation operations - including at transport level.
- Add transportType to segregate HTTP and SSH transport logging.
- SSH operations are now enriched with addr containing server address, and HTTP url.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-06-13 07:35:12 +01:00
Sanskar Jaiswal 4ce31319ee libgit2: refactor tests to use managed and unmanaged transport cleanly
Refactors libgit2 checkout tests to test managed and unmanaged
transport by making sure the tests requiring unmanaged transport are run
before, any tests that require managed transport (since disabling
managed transport isn't possible). This is done via arranging the tests
carefully in alphabetically sorted names, i.e. the tests with unmanaged
transport go in `checkout_test.go`, which forces golang to run the tests
in that file before any other tests.

Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
2022-06-09 21:01:17 +05:30
Soule BA 0d4d78fc6b
Fix repository cache regression
If implemented this make sure we don't unload an index before caching it
during a chart build phase.

Signed-off-by: Soule BA <soule@weave.works>
2022-06-06 13:15:10 +02:00
Stefan Prodan 2441f1f0e9
Log on new artifact and failure recovery
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-06-03 15:58:19 +03:00
Sunny 9fe287d912
helmrepo: rm stale condition when type switching
Remove stale condition from HelmRepo during garbage collection when a
type switch to OCI HelmRepo occurs. This ensures the OCI HelmRepo does
not have any conditions from the previous type.

Co-authored-by: Soule BA <soule@weave.works>
Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-05-31 03:53:33 +05:30