Commit Graph

123 Commits

Author SHA1 Message Date
Soule BA b30404fff1
Bind cached helm index to the maximum index size
Signed-off-by: Soule BA <bah.soule@gmail.com>
2024-04-22 17:02:33 +02:00
Austin Abro ab6d560002
allow insecure registry login
Signed-off-by: Austin Abro <AustinAbro321@gmail.com>

only add loginoptinsecure when already logging in

Signed-off-by: Austin Abro <AustinAbro321@gmail.com>

fix tests, move registry login options

Signed-off-by: Austin Abro <AustinAbro321@gmail.com>

cleaner code

Signed-off-by: Austin Abro <austinabro321@gmail.com>

change localhost to 127.0.0.1 in insecure suite test path

Signed-off-by: Austin Abro <austinabro321@gmail.com>

let myself run tests

Signed-off-by: Austin Abro <AustinAbro321@gmail.com>

allow myself to run tests

Signed-off-by: Austin Abro <AustinAbro321@gmail.com>

clearer error message in test

Signed-off-by: Austin Abro <AustinAbro321@gmail.com>

update test to reflect new behavior

Signed-off-by: Austin Abro <AustinAbro321@gmail.com>

undo test config changes

Signed-off-by: Austin Abro <AustinAbro321@gmail.com>

remove extra whitespace

Signed-off-by: Austin Abro <AustinAbro321@gmail.com>
2024-04-18 13:12:18 +00:00
Soule BA 51a7c72d4c
Update controller to Helm v3.14.2 and Kubernetes v1.29.0
Signed-off-by: Soule BA <bah.soule@gmail.com>
2024-04-02 15:00:52 +02:00
JasonTheDeveloper 553945ab8e Add verification support for notation signed artifacts
Introduces a new verification provider `notation` to verify notation signed artifacts. Currently only cosign is supported and that is a problem if the end user utilises notation.

---------

Signed-off-by: Jason <jagoodse@microsoft.com>
Signed-off-by: JasonTheDeveloper <jagoodse@microsoft.com>
Signed-off-by: Jagpreet Singh Tamber <jagpreetstamber@gmail.com>
Co-authored-by: souleb <bah.soule@gmail.com>
Co-authored-by: Jagpreet Singh Tamber <jagpreetstamber@gmail.com>
Co-authored-by: Sunny <github@darkowlzz.space>
2024-03-26 21:32:23 +11:00
Sunny 84b30d1404
Improve chart name validation
Allow periods in the chart name and validate chart name before
packaging it.

Signed-off-by: Sunny <github@darkowlzz.space>
2024-02-20 14:27:57 +00:00
Sanskar Jaiswal 6a88d10bf1
oci: rename `OCIChartRepository.insecure` to `insecureHTTP`
Rename `OCIChartRepository.insecure` to `OCIChartRepository.insecureHTTP` in
`internal/helm/repository` to match the code docs and the related setter
method.

Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
2023-11-28 18:44:35 +05:30
Sanskar Jaiswal 4086c25acb
helmrepo: allow OCI helmrepos to connect to insecure registries
Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
2023-11-23 12:32:14 +05:30
Stefan Prodan 6e78779720
Add `.spec.insecure` to `HelmRepository`
Allow connecting to Helm OCI repositories over plain HTTP (non-TLS endpoint).

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2023-11-23 12:05:37 +05:30
Sunny 3dacb3140e optional interval and default timeout for helmrepo
With static HelmRepository OCI, the interval become optional. Make
interval optional in the API. Introduce getters for interval, in the
form of GetRequeueAfter(), and timeout with internal default values.

HelmRepository will not have interval and timeout fields unless it's
explicitly set.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2023-11-22 21:05:57 +05:30
Sanskar Jaiswal dfcede03f2
helmrepo: only configure tls login option when required
Modify `GetHelmClientOpts()` to only configure the TLS login option when
an authentication login option is configured. This prevents the
reconciler from trying to authenticate against public registries.

Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
2023-11-22 10:05:45 +05:30
Hidde Beydals 646089e07e
storage: change default file permissions
Signed-off-by: Hidde Beydals <hidde@hhh.computer>
2023-11-06 12:22:40 +01:00
Hidde Beydals 09772bd092
misc: set TLS certificate files perms to `0o600`
Signed-off-by: Hidde Beydals <hidde@hhh.computer>
2023-10-11 11:03:56 +02:00
Hidde Beydals 20230811e4
misc: address duplicate imports
Signed-off-by: Hidde Beydals <hidde@hhh.computer>
2023-10-11 11:03:54 +02:00
Hidde Beydals a74eb57811
misc: `math/rand.Read()` has been deprecated
See: https://pkg.go.dev/math/rand@go1.20#Read

Signed-off-by: Hidde Beydals <hidde@hhh.computer>
2023-10-11 11:03:54 +02:00
Sanskar Jaiswal f787fc7046
helmrepo: fix Secret type check for TLS via `.spec.secretRef`
This is a regression fix introduced in a302c71 which would wrongly check
for the type of the Secret specified in `.spec.secretRef` while
configuring TLS data.

Introduce `LegacyTLSClientConfigFromSecret` which does not check the
Secret type while constructing the TLS config.

Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
2023-09-06 00:27:45 +05:30
Somtochi Onyekwere 64139e7ec1 preserve url encoded path in helm repo
Signed-off-by: Somtochi Onyekwere <somtochionyekwere@gmail.com>
2023-08-23 10:01:27 +01:00
Sanskar Jaiswal 4bd6bcc9e9
helmrepo: adopt Kubernetes TLS secrets for `.spec.certSecretRef`
Adopt Kubernetes TLS secrets API to check for TLS data in the Secret
referred to by `.spec.certSecretRef`, i.e. check for keys `tls.crt` and
`tls.key` for the certificate and private key. Use `ca.crt` for the CA
certificate.

Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
2023-08-22 18:23:04 +05:30
Somtochi Onyekwere 1aa9cf2219 archive helm index in JSON format
Signed-off-by: Somtochi Onyekwere <somtochionyekwere@gmail.com>
2023-08-07 16:10:03 +02:00
Soule BA d45c08cba6
Add support for custom certificate and skip-tls-verify in helm OCI
If implemented user will be able to provide their own custom start and
bypass tls verification when interacting with OCI registries over https
to pull helmCharts.

Signed-off-by: Soule BA <soule@weave.works>
2023-08-07 16:07:04 +03:00
Sanskar Jaiswal 79adec586b
helm: add support for specifying TLS auth via `.spec.certSecretRef`
Add support for specifying TLS auth data via `.spec.certSecretRef` in
HelmRepository and log a deprecation warning if TLS is configured via
`.spec.secretRef`. Introduce (and refactor) Helm client builder and
auth helpers to reduce duplicated code and increase uniformity and
testability.

Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
2023-07-31 13:29:17 +05:30
Hidde Beydals b2da6f0647
api: Remove deprecated `Checksum` from `Artifact`
Signed-off-by: Hidde Beydals <hidde@hhh.computer>
2023-03-24 11:56:12 +01:00
Hidde Beydals 9509b62f40
helm/repository: address invalid test
Common mistake
(https://github.com/golang/go/wiki/CommonMistakes#using-goroutines-on-loop-iterator-variables),
but due to update now properly found by `go vet`.

In addition to making the test cases work in general.

Signed-off-by: Hidde Beydals <hidde@hhh.computer>
2023-03-02 16:14:01 +01:00
Hidde Beydals c712fede57 internal/helm: del deprecated ChartRepo#Revision
Signed-off-by: Hidde Beydals <hello@hidde.co>
2023-02-22 23:35:21 +01:00
Hidde Beydals c0a1099719 helm: only use Digest to calculcate index revision
In #1001 bits around the Helm repository reconciliation logic were
rewritten, mostly based on the documented behavior instead of the
actual code. This resulted in the reintroduction of a YAML marshal of
the (sorted) index YAML instead of reliance of just the checksum of the
file.

This to take situations into account in which a repository would e.g.
provide a new random order on every generation. However, this approach
is (extremely) expensive as the marshal goes through a JSON -> YAML
loop, eating lots of RAM in the process.

As the further (silently) introduced behavior has not resulted in any
reported issues, I deem this approach safe and better than e.g.
encoding to just JSON which would still require a substantial amount of
memory.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2023-02-22 23:33:59 +01:00
Somtochi Onyekwere 6f0384c50e Normalize path in url
Signed-off-by: Somtochi Onyekwere <somtochionyekwere@gmail.com>
2023-02-15 03:54:18 +01:00
Hidde Beydals d62f4dc0c6 misc: order imports and align digest aliases
Signed-off-by: Hidde Beydals <hello@hidde.co>
2023-02-14 12:48:36 +01:00
Hidde Beydals 0aaeeee5e9 controllers: RFC-0005 fmt for HelmRepository rev
This includes changes to the `ChartRepository`, to allow calculating
the revision and digest and tidy things.

In addition, the responsibility of caching the `IndexFile` has been
moved to the reconcilers. As this allowed to remove a lot of
complexities within the `ChartRepository`, and prevented passing on
the cache in general.

Change `HelmRepository`'s Revision to digest

Signed-off-by: Hidde Beydals <hello@hidde.co>
2023-02-14 12:48:36 +01:00
Hidde Beydals 909ece4092 helm: use digest lib for checksum calculation
Signed-off-by: Hidde Beydals <hello@hidde.co>
2023-02-14 12:48:36 +01:00
Sunny f9927e7d46 helm/oci: Add context to download failure
Add chart address in the OCI chart download failure error message to make
it clear about the chart URL that was attempted to download.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2023-01-31 19:18:22 +05:30
Sunny 869a9df6e9 helm: return ErrExternal when loading index fails
Loading index can fail due to network error. Return ErrExternal typed
error for it.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-12-19 16:12:39 +05:30
Sunny 8018b450a1 helm: typed errors from GetChartVersion()
Update the implementations of the helm repository downloaders to return
implementation specific typed error from GetChartVersion(). This is
needed to distinguish between persistent build error and transient build
error.
In the case of OCI charts, a transient network failure shouldn't be
considered a persistent build failure of the chart and should be
retried.

Two repository errors, ErrReference and ErrExternal are introduced for
the repository downloader implementations to provide enough context
about the failure which can be used by the caller to add appropriate
context as per the needs. In case of chart builder, it adds the build
error context based on the repository error value.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-12-19 16:12:39 +05:30
Soule BA 6b5d0b1335
Fix aliased chart dependencies resolution
If implemented, this fix the issue were aliased chart dependencies were
detected but not included in the final packaged chart.

Signed-off-by: Soule BA <soule@weave.works>
2022-12-16 23:12:27 +01:00
Soule BA 06a55590a5
Fix verification condition
Delete a failed verification condition at the beginning of the source
reconciliation and set `SourceVerifiedCondition` to false approprietly.

Set the `BuildOptions.Verify` to true as long as Verify is enabled in the
API fields.

Signed-off-by: Soule BA <soule@weave.works>
2022-10-21 15:21:10 +02:00
Soule BA 25673ac512
addressing review comments
Signed-off-by: Soule BA <soule@weave.works>
2022-10-21 10:00:08 +02:00
Soule BA 0e97547eeb
implement Cosign verification for HelmCharts
If implemented, users will be able to enable chart verification for OCI
based helm charts.

Signed-off-by: Soule BA <soule@weave.works>
2022-10-21 10:00:08 +02:00
Soule BA bb83270acc
Refactor to use authn for authentication as OCIrepository does
If implemented the oras registry loginOption will only be used internaly
with the specific ChartRepo struct.

This will permit reusing more easily feature developped with
googlecontainerregistry authn.

Signed-off-by: Soule BA <soule@weave.works>
2022-10-21 10:00:07 +02:00
Stefan Prodan ca3496e758
Build with Go 1.19
- Update Go to 1.19 in CI
- Use Go 1.19 in base image
- Update controller-gen v0.8.0 and regenerate manifests

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-09-28 15:35:26 +03:00
Max Jonas Werner 7a139647a2
add custom CA certificates to system certificates
When a custom CA certificate is provided in a Secret's `caCert` field
referenced in `HelmRelease.spec.secretRef` then that CA cert is now
added to the list of system certificates instead of it replacing the
system certificates. This makes HelmRepositories work in mixed
environments where charts are pulled from both, a public repository
and a private repository (e.g. through a chart dependency).

The test that is added as part of this change will fail without the
change and passes with it.

closes #866
closes fluxcd/helm-controller#519

Signed-off-by: Max Jonas Werner <max@e13.dev>
2022-09-23 10:57:00 +03:00
Soule BA ad3eb5ca47
Enable contextual login for helm OCI
If implemented, this pr will enable user to use the auto login feature
in order to automatically login to their provider of choice's container
registry (i.e. aws, gcr, acr).

Signed-off-by: Soule BA <soule@weave.works>
2022-08-25 22:27:35 +02:00
Max Jonas Werner 64c1b065a8
allow for charts from OCI registries to specify a chart path
This change allows for a HelmRepository to point to e.g. "ghcr.io" and
then a HelmRelease pointing to the chart "stefanprodan/charts/podinfo"
in its `.spec.chart.spec.chart` field.

Related discussion: https://github.com/fluxcd/flux2/discussions/2959

Signed-off-by: Max Jonas Werner <max@e13.dev>
2022-08-08 15:44:43 +02:00
Max Jonas Werner b86572b98e
don't fetch tags when exact version is used in HelmRepository
Taking this shortcut has two benefits:

1. It allows charts to be fetched from AWS's public container registry
   at public.ecr.aws
2. It makes reconciling a HelmChart faster by skipping one or more
   potentially expensive API calls to the registry.

I adapted the unit tests to the new behavior that the
OCIChartRepository doesn't fail anymore for the case where a specific
chart version has been requested that doesn't actually exist in the
registry.

refs #845

Signed-off-by: Max Jonas Werner <max@e13.dev>
2022-07-27 13:25:43 +02:00
York Chen d5a75f6b2f feat: cache helmrepo early after reconcile
1. moved chartRepo.Unload() from reconcileSource() to the defer func in reconcileArtifact to allow caching index in memory
2. added step to init memory cache in reconcileArtifact()
3. added step to save helmrepo index into memory cache in reconcileArtifact()

Signed-off-by: York Chen <ychen@d2iq.com>
2022-07-21 18:17:26 +01:00
Soule BA e4209dba8a
Fix SIGSEGV when resolving charts dependencies
If implemented, this make sure than we clear only referenced
downloaders.

It is also checked if the repository url is supported.

Signed-off-by: Soule BA <soule@weave.works>
2022-07-13 14:59:38 +02:00
Soule BA 361b975bf4
Enable remote dependencies from OCI repositories
If implemented, the source controller will be able to resolve charts
dependencies from OCI repositories.

The remote builder has been refactored as part of this work.

Signed-off-by: Soule BA <soule@weave.works>
2022-07-06 19:11:01 +02:00
Soule BA b402e546bc
Refactor repository logic
Signed-off-by: Soule BA <soule@weave.works>
2022-07-06 19:11:00 +02:00
Somtochi Onyekwere 35a7ea1efa Remove trailing slash
Signed-off-by: Somtochi Onyekwere <somtochionyekwere@gmail.com>
2022-06-29 11:05:08 +01:00
Soule BA a163ea1dff
Fix sorting semver from OCI repository tags
If implemented this fix the issue where we previously did a string
ordering of matching semver versions when retrieving a list of tags from
an OCI registry.

Signed-off-by: Soule BA <soule@weave.works>
2022-06-07 22:20:36 +02:00
Soule BA 0d4d78fc6b
Fix repository cache regression
If implemented this make sure we don't unload an index before caching it
during a chart build phase.

Signed-off-by: Soule BA <soule@weave.works>
2022-06-06 13:15:10 +02:00
Sunny f79fd034fd
registry: repo URL and dockerconfig URL mismatch
Registry login option should verify that the obtained dockerconfig
credentials are for the same host. When the helmrepo URL and the URL in
docker auth config don't match, the docker config store returns an
empty auth config, instead of failing. This results in accepting empty
username and password. The HelmRepo would appear to be ready in such
situation because the creds are empty, no login is attempted. But when
a HelmChart tries to use the login options, it'd fail.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-06-01 00:09:36 +05:30
Max Jonas Werner bb569bec1f
include Secret name in returned errors
Signed-off-by: Max Jonas Werner <mail@makk.es>
2022-05-24 10:32:42 +02:00