This commit reintroduces the use of goroutines for fetching objects,
but in the caller of the client interface rather than in a particular
client implementation.
Signed-off-by: Michael Bridgen <michael@weave.works>
The algorithm for conditionally downloading object files is the same,
whether you are using GCP storage or an S3/Minio-compatible
bucket. The only thing that differs is how the respective clients
handle enumerating through the objects in the bucket; by implementing
just that in each provider, I can have the select-and-fetch code in
once place.
This deliberately omits the parallelised fetching that the GCP client
had, for the sake of lining the clients up. It can be reintroduced (in
the factored out code) later.
Signed-off-by: Michael Bridgen <michael@weave.works>
General cleanup and function renaming
Signed-off-by: pa250194 <pa250194@ncr.com>
Fix: errgroup provides waitgroup within the Go function
Signed-off-by: pa250194 <pa250194@ncr.com>
For signed commit verification, this code errors out on line 303:
✗ GitRepository reconciliation failed: ''PGP public keys secret error: expected pointer, but got nil
Pointer was not initialized with a concrete instance of the Secret struct
Signed-off-by: Kingdon Barrett <yebyen@gmail.com>
This commit changes the `gogit` behavior for commit checkouts,
now allowing one to reference to just a commit while omitting any
branch reference. Doing this creates an Artifact with a
`HEAD/<commit>` revision.
If both a `branch` and `commit` are defined, the commit is expected
to exist within the branch. This results in a more efficient clone
of just the target branch, and also makes this change backwards
compatible.
Fixes#407Fixes#315
Signed-off-by: Hidde Beydals <hello@hidde.co>
This commit refactors the previous `Commit` interface into a
standardised `Commit` struct. This object contains sufficient
information for referencing, observating and (PGP) verification.
- `libgit2` commit checkout does now return `HEAD/<SHA1>` as
the branch is not taken into account.
- `git2go` objects are now properly `Free`d everywhere
- `Verify` logic is tested.
Signed-off-by: Hidde Beydals <hello@hidde.co>
This commit moves the previous `AuthStrategy` wiring to a more generic
`AuthOptions`, breaking free from implementation specific details in
the `git` package.
Signed-off-by: Hidde Beydals <hello@hidde.co>
This commit adds a `ReconcileStrategy` field to the `HelmChart` resource, which
allows defining when a new chart should be packaged and/or published if it
originates from a `Bucket` or `GitRepository` resource.
The two available strategies are:
- `ChartVersion`: creates a new artifact when the version of the Helm chart as
defined in the `Chart.yaml` from the Source is different from the current
version.
- `Revision`: creates a new artifact when the revision of the Source is
different from the current revision.
For the `Revision` strategy, the (checksum part of the) revision of the
artifact the chart originatesfrom is added as SemVer metadata.
A chart from a `GitRepository` with Artifact revision
`main/f0faacd5164a875ebdbd9e3fab778f49c5aadbbc` and a chart with e.g. SemVer
`0.1.0` will be published as `0.1.0+f0faacd5164a875ebdbd9e3fab778f49c5aadbbc`.
A chart from a `Bucket` with Artifact revision
`f0faacd5164a875ebdbd9e3fab778f49c5aadbbc` and a chart with e.g. SemVer `0.1.0`
will be published as `0.1.0+f0faacd5164a875ebdbd9e3fab778f49c5aadbbc`.
Signed-off-by: Dylan Arbour <arbourd@users.noreply.github.com>
Added Support for Google Cloud Storage with Workload Identity as Source Provider. This enables the use of GCP without enabling S3 compatible access.
Signed-off-by: pa250194 <pa250194@ncr.com>
This ensures the checksum is predictable, and not influenced by e.g.
different runtime configuration settings, or FS specific data.
Signed-off-by: Hidde Beydals <hello@hidde.co>
This commit changes the checksum method which is used to calculate the
revision of a Bucket source, so that the file paths are taken into
account and directory structure changes can be observed.
Signed-off-by: Hidde Beydals <hello@hidde.co>
controller-runtime's client package provides ObjectKeyFromObject() to
extract NamespacedName from a given object. ObjectKey() in
internal/util package is a helper for the same. Replace the internal
helper with controller-runtime's helper for the same.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
v3.6.1 is a a security update from Helm, ensuring that credentials are
always only passed to the defined repository host.
Based on Helm user reports, disabling this behavior may be required for
some Helm repository solutions like Artifactory, and may be done by
setting `PassCredentials` in the `HelmRepositorySpec`.
For more information, see:
https://github.com/helm/helm/security/advisories/GHSA-56hp-xqp3-w2jf
Signed-off-by: Hidde Beydals <hello@hidde.co>
Unlike Bucket resources which are matched by key as presented by S3,
ignore rules for GitRepository objects do have a domain: the temporary
directory of the Git repository.
Signed-off-by: Hidde Beydals <hello@hidde.co>
This likely happened because the byte buffer response was already
being read by the chart loader, making it empty by the time the
artifact was written to storage.
As an alternative, and because it makes the code a tiny bit less
obnoxious: write the data to a temp file first, and later decide
what file to copy over and use as an stored artifact.
Signed-off-by: Hidde Beydals <hello@hidde.co>
This commit makes the filtering applied during the archiving
configurable by introducing an optional `ArchiveFileFilter`
callback argument and a `SourceIgnoreFilter` implementation.
`SourceIgnoreFilter` filters out files matching
sourceignore.VCSPatterns and any of the provided patterns.
If an empty gitignore.Pattern slice is given, the matcher is set to
sourceignore.NewDefaultMatcher.
The `GitRepository` now loads the ignore patterns before archiving
the repository contents by calling `sourceignore.LoadIgnorePatterns`
and other helpers. The loading behavior is **breaking** as
`.sourceignore` files in the (subdirectories of the) repository are
now still taken into account if `spec.ignore` for a resource is
defined, overwriting is still possible by creating an overwriting
rule in the `spec.ignore` of the resource.
This change also makes it possible for the `BucketReconciler` to not
configure a callback at all and prevent looking for ignore
matches twice. To finalize the bucket refactor, a change to the
reconciler has been made to look for a `.sourceignore` file in
the root of the bucket to provide an additional way of configuring
(global) exclusions.
Signed-off-by: Hidde Beydals <hello@hidde.co>
Adds a test that loads the helmChart from the updated resource and
verifies that `testOverride` (the value overrode in the test fixtures)
changes from `false` to `true`.
Signed-off-by: Dylan Arbour <arbourd@users.noreply.github.com>
This commit adds a test specifically for RecurseSubmodules. It takes a
bit more preparation, since it needs a repo using submodules to start
with. go-git doesn't appear to support adding submodules
programmatically, so the preparation is done in part by execing `git`.
Signed-off-by: Michael Bridgen <michael@weave.works>
This commit fixes a bug where local chart dependencies would not be
detected correctly due to the absence of a repository URL.
Signed-off-by: Hidde Beydals <hello@hidde.co>
Michael Bridgen
pa250194
Kingdon Barrett
Sunny
Hidde Beydals
Dylan Arbour
Stefan Prodan
Philip Laine
Somtochi Onyekwere
Raffael Sahli
Philip Laine
Thomas Runyon
Léo Martinez