By providing the Generation of the object that is getting reconciled
as version metadata to the builder if any custom values files are
defined, the Artifact revision changes if the specification does,
ensuring consumers of the Artifact are able to react to changes in
values (and perform a release).
Signed-off-by: Hidde Beydals <hello@hidde.co>
This makes it possible to signal reference (validation) errors
happening before the build process actually starts dealing with
the chart.
At present, this does not have a more specific counterpart in the API,
but this is expected to change when the conditions logic is revised.
Signed-off-by: Hidde Beydals <hello@hidde.co>
- Add some more documentation around chart builders
- Ensure correct indentation in some doc comments
- Provide example of using `errors.Is` for typed `BuildError`
- Mention "bytes" in file size limit errors
- Add missing copyright header
Signed-off-by: Hidde Beydals <hello@hidde.co>
This allows custom configuration of the Helm file read limits, allowing
a user to overwrite them to their likenings if the defaults are too
restrictive for their specific setup using arguments:
`--helm-{index,chart,chart-file}-max-size`
Signed-off-by: Hidde Beydals <hello@hidde.co>
This commit introduces a typed `BuildError` to be returned by
`Builder.Build` in case of a failure.
The `Reason` field in combination with `BuildErrorReason` can be used
to signal (or determine) the reason of a returned error within the
context of the build process.
At present this is used to determine the correct Condition Reason, but
in a future iteration this can be used to determine the negative
polarity condition that should be set to indicate a precise failure to
the user.
Signed-off-by: Hidde Beydals <hello@hidde.co>
- For remote builds, if the build option has a version metadata, the
chart should be repackaged with the provided version.
- Update internal/helm/testdata/charts/helmchart-0.1.0.tgz to include
value files for testing merge chart values.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
Dealing with some loose ends around making observations, and code
style.
The loaded byes of a chart are used as a revision to ensure e.g.
periodic builds with unstable ordering of items do not trigger a false
positive.
Signed-off-by: Hidde Beydals <hello@hidde.co>
With all the logic that used to reside in the `controllers` package
factored into this package, it became cluttered. This commit tries to
bring a bit more structure in place.
Signed-off-by: Hidde Beydals <hello@hidde.co>
This commit starts wiring the factored out Helm chart build logic into
the reconciler to ensure, validating the API capabilities.
Signed-off-by: Hidde Beydals <hello@hidde.co>
This commit refactors the `ChartBuilder` that used to be a do-it-all
struct into an interace with two implementations:
- `LocalChartBuilder`: to build charts from a source on the local
filesystem, either from a directory or from a packaged chart.
- `RemoteChartBuilder`: to build charts from a remote Helm repository
index.
The new logic within the builders validates the size of the Helm size
it works with based on the `Max*Size` global variables in the internal
`helm` package, to address the recommendation from the security audit.
In addition, changes `ClientOptionsFromSecret` takes now a directory
argument which temporary files are placed in, making it easier to
perform a garbage collection of the whole directory at the end of a
reconcile run.
Signed-off-by: Hidde Beydals <hello@hidde.co>
This commit starts with the creation of a `ChartBuilder` to facilitate
the (conditional) build of a chart outside of the reconciler logic.
The builder can be configured with a set of (modifying) options, which
define together with the type of chart source what steps are taken
during the build.
To better facilitate the builder's needs and attempt to be more
efficient, changes have been made to the `DependencyBuilder` and
`ChartRepository` around (order of) operations and/or lazy-load
capabilities.
Signed-off-by: Hidde Beydals <hello@hidde.co>
This commit starts with the optimization of the `DepenendencyManager`,
ensuring the chart indexes are lazy loaded, and replacing the
(limitless) concurrency with a configurable number of workers with a
default of 1.
Signed-off-by: Hidde Beydals <hello@hidde.co>
This commits adds simple caching capabilities to the
`ChartRepository`, which makes it possible to load the `Index` from a
defined `CachePath` using `LoadFromCache()`, and to download the index
to a new `CachePath` using `CacheIndex()`.
In addition, the repository tests have been updated to make use of
Gomega, and some missing ones have been added.
Signed-off-by: Hidde Beydals <hello@hidde.co>
This commits adds `LoadChartMetadataFromArchive` and
`LoadChartMetadataFromDir` helpers to the internal `helm` package
to be able to make observations to the Helm metadata file without
loading the chart in full.
The helpers are compatible with charts of the v1 format (with a
separate `requirements.yaml` file), and an additional
`LoadChartMetadata` helper is available to automatically call the
right `LoadChartMetadataFrom*` version by looking at the file
description of the given path.
Signed-off-by: Hidde Beydals <hello@hidde.co>
To mitigate warnings for CVE-2021-41092. Because even if there is no
impact whatsoever, we are nice people.
Signed-off-by: Hidde Beydals <hello@hidde.co>
This mitigates another warning for CVE-2017-11468, which is mostly
triggered because a part of Helm depends on it that our code paths
never reach.
Signed-off-by: Hidde Beydals <hello@hidde.co>
- Ensure the proper path is garbage collected for libgit2 repositories,
as the `Path` method on the repository object returns the `.git`
directory, and not the root path.
- Ensure the Helm test server does not get swapped during tests,
with as side-effect that no obsolete temporary directories remain.
Signed-off-by: Hidde Beydals <hello@hidde.co>
This changes the format of the Artifact checksum from SHA1 to SHA256 to
mitigate chosen-prefix and length extension attacks, and ensures it can
be used to secure content against malicious modifications.
Source consumers (including our own {kustomize,helm}-controllers)
should ensure the SHA256 of a downloaded artifact matches the
advertised checksum before making use of it.
Signed-off-by: Hidde Beydals <hello@hidde.co>
- Adds tests for the libgit2 remote callbacks
- Adds tests for CheckoutStrategyForImplementation with context timeout
and verify timeout is respected by both the git implementations.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
Add SidebandProgressCallback to be able to cancel the network operation
before any transfer operation.
Add PushTransferProgressCallback to be able to cancel the push transfer
operation.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
In transferProgressCallback(), if the received objects is equal to the
total objects, return early with OK.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
With the information from the refactor still fresh in mind, I continue
to find new paths now I mentally tamed the git2go beast.
`libgit2` seems to assume that a transport will eventually tell by
itself that it has timed out. This also means that at present any
timeout configuration does not seem have an effect. It will continue
to transfer until the remote (or _something_ else) tells it is no
longer transfering.
This commit introduces a simple check (without tests) which was used
to confirm the theory in combination with the tests in
`pkg/git/strategy` (by setting it to a very low timeout and observing
it fail).
A future iteration should probably take the data given to the callback
into account to ensure it doesn't error out if the given data[1]
reports it has successfully received all objects. Another candidate
for this check may be `CompletionCallback`, but one should study the
C code (and likely some Go code as well) before this.
In addition, to ensure the same timeout is taken into account for push
operations, `PushTransferProgressCallback` may require a likewise
helper.
[1]: https://github.com/libgit2/git2go/blob/main/remote.go#L50-L58
Signed-off-by: Hidde Beydals <hello@hidde.co>
For signed commit verification, this code errors out on line 303:
✗ GitRepository reconciliation failed: ''PGP public keys secret error: expected pointer, but got nil
Pointer was not initialized with a concrete instance of the Secret struct
Signed-off-by: Kingdon Barrett <yebyen@gmail.com>
parseKnownHosts() uses golang.org/x/crypto/ssh's ParseKnownHosts() for
parsing known hosts. It returns EOF error when the input is not a host
public key, but a valid known_hosts content, like a comment line.
With this fix, lines causing EOF error are skipped and the parsing of
the known_hosts file continues. But invalid lines still cause parsing
failure.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
Hidde Beydals
Sunny
Hidde Beydals
Stefan Prodan
Kingdon Barrett