This commit starts with the creation of a `ChartBuilder` to facilitate
the (conditional) build of a chart outside of the reconciler logic.
The builder can be configured with a set of (modifying) options, which
define together with the type of chart source what steps are taken
during the build.
To better facilitate the builder's needs and attempt to be more
efficient, changes have been made to the `DependencyBuilder` and
`ChartRepository` around (order of) operations and/or lazy-load
capabilities.
Signed-off-by: Hidde Beydals <hello@hidde.co>
This commit starts with the optimization of the `DepenendencyManager`,
ensuring the chart indexes are lazy loaded, and replacing the
(limitless) concurrency with a configurable number of workers with a
default of 1.
Signed-off-by: Hidde Beydals <hello@hidde.co>
This commits adds simple caching capabilities to the
`ChartRepository`, which makes it possible to load the `Index` from a
defined `CachePath` using `LoadFromCache()`, and to download the index
to a new `CachePath` using `CacheIndex()`.
In addition, the repository tests have been updated to make use of
Gomega, and some missing ones have been added.
Signed-off-by: Hidde Beydals <hello@hidde.co>
This commits adds `LoadChartMetadataFromArchive` and
`LoadChartMetadataFromDir` helpers to the internal `helm` package
to be able to make observations to the Helm metadata file without
loading the chart in full.
The helpers are compatible with charts of the v1 format (with a
separate `requirements.yaml` file), and an additional
`LoadChartMetadata` helper is available to automatically call the
right `LoadChartMetadataFrom*` version by looking at the file
description of the given path.
Signed-off-by: Hidde Beydals <hello@hidde.co>
To mitigate warnings for CVE-2021-41092. Because even if there is no
impact whatsoever, we are nice people.
Signed-off-by: Hidde Beydals <hello@hidde.co>
This mitigates another warning for CVE-2017-11468, which is mostly
triggered because a part of Helm depends on it that our code paths
never reach.
Signed-off-by: Hidde Beydals <hello@hidde.co>
- Ensure the proper path is garbage collected for libgit2 repositories,
as the `Path` method on the repository object returns the `.git`
directory, and not the root path.
- Ensure the Helm test server does not get swapped during tests,
with as side-effect that no obsolete temporary directories remain.
Signed-off-by: Hidde Beydals <hello@hidde.co>
This changes the format of the Artifact checksum from SHA1 to SHA256 to
mitigate chosen-prefix and length extension attacks, and ensures it can
be used to secure content against malicious modifications.
Source consumers (including our own {kustomize,helm}-controllers)
should ensure the SHA256 of a downloaded artifact matches the
advertised checksum before making use of it.
Signed-off-by: Hidde Beydals <hello@hidde.co>
- Adds tests for the libgit2 remote callbacks
- Adds tests for CheckoutStrategyForImplementation with context timeout
and verify timeout is respected by both the git implementations.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
Add SidebandProgressCallback to be able to cancel the network operation
before any transfer operation.
Add PushTransferProgressCallback to be able to cancel the push transfer
operation.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
In transferProgressCallback(), if the received objects is equal to the
total objects, return early with OK.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
With the information from the refactor still fresh in mind, I continue
to find new paths now I mentally tamed the git2go beast.
`libgit2` seems to assume that a transport will eventually tell by
itself that it has timed out. This also means that at present any
timeout configuration does not seem have an effect. It will continue
to transfer until the remote (or _something_ else) tells it is no
longer transfering.
This commit introduces a simple check (without tests) which was used
to confirm the theory in combination with the tests in
`pkg/git/strategy` (by setting it to a very low timeout and observing
it fail).
A future iteration should probably take the data given to the callback
into account to ensure it doesn't error out if the given data[1]
reports it has successfully received all objects. Another candidate
for this check may be `CompletionCallback`, but one should study the
C code (and likely some Go code as well) before this.
In addition, to ensure the same timeout is taken into account for push
operations, `PushTransferProgressCallback` may require a likewise
helper.
[1]: https://github.com/libgit2/git2go/blob/main/remote.go#L50-L58
Signed-off-by: Hidde Beydals <hello@hidde.co>
For signed commit verification, this code errors out on line 303:
✗ GitRepository reconciliation failed: ''PGP public keys secret error: expected pointer, but got nil
Pointer was not initialized with a concrete instance of the Secret struct
Signed-off-by: Kingdon Barrett <yebyen@gmail.com>
parseKnownHosts() uses golang.org/x/crypto/ssh's ParseKnownHosts() for
parsing known hosts. It returns EOF error when the input is not a host
public key, but a valid known_hosts content, like a comment line.
With this fix, lines causing EOF error are skipped and the parsing of
the known_hosts file continues. But invalid lines still cause parsing
failure.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
Adds tests for git.CheckoutStrategy to check if both the git
implementations follow the same SemVer tag selection rules.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
This changes the logic of `credentialsCallback` so that it takes the
`allowedTypes` passed on by `git2go` into account. Reason for this
change is because this prepares it to work with `v33`, but also
because it can provide better guidance when `libgit2` has been
compiled with a different configuration, which e.g. doesn't
allow for "in-memory SSH keys".
Because `AuthOptions#Identity` now gets validated by the callback
and go-git does its own validaiton, the check has been removed
from `Validate` (and now does a simple check if the fields are set).
Signed-off-by: Hidde Beydals <hello@hidde.co>
Adds tests for git.CheckoutStrategy to check if both the git
implementations work with all the authentication methods.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
Main requirement for this is the image-automation-controller
depending on being able to get a working auth configuration.
Once the package is moved, we should add push logic to it,
so that the controller is able to use that instead.
Signed-off-by: Hidde Beydals <hello@hidde.co>
This commit changes the `gogit` behavior for commit checkouts,
now allowing one to reference to just a commit while omitting any
branch reference. Doing this creates an Artifact with a
`HEAD/<commit>` revision.
If both a `branch` and `commit` are defined, the commit is expected
to exist within the branch. This results in a more efficient clone
of just the target branch, and also makes this change backwards
compatible.
Fixes#407Fixes#315
Signed-off-by: Hidde Beydals <hello@hidde.co>
This commit refactors the previous `Commit` interface into a
standardised `Commit` struct. This object contains sufficient
information for referencing, observating and (PGP) verification.
- `libgit2` commit checkout does now return `HEAD/<SHA1>` as
the branch is not taken into account.
- `git2go` objects are now properly `Free`d everywhere
- `Verify` logic is tested.
Signed-off-by: Hidde Beydals <hello@hidde.co>
This commit moves the previous `AuthStrategy` wiring to a more generic
`AuthOptions`, breaking free from implementation specific details in
the `git` package.
Signed-off-by: Hidde Beydals <hello@hidde.co>
As part of fluxcd/.github#4, create DEVELOPMENT.md
Will link to this doc from fluxcd.io/contributing
so new folks can find this easily.
Signed-off-by: Daniel Holbach <daniel@weave.works>
Hidde Beydals
Hidde Beydals
Stefan Prodan
Sunny
Kingdon Barrett
Daniel Holbach