Commit Graph

896 Commits

Author SHA1 Message Date
Kingdon Barrett d400c8e9e8 Fix error during public key import
For signed commit verification, this code errors out on line 303:

✗ GitRepository reconciliation failed: ''PGP public keys secret error: expected pointer, but got nil

Pointer was not initialized with a concrete instance of the Secret struct

Signed-off-by: Kingdon Barrett <yebyen@gmail.com>
2021-10-29 10:54:16 -04:00
Hidde Beydals d3ea648454
Merge pull request #476 from fluxcd/release-v0.17.0 2021-10-28 15:29:49 +02:00
Hidde Beydals 4d7812ea64 Release v0.17.0
Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-28 15:11:58 +02:00
Hidde Beydals cd245069d1
Merge pull request #475 from darkowlzz/libgit2-parseKnownHosts-fix 2021-10-28 14:44:52 +02:00
Sunny 10d262032c libgit2: handle EOF in parseKnownHosts()
parseKnownHosts() uses golang.org/x/crypto/ssh's ParseKnownHosts() for
parsing known hosts. It returns EOF error when the input is not a host
public key, but a valid known_hosts content, like a comment line.

With this fix, lines causing EOF error are skipped and the parsing of
the known_hosts file continues. But invalid lines still cause parsing
failure.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2021-10-28 17:35:46 +05:30
Hidde Beydals 5ccb914620
Merge pull request #462 from fluxcd/git-auth-opts 2021-10-28 10:35:45 +02:00
Hidde Beydals d0ca107e55 docs: GitRepository commit without branch example
Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-27 17:00:11 +05:30
Sunny a7f2e870bf transportAuth(): Add checks for invalid transports
Update GitRepositoryReconciler to use a nil authOpts unless it's
configured.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2021-10-27 16:59:26 +05:30
Sunny f9a34045e1 Update gittestserver
New gittestserver fixes the issue with custom branch in an
initialized repo.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2021-10-27 00:43:31 +05:30
Sunny 5bd08a6960 Add Host field check in AuthOptions.Validate()
For ssh, Host field is required in AuthOptions.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2021-10-27 00:43:31 +05:30
Sunny 562af6d658 Add git.CheckoutStrategy SemVer checkout tests
Adds tests for git.CheckoutStrategy to check if both the git
implementations follow the same SemVer tag selection rules.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2021-10-27 00:43:31 +05:30
Hidde Beydals 99428f593e libgit2: change credentialsCallback logic
This changes the logic of `credentialsCallback` so that it takes the
`allowedTypes` passed on by `git2go` into account. Reason for this
change is because this prepares it to work with `v33`, but also
because it can provide better guidance when `libgit2` has been
compiled with a different configuration, which e.g. doesn't
allow for "in-memory SSH keys".

Because `AuthOptions#Identity` now gets validated by the callback
and go-git does its own validaiton, the check has been removed
from `Validate` (and now does a simple check if the fields are set).

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-27 00:43:31 +05:30
Hidde Beydals 7a5d8b116d git: styling nitpicks
Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-27 00:43:31 +05:30
Hidde Beydals c814e0fa26 git: remove ', error:' from returned error
As we properly nest errors.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-27 00:43:31 +05:30
Sunny 8c581ddfbc Add git.CheckoutStrategy auth tests
Adds tests for git.CheckoutStrategy to check if both the git
implementations work with all the authentication methods.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2021-10-27 00:43:31 +05:30
Sunny 942c310195 pkg/git: AuthOptions.Validate() test improvements
Adds more test cases for Validate() and an error for unknown transport.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2021-10-27 00:43:31 +05:30
Hidde Beydals 4a23126c6a libgit2: make RemoteCallbacks helper public
Main requirement for this is the image-automation-controller
depending on being able to get a working auth configuration.

Once the package is moved, we should add push logic to it,
so that the controller is able to use that instead.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-27 00:43:31 +05:30
Hidde Beydals b7376ce94c gogit: allow checkout of commit without branch
This commit changes the `gogit` behavior for commit checkouts,
now allowing one to reference to just a commit while omitting any
branch reference. Doing this creates an Artifact with a
`HEAD/<commit>` revision.

If both a `branch` and `commit` are defined, the commit is expected
to exist within the branch. This results in a more efficient clone
of just the target branch, and also makes this change backwards
compatible.

Fixes #407
Fixes #315

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-27 00:43:31 +05:30
Hidde Beydals 5a1fcc213b git: standardise commit and (PGP) verification
This commit refactors the previous `Commit` interface into a
standardised `Commit` struct. This object contains sufficient
information for referencing, observating and (PGP) verification.

- `libgit2` commit checkout does now return `HEAD/<SHA1>` as
  the branch is not taken into account.
- `git2go` objects are now properly `Free`d everywhere
- `Verify` logic is tested.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-27 00:43:31 +05:30
Hidde Beydals 80b9807550 gogit: add CheckoutStrategy tests
Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-27 00:43:31 +05:30
Hidde Beydals 0cf0d4e756 git: refactor AuthStrategy into AuthOptions
This commit moves the previous `AuthStrategy` wiring to a more generic
`AuthOptions`, breaking free from implementation specific details in
the `git` package.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-27 00:43:31 +05:30
Hidde Beydals 46a5b9c27d
Merge pull request #466 from dholbach/move-contributor-doc 2021-10-26 16:03:39 +02:00
Daniel Holbach 4411a6ec80 Centralise Contributor docs, add Dev doc
As part of fluxcd/.github#4, create DEVELOPMENT.md
	Will link to this doc from fluxcd.io/contributing
	so new folks can find this easily.

Signed-off-by: Daniel Holbach <daniel@weave.works>
2021-10-26 11:24:41 +02:00
Hidde Beydals db14c22923
Merge pull request #460 from fluxcd/release-v0.16.1 2021-10-22 14:40:45 +02:00
Hidde Beydals bef17ae851 Release v0.16.1
Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-22 14:24:22 +02:00
Hidde Beydals 7ff8da432e
Merge pull request #459 from fluxcd/libgit2-semver-flake 2021-10-22 13:43:40 +02:00
Hidde Beydals 9ff5334688 go-git: ensure original tag is used for TS lookup
Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-22 13:25:24 +02:00
Hidde Beydals 4c94f88ebb libgit2: ensure original tag is used for TS lookup
This adds an additional selection candidate, to ensure test flakiness
happens more often.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-22 13:19:34 +02:00
Hidde Beydals b35d7d88f2
Merge pull request #458 from fluxcd/libgit2-simple-tag 2021-10-22 12:40:45 +02:00
Hidde Beydals 56201f30fe libgit2: Free most objects
This commit ensures most of the `git2go` objects `Free` themselves from
the underlying C object.

Ensuring all objects are freed is not possible yet, due to the way
commits are wired in to facilitate verification later on. In a later
follow up, we should change this and e.g. validate as part of the
checkout process, and move the implementation specific authentication
configuration from `git` into `libgit2`.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-22 12:01:45 +02:00
Hidde Beydals d9473d008c libgit2: add remaining checkout strategy tests
This commit is a follow up on 4dc3185c5f
and adds tests for the remaining checkout strategies, while
consolidating some of the logic.

The consolidated logic ensures that (SemVer) tag and commit checkouts
happen using the same "checkout detached HEAD" logic.
The branch checkout is left unmodified, and simply checks out at the
current HEAD of the given branch.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-22 12:01:41 +02:00
Hidde Beydals 4dc3185c5f
Merge pull request #457 from fluxcd/libgit2-simple-tag 2021-10-21 17:34:45 +02:00
Hidde Beydals e736493730 libgit2: correctly resolve (annotated) tags
In d0560e5 the SemVer implementations were aligned, and the logic was
simplified a bit (or so I thought). This did however result in the
introduction of a regression, as it failed to take "simple tags" into
account.

This commit ensures both are taken into account again, and ensures it
is now covered by a proper test.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-21 17:10:43 +02:00
Hidde Beydals 79c19adf3f
Merge pull request #453 from dholbach/fix-contributing-sections 2021-10-18 16:52:24 +02:00
Daniel Holbach a7a1d06d83 'installing required deps' should not be under comms
Signed-off-by: Daniel Holbach <daniel@weave.works>
2021-10-18 15:55:45 +02:00
Hidde Beydals c574637c54
Merge pull request #434 from pa250194/gcp-bucket-provider 2021-10-14 23:05:17 +02:00
pa250194 8f0ea2e8af Merge branch 'gcp-bucket-provider' of https://github.com/pa250194/source-controller into gcp-bucket-provider 2021-10-14 14:57:58 -05:00
pa250194 c4e4b3928c Added Logger to closing GCP client
Signed-off-by: pa250194 <pa250194@ncr.com>

Added log for GCP provider auth error

Signed-off-by: pa250194 <pa250194@ncr.com>
2021-10-14 14:57:48 -05:00
pa250194 7921caf056 Updated docs to include GCP provider instructions
Signed-off-by: pa250194 <pa250194@ncr.com>
2021-10-14 14:26:33 -05:00
pa250194 5077c1f9f6 Added more tests and cleaned up GCP provider logic
Signed-off-by: pa250194 <pa250194@ncr.com>
2021-10-14 14:26:33 -05:00
pa250194 c98130548e Added Tests to GCP provider
Signed-off-by: pa250194 <pa250194@ncr.com>
2021-10-14 14:21:15 -05:00
pa250194 99c79bffaa Tests for GCP Bucket Provider
Signed-off-by: pa250194 <pa250194@ncr.com>
2021-10-14 14:19:19 -05:00
pa250194 be1ed50ac4 Service Account Key Authentication to GCP Provider
Signed-off-by: pa250194 <pa250194@ncr.com>
2021-10-14 14:16:53 -05:00
pa250194 2baa8a289c Added Comments for reconcileWithGCP and reconcileWithMinio
Signed-off-by: pa250194 <pa250194@ncr.com>
2021-10-14 14:10:29 -05:00
pa250194 39811ed46a Add Support for GCP storage with workload identity
Added Support for Google Cloud Storage with Workload Identity as
Source Provider. This enables the use of GCP without enabling S3
compatible access.

Signed-off-by: pa250194 <pa250194@ncr.com>
2021-10-14 14:07:11 -05:00
pa250194 38bf4d9859 Fixed spelling and capitalization
Signed-off-by: pa250194 <pa250194@ncr.com>
2021-10-14 13:51:02 -05:00
Hidde Beydals e2548cbe57 Update fluxcd/golang-with-libgit2 to 1.1.1-1
This includes a tiny fix for Darwin to ensure the generated `.pc`
file includes the right paths.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-14 13:49:52 -05:00
Hidde Beydals c2495ae408 Fix generation of API documentation
The version was accidentally set to an invalid version, causing the
API documentation generation to fail.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-14 13:49:52 -05:00
Hidde Beydals 96ab646cd4 Release v0.16.0
Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-14 13:49:52 -05:00
Dylan Arbour 5e6abae9e8 Add ReconcileStrategy to HelmChart
This commit adds a `ReconcileStrategy` field to the `HelmChart` resource, which
allows defining when a new chart should be packaged and/or published if it
originates from a `Bucket` or `GitRepository` resource.

The two available strategies are:

- `ChartVersion`: creates a new artifact when the version of the Helm chart as
  defined in the `Chart.yaml` from the Source is different from the current
  version.
- `Revision`: creates a new artifact when the revision of the Source is
  different from the current revision.

For the `Revision` strategy, the (checksum part of the) revision of the
artifact the chart originatesfrom is added as SemVer metadata.

A chart from a `GitRepository` with Artifact revision
`main/f0faacd5164a875ebdbd9e3fab778f49c5aadbbc` and a chart with e.g. SemVer
`0.1.0` will be published as `0.1.0+f0faacd5164a875ebdbd9e3fab778f49c5aadbbc`.

A chart from a `Bucket` with Artifact revision
`f0faacd5164a875ebdbd9e3fab778f49c5aadbbc` and a chart with e.g. SemVer `0.1.0`
will be published as `0.1.0+f0faacd5164a875ebdbd9e3fab778f49c5aadbbc`.

Signed-off-by: Dylan Arbour <arbourd@users.noreply.github.com>
2021-10-14 13:49:52 -05:00