Commit Graph

45 Commits

Author SHA1 Message Date
Kevin McDermott f7105ea736 Implement Size field on archived artifacts
This adds a Size field to Artifacts, which reflects the number of bytes
written to the artifact when it's being archived.

Signed-off-by: Kevin McDermott <bigkevmcd@gmail.com>
2022-02-28 16:01:34 +01:00
Sunny 234b7f4c9d
Remove redundant reconciling in reconcileArtifact
reconcileSource() adds reconciling condition with accurate information.
Remove setting reconciling condition in reconcileArtifact().

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-02-25 20:39:24 +05:30
Sunny e9ae0c232e Add kstatus client conformance tests
Use kstatus to compute the status of the objects.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-02-23 12:37:45 +01:00
Sunny acda998150 gitrepo: Use commit msg in NewArtifact message
Use commit message in the NewArtifact event message to make it more user
friendly.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-02-23 12:35:30 +01:00
Sunny d997876b07 Make generic SummarizeAndPatch()
summarizeAndPatch() was used by all the reconcilers with their own
object type. This creates a generic SummarizeAndPatch helper that takes
a conditions.Setter object and performs the same operations. All the
reconcilers are updated to use SummarizeAndPatch(). The process of
summarize and patch can be configured using the HelperOptions.

Introduce ResultProcessor to allow injecting middlewares in the
SummarizeAndPatch process.

Introduce RuntimeResultBuilder to allow defining how the reconciliation
result is computed for specific reconciler. This enabled different
reconcilers to have different meanings of the reconciliation results.

Introduce Conditions in summary package to store all the status
conditions related information of a reconciler. This is passed to
SummarizeAndPatch() to be used for summary and patch calculation.

Remove all the redundant summarizeAndPatch() tests per reconciler.

Add package internal/object containing helpers for interacting with
runtime.Object needed by the generic SummarizeAndPatch().

Add tests for ComputeReconcileResult().

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-02-23 12:35:30 +01:00
Hidde Beydals 849f7eeda3 controllers: replace `logr.NewContext` shims
Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-02-23 12:35:30 +01:00
Sunny c4fa79c85e gitrepo: Fix reconcileInclude() includes
The artifacts built in reconcileInclude() should be persisted by writing
it to includes. reconcileArtifact() later adds includes to the git repo
object status and persists it.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-02-23 12:35:30 +01:00
Sunny 2acb721c35 gitrepo: Use internal/util for creating temp dir
Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-02-23 12:35:30 +01:00
Sunny 5767291b58 gitrepo: Add more reconciler design improvements
- Remove ArtifactUnavailable condition and use Reconciling condition to
  convey the same.
- Make Reconciling condition affect the ready condition.
- Introduce summarizeAndPatch() to calculate the final status conditions
  and patch them.
- Introduce reconcile() to iterate through the sub-reconcilers and
  execute them.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-02-23 12:35:30 +01:00
Sunny 25ae83ad7d gitrepo: Fix SourceVerifiedCondition condition type
SourceVerifiedCondition is a normal condition, remove it from negative
polarity conditions. Add SourceVerifiedCondition in patch option
WithOwnedConditions.

Also, Update the signature of reconcileInclude() to remove include
being passed and overwritten in the first line. Include is available
as part of the passed source object.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-02-23 12:35:30 +01:00
Hidde Beydals 88dc2e6ed6 chore: ensure Git server dir is removed after test
Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-02-23 12:35:30 +01:00
Hidde Beydals a437ed69bd Consolidate condition types into `FetchFailed`
This commit consolidates the `DownloadFailed` and `CheckoutFailed`
Condition types into a new more generic `FetchFailed` type to simplify
the API and observations by consumers.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-02-23 12:35:30 +01:00
Sunny 3d5698856a gitrepo: test reconcileArtifact condtns & symlink
Adds test cases for reconcileArtifact to check if old status
conditions are removed after new artifact is created.
Adds a test case to verify that the latest artifact symlink points to
the created artifact.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-02-23 12:35:30 +01:00
Sunny 59b3e5da5d gitrepo: Add tests for old conditions update
This tests the status conditions update in the gitrepository reconciler.
Given a mix of old status conditions, on a successful reconciliation,
the status condition is set to Ready=True.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-02-23 12:35:30 +01:00
Sunny 7e71185594 gitrepo: reconcileInclude test assertion fixes
Use the created artifact server test storage in reconcileInclude
test's GitRepositoryReconciler and cleanup the created storage.

Fix the test assertions to check the copied artifact directories in
the correct path. Also, update the tests to expect artifacts in the
include `toPath` to exist.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-02-23 12:35:30 +01:00
Hidde Beydals 67bd2cb81c Replace %q in messages with '%s'
Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-02-23 12:35:30 +01:00
Sunny 18180776e7 controllers: Add more tests for reconcileArtifact
Fixes error returned from target path validation check and adds more
test cases for TestGitRepositoryReconciler_reconcileArtifact.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-02-23 12:35:30 +01:00
Sunny b814070bc2 Fixes to gitrepo reconciler tests
NOTE: This should be amended with the previous commit which has
commented out tests.

Update reconcileSource() to work with the test case where no secret is
set. A minimal auth options is created and used for git checkout.

Update TestGitRepositoryReconciler_verifyCommitSignature() to use the
new git.Commit type.

Update TestGitRepositoryReconciler_reconcileSource_checkoutStrategy to
add skipForImplementation for branch commit test case.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-02-23 12:35:30 +01:00
Hidde Beydals 31d2e6d65c Rewrite `GitRepositoryReconciler` to new standards
This commit rewrites the `GitRepositoryReconciler` to new standards,
while implementing the newly introduced Condition types, and trying
to adhere better to Kubernetes API conventions.

More specifically it introduces:

- Implementation of more explicit Condition types to highlight
  abnormalities.
- Extensive usage of the `conditions` subpackage from `runtime`.
- Better and more conflict-resilient (status)patching of reconciled
  objects using the `patch` subpackage from runtime.
- Proper implementation of kstatus' `Reconciling` and `Stalled`
  conditions.
- First (integration) tests that solely rely on `testenv` and do not
  use Ginkgo.

There are a couple of TODOs marked in-code, these are suggestions for
the future and should be non-blocking.
In addition to the TODOs, more complex and/or edge-case test scenarios
may be added as well.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-02-23 12:35:30 +01:00
Hidde Beydals 7c3c14997e Switch to v1beta2 API package
Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-02-23 12:35:25 +01:00
Sunny 649d33ca37
pkg/git: Include commit message and URL in error
go-git: Include the commit message in the returned commit object.
libgit2: Set the URL in the checkout error.

Add new method Commit.ShortMessage() for returning short commit
message.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-02-14 21:34:27 +05:30
Hidde Beydals c814e0fa26 git: remove ', error:' from returned error
As we properly nest errors.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-27 00:43:31 +05:30
Hidde Beydals b7376ce94c gogit: allow checkout of commit without branch
This commit changes the `gogit` behavior for commit checkouts,
now allowing one to reference to just a commit while omitting any
branch reference. Doing this creates an Artifact with a
`HEAD/<commit>` revision.

If both a `branch` and `commit` are defined, the commit is expected
to exist within the branch. This results in a more efficient clone
of just the target branch, and also makes this change backwards
compatible.

Fixes #407
Fixes #315

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-27 00:43:31 +05:30
Hidde Beydals 5a1fcc213b git: standardise commit and (PGP) verification
This commit refactors the previous `Commit` interface into a
standardised `Commit` struct. This object contains sufficient
information for referencing, observating and (PGP) verification.

- `libgit2` commit checkout does now return `HEAD/<SHA1>` as
  the branch is not taken into account.
- `git2go` objects are now properly `Free`d everywhere
- `Verify` logic is tested.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-27 00:43:31 +05:30
Hidde Beydals c4d7e46b90 Drop deprecated `io/ioutil`
The package has been deprecated since Go 1.16, see:
https://golang.org/doc/go1.16#ioutil

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-07-29 09:58:00 +02:00
Stefan Prodan add5444f16
Fix GitRepository include for nested paths
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-05-28 01:17:40 +03:00
Philip Laine fcf7048992 Add include property to GitRepositories
Signed-off-by: Philip Laine <philip.laine@gmail.com>
Signed-off-by: Philip Laine <philip.laine@xenit.se>
2021-05-11 09:46:50 +02:00
Michael Bridgen 681ddd5db0 Test RecurseSubmodules
This commit adds a test specifically for RecurseSubmodules. It takes a
bit more preparation, since it needs a repo using submodules to start
with. go-git doesn't appear to support adding submodules
programmatically, so the preparation is done in part by execing `git`.

Signed-off-by: Michael Bridgen <michael@weave.works>
2021-03-31 12:22:10 +01:00
Stefan Prodan f0016cfad1
Enable self-signed certs for go-git
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-03-29 13:23:32 +03:00
Philip Laine c063484761 Add custom certificate validation
Signed-off-by: Philip Laine <philip.laine@gmail.com>
2021-02-08 12:19:22 +01:00
Philip Laine f044ebc253 Rename git2go option to libgit2
Signed-off-by: Philip Laine <philip.laine@xenit.se>
2020-12-06 23:19:57 +01:00
Philip Laine 1b4eacc588 Refactor argument name to enable git2go
Signed-off-by: Philip Laine <philip.laine@xenit.se>
2020-12-04 10:07:17 +01:00
Philip Laine cb12736a84 Update pkg git server
Signed-off-by: Philip Laine <philip.laine@xenit.se>
2020-12-03 12:56:38 +01:00
Philip Laine 47e4e54550 Add test to show that libgit2 fails with self signed test
Signed-off-by: Philip Laine <philip.laine@gmail.com>
2020-12-02 23:28:32 +01:00
Aurel Canciu 00bb853d0e
Refactor to adopt k8s standardized Condition type
Updates to use metav1.Condition type and removes references for
deprecated corev1.Condition* constants and uses the new k8s api/meta
helpers in place of the old pkg/apis/meta types.

Signed-off-by: Aurel Canciu <aurelcanciu@gmail.com>
2020-11-19 13:08:33 +02:00
Hidde Beydals c9eb8f03c0 Switch to Masterminds/semver and pkg/version libs
Co-authored-by: Illia Ovchynnikov <illia.ovchynnikov@gmail.com>
Signed-off-by: Hidde Beydals <hello@hidde.co>
2020-10-28 12:21:08 +01:00
Hidde Beydals 582c2092b6 Change copyright to Flux authors
Signed-off-by: Hidde Beydals <hello@hidde.co>
2020-10-27 17:54:22 +01:00
stefanprodan f16e5f041a Promote API to v1beta1 2020-09-30 16:11:00 +03:00
Hidde Beydals 431ea05427 Implement `fluxcd/pkg/meta/api` in APIs 2020-09-30 10:18:43 +02:00
Hidde Beydals 3475d741a3 Switch to dedicated git and helm testserver pkgs 2020-08-18 12:11:29 +02:00
Hidde Beydals 9d947b8efa Use testserver package from fluxcd/pkg 2020-08-18 12:11:29 +02:00
Hidde Beydals 0bc28f7a42 testserver: move from internal to pkg 2020-06-30 11:48:14 +02:00
Hidde Beydals 24b77d37a8 controllers: GitRepository ref 2020-05-29 14:24:31 +02:00
Hidde Beydals b27c82dd1b controllers: GitRepository test refs 2020-05-29 14:24:31 +02:00
Hidde Beydals 77822bd4d1 controllers: simple GitRepository test 2020-05-29 14:24:31 +02:00