Commit Graph

1156 Commits

Author SHA1 Message Date
Sunny 6789aaf147 api: Embed runtime.Object in Source interface
Embedding runtime.Object in Source interface makes the Source type more
useful to interact with k8s API machinery.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-02-23 12:35:30 +01:00
Sunny dc9d8b7f66 Add gomega matcher for artifact
Signed-off-by: Sunny <darkowlzz@protonmail.com>

Co-authored-by: Hidde Beydals <hello@hidde.co>
2022-02-23 12:35:30 +01:00
Sunny b1233dc24f Move Artifact conditions to conditions
Also, introduce FetchFailedCondition for generic fetch failures.

Signed-off-by: Sunny <darkowlzz@protonmail.com>

Co-authored-by: Hidde Beydals <hello@hidde.co>
2022-02-23 12:35:30 +01:00
Sunny a1efbad15a Use new events and metrics helpers in main.go
Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-02-23 12:35:30 +01:00
Sunny 9eb6833d4d source: Add `GetRequeueAfter`
The problem with `GetInterval()` was that the returned type was of
`metav1.Duration`, while almost anywhere it was used, a type of
`time.Duration` was requested. The result of this was that we had to
call `GetInterval().Duration` all the time, which would become a bit
cumbersome after awhile.

To prevent this, we introduce a new `GetRequeueAfter() time.Duration`
method, which both results the right type, and bears a name that is
easier to remember where the value is used most; while setting the
`Result.RequeueAfter` during reconcile operations.

The introduction of this method deprecates `GetInterval()`, which
should be removed in a future MINOR release.

Signed-off-by: Sunny <darkowlzz@protonmail.com>

Co-authored-by: Hidde Beydals <hello@hidde.co>
2022-02-23 12:35:30 +01:00
Hidde Beydals e190059cc7 Introduce `artifactSet` to replace `hasArtifactUpdated`
NOTE: Remove `hasArtifactUpdated` in the future once it's no longer
used.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-02-23 12:35:30 +01:00
Hidde Beydals 349739b7e4 Implement new runtime interfaces, prepare testenv
This commit ensures all API objects implement the interfaces used by
the runtime package to work with conditions, etc., and prepares the
test suite to work with the `pkg/runtime/testenv` wrapper.

Changes are made in a backwards compatible way (that being: the
existing code can still be build and works as expected), but without
proper dependency boundaries. The result of this is that the API
package temporary depends on the runtime package, which is resolved
when all reconcilers have been refactored and the API package does
no longer contain condition modifying functions.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-02-23 12:35:30 +01:00
Hidde Beydals e42eedd392 Introduce more explicit Condition types
This commit introduces new Condition types to the v1beta1 API,
facilitating easier observation of (potentially) problematic state for
end-users.

- `ArtifactUnavailableCondition`: indicates there is no artifact
  available for the resource. This Condition should be set by the
  reconciler as soon as it observes the absence of an artifact for a
  source.
- `CheckoutFailedCondition`: indicates a transient or persistent
  checkout failure. This Condition should be set by the reconciler as
  soon as it observes a Git checkout failure, including any
  prerequisites like the unavailability of the referenced Secret used
  for authentication. It should be deleted as soon as a successful
  checkout has been observed again.
- `SourceVerifiedCondition`: indicates the integrity of the source has
  been verified. The Condition should be set to True or False by the
  reconciler based on the result of the integrity check.
  If there is no verification mode and/or secret configured, the
  Condition should be removed.
- `IncludeUnavailableCondition`: indicates one of the referenced
  includes is not available. This Condition should for example be set
  by the reconciler when the include does not exist, or does not have
  an artifact. If the includes become available, it should be deleted.
- `ArtifactOutdatedCondition`: indicates the current artifact of the
  source is outdated. This Condition should for example be set by the
  reconciler when it notices there is a newer revision for an artifact,
  or the previously included artifacts differ from the current available
  ones. The Condition should be removed after writing a new artifact
  to the storage.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-02-23 12:35:30 +01:00
Hidde Beydals 7c3c14997e Switch to v1beta2 API package
Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-02-23 12:35:25 +01:00
Sunny cc1e48243d Introduce v1beta2 API package
This commit introduces a v1beta2 API package for the staged breaking
changes around conditions and general usage of the API objects.

Signed-off-by: Hidde Beydals <hello@hidde.co>
Signed-off-by: Sunny <darkowlzz@protonmail.com>
Co-authored-by: Hidde Beydals <hello@hidde.co>
2022-02-23 12:34:35 +01:00
Hidde Beydals bd3d7817d0
Merge pull request #589 from fluxcd/libgit2-branch-checkout 2022-02-22 17:15:37 +01:00
Hidde Beydals eff40e22e9 git/libgit2: assert proper test of default branch
If there is no configuration set for `init.defaultBranch`, it does not
return an error but an empty string. We now take this into account so
we do not overwrite the default, and make the default `master` to match
with libgit2 defaults.

In addition, some comments have been added to not get confused about
what commits we are checking against.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-02-22 16:31:56 +01:00
Hidde Beydals 15c064abdf git/libgit2: set CheckoutForce on branch strategy
In the recent update from libgit2 1.1.x to 1.3.x, something seems to
have changed upstream. Resulting in the clone of a branch ending up
with a semi-bare file system state (in other words: without any files
present in the directory).

This commit patches the clone behavior to set the `CheckoutForce`
strategy as `CheckoutOption`, which mitigates the issue.

In addition, test cases have been added to ensure we do not run into
this again by asserting the state of the branch after cloning.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-02-22 16:31:56 +01:00
Paulo Gomes b65cf9d535
Merge pull request #583 from pjbgf/controller-runtime
Upgrade controller-runtime to v0.11.1 and docker/distribution to v2.8.0
2022-02-18 09:12:27 +00:00
Paulo Gomes 6e46d7fe55
Upgrade docker/distribution to v2.8.0
Fixes https://github.com/advisories/GHSA-qq97-vm5h-rrhg

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-02-17 20:51:48 +00:00
Paulo Gomes 00ff9fb2fa
Upgrade controller-runtime to v0.11.1
Fix for CVE-2022-21698 by upgrading the trasient dependency github.com/prometheus/client_golang.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-02-17 20:51:47 +00:00
Stefan Prodan 35d785ea62
Merge pull request #584 from pjbgf/maintainers
Add pjbgf to Maintainers
2022-02-17 13:42:01 +02:00
Paulo Gomes ec89a2d067
Add pjbgf to Maintainers
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-02-17 11:24:02 +00:00
Hidde Beydals 2c4818fda5
Merge pull request #573 from pjbgf/bump-libgit2 2022-02-16 13:55:42 +01:00
Paulo Gomes 8429708997
Upgrade libgit2 to libgit2-1.3.0-2
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-02-16 11:39:11 +00:00
Paulo Gomes f0d7a6bb48
Update libgit2 attributions
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-02-16 10:30:27 +00:00
Paulo Gomes 514126c4b8
Fix make test on arm64 runners
The environment variables set at the Makefile were causing go install to
yield a corrupted file for setup-envtest. To fix the issue, such operation
is now always executed in a clean bash.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-02-16 10:17:33 +00:00
Paulo Gomes e5d032fe9c
Add libgit2 checkout test with ED25519 key
This adds a test to detect any regression in libgit2's ED25519 key
support. go-git supports ED25519 but not the current version of
libgit2 used in flux. The updates to libgit2 in v1.2.0 adds support
for ED25519. This test would help ensure the right version of libgit2
is used.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-02-16 10:17:32 +00:00
Paulo Gomes f44302eed0
Bump libgit2 to 1.3.0 and git2go to V33.
Downstream breaking changes introduced since git2go@V31:
- git2go.ErrorCode was deprecated in favour of the native error type.
- FetchOptions no longer expects a pointer, but rather the actual value of git2go.FetchOptions.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-02-16 10:17:31 +00:00
Hidde Beydals 1dae5e7621
Merge pull request #580 from pjbgf/fuzz-san 2022-02-16 11:14:52 +01:00
Paulo Gomes db0f033409
Add support for multiple fuzz sanitizers
The oss-fuzz repository contains the configuration for fluxcd supported sanitizers.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-02-15 16:57:38 +00:00
Sunny 69ae21be63
Merge pull request #579 from fluxcd/git-commit-msg-url
pkg/git: Include commit message and URL in checkout error
2022-02-14 21:57:21 +05:30
Sunny 649d33ca37
pkg/git: Include commit message and URL in error
go-git: Include the commit message in the returned commit object.
libgit2: Set the URL in the checkout error.

Add new method Commit.ShortMessage() for returning short commit
message.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-02-14 21:34:27 +05:30
Hidde Beydals af0226b90c
Merge pull request #577 from pjbgf/fix-fuzz 2022-02-14 15:46:18 +01:00
Paulo Gomes 03c56e79a0
Download libgit2 libraries for fuzzing
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-02-11 12:30:46 +00:00
Stefan Prodan 3fc70ff658
Merge pull request #574 from pjbgf/fix-push
Validate LIBGIT2 args are set correctly
2022-02-09 18:55:37 +02:00
Paulo Gomes fcb62eedc7
Validate LIBGIT2 args are set correctly
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-02-09 15:52:46 +00:00
Hidde Beydals 657f80bf6c
Merge pull request #572 from pjbgf/new-fuzz 2022-02-09 11:39:55 +01:00
Paulo Gomes cd6d33c101
Increase gingko timeout to 60s
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-02-08 17:49:51 +00:00
Paulo Gomes d8da814d2e
Initial fuzzing tests
This PR refactors and supersedes: https://github.com/fluxcd/source-controller/pull/443

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
Co-authored-by: Sanskar Jaiswal <sanskar.jaiswal@weave.works>
Co-authored-by: AdamKorcz <adam@adalogics.com>
2022-02-08 17:49:34 +00:00
Hidde Beydals e0d034440a
Merge pull request #562 from pjbgf/test-alpine 2022-02-08 13:21:33 +01:00
Paulo Gomes 80e1d243e6
Upgrade to libgit2-1.1.1-6
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-02-08 10:51:20 +00:00
Paulo Gomes c5e2e5e30b
Add make test execution for macos-10.15
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-02-07 18:36:18 +00:00
Paulo Gomes fa00ec8fc7
Migrate from deprecated ginkgo async testing
https://github.com/onsi/ginkgo/blob/ver2/docs/MIGRATING_TO_V2.md\#removed-async-testing

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-02-07 15:08:03 +00:00
Paulo Gomes 21e71c3750
Set u+w to envtest folders
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-02-07 13:00:58 +00:00
Paulo Gomes 60c98b5471
Update libgit2 to 1.1.1-6
Fix issues developing in amd64, arm64 and apple silicon

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-02-07 13:00:57 +00:00
Paulo Gomes 62928217a0
Optimise cross compilation time
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-02-07 13:00:56 +00:00
Paulo Gomes 990b4a361b
Use static libraries from built image
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-02-07 13:00:55 +00:00
Stefan Prodan ddc6cfeba0
Merge pull request #571 from fluxcd/release-v0.21.2
Release v0.21.2
2022-02-07 12:12:23 +02:00
Stefan Prodan e3e9deb3fd
Release v0.21.2
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-02-07 11:50:32 +02:00
Stefan Prodan 5aec67b7f1
Merge pull request #570 from fluxcd/default-timeout
Increase default timeout to 60s
2022-02-07 11:43:55 +02:00
Stefan Prodan fb45c9caf6
Increase default timeout to 60s
Increase the default timeout from 20s to 60s for Git repository cloning and Bucket download

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-02-07 10:29:43 +02:00
Hidde Beydals f1e69d1f26
Merge pull request #563 from fluxcd/release-0.21.1 2022-01-27 11:51:58 +01:00
Hidde Beydals 01946f22a3 Release v0.21.1
Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-01-27 11:16:21 +01:00
Hidde Beydals 93c636e42c
Merge pull request #561 from pjbgf/fix-host 2022-01-26 18:56:29 +01:00