Commit Graph

157 Commits

Author SHA1 Message Date
Paulo Gomes a980b894e0
Block the creation of empty commits
In specific cases, the update mechanism may lead to no files being
changed. The changes short-circuit the process to exit early when
0 files have been modified.

go-git has been changed to return an error when an empty commit
is detected, which is used as a last resort.

When the push branch is different than the clone branch, the controller
will now download all branch references from the Git server, in order
to enable a single fetch operation.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-12-20 10:11:22 +00:00
Paulo Gomes 5048e38508
features: Remove ForceGoGitImplementation
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-12-09 13:43:24 +00:00
Paulo Gomes 7a05f4856c
libgit2: Remove references to libgit2 from code
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-12-09 11:30:55 +00:00
Paulo Gomes 38d76cc407
libgit2: Migrate tests to go-git
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-12-09 11:27:51 +00:00
Paulo Gomes e846e29458
Add GitShallowClone feature
This feature enables the use of shallow clones when pulling source from
Git repositories.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-12-06 15:22:34 +00:00
Paulo Gomes 5fefee34b7
build: Improve fuzz tests' reliability
Establish conventions which aligns with what is supported upstream
today, whilst expanding on documentation to ensure folks have
pointers on how to debug/check for issues going forwards.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-11-23 15:48:14 +00:00
Sanskar Jaiswal 1cacafa5fc bump pkg/git, pkg/git/gogit and pkg/git/libgit2
* Bump `pkg/git` to `v0.7.0`
* Bump `pkg/git/gogit` to `v0.2.0`
* Bump `pkg/git/libgit2` to `v0.3.0`

Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
2022-11-16 20:00:26 +05:30
Stefan Prodan cad8850c11
Use Flux Event API v1beta1
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-11-09 12:11:43 +02:00
Paulo Gomes 0191d6b4cc
gogit: Add new ForceGoGitImplementation FeatureGate
ForceGoGitImplementation ignores the value set for gitImplementation
and ensures that go-git is used for all GitRepository objects.
This can be used to confirm that Flux instances won't break if/when
the libgit2 implementation was to be deprecated.

When enabled, libgit2 won't be initialized, nor will any git2go cgo
code be called.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-11-04 11:16:26 +00:00
Paulo Gomes 842c6d78c4
Add support for Git force push
Automations using a different push branch can now
enable force push to ensure that it gets refreshed
with the latest version of the branch used for the
clone operation.

Initially this was going to be added in combination
with Shallow clones for the `go-git` implementation.
However, force push is required for feature completeness
whilst Shallow clones are mostly an optimisation, and
will therefore be implemented at a later stage.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-11-02 13:35:36 +00:00
Paulo Gomes b00625bffc
Remove redundant repoAccess
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-11-01 12:42:48 +00:00
Sanskar Jaiswal 7c6a117523
use gitrepo implementation for git ops
Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
2022-10-27 14:43:03 +01:00
Paulo Gomes 26bd330f18
build: Bump golang-with-libgit2 to v0.4.0
The new version uses libgit2 1.5.0 and requires git2go/v34.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-10-27 14:43:02 +01:00
Sanskar Jaiswal 0296f7acf0
refactor reconciler to use fluxcd/pkg/git
Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
2022-10-27 14:42:13 +01:00
Stefan Prodan 79ca3ceec1
Build with Go 1.19
- Update Go to 1.19 in CI
- Use Go 1.19 in base image
- Update Alpine to 3.16 in the final image
- Update controller-gen v0.8.0 (v0.7 fails with Go 1.19) and regenerate manifests

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-09-28 18:38:55 +03:00
Paulo Gomes ae7c00a15a
fuzz: Refactor Fuzzers based on Go native fuzzing
The existing fuzzers were converted into the Go native format.
Based on how the code was structured on this project, the fuzzers
can be quite effective, allowing for entire E2E fuzzing in some
cases, but with very low execution cost.

The previous controller fuzzer was converted but will need
to be refactored once the controller has been rewritten.
Note that due to the libgit2 dependencies a copy of
compile_native_go_fuzzer had to be made and modified so
that we can inject libgit2 during the linking process.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-09-08 20:50:12 +01:00
Paulo Gomes 06f4acd118
Use helper.Metrics for recording metrics
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-09-05 14:03:08 +01:00
Paulo Gomes 735ab61911
Log when no changes were made
This gets the behaviour a step closer to Source Controller
which logs after every reconciliation, even when no changes
were made.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-09-05 14:02:51 +01:00
Pierre-Barthelemy Fiocconi 3c06e94877
Fix fetch error in push branch
Fixes error when push branch and reference branch are equal
by skipping switch to push branch

Signed-off-by: Pierre-Barthelemy Fiocconi <pierre-barthelemy.fi@doctolib.com>
2022-08-25 12:01:06 +02:00
Paulo Gomes 0fdcf7fe6d
Enable RecoverPanic
The controller-runtime provides the ability to recover from
panics that have arisen from a reconciliation. This change
enables this functionality by default.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-08-10 09:59:19 +01:00
Paulo Gomes 4a8ddfad49
git: Decommission libgit2 Unmanaged Transport
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-08-09 13:47:48 +01:00
Paulo Gomes 3990e9c828
Remove MUSL and enable threadless libgit2 support
Use of MUSL was a temporary solution to mitigate cross-platform
issues while building openssl and libssh2. Since Unmanaged transport has
been deprecated, openssl and libssh2 dependencies are no longer required
and by extension MUSL.

Enables libgit2 threadless support and provides a regression assurance
for #339.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-07-31 20:06:46 +01:00
Paulo Gomes f51050e8c0
Enrich 'early EOF' error message
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-07-26 11:17:12 +01:00
Stefan Prodan 0d43488ba6
Fix flaky tests by waiting for finalizers
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-06-24 12:54:23 +03:00
Stefan Prodan 34413b041f
Add finalizer to ImageUpdateAutomation resources
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-06-24 11:46:33 +03:00
Paulo Gomes ec89ad41b9
Update source-controller to v0.25.7
This version of source-controller introduces a fix for the
SSH connections leak issue reported at:
https://github.com/fluxcd/image-automation-controller/issues/334

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-06-22 15:55:50 +05:30
Paulo Gomes a996462048
Fix subtest references
Subtests are not supposed to use the parent t reference
but rather to use their own. This change ensures that
is the case.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-06-16 14:12:56 +01:00
Sunny 9e560841c2
Update source-controller with libgit2 race fixes
Also, removes the panic recovery code added in cloneInto() as the
underlying Checkout() in SC already has a panic handler. The issues were
due to goroutine panicking, which are now handled in the SC libgit
managed transport.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-06-03 02:59:05 +05:30
Sanskar Jaiswal 175f91ea0e recover from panics in cloneInto
Signed-off-by: Sanskar Jaiswal <sanskar.jaiswal@weave.works>
2022-06-02 19:23:56 +05:30
Sanskar Jaiswal 61f76d3f6d use context for cloning in tests
Signed-off-by: Sanskar Jaiswal <sanskar.jaiswal@weave.works>
2022-06-02 19:10:08 +05:30
Sanskar Jaiswal 5ee6446764 fix regression in switchToBranch
Fixes regression in which we fail to push to a branch after switching to
a branch, if origin is ahead of local. Fixed by setting the upstream
commit as the local branch target.

Regression introduced in #330, and partially addressed in #369.

Signed-off-by: Sanskar Jaiswal <sanskar.jaiswal@weave.works>
2022-06-02 19:09:01 +05:30
Sanskar Jaiswal 74420bd1a7 refactor controller and git tests to use managed transport
Signed-off-by: Sanskar Jaiswal <sanskar.jaiswal@weave.works>
2022-06-02 15:52:55 +05:30
Paulo Gomes ebc40b2d88
Fix regression on push branches
Fixes regression in which IAC will fail to update push branch
when the push branch already exists and checkout branch is
ahead.

The reconciliation errors with: 'cannot push because a reference
that you are trying to update on the remote contains commits
that are not present locally.'

Regression introduced on #330.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-06-01 21:49:04 +03:00
Paulo Gomes 4f044661fd
Update source-controller
- Panic recovery for Git operations.
- Improved SSH connection management without use of caching.
- Enforce context timeout for managed SSH.
- Remove dependency to callback functions.
- Add support for hashed known_hosts.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-06-01 21:04:12 +03:00
Sunny 0e87398e59
Remove legacy_suite_test.go and ginkgo deps
Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-04-26 01:22:21 +05:30
Sunny 37022d0c88
Refactor update_test.go to use testenv
Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-04-26 01:22:14 +05:30
Sunny 985d879e31
Move ginkgo suite to legacy & add testenv
Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-04-21 18:59:05 +05:30
Paulo Gomes 4a53f9defa
Add flags to configure exponential back-off retry
Add two new flags to enable users to configure exponential
back-off for Flux objects. The default values are now
set to 750ms for minimum retry time, and 15min for max.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-04-12 13:01:36 +01:00
Paulo Gomes 859a6ca93a
Update source-controller to v0.22.4
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-03-28 15:22:19 +01:00
Paulo Gomes 4b35fe3ee8
Fix bug when pushing into different branches
Changes replacing go-git with git2go introduced a bug
in which pushes into new branches squashes all commits
into one.

xref: https://github.com/fluxcd/image-automation-controller/pull/324
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-03-23 19:32:24 +00:00
Paulo Gomes 5b15bb7f94
Implement Managed Transport for libgit2
libgit2 network operations are blocking and do not provide timeout nor context capabilities,
leading to several reports of the controllers hanging indefinitely.

By using managed transport, golang primitives such as http.Transport and net.Dial can be used
to ensure timeouts are enforced.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-03-21 15:19:00 +00:00
Paulo Gomes 4d8ebe69a0 Ensure libgit2 resources are released
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-03-21 13:06:38 +01:00
Sunny 146a86865c Use new standardized runtime and meta package
Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-03-21 12:45:55 +01:00
Paulo Gomes 1a52581356
Remove direct dependency to go-git
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-03-04 11:10:22 +00:00
Paulo Gomes d20cdb532f
Refactor initGitRepo to use libgit2 instead of go-git
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-03-01 14:48:15 +00:00
Paulo Gomes 4ab219e64b
Fix tests after upgrading to libgit2 1.3.0
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-03-01 14:48:13 +00:00
Paulo Gomes b898759d26
Update libgit2 to 1.3.0 and update source-controller to v0.21.3
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-03-01 14:48:12 +00:00
Somtochi Onyekwere 0ca01bd254 Add flag to disable cross-namespace referencing
Signed-off-by: Somtochi Onyekwere <somtochionyekwere@gmail.com>
2022-01-28 08:32:21 +01:00
Aurel Canciu aa3b6cc4b7
Update kyaml to v0.13.0
Updating kyaml to match the version we use in other components. This
version's most significant change for us would be that kyaml will no
longer override indentations in the targeted files.

Signed-off-by: Aurel Canciu <aurelcanciu@gmail.com>
2022-01-26 15:56:25 +01:00
Sanskar Jaiswal 3de51e7a1e add support for cross-namespace sourceRef in ImageUpdateAutomation
ImageUpdateAutomation objects can now refer to GitRepository objects in other
namespaces. Implemented by switching sourceRef from a SourceReference to a
dependency.CrossNamespaceDependencyReference.

Signed-off-by: Sanskar Jaiswal <sanskar.jaiswal@weave.works>
2022-01-25 14:21:37 +05:30