Commit Graph

143 Commits

Author SHA1 Message Date
Stefan Prodan 79ca3ceec1
Build with Go 1.19
- Update Go to 1.19 in CI
- Use Go 1.19 in base image
- Update Alpine to 3.16 in the final image
- Update controller-gen v0.8.0 (v0.7 fails with Go 1.19) and regenerate manifests

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-09-28 18:38:55 +03:00
Paulo Gomes ae7c00a15a
fuzz: Refactor Fuzzers based on Go native fuzzing
The existing fuzzers were converted into the Go native format.
Based on how the code was structured on this project, the fuzzers
can be quite effective, allowing for entire E2E fuzzing in some
cases, but with very low execution cost.

The previous controller fuzzer was converted but will need
to be refactored once the controller has been rewritten.
Note that due to the libgit2 dependencies a copy of
compile_native_go_fuzzer had to be made and modified so
that we can inject libgit2 during the linking process.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-09-08 20:50:12 +01:00
Paulo Gomes 06f4acd118
Use helper.Metrics for recording metrics
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-09-05 14:03:08 +01:00
Paulo Gomes 735ab61911
Log when no changes were made
This gets the behaviour a step closer to Source Controller
which logs after every reconciliation, even when no changes
were made.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-09-05 14:02:51 +01:00
Pierre-Barthelemy Fiocconi 3c06e94877
Fix fetch error in push branch
Fixes error when push branch and reference branch are equal
by skipping switch to push branch

Signed-off-by: Pierre-Barthelemy Fiocconi <pierre-barthelemy.fi@doctolib.com>
2022-08-25 12:01:06 +02:00
Paulo Gomes 0fdcf7fe6d
Enable RecoverPanic
The controller-runtime provides the ability to recover from
panics that have arisen from a reconciliation. This change
enables this functionality by default.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-08-10 09:59:19 +01:00
Paulo Gomes 4a8ddfad49
git: Decommission libgit2 Unmanaged Transport
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-08-09 13:47:48 +01:00
Paulo Gomes 3990e9c828
Remove MUSL and enable threadless libgit2 support
Use of MUSL was a temporary solution to mitigate cross-platform
issues while building openssl and libssh2. Since Unmanaged transport has
been deprecated, openssl and libssh2 dependencies are no longer required
and by extension MUSL.

Enables libgit2 threadless support and provides a regression assurance
for #339.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-07-31 20:06:46 +01:00
Paulo Gomes f51050e8c0
Enrich 'early EOF' error message
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-07-26 11:17:12 +01:00
Stefan Prodan 0d43488ba6
Fix flaky tests by waiting for finalizers
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-06-24 12:54:23 +03:00
Stefan Prodan 34413b041f
Add finalizer to ImageUpdateAutomation resources
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-06-24 11:46:33 +03:00
Paulo Gomes ec89ad41b9
Update source-controller to v0.25.7
This version of source-controller introduces a fix for the
SSH connections leak issue reported at:
https://github.com/fluxcd/image-automation-controller/issues/334

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-06-22 15:55:50 +05:30
Paulo Gomes a996462048
Fix subtest references
Subtests are not supposed to use the parent t reference
but rather to use their own. This change ensures that
is the case.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-06-16 14:12:56 +01:00
Sunny 9e560841c2
Update source-controller with libgit2 race fixes
Also, removes the panic recovery code added in cloneInto() as the
underlying Checkout() in SC already has a panic handler. The issues were
due to goroutine panicking, which are now handled in the SC libgit
managed transport.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-06-03 02:59:05 +05:30
Sanskar Jaiswal 175f91ea0e recover from panics in cloneInto
Signed-off-by: Sanskar Jaiswal <sanskar.jaiswal@weave.works>
2022-06-02 19:23:56 +05:30
Sanskar Jaiswal 61f76d3f6d use context for cloning in tests
Signed-off-by: Sanskar Jaiswal <sanskar.jaiswal@weave.works>
2022-06-02 19:10:08 +05:30
Sanskar Jaiswal 5ee6446764 fix regression in switchToBranch
Fixes regression in which we fail to push to a branch after switching to
a branch, if origin is ahead of local. Fixed by setting the upstream
commit as the local branch target.

Regression introduced in #330, and partially addressed in #369.

Signed-off-by: Sanskar Jaiswal <sanskar.jaiswal@weave.works>
2022-06-02 19:09:01 +05:30
Sanskar Jaiswal 74420bd1a7 refactor controller and git tests to use managed transport
Signed-off-by: Sanskar Jaiswal <sanskar.jaiswal@weave.works>
2022-06-02 15:52:55 +05:30
Paulo Gomes ebc40b2d88
Fix regression on push branches
Fixes regression in which IAC will fail to update push branch
when the push branch already exists and checkout branch is
ahead.

The reconciliation errors with: 'cannot push because a reference
that you are trying to update on the remote contains commits
that are not present locally.'

Regression introduced on #330.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-06-01 21:49:04 +03:00
Paulo Gomes 4f044661fd
Update source-controller
- Panic recovery for Git operations.
- Improved SSH connection management without use of caching.
- Enforce context timeout for managed SSH.
- Remove dependency to callback functions.
- Add support for hashed known_hosts.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-06-01 21:04:12 +03:00
Sunny 0e87398e59
Remove legacy_suite_test.go and ginkgo deps
Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-04-26 01:22:21 +05:30
Sunny 37022d0c88
Refactor update_test.go to use testenv
Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-04-26 01:22:14 +05:30
Sunny 985d879e31
Move ginkgo suite to legacy & add testenv
Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-04-21 18:59:05 +05:30
Paulo Gomes 4a53f9defa
Add flags to configure exponential back-off retry
Add two new flags to enable users to configure exponential
back-off for Flux objects. The default values are now
set to 750ms for minimum retry time, and 15min for max.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-04-12 13:01:36 +01:00
Paulo Gomes 859a6ca93a
Update source-controller to v0.22.4
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-03-28 15:22:19 +01:00
Paulo Gomes 4b35fe3ee8
Fix bug when pushing into different branches
Changes replacing go-git with git2go introduced a bug
in which pushes into new branches squashes all commits
into one.

xref: https://github.com/fluxcd/image-automation-controller/pull/324
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-03-23 19:32:24 +00:00
Paulo Gomes 5b15bb7f94
Implement Managed Transport for libgit2
libgit2 network operations are blocking and do not provide timeout nor context capabilities,
leading to several reports of the controllers hanging indefinitely.

By using managed transport, golang primitives such as http.Transport and net.Dial can be used
to ensure timeouts are enforced.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-03-21 15:19:00 +00:00
Paulo Gomes 4d8ebe69a0 Ensure libgit2 resources are released
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-03-21 13:06:38 +01:00
Sunny 146a86865c Use new standardized runtime and meta package
Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-03-21 12:45:55 +01:00
Paulo Gomes 1a52581356
Remove direct dependency to go-git
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-03-04 11:10:22 +00:00
Paulo Gomes d20cdb532f
Refactor initGitRepo to use libgit2 instead of go-git
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-03-01 14:48:15 +00:00
Paulo Gomes 4ab219e64b
Fix tests after upgrading to libgit2 1.3.0
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-03-01 14:48:13 +00:00
Paulo Gomes b898759d26
Update libgit2 to 1.3.0 and update source-controller to v0.21.3
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-03-01 14:48:12 +00:00
Somtochi Onyekwere 0ca01bd254 Add flag to disable cross-namespace referencing
Signed-off-by: Somtochi Onyekwere <somtochionyekwere@gmail.com>
2022-01-28 08:32:21 +01:00
Aurel Canciu aa3b6cc4b7
Update kyaml to v0.13.0
Updating kyaml to match the version we use in other components. This
version's most significant change for us would be that kyaml will no
longer override indentations in the targeted files.

Signed-off-by: Aurel Canciu <aurelcanciu@gmail.com>
2022-01-26 15:56:25 +01:00
Sanskar Jaiswal 3de51e7a1e add support for cross-namespace sourceRef in ImageUpdateAutomation
ImageUpdateAutomation objects can now refer to GitRepository objects in other
namespaces. Implemented by switching sourceRef from a SourceReference to a
dependency.CrossNamespaceDependencyReference.

Signed-off-by: Sanskar Jaiswal <sanskar.jaiswal@weave.works>
2022-01-25 14:21:37 +05:30
Aurélien GARNIER 9b82e81af0 Use libgit2 auto proxy configuration
Signed-off-by: Aurélien GARNIER <aurelien.garnier@atos.net>
2022-01-20 17:14:18 +01:00
Sanskar Jaiswal 942e980616 fix makefile envtest and controller-gen usage
Refactor logic to install helper tools into one function in the
Makefile. Add support for envtest to help install tools like kubectl,
etcd which helps users run tests more conveniently.

Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
2022-01-13 16:46:12 +05:30
Luke Mallon (Nalum) c026e67484
[refactor] Check for a nil lastTime before use
Signed-off-by: Luke Mallon (Nalum) <luke@mallon.ie>
2022-01-11 12:31:02 +00:00
Aurel Canciu 24594ebf20
Update flux pkg components
Signed-off-by: Aurel Canciu <aurelcanciu@gmail.com>
2022-01-07 16:53:29 +01:00
Michael Bridgen 8e1f31fe12 Log the error when removing the working dir fails
os.RemoveAll can return an error; even if we're exiting the procedure
at the time, it's worth knowing when it's failing, because typically
the working directory is a memory-backed volume and you can run out
quite easily.

Signed-off-by: Michael Bridgen <michael@weave.works>
2022-01-05 17:55:06 +00:00
Luke Mallon (Nalum) f75315e44d
[refactor] Remove else as redundant in this use
Signed-off-by: Luke Mallon (Nalum) <luke@mallon.ie>
2021-12-21 15:58:58 +00:00
Luke Mallon (Nalum) 864ad1182e
[refactor] Move Path check into switch case
Signed-off-by: Luke Mallon (Nalum) <luke@mallon.ie>
2021-12-20 16:20:21 +00:00
Sunny 16690b3fb5 Update tests to be compatible with CR > v0.10.0
Also updates the tests to be compatible with the breaking changes in
controller-runtime > v0.10.0.

Cherry-picked from 467d97a9bf.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2021-11-22 17:15:56 +01:00
Stefan Prodan bfcb9ef8db
Remove deprecated io/ioutil
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-11-22 16:08:28 +02:00
Stefan Prodan 469988105d
Add the commit message to the event body
Drop "no updates made" event leave only the debug log in place.

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-11-09 12:26:52 +02:00
Aurel Canciu fe40819140
Fix unhandled error in signing key retrieval
Fixes #242

Signed-off-by: Aurel Canciu <aurelcanciu@gmail.com>
2021-11-04 22:12:10 +01:00
Sunny b299a9db59 Pass context to libgit2.RemoteCallbacks
Pass cancellable context to libgit2.RemoteCallbacks to be able to cancel
the remote operations when the context is cancelled.
For git clone, fetch and push, a context is created with the timeout of
the target GitRepository.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2021-11-04 18:36:13 +05:30
Hidde Beydals b29b87a7c5 controller: `Free` remaining `libgit2` objects
This supersedes #238.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-28 15:36:08 +02:00
Hidde Beydals c4b8b42164 controllers: use new `git` contract
This commit makes use of the refactored `git` package, which has
been reworked to increase stability and test coverage, and ensures
implementation details do not leak out into the "main wrapper".

This indirectly seems to resolve a memory leak that happenedd with
the previous wiring, thereby fixing #247.

The code changes for this controller itself are minimal, mostly
ensuring the auth and checkout configurations are created in the
"new way".

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-28 15:35:59 +02:00