Commit Graph

134 Commits

Author SHA1 Message Date
Stefan Prodan 0d43488ba6
Fix flaky tests by waiting for finalizers
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-06-24 12:54:23 +03:00
Stefan Prodan 34413b041f
Add finalizer to ImageUpdateAutomation resources
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-06-24 11:46:33 +03:00
Paulo Gomes ec89ad41b9
Update source-controller to v0.25.7
This version of source-controller introduces a fix for the
SSH connections leak issue reported at:
https://github.com/fluxcd/image-automation-controller/issues/334

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-06-22 15:55:50 +05:30
Paulo Gomes a996462048
Fix subtest references
Subtests are not supposed to use the parent t reference
but rather to use their own. This change ensures that
is the case.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-06-16 14:12:56 +01:00
Sunny 9e560841c2
Update source-controller with libgit2 race fixes
Also, removes the panic recovery code added in cloneInto() as the
underlying Checkout() in SC already has a panic handler. The issues were
due to goroutine panicking, which are now handled in the SC libgit
managed transport.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-06-03 02:59:05 +05:30
Sanskar Jaiswal 175f91ea0e recover from panics in cloneInto
Signed-off-by: Sanskar Jaiswal <sanskar.jaiswal@weave.works>
2022-06-02 19:23:56 +05:30
Sanskar Jaiswal 61f76d3f6d use context for cloning in tests
Signed-off-by: Sanskar Jaiswal <sanskar.jaiswal@weave.works>
2022-06-02 19:10:08 +05:30
Sanskar Jaiswal 5ee6446764 fix regression in switchToBranch
Fixes regression in which we fail to push to a branch after switching to
a branch, if origin is ahead of local. Fixed by setting the upstream
commit as the local branch target.

Regression introduced in #330, and partially addressed in #369.

Signed-off-by: Sanskar Jaiswal <sanskar.jaiswal@weave.works>
2022-06-02 19:09:01 +05:30
Sanskar Jaiswal 74420bd1a7 refactor controller and git tests to use managed transport
Signed-off-by: Sanskar Jaiswal <sanskar.jaiswal@weave.works>
2022-06-02 15:52:55 +05:30
Paulo Gomes ebc40b2d88
Fix regression on push branches
Fixes regression in which IAC will fail to update push branch
when the push branch already exists and checkout branch is
ahead.

The reconciliation errors with: 'cannot push because a reference
that you are trying to update on the remote contains commits
that are not present locally.'

Regression introduced on #330.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-06-01 21:49:04 +03:00
Paulo Gomes 4f044661fd
Update source-controller
- Panic recovery for Git operations.
- Improved SSH connection management without use of caching.
- Enforce context timeout for managed SSH.
- Remove dependency to callback functions.
- Add support for hashed known_hosts.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-06-01 21:04:12 +03:00
Sunny 0e87398e59
Remove legacy_suite_test.go and ginkgo deps
Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-04-26 01:22:21 +05:30
Sunny 37022d0c88
Refactor update_test.go to use testenv
Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-04-26 01:22:14 +05:30
Sunny 985d879e31
Move ginkgo suite to legacy & add testenv
Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-04-21 18:59:05 +05:30
Paulo Gomes 4a53f9defa
Add flags to configure exponential back-off retry
Add two new flags to enable users to configure exponential
back-off for Flux objects. The default values are now
set to 750ms for minimum retry time, and 15min for max.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-04-12 13:01:36 +01:00
Paulo Gomes 859a6ca93a
Update source-controller to v0.22.4
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-03-28 15:22:19 +01:00
Paulo Gomes 4b35fe3ee8
Fix bug when pushing into different branches
Changes replacing go-git with git2go introduced a bug
in which pushes into new branches squashes all commits
into one.

xref: https://github.com/fluxcd/image-automation-controller/pull/324
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-03-23 19:32:24 +00:00
Paulo Gomes 5b15bb7f94
Implement Managed Transport for libgit2
libgit2 network operations are blocking and do not provide timeout nor context capabilities,
leading to several reports of the controllers hanging indefinitely.

By using managed transport, golang primitives such as http.Transport and net.Dial can be used
to ensure timeouts are enforced.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-03-21 15:19:00 +00:00
Paulo Gomes 4d8ebe69a0 Ensure libgit2 resources are released
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-03-21 13:06:38 +01:00
Sunny 146a86865c Use new standardized runtime and meta package
Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-03-21 12:45:55 +01:00
Paulo Gomes 1a52581356
Remove direct dependency to go-git
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-03-04 11:10:22 +00:00
Paulo Gomes d20cdb532f
Refactor initGitRepo to use libgit2 instead of go-git
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-03-01 14:48:15 +00:00
Paulo Gomes 4ab219e64b
Fix tests after upgrading to libgit2 1.3.0
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-03-01 14:48:13 +00:00
Paulo Gomes b898759d26
Update libgit2 to 1.3.0 and update source-controller to v0.21.3
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-03-01 14:48:12 +00:00
Somtochi Onyekwere 0ca01bd254 Add flag to disable cross-namespace referencing
Signed-off-by: Somtochi Onyekwere <somtochionyekwere@gmail.com>
2022-01-28 08:32:21 +01:00
Aurel Canciu aa3b6cc4b7
Update kyaml to v0.13.0
Updating kyaml to match the version we use in other components. This
version's most significant change for us would be that kyaml will no
longer override indentations in the targeted files.

Signed-off-by: Aurel Canciu <aurelcanciu@gmail.com>
2022-01-26 15:56:25 +01:00
Sanskar Jaiswal 3de51e7a1e add support for cross-namespace sourceRef in ImageUpdateAutomation
ImageUpdateAutomation objects can now refer to GitRepository objects in other
namespaces. Implemented by switching sourceRef from a SourceReference to a
dependency.CrossNamespaceDependencyReference.

Signed-off-by: Sanskar Jaiswal <sanskar.jaiswal@weave.works>
2022-01-25 14:21:37 +05:30
Aurélien GARNIER 9b82e81af0 Use libgit2 auto proxy configuration
Signed-off-by: Aurélien GARNIER <aurelien.garnier@atos.net>
2022-01-20 17:14:18 +01:00
Sanskar Jaiswal 942e980616 fix makefile envtest and controller-gen usage
Refactor logic to install helper tools into one function in the
Makefile. Add support for envtest to help install tools like kubectl,
etcd which helps users run tests more conveniently.

Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
2022-01-13 16:46:12 +05:30
Luke Mallon (Nalum) c026e67484
[refactor] Check for a nil lastTime before use
Signed-off-by: Luke Mallon (Nalum) <luke@mallon.ie>
2022-01-11 12:31:02 +00:00
Aurel Canciu 24594ebf20
Update flux pkg components
Signed-off-by: Aurel Canciu <aurelcanciu@gmail.com>
2022-01-07 16:53:29 +01:00
Michael Bridgen 8e1f31fe12 Log the error when removing the working dir fails
os.RemoveAll can return an error; even if we're exiting the procedure
at the time, it's worth knowing when it's failing, because typically
the working directory is a memory-backed volume and you can run out
quite easily.

Signed-off-by: Michael Bridgen <michael@weave.works>
2022-01-05 17:55:06 +00:00
Luke Mallon (Nalum) f75315e44d
[refactor] Remove else as redundant in this use
Signed-off-by: Luke Mallon (Nalum) <luke@mallon.ie>
2021-12-21 15:58:58 +00:00
Luke Mallon (Nalum) 864ad1182e
[refactor] Move Path check into switch case
Signed-off-by: Luke Mallon (Nalum) <luke@mallon.ie>
2021-12-20 16:20:21 +00:00
Sunny 16690b3fb5 Update tests to be compatible with CR > v0.10.0
Also updates the tests to be compatible with the breaking changes in
controller-runtime > v0.10.0.

Cherry-picked from 467d97a9bf.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2021-11-22 17:15:56 +01:00
Stefan Prodan bfcb9ef8db
Remove deprecated io/ioutil
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-11-22 16:08:28 +02:00
Stefan Prodan 469988105d
Add the commit message to the event body
Drop "no updates made" event leave only the debug log in place.

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-11-09 12:26:52 +02:00
Aurel Canciu fe40819140
Fix unhandled error in signing key retrieval
Fixes #242

Signed-off-by: Aurel Canciu <aurelcanciu@gmail.com>
2021-11-04 22:12:10 +01:00
Sunny b299a9db59 Pass context to libgit2.RemoteCallbacks
Pass cancellable context to libgit2.RemoteCallbacks to be able to cancel
the remote operations when the context is cancelled.
For git clone, fetch and push, a context is created with the timeout of
the target GitRepository.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2021-11-04 18:36:13 +05:30
Hidde Beydals b29b87a7c5 controller: `Free` remaining `libgit2` objects
This supersedes #238.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-28 15:36:08 +02:00
Hidde Beydals c4b8b42164 controllers: use new `git` contract
This commit makes use of the refactored `git` package, which has
been reworked to increase stability and test coverage, and ensures
implementation details do not leak out into the "main wrapper".

This indirectly seems to resolve a memory leak that happenedd with
the previous wiring, thereby fixing #247.

The code changes for this controller itself are minimal, mostly
ensuring the auth and checkout configurations are created in the
"new way".

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-28 15:35:59 +02:00
Soule BA bc3d7b2112 Add support for the sprig functions library
All functions are not supported, The following are discarded:
            // Date functions
            "date",
            "date_in_zone",
            "date_modify",
            "now",
            "htmlDate",
            "htmlDateInZone",
            "dateInZone",
            "dateModify",

            // Strings
            "randAlphaNum",
            "randAlpha",
            "randAscii",
            "randNumeric",
            "uuidv4",

            // OS
            "env",
            "expandenv",

            // Network
            "getHostByName",

Signed-off-by: Soule BA <bah.soule@gmail.com>
2021-10-12 11:41:03 +02:00
Stefan Prodan 78b565dc90
Use NamespacedObjectReference in tests
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-10-08 19:05:54 +03:00
Hidde Beydals 58e21f37fb Update github.com/libgit2/git2go to v31.6.1
This commit updates `github.com/libgit2/git2go` to `v31.6.1` (with
`libgit2` `1.1.1`), and changes the container image build process so
that it makes use of `ghcr.io/hiddeco/golang-with-libgit2`.

This image provides a pre-build dynamic `libgit2` dependency linked
against OpenSSL and LibSSH2 (without gcrypt), and a set of cross-compile
build tools (see
[rationale](https://github.com/hiddeco/golang-with-libgit2#rationale) and
[usage](https://github.com/hiddeco/golang-with-libgit2#usage) for more
detailed information).

The linked set of dependency should solve most known issues around
unsupport private key types, but does not resolve the issues with ECDSA*
and ED25519 hostkeys yet. Solving this requires a newer version of
`libgit2` (`>=1.2.0`), which currently does not seem to work properly
with `git2go/v32`.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-08 16:26:15 +02:00
Michael Bridgen b19eaf67f4 Use pkg/runtime consts for log levels
github.com/fluxcd/pkg/runtime/logger now supports the trace log level,
and has consts for the V(...) argument to supply. This commit updates
that module so to use those consts.

Previously, trace messages were output at the debug level; after this
commit they will be in their own level, so you need

    --log-level=trace

to see them.

Signed-off-by: Michael Bridgen <michael@weave.works>
2021-10-06 16:10:05 +01:00
David Korczynski 5a831a8a00
Fix nil-dereference in controller
Signed-off-by: David Korczynski <david@adalogics.com>
2021-09-29 18:06:11 +01:00
Michael Bridgen 3476ecb950 Fail push if a ref update is rejected
libgit2's Push method will succeed even when ref updates are rejected,
meaning it can silently fail if you e.g., use branch protection in
GitHub.

To make these errors visible, a callback is supplied to Push, which
checks for a non-empty status (on the advice of
https://libgit2.org/libgit2/#HEAD/group/callback/git_push_update_reference_cb).

For whatever reason, gogit seems overly sensitive to hook errors (in a
way that `git` and libgit2 aren't), and reports "invalid pkg-len
found" when it sees a rejected ref message. This doesn't affect the
runtime code, since that uses libgit2 -- but it does affect the test
code, which initialises the git repo used in many tests, so more care
is needed to push only the main branch, so as not to trigger a
rejection.

Signed-off-by: Michael Bridgen <michael@weave.works>
2021-07-13 16:06:17 +01:00
Michael Bridgen fd52463ac5 Use v1beta1 of the API from image reflector
This bumps the version of the image reflector types to v1beta1. This
doesn't technically make a difference for the minute, since the
Kubernetes API server would convert between versions, but is tidier.

Signed-off-by: Michael Bridgen <michael@weave.works>
2021-06-28 13:59:34 +01:00
Michael Bridgen 5b9631327f Graduate v1alpha2 API to v1beta1
This does the following:

 - copies the type definitions from v1alpha2 to v1beta1
 - changes the "stored" version to v1beta1
 - gives the CRD a conversion strategy of None, meaning just rewrite the version
 - switches the controller to use v1beta1
 - moves the generated documentation to v1beta1

This effectively rebadges the v1alpha2 version of this part of the
image API to v1beta1. The v1alpha2 version is left in place; there are
no conversion issues, as with v1alpha1->v1alpha2. The CRD specifies
that converting between v1alpha2 and v1beta1 just means changing the
version (i.e., the schema and semantics are the same).

Signed-off-by: Michael Bridgen <michael@weave.works>
2021-06-28 11:26:59 +01:00
Max Jonas Werner 93782755ef
feat: add controller concurrency
The controller is now working with 4 concurrent workers by default.
This value is configurable through the `--concurrent` flag.

Signed-off-by: Max Jonas Werner <mail@makk.es>
2021-06-24 20:15:00 +02:00