Commit Graph

121 Commits

Author SHA1 Message Date
Hidde Beydals d19c53f628 Update dependencies
- github.com/fluxcd/pkg/apis/meta to v0.19.0
- github.com/fluxcd/source-controller/api to v0.34.0
- github.com/fluxcd/image-reflector-controller/api to v0.24.0
- k8s.io/api to v0.26.1
- k8s.io/apimachinery to v0.26.1
- k8s.io/client-go to v0.26.1
- sigs.k8s.io/controller-runtime to v0.14.2

Signed-off-by: Hidde Beydals <hello@hidde.co>
2023-02-01 15:55:30 +00:00
Paulo Gomes 9d242c54f0
Add feature gate GitAllBranchReferences
The new feature gate enables users to toggle the download of all branch
head references when push branches are configured.

Tests were refactored to ensure that they are feature gate sensitive.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-12-20 16:57:39 +00:00
Paulo Gomes a980b894e0
Block the creation of empty commits
In specific cases, the update mechanism may lead to no files being
changed. The changes short-circuit the process to exit early when
0 files have been modified.

go-git has been changed to return an error when an empty commit
is detected, which is used as a last resort.

When the push branch is different than the clone branch, the controller
will now download all branch references from the Git server, in order
to enable a single fetch operation.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-12-20 10:11:22 +00:00
Paulo Gomes 5048e38508
features: Remove ForceGoGitImplementation
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-12-09 13:43:24 +00:00
Paulo Gomes 7a05f4856c
libgit2: Remove references to libgit2 from code
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-12-09 11:30:55 +00:00
Paulo Gomes e846e29458
Add GitShallowClone feature
This feature enables the use of shallow clones when pulling source from
Git repositories.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-12-06 15:22:34 +00:00
Sanskar Jaiswal 1cacafa5fc bump pkg/git, pkg/git/gogit and pkg/git/libgit2
* Bump `pkg/git` to `v0.7.0`
* Bump `pkg/git/gogit` to `v0.2.0`
* Bump `pkg/git/libgit2` to `v0.3.0`

Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
2022-11-16 20:00:26 +05:30
Stefan Prodan cad8850c11
Use Flux Event API v1beta1
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-11-09 12:11:43 +02:00
Paulo Gomes 0191d6b4cc
gogit: Add new ForceGoGitImplementation FeatureGate
ForceGoGitImplementation ignores the value set for gitImplementation
and ensures that go-git is used for all GitRepository objects.
This can be used to confirm that Flux instances won't break if/when
the libgit2 implementation was to be deprecated.

When enabled, libgit2 won't be initialized, nor will any git2go cgo
code be called.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-11-04 11:16:26 +00:00
Paulo Gomes 842c6d78c4
Add support for Git force push
Automations using a different push branch can now
enable force push to ensure that it gets refreshed
with the latest version of the branch used for the
clone operation.

Initially this was going to be added in combination
with Shallow clones for the `go-git` implementation.
However, force push is required for feature completeness
whilst Shallow clones are mostly an optimisation, and
will therefore be implemented at a later stage.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-11-02 13:35:36 +00:00
Paulo Gomes b00625bffc
Remove redundant repoAccess
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-11-01 12:42:48 +00:00
Sanskar Jaiswal 7c6a117523
use gitrepo implementation for git ops
Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
2022-10-27 14:43:03 +01:00
Sanskar Jaiswal 0296f7acf0
refactor reconciler to use fluxcd/pkg/git
Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
2022-10-27 14:42:13 +01:00
Paulo Gomes 06f4acd118
Use helper.Metrics for recording metrics
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-09-05 14:03:08 +01:00
Paulo Gomes 735ab61911
Log when no changes were made
This gets the behaviour a step closer to Source Controller
which logs after every reconciliation, even when no changes
were made.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-09-05 14:02:51 +01:00
Pierre-Barthelemy Fiocconi 3c06e94877
Fix fetch error in push branch
Fixes error when push branch and reference branch are equal
by skipping switch to push branch

Signed-off-by: Pierre-Barthelemy Fiocconi <pierre-barthelemy.fi@doctolib.com>
2022-08-25 12:01:06 +02:00
Paulo Gomes 0fdcf7fe6d
Enable RecoverPanic
The controller-runtime provides the ability to recover from
panics that have arisen from a reconciliation. This change
enables this functionality by default.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-08-10 09:59:19 +01:00
Paulo Gomes 4a8ddfad49
git: Decommission libgit2 Unmanaged Transport
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-08-09 13:47:48 +01:00
Paulo Gomes f51050e8c0
Enrich 'early EOF' error message
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-07-26 11:17:12 +01:00
Stefan Prodan 34413b041f
Add finalizer to ImageUpdateAutomation resources
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-06-24 11:46:33 +03:00
Sunny 9e560841c2
Update source-controller with libgit2 race fixes
Also, removes the panic recovery code added in cloneInto() as the
underlying Checkout() in SC already has a panic handler. The issues were
due to goroutine panicking, which are now handled in the SC libgit
managed transport.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-06-03 02:59:05 +05:30
Sanskar Jaiswal 175f91ea0e recover from panics in cloneInto
Signed-off-by: Sanskar Jaiswal <sanskar.jaiswal@weave.works>
2022-06-02 19:23:56 +05:30
Sanskar Jaiswal 5ee6446764 fix regression in switchToBranch
Fixes regression in which we fail to push to a branch after switching to
a branch, if origin is ahead of local. Fixed by setting the upstream
commit as the local branch target.

Regression introduced in #330, and partially addressed in #369.

Signed-off-by: Sanskar Jaiswal <sanskar.jaiswal@weave.works>
2022-06-02 19:09:01 +05:30
Sanskar Jaiswal 74420bd1a7 refactor controller and git tests to use managed transport
Signed-off-by: Sanskar Jaiswal <sanskar.jaiswal@weave.works>
2022-06-02 15:52:55 +05:30
Paulo Gomes ebc40b2d88
Fix regression on push branches
Fixes regression in which IAC will fail to update push branch
when the push branch already exists and checkout branch is
ahead.

The reconciliation errors with: 'cannot push because a reference
that you are trying to update on the remote contains commits
that are not present locally.'

Regression introduced on #330.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-06-01 21:49:04 +03:00
Paulo Gomes 4f044661fd
Update source-controller
- Panic recovery for Git operations.
- Improved SSH connection management without use of caching.
- Enforce context timeout for managed SSH.
- Remove dependency to callback functions.
- Add support for hashed known_hosts.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-06-01 21:04:12 +03:00
Paulo Gomes 4a53f9defa
Add flags to configure exponential back-off retry
Add two new flags to enable users to configure exponential
back-off for Flux objects. The default values are now
set to 750ms for minimum retry time, and 15min for max.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-04-12 13:01:36 +01:00
Paulo Gomes 859a6ca93a
Update source-controller to v0.22.4
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-03-28 15:22:19 +01:00
Paulo Gomes 4b35fe3ee8
Fix bug when pushing into different branches
Changes replacing go-git with git2go introduced a bug
in which pushes into new branches squashes all commits
into one.

xref: https://github.com/fluxcd/image-automation-controller/pull/324
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-03-23 19:32:24 +00:00
Paulo Gomes 5b15bb7f94
Implement Managed Transport for libgit2
libgit2 network operations are blocking and do not provide timeout nor context capabilities,
leading to several reports of the controllers hanging indefinitely.

By using managed transport, golang primitives such as http.Transport and net.Dial can be used
to ensure timeouts are enforced.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-03-21 15:19:00 +00:00
Paulo Gomes 4d8ebe69a0 Ensure libgit2 resources are released
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-03-21 13:06:38 +01:00
Sunny 146a86865c Use new standardized runtime and meta package
Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-03-21 12:45:55 +01:00
Paulo Gomes 1a52581356
Remove direct dependency to go-git
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-03-04 11:10:22 +00:00
Paulo Gomes b898759d26
Update libgit2 to 1.3.0 and update source-controller to v0.21.3
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-03-01 14:48:12 +00:00
Somtochi Onyekwere 0ca01bd254 Add flag to disable cross-namespace referencing
Signed-off-by: Somtochi Onyekwere <somtochionyekwere@gmail.com>
2022-01-28 08:32:21 +01:00
Sanskar Jaiswal 3de51e7a1e add support for cross-namespace sourceRef in ImageUpdateAutomation
ImageUpdateAutomation objects can now refer to GitRepository objects in other
namespaces. Implemented by switching sourceRef from a SourceReference to a
dependency.CrossNamespaceDependencyReference.

Signed-off-by: Sanskar Jaiswal <sanskar.jaiswal@weave.works>
2022-01-25 14:21:37 +05:30
Aurélien GARNIER 9b82e81af0 Use libgit2 auto proxy configuration
Signed-off-by: Aurélien GARNIER <aurelien.garnier@atos.net>
2022-01-20 17:14:18 +01:00
Luke Mallon (Nalum) c026e67484
[refactor] Check for a nil lastTime before use
Signed-off-by: Luke Mallon (Nalum) <luke@mallon.ie>
2022-01-11 12:31:02 +00:00
Aurel Canciu 24594ebf20
Update flux pkg components
Signed-off-by: Aurel Canciu <aurelcanciu@gmail.com>
2022-01-07 16:53:29 +01:00
Michael Bridgen 8e1f31fe12 Log the error when removing the working dir fails
os.RemoveAll can return an error; even if we're exiting the procedure
at the time, it's worth knowing when it's failing, because typically
the working directory is a memory-backed volume and you can run out
quite easily.

Signed-off-by: Michael Bridgen <michael@weave.works>
2022-01-05 17:55:06 +00:00
Luke Mallon (Nalum) f75315e44d
[refactor] Remove else as redundant in this use
Signed-off-by: Luke Mallon (Nalum) <luke@mallon.ie>
2021-12-21 15:58:58 +00:00
Luke Mallon (Nalum) 864ad1182e
[refactor] Move Path check into switch case
Signed-off-by: Luke Mallon (Nalum) <luke@mallon.ie>
2021-12-20 16:20:21 +00:00
Stefan Prodan bfcb9ef8db
Remove deprecated io/ioutil
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-11-22 16:08:28 +02:00
Stefan Prodan 469988105d
Add the commit message to the event body
Drop "no updates made" event leave only the debug log in place.

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-11-09 12:26:52 +02:00
Aurel Canciu fe40819140
Fix unhandled error in signing key retrieval
Fixes #242

Signed-off-by: Aurel Canciu <aurelcanciu@gmail.com>
2021-11-04 22:12:10 +01:00
Sunny b299a9db59 Pass context to libgit2.RemoteCallbacks
Pass cancellable context to libgit2.RemoteCallbacks to be able to cancel
the remote operations when the context is cancelled.
For git clone, fetch and push, a context is created with the timeout of
the target GitRepository.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2021-11-04 18:36:13 +05:30
Hidde Beydals b29b87a7c5 controller: `Free` remaining `libgit2` objects
This supersedes #238.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-28 15:36:08 +02:00
Hidde Beydals c4b8b42164 controllers: use new `git` contract
This commit makes use of the refactored `git` package, which has
been reworked to increase stability and test coverage, and ensures
implementation details do not leak out into the "main wrapper".

This indirectly seems to resolve a memory leak that happenedd with
the previous wiring, thereby fixing #247.

The code changes for this controller itself are minimal, mostly
ensuring the auth and checkout configurations are created in the
"new way".

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-28 15:35:59 +02:00
Soule BA bc3d7b2112 Add support for the sprig functions library
All functions are not supported, The following are discarded:
            // Date functions
            "date",
            "date_in_zone",
            "date_modify",
            "now",
            "htmlDate",
            "htmlDateInZone",
            "dateInZone",
            "dateModify",

            // Strings
            "randAlphaNum",
            "randAlpha",
            "randAscii",
            "randNumeric",
            "uuidv4",

            // OS
            "env",
            "expandenv",

            // Network
            "getHostByName",

Signed-off-by: Soule BA <bah.soule@gmail.com>
2021-10-12 11:41:03 +02:00
Hidde Beydals 58e21f37fb Update github.com/libgit2/git2go to v31.6.1
This commit updates `github.com/libgit2/git2go` to `v31.6.1` (with
`libgit2` `1.1.1`), and changes the container image build process so
that it makes use of `ghcr.io/hiddeco/golang-with-libgit2`.

This image provides a pre-build dynamic `libgit2` dependency linked
against OpenSSL and LibSSH2 (without gcrypt), and a set of cross-compile
build tools (see
[rationale](https://github.com/hiddeco/golang-with-libgit2#rationale) and
[usage](https://github.com/hiddeco/golang-with-libgit2#usage) for more
detailed information).

The linked set of dependency should solve most known issues around
unsupport private key types, but does not resolve the issues with ECDSA*
and ED25519 hostkeys yet. Solving this requires a newer version of
`libgit2` (`>=1.2.0`), which currently does not seem to work properly
with `git2go/v32`.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-08 16:26:15 +02:00