Commit Graph

72 Commits

Author SHA1 Message Date
Hidde Beydals 49639638b9 Rewrite `HelmRepositoryReconciler` to new standards
This commit rewrites the `HelmRepositoryReconciler` to new standards,
while implementing the newly introduced Condition types, and trying to
adhere better to Kubernetes API conventions.

More specifically it introduces:

- Implementation of more explicit Condition types to highlight
  abnormalities.
- Extensive usage of the `conditions` subpackage from `runtime`.
- Better and more conflict-resilient (status)patching of reconciled
  objects using the `patch` subpackage from runtime.
- Proper implementation of kstatus' `Reconciling` and `Stalled`
  conditions.
- Refactoring of some Helm elements to make them easier to use within
  the new reconciler logic.
- Integration tests that solely rely on `testenv` and do not
  use Ginkgo.

There are a couple of TODOs marked in-code, these are suggestions for
the future and should be non-blocking.
In addition to the TODOs, more complex and/or edge-case test scenarios
may be added as well.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-08-12 11:47:56 +02:00
Stefan Prodan 525be388ec Add ACL optional field to Source API
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-08-12 10:04:35 +02:00
Hidde Beydals 2ca0b47fea Use same SemVer logic in both Git implementations
Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-07-30 19:36:58 +02:00
Hidde Beydals 08ce0c95fc Rewrite `GitRepositoryReconciler` to new standards
This commit rewrites the `GitRepositoryReconciler` to new standards,
while implementing the newly introduced Condition types, and trying
to adhere better to Kubernetes API conventions.

More specifically it introduces:

- Implementation of more explicit Condition types to highlight
  abnormalities.
- Extensive usage of the `conditions` subpackage from `runtime`.
- Better and more conflict-resilient (status)patching of reconciled
  objects using the `patch` subpackage from runtime.
- Proper implementation of kstatus' `Reconciling` and `Stalled`
  conditions.
- First (integration) tests that solely rely on `testenv` and do not
  use Ginkgo.

There are a couple of TODOs marked in-code, these are suggestions for
the future and should be non-blocking.
In addition to the TODOs, more complex and/or edge-case test scenarios
may be added as well.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-07-30 19:36:32 +02:00
Hidde Beydals 82583f2247 Implement new runtime interfaces, prepare testenv
This commit ensures all API objects implement the interfaces used by
the runtime package to work with conditions, etc., and prepares the
test suite to work with the `pkg/runtime/testenv` wrapper.

Changes are made in a backwards compatible way (that being: the
existing code can still be build and works as expected), but without
proper dependency boundaries. The result of this is that the API
package temporary depends on the runtime package, which is resolved
when all reconcilers have been refactored and the API package does
no longer contain condition modifying functions.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-07-30 12:33:18 +02:00
Hidde Beydals 5e8e0ab65c Update Helm to v3.6.3
Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-07-14 23:24:26 +02:00
Hidde Beydals 1f27410b34 Update Helm to v3.6.1
v3.6.1 is a a security update from Helm, ensuring that credentials are
always only passed to the defined repository host.

Based on Helm user reports, disabling this behavior may be required for
some Helm repository solutions like Artifactory, and may be done by
setting `PassCredentials` in the `HelmRepositorySpec`.

For more information, see:
https://github.com/helm/helm/security/advisories/GHSA-56hp-xqp3-w2jf

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-06-18 12:31:23 +02:00
Hidde Beydals 2d8a540f0c Unpin docker Go Module dependency
No longer required as we do now depend on Helm v3.6.0, which no longer
pins the version.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-06-10 16:48:00 +02:00
Hidde Beydals a76b10cebb Update K8s, controller-runtime and fluxcd/pkg deps
Controller-runtime has been updated to `v0.9.0`, K8s dependencies to
`v0.21.1`, and all `fluxcd/pkg` and other dependencies to the versions
that have matching dependencies and/or build constraints.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-06-08 13:41:34 +02:00
Hidde Beydals e1682da795 Update go-git to v5.4.2
This should resolve `object not found` and
`empty git-upload-pack given` errors that were thrown for Git
repositories that used to work fine before the `v0.13.0` release.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-06-02 11:20:23 +02:00
Stefan Prodan 4f5b958806
Update libgit2/git2go to v31.4.14
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-05-26 14:08:01 +03:00
Stefan Prodan 79b180376d
Update go-git to v5.4.1
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-05-26 14:06:41 +03:00
Philip Laine fcf7048992 Add include property to GitRepositories
Signed-off-by: Philip Laine <philip.laine@gmail.com>
Signed-off-by: Philip Laine <philip.laine@xenit.se>
2021-05-11 09:46:50 +02:00
Stefan Prodan bd0f1fdd19
Release v0.12.0
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-04-21 17:57:50 +03:00
Stefan Prodan cb491c9f10
Update fluxcd/pkg/runtime to v0.11.0
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-04-21 15:04:16 +03:00
Stefan Prodan 51a62a6043
Update fluxcd/pkg/runtime to v0.10.2
Followup: https://github.com/fluxcd/pkg/pull/96

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-04-19 15:05:40 +03:00
Stefan Prodan d7c90a533d
Update Helm to v3.5.4
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-04-15 11:57:01 +03:00
Stefan Prodan f0016cfad1
Enable self-signed certs for go-git
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-03-29 13:23:32 +03:00
Stefan Prodan a034c5d955
Set leader election deadline to 30s
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-03-25 13:43:22 +02:00
Somtochi Onyekwere 2624ba93a3 Record suspension metric
Signed-off-by: Somtochi Onyekwere <somtochionyekwere@gmail.com>
2021-03-17 14:04:21 +01:00
Stefan Prodan 2f04bd3043
Update dependencies
- helm.sh/helm/v3 v3.5.3
- github.com/minio/minio-go/v7 v7.0.10
- sigs.k8s.io/controller-runtime v0.8.3

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-03-12 12:39:23 +02:00
Michael Bridgen f7e08c4738 Tidy git clone errors
In some circumstances (that are rather hard to reproduce), cloning
from a GitLab repo gets a multiline response as described in
https://github.com/fluxcd/image-automation-controller/pull/115.

This uses the same remedy as in that PR, by calling the funcs provided
by fluxcd/pkg/gitutil on any error returned by libgit2 or gogit clone
operations.

Signed-off-by: Michael Bridgen <mikeb@squaremobius.net>
2021-03-03 13:41:53 +00:00
Stefan Prodan 22f5a6f6fb
Update dependencies
- helm.sh/helm/v3 v3.5.2 (forces us into pinning docker)
- sigs.k8s.io/controller-tools/cmd/controller-gen v0.4.1
- sigs.k8s.io/controller-runtime v0.8.2
- fluxcd/pkg/apis/meta v0.8.0
- fluxcd/pkg/runtime v0.8.3

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-02-23 13:18:21 +02:00
Philip Laine c063484761 Add custom certificate validation
Signed-off-by: Philip Laine <philip.laine@gmail.com>
2021-02-08 12:19:22 +01:00
Hidde Beydals 2764c00a7e Enable pprof endpoints on metrics server
Using the helper from `pkg/runtime/pprof`, which follows the suggestion
from controller-runtime to use `AddMetricsExtraHandler`.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-02-07 11:43:27 +01:00
Stefan Prodan 8e3700ebd8
Add QPS/Burst flags using fluxcd/pkg/runtime v0.8
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-01-21 14:13:07 +02:00
Stefan Prodan ab16dd7c2e
Update Kubernetes and Helm packages
- k8s.io/api v0.20.2
- controller-runtime v0.8.0
- helm v3.5.0

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-01-16 14:38:28 +02:00
Hidde Beydals 566ce8aa15 Upgrade runtime package to v0.6.2
To include a bug fix to the `ReconcilateAtChangedPredicate`
and renaming to `ReconcileRequestedPredicate`.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-01-14 11:39:47 +01:00
Hidde Beydals fda481efe9 Upgrade controller-runtime to v0.7.0
This commit upgrades the `controller-runtime` dependency to `v0.7.0`,
including all changes required to make all wiring work again.

- Upgrade `runtime` to v0.6.0 to include `controller-runtime` changes.
- Loggers have been removed from the reconcilers and are now retrieved
  from the `context.Context` passed to the `Reconcile` method and
  downwards functions.
- Logger configuration flags are now bound to the flag set using
  `BindFlags` from `runtime/logger`, ensuring the same contract across
  GitOps Toolkit controllers, and the `--log-json` flag has been
  deprecated in favour of the `--log-encoding=json` default.
- The `ChangePredicate` from `runtime` has changed to a
  `ReconcilateAtChangedPredicate`, and is now chained with the
  `GenerationChangedPredicate` from `controller-runtime` using
  `predicate.Or`.
- Signatures that made use of `runtime.Object` have changed to
  `client.Object`, removing the requirement to e.g. call
  `runtime.Object#Object`.
- The `client.MatchingField` function was deprecated, and has been
  replaced with `client.MatchingFields{}`.
- The `leader-election-role` was changed, as leader election now works
  via the `coordination/v1` API.

Other notable changes:

- `util.ObjectKey` was added to easily construct a `client.ObjectKey` /
  `types.NamespacedName` from a `metav1.Object`.
- The `SourceIndexKey` constant has been split out into
  `{GitRepository,HelmRepository,Bucket}IndexKey` constants.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2020-12-18 19:17:10 +01:00
Hidde Beydals 830b350339 Upgrade dependencies
* `github.com/fluxcd/pkg/apis/meta` to `v0.5.0`
* `github.com/fluxcd/pkg/helmtestserver` to `v0.1.0`
* `github.com/fluxcd/pkg/runtime` to `v0.4.0`
* `helm.sh/helm/v3` to `v3.4.2`
* `k8s.io/{api,apimachinery,client-go}` to `v0.19.4`
* `sigs.k8s.io/controller-runtime` to `v0.6.4`

Signed-off-by: Hidde Beydals <hello@hidde.co>
2020-12-09 18:22:27 +01:00
Philip Laine 1b4eacc588 Refactor argument name to enable git2go
Signed-off-by: Philip Laine <philip.laine@xenit.se>
2020-12-04 10:07:17 +01:00
Philip Laine cb12736a84 Update pkg git server
Signed-off-by: Philip Laine <philip.laine@xenit.se>
2020-12-03 12:56:38 +01:00
Philip Laine 47e4e54550 Add test to show that libgit2 fails with self signed test
Signed-off-by: Philip Laine <philip.laine@gmail.com>
2020-12-02 23:28:32 +01:00
Philip Laine 2cb3754953 Add support for gitv2 servers
Signed-off-by: Philip Laine <philip.laine@xenit.se>
2020-12-02 20:18:29 +01:00
Aurel Canciu b80f450b68
Switch to new pkg/apis/meta SetResourceCondition
Use SetResourceCondition as a generic method to set conditions for CRs,
implmeneting the ObjectWithStatusConditions interface used as input
type.

Signed-off-by: Aurel Canciu <aurelcanciu@gmail.com>
2020-11-19 13:08:33 +02:00
Aurel Canciu 00bb853d0e
Refactor to adopt k8s standardized Condition type
Updates to use metav1.Condition type and removes references for
deprecated corev1.Condition* constants and uses the new k8s api/meta
helpers in place of the old pkg/apis/meta types.

Signed-off-by: Aurel Canciu <aurelcanciu@gmail.com>
2020-11-19 13:08:33 +02:00
Aurel Canciu 60d1dc8219
Update pkg components and k8s to 1.19
Signed-off-by: Aurel Canciu <aurelcanciu@gmail.com>
2020-11-01 18:11:27 +02:00
Aurel Canciu adb9f86fee
Helm dependency update to v3.4.0
Signed-off-by: Aurel Canciu <aurelcanciu@gmail.com>
2020-10-30 21:41:30 +02:00
Stefan Prodan 08c86bb2d0
Update fluxcd/pkg/runtime to v0.1.2
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-10-28 13:59:19 +02:00
Hidde Beydals c9eb8f03c0 Switch to Masterminds/semver and pkg/version libs
Co-authored-by: Illia Ovchynnikov <illia.ovchynnikov@gmail.com>
Signed-off-by: Hidde Beydals <hello@hidde.co>
2020-10-28 12:21:08 +01:00
Stefan Prodan c8c2eec3a6
Update fluxcd/pkg/runtime to v0.1.0
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2020-10-13 13:34:47 +03:00
Hidde Beydals c176dcf3a9 Release v0.1.0 2020-09-30 17:37:23 +02:00
Hidde Beydals 431ea05427 Implement `fluxcd/pkg/meta/api` in APIs 2020-09-30 10:18:43 +02:00
stefanprodan 8d95c18342 Update Helm to v3.3.4 2020-09-23 12:59:14 +03:00
stefanprodan fd27e378b2 Update Helm to v3.3.3 2020-09-21 10:53:16 +03:00
stefanprodan 924e0124b4 Release v0.0.17 2020-09-18 16:44:59 +03:00
stefanprodan c2c61b10a1 Implement S3 Bucket controller 2020-09-18 16:11:24 +03:00
Hidde Beydals 42706a342b Calculate checksums during file writes 2020-09-10 13:00:08 +02:00
Michael Bridgen 901463eaaa Use pkg/runtime for predicate and logger 2020-09-09 17:57:16 +01:00
Hidde Beydals 8f08ce20a4 Upgrade Helm dependency
* helm.sh/helm/v3 to v3.3.1
2020-09-01 20:32:10 +02:00