This deals with various breaking changes in controller-runtime, as
documented in the release notes:
https://github.com/kubernetes-sigs/controller-runtime/releases/tag/v0.15.0
In short:
- `Watches` now use a `client.Object` instead of a `source.Kind`.
- `handler.MapFunc` signature accepts a Go context, which is used to
log any errors, instead of silently ignoring them and/or panicking.
- Fake clients used in tests are now configured using
`WithStatusSubresource` to enable the correct behavior for status
updates and patches.
- Max concurrent reconciles is configured on the manager, instead of
configuring them per reconciler instance.
- Various manager configuration options have been moved to new
structures and/or fields.
In addition to this, all other dependencies which had updates are
updated to their latest (compatible) versions as well.
Signed-off-by: Hidde Beydals <hidde@hhh.computer>
With an update to github.com/fluxcd/pkg/oci v0.22.0.
This includes a pin of `github.com/docker/docker` to `v20.10.x`, to
prevent Oras from complaining.
Co-authored-by: Hidde Beydals <hidde@hhh.computer>
Signed-off-by: Somtochi Onyekwere <somtochionyekwere@gmail.com>
- cloud.google.com/go/storage to v1.30.1
- github.com/minio/minio-go/v7 to v7.0.50
- google.golang.org/api to v0.114.0
- k8s.io/utils to v0.0.0-20230313181309-38a27ef9d749
- github.com/opencontainers/runc to v1.1.5
Signed-off-by: Hidde Beydals <hidde@hhh.computer>
With this enhancement, the controller can be configured with
`--watch-label-selector`, after which only objects with this label will
be reconciled by the controller.
This allows for horizontal scaling of the source-controller, where each
controller can be deployed multiple times with a unique label selector
which is used as the sharding key.
Note that this also requires configuration of the `--storage-adv-addr`
to a unique address (in combination with a proper Service definition).
This to ensure the Artifacts handled by the sharding controller point
to a unique endpoint.
In addition, Source object kinds which have a dependency on another
kind (i.e. a HelmChart on a HelmRepository) need to have the same
labels applied to work as expected.
Signed-off-by: Hidde Beydals <hidde@hhh.computer>
- github.com/Azure/azure-sdk-for-go/sdk/azcore to v1.4.0
- github.com/fluxcd/pkg/git/gogit to v0.8.1
- github.com/fluxcd/pkg/gittestserver to v0.8.2
- github.com/fluxcd/pkg/oci to v0.21.1
- github.com/fluxcd/pkg/ssh to v0.7.3
- github.com/google/go-containerregistry/pkg/authn/k8schain to v0.0.0-20230307034325-57f010d26af8
- golang.org/x/crypto to v0.7.0
Signed-off-by: Hidde Beydals <hidde@hhh.computer>
This uses the newly introduced helper from runtime, which also
configures the logger for `klog`.
Resulting in all logs now being properly formatted in, even when logged
by internal Kubernetes elements like the leader election or a dynamic
client.
Signed-off-by: Hidde Beydals <hidde@hhh.computer>
- github.com/fluxcd/pkg/apis/event to v0.4.1
- github.com/fluxcd/pkg/apis/meta to v0.19.1
- github.com/fluxcd/pkg/oci to v0.20.1
- github.com/fluxcd/pkg/runtime to v0.30.0
- github.com/fluxcd/pkg/ssh to v0.7.2
- github.com/google/go-containerregistry/pkg/authn/k8schain to v0.0.0-20230227161101-1b8dc2babc55
- github.com/onsi/gomega to v1.27.2
- google.golang.org/api to v0.111.0
- k8s.io/api to v0.26.2
- k8s.io/apimachinery to v0.26.2
- k8s.io/client-go to v0.26.2
- sigs.k8s.io/controller-runtime to v0.14.5
- Unpin github.com/emicklei/go-restful as it is no longer an active
dependency.
Signed-off-by: Hidde Beydals <hidde@hhh.computer>
Use `commit.AbsoluteReference()` to show the full reference when
`.spec.ref.name` is provided. For eg: `refs/heads/main@sha1:<SHA>`.
Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
- github.com/distribution/distribution/v3 to v3.0.0-20230223072852-e5d5810851d1
- github.com/fluxcd/pkg/oci to v0.20.0
- github.com/fluxcd/pkg/sourceignore to v0.3.2
- github.com/google/go-containerregistry/pkg/authn/k8schain to v0.0.0-20230217043738-4a0e0af4bf95
- github.com/minio/minio-go/v7 to v7.0.49
- github.com/onsi/gomega to v1.27.1
- github.com/sigstore/sigstore to v1.5.2
- k8s.io/utils to v0.0.0-20230220204549-a5ecb0141aa5
Signed-off-by: Hidde Beydals <hello@hidde.co>
- github.com/distribution/distribution/v3 to v3.0.0-20230214150026-36d8c594d7aa
- github.com/fluxcd/pkg/git to v0.10.0
- github.com/fluxcd/pkg/git/gogit to v0.7.1
- github.com/fluxcd/pkg/gittestserver to v0.8.1
- github.com/fluxcd/pkg/helmtestserver to v0.11.1
- github.com/fluxcd/pkg/oci to v0.19.1
- github.com/fluxcd/pkg/runtime to v0.29.0
- github.com/fluxcd/pkg/sourceignore to v0.3.1
- github.com/fluxcd/pkg/ssh to v0.7.1
- github.com/fluxcd/pkg/version to v0.2.1
Signed-off-by: Hidde Beydals <hello@hidde.co>
This allows using the condition checker as a test helper with proper
test like assertion failure and stacktrace.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
- cloud.google.com/go/storage to version 1.28.1.
- github.com/AdaLogics/go-fuzz-headers to version 0.0.0-20221206110420-d395f97c4830.
- github.com/Azure/azure-sdk-for-go/sdk/storage/azblob to version 0.6.1.
- github.com/distribution/distribution/v3 to version 3.0.0-20221208165359-362910506bc2.
- github.com/google/go-containerregistry/pkg/authn/k8schain to version 0.0.0-20221213180026-23d895d08035.
- github.com/minio/minio-go/v7 to version 7.0.45.
- github.com/onsi/gomega to version 1.24.2.
- github.com/sigstore/sigstore to version 1.5.0.
- golang.org/x/crypto to version 0.4.0.
- google.golang.org/api to version 0.105.0.
- k8s.io/utils to version 0.0.0-20221128185143-99ec85e7a448.
- github.com/docker/cli to version 20.10.22+incompatible.
- github.com/fluxcd/pkg/git/gogit to version 0.4.0.
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
This drops the usage of `play.min.io` in favor of running a local
container image for tests. Which is a better practice, and resolves
test flakiness due to the endpoint being unavailable once in awhile.
Signed-off-by: Hidde Beydals <hello@hidde.co>
- cloud.google.com/go/storage to v1.28.0
- github.com/distribution/distribution/v3 to v3.0.0-20221108081720-e9a25da7a47e
- github.com/docker/cli to v20.10.21+incompatible
- github.com/google/go-containerregistry to v0.12.0
- github.com/google/go-containerregistry/pkg/authn/k8schain to v0.0.0-20221103173901-353a117661ae
- github.com/minio/minio-go/v7 to v7.0.43
- github.com/onsi/gomega to v1.24.0
- github.com/prometheus/client_golang to v1.14.0
- github.com/sigstore/sigstore to v1.4.5
- google.golang.org/api to v0.102.0
- k8s.io/utils to v0.0.0-20221107191617-1a15be271d1d
- sigs.k8s.io/cli-utils to v0.34.0
- sigs.k8s.io/controller-runtime to v0.13.1
Signed-off-by: Hidde Beydals <hello@hidde.co>
Dependencies updated:
- cloud.google.com/go/storage to version 1.25.0.
- github.com/Azure/azure-sdk-for-go/sdk/azcore to version 1.1.2.
- github.com/ProtonMail/go-crypto to version 0.0.0-20220812175011-7fcef0dbe794.
- github.com/distribution/distribution/v3 to version 3.0.0-20220816150328-6c237953cbbe.
- github.com/fluxcd/gitkit to version 0.6.0.
- github.com/fluxcd/pkg/gittestserver to version 0.6.0.
- github.com/fluxcd/pkg/oci to version 0.5.1.
- github.com/google/go-containerregistry to version 0.11.0.
- github.com/google/go-containerregistry/pkg/authn/k8schain to version 0.0.0-20220808030257-7196cf3dc436.
- github.com/minio/minio-go/v7 to version 7.0.34.
- github.com/onsi/gomega to version 1.20.0.
- github.com/prometheus/client_golang to version 1.13.0.
- golang.org/x/crypto to version 0.0.0-20220722155217-630584e8d5aa.
- golang.org/x/net to version 0.0.0-20220812174116-3211cb980234.
- golang.org/x/sync to version 0.0.0-20220722155255-886fb9371eb4.
- google.golang.org/api to version 0.93.0.
- helm.sh/helm/v3 to version 3.9.3.
- k8s.io/api to version 0.24.3.
- k8s.io/apimachinery to version 0.24.3.
- k8s.io/client-go to version 0.24.3.
- k8s.io/utils to version 0.0.0-20220812165043-ad590609e2e5.
Replace removed for gopkg.in/yaml.v3@v3.0.1 and github.com/containerd/containerd@v1.6.6,
as they no longer need to be enforced.
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
Use of MUSL was a temporary solution to mitigate cross-platform
issues while building openssl and libssh2. Since Unmanaged transport has
been deprecated, openssl and libssh2 dependencies are no longer required
and by extension MUSL.
Enables libgit2 threadless support and provides a regression assurance
for fluxcd/image-automation-controller#339.
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
- cloud.google.com/go/storage to version 1.23.0.
- github.com/ProtonMail/go-crypto to version 0.0.0-20220623141421-5afb4c282135.
- github.com/distribution/distribution/v3 to version 3.0.0-20220702071910-8857a1948739.
- github.com/minio/minio-go/v7 to version 7.0.31.
- golang.org/x/crypto to version 0.0.0-20220622213112-05595931fe9d.
- golang.org/x/net to version 0.0.0-20220706163947-c90051bbdb60.
- google.golang.org/api to version 0.86.0.
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
- github.com/fluxcd/pkg/apis/meta to version 0.14.2.
- github.com/fluxcd/pkg/runtime to version 0.16.2.
- google.golang.org/api to version 0.83.0.
- k8s.io/api to version 0.24.1.
- github.com/fluxcd/pkg/apis/meta to version 0.14.2.
- k8s.io/apimachinery to version 0.24.1.
- github.com/fluxcd/pkg/helmtestserver to version 0.7.4.
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
Earlier, host key verification could potentially fail if there were
multiple entries in the known_hosts file and if the intended encryption
algorithm wasn't the first entry. This happened because we used the same
hasher object to compute the sum of all the public keys present in the
known_hosts file, which led to invalid hashes, resulting in a mismatch
when compared with the hash of the advertised public key. This is fixed,
by not creating the hasher ourselves and instead delegating that to the
function actually doing the matching, ensuring that a new hasher is used
for each comparison.
Regression introduced in v0.25.0 and reported in
https://github.com/fluxcd/image-automation-controller/issues/378
Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
- github.com/distribution/distribution/v3 to version 3.0.0-20220526142353-ffbd94cbe269.
- github.com/docker/cli to version 20.10.17+incompatible.
- github.com/elazarl/goproxy to version 0.0.0-20220529153421-8ea89ba92021.
- github.com/fluxcd/pkg/gittestserver to version 0.5.4.
- github.com/fluxcd/pkg/helmtestserver to version 0.7.3.
- github.com/fluxcd/pkg/ssh to version 0.4.1.
- github.com/minio/minio-go/v7 to version 7.0.27.
- golang.org/x/crypto to version 0.0.0-20220525230936-793ad666bf5e.
- golang.org/x/net to version 0.0.0-20220607020251-c690dde0001d.
- golang.org/x/sync to version 0.0.0-20220601150217-0de741cfad7f.
- google.golang.org/api to version 0.82.0.
- github.com/containerd/containerd to version v1.6.6.
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
- cloud.google.com/go/storage to v1.22.1
- github.com/ProtonMail/go-crypto to v0.0.0-20220517143526-88bb52951d5b
- github.com/distribution/distribution/v3 to v3.0.0-20220516112011-c202b9b0d7b7
- github.com/docker/cli to v20.10.16+incompatible
- github.com/fluxcd/gitkit to v0.5.1
- github.com/fluxcd/pkg/helmtestserver to v0.7.2
- github.com/fluxcd/pkg/ssh to v0.4.0
- github.com/phayes/freeport to v0.0.0-20220201140144-74d24b5ae9f5
- github.com/prometheus/client_golang to v1.12.2
- golang.org/x/crypto to v0.0.0-20220518034528-6f7dac969898
- golang.org/x/net to v0.0.0-20220524220425-1d687d428aca
- golang.org/x/sync to v0.0.0-20220513210516-0976fa681c29
- google.golang.org/api to v0.81.0
- sigs.k8s.io/cli-utils to v0.31.1
Signed-off-by: Hidde Beydals <hello@hidde.co>
- fluxcd/pkg/apis/meta v0.14.0
- fluxcd/pkg/runtime v0.16.0
- k8s.io/* v0.24.0
- helm.sh/helm/v3 v3.9.0-rc.1 (required by breaking changes in Kubernetes 1.24)
Note that fluxcd/pkg/runtime v0.16 comes with support for Kubernetes API Priority and Fairness feature.
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
* Add OCI Helm support
* users will be able to declare OCI HelmRepository by using the `.spec.type` field of the HelmRepository API. Contrary to the HTTP/S HelmRepository no index.yaml is reconciled from source, instead a simple url and credentials validation is performed.
* For backwards-compatibility, an empty `.spec.type` field leads to the HelmRepository being treated as a plain old HTTP Helm repository.
* users will be able to declare the new OCI HelmRepository type as source using the .Spec.SourceRef field of the HelmChart API. This will result in reconciling a chart from an OCI repository.
* Add registryTestServer in the test suite and OCI HelmRepository test case
* Add a new OCI chart repository type that manage tags and charts from an OCI registry.
* Adapat RemoteBuilder to accept both repository types
* discard output from OCI registry client; The client has no way to set a verbosity level and spamming the controller logs with "Login succeeded" every time the object is reconciled doesn't help much.
Signed-off-by: Soule BA <soule@weave.works>
Signed-off-by: Max Jonas Werner <mail@makk.es>
Co-authored-by: Soule BA <soule@weave.works>
Hashed known_hosts was previously only supported when using
go-git. Now both Git implementations benefit from this
features, and the code coverage across them can ensure no
future regression.
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
OptimizedGitClones decreases resource utilization for GitRepository
reconciliations. It supports both go-git and libgit2 implementations
when cloning repositories using branches or tags.
This is an opt-out feature, which can be disabled by starting the
controller with the argument '--feature-gates=OptimizedGitClones=false'.
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
- github.com/fluxcd/pkg/runtime to v0.14.2
- golang.org/x/crypto to v0.0.0-20220427172511-eb4f295cb31f
- google.golang.org/api to v0.77.0
- k8s.io/api to v0.23.6
- k8s.io/client-go to v0.23.6
Signed-off-by: Hidde Beydals <hello@hidde.co>
Azure SDK dependencies cannot be updated, as this requires us to move to
Go 1.18.
- cloud.google.com/go/storage to v1.22.0
- github.com/ProtonMail/go-crypto to v0.0.0-20220407094043-a94812496cf5
- github.com/darkowlzz/controller-check to v0.0.0-20220325122359-11f5827b7981
- github.com/elazarl/goproxy to v0.0.0-20220403042543-a53172b9392e
- github.com/fluxcd/pkg/gittestserver to v0.5.2
- github.com/go-logr/logr to v1.2.3
- github.com/minio/minio-go/v7 to v7.0.24
- github.com/onsi/gomega to v1.19.0
- golang.org/x/crypto to v0.0.0-20220411220226-7b82a4e95df4
- google.golang.org/api to v0.74.0
Signed-off-by: Hidde Beydals <hello@hidde.co>
Add two new flags to enable users to configure exponential
back-off for Flux objects. The default values are now
set to 750ms for minimum retry time, and 15min for max.
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
Introduce separate positive polarity conditions which are used to set
Ready condition. Move the "artifact stored" ready condition into
ArtifactInStorage positive polarity condition. If ArtifactInStorage is
True and there's no negative polarity condition present, the Ready
condition is summarized with ArtifactInStorage condition value.
Also, update the priorities of the conditions. ArtifactInStorage has
higher priority than SourceVerfied condition. If both are present, the
Ready condition will have ArtifactInStorage.
The negative polarity conditions are reordered to have the most likely
actual cause of failure condition the highest priority, for example
StorageOperationFailed, followed by the conditions that are reconciled
first in the whole reconciliation so as to prioritize the first failure
which may be the cause of subsequent failures.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
- cloud.google.com/go/storage to v1.21.0
- github.com/Azure/azure-sdk-for-go/sdk/azcore to v0.22.0
- github.com/Azure/azure-sdk-for-go/sdk/azidentity to v0.13.2
- github.com/elazarl/goproxy to v0.0.0-20220115173737-adb46da277ac
- github.com/fluxcd/pkg/apis/meta to v0.12.1
- github.com/fluxcd/pkg/gittestserver to v0.5.1
- github.com/fluxcd/pkg/helmtestserver to v0.7.1
- github.com/fluxcd/pkg/runtime to v0.13.2
- github.com/fluxcd/pkg/ssh to v0.3.2
- github.com/libgit2/git2go/v33 to v33.0.9
- github.com/minio/minio-go/v7 to v7.0.23
- github.com/onsi/gomega to v1.18.1
- google.golang.org/api to v0.73.0
- sigs.k8s.io/cli-utils to v0.29.3
Signed-off-by: Hidde Beydals <hello@hidde.co>
This commit introduces an Azure Blob BucketProvider implementation,
capable of fetching from objects from public and private "container"
buckets.
The supported credential types are:
- ManagedIdentity with a `resourceId` Secret data field.
- ManagedIdentity with a `clientId` Secret data field.
- ClientSecret with `tenantId`, `clientId` and `clientSecret` Secret
data fields.
- SharedKey with `accountKey` Secret data field, the Account Name is
extracted from the endpoint URL specified on the object.
If no Secret is provided, the Bucket is assumed to be public.
Co-authored-by: Zhongcheng Lao <Zhongcheng.Lao@microsoft.com>
Signed-off-by: Hidde Beydals <hello@hidde.co>
Reuses the same transport across different helm chart downloads,
whilst resetting the tlsconfig to avoid cross-contamination.
Crypto material is now only processed in-memory and does not
touch the disk.
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
This commit updates to a version of Helm 3.8.0, with patches applied to
deal with memory leak and HTTP transport issues. The latter being
described in https://github.com/fluxcd/source-controller/issues/578.
Signed-off-by: Hidde Beydals <hello@hidde.co>
- Remove ArtifactUnavailable condition and use Reconciling condition to
convey the same.
- Make Reconciling condition affect the ready condition.
- Introduce summarizeAndPatch() to calculate the final status conditions
and patch them.
- Introduce reconcile() to iterate through the sub-reconcilers and
execute them.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
This commit ensures all API objects implement the interfaces used by
the runtime package to work with conditions, etc., and prepares the
test suite to work with the `pkg/runtime/testenv` wrapper.
Changes are made in a backwards compatible way (that being: the
existing code can still be build and works as expected), but without
proper dependency boundaries. The result of this is that the API
package temporary depends on the runtime package, which is resolved
when all reconcilers have been refactored and the API package does
no longer contain condition modifying functions.
Signed-off-by: Hidde Beydals <hello@hidde.co>
Downstream breaking changes introduced since git2go@V31:
- git2go.ErrorCode was deprecated in favour of the native error type.
- FetchOptions no longer expects a pointer, but rather the actual value of git2go.FetchOptions.
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
This configures ProxyOptions for all libgit2 Checkout functions when
cloning and configures the options based on current environment
settings using the git2go.ProxyTypeAuto option.
Refs: #131
Signed-off-by: Robert Clarke <rob@robertandrewclarke.com>
Co-authored-by: Aurélien GARNIER <aurelien.garnier@atos.net>
To mitigate warnings for CVE-2021-41092. Because even if there is no
impact whatsoever, we are nice people.
Signed-off-by: Hidde Beydals <hello@hidde.co>
This mitigates another warning for CVE-2017-11468, which is mostly
triggered because a part of Helm depends on it that our code paths
never reach.
Signed-off-by: Hidde Beydals <hello@hidde.co>
- Adds tests for the libgit2 remote callbacks
- Adds tests for CheckoutStrategyForImplementation with context timeout
and verify timeout is respected by both the git implementations.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
Adds tests for git.CheckoutStrategy to check if both the git
implementations work with all the authentication methods.
Signed-off-by: Sunny <darkowlzz@protonmail.com>
Added Support for Google Cloud Storage with Workload Identity as
Source Provider. This enables the use of GCP without enabling S3
compatible access.
Signed-off-by: pa250194 <pa250194@ncr.com>
This commit updates `github.com/libgit2/git2go` to `v31.6.1` (with
`libgit2` `1.1.1`), and changes the container image build process so
that it makes use of `ghcr.io/hiddeco/golang-with-libgit2`.
This image provides a pre-build dynamic `libgit2` dependency linked
against OpenSSL and LibSSH2 (without gcrypt), and a set of cross-compile
build tools (see
[rationale](https://github.com/hiddeco/golang-with-libgit2#rationale) and
[usage](https://github.co/hiddeco/golang-with-libgit2#usage) for more
detailed information).
The linked set of dependency should solve most known issues around
unsupport private key types, but does not resolve the issues with ECDSA*
and ED25519 hostkeys yet. Solving this requires a newer version of
`libgit2` (`>=1.2.0`), which currently does not seem to work properly
with `git2go/v32`.
Some small changes have been made to the `libgit2` package to address
(future) deprecations.
Signed-off-by: Hidde Beydals <hello@hidde.co>
Added Support for Google Cloud Storage with Workload Identity as Source Provider. This enables the use of GCP without enabling S3 compatible access.
Signed-off-by: pa250194 <pa250194@ncr.com>
v3.6.1 is a a security update from Helm, ensuring that credentials are
always only passed to the defined repository host.
Based on Helm user reports, disabling this behavior may be required for
some Helm repository solutions like Artifactory, and may be done by
setting `PassCredentials` in the `HelmRepositorySpec`.
For more information, see:
https://github.com/helm/helm/security/advisories/GHSA-56hp-xqp3-w2jf
Signed-off-by: Hidde Beydals <hello@hidde.co>
Controller-runtime has been updated to `v0.9.0`, K8s dependencies to
`v0.21.1`, and all `fluxcd/pkg` and other dependencies to the versions
that have matching dependencies and/or build constraints.
Signed-off-by: Hidde Beydals <hello@hidde.co>
This should resolve `object not found` and
`empty git-upload-pack given` errors that were thrown for Git
repositories that used to work fine before the `v0.13.0` release.
Signed-off-by: Hidde Beydals <hello@hidde.co>
Sanskar Jaiswal