Commit Graph

148 Commits

Author SHA1 Message Date
Stefan Prodan 6a367ec70b
Update Kubernetes packages to v1.25.0
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-08-26 12:26:38 +03:00
Paulo Gomes aac18f5273
Update dependencies
Dependencies updated:
- cloud.google.com/go/storage to version 1.25.0.
- github.com/Azure/azure-sdk-for-go/sdk/azcore to version 1.1.2.
- github.com/ProtonMail/go-crypto to version 0.0.0-20220812175011-7fcef0dbe794.
- github.com/distribution/distribution/v3 to version 3.0.0-20220816150328-6c237953cbbe.
- github.com/fluxcd/gitkit to version 0.6.0.
- github.com/fluxcd/pkg/gittestserver to version 0.6.0.
- github.com/fluxcd/pkg/oci to version 0.5.1.
- github.com/google/go-containerregistry to version 0.11.0.
- github.com/google/go-containerregistry/pkg/authn/k8schain to version 0.0.0-20220808030257-7196cf3dc436.
- github.com/minio/minio-go/v7 to version 7.0.34.
- github.com/onsi/gomega to version 1.20.0.
- github.com/prometheus/client_golang to version 1.13.0.
- golang.org/x/crypto to version 0.0.0-20220722155217-630584e8d5aa.
- golang.org/x/net to version 0.0.0-20220812174116-3211cb980234.
- golang.org/x/sync to version 0.0.0-20220722155255-886fb9371eb4.
- google.golang.org/api to version 0.93.0.
- helm.sh/helm/v3 to version 3.9.3.
- k8s.io/api to version 0.24.3.
- k8s.io/apimachinery to version 0.24.3.
- k8s.io/client-go to version 0.24.3.
- k8s.io/utils to version 0.0.0-20220812165043-ad590609e2e5.

Replace removed for gopkg.in/yaml.v3@v3.0.1 and github.com/containerd/containerd@v1.6.6,
as they no longer need to be enforced.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-08-17 16:57:34 +01:00
Stefan Prodan f873d71ec5
Use sourceignore from fluxcd/pkg
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-08-17 18:04:48 +03:00
Somtochi Onyekwere 35268638ba Add Support for SAS keys in Azure Blob
Signed-off-by: Somtochi Onyekwere <somtochionyekwere@gmail.com>
2022-08-11 20:49:59 +01:00
Stefan Prodan 94e98ee5ca
Add the opencontainers annotations to API docs
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-08-08 12:58:04 +03:00
Stefan Prodan 63c94397f7
Implement OCI auth for cloud providers
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-08-04 13:29:03 +03:00
Somtochi Onyekwere 25b88256ef
Add tests for reconcile delete
Signed-off-by: Somtochi Onyekwere <somtochionyekwere@gmail.com>
2022-08-02 13:11:08 +03:00
Philip Laine fdf6b01817
Update Azure Go SDK
Signed-off-by: Philip Laine <philip.laine@gmail.com>
2022-08-01 12:27:01 +03:00
Paulo Gomes f51f9d3305
Remove MUSL and enable threadless libgit2 support
Use of MUSL was a temporary solution to mitigate cross-platform
issues while building openssl and libssh2. Since Unmanaged transport has
been deprecated, openssl and libssh2 dependencies are no longer required
and by extension MUSL.

Enables libgit2 threadless support and provides a regression assurance
for fluxcd/image-automation-controller#339.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-07-31 18:38:27 +01:00
Soule BA 3717f02d46
Bump Helm to v3.9.1
Signed-off-by: Soule BA <soule@weave.works>
2022-07-14 01:18:27 +02:00
Paulo Gomes 73ce792414
Fix github.com/emicklei/go-restful (CVE-2022-1996)
This addresses CVE-2022-1996, due to v2.16.0 including
emicklei/go-restful@9266625.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-07-13 09:49:18 +01:00
Paulo Gomes 0633c57629
Update dependencies
- cloud.google.com/go/storage to version 1.23.0.
- github.com/ProtonMail/go-crypto to version 0.0.0-20220623141421-5afb4c282135.
- github.com/distribution/distribution/v3 to version 3.0.0-20220702071910-8857a1948739.
- github.com/minio/minio-go/v7 to version 7.0.31.
- golang.org/x/crypto to version 0.0.0-20220622213112-05595931fe9d.
- golang.org/x/net to version 0.0.0-20220706163947-c90051bbdb60.
- google.golang.org/api to version 0.86.0.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-07-07 17:23:22 +01:00
Sunny f941f5ce24
Update go-yaml to v3.0.1
Fix CVE-2022-28948

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-06-30 00:21:59 +05:30
Soule BA 988fd1fba9
disable cache in oci registry client
If implemented, the oci registry client is created with cache disabled.

Signed-off-by: Soule BA <soule@weave.works>
2022-06-29 12:47:40 +02:00
Paulo Gomes ea7027ac54
Update dependencies
- github.com/fluxcd/pkg/apis/meta to version 0.14.2.
- github.com/fluxcd/pkg/runtime to version 0.16.2.
- google.golang.org/api to version 0.83.0.
- k8s.io/api to version 0.24.1.
- github.com/fluxcd/pkg/apis/meta to version 0.14.2.
- k8s.io/apimachinery to version 0.24.1.
- github.com/fluxcd/pkg/helmtestserver to version 0.7.4.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-06-08 10:19:05 +01:00
Sanskar Jaiswal 47692530d7 libgit2: fix ssh host key verification regression
Earlier, host key verification could potentially fail if there were
multiple entries in the known_hosts file and if the intended encryption
algorithm wasn't the first entry. This happened because we used the same
hasher object to compute the sum of all the public keys present in the
known_hosts file, which led to invalid hashes, resulting in a mismatch
when compared with the hash of the advertised public key. This is fixed,
by not creating the hasher ourselves and instead delegating that to the
function actually doing the matching, ensuring that a new hasher is used
for each comparison.

Regression introduced in v0.25.0 and reported in
https://github.com/fluxcd/image-automation-controller/issues/378

Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
2022-06-08 13:59:32 +05:30
Paulo Gomes 7c9fb1dd25
Update dependencies
- github.com/distribution/distribution/v3 to version 3.0.0-20220526142353-ffbd94cbe269.
- github.com/docker/cli to version 20.10.17+incompatible.
- github.com/elazarl/goproxy to version 0.0.0-20220529153421-8ea89ba92021.
- github.com/fluxcd/pkg/gittestserver to version 0.5.4.
- github.com/fluxcd/pkg/helmtestserver to version 0.7.3.
- github.com/fluxcd/pkg/ssh to version 0.4.1.
- github.com/minio/minio-go/v7 to version 7.0.27.
- golang.org/x/crypto to version 0.0.0-20220525230936-793ad666bf5e.
- golang.org/x/net to version 0.0.0-20220607020251-c690dde0001d.
- golang.org/x/sync to version 0.0.0-20220601150217-0de741cfad7f.
- google.golang.org/api to version 0.82.0.
- github.com/containerd/containerd to version v1.6.6.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-06-07 15:29:06 +01:00
Sanskar Jaiswal 7d2bc64f47 fix panics on unmanaged http and proxy on managed http
Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
2022-05-27 14:01:23 +05:30
Hidde Beydals 7fc9798405
Update dependencies
- cloud.google.com/go/storage to v1.22.1
- github.com/ProtonMail/go-crypto to v0.0.0-20220517143526-88bb52951d5b
- github.com/distribution/distribution/v3 to v3.0.0-20220516112011-c202b9b0d7b7
- github.com/docker/cli to v20.10.16+incompatible
- github.com/fluxcd/gitkit to v0.5.1
- github.com/fluxcd/pkg/helmtestserver to v0.7.2
- github.com/fluxcd/pkg/ssh to v0.4.0
- github.com/phayes/freeport to v0.0.0-20220201140144-74d24b5ae9f5
- github.com/prometheus/client_golang to v1.12.2
- golang.org/x/crypto to v0.0.0-20220518034528-6f7dac969898
- golang.org/x/net to v0.0.0-20220524220425-1d687d428aca
- golang.org/x/sync to v0.0.0-20220513210516-0976fa681c29
- google.golang.org/api to v0.81.0
- sigs.k8s.io/cli-utils to v0.31.1

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-05-27 11:11:44 +03:00
Stefan Prodan 441f4c53c8
Update runc to v1.1.2
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-05-27 10:59:40 +03:00
Paulo Gomes b0c59d1fbb
Update dependencies
- fluxcd/pkg/apis/meta v0.14.1
- fluxcd/pkg/runtime v0.16.1
- gopkg.in/yaml.v3 v3.0.0

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-05-26 14:48:36 +01:00
Stefan Prodan f3a372da60
Update dependencies
- fluxcd/pkg/apis/meta v0.14.0
- fluxcd/pkg/runtime v0.16.0
- k8s.io/* v0.24.0
- helm.sh/helm/v3 v3.9.0-rc.1 (required by breaking changes in Kubernetes 1.24)

Note that fluxcd/pkg/runtime v0.16 comes with support for Kubernetes API Priority and Fairness feature.

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-05-24 14:48:32 +03:00
Max Jonas Werner 841ed7ae66
[RFC 0002] Flux OCI support for Helm (#690)
* Add OCI Helm support

* users will be able to declare OCI HelmRepository by using the `.spec.type` field of the HelmRepository API. Contrary to the HTTP/S HelmRepository no index.yaml is reconciled from source, instead a simple url and credentials validation is performed.
* For backwards-compatibility, an empty `.spec.type` field leads to the HelmRepository being treated as a plain old HTTP Helm repository.
* users will be able to declare the new OCI HelmRepository type as source using the .Spec.SourceRef field of the HelmChart API. This will result in reconciling a chart from an OCI repository.
* Add registryTestServer in the test suite and OCI HelmRepository test case
* Add a new OCI chart repository type that manage tags and charts from an OCI registry.
* Adapat RemoteBuilder to accept both repository types
* discard output from OCI registry client; The client has no way to set a verbosity level and spamming the controller logs with "Login succeeded" every time the object is reconciled doesn't help much.

Signed-off-by: Soule BA <soule@weave.works>
Signed-off-by: Max Jonas Werner <mail@makk.es>
Co-authored-by: Soule BA <soule@weave.works>
2022-05-19 14:50:16 +02:00
Paulo Gomes 8b50367849
libgit2: Add support for hashed known_hosts
Hashed known_hosts was previously only supported when using
go-git. Now both Git implementations benefit from this
features, and the code coverage across them can ensure no
future regression.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-05-16 16:57:22 +01:00
Paulo Gomes 262efc08bc
Add git-optimized-clones feature gate
OptimizedGitClones decreases resource utilization for GitRepository
reconciliations. It supports both go-git and libgit2 implementations
when cloning repositories using branches or tags.

This is an opt-out feature, which can be disabled by starting the
controller with the argument '--feature-gates=OptimizedGitClones=false'.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-05-11 14:39:21 +01:00
Paulo Gomes 2b59150fbe
tests: algorithms test coverage for go-git and libgit2
Assures support for:
- Authentication Key Types
  - rsa
  - ecdsa P256
  - ecdsa P384
  - ecdsa P521
  - ed25519
- Key Exchange Algoritms:
  - diffie-hellman-group14-sha1
  - diffie-hellman-group14-sha256
  - curve25519-sha256
  - ecdh-sha2-nistp256
  - ecdh-sha2-nistp384
  - ecdh-sha2-nistp521
  - curve25519-sha256@libssh.org
- HostKey Algoritms:
  - ssh-rsa
  - rsa-sha2-256
  - rsa-sha2-512
  - ecdsa-sha2-nistp256
  - ecdsa-sha2-nistp384
  - ecdsa-sha2-nistp521
  - ssh-ed25519

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-05-09 13:31:54 +01:00
Hidde Beydals 5b32ef8269 Update dependencies
- github.com/fluxcd/pkg/runtime to v0.14.2
- golang.org/x/crypto to v0.0.0-20220427172511-eb4f295cb31f
- google.golang.org/api to v0.77.0
- k8s.io/api to v0.23.6
- k8s.io/client-go to v0.23.6

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-05-03 12:44:15 +02:00
Hidde Beydals 67ef697630 Update github.com/elazarl/goproxy
Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-04-19 12:19:54 +02:00
Hidde Beydals 18b59351c1 Update dependencies
Azure SDK dependencies cannot be updated, as this requires us to move to
Go 1.18.

- cloud.google.com/go/storage to v1.22.0
- github.com/ProtonMail/go-crypto to v0.0.0-20220407094043-a94812496cf5
- github.com/darkowlzz/controller-check to v0.0.0-20220325122359-11f5827b7981
- github.com/elazarl/goproxy to v0.0.0-20220403042543-a53172b9392e
- github.com/fluxcd/pkg/gittestserver to v0.5.2
- github.com/go-logr/logr to v1.2.3
- github.com/minio/minio-go/v7 to v7.0.24
- github.com/onsi/gomega to v1.19.0
- golang.org/x/crypto to v0.0.0-20220411220226-7b82a4e95df4
- google.golang.org/api to v0.74.0

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-04-15 16:41:42 +02:00
Hidde Beydals 59df52d332 Update sigs.k8s.io/cli-utils to v0.29.4
Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-04-15 15:47:41 +02:00
Stefan Prodan 0f64fef2a1
Update dependencies
- helm.sh/helm/v3 v3.8.2
- k8s.io/api v0.23.5
- fluxcd/pkg/runtime v0.14.1

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-04-15 10:49:31 +03:00
Paulo Gomes 4198191759
Add flags to configure exponential back-off retry
Add two new flags to enable users to configure exponential
back-off for Flux objects. The default values are now
set to 750ms for minimum retry time, and 15min for max.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-04-12 10:44:46 +01:00
Sunny d939e98ec2
Introduce separate positive polarity conditions
Introduce separate positive polarity conditions which are used to set
Ready condition. Move the "artifact stored" ready condition into
ArtifactInStorage positive polarity condition. If ArtifactInStorage is
True and there's no negative polarity condition present, the Ready
condition is summarized with ArtifactInStorage condition value.

Also, update the priorities of the conditions. ArtifactInStorage has
higher priority than SourceVerfied condition. If both are present, the
Ready condition will have ArtifactInStorage.
The negative polarity conditions are reordered to have the most likely
actual cause of failure condition the highest priority, for example
StorageOperationFailed, followed by the conditions that are reconciled
first in the whole reconciliation so as to prioritize the first failure
which may be the cause of subsequent failures.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-03-30 17:02:59 +05:30
Paulo Gomes b73f18a091
Update golang.org/x/crypto
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-03-25 19:08:53 +00:00
Hidde Beydals 55b07f97f0 Update various dependencies
- cloud.google.com/go/storage to v1.21.0
- github.com/Azure/azure-sdk-for-go/sdk/azcore to v0.22.0
- github.com/Azure/azure-sdk-for-go/sdk/azidentity to v0.13.2
- github.com/elazarl/goproxy to v0.0.0-20220115173737-adb46da277ac
- github.com/fluxcd/pkg/apis/meta to v0.12.1
- github.com/fluxcd/pkg/gittestserver to v0.5.1
- github.com/fluxcd/pkg/helmtestserver to v0.7.1
- github.com/fluxcd/pkg/runtime to v0.13.2
- github.com/fluxcd/pkg/ssh to v0.3.2
- github.com/libgit2/git2go/v33 to v33.0.9
- github.com/minio/minio-go/v7 to v7.0.23
- github.com/onsi/gomega to v1.18.1
- google.golang.org/api to v0.73.0
- sigs.k8s.io/cli-utils to v0.29.3

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-03-16 11:15:59 +01:00
Hidde Beydals 44198da663 Update golang.org/x/crypto to latest
This pins it to the latest version which introduces support for
rsa-ssh2-256/512.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-03-15 22:17:01 +01:00
Hidde Beydals 458d67d0a2 Remove github.com/docker/cli replace
This was required by an older version of Helm, which has now updated
the dependency itself.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-03-15 22:11:52 +01:00
Hidde Beydals c5bfe4f227 Update github.com/docker/distribution to v2.8.0
This mitigates GMS-2022-20.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-03-15 22:10:53 +01:00
Hidde Beydals 91aba67781 Update Helm to patched v3.8.1
(Forked) version still includes patch to manage HTTP transport.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-03-11 12:53:30 +01:00
Hidde Beydals ec5bc1ac9a Implement Azure Blob BucketProvider
This commit introduces an Azure Blob BucketProvider implementation,
capable of fetching from objects from public and private "container"
buckets.

The supported credential types are:

- ManagedIdentity with a `resourceId` Secret data field.
- ManagedIdentity with a `clientId` Secret data field.
- ClientSecret with `tenantId`, `clientId` and `clientSecret` Secret
  data fields.
- SharedKey with `accountKey` Secret data field, the Account Name is
  extracted from the endpoint URL specified on the object.

If no Secret is provided, the Bucket is assumed to be public.

Co-authored-by: Zhongcheng Lao <Zhongcheng.Lao@microsoft.com>
Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-03-03 13:20:48 +01:00
Stefan Prodan fe773c510d
Update dependencies
- runc v1.1.0 (CVE fix https://github.com/advisories/GHSA-fgv8-vj5c-2ppq)
- containerd v1.6.1 (CVE fix https://github.com/containerd/containerd/security/advisories/GHSA-crp2-qrr5-8pq7)

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-03-03 12:49:01 +02:00
Paulo Gomes 7d61553463
Remove dependency to helm v2
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-03-02 17:58:15 +00:00
Paulo Gomes d9d789fdb1
Reuse transport for helm chart download
Reuses the same transport across different helm chart downloads,
whilst resetting the tlsconfig to avoid cross-contamination.

Crypto material is now only processed in-memory and does not
touch the disk.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-03-02 13:02:11 +00:00
Hidde Beydals b28669e446
Update to patched version of Helm 3.8.0
This commit updates to a version of Helm 3.8.0, with patches applied to
deal with memory leak and HTTP transport issues. The latter being
described in https://github.com/fluxcd/source-controller/issues/578.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-03-02 13:02:07 +00:00
Paulo Gomes be02e15410
Update github.com/sosedoff/gitkit (indirect) to v0.3.0 (CVE fix)
Fixes https://snyk.io/vuln/SNYK-GOLANG-GITHUBCOMSATORIGOUUID-72488 in the indirect dependency github.com/satori/go.uuid.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-02-25 13:25:03 +00:00
Sunny e9ae0c232e Add kstatus client conformance tests
Use kstatus to compute the status of the objects.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-02-23 12:37:45 +01:00
Hidde Beydals 07a539e3d6 build: update meta API and pkg/runtime to non RC
Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-02-23 12:35:30 +01:00
Sunny 5767291b58 gitrepo: Add more reconciler design improvements
- Remove ArtifactUnavailable condition and use Reconciling condition to
  convey the same.
- Make Reconciling condition affect the ready condition.
- Introduce summarizeAndPatch() to calculate the final status conditions
  and patch them.
- Introduce reconcile() to iterate through the sub-reconcilers and
  execute them.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-02-23 12:35:30 +01:00
Hidde Beydals 349739b7e4 Implement new runtime interfaces, prepare testenv
This commit ensures all API objects implement the interfaces used by
the runtime package to work with conditions, etc., and prepares the
test suite to work with the `pkg/runtime/testenv` wrapper.

Changes are made in a backwards compatible way (that being: the
existing code can still be build and works as expected), but without
proper dependency boundaries. The result of this is that the API
package temporary depends on the runtime package, which is resolved
when all reconcilers have been refactored and the API package does
no longer contain condition modifying functions.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-02-23 12:35:30 +01:00
Paulo Gomes 6e46d7fe55
Upgrade docker/distribution to v2.8.0
Fixes https://github.com/advisories/GHSA-qq97-vm5h-rrhg

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-02-17 20:51:48 +00:00