Commit Graph

1206 Commits

Author SHA1 Message Date
Hidde Beydals 6ebe460676 controllers: store HelmChart Artifact with suffix
This adds a Unix suffix to the HelmChart Artifact filename, to ensure
it is unique for sequential builds triggered due to e.g. a controller
restart.

The result of this is that consumers who _think_ they are fetching an
Artifact with a certain checksum run into a 404 when attempting to
download a previously advertised but now unavailable file, instead of
running into a checksum validation error (due to non-repetitive Helm
builds).

For more information, see:
https://github.com/fluxcd/source-controller/issues/610

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-03-11 22:07:04 +01:00
Hidde Beydals 02c8fbae7f
Merge pull request #609 from fluxcd/helm-3.8.1-patched 2022-03-11 13:16:32 +01:00
Hidde Beydals 91aba67781 Update Helm to patched v3.8.1
(Forked) version still includes patch to manage HTTP transport.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-03-11 12:53:30 +01:00
Hidde Beydals 4efca19efb
Merge pull request #607 from fluxcd/integration-tests 2022-03-11 10:57:41 +01:00
Hidde Beydals d17e724124 build: properly wire integration tests
- Ensure secrets are set as env variables
- Wire `GO_TAGS` together with `GO_STATIC_FLAGS`

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-03-11 10:43:35 +01:00
Hidde Beydals 7c41249caf
Merge pull request #599 from fluxcd/spec-docs-v1beta2 2022-03-11 10:35:19 +01:00
Hidde Beydals 971caf92d5 controllers: finetune `eventLogf` (variant) docs
Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-03-11 10:04:14 +01:00
Hidde Beydals ff536e10bb doc/spec: use --field-manager in kubectl examples
To align with:
https://fluxcd.io/docs/faq/#why-are-kubectl-edits-rolled-back-by-flux

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-03-11 10:04:14 +01:00
Hidde Beydals abf29da106 docs/spec: document Bucket's Azure Blob support
Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-03-11 10:04:14 +01:00
Sunny 8d4fce1955 Add more details in helmchart spec doc
Details about the source reference, reconcile strategy and artifact
revision value based on the reconcile strategy.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-03-11 10:04:14 +01:00
Sunny 9e73bfab9e docs/spec: Update the outputs
Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-03-11 10:04:14 +01:00
Sunny f59b48c339 Document HelmChart API v1beta2 spec
Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-03-11 10:04:14 +01:00
Hidde Beydals cbffd82d8f Document GitRepository API v1beta2 spec
Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-03-11 10:04:14 +01:00
Hidde Beydals 86d1d80bf2 Document HelmRepository API v1beta2 spec
Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-03-11 10:04:14 +01:00
Hidde Beydals 5832296ef5 Document Bucket API v1beta2 spec
Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-03-11 10:04:14 +01:00
Hidde Beydals ccadce6d16
Merge pull request #598 from fluxcd/azure-blob-bucket-provider 2022-03-10 10:08:59 +01:00
Hidde Beydals ccb65c74f0 Add ManagedIdentity with `AZURE_CLIENT_ID`
This ensures the Managed Identity authentication works with multiple
identities assigned to a single node.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-03-08 14:57:56 +01:00
Hidde Beydals d7ce67979f Reorder Azure token credential composition
- `authorityHost` and `clientCertificateSendChain` can now be set where
  applicable.
- AZ CLI fields have been removed.
- Fallback to `ChainedTokenCredential` with `EnvironmentCredential` and
  `ManagedIdentityCredential` with defaults if no Secret is given.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-03-08 14:57:38 +01:00
Hidde Beydals c5c9160ec5 Support fields from `az` generated Azure SP
This supports the fields as documented in the AKS documentation:
https://docs.microsoft.com/en-us/azure/aks/kubernetes-service-principal?tabs=azure-cli#manually-create-a-service-principal

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-03-08 14:48:51 +01:00
Hidde Beydals cc805b4c55 Change Azure authentication order
Based on recommendations from Microsoft, change the order valid
authentication options are taken into account. Mainly to ensure it works
as expected when multiple Managed Identities are bound on the same VM
node.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-03-08 14:48:51 +01:00
Hidde Beydals 37e602a2e3 Address review nitpicks
- Use octal syntax for permissions.
- Fix typo.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-03-08 14:48:51 +01:00
Hidde Beydals d55a75930c Enable Azure integration tests
Tests are configured in such a way that they only run for `main`.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-03-03 14:03:00 +01:00
Hidde Beydals bd12cdba17 Allow Azure ClientCertificate authentication
This commit allows for a Secret to be configured with `tenantId`,
`clientId` and `clientCertificate` data fields (with optionally
`clientCertificatePassword`) to authenticate using TLS.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-03-03 13:20:48 +01:00
Hidde Beydals 94c8185d87 controllers: wire Azure Blob Bucket Provider
Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-03-03 13:20:48 +01:00
Hidde Beydals ec5bc1ac9a Implement Azure Blob BucketProvider
This commit introduces an Azure Blob BucketProvider implementation,
capable of fetching from objects from public and private "container"
buckets.

The supported credential types are:

- ManagedIdentity with a `resourceId` Secret data field.
- ManagedIdentity with a `clientId` Secret data field.
- ClientSecret with `tenantId`, `clientId` and `clientSecret` Secret
  data fields.
- SharedKey with `accountKey` Secret data field, the Account Name is
  extracted from the endpoint URL specified on the object.

If no Secret is provided, the Bucket is assumed to be public.

Co-authored-by: Zhongcheng Lao <Zhongcheng.Lao@microsoft.com>
Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-03-03 13:20:48 +01:00
Hidde Beydals 2167498736 api: add AzureBucketProvider to Bucket
Co-authored-by: Zhongcheng Lao <Zhongcheng.Lao@microsoft.com>
Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-03-03 12:53:10 +01:00
Stefan Prodan 2410766a29
Merge pull request #600 from fluxcd/cve-fixes
Update dependencies
2022-03-03 13:47:59 +02:00
Stefan Prodan fe773c510d
Update dependencies
- runc v1.1.0 (CVE fix https://github.com/advisories/GHSA-fgv8-vj5c-2ppq)
- containerd v1.6.1 (CVE fix https://github.com/containerd/containerd/security/advisories/GHSA-crp2-qrr5-8pq7)

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-03-03 12:49:01 +02:00
Paulo Gomes 59108f48a0
Merge pull request #590 from fluxcd/helm-getter-http-transport
Reuse transport for Helm downloads
2022-03-02 18:17:28 +00:00
Paulo Gomes 7d61553463
Remove dependency to helm v2
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-03-02 17:58:15 +00:00
Paulo Gomes f63681f372
Improve TransportPool documentation
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-03-02 13:53:31 +00:00
Paulo Gomes 3b4cc52419
Use uppercase TLS in error messages
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-03-02 13:02:11 +00:00
Paulo Gomes d9d789fdb1
Reuse transport for helm chart download
Reuses the same transport across different helm chart downloads,
whilst resetting the tlsconfig to avoid cross-contamination.

Crypto material is now only processed in-memory and does not
touch the disk.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-03-02 13:02:11 +00:00
Hidde Beydals b28669e446
Update to patched version of Helm 3.8.0
This commit updates to a version of Helm 3.8.0, with patches applied to
deal with memory leak and HTTP transport issues. The latter being
described in https://github.com/fluxcd/source-controller/issues/578.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-03-02 13:02:07 +00:00
Hidde Beydals a4012f2022
Merge pull request #596 from fluxcd/bucket-provider-interface-dev 2022-03-01 10:31:10 +01:00
pa250194 ed6c6ebc3c Introduce BucketProvider interface
This commit introduces a BucketProvider interface for fetch operations
against object storage provider buckets. Allowing for easier
introduction of new provider implementations.

The algorithm for conditionally downloading object files is the same,
whether you are using GCP storage or an S3/Minio-compatible
bucket. The only thing that differs is how the respective clients
handle enumerating through the objects in the bucket; by implementing
just that in each provider, I can have the select-and-fetch code in
once place.

The client implementations do now include safe-guards to ensure the
fetched object is the same as metadata has been collected for. In
addition, minor changes have been made to the object fetch operation
to take into account that:

- Etags can change between composition of index and actual fetch, in
  which case the etag is now updated.
- Objects can disappear between composition of index and actual fetch,
  in which case the item is removed from the index.

Lastly, the requirement for authentication has been removed (and not
referring to a Secret at all is thus allowed), to provide support
for e.g. public buckets.

Co-authored-by: Hidde Beydals <hello@hidde.co>
Co-authored by: Michael Bridgen <michael@weave.works>
Signed-off-by: pa250194 <pa250194@ncr.com>
2022-03-01 10:15:10 +01:00
Hidde Beydals bae1b1094a
Merge pull request #597 from fluxcd/artifact-size 2022-03-01 07:30:29 +01:00
Kevin McDermott f7105ea736 Implement Size field on archived artifacts
This adds a Size field to Artifacts, which reflects the number of bytes
written to the artifact when it's being archived.

Signed-off-by: Kevin McDermott <bigkevmcd@gmail.com>
2022-02-28 16:01:34 +01:00
Sunny 25e6e16a75
Merge pull request #595 from fluxcd/redundant-reconciling-condition
controllers: Remove redundant reconciling condition in reconcileArtifact
2022-02-25 22:20:35 +05:30
Sunny 234b7f4c9d
Remove redundant reconciling in reconcileArtifact
reconcileSource() adds reconciling condition with accurate information.
Remove setting reconciling condition in reconcileArtifact().

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-02-25 20:39:24 +05:30
Hidde Beydals e4bfab2deb
Merge pull request #594 from pjbgf/patch-250222 2022-02-25 15:23:43 +01:00
Paulo Gomes be02e15410
Update github.com/sosedoff/gitkit (indirect) to v0.3.0 (CVE fix)
Fixes https://snyk.io/vuln/SNYK-GOLANG-GITHUBCOMSATORIGOUUID-72488 in the indirect dependency github.com/satori/go.uuid.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-02-25 13:25:03 +00:00
Sunny a5f65a2e32
Merge pull request #592 from fluxcd/printcolumn-reorder
api: Move Status in CRD printcolumn to the end
2022-02-25 15:06:06 +05:30
Sunny 0659a69016
api: Move Status in CRD printcolumn to the end
Status content could be very long compare to other fields. Moving it to
the end helps improve the visibility of other fields.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-02-24 01:31:58 +05:30
Hidde Beydals 6497680d0d
Merge pull request #591 from fluxcd/update-samples 2022-02-23 14:53:57 +01:00
Sunny d55c933a4f
Update object API version in the sample configs
Use the latest API version v1beta2.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-02-23 18:26:48 +05:30
Hidde Beydals ff94d41f05
Merge pull request #586 from fluxcd/rewrite-dev 2022-02-23 13:30:08 +01:00
Sunny 9c7661dcbd helmrepo: Make NewArtifact event human friendly
Inform index size and repo instead of a revision.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-02-23 12:37:48 +01:00
Sunny 84bf8c8572 fuzz: Update to use v1beta2 APIs
Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-02-23 12:37:48 +01:00
Sunny e9ae0c232e Add kstatus client conformance tests
Use kstatus to compute the status of the objects.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-02-23 12:37:45 +01:00