Commit Graph

808 Commits

Author SHA1 Message Date
pa250194 57b54c8592 Service Account Key Authentication to GCP Provider
Signed-off-by: pa250194 <pa250194@ncr.com>
2021-10-14 10:16:11 +01:00
pa250194 2cc48fefb1 Added initial testing for new GCP provider
Signed-off-by: pa250194 <pa250194@ncr.com>
2021-10-14 10:16:11 +01:00
pa250194 a600528729 Added Comments for reconcileWithGCP and reconcileWithMinio
Signed-off-by: pa250194 <pa250194@ncr.com>
2021-10-14 10:16:11 +01:00
pa250194 572eed74e5 Add Support for GCP storage with workload identity
Added Support for Google Cloud Storage with Workload Identity as
Source Provider. This enables the use of GCP without enabling S3
compatible access.

Signed-off-by: pa250194 <pa250194@ncr.com>
2021-10-14 10:16:11 +01:00
Hidde Beydals d9f19a83ef
Merge pull request #449 from fluxcd/libgit2-1.1.1-1
Update fluxcd/golang-with-libgit2 to 1.1.1-1
2021-10-08 16:33:27 +02:00
Hidde Beydals 2ffd15a548 Update fluxcd/golang-with-libgit2 to 1.1.1-1
This includes a tiny fix for Darwin to ensure the generated `.pc`
file includes the right paths.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-08 15:50:55 +02:00
Hidde Beydals 6cf20fb721
Merge pull request #448 from fluxcd/fix-api-docs
Fix generation of API documentation
2021-10-08 13:36:05 +02:00
Hidde Beydals 4a4c7e5cb3 Fix generation of API documentation
The version was accidentally set to an invalid version, causing the
API documentation generation to fail.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-08 13:15:17 +02:00
Hidde Beydals 64f0cf06c5
Merge pull request #447 from fluxcd/release-v0.16.0
Release v0.16.0
2021-10-08 12:18:43 +02:00
Hidde Beydals 7cc6124a4e Release v0.16.0
Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-08 12:00:03 +02:00
Hidde Beydals d382eca383
Merge pull request #308 from arbourd/ignore-version
Add reconcile strategy for HelmCharts
2021-10-08 11:31:55 +02:00
Dylan Arbour 27c385b957 Add ReconcileStrategy to HelmChart
This commit adds a `ReconcileStrategy` field to the `HelmChart` resource, which
allows defining when a new chart should be packaged and/or published if it
originates from a `Bucket` or `GitRepository` resource.

The two available strategies are:

- `ChartVersion`: creates a new artifact when the version of the Helm chart as
  defined in the `Chart.yaml` from the Source is different from the current
  version.
- `Revision`: creates a new artifact when the revision of the Source is
  different from the current revision.

For the `Revision` strategy, the (checksum part of the) revision of the
artifact the chart originatesfrom is added as SemVer metadata.

A chart from a `GitRepository` with Artifact revision
`main/f0faacd5164a875ebdbd9e3fab778f49c5aadbbc` and a chart with e.g. SemVer
`0.1.0` will be published as `0.1.0+f0faacd5164a875ebdbd9e3fab778f49c5aadbbc`.

A chart from a `Bucket` with Artifact revision
`f0faacd5164a875ebdbd9e3fab778f49c5aadbbc` and a chart with e.g. SemVer `0.1.0`
will be published as `0.1.0+f0faacd5164a875ebdbd9e3fab778f49c5aadbbc`.

Signed-off-by: Dylan Arbour <arbourd@users.noreply.github.com>
2021-10-08 11:07:11 +02:00
Stefan Prodan c4cc0a7ccf
Merge pull request #446 from fluxcd/oci-cve-fix
Update containerd and runc to fix CVEs
2021-10-08 11:46:05 +03:00
Stefan Prodan 66f54ac422
Update containerd and runc to fix CVEs
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-10-08 11:22:44 +03:00
Hidde Beydals eb167bc68d
Merge pull request #437 from fluxcd/update-libgit2
Update github.com/libgit2/git2go to v31.6.1
2021-10-08 10:04:23 +02:00
Hidde Beydals fa9a2d8512 CONTRIBUTING: include pkg-config as macOS dep
As this isn't available on Darwin by default, unlike on most Linux
distributions.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-08 09:40:02 +02:00
Hidde Beydals a871e0b527 Update Dockerfile used in tests as well
This ensures the Dockerfile used for testing is making use of the
same scratch image to compile `libgit2` as the actual application
image.

In a future iteration we should restructure our GitHub Action
workflows to re-use the application image, saving us an additional
Dockerfile and a duplicate build. Inspiration for this (which makes
use of a local registry for the duration of the build) can be found
at: https://github.com/fluxcd/golang-with-libgit2/blob/main/.github/workflows/build.yaml

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-08 09:38:20 +02:00
Hidde Beydals 9fd16d8e45 Switch to scratch based libgit2 container image
This moves the `libgit2` compilation to the image, to ensure it
can be build on builders that aren't backed by AMD64.

The image is structured in such a way that e.g. running nightly
builds targeting a different Go version, or targeting a different
OS vendor would be possible in the future via build arguments.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-08 09:38:20 +02:00
Hidde Beydals b87448864a Document libgit2 build behavior in CONTRIBUTING.md
Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-04 15:36:43 +02:00
Hidde Beydals 029617cb49 Detect macOS produced libgit2.dylib on Darwin
Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-04 15:36:43 +02:00
Hidde Beydals a5a90fd4f6 Allow libgit2 build to be enforced
This can be useful on machines where libgit2 is installed due to other
applications depending on it, but where the composition of this
installation does not properly work with the controller.

Reason the system version is still preferred, is because this lowers the
barrier for drive-by contributors, as a working set of (Git) dependencies
should only really be required if you are going to perform work in that
domain.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-04 15:36:43 +02:00
Hidde Beydals 981d91e363 Update base image to version with Darwin detection
To provide a better (contributing) experience to those with Apple
machines, as determining the correct paths there is a bit harder.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-01 19:08:59 +02:00
Hidde Beydals 0358ebc43d Change image to image under Flux organization
Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-09-30 16:37:05 +02:00
Hidde Beydals 7a212b6570 Add `docker-buildx` target to `Makefile`
To allow building a multi-platform container image using `buildx`.

Various configuration flags allow for fine(r)-grain control over the
build process:

- `BASE_IMG`: FQDN of the base image that should be used, without a
  tag.
- `BASE_TAG: tag of the base image that should be used. Allows checksum
  sum to be included.
- `BUILDX_PLATFORMS`: platforms to target for the final container
  image.
- `BUILDX_ARGS`: additional `docker buildx build` arguments, e.g.
  `--push` to push the result to a (local) image registry.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-09-30 16:37:05 +02:00
Hidde Beydals d3c27c37f8 Update github.com/libgit2/git2go to v31.6.1
This commit updates `github.com/libgit2/git2go` to `v31.6.1` (with
`libgit2` `1.1.1`), and changes the container image build process so
that it makes use of `ghcr.io/hiddeco/golang-with-libgit2`.

This image provides a pre-build dynamic `libgit2` dependency linked
against OpenSSL and LibSSH2 (without gcrypt), and a set of cross-compile
build tools (see
[rationale](https://github.com/hiddeco/golang-with-libgit2#rationale) and
[usage](https://github.co/hiddeco/golang-with-libgit2#usage) for more
detailed information).

The linked set of dependency should solve most known issues around
unsupport private key types, but does not resolve the issues with ECDSA*
and ED25519 hostkeys yet. Solving this requires a newer version of
`libgit2` (`>=1.2.0`), which currently does not seem to work properly
with `git2go/v32`.

Some small changes have been made to the `libgit2` package to address
(future) deprecations.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-09-30 16:36:20 +02:00
Hidde Beydals d7afc3596b
Merge pull request #418 from fluxcd/release-v0.15.4
Release v0.15.4
2021-08-05 14:03:58 +02:00
Hidde Beydals fd1b3ce0bb Release v0.15.4
Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-08-05 13:45:07 +02:00
Hidde Beydals c30b9408bd
Merge pull request #417 from fluxcd/cherry-pick-fixes 2021-08-05 12:57:13 +02:00
Hidde Beydals 3ac39b6137 storage: strip env specific data during archive
This ensures the checksum is predictable, and not influenced by e.g.
different runtime configuration settings, or FS specific data.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-08-05 12:42:40 +02:00
Hidde Beydals d0560e5dbe Use same SemVer logic in both Git implementations
Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-08-05 12:40:54 +02:00
Hidde Beydals 77d1c9ccba Ensure rel path never traverses outside Storage
Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-08-05 12:40:21 +02:00
Stefan Prodan edb4595538
Merge pull request #416 from fluxcd/update-deps
Update dependencies
2021-08-05 13:22:22 +03:00
Stefan Prodan 7001b34f10
Update dependencies
- k8s.io/* v0.21.3
- controller-runtime v0.9.5
- kubectl 1.21.3

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-08-05 12:43:08 +03:00
Stefan Prodan aef56d83e5
Merge pull request #404 from darkowlzz/setup-envtest
Add setup-envtest in Makefile
2021-08-02 10:00:25 +03:00
Sunny cb3fb3bb4e
workflows/e2e: Use fluxcd/pkg/actions/envtest
Use fluxcd/pkg/actions/envtest to setup envtest.

Since the run-test action runs in a container, the host $PATH is not
passed to the test container. The ENV directive in actions/run-tests
Dockerfile is used to append the PATH with envtest binary path. This
helps avoid redownload of the envtest binaries when it's not found in
the path via the Makefile targets.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2021-08-01 21:07:27 +05:30
Sunny 4ce7c0d9aa
Add setup-envtest in Makefile
Use setup-envtest for installing the binaries required for using
envtest.
The latest envtest binaries for k8s v1.21.2 are not compatible with
kubebuilder < v3, which results in envtest suite test tear down
failure. envtest fails to stop the kube-apiserver.

```
timeout waiting for process kube-apiserver to stop
```

Pin to k8s v1.19.2 envtest binaries that work with kubebuilder v2.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2021-07-30 18:45:17 +05:30
Hidde Beydals 273f8b155e
Merge pull request #409 from fluxcd/dep-ioutil 2021-07-29 11:36:46 +02:00
Hidde Beydals c4d7e46b90 Drop deprecated `io/ioutil`
The package has been deprecated since Go 1.16, see:
https://golang.org/doc/go1.16#ioutil

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-07-29 09:58:00 +02:00
Hidde Beydals be5d10eaac
Merge pull request #403 from fluxcd/bucket-revision-path
Take relative paths in account for Bucket revision
2021-07-26 14:01:44 +02:00
Hidde Beydals f5cb441a82 Take relative paths in account for Bucket revision
This commit changes the checksum method which is used to calculate the
revision of a Bucket source, so that the file paths are taken into
account and directory structure changes can be observed.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-07-26 10:54:39 +02:00
Hidde Beydals dbaf8bf458
Merge pull request #405 from darkowlzz/client-objectkeyfromobject
Use ObjectKeyFromObject instead of ObjectKey
2021-07-26 10:48:19 +02:00
Sunny 9825a60b74
Use ObjectKeyFromObject instead of ObjectKey
controller-runtime's client package provides ObjectKeyFromObject() to
extract NamespacedName from a given object. ObjectKey() in
internal/util package is a helper for the same. Replace the internal
helper with controller-runtime's helper for the same.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2021-07-26 04:35:08 +05:30
Hidde Beydals 40a47670aa
Merge pull request #400 from fluxcd/update-deps
Update Helm to v3.6.3
2021-07-15 11:10:29 +02:00
Hidde Beydals 5e8e0ab65c Update Helm to v3.6.3
Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-07-14 23:24:26 +02:00
Stefan Prodan c6ae5e779a
Merge pull request #396 from fluxcd/debug-e2e
Print pods status on e2e failures
2021-06-29 17:04:00 +03:00
Stefan Prodan 25a2428b80
Print pods status on e2e failures
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-06-29 16:40:15 +03:00
Stefan Prodan 5c170bfdc3
Merge pull request #395 from fluxcd/release-v0.15.3
Release v0.15.3
2021-06-29 13:37:28 +03:00
Stefan Prodan be9f05baf1
Release v0.15.3
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-06-29 12:27:52 +03:00
Stefan Prodan 887d2c5e50
Merge pull request #394 from bburky/libgit2-tag-checkout
Fix tag checkout with libgit2
2021-06-29 11:54:03 +03:00
Blake Burkhart 0df2b0e0f0 Fix tag checkout with libgit2
SetHeadDetached (git_repository_set_head_detached) only changes HEAD,
and does not actually checkout the files on disk. Use CheckoutHead with
the CheckoutForce Strategy to actually check the files out on disk.

Additionally add a test that validates the hash of a checked out file's
contents.

Previously, the hash of the desired tag was being reported as the
checked out revision by the GitRepository. However the wrong files were
checked out and an incorrect revision would be deployed by Flux.

Signed-off-by: Blake Burkhart <blake.burkhart@us.af.mil>
2021-06-28 20:03:11 -05:00