Commit Graph

808 Commits

Author SHA1 Message Date
Stefan Prodan 6fe6f07d5e Update containerd and runc to fix CVEs
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-10-14 13:49:52 -05:00
Hidde Beydals 66fffe103e CONTRIBUTING: include pkg-config as macOS dep
As this isn't available on Darwin by default, unlike on most Linux
distributions.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-14 13:48:15 -05:00
Hidde Beydals 610131956e Update Dockerfile used in tests as well
This ensures the Dockerfile used for testing is making use of the
same scratch image to compile `libgit2` as the actual application
image.

In a future iteration we should restructure our GitHub Action
workflows to re-use the application image, saving us an additional
Dockerfile and a duplicate build. Inspiration for this (which makes
use of a local registry for the duration of the build) can be found
at: https://github.com/fluxcd/golang-with-libgit2/blob/main/.github/workflows/build.yaml

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-14 13:48:15 -05:00
Hidde Beydals d04c532461 Switch to scratch based libgit2 container image
This moves the `libgit2` compilation to the image, to ensure it
can be build on builders that aren't backed by AMD64.

The image is structured in such a way that e.g. running nightly
builds targeting a different Go version, or targeting a different
OS vendor would be possible in the future via build arguments.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-14 13:48:15 -05:00
Hidde Beydals 153b122970 Document libgit2 build behavior in CONTRIBUTING.md
Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-14 13:48:15 -05:00
Hidde Beydals cc01df231e Detect macOS produced libgit2.dylib on Darwin
Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-14 13:48:15 -05:00
Hidde Beydals 1b11e11a90 Allow libgit2 build to be enforced
This can be useful on machines where libgit2 is installed due to other
applications depending on it, but where the composition of this
installation does not properly work with the controller.

Reason the system version is still preferred, is because this lowers the
barrier for drive-by contributors, as a working set of (Git) dependencies
should only really be required if you are going to perform work in that
domain.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-14 13:48:15 -05:00
Hidde Beydals 500d0aeda0 Update base image to version with Darwin detection
To provide a better (contributing) experience to those with Apple
machines, as determining the correct paths there is a bit harder.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-14 13:48:15 -05:00
Hidde Beydals b283e3e9c5 Change image to image under Flux organization
Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-14 13:48:15 -05:00
Hidde Beydals c9e3f97470 Add `docker-buildx` target to `Makefile`
To allow building a multi-platform container image using `buildx`.

Various configuration flags allow for fine(r)-grain control over the
build process:

- `BASE_IMG`: FQDN of the base image that should be used, without a
  tag.
- `BASE_TAG: tag of the base image that should be used. Allows checksum
  sum to be included.
- `BUILDX_PLATFORMS`: platforms to target for the final container
  image.
- `BUILDX_ARGS`: additional `docker buildx build` arguments, e.g.
  `--push` to push the result to a (local) image registry.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-14 13:48:15 -05:00
Hidde Beydals 869c7960e3 Update github.com/libgit2/git2go to v31.6.1
This commit updates `github.com/libgit2/git2go` to `v31.6.1` (with
`libgit2` `1.1.1`), and changes the container image build process so
that it makes use of `ghcr.io/hiddeco/golang-with-libgit2`.

This image provides a pre-build dynamic `libgit2` dependency linked
against OpenSSL and LibSSH2 (without gcrypt), and a set of cross-compile
build tools (see
[rationale](https://github.com/hiddeco/golang-with-libgit2#rationale) and
[usage](https://github.co/hiddeco/golang-with-libgit2#usage) for more
detailed information).

The linked set of dependency should solve most known issues around
unsupport private key types, but does not resolve the issues with ECDSA*
and ED25519 hostkeys yet. Solving this requires a newer version of
`libgit2` (`>=1.2.0`), which currently does not seem to work properly
with `git2go/v32`.

Some small changes have been made to the `libgit2` package to address
(future) deprecations.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-14 13:48:15 -05:00
pa250194 7c0d4c070e Refactor comments and method names
Signed-off-by: pa250194 <pa250194@ncr.com>
2021-09-23 13:43:08 -05:00
pa250194 38be5ed111 Cleanup obsolete comments
Signed-off-by: pa250194 <pa250194@ncr.com>
2021-09-23 12:40:55 -05:00
pa250194 057c65e939 Removed resumable downloads
Signed-off-by: pa250194 <pa250194@ncr.com>
2021-09-23 12:39:00 -05:00
pa250194 0b971511ad Revert change to doc/api/source.md
Signed-off-by: pa250194 <pa250194@ncr.com>
2021-09-16 12:30:58 -05:00
pa250194 a6be9c8a21 Updated docs to include GCP provider instructions
Signed-off-by: pa250194 <pa250194@ncr.com>
2021-09-16 12:16:17 -05:00
pa250194 fa8c4ca096 Fix nil pointer dereference
Signed-off-by: pa250194 <pa250194@ncr.com>
2021-09-16 09:57:27 -05:00
pa250194 6ff5970fe1 Added more tests and cleaned up GCP provider logic
Signed-off-by: pa250194 <pa250194@ncr.com>
2021-09-16 09:49:56 -05:00
pa250194 c204f6a8ee Added Tests to GCP provider
Signed-off-by: pa250194 <pa250194@ncr.com>
2021-09-16 09:49:56 -05:00
pa250194 eeb38bdf2c Tests for GCP Bucket Provider
Signed-off-by: pa250194 <pa250194@ncr.com>
2021-09-16 09:49:56 -05:00
pa250194 0444c6e16d Service Account Key Authentication to GCP Provider
Signed-off-by: pa250194 <pa250194@ncr.com>
2021-09-16 09:49:56 -05:00
pa250194 90395f426a Remove .DS_STORE file
Signed-off-by: pa250194 <pa250194@ncr.com>
2021-09-16 09:49:56 -05:00
pa250194 78379ddcd5 Added initial testing for new GCP provider
Signed-off-by: pa250194 <pa250194@ncr.com>
2021-09-16 09:49:56 -05:00
pa250194 a5588fb191 Added Comments for reconcileWithGCP and reconcileWithMinio
Signed-off-by: pa250194 <pa250194@ncr.com>
2021-09-16 09:49:56 -05:00
pa250194 7da9619b58 Feature: Add Support for Google Cloud Storage along with Workload Identity
Added Support for Google Cloud Storage with Workload Identity as Source Provider. This enables the use of GCP without enabling S3 compatible access.

Signed-off-by: pa250194 <pa250194@ncr.com>
2021-09-16 09:49:56 -05:00
Hidde Beydals d7afc3596b
Merge pull request #418 from fluxcd/release-v0.15.4
Release v0.15.4
2021-08-05 14:03:58 +02:00
Hidde Beydals fd1b3ce0bb Release v0.15.4
Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-08-05 13:45:07 +02:00
Hidde Beydals c30b9408bd
Merge pull request #417 from fluxcd/cherry-pick-fixes 2021-08-05 12:57:13 +02:00
Hidde Beydals 3ac39b6137 storage: strip env specific data during archive
This ensures the checksum is predictable, and not influenced by e.g.
different runtime configuration settings, or FS specific data.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-08-05 12:42:40 +02:00
Hidde Beydals d0560e5dbe Use same SemVer logic in both Git implementations
Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-08-05 12:40:54 +02:00
Hidde Beydals 77d1c9ccba Ensure rel path never traverses outside Storage
Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-08-05 12:40:21 +02:00
Stefan Prodan edb4595538
Merge pull request #416 from fluxcd/update-deps
Update dependencies
2021-08-05 13:22:22 +03:00
Stefan Prodan 7001b34f10
Update dependencies
- k8s.io/* v0.21.3
- controller-runtime v0.9.5
- kubectl 1.21.3

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-08-05 12:43:08 +03:00
Stefan Prodan aef56d83e5
Merge pull request #404 from darkowlzz/setup-envtest
Add setup-envtest in Makefile
2021-08-02 10:00:25 +03:00
Sunny cb3fb3bb4e
workflows/e2e: Use fluxcd/pkg/actions/envtest
Use fluxcd/pkg/actions/envtest to setup envtest.

Since the run-test action runs in a container, the host $PATH is not
passed to the test container. The ENV directive in actions/run-tests
Dockerfile is used to append the PATH with envtest binary path. This
helps avoid redownload of the envtest binaries when it's not found in
the path via the Makefile targets.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2021-08-01 21:07:27 +05:30
Sunny 4ce7c0d9aa
Add setup-envtest in Makefile
Use setup-envtest for installing the binaries required for using
envtest.
The latest envtest binaries for k8s v1.21.2 are not compatible with
kubebuilder < v3, which results in envtest suite test tear down
failure. envtest fails to stop the kube-apiserver.

```
timeout waiting for process kube-apiserver to stop
```

Pin to k8s v1.19.2 envtest binaries that work with kubebuilder v2.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2021-07-30 18:45:17 +05:30
Hidde Beydals 273f8b155e
Merge pull request #409 from fluxcd/dep-ioutil 2021-07-29 11:36:46 +02:00
Hidde Beydals c4d7e46b90 Drop deprecated `io/ioutil`
The package has been deprecated since Go 1.16, see:
https://golang.org/doc/go1.16#ioutil

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-07-29 09:58:00 +02:00
Hidde Beydals be5d10eaac
Merge pull request #403 from fluxcd/bucket-revision-path
Take relative paths in account for Bucket revision
2021-07-26 14:01:44 +02:00
Hidde Beydals f5cb441a82 Take relative paths in account for Bucket revision
This commit changes the checksum method which is used to calculate the
revision of a Bucket source, so that the file paths are taken into
account and directory structure changes can be observed.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-07-26 10:54:39 +02:00
Hidde Beydals dbaf8bf458
Merge pull request #405 from darkowlzz/client-objectkeyfromobject
Use ObjectKeyFromObject instead of ObjectKey
2021-07-26 10:48:19 +02:00
Sunny 9825a60b74
Use ObjectKeyFromObject instead of ObjectKey
controller-runtime's client package provides ObjectKeyFromObject() to
extract NamespacedName from a given object. ObjectKey() in
internal/util package is a helper for the same. Replace the internal
helper with controller-runtime's helper for the same.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2021-07-26 04:35:08 +05:30
Hidde Beydals 40a47670aa
Merge pull request #400 from fluxcd/update-deps
Update Helm to v3.6.3
2021-07-15 11:10:29 +02:00
Hidde Beydals 5e8e0ab65c Update Helm to v3.6.3
Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-07-14 23:24:26 +02:00
Stefan Prodan c6ae5e779a
Merge pull request #396 from fluxcd/debug-e2e
Print pods status on e2e failures
2021-06-29 17:04:00 +03:00
Stefan Prodan 25a2428b80
Print pods status on e2e failures
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-06-29 16:40:15 +03:00
Stefan Prodan 5c170bfdc3
Merge pull request #395 from fluxcd/release-v0.15.3
Release v0.15.3
2021-06-29 13:37:28 +03:00
Stefan Prodan be9f05baf1
Release v0.15.3
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-06-29 12:27:52 +03:00
Stefan Prodan 887d2c5e50
Merge pull request #394 from bburky/libgit2-tag-checkout
Fix tag checkout with libgit2
2021-06-29 11:54:03 +03:00
Blake Burkhart 0df2b0e0f0 Fix tag checkout with libgit2
SetHeadDetached (git_repository_set_head_detached) only changes HEAD,
and does not actually checkout the files on disk. Use CheckoutHead with
the CheckoutForce Strategy to actually check the files out on disk.

Additionally add a test that validates the hash of a checked out file's
contents.

Previously, the hash of the desired tag was being reported as the
checked out revision by the GitRepository. However the wrong files were
checked out and an incorrect revision would be deployed by Flux.

Signed-off-by: Blake Burkhart <blake.burkhart@us.af.mil>
2021-06-28 20:03:11 -05:00