Commit Graph

56 Commits

Author SHA1 Message Date
Paulo Gomes c936fc145b
Ensure git status is tracked at the correct time
The previous version would verify git status at time of `make verify` execution, but before all its dependencies executed.
If one of the dependencies resulted in a dirty git, the check would still pass.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-05-09 15:54:55 +01:00
Max Jonas Werner 1e83f0ecf5 fix API docs link
v0.3.0 of gen-crd-api-reference-docs has a bug that leads to it
rendering incorrect links for local types. This is fixed on the master
branch but not released, yet, so I'm pinning the version of the tool
to the latest current commit.

Signed-off-by: Max Jonas Werner <mail@makk.es>
2022-04-29 19:27:53 +02:00
Max Jonas Werner 9c6a6d91e8 build: let users pass test flags to `make test`
This comes in handy e.g. to only run a single test during development.

Signed-off-by: Max Jonas Werner <mail@makk.es>
2022-04-28 06:31:45 +02:00
Alexander Block 7c8b175258 tests: Set GIT_CONFIG_GLOBAL=/dev/null for tests run via make test
This ensures that no host specific git configuration is pulled in while
testing. One such example would be "init.defaultBranch" which is set to
"main" on some hosts, which then causes tests to fail as they keep looking
for "master" branches.

Signed-off-by: Alexander Block <ablock84@gmail.com>
2022-04-25 14:30:16 +02:00
Paulo Gomes 77fcc88805
Update libgit2 to 1.3.1
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-04-14 09:59:01 +01:00
Paulo Gomes da91e47036 Improve documentation on Debugging source controller
Expands on the current documentation to help contributors debug the controller
regardless of all its existing dependencies.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-03-30 10:50:40 +02:00
Hidde Beydals d17e724124 build: properly wire integration tests
- Ensure secrets are set as env variables
- Wire `GO_TAGS` together with `GO_STATIC_FLAGS`

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-03-11 10:43:35 +01:00
Hidde Beydals ec5bc1ac9a Implement Azure Blob BucketProvider
This commit introduces an Azure Blob BucketProvider implementation,
capable of fetching from objects from public and private "container"
buckets.

The supported credential types are:

- ManagedIdentity with a `resourceId` Secret data field.
- ManagedIdentity with a `clientId` Secret data field.
- ClientSecret with `tenantId`, `clientId` and `clientSecret` Secret
  data fields.
- SharedKey with `accountKey` Secret data field, the Account Name is
  extracted from the endpoint URL specified on the object.

If no Secret is provided, the Bucket is assumed to be public.

Co-authored-by: Zhongcheng Lao <Zhongcheng.Lao@microsoft.com>
Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-03-03 13:20:48 +01:00
Sunny cc1e48243d Introduce v1beta2 API package
This commit introduces a v1beta2 API package for the staged breaking
changes around conditions and general usage of the API objects.

Signed-off-by: Hidde Beydals <hello@hidde.co>
Signed-off-by: Sunny <darkowlzz@protonmail.com>
Co-authored-by: Hidde Beydals <hello@hidde.co>
2022-02-23 12:34:35 +01:00
Paulo Gomes 8429708997
Upgrade libgit2 to libgit2-1.3.0-2
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-02-16 11:39:11 +00:00
Paulo Gomes 514126c4b8
Fix make test on arm64 runners
The environment variables set at the Makefile were causing go install to
yield a corrupted file for setup-envtest. To fix the issue, such operation
is now always executed in a clean bash.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-02-16 10:17:33 +00:00
Paulo Gomes 03c56e79a0
Download libgit2 libraries for fuzzing
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-02-11 12:30:46 +00:00
Paulo Gomes fcb62eedc7
Validate LIBGIT2 args are set correctly
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-02-09 15:52:46 +00:00
Paulo Gomes d8da814d2e
Initial fuzzing tests
This PR refactors and supersedes: https://github.com/fluxcd/source-controller/pull/443

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
Co-authored-by: Sanskar Jaiswal <sanskar.jaiswal@weave.works>
Co-authored-by: AdamKorcz <adam@adalogics.com>
2022-02-08 17:49:34 +00:00
Paulo Gomes 80e1d243e6
Upgrade to libgit2-1.1.1-6
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-02-08 10:51:20 +00:00
Paulo Gomes 21e71c3750
Set u+w to envtest folders
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-02-07 13:00:58 +00:00
Paulo Gomes 60c98b5471
Update libgit2 to 1.1.1-6
Fix issues developing in amd64, arm64 and apple silicon

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-02-07 13:00:57 +00:00
Paulo Gomes 990b4a361b
Use static libraries from built image
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-02-07 13:00:55 +00:00
Paulo Gomes d163ebae14
Statically build using musl toolchain and target alpine
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-01-26 12:11:49 +00:00
Sanskar Jaiswal 2b8ede12cc fix makefile envtest and controller-gen usage
Refactor logic to install helper tools into one function in the
Makefile. Add support for envtest to help install tools like kubectl,
etcd which helps users run tests more conveniently.

Signed-off-by: Sanskar Jaiswal <sanskar.jaiswal@weave.works>
2022-01-13 16:48:04 +05:30
Paulo Gomes ab934579bb
Bump libgit2 dependencies to debian bookworm
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-01-13 10:16:15 +00:00
Paulo Gomes b63780505e
Add api-docs and manifests to make verify
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-01-13 10:16:13 +00:00
Paulo Gomes 7abe7591f4
Add make verify
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-01-13 10:16:10 +00:00
Paulo Gomes 0c9078c5d1
Bump LIBGIT2_TAG to libgit2-1.1.1-2
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-01-13 10:16:09 +00:00
Paulo Gomes a76cef6bde
Extract e2e tests from github actions
Enable users to run the exact same tests locally as they would be executed
as part of their PRs.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-01-13 10:16:05 +00:00
Paulo Gomes 6ab959d6fb
Add copyright attributions
Adds an initial ATTRIBUTIONS.md file containing the open source
components that source-controller depends on, and their copyright
information.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-01-13 10:16:03 +00:00
Paulo Gomes 7f886f0bcf
Statically build source-controller into distroless image
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-01-13 10:16:02 +00:00
Aurel Canciu 22d0880e4d
Update flux pkg components
Signed-off-by: Aurel Canciu <aurelcanciu@gmail.com>
2021-12-20 14:47:44 +01:00
Sanskar Jaiswal d174bc9449 fix potentially broken support for macos
macOS support is broken for users who rely on the Makefile to install
libgit2 for them. libgit2.1.1.dylib could not be dynamically linked at runtime
because it couldn't be found. This patch makes the following changes to
the Makefile:
1) Respects the user's PKG_CONFIG_PATH present in the env so that both
   libgit2.pc and openssl.pc are discoverable.
2) Embeds the required rpath in the binary at compile time, so that
   libgit2.1.1.dylib can be found at runtime. For more info see:
   https://github.com/fluxcd/source-controller/pull/515#discussion_r764245029
Signed-off-by: Sanskar Jaiswal <sanskar.jaiswal@weave.works>
2021-12-13 15:28:13 +05:30
Hidde Beydals 2ffd15a548 Update fluxcd/golang-with-libgit2 to 1.1.1-1
This includes a tiny fix for Darwin to ensure the generated `.pc`
file includes the right paths.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-08 15:50:55 +02:00
Hidde Beydals 4a4c7e5cb3 Fix generation of API documentation
The version was accidentally set to an invalid version, causing the
API documentation generation to fail.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-08 13:15:17 +02:00
Hidde Beydals 9fd16d8e45 Switch to scratch based libgit2 container image
This moves the `libgit2` compilation to the image, to ensure it
can be build on builders that aren't backed by AMD64.

The image is structured in such a way that e.g. running nightly
builds targeting a different Go version, or targeting a different
OS vendor would be possible in the future via build arguments.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-08 09:38:20 +02:00
Hidde Beydals 029617cb49 Detect macOS produced libgit2.dylib on Darwin
Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-04 15:36:43 +02:00
Hidde Beydals a5a90fd4f6 Allow libgit2 build to be enforced
This can be useful on machines where libgit2 is installed due to other
applications depending on it, but where the composition of this
installation does not properly work with the controller.

Reason the system version is still preferred, is because this lowers the
barrier for drive-by contributors, as a working set of (Git) dependencies
should only really be required if you are going to perform work in that
domain.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-04 15:36:43 +02:00
Hidde Beydals 981d91e363 Update base image to version with Darwin detection
To provide a better (contributing) experience to those with Apple
machines, as determining the correct paths there is a bit harder.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-01 19:08:59 +02:00
Hidde Beydals 0358ebc43d Change image to image under Flux organization
Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-09-30 16:37:05 +02:00
Hidde Beydals 7a212b6570 Add `docker-buildx` target to `Makefile`
To allow building a multi-platform container image using `buildx`.

Various configuration flags allow for fine(r)-grain control over the
build process:

- `BASE_IMG`: FQDN of the base image that should be used, without a
  tag.
- `BASE_TAG: tag of the base image that should be used. Allows checksum
  sum to be included.
- `BUILDX_PLATFORMS`: platforms to target for the final container
  image.
- `BUILDX_ARGS`: additional `docker buildx build` arguments, e.g.
  `--push` to push the result to a (local) image registry.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-09-30 16:37:05 +02:00
Hidde Beydals d3c27c37f8 Update github.com/libgit2/git2go to v31.6.1
This commit updates `github.com/libgit2/git2go` to `v31.6.1` (with
`libgit2` `1.1.1`), and changes the container image build process so
that it makes use of `ghcr.io/hiddeco/golang-with-libgit2`.

This image provides a pre-build dynamic `libgit2` dependency linked
against OpenSSL and LibSSH2 (without gcrypt), and a set of cross-compile
build tools (see
[rationale](https://github.com/hiddeco/golang-with-libgit2#rationale) and
[usage](https://github.co/hiddeco/golang-with-libgit2#usage) for more
detailed information).

The linked set of dependency should solve most known issues around
unsupport private key types, but does not resolve the issues with ECDSA*
and ED25519 hostkeys yet. Solving this requires a newer version of
`libgit2` (`>=1.2.0`), which currently does not seem to work properly
with `git2go/v32`.

Some small changes have been made to the `libgit2` package to address
(future) deprecations.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-09-30 16:36:20 +02:00
Sunny 4ce7c0d9aa
Add setup-envtest in Makefile
Use setup-envtest for installing the binaries required for using
envtest.
The latest envtest binaries for k8s v1.21.2 are not compatible with
kubebuilder < v3, which results in envtest suite test tear down
failure. envtest fails to stop the kube-apiserver.

```
timeout waiting for process kube-apiserver to stop
```

Pin to k8s v1.19.2 envtest binaries that work with kubebuilder v2.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2021-07-30 18:45:17 +05:30
Hidde Beydals a76b10cebb Update K8s, controller-runtime and fluxcd/pkg deps
Controller-runtime has been updated to `v0.9.0`, K8s dependencies to
`v0.21.1`, and all `fluxcd/pkg` and other dependencies to the versions
that have matching dependencies and/or build constraints.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-06-08 13:41:34 +02:00
Philip Laine fcf7048992 Add include property to GitRepositories
Signed-off-by: Philip Laine <philip.laine@gmail.com>
Signed-off-by: Philip Laine <philip.laine@xenit.se>
2021-05-11 09:46:50 +02:00
Stefan Prodan 22f5a6f6fb
Update dependencies
- helm.sh/helm/v3 v3.5.2 (forces us into pinning docker)
- sigs.k8s.io/controller-tools/cmd/controller-gen v0.4.1
- sigs.k8s.io/controller-runtime v0.8.2
- fluxcd/pkg/apis/meta v0.8.0
- fluxcd/pkg/runtime v0.8.3

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-02-23 13:18:21 +02:00
Stefan Prodan ab16dd7c2e
Update Kubernetes and Helm packages
- k8s.io/api v0.20.2
- controller-runtime v0.8.0
- helm v3.5.0

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-01-16 14:38:28 +02:00
Philip Laine 934f5df668 Remove manifests run on dev deploy
Signed-off-by: Philip Laine <philip.laine@xenit.se>
2020-12-02 20:18:29 +01:00
stefanprodan f16e5f041a Promote API to v1beta1 2020-09-30 16:11:00 +03:00
Hidde Beydals 0f9a6ff939 Remove `find -execdir` from `Makefile`
As it does not allow for error codes to surface
2020-09-11 09:58:07 +02:00
Hidde Beydals 7f0c99cf03 Tweak `make manifest` target to reenable RBAC gen 2020-09-10 21:17:06 +02:00
Hidde Beydals 157ad2f51a Make various targets work after api pkg change 2020-08-20 10:32:24 +02:00
stefanprodan 7e2fe14efa Upgrade CRDs to apiextensions.k8s.io/v1 2020-07-20 12:21:34 +03:00
Hidde Beydals 7e647a6bb2 Generate API documentation 2020-06-19 16:45:05 +02:00