Commit Graph

71 Commits

Author SHA1 Message Date
Paulo Gomes 6c06f4e222
The libgit2 libraries are downloaded and verified before
some of the make targets are executed. This assures the
provenance of such files before using them and is very
important specially for end users running such tests on
their machines.

Note that has been disabled specially due to recent issues
we experienced at CI which can be seen in:
fluxcd/source-controller#899

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-09-29 07:01:36 +01:00
Paulo Gomes 1ab76264de
Bump libgit2 image to v0.3.0
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-09-29 06:55:18 +01:00
Stefan Prodan ca3496e758
Build with Go 1.19
- Update Go to 1.19 in CI
- Use Go 1.19 in base image
- Update controller-gen v0.8.0 and regenerate manifests

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-09-28 15:35:26 +03:00
Paulo Gomes e26f8b4fc6
fuzz: Reuse go cache from host
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-09-02 17:51:28 +01:00
Paulo Gomes 50ef51b59d
fuzz: Ensure latest base images are used
Latest base image should contain Go 1.18, removing
the need of updating that ourselves, apart from
benefiting from latest changes upstream.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-09-02 17:51:11 +01:00
Stefan Prodan 799d7df457
Add filter option when running tests
Allow running specific controller tests by specifying a prefix e.g.
GO_TEST_PREFIX=TestOCI make test-ctrl

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-08-02 13:11:12 +03:00
Paulo Gomes f51f9d3305
Remove MUSL and enable threadless libgit2 support
Use of MUSL was a temporary solution to mitigate cross-platform
issues while building openssl and libssh2. Since Unmanaged transport has
been deprecated, openssl and libssh2 dependencies are no longer required
and by extension MUSL.

Enables libgit2 threadless support and provides a regression assurance
for fluxcd/image-automation-controller#339.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-07-31 18:38:27 +01:00
Philip Laine b47d468e8f
Upgrade to go 1.18
Signed-off-by: Philip Laine <philip.laine@gmail.com>
2022-07-27 19:02:13 +03:00
Sanskar Jaiswal 536bacbea3 libgit2: upgrade to golang-with-libgit2 v0.1.4
Upgrade to golang-with-libgit2 v0.1.4 and use libgit2 which is compiled
without being linked to libssh2 and openssl.

Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
2022-07-26 15:10:29 +05:30
Paulo Gomes aeaeec881e
Update libgit2 to v1.3.2
Updates golang-with-libgit2-all to v0.1.2 which contains libgit2-1.3.2.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-07-14 12:24:26 +01:00
Paulo Gomes afd02d3cfc
build: provenance and tampering checks for libgit2
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-07-13 10:10:42 +01:00
Paulo Gomes 02f5b94baf
Update to golang-with-libgit2-all image
This dependency now releases two different images, one
containing the entire dependency chain for libgit2, and
another containing just the library itself. The latter
will be later used once Managed Transport is completely
removed from source controller.

As part of this update, the image now follows a new tag
format which is semver based and starts at 0.1.0.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-07-08 09:39:06 +01:00
Hidde Beydals 7a5f0ccd89
build: enable `-race` for `go test`
Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-06-14 09:25:51 +01:00
Paulo Gomes 8dfb9090d0
tests: ignore proxy settings when running tests
Users environmental proxy settings should not impact
the execution of the tests. The changes override both
HTTP_PROXY and HTTPS_PROXY to ensure that is the case.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-05-25 13:57:54 +01:00
Stefan Prodan f3a372da60
Update dependencies
- fluxcd/pkg/apis/meta v0.14.0
- fluxcd/pkg/runtime v0.16.0
- k8s.io/* v0.24.0
- helm.sh/helm/v3 v3.9.0-rc.1 (required by breaking changes in Kubernetes 1.24)

Note that fluxcd/pkg/runtime v0.16 comes with support for Kubernetes API Priority and Fairness feature.

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-05-24 14:48:32 +03:00
Paulo Gomes c936fc145b
Ensure git status is tracked at the correct time
The previous version would verify git status at time of `make verify` execution, but before all its dependencies executed.
If one of the dependencies resulted in a dirty git, the check would still pass.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-05-09 15:54:55 +01:00
Max Jonas Werner 1e83f0ecf5 fix API docs link
v0.3.0 of gen-crd-api-reference-docs has a bug that leads to it
rendering incorrect links for local types. This is fixed on the master
branch but not released, yet, so I'm pinning the version of the tool
to the latest current commit.

Signed-off-by: Max Jonas Werner <mail@makk.es>
2022-04-29 19:27:53 +02:00
Max Jonas Werner 9c6a6d91e8 build: let users pass test flags to `make test`
This comes in handy e.g. to only run a single test during development.

Signed-off-by: Max Jonas Werner <mail@makk.es>
2022-04-28 06:31:45 +02:00
Alexander Block 7c8b175258 tests: Set GIT_CONFIG_GLOBAL=/dev/null for tests run via make test
This ensures that no host specific git configuration is pulled in while
testing. One such example would be "init.defaultBranch" which is set to
"main" on some hosts, which then causes tests to fail as they keep looking
for "master" branches.

Signed-off-by: Alexander Block <ablock84@gmail.com>
2022-04-25 14:30:16 +02:00
Paulo Gomes 77fcc88805
Update libgit2 to 1.3.1
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-04-14 09:59:01 +01:00
Paulo Gomes da91e47036 Improve documentation on Debugging source controller
Expands on the current documentation to help contributors debug the controller
regardless of all its existing dependencies.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-03-30 10:50:40 +02:00
Hidde Beydals d17e724124 build: properly wire integration tests
- Ensure secrets are set as env variables
- Wire `GO_TAGS` together with `GO_STATIC_FLAGS`

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-03-11 10:43:35 +01:00
Hidde Beydals ec5bc1ac9a Implement Azure Blob BucketProvider
This commit introduces an Azure Blob BucketProvider implementation,
capable of fetching from objects from public and private "container"
buckets.

The supported credential types are:

- ManagedIdentity with a `resourceId` Secret data field.
- ManagedIdentity with a `clientId` Secret data field.
- ClientSecret with `tenantId`, `clientId` and `clientSecret` Secret
  data fields.
- SharedKey with `accountKey` Secret data field, the Account Name is
  extracted from the endpoint URL specified on the object.

If no Secret is provided, the Bucket is assumed to be public.

Co-authored-by: Zhongcheng Lao <Zhongcheng.Lao@microsoft.com>
Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-03-03 13:20:48 +01:00
Sunny cc1e48243d Introduce v1beta2 API package
This commit introduces a v1beta2 API package for the staged breaking
changes around conditions and general usage of the API objects.

Signed-off-by: Hidde Beydals <hello@hidde.co>
Signed-off-by: Sunny <darkowlzz@protonmail.com>
Co-authored-by: Hidde Beydals <hello@hidde.co>
2022-02-23 12:34:35 +01:00
Paulo Gomes 8429708997
Upgrade libgit2 to libgit2-1.3.0-2
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-02-16 11:39:11 +00:00
Paulo Gomes 514126c4b8
Fix make test on arm64 runners
The environment variables set at the Makefile were causing go install to
yield a corrupted file for setup-envtest. To fix the issue, such operation
is now always executed in a clean bash.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-02-16 10:17:33 +00:00
Paulo Gomes 03c56e79a0
Download libgit2 libraries for fuzzing
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-02-11 12:30:46 +00:00
Paulo Gomes fcb62eedc7
Validate LIBGIT2 args are set correctly
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-02-09 15:52:46 +00:00
Paulo Gomes d8da814d2e
Initial fuzzing tests
This PR refactors and supersedes: https://github.com/fluxcd/source-controller/pull/443

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
Co-authored-by: Sanskar Jaiswal <sanskar.jaiswal@weave.works>
Co-authored-by: AdamKorcz <adam@adalogics.com>
2022-02-08 17:49:34 +00:00
Paulo Gomes 80e1d243e6
Upgrade to libgit2-1.1.1-6
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-02-08 10:51:20 +00:00
Paulo Gomes 21e71c3750
Set u+w to envtest folders
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-02-07 13:00:58 +00:00
Paulo Gomes 60c98b5471
Update libgit2 to 1.1.1-6
Fix issues developing in amd64, arm64 and apple silicon

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-02-07 13:00:57 +00:00
Paulo Gomes 990b4a361b
Use static libraries from built image
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-02-07 13:00:55 +00:00
Paulo Gomes d163ebae14
Statically build using musl toolchain and target alpine
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-01-26 12:11:49 +00:00
Sanskar Jaiswal 2b8ede12cc fix makefile envtest and controller-gen usage
Refactor logic to install helper tools into one function in the
Makefile. Add support for envtest to help install tools like kubectl,
etcd which helps users run tests more conveniently.

Signed-off-by: Sanskar Jaiswal <sanskar.jaiswal@weave.works>
2022-01-13 16:48:04 +05:30
Paulo Gomes ab934579bb
Bump libgit2 dependencies to debian bookworm
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-01-13 10:16:15 +00:00
Paulo Gomes b63780505e
Add api-docs and manifests to make verify
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-01-13 10:16:13 +00:00
Paulo Gomes 7abe7591f4
Add make verify
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-01-13 10:16:10 +00:00
Paulo Gomes 0c9078c5d1
Bump LIBGIT2_TAG to libgit2-1.1.1-2
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-01-13 10:16:09 +00:00
Paulo Gomes a76cef6bde
Extract e2e tests from github actions
Enable users to run the exact same tests locally as they would be executed
as part of their PRs.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-01-13 10:16:05 +00:00
Paulo Gomes 6ab959d6fb
Add copyright attributions
Adds an initial ATTRIBUTIONS.md file containing the open source
components that source-controller depends on, and their copyright
information.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-01-13 10:16:03 +00:00
Paulo Gomes 7f886f0bcf
Statically build source-controller into distroless image
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-01-13 10:16:02 +00:00
Aurel Canciu 22d0880e4d
Update flux pkg components
Signed-off-by: Aurel Canciu <aurelcanciu@gmail.com>
2021-12-20 14:47:44 +01:00
Sanskar Jaiswal d174bc9449 fix potentially broken support for macos
macOS support is broken for users who rely on the Makefile to install
libgit2 for them. libgit2.1.1.dylib could not be dynamically linked at runtime
because it couldn't be found. This patch makes the following changes to
the Makefile:
1) Respects the user's PKG_CONFIG_PATH present in the env so that both
   libgit2.pc and openssl.pc are discoverable.
2) Embeds the required rpath in the binary at compile time, so that
   libgit2.1.1.dylib can be found at runtime. For more info see:
   https://github.com/fluxcd/source-controller/pull/515#discussion_r764245029
Signed-off-by: Sanskar Jaiswal <sanskar.jaiswal@weave.works>
2021-12-13 15:28:13 +05:30
Hidde Beydals 2ffd15a548 Update fluxcd/golang-with-libgit2 to 1.1.1-1
This includes a tiny fix for Darwin to ensure the generated `.pc`
file includes the right paths.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-08 15:50:55 +02:00
Hidde Beydals 4a4c7e5cb3 Fix generation of API documentation
The version was accidentally set to an invalid version, causing the
API documentation generation to fail.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-08 13:15:17 +02:00
Hidde Beydals 9fd16d8e45 Switch to scratch based libgit2 container image
This moves the `libgit2` compilation to the image, to ensure it
can be build on builders that aren't backed by AMD64.

The image is structured in such a way that e.g. running nightly
builds targeting a different Go version, or targeting a different
OS vendor would be possible in the future via build arguments.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-08 09:38:20 +02:00
Hidde Beydals 029617cb49 Detect macOS produced libgit2.dylib on Darwin
Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-04 15:36:43 +02:00
Hidde Beydals a5a90fd4f6 Allow libgit2 build to be enforced
This can be useful on machines where libgit2 is installed due to other
applications depending on it, but where the composition of this
installation does not properly work with the controller.

Reason the system version is still preferred, is because this lowers the
barrier for drive-by contributors, as a working set of (Git) dependencies
should only really be required if you are going to perform work in that
domain.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-04 15:36:43 +02:00
Hidde Beydals 981d91e363 Update base image to version with Darwin detection
To provide a better (contributing) experience to those with Apple
machines, as determining the correct paths there is a bit harder.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2021-10-01 19:08:59 +02:00