Commit Graph

1474 Commits

Author SHA1 Message Date
Paulo Gomes 860d7051da
libgit2: optimise checkout branch clones
No-op reconciliations are very inefficient, as they carry out
a full clone operation of the target repository even when
no changes have taken place.

This change will execute a remote-ls operation, and cancel
the clone operation if the remote tip commit is still the same
as the one observed on the last reconcilation. In such cases,
an git.NoChangesError is returned.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-05-11 11:40:04 +01:00
Paulo Gomes 5b4750b87d
Merge pull request #715 from pjbgf/minio-region
Remove minio region
2022-05-10 15:41:17 +01:00
Paulo Gomes c666f50424
Remove minio region
Twice recently the minio region seemed to have changed
unsetting it seems to work consistently.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-05-10 14:28:54 +01:00
Paulo Gomes 39f1c5c217
Merge pull request #711 from pjbgf/add-hostkey-flag
Add new flag `--ssh-hostkey-algos`
2022-05-10 11:57:59 +01:00
Paulo Gomes 9fddf6947b
Add new flag --ssh-hostkey-algos
The new flag allow users to set the list of hostkey algorithms
to use for ssh connections, enabling them to ensure specific
are/aren't used.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-05-09 16:29:35 +01:00
Paulo Gomes 1a06b7ae49
Merge pull request #575 from pjbgf/fix-push
Ensure git status is checked at the correct time
2022-05-09 16:28:55 +01:00
Paulo Gomes 50cb97f331
Fix make verify failures
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-05-09 16:16:04 +01:00
Paulo Gomes c936fc145b
Ensure git status is tracked at the correct time
The previous version would verify git status at time of `make verify` execution, but before all its dependencies executed.
If one of the dependencies resulted in a dirty git, the check would still pass.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-05-09 15:54:55 +01:00
Paulo Gomes 43235df2b8
Merge pull request #708 from pjbgf/algo-tests
Add Git test coverage for supported algorithms
2022-05-09 14:24:03 +01:00
Paulo Gomes 2b59150fbe
tests: algorithms test coverage for go-git and libgit2
Assures support for:
- Authentication Key Types
  - rsa
  - ecdsa P256
  - ecdsa P384
  - ecdsa P521
  - ed25519
- Key Exchange Algoritms:
  - diffie-hellman-group14-sha1
  - diffie-hellman-group14-sha256
  - curve25519-sha256
  - ecdh-sha2-nistp256
  - ecdh-sha2-nistp384
  - ecdh-sha2-nistp521
  - curve25519-sha256@libssh.org
- HostKey Algoritms:
  - ssh-rsa
  - rsa-sha2-256
  - rsa-sha2-512
  - ecdsa-sha2-nistp256
  - ecdsa-sha2-nistp384
  - ecdsa-sha2-nistp521
  - ssh-ed25519

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-05-09 13:31:54 +01:00
Paulo Gomes 4e3e62923b
git: Add git.HostKeyAlgos
Enables the setting of HostKey algorithms to be used from
a client perspective. This implementation supports go-git
and libgit2 when in ManagedTransport.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-05-06 17:58:09 +01:00
Paulo Gomes d425923a54
Merge pull request #707 from pjbgf/libgit2_panic
libgit2: recover from git2go panic
2022-05-06 09:31:53 +01:00
Paulo Gomes d86ea25e87
Fix minio test by changing region to us-west-2
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-05-06 08:59:56 +01:00
Paulo Gomes c0c5046e5f
libgit2: recover from git2go panic
Some specific scenarios may lead libgit2 or git2go to panic.
Adding a recovery logic, ensures a predictable execution path
for callers, and safeguards the controller's stability.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-05-06 08:39:00 +01:00
Hidde Beydals a952c25967
Merge pull request #706 from fluxcd/release-v0.24.4 2022-05-03 13:53:47 +02:00
Hidde Beydals fb383c9fcd Release v0.24.4
Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-05-03 13:15:04 +02:00
Hidde Beydals dbd3113a79
Merge pull request #705 from fluxcd/update-deps
Update dependencies
2022-05-03 12:59:21 +02:00
Hidde Beydals 5b32ef8269 Update dependencies
- github.com/fluxcd/pkg/runtime to v0.14.2
- golang.org/x/crypto to v0.0.0-20220427172511-eb4f295cb31f
- google.golang.org/api to v0.77.0
- k8s.io/api to v0.23.6
- k8s.io/client-go to v0.23.6

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-05-03 12:44:15 +02:00
Hidde Beydals a5e5fe13bd api: update dependencies
- github.com/fluxcd/pkg/apis/meta to v0.13.0
- k8s.io/apimachinery to v0.23.6

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-05-03 12:44:08 +02:00
Stefan Prodan 4e30a4f63e
Merge pull request #703 from fluxcd/target-condition-in-result
summarize: Consider obj status condition in result
2022-05-03 13:26:25 +03:00
Sunny 2240106c8e
summarize: Consider obj status condition in result
SummarizeAndPatch() should also consider the object's status conditions
when computing and returning the runtime results to avoid any
inconsistency in the runtime result and status condition of the object.
When an object's Ready condition is False, the reconciler should retry
unless it's in stalled condition.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-04-30 10:12:21 +05:30
Alexander Block 065a760752 docs: Remove all traces of "identity.pub" from docs
"identity.pub" is referenced multiple times in CRDs and docs. This secret
is however never used in any place. Instead, the public key is derived from
the "identity" private key.

This commit/PR removes all traces of "identity.pub" from v1beta2 CRDs and
docs.

Signed-off-by: Alexander Block <ablock84@gmail.com>
2022-04-29 21:17:53 +02:00
Max Jonas Werner 1e83f0ecf5 fix API docs link
v0.3.0 of gen-crd-api-reference-docs has a bug that leads to it
rendering incorrect links for local types. This is fixed on the master
branch but not released, yet, so I'm pinning the version of the tool
to the latest current commit.

Signed-off-by: Max Jonas Werner <mail@makk.es>
2022-04-29 19:27:53 +02:00
Hidde Beydals 89a4e52fe3
Merge pull request #701 from Juneezee/test/t.TempDir 2022-04-29 17:39:23 +02:00
Eng Zer Jun 10b92da433
test: use `T.TempDir` to create temporary test directory
This commit replaces `os.MkdirTemp` with `t.TempDir` in tests. The
directory created by `t.TempDir` is automatically removed when the test
and all its subtests complete.

Prior to this commit, temporary directory created using `os.MkdirTemp`
needs to be removed manually by calling `os.RemoveAll`, which is omitted
in some tests. The error handling boilerplate e.g.
	defer func() {
		if err := os.RemoveAll(dir); err != nil {
			t.Fatal(err)
		}
	}
is also tedious, but `t.TempDir` handles this for us nicely.

Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-04-29 23:01:39 +08:00
Hidde Beydals bc5a47e821
Merge pull request #695 from fluxcd/release-v0.24.3 2022-04-28 09:43:25 +02:00
Hidde Beydals 57e7c79098 Release v0.24.3
Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-04-28 09:11:42 +02:00
Max Jonas Werner 9c6a6d91e8 build: let users pass test flags to `make test`
This comes in handy e.g. to only run a single test during development.

Signed-off-by: Max Jonas Werner <mail@makk.es>
2022-04-28 06:31:45 +02:00
Sunny 745d6ee0c5
Merge pull request #691 from fluxcd/cached-helmrepo-diff-checksum
helmrepo: same revision different checksum scenario
2022-04-27 15:28:29 +05:30
Sunny eeaa958866
helmrepo: same revision different checksum condn
This change prevents Reconciling and ArtifactOutdated conditions to be
set on HelmRepo when the checksum of a cached repo index changes.

Adds some tests to ensure that when the repo index is cached, the
revision and checksum of the returned artifact are the same as on the
existing object status.
Also adds checks for the returned artifact and chartRepo from
reconcileSource, to ensure that chartRepo is populated and the checksum
of a new potential artifact is always empty, as it's populated when the
artifact is written in the storage.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-04-27 13:03:51 +05:30
Hidde Beydals dd40748e96
Merge pull request #688 from fluxcd/release-v0.24.2 2022-04-26 09:12:18 +02:00
Hidde Beydals 4c0f22171c Release v0.24.2
Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-04-26 08:53:17 +02:00
Hidde Beydals d7e36e2ec0
Merge pull request #685 from pjbgf/optimise-helm-load
helm: optimise repository index loading
2022-04-26 08:10:28 +02:00
Paulo Gomes 009504b294 helm: optimise repository index loading
Avoid validating (and thus loading) indexes if the checksum already exists in storage.
In other words, if the YAML is identical to the Artifact in storage, the reconciliation should
be a no-op, and therefore can short-circuit long/heavy operations.

Co-authored-by: Hidde Beydals <hello@hidde.co>
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-04-25 17:00:27 +02:00
Paulo Gomes 3c67efaf8f
Merge pull request #686 from codablock/fix-git-tests
tests: Fix flakiness of git related tests
2022-04-25 15:53:34 +01:00
Alexander Block fa09aa2a39 tests: Use proper timeouts in gitrepository and helmrepository tests
I assume using "interval" for timeouts was an accident and "timeout" was
actually meant to be used. This also fixes flakiness of tests.

Signed-off-by: Alexander Block <ablock84@gmail.com>
2022-04-25 14:30:25 +02:00
Alexander Block 7c8b175258 tests: Set GIT_CONFIG_GLOBAL=/dev/null for tests run via make test
This ensures that no host specific git configuration is pulled in while
testing. One such example would be "init.defaultBranch" which is set to
"main" on some hosts, which then causes tests to fail as they keep looking
for "master" branches.

Signed-off-by: Alexander Block <ablock84@gmail.com>
2022-04-25 14:30:16 +02:00
Hidde Beydals cfbe6ac124
Merge pull request #684 from fluxcd/release-v0.24.1 2022-04-22 11:06:37 +02:00
Hidde Beydals d2132086d0 Release v0.24.1
Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-04-22 10:50:29 +02:00
Hidde Beydals 0836d882ba
Merge pull request #683 from fluxcd/fix-panic 2022-04-22 10:42:17 +02:00
Max Jonas Werner c3d5dac4a8
fix panic when HelmRepository's artifact size is nil
This fixes the immediate issue of the nil pointer dereference but we
still haven't isolated the actual cause of the size being nil to begin
with. This is ongoing work and as soon as we have boiled that down to
the simplest case we will provide a regression test for that case.

closes #680

Signed-off-by: Max Jonas Werner <mail@makk.es>
Co-authored-by: Hidde Beydals <hiddeco@users.noreply.github.com>
2022-04-22 10:29:56 +02:00
Hidde Beydals c51b359a6a
Merge pull request #678 from fluxcd/release-v0.24.0 2022-04-19 13:06:47 +02:00
Hidde Beydals c83f0ad96c Release v0.24.0
Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-04-19 12:53:03 +02:00
Hidde Beydals 3559e7aac3
Merge pull request #677 from fluxcd/update-deps 2022-04-19 12:52:21 +02:00
Hidde Beydals 67ef697630 Update github.com/elazarl/goproxy
Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-04-19 12:19:54 +02:00
Hidde Beydals b9b123c063
Merge pull request #667 from souleb/use-cache-when-building-deps 2022-04-19 12:05:20 +02:00
Soule BA 0df8dcccec
Enable dependency manager to use in memory cache
If implemented this will:
- enable the helmCharts dependency manager to use the helm in memry
  cache to retrieve reconciled HelmRepositories indexes.
- record cache events.

Signed-off-by: Soule BA <soule@weave.works>
2022-04-19 11:46:27 +02:00
Hidde Beydals a4d339bf25
Merge pull request #676 from fluxcd/update-deps 2022-04-15 16:58:20 +02:00
Hidde Beydals 18b59351c1 Update dependencies
Azure SDK dependencies cannot be updated, as this requires us to move to
Go 1.18.

- cloud.google.com/go/storage to v1.22.0
- github.com/ProtonMail/go-crypto to v0.0.0-20220407094043-a94812496cf5
- github.com/darkowlzz/controller-check to v0.0.0-20220325122359-11f5827b7981
- github.com/elazarl/goproxy to v0.0.0-20220403042543-a53172b9392e
- github.com/fluxcd/pkg/gittestserver to v0.5.2
- github.com/go-logr/logr to v1.2.3
- github.com/minio/minio-go/v7 to v7.0.24
- github.com/onsi/gomega to v1.19.0
- golang.org/x/crypto to v0.0.0-20220411220226-7b82a4e95df4
- google.golang.org/api to v0.74.0

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-04-15 16:41:42 +02:00
Hidde Beydals 59df52d332 Update sigs.k8s.io/cli-utils to v0.29.4
Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-04-15 15:47:41 +02:00