Commit Graph

217 Commits

Author SHA1 Message Date
Stefan Prodan 34bb9862e6
Promote OCIRepository API to v1 (GA)
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2025-05-27 08:19:51 +03:00
Stefan Prodan 5be0c53729
Promote Bucket API to v1
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2024-08-23 13:25:57 +03:00
Matheus Pimenta 02d492bc43 Introduce Bucket provider constants with the common part as a prefix
Signed-off-by: Matheus Pimenta <matheuscscp@gmail.com>
2024-08-22 14:48:53 -03:00
Matheus Pimenta 10ac11314d Add LDAP provider for Bucket STS API
Signed-off-by: Matheus Pimenta <matheuscscp@gmail.com>
2024-08-22 08:41:23 -03:00
Matheus Pimenta b6bd2abe2d Add proxy support for Azure buckets
Signed-off-by: Matheus Pimenta <matheuscscp@gmail.com>
2024-08-09 22:17:06 +01:00
Sunny 0618f54717
azure-blob: Fix VisitObjects() in integration test
Signed-off-by: Sunny <github@darkowlzz.space>
2024-08-09 18:58:17 +00:00
Matheus Pimenta 31ed900a90 Add proxy support for GCS buckets
Signed-off-by: Matheus Pimenta <matheuscscp@gmail.com>
2024-08-09 19:05:10 +01:00
Matheus Pimenta 7536ab4b02 Add support for AWS STS endpoint in the Bucket API
Signed-off-by: Matheus Pimenta <matheuscscp@gmail.com>
2024-08-08 16:35:01 +01:00
Matheus Pimenta 4d9c87cf3f Add support for .spec.proxySecretRef for generic provider of Bucket API
Signed-off-by: Matheus Pimenta <matheuscscp@gmail.com>
2024-07-03 15:10:47 +01:00
Matheus Pimenta bc16734391 Add .certSecretRef for Bucket API
Signed-off-by: Matheus Pimenta <matheuscscp@gmail.com>
2024-05-15 19:44:05 +01:00
Stefan Prodan 6a2b26fcdf
Adapt GCS mock server for `cloud.google.com/go/storage v1.39.1`
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2024-04-11 10:04:43 +03:00
Stefan Prodan 674c287dbd
Bucket: add prefix support for GCP
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2023-10-17 15:10:33 +03:00
Stefan Prodan b655149786
bucket: Add prefix filtering capability
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2023-10-17 14:13:04 +03:00
Hidde Beydals b0d94ce6d0
misc: do not capitalize err string and fix wording
Signed-off-by: Hidde Beydals <hidde@hhh.computer>
2023-10-11 11:03:56 +02:00
Hidde Beydals 20230811e4
misc: address duplicate imports
Signed-off-by: Hidde Beydals <hidde@hhh.computer>
2023-10-11 11:03:54 +02:00
Hidde Beydals a74eb57811
misc: `math/rand.Read()` has been deprecated
See: https://pkg.go.dev/math/rand@go1.20#Read

Signed-off-by: Hidde Beydals <hidde@hhh.computer>
2023-10-11 11:03:54 +02:00
Hidde Beydals d9dbd1bdb7
misc: `rand.Seed` deprecation
See: https://pkg.go.dev/math/rand@go1.20#Seed

Signed-off-by: Hidde Beydals <hidde@hhh.computer>
2023-10-10 09:40:44 +02:00
Hidde Beydals 691d925add
misc: ensure return errs are captured
Signed-off-by: Hidde Beydals <hidde@hhh.computer>
2023-10-09 15:11:09 +02:00
Hidde Beydals 25400d8810
misc: remove redundant return statements
Signed-off-by: Hidde Beydals <hidde@hhh.computer>
2023-10-09 15:00:04 +02:00
Zhiyu Wang 1ab089e19e Chore: use auto bucket lookup type
Signed-off-by: Zhiyu Wang <zhiyuwang.newbis@gmail.com>
2023-08-31 17:59:41 +08:00
Hidde Beydals 24d37a8e77
gcp: update recognized mock storage server path
This is a candidate for a refactoring / merge together with `mock/gcs`,
but this deals with the issue for now.

Signed-off-by: Hidde Beydals <hidde@hhh.computer>
2023-07-03 14:09:58 +02:00
Stefan Prodan f58c229bc6 Update cosign to v2
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2023-05-22 11:08:12 +02:00
Somtochi Onyekwere 37e113a422 Support Azure Workload Identity
With an update to github.com/fluxcd/pkg/oci v0.22.0.

This includes a pin of `github.com/docker/docker` to `v20.10.x`, to
prevent Oras from complaining.

Co-authored-by: Hidde Beydals <hidde@hhh.computer>
Signed-off-by: Somtochi Onyekwere <somtochionyekwere@gmail.com>
2023-03-30 11:14:55 +02:00
Sunny 3a17461777 purge minio test container at the end of tests
Also, add container option to auto-remove a container when stopped, in
case a container is left behind due to some reason.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-12-23 03:46:01 +05:30
Hidde Beydals 57543ae625 minio: use container image for tests
This drops the usage of `play.min.io` in favor of running a local
container image for tests. Which is a better practice, and resolves
test flakiness due to the endpoint being unavailable once in awhile.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-12-14 11:56:53 +00:00
Hidde Beydals 76c1f531bd Update to azure-sdk-for-go/sdk/azcore v1.2.0
Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-11-08 13:42:00 +00:00
Somtochi Onyekwere e50b6ba8da Update Azure Blob Storage SDK to v0.5.1
Signed-off-by: Somtochi Onyekwere <somtochionyekwere@gmail.com>
2022-11-08 13:41:52 +00:00
Somtochi Onyekwere 754b20b9e0 Update Azure Blob Storage SDK to v0.5.0
Signed-off-by: Somtochi Onyekwere <somtochionyekwere@gmail.com>
2022-11-08 13:40:08 +00:00
Sanskar Jaiswal b6d6b593c8
gitrepo: refactor reconciler to use fluxcd/pkg/git
Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
2022-10-26 14:04:00 +01:00
Somtochi Onyekwere 5eeaa6455d List objects instead when checking if bucket exists in Azure
Signed-off-by: Somtochi Onyekwere <somtochionyekwere@gmail.com>
2022-10-07 20:11:04 +01:00
Stefan Prodan ca3496e758
Build with Go 1.19
- Update Go to 1.19 in CI
- Use Go 1.19 in base image
- Update controller-gen v0.8.0 and regenerate manifests

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-09-28 15:35:26 +03:00
Stefan Prodan 6a367ec70b
Update Kubernetes packages to v1.25.0
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-08-26 12:26:38 +03:00
Stefan Prodan f873d71ec5
Use sourceignore from fluxcd/pkg
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-08-17 18:04:48 +03:00
Somtochi Onyekwere 35268638ba Add Support for SAS keys in Azure Blob
Signed-off-by: Somtochi Onyekwere <somtochionyekwere@gmail.com>
2022-08-11 20:49:59 +01:00
Philip Laine fdf6b01817
Update Azure Go SDK
Signed-off-by: Philip Laine <philip.laine@gmail.com>
2022-08-01 12:27:01 +03:00
Sanskar Jaiswal cd03c26083 libgit2: enforce timeouts for http transports
Attach context to HTTP requests at the transport level to honour
timeouts.

Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
2022-07-20 18:56:37 +05:30
Sanskar Jaiswal a252532c65 libgit2: overwrite remote url if it exists
Update initRepoWithRemote() so that it overwrites the remote url with
the provided url if the remote already exists, instead of erroring out.

Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
2022-07-20 18:56:37 +05:30
Sanskar Jaiswal f5ada743d5 libgit2: decommission unmanaged transport
Decommission libgit2 unmanaged transport and remove the related feature
gate, making managed transport the default.

Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
2022-07-20 18:56:37 +05:30
Sunny 7325430bc9
libgit2/managed/http: test for incomplete creds
Add test for createClientRequest() where the credentials are incomplete.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-06-23 15:31:46 +05:30
Sanskar Jaiswal 7a708ac6d2 set request auth if both username and password are non empty
BitBucket servers don't accept a username with an empty password, so a
secret with no http auth creds will result in a 401, since we
fall back to "git" for the username and used to set basic auth with that
username without a password.

Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
2022-06-23 14:44:57 +05:30
Paulo Gomes b490a6aae8 libgit2: improve known_hosts error messages
Known hosts can be a difficult problem to troubleshoot.
To make it easier for end users, the generic message has
now been changed with a much more user friendly one.

Now if a known_host is not set, an error message will be
returned, instead of it simply being ignored.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-06-21 20:01:29 +05:30
Sunny a530c5dee2
libgit2/ssh: Embed connection fields in Subtransport
The connection type was created to group the connection related fields
and use mutex to prevent race conditions. Since that's no longer the
case, this puts back those fields in sshSmartSubtransport.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-06-21 17:55:21 +05:30
Paulo Gomes 09fae634df
libgit2: remove deadlock
Some scenarios may lead to deadlocks, specially
in image automation controller.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-06-16 13:54:38 +01:00
Paulo Gomes a7ffb8c8aa
git: Update proxy tests
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-06-14 09:25:53 +01:00
Sunny d38086bd72
Fix race condition in git proxy tests
The variable used to store the information about proxied request was
being written to in the proxy server request handler and read for
assertion at the end of the test.
Replace the boolean variable with an atomic counter to count the number
of requests proxied, preventing the race condition.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-06-14 09:25:52 +01:00
Paulo Gomes cea9ea9142
libgit2: improve subtransport logging
Debugging connection issues can be extremely difficult, even more so at scale or when
concurrent connections are required to trigger specific issues.

Changes:
- Add a correlation identifier for each reconciliation, which allows for greater traceability when
going through all the reconciliation operations - including at transport level.
- Add transportType to segregate HTTP and SSH transport logging.
- SSH operations are now enriched with addr containing server address, and HTTP url.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-06-13 07:35:12 +01:00
Sanskar Jaiswal 590aa325b0 libgit2: fix managed transport enabled flag updation
Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
2022-06-10 18:12:58 +05:30
Sanskar Jaiswal 45ee564e27 assert state of managed transport acc for each test
Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
2022-06-09 22:21:10 +05:30
Sanskar Jaiswal 4ce31319ee libgit2: refactor tests to use managed and unmanaged transport cleanly
Refactors libgit2 checkout tests to test managed and unmanaged
transport by making sure the tests requiring unmanaged transport are run
before, any tests that require managed transport (since disabling
managed transport isn't possible). This is done via arranging the tests
carefully in alphabetically sorted names, i.e. the tests with unmanaged
transport go in `checkout_test.go`, which forces golang to run the tests
in that file before any other tests.

Signed-off-by: Sanskar Jaiswal <jaiswalsanskar078@gmail.com>
2022-06-09 21:01:17 +05:30
Paulo Gomes bfa4e38b84
libgit2: dispose connections in SubTransport.Close
The average SubTransport lifecycle encompass two Actions calls. Previously,
it was attempted to share the same connection across both calls. That did
not work as some Git Servers do not support multiple sessions from the same
connection. The implementation was not fully transitioned into the
"one connection per action" model, which led to connection being leaked.

The transition to RW mutex was to avoid the unnecessary blocking in the
goroutine at the start of the second action call.

It is worth mentioning that now when the context is done,  the client level
resources (connection) will also be freed. This ensures that SSH connections
will not outlive the subtransport.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-06-08 19:39:50 +01:00