Commit Graph

15 Commits

Author SHA1 Message Date
Stefan Prodan 5be0c53729
Promote Bucket API to v1
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2024-08-23 13:25:57 +03:00
Matheus Pimenta 02d492bc43 Introduce Bucket provider constants with the common part as a prefix
Signed-off-by: Matheus Pimenta <matheuscscp@gmail.com>
2024-08-22 14:48:53 -03:00
Matheus Pimenta 10ac11314d Add LDAP provider for Bucket STS API
Signed-off-by: Matheus Pimenta <matheuscscp@gmail.com>
2024-08-22 08:41:23 -03:00
Matheus Pimenta 7536ab4b02 Add support for AWS STS endpoint in the Bucket API
Signed-off-by: Matheus Pimenta <matheuscscp@gmail.com>
2024-08-08 16:35:01 +01:00
Matheus Pimenta 4d9c87cf3f Add support for .spec.proxySecretRef for generic provider of Bucket API
Signed-off-by: Matheus Pimenta <matheuscscp@gmail.com>
2024-07-03 15:10:47 +01:00
Matheus Pimenta bc16734391 Add .certSecretRef for Bucket API
Signed-off-by: Matheus Pimenta <matheuscscp@gmail.com>
2024-05-15 19:44:05 +01:00
Stefan Prodan b655149786
bucket: Add prefix filtering capability
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2023-10-17 14:13:04 +03:00
Zhiyu Wang 1ab089e19e Chore: use auto bucket lookup type
Signed-off-by: Zhiyu Wang <zhiyuwang.newbis@gmail.com>
2023-08-31 17:59:41 +08:00
Sunny 3a17461777 purge minio test container at the end of tests
Also, add container option to auto-remove a container when stopped, in
case a container is left behind due to some reason.

Signed-off-by: Sunny <darkowlzz@protonmail.com>
2022-12-23 03:46:01 +05:30
Hidde Beydals 57543ae625 minio: use container image for tests
This drops the usage of `play.min.io` in favor of running a local
container image for tests. Which is a better practice, and resolves
test flakiness due to the endpoint being unavailable once in awhile.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-12-14 11:56:53 +00:00
Stefan Prodan f873d71ec5
Use sourceignore from fluxcd/pkg
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-08-17 18:04:48 +03:00
Paulo Gomes c666f50424
Remove minio region
Twice recently the minio region seemed to have changed
unsetting it seems to work consistently.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-05-10 14:28:54 +01:00
Paulo Gomes d86ea25e87
Fix minio test by changing region to us-west-2
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-05-06 08:59:56 +01:00
Eng Zer Jun 10b92da433
test: use `T.TempDir` to create temporary test directory
This commit replaces `os.MkdirTemp` with `t.TempDir` in tests. The
directory created by `t.TempDir` is automatically removed when the test
and all its subtests complete.

Prior to this commit, temporary directory created using `os.MkdirTemp`
needs to be removed manually by calling `os.RemoveAll`, which is omitted
in some tests. The error handling boilerplate e.g.
	defer func() {
		if err := os.RemoveAll(dir); err != nil {
			t.Fatal(err)
		}
	}
is also tedious, but `t.TempDir` handles this for us nicely.

Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-04-29 23:01:39 +08:00
pa250194 ed6c6ebc3c Introduce BucketProvider interface
This commit introduces a BucketProvider interface for fetch operations
against object storage provider buckets. Allowing for easier
introduction of new provider implementations.

The algorithm for conditionally downloading object files is the same,
whether you are using GCP storage or an S3/Minio-compatible
bucket. The only thing that differs is how the respective clients
handle enumerating through the objects in the bucket; by implementing
just that in each provider, I can have the select-and-fetch code in
once place.

The client implementations do now include safe-guards to ensure the
fetched object is the same as metadata has been collected for. In
addition, minor changes have been made to the object fetch operation
to take into account that:

- Etags can change between composition of index and actual fetch, in
  which case the etag is now updated.
- Objects can disappear between composition of index and actual fetch,
  in which case the item is removed from the index.

Lastly, the requirement for authentication has been removed (and not
referring to a Secret at all is thus allowed), to provide support
for e.g. public buckets.

Co-authored-by: Hidde Beydals <hello@hidde.co>
Co-authored by: Michael Bridgen <michael@weave.works>
Signed-off-by: pa250194 <pa250194@ncr.com>
2022-03-01 10:15:10 +01:00