Commit Graph

9 Commits

Author SHA1 Message Date
Stefan Prodan 5be0c53729
Promote Bucket API to v1
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2024-08-23 13:25:57 +03:00
Matheus Pimenta 02d492bc43 Introduce Bucket provider constants with the common part as a prefix
Signed-off-by: Matheus Pimenta <matheuscscp@gmail.com>
2024-08-22 14:48:53 -03:00
Matheus Pimenta 10ac11314d Add LDAP provider for Bucket STS API
Signed-off-by: Matheus Pimenta <matheuscscp@gmail.com>
2024-08-22 08:41:23 -03:00
Matheus Pimenta 7536ab4b02 Add support for AWS STS endpoint in the Bucket API
Signed-off-by: Matheus Pimenta <matheuscscp@gmail.com>
2024-08-08 16:35:01 +01:00
Matheus Pimenta 4d9c87cf3f Add support for .spec.proxySecretRef for generic provider of Bucket API
Signed-off-by: Matheus Pimenta <matheuscscp@gmail.com>
2024-07-03 15:10:47 +01:00
Matheus Pimenta bc16734391 Add .certSecretRef for Bucket API
Signed-off-by: Matheus Pimenta <matheuscscp@gmail.com>
2024-05-15 19:44:05 +01:00
Stefan Prodan b655149786
bucket: Add prefix filtering capability
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2023-10-17 14:13:04 +03:00
Zhiyu Wang 1ab089e19e Chore: use auto bucket lookup type
Signed-off-by: Zhiyu Wang <zhiyuwang.newbis@gmail.com>
2023-08-31 17:59:41 +08:00
pa250194 ed6c6ebc3c Introduce BucketProvider interface
This commit introduces a BucketProvider interface for fetch operations
against object storage provider buckets. Allowing for easier
introduction of new provider implementations.

The algorithm for conditionally downloading object files is the same,
whether you are using GCP storage or an S3/Minio-compatible
bucket. The only thing that differs is how the respective clients
handle enumerating through the objects in the bucket; by implementing
just that in each provider, I can have the select-and-fetch code in
once place.

The client implementations do now include safe-guards to ensure the
fetched object is the same as metadata has been collected for. In
addition, minor changes have been made to the object fetch operation
to take into account that:

- Etags can change between composition of index and actual fetch, in
  which case the etag is now updated.
- Objects can disappear between composition of index and actual fetch,
  in which case the item is removed from the index.

Lastly, the requirement for authentication has been removed (and not
referring to a Secret at all is thus allowed), to provide support
for e.g. public buckets.

Co-authored-by: Hidde Beydals <hello@hidde.co>
Co-authored by: Michael Bridgen <michael@weave.works>
Signed-off-by: pa250194 <pa250194@ncr.com>
2022-03-01 10:15:10 +01:00