Commit Graph

248 Commits

Author SHA1 Message Date
Peter Rifel 701f01a26b
Use switch/case statement 2025-08-22 20:36:02 -05:00
Ciprian Hacman 20b24078d4 azure: Add e2e testing 2025-07-19 16:26:14 +03:00
Ciprian Hacman ef3df171a8 hetzner: Add support for Object Storage 2025-06-22 09:29:28 +03:00
Peter Rifel e632f18644
Use SDK's built-in resolver for S3Path.GetHTTPsUrl 2025-01-07 23:10:40 -06:00
Peter Rifel 7663a76e85
Add unit test for S3Path.GetHTTPsUrl 2025-01-07 23:09:59 -06:00
justinsb 5170257008 s3 vfs: fix delete all versions to handle errors
We were not checking for errors when deleting all versions of a file,
as these errors were not returned at the call level, but instead
we need to check the returned DeleteObjectsOutput.
2024-12-27 13:08:57 -05:00
Pierre Prinetti 1005dda58e
Upgrade to Gophercloud v2
Essentially:

* Add a context argument to Gophercloud functions
* Refactor scheduler hints for server creation
2024-10-23 12:33:34 +02:00
Peter Rifel 8e191496e1
Only set aws_s3_object SSE and ACL if not empty 2024-09-07 10:48:51 -05:00
justinsb a8a0d22a9d vfs: fix file replacement for SFTP
This makes it consistent with other VFS systems, where we can replace existing files.
2024-08-30 09:13:00 -04:00
justinsb a337ca983b refactor: allow access to S3 bucket region
This is useful if we want to build an AWSCloud instance for the S3
bucket, rather than where we are installing resources.
2024-07-04 15:09:12 -04:00
Ciprian Hacman 490a57501f aws: Use EndpointResolverV2 for enabling dual-stack 2024-05-17 06:55:12 +03:00
Ciprian Hacman 017f98c906 azure: Remove unused VFS client code 2024-05-13 07:54:26 +03:00
Ciprian Hacman f073917ef1 aliyun: Remove some leftovers 2024-04-26 12:13:15 +03:00
Peter Rifel 86ad932559
Migrate VFS S3 to aws-sdk-go-v2 2024-03-24 20:01:05 -05:00
Peter Rifel 1e14df43b6
Add missing defer statements from imds queries 2024-03-24 18:05:25 -05:00
Peter Rifel 0290a7e8d7
Migrate some packages to aws-sdk-go-v2 2024-03-23 19:42:28 -05:00
Matt Morrison 1944176ac1
Update GCE storage service scope to FullControl 2024-02-15 16:26:05 +13:00
Ciprian Hacman f8ebec5009 azure: Migrate vfs to new SDK version 2024-02-09 05:58:45 +02:00
Kubernetes Prow Robot a4bd641630
Merge pull request #16050 from sl1pm4t/gcp-sa-issuer
gce: Add support for publishing Service Account Issuer documents to GCS
2023-12-03 01:41:43 +01:00
justinsb d8c449a4f8 Basic OpenTelemetry support
We initially support capturing to a file (in our own format, as it
doesn't appear a suitable format exists).  This means we don't need a
server to capture the traces, and can start capturing through prow
without a lot of infrastructure changes.

Co-authored-by: Peter Rifel <rifelpet@users.noreply.github.com>
2023-11-18 09:12:19 -05:00
Matt Morrison 811f7d0420
Push issuer documents to GCS 2023-10-24 07:57:21 +13:00
justinsb 27dd371c08 Refactor: Add context.Context to more VFS methods
This allows for propagation of opentelemetry traces.
2023-10-14 14:31:05 -04:00
Mia-Cross e12f3dc07b
scaleway: terraform support (#15892)
* tasks dependencies

* FindAddresses should proceed even if LBID is empty

* refacto instance: handle several instances in IG + volumeSize in model

* refacto lb_backend: set servers' IPs in backend task rather than instance

* dns changes for tf support

* render Scaleway s3 objects + add zone to outputs

* add terraform support to documentation

* new instance unique name should be given according to actual instances names

* avoid untimely changes in tf + tag instance IPs to avoid orphaned IPs at cluster deletion

* update integration tests

* review comments

* add IPs to the resources to be deleted with the cluster

* add task + model for DNS record
2023-09-29 04:22:42 -07:00
John Gardiner Myers 0d1c6371d6 Get VFSContext from Factory in channels 2023-07-15 15:32:50 -07:00
Kubernetes Prow Robot 436732b724
Merge pull request #15395 from Codelax/vfs-remove-all
add removeAll to vfs
2023-07-15 14:55:13 -07:00
John Gardiner Myers 11304807f2 Hold reference to VFSContext from simple.Clientset 2023-07-06 19:41:45 -07:00
guoguangwu da8c0ca830 chore: remove refs to deprecated io/ioutil 2023-06-07 18:02:33 +08:00
Jules Casteran e27ae94daf add removeAll to vfs 2023-05-10 15:24:15 +02:00
srikiz efdccb2e65 incorporate review comments 2023-04-11 07:51:36 +05:30
srikiz e77e660ed9 More changes based on testing 2023-04-11 07:51:36 +05:30
Kubernetes Prow Robot dfb9c424d5
Merge pull request #14954 from justinsb/use_google_storage_object_acl
gce terraform: Use google_storage_object_acl
2023-02-11 21:59:30 -08:00
Ciprian Hacman fbabea98e9 aws: Remove S3 region validation 2023-02-04 06:55:33 +02:00
justinsb 0b8cb933ad gce terraform: Use google_storage_object_acl
We are using the schema of google_storage_object_acl.
2023-01-22 10:15:25 -05:00
justinsb c2810a5c96 Refactor to reduce vfs -> terraformWriter dependency 2023-01-04 13:38:47 -05:00
Kubernetes Prow Robot aeca5f9b54
Merge pull request #14926 from justinsb/multiple_tfproviders
Support multiple terraform file providers
2023-01-04 09:24:02 -08:00
justinsb 50556d6ab1 vfs: don't cache bucketDetails in S3Path
We cache it in the S3Context anyway, and this potentially avoids some
race conditions (albeit likely harmless race conditions).

Co-authored-by: John Gardiner Myers <jgmyers@proofpoint.com>
2023-01-04 08:26:48 -05:00
justinsb 707758f1e9 Support multiple terraform file providers
By adding them on-demand, it is arguably a little simpler as well.

Co-authored-by: John Gardiner Myers <jgmyers@proofpoint.com>
2023-01-04 08:14:18 -05:00
justinsb 6c2edaee7e Add Context arg to vfs ReadFile
This is an "action" method, so should take a context.
2023-01-01 09:51:44 -05:00
justinsb ae074bbc6b VFS: Add context to CreateFile & WriteFile
As "request" methods, these should have context parameters.
2022-12-23 10:39:52 -05:00
Kubernetes Prow Robot 5922daed00
Merge pull request #14852 from justinsb/ssh_vfscontext
vfs: wire context into SSHPath
2022-12-22 09:31:38 -08:00
Kubernetes Prow Robot c15ab50738
Merge pull request #14851 from justinsb/azure_context
vfs: azure client support for context and defer client construction
2022-12-22 09:31:26 -08:00
justinsb 4c789c7295 vfs: cleanup of SSH code
Generally cleaning up the SSH/SFTP code, only real change is to use posix rename.
2022-12-22 10:42:40 -05:00
justinsb 0a24aa99be vfs: wire context into SSHPath
It doesn't follow quite the same patterns, but we still need to lay
the groundwork here.
2022-12-22 10:37:50 -05:00
justinsb c81ec7f435 vfs: azure client support for context and defer client construction
Updating azure to follow the context patterns we've defined, where
BuildVFSPath no longer makes network calls.
2022-12-22 10:32:11 -05:00
justinsb 6406bfe150 VFS: Add context to internals of Openstack Swift client
Setting up context usage internally ready for the context to be
exposed.

Also avoid initializing the client until the first usage, making
building VFS path more of a builder function (deterministic, not
expected to fail for good inputs).
2022-12-21 10:07:03 -05:00
Kubernetes Prow Robot da9775972c
Merge pull request #14832 from justinsb/aws_s3_context
vfs: add context to s3 functions
2022-12-20 16:25:25 -08:00
Kubernetes Prow Robot 93a5be1599
Merge pull request #14833 from justinsb/cleanup_double_license_header
Cleanup double-license header in file
2022-12-20 14:11:35 -08:00
justinsb b233bdd67d Cleanup double-license header in file
We somehow ended up with 2 license headers here (the same, though with
different dates).
2022-12-20 15:36:31 -05:00
justinsb bba2eaf946 vfs: add context to s3 functions
We don't have the early initialization problem, but we do want to pass
the context through to the requests e.g. for tracing.
2022-12-20 15:34:42 -05:00
justinsb 46f27168b8 vfs: avoid early initialization of GCS client
By initializing on demand, we avoid the need for some context.Context
during "build" time and better reflect the notion of (passive)
builders vs (active) requests.
2022-12-20 15:23:16 -05:00