Commit Graph

57 Commits

Author SHA1 Message Date
Ciprian Hacman ef3df171a8 hetzner: Add support for Object Storage 2025-06-22 09:29:28 +03:00
Pierre Prinetti 1005dda58e
Upgrade to Gophercloud v2
Essentially:

* Add a context argument to Gophercloud functions
* Refactor scheduler hints for server creation
2024-10-23 12:33:34 +02:00
Ciprian Hacman f073917ef1 aliyun: Remove some leftovers 2024-04-26 12:13:15 +03:00
Peter Rifel 1e14df43b6
Add missing defer statements from imds queries 2024-03-24 18:05:25 -05:00
Peter Rifel 0290a7e8d7
Migrate some packages to aws-sdk-go-v2 2024-03-23 19:42:28 -05:00
Matt Morrison 1944176ac1
Update GCE storage service scope to FullControl 2024-02-15 16:26:05 +13:00
Ciprian Hacman f8ebec5009 azure: Migrate vfs to new SDK version 2024-02-09 05:58:45 +02:00
John Gardiner Myers 0d1c6371d6 Get VFSContext from Factory in channels 2023-07-15 15:32:50 -07:00
John Gardiner Myers 11304807f2 Hold reference to VFSContext from simple.Clientset 2023-07-06 19:41:45 -07:00
justinsb 6c2edaee7e Add Context arg to vfs ReadFile
This is an "action" method, so should take a context.
2023-01-01 09:51:44 -05:00
justinsb c81ec7f435 vfs: azure client support for context and defer client construction
Updating azure to follow the context patterns we've defined, where
BuildVFSPath no longer makes network calls.
2022-12-22 10:32:11 -05:00
justinsb 6406bfe150 VFS: Add context to internals of Openstack Swift client
Setting up context usage internally ready for the context to be
exposed.

Also avoid initializing the client until the first usage, making
building VFS path more of a builder function (deterministic, not
expected to fail for good inputs).
2022-12-21 10:07:03 -05:00
justinsb 46f27168b8 vfs: avoid early initialization of GCS client
By initializing on demand, we avoid the need for some context.Context
during "build" time and better reflect the notion of (passive)
builders vs (active) requests.
2022-12-20 15:23:16 -05:00
justinsb 5fde739814 Introduce context.Context into some of our "Context" objects
The rule of thumb is that we shouldn't be embedding a context.Context,
but it is reasonable when the lifetime is similar and when the
refactor would otherwise be unacceptably large.

This is a minimal way to introduce it, based on adding the support
needed in the GCS support for serviceAccountIssuerDiscovery.  We will
need to plumb through the context in many more places over time.
2022-12-12 09:56:09 -05:00
Ciprian Hacman dbef6209c2 Remove support for using Vault as state store
Co-authored-by: John Gardiner Myers <jgmyers@proofpoint.com>
2022-11-30 18:38:21 +02:00
Leïla MARABESE 76562f80d5 support for scaleway in s3 buckets 2022-09-01 12:06:23 +02:00
Ciprian Hacman cb99db0757 Run make goimports 2022-08-17 07:03:33 +03:00
John Gardiner Myers a0736b3c29 Remove support for Aliyun/Alibaba Cloud 2021-12-11 21:49:13 -08:00
Eng Zer Jun 425173ae9f
refactor: move from io/ioutil to io and os packages
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2021-11-12 15:37:18 +08:00
Kenji Kaneda 40c944aa5c Fix a typo in an error message returned from buildAzureBlobPath
invalid Azure Blob schem -> invalid Azure Blob scheme
2020-11-23 08:16:39 -08:00
Kenji Kaneda 4555c0b2df Add support of Azure Blob storage to VFS
The schema is "azureblob".

azureClient provides two ways to set up credential. One approach is to
use an account key stored in env var. This approach is used when
accessing Blob from kops CLI. The second approach is to retrieve
credentials from Instance Metadata Service. This works only when
azureClient is created on a VM that has sufficient privilege to access
a specified blob. This approach is used from nodeup, etcd-manager,
etc.
2020-11-19 10:47:03 -08:00
Ciprian Hacman 0c6f1c733c Use all kops mirrors to determine artifacts hashes 2020-09-18 09:44:37 +03:00
Peter Rifel 4d9f0128a3
Upgrade to klog2
This splits up the kubernetes 1.19 PR to make it easier to keep up to date until we get it sorted out.
2020-08-16 20:56:48 -05:00
Ole Markus With d1479fb666 Add support for reading openstack metadata in vfs 2020-08-04 08:22:00 +02:00
Ole Markus With acaa1e1dfc Implement VFS for vault 2020-06-18 13:02:37 +02:00
Justin SB 7d7b8969ea Use AWS SDK to fetch metadata
Previously the EC2 metadata service was straightforward HTTP, but IMDS
v2 now requires managing a session token (and is more secure for it).

We now use the AWS SDK when retrieving metadata; it automatically
supports IMDS v2.
2020-05-31 17:23:01 -04:00
Justin SB 5ed11fd9c7 GCE: don't rely on hostname being correct
Distros that use systemd for DHCP often don't have the hostname
correct, due to e.g. the requirement for policy kit.

We don't rely on it being set correctly on other clouds; no real
reason to require it on GCP either!
2020-05-17 15:20:58 -04:00
Justin Santa Barbara 108d1eee5d Replace deprecated x/net/context with context
It's not x-perimental any more!
2020-04-09 23:58:19 -04:00
Justin Santa Barbara 5ebbfc96b9
Replace deprecated method calls to google cloud libraries
Required for static-check to pass.
2020-01-17 06:38:43 -05:00
tanjunchen 3f9400a588 util/pkg/vfs/:staticcheck 2019-12-23 10:20:56 +08:00
Xiaoyu Zhong 002ddbb270 Alicloud: add hostname override 2019-10-16 21:53:40 +08:00
mikesplain 9e55b8230a Update copyright notices
Also cleans some white spaces
2019-09-09 14:47:51 -04:00
Justin SB eca2ac6b80
Look for sha256 and sha1 files for artifacts 2019-08-23 18:26:25 -04:00
Justin SB 3e33ac7682
Change code from glog to klog
We don't call klog.InitFlags yet, because that will cause a flag
redefinition error until we get everyone to stop using glog.  That
will happen when we update to k8s 1.13.
2019-05-06 12:54:51 -04:00
Justin SB 4522a9bc66
Always log when a retry loop fails
We want to be sure the retry loop is working, and we want to know when
we're incurring retry failures (if something is expected to fail).
2018-12-21 14:16:51 -05:00
andrewsykim 54bee09f47 digitalocean: add kubelet hostname override 2018-04-03 01:16:50 -04:00
Xiao An 4aa68d2de9 a few updates based on suggestions
Signed-off-by: Xiao An <hac@zju.edu.cn>
2018-04-02 15:29:18 +08:00
Xiao An 18e160748e add VFS implementation with Aliyun OSS
Signed-off-by: Xiao An <hac@zju.edu.cn>
2018-04-02 15:23:36 +08:00
andrewsykim 6fa37bf005 add digitalocean VFS 2018-04-01 23:05:46 -04:00
Justin Santa Barbara 82b9a54332 VFS: Recognize file:// paths 2018-01-27 15:03:05 -08:00
zengchen1024 bbfd1e18a3 implement vfs with openstack swift 2017-11-02 17:08:16 +08:00
chrislovecnm 8d1ee1fa16 updating files for goimports 2017-11-01 12:51:43 -06:00
Justin Santa Barbara d1ee8026ac GCE: Tasks for object & bucket level permissions
We also switch to setting a bucket-level ACL permission, as this
requires less permissions.
2017-10-29 18:08:08 -04:00
Justin Santa Barbara e945322cab Support generated clientset as alternative to vfs clientset
We modelled our VFS clientset (for API objects backed by a VFS path)
after the "real" clientsets, so now it is relatively easy to add a
second implementation that will be backed by a real clientset.

The snafu here is that we weren't really using namespaces previously.
Namespaces do seem to be the primary RBAC scoping mechanism though, so
we start using them with the real clientset.

The namespace is currently inferred from the cluster name.  We map dots
to dashes, because of namespace limitations, which could yield
collisions, but we'll deal with this by simply preventing users from
creating conflicting cluster names - i.e. you simply won't be able to
create a.b.example.com and a-b.example.com
2017-06-20 22:29:37 -04:00
yissachar e873950633 Merge pull request #2113 from justinsb/vfs_path_validation_error
Fix error for invalid vfs paths
2017-03-18 21:36:26 -04:00
Justin Santa Barbara cb4641fea3 Code updates 2017-03-16 02:40:50 -04:00
Justin Santa Barbara 32b9e1b5b1 Fix error for invalid vfs paths
We should be printing the path when it fails validation
2017-03-13 10:42:31 -04:00
Justin Santa Barbara 645f330dad Re-enable GCE support
We move everything to the models.  We feature-flag it, because we
probably want to change the names etc, and we aren't going to be able to
offer smooth upgrades until that is done.
2017-02-28 20:08:03 -05:00
Justin Santa Barbara 1e5dab5a6c Actually retry on HTTP 404s
We were returning the "don't retry" value (true).

Also tidy up the logic a little.
2017-02-05 13:17:09 -05:00
Justin Santa Barbara bb42ae6723 Recognize file:// urls 2017-01-25 23:33:53 -05:00