Commit Graph

20 Commits

Author SHA1 Message Date
Otto Yiu e4427e9672 improve S3 url parsing for vfsPath to support more naming conventions 2018-01-12 16:07:18 -08:00
Justin Santa Barbara 1ac08b5cf1 Add missed error handling on session.NewSession 2017-08-28 07:52:11 -04:00
Justin Santa Barbara a26b28576e AWS: Always use verbose errors
As otherwise very difficult to diagnose errors
2017-08-25 23:08:39 -04:00
Lars Lehtonen a9bbe3af24
Wrap AWS NewSession() errors in vfs package 2017-08-25 13:25:04 -07:00
Lars Lehtonen afea9d05c5
Replace deprecated aws session.New() with session.NewSession() 2017-08-22 17:28:55 -07:00
Lars Lehtonen 4e847ce020
Fix swallowed err variable in vfs package 2017-07-18 18:09:39 -07:00
Miao Luo 22c49e76cb Fix user-defined s3 endpoint support.
Address review feedbacks and remove unintended space.
2017-04-19 23:46:05 -07:00
Miao Luo 76437a77d4 Support user-defined s3 endpoint. 2017-04-18 11:27:07 -07:00
Justin Santa Barbara 8104ba2cea More logging around errors in s3 write path
Hopefully to shed some light on issues like #2108
2017-03-15 09:52:32 -04:00
Seth Pollack b1702d749e
validate region 2017-02-23 12:03:03 -05:00
Vincent Ambo 8c85935f1f
fix s3context: Attempt a normal S3 call before bruteforcing location
In cases where the user is the bucket owner an initial call to
s3.GetBucketLocation will succeed. If it does return an error we
fall back to the bruteforce method.

This effectively makes the behaviour unchanged from previous versions
for bucket owners.
2017-01-30 08:01:39 +01:00
Vincent Ambo cd6f9eb66a refactor s3context: Use Go time constants 2017-01-27 13:39:15 +01:00
Vincent Ambo 0728b7c9fa refactor vfs: Create location request further down
Minor refactor, the request was created one level up originally
because I had added two separate steps for initially determining
whether we have to use the bruteforce method.

However this is a premature optimisation and unnecessary due to the
concurrency behaviour we've got now.
2017-01-26 19:52:23 +01:00
Vincent Ambo 3b6e3bda56 chore: Run gofmt on new VFS code 2017-01-26 12:30:04 +01:00
Vincent Ambo 5a25a96c93 feat vfs: Fetch S3 bucket locations cross-account
The AWS API makes it difficult to retrieve S3 bucket locations from shared buckets
with bucket-policy based access delegations. This introduces a workaround for the
issue.

AWS is aware of the issue but for the time being they can not provide information
about when it will be fixed.

See #1247 for more information.
2017-01-26 11:47:09 +01:00
Justin Santa Barbara b36b75ad0c Always set CredentialsChainVerboseErrors when initializing AWS
Fix #605
2017-01-03 20:03:03 -05:00
yancl aec534a93f change KOPS_AWS_REGION to AWS_REGION 2016-12-05 15:04:03 +00:00
yancl bc444fd535 fix the aws region that hardcoded to "us-east-1" which doesn't work in
some isolated regions(cn-north-1 and GovCloud, for example)
2016-11-21 08:53:45 +00:00
chrislovecnm 8fa2aac99f fixing more headers 2016-10-15 19:20:56 -06:00
Justin Santa Barbara 767c9a6416 Fix tests; don't hit s3 until we need to 2016-10-07 01:38:44 -04:00