Commit Graph

65 Commits

Author SHA1 Message Date
Vincent Ambo 5a25a96c93 feat vfs: Fetch S3 bucket locations cross-account
The AWS API makes it difficult to retrieve S3 bucket locations from shared buckets
with bucket-policy based access delegations. This introduces a workaround for the
issue.

AWS is aware of the issue but for the time being they can not provide information
about when it will be fixed.

See #1247 for more information.
2017-01-26 11:47:09 +01:00
Vincent Ambo db0155b5b5 feat s3fs: Allow bucket object ACL override
When sharing S3 buckets across accounts it may be necessary to override ACLs
per object to avoid locking out different accounts.

This commit lets users specify a `KOPS_STATE_S3_ACL` environment variable which
(if specified) overrides the ACL in the PutObject request.

Fixes #907
2017-01-26 11:47:09 +01:00
Justin Santa Barbara bb42ae6723 Recognize file:// urls 2017-01-25 23:33:53 -05:00
Justin Santa Barbara 8c84ed3fe8 Retry readHttpLocation on 500 errors
Fix #1441
2017-01-15 20:59:02 -05:00
Justin Santa Barbara 0a56d3d2e1 Raise an error on an invalid s3 path
Fix #902
2017-01-08 14:37:46 -05:00
Justin Santa Barbara b36b75ad0c Always set CredentialsChainVerboseErrors when initializing AWS
Fix #605
2017-01-03 20:03:03 -05:00
Justin Santa Barbara afd0c25abe First model -> tf test 2016-12-11 17:11:10 -05:00
yancl aec534a93f change KOPS_AWS_REGION to AWS_REGION 2016-12-05 15:04:03 +00:00
yancl bc444fd535 fix the aws region that hardcoded to "us-east-1" which doesn't work in
some isolated regions(cn-north-1 and GovCloud, for example)
2016-11-21 08:53:45 +00:00
chrislovecnm 8fa2aac99f fixing more headers 2016-10-15 19:20:56 -06:00
Justin Santa Barbara defa53bb89 Fix S3 initialization 2016-10-07 01:52:01 -04:00
Justin Santa Barbara 767c9a6416 Fix tests; don't hit s3 until we need to 2016-10-07 01:38:44 -04:00
Justin Santa Barbara f7fa324858 Initial k8s-style kops API
We try to emulate the k8s Clientset approach
2016-10-07 01:13:02 -04:00
Justin Santa Barbara 39c3c85262 Skip directory objects in S3 when listing files
Issue #520
Issue #548
2016-09-30 10:07:15 -04:00
Justin Santa Barbara 5f8d68ec85 Refactor shared packages into util directory 2016-09-25 18:27:09 -04:00