Ciprian Hacman
8b8dd8d2dd
Use aws_s3_object instead of deprecated aws_s3_bucket_object
2022-04-13 16:49:03 +03:00
Ole Markus With
2fa53989c4
Configure dualstack endpoint for s3
...
Use dualstack https endpoints on ipv6only cluster. Always use
dualstack endpoints through the SDK
2021-11-20 08:00:00 +01:00
Eng Zer Jun
425173ae9f
refactor: move from io/ioutil to io and os packages
...
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil . This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2021-11-12 15:37:18 +08:00
Peter Rifel
7eaa647b2a
Include the provider alias on TerraformPath resources that reference their provider
2021-09-17 18:08:59 -05:00
Peter Rifel
c3a7f9f75a
Add TerraformProvider definition to TerraformPath interface
2021-09-17 18:08:58 -05:00
Ole Markus With
67b4024694
Reconcile if managedFile is public or not
2021-08-13 20:20:44 +02:00
John Gardiner Myers
eb076e3713
Render managed files with Terraform
2021-06-28 12:15:15 -07:00
AkiraFukushima
d52ec60c02
Fix issuer and jwks object path for IRSA
2021-06-01 23:35:21 +09:00
Peter Rifel
7c900b7fae
Generate and upload keys.json + discovery.json to public store
...
Generate and upload keys.json + discovery.json to public store
Don't enable anonymous auth on publicjwks
Remove tests that won't work using FS VFS anymore
2021-03-19 20:03:26 +01:00
Peter Rifel
4d9f0128a3
Upgrade to klog2
...
This splits up the kubernetes 1.19 PR to make it easier to keep up to date until we get it sorted out.
2020-08-16 20:56:48 -05:00
Justin SB
319ddcc333
S3 DeleteAllVersions: use pagination
...
This way we're not limited to one page of versions. This is likely a
purely theoretical concern, at least as we're using it today.
2020-05-31 18:21:05 -04:00
Ciprian Hacman
d54aadc89c
Fix nits for removal of S3 file versions
2020-05-28 06:50:32 +03:00
Ciprian Hacman
9675692b84
Implement RemoveAll() for S3 paths
2020-05-25 07:46:32 +03:00
Ciprian Hacman
b565122875
Remove delete markers also from S3 bucket
2020-05-24 17:42:02 +03:00
Ciprian Hacman
a48ccfa06c
Return warning instead of error to hide issues during cluster teardown
2020-05-24 15:20:20 +03:00
Ciprian Hacman
1a38a3feaa
Return os.ErrNotExist when no versions are found
2020-05-24 11:42:18 +03:00
Ciprian Hacman
56af880c53
Remove TODO that was not addressed for a long time
2020-05-24 10:11:56 +03:00
Ciprian Hacman
831e3f0e57
Remove all versions of a file form the S3 bucket
2020-05-24 08:38:46 +03:00
feifei.zhang@huawei.com
48ebd260d3
fix golint failures
2019-11-24 16:38:58 +08:00
mikesplain
9e55b8230a
Update copyright notices
...
Also cleans some white spaces
2019-09-09 14:47:51 -04:00
Justin SB
3e33ac7682
Change code from glog to klog
...
We don't call klog.InitFlags yet, because that will cause a flag
redefinition error until we get everyone to stop using glog. That
will happen when we update to k8s 1.13.
2019-05-06 12:54:51 -04:00
Justin Santa Barbara
85d47cd67d
s3: lazy-evaluate encryption policy
...
Should help performance a little bit, and should be a little faster.
2018-10-11 06:46:34 -07:00
David Archer
83db56fab0
Use appropriate log level for KOPS_STATE_S3_ACL debug message
2018-08-30 09:58:23 -04:00
Justin Santa Barbara
1753423027
DigitalOcean: don't try to set SSE
...
We lost the p.sse check in a bad merge; restoring it here.
Fix #5519
2018-08-14 21:26:18 -04:00
Christian Jantz
6fba37ea63
Merge branch 'master' of github.com:gekart/kops into feature/s3_bucket_encryption
2018-05-23 10:49:21 +02:00
andrewsykim
2947bb1b9e
allow s3 vfs scheme and sse to be configurable
2018-04-01 23:05:11 -04:00
Grischa Ekart
7c41e35bbc
Implement AWS Default Bucket Encryption PR #4235
2018-03-07 23:26:28 +01:00
Justin Santa Barbara
412cf377c2
VFS: WriteFile takes an io.ReadSeeker
...
Means we don't have to buffer big files in memory, in combination with
WriteTo for reading.
2018-02-26 09:09:17 -05:00
Justin Santa Barbara
6f6bafb65e
VFS: Support io.WriterTo interface
...
Allows us to handle much bigger files (no need to buffer in-memory)
2018-01-08 22:34:27 -05:00
chrislovecnm
7057aaf1bb
Enabling the file assets
...
File assets and the SHA files are uploaded to the new location. Files
when are users uses s3 are upload public read only. The copyfile task
uses only the existing SHA value.
This PR include major refactoring of the use of URLs. Strings are no
longer categnated, but converted into a URL struct and path.Join is
utlilized.
A new values.go file is included so that we can start refactoring more
code out of the "fi" package.
A
2017-12-17 15:26:57 -07:00
chrislovecnm
8d1ee1fa16
updating files for goimports
2017-11-01 12:51:43 -06:00
Justin Santa Barbara
b2bcba4a6d
GCE: Use object-level permissions for files in GCS
...
This lets us configure cross-project permissions while ourselves needing
minimal permissions, but also gives us a nice hook for future lockdown
of object-level permissions.
2017-10-29 19:17:00 -04:00
Justin Santa Barbara
106875115d
Support for deleting tokens & keypairs
...
This now allows for deleting all secrets, which means we can have a
procedure for rotating all keys.
2017-09-09 01:04:45 -04:00
Miao Luo
22c49e76cb
Fix user-defined s3 endpoint support.
...
Address review feedbacks and remove unintended space.
2017-04-19 23:46:05 -07:00
Miao Luo
76437a77d4
Support user-defined s3 endpoint.
2017-04-18 11:27:07 -07:00
Justin Santa Barbara
8104ba2cea
More logging around errors in s3 write path
...
Hopefully to shed some light on issues like #2108
2017-03-15 09:52:32 -04:00
Justin Santa Barbara
45a62a02aa
s3 path: apply suffix to ReadTree also
...
Otherwise we were matching directories with the same prefix.
2017-01-29 21:35:26 -05:00
Vincent Ambo
3b6e3bda56
chore: Run gofmt on new VFS code
2017-01-26 12:30:04 +01:00
Vincent Ambo
db0155b5b5
feat s3fs: Allow bucket object ACL override
...
When sharing S3 buckets across accounts it may be necessary to override ACLs
per object to avoid locking out different accounts.
This commit lets users specify a `KOPS_STATE_S3_ACL` environment variable which
(if specified) overrides the ACL in the PutObject request.
Fixes #907
2017-01-26 11:47:09 +01:00
Justin Santa Barbara
0a56d3d2e1
Raise an error on an invalid s3 path
...
Fix #902
2017-01-08 14:37:46 -05:00
chrislovecnm
8fa2aac99f
fixing more headers
2016-10-15 19:20:56 -06:00
Justin Santa Barbara
767c9a6416
Fix tests; don't hit s3 until we need to
2016-10-07 01:38:44 -04:00
Justin Santa Barbara
39c3c85262
Skip directory objects in S3 when listing files
...
Issue #520
Issue #548
2016-09-30 10:07:15 -04:00
Justin Santa Barbara
5f8d68ec85
Refactor shared packages into util directory
2016-09-25 18:27:09 -04:00