Commit Graph

321 Commits

Author SHA1 Message Date
justinsb 689f66a390 Recognize debian bullseye as having "broken" resolv.conf
Debian bullseye also has the loopback resolv.conf that needs special
handling.
2021-12-11 10:15:46 -05:00
Ciprian Hacman e19a1bbad9 Remove support for RHEL/CentOS 7 2021-12-03 21:40:10 +02:00
Ciprian Hacman 2030055c54 Remove support for Ubuntu 18.04 2021-12-03 21:28:12 +02:00
Ciprian Hacman 45094241f6 Remove support for Ubuntu 16.04 2021-12-03 21:28:12 +02:00
Ciprian Hacman 2055eec0f9 Remove support for Debian 9 2021-12-03 21:12:28 +02:00
Kubernetes Prow Robot ca966310e8
Merge pull request #12121 from rifelpet/gcs-tf
Support GSFS Terraform Managed Files
2021-12-03 06:32:27 -08:00
John Gardiner Myers c070c4def5 Remove support for CentOS 8 2021-12-02 23:09:08 -08:00
Peter Rifel f5f52eec92
Add mock GCP credentials to fix unit tests 2021-12-02 08:01:35 -06:00
Peter Rifel f56f98154a
Add TerraformProvider 2021-12-02 08:01:35 -06:00
Peter Rifel 675754edeb
Add Terraform support for GS Paths 2021-12-02 08:01:34 -06:00
Peter Rifel 5813b7f0e0
Add a unit test for S3Path.RenderTerraform
Unfortunately it needs to be in a different package to avoid a rather complex import loop: terraform -> kops api -> vfs -> terraform
2021-12-02 07:55:45 -06:00
Ciprian Hacman ea7df00719 Run hack/update-gofmt.sh 2021-12-01 22:39:50 +02:00
Ole Markus With 2fa53989c4 Configure dualstack endpoint for s3
Use dualstack https endpoints on ipv6only cluster. Always use
dualstack endpoints through the SDK
2021-11-20 08:00:00 +01:00
John Gardiner Myers 18d58f900c Support setting empty maps and structs 2021-11-13 19:10:29 -08:00
Kubernetes Prow Robot 49162378aa
Merge pull request #12722 from Juneezee/deprecate-ioutil
refactor: move from io/ioutil to io and os packages
2021-11-12 20:50:53 -08:00
Ciprian Hacman 471bcefb89 Recognize Ubuntu 22.04 (Jammy Jellyfish) 2021-11-13 05:23:09 +02:00
Eng Zer Jun 425173ae9f
refactor: move from io/ioutil to io and os packages
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2021-11-12 15:37:18 +08:00
Peter Rifel 7eaa647b2a
Include the provider alias on TerraformPath resources that reference their provider 2021-09-17 18:08:59 -05:00
Peter Rifel c3a7f9f75a
Add TerraformProvider definition to TerraformPath interface 2021-09-17 18:08:58 -05:00
Kubernetes Prow Robot 1b431b4c9c
Merge pull request #11628 from olemarkus/gpu-runtime
Pre-install nvidia container runtime + drivers on GPU instances
2021-09-11 13:00:07 -07:00
Ciprian Hacman a7bb55b72f Recognize Ubuntu 21.10 (Impish Indri) 2021-09-11 08:56:33 +03:00
Ole Markus With 4ab75b01cb Have instances learn about their GPU capabilities 2021-09-05 20:09:04 +02:00
justinsb c2914ccba7 Warn if KOPS_ARCH is not a recognized value
Otherwise we were silently rejecting it.
2021-08-23 08:41:07 -04:00
Yadnesh Kulkarni 670fa3aaf7 Incorrect url to the swift container
Swift path prefix in ReadTree and ReadDir should not be empty
when adding "/" to it.
2021-08-15 04:21:34 -04:00
Ole Markus With 67b4024694 Reconcile if managedFile is public or not 2021-08-13 20:20:44 +02:00
Reilly Brogan 850bca8db6 Support Debian 11 Bullseye 2021-08-06 12:52:16 -05:00
Ole Markus With 4bf0fae33a Add function for getting memfs location 2021-07-01 08:27:48 +02:00
John Gardiner Myers eb076e3713 Render managed files with Terraform 2021-06-28 12:15:15 -07:00
John Gardiner Myers ead0941ae8 Make SpecOverride append to slices 2021-06-14 14:01:22 -07:00
John Gardiner Myers 76fc012f96 Allow unsetting fields from the command line 2021-06-14 08:52:32 -07:00
Kubernetes Prow Robot 3c4b6068b9
Merge pull request #11649 from h3poteto/fix-jwks-location
Fix jwks object path in S3 for IRSA
2021-06-01 08:26:27 -07:00
AkiraFukushima d52ec60c02
Fix issuer and jwks object path for IRSA 2021-06-01 23:35:21 +09:00
John Gardiner Myers e896a8a215 Fix detection of virtual-hosted-style S3 urls in us-east-1 2021-05-31 19:07:56 -07:00
Kubernetes Prow Robot 3a376e9048
Merge pull request #11387 from johngmyers/aws-config
Enable reading shared config when possibly from CLI
2021-05-23 15:15:38 -07:00
Ciprian Hacman a39d829f1f Set canonical location for downloads to artifacts.k8s.io
And remove the legacy location for downloads.
2021-05-14 00:41:56 +03:00
John Gardiner Myers 23de00da6e Enable reading shared config when possibly from CLI 2021-05-05 22:08:54 -07:00
Ciprian Hacman e0eab51c5c Recognize Ubuntu 21.04 2021-04-27 12:54:42 +03:00
Peter Rifel 7c900b7fae Generate and upload keys.json + discovery.json to public store
Generate and upload keys.json + discovery.json to public store

Don't enable anonymous auth on publicjwks

Remove tests that won't work using FS VFS anymore
2021-03-19 20:03:26 +01:00
Kubernetes Prow Robot 2083133cfc
Merge pull request #11047 from bmelbourne/update-misc-go-mods
Update Go modules to latest versions
2021-03-15 00:11:04 -07:00
Peter Rifel 21389c8276
Cleanup some nodeup & protokube logging
Also log a channels error that we're seeing on flatcar to help with troubleshooting
2021-03-14 21:49:35 -05:00
Barry Melbourne e30bf1cf35 Update Go modules to latest versions 2021-03-14 15:08:27 +00:00
Bharath Vedartham e5aa8177b6 Add protokube and channels urls 2021-03-06 00:31:18 +05:30
Kubernetes Prow Robot 016b0e5500
Merge pull request #10732 from zetaab/feature/uagent
add user agent to openstack api requests
2021-02-06 23:53:12 -08:00
Jesse Haka 034dad258c modify names 2021-02-05 09:57:54 +02:00
Jesse Haka 41d04d8d4b add user agent to openstack api requests 2021-02-04 23:04:06 +02:00
Bharath Vedartham 49f3ab0703 Throw error if path being set by kops set is not present in struct 2021-01-31 12:00:42 +05:30
Kubernetes Prow Robot 95e9cbf8ab
Merge pull request #10566 from rifelpet/fs-err
Fix file not found error detection in fs://
2021-01-12 22:42:35 -08:00
Peter Rifel 0df5f6c24d
Fix file not found error detection in fs:// 2021-01-12 20:57:33 -06:00
Justin Santa Barbara 78b139465c Refactor and centralize distribution logic
Use of a struct makes it more sustainable, centralizing into the
distribution package makes it simpler to follow.
2021-01-05 11:50:23 -05:00
Justin SB b17e44b709 Recognize ubuntu 20.10
Teach nodeup about ubuntu 20.10, including the unusual
/etc/resolv.conf configuration.
2021-01-05 10:53:40 -05:00
Kenji Kaneda a61caea8d2 Add Azure support
This commit contains all changes required to support Azure
(https://github.com/kubernetes/kops/issues/3957).
2020-12-21 08:27:54 -08:00
Justin SB 7d9ff3ba96 Refactor MirroredAsset into mirrors package
This means we can use MirroredAsset for nodeup without circular
dependencies.  Also removes a duplicate constant that was declared
twice.
2020-12-19 18:39:09 -05:00
Kenji Kaneda 40c944aa5c Fix a typo in an error message returned from buildAzureBlobPath
invalid Azure Blob schem -> invalid Azure Blob scheme
2020-11-23 08:16:39 -08:00
Kenji Kaneda 4555c0b2df Add support of Azure Blob storage to VFS
The schema is "azureblob".

azureClient provides two ways to set up credential. One approach is to
use an account key stored in env var. This approach is used when
accessing Blob from kops CLI. The second approach is to retrieve
credentials from Instance Metadata Service. This works only when
azureClient is created on a VM that has sufficient privilege to access
a specified blob. This approach is used from nodeup, etcd-manager,
etc.
2020-11-19 10:47:03 -08:00
Jesse Haka 67d69f16a9 allow reauth for openstack client 2020-10-30 08:57:49 +02:00
Ole Markus With f6ce70e5c3 Minor fixes to swiftfs.go 2020-10-08 20:32:29 +02:00
Ciprian Hacman 0c6f1c733c Use all kops mirrors to determine artifacts hashes 2020-09-18 09:44:37 +03:00
Kubernetes Prow Robot a5fc8895dc
Merge pull request #9857 from hakman/detect-aws-region
Detect AWS region for S3 inside containers
2020-09-09 23:17:44 -07:00
John Gardiner Myers 1e92c7740c Map ENOENT to ErrNotExist in FSPath 2020-09-05 21:46:57 -07:00
Ciprian Hacman 32e6da7576 Detect AWS region for S3 inside containers 2020-09-02 06:41:12 +03:00
Kubernetes Prow Robot fc66e0161e
Merge pull request #9836 from justinsb/openstack_reauth
Always use OpenStack Swift reauthentication
2020-08-31 00:40:20 -07:00
Justin SB c63ce4b5ab Implement setter by reflection
This means we no longer have to individually hard-code the `kops set`
fields, however we use the "language" we're now demonstrated.

We add tests to ensure we have parity with our existing (hard-coded)
setter logic.
2020-08-30 09:59:52 -04:00
Justin SB d4480e4721 Always use OpenStack Swift reauthentication
If we were using credentials from env vars, we would not do
reauthentication with Swift.
2020-08-29 08:25:59 -04:00
Ciprian Hacman 0da3980865 Use /etc/os-release to identify the distribution 2020-08-17 07:25:44 +03:00
Ciprian Hacman e29b84da01 Add tests for distributions 2020-08-17 07:25:43 +03:00
Ciprian Hacman 22ec1512dc Use numbers for distribution names 2020-08-17 07:25:43 +03:00
Ciprian Hacman 3825f657cc Use const for architectures and distributions 2020-08-17 07:25:43 +03:00
Ciprian Hacman 795373a499 Remove unused function 2020-08-17 07:25:43 +03:00
Ciprian Hacman 3f43d047dd Remove confusing comment 2020-08-17 07:25:43 +03:00
Ciprian Hacman e68ee80a93 Move and rename the "distros" package 2020-08-17 07:25:43 +03:00
Peter Rifel 4d9f0128a3
Upgrade to klog2
This splits up the kubernetes 1.19 PR to make it easier to keep up to date until we get it sorted out.
2020-08-16 20:56:48 -05:00
Ole Markus With a708a96c05 Adds support for using OS application credentials
Application credentials allows you to export a purpose-specific set of
credentials for a user instead of exposing user login credentials.
Especially useful when using LDAP or similar for Openstack users.
Also lets you rotate credentials more easily since multiple application
credentials can be provisioned per user.

Update pkg/model/bootstrapscript.go

Co-authored-by: Ciprian Hacman <ciprianhacman@gmail.com>
2020-08-07 14:26:47 +02:00
Ole Markus With d1479fb666 Add support for reading openstack metadata in vfs 2020-08-04 08:22:00 +02:00
John Gardiner Myers fed5587d77 Improve locking in memfs 2020-07-19 16:01:47 -07:00
Kubernetes Prow Robot aa7b67124f
Merge pull request #9535 from hakman/env-arch-amd64
Force single arch support via env var
2020-07-09 09:00:31 -07:00
Ciprian Hacman 3c84d83d37 Address review comments 2020-07-09 17:07:37 +03:00
Zhou Hao d6695b822f Add err judgment to os.RemoveAll
Signed-off-by: Zhou Hao <zhouhao@cn.fujitsu.com>
2020-07-09 16:48:35 +08:00
Zhou Hao 34931ed930 Add err judgment to ioutil.TempDir
Signed-off-by: Zhou Hao <zhouhao@cn.fujitsu.com>
2020-07-09 16:45:12 +08:00
Ciprian Hacman 9b77f372f1 Fix typo in function name 2020-07-09 10:45:50 +03:00
Ciprian Hacman 1f296e58f6 Force AMD64 only support via env var 2020-07-09 09:31:54 +03:00
Li Zhijian c3fc293ede cleanup tempfiles for fs_test
Signed-off-by: Li Zhijian <lizhijian@cn.fujitsu.com>
2020-07-02 10:09:34 +08:00
Xiaoyu Zhong 98c35cd220 Rename accessKeyId to accessKeyID 2020-06-22 09:32:20 +08:00
Kubernetes Prow Robot 8b371acef0
Merge pull request #9094 from olemarkus/vault-vfs
Implement VFS for vault
2020-06-20 12:02:39 -07:00
Ciprian Hacman a49879c4bc ARM64 support - Add architecture 2020-06-19 04:42:11 +03:00
Ole Markus With acaa1e1dfc Implement VFS for vault 2020-06-18 13:02:37 +02:00
ZouYu 2fc52ec6be fix some go-lint warning
Signed-off-by: ZouYu <zouy.fnst@cn.fujitsu.com>
2020-06-09 08:52:50 +08:00
Ciprian Hacman 654a0d2d8a Detect supported architecture during node setup 2020-06-03 17:23:59 +03:00
Kubernetes Prow Robot 5e27f74dd8
Merge pull request #9228 from justinsb/paginate_delete_all_versions
S3 DeleteAllVersions: use pagination
2020-05-31 15:59:54 -07:00
Justin SB 319ddcc333 S3 DeleteAllVersions: use pagination
This way we're not limited to one page of versions.  This is likely a
purely theoretical concern, at least as we're using it today.
2020-05-31 18:21:05 -04:00
Justin SB 7d7b8969ea Use AWS SDK to fetch metadata
Previously the EC2 metadata service was straightforward HTTP, but IMDS
v2 now requires managing a session token (and is more secure for it).

We now use the AWS SDK when retrieving metadata; it automatically
supports IMDS v2.
2020-05-31 17:23:01 -04:00
Ciprian Hacman d54aadc89c Fix nits for removal of S3 file versions 2020-05-28 06:50:32 +03:00
Kubernetes Prow Robot 92f8e22002
Merge pull request #9174 from johngmyers/remove-vfsscan
Remove unused VFSScan
2020-05-27 09:24:48 -07:00
Ciprian Hacman 9675692b84 Implement RemoveAll() for S3 paths 2020-05-25 07:46:32 +03:00
John Gardiner Myers 62ebbc5a5d Remove unused VFSScan 2020-05-24 21:27:28 -07:00
Ciprian Hacman b565122875 Remove delete markers also from S3 bucket 2020-05-24 17:42:02 +03:00
Ciprian Hacman a48ccfa06c Return warning instead of error to hide issues during cluster teardown 2020-05-24 15:20:20 +03:00
Ciprian Hacman 1a38a3feaa Return os.ErrNotExist when no versions are found 2020-05-24 11:42:18 +03:00
Ciprian Hacman 56af880c53 Remove TODO that was not addressed for a long time 2020-05-24 10:11:56 +03:00
Ciprian Hacman 831e3f0e57 Remove all versions of a file form the S3 bucket 2020-05-24 08:38:46 +03:00
Justin SB 5ed11fd9c7 GCE: don't rely on hostname being correct
Distros that use systemd for DHCP often don't have the hostname
correct, due to e.g. the requirement for policy kit.

We don't rely on it being set correctly on other clouds; no real
reason to require it on GCP either!
2020-05-17 15:20:58 -04:00