Commit Graph

254 Commits

Author SHA1 Message Date
Ole Markus With 37271998e1 Use sets for aws lbc permissions 2021-07-01 10:19:40 +02:00
Ole Markus With c7bd1c1529 Add s3 policies to integration tests 2021-07-01 09:26:58 +02:00
Ole Markus With 19833e6b73 Use sets for ebscsidriver permissions 2021-07-01 09:02:04 +02:00
John Gardiner Myers 0f1de5cfc8 hack/update-expected.sh 2021-06-30 18:55:35 -07:00
John Gardiner Myers e90f2cc834 hack/update-expected.sh 2021-06-28 13:48:35 -07:00
Kubernetes Prow Robot ee048e89e7
Merge pull request #11872 from johngmyers/refactor-serviceaccount
Refactor nodeup APIServer builder, part one
2021-06-28 10:42:01 -07:00
John Gardiner Myers 7dfe9d82ab hack/update-expected.sh 2021-06-27 08:45:06 -07:00
John Gardiner Myers fdf034058d hack/update-expected.sh 2021-06-27 08:45:05 -07:00
Ciprian Hacman 348eed772a Avoid spurious changes for ASG InstanceProtection and LT InstanceMonitoring 2021-06-27 10:08:13 +03:00
John Gardiner Myers 89209df150 hack/update-expected.sh 2021-06-25 22:25:50 -07:00
Ciprian Hacman d7f405f65a Decrease default values for net.ipv4.tcp_rmem and net.ipv4.tcp_wmem 2021-06-25 21:27:56 +03:00
John Gardiner Myers 7dea5af9be hack/update-expected.sh 2021-06-21 19:37:24 -07:00
John Gardiner Myers 48c42fe37f hack/update-expected.sh 2021-06-21 16:10:07 -07:00
Ole Markus With 79a2c111f2 Remove redundant permissions 2021-06-21 08:59:54 +02:00
Ole Markus With b3f274e140 Apply permissions to master role when irsa is not used 2021-06-21 08:56:11 +02:00
John Gardiner Myers 0700ef64a0 hack/update-expected.sh 2021-06-19 10:56:24 -07:00
Ole Markus With b37bc7578e Reduce master policy size for lb controller 2021-06-19 10:12:22 +02:00
Kubernetes Prow Robot 135cdf3461
Merge pull request #11789 from johngmyers/seed-rng
Seed the random number generator on AWS
2021-06-18 08:48:06 -07:00
Ole Markus With 33a7de60a7 Enable IRSA for EBS CSI Driver 2021-06-18 08:05:59 +02:00
John Gardiner Myers b1e77af664 hack/update-expected.sh 2021-06-17 23:03:52 -07:00
Ole Markus With 6e8e027aff Enable IRSA for Cluster Autoscaler 2021-06-16 18:03:11 +02:00
Ole Markus With e7fa3fa82c Set containerd config on nodeup.Config instead of clusterspec
This allows us to set a default containerd config per IG (e.g add a different config for GPU IGs)

Can also be considered a cleanup as we no longer use containerd.overrideConfig as a mechanism for bringing the default containerd config from cloudup to nodeup.
2021-06-15 11:08:22 +02:00
John Gardiner Myers 74a44c2270 Don't restrict nodeup download to IPv4 2021-06-13 21:46:58 -07:00
Kubernetes Prow Robot cfc93e5178
Merge pull request #9294 from johngmyers/refactor-nodeup-context
Remove InstanceGroup from NodeupModelContext
2021-06-12 13:43:01 -07:00
Matthew Wong b6266ce5f0 Run hack/update-expected.sh 2021-06-09 13:53:07 -07:00
John Gardiner Myers 1db6e318a1 hack/update-expected.sh 2021-06-03 21:30:06 -07:00
John Gardiner Myers 91d81e5a1a hack/update-expected.sh 2021-06-03 21:26:51 -07:00
John Gardiner Myers 4bf9150ab6 hack/update-expected.sh 2021-06-03 21:20:43 -07:00
John Gardiner Myers 1d44ee3116 hack/update-expected.sh 2021-06-03 20:41:05 -07:00
Ciprian Hacman 62f54d1401 Run hack/update-expected.sh 2021-06-03 11:16:08 +03:00
Ciprian Hacman 3b80de3bcc Convert all indents to spaces in node bootstrap script 2021-05-27 11:21:52 +03:00
Ole Markus With 46e13c0009 Bump snapshot-controller version
Update upup/models/cloudup/resources/addons/storage-aws.addons.k8s.io/v1.15.0.yaml.template

Co-authored-by: Peter Rifel <rifelpet@users.noreply.github.com>

Update upup/models/cloudup/resources/addons/storage-aws.addons.k8s.io/v1.15.0.yaml.template

Co-authored-by: Peter Rifel <rifelpet@users.noreply.github.com>
2021-05-21 15:40:40 +02:00
Alexander Block 5306e27646 Run ./hack/update-expected.sh 2021-05-20 08:11:35 +02:00
Ciprian Hacman c0e71d802d Run hack/update-expected.sh 2021-05-19 20:31:13 +03:00
John Gardiner Myers 65711d05c0 hack/update-expected.sh 2021-05-19 08:02:10 -07:00
John Gardiner Myers fbd7663606 hack/update-expected.sh 2021-05-18 21:49:39 -07:00
Ciprian Hacman 57feaa65c3 Run hack/update-expected.sh 2021-05-19 06:21:07 +03:00
Ole Markus With d3581ebb84 bump aws lb controller to 2.2.0 2021-05-16 18:26:23 +02:00
Ciprian Hacman a39d829f1f Set canonical location for downloads to artifacts.k8s.io
And remove the legacy location for downloads.
2021-05-14 00:41:56 +03:00
John Gardiner Myers 5c4f1c4f6c Adjust sorting of resources in hcl2 2021-05-02 19:39:23 -07:00
Ole Markus With 1ec0bd18e8 Enable support for the ASG WarmPool lifecycle hook
Update pkg/model/iam/iam_builder.go

Co-authored-by: Ciprian Hacman <ciprianhacman@gmail.com>
2021-04-24 09:40:52 +02:00
Kubernetes Prow Robot c771b7622e
Merge pull request #11216 from olemarkus/warm-nodeup
Don't start kubelet if instance is entering the warm pool
2021-04-15 00:07:49 -07:00
Ciprian Hacman 27e102bd04 Add support for Docker v20.10.6 2021-04-15 07:01:36 +03:00
Ole Markus With af92896dc7 Don't start kubelet if we are warming 2021-04-14 11:05:50 +02:00
Barry Melbourne 6575b6113d Update integration tests to k8s v1.21.0 2021-04-11 17:07:17 +01:00
Peter Rifel 80ceb4200d
Update test outputs 2021-03-26 11:24:11 -07:00
Barry Melbourne 05123faf5a Update containerd to v1.3.10/v1.4.4 2021-03-23 17:02:01 +00:00
Bharath Vedartham 1d721c3ff8 Update integration tests 2021-03-06 00:33:20 +05:30
Ole Markus With 063e3f6c7b Use internal api url for jwks when required
The public api url cannot be used by pods and nodes if access is restricted. So by default we need to use the internal one.
This should finally pass the OIDC e2e test

For public access, api server must be publically available and anonymous
auth must be enabled
2021-03-05 06:52:51 +01:00
Ole Markus With 32fce0d59c Exclude CP nodes from load balancers 2021-02-27 20:14:31 +01:00
Ole Markus With 9a13837629 Fix JWKS path for volume projection 2021-02-18 22:07:35 +01:00
Alexander Block 16f3de29fb Run ./hack/update-expected.sh 2021-02-16 18:46:00 +01:00
Ciprian Hacman 1c4da19881 Update integration test outputs with new mock version 2021-02-16 14:21:15 +02:00
Ole Markus With 783b6c0d6c Make protokube CP label setting consistent with kops-controller 2021-02-12 08:17:14 +01:00
Steven E. Harris d44612cc84 Capture outcome of "hack/update-expected.sh" run 2021-02-11 10:49:49 -05:00
Ciprian Hacman f8d3b76556 Default IMDSv2 to "optional" for AWS 2021-01-29 14:02:14 +02:00
Kubernetes Prow Robot 3d39be7721
Merge pull request #10661 from hakman/etcd-manager-defaults
Update AWS etcd-manager volumes defaults
2021-01-28 22:01:41 -08:00
Ciprian Hacman 5fcd4e4b28 Allow attaching same external load balancer to multiple instance groups 2021-01-27 16:25:39 +02:00
Ciprian Hacman 4c5d7ddabf Remove workaround for volume throughput when using Terraform 2021-01-27 06:33:15 +02:00
Ciprian Hacman ca408f7e8f Set default volume type to "gp3" for etcd-manager volumes in AWS 2021-01-27 06:23:27 +02:00
Justin SB 1d76a15f69 Set the tcp_rmem sysctl in bootstrap script
This ensures that we're using our settings for downloading nodeup
itself and any assets that nodeup downloads.  This is a workaround for
reported problems with the initial download on some kernels otherwise.

Issue #10206
2021-01-24 21:50:45 -05:00
Ciprian Hacman d889d61ddb Set default IMDS v2 to "required" for instances in AWS 2021-01-21 11:35:41 +02:00
Ciprian Hacman c8a9b2fb3e Set default volume encryption to "true" for instances in AWS 2021-01-21 11:27:02 +02:00
Ciprian Hacman 18bb14ffed Set default volume type to "gp3" for instances in AWS 2021-01-21 11:27:02 +02:00
Barry Melbourne 337c9c4c66 Set default container runtime to containerd 2021-01-16 14:55:35 +00:00
Ciprian Hacman b0cb0c77d4 Update integration tests for "update cluster" 2021-01-15 15:51:02 +02:00
Ole Markus With afbd057286 Use consistent naming for the remaining SGRs 2021-01-14 12:57:33 +01:00
Ole Markus With 2b3a8f133e Add control-plane node role annotation to cp nodes
Update docs/releases/1.20-NOTES.md

Co-authored-by: John Gardiner Myers <jgmyers@proofpoint.com>
2021-01-08 12:39:42 +01:00
Peter Rifel 5406744c55
Update integration test output 2020-12-23 15:13:45 -06:00
Bharath Vedartham cebe171805 Explicitly specify http_endpoint in launch_template terraform
http_endpoint has to be explicitly specified in the metadata_options block
of the launch template terraform according to issue
https://github.com/hashicorp/terraform-provider-aws/issues/12564
2020-12-10 01:37:15 +05:30
Kubernetes Prow Robot bee16c052d
Merge pull request #10324 from bharath-123/feature/aws-imdv2
Add support for AWS IMDS v2
2020-12-07 22:55:11 -08:00
Ciprian Hacman 174f405e39 Update expected outputs of integration tests 2020-12-08 07:08:32 +02:00
Bharath Vedartham ee5d8a3435 update integration tests 2020-12-07 02:57:32 +05:30
Ciprian Hacman e57cd534b5 Allow attaching same external target group to multiple instance groups 2020-12-03 06:59:59 +02:00
Rodrigo Menezes c9af4de9cf Remove copywrite from nodeup scripts to reduce the user-data size 2020-11-30 12:49:25 -08:00
Ciprian Hacman d5bee0b867 Update integration test for ExternalLoadBalancers 2020-11-21 21:45:57 +02:00
Ciprian Hacman 0c3e3784c8 Use LaunchTemplate name instead of name_prefix for Terraform 2020-11-09 21:40:36 +02:00
Kubernetes Prow Robot 6a57543f6e
Merge pull request #10179 from olemarkus/sgr-consistent-naming
Consistent naming of security group rules
2020-11-07 02:07:37 -08:00
Ole Markus With fab694d290 Add ability to consistently name sgrs
In order to let kops fully control the rules for each security group we need to be able to generate names from the info in AWS. This is similar to the approach we used for openstack

Update pkg/model/firewall.go

Co-authored-by: Ciprian Hacman <ciprianhacman@gmail.com>
2020-11-07 10:27:19 +01:00
Ciprian Hacman a3a0b91b5f Order policy document sections alphabetically 2020-11-04 16:15:00 +02:00
Ciprian Hacman 51e70083f8 Add missing resource names for CF 2020-11-04 16:15:00 +02:00
Peter Rifel 11bf3ff202
Add missing cloudmock functionality for enabling lifecycle integration test on complex cluster 2020-10-30 10:06:36 -05:00
Kubernetes Prow Robot b7f66a6d98
Merge pull request #10109 from bmelbourne/set-minimum-terraform-0.12
Set minimum Terraform version to 0.12.26/0.13.0
2020-10-29 01:52:58 -07:00
Barry Melbourne 84417c330b Set minimum Terraform version to 0.12.26/0.13.0 2020-10-28 20:24:41 +00:00
Rodrigo Menezes dbbd0dd802 Move external LB and target group to inline as well. 2020-10-26 17:30:06 -07:00
Kubernetes Prow Robot c9aa53895a
Merge pull request #10048 from hakman/container-runtime-assets
Install container runtime packages as assets
2020-10-25 21:03:01 -07:00
Kubernetes Prow Robot fbb172c08c
Merge pull request #9575 from johngmyers/node-labels
Take node labels from cloud tags on AWS
2020-10-23 04:01:45 -07:00
binkkatal 86b605d10b RUN: ./hack/update-expected.sh 2020-10-20 19:28:20 +05:30
Ciprian Hacman e44037f825 Install container runtime packages as assets - Integration 2020-10-14 15:41:51 +03:00
John Gardiner Myers 54c280eed5 update-expected.sh 2020-09-10 20:59:28 -07:00
Ciprian Hacman 43e0b2332c Update expected outputs of integration tests 2020-09-08 10:12:57 +03:00
Ciprian Hacman d1bdc1632d Update expected outputs of integration tests 2020-09-08 08:46:09 +03:00
Ciprian Hacman 527b1eb36f ARM64 support - Update expected tests output 2020-08-10 12:05:24 +03:00
John Gardiner Myers cd2941b56c update-expected.sh 2020-07-29 17:24:39 -07:00
Peter Rifel f3e6ac874d
Update integration test outputs with new ssh key tags 2020-07-28 13:35:10 -05:00
John Gardiner Myers be3e311c12 update-expected.sh 2020-07-17 19:32:53 -07:00
John Gardiner Myers c0774d7ffa Stop using legacy IAM in integration tests 2020-07-17 19:32:48 -07:00
Peter Rifel f213854330
Update integration test outputs with filebase64 2020-07-08 23:24:11 -05:00
Ciprian Hacman 827d8c041f Update mock version to 1.19.0-alpha.1 2020-07-08 18:31:18 +03:00
Peter Rifel 69f2c71cc3
Add tag support to AWS launch templates
In addition to TagSpecifications which allow tagging of instances and volumes, launch templates support tags of their own.

This adds the usual tags to LTs, as seen in the kubernetes.tf additions. Cloudformation does not yet support it [0], so only "api" and "terraform" targets are updated.

[0] https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-ec2-launchtemplate-launchtemplatedata.html
2020-07-07 22:43:45 -05:00
John Gardiner Myers 5f45389124 update-expected.sh 2020-06-28 18:52:03 -07:00
John Gardiner Myers 35645b49c4 update-expected.sh 2020-06-28 18:51:42 -07:00
John Gardiner Myers 64167b7420 update-expected.sh 2020-06-28 18:51:16 -07:00
Ciprian Hacman 80295961fb ARM64 support - Update tests output 2020-06-19 04:42:11 +03:00
John Gardiner Myers 91c741d8fb update-expected.sh 2020-06-17 09:09:24 -07:00
Peter Rifel 964d402eb5
Store terraform launchtemplate userdata in plaintext rather than b64
This makes it easier to grok the userdata contents
2020-06-16 16:53:49 -05:00
Ciprian Hacman d888b1f517 Update integration tests 2020-06-12 06:24:08 +03:00
John Gardiner Myers 1b03e7237b hack/update-expected.sh 2020-06-10 09:34:48 -07:00
Ciprian Hacman 3eda07aec0 Update tests output 2020-06-03 17:25:53 +03:00
Ciprian Hacman c0b62a74ca Remove all traces of utils.tar.gz 2020-06-03 09:55:45 +03:00
Ciprian Hacman 78de18a803 Update k8s.gcr.io/pause image to version 3.2 2020-05-01 22:12:42 +03:00
Peter Rifel fd17a118f5
Remove irrelevant TODO comment from userdata
This comment is from 2016, isn't clear, and I believe may be no longer relevant.

This will cleanup the TODO comment from everyone's userdata.
2020-04-18 08:54:35 -05:00
Peter Rifel fa2d64c3a3 Update expected outputs of terraform tests 2020-04-05 21:24:49 -05:00
Peter Rifel a0e1672cdc add terraform testing 2020-03-31 14:50:45 -05:00
Peter Rifel 833b22575f Disable anonymous auth in integration test clusters
This silences the anonymousAuth warning printed during `update cluster`, reducing the integration test output by >500 lines.
2020-03-29 22:21:10 -05:00
Ciprian Hacman 661bd6dfbd Add "owned" tag for all instance groups related resources 2020-03-02 10:39:50 +02:00
Justin SB 5d7fc87102
Update expected output to include v in github URLs 2020-02-05 20:58:03 -05:00
Peter Rifel e0cebf30e5 Update terraform resource names to be 0.12 compatible.
According to the upgrade guide [0] resource names cannot start with digits.
Currently both routes and VPC CIDR associations start with digits, so this adds prefixes to them so that they are valid resource identifiers in 0.12.

This is a significant change because on its own, terraform will destroy and recreate the route which impact the cluster networking.
To avoid this, existing clusters this will require moving the resources within the terraform state prior to the next `apply`.

```
kops update cluster --target terraform --out ./
terraform state mv aws_route.0-0-0-0--0 aws_route.route-0-0-0-0--0 # repeat for all aws_route resources
terraform plan
terraform apply
```

The exact terraform state command may vary depending on how Kops' terraform output is used.
See the command documentation [1] for more details.
Always run a terraform plan first to ensure the `aws_route` and `aws_vpc_ipv4_cidr_block_association` resources are not getting recreated.

Due to the potential impact, this notice should be very prominant in the Kops release notes

[0] https://www.terraform.io/upgrade-guides/0-12.html
[1] https://www.terraform.io/docs/commands/state/mv.html
2020-01-17 07:51:39 -06:00
Kubernetes Prow Robot 63930904c3
Merge pull request #8269 from justinsb/ops_kops_writeable
Make /opt/kops writeable on COS
2020-01-06 01:33:34 -08:00
Kubernetes Prow Robot e4da659509
Merge pull request #8196 from justinsb/dont_preload_unused_images
For dev, don't preload docker images on nodes
2020-01-04 16:15:40 -08:00
Justin Santa Barbara 6c41748640
Download nodeup into a writeable path on COS
/opt/ isn't writeable on COS, so we download to /var/lib/toolbox/kops
instead.
2020-01-04 18:16:08 -05:00
Kubernetes Prow Robot 78267482bf
Merge pull request #8212 from hakman/move-nodeup
Move nodeup to /opt/kops/bin
2020-01-04 14:49:40 -08:00
Justin SB 976724dfec
Update tests for leaner configuration 2020-01-04 14:25:21 -05:00
Peter Rifel 21c5a82ed3 Update mock kops version 2020-01-03 08:17:02 -06:00
Ciprian Hacman 474cb76827 Update tests 2019-12-28 22:17:16 +02:00
Ciprian Hacman 760c7b06b4 Use containerd 1.2.4 with Docker 18.09.3 2019-12-23 19:26:38 +02:00
Ciprian Hacman f1ffd9a7f6 Rename config option contained.configFile to containerd.configOverride 2019-12-16 00:59:26 +02:00
Ciprian Hacman 1a4d8bf49c Add support for containerd container runtime - tests 2019-12-15 21:37:57 +02:00
Kubernetes Prow Robot 1bb829bb4a
Merge pull request #7378 from justinsb/add_artifacts_k8s_io
Add artifacts.k8s.io to mirror list
2019-10-12 11:18:37 -07:00
Justin SB 1a7bb9d21e
Update expected test output 2019-10-05 16:40:31 -04:00
Justin SB 4e1e596fc9
Add artifacts.k8s.io to mirror list
The official mirror is coming!
2019-09-30 12:03:53 -04:00
Justin SB 1580d4c82b
Update tests to use k8s >= 1.14
Where tests are using a version of k8s that is now really deprecated,
replace it with k8s 1.14.0.
2019-09-14 13:58:50 -04:00
Justin SB 21cb8ece06
Label AWS ASGs with kops.k8s.io/instancegroup
We will use this to map an AWS instance to the instance group.
2019-09-02 09:26:23 -07:00
Justin SB 4733317a64
Update expected test for new script 2019-08-25 21:15:14 -04:00
Kubernetes Prow Robot 675eeaa07b
Merge pull request #7447 from justinsb/use_mirrors
nodeup download should try all mirrors
2019-08-22 07:10:43 -07:00
Justin SB 05bbf9cbcc
bash script: don't assume nodeup filename
We were previously assuming that the name of the URL matched the name
we wanted to download to, but also assuming that the name of the file
was nodeup.  Change to download to a known file location, whatever the
mirror url.
2019-08-22 01:13:47 -04:00
Justin SB 2462ab1c02
nodeup download should try all mirrors
On an error, we were bailing out early!
2019-08-22 01:05:16 -04:00
Justin SB 330ccfcd55
Support mirroring for nodeup also
We weren't using mirrors for nodeup (just protokube), though the
existing bash script supports mirrors.
2019-08-04 01:34:39 -04:00
Justin SB ba9a4afc43
apigroup -> kops.k8s.io: Tests & supporting changes
Autogenerated code, docs & test changes

Also a new test for the older "kops" apiGroup, to make sure that
continues to work.
2019-05-06 22:26:29 -04:00
Ryan Bonham 54ef99ef54 Update Tests 2019-04-30 09:15:08 -05:00
Justin SB 4f469fab9a
Support mirrors with restricted characters
Github doesn't allow us to have slashes in our release artifact names;
we therefore support a configurable per-mirror set of substitutions.
We use that to map `/` to `-` for github.
2019-03-26 01:02:31 -04:00
Justin SB 7bd4a7e262
Support download protokube from mirror 2019-03-26 00:45:54 -04:00
Justin SB ec2151f229
Update tests for mirrors 2019-02-20 14:32:00 -05:00
Ariel Mashraki d63c47149d integration: Fix broken test in externallb
Running `terraform validate` on `externallb/kubernetes.tf` was failing
because resource name can't contain ":".
2018-10-20 18:27:41 +03:00
Justin Santa Barbara 237043dded Update expected test output for script changes 2018-09-21 14:51:45 -04:00
k8s-ci-robot b1c446f8f3
Merge pull request #5503 from mikesplain/fix_suspendprocess
Fix suspendprocess
2018-09-03 16:28:31 -07:00
Justin Santa Barbara 16985c3abc Remove _kubernetes_master tag
We can get the master role just as readily from the InstanceGroup spec
2018-08-14 21:01:07 -04:00
Justin Santa Barbara 1c3949bdfe Don't assume that we only have one subnet per AZ
I made a mistaken assumption in
dde2100a19 that we only had one subnet
per AZ, but as demonstrated in #5587 this was not the case.

What I was trying to achieve was not to include the cluster name, so
for the case of subnets this commit just uses the subnet name from the
cluster spec, which should be unique and stable.  That is hopefully at
least as meaningful.

Thankfully we hadn't released a version with the erroneous naming.

Fix #5587
2018-08-13 13:15:03 -04:00
Justin Santa Barbara a7b22b4876 Remove GetAsgForInstance IAM permission
It isn't a valid IAM permission - it was introduced in error, but IAM
is kind enough to ignore it.

Fixes #5549
2018-08-02 11:27:29 -04:00
Mike Splain a93ca798ec Fix tests 2018-07-23 20:55:44 -04:00