Commit Graph

584 Commits

Author SHA1 Message Date
Peter Rifel 5406744c55
Update integration test output 2020-12-23 15:13:45 -06:00
Ciprian Hacman ff6a782303 Add config options for container runtime package URL and Hash 2020-12-23 13:29:22 +02:00
Ciprian Hacman eff2af2fe2 Update CNI plugins to v0.8.7 2020-12-21 11:07:57 +02:00
Ciprian Hacman 472faf82d2 Drop support for containerd 1.2 2020-12-21 10:08:24 +02:00
Ole Markus With d89a7a55ce Add template function for upgrade version 2020-12-15 09:00:28 +01:00
Ole Markus With 2b0e84b432 Move mock channel to own package 2020-12-15 08:48:13 +01:00
Peter Rifel a5071e08cf
Don't allow ebs volume TF resource names to begin with digit 2020-12-14 23:15:36 -06:00
Bharath Vedartham cebe171805 Explicitly specify http_endpoint in launch_template terraform
http_endpoint has to be explicitly specified in the metadata_options block
of the launch template terraform according to issue
https://github.com/hashicorp/terraform-provider-aws/issues/12564
2020-12-10 01:37:15 +05:30
Kubernetes Prow Robot bee16c052d
Merge pull request #10324 from bharath-123/feature/aws-imdv2
Add support for AWS IMDS v2
2020-12-07 22:55:11 -08:00
Ciprian Hacman 174f405e39 Update expected outputs of integration tests 2020-12-08 07:08:32 +02:00
Ciprian Hacman 265bf4d106 Add option for setting the volume encryption key in AWS 2020-12-08 07:08:09 +02:00
Bharath Vedartham ee5d8a3435 update integration tests 2020-12-07 02:57:32 +05:30
Ciprian Hacman e11d934268 Add option to reuse existing Elastic IPs for NAT gateways 2020-12-06 09:37:17 +02:00
Ciprian Hacman 82eee758e0 Update expected outputs of integration tests 2020-12-05 05:16:55 +02:00
Kubernetes Prow Robot 0f9c0c03ef
Merge pull request #10365 from hakman/test-ha-shared-zone
Add integration test for creating an HA cluster in shared zone
2020-12-04 14:15:26 -08:00
Kubernetes Prow Robot ec691116a9
Merge pull request #10357 from rdrgmnzs/gzip-nodeup-heredocs
Give users the option to gzip and base64 encode the heredocs in the nodeup.sh user-data
2020-12-04 13:37:38 -08:00
Rodrigo Menezes 3fb12c66ae gzip and base64 encode the heredocs in the nodeup.sh portion of user-data 2020-12-04 10:46:18 -08:00
Ciprian Hacman afbb6475fe Add integration test for creating an HA cluster in shared zone 2020-12-04 20:16:38 +02:00
Kubernetes Prow Robot 1b45f876a4
Merge pull request #10335 from hakman/same-tg-multiple-igs
Allow attaching same external target group to multiple instance groups
2020-12-02 21:38:59 -08:00
Ciprian Hacman e57cd534b5 Allow attaching same external target group to multiple instance groups 2020-12-03 06:59:59 +02:00
Kubernetes Prow Robot c86e509dbc
Merge pull request #10341 from hakman/docker-19.03.13
Update containerd and Docker versions
2020-12-02 04:22:50 -08:00
Ciprian Hacman 5510d946e9 Update expected outputs of integration tests 2020-12-02 10:11:27 +02:00
MoShitrit c8b2d7b9cd update-expected.sh 2020-12-01 22:12:18 -05:00
Rodrigo Menezes c9af4de9cf Remove copywrite from nodeup scripts to reduce the user-data size 2020-11-30 12:49:25 -08:00
Kubernetes Prow Robot e789c24c3a
Merge pull request #10275 from rdrgmnzs/kubeapi-mem-cpu-request-limit
Allow setting CPU limit and Mem request / limit for kube API server
2020-11-23 11:23:02 -08:00
Rodrigo Menezes da773ba35c Allow setting CPU limit and Mem request / limit for kube API 2020-11-23 10:03:34 -08:00
Ciprian Hacman d5bee0b867 Update integration test for ExternalLoadBalancers 2020-11-21 21:45:57 +02:00
Frank Yang 93dcaddc48 feat(aws): add PolicyNames for ELB to change listener's security policy 2020-11-19 16:07:21 +08:00
Ciprian Hacman 4579a1bcdc Validate external IAM policies 2020-11-12 14:34:35 +02:00
Kubernetes Prow Robot 9b3f13d93f
Merge pull request #10151 from hakman/launch-template-versions
Use LaunchTemplate versions instead of timestamped LaunchTemplates
2020-11-10 23:23:48 -08:00
Kubernetes Prow Robot e43efbe102
Merge pull request #10157 from rifelpet/acm-nlb
Setup a second NLB listener when an AWS ACM certificate is used
2020-11-10 10:36:41 -08:00
Ciprian Hacman 0c3e3784c8 Use LaunchTemplate name instead of name_prefix for Terraform 2020-11-09 21:40:36 +02:00
Kubernetes Prow Robot 6a57543f6e
Merge pull request #10179 from olemarkus/sgr-consistent-naming
Consistent naming of security group rules
2020-11-07 02:07:37 -08:00
Ole Markus With fab694d290 Add ability to consistently name sgrs
In order to let kops fully control the rules for each security group we need to be able to generate names from the info in AWS. This is similar to the approach we used for openstack

Update pkg/model/firewall.go

Co-authored-by: Ciprian Hacman <ciprianhacman@gmail.com>
2020-11-07 10:27:19 +01:00
Peter Rifel 54decbc479
Always use TCP health check protocol for target groups 2020-11-06 11:09:38 -06:00
Peter Rifel 370092cb5a
Update TG ports rather than protocols when adding/removing ACM certs from listeners
This also renames the TGs to be more descriptive, with tcp and tls prefixes.
2020-11-06 11:09:38 -06:00
Peter Rifel 316c1eec8a
Update complex integration test for ACM cert and second listener 2020-11-06 11:09:37 -06:00
Ciprian Hacman a3a0b91b5f Order policy document sections alphabetically 2020-11-04 16:15:00 +02:00
Ciprian Hacman b5e4b7ad48 Add missing instance monitoring for CF and TF 2020-11-04 16:15:00 +02:00
Ciprian Hacman 51e70083f8 Add missing resource names for CF 2020-11-04 16:15:00 +02:00
Kubernetes Prow Robot 578920e921
Merge pull request #10162 from rifelpet/nlb-sg
Fix additionalSecurityGroups support for NLB
2020-11-03 08:02:16 -08:00
Peter Rifel 860249f6b7
Fix additionalSecurityGroups support for NLB
We were correctly adding the security groups to the master ASGs but identified them incorrectly.
2020-11-03 08:22:24 -06:00
Peter Rifel f08284834e
Move NLB's VPC CIDR security group rule logic into model
This way the security group rule task doesn't need to be aware of VPCs, since we know the VPC CIDR ahead of time via cluster spec.

This also fixes the terraform and cloudformation rendering of this rule (see the added cidr block in the integration test outputs)

These rules are for NLB's health checks. The AWS docs recommend allowing access from the entire VPC CIDRs
Also add rules for additionalNetworkCIDRs, supporting VPCs with multiple CIDR blocks.
2020-11-03 08:13:32 -06:00
Peter Rifel 0cd3854605
Fix NLB naming for terraform and cloudformation targets 2020-11-02 19:38:01 -06:00
Christian Joun e91ed11449
Implement API load balancer class with NLB and ELB support on AWS (#9011)
* refactor TargetLoadBalancer to use DNSTarget interface instead of LoadBalancer

* add LoadBalancerClass fields into api

* make api machinery

* WIP: Implemented API loadbalancer class, allowing NLB and ELB support on AWS for new clusters.

* perform vendoring related tasks and apply fixes identified from hack/

dissallow spotinst + nlb
remove reflection in status_discovery.go
Add precreated additional security groups to the Master nodes in case of NLB
Remove support for attaching individual instances to NLB; only rely on ASG attachments
Don't specify Classic loadbalancer in GCE integration test

* add utility function to the kops model context to make LoadBalancer comparisons simpler

* use DNSTarget interface when locating DNSName of API ELB

* wip: create target group task

* Consolidate TargetGroup tasks

* Use context helper for determining api load balancer type to avoid nil pointers

* Update NLB creation to use target group ARN from separate task rather than creating a TG in-line

* Address staticcheck and bazel failures

* Removing NLB Attachment tasks because they're not used since we switched to defining them as a part of the ASGs

* Address PR review feedback

* Only set LB Class field for AWS clusters, fix nil pointer

* Move target group attributes from NLB task to TG task, removing unused attributes

* Add terraform and cloudformation support for NLBs, listeners, and target groups

* Update integration test for NLB support

* Fix NLB name format to pass terraform validation

* Preserve security group rule names when switching ELB to NLB to reduce destructive terraform changes

* Use elbv2 enums and address some TODOs

* Set healthcheck values in target group

* Find TG tags, fix NLB name detection

* Fix more spurious changes reported by lifecycle integration test

* Fix spotinst validation, more code cleanup

* Address more PR feedback

* ReconcileTargetGroups unit test + more code simplification

* Addressing PR feedback Renaming task 1. awstasks.LoadBalancer -> awstasks.ClassicLoadBalancer

* Addressing PR feedback Renaming task: ELBName() -> CLBName() / LinkToELB() -> LinkToCLB()

* Addressing PR feedback: Various text changes

* fix export of kubecfg

* address TargetGroup should have the same name as the NLB

* should address error when fetching tags due to missing ARN

* Update expected and crds

* Add feature table to NLB docs

* Address more feedback and remove some TODOs that arent applicable anymore

* Update spotinst validation error message

Co-authored-by: Peter Rifel <pgrifel@gmail.com>
2020-11-02 05:28:52 -08:00
Kubernetes Prow Robot 999ae25932
Merge pull request #10143 from hakman/deprecate-calico-mv
Deprecate field calico.majorVersion
2020-10-30 11:55:47 -07:00
Kubernetes Prow Robot 43293d551f
Merge pull request #10140 from rifelpet/elbv2-cloudmock
Update AWS Cloudmock for complex and externallb integration test clusters
2020-10-30 10:02:07 -07:00
Peter Rifel 11bf3ff202
Add missing cloudmock functionality for enabling lifecycle integration test on complex cluster 2020-10-30 10:06:36 -05:00
Ciprian Hacman a999ac2d94 Deprecate field calico.majorVersion 2020-10-30 08:37:21 +02:00
John Gardiner Myers edfa6a0e94 Adjust Kubernetes version removals and deprecations for 1.20 2020-10-29 13:25:37 -07:00
Kubernetes Prow Robot b7f66a6d98
Merge pull request #10109 from bmelbourne/set-minimum-terraform-0.12
Set minimum Terraform version to 0.12.26/0.13.0
2020-10-29 01:52:58 -07:00
Kubernetes Prow Robot f466403912
Merge pull request #9794 from rdrgmnzs/lb-attachment
Prevent unintended resource updates to LB attatchments
2020-10-28 15:18:59 -07:00
Barry Melbourne 84417c330b Set minimum Terraform version to 0.12.26/0.13.0 2020-10-28 20:24:41 +00:00
Peter Rifel 6c4273d985
Fix cloudformation lint errors
```
E3002 Invalid Property Resources/AWSAutoScalingAutoScalingGroupnodesmixedinstancesexamplecom/Properties/MixedInstancesPolicy/InstancesDistribution/SpotInstancePool
//kops/tests/integration/update_cluster/mixed_instances_spot/cloudformation.json:321:13

E3031 CidrIp contains invalid characters (Pattern: x.x.x.x/y) at Resources/AWSEC2SecurityGroupIngresssshexternaltonode2001085a348/Properties/CidrIp
//kops/tests/integration/update_cluster/complex/cloudformation.json:833:9
```

I'm using the strings.Contains to determine if a CIDR is v6 since it seems simplest given that we know the value wont have a port number [0]

[0] https://stackoverflow.com/questions/22751035/golang-distinguish-ipv4-ipv6
2020-10-28 10:16:29 -05:00
Rodrigo Menezes dbbd0dd802 Move external LB and target group to inline as well. 2020-10-26 17:30:06 -07:00
Rodrigo Menezes 82d0ebdb56 Prevent unintended resource updates to LB attatchments 2020-10-26 17:29:07 -07:00
Kubernetes Prow Robot c9aa53895a
Merge pull request #10048 from hakman/container-runtime-assets
Install container runtime packages as assets
2020-10-25 21:03:01 -07:00
Kubernetes Prow Robot fbb172c08c
Merge pull request #9575 from johngmyers/node-labels
Take node labels from cloud tags on AWS
2020-10-23 04:01:45 -07:00
binkkatal 86b605d10b RUN: ./hack/update-expected.sh 2020-10-20 19:28:20 +05:30
Nicolas Vanheuverzwijn 5f59b86c78
Update tests/integration/channel/integration_test.go
Co-authored-by: Ciprian Hacman <ciprianhacman@gmail.com>
2020-10-14 13:23:04 -04:00
Nicolas Vanheuverzwijn b0fd89a193 upgrade-cluster: test that new image in stable or alpha channel will receive automated update 2020-10-14 10:14:47 -04:00
Ciprian Hacman e44037f825 Install container runtime packages as assets - Integration 2020-10-14 15:41:51 +03:00
Ciprian Hacman dc86205faf Update expected outputs of integration tests 2020-10-09 17:12:37 +03:00
Ciprian Hacman 9c888bcdcf Update expected outputs of integration tests 2020-10-09 09:33:39 +03:00
Peter Rifel 4bcfebebcc
Fix the detection and rendering of multiple additionalNetworkCIDR blocks 2020-09-27 20:12:09 -05:00
Peter Rifel 0978cd97e8
Add second additionalNetworkCIDR w/ incorrect test output 2020-09-27 20:08:09 -05:00
Justin SB 2a44cb7f16 Update script and testdata CA keypair
If we generate with the wrong type (usages), the keypair will be
regenerated.
2020-09-12 16:21:15 -04:00
John Gardiner Myers 54c280eed5 update-expected.sh 2020-09-10 20:59:28 -07:00
Justin SB 6fa8be2716 JSON formatting of IAM: Workaround for optional fields
AWS IAM is very strict and doesn't support `Resource: []` for example.
We implement a custom MarshalJSON method to work around that.
2020-09-09 09:57:07 -04:00
Justin SB a61ecf4c58 Refactor to use interface for iam Subjects
Hat-tip to johngmyers for the idea!
2020-09-09 09:57:07 -04:00
Justin SB ccc814dfbc Create tests for JWKS scenarios 2020-09-09 09:57:06 -04:00
Kubernetes Prow Robot 4604fa53b3
Merge pull request #9899 from olemarkus/remove-insecure-bind-address
Don't explicitly set insecure-bind-address on newer k8s
2020-09-09 03:25:53 -07:00
Ole Markus With 886b4c97cb Don't explicitly set insecure-bind-address on newer k8s 2020-09-09 11:41:51 +02:00
Ciprian Hacman 43e0b2332c Update expected outputs of integration tests 2020-09-08 10:12:57 +03:00
Ciprian Hacman d1bdc1632d Update expected outputs of integration tests 2020-09-08 08:46:09 +03:00
Kubernetes Prow Robot 277038d419
Merge pull request #9839 from rifelpet/tf-cert-id
Dont generate the ssl_certificate_id field on TCP listeners in Terraform
2020-08-31 06:14:21 -07:00
Kubernetes Prow Robot 56bab9fa4f
Merge pull request #9813 from justinsb/expose_jwks
Expose JWKS via a feature-flag
2020-08-30 21:06:20 -07:00
Peter Rifel 0326ce4ad7
Dont generate the ssl_certificate_id field on TCP listeners in Terraform 2020-08-30 09:44:16 -05:00
Justin SB 786423f617 Expose JWKS via a feature-flag
When the PublicJWKS feature-flag is set, we expose the apiserver JWKS
document publicly (including enabling anonymous access).  This is a
stepping stone to a more hardened configuration where we copy the JWKS
document to S3/GCS/etc.

Co-authored-by: John Gardiner Myers <jgmyers@proofpoint.com>
2020-08-30 10:15:11 -04:00
Justin SB c63ce4b5ab Implement setter by reflection
This means we no longer have to individually hard-code the `kops set`
fields, however we use the "language" we're now demonstrated.

We add tests to ensure we have parity with our existing (hard-coded)
setter logic.
2020-08-30 09:59:52 -04:00
Peter Rifel 55f33c68b3
Cleanup old v1alpha1 test outputs 2020-08-27 14:30:10 -05:00
Peter Rifel 64f6f5e2cb
Add integration test for GCE private topology with bastion 2020-08-27 14:28:26 -05:00
Ciprian Hacman cd82550088 Update integration test for Calico after validation changes 2020-08-24 12:54:15 +03:00
Ciprian Hacman 3f8edd74fe Update integration test for Calico 2020-08-24 12:30:37 +03:00
Ciprian Hacman 2880e22bce Add flag for root volume encryption 2020-08-21 18:31:21 +03:00
Kubernetes Prow Robot 8a81d94c7b
Merge pull request #9773 from victorfrancax1/7286
Adding support for permission boundaries for AWS IAM Roles
2020-08-19 06:51:11 -07:00
Victor Ferreira 3aaa9a7c0f feat(aws): adding support to permission boundaries for IAM Roles 2020-08-19 01:16:13 -03:00
Kubernetes Prow Robot 96ab8423b1
Merge pull request #9566 from hakman/arm64-images
Add ARM64 support for masters
2020-08-14 20:46:17 -07:00
Ole Markus With 9890839cec Add an integration test for openstack floating ip
* Integration test for floatingip cluster
* Implements mocking of floatingIP (only list for now)
* Expands various cloudmocks
* Fixes an NPR in openstack validation
* Fixes a bug where kops tries to use DNS even if the cluster is gossip
2020-08-12 12:59:30 +02:00
Kubernetes Prow Robot 9b46e3fb9a
Merge pull request #9721 from hakman/ghw-win-2019
Add windows job to GH workflows
2020-08-11 11:20:18 -07:00
Ciprian Hacman f9f8ecfcf5 Fix API conversion tests for Windows 2020-08-11 14:28:37 +03:00
Ciprian Hacman 2359a25b84 Fix integration tests EOLs for Windows 2020-08-11 14:28:37 +03:00
Peter Rifel 6991655921
Add openstack integration test.
This will create / update / update / delete an openstack cluster using cloudmock, ensuring there are no lingering changes reported or orphaned resources
2020-08-10 15:22:49 -05:00
Ciprian Hacman 527b1eb36f ARM64 support - Update expected tests output 2020-08-10 12:05:24 +03:00
Kubernetes Prow Robot 2fbef78143
Merge pull request #9650 from johngmyers/update-notag
Remove tags from NodeupConfig
2020-07-31 02:27:08 -07:00
John Gardiner Myers d434733254 update-expected.sh 2020-07-30 19:42:07 -07:00
John Gardiner Myers cd2941b56c update-expected.sh 2020-07-29 17:24:39 -07:00
Peter Rifel f3e6ac874d
Update integration test outputs with new ssh key tags 2020-07-28 13:35:10 -05:00
John Gardiner Myers be3e311c12 update-expected.sh 2020-07-17 19:32:53 -07:00
John Gardiner Myers c0774d7ffa Stop using legacy IAM in integration tests 2020-07-17 19:32:48 -07:00