Commit Graph

101 Commits

Author SHA1 Message Date
Ciprian Hacman a3a0b91b5f Order policy document sections alphabetically 2020-11-04 16:15:00 +02:00
Ciprian Hacman b5e4b7ad48 Add missing instance monitoring for CF and TF 2020-11-04 16:15:00 +02:00
Ciprian Hacman 51e70083f8 Add missing resource names for CF 2020-11-04 16:15:00 +02:00
Kubernetes Prow Robot 578920e921
Merge pull request #10162 from rifelpet/nlb-sg
Fix additionalSecurityGroups support for NLB
2020-11-03 08:02:16 -08:00
Peter Rifel 860249f6b7
Fix additionalSecurityGroups support for NLB
We were correctly adding the security groups to the master ASGs but identified them incorrectly.
2020-11-03 08:22:24 -06:00
Peter Rifel f08284834e
Move NLB's VPC CIDR security group rule logic into model
This way the security group rule task doesn't need to be aware of VPCs, since we know the VPC CIDR ahead of time via cluster spec.

This also fixes the terraform and cloudformation rendering of this rule (see the added cidr block in the integration test outputs)

These rules are for NLB's health checks. The AWS docs recommend allowing access from the entire VPC CIDRs
Also add rules for additionalNetworkCIDRs, supporting VPCs with multiple CIDR blocks.
2020-11-03 08:13:32 -06:00
Peter Rifel 0cd3854605
Fix NLB naming for terraform and cloudformation targets 2020-11-02 19:38:01 -06:00
Christian Joun e91ed11449
Implement API load balancer class with NLB and ELB support on AWS (#9011)
* refactor TargetLoadBalancer to use DNSTarget interface instead of LoadBalancer

* add LoadBalancerClass fields into api

* make api machinery

* WIP: Implemented API loadbalancer class, allowing NLB and ELB support on AWS for new clusters.

* perform vendoring related tasks and apply fixes identified from hack/

dissallow spotinst + nlb
remove reflection in status_discovery.go
Add precreated additional security groups to the Master nodes in case of NLB
Remove support for attaching individual instances to NLB; only rely on ASG attachments
Don't specify Classic loadbalancer in GCE integration test

* add utility function to the kops model context to make LoadBalancer comparisons simpler

* use DNSTarget interface when locating DNSName of API ELB

* wip: create target group task

* Consolidate TargetGroup tasks

* Use context helper for determining api load balancer type to avoid nil pointers

* Update NLB creation to use target group ARN from separate task rather than creating a TG in-line

* Address staticcheck and bazel failures

* Removing NLB Attachment tasks because they're not used since we switched to defining them as a part of the ASGs

* Address PR review feedback

* Only set LB Class field for AWS clusters, fix nil pointer

* Move target group attributes from NLB task to TG task, removing unused attributes

* Add terraform and cloudformation support for NLBs, listeners, and target groups

* Update integration test for NLB support

* Fix NLB name format to pass terraform validation

* Preserve security group rule names when switching ELB to NLB to reduce destructive terraform changes

* Use elbv2 enums and address some TODOs

* Set healthcheck values in target group

* Find TG tags, fix NLB name detection

* Fix more spurious changes reported by lifecycle integration test

* Fix spotinst validation, more code cleanup

* Address more PR feedback

* ReconcileTargetGroups unit test + more code simplification

* Addressing PR feedback Renaming task 1. awstasks.LoadBalancer -> awstasks.ClassicLoadBalancer

* Addressing PR feedback Renaming task: ELBName() -> CLBName() / LinkToELB() -> LinkToCLB()

* Addressing PR feedback: Various text changes

* fix export of kubecfg

* address TargetGroup should have the same name as the NLB

* should address error when fetching tags due to missing ARN

* Update expected and crds

* Add feature table to NLB docs

* Address more feedback and remove some TODOs that arent applicable anymore

* Update spotinst validation error message

Co-authored-by: Peter Rifel <pgrifel@gmail.com>
2020-11-02 05:28:52 -08:00
Kubernetes Prow Robot b7f66a6d98
Merge pull request #10109 from bmelbourne/set-minimum-terraform-0.12
Set minimum Terraform version to 0.12.26/0.13.0
2020-10-29 01:52:58 -07:00
Kubernetes Prow Robot f466403912
Merge pull request #9794 from rdrgmnzs/lb-attachment
Prevent unintended resource updates to LB attatchments
2020-10-28 15:18:59 -07:00
Barry Melbourne 84417c330b Set minimum Terraform version to 0.12.26/0.13.0 2020-10-28 20:24:41 +00:00
Peter Rifel 6c4273d985
Fix cloudformation lint errors
```
E3002 Invalid Property Resources/AWSAutoScalingAutoScalingGroupnodesmixedinstancesexamplecom/Properties/MixedInstancesPolicy/InstancesDistribution/SpotInstancePool
//kops/tests/integration/update_cluster/mixed_instances_spot/cloudformation.json:321:13

E3031 CidrIp contains invalid characters (Pattern: x.x.x.x/y) at Resources/AWSEC2SecurityGroupIngresssshexternaltonode2001085a348/Properties/CidrIp
//kops/tests/integration/update_cluster/complex/cloudformation.json:833:9
```

I'm using the strings.Contains to determine if a CIDR is v6 since it seems simplest given that we know the value wont have a port number [0]

[0] https://stackoverflow.com/questions/22751035/golang-distinguish-ipv4-ipv6
2020-10-28 10:16:29 -05:00
Rodrigo Menezes 82d0ebdb56 Prevent unintended resource updates to LB attatchments 2020-10-26 17:29:07 -07:00
Kubernetes Prow Robot c9aa53895a
Merge pull request #10048 from hakman/container-runtime-assets
Install container runtime packages as assets
2020-10-25 21:03:01 -07:00
Kubernetes Prow Robot fbb172c08c
Merge pull request #9575 from johngmyers/node-labels
Take node labels from cloud tags on AWS
2020-10-23 04:01:45 -07:00
binkkatal 86b605d10b RUN: ./hack/update-expected.sh 2020-10-20 19:28:20 +05:30
Ciprian Hacman e44037f825 Install container runtime packages as assets - Integration 2020-10-14 15:41:51 +03:00
Peter Rifel 4bcfebebcc
Fix the detection and rendering of multiple additionalNetworkCIDR blocks 2020-09-27 20:12:09 -05:00
Peter Rifel 0978cd97e8
Add second additionalNetworkCIDR w/ incorrect test output 2020-09-27 20:08:09 -05:00
John Gardiner Myers 54c280eed5 update-expected.sh 2020-09-10 20:59:28 -07:00
Ciprian Hacman 43e0b2332c Update expected outputs of integration tests 2020-09-08 10:12:57 +03:00
Ciprian Hacman d1bdc1632d Update expected outputs of integration tests 2020-09-08 08:46:09 +03:00
Peter Rifel 0326ce4ad7
Dont generate the ssl_certificate_id field on TCP listeners in Terraform 2020-08-30 09:44:16 -05:00
Ciprian Hacman 2880e22bce Add flag for root volume encryption 2020-08-21 18:31:21 +03:00
Kubernetes Prow Robot 8a81d94c7b
Merge pull request #9773 from victorfrancax1/7286
Adding support for permission boundaries for AWS IAM Roles
2020-08-19 06:51:11 -07:00
Victor Ferreira 3aaa9a7c0f feat(aws): adding support to permission boundaries for IAM Roles 2020-08-19 01:16:13 -03:00
Ciprian Hacman 527b1eb36f ARM64 support - Update expected tests output 2020-08-10 12:05:24 +03:00
John Gardiner Myers cd2941b56c update-expected.sh 2020-07-29 17:24:39 -07:00
John Gardiner Myers be3e311c12 update-expected.sh 2020-07-17 19:32:53 -07:00
John Gardiner Myers c0774d7ffa Stop using legacy IAM in integration tests 2020-07-17 19:32:48 -07:00
Peter Rifel f213854330
Update integration test outputs with filebase64 2020-07-08 23:24:11 -05:00
Ciprian Hacman 827d8c041f Update mock version to 1.19.0-alpha.1 2020-07-08 18:31:18 +03:00
Peter Rifel 69f2c71cc3
Add tag support to AWS launch templates
In addition to TagSpecifications which allow tagging of instances and volumes, launch templates support tags of their own.

This adds the usual tags to LTs, as seen in the kubernetes.tf additions. Cloudformation does not yet support it [0], so only "api" and "terraform" targets are updated.

[0] https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-ec2-launchtemplate-launchtemplatedata.html
2020-07-07 22:43:45 -05:00
John Gardiner Myers 5f45389124 update-expected.sh 2020-06-28 18:52:03 -07:00
John Gardiner Myers 35645b49c4 update-expected.sh 2020-06-28 18:51:42 -07:00
John Gardiner Myers 64167b7420 update-expected.sh 2020-06-28 18:51:16 -07:00
Peter Rifel 75ccf45eb7
Fold multiple integration test cases into the complex cluster test
Each integration test cluster adds many LoC and some overhead in running the integration tests.
Since many of the tests are only testing a specific feature, it would be simpler to combine all of the non-mutually exclusive features into the complex cluster.
2020-06-19 22:09:22 -05:00
Ciprian Hacman 80295961fb ARM64 support - Update tests output 2020-06-19 04:42:11 +03:00
John Gardiner Myers 91c741d8fb update-expected.sh 2020-06-17 09:09:24 -07:00
Peter Rifel 964d402eb5
Store terraform launchtemplate userdata in plaintext rather than b64
This makes it easier to grok the userdata contents
2020-06-16 16:53:49 -05:00
Ciprian Hacman d888b1f517 Update integration tests 2020-06-12 06:24:08 +03:00
John Gardiner Myers 1b03e7237b hack/update-expected.sh 2020-06-10 09:34:48 -07:00
Ciprian Hacman 3eda07aec0 Update tests output 2020-06-03 17:25:53 +03:00
Ciprian Hacman c0b62a74ca Remove all traces of utils.tar.gz 2020-06-03 09:55:45 +03:00
Ciprian Hacman 78de18a803 Update k8s.gcr.io/pause image to version 3.2 2020-05-01 22:12:42 +03:00
Peter Rifel fd17a118f5
Remove irrelevant TODO comment from userdata
This comment is from 2016, isn't clear, and I believe may be no longer relevant.

This will cleanup the TODO comment from everyone's userdata.
2020-04-18 08:54:35 -05:00
Peter Rifel 460cdbfc67 Add CloudLabels tags to additional AWS resources
This adds any labels defined in the Cluster spec's CloudLabels to the tags of the following AWS resource types:

Elastic IP
Internet Gateway
NAT Gateway
Route Table
Security Group
Subnet
VPC DHCP Options
VPC
2020-04-11 13:25:52 -05:00
Peter Rifel fa2d64c3a3 Update expected outputs of terraform tests 2020-04-05 21:24:49 -05:00
Peter Rifel a0e1672cdc add terraform testing 2020-03-31 14:50:45 -05:00
Peter Rifel 833b22575f Disable anonymous auth in integration test clusters
This silences the anonymousAuth warning printed during `update cluster`, reducing the integration test output by >500 lines.
2020-03-29 22:21:10 -05:00