Commit Graph

154 Commits

Author SHA1 Message Date
John Gardiner Myers a1db8f1e82 Refactor LaunchTemplate.InstanceInterruptionBehavior 2021-05-10 23:11:17 -07:00
John Gardiner Myers d0793bd6ed Refactor LaunchTemplate.ImageID 2021-05-10 23:08:21 -07:00
John Gardiner Myers bfd8034cce Refactor LaunchTemplate.IAMInstanceProfile 2021-05-10 23:08:21 -07:00
John Gardiner Myers 07aa346e68 Refactor LaunchTemplate.HTTPTokens 2021-05-10 23:08:20 -07:00
John Gardiner Myers 98502cd0b2 Refactor LaunchTemplate.HTTPPutResponseHopLimit 2021-05-10 23:08:16 -07:00
John Gardiner Myers 33590eb617 Refactor LaunchTemplate.CPUCredits 2021-05-10 23:07:24 -07:00
John Gardiner Myers 0557414111 Refactor LaunchTemplate.BlockDeviceMappings 2021-05-10 22:51:00 -07:00
John Gardiner Myers 4657cb94d6 Refactor LaunchTemplate.AssociatePublicIP 2021-05-10 22:47:48 -07:00
Ole Markus With f16cafb8ef Make hook task name unique while the hook name is consistent
Since tasks need to be unique, but we need to reuse the hookname across all ASGs, we distinguish between task and actual name of the hook
2021-04-27 20:57:19 +02:00
John Gardiner Myers 428041bc0f Add cluster-level warmPool settings 2021-04-25 20:22:04 -07:00
John Gardiner Myers 5ad32230bb Fix typo 2021-04-25 13:42:12 -07:00
John Gardiner Myers 044b5f6d0d Allow disabling warm pool by setting WarmPool.MaxSize to 0 2021-04-24 16:35:46 -07:00
Ole Markus With 1ec0bd18e8 Enable support for the ASG WarmPool lifecycle hook
Update pkg/model/iam/iam_builder.go

Co-authored-by: Ciprian Hacman <ciprianhacman@gmail.com>
2021-04-24 09:40:52 +02:00
Ole Markus With 020652e096 Add ability to enable/configure warm pool for ASG
Apply suggestions from code review

Co-authored-by: John Gardiner Myers <jgmyers@proofpoint.com>

Apply suggestions from code review

Co-authored-by: John Gardiner Myers <jgmyers@proofpoint.com>
2021-04-20 09:02:09 +02:00
Ole Markus With 20bd724f5e Add support for scaling out the control plane with dedicated apiserver nodes
Ensure apiserver role can only be used on AWS (because of firewalling)

Apply api-server label to CP as well

Consolidate node not ready validation message

Guard apiserver nodes with a feature flag

Rename Apiserver role to APIServer

Add an integration test for apiserver nodes

Rename Apiserver role to APIServer

Enumerate all roles in rolling update docs

Apply suggestions from code review

Co-authored-by: Steven E. Harris <seh@panix.com>
2021-03-20 20:57:00 +01:00
Bharath Vedartham 0c0767c0c9 Remove support for launch configurations 2021-03-09 09:04:15 +02:00
Ole Markus With 896f1740c6 Rename spotinst symbols and merge spotinstmodel with awsmodel 2021-03-07 22:06:12 +01:00
Peter Rifel ce51ec44bc
Use new CPUCredits IG spec field in launch templates 2021-03-02 22:54:29 -06:00
liranp 2abdb90c54
fix: don't skip lb attachments when hybrid is enabled 2021-03-01 14:07:22 +02:00
Peter Rifel e7ede2b13e
Use EnsureTask instead of prepending IG names to external ELB tasks
This way we end up with one CLB task per CLB regardless of how many ASGs to which it is attached.
2021-02-07 10:45:38 -06:00
Ciprian Hacman f8d3b76556 Default IMDSv2 to "optional" for AWS 2021-01-29 14:02:14 +02:00
Ciprian Hacman 5fcd4e4b28 Allow attaching same external load balancer to multiple instance groups 2021-01-27 16:25:39 +02:00
Ciprian Hacman d889d61ddb Set default IMDS v2 to "required" for instances in AWS 2021-01-21 11:35:41 +02:00
Ciprian Hacman c8a9b2fb3e Set default volume encryption to "true" for instances in AWS 2021-01-21 11:27:02 +02:00
Ciprian Hacman 18bb14ffed Set default volume type to "gp3" for instances in AWS 2021-01-21 11:27:02 +02:00
Ciprian Hacman 85fbf1c6a2 Add iops field for gp3 volumes only with launch templates 2021-01-21 11:27:02 +02:00
Ciprian Hacman a7bb949936 Add possibility to set volume throughput for gp3 volumes 2021-01-05 13:18:32 +02:00
Steven E. Harris 252d4177f0 Only include API server SGs in IGs for masters
When using an AWS NLB in front of the Kubernetes API servers, we can't
attach the EC2 security groups nominated in the Cluster
"spec.api.loadBalancer.additionalSecurityGroups" field directly to the
load balancer, as NLBs don't have associated security groups. Instead,
we intend to attach those nominated security groups to the machines
that will receive network traffic forwarded from the NLB's
listeners. For the API servers, since that program runs only on the
master or control plane machines, we need only attach those security
groups to the machines that will host the "kube-apiserver" program, by
way of the ASG launch templates that come from kOps InstanceGroups of
role "master."

We were mistakenly including these security groups in launch templates
derived from InstanceGroups of all of our three current roles:
"bastion," "master," and "node." Instead, skip InstanceGroups of the
"bastion" and "node" roles and only target those of role "master."
2021-01-04 08:38:25 -05:00
Kubernetes Prow Robot bee16c052d
Merge pull request #10324 from bharath-123/feature/aws-imdv2
Add support for AWS IMDS v2
2020-12-07 22:55:11 -08:00
Ciprian Hacman 265bf4d106 Add option for setting the volume encryption key in AWS 2020-12-08 07:08:09 +02:00
Bharath Vedartham 7f6e125733 Add support for aws ec2 instance metadata v2
A new field is add to the InstanceGroup spec with 2 sub fields,
HTTPPutResponseHopLimit and HTTPTokens. These fields enable the user
to disable IMDv1 for instances within an instance group.

By default, both IMDv1 and IMDv2 are enabled in instances in an instance group.
2020-12-07 02:57:02 +05:30
Ciprian Hacman e57cd534b5 Allow attaching same external target group to multiple instance groups 2020-12-03 06:59:59 +02:00
Ciprian Hacman 19345c3f7f Order attached TargetGrups list by name 2020-11-20 10:40:27 +02:00
Ciprian Hacman fdcc2607bf Parse TargetGrup names from ARNs 2020-11-20 10:40:26 +02:00
Kubernetes Prow Robot e43efbe102
Merge pull request #10157 from rifelpet/acm-nlb
Setup a second NLB listener when an AWS ACM certificate is used
2020-11-10 10:36:41 -08:00
Ciprian Hacman 0934374fe2 Fix various NLB nits 2020-11-10 17:30:23 +02:00
Ciprian Hacman 32658075d3 Fix disabling spot instances when using launch templates 2020-11-08 19:11:45 +02:00
Peter Rifel 370092cb5a
Update TG ports rather than protocols when adding/removing ACM certs from listeners
This also renames the TGs to be more descriptive, with tcp and tls prefixes.
2020-11-06 11:09:38 -06:00
Peter Rifel 9242c34a38
Setup a second NLB listener on 8443 when sslCertificate is set 2020-11-06 11:09:37 -06:00
Peter Rifel 860249f6b7
Fix additionalSecurityGroups support for NLB
We were correctly adding the security groups to the master ASGs but identified them incorrectly.
2020-11-03 08:22:24 -06:00
Christian Joun e91ed11449
Implement API load balancer class with NLB and ELB support on AWS (#9011)
* refactor TargetLoadBalancer to use DNSTarget interface instead of LoadBalancer

* add LoadBalancerClass fields into api

* make api machinery

* WIP: Implemented API loadbalancer class, allowing NLB and ELB support on AWS for new clusters.

* perform vendoring related tasks and apply fixes identified from hack/

dissallow spotinst + nlb
remove reflection in status_discovery.go
Add precreated additional security groups to the Master nodes in case of NLB
Remove support for attaching individual instances to NLB; only rely on ASG attachments
Don't specify Classic loadbalancer in GCE integration test

* add utility function to the kops model context to make LoadBalancer comparisons simpler

* use DNSTarget interface when locating DNSName of API ELB

* wip: create target group task

* Consolidate TargetGroup tasks

* Use context helper for determining api load balancer type to avoid nil pointers

* Update NLB creation to use target group ARN from separate task rather than creating a TG in-line

* Address staticcheck and bazel failures

* Removing NLB Attachment tasks because they're not used since we switched to defining them as a part of the ASGs

* Address PR review feedback

* Only set LB Class field for AWS clusters, fix nil pointer

* Move target group attributes from NLB task to TG task, removing unused attributes

* Add terraform and cloudformation support for NLBs, listeners, and target groups

* Update integration test for NLB support

* Fix NLB name format to pass terraform validation

* Preserve security group rule names when switching ELB to NLB to reduce destructive terraform changes

* Use elbv2 enums and address some TODOs

* Set healthcheck values in target group

* Find TG tags, fix NLB name detection

* Fix more spurious changes reported by lifecycle integration test

* Fix spotinst validation, more code cleanup

* Address more PR feedback

* ReconcileTargetGroups unit test + more code simplification

* Addressing PR feedback Renaming task 1. awstasks.LoadBalancer -> awstasks.ClassicLoadBalancer

* Addressing PR feedback Renaming task: ELBName() -> CLBName() / LinkToELB() -> LinkToCLB()

* Addressing PR feedback: Various text changes

* fix export of kubecfg

* address TargetGroup should have the same name as the NLB

* should address error when fetching tags due to missing ARN

* Update expected and crds

* Add feature table to NLB docs

* Address more feedback and remove some TODOs that arent applicable anymore

* Update spotinst validation error message

Co-authored-by: Peter Rifel <pgrifel@gmail.com>
2020-11-02 05:28:52 -08:00
Peter Rifel 6318e90128
Ignore changes reported by subsequent updates
Usually this is an "actual.Foo = e.Foo" one-liner but we don't know which LB attached to an ASG is the API ELB so it's a bit more complicated
2020-10-29 12:34:20 -05:00
Peter Rifel 7497edaf7c
Lookup LoadBalancerName when only the LB task name is known 2020-10-29 12:13:23 -05:00
Rodrigo Menezes 41adf07e15 cleanup code 2020-10-28 11:11:58 -07:00
Rodrigo Menezes 9bd0a7aedb Add instruction for no downtime 2020-10-26 18:11:46 -07:00
Rodrigo Menezes dbbd0dd802 Move external LB and target group to inline as well. 2020-10-26 17:30:06 -07:00
Rodrigo Menezes 82d0ebdb56 Prevent unintended resource updates to LB attatchments 2020-10-26 17:29:07 -07:00
binkkatal e32717f31d FIX: Change int fields to string
The ./hack/update-expected.sh script generates some fields which are
required to be string fields and hence results in linting errors.

This PR changes those fields to string/*string and removes lint
warnings.
2020-10-20 19:28:20 +05:30
Peter Rifel 7d9f0a06cf
Update API slice fields to not use pointers
This is causing problems with the Kubernetes 1.19 code-generator.
A nil entry in these slices wouldn't be valid anyways, so this should have no impact.
2020-08-24 07:46:38 -05:00
Ciprian Hacman 2880e22bce Add flag for root volume encryption 2020-08-21 18:31:21 +03:00
Peter Rifel 4d9f0128a3
Upgrade to klog2
This splits up the kubernetes 1.19 PR to make it easier to keep up to date until we get it sorted out.
2020-08-16 20:56:48 -05:00
John Gardiner Myers 013f9bf914 Create bootstrap script in a Task 2020-06-26 19:11:40 -07:00
John Gardiner Myers cef5b175c7 Rename BootstrapScript to BootstrapScriptBuilder 2020-06-26 10:57:36 -07:00
liranp 04d83c6c04
fix(spotinst): rename the label to match the feature flag 2020-05-23 21:39:07 +03:00
liranp 23c0cdab36
feat(spotinst): new hybrid mode 2020-05-23 21:39:04 +03:00
Martin Tomes c66180bc58 Added support for instance interruption behavior 2020-04-29 14:53:17 +02:00
Thejas B dda8dc3f37 Add support for Spot block in launch template
- Launch configuration does not support the field SpotDurationInMinutes which is used to reserve the spot instances, but however Launch Template does
2020-03-31 20:07:01 +05:30
John Gardiner Myers 4b33efedaa Tag EBS volumes when using launch templates with AWS API target 2020-02-01 22:32:37 -08:00
Austin Moore 4a88f7b5a5
Add ability to specify no ssh key by setting sshKeyName to empty
Add tests for no ssh key functionality

Add docs for setting no ssh key

Disable sshKey rendering for cloudformation if nosshkey is set

Fix broken test

make goimports

Fix

Formatting fix

Update kubernetes version for tests

Update expected test output

Fix imports in mesh.pb.go

Run hack/update-expected.sh

Change digital ocean logic to handle *string for SSHKeyName

Fix expected output

Missed a few
2020-01-15 15:24:32 -05:00
Gabriel Tiossi 3b902262f7 feat(awsmodel): Logic for EBS DeleteOnTermination
Implement separated logics for root volume and additional volumes
2019-12-03 04:28:35 -03:00
tanjunchen 43bf6bdab6 fix-up gosimple check error 2019-10-07 11:59:57 +08:00
chentanjun 3d1966df84 fix-up some spelling mistakes in /pkg 2019-09-26 10:15:44 +08:00
mikesplain 9e55b8230a Update copyright notices
Also cleans some white spaces
2019-09-09 14:47:51 -04:00
mikesplain adaf3ad5fd First pass at instance protection 2019-06-21 14:00:37 -04:00
Peter Rifel adef332fe3 Add integration test for mixed instances / launch templates 2019-06-07 14:29:34 -07:00
Peter Rifel dc3c88af9c Add support for SpotPrice and Mixed Instance ASGs 2019-06-07 14:12:17 -07:00
Rohith b37b5cba0c - fixing up the checking of the mixed instance policy attribute as it was a interface to a thing 2019-02-26 10:17:10 +00:00
Rohith 74af751dc6 - fixing up the conflicts post the additional volumes merge 2019-02-04 14:15:16 +00:00
Rohith 098b4486f5 - fixing up the spelling mistake and the package update 2019-02-04 11:59:43 +00:00
Rohith 5f082e2158 - fixing up the logic to determine the public ip based on subnets and choice 2019-02-04 11:59:43 +00:00
Rohith 761d760c11 - updating the aws model to make use of the aws template and mixed instance policies 2019-02-04 11:59:42 +00:00
Rohith 0bd9126387 - fixing up the iops issue, masking when not required i.e. when the volume type is not io1 for now 2019-01-22 14:29:02 +00:00
Rohith 0e155b4c78 - changed tack and making them two separate features for now, one adding additional volumes and two mounting them. This should always allow for user to use epherimal devices as well
- updated the api specs and machinery
- adding the dependecies on the services when the volume mounts are enable (should probably false this if they don't effect the docker filesystem)
2019-01-18 22:49:38 +00:00
Rohith 08fb11c8cf - fixing up the iops to only be includes on io1 volume types 2019-01-18 22:45:05 +00:00
Rohith 5e262932ae - fixing up various linting issues and formatting 2019-01-18 22:45:05 +00:00
Rohith 5bfed5c63f - adding the additional volumes to the launchconfiguration 2019-01-18 22:45:05 +00:00
Justin SB 26bd75aecb
Bulk spelling fixes
Experimenting with my own spelling checker, these are the typos it caught.
2018-12-20 17:43:56 -05:00
Rodrigo Menezes a7903adfe8 Fix for when node and master use the same SG. 2018-12-06 01:05:54 -08:00
Liran Polak 46b1c70b96 fix: ignore additional instance types 2018-10-14 11:37:31 +03:00
Rodrigo Menezes a82f548ff8 Allow using existing/shared Security Groups
Verbosely log when a user overwrites LB or IG security groups

Change SecurityGroup to SecurityGroupOverride

Allow using existing/shared Security Groups

Update tests
2018-10-02 00:51:39 -07:00
Mike Splain 84d63cbe60 Fix suspend proccesst to also resume
Also fixed internal consistency error by switching from
[]*string to *[]string.
2018-07-23 20:44:37 -04:00
Gijs Kunze b605a27cb5 Added external load balancer support for terraform 2018-06-29 11:39:21 +02:00
Gijs Kunze b3346c28cd Added the ability to specify external loadbalancers in instancegroups 2018-06-29 11:39:20 +02:00
Peter Rifel 5f0b63100d Add support for using existing instance profiles 2018-06-08 10:33:09 -07:00
Justin Santa Barbara e158f84e9f Set AWS_REGION into bootstrapscript
Fix #4451
2018-04-12 17:39:24 -04:00
Sergey Lanzman dc4a1a35b6 Validation around IOPS fields 2018-04-02 08:40:05 +03:00
devops d734f35126 Add support for instance monitoring
Add API backward comptability

Add api generated files

documenting detailed-monitoring

instance-group json typo

Update test expected result
to support enable_monitroing

Add instance-monitroing support to CF

Improve doc, test and rename InstanceMonitoring

Rename instnace-monitoring to fit the YAML kops form

typo detailedInstanceMonitoring
2018-03-21 23:06:38 +02:00
Mike Splain fdc2695fe1 Add suspendProcesses to IGs 2018-02-20 09:30:18 -05:00
Justin Santa Barbara 6e562e0742 Support updating autoscaling metrics 2018-02-20 00:53:24 -05:00
k8s-ci-robot f9748b7ebf
Merge pull request #4270 from int128/root-volume-type
Fix rootVolumeType accepts all volume types
2018-01-26 11:36:35 -08:00
Johannes Würbach 02855fc13f
Enable metrics for AutoScalingGroups
Those metrics simplify monitoring the created ASGs and are free [0].

Enable them by default.

[0] https://aws.amazon.com/about-aws/whats-new/2016/08/free-auto-scaling-group-metrics-with-graphs/
2018-01-25 20:06:10 +01:00
Hidetake Iwata d97e0e6205 Fix rootVolumeType accepts all volume types #4256 2018-01-14 17:57:35 +09:00
chrislovecnm 88baba38de fixing security lifecycle for additional security groups 2017-12-30 09:00:11 -07:00
Justin Santa Barbara 66b174321f Cleanup signature of default volume-size method
Because the default doesn't depend on the user-specified value, it's
misleading to pass it in.
2017-09-30 21:24:51 -04:00
chrislovecnm c4c63b2b0c using same disk sizes for gce 2017-09-29 16:07:38 -06:00
Kashif Saadat e0461b92a9 Add ability to store partial cluster and instancegroup spec in userdata,
so component config changes are detected and causes nodes to be updated
2017-08-09 14:15:02 +01:00
Derek VerLee ffa95b8112 Add support for cluster using http forward proxy 2017-08-07 14:30:42 -04:00
Ionut Craciunescu 07397ec2f0 ran make gofmt 2017-08-01 00:34:39 +01:00
Ionut Craciunescu 847bf0c2e1 updates to support aws disks with provisioned iops 2017-08-01 00:28:21 +01:00
Justin Santa Barbara 3dfe48e5ae Wiring up lifecycle 2017-07-15 22:03:54 -04:00