Commit Graph

149 Commits

Author SHA1 Message Date
Jason Haugen d07b067249 Add NTH queue-processor mode 2021-04-19 15:43:05 -05:00
Ole Markus With 09615935fd Make kOps CLI handle ASG warm pools 2021-04-15 11:10:23 +02:00
Ole Markus With c6e5c4364d Allow setting dedicated apiserver node count from create cluster cmd 2021-03-27 08:59:45 +01:00
AkiraFukushima 36acadca59 Fill Role names in kops-controller-config instead of instance profile names when it is specified
The role names are checked in node bootstrap.
If profile names are provided, bootstrap will fail.
Because profile name and role name do not always mactch in AWS IAM
2021-02-11 14:28:49 +09:00
Ciprian Hacman 4acc1d4f5d Update docs for CentOS 8 2020-12-09 09:36:43 +02:00
Markos Chandras 0eb114d751
Tolerate missing detached EC2 instances
Sometimes we see the following error during a rolling update:

I1125 18:12:46.467059     165 instancegroups.go:340] Draining the node: "ip-X-X-X-X.X.compute.internal".
I1125 18:12:46.473365     165 instancegroups.go:359] deleting node "ip-X-X-X-X.X.compute.internal" from kubernetes
I1125 18:12:46.476756     165 instancegroups.go:486] Stopping instance "i-XXXXXXXX", node "ip-X-X-X-X.X.compute.internal", in group "X" (this may take a while).
E1125 18:12:46.523269     165 instancegroups.go:367] error deleting instance "i-XXXXXXXX", node "ip-X-X-X-X.X.compute.internal": error deleting instance "i-XXXXXXXX", node "ip-X-X-X-X.X.compute.internal": error deleting instance "i-XXXXXXXX": InvalidInstanceID.NotFound: The instance ID 'i-XXXXXXXXX' does not exist
	status code: 400, request id: 91238c21-1caf-41eb-91d7-534d4ca67ed0

It's possible that the EC2 instance to have disappeared by the time it
was detached (it may have been a spot instance for example)

In any case, we can't do much when we do not find an instance id, and
throwing this error during the update is not very user friendly.

As such, we can simply report and tolerate this problem instead of
exiting with non-zero code. This is similar to how we handle missing
EC2 when updating an IG[1]

[1] https://github.com/kubernetes/kops/pull/594
2020-11-26 08:55:38 +00:00
Ciprian Hacman 1d6a51aff9 Address review comments 2020-11-09 21:41:58 +02:00
Ciprian Hacman 7b04b97999 Use LaunchTemplate id instead of name for rolling updates 2020-11-09 21:40:36 +02:00
Ciprian Hacman 057ab28d4c Update LaunchTemplate tags on changes 2020-11-09 21:40:36 +02:00
Ciprian Hacman 565adceab9 Use LaunchTemplate versions instead of timestamped LaunchTemplates 2020-11-09 21:40:33 +02:00
Christian Joun e91ed11449
Implement API load balancer class with NLB and ELB support on AWS (#9011)
* refactor TargetLoadBalancer to use DNSTarget interface instead of LoadBalancer

* add LoadBalancerClass fields into api

* make api machinery

* WIP: Implemented API loadbalancer class, allowing NLB and ELB support on AWS for new clusters.

* perform vendoring related tasks and apply fixes identified from hack/

dissallow spotinst + nlb
remove reflection in status_discovery.go
Add precreated additional security groups to the Master nodes in case of NLB
Remove support for attaching individual instances to NLB; only rely on ASG attachments
Don't specify Classic loadbalancer in GCE integration test

* add utility function to the kops model context to make LoadBalancer comparisons simpler

* use DNSTarget interface when locating DNSName of API ELB

* wip: create target group task

* Consolidate TargetGroup tasks

* Use context helper for determining api load balancer type to avoid nil pointers

* Update NLB creation to use target group ARN from separate task rather than creating a TG in-line

* Address staticcheck and bazel failures

* Removing NLB Attachment tasks because they're not used since we switched to defining them as a part of the ASGs

* Address PR review feedback

* Only set LB Class field for AWS clusters, fix nil pointer

* Move target group attributes from NLB task to TG task, removing unused attributes

* Add terraform and cloudformation support for NLBs, listeners, and target groups

* Update integration test for NLB support

* Fix NLB name format to pass terraform validation

* Preserve security group rule names when switching ELB to NLB to reduce destructive terraform changes

* Use elbv2 enums and address some TODOs

* Set healthcheck values in target group

* Find TG tags, fix NLB name detection

* Fix more spurious changes reported by lifecycle integration test

* Fix spotinst validation, more code cleanup

* Address more PR feedback

* ReconcileTargetGroups unit test + more code simplification

* Addressing PR feedback Renaming task 1. awstasks.LoadBalancer -> awstasks.ClassicLoadBalancer

* Addressing PR feedback Renaming task: ELBName() -> CLBName() / LinkToELB() -> LinkToCLB()

* Addressing PR feedback: Various text changes

* fix export of kubecfg

* address TargetGroup should have the same name as the NLB

* should address error when fetching tags due to missing ARN

* Update expected and crds

* Add feature table to NLB docs

* Address more feedback and remove some TODOs that arent applicable anymore

* Update spotinst validation error message

Co-authored-by: Peter Rifel <pgrifel@gmail.com>
2020-11-02 05:28:52 -08:00
Peter Rifel db1b4e301c
Reconcile deletion of VPC CIDR block associations 2020-09-30 09:34:22 -05:00
Kubernetes Prow Robot 255cd59b67
Merge pull request #9964 from rifelpet/sa-partition
Add AWS partition support to iam service account roles
2020-09-18 06:48:46 -07:00
Jesse Haka 5d455f655b add clustername for filters 2020-09-17 23:55:21 +03:00
Peter Rifel d4d4545345
Add AWS partition support to iam service account roles 2020-09-17 10:01:27 -05:00
Kubernetes Prow Robot 7a81b3e10e
Merge pull request #9900 from olemarkus/kops-get-instances-nil-pointer
Continue if asg instance is unknown
2020-09-13 16:14:57 -07:00
Ole Markus With f6abac3ec2 Continue if asg instance is unknown
Most likely this comes from ASG thinking the instance is inService, but it is terminating when we run describe from EC2
2020-09-10 07:09:05 +02:00
Justin SB 8498ac9dbb Create PublicJWKS feature flag
This should be much easier to start and to get under testing; it only
works with a load balancer, it sets the apiserver into anonymous-auth
allowed, it grants the anonymous auth user permission to read our jwks
tokens.  But it shouldn't need a second bucket or anything of that
nature.

Co-authored-by: John Gardiner Myers <jgmyers@proofpoint.com>
2020-09-09 09:57:06 -04:00
Ole Markus With 8cce4756d9 Add instance info to detached nodes 2020-09-02 15:33:13 +02:00
Ole Markus With 0ec71686b9 Refactor cloudinstancegroupmember in a more independent cloud instance representation
Apply suggestions from code review

Co-authored-by: John Gardiner Myers <jgmyers@proofpoint.com>
2020-08-30 21:37:03 +02:00
Ole Markus With ff6c04938d Add kops delete instance command
Add support for deleting instance by k8s node name

Add yes flag
2020-08-28 08:43:30 +02:00
Peter Rifel 4d9f0128a3
Upgrade to klog2
This splits up the kubernetes 1.19 PR to make it easier to keep up to date until we get it sorted out.
2020-08-16 20:56:48 -05:00
Peter Rifel d923354d4f
Address review comments 2020-06-09 17:07:44 -05:00
Peter Rifel 95b6a3f837
Address feedback 2020-06-09 10:58:24 -05:00
Peter Rifel 64ef8c2d42
Add DescribeInstanceTypes to AWSCloud 2020-06-09 10:13:00 -05:00
liranp 23c0cdab36
feat(spotinst): new hybrid mode 2020-05-23 21:39:04 +03:00
John Gardiner Myers 154833e652 Fail cluster validation if too few nodes for ig's target size 2020-05-12 22:28:26 -07:00
John Gardiner Myers 1b7c5139e0 Merge branch 'master' into surge 2020-03-03 17:53:18 -08:00
John Gardiner Myers 9f9b98e9f4 Merge branch 'master' into surge 2020-02-17 09:17:26 -08:00
Hanfei Shen 5ff9b2b91e Treat nil of LaunchTemplateSpecification.Version as $Default [0]
[0] https://docs.aws.amazon.com/autoscaling/ec2/APIReference/API_LaunchTemplateSpecification.html
2020-02-15 11:38:11 +08:00
Ciprian Hacman 5f930683ed Update support for Amazon Linux 2 2020-01-28 08:13:45 +02:00
John Gardiner Myers cc5b6f4b8f Add fi.Cloud.DetachInstance() 2020-01-27 20:15:11 -08:00
John Gardiner Myers 640f5f5b74 Terminate AWS instances through EC2 instead of Autoscaling 2020-01-27 20:15:10 -08:00
Ciprian Hacman 7a42cf42cd Set the default instance type to t3.medium for AWS 2020-01-18 08:58:05 +02:00
tanjunchen f01e8e4cdd upup/pkg/fi/cloudup/ staticcheck 2019-12-31 16:15:28 +08:00
Justin SB 802b1add43
Refactor: Add Region() method to fi.Cloud
This enables us to simply the apply logic.
2019-12-22 13:40:38 -05:00
tanjunchen d5fef40863 upup/pkg/fi/cloudup/ upup/pkg/fi/nodeup/ hack/.staticcheck_failures : simplify code and remove code 2019-12-18 09:47:58 +08:00
Ryan Bonham 44888cec22
Fix typo 2019-12-04 10:06:04 -05:00
Ryan Bonham 9a127347f2
Fix Handling of LaunchTemplate Versions for MixedInstancePolicy 2019-12-03 13:26:57 -05:00
John Gardiner Myers 29d4471c06 Fix extraneous whitespace in warning message 2019-11-02 21:10:01 -07:00
Kubernetes Prow Robot 082eda37f0
Merge pull request #7445 from hippolin/fix-needsupdate-state-in-rolling-update
Fixed "NeedsUpdate" status of nodes in mixedinstancegroups after rolling update
2019-09-20 07:12:59 -07:00
mikesplain 9e55b8230a Update copyright notices
Also cleans some white spaces
2019-09-09 14:47:51 -04:00
Hippo 8be5e0f872
Add print error 2019-09-09 14:32:54 +08:00
Justin SB b1f8f84306
Code changes for 1.15 2019-08-25 16:00:39 -04:00
Hippo 9326ff006e
Fix that the node of the instance group of the mixed instance type is in the "NeedsUpdate" state even after the rolling update 2019-08-22 09:56:52 +08:00
liranp 80020052d3
feat(spotinst): new instance group type: ocean 2019-06-03 12:38:46 +03:00
Justin SB 3a41588efd
Don't panic when deleting instancegroups
Issue #6999
2019-05-16 09:17:17 -07:00
Pavlo Kutishchev 7a40da5016 LaunchTemplate - support for deletion of instance group which using launch template 2019-05-13 16:24:43 +02:00
Justin SB 3e33ac7682
Change code from glog to klog
We don't call klog.InitFlags yet, because that will cause a flag
redefinition error until we get everyone to stop using glog.  That
will happen when we update to k8s 1.13.
2019-05-06 12:54:51 -04:00
Ryan Bonham eca2b0a27a User version number as well as name when checking LaunchTemplates 2019-04-08 14:11:02 -05:00