Commit Graph

1163 Commits

Author SHA1 Message Date
Christian Joun e91ed11449
Implement API load balancer class with NLB and ELB support on AWS (#9011)
* refactor TargetLoadBalancer to use DNSTarget interface instead of LoadBalancer

* add LoadBalancerClass fields into api

* make api machinery

* WIP: Implemented API loadbalancer class, allowing NLB and ELB support on AWS for new clusters.

* perform vendoring related tasks and apply fixes identified from hack/

dissallow spotinst + nlb
remove reflection in status_discovery.go
Add precreated additional security groups to the Master nodes in case of NLB
Remove support for attaching individual instances to NLB; only rely on ASG attachments
Don't specify Classic loadbalancer in GCE integration test

* add utility function to the kops model context to make LoadBalancer comparisons simpler

* use DNSTarget interface when locating DNSName of API ELB

* wip: create target group task

* Consolidate TargetGroup tasks

* Use context helper for determining api load balancer type to avoid nil pointers

* Update NLB creation to use target group ARN from separate task rather than creating a TG in-line

* Address staticcheck and bazel failures

* Removing NLB Attachment tasks because they're not used since we switched to defining them as a part of the ASGs

* Address PR review feedback

* Only set LB Class field for AWS clusters, fix nil pointer

* Move target group attributes from NLB task to TG task, removing unused attributes

* Add terraform and cloudformation support for NLBs, listeners, and target groups

* Update integration test for NLB support

* Fix NLB name format to pass terraform validation

* Preserve security group rule names when switching ELB to NLB to reduce destructive terraform changes

* Use elbv2 enums and address some TODOs

* Set healthcheck values in target group

* Find TG tags, fix NLB name detection

* Fix more spurious changes reported by lifecycle integration test

* Fix spotinst validation, more code cleanup

* Address more PR feedback

* ReconcileTargetGroups unit test + more code simplification

* Addressing PR feedback Renaming task 1. awstasks.LoadBalancer -> awstasks.ClassicLoadBalancer

* Addressing PR feedback Renaming task: ELBName() -> CLBName() / LinkToELB() -> LinkToCLB()

* Addressing PR feedback: Various text changes

* fix export of kubecfg

* address TargetGroup should have the same name as the NLB

* should address error when fetching tags due to missing ARN

* Update expected and crds

* Add feature table to NLB docs

* Address more feedback and remove some TODOs that arent applicable anymore

* Update spotinst validation error message

Co-authored-by: Peter Rifel <pgrifel@gmail.com>
2020-11-02 05:28:52 -08:00
Peter Rifel 7d5a39974f
Add lifecycle integration tests for complex and externallb clusters 2020-10-30 10:06:36 -05:00
Kubernetes Prow Robot b7f66a6d98
Merge pull request #10109 from bmelbourne/set-minimum-terraform-0.12
Set minimum Terraform version to 0.12.26/0.13.0
2020-10-29 01:52:58 -07:00
Barry Melbourne 84417c330b Set minimum Terraform version to 0.12.26/0.13.0 2020-10-28 20:24:41 +00:00
Ciprian Hacman f69ffeaa63 Update cluster and state store names in CLI docs 2020-10-28 18:22:53 +02:00
Kubernetes Prow Robot d739bae871
Merge pull request #10106 from johngmyers/tf-json
Remove dependency of TerraformJSON feature flag
2020-10-26 07:23:01 -07:00
Justin SB e03bb72c2c Default to exporting a kubecfg, even without credentials
We do log a hint for the user when we have exported an empty kubecfg,
but this now supports the "current cluster" UX.

Issue #9990
2020-10-25 14:30:32 -04:00
John Gardiner Myers f92d486197 Remove dependency of TerraformJSON feature flag 2020-10-25 10:49:59 -07:00
Kubernetes Prow Robot fbb172c08c
Merge pull request #9575 from johngmyers/node-labels
Take node labels from cloud tags on AWS
2020-10-23 04:01:45 -07:00
Nicolas Vanheuverzwijn b0fd89a193 upgrade-cluster: test that new image in stable or alpha channel will receive automated update 2020-10-14 10:14:47 -04:00
Ole Markus With b122d0e3ba Fix nil pointer when deleting instance 2020-10-13 13:23:22 +02:00
Ole Markus With aa66c4f6d8 Add rolling upgrade to openstack 2020-10-01 20:07:44 +02:00
Ole Markus With 7c8ff94631 Make setupmockopenstack standalone 2020-10-01 19:15:39 +02:00
Kubernetes Prow Robot d6f60b9ee5
Merge pull request #9981 from olemarkus/cleanup-cloud-2
More removals of BuildCloud
2020-10-01 05:18:54 -07:00
Kubernetes Prow Robot 257f85962e
Merge pull request #9974 from olemarkus/cleanup-rolling-update-ctx
Set ctx and cluster on the rolling update struct instead of passing it around everywhere
2020-09-27 02:24:48 -07:00
Derrik Campau ca70ac2203 Fix small typo in create cluster help output
Fixes typo in cmd/kops/create_cluster.go and
docs/cli/kops_create_cluster.md where example output had filename.yamlh,
changed to filename.yaml
2020-09-26 14:45:03 -07:00
Ole Markus With 5df2d2adbd Fix nil pointer when instance has not joined the cluster 2020-09-24 20:31:19 +02:00
Ole Markus With 1d922af364 Pass cloud into populate cluster 2020-09-24 07:22:13 +02:00
Ole Markus With 1a905d2063 Pass cloud into ApplyCluster 2020-09-23 19:57:43 +02:00
Ole Markus With 63f13322d5 Don't pass ctx and cluster everywhere 2020-09-23 08:30:24 +02:00
Ole Markus With 7bc17f4b1f Build cloud outside of PerformAssignments
We tend to build cloud, call some method, and then build cloud over
again. It would be easier to just pass the first one along.

Passing along cloud would also make it easier to mock cloud.
2020-09-23 07:54:28 +02:00
Ole Markus With 31ee079c7b Improve kops get instances when api is unavailable
When the api is unavailable, kops will say all the nodes have not yet
joined the cluster. That is not the case simply because e.g the admin
credentials have been expired. This PR makes it a bit more clear that we
cannot know the node name when the API is unavailable.
2020-09-19 08:43:53 +02:00
John Gardiner Myers f4cecc58ac Ignore lack of tags on launch templates 2020-09-10 20:59:28 -07:00
Justin SB ccc814dfbc Create tests for JWKS scenarios 2020-09-09 09:57:06 -04:00
Ciprian Hacman c7bc3d4397 Update mock version to 1.19.0-alpha.3 2020-09-08 08:45:25 +03:00
Ole Markus With a483945711 Refactor based on changes to cloud instances 2020-09-01 08:41:53 +02:00
Ole Markus With c01455cf91 Keep the good part from last attempt 2020-09-01 08:30:03 +02:00
Kubernetes Prow Robot e11146c0df
Merge pull request #9799 from olemarkus/cloudinstances-refactor
Cloudinstances refactor
2020-08-31 23:23:50 -07:00
Kubernetes Prow Robot 5d09a9a95b
Merge pull request #9667 from justinsb/kubectl_auth_helper
Support authentication helper for kubectl
2020-08-30 21:46:21 -07:00
Ole Markus With 0ec71686b9 Refactor cloudinstancegroupmember in a more independent cloud instance representation
Apply suggestions from code review

Co-authored-by: John Gardiner Myers <jgmyers@proofpoint.com>
2020-08-30 21:37:03 +02:00
Justin SB 8757a2ce2a kubeconfig generation: add tests for kops plugin
Also slightly simplify the tests and Kubecfg Builder signature by
passing in the ConfigAccess only when needed.
2020-08-30 15:17:36 -04:00
Justin SB 0cda0f5068 Support authentication helper for kubectl
We create a simple exec plugin command which can create and renew
short-lived admin credentials on the fly, essentially leveraging the
security of the underlying cloud credentials.

Co-authored-by: John Gardiner Myers <jgmyers@proofpoint.com>
2020-08-30 15:16:20 -04:00
Justin SB 786423f617 Expose JWKS via a feature-flag
When the PublicJWKS feature-flag is set, we expose the apiserver JWKS
document publicly (including enabling anonymous access).  This is a
stepping stone to a more hardened configuration where we copy the JWKS
document to S3/GCS/etc.

Co-authored-by: John Gardiner Myers <jgmyers@proofpoint.com>
2020-08-30 10:15:11 -04:00
Justin Santa Barbara f32fcc35fa Addons: Support arbitrary additional objects
We will be managing cluster addons using CRDs, and so we want to be
able to apply arbitrary objects as part of cluster bringup.

Start by allowing (behind a feature-flag) for arbitrary objects to be
specified.

Co-authored-by: John Gardiner Myers <jgmyers@proofpoint.com>
2020-08-28 09:03:41 -04:00
Kubernetes Prow Robot e537846b41
Merge pull request #9784 from olemarkus/kops-delete-instance
Add kops delete instance command
2020-08-28 00:36:53 -07:00
Ole Markus With ff6c04938d Add kops delete instance command
Add support for deleting instance by k8s node name

Add yes flag
2020-08-28 08:43:30 +02:00
Peter Rifel 64f6f5e2cb
Add integration test for GCE private topology with bastion 2020-08-27 14:28:26 -05:00
Peter Rifel d0b8c654bd
Add --internal flag for export kubecfg that targets the internal dns name
Kops creates an "api.internal.$clustername" dns A record that points to the master IP(s)

This adds a flag that will use that name and force the CA cert to be included.
This is a workaround for client certificate authentication not working on API ELBs with ACM certificates.
The ELB has a TLS listener rather than TCP, so the client certificate is not passed through to the apiserver.
Using --internal will bypass the API ELB so that the client certificate will be passed directly to the apiserver.
This also requires that the masters' security groups allow 443 access from the client which this does not handle automatically.
2020-08-26 21:15:18 -05:00
Peter Rifel 4d9f0128a3
Upgrade to klog2
This splits up the kubernetes 1.19 PR to make it easier to keep up to date until we get it sorted out.
2020-08-16 20:56:48 -05:00
Kubernetes Prow Robot ec8b47d725
Merge pull request #9593 from johngmyers/kubectl-lifetime
Reduce the lifetime of exported kubecfg credentials
2020-08-14 19:24:18 -07:00
Ole Markus With 9890839cec Add an integration test for openstack floating ip
* Integration test for floatingip cluster
* Implements mocking of floatingIP (only list for now)
* Expands various cloudmocks
* Fixes an NPR in openstack validation
* Fixes a bug where kops tries to use DNS even if the cluster is gossip
2020-08-12 12:59:30 +02:00
Kubernetes Prow Robot b7871e2e72
Merge pull request #9478 from bwagner5/feat-instance-selector
Add instance-selector cmd to toolbox
2020-08-11 14:15:45 -07:00
Brandon Wagner c4e2497a8a change defaults 2020-08-11 15:40:54 -05:00
Brandon Wagner e1136f6d9a fix new cli api for byte quantities 2020-08-10 17:13:43 -05:00
Brandon Wagner 602564d26c use byte quantity flag instead of int MiBs for memory args 2020-08-10 16:16:51 -05:00
Brandon Wagner 89c90c8b49 cpuarch amd64 is now supported in upstream lib 2020-08-10 16:16:51 -05:00
Brandon Wagner b4bc9b5d56 update cli docs for instance-selector 2020-08-10 16:16:51 -05:00
Brandon Wagner 2a33b98317 ove instance-group-name to arg like create ig 2020-08-10 16:16:51 -05:00
Brandon Wagner 1bb593aa1a move from zones input to subnets input 2020-08-10 16:16:51 -05:00
Brandon Wagner 8d81c225a9 pr comments 2020-08-10 16:16:51 -05:00