Commit Graph

1243 Commits

Author SHA1 Message Date
Christian Joun e91ed11449
Implement API load balancer class with NLB and ELB support on AWS (#9011)
* refactor TargetLoadBalancer to use DNSTarget interface instead of LoadBalancer

* add LoadBalancerClass fields into api

* make api machinery

* WIP: Implemented API loadbalancer class, allowing NLB and ELB support on AWS for new clusters.

* perform vendoring related tasks and apply fixes identified from hack/

dissallow spotinst + nlb
remove reflection in status_discovery.go
Add precreated additional security groups to the Master nodes in case of NLB
Remove support for attaching individual instances to NLB; only rely on ASG attachments
Don't specify Classic loadbalancer in GCE integration test

* add utility function to the kops model context to make LoadBalancer comparisons simpler

* use DNSTarget interface when locating DNSName of API ELB

* wip: create target group task

* Consolidate TargetGroup tasks

* Use context helper for determining api load balancer type to avoid nil pointers

* Update NLB creation to use target group ARN from separate task rather than creating a TG in-line

* Address staticcheck and bazel failures

* Removing NLB Attachment tasks because they're not used since we switched to defining them as a part of the ASGs

* Address PR review feedback

* Only set LB Class field for AWS clusters, fix nil pointer

* Move target group attributes from NLB task to TG task, removing unused attributes

* Add terraform and cloudformation support for NLBs, listeners, and target groups

* Update integration test for NLB support

* Fix NLB name format to pass terraform validation

* Preserve security group rule names when switching ELB to NLB to reduce destructive terraform changes

* Use elbv2 enums and address some TODOs

* Set healthcheck values in target group

* Find TG tags, fix NLB name detection

* Fix more spurious changes reported by lifecycle integration test

* Fix spotinst validation, more code cleanup

* Address more PR feedback

* ReconcileTargetGroups unit test + more code simplification

* Addressing PR feedback Renaming task 1. awstasks.LoadBalancer -> awstasks.ClassicLoadBalancer

* Addressing PR feedback Renaming task: ELBName() -> CLBName() / LinkToELB() -> LinkToCLB()

* Addressing PR feedback: Various text changes

* fix export of kubecfg

* address TargetGroup should have the same name as the NLB

* should address error when fetching tags due to missing ARN

* Update expected and crds

* Add feature table to NLB docs

* Address more feedback and remove some TODOs that arent applicable anymore

* Update spotinst validation error message

Co-authored-by: Peter Rifel <pgrifel@gmail.com>
2020-11-02 05:28:52 -08:00
Peter Rifel 7d5a39974f
Add lifecycle integration tests for complex and externallb clusters 2020-10-30 10:06:36 -05:00
Kubernetes Prow Robot b7f66a6d98
Merge pull request #10109 from bmelbourne/set-minimum-terraform-0.12
Set minimum Terraform version to 0.12.26/0.13.0
2020-10-29 01:52:58 -07:00
Barry Melbourne 84417c330b Set minimum Terraform version to 0.12.26/0.13.0 2020-10-28 20:24:41 +00:00
Ciprian Hacman f69ffeaa63 Update cluster and state store names in CLI docs 2020-10-28 18:22:53 +02:00
Kubernetes Prow Robot d739bae871
Merge pull request #10106 from johngmyers/tf-json
Remove dependency of TerraformJSON feature flag
2020-10-26 07:23:01 -07:00
Justin SB e03bb72c2c Default to exporting a kubecfg, even without credentials
We do log a hint for the user when we have exported an empty kubecfg,
but this now supports the "current cluster" UX.

Issue #9990
2020-10-25 14:30:32 -04:00
John Gardiner Myers f92d486197 Remove dependency of TerraformJSON feature flag 2020-10-25 10:49:59 -07:00
Kubernetes Prow Robot fbb172c08c
Merge pull request #9575 from johngmyers/node-labels
Take node labels from cloud tags on AWS
2020-10-23 04:01:45 -07:00
Nicolas Vanheuverzwijn b0fd89a193 upgrade-cluster: test that new image in stable or alpha channel will receive automated update 2020-10-14 10:14:47 -04:00
Ole Markus With b122d0e3ba Fix nil pointer when deleting instance 2020-10-13 13:23:22 +02:00
Ole Markus With 466dcd001e Apply suggestions from code review
Co-authored-by: John Gardiner Myers <jgmyers@proofpoint.com>
2020-10-09 08:27:08 +02:00
Ole Markus With 809aa93634 Make use of kubelet service certificate 2020-10-09 08:27:08 +02:00
Ole Markus With aa66c4f6d8 Add rolling upgrade to openstack 2020-10-01 20:07:44 +02:00
Ole Markus With 7c8ff94631 Make setupmockopenstack standalone 2020-10-01 19:15:39 +02:00
Kubernetes Prow Robot d6f60b9ee5
Merge pull request #9981 from olemarkus/cleanup-cloud-2
More removals of BuildCloud
2020-10-01 05:18:54 -07:00
Kubernetes Prow Robot 257f85962e
Merge pull request #9974 from olemarkus/cleanup-rolling-update-ctx
Set ctx and cluster on the rolling update struct instead of passing it around everywhere
2020-09-27 02:24:48 -07:00
Derrik Campau ca70ac2203 Fix small typo in create cluster help output
Fixes typo in cmd/kops/create_cluster.go and
docs/cli/kops_create_cluster.md where example output had filename.yamlh,
changed to filename.yaml
2020-09-26 14:45:03 -07:00
Ole Markus With 5df2d2adbd Fix nil pointer when instance has not joined the cluster 2020-09-24 20:31:19 +02:00
Ole Markus With 1d922af364 Pass cloud into populate cluster 2020-09-24 07:22:13 +02:00
Ole Markus With 1a905d2063 Pass cloud into ApplyCluster 2020-09-23 19:57:43 +02:00
Ole Markus With 63f13322d5 Don't pass ctx and cluster everywhere 2020-09-23 08:30:24 +02:00
Ole Markus With 7bc17f4b1f Build cloud outside of PerformAssignments
We tend to build cloud, call some method, and then build cloud over
again. It would be easier to just pass the first one along.

Passing along cloud would also make it easier to mock cloud.
2020-09-23 07:54:28 +02:00
Ole Markus With 31ee079c7b Improve kops get instances when api is unavailable
When the api is unavailable, kops will say all the nodes have not yet
joined the cluster. That is not the case simply because e.g the admin
credentials have been expired. This PR makes it a bit more clear that we
cannot know the node name when the API is unavailable.
2020-09-19 08:43:53 +02:00
John Gardiner Myers f4cecc58ac Ignore lack of tags on launch templates 2020-09-10 20:59:28 -07:00
John Gardiner Myers 7069aaabf6 Take node labels from cloud tags on AWS 2020-09-10 20:59:24 -07:00
John Gardiner Myers 24ff622d8e Rename NodeReconciler to LegacyNodeReconciler 2020-09-10 20:42:56 -07:00
Kubernetes Prow Robot 036ea69525
Merge pull request #9352 from justinsb/irsa_with_public
Simplified form of IAM Roles for ServiceAccounts
2020-09-09 22:23:44 -07:00
Kubernetes Prow Robot 4508406515
Merge pull request #9908 from rdrgmnzs/CacheNodeidentityInfo
Allow caching of Nodeidentity Info in kops-controller for AWS.
2020-09-09 13:01:44 -07:00
Rodrigo Menezes 4c057f138a Allow caching of Nodeidentity Info in kops-controller for AWS to reduce the number of DescribeInstances API calls. 2020-09-09 22:11:29 +03:00
Justin SB ccc814dfbc Create tests for JWKS scenarios 2020-09-09 09:57:06 -04:00
Ciprian Hacman c7bc3d4397 Update mock version to 1.19.0-alpha.3 2020-09-08 08:45:25 +03:00
Ole Markus With a483945711 Refactor based on changes to cloud instances 2020-09-01 08:41:53 +02:00
Ole Markus With c01455cf91 Keep the good part from last attempt 2020-09-01 08:30:03 +02:00
Kubernetes Prow Robot e11146c0df
Merge pull request #9799 from olemarkus/cloudinstances-refactor
Cloudinstances refactor
2020-08-31 23:23:50 -07:00
Kubernetes Prow Robot 5d09a9a95b
Merge pull request #9667 from justinsb/kubectl_auth_helper
Support authentication helper for kubectl
2020-08-30 21:46:21 -07:00
Ole Markus With 0ec71686b9 Refactor cloudinstancegroupmember in a more independent cloud instance representation
Apply suggestions from code review

Co-authored-by: John Gardiner Myers <jgmyers@proofpoint.com>
2020-08-30 21:37:03 +02:00
Justin SB 8757a2ce2a kubeconfig generation: add tests for kops plugin
Also slightly simplify the tests and Kubecfg Builder signature by
passing in the ConfigAccess only when needed.
2020-08-30 15:17:36 -04:00
Justin SB 0cda0f5068 Support authentication helper for kubectl
We create a simple exec plugin command which can create and renew
short-lived admin credentials on the fly, essentially leveraging the
security of the underlying cloud credentials.

Co-authored-by: John Gardiner Myers <jgmyers@proofpoint.com>
2020-08-30 15:16:20 -04:00
Justin SB 786423f617 Expose JWKS via a feature-flag
When the PublicJWKS feature-flag is set, we expose the apiserver JWKS
document publicly (including enabling anonymous access).  This is a
stepping stone to a more hardened configuration where we copy the JWKS
document to S3/GCS/etc.

Co-authored-by: John Gardiner Myers <jgmyers@proofpoint.com>
2020-08-30 10:15:11 -04:00
Justin Santa Barbara f32fcc35fa Addons: Support arbitrary additional objects
We will be managing cluster addons using CRDs, and so we want to be
able to apply arbitrary objects as part of cluster bringup.

Start by allowing (behind a feature-flag) for arbitrary objects to be
specified.

Co-authored-by: John Gardiner Myers <jgmyers@proofpoint.com>
2020-08-28 09:03:41 -04:00
Kubernetes Prow Robot e537846b41
Merge pull request #9784 from olemarkus/kops-delete-instance
Add kops delete instance command
2020-08-28 00:36:53 -07:00
Ole Markus With ff6c04938d Add kops delete instance command
Add support for deleting instance by k8s node name

Add yes flag
2020-08-28 08:43:30 +02:00
Peter Rifel 64f6f5e2cb
Add integration test for GCE private topology with bastion 2020-08-27 14:28:26 -05:00
Peter Rifel d0b8c654bd
Add --internal flag for export kubecfg that targets the internal dns name
Kops creates an "api.internal.$clustername" dns A record that points to the master IP(s)

This adds a flag that will use that name and force the CA cert to be included.
This is a workaround for client certificate authentication not working on API ELBs with ACM certificates.
The ELB has a TLS listener rather than TCP, so the client certificate is not passed through to the apiserver.
Using --internal will bypass the API ELB so that the client certificate will be passed directly to the apiserver.
This also requires that the masters' security groups allow 443 access from the client which this does not handle automatically.
2020-08-26 21:15:18 -05:00
John Gardiner Myers 07220797b4 Issue the cilium etcd client cert out of kops-controller 2020-08-17 21:15:34 -07:00
Peter Rifel bae8150e12
Update more klog v1 references to v2
I missed these in the previous PR. This removes the direct dependency on v1 entirely.
The kubernetes 1.19 upgrade will remove the indirect reference on v1.
2020-08-17 07:44:48 -05:00
John Gardiner Myers d05f9a3eff Don't issue certs for features not enabled 2020-08-16 23:40:43 -07:00
John Gardiner Myers b6947ccaee Use kops-controller to issue kube-router cert 2020-08-16 23:40:38 -07:00
John Gardiner Myers 8e43c1d637 Use kops-controller to issue kube-proxy cert 2020-08-16 23:36:42 -07:00