Commit Graph

76 Commits

Author SHA1 Message Date
justinsb caff7e36ad gce: open node->master ports for calico and cilium
We're taking the opportunity to pursue a locked-down model, but this
means we need to open ports explicitly.
2021-10-25 08:31:21 -04:00
John Gardiner Myers 0fd4dca30e Remove dead code 2021-10-02 20:58:55 -07:00
justinsb e2f7895700 GCE: When using calico, need to open up ipip protocol
We need to open up the ipip protocol, which wasn't previously enabled.

Future work could construct the firewall rules in a common library,
and then adapt them to the various clouds.
2021-09-21 21:20:24 -04:00
John Gardiner Myers be8933b577 Remove code for unsupported features 2021-08-28 13:49:55 -07:00
Ole Markus With 0439bb0d76 Remove UseServiceAccountIAM feature flag and rename feature to UseServiceAccountExternalPermissions 2021-08-07 21:20:03 +02:00
John Gardiner Myers e6ede8f4a9 Don't provision SSH key by default on AWS 2021-07-17 16:33:26 -07:00
Ciprian Hacman 7bc629b683 Use DualStack API NLB for IPv6 2021-06-26 19:16:46 +03:00
John Gardiner Myers c0b54d980d Enable IPv6 support for Cilium 2021-06-13 20:47:44 -07:00
Ciprian Hacman c49b71feb5 Use same method receiver names everywhere 2021-04-30 11:31:14 +03:00
Jason Haugen 36722afb0f change casing Asg->ASG 2021-04-22 13:07:01 -05:00
Jason Haugen 7e48dad4d2 add ManagedAsgTag, merge templates, improve docs 2021-04-19 16:51:08 -05:00
Jason Haugen d07b067249 Add NTH queue-processor mode 2021-04-19 15:43:05 -05:00
Akiros001 857cb64fad
'sv' may have 'nil' or other unexpected value (#11161)
* 'sv' may have 'nil' or other unexpected value as its corresponding error variable may be not 'nil'

* Apply suggestions from code review

Co-authored-by: Ciprian Hacman <ciprian@hakman.dev>
2021-04-15 02:13:49 -07:00
Ole Markus With 20bd724f5e Add support for scaling out the control plane with dedicated apiserver nodes
Ensure apiserver role can only be used on AWS (because of firewalling)

Apply api-server label to CP as well

Consolidate node not ready validation message

Guard apiserver nodes with a feature flag

Rename Apiserver role to APIServer

Add an integration test for apiserver nodes

Rename Apiserver role to APIServer

Enumerate all roles in rolling update docs

Apply suggestions from code review

Co-authored-by: Steven E. Harris <seh@panix.com>
2021-03-20 20:57:00 +01:00
Ciprian Hacman e57cd534b5 Allow attaching same external target group to multiple instance groups 2020-12-03 06:59:59 +02:00
Ciprian Hacman ffe0af8629 Remove support for using legacy ELB name 2020-11-22 08:24:12 +02:00
Christian Joun e91ed11449
Implement API load balancer class with NLB and ELB support on AWS (#9011)
* refactor TargetLoadBalancer to use DNSTarget interface instead of LoadBalancer

* add LoadBalancerClass fields into api

* make api machinery

* WIP: Implemented API loadbalancer class, allowing NLB and ELB support on AWS for new clusters.

* perform vendoring related tasks and apply fixes identified from hack/

dissallow spotinst + nlb
remove reflection in status_discovery.go
Add precreated additional security groups to the Master nodes in case of NLB
Remove support for attaching individual instances to NLB; only rely on ASG attachments
Don't specify Classic loadbalancer in GCE integration test

* add utility function to the kops model context to make LoadBalancer comparisons simpler

* use DNSTarget interface when locating DNSName of API ELB

* wip: create target group task

* Consolidate TargetGroup tasks

* Use context helper for determining api load balancer type to avoid nil pointers

* Update NLB creation to use target group ARN from separate task rather than creating a TG in-line

* Address staticcheck and bazel failures

* Removing NLB Attachment tasks because they're not used since we switched to defining them as a part of the ASGs

* Address PR review feedback

* Only set LB Class field for AWS clusters, fix nil pointer

* Move target group attributes from NLB task to TG task, removing unused attributes

* Add terraform and cloudformation support for NLBs, listeners, and target groups

* Update integration test for NLB support

* Fix NLB name format to pass terraform validation

* Preserve security group rule names when switching ELB to NLB to reduce destructive terraform changes

* Use elbv2 enums and address some TODOs

* Set healthcheck values in target group

* Find TG tags, fix NLB name detection

* Fix more spurious changes reported by lifecycle integration test

* Fix spotinst validation, more code cleanup

* Address more PR feedback

* ReconcileTargetGroups unit test + more code simplification

* Addressing PR feedback Renaming task 1. awstasks.LoadBalancer -> awstasks.ClassicLoadBalancer

* Addressing PR feedback Renaming task: ELBName() -> CLBName() / LinkToELB() -> LinkToCLB()

* Addressing PR feedback: Various text changes

* fix export of kubecfg

* address TargetGroup should have the same name as the NLB

* should address error when fetching tags due to missing ARN

* Update expected and crds

* Add feature table to NLB docs

* Address more feedback and remove some TODOs that arent applicable anymore

* Update spotinst validation error message

Co-authored-by: Peter Rifel <pgrifel@gmail.com>
2020-11-02 05:28:52 -08:00
John Gardiner Myers 2ac17bee69 Remove code for no-longer-supported k8s releases 2020-10-29 16:45:53 -07:00
John Gardiner Myers 7069aaabf6 Take node labels from cloud tags on AWS 2020-09-10 20:59:24 -07:00
Justin Santa Barbara d8895c57ec Add version logic to UseServiceAccountIAM
Co-authored-by: John Gardiner Myers <jgmyers@proofpoint.com>
2020-09-09 09:57:07 -04:00
Justin SB 8498ac9dbb Create PublicJWKS feature flag
This should be much easier to start and to get under testing; it only
works with a load balancer, it sets the apiserver into anonymous-auth
allowed, it grants the anonymous auth user permission to read our jwks
tokens.  But it shouldn't need a second bucket or anything of that
nature.

Co-authored-by: John Gardiner Myers <jgmyers@proofpoint.com>
2020-09-09 09:57:06 -04:00
Justin SB b158ffab04 Refactor: KopsModelContext embeds IAMModelContext
go syntax makes this an annoying change, unfortunately.
2020-08-25 11:22:34 -04:00
Peter Rifel 4d9f0128a3
Upgrade to klog2
This splits up the kubernetes 1.19 PR to make it easier to keep up to date until we get it sorted out.
2020-08-16 20:56:48 -05:00
John Gardiner Myers c5871df319 Get kubelet certificate from kops-controller 2020-08-15 10:30:20 -07:00
John Gardiner Myers 00c60ddff6 Add server code to kops-controller 2020-08-15 09:46:30 -07:00
Ciprian Hacman a7c8d2087c Use github.com/blang/semver/v4 2020-07-01 08:54:42 +03:00
Peter Rifel 460cdbfc67 Add CloudLabels tags to additional AWS resources
This adds any labels defined in the Cluster spec's CloudLabels to the tags of the following AWS resource types:

Elastic IP
Internet Gateway
NAT Gateway
Route Table
Security Group
Subnet
VPC DHCP Options
VPC
2020-04-11 13:25:52 -05:00
Ciprian Hacman 661bd6dfbd Add "owned" tag for all instance groups related resources 2020-03-02 10:39:50 +02:00
Ciprian Hacman e8c107f767 Update tags support for LaunchTemplates 2020-02-21 10:12:14 +02:00
Kubernetes Prow Robot 77d6d381c3
Merge pull request #8327 from johngmyers/remove-code
Remove code for unsupported Kubernetes versions
2020-01-15 14:54:22 -08:00
Austin Moore 4a88f7b5a5
Add ability to specify no ssh key by setting sshKeyName to empty
Add tests for no ssh key functionality

Add docs for setting no ssh key

Disable sshKey rendering for cloudformation if nosshkey is set

Fix broken test

make goimports

Fix

Formatting fix

Update kubernetes version for tests

Update expected test output

Fix imports in mesh.pb.go

Run hack/update-expected.sh

Change digital ocean logic to handle *string for SSHKeyName

Fix expected output

Missed a few
2020-01-15 15:24:32 -05:00
John Gardiner Myers e428903871 Remove code for unsupported k8s versions from pkg/model 2020-01-12 19:30:37 -08:00
Justin SB 4fe3db7faa
Don't share /etc/hosts when using etcd-manager
With etcd-manager the DNS names should only be used by the
etcd-manager pod itself, so we don't need to share /etc/hosts with the
host.

By not sharing we avoid:

(1) the temptation to address etcd directly
(2) problems of concurrent updates to /etc/hosts being hard from within a container (because locking is difficult across bind mounts)

Introducing with kubernetes 1.17 to avoid changing behavior of existing versions.
2020-01-12 11:56:14 -05:00
tanjunchen 28fdb358f8 fix-up staticcheck error 2019-10-08 13:53:04 +08:00
Justin SB 728e582360
Fill out kops controller functionality
k8s 1.16 requires that we move label setting away from the kubelet, to
a central controller.  kops-controller is that controller.
2019-09-25 12:04:34 -04:00
mikesplain 9e55b8230a Update copyright notices
Also cleans some white spaces
2019-09-09 14:47:51 -04:00
Justin SB 21cb8ece06
Label AWS ASGs with kops.k8s.io/instancegroup
We will use this to map an AWS instance to the instance group.
2019-09-02 09:26:23 -07:00
Justin SB 3e33ac7682
Change code from glog to klog
We don't call klog.InitFlags yet, because that will cause a flag
redefinition error until we get everyone to stop using glog.  That
will happen when we update to k8s 1.13.
2019-05-06 12:54:51 -04:00
Justin SB 31f408c978
Support etcd-manager in kops 1.12
In 1.12 (kops & kubenetes):

* We default etcd-manager on
* We default to etcd3
* We default to full TLS for etcd (client and peer)
* We stop allowing external access to etcd
2019-03-14 23:13:06 -04:00
Rohith 761d760c11 - updating the aws model to make use of the aws template and mixed instance policies 2019-02-04 11:59:42 +00:00
Rob Graham 8ccf42f4a2 GH-4252 Better name for the config value and also add to v1alpha1 API 2018-07-23 13:48:35 +01:00
Rob Graham e307021a6c remove log 2018-07-03 10:26:13 +01:00
Rob Graham ae327e1e8c wrestling with the api stuff 2018-07-02 15:16:37 +01:00
Rob Graham cc589ae538 Reworked to use loadbalancer only if config is specified 2018-07-02 12:02:50 +01:00
Rohith d2bae64dd1 - adding the enable-bootstrap-token-auth to the kubeapi and fixing up the various compoents 2018-06-11 09:57:26 +01:00
Rohith 2d5bd2cfd9 - update the IAM policy to ensure the kubelet permision is skipped
- update the PKI to ensure on new clusters the certificate it not created
2018-06-11 09:57:26 +01:00
Rohith 4531384649 This PR attempts to solve two issues
a) The current implementation use's a static kubelet which doesn't not conform to the Node authorization mode (i.e. system:nodes:<nodename>)
b) As present the kubeconfig is static and reused across all the masters and nodes

The PR firstly introduces the ability for users to use bootstrap tokens and secondly when enabled ensure the kubelets for the masters as have unique usernames.  Note, this PR does not attempt to address the distribution of the bootstrap tokens themselves, that's for cluster admins. One solution for this would be a daemonset on the masters running on hostNetwork and reuse dns-controller to annotated the pods and give as the DNS

Notes:
- the master node do not use bootstrap tokens, instead given they have access to the ca anyhow, we generate certificates for each.
- when bootstrap token is not enabled the behaviour will stay the same; i.e. a kubelet configuration brought down from the store.
- when bootstrap tokens are enabled, the Nodes sit in a timeout loop waiting for the configuration to appear (by third party).
- given the nodeup docker and manifests builders are executed before the kubelet builder, the assumption here is a unit file kicks of a custom container to bootstrap the rest.
- the current firewalls on between the master and nodes are fairly open so no need to open ports between the two
- much of the work was ported from @justinsb PR [here](https://github.com/kubernetes/kops/pull/4134/)
- we add a very presumptuous server and client certificates for use with an authorizer (node-bootstrap-internal.dns_zone)

I do have an additional PR which performs the entire thing. The process being a node_authorizer which runs on the master nodes via a daemonset, the service implements a series of authorizers (i.e. alwaysallow, aws, gce etc). For aws, the process is similar to how vault authorizes nodes [here](https://www.vaultproject.io/docs/auth/aws.html). Nodeup no then calls out to the node_authorizer on bootstrap and provisions the kubelet.
2018-06-11 09:56:32 +01:00
Justin Santa Barbara a7f82a6380 Fix shared subnet/vpc tags
* Stop setting the Name tag on a shared subnet/vpc

* Stop setting the legacy KubernetesCluster tag on a shared subnet/vpc
that is new enough (>=1.6); we rely on the shared tags instead

* Set tags on shared subnets; i.e. we _do_ set the shared tag on a
shared subnet; that is important for ELBs

* Set tags on shared VPCs; i.e. we _do_ set the shared tag on a shared
VPC; that is not used but consistent with subnets.

* Add tests for shared subnet
2017-10-23 11:30:25 -04:00
Justin Santa Barbara 7fd1196708 Add Zones field to InstanceGroup
The Zones field can specify zones where they are not specified on a
Subnet, for example on GCE where we have regional subnets.
2017-09-30 19:44:35 -04:00
Justin Santa Barbara b29f3a7505 Honor ServiceNodePortRange when opening NodePort access 2017-09-15 00:39:41 -04:00