When we create a external load balancer on GCE, we now also create an
internal load balancer. The internal load balancer is used for
node/pod -> control-plane traffic, the external load balancer is used
for other traffic (e.g. "user" traffic to kube-apiserver).
This means that we can apply more granular firewall rules, and
generally avoid complex logic around discovery of the internal control
plane addresses for GCE.
If we're not masquerading the pod IPs, we need an explicit firewall
rule for the pods to reach the kube-apiserver. Normally this is
permitted anyway, but if the apiserver has a locked-down CIDR range
(as the e2e tests do) then we need our own rule.
* Add ILBs, broadly following the AWS model. The following new
capabilities are added for clusters in GCP:
* Cluster's spec.api.loadBalancer can be set to 'type: internal' on
GCP.
* Therefore, GCP can now create:
* regional backend services
* regional (non-legacy) healthchecks
* firewall rules with "internal" load-balancing scheme
* firewall rules with dot-notation-specified IP addresses
* Cluster's spec.api.loadBalancer's 'subnets' field functions
as in the AWS model.
A few incidental changes are included, either because this change
touched the relevant code or because my use case happened to trigger the
issues that are fixed here.
* Cluster's spec.networkID field can be prefixed by project to use
GCP's common cross-project networking model.
* The presumption is that all specified subnets belong to this
network and therefore this project.
* Add missing operation wait on forwarding rule creation.
* Some Terraform output improvements:
* Permit no-ACL files in GCS buckets in Terraform output.
* Enable marginally better cross-resource reference in Terraform outputs
* Add project to network + subnetwork literals in Terraform output.
* Add terraform output to backend services and health checks.
Testing:
* Add mocks for backend services and health checks.
* Add minimal integration test - copied from gce_private and ilb added.
* Add update cluster goldens.
Co-authored-by: Travis Reid <travis_reid@apple.com>
Supporting IPv6 values where they can be set by the user, and ensuring
that IPv4 and IPv6 firewall rules are split because on GCP they cannot
be in the same rule.
justinsb
Ciprian Hacman
upodroid
John Gardiner Myers
Nat Henderson
Jesse Haka
mikesplain
chrislovecnm
Justin Santa Barbara