Adding documentation for Topology Aware Hints

content/en/docs/concepts/services-networking/service-topology.md

@@ -15,7 +15,16 @@ weight: 10
 
 <!-- overview -->
 
-{{< feature-state for_k8s_version="v1.17" state="alpha" >}}
+{{< feature-state for_k8s_version="v1.21" state="deprecated" >}}
+
+{{< note >}}
+
+This feature, specifically the alpha topologyKeys API, is deprecated in
+Kubernetes v1.21. [Topology Aware
+Hints](/docs/concepts/services-networking/topology-aware-hints) was introduced
+in Kubernetes v1.21 and provides similar functionality.
+
+{{</ note >}}
 
 _Service Topology_ enables a service to route traffic based upon the Node
 topology of the cluster. For example, a service can specify that traffic be

content/en/docs/concepts/services-networking/topology-aware-hints.md

@@ -0,0 +1,156 @@
+---
+reviewers:
+- robscott
+title: Topology Aware Hints
+content_type: concept
+weight: 10
+---
+
+
+<!-- overview -->
+
+{{< feature-state for_k8s_version="v1.21" state="alpha" >}}
+
+_Topology Aware Hints_ enable topology aware routing by including suggestions
+for how clients should consume endpoints. This approach adds metadata to enable
+consumers of Endpoint(Slice) to be able to route traffic closer to where it is
+originated. For example, users can route traffic within a locality to reduce
+costs and improve performance.
+
+<!-- body -->
+
+## Motivation
+
+Kubernetes clusters are increasingly deployed in multi-zone environments.
+_Topology Aware Hints_ provides a mechanism to help keep traffic within the zone
+it originated from. This concept is commonly referred to as "Topology Aware
+Routing". When calculating the endpoints for a Service, the EndpointSlice
+controller considers the topology (region and zone) of each endpoint and
+populates the hints field to allocate it to a zone. These hints are then
+consumed by components like kube-proxy as they configure how requests are
+routed.
+
+## Using Topology Aware Hints
+
+You can enable Topology Aware Hints for a Service by setting the
+`service.kubernetes.io/topology-aware-hints` annotation to `auto`. This tells
+the EndpointSlice controller to set topology hints if it is deemed safe.
+Importantly, this does not guarantee that hints will always be set.
+
+## How it Works
+
+The functionality enabling this feature is split into two components: The
+EndpointSlice controller and Kube-Proxy. This provides a high level overview of
+how each component implements this feature.
+
+### EndpointSlice controller
+
+The EndpointSlice controller is responsible for setting hints on EndpointSlices
+when this feature is enabled. The controller allocates a proportional amount of
+endpoints to each zone. This proportion is based on the
+[allocatable](/docs/tasks/administer-cluster/reserve-compute-resources/#node-allocatable)
+CPU cores for nodes running in that zone. For example, if one zone had 2 CPU
+cores and another zone only had 1 CPU core, the controller would allocated twice
+as many endpoints to the zone with 2 CPU cores.
+
+The following example shows what an EndpointSlice looks like when hints have
+been populated:
+
+```yaml
+apiVersion: discovery.k8s.io/v1
+kind: EndpointSlice
+metadata:
+  name: example-hints
+  labels:
+    kubernetes.io/service-name: example-svc
+addressType: IPv4
+ports:
+  - name: http
+    protocol: TCP
+    port: 80
+endpoints:
+  - addresses:
+      - "10.1.2.3"
+    conditions:
+      ready: true
+    hostname: pod-1
+    zone: zone-a
+    hints:
+      forZones:
+        - name: "zone-a"
+```
+
+### Kube-Proxy
+
+Kube-Proxy filters the endpoints it routes to based on the hints set by the
+EndpointSlice controller. In most cases, this means that kube-proxy will route
+to endpoints in the same zone. Sometimes the controller allocates endpoints from
+a different zone to ensure more even distribution of endpoints between zones.
+This would result in some traffic being routed to other zones.
+
+## Safeguards
+
+The Kubernetes control plane and the kube-proxy on each node apply some
+safeguard rules before using Topology Aware Hints. If these don't check out,
+kube-proxy selects endpoints from anywhere in your cluster, regardless of the
+zone.
+
+1. **Insufficient number of endpoints:** If there are less endpoints than zones
+   in a cluster, the controller will not assign any hints. 
+
+2. **Impossible to achieve balanced allocation:** In some cases, it will be
+   impossible to achieve a balanced allocation of endpoints among zones. For
+   example, if zone-a is twice as large as zone-b, but there are only 2
+   endpoints, an endpoint allocated to zone-a may receive twice as much traffic
+   as zone-b. The controller wil not assign hints if it can't get this "expected
+   overload" value below an acceptable threshold for each zone. Importantly this
+   is not based on real-time feedback. It is still possible for individual
+   endpoints to become overloaded.
+
+3. **One or more Nodes has insufficient information:** If any node does not have
+   a `topology.kubernetes.io/zone` label or is not reporting a value for
+   allocatable CPU, the control plane does not set any topology-aware endpoint
+   hints and so kube-proxy does not filter endpoints by zone.
+
+4. **One or more endpoints does not have a zone hint:** When this happens,
+   kube-proxy assumes that a transition from or to Topology Aware Hints is
+   underway. Filtering endpoints for a Service in this state would be dangerous
+   so Kube-Proxy falls back to using all endpoints.
+
+5. **A zone is not represented in hints:** If kube-proxy is unable to find at
+   least one endpoint with a hint targeting the zone it is running in, it will
+   fall back to using endpoints from all zones. This is most likely to happen as
+   a new zone is being added to a cluster.
+
+## Constraints
+
+* Topology Aware Hints are not used when either `externalTrafficPolicy` or
+  `internalTrafficPolicy` is set to `Local` on a Service. It is possible to use
+  both features in the same cluster on different Services, just not on the same
+  Service.
+
+* This approach will not work well for Services that have a large proportion of
+  traffic originating from a subset of zones. Instead this assumes that incoming
+  traffic will be roughly proportional to the capacity of the Nodes in each
+  zone.
+
+* The EndpointSlice controller ignores unready nodes as it calculates the
+  proportions of each zone. This could have unintended consequences if a large
+  portion of nodes are unready.
+
+* The EndpointSlice controller does not take into account {{< glossary_tooltip
+  text="tolerations" term_id="toleration" >}} when deploying calculating the
+  proportions of each zone. If the Pods backing a Service are limited to a
+  subset of Nodes in the cluster, this will not be taken into account.
+
+* This may not work well with autoscaling. For example, if a lot of traffic is
+  originating from a single zone, only the endpoints allocated to that zone will
+  be handling that traffic. That could result in {{< glossary_tooltip
+  text="Horizontal Pod Autoscaler" term_id="horizontal-pod-autoscaler" >}}
+  either not picking up on this event, or newly added pods starting in a
+  different zone.
+
+## {{% heading "whatsnext" %}}
+
+* Read about [enabling Topology Aware Hints](/docs/tasks/administer-cluster/enabling-topology-aware-hints)
+* Read [Connecting Applications with Services](/docs/concepts/services-networking/connect-applications-service/)

content/en/docs/reference/command-line-tools-reference/feature-gates.md

@@ -171,6 +171,7 @@ different Kubernetes components.
 | `StorageVersionHash` | `true` | Beta | 1.15 | |
 | `Sysctls` | `true` | Beta | 1.11 | |
 | `TTLAfterFinished` | `false` | Alpha | 1.12 | |
+| `TopologyAwareHints` | `false` | Alpha | 1.21 | |
 | `TopologyManager` | `false` | Alpha | 1.16 | 1.17 |
 | `TopologyManager` | `true` | Beta | 1.18 | |
 | `ValidateProxyRedirects` | `false` | Alpha | 1.12 | 1.13 |
@@ -779,6 +780,10 @@ Each feature gate is designed for enabling/disabling a specific feature:
 - `TokenRequest`: Enable the `TokenRequest` endpoint on service account resources.
 - `TokenRequestProjection`: Enable the injection of service account tokens into a
   Pod through a [`projected` volume](/docs/concepts/storage/volumes/#projected).
+- `TopologyAwareHints`: Enables topology aware routing based on topology hints
+  in EndpointSlices. See [Topology Aware
+  Hints](/docs/concepts/services-networking/topology-aware-hints/) for more
+  details.
 - `TopologyManager`: Enable a mechanism to coordinate fine-grained hardware resource
   assignments for different components in Kubernetes. See
   [Control Topology Management Policies on a node](/docs/tasks/administer-cluster/topology-manager/).

content/en/docs/tasks/administer-cluster/enabling-service-topology.md

@@ -8,9 +8,12 @@ content_type: task
 ---
 
 <!-- overview -->
-This page provides an overview of enabling Service Topology in Kubernetes.
+{{< feature-state for_k8s_version="v1.21" state="deprecated" >}}
-
 
+This feature, specifically the alpha topologyKeys API, is deprecated in
+Kubernetes v1.21. [Topology Aware
+Hints](/docs/concepts/services-networking/topology-aware-hints) was introduced
+in Kubernetes v1.21 and provides similar functionality.
 
 ## {{% heading "prerequisites" %}}
 

content/en/docs/tasks/administer-cluster/enabling-topology-aware-hints.md

@@ -0,0 +1,39 @@
+---
+reviewers:
+- robscott
+title: Enabling Topology Aware Hints
+content_type: task
+min-kubernetes-server-version: 1.21
+---
+
+<!-- overview -->
+{{< feature-state for_k8s_version="v1.21" state="alpha" >}}
+
+_Topology Aware Hints_ enable topology aware routing with topology hints
+included in EndpointSlices. This approach tries to keep traffic close to where
+it originated from. This can result in decreased costs and improved performance.
+
+## {{% heading "prerequisites" %}}
+
+  {{< include "task-tutorial-prereqs.md" >}} {{< version-check >}}
+
+The following prerequisite is needed in order to enable topology aware hints:
+
+* {{< glossary_tooltip text="Kube-proxy" term_id="kube-proxy" >}} running in
+  iptables mode or IPVS mode
+* Ensure that you have not disabled EndpointSlices
+
+## Enable Topology Aware Hints
+
+To enable service topology, enable the `TopologyAwareHints` [feature
+gate](docs/reference/command-line-tools-reference/feature-gates/) for the
+kube-apiserver, kube-controller-manager, and kube-proxy:
+
+```
+--feature-gates="TopologyAwareHints=true"
+```
+
+## {{% heading "whatsnext" %}}
+
+* Read about [Topology Aware Hints](/docs/concepts/services-networking/topology-aware-hints) for Services
+* Read [Connecting Applications with Services](/docs/concepts/services-networking/connect-applications-service/)