Added documentation to support Topology Manager feature in Kubelet. (#15716)

* Added documentation to support Topology Manager feature in Kubelet.

	* Added new document outlining feature
	* Updated feature-gates.md to include feature gate for feature
	* Update kubelet.md to include kubelet flags for feature
	* Added Topology Manager reference to relevant pages

Co-authored-by: Tim Bannister <tim@scalefactory.com>

* Update Topology Manager Policies section

content/en/docs/concepts/architecture/nodes.md

@@ -290,6 +290,13 @@ includes all containers started by the kubelet, but not containers started direc
 If you want to explicitly reserve resources for non-Pod processes, follow this tutorial to
 [reserve resources for system daemons](/docs/tasks/administer-cluster/reserve-compute-resources/#system-reserved).
 
+## Node topology
+
+{{< feature-state state="alpha" >}}
+
+If you have enabled the `TopologyManager`
+[feature gate](/docs/reference/command-line-tools-reference/feature-gates/), then
+the kubelet can use topology hints when making resource assignment decisions.
 
 ## API Object
 
@@ -298,3 +305,7 @@ API object can be found at:
 [Node API object](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#node-v1-core).
 
 {{% /capture %}}
+{{% capture whatsnext %}}
+* Read about [node components](https://kubernetes.io/docs/concepts/overview/components/#node-components)
+* Read about node-level topology: [Control Topology Management Policies on a node](/docs/tasks/administer-cluster/topology-manager/)
+{{% /capture %}}

content/en/docs/concepts/configuration/assign-pod-node.md

@@ -397,4 +397,8 @@ The design documents for
 [node affinity](https://git.k8s.io/community/contributors/design-proposals/scheduling/nodeaffinity.md)
 and for [inter-pod affinity/anti-affinity](https://git.k8s.io/community/contributors/design-proposals/scheduling/podaffinity.md) contain extra background information about these features.
 
+Once a Pod is assigned to a Node, the kubelet runs the Pod and allocates node-local resources.
+The [topology manager](/docs/tasks/administer-cluster/topology-manager/) can take part in node-level
+resource allocation decisions. 
+
 {{% /capture %}}

content/en/docs/concepts/scheduling/kube-scheduler.md

@@ -183,6 +183,7 @@ kube-scheduler has a default set of scheduling policies.
 * Read about [scheduler performance tuning](/docs/concepts/scheduling/scheduler-perf-tuning/)
 * Read about [Pod topology spread constraints](/docs/concepts/workloads/pods/pod-topology-spread-constraints/)
 * Read the [reference documentation](/docs/reference/command-line-tools-reference/kube-scheduler/) for kube-scheduler
-* Learn about [configuring multiple schedulers](https://kubernetes.io/docs/tasks/administer-cluster/configure-multiple-schedulers/)
+* Learn about [configuring multiple schedulers](/docs/tasks/administer-cluster/configure-multiple-schedulers/)
+* Learn about [topology management policies](/docs/tasks/administer-cluster/topology-manager/)
 * Learn about [Pod Overhead](/docs/concepts/configuration/pod-overhead/)
 {{% /capture %}}

content/en/docs/reference/command-line-tools-reference/feature-gates.md

@@ -177,6 +177,7 @@ different Kubernetes components.
 | `TokenRequestProjection` | `false` | Alpha | 1.11 | 1.11 |
 | `TokenRequestProjection` | `true` | Beta | 1.12 | |
 | `TTLAfterFinished` | `false` | Alpha | 1.12 | |
+| `TopologyManager` | `false` | Alpha | 1.16 | |
 | `VolumePVCDataSource` | `false` | Alpha | 1.15 | 1.15 |
 | `VolumePVCDataSource` | `true` | Beta | 1.16 | |
 | `VolumeScheduling` | `false` | Alpha | 1.9 | 1.9 |

content/en/docs/tasks/administer-cluster/topology-manager.md

@@ -0,0 +1,153 @@
+---
+title: Control Topology Management Policies on a node
+reviewers:
+- ConnorDoyle
+- klueska
+- lmdaly
+- nolancon
+
+content_template: templates/task
+---
+
+{{% capture overview %}}
+
+{{< feature-state state="alpha" >}}
+
+An increasing number of systems leverage a combination of CPUs and hardware accelerators to support latency-critical execution and high-throughput parallel computation. These include workloads in fields such as telecommunications, scientific computing, machine learning, financial services and data analytics. Such hybrid systems comprise a high performance environment.
+
+In order to extract the best performance, optimizations related to CPU isolation, memory and device locality are required. However, in Kubernetes, these optimizations are handled by a disjoint set of components.
+
+_Topology Manager_ is a Kubelet component that aims to co-ordinate the set of components that are resposible for these optimizations.
+ 
+{{% /capture %}}
+
+{{% capture prerequisites %}}
+
+{{< include "task-tutorial-prereqs.md" >}} {{< version-check >}}
+
+{{% /capture %}}
+
+{{% capture steps %}}
+
+## How Topology Manager Works
+
+Prior to the introduction of Topology Manager, the CPU and Device Manager in Kubernetes make resource allocation decisions independently of each other.
+This can result in undesirable allocations on multiple-socketed systems, performance/latency sensitive applications will suffer due to these undesirable allocations. 
+ Undesirable in this case meaning for example, CPUs and devices being allocated from different NUMA Nodes thus, incurring additional latency.
+
+The Topology Manager is a Kubelet component, which acts as a source of truth so that other Kubelet components can make topology aligned resource allocation choices.
+
+The Topology Manager provides an interface for components, called *Hint Providers*, to send and receive topology information. Topology Manager has a set of node level policies which are explained below.
+
+The Topology manager receives Topology information from the *Hint Providers* as a bitmask denoting NUMA Nodes available and a preferred allocation indication. The Topology Manager polices preform a set of operations on the hints provided and converge on the hint determined by the policy to give the optimal result, if a undesirable hint is stored the preferred field for the hint will be set to false. In the current policies preferred is the narrowest preferred mask.
+The selected hint is stored as part of the Topology Manager. Depending on the policy configured the pod can be accepted or rejected from the node based on the selected hint.
+The hint is then stored in the Topology Manager for use by the *Hint Providers* when making the resource allocation decisions.
+
+### Topology Manager Policies
+
+The Topology Manager currently:
+
+ - Works on Nodes with the `static` CPU Manager Policy enabled. See [control CPU Management Policies](https://kubernetes.io/docs/tasks/administer-cluster/cpu-management-policies/)
+ - Works on Pods in the `Guaranteed` {{< glossary_tooltip text="QoS class" term_id="qos-class" >}}
+If these conditions are met, Topology Manager will align CPU and device requests.
+
+Topology Manager supports four allocation policies. You can set a policy via a Kubelet flag, `--topology-manager-policy`.
+There are four supported policies:
+
+* `none` (default)
+* `best-effort`
+* `restricted`
+* `single-numa-node`
+
+### none policy {#policy-none}
+
+This is the default policy and does not perform any topology alignment.
+
+### best-effort policy {#policy-best-effort}
+
+For each container in a Guaranteed Pod, kubelet, with `best-effort` topology 
+management policy, calls each Hint Provider to discover their resource availability.
+Using this information, the Topology Manager stores the 
+preferred NUMA Node affinity for that container. If the affinity is not preferred, 
+Topology Manager will store this and admit the pod to the node anyway.
+
+The *Hint Providers* can then use this information when making the 
+resource allocation decision.
+
+### restricted policy {#policy-restricted}
+
+For each container in a Guaranteed Pod, kubelet, with `restricted` topology 
+management policy, calls each Hint Provider to discover their resource availability.
+Using this information, the Topology Manager stores the 
+preferred NUMA Node affinity for that container. If the affinity is not preferred, 
+Topology Manager will reject this pod from the node. This will result in a pod in a `Terminated` state with a pod admission failure.
+
+If the pod is admitted, the *Hint Providers* can then use this information when making the 
+resource allocation decision.
+
+### single-numa-node policy {#policy-single-numa-node}
+
+For each container in a Guaranteed Pod, kubelet, with `single-numa-node` topology 
+management policy, calls each Hint Provider to discover their resource availability.
+Using this information, the Topology Manager determines if a single NUMA Node affinity is possible.
+If it is Topology Manager will store this and the *Hint Providers* can then use this information when making the 
+resource allocation decision.
+If, however, this is not possible the Topology Manager will reject the pod from the node. This will result in a pod in a `Terminated` state with a pod admission failure.
+
+
+### Pod Interactions with Topology Manager Policies
+
+Consider the containers in the following pod specs:
+
+```yaml
+spec:
+  containers:
+  - name: nginx
+    image: nginx
+```
+
+This pod runs in the `BestEffort` QoS class because no resource `requests` or
+`limits` are specified.
+
+```yaml
+spec:
+  containers:
+  - name: nginx
+    image: nginx
+    resources:
+      limits:
+        memory: "200Mi"
+      requests:
+        memory: "100Mi"
+```
+
+This pod runs in the `Burstable` QoS class because requests are less than limits.
+
+If the selected policy is anything other than `none` , Topology Manager would not consider either of these Pod
+specifications. 
+
+
+```yaml
+spec:
+  containers:
+  - name: nginx
+    image: nginx
+    resources:
+      limits:
+        memory: "200Mi"
+        cpu: "2"
+        example.com/device: "1"
+      requests:
+        memory: "200Mi"
+        cpu: "2"
+        example.com/device: "1"
+```
+
+This pod runs in the `Guaranteed` QoS class because `requests` are equal to `limits`.
+
+Topology Manager would consider this Pod. The Topology Manager consults the CPU Manager `static` policy, which returns the topology of available CPUs. 
+Topology Manager also consults Device Manager to discover the topology of available devices for example.com/device.
+
+Topology Manager will use this information to store the best Topology for this container. In the case of this Pod, CPU and Device Manager will use this stored information at the resource allocation stage.
+
+{{% /capture %}}

content/en/docs/tasks/configure-pod-container/quality-service-pod.md

@@ -261,9 +261,10 @@ kubectl delete namespace qos-example
 * [Configure a Pod Quota for a Namespace](/docs/tasks/administer-cluster/quota-pod-namespace/)
 
 * [Configure Quotas for API Objects](/docs/tasks/administer-cluster/quota-api-object/)
+
+* [Control Topology Management policies on a node](/docs/tasks/administer-cluster/topology-manager/)
 {{% /capture %}}
 
 
 
 
-