Add concept page about Controllers (#15733)

* Add controller concept

* Fix controller glossary entries

* Link from Understanding Kubernetes Objects to Controllers

content/en/docs/concepts/_index.md

@@ -17,7 +17,7 @@ The Concepts section helps you learn about the parts of the Kubernetes system an
 
 To work with Kubernetes, you use *Kubernetes API objects* to describe your cluster's *desired state*: what applications or other workloads you want to run, what container images they use, the number of replicas, what network and disk resources you want to make available, and more. You set your desired state by creating objects using the Kubernetes API, typically via the command-line interface, `kubectl`. You can also use the Kubernetes API directly to interact with the cluster and set or modify your desired state.
 
-Once you've set your desired state, the *Kubernetes Control Plane* makes the cluster's current state match the desired state via the Pod Lifecycle Event Generator (PLEG). To do so, Kubernetes performs a variety of tasks automatically--such as starting or restarting containers, scaling the number of replicas of a given application, and more. The Kubernetes Control Plane consists of a collection of processes running on your cluster: 
+Once you've set your desired state, the *Kubernetes Control Plane* makes the cluster's current state match the desired state via the Pod Lifecycle Event Generator (PLEG). To do so, Kubernetes performs a variety of tasks automatically--such as starting or restarting containers, scaling the number of replicas of a given application, and more. The Kubernetes Control Plane consists of a collection of processes running on your cluster:
 
 * The **Kubernetes Master** is a collection of three processes that run on a single node in your cluster, which is designated as the master node. Those processes are: [kube-apiserver](/docs/admin/kube-apiserver/), [kube-controller-manager](/docs/admin/kube-controller-manager/) and [kube-scheduler](/docs/admin/kube-scheduler/).
 * Each individual non-master node in your cluster runs two processes:
@@ -26,7 +26,7 @@ Once you've set your desired state, the *Kubernetes Control Plane* makes the clu
 
 ## Kubernetes Objects
 
-Kubernetes contains a number of abstractions that represent the state of your system: deployed containerized applications and workloads, their associated network and disk resources, and other information about what your cluster is doing. These abstractions are represented by objects in the Kubernetes API; see the [Kubernetes Objects overview](/docs/concepts/abstractions/overview/) for more details. 
+Kubernetes contains a number of abstractions that represent the state of your system: deployed containerized applications and workloads, their associated network and disk resources, and other information about what your cluster is doing. These abstractions are represented by objects in the Kubernetes API. See [Understanding Kubernetes Objects](/docs/concepts/overview/working-with-objects/kubernetes-objects/) for more details.
 
 The basic Kubernetes objects include:
 
@@ -35,12 +35,12 @@ The basic Kubernetes objects include:
 * [Volume](/docs/concepts/storage/volumes/)
 * [Namespace](/docs/concepts/overview/working-with-objects/namespaces/)
 
-In addition, Kubernetes contains a number of higher-level abstractions called Controllers. Controllers build upon the basic objects, and provide additional functionality and convenience features. They include:
+Kubernetes also contains higher-level abstractions that rely on [Controllers](/docs/concepts/architecture/controller/) to build upon the basic objects, and provide additional functionality and convenience features. These include:
 
-* [ReplicaSet](/docs/concepts/workloads/controllers/replicaset/)
 * [Deployment](/docs/concepts/workloads/controllers/deployment/)
-* [StatefulSet](/docs/concepts/workloads/controllers/statefulset/)
 * [DaemonSet](/docs/concepts/workloads/controllers/daemonset/)
+* [StatefulSet](/docs/concepts/workloads/controllers/statefulset/)
+* [ReplicaSet](/docs/concepts/workloads/controllers/replicaset/)
 * [Job](/docs/concepts/workloads/controllers/jobs-run-to-completion/)
 
 ## Kubernetes Control Plane

content/en/docs/concepts/architecture/cloud-controller.md

@@ -1,7 +1,7 @@
 ---
 title: Concepts Underlying the Cloud Controller Manager
 content_template: templates/concept
-weight: 30
+weight: 40
 ---
 
 {{% capture overview %}}

content/en/docs/concepts/architecture/controller.md

@@ -0,0 +1,162 @@
+---
+title: Controllers
+content_template: templates/concept
+weight: 30
+---
+
+{{% capture overview %}}
+
+In robotics and automation, a _control loop_ is
+a non-terminating loop that regulates the state of a system.
+
+Here is one example of a control loop: a thermostat in a room.
+
+When you set the temperature, that's telling the thermostat
+about your *desired state*. The actual room temperature is the
+*current state*. The thermostat acts to bring the current state
+closer to the desired state, by turning equipment on or off.
+
+{{< glossary_definition term_id="controller" length="short">}}
+
+{{% /capture %}}
+
+
+{{% capture body %}}
+
+## Controller pattern
+
+A controller tracks at least one Kubernetes resource type.
+These [objects](/docs/concepts/overview/working-with-objects/kubernetes-objects/)
+have a spec field that represents the desired state. The
+controller(s) for that resource are responsible for making the current
+state come closer to that desired state.
+
+The controller might carry the action out itself; more commonly, in Kubernetes,
+a controller will send messages to the
+{{< glossary_tooltip text="API server" term_id="kube-apiserver" >}} that have
+useful side effects. You'll see examples of this below.
+
+{{< comment >}}
+Some built-in controllers, such as the namespace controller, act on objects
+that do not have a spec. For simplicity, this page omits explaining that
+detail.
+{{< /comment >}}
+
+### Control via API server
+
+The {{< glossary_tooltip term_id="job" >}} controller is an example of a
+Kubernetes built-in controller. Built-in controllers manage state by
+interacting with the cluster API server.
+
+Job is a Kubernetes resource that runs a
+{{< glossary_tooltip term_id="pod" >}}, or perhaps several Pods, to carry out
+a task and then stop.
+
+(Once [scheduled](/docs/concepts/scheduling/), Pod objects become part of the
+desired state for a kubelet).
+
+When the Job controller sees a new task it makes sure that, somewhere
+in your cluster, the kubelets on a set of Nodes are running the right
+number of Pods to get the work done.
+The Job controller does not run any Pods or containers
+itself. Instead, the Job controller tells the API server to create or remove
+Pods.
+Other components in the
+{{< glossary_tooltip text="control plane" term_id="control-plane" >}}
+act on the new information (there are new Pods to schedule and run),
+and eventually the work is done.
+
+After you create a new Job, the desired state is for that Job to be completed.
+The Job controller makes the current state for that Job be nearer to your
+desired state: creating Pods that do the work you wanted for that Job, so that
+the Job is closer to completion.
+
+Controllers also update the objects that configure them.
+For example: once the work is done for a Job, the Job controller
+updates that Job object to mark it `Finished`.
+
+(This is a bit like how some thermostats turn a light off to
+indicate that the your room is now at the temperature you set).
+
+### Direct control
+
+By contrast with Job, some controllers need to make changes to
+things outside of your cluster.
+
+For example, if you use a control loop to make sure there
+are enough {{< glossary_tooltip text="Nodes" term_id="node" >}}
+in your cluster, then that controller needs something outside the
+current cluster to set up new Nodes when needed.
+
+Controllers that interact with external state find their desired state from
+the API server, then communicate directly with an external system to bring
+the current state closer in line.
+
+(There actually is a controller that horizontally scales the
+nodes in your cluster. See
+[Cluster autoscaling](https://kubernetes.io/docs/tasks/administer-cluster/cluster-management/#cluster-autoscaling)).
+
+## Desired versus current state {#desired-vs-current}
+
+Kubernetes takes a cloud-native view of systems, and is able to handle
+constant change.
+
+Your cluster could be changing at any point as work happens and
+control loops automatically fix failures. This means that,
+potentially, your cluster never reaches a stable state.
+
+As long as the controllers for your cluster are running and able to make
+useful changes, it doesn't matter if the overall state is or is not stable.
+
+## Design
+
+As a tenet of its design, Kubernetes uses lots of controllers that each manage
+a particular aspect of cluster state. Most commonly, a particular control loop
+(controller) uses one kind of resource as its desired state, and has a different
+kind of resource that it manages to make that desired state happen.
+
+It's useful to have simple controllers rather than one, monolithic set of control
+loops that are interlinked. Controllers can fail, so Kubernetes is designed to
+allow for that.
+
+For example: a controller for Jobs tracks Job objects (to discover
+new work) and Pod object (to run the Jobs, and then to see when the work is
+finished). In this case something else creates the Jobs, whereas the Job
+controller creates Pods.
+
+{{< note >}}
+There can be several controllers that create or update the same kind of object.
+Behind the scenes, Kubernetes controllers make sure that they only pay attention
+to the resources linked to their controlling resource.
+
+For example, you can have Deployments and Jobs; these both create Pods.
+The Job controller does not delete the Pods that your Deployment created,
+because there is information ({{< glossary_tooltip term_id="label" text="labels" >}})
+the controllers can use to tell those Pods apart.
+{{< /note >}}
+
+## Ways of running controllers {#running-controllers}
+
+Kubernetes comes with a set of built-in controllers that run inside
+the {{< glossary_tooltip term_id="kube-controller-manager" >}}. These
+built-in controllers provide important core behaviors.
+
+The Deployment controller and Job controller are examples of controllers that
+come as part of Kubernetes itself (“built-in” controllers).
+Kubernetes lets you run a resilient control plane, so that if any of the built-in
+controllers were to fail, another part of the control plane will take over the work.
+
+You can find controllers that run outside the control plane, to extend Kubernetes.
+Or, if you want, you can write a new controller yourself.
+You can run your own controller as a set of Pods,
+or externally to Kubernetes. What fits best will depend on what that particular
+controller does.
+
+{{% /capture %}}
+
+{{% capture whatsnext %}}
+* Read about the [Kubernetes control plane](https://kubernetes.io/docs/concepts/#kubernetes-control-plane)
+* Discover some of the basic [Kubernetes objects](https://kubernetes.io/docs/concepts/#kubernetes-objects)
+* Learn more about the [Kubernetes API](/docs/concepts/overview/kubernetes-api/)
+* If you want to write your own controller, see [Extension Patterns](/docs/concepts/extend-kubernetes/extend-cluster/#extension-patterns) in Extending Kubernetes.
+{{% /capture %}}

content/en/docs/concepts/overview/components.md

@@ -115,6 +115,7 @@ saving container logs to a central log store with search/browsing interface.
 {{% /capture %}}
 {{% capture whatsnext %}}
 * Learn about [Nodes](/docs/concepts/architecture/nodes/)
+* Learn about [Controllers](/docs/concepts/architecture/controller/)
 * Learn about [kube-scheduler](/docs/concepts/scheduling/kube-scheduler/)
 * Read etcd's official [documentation](https://etcd.io/docs/)
 {{% /capture %}}

content/en/docs/concepts/overview/working-with-objects/kubernetes-objects.md

@@ -73,6 +73,7 @@ and the `spec` format for a `Deployment` can be found
 
 {{% capture whatsnext %}}
 * Learn about the most important basic Kubernetes objects, such as [Pod](/docs/concepts/workloads/pods/pod-overview/).
+* Learn about [controllers](/docs/concepts/architecture/controller/) in Kubernetes
 {{% /capture %}}
 
 

content/en/docs/concepts/workloads/controllers/cron-jobs.md

@@ -36,7 +36,7 @@ If `startingDeadlineSeconds` is set to a large value or left unset (the default)
 and if `concurrencyPolicy` is set to `Allow`, the jobs will always run
 at least once.
 
-For every CronJob, the CronJob controller checks how many schedules it missed in the duration from its last scheduled time until now. If there are more than 100 missed schedules, then it does not start the job and logs the error
+For every CronJob, the CronJob {{< glossary_tooltip term_id="controller" >}} checks how many schedules it missed in the duration from its last scheduled time until now. If there are more than 100 missed schedules, then it does not start the job and logs the error
 
 ````
 Cannot determine if job needs to be started. Too many missed start time (> 100). Set or decrease .spec.startingDeadlineSeconds or check clock skew.

content/en/docs/concepts/workloads/controllers/daemonset.md

@@ -86,10 +86,10 @@ When the two are specified the result is ANDed.
 If the `.spec.selector` is specified, it must match the `.spec.template.metadata.labels`. Config with these not matching will be rejected by the API.
 
 Also you should not normally create any Pods whose labels match this selector, either directly, via
-another DaemonSet, or via other controller such as ReplicaSet.  Otherwise, the DaemonSet
-controller will think that those Pods were created by it.  Kubernetes will not stop you from doing
-this.  One case where you might want to do this is manually create a Pod with a different value on
-a node for testing.
+another DaemonSet, or via another workload resource such as ReplicaSet.  Otherwise, the DaemonSet
+{{< glossary_tooltip term_id="controller" >}} will think that those Pods were created by it.
+Kubernetes will not stop you from doing this. One case where you might want to do this is manually
+create a Pod with a different value on a node for testing.
 
 ### Running Pods on Only Some Nodes
 

content/en/docs/concepts/workloads/controllers/deployment.md

@@ -13,10 +13,10 @@ weight: 30
 
 {{% capture overview %}}
 
-A _Deployment_ controller provides declarative updates for [Pods](/docs/concepts/workloads/pods/pod/) and
+A _Deployment_ provides declarative updates for [Pods](/docs/concepts/workloads/pods/pod/) and
 [ReplicaSets](/docs/concepts/workloads/controllers/replicaset/).
 
-You describe a _desired state_ in a Deployment, and the Deployment controller changes the actual state to the desired state at a controlled rate. You can define Deployments to create new ReplicaSets, or to remove existing Deployments and adopt all their resources with new Deployments.
+You describe a _desired state_ in a Deployment, and the Deployment {{< glossary_tooltip term_id="controller" >}} changes the actual state to the desired state at a controlled rate. You can define Deployments to create new ReplicaSets, or to remove existing Deployments and adopt all their resources with new Deployments.
 
 {{< note >}}
 Do not manage ReplicaSets owned by a Deployment. Consider opening an issue in the main Kubernetes repository if your use case is not covered below.

content/en/docs/concepts/workloads/controllers/jobs-run-to-completion.md

@@ -173,10 +173,10 @@ parallelism, for a variety of reasons:
 - For _fixed completion count_ Jobs, the actual number of pods running in parallel will not exceed the number of
   remaining completions.   Higher values of `.spec.parallelism` are effectively ignored.
 - For _work queue_ Jobs, no new Pods are started after any Pod has succeeded -- remaining Pods are allowed to complete, however.
-- If the controller has not had time to react.
-- If the controller failed to create Pods for any reason (lack of `ResourceQuota`, lack of permission, etc.),
+- If the Job {{< glossary_tooltip term_id="controller" >}} has not had time to react.
+- If the Job controller failed to create Pods for any reason (lack of `ResourceQuota`, lack of permission, etc.),
   then there may be fewer pods than requested.
-- The controller may throttle new Pod creation due to excessive previous pod failures in the same Job.
+- The Job controller may throttle new Pod creation due to excessive previous pod failures in the same Job.
 - When a Pod is gracefully shut down, it takes time to stop.
 
 ## Handling Pod and Container Failures

content/en/docs/concepts/workloads/controllers/replicaset.md

@@ -32,7 +32,7 @@ ReplicaSet's identifying information within their ownerReferences field. It's th
 knows of the state of the Pods it is maintaining and plans accordingly.
 
 A ReplicaSet identifies new Pods to acquire by using its selector. If there is a Pod that has no OwnerReference or the
-OwnerReference is not a controller and it matches a ReplicaSet's selector, it will be immediately acquired by said
+OwnerReference is not a {{< glossary_tooltip term_id="controller" >}} and it matches a ReplicaSet's selector, it will be immediately acquired by said
 ReplicaSet.
 
 ## When to use a ReplicaSet

content/en/docs/concepts/workloads/controllers/statefulset.md

@@ -32,8 +32,8 @@ following.
 
 In the above, stable is synonymous with persistence across Pod (re)scheduling.
 If an application doesn't require any stable identifiers or ordered deployment,
-deletion, or scaling, you should deploy your application with a controller that
-provides a set of stateless replicas. Controllers such as
+deletion, or scaling, you should deploy your application using a workload object
+that provides a set of stateless replicas.
 [Deployment](/docs/concepts/workloads/controllers/deployment/) or
 [ReplicaSet](/docs/concepts/workloads/controllers/replicaset/) may be better suited to your stateless needs.
 
@@ -164,8 +164,9 @@ This must be done manually.
 
 ### Pod Name Label
 
-When the StatefulSet controller creates a Pod, it adds a label, `statefulset.kubernetes.io/pod-name`, 
-that is set to the name of the Pod. This label allows you to attach a Service to a specific Pod in 
+When the StatefulSet {{< glossary_tooltip term_id="controller" >}} creates a Pod,
+it adds a label, `statefulset.kubernetes.io/pod-name`, that is set to the name of
+the Pod. This label allows you to attach a Service to a specific Pod in
 the StatefulSet.
 
 ## Deployment and Scaling Guarantees

content/en/docs/reference/glossary/controller.md

@@ -2,7 +2,7 @@
 title: Controller
 id: controller
 date: 2018-04-12
-full_link: /docs/admin/kube-controller-manager/
+full_link: /docs/concepts/architecture/controller/
 short_description: >
   A control loop that watches the shared state of the cluster through the apiserver and makes changes attempting to move the current state towards the desired state.
 
@@ -11,9 +11,20 @@ tags:
 - architecture
 - fundamental
 ---
- A control loop that watches the shared state of the cluster through the {{< glossary_tooltip text="apiserver" term_id="kube-apiserver" >}} and makes changes attempting to move the current state towards the desired state.
+In Kubernetes, controllers are control loops that watch the state of your
+{{< glossary_tooltip term_id="cluster" text="cluster">}}, then make or request
+changes where needed.
+Each controller tries to move the current cluster state closer to the desired
+state.
 
-<!--more--> 
+<!--more-->
 
-Examples of controllers that ship with Kubernetes today are the replication controller, endpoints controller, namespace controller, and serviceaccounts controller.
+Controllers watch the shared state of your cluster through the
+{{< glossary_tooltip text="apiserver" term_id="kube-apiserver" >}} (part of the
+{{< glossary_tooltip term_id="control-plane" >}}).
 
+Some controllers also run inside the control plane, providing control loops that
+are core to Kubernetes' operations. For example: the deployment controller, the
+daemonset controller, the namespace controller, and the persistent volume
+controller (and others) all run within the
+{{< glossary_tooltip term_id="kube-controller-manager" >}}.

content/en/docs/reference/glossary/kube-controller-manager.md

@@ -2,7 +2,7 @@
 title: kube-controller-manager
 id: kube-controller-manager
 date: 2018-04-12
-full_link: /docs/reference/generated/kube-controller-manager/
+full_link: /docs/reference/command-line-tools-reference/kube-controller-manager/
 short_description: >
   Component on the master that runs controllers.
 
@@ -13,7 +13,6 @@ tags:
 ---
  Component on the master that runs {{< glossary_tooltip text="controllers" term_id="controller" >}}.
 
-<!--more--> 
+<!--more-->
 
 Logically, each {{< glossary_tooltip text="controller" term_id="controller" >}} is a separate process, but to reduce complexity, they are all compiled into a single binary and run in a single process.
-

content/en/docs/reference/glossary/statefulset.md

@@ -18,6 +18,3 @@ tags:
 <!--more--> 
 
 Like a {{< glossary_tooltip term_id="deployment" >}}, a StatefulSet manages Pods that are based on an identical container spec. Unlike a Deployment, a StatefulSet maintains a sticky identity for each of their Pods. These pods are created from the same spec, but are not interchangeable&#58; each has a persistent identifier that it maintains across any rescheduling.
-
-A StatefulSet operates under the same pattern as any other Controller. You define your desired state in a StatefulSet *object*, and the StatefulSet *controller* makes any necessary updates to get there from the current state.
-