Update architecture.md
This commit is contained in:
Brian Grant
GitHub
parent
7a444fa1cc
commit
ece19783f2
|
|
@ -1,6 +1,236 @@
|
|||
# Kubernetes architecture
|
||||
# Kubernetes Design and Architecture
|
||||
|
||||
Please see the [Kubernetes design overview](README.md).
|
||||
## Overview
|
||||
|
||||
Kubernetes is production-grade, open-source infrastructure for the deployment, scaling,
|
||||
management, and composition of application containers across clusters of hosts, inspired
|
||||
by [previous work at Google](https://research.google.com/pubs/pub44843.html). Kubernetes
|
||||
is more than just a “container orchestrator”. It aims to eliminate the burden of orchestrating
|
||||
physical/virtual compute, network, and storage infrastructure, and enable application operators
|
||||
and developers to focus entirely on container-centric primitives for self-service operation.
|
||||
Kubernetes also provides a stable, portable foundation (a platform) for building customized
|
||||
workflows and higher-level automation.
|
||||
|
||||
Kubernetes is primarily targeted at applications composed of multiple containers. It therefore
|
||||
groups containers using *pods* and *labels* into tightly coupled and loosely coupled formations
|
||||
for easy management and discovery.
|
||||
|
||||
## Scope
|
||||
|
||||
Kubernetes is a [platform for deploying and managing containers]
|
||||
(https://kubernetes.io/docs/whatisk8s/). Kubernetes provides a container runtime, container
|
||||
orchestration, container-centric infrastructure orchestration, self-healing mechanisms such as health checking and re-scheduling, and service discovery and load balancing.
|
||||
|
||||
Kubernetes aspires to be an extensible, pluggable, building-block OSS
|
||||
platform and toolkit. Therefore, architecturally, we want Kubernetes to be built
|
||||
as a collection of pluggable components and layers, with the ability to use
|
||||
alternative schedulers, controllers, storage systems, and distribution
|
||||
mechanisms, and we're evolving its current code in that direction. Furthermore,
|
||||
we want others to be able to extend Kubernetes functionality, such as with
|
||||
higher-level PaaS functionality or multi-cluster layers, without modification of
|
||||
core Kubernetes source. Therefore, its API isn't just (or even necessarily
|
||||
mainly) targeted at end users, but at tool and extension developers. Its APIs
|
||||
are intended to serve as the foundation for an open ecosystem of tools,
|
||||
automation systems, and higher-level API layers. Consequently, there are no
|
||||
"internal" inter-component APIs. All APIs are visible and available, including
|
||||
the APIs used by the scheduler, the node controller, the replication-controller
|
||||
manager, Kubelet's API, etc. There's no glass to break -- in order to handle
|
||||
more complex use cases, one can just access the lower-level APIs in a fully
|
||||
transparent, composable manner.
|
||||
|
||||
## Goals
|
||||
|
||||
The project is committed to the following (aspirational) [design ideals](principles.md):
|
||||
* _Portable_. Kubernetes runs everywhere -- public cloud, private cloud, bare metal, laptop --
|
||||
with consistent behavior so that applications and tools are portable throughout the ecosystem
|
||||
as well as between development and production environments.
|
||||
* _General-purpose_. Kubernetes should run all major categories of workloads to enable you to run
|
||||
all of your workloads on a single infrastructure, stateless and stateful, microservices and
|
||||
monoliths, services and batch, greenfield and legacy.
|
||||
* _Meet users partway_. Kubernetes doesn’t just cater to purely greenfield cloud-native
|
||||
applications, nor does it meet all users where they are. It focuses on deployment and management
|
||||
of microservices and cloud-native applications, but provides some mechanisms to facilitate
|
||||
migration of monolithic and legacy applications.
|
||||
* _Flexible_. Kubernetes functionality can be consumed a la carte and (in most cases) Kubernetes
|
||||
does not prevent you from using your own solutions in lieu of built-in functionality.
|
||||
* _Extensible_. Kubernetes enables you to integrate it into your environment and to add the
|
||||
additional capabilities you need, by exposing the same interfaces used by built-in
|
||||
functionality.
|
||||
* _Automatable_. Kubernetes aims to dramatically reduce the burden of manual operations. It
|
||||
supports both declarative control by specifying users’ desired intent via its API, as well as
|
||||
imperative control to support higher-level orchestration and automation. The declarative
|
||||
approach is key to the system’s self-healing and autonomic capabilities.
|
||||
* _Advance the state of the art_. While Kubernetes intends to support non-cloud-native
|
||||
applications, it also aspires to advance the cloud-native and DevOps state of the art, such as
|
||||
in the [participation of applications in their own management]
|
||||
(http://blog.kubernetes.io/2016/09/cloud-native-application-interfaces.html). However, in doing
|
||||
so, we strive not to force applications to lock themselves into Kubernetes APIs, which is, for
|
||||
example, why we prefer configuration over convention in the [downward API]
|
||||
(https://kubernetes.io/docs/user-guide/downward-api/). Additionally, Kubernetes is not bound by
|
||||
the lowest common denominator of systems upon which it depends, such as container runtimes and
|
||||
cloud providers. An example where we pushed the envelope of what was achievable was in its [IP
|
||||
per Pod networking model](https://kubernetes.io/docs/admin/networking/#kubernetes-model).
|
||||
|
||||
## Architecture
|
||||
|
||||
A running Kubernetes cluster contains node agents (kubelet) and a cluster control plane (AKA
|
||||
*master*), with cluster state backed by a distributed storage system
|
||||
([etcd](https://github.com/coreos/etcd)).
|
||||
|
||||
### Cluster control plane (AKA *master*)
|
||||
|
||||
The Kubernetes [control plane](https://en.wikipedia.org/wiki/Control_plane) is split
|
||||
into a set of components, which can all run on a single *master* node, or can be replicated
|
||||
in order to support high-availability clusters, or can even be run on Kubernetes itself (AKA
|
||||
[self-hosted](self-hosted-kubernetes.md#what-is-self-hosted)).
|
||||
|
||||
Kubernetes provides a REST API supporting primarily CRUD operations on (mostly) persistent resources as the nucleus of its control plane. Kubernetes’s API provides IaaS-like
|
||||
container-centric primitives such as [Pods](https://kubernetes.io/docs/user-guide/pods/),
|
||||
[Services](https://kubernetes.io/docs/user-guide/services/), and [Ingress]
|
||||
(https://kubernetes.io/docs/user-guide/ingress/), and also lifecycle APIs to support orchestration
|
||||
(self-healing, scaling, updates, termination) of common types of workloads, such as [ReplicaSet]
|
||||
(https://kubernetes.io/docs/user-guide/replicasets/) (simple fungible/stateless app manager),
|
||||
[Deployment](https://kubernetes.io/docs/user-guide/deployments/) (orchestrates updates of
|
||||
stateless apps), [Job](https://kubernetes.io/docs/user-guide/jobs/) (batch), [CronJob]
|
||||
(https://kubernetes.io/docs/user-guide/cron-jobs/) (cron), [DaemonSet]
|
||||
(https://kubernetes.io/docs/admin/daemons/) (cluster services), and [StatefulSet]
|
||||
(https://kubernetes.io/docs/concepts/abstractions/controllers/statefulsets/) (stateful apps).
|
||||
We deliberately decoupled service naming/discovery and load balancing from application
|
||||
implementation, since the latter is diverse and open-ended.
|
||||
|
||||
Both user clients and components containing asynchronous controllers interact with the same API resources, which serve as coordination points, common intermediate representation, and shared state. Most resources contain metadata, including [labels](https://kubernetes.io/docs/user-guide/labels/) and [annotations](https://kubernetes.io/docs/user-guide/annotations/), fully elaborated desired state (spec), including default values, and observed state (status).
|
||||
|
||||
Controllers work continuously to drive the actual state towards the desired state, while reporting back the currently observed state for users and for other controllers.
|
||||
|
||||
While the controllers are [level-based]
|
||||
(http://gengnosis.blogspot.com/2007/01/level-triggered-and-edge-triggered.html) to maximize fault
|
||||
tolerance, they typically `watch` for changes to relevant resources in order to minimize reaction
|
||||
latency and redundant work. This enables decentralized and decoupled
|
||||
[choreography-like](https://en.wikipedia.org/wiki/Service_choreography) coordination without a
|
||||
message bus.
|
||||
|
||||
#### API Server
|
||||
|
||||
The [API server](https://kubernetes.io/docs/admin/kube-apiserver/) serves up the
|
||||
[Kubernetes API](https://kubernetes.io/docs/api/). It is intended to be a relatively simple
|
||||
server, with most/all business logic implemented in separate components or in plug-ins. It mainly
|
||||
processes REST operations, validates them, and updates the corresponding objects in `etcd` (and
|
||||
perhaps eventually other stores). Note that, for a number of reasons, Kubernetes deliberately does
|
||||
not support atomic transactions across multiple resources.
|
||||
|
||||
Kubernetes cannot function without this basic API machinery, which includes:
|
||||
* REST semantics, watch, durability and consistency guarantees, API versioning, defaulting, and
|
||||
validation
|
||||
* Built-in admission-control semantics, synchronous admission-control hooks, and asynchronous
|
||||
resource initialization
|
||||
* API registration and discovery
|
||||
|
||||
Additionally, the API server acts as the gateway to the cluster. By definition, the API server
|
||||
must be accessible by clients from outside the cluster, whereas the nodes, and certainly
|
||||
containers, may not be. Clients authenticate the API server and also use it as a bastion and
|
||||
proxy/tunnel to nodes and pods (and services).
|
||||
|
||||
#### Cluster state store
|
||||
|
||||
All persistent cluster state is stored in an instance of `etcd`. This provides a way to store
|
||||
configuration data reliably. With `watch` support, coordinating components can be notified very
|
||||
quickly of changes.
|
||||
|
||||
|
||||
#### Controller-Manager Server
|
||||
|
||||
Most other cluster-level functions are currently performed by a separate process, called the
|
||||
[Controller Manager](https://kubernetes.io/docs/admin/kube-controller-manager/). It performs
|
||||
both lifecycle functions (e.g., namespace creation and lifecycle, event garbage collection,
|
||||
terminated-pod garbage collection, cascading-deletion garbage collection, node garbage collection)
|
||||
and API business logic (e.g., scaling of pods controlled by a [ReplicaSet]
|
||||
(https://kubernetes.io/docs/user-guide/replicasets/)).
|
||||
|
||||
The application management and composition layer, providing self-healing, scaling, application lifecycle management, service discovery, routing, and service binding and provisioning.
|
||||
|
||||
These functions may eventually be split into separate components to make them more easily
|
||||
extended or replaced.
|
||||
|
||||
#### Scheduler
|
||||
|
||||
|
||||
Kubernetes enables users to ask a cluster to run a set of containers. The scheduler
|
||||
component automatically chooses hosts to run those containers on.
|
||||
|
||||
The scheduler watches for unscheduled pods and binds them to nodes via the `/binding` pod
|
||||
subresource API, according to the availability of the requested resources, quality of service
|
||||
requirements, affinity and anti-affinity specifications, and other constraints.
|
||||
|
||||
Kubernetes supports user-provided schedulers and multiple concurrent cluster schedulers,
|
||||
using the shared-state approach pioneered by [Omega]
|
||||
(https://research.google.com/pubs/pub41684.html). In addition to the disadvantages of
|
||||
pessimistic concurrency described by the Omega paper, [two-level scheduling models]
|
||||
(http://mesos.berkeley.edu/mesos_tech_report.pdf) that hide information from the upper-level
|
||||
schedulers need to implement all of the same features in the lower-level scheduler as required by
|
||||
all upper-layer schedulers in order to ensure that their scheduling requests can be satisfied by
|
||||
available desired resources.
|
||||
|
||||
|
||||
### The Kubernetes Node
|
||||
|
||||
The Kubernetes node has the services necessary to run application containers and
|
||||
be managed from the master systems.
|
||||
|
||||
Each node runs a container runtime (like docker, rkt, or Hyper). The container
|
||||
runtime is responsible for downloading images and running containers.
|
||||
|
||||
#### Kubelet
|
||||
|
||||
|
||||
The most important and most prominent controller in Kubernetes is the Kubelet, which is the
|
||||
primary implementer of the Pod and Node APIs that drive the container execution layer. Without
|
||||
these APIs, Kubernetes would just be a CRUD-oriented REST application framework backed by a
|
||||
key-value store (and perhaps the API machinery will eventually be spun out as an independent
|
||||
project).
|
||||
|
||||
Kubernetes executes isolated application containers as its default, native mode of execution, as
|
||||
opposed to processes and traditional operating-system packages. Not only are application
|
||||
containers isolated from each other, but they are also isolated from the hosts on which they
|
||||
execute, which is critical to decoupling management of individual applications from each other and
|
||||
from management of the underlying cluster physical/virtual infrastructure.
|
||||
|
||||
Kubernetes provides [Pods](https://kubernetes.io/docs/user-guide/pods/) that can host multiple
|
||||
containers and storage volumes as its fundamental execution primitive in order to facilitate
|
||||
packaging a single application per container, decoupling deployment-time concerns from build-time
|
||||
concerns, and migration from physical/virtual machines. The Pod primitive is key to glean the
|
||||
[primary benefits](https://kubernetes.io/docs/whatisk8s/#why-containers) of deployment on modern
|
||||
cloud platforms, such as Kubernetes.
|
||||
|
||||
#### Container runtime
|
||||
|
||||
TODO
|
||||
|
||||
#### `kube-proxy`
|
||||
|
||||
The [service](https://kubernetes.io/docs/user-guide/services/) abstraction provides a way to
|
||||
group pods under a common access policy (e.g., load-balanced). The implementation of this creates
|
||||
A virtual IP which clients can access and which is transparently proxied to the pods in a Service.
|
||||
Each node runs a [kube-proxy](https://kubernetes.io/docs/admin/kube-proxy/) process which programs
|
||||
`iptables` rules to trap access to service IPs and redirect them to the correct backends. This provides a highly-available load-balancing solution with low performance overhead by balancing
|
||||
client traffic from a node on that same node.
|
||||
|
||||
Service endpoints are found primarily via [DNS](https://kubernetes.io/docs/admin/dns/).
|
||||
|
||||
### Add-ons and other dependencies
|
||||
|
||||
A number of components, called [*add-ons*]
|
||||
(https://github.com/kubernetes/kubernetes/tree/master/cluster/addons) typically run on Kubernetes
|
||||
itself:
|
||||
* [DNS](https://github.com/kubernetes/kubernetes/tree/master/cluster/addons/dns)
|
||||
* [Ingress controller](https://github.com/kubernetes/ingress/tree/master/controllers)
|
||||
* [Heapster](https://github.com/kubernetes/heapster/) (resource monitoring)
|
||||
* [Dashboard](https://github.com/kubernetes/dashboard/) (GUI)
|
||||
|
||||
### Federation
|
||||
|
||||
A single Kubernetes cluster may span multiple availability zones.
|
||||
|
||||
However, for the highest availability, we recommend using [cluster federation](federation.md).
|
||||
|
||||
<!-- BEGIN MUNGE: GENERATED_ANALYTICS -->
|
||||
[]()
|
||||
|
|
|
|||
Loading…
Reference in New Issue