kubernetes
/
website
mirror of https://github.com/kubernetes/website.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #5 from kubernetes/master 

merge
This commit is contained in:
Yong Zhang 2020-08-29 14:30:02 +08:00 committed by GitHub
parent 73e4ccc828 94b2f857bb
commit e8d464b800
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1063 changed files with 100887 additions and 25802 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
OWNERS_ALIASES
View File

@ -107,10 +107,12 @@ aliases:
- danninov
sig-docs-it-owners: # Admins for Italian content
- fabriziopandini
- Fale
- mattiaperi
- micheleberardi
sig-docs-it-reviews: # PR reviews for Italian content
- fabriziopandini
- Fale
- mattiaperi
- micheleberardi
sig-docs-ja-owners: # Admins for Japanese content
@ -138,6 +140,7 @@ aliases:
- ianychoi
- seokho-son
- ysyukr
- pjhwa
sig-docs-leads: # Website chairs and tech leads
- jimangel
- kbarnard10
@ -214,4 +217,4 @@ aliases:
- butuzov
- idvoretskyi
- MaxymVlasov
- Potapy4
- Potapy4

95
README.md
View File

@ -4,37 +4,98 @@
This repository contains the assets required to build the [Kubernetes website and documentation](https://kubernetes.io/). We're glad that you want to contribute!
## Running the website locally using Hugo
# Using this repository
See the [official Hugo documentation](https://gohugo.io/getting-started/installing/) for Hugo installation instructions. Make sure to install the Hugo extended version specified by the `HUGO_VERSION` environment variable in the [`netlify.toml`](netlify.toml#L10) file.
You can run the website locally using Hugo, or you can run it in a container runtime. We strongly recommend using the container runtime, as it gives deployment consistency with the live website.
Before building the site, clone the Kubernetes website repository:
## Prerequisites
```bash
To use this repository, you need the following installed locally:
- [yarn](https://yarnpkg.com/)
- [npm](https://www.npmjs.com/)
- [Go](https://golang.org/)
- [Hugo](https://gohugo.io/)
- A container runtime, like [Docker](https://www.docker.com/).
Before you start, install the dependencies. Clone the repository and navigate to the directory:
```
git clone https://github.com/kubernetes/website.git
cd website
```
The Kubernetes website uses the [Docsy Hugo theme](https://github.com/google/docsy#readme). Even if you plan to run the website in a container, we strongly recommend pulling in the submodule and other development dependencies by running the following:
```
# install dependencies
yarn
# pull in the Docsy submodule
git submodule update --init --recursive --depth 1
```
**Note:** The Kubernetes website deploys the [Docsy Hugo theme](https://github.com/google/docsy#readme).
If you have not updated your website repository, the `website/themes/docsy` directory is empty. The site cannot build
without a local copy of the theme.
## Running the website using a container
Update the website theme:
To build the site in a container, run the following to build the container image and run it:
```bash
git submodule update --init --recursive --depth 1
```
make container-image
make container-serve
```
Open up your browser to http://localhost:1313 to view the website. As you make changes to the source files, Hugo updates the website and forces a browser refresh.
## Running the website locally using Hugo
Make sure to install the Hugo extended version specified by the `HUGO_VERSION` environment variable in the [`netlify.toml`](netlify.toml#L10) file.
To build and test the site locally, run:
```bash
hugo server --buildFuture
make serve
```
This will start the local Hugo server on port 1313. Open up your browser to http://localhost:1313 to view the website. As you make changes to the source files, Hugo updates the website and forces a browser refresh.
## Get involved with SIG Docs
### Troubleshooting macOS for too many open files
If you run `make serve` on macOS and receive the following error:
```
ERROR 2020/08/01 19:09:18 Error: listen tcp 127.0.0.1:1313: socket: too many open files
make: *** [serve] Error 1
```
Try checking the current limit for open files:
`launchctl limit maxfiles`
Then run the following commands:
```
#!/bin/sh
# These are the original gist links, linking to my gists now.
# curl -O https://gist.githubusercontent.com/a2ikm/761c2ab02b7b3935679e55af5d81786a/raw/ab644cb92f216c019a2f032bbf25e258b01d87f9/limit.maxfiles.plist
# curl -O https://gist.githubusercontent.com/a2ikm/761c2ab02b7b3935679e55af5d81786a/raw/ab644cb92f216c019a2f032bbf25e258b01d87f9/limit.maxproc.plist
curl -O https://gist.githubusercontent.com/tombigel/d503800a282fcadbee14b537735d202c/raw/ed73cacf82906fdde59976a0c8248cce8b44f906/limit.maxfiles.plist
curl -O https://gist.githubusercontent.com/tombigel/d503800a282fcadbee14b537735d202c/raw/ed73cacf82906fdde59976a0c8248cce8b44f906/limit.maxproc.plist
sudo mv limit.maxfiles.plist /Library/LaunchDaemons
sudo mv limit.maxproc.plist /Library/LaunchDaemons
sudo chown root:wheel /Library/LaunchDaemons/limit.maxfiles.plist
sudo chown root:wheel /Library/LaunchDaemons/limit.maxproc.plist
sudo launchctl load -w /Library/LaunchDaemons/limit.maxfiles.plist
```
This works for Catalina as well as Mojave macOS.
# Get involved with SIG Docs
Learn more about SIG Docs Kubernetes community and meetings on the [community page](https://github.com/kubernetes/community/tree/master/sig-docs#meetings).
@ -43,7 +104,7 @@ You can also reach the maintainers of this project at:
- [Slack](https://kubernetes.slack.com/messages/sig-docs)
- [Mailing List](https://groups.google.com/forum/#!forum/kubernetes-sig-docs)
## Contributing to the docs
# Contributing to the docs
You can click the **Fork** button in the upper-right area of the screen to create a copy of this repository in your GitHub account. This copy is called a *fork*. Make any changes you want in your fork, and when you are ready to send those changes to us, go to your fork and create a new pull request to let us know about it.
@ -60,7 +121,7 @@ For more information about contributing to the Kubernetes documentation, see:
* [Documentation Style Guide](https://kubernetes.io/docs/contribute/style/style-guide/)
* [Localizing Kubernetes Documentation](https://kubernetes.io/docs/contribute/localization/)
## Localization `README.md`'s
# Localization `README.md`'s
| Language | Language |
|---|---|
@ -72,10 +133,10 @@ For more information about contributing to the Kubernetes documentation, see:
|[Italian](README-it.md)|[Ukrainian](README-uk.md)|
|[Japanese](README-ja.md)|[Vietnamese](README-vi.md)|
## Code of conduct
# Code of conduct
Participation in the Kubernetes community is governed by the [CNCF Code of Conduct](https://github.com/cncf/foundation/blob/master/code-of-conduct.md).
## Thank you!
# Thank you!
Kubernetes thrives on community participation, and we appreciate your contributions to our website and our documentation!
Kubernetes thrives on community participation, and we appreciate your contributions to our website and our documentation!

22
SECURITY.md Normal file
View File

@ -0,0 +1,22 @@
# Security Policy
## Security Announcements
Join the [kubernetes-security-announce] group for security and vulnerability announcements.
You can also subscribe to an RSS feed of the above using [this link][kubernetes-security-announce-rss].
## Reporting a Vulnerability
Instructions for reporting a vulnerability can be found on the
[Kubernetes Security and Disclosure Information] page.
## Supported Versions
Information about supported Kubernetes versions can be found on the
[Kubernetes version and version skew support policy] page on the Kubernetes website.
[kubernetes-security-announce]: https://groups.google.com/forum/#!forum/kubernetes-security-announce
[kubernetes-security-announce-rss]: https://groups.google.com/forum/feed/kubernetes-security-announce/msgs/rss_v2_0.xml?num=50
[Kubernetes version and version skew support policy]: https://kubernetes.io/docs/setup/release/version-skew-policy/#supported-versions
[Kubernetes Security and Disclosure Information]: https://kubernetes.io/docs/reference/issues-security/security/#report-a-vulnerability

13
assets/scss/_base.scss
View File

@ -832,3 +832,16 @@ section#cncf {
font-size: 1rem;
}
}
/* DOCUMENTATION */
body.td-documentation {
header > .header-filler {
height: $hero-padding-top;
background-color: black;
}
/* Special case for if an announcement is active */
header section#announcement ~ .header-filler {
display: none;
}
}

123
assets/scss/_custom.scss
View File

@ -1,3 +1,5 @@
$announcement-size-adjustment: 8px;
/* GLOBAL */
.td-main {
.row {
@ -284,6 +286,82 @@ blockquote {
border-left-color: #d9534f !important;
}
.deprecation-warning {
padding: 20px;
margin: 20px 0;
background-color: #faf5b6;
color: #000;
}
body.td-home .deprecation-warning, body.td-blog .deprecation-warning, body.td-documentation .deprecation-warning {
border-radius: 3px;
}
body.td-home #deprecation-warning {
max-width: 1000px;
margin-top: 2.5rem;
margin-left: auto;
margin-right: auto;
}
#caseStudies body > #deprecation-warning, body.cid-casestudies > #deprecation-warning, body.cid-community > #deprecation-warning {
display: inline-block;
vertical-align: top;
position: relative;
background-color: #326ce5; // Kubernetes blue
color: #fff;
padding: 0;
margin: 0;
width: 100vw;
}
#caseStudies body > #deprecation-warning, body.cid-casestudies > #deprecation-warning {
padding-top: 32px;
}
body.cid-partners > #deprecation-warning {
padding: 0;
margin-right: 0;
margin-left: 0;
margin-top: 0;
width: 100vw;
}
body.cid-partners > #deprecation-warning > .content {
width: 100%;
max-width: initial;
margin-right: 0;
margin-left: 0;
margin-top: 0;
padding-left: 5vw;
padding-right: 5vw;
padding-top: 2rem;
padding-bottom: 2rem;
}
body.cid-community > #deprecation-warning > .deprecation-warning {
margin-left: 20px;
margin-right: 20px;
color: #faf5b6;
background-color: inherit;
}
body.cid-community > #deprecation-warning > .deprecation-warning > * {
color: inherit;
background-color: inherit;
}
#caseStudies body > #deprecation-warning > .deprecation-warning, body.cid-casestudies > #deprecation-warning > .deprecation-warning {
color: inherit;
background: inherit;
width: 80%;
margin: 0;
margin-top: 120px;
margin-left: auto;
margin-right: auto;
border-radius: initial;
}
#deprecation-warning > .deprecation-warning a {
background: transparent;
color: inherit;
text-decoration: underline;
}
// search & sidebar
.td-sidebar {
@media only screen and (min-width: 768px) {
@ -390,4 +468,47 @@ main.content {
}
}
}
}
}
/* ANNOUNCEMENTS */
section#fp-announcement ~ .header-hero {
padding: $announcement-size-adjustment 0;
> div {
margin-top: $announcement-size-adjustment;
margin-bottom: $announcement-size-adjustment;
}
h1, h2, h3, h4, h5 {
margin: $announcement-size-adjustment 0;
}
}
section#announcement ~ .header-hero {
padding: #{$announcement-size-adjustment / 2} 0;
> div {
margin-top: #{$announcement-size-adjustment / 2};
margin-bottom: #{$announcement-size-adjustment / 2};
padding-bottom: #{$announcement-size-adjustment / 2};
}
h1, h2, h3, h4, h5 {
margin: #{$announcement-size-adjustment / 2} 0;
}
}
/* DOCUMENTATION */
/* Don't show lead text */
body.td-documentation {
main {
@media only screen {
> * {
> .lead:first-of-type {
display: none;
}
}
}
}
}

2
assets/scss/_tablet.scss
View File

@ -23,7 +23,7 @@ $main-nav-left-button-size: 50px;
$main-nav-left-button-font-size: 18px;
// hero
$hero-padding-top: 136px;
$hero-padding-top: 116px;
$headline-wrapper-margin-bottom: 40px;
$quickstart-button-padding: 0 50px;
$vendor-strip-font-size: 16px;

41
config.toml
View File

@ -115,10 +115,10 @@ time_format_blog = "Monday, January 02, 2006"
description = "Production-Grade Container Orchestration"
showedit = true
latest = "v1.18"
latest = "v1.19"
fullversion = "v1.18.0"
version = "v1.18"
fullversion = "v1.19.0"
version = "v1.19"
githubbranch = "master"
docsbranch = "master"
deprecated = false
@ -157,39 +157,40 @@ js = [
]
[[params.versions]]
fullversion = "v1.18.0"
version = "v1.18"
githubbranch = "v1.18.0"
docsbranch = "release-1.18"
fullversion = "v1.19.0"
version = "v1.19"
githubbranch = "v1.19.0"
docsbranch = "master"
url = "https://kubernetes.io"
[[params.versions]]
fullversion = "v1.17.4"
fullversion = "v1.18.8"
version = "v1.18"
githubbranch = "v1.18.8"
docsbranch = "release-1.18"
url = "https://v1-18.docs.kubernetes.io"
[[params.versions]]
fullversion = "v1.17.11"
version = "v1.17"
githubbranch = "v1.17.4"
githubbranch = "v1.17.11"
docsbranch = "release-1.17"
url = "https://v1-17.docs.kubernetes.io"
[[params.versions]]
fullversion = "v1.16.8"
fullversion = "v1.16.14"
version = "v1.16"
githubbranch = "v1.16.8"
githubbranch = "v1.16.14"
docsbranch = "release-1.16"
url = "https://v1-16.docs.kubernetes.io"
[[params.versions]]
fullversion = "v1.15.11"
fullversion = "v1.15.12"
version = "v1.15"
githubbranch = "v1.15.11"
githubbranch = "v1.15.12"
docsbranch = "release-1.15"
url = "https://v1-15.docs.kubernetes.io"
[[params.versions]]
fullversion = "v1.14.10"
version = "v1.14"
githubbranch = "v1.14.10"
docsbranch = "release-1.14"
url = "https://v1-14.docs.kubernetes.io"
# User interface configuration
[params.ui]
@ -230,7 +231,7 @@ no = 'Sorry to hear that. Please <a href="https://github.com/USERNAME/REPOSITORY
[[params.links.user]]
name = "Calendar"
url = "https://calendar.google.com/calendar/embed?src=nt2tcnbtbied3l6gi2h29slvc0%40group.calendar.google.com"
url = "https://calendar.google.com/calendar/embed?src=calendar%40kubernetes.io"
icon = "fas fa-calendar-alt"
desc = "Google Calendar for Kubernetes"

1
content/de/_index.html
View File

@ -42,7 +42,6 @@ Kubernetes ist Open Source und bietet Dir die Freiheit, die Infrastruktur vor Or
<button id="desktopShowVideoButton" onclick="kub.showVideo()">Video ansehen</button>
<br>
<br>
<br>
<a href="https://events.linuxfoundation.org/kubecon-cloudnativecon-europe/?utm_source=kubernetes.io&utm_medium=nav&utm_campaign=kccnceu20" button id="desktopKCButton">Besuche die KubeCon - 13-16 August 2020 in Amsterdam</a>
<br>
<br>

2
content/de/docs/concepts/architecture/nodes.md
View File

@ -123,7 +123,7 @@ Wenn Sie beispielsweise versuchen, einen Node aus folgendem Inhalt zu erstellen:
```
Kubernetes erstellt intern ein Node-Oject (die Darstellung) und validiert den Node durch Zustandsprüfung basierend auf dem Feld `metadata.name`.
Kubernetes erstellt intern ein Node-Objekt (die Darstellung) und validiert den Node durch Zustandsprüfung basierend auf dem Feld `metadata.name`.
Wenn der Node gültig ist, d.h. wenn alle notwendigen Dienste ausgeführt werden, ist er berechtigt, einen Pod auszuführen.
Andernfalls wird er für alle Clusteraktivitäten ignoriert, bis er gültig wird.

6
content/de/docs/setup/minikube.md
View File

@ -428,11 +428,11 @@ Weitere Informationen zu Minikube finden Sie im [Vorschlag](https://git.k8s.io/c
## Zusätzliche Links
* **Ziele und Nichtziele**: Die Ziele und Nichtziele des Minikube-Projekts finden Sie in unserer [Roadmap](https://git.k8s.io/minikube/docs/contributors/roadmap.md).
* **Ziele und Nichtziele**: Die Ziele und Nichtziele des Minikube-Projekts finden Sie in unserer [Roadmap](https://minikube.sigs.k8s.io/docs/contrib/roadmap/).
* **Entwicklungshandbuch**: Lesen Sie [CONTRIBUTING.md](https://git.k8s.io/minikube/CONTRIBUTING.md) für einen Überblick über das Senden von Pull-Requests.
* **Minikube bauen**: Anweisungen zum Erstellen/Testen von Minikube aus dem Quellcode finden Sie im [build Handbuch](https://git.k8s.io/minikube/docs/contributors/build_guide.md).
* **Minikube bauen**: Anweisungen zum Erstellen/Testen von Minikube aus dem Quellcode finden Sie im [build Handbuch](https://minikube.sigs.k8s.io/docs/contrib/building/).
* **Neue Abhängigkeit hinzufügen**: Anweisungen zum Hinzufügen einer neuen Abhängigkeit zu Minikube finden Sie in der [Anleitung zum Hinzufügen von Abhängigkeiten](https://minikube.sigs.k8s.io/docs/drivers/).
* **Neues Addon hinzufügen**: Anweisungen zum Hinzufügen eines neuen Addons für Minikube finden Sie im [Anleitung zum Hinzufügen eines Addons](https://git.k8s.io/minikube/docs/contributors/adding_an_addon.md).
* **Neues Addon hinzufügen**: Anweisungen zum Hinzufügen eines neuen Addons für Minikube finden Sie im [Anleitung zum Hinzufügen eines Addons](https://minikube.sigs.k8s.io/docs/handbook/addons/).
* **MicroK8s**: Linux-Benutzer, die die Ausführung einer virtuellen Maschine vermeiden möchten, sollten [MicroK8s](https://microk8s.io/) als Alternative in Betracht ziehen.
## Community

1
content/de/docs/tasks/administer-cluster/_index.md
View File

@ -1,5 +1,6 @@
---
title: "Einen Cluster verwalten"
description: Lerne allgemeine Aufgaben zur Verwaltung eines Clusters kennen.
weight: 20
---

5
content/de/docs/tasks/administer-cluster/kubeadm/_index.md Normal file
View File

@ -0,0 +1,5 @@
---
title: "Verwaltung mit kubeadm"
weight: 10
---

10
content/de/docs/tutorials/_index.md
View File

@ -46,16 +46,6 @@ Bevor Sie die einzelnen Lernprogramme durchgehen, möchten Sie möglicherweise e
* [ZooKeeper, ein verteiltes CP-System](/docs/tutorials/stateful-application/zookeeper/)
## CI/CD Pipeline
* [Einrichten einer CI/CD-Pipeline mit Kubernetes Teil 1: Übersicht](https://www.linux.com/blog/learn/chapter/Intro-to-Kubernetes/2017/5/set-cicd-pipeline-kubernetes-part-1-overview)
* [Einrichten einer CI/CD-Pipeline mit einem Jenkins-Pod in Kubernetes (Teil 2)](https://www.linux.com/blog/learn/chapter/Intro-to-Kubernetes/2017/6/set-cicd-pipeline-jenkins-pod-kubernetes-part-2)
* [Ausführen und Skalieren einer verteilten Kreuzworträtsel-App mit CI/CD auf Kubernetes (Teil 3)](https://www.linux.com/blog/learn/chapter/intro-to-kubernetes/2017/6/run-and-scale-distributed-crossword-puzzle-app-cicd-kubernetes-part-3)
* [CI/CD für eine verteilte Kreuzworträtsel-App auf Kubernetes einrichten (Teil 4)](https://www.linux.com/blog/learn/chapter/intro-to-kubernetes/2017/6/set-cicd-distributed-crossword-puzzle-app-kubernetes-part-4)
## Clusters
* [AppArmor](/docs/tutorials/clusters/apparmor/)

7
content/en/_index.html
View File

@ -41,12 +41,7 @@ Kubernetes is open source giving you the freedom to take advantage of on-premise
<button id="desktopShowVideoButton" onclick="kub.showVideo()">Watch Video</button>
<br>
<br>
<a href="https://events.linuxfoundation.org/kubecon-cloudnativecon-europe/?utm_source=kubernetes.io&utm_medium=nav&utm_campaign=kccnceu20" button id="desktopKCButton">Attend KubeCon EU virtually on August 17-20, 2020</a>
<br>
<br>
<br>
<br>
<a href="https://events.linuxfoundation.org/kubecon-cloudnativecon-north-america/?utm_source=kubernetes.io&utm_medium=nav&utm_campaign=kccncna20" button id="desktopKCButton">Attend KubeCon in Boston on November 17-20, 2020</a>
<a href="https://events.linuxfoundation.org/kubecon-cloudnativecon-north-america/?utm_source=kubernetes.io&utm_medium=nav&utm_campaign=kccncna20" button id="desktopKCButton">Attend KubeCon NA virtually on November 17-20, 2020</a>
</div>
<div id="videoPlayer">
<iframe data-url="https://www.youtube.com/embed/H06qrNmGqyE?autoplay=1" frameborder="0" allowfullscreen></iframe>

2
content/en/blog/_posts/2018-12-04-kubeadm-ga-release.md
View File

@ -23,7 +23,7 @@ To keep kubeadm lean, focused, and vendor/infrastructure agnostic, the following
- Non-critical add-ons, e.g. for monitoring, logging, and visualization
- Specific cloud provider integrations
Infrastructure provisioning, for example, is left to other SIG Cluster Lifecycle projects, such as the [Cluster API](https://github.com/kubernetes-sigs/cluster-api). Instead, kubeadm covers only the common denominator in every Kubernetes cluster: the [control plane](/docs/concepts/#kubernetes-control-plane). The user may install their preferred networking solution and other add-ons on top of Kubernetes *after* cluster creation.
Infrastructure provisioning, for example, is left to other SIG Cluster Lifecycle projects, such as the [Cluster API](https://github.com/kubernetes-sigs/cluster-api). Instead, kubeadm covers only the common denominator in every Kubernetes cluster: the [control plane](/docs/concepts/overview/components/#control-plane-components). The user may install their preferred networking solution and other add-ons on top of Kubernetes *after* cluster creation.
### What kubeadm's GA release means

2
content/en/blog/_posts/2019-06-24-kubeadm-ha-v115.md
View File

@ -43,7 +43,7 @@ Those tasks are addressed by other SIG Cluster Lifecycle projects, such as the
[Cluster API](https://github.com/kubernetes-sigs/cluster-api) for infrastructure provisioning and management.
Instead, kubeadm covers only the common denominator in every Kubernetes cluster: the
[control plane](https://kubernetes.io/docs/concepts/#kubernetes-control-plane).
[control plane](/docs/concepts/overview/components/#control-plane-components).
![Cluster Lifecycle Layers](/images/blog/2019-06-24-kubeadm-ha-v115/overview.png)

2
content/en/blog/_posts/2020-05-29-K8s-KPIs-with-Kuberhealthy.md
View File

@ -175,7 +175,7 @@ We calculate this by measuring Kuberhealthy's [deployment check](https://github.
- PromQL Query (Availability % over the past 30 days):
```promql
1 - (sum(count_over_time(kuberhealthy_check{check="kuberhealthy/deployment", status="0"}[30d])) OR vector(0))/(sum(count_over_time(kuberhealthy_check{check="kuberhealthy/deployment", status="1"}[30d])) * 100)
1 - (sum(count_over_time(kuberhealthy_check{check="kuberhealthy/deployment", status="0"}[30d])) OR vector(0)) / sum(count_over_time(kuberhealthy_check{check="kuberhealthy/deployment", status="1"}[30d]))
```
*Utilization*

1
content/en/blog/_posts/2020-06-30-SIG-Windows-Spotlight/index.md
View File

@ -5,7 +5,6 @@ date: 2020-06-30
slug: sig-windows-spotlight-2020
---
# SIG-Windows Spotlight
_This post tells the story of how Kubernetes contributors work together to provide a container orchestrator that works for both Linux and Windows._
<img alt="Image of a computer with Kubernetes logo" width="30%" src="KubernetesComputer_transparent.png">

219
content/en/blog/_posts/2020-07-27-kubernetes-1-17-release-interview.md Normal file
View File

@ -0,0 +1,219 @@
---
layout: blog
title: "Music and math: the Kubernetes 1.17 release interview"
date: 2020-07-27
---
**Author**: Adam Glick (Google)
Every time the Kubernetes release train stops at the station, we like to ask the release lead to take a moment to reflect on their experience. That takes the form of an interview on the weekly [Kubernetes Podcast from Google](https://kubernetespodcast.com/) that I co-host with [Craig Box](https://twitter.com/craigbox). If you're not familiar with the show, every week we summarise the new in the Cloud Native ecosystem, and have an insightful discussion with an interesting guest from the broader Kubernetes community.
At the time of the 1.17 release in December, we [talked to release team lead Guinevere Saenger](https://kubernetespodcast.com/episode/083-kubernetes-1.17/). We have [shared](https://kubernetes.io/blog/2018/07/16/how-the-sausage-is-made-the-kubernetes-1.11-release-interview-from-the-kubernetes-podcast/) [the](https://kubernetes.io/blog/2019/05/13/cat-shirts-and-groundhog-day-the-kubernetes-1.14-release-interview/) [transcripts](https://kubernetes.io/blog/2019/12/06/when-youre-in-the-release-team-youre-family-the-kubernetes-1.16-release-interview/) of previous interviews on the Kubernetes blog, and we're very happy to share another today.
Next week we will bring you up to date with the story of Kubernetes 1.18, as we gear up for the release of 1.19 next month. [Subscribe to the show](https://kubernetespodcast.com/subscribe/) wherever you get your podcasts to make sure you don't miss that chat!
---
**ADAM GLICK: You have a nontraditional background for someone who works as a software engineer. Can you explain that background?**
GUINEVERE SAENGER: My first career was as a [collaborative pianist](https://en.wikipedia.org/wiki/Collaborative_piano), which is an academic way of saying "piano accompanist". I was a classically trained pianist who spends most of her time onstage, accompanying other people and making them sound great.
**ADAM GLICK: Is that the piano equivalent of pair-programming?**
GUINEVERE SAENGER: No one has said it to me like that before, but all sorts of things are starting to make sense in my head right now. I think that's a really great way of putting it.
**ADAM GLICK: That's a really interesting background, as someone who also has a background with music. What made you decide to get into software development?**
GUINEVERE SAENGER: I found myself in a life situation where I needed more stable source of income, and teaching music, and performing for various gig opportunities, was really just not cutting it anymore. And I found myself to be working really, really hard with not much to show for it. I had a lot of friends who were software engineers. I live in Seattle. That's sort of a thing that happens to you when you live in Seattle — you get to know a bunch of software engineers, one way or the other.
The ones I met were all lovely people, and they said, hey, I'm happy to show you how to program in Python. And so I did that for a bit, and then I heard about this program called [Ada Developers Academy](https://adadevelopersacademy.org/). That's a year long coding school, targeted at women and non-binary folks that are looking for a second career in tech. And so I applied for that.
**CRAIG BOX: What can you tell us about that program?**
GUINEVERE SAENGER: It's incredibly selective, for starters. It's really popular in Seattle and has gotten quite a good reputation. It took me three tries to get in. They do two classes a year, and so it was a while before I got my response saying 'congratulations, we are happy to welcome you into Cohort 6'. I think what sets Ada Developers Academy apart from other bootcamp style coding programs are three things, I think? The main important one is that if you get in, you pay no tuition. The entire program is funded by company sponsors.
**CRAIG BOX: Right.**
GUINEVERE SAENGER: The other thing that really convinced me is that five months of the 11-month program are an industry internship, which means you get both practical experience, mentorship, and potential job leads at the end of it.
**CRAIG BOX: So very much like a condensed version of the University of Waterloo degree, where you do co-op terms.**
GUINEVERE SAENGER: Interesting. I didn't know about that.
**CRAIG BOX: Having lived in Waterloo for a while, I knew a lot of people who did that. But what would you say the advantages were of going through such a condensed schooling process in computer science?**
GUINEVERE SAENGER: I'm not sure that the condensed process is necessarily an advantage. I think it's a necessity, though. People have to quit their jobs to go do this program. It's not an evening school type of thing.
**CRAIG BOX: Right.**
GUINEVERE SAENGER: And your internship is basically a full-time job when you do it. One thing that Ada was really, really good at is giving us practical experience that directly relates to the workplace. We learned how to use Git. We learned how to design websites using [Rails](https://rubyonrails.org/). And we also learned how to collaborate, how to pair-program. We had a weekly retrospective, so we sort of got a soft introduction to workflows at a real workplace. Adding to that, the internship, and I think the overall experience is a little bit more 'practical workplace oriented' and a little bit less academic.
When you're done with it, you don't have to relearn how to be an adult in a working relationship with other people. You come with a set of previous skills. There are Ada graduates who have previously been campaign lawyers, and veterinarians, and nannies, cooks, all sorts of people. And it turns out these skills tend to translate, and they tend to matter.
**ADAM GLICK: With your background in music, what do you think that that allows you to bring to software development that could be missing from, say, standard software development training that people go through?**
GUINEVERE SAENGER: People tend to really connect the dots when I tell them I used to be a musician. Of course, I still consider myself a musician, because you don't really ever stop being a musician. But they say, 'oh, yeah, music and math', and that's just a similar sort of brain. And that makes so much sense. And I think there's a little bit of a point to that. When you learn a piece of music, you have to start recognizing patterns incredibly quickly, almost intuitively.
And I think that is the main skill that translates into programming— recognizing patterns, finding the things that work, finding the things that don't work. And for me, especially as a collaborative pianist, it's the communicating with people, the finding out what people really want, where something is going, how to figure out what the general direction is that we want to take, before we start writing the first line of code.
**CRAIG BOX: In your experience at Ada or with other experiences you've had, have you been able to identify patterns in other backgrounds for people that you'd recommend, 'hey, you're good at music, so therefore you might want to consider doing something like a course in computer science'?**
GUINEVERE SAENGER: Overall, I think ultimately writing code is just giving a set of instructions to a computer. And we do that in daily life all the time. We give instructions to our kids, we give instructions to our students. We do math, we write textbooks. We give instructions to a room full of people when you're in court as a lawyer.
Actually, the entrance exam to Ada Developers Academy used to have questions from the [LSAT](https://en.wikipedia.org/wiki/Law_School_Admission_Test) on it to see if you were qualified to join the program. They changed that when I applied, but I think that's a thing that happened at one point. So, overall, I think software engineering is a much more varied field than we give it credit for, and that there are so many ways in which you can apply your so-called other skills and bring them under the umbrella of software engineering.
**CRAIG BOX: I do think that programming is effectively half art and half science. There's creativity to be applied. There is perhaps one way to solve a problem most efficiently. But there are many different ways that you can choose to express how you compiled something down to that way.**
GUINEVERE SAENGER: Yeah, I mean, that's definitely true. I think one way that you could probably prove that is that if you write code at work and you're working on something with other people, you can probably tell which one of your co-workers wrote which package, just by the way it's written, or how it is documented, or how it is styled, or any of those things. I really do think that the human character shines through.
**ADAM GLICK: What got you interested in Kubernetes and open source?**
GUINEVERE SAENGER: The honest answer is absolutely nothing. Going back to my programming school— and remember that I had to do a five-month internship as part of my training— the way that the internship works is that sponsor companies for the program get interns in according to how much they sponsored a specific cohort of students.
So at the time, Samsung and SDS offered to host two interns for five months on their [Cloud Native Computing team](https://samsung-cnct.github.io/) and have that be their practical experience. So I go out of a Ruby on Rails full stack web development bootcamp and show up at my internship, and they said, "Welcome to Kubernetes. Try to bring up a cluster." And I said, "Kuber what?"
**CRAIG BOX: We've all said that on occasion.**
**ADAM GLICK: Trial by fire, wow.**
GUINEVERE SAENGER: I will say that that entire team was absolutely wonderful, delightful to work with, incredibly helpful. And I will forever be grateful for all of the help and support that I got in that environment. It was a great place to learn.
**CRAIG BOX: You now work on GitHub's Kubernetes infrastructure. Obviously, there was GitHub before there was a Kubernetes, so a migration happened. What can you tell us about the transition that GitHub made to running on Kubernetes?**
GUINEVERE SAENGER: A disclaimer here— I was not at GitHub at the time that the transition to Kubernetes was made. However, to the best of my knowledge, the decision to transition to Kubernetes was made and people decided, yes, we want to try Kubernetes. We want to use Kubernetes. And mostly, the only decision left was, which one of our applications should we move over to Kubernetes?
**CRAIG BOX: I thought GitHub was written on Rails, so there was only one application.**
GUINEVERE SAENGER: [LAUGHING] We have a lot of supplementary stuff under the covers.
**CRAIG BOX: I'm sure.**
GUINEVERE SAENGER: But yes, GitHub is written in Rails. It is still written in Rails. And most of the supplementary things are currently running on Kubernetes. We have a fair bit of stuff that currently does not run on Kubernetes. Mainly, that is GitHub Enterprise related things. I would know less about that because I am on the platform team that helps people use the Kubernetes infrastructure. But back to your question, leadership at the time decided that it would be a good idea to start with GitHub the Rails website as the first project to move to Kubernetes.
**ADAM GLICK: High stakes!**
GUINEVERE SAENGER: The reason for this was that they decided if they were going to not start big, it really wasn't going to transition ever. It was really not going to happen. So they just decided to go all out, and it was successful, for which I think the lesson would probably be commit early, commit big.
**CRAIG BOX: Are there any other lessons that you would take away or that you've learned kind of from the transition that the company made, and might be applicable to other people who are looking at moving their companies from a traditional infrastructure to a Kubernetes infrastructure?**
GUINEVERE SAENGER: I'm not sure this is a lesson specifically, but I was on support recently, and it turned out that, due to unforeseen circumstances and a mix of human error, a bunch of the namespaces on one of our Kubernetes clusters got deleted.
**ADAM GLICK: Oh, my.**
GUINEVERE SAENGER: It should not have affected any customers, I should mention, at this point. But all in all, it took a few of us a few hours to almost completely recover from this event. I think that, without Kubernetes, this would not have been possible.
**CRAIG BOX: Generally, deleting something like that is quite catastrophic. We've seen a number of other vendors suffer large outages when someone's done something to that effect, which is why we get [#hugops](https://twitter.com/hashtag/hugops) on Twitter all the time.**
GUINEVERE SAENGER: People did send me #hugops, that is a thing that happened. But overall, something like this was an interesting stress test and sort of proved that it wasn't nearly as catastrophic as a worst case scenario.
**CRAIG BOX: GitHub [runs its own data centers](https://githubengineering.com/githubs-metal-cloud/). Kubernetes was largely built for running on the cloud, but a lot of people do choose to run it on their own, bare metal. How do you manage clusters and provisioning of the machinery you run?**
GUINEVERE SAENGER: When I started, my onboarding project was to deprovision an old cluster, make sure all the traffic got moved to somewhere where it would keep running, provision a new cluster, and then move website traffic onto the new cluster. That was a really exciting onboarding project. At the time, we provisioned bare metal machines using Puppet. We still do that to a degree, but I believe the team that now runs our computing resources actually inserts virtual machines as an extra layer between the bare metal and the Kubernetes nodes.
Again, I was not intrinsically part of that decision, but my understanding is that it just makes for a greater reliability and reproducibility across the board. We've had some interesting hardware dependency issues come up, and the virtual machines basically avoid those.
**CRAIG BOX: You've been working with Kubernetes for a couple of years now. How did you get involved in the release process?**
GUINEVERE SAENGER: When I first started in the project, I started at the [special interest group for contributor experience](https://github.com/kubernetes/community/tree/master/sig-contributor-experience#readme), namely because one of my co-workers at the time, Aaron Crickenberger, was a big Kubernetes community person. Still is.
**CRAIG BOX: We've [had him on the show](https://kubernetespodcast.com/episode/046-kubernetes-1.14/) for one of these very release interviews!**
GUINEVERE SAENGER: In fact, this is true! So Aaron and I actually go way back to Samsung SDS. Anyway, Aaron suggested that I should write up a contribution to the Kubernetes project, and I said, me? And he said, yes, of course. You will be [speaking at KubeCon](https://www.youtube.com/watch?v=TkCDUFR6xqw), so you should probably get started with a PR or something. So I tried, and it was really, really hard. And I complained about it [in a public GitHub issue](https://github.com/kubernetes/community/issues/141), and people said, yeah. Yeah, we know it's hard. Do you want to help with that?
And so I started getting really involved with the [process for new contributors to get started](https://github.com/kubernetes/community/tree/master/contributors/guide) and have successes, kind of getting a foothold into a project that's as large and varied as Kubernetes. From there on, I began to talk to people, get to know people. The great thing about the Kubernetes community is that there is so much mentorship to go around.
**ADAM GLICK: Right.**
GUINEVERE SAENGER: There are so many friendly people willing to help. It's really funny when I talk to other people about it. They say, what do you mean, your coworker? And I said, well, he's really a colleague. He really works for another company.
**CRAIG BOX: He's sort-of officially a competitor.**
GUINEVERE SAENGER: Yeah.
**CRAIG BOX: But we're friends.**
GUINEVERE SAENGER: But he totally helped me when I didn't know how to git patch my borked pull request. So that happened. And eventually, somebody just suggested that I start following along in the release process and shadow someone on their release team role. And that, at the time, was Tim Pepper, who was bug triage lead, and I shadowed him for that role.
**CRAIG BOX: Another [podcast guest](https://kubernetespodcast.com/episode/010-kubernetes-1.11/) on the interview train.**
GUINEVERE SAENGER: This is a pattern that probably will make more sense once I explain to you about the shadow process of the release team.
**ADAM GLICK: Well, let's turn to the Kubernetes release and the release process. First up, what's new in this release of 1.17?**
GUINEVERE SAENGER: We have only a very few new things. The one that I'm most excited about is that we have moved [IPv4 and IPv6 dual stack](https://github.com/kubernetes/enhancements/issues/563) support to alpha. That is the most major change, and it has been, I think, a year and a half in coming. So this is the very first cut of that feature, and I'm super excited about that.
**CRAIG BOX: The people who have been promised IPv6 for many, many years and still don't really see it, what will this mean for them?**
**ADAM GLICK: And most importantly, why did we skip IPv5 support?**
GUINEVERE SAENGER: I don't know!
**CRAIG BOX: Please see [the appendix to this podcast](https://softwareengineering.stackexchange.com/questions/185380/ipv4-to-ipv6-where-is-ipv5) for technical explanations.**
GUINEVERE SAENGER: Having a dual stack configuration obviously enables people to have a much more flexible infrastructure and not have to worry so much about making decisions that will become outdated or that may be over-complicated. This basically means that pods can have dual stack addresses, and nodes can have dual stack addresses. And that basically just makes communication a lot easier.
**CRAIG BOX: What about features that didn't make it into the release? We had a conversation with Lachie in the [1.16 interview](https://kubernetespodcast.com/episode/072-kubernetes-1.16/), where he mentioned [sidecar containers](https://github.com/kubernetes/enhancements/blob/master/keps/sig-apps/sidecarcontainers.md). They unfortunately didn't make it into that release. And I see now that they haven't made this one either.**
GUINEVERE SAENGER: They have not, and we are actually currently undergoing an effort of tracking features that flip multiple releases.
As a community, we need everyone's help. There are a lot of features that people want. There is also a lot of cleanup that needs to happen. And we have started talking at previous KubeCons repeatedly about problems with maintainer burnout, reviewer burnout, have a hard time finding reviews for your particular contributions, especially if you are not an entrenched member of the community. And it has become very clear that this is an area where the entire community needs to improve.
So the unfortunate reality is that sometimes life happens, and people are busy. This is an open source project. This is not something that has company mandated OKRs. Particularly during the fourth quarter of the year in North America, but around the world, we have a lot of holidays. It is the end of the year. Kubecon North America happened as well. This makes it often hard to find a reviewer in time or to rally the support that you need for your enhancement proposal. Unfortunately, slipping releases is fairly common and, at this point, expected. We started out with having 42 enhancements and [landed with roughly half of that](https://docs.google.com/spreadsheets/d/1ebKGsYB1TmMnkx86bR2ZDOibm5KWWCs_UjV3Ys71WIs/edit#gid=0).
**CRAIG BOX: I was going to ask about the truncated schedule due to the fourth quarter of the year, where there are holidays in large parts of the world. Do you find that the Q4 release on the whole is smaller than others, if not for the fact that it's some week shorter?**
GUINEVERE SAENGER: Q4 releases are shorter by necessity because we are trying to finish the final release of the year before the end of the year holidays. Often, releases are under pressure of KubeCons, during which finding reviewers or even finding the time to do work can be hard to do, if you are attending. And even if you're not attending, your reviewers might be attending.
It has been brought up last year to make the final release more of a stability release, meaning no new alpha features. In practice, for this release, this is actually quite close to the truth. We have four features graduating to beta and most of our features are graduating to stable. I am hoping to use this as a precedent to change our process to make the final release a stability release from here on out. The timeline fits. The past experience fits this model.
**ADAM GLICK: On top of all of the release work that was going on, there was also KubeCon that happened. And you were involved in the [contributor summit](https://github.com/kubernetes/community/tree/master/events/2019/11-contributor-summit). How was the summit?**
GUINEVERE SAENGER: This was the first contributor summit where we had an organized events team with events organizing leads, and handbooks, and processes. And I have heard from multiple people— this is just word of mouth— that it was their favorite contributor summit ever.
**CRAIG BOX: Was someone allocated to hat production? [Everyone had sailor hats](https://flickr.com/photos/143247548@N03/49093218951/).**
GUINEVERE SAENGER: Yes, the entire event staff had sailor hats with their GitHub handle on them, and it was pretty fantastic. You can probably see me wearing one in some of the pictures from the contributor summit. That literally was something that was pulled out of a box the morning of the contributor summit, and no one had any idea. But at first, I was a little skeptical, but then I put it on and looked at myself in the mirror. And I was like, yes. Yes, this is accurate. We should all wear these.
**ADAM GLICK: Did getting everyone together for the contributor summit help with the release process?**
GUINEVERE SAENGER: It did not. It did quite the opposite, really. Well, that's too strong.
**ADAM GLICK: Is that just a matter of the time taken up?**
GUINEVERE SAENGER: It's just a completely different focus. Honestly, it helped getting to know people face-to-face that I had currently only interacted with on video. But we did have to cancel the release team meeting the day of the contributor summit because there was kind of no sense in having it happen. We moved it to the Tuesday, I believe.
**CRAIG BOX: The role of the release team leader has been described as servant leadership. Do you consider the position proactive or reactive?**
GUINEVERE SAENGER: Honestly, I think that depends on who's the release team lead, right? There are some people who are very watchful and look for trends, trying to detect problems before they happen. I tend to be in that camp, but I also know that sometimes it's not possible to predict things. There will be last minute bugs sometimes, sometimes not. If there is a last minute bug, you have to be ready to be on top of that. So for me, the approach has been I want to make sure that I have my priorities in order and also that I have backups in case I can't be available.
**ADAM GLICK: What was the most interesting part of the release process for you?**
GUINEVERE SAENGER: A release lead has to have served in other roles on the release team prior to being release team lead. To me, it was very interesting to see what other roles were responsible for, ones that I hadn't seen from the inside before, such as docs, CI signal. I had helped out with CI signal for a bit, but I want to give a big shout out to CI signal lead, Alena Varkockova, who was able to communicate effectively and kindly with everyone who was running into broken tests, failing tests. And she was very effective in getting all of our tests up and running.
So that was actually really cool to see. And yeah, just getting to see more of the workings of the team, for me, it was exciting. The other big exciting thing, of course, was to see all the changes that were going in and all the efforts that were being made.
**CRAIG BOX: The release lead for 1.18 has just been announced as [Jorge Alarcon](https://twitter.com/alejandrox135). What are you going to put in the proverbial envelope as advice for him?**
GUINEVERE SAENGER: I would want Jorge to be really on top of making sure that every Special Interest Group that enters a change, that has an enhancement for 1.18, is on top of the timelines and is responsive. Communication tends to be a problem. And I had hinted at this earlier, but some enhancements slipped simply because there wasn't enough reviewer bandwidth.
Greater communication of timelines and just giving people more time and space to be able to get in their changes, or at least, seemingly give them more time and space by sending early warnings, is going to be helpful. Of course, he's going to have a slightly longer release, too, than I did. This might be related to a unique Q4 challenge. Overall, I would encourage him to take more breaks, to rely more on his release shadows, and split out the work in a fashion that allows everyone to have a turn and everyone to have a break as well.
**ADAM GLICK: What would your advice be to someone who is hearing your experience and is inspired to get involved with the Kubernetes release or contributer process?**
GUINEVERE SAENGER: Those are two separate questions. So let me tackle the Kubernetes release question first. Kubernetes [SIG Release](https://github.com/kubernetes/sig-release/#readme) has, in my opinion, a really excellent onboarding program for new members. We have what is called the [Release Team Shadow Program](https://github.com/kubernetes/sig-release/blob/master/release-team/shadows.md). We also have the Release Engineering Shadow Program, or the Release Management Shadow Program. Those are two separate subprojects within SIG Release. And each subproject has a team of roles, and each role can have two to four shadows that are basically people who are part of that role team, and they are learning that role as they are doing it.
So for example, if I am the lead for bug triage on the release team, I may have two, three or four people that I closely work with on the bug triage tasks. These people are my shadows. And once they have served one release cycle as a shadow, they are now eligible to be lead in that role. We have an application form for this process, and it should probably be going up in January. It usually happens the first week of the release once all the release leads are put together.
**CRAIG BOX: Do you think being a member of the release team is something that is a good first contribution to the Kubernetes project overall?**
GUINEVERE SAENGER: It depends on what your goals are, right? I believe so. I believe, for me, personally, it has been incredibly helpful looking into corners of the project that I don't know very much about at all, like API machinery, storage. It's been really exciting to look over all the areas of code that I normally never touch.
It depends on what you want to get out of it. In general, I think that being a release team shadow is a really, really great on-ramp to being a part of the community because it has a paved path solution to contributing. All you have to do is show up to the meetings, ask questions of your lead, who is required to answer those questions.
And you also do real work. You really help, you really contribute. If you go across the issues and pull requests in the repo, you will see, 'Hi, my name is so-and-so. I am shadowing the CI signal lead for the current release. Can you help me out here?' And that's a valuable contribution, and it introduces people to others. And then people will recognize your name. They'll see a pull request by you, and they're like oh yeah, I know this person. They're legit.
---
_[Guinevere Saenger](https://twitter.com/guincodes) is a software engineer for GitHub and served as the Kubernetes 1.17 release team lead._
_You can find the [Kubernetes Podcast from Google](http://www.kubernetespodcast.com/) at [@KubernetesPod](https://twitter.com/KubernetesPod) on Twitter, and you can [subscribe](https://kubernetespodcast.com/subscribe/) so you never miss an episode._

215
content/en/blog/_posts/2020-08-03-kubernetes-1-18-release-interview.md Normal file
View File

@ -0,0 +1,215 @@
---
layout: blog
title: "Physics, politics and Pull Requests: the Kubernetes 1.18 release interview"
date: 2020-08-03
---
**Author**: Craig Box (Google)
The start of the COVID-19 pandemic couldn't delay the release of Kubernetes 1.18, but unfortunately [a small bug](https://github.com/kubernetes/utils/issues/141) could — thankfully only by a day. This was the last cat that needed to be herded by 1.18 release lead [Jorge Alarcón](https://twitter.com/alejandrox135) before the [release on March 25](https://kubernetes.io/blog/2020/03/25/kubernetes-1-18-release-announcement/).
One of the best parts about co-hosting the weekly [Kubernetes Podcast from Google](https://kubernetespodcast.com/) is the conversations we have with the people who help bring Kubernetes releases together. [Jorge was our guest on episode 96](https://kubernetespodcast.com/episode/096-kubernetes-1.18/) back in March, and [just like last week](https://kubernetes.io/blog/2020/07/27/music-and-math-the-kubernetes-1.17-release-interview/) we are delighted to bring you the transcript of this interview.
If you'd rather enjoy the "audiobook version", including another interview when 1.19 is released later this month, [subscribe to the show](https://kubernetespodcast.com/subscribe/) wherever you get your podcasts.
In the last few weeks, we've talked to long-time Kubernetes contributors and SIG leads [David Oppenheimer](https://kubernetespodcast.com/episode/114-scheduling/), [David Ashpole](https://kubernetespodcast.com/episode/113-instrumentation-and-cadvisor/) and [Wojciech Tyczynski](https://kubernetespodcast.com/episode/111-scalability/). All are worth taking the dog for a longer walk to listen to!
---
**ADAM GLICK: You're a former physicist. I have to ask, what kind of physics did you work on?**
JORGE ALARCÓN: Back in my days of math and all that, I used to work in [computational biology](https://en.wikipedia.org/wiki/Computational_biology) and a little bit of high energy physics. Computational biology was, for the most part, what I spent most of my time on. And it was essentially exploring the big idea of we have the structure of proteins. We know what they're made of. Now, based on that structure, we want to be able to predict [how they're going to fold](https://en.wikipedia.org/wiki/Protein_folding) and how they're going to behave, which essentially translates into the whole idea of designing pharmaceuticals, designing vaccines, or anything that you can possibly think of that has any connection whatsoever to a living organism.
**ADAM GLICK: That would seem to ladder itself well into maybe going to something like bioinformatics. Did you take a tour into that, or did you decide to go elsewhere directly?**
JORGE ALARCÓN: It is related, and I worked a little bit with some people that did focus on bioinformatics on the field specifically, but I never took a detour into it. Really, my big idea with computational biology, to be honest, it wasn't even the biology. That's usually what sells it, what people are really interested in, because protein engineering, all the cool and amazing things that you can do.
Which is definitely good, and I don't want to take away from it. But my big thing is because biology is such a real thing, it is amazingly complicated. And the math— the models that you have to design to study those systems, to be able to predict something that people can actually experiment and measure, it just captivated me. The level of complexity, the beauty, the mechanisms, all the structures that you see once you got through the math and look at things, it just kind of got to me.
**ADAM GLICK: How did you go from that world into the world of Kubernetes?**
JORGE ALARCÓN: That's both a really boring story and an interesting one.
[LAUGHING]
I did my thing with physics, and it was good. It was fun. But at some point, I wanted— working in academia— at least my feeling for it is that generally all the people that you're surrounded with are usually academics. Just another bunch of physics, a bunch of mathematicians.
But very seldom do you actually get the opportunity to take what you're working on and give it to someone else to use. Even with the mathematicians and physicists, the things that we're working on are super specialized, and you can probably find three, four, five people that can actually understand everything that you're saying. A lot of people are going to get the gist of it, but understanding the details, it's somewhat rare.
One of the things that I absolutely love about tech, about software engineering, coding, all that, is how open and transparent everything is. You can write your library in Python, you can publish it, and suddenly the world is going to actually use it, actually consume it. And because normally, I've seen that it has a large avenue where you can work in something really complicated, you can communicate it, and people can actually go ahead and take it and run with it in their given direction. And that is kind of what happened.
At some point, by pure accident and chance, I came across this group of people on the internet, and they were in the stages of making up this new group that's called [Data for Democracy](https://datafordemocracy.org/), a non-profit. And the whole idea was the internet, especially Twitter— that's how we congregated— Twitter, the internet. We have a ton of data scientists, people who work as software engineers, and the like. What if we all come together and try to solve some issues that actually affect the daily lives of people. And there were a ton of projects. Helping the ACLU gather data for something interesting that they were doing, gather data and analyze it for local governments— where do you have potholes, how much water is being consumed.
Try to apply all the science that we knew, combined with all the code that we could write, and offer a good and digestible idea for people to say, OK, this makes sense, let's do something about it— policy, action, whatever. And I started working with this group, Data for Democracy— wonderful set of people. And the person who I believe we can blame for Data for Democracy— the one who got the idea and got it up and running, his name is Jonathan Morgan. And eventually, we got to work together. He started a startup, and I went to work with the startup. And that was essentially the thing that took me away from physics and into the world of software engineering— Data for Democracy, definitely.
**ADAM GLICK: Were you using Kubernetes as part of that work there?**
JORGE ALARCÓN: No, it was simple as it gets. You just try to get some data. You create a couple [IPython notebooks](https://ipython.org/), some setting up of really simple MySQL databases, and that was it.
**ADAM GLICK: Where did you get started using Kubernetes? And was it before you started contributing to it and being a part, or did you decide to jump right in?**
JORGE ALARCÓN: When I first started using Kubernetes, it was also on my first job. So there wasn't a lot of specific training in regards to software engineering or anything of the sort that I did before I actually started working as a software engineer. I just went from physicist to engineer. And in my days of physics, at least on the computer side, I was completely trained in the super old school system administrator, where you have your 10, 20 computers. You know physically where they are, and you have to connect the cables.
**ADAM GLICK: All pets— all pets all the time.**
JORGE ALARCÓN: [LAUGHING] You have to have your huge Python, bash scripts, three, five major versions, all because doing an upgrade will break something really important and you have no idea how to work on it. And that was my training. That was the way that I learned how to do things. Those were the kind of things that I knew how to do.
And when I got to this company— startup— we were pretty much starting from scratch. We were building a couple applications. We work testing them, we were deploying them on a couple of managed instances. But like everything, there was a lot of toil that we wanted to automate. The whole issue of, OK, after days of work, we finally managed to get this version of the application up and running in these machines.
It's open to the internet. People can test it out. But it turns out that it is now two weeks behind the latest on all the master branches for this repo, so now we want to update. And we have to go through the process of bringing it back up, creating new machines, do that whole thing. And I had no idea what Kubernetes was, to be honest. My boss at the moment mentioned it to me like, hey, we should use Kubernetes because apparently, Kubernetes is something that might be able to help us here. And we did some— I want to call it research and development.
It was actually just making— again, startup, small company, small team, so really me just playing around with Kubernetes trying to get it to work, trying to get it to run. I was so lost. I had no idea what I was doing— not enough. I didn't have an idea of how Kubernetes was supposed to help me. And at that point, I did the best Googling that I could manage. Didn't really find a lot of examples. Didn't find a lot of blog posts. It was early.
**ADAM GLICK: What time frame was this?**
JORGE ALARCÓN: Three, four years ago, so definitely not 1.13. That's the best guesstimate that I can give at this point. But I wasn't able to find any good examples, any tutorials. The only book that I was able to get my hands on was the one written by Joe Beda, Kelsey Hightower, and I forget the other author. But what is it? "[Kubernetes— Up and Running](](http://shop.oreilly.com/product/0636920223788.do))"?
And in general, right now I use it as reference— it's really good. But as a beginner, I still was lost. They give all these amazing examples, they provide the applications, but I had no idea why someone might need a Pod, why someone might need a Deployment. So my last resort was to try and find someone who actually knew Kubernetes.
By accident, during my eternal Googling, I actually found a link to the [Kubernetes Slack](http://slack.kubernetes.io/). I jumped into the Kubernetes Slack hoping that someone might be able to help me out. And that was my entry point into the Kubernetes community. I just kept on exploring the Slack, tried to see what people were talking about, what they were asking to try to make sense of it, and just kept on iterating. And at some point, I think I got the hang of it.
**ADAM GLICK: What made you decide to be a release lead?**
JORGE ALARCÓN: The answer to this is my answer to why I have been contributing to Kubernetes. I really just want to be able to help out the community. Kubernetes is something that I absolutely adore.
Comparing Kubernetes to old school system administration, a handful of years ago, it took me like a week to create a node for an application to run. It took me months to get something that vaguely looked like an Ingress resource— just setting up the Nginx, and allowing someone else to actually use my application. And the fact that I could do all of that in five minutes, it really captivated me. Plus I've got to blame it on the physics. The whole idea with physics, I really like the patterns, and I really like the design of Kubernetes.
Once I actually got the hang of it, I loved the idea of how everything was designed, and I just wanted to learn a lot more about it. And I wanted to help the contributors. I wanted to help the people who actually build it. I wanted to help maintain it, and help provide the information for new contributors or new users. So instead of taking months for them to be up and running, let's just chat about what your issue is, and let's try to get a fix within the next hour or so.
**ADAM GLICK: You work for a stealth startup right now. Is it fair to assume that they're using Kubernetes?**
JORGE ALARCÓN: Yes—
[LAUGHING]
—for everything.
**ADAM GLICK: Are you able to say what [Searchable](https://www.searchable.ai/) does?**
JORGE ALARCÓN: The thing that we are trying to build is kind of like a search engine for your documents. Usually, if people have a question, they jump on Google. And for the most part, you're going to be able to get a good answer. You can ask something really random, like 'what is the weight of an elephant?'
Which, if you think about it, it's kind of random, but Google is going to give you an answer. And the thing that we are trying to build is something similar to that, but for files. So essentially, a search engine for your files. And most people, you have your local machine loaded up with— at least mine, I have a couple tens of gigabytes of different files.
I have Google Drive. I have a lot of documents that live in my email and the like. So the idea is to kind of build a search engine that is going to be able to connect all of those pieces. And besides doing simple word searches— for example, 'Kubernetes interview', and bring me the documents that we're looking at with all the questions— I can also ask things like what issue did I find last week while testing Prometheus. And it's going to be able to read my files, like through natural language processing, understand it, and be able to give me an answer.
**ADAM GLICK: It is a Google for your personal and non-public information, essentially?**
JORGE ALARCÓN: Hopefully.
**ADAM GLICK: Is the work that you do with Kubernetes as the release lead— is that part of your day job, or is that something that you're doing kind of nights and weekends separate from your day job?**
JORGE ALARCÓN: Both. Strictly speaking, my day job is just keep working on the application, build the things that it needs, maintain the infrastructure, and all that. When I started working at the company— which by the way, the person who brought me into the company was also someone that I met from my days in Data for Democracy— we started talking about the work.
I mentioned that I do a lot of work with the Kubernetes community and if it was OK that I continue doing it. And to my surprise, the answer was not only a yes, but yeah, you can do it during your day work. And at least for the time being, I just balance— I try to keep things organized.
Some days I just focus on Kubernetes. Some mornings I do Kubernetes. And then afternoon, I do Searchable, vice-versa, or just go back and forth, and try to balance the work as much as possible. But being release lead, definitely, it is a lot, so nights and weekends.
**ADAM GLICK: How much time does it take to be the release lead?**
JORGE ALARCÓN: It varies, but probably, if I had to give an estimate, at the very least you have to be able to dedicate four hours most days.
**ADAM GLICK: Four hours a day?**
JORGE ALARCÓN: Yeah, most days. It varies a lot. For example, at the beginning of the release cycle, you don't need to put in that much work because essentially, you're just waiting and helping people get set up, and people are writing their [Kubernetes Enhancement Proposals](https://github.com/kubernetes/enhancements/tree/master/keps), they are implementing it, and you can answer some questions. It's relatively easy, but for the most part, a lot of the time the four hours go into talking with people, just making sure that, hey, are people actually writing their enhancements, do we have all the enhancements that we want. And most of those fours hours, going around, chatting with people, and making sure that things are being done. And if, for some reason, someone needs help, just directing them to the right place to get their answer.
**ADAM GLICK: What does Searchable get out of you doing this work?**
JORGE ALARCÓN: Physically, nothing. The thing that we're striving for is to give back to the community. My manager/boss/homeslice— I told him I was going to call him my homeslice— both of us have experience working in open source. At some point, he was also working on a project that I'm probably going to mispronounce, but Mahout with Apache.
And he also has had this experience. And both of us have this general idea and strive to build something for Searchable that's going to be useful for people, but also build knowledge, build guides, build applications that are going to be useful for the community. And at least one of the things that I was able to do right now is be the lead for the Kubernetes team. And this is a way of giving back to the community. We're using Kubernetes to run our things, so let's try to balance how things work.
**ADAM GLICK: Lachlan Evenson was the release lead on 1.16 as well as [our guest back in episode 72](https://kubernetespodcast.com/episode/072-kubernetes-1.16/), and he's returned on this release as the [emeritus advisor](https://github.com/kubernetes/sig-release/tree/master/release-team/role-handbooks/emeritus-adviser). What did you learn from him?**
JORGE ALARCÓN: Oh, everything. And it actually all started back on 1.16. So like you said, an amazing person— he's an amazing individual. And it's truly an opportunity to be able to work with him. During 1.16, I was the CI Signal lead, and Lachie is very hands on.
He's not the kind of person to just give you a list of things and say, do them. He actually comes to you, has a conversation, and he works with you more than anything. And when we were working together on 1.16, I got to learn a lot from him in terms of CI Signal. And especially because we talked about everything just to make sure that 1.16 was ready to go, I also got to pick up a couple of things that a release lead has to know, has to be able to do, has to work on to get a release out the door.
And now, during this release, there is a lot of information that's really useful, and there's a lot of advice and general wisdom that comes in handy. For most of the things that impact a lot of things, we are always in communication. Like, I'm doing this, you're doing that, advice. And essentially, every single thing that we do is pretty much a code review. You do it, and then you wait for someone else to give you comments. And that's been a strong part of our relationship working.
**ADAM GLICK: What would you say the theme for this release is?**
JORGE ALARCÓN: I think one of the themes is "fit and finish". There are a lot of features that we are bumping from alpha to beta, from beta to stable. And we want to make sure that people have a good user experience. Operators and developers alike just want to get rid of as many bugs as possible, improve the flow of things.
But the other really cool thing is we have about an equal distribution between alpha, beta, and stable. We are also bringing up a lot of new features. So besides making Kubernetes more stable for all the users that are already using it, we are working on bringing up new things that people can try out for the next release and see how it goes in the future.
**ADAM GLICK: Did you have a release team mascot?**
JORGE ALARCÓN: Kind of.
**ADAM GLICK: Who/what was it?**
JORGE ALARCÓN: [LAUGHING] I say kind of because I'm using the mascot in the [logo](https://twitter.com/KubernetesPod/status/1242953121380392963), and the logo is inspired by the Large Hadron Collider.
**ADAM GLICK: Oh, fantastic.**
JORGE ALARCÓN: Being the release lead, I really had to take a chance on this opportunity to use the LHC as the mascot.
**ADAM GLICK: We've had [some of the folks from the LHC on the show](https://kubernetespodcast.com/episode/062-cern/), and I know they listen, and they will be thrilled with that.**
JORGE ALARCÓN: [LAUGHING] Hopefully, they like the logo.
**ADAM GLICK: If you look at this release, what part of this release, what thing that has been added to it are you personally most excited about?**
JORGE ALARCÓN: Like a parent can't choose which child is his or her favorite, you really can't choose a specific thing.
**ADAM GLICK: We have been following online and in the issues an enhancement that's called [sidecar containers](https://github.com/kubernetes/enhancements/issues/753). You'd be able to mark the order of containers starting in a pod. Tim Hockin posted [a long comment on behalf of a number of SIG Node contributors](https://github.com/kubernetes/enhancements/issues/753#issuecomment-597372056) citing social, procedural, and technical concerns about what's going on with that— in particular, that it moved out of 1.18 and is now moving to 1.19. Did you have any thoughts on that?**
JORGE ALARCÓN: The sidecar enhancement has definitely been an interesting one. First off, thank you very much to Joseph Irving, the author of the KEP. And thank you very much to Tim Hockin, who voiced out the point of view of the approvers, maintainers of SIG Node. And I guess a little bit of context before we move on is, in the Kubernetes community, we have contributors, we have reviewers, and we have approvers.
Contributors are people who write PRs, who file issues, who troubleshoot issues. Reviewers are contributors who focus on one or multiple specific areas within the project, and then approvers are maintainers for the specific area, for one or multiple specific areas, of the project. So you can think of approvers as people who have write access in a repo or someplace within a repo.
The issue with the sidecar enhancement is that it has been deferred for multiple releases now, and that's been because there hasn't been a lot of collaboration between the KEP authors and the approvers for specific parts of the project. Something worthwhile to mention— and this was brought up during the original discussion— is this can obviously be frustrating for both contributors and for approvers. From the contributor's side of things, you are working on something. You are doing your best to make sure that it works.
And to build something that's going to be used by people, both from the approver side of things and, I think, for the most part, every single person in the Kubernetes community, we are all really excited to see this project grow. We want to help improve it, and we love when new people come in and work on new enhancements, bug fixes, and the like.
But one of the limitations is the day only has so many hours, and there are only so many things that we can work on at a time. So people prioritize in whatever way works best, and some things just fall behind. And a lot of the time, the things that fall behind are not because people don't want them to continue moving forward, but it's just a limited amount of resources, a limited amount of people.
And I think this discussion around the sidecar enhancement proposal has been very useful, and it points us to the need for more standardized mentoring programs. This is something that multiple SIGs are working on. For example, SIG Contribex, SIG Cluster Lifecycle, SIG Release. The idea is to standardize some sort of mentoring experience so that we can better prepare new contributors to become reviewers and ultimately approvers.
Because ultimately at the end of the day, if we have more people who are knowledgeable about Kubernetes, or even some specific area of Kubernetes, we can better distribute the load, and we can better collaborate on whatever new things come up. I think the sidecar enhancement has shown us mentoring is something worthwhile, and we need a lot more of it. Because as much work as we do, more things are going to continue popping in throughout the project. And the more people we have who are comfortable working in these really complicated areas of Kubernetes, the better off that we are going to be.
**ADAM GLICK: Was there any talk of delaying 1.18 due to the current worldwide health situation?**
JORGE ALARCÓN: We thought about it, and the plan was to just wait and see how people felt. Tried make sure that people were comfortable continuing to work and all the people were landing in new enhancements, or fixing tests, or members of the release team who were making sure that things were happening. We wanted to see that people were comfortable, that they could continue doing their job. And for a moment, I actually thought about delaying just outright— we're going to give it more time, and hopefully at some point, things are going to work out.
But people just continue doing their amazing work. There was no delay. There was no hitch throughout the process. So at some point, I just figured we stay with the current timeline and see how we went. And at this point, things are more or less set.
**ADAM GLICK: Amazing power of a distributed team.**
JORGE ALARCÓN: Yeah, definitely.
[LAUGHING]
**ADAM GLICK: [Taylor Dolezal was announced as the 1.19 release lead](https://twitter.com/alejandrox135/status/1239629281766096898). Do you know how that choice was made, and by whom?**
JORGE ALARCÓN: I actually got to choose the lead. The practice is the current lead for the release team is going to look at people and see, first off, who's interested and out of the people interested, who can do the job, who's comfortable enough with the release team, with the Kubernetes community at large who can actually commit the amount of hours throughout the next, hopefully, three months.
And for one, I think Taylor has been part of my team. So there is the release team. Then the release team has multiple subgroups. One of those subgroups is actually just for me and my shadows. So for this release, it was mrbobbytables and Taylor. And Taylor volunteered to take over 1.19, and I'm sure that he will do an amazing job.
**ADAM GLICK: I am as well. What advice will you give Taylor?**
JORGE ALARCÓN: Over-communicate as much as possible. Normally, if you made it to the point that you are the lead for a release, or even the shadow for a release, you more or less are familiar with a lot of the work— CI Signal, enhancements, documentation, and the like. And a lot of people, if they know how to do their job, they might tell themselves, yeah, I could do it— no need to worry about it. I'm just going to go ahead and sign this PR, debug this test, whatever.
But one of the interesting aspects is whenever we are actually working in a release, 50% of the work has to go into actually making the release happen. The other 50% of the work has to go into mentoring people, and making sure the newcomers, new members are able to learn everything that they need to learn to do your job, you being in the lead for a subgroup or the entire team. And whenever you actually see that things need to happen, just over-communicate.
Try to provide the opportunity for someone else to do the work, and over-communicate with them as much as possible to make sure that they are learning whatever it is that they need to learn. If neither you or the other person knows what's going on, then I can over-communicate, so someone hopefully will see your messages and come to the rescue. That happens a lot. There's a lot of really nice and kind people who will come out and tell you how something works, help you fix it.
**ADAM GLICK: If you were to sum up your experience running this release, what would it be?**
JORGE ALARCÓN: It's been super fun and a little bit stressing, to be honest. Being the release lead is definitely amazing. You're kind of sitting at the center of Kubernetes.
You not only see the people who are working on things— the things that are broken, and the users filling out issues, and saying what broke, and the like. But you also get the opportunity to work with a lot of people who do a lot of non-code related work. Docs is one of the most obvious things. There's a lot of work that goes into communications, contributor experience, public relations.
And being connected, getting to talk with those people mostly every other day, it's really fun. It's a really good experience in terms of becoming a better contributor to the community, but also taking some of that knowledge home with you and applying it somewhere else. If you are a software engineer, if you are a project manager, whatever, it's amazing how much you can learn.
**ADAM GLICK: I know the community likes to rotate around who are the release leads. But if you were given the opportunity to be a release lead for a future release of Kubernetes, would you do it again?**
JORGE ALARCÓN: Yeah, it's a fun job. To be honest, it can be really stressing. Especially, as I mentioned, at some point, most of that work is just going to be talking with people, and talking requires a lot more thought and effort than just sitting down and thinking about things sometimes. And some of that can be really stressful.
But the job itself, it is definitely fun. And at some distant point in the future, if for some reason it was a possibility, I will think about it. But definitely, as you mentioned, one thing that we try to do is cycle out, because I can have fun in it, and that's all good and nice. And hopefully I can help another release go out the door. But providing the opportunity for other people to learn I think is a lot more important than just being the lead itself.
---
_[Jorge Alarcón](https://twitter.com/alejandrox135) is a site reliability engineer with Searchable AI and served as the Kubernetes 1.18 release team lead._
_You can find the [Kubernetes Podcast from Google](http://www.kubernetespodcast.com/) at [@KubernetesPod](https://twitter.com/KubernetesPod) on Twitter, and you can [subscribe](https://kubernetespodcast.com/subscribe/) so you never miss an episode._

199
content/en/blog/_posts/2020-08-14-introducing-hierarchical-namespaces.md Normal file
View File

@ -0,0 +1,199 @@
---
layout: blog
title: "Introducing Hierarchical Namespaces"
date: 2020-08-14
---
**Author**: Adrian Ludwin (Google)
Safely hosting large numbers of users on a single Kubernetes cluster has always
been a troublesome task. One key reason for this is that different organizations
use Kubernetes in different ways, and so no one tenancy model is likely to suit
everyone. Instead, Kubernetes offers you building blocks to create your own
tenancy solution, such as Role Based Access Control (RBAC) and NetworkPolicies;
the better these building blocks, the easier it is to safely build a multitenant
cluster.
# Namespaces for tenancy
By far the most important of these building blocks is the namespace, which forms
the backbone of almost all Kubernetes control plane security and sharing
policies. For example, RBAC, NetworkPolicies and ResourceQuotas all respect
namespaces by default, and objects such as Secrets, ServiceAccounts and
Ingresses are freely usable _within_ any one namespace, but fully segregated
from _other_ namespaces.
Namespaces have two key properties that make them ideal for policy enforcement.
Firstly, they can be used to **represent ownership**. Most Kubernetes objects
_must_ be in a namespace, so if you use namespaces to represent ownership, you
can always count on there being an owner.
Secondly, namespaces have **authorized creation and use**. Only
highly-privileged users can create namespaces, and other users require explicit
permission to use those namespaces - that is, create, view or modify objects in
those namespaces. This allows them to be carefully created with appropriate
policies, before unprivileged users can create “regular” objects like pods and
services.
# The limits of namespaces
However, in practice, namespaces are not flexible enough to meet some common use
cases. For example, let’s say that one team owns several microservices with
different secrets and quotas. Ideally, they should place these services into
different namespaces in order to isolate them from each other, but this presents
two problems.
Firstly, these namespaces have no common concept of ownership, even though
they’re both owned by the same team. This means that if the team controls
multiple namespaces, not only does Kubernetes not have any record of their
common owner, but namespaced-scoped policies cannot be applied uniformly across
them.
Secondly, teams generally work best if they can operate autonomously, but since
namespace creation is highly privileged, it’s unlikely that any member of the
dev team is allowed to create namespaces. This means that whenever a team wants
a new namespace, they must raise a ticket to the cluster administrator. While
this is probably acceptable for small organizations, it generates unnecessary
toil as the organization grows.
# Introducing hierarchical namespaces
[Hierarchical
namespaces](https://github.com/kubernetes-sigs/multi-tenancy/blob/master/incubator/hnc/docs/user-guide/concepts.md#basic)
are a new concept developed by the [Kubernetes Working Group for Multi-Tenancy
(wg-multitenancy)](https://github.com/kubernetes-sigs/multi-tenancy) in order to
solve these problems. In its simplest form, a hierarchical namespace is a
regular Kubernetes namespace that contains a small custom resource that
identifies a single, optional, parent namespace. This establishes the concept of
ownership _across_ namespaces, not just _within_ them.
This concept of ownership enables two additional types of behaviours:
* **Policy inheritance:** if one namespace is a child of another, policy objects
such as RBAC RoleBindings are [copied from the parent to the
child](https://github.com/kubernetes-sigs/multi-tenancy/blob/master/incubator/hnc/docs/user-guide/concepts.md#basic-propagation).
* **Delegated creation:** you usually need cluster-level privileges to create a
namespace, but hierarchical namespaces adds an alternative:
[_subnamespaces_](https://github.com/kubernetes-sigs/multi-tenancy/blob/master/incubator/hnc/docs/user-guide/concepts.md#basic-subns),
which can be manipulated using only limited permissions in the parent
namespace.
This solves both of the problems for our dev team. The cluster administrator can
create a single “root” namespace for the team, along with all necessary
policies, and then delegate permission to create subnamespaces to members of
that team. Those team members can then create subnamespaces for their own use,
without violating the policies that were imposed by the cluster administrators.
# Hands-on with hierarchical namespaces
Hierarchical namespaces are provided by a Kubernetes extension known as the
[**Hierarchical Namespace
Controller**](https://github.com/kubernetes-sigs/multi-tenancy/tree/master/incubator/hnc),
or **HNC**. The HNC consists of two components:
* The **manager** runs on your cluster, manages subnamespaces, propagates policy
objects, ensures that your hierarchies are legal and manages extension points.
* The **kubectl plugin**, called `kubectl-hns`, makes it easy for users to
interact with the manager.
Both can be easily installed from the [releases page of our
repo](https://github.com/kubernetes-sigs/multi-tenancy/releases).
Let’s see HNC in action. Imagine that I do not have namespace creation
privileges, but I can view the namespace `team-a` and create subnamespaces
within it<sup>[1](#note-1)</sup>. Using the plugin, I can now say:
```bash
$ kubectl hns create svc1-team-a -n team-a
```
This creates a subnamespace called `svc1-team-a`. Note that since subnamespaces
are just regular Kubernetes namespaces, all subnamespace names must still be
unique.
I can view the structure of these namespaces by asking for a tree view:
```bash
$ kubectl hns tree team-a
# Output:
team-a
└── svc1-team-a
```
And if there were any policies in the parent namespace, these now appear in the
child as well<sup>[2](#note-2)</sup>. For example, let’s say that `team-a` had
an RBAC RoleBinding called `sres`. This rolebinding will also be present in the
subnamespace:
```bash
$ kubectl describe rolebinding sres -n svc1-team-a
# Output:
Name: sres
Labels: hnc.x-k8s.io/inheritedFrom=team-a # inserted by HNC
Annotations: <none>
Role:
Kind: ClusterRole
Name: admin
Subjects: ...
```
Finally, HNC adds labels to these namespaces with useful information about the
hierarchy which you can use to apply other policies. For example, you can create
the following NetworkPolicy:
```yaml
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: allow-team-a
namespace: team-a
spec:
ingress:
- from:
- namespaceSelector:
matchExpressions:
- key: 'team-a.tree.hnc.x-k8s.io/depth' # Label created by HNC
operator: Exists
```
This policy will both be propagated to all descendants of `team-a`, and will
_also_ allow ingress traffic between all of those namespaces. The “tree” label
can only be applied by HNC, and is guaranteed to reflect the latest hierarchy.
You can learn all about the features of HNC from the [user
guide](https://github.com/kubernetes-sigs/multi-tenancy/tree/master/incubator/hnc/docs/user-guide).
# Next steps and getting involved
If you think that hierarchical namespaces can work for your organization, [HNC
v0.5.1 is available on
GitHub](https://github.com/kubernetes-sigs/multi-tenancy/releases/tag/hnc-v0.5.1).
We’d love to know what you think of it, what problems you’re using it to solve
and what features you’d most like to see added. As with all early software, you
should be cautious about using HNC in production environments, but the more
feedback we get, the sooner we’ll be able to drive to HNC 1.0.
We’re also open to additional contributors, whether it’s to fix or report bugs,
or help prototype new features such as exceptions, improved monitoring,
hierarchical resource quotas or fine-grained configuration.
Please get in touch with us via our
[repo](https://github.com/kubernetes-sigs/multi-tenancy), [mailing
list](https://groups.google.com/g/kubernetes-wg-multitenancy) or on
[Slack](https://kubernetes.slack.com/messages/wg-multitenancy) - we look forward
to hearing from you!
---
_[Adrian Ludwin](https://twitter.com/aludwin) is a software engineer and the
tech lead for the Hierarchical Namespace Controller._
<a name="note-1"/>
_Note 1: technically, you create a small object called a "subnamespace anchor"
in the parent namespace, and then HNC creates the subnamespace for you._
<a name="note-2"/>
_Note 2: By default, only RBAC Roles and RoleBindings are propagated, but you
can configure HNC to propagate any namespaced Kubernetes object._

47
content/en/blog/_posts/2020-08-21-Moving-Forward-From-Beta/feature_stages.svg Normal file
View File

@ -0,0 +1,47 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg width="120mm" height="50mm" version="1.1" viewBox="0 0 120 50" xmlns="http://www.w3.org/2000/svg" xmlns:cc="http://creativecommons.org/ns#" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
<style>
text {
font-family: "Open Sans", sans-serif;
font-weight: initial;
font-size: 3px;
letter-spacing: 0;
word-spacing: 0;
}
@media (max-width: 1200px) {
text {
font-size: 3.5px;
}
}
@media (max-width: 600px) {
text {
font-size: 4px;
}
}
</style>
<path d="m80.515 23.737c-0.07154 5e-3 -0.10954 0.08683-0.06718 0.14469 0.21444 0.29879 0.32852 0.6427 0.35295 0.99012h-10.835v0.26407h10.835c-0.02397 0.34688-0.13762 0.68768-0.35295 0.9803-0.05673 0.0732 0.01926 0.17452 0.10542 0.14056l3.1853-1.172c0.08057-0.02881 0.08057-0.14275 0-0.17157l-3.1853-1.17c-0.0081-0.0033-0.01662-0.0054-0.02532-0.0062-0.0043-3.17e-4 -0.0086-3.17e-4 -0.01291 0z" color="#000000" dominant-baseline="auto" fill-rule="evenodd" shape-rendering="auto" solid-color="#000000" stop-color="#000000" style="inline-size:0;isolation:auto;mix-blend-mode:normal;shape-margin:0;shape-padding:0;"/>
<path d="m43.342 23.737c-0.07154 5e-3 -0.10954 0.08683-0.06718 0.14469 0.21444 0.29879 0.32852 0.6427 0.35295 0.99012h-10.835v0.26407h10.835c-0.02397 0.34688-0.13762 0.68768-0.35295 0.9803-0.05673 0.0732 0.01926 0.17452 0.10542 0.14056l3.1853-1.172c0.08057-0.02881 0.08057-0.14275 0-0.17157l-3.1853-1.17c-0.0081-0.0033-0.01662-0.0054-0.02532-0.0062-0.0043-3.17e-4 -0.0086-3.17e-4 -0.01291 0z" color="#000000" dominant-baseline="auto" fill-rule="evenodd" shape-rendering="auto" solid-color="#000000" stop-color="#000000" style="inline-size:0;isolation:auto;mix-blend-mode:normal;shape-margin:0;shape-padding:0;"/>
<rect x="9.4527" y="16.25" width="23.568" height="17.5" fill="#3573e3" />
<path d="m16.434 26.129q0.16298 0 0.28646-0.0049 0.12841-0.0099 0.21237-0.02963v-0.76553q-0.04939-0.02469-0.16298-0.03951-0.10866-0.01975-0.2667-0.01975-0.10372 0-0.22225 0.01482-0.11359 0.01482-0.21237 0.0642-0.09384 0.04445-0.15804 0.12841-0.06421 0.07902-0.06421 0.21237 0 0.24694 0.15804 0.34572 0.15804 0.09384 0.42968 0.09384zm-0.03951-2.3015q0.27658 0 0.46426 0.07408 0.19262 0.06915 0.30621 0.2025 0.11853 0.12841 0.16792 0.31115 0.04939 0.1778 0.04939 0.39511v1.6051q-0.05927 0.0099-0.16792 0.02963-0.10372 0.01482-0.23707 0.02963-0.13335 0.01482-0.2914 0.02469-0.1531 0.01482-0.30621 0.01482-0.21731 0-0.40005-0.04445t-0.31609-0.13829q-0.13335-0.09878-0.20743-0.25682-0.07408-0.15804-0.07408-0.3803 0-0.21237 0.08396-0.36548 0.0889-0.15311 0.23707-0.24694 0.14817-0.09384 0.34572-0.13829 0.19756-0.04445 0.41487-0.04445 0.06915 0 0.14323 0.0099 0.07408 0.0049 0.13829 0.01975 0.06914 0.0099 0.11853 0.01976 0.04939 0.0099 0.06914 0.01482v-0.12841q0-0.1136-0.02469-0.22225-0.02469-0.11359-0.0889-0.19756-0.0642-0.0889-0.1778-0.13829-0.10866-0.05433-0.28646-0.05433-0.22719 0-0.40005 0.03457-0.16792 0.02963-0.25188 0.06421l-0.05433-0.3803q0.0889-0.03951 0.29633-0.07408 0.20743-0.03951 0.44944-0.03951z" fill="#fff" />
<path d="m18.911 26.509q-0.42474-0.0099-0.60254-0.18274-0.1778-0.17286-0.1778-0.53834v-3.0819l0.45932-0.07902v3.0868q0 0.11359 0.01976 0.18768 0.01975 0.07408 0.06421 0.11853 0.04445 0.04445 0.11853 0.06914 0.07408 0.01975 0.18274 0.03457z" fill="#fff" />
<path d="m21.274 25.18q0-0.44944-0.22225-0.69144-0.22225-0.242-0.59267-0.242-0.20743 0-0.32597 0.01482-0.1136 0.01482-0.18274 0.03457v1.62q0.08396 0.06914 0.242 0.13335t0.34572 0.06421q0.19756 0 0.33584-0.06914 0.14323-0.07408 0.23213-0.19756 0.0889-0.12841 0.12841-0.29633 0.03951-0.17286 0.03951-0.37042zm0.47907 0q0 0.29139-0.07902 0.53834-0.07408 0.24694-0.22225 0.42474-0.14817 0.1778-0.36548 0.27658-0.21237 0.09878-0.48895 0.09878-0.22225 0-0.39511-0.05927-0.16792-0.05927-0.25188-0.11359v1.0273h-0.45932v-3.4029q0.16298-0.03951 0.40499-0.08396 0.24694-0.04939 0.56797-0.04939 0.29633 0 0.5334 0.09384 0.23707 0.09384 0.40499 0.2667 0.16792 0.17286 0.25682 0.42474 0.09384 0.24694 0.09384 0.5581z" fill="#fff" />
<path d="m22.404 26.459v-3.7536l0.45932-0.07902v1.3137q0.12841-0.04939 0.27164-0.07408 0.14817-0.02963 0.29139-0.02963 0.30621 0 0.50871 0.0889 0.2025 0.08396 0.32103 0.242 0.12347 0.15311 0.17286 0.37042 0.04939 0.21731 0.04939 0.47907v1.4422h-0.45932v-1.3434q0-0.23707-0.03457-0.40499-0.02963-0.16792-0.10372-0.27164-0.07408-0.10372-0.19756-0.14817-0.12347-0.04939-0.30621-0.04939-0.07408 0-0.15311 0.0099-0.07902 0.0099-0.15311 0.02469-0.06914 0.0099-0.12841 0.02469-0.05433 0.01482-0.07902 0.0247v2.1336z" fill="#fff" />
<path d="m26.09 26.129q0.16298 0 0.28646-0.0049 0.12841-0.0099 0.21237-0.02963v-0.76553q-0.04939-0.02469-0.16298-0.03951-0.10866-0.01975-0.2667-0.01975-0.10372 0-0.22225 0.01482-0.1136 0.01482-0.21237 0.0642-0.09384 0.04445-0.15804 0.12841-0.06421 0.07902-0.06421 0.21237 0 0.24694 0.15804 0.34572 0.15804 0.09384 0.42968 0.09384zm-0.03951-2.3015q0.27658 0 0.46426 0.07408 0.19262 0.06915 0.30621 0.2025 0.11853 0.12841 0.16792 0.31115 0.04939 0.1778 0.04939 0.39511v1.6051q-0.05927 0.0099-0.16792 0.02963-0.10372 0.01482-0.23707 0.02963-0.13335 0.01482-0.29139 0.02469-0.15311 0.01482-0.30621 0.01482-0.21731 0-0.40005-0.04445t-0.31609-0.13829q-0.13335-0.09878-0.20743-0.25682-0.07408-0.15804-0.07408-0.3803 0-0.21237 0.08396-0.36548 0.0889-0.15311 0.23707-0.24694 0.14817-0.09384 0.34572-0.13829 0.19756-0.04445 0.41487-0.04445 0.06914 0 0.14323 0.0099 0.07408 0.0049 0.13829 0.01975 0.06914 0.0099 0.11853 0.01976 0.04939 0.0099 0.06914 0.01482v-0.12841q0-0.1136-0.02469-0.22225-0.02469-0.11359-0.0889-0.19756-0.06421-0.0889-0.1778-0.13829-0.10866-0.05433-0.28646-0.05433-0.22719 0-0.40005 0.03457-0.16792 0.02963-0.25188 0.06421l-0.05433-0.3803q0.0889-0.03951 0.29633-0.07408 0.20743-0.03951 0.44944-0.03951z" fill="#fff" />
<g transform="translate(-1.59)">
<rect x="48.216" y="16.25" width="23.568" height="17.5" fill="#3573e3" />
<path d="m55.727 24.432q0.08396-0.05433 0.25188-0.11359 0.17286-0.05927 0.39511-0.05927 0.27658 0 0.48895 0.09878 0.21731 0.09878 0.36548 0.27658 0.14817 0.1778 0.22225 0.42474 0.07902 0.24695 0.07902 0.54328 0 0.31115-0.09384 0.56304-0.0889 0.24694-0.25682 0.4198t-0.40499 0.2667q-0.23707 0.09384-0.5334 0.09384-0.32103 0-0.56797-0.04445-0.24694-0.04445-0.40499-0.0889v-3.6795l0.45932-0.07902zm0 2.0546q0.06914 0.01976 0.19262 0.03951 0.12841 0.01482 0.31609 0.01482 0.37042 0 0.59267-0.242 0.22225-0.24695 0.22225-0.69638 0-0.19756-0.03951-0.37042-0.03951-0.17286-0.12841-0.29633-0.0889-0.12841-0.23213-0.19756-0.13829-0.07408-0.33584-0.07408-0.18768 0-0.34572 0.06421-0.15804 0.0642-0.24201 0.13335z" fill="#fff" />
<path d="m58.037 25.607q0-0.34078 0.09878-0.59267 0.09878-0.25682 0.26176-0.42474 0.16298-0.16792 0.37536-0.25188 0.21237-0.08396 0.43462-0.08396 0.51858 0 0.79516 0.32597 0.27658 0.32103 0.27658 0.98284 0 0.02963 0 0.07902 0 0.04445-0.0049 0.08396h-1.7582q0.02963 0.40005 0.23213 0.60748 0.2025 0.20743 0.63218 0.20743 0.24201 0 0.40499-0.03951 0.16792-0.04445 0.25188-0.08396l0.06421 0.38523q-0.08396 0.04445-0.29633 0.09384-0.20743 0.04939-0.47413 0.04939-0.33584 0-0.58279-0.09878-0.242-0.10372-0.40005-0.28152-0.15804-0.1778-0.23707-0.41981-0.07408-0.24694-0.07408-0.53834zm1.7632-0.25188q0.0049-0.31115-0.15804-0.5087-0.15804-0.2025-0.43956-0.2025-0.15804 0-0.28152 0.06421-0.11853 0.05927-0.2025 0.15804-0.08396 0.09878-0.13335 0.22719-0.04445 0.12841-0.05927 0.26176z" fill="#fff" />
<path d="m61.369 24.318h0.97296v0.38523h-0.97296v1.1853q0 0.19262 0.02963 0.32103 0.02963 0.12347 0.0889 0.19756 0.05927 0.06914 0.14817 0.09878 0.0889 0.02963 0.20743 0.02963 0.20743 0 0.33091-0.04445 0.12841-0.04939 0.1778-0.06914l0.0889 0.38029q-0.06914 0.03457-0.24201 0.08396-0.17286 0.05433-0.39511 0.05433-0.26176 0-0.43462-0.06421-0.16792-0.06914-0.27164-0.20249-0.10372-0.13335-0.14817-0.32597-0.03951-0.19756-0.03951-0.45438v-2.2916l0.45932-0.07902z" fill="#fff" />
<path d="m63.796 26.556q0.16298 0 0.28646-0.0049 0.12841-0.0099 0.21237-0.02963v-0.76553q-0.04939-0.0247-0.16298-0.03951-0.10866-0.01976-0.2667-0.01976-0.10372 0-0.22225 0.01482-0.1136 0.01482-0.21237 0.06421-0.09384 0.04445-0.15804 0.12841-0.06421 0.07902-0.06421 0.21237 0 0.24694 0.15804 0.34572 0.15804 0.09384 0.42968 0.09384zm-0.03951-2.3015q0.27658 0 0.46426 0.07408 0.19262 0.06914 0.30621 0.20249 0.11853 0.12841 0.16792 0.31115 0.04939 0.1778 0.04939 0.39511v1.6051q-0.05927 0.0099-0.16792 0.02963-0.10372 0.01482-0.23707 0.02963-0.13335 0.01482-0.29139 0.0247-0.15311 0.01482-0.30621 0.01482-0.21731 0-0.40005-0.04445-0.18274-0.04445-0.31609-0.13829-0.13335-0.09878-0.20743-0.25682-0.07408-0.15804-0.07408-0.38029 0-0.21237 0.08396-0.36548 0.0889-0.15311 0.23707-0.24694 0.14817-0.09384 0.34572-0.13829 0.19756-0.04445 0.41487-0.04445 0.06914 0 0.14323 0.0099 0.07408 0.0049 0.13829 0.01976 0.06915 0.0099 0.11853 0.01975 0.04939 0.0099 0.06914 0.01482v-0.12841q0-0.1136-0.02469-0.22225-0.0247-0.11359-0.0889-0.19756-0.06421-0.0889-0.1778-0.13829-0.10866-0.05433-0.28646-0.05433-0.22719 0-0.40005 0.03457-0.16792 0.02963-0.25188 0.06421l-0.05433-0.38029q0.0889-0.03951 0.29633-0.07408 0.20743-0.03951 0.44944-0.03951z" fill="#fff" />
</g>
<rect x="83.799" y="16.25" width="23.568" height="17.5" fill="#3573e3" />
<path d="m89.795 26.556q0.28152 0 0.41487-0.07408 0.13829-0.07408 0.13829-0.23707 0-0.16792-0.13335-0.2667-0.13335-0.09878-0.43956-0.22225-0.14817-0.05927-0.28646-0.11853-0.13335-0.06421-0.23213-0.14817-0.09878-0.08396-0.15804-0.2025-0.05927-0.11853-0.05927-0.29139 0-0.34078 0.25188-0.53834 0.25188-0.2025 0.6865-0.2025 0.10866 0 0.21731 0.01482 0.10866 0.0099 0.2025 0.02963 0.09384 0.01482 0.16298 0.03457 0.07408 0.01976 0.1136 0.03457l-0.08396 0.39511q-0.07408-0.03951-0.23213-0.07902-0.15804-0.04445-0.3803-0.04445-0.19262 0-0.33584 0.07902-0.14323 0.07408-0.14323 0.23707 0 0.08396 0.02963 0.14816 0.03457 0.06421 0.09878 0.11854 0.06914 0.04939 0.16792 0.09384 0.09878 0.04445 0.23707 0.09384 0.18274 0.06914 0.32597 0.13829 0.14323 0.06421 0.242 0.15311 0.10372 0.0889 0.15804 0.21731 0.05433 0.12347 0.05433 0.30621 0 0.3556-0.2667 0.53834-0.26176 0.18274-0.75071 0.18274-0.34078 0-0.5334-0.05927-0.19262-0.05433-0.26176-0.08396l0.08396-0.39511q0.07902 0.02963 0.25188 0.0889 0.17286 0.05927 0.45932 0.05927z" fill="#fff" />
<path d="m91.826 24.318h0.97296v0.38523h-0.97296v1.1853q0 0.19262 0.02963 0.32103 0.02963 0.12347 0.0889 0.19756 0.05927 0.06914 0.14817 0.09878 0.0889 0.02963 0.20743 0.02963 0.20743 0 0.33091-0.04445 0.12841-0.04939 0.1778-0.06914l0.0889 0.38029q-0.06915 0.03457-0.24201 0.08396-0.17286 0.05433-0.39511 0.05433-0.26176 0-0.43462-0.06421-0.16792-0.06914-0.27164-0.20249-0.10372-0.13335-0.14817-0.32597-0.03951-0.19756-0.03951-0.45438v-2.2916l0.45932-0.07902z" fill="#fff" />
<path d="m94.253 26.556q0.16298 0 0.28646-0.0049 0.12841-0.0099 0.21237-0.02963v-0.76553q-0.04939-0.0247-0.16298-0.03951-0.10866-0.01976-0.2667-0.01976-0.10372 0-0.22225 0.01482-0.1136 0.01482-0.21237 0.06421-0.09384 0.04445-0.15804 0.12841-0.06421 0.07902-0.06421 0.21237 0 0.24694 0.15804 0.34572 0.15804 0.09384 0.42968 0.09384zm-0.03951-2.3015q0.27658 0 0.46426 0.07408 0.19262 0.06914 0.30621 0.20249 0.11853 0.12841 0.16792 0.31115 0.04939 0.1778 0.04939 0.39511v1.6051q-0.05927 0.0099-0.16792 0.02963-0.10372 0.01482-0.23707 0.02963-0.13335 0.01482-0.29139 0.0247-0.15311 0.01482-0.30621 0.01482-0.21731 0-0.40005-0.04445-0.18274-0.04445-0.31609-0.13829-0.13335-0.09878-0.20743-0.25682-0.07408-0.15804-0.07408-0.38029 0-0.21237 0.08396-0.36548 0.0889-0.15311 0.23707-0.24694 0.14817-0.09384 0.34572-0.13829 0.19756-0.04445 0.41487-0.04445 0.06914 0 0.14323 0.0099 0.07408 0.0049 0.13829 0.01976 0.06915 0.0099 0.11853 0.01975 0.04939 0.0099 0.06914 0.01482v-0.12841q0-0.1136-0.02469-0.22225-0.02469-0.11359-0.0889-0.19756-0.06421-0.0889-0.1778-0.13829-0.10866-0.05433-0.28646-0.05433-0.22719 0-0.40005 0.03457-0.16792 0.02963-0.25188 0.06421l-0.05433-0.38032q0.0889-0.03951 0.29633-0.07408 0.20743-0.03951 0.44944-0.03951z" fill="#fff" />
<path d="m96.419 24.432q0.08396-0.05433 0.25188-0.11359 0.17286-0.05927 0.39511-0.05927 0.27658 0 0.48895 0.09878 0.21731 0.09878 0.36548 0.27658 0.14817 0.1778 0.22225 0.42474 0.07902 0.24695 0.07902 0.54328 0 0.31115-0.09384 0.56304-0.0889 0.24694-0.25682 0.4198t-0.40499 0.2667q-0.23707 0.09384-0.5334 0.09384-0.32103 0-0.56797-0.04445-0.24694-0.04445-0.40499-0.0889v-3.6795l0.45932-0.07902zm0 2.0546q0.06914 0.01976 0.19262 0.03951 0.12841 0.01482 0.31609 0.01482 0.37042 0 0.59267-0.242 0.22225-0.24695 0.22225-0.69638 0-0.19756-0.03951-0.37042-0.03951-0.17286-0.12841-0.29633-0.0889-0.12841-0.23213-0.19756-0.13829-0.07408-0.33584-0.07408-0.18768 0-0.34572 0.06421-0.15804 0.0642-0.24201 0.13335z" fill="#fff" />
<path d="m99.643 26.936q-0.42474-0.0099-0.60254-0.18274-0.1778-0.17286-0.1778-0.53834v-3.0819l0.45932-0.07902v3.0868q0 0.1136 0.01976 0.18768 0.01976 0.07408 0.06421 0.11853 0.04445 0.04445 0.11853 0.06914 0.07408 0.01975 0.18274 0.03457z" fill="#fff" />
<path d="m100.08 25.607q0-0.34078 0.0988-0.59267 0.0988-0.25682 0.26176-0.42474 0.16298-0.16792 0.37535-0.25188 0.21238-0.08396 0.43463-0.08396 0.51858 0 0.79516 0.32597 0.27658 0.32103 0.27658 0.98284 0 0.02963 0 0.07902 0 0.04445-5e-3 0.08396h-1.7582q0.0296 0.40005 0.23213 0.60748 0.20249 0.20743 0.63218 0.20743 0.242 0 0.40499-0.03951 0.16792-0.04445 0.25188-0.08396l0.0642 0.38523q-0.084 0.04445-0.29634 0.09384-0.20743 0.04939-0.47413 0.04939-0.33585 0-0.58279-0.09878-0.24201-0.10372-0.40005-0.28152t-0.23707-0.41981q-0.0741-0.24694-0.0741-0.53834zm1.7632-0.25188q5e-3 -0.31115-0.15804-0.5087-0.15805-0.2025-0.43956-0.2025-0.15805 0-0.28152 0.06421-0.11853 0.05927-0.20249 0.15804-0.084 0.09878-0.13335 0.22719-0.0445 0.12841-0.0593 0.26176z" fill="#fff" />
<text x="82.5" y="38.5" fill="#000" xml:space="preserve"><tspan x="94.5" y="38.5" text-anchor="middle">(general availability)</tspan></text>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 14 KiB

111
content/en/blog/_posts/2020-08-21-Moving-Forward-From-Beta/index.md Normal file
View File

@ -0,0 +1,111 @@
---
layout: blog
title: "Moving Forward From Beta"
date: 2020-08-21
slug: moving-forward-from-beta
---
**Author**: Tim Bannister, The Scale Factory
In Kubernetes, features follow a defined
[lifecycle](/docs/reference/command-line-tools-reference/feature-gates/#feature-stages).
First, as the twinkle of an eye in an interested developer. Maybe, then,
sketched in online discussions, drawn on the online equivalent of a cafe
napkin. This rough work typically becomes a
[Kubernetes Enhancement Proposal](https://github.com/kubernetes/enhancements/blob/master/keps/0001-kubernetes-enhancement-proposal-process.md#kubernetes-enhancement-proposal-process) (KEP), and
from there it usually turns into code.
For Kubernetes v1.20 and onwards, we're focusing on helping that code
graduate into stable features.
That lifecycle I mentioned runs as follows:
![Alpha → Beta → General Availability](feature_stages.svg)
Usually, alpha features aren't enabled by default. You turn them on by setting a feature
gate; usually, by setting a command line flag on each of the components that use the
feature.
(If you use Kubernetes through a managed service offering such as AKS, EKS, GKE, etc then
the vendor who runs that service may have decided what feature gates are enabled for you).
There's a defined process for graduating an existing, alpha feature into the beta phase.
This is important because **beta features are enabled by default**, with the feature flag still
there so cluster operators can opt out if they want.
A similar but more thorough set of graduation criteria govern the transition to general
availability (GA), also known as "stable". GA features are part of Kubernetes, with a
commitment that they are staying in place throughout the current major version.
Having beta features on by default lets Kubernetes and its contributors get valuable
real-world feedback. However, there's a mismatch of incentives. Once a feature is enabled
by default, people will use it. Even if there might be a few details to shake out,
the way Kubernetes' REST APIs and conventions work mean that any future stable API is going
to be compatible with the most recent beta API: your API objects won't stop working when
a beta feature graduates to GA.
For the API and its resources in particular, there's a much less strong incentive to move
features from beta to GA than from alpha to beta. Vendors who want a particular feature
have had good reason to help get code to the point where features are enabled by default,
and beyond that the journey has been less clear.
KEPs track more than code improvements. Essentially, anything that would need
communicating to the wider community merits a KEP. That said, most KEPs cover
Kubernetes features (and the code to implement them).
You might know that [Ingress](/docs/concepts/services-networking/ingress/)
has been in Kubernetes for a while, but did you realize that it actually went beta in 2015? To help
drive things forward, Kubernetes' Architecture Special Interest Group (SIG) have a new approach in
mind.
## Avoiding permanent beta
For Kubernetes REST APIs, when a new feature's API reaches beta, that starts a countdown.
The beta-quality API now has **nine calendar months** to either:
- reach GA, and deprecate the beta, or
- have a new beta version (_and deprecate the previous beta_).
To be clear, at this point **only REST APIs are affected**. For example, _APIListChunking_ is
a beta feature but isn't itself a REST API. Right now there are no plans to automatically
deprecate _APIListChunking_ nor any other features that aren't REST APIs.
If a REST API reaches the end of that 9 month countdown, then the next Kubernetes release
will deprecate that API version. There's no option for the REST API to stay at the same
beta version beyond the first Kubernetes release to come out after the 9 month window.
### What this means for you
If you're using Kubernetes, there's a good chance that you're using a beta feature. Like
I said, there are lots of them about.
As well as Ingress, you might be using [CronJob](/docs/concepts/workloads/controllers/cron-jobs/),
or [PodSecurityPolicy](/docs/concepts/policy/pod-security-policy/), or others.
There's an even bigger chance that you're running on a control plane with at least one beta
feature enabled.
If you're using or generating Kubernetes manifests that use beta APIs like Ingress, you'll
need to plan to revise those. The current APIs are going to be deprecated following a
schedule (the 9 months I mentioned earlier) and after a further 9 months those deprecated
APIs will be removed. At that point, to stay current with Kubernetes, you should already
have migrated.
### What this means for Kubernetes contributors
The motivation here seems pretty clear: get features stable. Guaranteeing that beta
features will be deprecated adds a pretty big incentive so that people who want the
feature continue their effort until the code, documentation and tests are ready for this
feature to graduate to stable, backed by several Kubernetes' releases of evidence in
real-world use.
### What this means for the ecosystem
In my opinion, these harsh-seeming measures make a lot of sense, and are going to be
good for Kubernetes. Deprecating existing APIs, through a rule that applies across all
the different Special Interest Groups (SIGs), helps avoid stagnation and encourages
fixes.
Let's say that an API goes to beta and then real-world experience shows that it
just isn't right - that, fundamentally, the API has shortcomings. With that 9 month
countdown ticking, the people involved have the means and the justification to revise
and release an API that deals with the problem cases. Anyone who wants to live with
the deprecated API is welcome to - Kubernetes is open source - but their needs do not
have to hold up progress on the feature.

127
content/en/blog/_posts/2020-08-26-kubernetes-release-1.19.md Normal file
View File

@ -0,0 +1,127 @@
---
layout: blog
title: 'Kubernetes 1.19: Accentuate the Paw-sitive'
date: 2020-08-26
slug: kubernetes-release-1.19-accentuate-the-paw-sitive
---
**Authors:** [Kubernetes 1.19 Release Team](https://github.com/kubernetes/sig-release/blob/master/releases/release-1.19/release_team.md)
Finally, we have arrived with Kubernetes 1.19, the second release for 2020, and by far the longest release cycle lasting 20 weeks in total. It consists of 34 enhancements: 10 enhancements are moving to stable, 15 enhancements in beta, and 9 enhancements in alpha.
The 1.19 release was quite different from a regular release due to COVID-19, the George Floyd protests, and several other global events that we experienced as a release team. Due to these events, we made the decision to adjust our timeline and allow the SIGs, Working Groups, and contributors more time to get things done. The extra time also allowed for people to take time to focus on their lives outside of the Kubernetes project, and ensure their mental wellbeing was in a good place.
Contributors are the heart of Kubernetes, not the other way around. The Kubernetes code of conduct asks that people be excellent to one another and despite the unrest in our world, we saw nothing but greatness and humility from the community.
## Major Themes
### Increase Kubernetes support window to one year
A survey conducted in early 2019 by the [Long Term Support (LTS) working group](https://github.com/kubernetes/community/tree/master/wg-lts#readme) showed that a significant subset of Kubernetes end-users fail to upgrade within the current 9-month support period.
This, and other responses from the survey, suggest that 30% of users would be able to keep their deployments on supported versions if the patch support period were extended to 12-14 months. This appears to be true regardless of whether the users are on self build or commercially vendored distributions. An extension would thus lead to more than 80% of users being on supported versions, instead of the 50-60% we have now.
A yearly support period provides the cushion end-users appear to desire, and is more in harmony with familiar annual planning cycles.
From Kubernetes version 1.19 on, the support window will be extended to one year.
### Storage capacity tracking
Traditionally, the Kubernetes scheduler was based on the assumptions that additional persistent storage is available everywhere in the cluster and has infinite capacity. Topology constraints addressed the first point, but up to now pod scheduling was still done without considering that the remaining storage capacity may not be enough to start a new pod. [Storage capacity tracking](/docs/concepts/storage/storage-capacity/), a new alpha feature, addresses that by adding an API for a CSI driver to report storage capacity and uses that information in the Kubernetes scheduler when choosing a node for a pod. This feature serves as a stepping stone for supporting dynamic provisioning for local volumes and other volume types that are more capacity constrained.
#### Generic ephemeral volumes
Kubernetes provides volume plugins whose lifecycle is tied to a pod and can be used as scratch space (e.g. the builtin `emptydir` volume type) or to load some data in to a pod (e.g. the builtin `configmap` and `secret` volume types, or “CSI inline volumes”). The new [generic ephemeral volumes](/docs/concepts/storage/ephemeral-volumes/#generic-ephemeral-volumes) alpha feature allows any existing storage driver that supports dynamic provisioning to be used as an ephemeral volume with the volume’s lifecycle bound to the Pod.
It can be used to provide scratch storage that is different from the root disk, for example persistent memory, or a separate local disk on that node.
All StorageClass parameters for volume provisioning are supported.
All features supported with PersistentVolumeClaims are supported, such as storage capacity tracking, snapshots and restore, and volume resizing.
#### CSI Volume Health Monitoring
The alpha version of CSI health monitoring is being released with Kubernetes 1.19. This feature enables CSI Drivers to share abnormal volume conditions from the underlying storage systems with Kubernetes so that they can be reported as events on PVCs or Pods. This feature serves as a stepping stone towards programmatic detection and resolution of individual volume health issues by Kubernetes.
### Ingress graduates to General Availability
In terms of moving the Ingress API towards GA, the API itself has been available in beta for so long that it has attained de facto GA status through usage and adoption (both by users and by load balancer / ingress controller providers). Abandoning it without a full replacement is not a viable approach. It is clearly a useful API and captures a non-trivial set of use cases. At this point, it seems more prudent to declare the current API as something the community will support as a V1, codifying its status, while working on either a V2 Ingress API or an entirely different API with a superset of features.
### Structured logging
Before v1.19, logging in the Kubernetes control plane couldn't guarantee any uniform structure for log messages and references to Kubernetes objects in those logs. This makes parsing, processing, storing, querying and analyzing logs hard and forces administrators and developers to rely on ad-hoc solutions in most cases based on some regular expressions. Due to those problems any analytical solution based on those logs is hard to implement and maintain.
#### New klog methods
This Kubernetes release introduces new methods to the _klog_ library that provide a more structured interface for formatting log messages. Each existing formatted log method (`Infof`, `Errorf`) is now matched by a structured method (`InfoS`, `ErrorS`). The new logging methods accept log messages as a first argument and a list of key-values pairs as a variadic second argument. This approach allows incremental adoption of structured logging without converting **all** of Kubernetes to a new API at one time.
### Client TLS certificate rotation for kubelet
A kubelet authenticates the kubelet to the kube-apiserver using a private key and certificate. The certificate is supplied to the kubelet when it is first booted, via an out-of-cluster mechanism. Since Kubernetes v1.8, clusters have included a (beta) process for obtaining the initial cert/key pair and rotating it as expiration of the certificate approaches. In Kubernetes v1.19 this graduates to stable.
During the kubelet start-up sequence, the filesystem is scanned for an existing cert/key pair, which is managed by the certificate manager. In the case that a cert/key is available it will be loaded. If not, the kubelet checks its config file for an encoded certificate value or a file reference in the kubeconfig. If the certificate is a bootstrap certificate, this will be used to generate a key, create a certificate signing request and request a signed certificate from the API server.
When an expiration approaches the cert manager takes care of providing the correct certificate, generating new private keys and requesting new certificates. With the kubelet requesting certificates be signed as part of its boot sequence, and on an ongoing basis, certificate signing requests from the kubelet need to be auto approved to make cluster administration manageable.
## Other Updates
### Graduated to Stable
* [Seccomp](https://github.com/kubernetes/enhancements/issues/135)
* [Kubelet client TLS certificate rotation](https://github.com/kubernetes/enhancements/issues/266)
* [Limit node access to API](https://github.com/kubernetes/enhancements/issues/279)
* [Redesign Event API](https://github.com/kubernetes/enhancements/issues/383)
* [Graduate Ingress to V1](https://github.com/kubernetes/enhancements/issues/1453)
* [CertificateSigningRequest API](https://github.com/kubernetes/enhancements/issues/1513)
* [Building Kubelet without Docker](https://github.com/kubernetes/enhancements/issues/1547)
### Major Changes
* [Node Topology Manager](https://github.com/kubernetes/enhancements/issues/693)
* [New Endpoint API](https://github.com/kubernetes/enhancements/issues/752)
* [Increase Kubernetes support window to one year](https://github.com/kubernetes/enhancements/issues/1498)
### Other Notable Features
* [Run multiple Scheduling Profiles](https://github.com/kubernetes/enhancements/issues/1451)
* [CertificateSigningRequest API](https://github.com/kubernetes/enhancements/issues/1513)
* [Immutable Secrets and ConfigMaps](https://github.com/kubernetes/enhancements/issues/1412)
## Release Notes
Check out the full details of the Kubernetes 1.19 release in our [release notes](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.19.md).
## Availability
Kubernetes 1.19 is available for download on [GitHub](https://github.com/kubernetes/kubernetes/releases/tag/v1.19.0). To get started with Kubernetes, check out these [interactive tutorials](https://kubernetes.io/docs/tutorials/) or run local Kubernetes clusters using Docker container “nodes” with [KinD](https://kind.sigs.k8s.io/) (Kubernetes in Docker). You can also easily install 1.19 using [kubeadm](https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/).
## Release Team
This release is made possible through the efforts of hundreds of individuals who contributed both technical and non-technical content. Special thanks to the [release team](https://github.com/kubernetes/sig-release/blob/master/releases/release-1.19/release_team.md) led by Taylor Dolezal, Senior Developer Advocate at HashiCorp. The 34 release team members coordinated many aspects of the release, from documentation to testing, validation, and feature completeness.
As the Kubernetes community has grown, our release process represents an amazing demonstration of collaboration in open source software development. Kubernetes continues to gain new users at a rapid pace. This growth creates a positive feedback cycle where more contributors commit code creating a more vibrant ecosystem. Kubernetes has had over [49,000 individual contributors](https://k8s.devstats.cncf.io/d/24/overall-project-statistics?orgId=1) to date and an active community of more than 3,000 people.
## Release Logo
All of you inspired this Kubernetes 1.19 release logo! This release was a bit more of a marathon and a testament to when the world is a wild place, we can come together and do unbelievable things.
![Kubernetes 1.19 Release Logo](/images/blog/2020-08-26-kubernetes-1.19-release-announcement/accentuate.png)
"Accentuate the Paw-sitive" was chosen as the release theme because it captures the positive outlook that the release team had, despite the state of the world. The characters pictured in the 1.19 logo represent everyone's personalities on our release team, from emo to peppy, and beyond!
About the designer: Hannabeth Lagerlof is a Visual Designer based in Los Angeles, California, and she has an extensive background in Environments and Graphic Design. Hannabeth creates art and user experiences that inspire connection. You can find Hannabeth on Twitter as @emanate_design.
## The Long Run
The release was also different from the enhancements side of things. Traditionally, we have had 3-4 weeks between the call for enhancements and Enhancements Freeze, which ends the phase in which contributors can acknowledge whether a particular feature will be part of the cycle. This release cycle, being unique, we had five weeks for the same milestone. The extended duration gave the contributors more time to plan and decide about the graduation of their respective features.
The milestone until which contributors implement the features was extended from the usual five weeks to 7 weeks. Contributors were provided with 40% more time to work on their features, resulting in reduced fatigue and more to think through about the implementation. We also noticed a considerable reduction in last-minute hustles. There were also a lesser number of exception requests this cycle - 6 compared to 14 the previous release cycle.
## User Highlights
* The CNCF grants Zalando, Europe’s leading online platform for fashion and lifestyle, the [Top End User Award](https://www.cncf.io/announcement/2020/08/20/cloud-native-computing-foundation-grants-zalando-the-top-end-user-award/). Zalando leverages numerous CNCF projects and open sourced multiple of their own development.
## Ecosystem Updates
* The CNCF just concluded its very first Virtual KubeCon. All talks are [on-demand]( https://events.linuxfoundation.org/kubecon-cloudnativecon-europe/) for anyone registered, it's not too late!
* The [Certified Kubernetes Security Specialist](https://www.cncf.io/blog/2020/07/15/certified-kubernetes-security-specialist-cks-coming-in-november/) (CKS) coming in November! CKS focuses on cluster & system hardening, minimizing microservice vulnerabilities and the security of the supply chain.
* CNCF published the second [State of Cloud Native Development](https://www.cncf.io/blog/2020/08/14/state-of-cloud-native-development/), showing the massively growing number of cloud native developer using container and serverless technology.
* [Kubernetes.dev](https://www.kubernetes.dev), a Kubernetes contributor focused website has been launched. It brings the contributor documentation, resources and project event information into one central location.
## Project Velocity
The [Kubernetes DevStats dashboard](https://k8s.devstats.cncf.io/d/12/dashboards?orgId=1) illustrates the breakdown of contributions from major company contributors, as well as an impressive set of preconfigured reports on everything from individual contributors to pull request lifecycle times. If you want to gather numbers, facts and figures from Kubernetes and the CNCF community it is the best place to start.
During this release cycle from April till August, 382 different companies and over 2,464 individuals contributed to Kubernetes. [Check out DevStats](https://k8s.devstats.cncf.io/d/11/companies-contributing-in-repository-groups?orgId=1&var-period=m&var-repogroup_name=All&from=1585692000000&to=1598392799000) to learn more about the overall velocity of the Kubernetes project and community.
## Upcoming release webinar
Join the members of the Kubernetes 1.19 release team on September 25th, 2020 to learn about the major features in this release including storage capacity tracking, structured logging, Ingress V1 GA, and many more. Register here: https://www.cncf.io/webinars/kubernetes-1-19/.
## Get Involved
The simplest way to get involved with Kubernetes is by joining one of the many [Special Interest Groups](https://github.com/kubernetes/community/blob/master/sig-list.md) (SIGs) that align with your interests. Have something you’d like to broadcast to the Kubernetes community? Share your voice at our monthly [community meeting](https://github.com/kubernetes/community/tree/master/communication), and through the channels below. Thank you for your continued feedback and support.
* Find out more about contributing to Kubernetes at the new [Kubernetes Contributor website](https://www.kubernetes.dev/)
* Follow us on Twitter [@Kubernetesio](https://twitter.com/kubernetesio) for latest updates
* Join the community discussion on [Discuss](https://discuss.kubernetes.io/)
* Join the community on [Slack](http://slack.k8s.io/)
* Share your Kubernetes [story](https://docs.google.com/a/linuxfoundation.org/forms/d/e/1FAIpQLScuI7Ye3VQHQTwBASrgkjQDSS5TP0g3AXfFhwSM9YpHgxRKFA/viewform)
* Read more about what’s happening with Kubernetes on the [blog](https://kubernetes.io/blog/)
* Learn more about the [Kubernetes Release Team](https://github.com/kubernetes/sig-release/tree/master/release-team)

1
content/en/docs/_index.md
View File

@ -1,3 +1,4 @@
---
linktitle: Kubernetes Documentation
title: Documentation
---

2
content/en/docs/concepts/architecture/cloud-controller.md
View File

@ -23,7 +23,7 @@ mechanism that allows different cloud providers to integrate their platforms wit
## Design
![Kubernetes components](/images/docs/components-of-kubernetes.png)
![Kubernetes components](/images/docs/components-of-kubernetes.svg)
The cloud controller manager runs in the control plane as a replicated set of processes
(usually, these are containers in Pods). Each cloud-controller-manager implements

2
content/en/docs/concepts/architecture/control-plane-node-communication.md
View File

@ -46,7 +46,7 @@ These connections terminate at the kubelet's HTTPS endpoint. By default, the api
To verify this connection, use the `--kubelet-certificate-authority` flag to provide the apiserver with a root certificate bundle to use to verify the kubelet's serving certificate.
If that is not possible, use [SSH tunneling](/docs/concepts/architecture/master-node-communication/#ssh-tunnels) between the apiserver and kubelet if required to avoid connecting over an
If that is not possible, use [SSH tunneling](#ssh-tunnels) between the apiserver and kubelet if required to avoid connecting over an
untrusted or public network.
Finally, [Kubelet authentication and/or authorization](/docs/admin/kubelet-authentication-authorization/) should be enabled to secure the kubelet API.

11
content/en/docs/concepts/architecture/controller.md
View File

@ -113,9 +113,9 @@ useful changes, it doesn't matter if the overall state is or is not stable.
As a tenet of its design, Kubernetes uses lots of controllers that each manage
a particular aspect of cluster state. Most commonly, a particular control loop
(controller) uses one kind of resource as its desired state, and has a different
kind of resource that it manages to make that desired state happen. For example,
kind of resource that it manages to make that desired state happen. For example,
a controller for Jobs tracks Job objects (to discover new work) and Pod objects
(to run the Jobs, and then to see when the work is finished). In this case
(to run the Jobs, and then to see when the work is finished). In this case
something else creates the Jobs, whereas the Job controller creates Pods.
It's useful to have simple controllers rather than one, monolithic set of control
@ -140,7 +140,7 @@ the {{< glossary_tooltip term_id="kube-controller-manager" >}}. These
built-in controllers provide important core behaviors.
The Deployment controller and Job controller are examples of controllers that
come as part of Kubernetes itself (“built-in” controllers).
come as part of Kubernetes itself ("built-in" controllers).
Kubernetes lets you run a resilient control plane, so that if any of the built-in
controllers were to fail, another part of the control plane will take over the work.
@ -154,8 +154,7 @@ controller does.
## {{% heading "whatsnext" %}}
* Read about the [Kubernetes control plane](/docs/concepts/#kubernetes-control-plane)
* Discover some of the basic [Kubernetes objects](/docs/concepts/#kubernetes-objects)
* Read about the [Kubernetes control plane](/docs/concepts/overview/components/#control-plane-components)
* Discover some of the basic [Kubernetes objects](/docs/concepts/overview/working-with-objects/kubernetes-objects/)
* Learn more about the [Kubernetes API](/docs/concepts/overview/kubernetes-api/)
* If you want to write your own controller, see [Extension Patterns](/docs/concepts/extend-kubernetes/extend-cluster/#extension-patterns) in Extending Kubernetes.

8
content/en/docs/concepts/architecture/nodes.md
View File

@ -23,8 +23,6 @@ The [components](/docs/concepts/overview/components/#node-components) on a node
{{< glossary_tooltip text="container runtime" term_id="container-runtime" >}}, and the
{{< glossary_tooltip text="kube-proxy" term_id="kube-proxy" >}}.
<!-- body -->
## Management
@ -100,7 +98,7 @@ You can modify Node objects regardless of the setting of `--register-node`.
For example, you can set labels on an existing Node, or mark it unschedulable.
You can use labels on Nodes in conjunction with node selectors on Pods to control
scheduling. For example, you can to constrain a Pod to only be eligible to run on
scheduling. For example, you can constrain a Pod to only be eligible to run on
a subset of the available nodes.
Marking a node as unschedulable prevents the scheduler from placing new pods onto
@ -195,7 +193,7 @@ The node lifecycle controller automatically creates
The scheduler takes the Node's taints into consideration when assigning a Pod to a Node.
Pods can also have tolerations which let them tolerate a Node's taints.
See [Taint Nodes by Condition](/docs/concepts/configuration/taint-and-toleration/#taint-nodes-by-condition)
See [Taint Nodes by Condition](/docs/concepts/scheduling-eviction/taint-and-toleration/#taint-nodes-by-condition)
for more details.
### Capacity and Allocatable {#capacity}
@ -339,6 +337,6 @@ for more information.
* Read the [API definition for Node](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#node-v1-core).
* Read the [Node](https://git.k8s.io/community/contributors/design-proposals/architecture/architecture.md#the-kubernetes-node)
section of the architecture design document.
* Read about [taints and tolerations](/docs/concepts/configuration/taint-and-toleration/).
* Read about [taints and tolerations](/docs/concepts/scheduling-eviction/taint-and-toleration/).
* Read about [cluster autoscaling](/docs/tasks/administer-cluster/cluster-management/#cluster-autoscaling).

10
content/en/docs/concepts/cluster-administration/_index.md
View File

@ -34,14 +34,14 @@ Before choosing a guide, here are some considerations:
- Do you **just want to run a cluster**, or do you expect to do **active development of Kubernetes project code**? If the
latter, choose an actively-developed distro. Some distros only use binary releases, but
offer a greater variety of choices.
- Familiarize yourself with the [components](/docs/admin/cluster-components/) needed to run a cluster.
- Familiarize yourself with the [components](/docs/concepts/overview/components/) needed to run a cluster.
## Managing a cluster
* [Managing a cluster](/docs/tasks/administer-cluster/cluster-management/) describes several topics related to the lifecycle of a cluster: creating a new cluster, upgrading your cluster’s master and worker nodes, performing node maintenance (e.g. kernel upgrades), and upgrading the Kubernetes API version of a running cluster.
* [Managing a cluster](/docs/tasks/administer-cluster/cluster-management/) describes several topics related to the lifecycle of a cluster: creating a new cluster, upgrading your cluster's master and worker nodes, performing node maintenance (e.g. kernel upgrades), and upgrading the Kubernetes API version of a running cluster.
* Learn how to [manage nodes](/docs/concepts/nodes/node/).
* Learn how to [manage nodes](/docs/concepts/architecture/nodes/).
* Learn how to set up and manage the [resource quota](/docs/concepts/policy/resource-quotas/) for shared clusters.
@ -59,14 +59,14 @@ Before choosing a guide, here are some considerations:
* [Using Admission Controllers](/docs/reference/access-authn-authz/admission-controllers/) explains plug-ins which intercepts requests to the Kubernetes API server after authentication and authorization.
* [Using Sysctls in a Kubernetes Cluster](/docs/concepts/cluster-administration/sysctl-cluster/) describes to an administrator how to use the `sysctl` command-line tool to set kernel parameters .
* [Using Sysctls in a Kubernetes Cluster](/docs/tasks/administer-cluster/sysctl-cluster/) describes to an administrator how to use the `sysctl` command-line tool to set kernel parameters .
* [Auditing](/docs/tasks/debug-application-cluster/audit/) describes how to interact with Kubernetes' audit logs.
### Securing the kubelet
* [Control Plane-Node communication](/docs/concepts/architecture/control-plane-node-communication/)
* [TLS bootstrapping](/docs/reference/command-line-tools-reference/kubelet-tls-bootstrapping/)
* [Kubelet authentication/authorization](/docs/admin/kubelet-authentication-authorization/)
* [Kubelet authentication/authorization](/docs/reference/command-line-tools-reference/kubelet-authentication-authorization/)
## Optional Cluster Services

15
content/en/docs/concepts/cluster-administration/addons.md
View File

@ -5,36 +5,31 @@ content_type: concept
<!-- overview -->
Add-ons extend the functionality of Kubernetes.
This page lists some of the available add-ons and links to their respective installation instructions.
Add-ons in each section are sorted alphabetically - the ordering does not imply any preferential status.
<!-- body -->
## Networking and Network Policy
* [ACI](https://www.github.com/noironetworks/aci-containers) provides integrated container networking and network security with Cisco ACI.
* [Calico](https://docs.projectcalico.org/latest/introduction/) is a networking and network policy provider. Calico supports a flexible set of networking options so you can choose the most efficient option for your situation, including non-overlay and overlay networks, with or without BGP. Calico uses the same engine to enforce network policy for hosts, pods, and (if using Istio & Envoy) applications at the service mesh layer.
* [Canal](https://github.com/tigera/canal/tree/master/k8s-install) unites Flannel and Calico, providing networking and network policy.
* [Cilium](https://github.com/cilium/cilium) is a L3 network and network policy plugin that can enforce HTTP/API/L7 policies transparently. Both routing and overlay/encapsulation mode are supported, and it can work on top of other CNI plugins.
* [CNI-Genie](https://github.com/Huawei-PaaS/CNI-Genie) enables Kubernetes to seamlessly connect to a choice of CNI plugins, such as Calico, Canal, Flannel, Romana, or Weave.
* [Contiv](http://contiv.github.io) provides configurable networking (native L3 using BGP, overlay using vxlan, classic L2, and Cisco-SDN/ACI) for various use cases and a rich policy framework. Contiv project is fully [open sourced](http://github.com/contiv). The [installer](http://github.com/contiv/install) provides both kubeadm and non-kubeadm based installation options.
* [Contrail](http://www.juniper.net/us/en/products-services/sdn/contrail/contrail-networking/), based on [Tungsten Fabric](https://tungsten.io), is an open source, multi-cloud network virtualization and policy management platform. Contrail and Tungsten Fabric are integrated with orchestration systems such as Kubernetes, OpenShift, OpenStack and Mesos, and provide isolation modes for virtual machines, containers/pods and bare metal workloads.
* [Contiv](https://contiv.github.io) provides configurable networking (native L3 using BGP, overlay using vxlan, classic L2, and Cisco-SDN/ACI) for various use cases and a rich policy framework. Contiv project is fully [open sourced](https://github.com/contiv). The [installer](https://github.com/contiv/install) provides both kubeadm and non-kubeadm based installation options.
* [Contrail](https://www.juniper.net/us/en/products-services/sdn/contrail/contrail-networking/), based on [Tungsten Fabric](https://tungsten.io), is an open source, multi-cloud network virtualization and policy management platform. Contrail and Tungsten Fabric are integrated with orchestration systems such as Kubernetes, OpenShift, OpenStack and Mesos, and provide isolation modes for virtual machines, containers/pods and bare metal workloads.
* [Flannel](https://github.com/coreos/flannel/blob/master/Documentation/kubernetes.md) is an overlay network provider that can be used with Kubernetes.
* [Knitter](https://github.com/ZTE/Knitter/) is a plugin to support multiple network interfaces in a Kubernetes pod.
* [Multus](https://github.com/Intel-Corp/multus-cni) is a Multi plugin for multiple network support in Kubernetes to support all CNI plugins (e.g. Calico, Cilium, Contiv, Flannel), in addition to SRIOV, DPDK, OVS-DPDK and VPP based workloads in Kubernetes.
* [OVN4NFV-K8S-Plugin](https://github.com/opnfv/ovn4nfv-k8s-plugin) is OVN based CNI controller plugin to provide cloud native based Service function chaining(SFC), Multiple OVN overlay networking, dynamic subnet creation, dynamic creation of virtual networks, VLAN Provider network, Direct provider network and pluggable with other Multi-network plugins, ideal for edge based cloud native workloads in Multi-cluster networking
* [NSX-T](https://docs.vmware.com/en/VMware-NSX-T/2.0/nsxt_20_ncp_kubernetes.pdf) Container Plug-in (NCP) provides integration between VMware NSX-T and container orchestrators such as Kubernetes, as well as integration between NSX-T and container-based CaaS/PaaS platforms such as Pivotal Container Service (PKS) and OpenShift.
* [Nuage](https://github.com/nuagenetworks/nuage-kubernetes/blob/v5.1.1-1/docs/kubernetes-1-installation.rst) is an SDN platform that provides policy-based networking between Kubernetes Pods and non-Kubernetes environments with visibility and security monitoring.
* [Romana](http://romana.io) is a Layer 3 networking solution for pod networks that also supports the [NetworkPolicy API](/docs/concepts/services-networking/network-policies/). Kubeadm add-on installation details available [here](https://github.com/romana/romana/tree/master/containerize).
* [Weave Net](https://www.weave.works/docs/net/latest/kube-addon/) provides networking and network policy, will carry on working on both sides of a network partition, and does not require an external database.
* [Romana](https://romana.io) is a Layer 3 networking solution for pod networks that also supports the [NetworkPolicy API](/docs/concepts/services-networking/network-policies/). Kubeadm add-on installation details available [here](https://github.com/romana/romana/tree/master/containerize).
* [Weave Net](https://www.weave.works/docs/net/latest/kubernetes/kube-addon/) provides networking and network policy, will carry on working on both sides of a network partition, and does not require an external database.
## Service Discovery
@ -54,5 +49,3 @@ Add-ons in each section are sorted alphabetically - the ordering does not imply
There are several other add-ons documented in the deprecated [cluster/addons](https://git.k8s.io/kubernetes/cluster/addons) directory.
Well-maintained ones should be linked to here. PRs welcome!

96
content/en/docs/concepts/cluster-administration/cloud-providers.md
View File

@ -6,9 +6,7 @@ weight: 30
<!-- overview -->
This page explains how to manage Kubernetes running on a specific
cloud provider.
cloud provider. There are many other third-party cloud provider projects, but this list is specific to projects embedded within, or relied upon by Kubernetes itself.
<!-- body -->
### kubeadm
@ -46,8 +44,10 @@ controllerManager:
```
The in-tree cloud providers typically need both `--cloud-provider` and `--cloud-config` specified in the command lines
for the [kube-apiserver](/docs/admin/kube-apiserver/), [kube-controller-manager](/docs/admin/kube-controller-manager/) and the
[kubelet](/docs/admin/kubelet/). The contents of the file specified in `--cloud-config` for each provider is documented below as well.
for the [kube-apiserver](/docs/reference/command-line-tools-reference/kube-apiserver/),
[kube-controller-manager](/docs/reference/command-line-tools-reference/kube-controller-manager/) and the
[kubelet](/docs/reference/command-line-tools-reference/kubelet/).
The contents of the file specified in `--cloud-config` for each provider is documented below as well.
For all external cloud providers, please follow the instructions on the individual repositories,
which are listed under their headings below, or one may view [the list of all repositories](https://github.com/kubernetes?q=cloud-provider-&type=&language=)
@ -94,7 +94,7 @@ Different settings can be applied to a load balancer service in AWS using _annot
* `service.beta.kubernetes.io/aws-load-balancer-access-log-s3-bucket-prefix`: Used to specify access log s3 bucket prefix.
* `service.beta.kubernetes.io/aws-load-balancer-additional-resource-tags`: Used on the service to specify a comma-separated list of key-value pairs which will be recorded as additional tags in the ELB. For example: `"Key1=Val1,Key2=Val2,KeyNoVal1=,KeyNoVal2"`.
* `service.beta.kubernetes.io/aws-load-balancer-backend-protocol`: Used on the service to specify the protocol spoken by the backend (pod) behind a listener. If `http` (default) or `https`, an HTTPS listener that terminates the connection and parses headers is created. If set to `ssl` or `tcp`, a "raw" SSL listener is used. If set to `http` and `aws-load-balancer-ssl-cert` is not used then a HTTP listener is used.
* `service.beta.kubernetes.io/aws-load-balancer-ssl-cert`: Used on the service to request a secure listener. Value is a valid certificate ARN. For more, see [ELB Listener Config](http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/elb-listener-config.html) CertARN is an IAM or CM certificate ARN, for example `arn:aws:acm:us-east-1:123456789012:certificate/12345678-1234-1234-1234-123456789012`.
* `service.beta.kubernetes.io/aws-load-balancer-ssl-cert`: Used on the service to request a secure listener. Value is a valid certificate ARN. For more, see [ELB Listener Config](https://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/elb-listener-config.html) CertARN is an IAM or CM certificate ARN, for example `arn:aws:acm:us-east-1:123456789012:certificate/12345678-1234-1234-1234-123456789012`.
* `service.beta.kubernetes.io/aws-load-balancer-connection-draining-enabled`: Used on the service to enable or disable connection draining.
* `service.beta.kubernetes.io/aws-load-balancer-connection-draining-timeout`: Used on the service to specify a connection draining timeout.
* `service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout`: Used on the service to specify the idle connection timeout.
@ -116,15 +116,6 @@ If you wish to use the external cloud provider, its repository is [kubernetes/cl
The Azure cloud provider uses the hostname of the node (as determined by the kubelet or overridden with `--hostname-override`) as the name of the Kubernetes Node object.
Note that the Kubernetes Node name must match the Azure VM name.
## CloudStack
If you wish to use the external cloud provider, its repository is [apache/cloudstack-kubernetes-provider](https://github.com/apache/cloudstack-kubernetes-provider)
### Node Name
The CloudStack cloud provider uses the hostname of the node (as determined by the kubelet or overridden with `--hostname-override`) as the name of the Kubernetes Node object.
Note that the Kubernetes Node name must match the CloudStack VM name.
## GCE
If you wish to use the external cloud provider, its repository is [kubernetes/cloud-provider-gcp](https://github.com/kubernetes/cloud-provider-gcp#readme)
@ -138,11 +129,6 @@ Note that the first segment of the Kubernetes Node name must match the GCE insta
If you wish to use the external cloud provider, its repository is [kubernetes-sigs/cloud-provider-huaweicloud](https://github.com/kubernetes-sigs/cloud-provider-huaweicloud).
### Node Name
The HUAWEI CLOUD provider needs the private IP address of the node as the name of the Kubernetes Node object.
Please make sure indicating `--hostname-override=<node private IP>` when starting kubelet on the node.
## OpenStack
This section describes all the possible configurations which can
be used when using OpenStack with Kubernetes.
@ -251,11 +237,9 @@ file:
values are `v1` or `v2`. Where no value is provided automatic detection will
select the highest supported version exposed by the underlying OpenStack
cloud.
* `use-octavia` (Optional): Used to determine whether to look for and use an
Octavia LBaaS V2 service catalog endpoint. Valid values are `true` or `false`.
Where `true` is specified and an Octaiva LBaaS V2 entry can not be found, the
provider will fall back and attempt to find a Neutron LBaaS V2 endpoint
instead. The default value is `false`.
* `use-octavia`(Optional): Whether or not to use Octavia for LoadBalancer type
of Service implementation instead of using Neutron-LBaaS. Default: true
Attention: Openstack CCM use Octavia as default load balancer implementation since v1.17.0
* `subnet-id` (Optional): Used to specify the id of the subnet you want to
create your loadbalancer on. Can be found at Network > Networks. Click on the
respective network to get its subnets.
@ -358,27 +342,12 @@ Kubernetes network plugin and should appear in the `[Route]` section of the
the `extraroutes` extension then use `router-id` to specify a router to add
routes to. The router chosen must span the private networks containing your
cluster nodes (typically there is only one node network, and this value should be
the default router for the node network). This value is required to use [kubenet]
the default router for the node network). This value is required to use
[kubenet](/docs/concepts/cluster-administration/network-plugins/#kubenet)
on OpenStack.
[kubenet]: /docs/concepts/cluster-administration/network-plugins/#kubenet
## OVirt
### Node Name
The OVirt cloud provider uses the hostname of the node (as determined by the kubelet or overridden with `--hostname-override`) as the name of the Kubernetes Node object.
Note that the Kubernetes Node name must match the VM FQDN (reported by OVirt under `<vm><guest_info><fqdn>...</fqdn></guest_info></vm>`)
## Photon
### Node Name
The Photon cloud provider uses the hostname of the node (as determined by the kubelet or overridden with `--hostname-override`) as the name of the Kubernetes Node object.
Note that the Kubernetes Node name must match the Photon VM name (or if `overrideIP` is set to true in the `--cloud-config`, the Kubernetes Node name must match the Photon VM IP address).
## vSphere
{{< tabs name="vSphere cloud provider" >}}
@ -391,46 +360,3 @@ If you are running vSphere < 6.7U3, the in-tree vSphere cloud provider is recomm
{{< /tabs >}}
For in-depth documentation on the vSphere cloud provider, visit the [vSphere cloud provider docs site](https://cloud-provider-vsphere.sigs.k8s.io).
## IBM Cloud Kubernetes Service
### Compute nodes
By using the IBM Cloud Kubernetes Service provider, you can create clusters with a mixture of virtual and physical (bare metal) nodes in a single zone or across multiple zones in a region. For more information, see [Planning your cluster and worker node setup](https://cloud.ibm.com/docs/containers?topic=containers-planning_worker_nodes).
The name of the Kubernetes Node object is the private IP address of the IBM Cloud Kubernetes Service worker node instance.
### Networking
The IBM Cloud Kubernetes Service provider provides VLANs for quality network performance and network isolation for nodes. You can set up custom firewalls and Calico network policies to add an extra layer of security for your cluster, or connect your cluster to your on-prem data center via VPN. For more information, see [Planning your cluster network setup](https://cloud.ibm.com/docs/containers?topic=containers-plan_clusters).
To expose apps to the public or within the cluster, you can leverage NodePort, LoadBalancer, or Ingress services. You can also customize the Ingress application load balancer with annotations. For more information, see [Choosing an app exposure service](https://cloud.ibm.com/docs/containers?topic=containers-cs_network_planning#cs_network_planning).
### Storage
The IBM Cloud Kubernetes Service provider leverages Kubernetes-native persistent volumes to enable users to mount file, block, and cloud object storage to their apps. You can also use database-as-a-service and third-party add-ons for persistent storage of your data. For more information, see [Planning highly available persistent storage](https://cloud.ibm.com/docs/containers?topic=containers-storage_planning#storage_planning).
## Baidu Cloud Container Engine
### Node Name
The Baidu cloud provider uses the private IP address of the node (as determined by the kubelet or overridden with `--hostname-override`) as the name of the Kubernetes Node object.
Note that the Kubernetes Node name must match the Baidu VM private IP.
## Tencent Kubernetes Engine
If you wish to use the external cloud provider, its repository is [TencentCloud/tencentcloud-cloud-controller-manager](https://github.com/TencentCloud/tencentcloud-cloud-controller-manager).
### Node Name
The Tencent cloud provider uses the hostname of the node (as determined by the kubelet or overridden with `--hostname-override`) as the name of the Kubernetes Node object.
Note that the Kubernetes Node name must match the Tencent VM private IP.
## Alibaba Cloud Kubernetes
If you wish to use the external cloud provider, its repository is [kubernetes/cloud-provider-alibaba-cloud](https://github.com/kubernetes/cloud-provider-alibaba-cloud).
### Node Name
Alibaba Cloud does not require the format of node name, but the kubelet needs to add `--provider-id=${REGION_ID}.${INSTANCE_ID}`. The parameter `${REGION_ID}` represents the region id of the Kubernetes and `${INSTANCE_ID}` denotes the Alibaba ECS (Elastic Compute Service) ID.
### Load Balancers
You can setup external load balancers to use specific features in Alibaba Cloud by configuring the [annotations](https://www.alibabacloud.com/help/en/doc-detail/86531.htm) .

137
content/en/docs/concepts/cluster-administration/flow-control.md
View File

@ -162,6 +162,31 @@ are built in and may not be overwritten:
that only matches the `catch-all` FlowSchema will be rejected with an HTTP 429
error.
## Health check concurrency exemption
The suggested configuration gives no special treatment to the health
check requests on kube-apiservers from their local kubelets --- which
tend to use the secured port but supply no credentials. With the
suggested config, these requests get assigned to the `global-default`
FlowSchema and the corresponding `global-default` priority level,
where other traffic can crowd them out.
If you add the following additional FlowSchema, this exempts those
requests from rate limiting.
{{< caution >}}
Making this change also allows any hostile party to then send
health-check requests that match this FlowSchema, at any volume they
like. If you have a web traffic filter or similar external security
mechanism to protect your cluster's API server from general internet
traffic, you can configure rules to block any health check requests
that originate from outside your cluster.
{{< /caution >}}
{{< codenew file="priority-and-fairness/health-for-strangers.yaml" >}}
## Resources
The flow control API involves two kinds of resources.
[PriorityLevelConfigurations](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#prioritylevelconfiguration-v1alpha1-flowcontrol-apiserver-k8s-io)
@ -311,10 +336,12 @@ exports additional metrics. Monitoring these can help you determine whether your
configuration is inappropriately throttling important traffic, or find
poorly-behaved workloads that may be harming system health.
* `apiserver_flowcontrol_rejected_requests_total` counts requests that
were rejected, grouped by the name of the assigned priority level,
the name of the assigned FlowSchema, and the reason for rejection.
The reason will be one of the following:
* `apiserver_flowcontrol_rejected_requests_total` is a counter vector
(cumulative since server start) of requests that were rejected,
broken down by the labels `flowSchema` (indicating the one that
matched the request), `priorityLevel` (indicating the one to which
the request was assigned), and `reason`. The `reason` label will be
have one of the following values:
* `queue-full`, indicating that too many requests were already
queued,
* `concurrency-limit`, indicating that the
@ -323,23 +350,72 @@ poorly-behaved workloads that may be harming system health.
* `time-out`, indicating that the request was still in the queue
when its queuing time limit expired.
* `apiserver_flowcontrol_dispatched_requests_total` counts requests
that began executing, grouped by the name of the assigned priority
level and the name of the assigned FlowSchema.
* `apiserver_flowcontrol_dispatched_requests_total` is a counter
vector (cumulative since server start) of requests that began
executing, broken down by the labels `flowSchema` (indicating the
one that matched the request) and `priorityLevel` (indicating the
one to which the request was assigned).
* `apiserver_flowcontrol_current_inqueue_requests` gives the
instantaneous total number of queued (not executing) requests,
grouped by priority level and FlowSchema.
* `apiserver_current_inqueue_requests` is a gauge vector of recent
high water marks of the number of queued requests, grouped by a
label named `request_kind` whose value is `mutating` or `readOnly`.
These high water marks describe the largest number seen in the one
second window most recently completed. These complement the older
`apiserver_current_inflight_requests` gauge vector that holds the
last window's high water mark of number of requests actively being
served.
* `apiserver_flowcontrol_current_executing_requests` gives the instantaneous
total number of executing requests, grouped by priority level and FlowSchema.
* `apiserver_flowcontrol_read_vs_write_request_count_samples` is a
histogram vector of observations of the then-current number of
requests, broken down by the labels `phase` (which takes on the
values `waiting` and `executing`) and `request_kind` (which takes on
the values `mutating` and `readOnly`). The observations are made
periodically at a high rate.
* `apiserver_flowcontrol_request_queue_length_after_enqueue` gives a
histogram of queue lengths for the queues, grouped by priority level
and FlowSchema, as sampled by the enqueued requests. Each request
that gets queued contributes one sample to its histogram, reporting
the length of the queue just after the request was added. Note that
this produces different statistics than an unbiased survey would.
* `apiserver_flowcontrol_read_vs_write_request_count_watermarks` is a
histogram vector of high or low water marks of the number of
requests broken down by the labels `phase` (which takes on the
values `waiting` and `executing`) and `request_kind` (which takes on
the values `mutating` and `readOnly`); the label `mark` takes on
values `high` and `low`. The water marks are accumulated over
windows bounded by the times when an observation was added to
`apiserver_flowcontrol_read_vs_write_request_count_samples`. These
water marks show the range of values that occurred between samples.
* `apiserver_flowcontrol_current_inqueue_requests` is a gauge vector
holding the instantaneous number of queued (not executing) requests,
broken down by the labels `priorityLevel` and `flowSchema`.
* `apiserver_flowcontrol_current_executing_requests` is a gauge vector
holding the instantaneous number of executing (not waiting in a
queue) requests, broken down by the labels `priorityLevel` and
`flowSchema`.
* `apiserver_flowcontrol_priority_level_request_count_samples` is a
histogram vector of observations of the then-current number of
requests broken down by the labels `phase` (which takes on the
values `waiting` and `executing`) and `priorityLevel`. Each
histogram gets observations taken periodically, up through the last
activity of the relevant sort. The observations are made at a high
rate.
* `apiserver_flowcontrol_priority_level_request_count_watermarks` is a
histogram vector of high or low water marks of the number of
requests broken down by the labels `phase` (which takes on the
values `waiting` and `executing`) and `priorityLevel`; the label
`mark` takes on values `high` and `low`. The water marks are
accumulated over windows bounded by the times when an observation
was added to
`apiserver_flowcontrol_priority_level_request_count_samples`. These
water marks show the range of values that occurred between samples.
* `apiserver_flowcontrol_request_queue_length_after_enqueue` is a
histogram vector of queue lengths for the queues, broken down by
the labels `priorityLevel` and `flowSchema`, as sampled by the
enqueued requests. Each request that gets queued contributes one
sample to its histogram, reporting the length of the queue just
after the request was added. Note that this produces different
statistics than an unbiased survey would.
{{< note >}}
An outlier value in a histogram here means it is likely that a single flow
(i.e., requests by one user or for one namespace, depending on
@ -349,14 +425,17 @@ poorly-behaved workloads that may be harming system health.
to increase that PriorityLevelConfiguration's concurrency shares.
{{< /note >}}
* `apiserver_flowcontrol_request_concurrency_limit` gives the computed
concurrency limit (based on the API server's total concurrency limit and PriorityLevelConfigurations'
concurrency shares) for each PriorityLevelConfiguration.
* `apiserver_flowcontrol_request_concurrency_limit` is a gauge vector
hoding the computed concurrency limit (based on the API server's
total concurrency limit and PriorityLevelConfigurations' concurrency
shares), broken down by the label `priorityLevel`.
* `apiserver_flowcontrol_request_wait_duration_seconds` gives a histogram of how
long requests spent queued, grouped by the FlowSchema that matched the
request, the PriorityLevel to which it was assigned, and whether or not the
request successfully executed.
* `apiserver_flowcontrol_request_wait_duration_seconds` is a histogram
vector of how long requests spent queued, broken down by the labels
`flowSchema` (indicating which one matched the request),
`priorityLevel` (indicating the one to which the request was
assigned), and `execute` (indicating whether the request started
executing).
{{< note >}}
Since each FlowSchema always assigns requests to a single
PriorityLevelConfiguration, you can add the histograms for all the
@ -364,9 +443,11 @@ poorly-behaved workloads that may be harming system health.
requests assigned to that priority level.
{{< /note >}}
* `apiserver_flowcontrol_request_execution_seconds` gives a histogram of how
long requests took to actually execute, grouped by the FlowSchema that matched the
request and the PriorityLevel to which it was assigned.
* `apiserver_flowcontrol_request_execution_seconds` is a histogram
vector of how long requests took to actually execute, broken down by
the labels `flowSchema` (indicating which one matched the request)
and `priorityLevel` (indicating the one to which the request was
assigned).
### Debug endpoints

10
content/en/docs/concepts/cluster-administration/logging.md
View File

@ -9,13 +9,10 @@ weight: 60
<!-- overview -->
Application and systems logs can help you understand what is happening inside your cluster. The logs are particularly useful for debugging problems and monitoring cluster activity. Most modern applications have some kind of logging mechanism; as such, most container engines are likewise designed to support some kind of logging. The easiest and most embraced logging method for containerized applications is to write to the standard output and standard error streams.
Application logs can help you understand what is happening inside your application. The logs are particularly useful for debugging problems and monitoring cluster activity. Most modern applications have some kind of logging mechanism; as such, most container engines are likewise designed to support some kind of logging. The easiest and most embraced logging method for containerized applications is to write to the standard output and standard error streams.
However, the native functionality provided by a container engine or runtime is usually not enough for a complete logging solution. For example, if a container crashes, a pod is evicted, or a node dies, you'll usually still want to access your application's logs. As such, logs should have a separate storage and lifecycle independent of nodes, pods, or containers. This concept is called _cluster-level-logging_. Cluster-level logging requires a separate backend to store, analyze, and query logs. Kubernetes provides no native storage solution for log data, but you can integrate many existing logging solutions into your Kubernetes cluster.
<!-- body -->
Cluster-level logging architectures are described in assumption that
@ -97,6 +94,7 @@ the rotation and there are two files, one 10MB in size and one empty,
`kubectl logs` will return an empty response.
{{< /note >}}
[cosConfigureHelper]: https://github.com/kubernetes/kubernetes/blob/{{< param "githubbranch" >}}/cluster/gce/gci/configure-helper.sh
### System component logs
There are two types of system components: those that run in a container and those
@ -135,7 +133,7 @@ Because the logging agent must run on every node, it's common to implement it as
Using a node-level logging agent is the most common and encouraged approach for a Kubernetes cluster, because it creates only one agent per node, and it doesn't require any changes to the applications running on the node. However, node-level logging _only works for applications' standard output and standard error_.
Kubernetes doesn't specify a logging agent, but two optional logging agents are packaged with the Kubernetes release: [Stackdriver Logging](/docs/user-guide/logging/stackdriver) for use with Google Cloud Platform, and [Elasticsearch](/docs/user-guide/logging/elasticsearch). You can find more information and instructions in the dedicated documents. Both use [fluentd](http://www.fluentd.org/) with custom configuration as an agent on the node.
Kubernetes doesn't specify a logging agent, but two optional logging agents are packaged with the Kubernetes release: [Stackdriver Logging](/docs/tasks/debug-application-cluster/logging-stackdriver/) for use with Google Cloud Platform, and [Elasticsearch](/docs/tasks/debug-application-cluster/logging-elasticsearch-kibana/). You can find more information and instructions in the dedicated documents. Both use [fluentd](https://www.fluentd.org/) with custom configuration as an agent on the node.
### Using a sidecar container with the logging agent
@ -242,7 +240,7 @@ a [ConfigMap](/docs/tasks/configure-pod-container/configure-pod-configmap/) to c
{{< note >}}
The configuration of fluentd is beyond the scope of this article. For
information about configuring fluentd, see the
[official fluentd documentation](http://docs.fluentd.org/).
[official fluentd documentation](https://docs.fluentd.org/).
{{< /note >}}
The second file describes a pod that has a sidecar container running fluentd.

6
content/en/docs/concepts/cluster-administration/manage-deployment.md
View File

@ -10,9 +10,6 @@ weight: 40
You've deployed your application and exposed it via a service. Now what? Kubernetes provides a number of tools to help you manage your application deployment, including scaling and updating. Among the features that we will discuss in more depth are [configuration files](/docs/concepts/configuration/overview/) and [labels](/docs/concepts/overview/working-with-objects/labels/).
<!-- body -->
## Organizing resource configurations
@ -356,7 +353,8 @@ Sometimes it's necessary to make narrow, non-disruptive updates to resources you
### kubectl apply
It is suggested to maintain a set of configuration files in source control (see [configuration as code](http://martinfowler.com/bliki/InfrastructureAsCode.html)),
It is suggested to maintain a set of configuration files in source control
(see [configuration as code](https://martinfowler.com/bliki/InfrastructureAsCode.html)),
so that they can be maintained and versioned along with the code for the resources they configure.
Then, you can use [`kubectl apply`](/docs/reference/generated/kubectl/kubectl-commands/#apply) to push your configuration changes to the cluster.

35
content/en/docs/concepts/cluster-administration/networking.md
View File

@ -17,9 +17,6 @@ problems to address:
3. Pod-to-Service communications: this is covered by [services](/docs/concepts/services-networking/service/).
4. External-to-Service communications: this is covered by [services](/docs/concepts/services-networking/service/).
<!-- body -->
Kubernetes is all about sharing machines between applications. Typically,
@ -93,7 +90,7 @@ Thanks to the "programmable" characteristic of Open vSwitch, Antrea is able to i
### AOS from Apstra
[AOS](http://www.apstra.com/products/aos/) is an Intent-Based Networking system that creates and manages complex datacenter environments from a simple integrated platform. AOS leverages a highly scalable distributed design to eliminate network outages while minimizing costs.
[AOS](https://www.apstra.com/products/aos/) is an Intent-Based Networking system that creates and manages complex datacenter environments from a simple integrated platform. AOS leverages a highly scalable distributed design to eliminate network outages while minimizing costs.
The AOS Reference Design currently supports Layer-3 connected hosts that eliminate legacy Layer-2 switching problems. These Layer-3 hosts can be Linux servers (Debian, Ubuntu, CentOS) that create BGP neighbor relationships directly with the top of rack switches (TORs). AOS automates the routing adjacencies and then provides fine grained control over the route health injections (RHI) that are common in a Kubernetes deployment.
@ -101,7 +98,7 @@ AOS has a rich set of REST API endpoints that enable Kubernetes to quickly chang
AOS supports the use of common vendor equipment from manufacturers including Cisco, Arista, Dell, Mellanox, HPE, and a large number of white-box systems and open network operating systems like Microsoft SONiC, Dell OPX, and Cumulus Linux.
Details on how the AOS system works can be accessed here: http://www.apstra.com/products/how-it-works/
Details on how the AOS system works can be accessed here: https://www.apstra.com/products/how-it-works/
### AWS VPC CNI for Kubernetes
@ -123,7 +120,7 @@ Azure CNI is available natively in the [Azure Kubernetes Service (AKS)] (https:/
With the help of the Big Cloud Fabric's virtual pod multi-tenant architecture, container orchestration systems such as Kubernetes, RedHat OpenShift, Mesosphere DC/OS & Docker Swarm will be natively integrated alongside with VM orchestration systems such as VMware, OpenStack & Nutanix. Customers will be able to securely inter-connect any number of these clusters and enable inter-tenant communication between them if needed.
BCF was recognized by Gartner as a visionary in the latest [Magic Quadrant](http://go.bigswitch.com/17GatedDocuments-MagicQuadrantforDataCenterNetworking_Reg.html). One of the BCF Kubernetes on-premises deployments (which includes Kubernetes, DC/OS & VMware running on multiple DCs across different geographic regions) is also referenced [here](https://portworx.com/architects-corner-kubernetes-satya-komala-nio/).
BCF was recognized by Gartner as a visionary in the latest [Magic Quadrant](https://go.bigswitch.com/17GatedDocuments-MagicQuadrantforDataCenterNetworking_Reg.html). One of the BCF Kubernetes on-premises deployments (which includes Kubernetes, DC/OS & VMware running on multiple DCs across different geographic regions) is also referenced [here](https://portworx.com/architects-corner-kubernetes-satya-komala-nio/).
### Cilium
@ -135,7 +132,7 @@ addressing, and it can be used in combination with other CNI plugins.
### CNI-Genie from Huawei
[CNI-Genie](https://github.com/Huawei-PaaS/CNI-Genie) is a CNI plugin that enables Kubernetes to [simultaneously have access to different implementations](https://github.com/Huawei-PaaS/CNI-Genie/blob/master/docs/multiple-cni-plugins/README.md#what-cni-genie-feature-1-multiple-cni-plugins-enables) of the [Kubernetes network model](https://github.com/kubernetes/website/blob/master/content/en/docs/concepts/cluster-administration/networking.md#the-kubernetes-network-model) in runtime. This includes any implementation that runs as a [CNI plugin](https://github.com/containernetworking/cni#3rd-party-plugins), such as [Flannel](https://github.com/coreos/flannel#flannel), [Calico](http://docs.projectcalico.org/), [Romana](http://romana.io), [Weave-net](https://www.weave.works/products/weave-net/).
[CNI-Genie](https://github.com/Huawei-PaaS/CNI-Genie) is a CNI plugin that enables Kubernetes to [simultaneously have access to different implementations](https://github.com/Huawei-PaaS/CNI-Genie/blob/master/docs/multiple-cni-plugins/README.md#what-cni-genie-feature-1-multiple-cni-plugins-enables) of the [Kubernetes network model](/docs/concepts/cluster-administration/networking/#the-kubernetes-network-model) in runtime. This includes any implementation that runs as a [CNI plugin](https://github.com/containernetworking/cni#3rd-party-plugins), such as [Flannel](https://github.com/coreos/flannel#flannel), [Calico](https://docs.projectcalico.org/), [Romana](https://romana.io), [Weave-net](https://www.weave.works/products/weave-net/).
CNI-Genie also supports [assigning multiple IP addresses to a pod](https://github.com/Huawei-PaaS/CNI-Genie/blob/master/docs/multiple-ips/README.md#feature-2-extension-cni-genie-multiple-ip-addresses-per-pod), each from a different CNI plugin.
@ -157,11 +154,11 @@ network complexity required to deploy Kubernetes at scale within AWS.
### Contiv
[Contiv](https://github.com/contiv/netplugin) provides configurable networking (native l3 using BGP, overlay using vxlan, classic l2, or Cisco-SDN/ACI) for various use cases. [Contiv](http://contiv.io) is all open sourced.
[Contiv](https://github.com/contiv/netplugin) provides configurable networking (native l3 using BGP, overlay using vxlan, classic l2, or Cisco-SDN/ACI) for various use cases. [Contiv](https://contiv.io) is all open sourced.
### Contrail / Tungsten Fabric
[Contrail](http://www.juniper.net/us/en/products-services/sdn/contrail/contrail-networking/), based on [Tungsten Fabric](https://tungsten.io), is a truly open, multi-cloud network virtualization and policy management platform. Contrail and Tungsten Fabric are integrated with various orchestration systems such as Kubernetes, OpenShift, OpenStack and Mesos, and provide different isolation modes for virtual machines, containers/pods and bare metal workloads.
[Contrail](https://www.juniper.net/us/en/products-services/sdn/contrail/contrail-networking/), based on [Tungsten Fabric](https://tungsten.io), is a truly open, multi-cloud network virtualization and policy management platform. Contrail and Tungsten Fabric are integrated with various orchestration systems such as Kubernetes, OpenShift, OpenStack and Mesos, and provide different isolation modes for virtual machines, containers/pods and bare metal workloads.
### DANM
@ -242,7 +239,7 @@ traffic to the internet.
### Kube-router
[Kube-router](https://github.com/cloudnativelabs/kube-router) is a purpose-built networking solution for Kubernetes that aims to provide high performance and operational simplicity. Kube-router provides a Linux [LVS/IPVS](http://www.linuxvirtualserver.org/software/ipvs.html)-based service proxy, a Linux kernel forwarding-based pod-to-pod networking solution with no overlays, and iptables/ipset-based network policy enforcer.
[Kube-router](https://github.com/cloudnativelabs/kube-router) is a purpose-built networking solution for Kubernetes that aims to provide high performance and operational simplicity. Kube-router provides a Linux [LVS/IPVS](https://www.linuxvirtualserver.org/software/ipvs.html)-based service proxy, a Linux kernel forwarding-based pod-to-pod networking solution with no overlays, and iptables/ipset-based network policy enforcer.
### L2 networks and linux bridging
@ -252,8 +249,8 @@ Note that these instructions have only been tried very casually - it seems to
work, but has not been thoroughly tested. If you use this technique and
perfect the process, please let us know.
Follow the "With Linux Bridge devices" section of [this very nice
tutorial](http://blog.oddbit.com/2014/08/11/four-ways-to-connect-a-docker/) from
Follow the "With Linux Bridge devices" section of
[this very nice tutorial](https://blog.oddbit.com/2014/08/11/four-ways-to-connect-a-docker/) from
Lars Kellogg-Stedman.
### Multus (a Multi Network plugin)
@ -274,7 +271,7 @@ Multus supports all [reference plugins](https://github.com/containernetworking/p
### Nuage Networks VCS (Virtualized Cloud Services)
[Nuage](http://www.nuagenetworks.net) provides a highly scalable policy-based Software-Defined Networking (SDN) platform. Nuage uses the open source Open vSwitch for the data plane along with a feature rich SDN Controller built on open standards.
[Nuage](https://www.nuagenetworks.net) provides a highly scalable policy-based Software-Defined Networking (SDN) platform. Nuage uses the open source Open vSwitch for the data plane along with a feature rich SDN Controller built on open standards.
The Nuage platform uses overlays to provide seamless policy-based networking between Kubernetes Pods and non-Kubernetes environments (VMs and bare metal servers). Nuage's policy abstraction model is designed with applications in mind and makes it easy to declare fine-grained policies for applications.The platform's real-time analytics engine enables visibility and security monitoring for Kubernetes applications.
@ -294,7 +291,7 @@ at [ovn-kubernetes](https://github.com/openvswitch/ovn-kubernetes).
### Project Calico
[Project Calico](http://docs.projectcalico.org/) is an open source container networking provider and network policy engine.
[Project Calico](https://docs.projectcalico.org/) is an open source container networking provider and network policy engine.
Calico provides a highly scalable networking and network policy solution for connecting Kubernetes pods based on the same IP networking principles as the internet, for both Linux (open source) and Windows (proprietary - available from [Tigera](https://www.tigera.io/essentials/)). Calico can be deployed without encapsulation or overlays to provide high-performance, high-scale data center networking. Calico also provides fine-grained, intent based network security policy for Kubernetes pods via its distributed firewall.
@ -302,7 +299,7 @@ Calico can also be run in policy enforcement mode in conjunction with other netw
### Romana
[Romana](http://romana.io) is an open source network and security automation solution that lets you deploy Kubernetes without an overlay network. Romana supports Kubernetes [Network Policy](/docs/concepts/services-networking/network-policies/) to provide isolation across network namespaces.
[Romana](https://romana.io) is an open source network and security automation solution that lets you deploy Kubernetes without an overlay network. Romana supports Kubernetes [Network Policy](/docs/concepts/services-networking/network-policies/) to provide isolation across network namespaces.
### Weave Net from Weaveworks
@ -312,13 +309,9 @@ Weave Net runs as a [CNI plug-in](https://www.weave.works/docs/net/latest/cni-pl
or stand-alone. In either version, it doesn't require any configuration or extra code
to run, and in both cases, the network provides one IP address per pod - as is standard for Kubernetes.
## {{% heading "whatsnext" %}}
The early design of the networking model and its rationale, and some future
plans are described in more detail in the [networking design
document](https://git.k8s.io/community/contributors/design-proposals/network/networking.md).
plans are described in more detail in the
[networking design document](https://git.k8s.io/community/contributors/design-proposals/network/networking.md).

119
content/en/docs/concepts/cluster-administration/system-logs.md Normal file
View File

@ -0,0 +1,119 @@
---
reviewers:
- dims
- 44past4
title: System Logs
content_type: concept
weight: 60
---
<!-- overview -->
System component logs record events happening in cluster, which can be very useful for debugging.
You can configure log verbosity to see more or less detail.
Logs can be as coarse-grained as showing errors within a component, or as fine-grained as showing step-by-step traces of events (like HTTP access logs, pod state changes, controller actions, or scheduler decisions).
<!-- body -->
## Klog
klog is the Kubernetes logging library. [klog](https://github.com/kubernetes/klog)
generates log messages for the Kubernetes system components.
For more information about klog configuration, see the [Command line tool reference](/docs/reference/command-line-tools-reference/).
An example of the klog native format:
```
I1025 00:15:15.525108 1 httplog.go:79] GET /api/v1/namespaces/kube-system/pods/metrics-server-v0.3.1-57c75779f-9p8wg: (1.512ms) 200 [pod_nanny/v0.0.0 (linux/amd64) kubernetes/$Format 10.56.1.19:51756]
```
### Structured Logging
{{< feature-state for_k8s_version="v1.19" state="alpha" >}}
{{<warning>}}
Migration to structured log messages is an ongoing process. Not all log messages are structured in this version. When parsing log files, you must also handle unstructured log messages.
Log formatting and value serialization are subject to change.
{{< /warning>}}
Structured logging is a effort to introduce a uniform structure in log messages allowing for easy extraction of information, making logs easier and cheaper to store and process.
New message format is backward compatible and enabled by default.
Format of structured logs:
```
<klog header> "<message>" <key1>="<value1>" <key2>="<value2>" ...
```
Example:
```
I1025 00:15:15.525108 1 controller_utils.go:116] "Pod status updated" pod="kube-system/kubedns" status="ready"
```
### JSON log format
{{< feature-state for_k8s_version="v1.19" state="alpha" >}}
{{<warning >}}
JSON output does not support many standard klog flags. For list of unsupported klog flags, see the [Command line tool reference](/docs/reference/command-line-tools-reference/).
Not all logs are guaranteed to be written in JSON format (for example, during process start). If you intend to parse logs, make sure you can handle log lines that are not JSON as well.
Field names and JSON serialization are subject to change.
{{< /warning >}}
The `--logging-format=json` flag changes the format of logs from klog native format to JSON format.
Example of JSON log format (pretty printed):
```json
{
"ts": 1580306777.04728,
"v": 4,
"msg": "Pod status updated",
"pod":{
"name": "nginx-1",
"namespace": "default"
},
"status": "ready"
}
```
Keys with special meaning:
* `ts` - timestamp as Unix time (required, float)
* `v` - verbosity (required, int, default 0)
* `err` - error string (optional, string)
* `msg` - message (required, string)
List of components currently supporting JSON format:
* {{< glossary_tooltip term_id="kube-controller-manager" text="kube-controller-manager" >}}
* {{< glossary_tooltip term_id="kube-apiserver" text="kube-apiserver" >}}
* {{< glossary_tooltip term_id="kube-scheduler" text="kube-scheduler" >}}
* {{< glossary_tooltip term_id="kubelet" text="kubelet" >}}
### Log verbosity level
The `-v` flag controls log verbosity. Increasing the value increases the number of logged events. Decreasing the value decreases the number of logged events.
Increasing verbosity settings logs increasingly less severe events. A verbosity setting of 0 logs only critical events.
### Log location
There are two types of system components: those that run in a container and those
that do not run in a container. For example:
* The Kubernetes scheduler and kube-proxy run in a container.
* The kubelet and container runtime, for example Docker, do not run in containers.
On machines with systemd, the kubelet and container runtime write to journald.
Otherwise, they write to `.log` files in the `/var/log` directory.
System components inside containers always write to `.log` files in the `/var/log` directory,
bypassing the default logging mechanism.
Similar to the container logs, you should rotate system component logs in the `/var/log` directory.
In Kubernetes clusters created by the `kube-up.sh` script, log rotation is configured by the `logrotate` tool.
The `logrotate` tool rotates logs daily, or once the log size is greater than 100MB.
## {{% heading "whatsnext" %}}
* Read about the [Kubernetes Logging Architecture](/docs/concepts/cluster-administration/logging/)
* Read about [Structured Logging](https://github.com/kubernetes/enhancements/tree/master/keps/sig-instrumentation/1602-structured-logging)
* Read about the [Conventions for logging severity](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-instrumentation/logging.md)

21
content/en/docs/concepts/cluster-administration/monitoring.md → content/en/docs/concepts/cluster-administration/system-metrics.md
View File

@ -1,22 +1,19 @@
---
title: Metrics For The Kubernetes Control Plane
title: Metrics For Kubernetes System Components
reviewers:
- brancz
- logicalhan
- RainbowMango
content_type: concept
weight: 60
aliases:
- controller-metrics.md
---
<!-- overview -->
System component metrics can give a better look into what is happening inside them. Metrics are particularly useful for building dashboards and alerts.
Metrics in Kubernetes control plane are emitted in [prometheus format](https://prometheus.io/docs/instrumenting/exposition_formats/) and are human readable.
Kubernetes components emit metrics in [Prometheus format](https://prometheus.io/docs/instrumenting/exposition_formats/).
This format is structured plain text, designed so that people and machines can both read it.
<!-- body -->
@ -39,7 +36,7 @@ Note that {{< glossary_tooltip term_id="kubelet" text="kubelet" >}} also exposes
If your cluster uses {{< glossary_tooltip term_id="rbac" text="RBAC" >}}, reading metrics requires authorization via a user, group or ServiceAccount with a ClusterRole that allows accessing `/metrics`.
For example:
```
```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
@ -101,6 +98,14 @@ Take metric `A` as an example, here assumed that `A` is deprecated in 1.n. Accor
If you're upgrading from release `1.12` to `1.13`, but still depend on a metric `A` deprecated in `1.12`, you should set hidden metrics via command line: `--show-hidden-metrics=1.12` and remember to remove this metric dependency before upgrading to `1.14`
## Disable accelerator metrics
The kubelet collects accelerator metrics through cAdvisor. To collect these metrics, for accelerators like NVIDIA GPUs, kubelet held an open handle on the driver. This meant that in order to perform infrastructure changes (for example, updating the driver), a cluster administrator needed to stop the kubelet agent.
The responsibility for collecting accelerator metrics now belongs to the vendor rather than the kubelet. Vendors must provide a container that collects metrics and exposes them to the metrics service (for example, Prometheus).
The [`DisableAcceleratorUsageMetrics` feature gate](/docs/references/command-line-tools-reference/feature-gate.md#feature-gates-for-alpha-or-beta-features:~:text= DisableAcceleratorUsageMetrics,-false) disables metrics collected by the kubelet, with a [timeline for enabling this feature by default](https://github.com/kubernetes/enhancements/tree/411e51027db842355bd489691af897afc1a41a5e/keps/sig-node/1867-disable-accelerator-usage-metrics#graduation-criteria).
## Component metrics
### kube-controller-manager metrics
@ -124,8 +129,6 @@ cloudprovider_gce_api_request_duration_seconds { request = "detach_disk"}
cloudprovider_gce_api_request_duration_seconds { request = "list_disk"}
```
## {{% heading "whatsnext" %}}
* Read about the [Prometheus text format](https://github.com/prometheus/docs/blob/master/content/docs/instrumenting/exposition_formats.md#text-based-format) for metrics

4
content/en/docs/concepts/configuration/configmap.md
View File

@ -213,9 +213,9 @@ when new keys are projected to the Pod can be as long as the kubelet sync period
propagation delay, where the cache propagation delay depends on the chosen cache type
(it equals to watch propagation delay, ttl of cache, or zero correspondingly).
{{< feature-state for_k8s_version="v1.18" state="alpha" >}}
{{< feature-state for_k8s_version="v1.19" state="beta" >}}
The Kubernetes alpha feature _Immutable Secrets and ConfigMaps_ provides an option to set
The Kubernetes beta feature _Immutable Secrets and ConfigMaps_ provides an option to set
individual Secrets and ConfigMaps as immutable. For clusters that extensively use ConfigMaps
(at least tens of thousands of unique ConfigMap to Pod mounts), preventing changes to their
data has the following advantages:

14
content/en/docs/concepts/configuration/manage-resources-containers.md
View File

@ -111,7 +111,7 @@ CPU is always requested as an absolute quantity, never as a relative quantity;
### Meaning of memory
Limits and requests for `memory` are measured in bytes. You can express memory as
a plain integer or as a fixed-point integer using one of these suffixes:
a plain integer or as a fixed-point number using one of these suffixes:
E, P, T, G, M, K. You can also use the power-of-two equivalents: Ei, Pi, Ti, Gi,
Mi, Ki. For example, the following represent roughly the same value:
@ -133,7 +133,7 @@ metadata:
spec:
containers:
- name: app
image: super.mycompany.com/app:v4
image: images.my-company.example/app:v4
env:
resources:
requests:
@ -143,7 +143,7 @@ spec:
memory: "128Mi"
cpu: "500m"
- name: log-aggregator
image: super.mycompany.com/log-aggregator:v6
image: images.my-company.example/log-aggregator:v6
resources:
requests:
memory: "64Mi"
@ -311,7 +311,7 @@ You can use _ephemeral-storage_ for managing local ephemeral storage. Each Conta
* `spec.containers[].resources.requests.ephemeral-storage`
Limits and requests for `ephemeral-storage` are measured in bytes. You can express storage as
a plain integer or as a fixed-point integer using one of these suffixes:
a plain integer or as a fixed-point number using one of these suffixes:
E, P, T, G, M, K. You can also use the power-of-two equivalents: Ei, Pi, Ti, Gi,
Mi, Ki. For example, the following represent roughly the same value:
@ -329,14 +329,14 @@ metadata:
spec:
containers:
- name: app
image: super.mycompany.com/app:v4
image: images.my-company.example/app:v4
resources:
requests:
ephemeral-storage: "2Gi"
limits:
ephemeral-storage: "4Gi"
- name: log-aggregator
image: super.mycompany.com/log-aggregator:v6
image: images.my-company.example/log-aggregator:v6
resources:
requests:
ephemeral-storage: "2Gi"
@ -752,4 +752,4 @@ You can see that the Container was terminated because of `reason:OOM Killed`, wh
* Read the [ResourceRequirements](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#resourcerequirements-v1-core) API reference
* Read about [project quotas](http://xfs.org/docs/xfsdocs-xml-dev/XFS_User_Guide/tmp/en-US/html/xfs-quotas.html) in XFS
* Read about [project quotas](https://xfs.org/docs/xfsdocs-xml-dev/XFS_User_Guide/tmp/en-US/html/xfs-quotas.html) in XFS

2
content/en/docs/concepts/configuration/overview.md
View File

@ -73,7 +73,7 @@ A desired state of an object is described by a Deployment, and if changes to tha
## Container Images
The [imagePullPolicy](/docs/concepts/containers/images/#updating-images) and the tag of the image affect when the [kubelet](/docs/admin/kubelet/) attempts to pull the specified image.
The [imagePullPolicy](/docs/concepts/containers/images/#updating-images) and the tag of the image affect when the [kubelet](/docs/reference/command-line-tools-reference/kubelet/) attempts to pull the specified image.
- `imagePullPolicy: IfNotPresent`: the image is pulled only if it is not already present locally.

44
content/en/docs/concepts/configuration/pod-priority-preemption.md
View File

@ -11,7 +11,7 @@ weight: 70
{{< feature-state for_k8s_version="v1.14" state="stable" >}}
[Pods](/docs/user-guide/pods) can have _priority_. Priority indicates the
[Pods](/docs/concepts/workloads/pods/pod/) can have _priority_. Priority indicates the
importance of a Pod relative to other Pods. If a Pod cannot be scheduled, the
scheduler tries to preempt (evict) lower priority Pods to make scheduling of the
pending Pod possible.
@ -51,40 +51,6 @@ Kubernetes already ships with two PriorityClasses:
These are common classes and are used to [ensure that critical components are always scheduled first](/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods/).
{{< /note >}}
## How to disable preemption
{{< caution >}}
Critical pods rely on scheduler preemption to be scheduled when a cluster
is under resource pressure. For this reason, it is not recommended to
disable preemption.
{{< /caution >}}
{{< note >}}
In Kubernetes 1.15 and later, if the feature `NonPreemptingPriority` is enabled,
PriorityClasses have the option to set `preemptionPolicy: Never`.
This will prevent pods of that PriorityClass from preempting other pods.
{{< /note >}}
Preemption is controlled by a kube-scheduler flag `disablePreemption`, which is
set to `false` by default.
If you want to disable preemption despite the above note, you can set
`disablePreemption` to `true`.
This option is available in component configs only and is not available in
old-style command line options. Below is a sample component config to disable
preemption:
```yaml
apiVersion: kubescheduler.config.k8s.io/v1alpha1
kind: KubeSchedulerConfiguration
algorithmSource:
provider: DefaultProvider
...
disablePreemption: true
```
## PriorityClass
A PriorityClass is a non-namespaced object that defines a mapping from a
@ -138,7 +104,7 @@ description: "This priority class should be used for XYZ service pods only."
## Non-preempting PriorityClass {#non-preempting-priority-class}
{{< feature-state for_k8s_version="v1.15" state="alpha" >}}
{{< feature-state for_k8s_version="v1.19" state="beta" >}}
Pods with `PreemptionPolicy: Never` will be placed in the scheduling queue
ahead of lower-priority pods,
@ -162,10 +128,6 @@ which will allow pods of that PriorityClass to preempt lower-priority pods
If `PreemptionPolicy` is set to `Never`,
pods in that PriorityClass will be non-preempting.
The use of the `PreemptionPolicy` field requires the `NonPreemptingPriority`
[feature gate](/docs/reference/command-line-tools-reference/feature-gates/)
to be enabled.
An example use case is for data science workloads.
A user may submit a job that they want to be prioritized above other workloads,
but do not wish to discard existing work by preempting running pods.
@ -397,7 +359,7 @@ The only component that considers both QoS and Pod priority is
[kubelet out-of-resource eviction](/docs/tasks/administer-cluster/out-of-resource/).
The kubelet ranks Pods for eviction first by whether or not their usage of the
starved resource exceeds requests, then by Priority, and then by the consumption
of the starved compute resource relative to the Pods’ scheduling requests.
of the starved compute resource relative to the Pods' scheduling requests.
See
[evicting end-user pods](/docs/tasks/administer-cluster/out-of-resource/#evicting-end-user-pods)
for more details.

2
content/en/docs/concepts/configuration/resource-bin-packing.md
View File

@ -172,7 +172,7 @@ Node Score:
intel.com/foo = resourceScoringFunction((2+2),8)
= (100 - ((8-4)*100/8)
= (100 - 25)
= (100 - 50)
= 50
= rawScoringFunction(50)
= 5

10
content/en/docs/concepts/configuration/secret.md
View File

@ -352,6 +352,8 @@ NAME TYPE DATA
db-user-pass-96mffmfh4k Opaque 2 51s
```
You can view a description of the secret:
```shell
kubectl describe secrets/db-user-pass-96mffmfh4k
```
@ -715,9 +717,9 @@ A container using a Secret as a
Secret updates.
{{< /note >}}
{{< feature-state for_k8s_version="v1.18" state="alpha" >}}
{{< feature-state for_k8s_version="v1.19" state="beta" >}}
The Kubernetes alpha feature _Immutable Secrets and ConfigMaps_ provides an option to set
The Kubernetes beta feature _Immutable Secrets and ConfigMaps_ provides an option to set
individual Secrets and ConfigMaps as immutable. For clusters that extensively use Secrets
(at least tens of thousands of unique Secret to Pod mounts), preventing changes to their
data has the following advantages:
@ -912,7 +914,7 @@ Create the Secret:
kubectl apply -f mysecret.yaml
```
Use `envFrom` to define all of the Secret’s data as container environment variables. The key from the Secret becomes the environment variable name in the Pod.
Use `envFrom` to define all of the Secret's data as container environment variables. The key from the Secret becomes the environment variable name in the Pod.
```yaml
apiVersion: v1
@ -1002,6 +1004,8 @@ The output is similar to:
secret "prod-db-secret" created
```
You can also create a secret for test environment credentials.
```shell
kubectl create secret generic test-db-secret --from-literal=username=testuser --from-literal=password=iluvtests
```

4
content/en/docs/concepts/containers/container-environment.md
View File

@ -28,7 +28,7 @@ The Kubernetes Container environment provides several important resources to Con
The *hostname* of a Container is the name of the Pod in which the Container is running.
It is available through the `hostname` command or the
[`gethostname`](http://man7.org/linux/man-pages/man2/gethostname.2.html)
[`gethostname`](https://man7.org/linux/man-pages/man2/gethostname.2.html)
function call in libc.
The Pod name and namespace are available as environment variables through the
@ -51,7 +51,7 @@ FOO_SERVICE_PORT=<the port the service is running on>
```
Services have dedicated IP addresses and are available to the Container via DNS,
if [DNS addon](http://releases.k8s.io/{{< param "githubbranch" >}}/cluster/addons/dns/) is enabled. 
if [DNS addon](https://releases.k8s.io/{{< param "githubbranch" >}}/cluster/addons/dns/) is enabled. 

2
content/en/docs/concepts/containers/container-lifecycle-hooks.md
View File

@ -30,7 +30,7 @@ There are two hooks that are exposed to Containers:
`PostStart`
This hook executes immediately after a container is created.
This hook is executed immediately after a container is created.
However, there is no guarantee that the hook will execute before the container ENTRYPOINT.
No parameters are passed to the handler.

6
content/en/docs/concepts/containers/images.md
View File

@ -19,8 +19,6 @@ before referring to it in a
This page provides an outline of the container image concept.
<!-- body -->
## Image names
@ -96,7 +94,7 @@ runtime to authenticate to a private container registry.
This approach is suitable if you can control node configuration.
{{< note >}}
Kubernetes as only supports the `auths` and `HttpHeaders` section in Docker configuration.
Default Kubernetes only supports the `auths` and `HttpHeaders` section in Docker configuration.
Docker credential helpers (`credHelpers` or `credsStore`) are not supported.
{{< /note >}}
@ -261,7 +259,7 @@ EOF
This needs to be done for each pod that is using a private registry.
However, setting of this field can be automated by setting the imagePullSecrets
in a [ServiceAccount](/docs/user-guide/service-accounts) resource.
in a [ServiceAccount](/docs/tasks/configure-pod-container/configure-service-account/) resource.
Check [Add ImagePullSecrets to a Service Account](/docs/tasks/configure-pod-container/configure-service-account/#add-imagepullsecrets-to-a-service-account) for detailed instructions.

53
content/en/docs/concepts/extend-kubernetes/_index.md
View File

@ -24,9 +24,6 @@ their work environment. Developers who are prospective {{< glossary_tooltip text
useful as an introduction to what extension points and patterns
exist, and their trade-offs and limitations.
<!-- body -->
## Overview
@ -37,14 +34,14 @@ Customization approaches can be broadly divided into *configuration*, which only
*Configuration files* and *flags* are documented in the Reference section of the online documentation, under each binary:
* [kubelet](/docs/admin/kubelet/)
* [kube-apiserver](/docs/admin/kube-apiserver/)
* [kube-controller-manager](/docs/admin/kube-controller-manager/)
* [kube-scheduler](/docs/admin/kube-scheduler/).
* [kubelet](/docs/reference/command-line-tools-reference/kubelet/)
* [kube-apiserver](/docs/reference/command-line-tools-reference/kube-apiserver/)
* [kube-controller-manager](/docs/reference/command-line-tools-reference/kube-controller-manager/)
* [kube-scheduler](/docs/reference/command-line-tools-reference/kube-scheduler/).
Flags and configuration files may not always be changeable in a hosted Kubernetes service or a distribution with managed installation. When they are changeable, they are usually only changeable by the cluster administrator. Also, they are subject to change in future Kubernetes versions, and setting them may require restarting processes. For those reasons, they should be used only when there are no other options.
*Built-in Policy APIs*, such as [ResourceQuota](/docs/concepts/policy/resource-quotas/), [PodSecurityPolicies](/docs/concepts/policy/pod-security-policy/), [NetworkPolicy](/docs/concepts/services-networking/network-policies/) and Role-based Access Control ([RBAC](/docs/reference/access-authn-authz/rbac/)), are built-in Kubernetes APIs. APIs are typically used with hosted Kubernetes services and with managed Kubernetes installations. They are declarative and use the same conventions as other Kubernetes resources like pods, so new cluster configuration can be repeatable and be managed the same way as applications. And, where they are stable, they enjoy a [defined support policy](/docs/reference/deprecation-policy/) like other Kubernetes APIs. For these reasons, they are preferred over *configuration files* and *flags* where suitable.
*Built-in Policy APIs*, such as [ResourceQuota](/docs/concepts/policy/resource-quotas/), [PodSecurityPolicies](/docs/concepts/policy/pod-security-policy/), [NetworkPolicy](/docs/concepts/services-networking/network-policies/) and Role-based Access Control ([RBAC](/docs/reference/access-authn-authz/rbac/)), are built-in Kubernetes APIs. APIs are typically used with hosted Kubernetes services and with managed Kubernetes installations. They are declarative and use the same conventions as other Kubernetes resources like pods, so new cluster configuration can be repeatable and be managed the same way as applications. And, where they are stable, they enjoy a [defined support policy](/docs/reference/using-api/deprecation-policy/) like other Kubernetes APIs. For these reasons, they are preferred over *configuration files* and *flags* where suitable.
## Extensions
@ -75,10 +72,9 @@ failure.
In the webhook model, Kubernetes makes a network request to a remote service.
In the *Binary Plugin* model, Kubernetes executes a binary (program).
Binary plugins are used by the kubelet (e.g. [Flex Volume
Plugins](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-storage/flexvolume.md)
and [Network
Plugins](/docs/concepts/cluster-administration/network-plugins/))
Binary plugins are used by the kubelet (e.g.
[Flex Volume Plugins](/docs/concepts/storage/volumes/#flexVolume)
and [Network Plugins](/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins/))
and by kubectl.
Below is a diagram showing how the extension points interact with the
@ -98,12 +94,12 @@ This diagram shows the extension points in a Kubernetes system.
<!-- image source diagrams: https://docs.google.com/drawings/d/1k2YdJgNTtNfW7_A8moIIkij-DmVgEhNrn3y2OODwqQQ/view -->
1. Users often interact with the Kubernetes API using `kubectl`. [Kubectl plugins](/docs/tasks/extend-kubectl/kubectl-plugins/) extend the kubectl binary. They only affect the individual user's local environment, and so cannot enforce site-wide policies.
2. The apiserver handles all requests. Several types of extension points in the apiserver allow authenticating requests, or blocking them based on their content, editing content, and handling deletion. These are described in the [API Access Extensions](/docs/concepts/overview/extending#api-access-extensions) section.
3. The apiserver serves various kinds of *resources*. *Built-in resource kinds*, like `pods`, are defined by the Kubernetes project and can't be changed. You can also add resources that you define, or that other projects have defined, called *Custom Resources*, as explained in the [Custom Resources](/docs/concepts/overview/extending#user-defined-types) section. Custom Resources are often used with API Access Extensions.
4. The Kubernetes scheduler decides which nodes to place pods on. There are several ways to extend scheduling. These are described in the [Scheduler Extensions](/docs/concepts/overview/extending#scheduler-extensions) section.
2. The apiserver handles all requests. Several types of extension points in the apiserver allow authenticating requests, or blocking them based on their content, editing content, and handling deletion. These are described in the [API Access Extensions](#api-access-extensions) section.
3. The apiserver serves various kinds of *resources*. *Built-in resource kinds*, like `pods`, are defined by the Kubernetes project and can't be changed. You can also add resources that you define, or that other projects have defined, called *Custom Resources*, as explained in the [Custom Resources](#user-defined-types) section. Custom Resources are often used with API Access Extensions.
4. The Kubernetes scheduler decides which nodes to place pods on. There are several ways to extend scheduling. These are described in the [Scheduler Extensions](#scheduler-extensions) section.
5. Much of the behavior of Kubernetes is implemented by programs called Controllers which are clients of the API-Server. Controllers are often used in conjunction with Custom Resources.
6. The kubelet runs on servers, and helps pods appear like virtual servers with their own IPs on the cluster network. [Network Plugins](/docs/concepts/overview/extending#network-plugins) allow for different implementations of pod networking.
7. The kubelet also mounts and unmounts volumes for containers. New types of storage can be supported via [Storage Plugins](/docs/concepts/overview/extending#storage-plugins).
6. The kubelet runs on servers, and helps pods appear like virtual servers with their own IPs on the cluster network. [Network Plugins](#network-plugins) allow for different implementations of pod networking.
7. The kubelet also mounts and unmounts volumes for containers. New types of storage can be supported via [Storage Plugins](#storage-plugins).
If you are unsure where to start, this flowchart can help. Note that some solutions may involve several types of extensions.
@ -119,7 +115,7 @@ Consider adding a Custom Resource to Kubernetes if you want to define new contro
Do not use a Custom Resource as data storage for application, user, or monitoring data.
For more about Custom Resources, see the [Custom Resources concept guide](/docs/concepts/api-extension/custom-resources/).
For more about Custom Resources, see the [Custom Resources concept guide](/docs/concepts/extend-kubernetes/api-extension/custom-resources/).
### Combining New APIs with Automation
@ -172,20 +168,21 @@ Kubelet call a Binary Plugin to mount the volume.
### Device Plugins
Device plugins allow a node to discover new Node resources (in addition to the
builtin ones like cpu and memory) via a [Device
Plugin](/docs/concepts/cluster-administration/device-plugins/).
builtin ones like cpu and memory) via a
[Device Plugin](/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins/).
### Network Plugins
Different networking fabrics can be supported via node-level [Network Plugins](/docs/admin/network-plugins/).
Different networking fabrics can be supported via node-level
[Network Plugins](/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins/).
### Scheduler Extensions
The scheduler is a special type of controller that watches pods, and assigns
pods to nodes. The default scheduler can be replaced entirely, while
continuing to use other Kubernetes components, or [multiple
schedulers](/docs/tasks/administer-cluster/configure-multiple-schedulers/)
continuing to use other Kubernetes components, or
[multiple schedulers](/docs/tasks/extend-kubernetes/configure-multiple-schedulers/)
can run at the same time.
This is a significant undertaking, and almost all Kubernetes users find they
@ -196,18 +193,14 @@ The scheduler also supports a
that permits a webhook backend (scheduler extension) to filter and prioritize
the nodes chosen for a pod.
## {{% heading "whatsnext" %}}
* Learn more about [Custom Resources](/docs/concepts/api-extension/custom-resources/)
* Learn more about [Custom Resources](/docs/concepts/extend-kubernetes/api-extension/custom-resources/)
* Learn about [Dynamic admission control](/docs/reference/access-authn-authz/extensible-admission-controllers/)
* Learn more about Infrastructure extensions
* [Network Plugins](/docs/concepts/cluster-administration/network-plugins/)
* [Device Plugins](/docs/concepts/cluster-administration/device-plugins/)
* [Network Plugins](/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins/)
* [Device Plugins](/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins/)
* Learn about [kubectl plugins](/docs/tasks/extend-kubectl/kubectl-plugins/)
* Learn about the [Operator pattern](/docs/concepts/extend-kubernetes/operator/)

5
content/en/docs/concepts/extend-kubernetes/api-extension/apiserver-aggregation.md
View File

@ -15,8 +15,6 @@ The additional APIs can either be ready-made solutions such as [service-catalog]
The aggregation layer is different from [Custom Resources](/docs/concepts/extend-kubernetes/api-extension/custom-resources/), which are a way to make the {{< glossary_tooltip term_id="kube-apiserver" text="kube-apiserver" >}} recognise new kinds of object.
<!-- body -->
## Aggregation layer
@ -34,11 +32,8 @@ If your extension API server cannot achieve that latency requirement, consider m
`EnableAggregatedDiscoveryTimeout=false` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/) on the kube-apiserver
to disable the timeout restriction. This deprecated feature gate will be removed in a future release.
## {{% heading "whatsnext" %}}
* To get the aggregator working in your environment, [configure the aggregation layer](/docs/tasks/extend-kubernetes/configure-aggregation-layer/).
* Then, [setup an extension api-server](/docs/tasks/extend-kubernetes/setup-extension-api-server/) to work with the aggregation layer.
* Also, learn how to [extend the Kubernetes API using Custom Resource Definitions](/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/).

8
content/en/docs/concepts/extend-kubernetes/api-extension/custom-resources.md
View File

@ -13,8 +13,6 @@ weight: 10
resource to your Kubernetes cluster and when to use a standalone service. It describes the two
methods for adding custom resources and how to choose between them.
<!-- body -->
## Custom resources
@ -28,7 +26,7 @@ many core Kubernetes functions are now built using custom resources, making Kube
Custom resources can appear and disappear in a running cluster through dynamic registration,
and cluster admins can update custom resources independently of the cluster itself.
Once a custom resource is installed, users can create and access its objects using
[kubectl](/docs/user-guide/kubectl-overview/), just as they do for built-in resources like
[kubectl](/docs/reference/kubectl/overview/), just as they do for built-in resources like
*Pods*.
## Custom controllers
@ -52,7 +50,9 @@ for specific applications into an extension of the Kubernetes API.
## Should I add a custom resource to my Kubernetes Cluster?
When creating a new API, consider whether to [aggregate your API with the Kubernetes cluster APIs](/docs/concepts/api-extension/apiserver-aggregation/) or let your API stand alone.
When creating a new API, consider whether to
[aggregate your API with the Kubernetes cluster APIs](/docs/concepts/extend-kubernetes/api-extension/apiserver-aggregation/)
or let your API stand alone.
| Consider API aggregation if: | Prefer a stand-alone API if: |
| ---------------------------- | ---------------------------- |

32
content/en/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins.md
View File

@ -19,8 +19,6 @@ The targeted devices include GPUs, high-performance NICs, FPGAs, InfiniBand adap
and other similar computing resources that may require vendor specific initialization
and setup.
<!-- body -->
## Device plugin registration
@ -39,7 +37,7 @@ During the registration, the device plugin needs to send:
* The name of its Unix socket.
* The Device Plugin API version against which it was built.
* The `ResourceName` it wants to advertise. Here `ResourceName` needs to follow the
[extended resource naming scheme](/docs/concepts/configuration/manage-compute-resources-container/#extended-resources)
[extended resource naming scheme](/docs/concepts/configuration/manage-resources-container/#extended-resources)
as `vendor-domain/resourcetype`.
(For example, an NVIDIA GPU is advertised as `nvidia.com/gpu`.)
@ -48,7 +46,7 @@ list of devices it manages, and the kubelet is then in charge of advertising tho
resources to the API server as part of the kubelet node status update.
For example, after a device plugin registers `hardware-vendor.example/foo` with the kubelet
and reports two healthy devices on a node, the node status is updated
to advertise that the node has 2 “Foo” devices installed and available.
to advertise that the node has 2 "Foo" devices installed and available.
Then, users can request devices in a
[Container](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#container-v1-core)
@ -94,6 +92,9 @@ The general workflow of a device plugin includes the following steps:
```gRPC
service DevicePlugin {
// GetDevicePluginOptions returns options to be communicated with Device Manager.
rpc GetDevicePluginOptions(Empty) returns (DevicePluginOptions) {}
// ListAndWatch returns a stream of List of Devices
// Whenever a Device state change or a Device disappears, ListAndWatch
// returns the new list
@ -103,9 +104,30 @@ The general workflow of a device plugin includes the following steps:
// Plugin can run device specific operations and instruct Kubelet
// of the steps to make the Device available in the container
rpc Allocate(AllocateRequest) returns (AllocateResponse) {}
// GetPreferredAllocation returns a preferred set of devices to allocate
// from a list of available ones. The resulting preferred allocation is not
// guaranteed to be the allocation ultimately performed by the
// devicemanager. It is only designed to help the devicemanager make a more
// informed allocation decision when possible.
rpc GetPreferredAllocation(PreferredAllocationRequest) returns (PreferredAllocationResponse) {}
// PreStartContainer is called, if indicated by Device Plugin during registeration phase,
// before each container start. Device plugin can run device specific operations
// such as resetting the device before making devices available to the container.
rpc PreStartContainer(PreStartContainerRequest) returns (PreStartContainerResponse) {}
}
```
{{< note >}}
Plugins are not required to provide useful implementations for
`GetPreferredAllocation()` or `PreStartContainer()`. Flags indicating which
(if any) of these calls are available should be set in the `DevicePluginOptions`
message sent back by a call to `GetDevicePluginOptions()`. The `kubelet` will
always call `GetDevicePluginOptions()` to see which optional functions are
available, before calling any of them directly.
{{< /note >}}
* The plugin registers itself with the kubelet through the Unix socket at host
path `/var/lib/kubelet/device-plugins/kubelet.sock`.
@ -186,7 +208,7 @@ Support for the "PodResources service" requires `KubeletPodResources` [feature g
## Device Plugin integration with the Topology Manager
{{< feature-state for_k8s_version="v1.17" state="alpha" >}}
{{< feature-state for_k8s_version="v1.18" state="beta" >}}
The Topology Manager is a Kubelet component that allows resources to be co-ordintated in a Topology aligned manner. In order to do this, the Device Plugin API was extended to include a `TopologyInfo` struct.

74
content/en/docs/concepts/extend-kubernetes/extend-cluster.md
View File

@ -15,14 +15,14 @@ Kubernetes is highly configurable and extensible. As a result,
there is rarely a need to fork or submit patches to the Kubernetes
project code.
This guide describes the options for customizing a Kubernetes
cluster. It is aimed at {{< glossary_tooltip text="cluster operators" term_id="cluster-operator" >}} who want to
understand how to adapt their Kubernetes cluster to the needs of
their work environment. Developers who are prospective {{< glossary_tooltip text="Platform Developers" term_id="platform-developer" >}} or Kubernetes Project {{< glossary_tooltip text="Contributors" term_id="contributor" >}} will also find it
useful as an introduction to what extension points and patterns
exist, and their trade-offs and limitations.
This guide describes the options for customizing a Kubernetes cluster. It is
aimed at {{< glossary_tooltip text="cluster operators" term_id="cluster-operator" >}}
who want to understand how to adapt their
Kubernetes cluster to the needs of their work environment. Developers who are prospective
{{< glossary_tooltip text="Platform Developers" term_id="platform-developer" >}}
or Kubernetes Project {{< glossary_tooltip text="Contributors" term_id="contributor" >}}
will also find it useful as an introduction to what extension points and
patterns exist, and their trade-offs and limitations.
<!-- body -->
@ -35,14 +35,14 @@ Customization approaches can be broadly divided into *configuration*, which only
*Configuration files* and *flags* are documented in the Reference section of the online documentation, under each binary:
* [kubelet](/docs/admin/kubelet/)
* [kube-apiserver](/docs/admin/kube-apiserver/)
* [kube-controller-manager](/docs/admin/kube-controller-manager/)
* [kube-scheduler](/docs/admin/kube-scheduler/).
* [kubelet](/docs/reference/command-line-tools-reference/kubelet/)
* [kube-apiserver](/docs/reference/command-line-tools-reference/kube-apiserver/)
* [kube-controller-manager](/docs/reference/command-line-tools-reference/kube-controller-manager/)
* [kube-scheduler](/docs/reference/command-line-tools-reference/kube-scheduler/).
Flags and configuration files may not always be changeable in a hosted Kubernetes service or a distribution with managed installation. When they are changeable, they are usually only changeable by the cluster administrator. Also, they are subject to change in future Kubernetes versions, and setting them may require restarting processes. For those reasons, they should be used only when there are no other options.
*Built-in Policy APIs*, such as [ResourceQuota](/docs/concepts/policy/resource-quotas/), [PodSecurityPolicies](/docs/concepts/policy/pod-security-policy/), [NetworkPolicy](/docs/concepts/services-networking/network-policies/) and Role-based Access Control ([RBAC](/docs/reference/access-authn-authz/rbac/)), are built-in Kubernetes APIs. APIs are typically used with hosted Kubernetes services and with managed Kubernetes installations. They are declarative and use the same conventions as other Kubernetes resources like pods, so new cluster configuration can be repeatable and be managed the same way as applications. And, where they are stable, they enjoy a [defined support policy](/docs/reference/deprecation-policy/) like other Kubernetes APIs. For these reasons, they are preferred over *configuration files* and *flags* where suitable.
*Built-in Policy APIs*, such as [ResourceQuota](/docs/concepts/policy/resource-quotas/), [PodSecurityPolicies](/docs/concepts/policy/pod-security-policy/), [NetworkPolicy](/docs/concepts/services-networking/network-policies/) and Role-based Access Control ([RBAC](/docs/reference/access-authn-authz/rbac/)), are built-in Kubernetes APIs. APIs are typically used with hosted Kubernetes services and with managed Kubernetes installations. They are declarative and use the same conventions as other Kubernetes resources like pods, so new cluster configuration can be repeatable and be managed the same way as applications. And, where they are stable, they enjoy a [defined support policy](/docs/reference/using-api/deprecation-policy/) like other Kubernetes APIs. For these reasons, they are preferred over *configuration files* and *flags* where suitable.
## Extensions
@ -73,10 +73,9 @@ failure.
In the webhook model, Kubernetes makes a network request to a remote service.
In the *Binary Plugin* model, Kubernetes executes a binary (program).
Binary plugins are used by the kubelet (e.g. [Flex Volume
Plugins](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-storage/flexvolume.md)
and [Network
Plugins](/docs/concepts/cluster-administration/network-plugins/))
Binary plugins are used by the kubelet (e.g.
[Flex Volume Plugins](/docs/concepts/storage/volumes/#flexVolume)
and [Network Plugins](/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins/))
and by kubectl.
Below is a diagram showing how the extension points interact with the
@ -95,13 +94,13 @@ This diagram shows the extension points in a Kubernetes system.
<!-- image source diagrams: https://docs.google.com/drawings/d/1k2YdJgNTtNfW7_A8moIIkij-DmVgEhNrn3y2OODwqQQ/view -->
1. Users often interact with the Kubernetes API using `kubectl`. [Kubectl plugins](/docs/tasks/extend-kubectl/kubectl-plugins/) extend the kubectl binary. They only affect the individual user's local environment, and so cannot enforce site-wide policies.
2. The apiserver handles all requests. Several types of extension points in the apiserver allow authenticating requests, or blocking them based on their content, editing content, and handling deletion. These are described in the [API Access Extensions](/docs/concepts/overview/extending#api-access-extensions) section.
3. The apiserver serves various kinds of *resources*. *Built-in resource kinds*, like `pods`, are defined by the Kubernetes project and can't be changed. You can also add resources that you define, or that other projects have defined, called *Custom Resources*, as explained in the [Custom Resources](/docs/concepts/overview/extending#user-defined-types) section. Custom Resources are often used with API Access Extensions.
4. The Kubernetes scheduler decides which nodes to place pods on. There are several ways to extend scheduling. These are described in the [Scheduler Extensions](/docs/concepts/overview/extending#scheduler-extensions) section.
5. Much of the behavior of Kubernetes is implemented by programs called Controllers which are clients of the API-Server. Controllers are often used in conjunction with Custom Resources.
6. The kubelet runs on servers, and helps pods appear like virtual servers with their own IPs on the cluster network. [Network Plugins](/docs/concepts/overview/extending#network-plugins) allow for different implementations of pod networking.
7. The kubelet also mounts and unmounts volumes for containers. New types of storage can be supported via [Storage Plugins](/docs/concepts/overview/extending#storage-plugins).
1. Users often interact with the Kubernetes API using `kubectl`. [Kubectl plugins](/docs/tasks/extend-kubectl/kubectl-plugins/) extend the kubectl binary. They only affect the individual user's local environment, and so cannot enforce site-wide policies.
2. The apiserver handles all requests. Several types of extension points in the apiserver allow authenticating requests, or blocking them based on their content, editing content, and handling deletion. These are described in the [API Access Extensions](/docs/concepts/extend-kubernetes/#api-access-extensions) section.
3. The apiserver serves various kinds of *resources*. *Built-in resource kinds*, like `pods`, are defined by the Kubernetes project and can't be changed. You can also add resources that you define, or that other projects have defined, called *Custom Resources*, as explained in the [Custom Resources](/docs/concepts/extend-kubernetes/#user-defined-types) section. Custom Resources are often used with API Access Extensions.
4. The Kubernetes scheduler decides which nodes to place pods on. There are several ways to extend scheduling. These are described in the [Scheduler Extensions](/docs/concepts/extend-kubernetes/#scheduler-extensions) section.
5. Much of the behavior of Kubernetes is implemented by programs called Controllers which are clients of the API-Server. Controllers are often used in conjunction with Custom Resources.
6. The kubelet runs on servers, and helps pods appear like virtual servers with their own IPs on the cluster network. [Network Plugins](/docs/concepts/extend-kubernetes/#network-plugins) allow for different implementations of pod networking.
7. The kubelet also mounts and unmounts volumes for containers. New types of storage can be supported via [Storage Plugins](/docs/concepts/extend-kubernetes/#storage-plugins).
If you are unsure where to start, this flowchart can help. Note that some solutions may involve several types of extensions.
@ -117,7 +116,7 @@ Consider adding a Custom Resource to Kubernetes if you want to define new contro
Do not use a Custom Resource as data storage for application, user, or monitoring data.
For more about Custom Resources, see the [Custom Resources concept guide](/docs/concepts/api-extension/custom-resources/).
For more about Custom Resources, see the [Custom Resources concept guide](/docs/concepts/extend-kubernetes/api-extension/custom-resources/).
### Combining New APIs with Automation
@ -162,28 +161,28 @@ After a request is authorized, if it is a write operation, it also goes through
### Storage Plugins
[Flex Volumes](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/storage/flexvolume-deployment.md
) allow users to mount volume types without built-in support by having the
[Flex Volumes](/docs/concepts/storage/volumes/#flexVolume)
allow users to mount volume types without built-in support by having the
Kubelet call a Binary Plugin to mount the volume.
### Device Plugins
Device plugins allow a node to discover new Node resources (in addition to the
builtin ones like cpu and memory) via a [Device
Plugin](/docs/concepts/cluster-administration/device-plugins/).
builtin ones like cpu and memory) via a
[Device Plugin](/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins/).
### Network Plugins
Different networking fabrics can be supported via node-level [Network Plugins](/docs/admin/network-plugins/).
Different networking fabrics can be supported via node-level
[Network Plugins](/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins/).
### Scheduler Extensions
The scheduler is a special type of controller that watches pods, and assigns
pods to nodes. The default scheduler can be replaced entirely, while
continuing to use other Kubernetes components, or [multiple
schedulers](/docs/tasks/administer-cluster/configure-multiple-schedulers/)
continuing to use other Kubernetes components, or
[multiple schedulers](/docs/tasks/extend-kubernetes/configure-multiple-schedulers/)
can run at the same time.
This is a significant undertaking, and almost all Kubernetes users find they
@ -195,16 +194,13 @@ that permits a webhook backend (scheduler extension) to filter and prioritize
the nodes chosen for a pod.
## {{% heading "whatsnext" %}}
* Learn more about [Custom Resources](/docs/concepts/api-extension/custom-resources/)
* Learn more about [Custom Resources](/docs/concepts/extend-kubernetes/api-extension/custom-resources/)
* Learn about [Dynamic admission control](/docs/reference/access-authn-authz/extensible-admission-controllers/)
* Learn more about Infrastructure extensions
* [Network Plugins](/docs/concepts/cluster-administration/network-plugins/)
* [Device Plugins](/docs/concepts/cluster-administration/device-plugins/)
* [Network Plugins](/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins/)
* [Device Plugins](/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins/)
* Learn about [kubectl plugins](/docs/tasks/extend-kubectl/kubectl-plugins/)
* Learn about the [Operator pattern](/docs/concepts/extend-kubernetes/operator/)

9
content/en/docs/concepts/extend-kubernetes/operator.md
View File

@ -6,13 +6,10 @@ weight: 30
<!-- overview -->
Operators are software extensions to Kubernetes that make use of [custom
resources](/docs/concepts/extend-kubernetes/api-extension/custom-resources/)
Operators are software extensions to Kubernetes that make use of
[custom resources](/docs/concepts/extend-kubernetes/api-extension/custom-resources/)
to manage applications and their components. Operators follow
Kubernetes principles, notably the [control loop](/docs/concepts/#kubernetes-control-plane).
Kubernetes principles, notably the [control loop](/docs/concepts/architecture/controller).
<!-- body -->

2
content/en/docs/concepts/overview/components.md
View File

@ -21,7 +21,7 @@ a complete and working Kubernetes cluster.
Here's the diagram of a Kubernetes cluster with all the components tied together.
![Components of Kubernetes](/images/docs/components-of-kubernetes.png)
![Components of Kubernetes](/images/docs/components-of-kubernetes.svg)

3
content/en/docs/concepts/overview/kubernetes-api.md
View File

@ -24,7 +24,6 @@ The Kubernetes API lets you query and manipulate the state of objects in the Kub
API endpoints, resource types and samples are described in the [API Reference](/docs/reference/kubernetes-api/).
<!-- body -->
## API changes
@ -135,7 +134,7 @@ There are several API groups in a cluster:
(e.g. `apiVersion: batch/v1`). The Kubernetes [API reference](/docs/reference/kubernetes-api/) has a
full list of available API groups.
There are two paths to extending the API with [custom resources](/docs/concepts/api-extension/custom-resources/):
There are two paths to extending the API with [custom resources](/docs/concepts/extend-kubernetes/api-extension/custom-resources/):
1. [CustomResourceDefinition](/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/)
lets you declaratively define how the API server should provide your chosen resource API.

16
content/en/docs/concepts/overview/what-is-kubernetes.md
View File

@ -59,17 +59,17 @@ That's how Kubernetes comes to the rescue! Kubernetes provides you with a framew
Kubernetes provides you with:
* **Service discovery and load balancing**
* **Service discovery and load balancing**
Kubernetes can expose a container using the DNS name or using their own IP address. If traffic to a container is high, Kubernetes is able to load balance and distribute the network traffic so that the deployment is stable.
* **Storage orchestration**
* **Storage orchestration**
Kubernetes allows you to automatically mount a storage system of your choice, such as local storages, public cloud providers, and more.
* **Automated rollouts and rollbacks**
* **Automated rollouts and rollbacks**
You can describe the desired state for your deployed containers using Kubernetes, and it can change the actual state to the desired state at a controlled rate. For example, you can automate Kubernetes to create new containers for your deployment, remove existing containers and adopt all their resources to the new container.
* **Automatic bin packing**
* **Automatic bin packing**
You provide Kubernetes with a cluster of nodes that it can use to run containerized tasks. You tell Kubernetes how much CPU and memory (RAM) each container needs. Kubernetes can fit containers onto your nodes to make the best use of your resources.
* **Self-healing**
Kubernetes restarts containers that fail, replaces containers, kills containers that don’t respond to your user-defined health check, and doesn’t advertise them to clients until they are ready to serve.
* **Secret and configuration management**
* **Self-healing**
Kubernetes restarts containers that fail, replaces containers, kills containers that don't respond to your user-defined health check, and doesn't advertise them to clients until they are ready to serve.
* **Secret and configuration management**
Kubernetes lets you store and manage sensitive information, such as passwords, OAuth tokens, and SSH keys. You can deploy and update secrets and application configuration without rebuilding your container images, and without exposing secrets in your stack configuration.
## What Kubernetes is not
@ -84,7 +84,7 @@ Kubernetes:
* Does not dictate logging, monitoring, or alerting solutions. It provides some integrations as proof of concept, and mechanisms to collect and export metrics.
* Does not provide nor mandate a configuration language/system (for example, Jsonnet). It provides a declarative API that may be targeted by arbitrary forms of declarative specifications.
* Does not provide nor adopt any comprehensive machine configuration, maintenance, management, or self-healing systems.
* Additionally, Kubernetes is not a mere orchestration system. In fact, it eliminates the need for orchestration. The technical definition of orchestration is execution of a defined workflow: first do A, then B, then C. In contrast, Kubernetes comprises a set of independent, composable control processes that continuously drive the current state towards the provided desired state. It shouldn’t matter how you get from A to C. Centralized control is also not required. This results in a system that is easier to use and more powerful, robust, resilient, and extensible.
* Additionally, Kubernetes is not a mere orchestration system. In fact, it eliminates the need for orchestration. The technical definition of orchestration is execution of a defined workflow: first do A, then B, then C. In contrast, Kubernetes comprises a set of independent, composable control processes that continuously drive the current state towards the provided desired state. It shouldn't matter how you get from A to C. Centralized control is also not required. This results in a system that is easier to use and more powerful, robust, resilient, and extensible.

4
content/en/docs/concepts/overview/working-with-objects/annotations.md
View File

@ -69,7 +69,7 @@ If the prefix is omitted, the annotation Key is presumed to be private to the us
The `kubernetes.io/` and `k8s.io/` prefixes are reserved for Kubernetes core components.
For example, here’s the configuration file for a Pod that has the annotation `imageregistry: https://hub.docker.com/` :
For example, here's the configuration file for a Pod that has the annotation `imageregistry: https://hub.docker.com/` :
```yaml
@ -85,7 +85,7 @@ spec:
image: nginx:1.14.2
ports:
- containerPort: 80
```

23
content/en/docs/concepts/overview/working-with-objects/labels.md
View File

@ -22,10 +22,9 @@ Each object can have a set of key/value labels defined. Each Key must be unique
}
```
Labels allow for efficient queries and watches and are ideal for use in UIs and CLIs. Non-identifying information should be recorded using [annotations](/docs/concepts/overview/working-with-objects/annotations/).
Labels allow for efficient queries and watches and are ideal for use in UIs
and CLIs. Non-identifying information should be recorded using
[annotations](/docs/concepts/overview/working-with-objects/annotations/).
<!-- body -->
@ -55,7 +54,7 @@ The `kubernetes.io/` and `k8s.io/` prefixes are reserved for Kubernetes core com
Valid label values must be 63 characters or less and must be empty or begin and end with an alphanumeric character (`[a-z0-9A-Z]`) with dashes (`-`), underscores (`_`), dots (`.`), and alphanumerics between.
For example, here’s the configuration file for a Pod that has two labels `environment: production` and `app: nginx` :
For example, here's the configuration file for a Pod that has two labels `environment: production` and `app: nginx` :
```yaml
@ -77,7 +76,7 @@ spec:
## Label selectors
Unlike [names and UIDs](/docs/user-guide/identifiers), labels do not provide uniqueness. In general, we expect many objects to carry the same label(s).
Unlike [names and UIDs](/docs/concepts/overview/working-with-objects/names/), labels do not provide uniqueness. In general, we expect many objects to carry the same label(s).
Via a _label selector_, the client/user can identify a set of objects. The label selector is the core grouping primitive in Kubernetes.
@ -186,7 +185,10 @@ kubectl get pods -l 'environment,environment notin (frontend)'
### Set references in API objects
Some Kubernetes objects, such as [`services`](/docs/user-guide/services) and [`replicationcontrollers`](/docs/user-guide/replication-controller), also use label selectors to specify sets of other resources, such as [pods](/docs/user-guide/pods).
Some Kubernetes objects, such as [`services`](/docs/concepts/services-networking/service/)
and [`replicationcontrollers`](/docs/concepts/workloads/controllers/replicationcontroller/),
also use label selectors to specify sets of other resources, such as
[pods](/docs/concepts/workloads/pods/pod/).
#### Service and ReplicationController
@ -210,7 +212,11 @@ this selector (respectively in `json` or `yaml` format) is equivalent to `compon
#### Resources that support set-based requirements
Newer resources, such as [`Job`](/docs/concepts/workloads/controllers/jobs-run-to-completion/), [`Deployment`](/docs/concepts/workloads/controllers/deployment/), [`ReplicaSet`](/docs/concepts/workloads/controllers/replicaset/), and [`DaemonSet`](/docs/concepts/workloads/controllers/daemonset/), support _set-based_ requirements as well.
Newer resources, such as [`Job`](/docs/concepts/workloads/controllers/job/),
[`Deployment`](/docs/concepts/workloads/controllers/deployment/),
[`ReplicaSet`](/docs/concepts/workloads/controllers/replicaset/), and
[`DaemonSet`](/docs/concepts/workloads/controllers/daemonset/),
support _set-based_ requirements as well.
```yaml
selector:
@ -228,4 +234,3 @@ selector:
One use case for selecting over labels is to constrain the set of nodes onto which a pod can schedule.
See the documentation on [node selection](/docs/concepts/scheduling-eviction/assign-pod-node/) for more information.

2
content/en/docs/concepts/overview/working-with-objects/names.md
View File

@ -54,7 +54,7 @@ Some resource types require their names to be able to be safely encoded as a
path segment. In other words, the name may not be "." or ".." and the name may
not contain "/" or "%".
Here’s an example manifest for a Pod named `nginx-demo`.
Here's an example manifest for a Pod named `nginx-demo`.
```yaml
apiVersion: v1

20
content/en/docs/concepts/overview/working-with-objects/namespaces.md
View File

@ -13,9 +13,6 @@ weight: 30
Kubernetes supports multiple virtual clusters backed by the same physical cluster.
These virtual clusters are called namespaces.
<!-- body -->
## When to Use Multiple Namespaces
@ -35,13 +32,14 @@ In future versions of Kubernetes, objects in the same namespace will have the sa
access control policies by default.
It is not necessary to use multiple namespaces just to separate slightly different
resources, such as different versions of the same software: use [labels](/docs/user-guide/labels) to distinguish
resources, such as different versions of the same software: use
[labels](/docs/concepts/overview/working-with-objects/labels) to distinguish
resources within the same namespace.
## Working with Namespaces
Creation and deletion of namespaces are described in the [Admin Guide documentation
for namespaces](/docs/admin/namespaces).
Creation and deletion of namespaces are described in the
[Admin Guide documentation for namespaces](/docs/tasks/administer-cluster/namespaces).
{{< note >}}
Avoid creating namespace with prefix `kube-`, since it is reserved for Kubernetes system namespaces.
@ -93,7 +91,8 @@ kubectl config view --minify | grep namespace:
## Namespaces and DNS
When you create a [Service](/docs/user-guide/services), it creates a corresponding [DNS entry](/docs/concepts/services-networking/dns-pod-service/).
When you create a [Service](/docs/concepts/services-networking/service/),
it creates a corresponding [DNS entry](/docs/concepts/services-networking/dns-pod-service/).
This entry is of the form `<service-name>.<namespace-name>.svc.cluster.local`, which means
that if a container just uses `<service-name>`, it will resolve to the service which
is local to a namespace. This is useful for using the same configuration across
@ -104,7 +103,8 @@ across namespaces, you need to use the fully qualified domain name (FQDN).
Most Kubernetes resources (e.g. pods, services, replication controllers, and others) are
in some namespaces. However namespace resources are not themselves in a namespace.
And low-level resources, such as [nodes](/docs/admin/node) and
And low-level resources, such as
[nodes](/docs/concepts/architecture/nodes/) and
persistentVolumes, are not in any namespace.
To see which Kubernetes resources are and aren't in a namespace:
@ -117,12 +117,8 @@ kubectl api-resources --namespaced=true
kubectl api-resources --namespaced=false
```
## {{% heading "whatsnext" %}}
* Learn more about [creating a new namespace](/docs/tasks/administer-cluster/namespaces/#creating-a-new-namespace).
* Learn more about [deleting a namespace](/docs/tasks/administer-cluster/namespaces/#deleting-a-namespace).

5
content/en/docs/concepts/overview/working-with-objects/object-management.md
View File

@ -10,7 +10,6 @@ Kubernetes objects. This document provides an overview of the different
approaches. Read the [Kubectl book](https://kubectl.docs.kubernetes.io) for
details of managing objects by Kubectl.
<!-- body -->
## Management techniques
@ -167,11 +166,8 @@ Disadvantages compared to imperative object configuration:
- Declarative object configuration is harder to debug and understand results when they are unexpected.
- Partial updates using diffs create complex merge and patch operations.
## {{% heading "whatsnext" %}}
- [Managing Kubernetes Objects Using Imperative Commands](/docs/tasks/manage-kubernetes-objects/imperative-command/)
- [Managing Kubernetes Objects Using Object Configuration (Imperative)](/docs/tasks/manage-kubernetes-objects/imperative-config/)
- [Managing Kubernetes Objects Using Object Configuration (Declarative)](/docs/tasks/manage-kubernetes-objects/declarative-config/)
@ -180,4 +176,3 @@ Disadvantages compared to imperative object configuration:
- [Kubectl Book](https://kubectl.docs.kubernetes.io)
- [Kubernetes API Reference](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/)

12
content/en/docs/concepts/policy/limit-range.md
View File

@ -8,13 +8,10 @@ weight: 10
<!-- overview -->
By default, containers run with unbounded [compute resources](/docs/user-guide/compute-resources) on a Kubernetes cluster.
By default, containers run with unbounded [compute resources](/docs/concepts/configuration/manage-resources-containers/) on a Kubernetes cluster.
With resource quotas, cluster administrators can restrict resource consumption and creation on a {{< glossary_tooltip text="namespace" term_id="namespace" >}} basis.
Within a namespace, a Pod or Container can consume as much CPU and memory as defined by the namespace's resource quota. There is a concern that one Pod or Container could monopolize all available resources. A LimitRange is a policy to constrain resource allocations (to Pods or Containers) in a namespace.
<!-- body -->
A _LimitRange_ provides constraints that can:
@ -54,11 +51,8 @@ there may be contention for resources. In this case, the Containers or Pods will
Neither contention nor changes to a LimitRange will affect already created resources.
## {{% heading "whatsnext" %}}
Refer to the [LimitRanger design document](https://git.k8s.io/community/contributors/design-proposals/resource-management/admission_control_limit_range.md) for more information.
For examples on using limits, see:
@ -68,7 +62,5 @@ For examples on using limits, see:
- [how to configure default CPU Requests and Limits per namespace](/docs/tasks/administer-cluster/manage-resources/cpu-default-namespace/).
- [how to configure default Memory Requests and Limits per namespace](/docs/tasks/administer-cluster/manage-resources/memory-default-namespace/).
- [how to configure minimum and maximum Storage consumption per namespace](/docs/tasks/administer-cluster/limit-storage-consumption/#limitrange-to-limit-requests-for-storage).
- a [detailed example on configuring quota per namespace](/docs/tasks/administer-cluster/quota-memory-cpu-namespace/).
- a [detailed example on configuring quota per namespace](/docs/tasks/administer-cluster/manage-resources/quota-memory-cpu-namespace/).

38
content/en/docs/concepts/policy/pod-security-policy.md
View File

@ -14,9 +14,6 @@ weight: 20
Pod Security Policies enable fine-grained authorization of pod creation and
updates.
<!-- body -->
## What is a Pod Security Policy?
@ -143,13 +140,13 @@ For a complete example of authorizing a PodSecurityPolicy, see
### Troubleshooting
- The [Controller Manager](/docs/admin/kube-controller-manager/) must be run
- The [Controller Manager](/docs/reference/command-line-tools-reference/kube-controller-manager/) must be run
against [the secured API port](/docs/reference/access-authn-authz/controlling-access/),
and must not have superuser permissions. Otherwise requests would bypass
authentication and authorization modules, all PodSecurityPolicy objects would be
allowed, and users would be able to create privileged containers. For more details
on configuring Controller Manager authorization, see [Controller
Roles](/docs/reference/access-authn-authz/rbac/#controller-roles).
on configuring Controller Manager authorization, see
[Controller Roles](/docs/reference/access-authn-authz/rbac/#controller-roles).
## Policy Order
@ -599,8 +596,11 @@ documentation](/docs/tutorials/clusters/apparmor/#podsecuritypolicy-annotations)
### Seccomp
The use of seccomp profiles in pods can be controlled via annotations on the
PodSecurityPolicy. Seccomp is an alpha feature in Kubernetes.
As of Kubernetes v1.19, you can use the `seccompProfile` field in the
`securityContext` of Pods or containers to [control use of seccomp
profiles](/docs/tutorials/clusters/seccomp). In prior versions, seccomp was
controlled by adding annotations to a Pod. The same PodSecurityPolicies can be
used with either version to enforce how these fields or annotations are applied.
**seccomp.security.alpha.kubernetes.io/defaultProfileName** - Annotation that
specifies the default seccomp profile to apply to containers. Possible values
@ -609,11 +609,18 @@ are:
- `unconfined` - Seccomp is not applied to the container processes (this is the
default in Kubernetes), if no alternative is provided.
- `runtime/default` - The default container runtime profile is used.
- `docker/default` - The Docker default seccomp profile is used. Deprecated as of
Kubernetes 1.11. Use `runtime/default` instead.
- `docker/default` - The Docker default seccomp profile is used. Deprecated as
of Kubernetes 1.11. Use `runtime/default` instead.
- `localhost/<path>` - Specify a profile as a file on the node located at
`<seccomp_root>/<path>`, where `<seccomp_root>` is defined via the
`--seccomp-profile-root` flag on the Kubelet.
`--seccomp-profile-root` flag on the Kubelet. If the `--seccomp-profile-root`
flag is not defined, the default path will be used, which is
`<root-dir>/seccomp` where `<root-dir>` is specified by the `--root-dir` flag.
{{< note >}}
The `--seccomp-profile-root` flag is deprecated since Kubernetes
v1.19. Users are encouraged to use the default path.
{{< /note >}}
**seccomp.security.alpha.kubernetes.io/allowedProfileNames** - Annotation that
specifies which values are allowed for the pod seccomp annotations. Specified as
@ -629,15 +636,12 @@ By default, all safe sysctls are allowed.
- `allowedUnsafeSysctls` - allows specific sysctls that had been disallowed by the default list, so long as these are not listed in `forbiddenSysctls`.
Refer to the [Sysctl documentation](
/docs/concepts/cluster-administration/sysctl-cluster/#podsecuritypolicy).
/docs/tasks/administer-cluster/sysctl-cluster/#podsecuritypolicy).
## {{% heading "whatsnext" %}}
- See [Pod Security Standards](/docs/concepts/security/pod-security-standards/) for policy recommendations.
See [Pod Security Standards](/docs/concepts/security/pod-security-standards/) for policy recommendations.
Refer to [Pod Security Policy Reference](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#podsecuritypolicy-v1beta1-policy) for the api details.
- Refer to [Pod Security Policy Reference](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#podsecuritypolicy-v1beta1-policy) for the api details.

28
content/en/docs/concepts/policy/resource-quotas.md
View File

@ -13,9 +13,6 @@ there is a concern that one team could use more than its fair share of resources
Resource quotas are a tool for administrators to address this concern.
<!-- body -->
A resource quota, defined by a `ResourceQuota` object, provides constraints that limit
@ -27,15 +24,21 @@ Resource quotas work like this:
- Different teams work in different namespaces. Currently this is voluntary, but
support for making this mandatory via ACLs is planned.
- The administrator creates one `ResourceQuota` for each namespace.
- Users create resources (pods, services, etc.) in the namespace, and the quota system
tracks usage to ensure it does not exceed hard resource limits defined in a `ResourceQuota`.
- If creating or updating a resource violates a quota constraint, the request will fail with HTTP
status code `403 FORBIDDEN` with a message explaining the constraint that would have been violated.
- If quota is enabled in a namespace for compute resources like `cpu` and `memory`, users must specify
requests or limits for those values; otherwise, the quota system may reject pod creation. Hint: Use
the `LimitRanger` admission controller to force defaults for pods that make no compute resource requirements.
See the [walkthrough](/docs/tasks/administer-cluster/quota-memory-cpu-namespace/) for an example of how to avoid this problem.
See the [walkthrough](/docs/tasks/administer-cluster/manage-resources/quota-memory-cpu-namespace/)
for an example of how to avoid this problem.
The name of a `ResourceQuota` object must be a valid
[DNS subdomain name](/docs/concepts/overview/working-with-objects/names#dns-subdomain-names).
@ -63,7 +66,7 @@ A resource quota is enforced in a particular namespace when there is a
## Compute Resource Quota
You can limit the total sum of [compute resources](/docs/user-guide/compute-resources) that can be requested in a given namespace.
You can limit the total sum of [compute resources](/docs/concepts/configuration/manage-resources-containers/) that can be requested in a given namespace.
The following resource types are supported:
@ -77,7 +80,7 @@ The following resource types are supported:
### Resource Quota For Extended Resources
In addition to the resources mentioned above, in release 1.10, quota support for
[extended resources](/docs/concepts/configuration/manage-compute-resources-container/#extended-resources) is added.
[extended resources](/docs/concepts/configuration/manage-resources-containers/#extended-resources) is added.
As overcommit is not allowed for extended resources, it makes no sense to specify both `requests`
and `limits` for the same extended resource in a quota. So for extended resources, only quota items
@ -494,8 +497,7 @@ kubectl create quota test --hard=count/deployments.extensions=2,count/replicaset
```
```shell
kubectl create deployment nginx --image=nginx --namespace=myspace
kubectl scale deployment nginx --replicas=2 --namespace=myspace
kubectl create deployment nginx --image=nginx --namespace=myspace --replicas=2
```
```shell
@ -554,7 +556,7 @@ plugins:
limitedResources:
- resource: pods
matchScopes:
- scopeName: PriorityClass
- scopeName: PriorityClass
operator: In
values: ["cluster-services"]
```
@ -573,7 +575,7 @@ plugins:
limitedResources:
- resource: pods
matchScopes:
- scopeName: PriorityClass
- scopeName: PriorityClass
operator: In
values: ["cluster-services"]
```
@ -596,11 +598,7 @@ See [LimitedResources](https://github.com/kubernetes/kubernetes/pull/36765) and
See a [detailed example for how to use resource quota](/docs/tasks/administer-cluster/quota-api-object/).
## {{% heading "whatsnext" %}}
See [ResourceQuota design doc](https://git.k8s.io/community/contributors/design-proposals/resource-management/admission_control_resource_quota.md) for more information.
- See [ResourceQuota design doc](https://git.k8s.io/community/contributors/design-proposals/resource-management/admission_control_resource_quota.md) for more information.

12
content/en/docs/concepts/scheduling-eviction/kube-scheduler.md
View File

@ -10,8 +10,6 @@ In Kubernetes, _scheduling_ refers to making sure that {{< glossary_tooltip text
are matched to {{< glossary_tooltip text="Nodes" term_id="node" >}} so that
{{< glossary_tooltip term_id="kubelet" >}} can run them.
<!-- body -->
## Scheduling overview {#scheduling}
@ -81,8 +79,8 @@ of the scheduler:
1. [Scheduling Policies](/docs/reference/scheduling/policies) allow you to
configure _Predicates_ for filtering and _Priorities_ for scoring.
1. [Scheduling Profiles](/docs/reference/scheduling/profiles) allow you to
configure Plugins that implement different scheduling stages, including:
1. [Scheduling Profiles](/docs/reference/scheduling/config/#profiles) allow you
to configure Plugins that implement different scheduling stages, including:
`QueueSort`, `Filter`, `Score`, `Bind`, `Reserve`, `Permit`, and others. You
can also configure the kube-scheduler to run different profiles.
@ -92,6 +90,10 @@ of the scheduler:
* Read about [scheduler performance tuning](/docs/concepts/scheduling-eviction/scheduler-perf-tuning/)
* Read about [Pod topology spread constraints](/docs/concepts/workloads/pods/pod-topology-spread-constraints/)
* Read the [reference documentation](/docs/reference/command-line-tools-reference/kube-scheduler/) for kube-scheduler
* Learn about [configuring multiple schedulers](/docs/tasks/administer-cluster/configure-multiple-schedulers/)
* Learn about [configuring multiple schedulers](/docs/tasks/extend-kubernetes/configure-multiple-schedulers/)
* Learn about [topology management policies](/docs/tasks/administer-cluster/topology-manager/)
* Learn about [Pod Overhead](/docs/concepts/configuration/pod-overhead/)
* Learn about scheduling of Pods that use volumes in:
* [Volume Topology Support](/docs/concepts/storage/storage-classes/#volume-binding-mode)
* [Storage Capacity Tracking](/docs/concepts/storage/storage-capacity/)
* [Node-specific Volume Limits](/docs/concepts/storage/storage-limits/)

61
content/en/docs/concepts/scheduling-eviction/scheduling-framework.md
View File

@ -75,6 +75,14 @@ node, the scheduler will call filter plugins in their configured order. If any
filter plugin marks the node as infeasible, the remaining plugins will not be
called for that node. Nodes may be evaluated concurrently.
### PostFilter {#post-filter}
These plugins are called after Filter phase, but only when no feasible nodes
were found for the node. Plugins are called in their configured order. If
any postFilter plugin marks the node as `Schedulable`, the remaining plugins
will not be called. A typical PostFilter implementation is preemption, which
tries to make the pod schedulable by preempting other Pods.
### PreScore {#pre-score}
These plugins are used to perform "pre-scoring" work, which generates a sharable
@ -129,17 +137,31 @@ Plugins wishing to perform "pre-reserve" work should use the
NormalizeScore extension point.
{{< /note >}}
### Reserve
### Reserve {#reserve}
This is an informational extension point. Plugins which maintain runtime state
(aka "stateful plugins") should use this extension point to be notified by the
scheduler when resources on a node are being reserved for a given Pod. This
happens before the scheduler actually binds the Pod to the Node, and it exists
to prevent race conditions while the scheduler waits for the bind to succeed.
A plugin that implements the Reserve extension has two methods, namely `Reserve`
and `Unreserve`, that back two informational scheduling phases called Reserve
and Unreserve, respectively. Plugins which maintain runtime state (aka "stateful
plugins") should use these phases to be notified by the scheduler when resources
on a node are being reserved and unreserved for a given Pod.
This is the last step in a scheduling cycle. Once a Pod is in the reserved
state, it will either trigger [Unreserve](#unreserve) plugins (on failure) or
[PostBind](#post-bind) plugins (on success) at the end of the binding cycle.
The Reserve phase happens before the scheduler actually binds a Pod to its
designated node. It exists to prevent race conditions while the scheduler waits
for the bind to succeed. The `Reserve` method of each Reserve plugin may succeed
or fail; if one `Reserve` method call fails, subsequent plugins are not executed
and the Reserve phase is considered to have failed. If the `Reserve` method of
all plugins succeed, the Reserve phase is considered to be successful and the
rest of the scheduling cycle and the binding cycle are executed.
The Unreserve phase is triggered if the Reserve phase or a later phase fails.
When this happens, the `Unreserve` method of **all** Reserve plugins will be
executed in the reverse order of `Reserve` method calls. This phase exists to
clean up the state associated with the reserved Pod.
{{< caution >}}
The implementation of the `Unreserve` method in Reserve plugins must be
idempotent and may not fail.
{{< /caution >}}
### Permit
@ -152,14 +174,14 @@ the three things:
1. **deny** \
If any Permit plugin denies a Pod, it is returned to the scheduling queue.
This will trigger [Unreserve](#unreserve) plugins.
This will trigger the Unreserve phase in [Reserve plugins](#reserve).
1. **wait** (with a timeout) \
If a Permit plugin returns "wait", then the Pod is kept in an internal "waiting"
Pods list, and the binding cycle of this Pod starts but directly blocks until it
gets approved. If a timeout occurs, **wait** becomes **deny**
and the Pod is returned to the scheduling queue, triggering [Unreserve](#unreserve)
plugins.
and the Pod is returned to the scheduling queue, triggering the
Unreserve phase in [Reserve plugins](#reserve).
{{< note >}}
While any plugin can access the list of "waiting" Pods and approve them
@ -174,7 +196,7 @@ These plugins are used to perform any work required before a Pod is bound. For
example, a pre-bind plugin may provision a network volume and mount it on the
target node before allowing the Pod to run there.
If any PreBind plugin returns an error, the Pod is [rejected](#unreserve) and
If any PreBind plugin returns an error, the Pod is [rejected](#reserve) and
returned to the scheduling queue.
### Bind
@ -191,15 +213,6 @@ This is an informational extension point. Post-bind plugins are called after a
Pod is successfully bound. This is the end of a binding cycle, and can be used
to clean up associated resources.
### Unreserve
This is an informational extension point. If a Pod was reserved and then
rejected in a later phase, then unreserve plugins will be notified. Unreserve
plugins should clean up state associated with the reserved Pod.
Plugins that use this extension point usually should also use
[Reserve](#reserve).
## Plugin API
There are two steps to the plugin API. First, plugins must register and get
@ -228,7 +241,7 @@ type PreFilterPlugin interface {
You can enable or disable plugins in the scheduler configuration. If you are using
Kubernetes v1.18 or later, most scheduling
[plugins](/docs/reference/scheduling/profiles/#scheduling-plugins) are in use and
[plugins](/docs/reference/scheduling/config/#scheduling-plugins) are in use and
enabled by default.
In addition to default plugins, you can also implement your own scheduling
@ -237,5 +250,5 @@ plugins and get them configured along with default plugins. You can visit
If you are using Kubernetes v1.18 or later, you can configure a set of plugins as
a scheduler profile and then define multiple profiles to fit various kinds of workload.
Learn more at [multiple profiles](/docs/reference/scheduling/profiles/#multiple-profiles).
Learn more at [multiple profiles](/docs/reference/scheduling/config/#multiple-profiles).

4
content/en/docs/concepts/scheduling-eviction/taint-and-toleration.md
View File

@ -62,7 +62,7 @@ tolerations:
effect: "NoSchedule"
```
Here’s an example of a pod that uses tolerations:
Here's an example of a pod that uses tolerations:
{{< codenew file="pods/pod-with-toleration.yaml" >}}
@ -175,7 +175,7 @@ toleration to pods that use the special hardware. As in the dedicated nodes use
it is probably easiest to apply the tolerations using a custom
[admission controller](/docs/reference/access-authn-authz/admission-controllers/).
For example, it is recommended to use [Extended
Resources](/docs/concepts/configuration/manage-compute-resources-container/#extended-resources)
Resources](/docs/concepts/configuration/manage-resources-containers/#extended-resources)
to represent the special hardware, taint your special hardware nodes with the
extended resource name and run the
[ExtendedResourceToleration](/docs/reference/access-authn-authz/admission-controllers/#extendedresourcetoleration)

19
content/en/docs/concepts/security/pod-security-standards.md
View File

@ -209,11 +209,11 @@ well as lower-trust users.The following listed controls should be enforced/disal
<tr>
<td>Privilege Escalation</td>
<td>
Privilege escalation to root should not be allowed.<br>
Privilege escalation (such as via set-user-ID or set-group-ID file mode) should not be allowed.<br>
<br><b>Restricted Fields:</b><br>
spec.containers[*].securityContext.privileged<br>
spec.initContainers[*].securityContext.privileged<br>
<br><b>Allowed Values:</b> false, undefined/nil<br>
spec.containers[*].securityContext.allowPrivilegeEscalation<br>
spec.initContainers[*].securityContext.allowPrivilegeEscalation<br>
<br><b>Allowed Values:</b> false<br>
</td>
</tr>
<tr>
@ -245,13 +245,14 @@ well as lower-trust users.The following listed controls should be enforced/disal
<tr>
<td>Seccomp</td>
<td>
The 'runtime/default' seccomp profile must be required, or allow specific additional profiles.<br>
The RuntimeDefault seccomp profile must be required, or allow specific additional profiles.<br>
<br><b>Restricted Fields:</b><br>
metadata.annotations['seccomp.security.alpha.kubernetes.io/pod']<br>
metadata.annotations['container.seccomp.security.alpha.kubernetes.io/*']<br>
spec.securityContext.seccompProfile.type<br>
spec.containers[*].securityContext.seccompProfile<br>
spec.initContainers[*].securityContext.seccompProfile<br>
<br><b>Allowed Values:</b><br>
'runtime/default'<br>
undefined (container annotation)<br>
undefined / nil<br>
</td>
</tr>
</tbody>
@ -316,6 +317,6 @@ restrict privileged permissions is lessened when the workload is isolated from t
kernel. This allows for workloads requiring heightened permissions to still be isolated.
Additionally, the protection of sandboxed workloads is highly dependent on the method of
sandboxing. As such, no single ‘recommended’ policy is recommended for all sandboxed workloads.
sandboxing. As such, no single recommended policy is recommended for all sandboxed workloads.

4
content/en/docs/concepts/services-networking/connect-applications-service.md
View File

@ -15,7 +15,7 @@ weight: 30
Now that you have a continuously running, replicated application you can expose it on a network. Before discussing the Kubernetes approach to networking, it is worthwhile to contrast it with the "normal" way networking works with Docker.
By default, Docker uses host-private networking, so containers can talk to other containers only if they are on the same machine. In order for Docker containers to communicate across nodes, there must be allocated ports on the machine’s own IP address, which are then forwarded or proxied to the containers. This obviously means that containers must either coordinate which ports they use very carefully or ports must be allocated dynamically.
By default, Docker uses host-private networking, so containers can talk to other containers only if they are on the same machine. In order for Docker containers to communicate across nodes, there must be allocated ports on the machine's own IP address, which are then forwarded or proxied to the containers. This obviously means that containers must either coordinate which ports they use very carefully or ports must be allocated dynamically.
Coordinating port allocations across multiple developers or teams that provide containers is very difficult to do at scale, and exposes users to cluster-level issues outside of their control. Kubernetes assumes that pods can communicate with other pods, regardless of which host they land on. Kubernetes gives every pod its own cluster-private IP address, so you do not need to explicitly create links between pods or map container ports to host ports. This means that containers within a Pod can all reach each other's ports on localhost, and all pods in a cluster can see each other without NAT. The rest of this document elaborates on how you can run reliable services on such a networking model.
@ -133,7 +133,7 @@ about the [service proxy](/docs/concepts/services-networking/service/#virtual-ip
Kubernetes supports 2 primary modes of finding a Service - environment variables
and DNS. The former works out of the box while the latter requires the
[CoreDNS cluster addon](http://releases.k8s.io/{{< param "githubbranch" >}}/cluster/addons/dns/coredns).
[CoreDNS cluster addon](https://releases.k8s.io/{{< param "githubbranch" >}}/cluster/addons/dns/coredns).
{{< note >}}
If the service environment variables are not desired (because possible clashing with expected program ones,
too many variables to process, only using DNS, etc) you can disable this mode by setting the `enableServiceLinks`

36
content/en/docs/concepts/services-networking/dns-pod-service.md
View File

@ -68,10 +68,19 @@ of the form `auto-generated-name.my-svc.my-namespace.svc.cluster-domain.example`
### A/AAAA records
Any pods created by a Deployment or DaemonSet have the following
DNS resolution available:
In general a pod has the following DNS resolution:
`pod-ip-address.deployment-name.my-namespace.svc.cluster-domain.example.`
`pod-ip-address.my-namespace.pod.cluster-domain.example`.
For example, if a pod in the `default` namespace has the IP address 172.17.0.3,
and the domain name for your cluster is `cluster.local`, then the Pod has a DNS name:
`172-17-0-3.default.pod.cluster.local`.
Any pods created by a Deployment or DaemonSet exposed by a Service have the
following DNS resolution available:
`pod-ip-address.deployment-name.my-namespace.svc.cluster-domain.example`.
### Pod's hostname and subdomain fields
@ -157,6 +166,24 @@ pointing to the Pod's IP address. Also, Pod needs to become ready in order to ha
record unless `publishNotReadyAddresses=True` is set on the Service.
{{< /note >}}
### Pod's setHostnameAsFQDN field {#pod-sethostnameasfqdn-field}
{{< feature-state for_k8s_version="v1.19" state="alpha" >}}
**Prerequisites**: The `SetHostnameAsFQDN` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/)
must be enabled for the
{{< glossary_tooltip text="API Server" term_id="kube-apiserver" >}}
When a Pod is configured to have fully qualified domain name (FQDN), its hostname is the short hostname. For example, if you have a Pod with the fully qualified domain name `busybox-1.default-subdomain.my-namespace.svc.cluster-domain.example`, then by default the `hostname` command inside that Pod returns `busybox-1` and the `hostname --fqdn` command returns the FQDN.
When you set `setHostnameAsFQDN: true` in the Pod spec, the kubelet writes the Pod's FQDN into the hostname for that Pod's namespace. In this case, both `hostname` and `hostname --fqdn` return the Pod's FQDN.
{{< note >}}
In Linux, the hostname field of the kernel (the `nodename` field of `struct utsname`) is limited to 64 characters.
If a Pod enables this feature and its FQDN is longer than 64 character, it will fail to start. The Pod will remain in `Pending` status (`ContainerCreating` as seen by `kubectl`) generating error events, such as Failed to construct FQDN from pod hostname and cluster domain, FQDN `long-FDQN` is too long (64 characters is the max, 70 characters requested). One way of improving user experience for this scenario is to create an [admission webhook controller](/docs/reference/access-authn-authz/extensible-admission-controllers/#admission-webhooks) to control FQDN size when users create top level objects, for example, Deployment.
{{< /note >}}
### Pod's DNS Policy
DNS policies can be set on a per-pod basis. Currently Kubernetes supports the
@ -182,7 +209,7 @@ following pod-specific DNS policies. These policies are specified in the
{{< note >}}
"Default" is not the default DNS policy. If `dnsPolicy` is not
explicitly specified, then “ClusterFirst” is used.
explicitly specified, then "ClusterFirst" is used.
{{< /note >}}
@ -276,4 +303,3 @@ The availability of Pod DNS Config and DNS Policy "`None`" is shown as below.
For guidance on administering DNS configurations, check
[Configure DNS Service](/docs/tasks/administer-cluster/dns-custom-nameservers/)

118
content/en/docs/concepts/services-networking/endpoint-slices.md
View File

@ -3,7 +3,7 @@ reviewers:
- freehan
title: EndpointSlices
content_type: concept
weight: 15
weight: 35
---
@ -23,7 +23,9 @@ Endpoints.
The Endpoints API has provided a simple and straightforward way of
tracking network endpoints in Kubernetes. Unfortunately as Kubernetes clusters
and Services have gotten larger, limitations of that API became more visible.
and {{< glossary_tooltip text="Services" term_id="service" >}} have grown to handle
send more traffic to more backend Pods, limitations of that original API became
more visible.
Most notably, those included challenges with scaling to larger numbers of
network endpoints.
@ -37,11 +39,12 @@ platform for additional features such as topological routing.
## EndpointSlice resources {#endpointslice-resource}
In Kubernetes, an EndpointSlice contains references to a set of network
endpoints. The EndpointSlice controller automatically creates EndpointSlices
for a Kubernetes Service when a {{< glossary_tooltip text="selector"
term_id="selector" >}} is specified. These EndpointSlices will include
references to any Pods that match the Service selector. EndpointSlices group
network endpoints together by unique Service and Port combinations.
endpoints. The control plane automatically creates EndpointSlices
for any Kubernetes Service that has a {{< glossary_tooltip text="selector"
term_id="selector" >}} specified. These EndpointSlices include
references to all the Pods that match the Service selector. EndpointSlices group
network endpoints together by unique combinations of protocol, port number, and
Service name.
The name of a EndpointSlice object must be a valid
[DNS subdomain name](/docs/concepts/overview/working-with-objects/names#dns-subdomain-names).
@ -71,15 +74,18 @@ endpoints:
topology.kubernetes.io/zone: us-west2-a
```
By default, EndpointSlices managed by the EndpointSlice controller will have no
more than 100 endpoints each. Below this scale, EndpointSlices should map 1:1
with Endpoints and Services and have similar performance.
By default, the control plane creates and manages EndpointSlices to have no
more than 100 endpoints each. You can configure this with the
`--max-endpoints-per-slice`
{{< glossary_tooltip text="kube-controller-manager" term_id="kube-controller-manager" >}}
flag, up to a maximum of 1000.
EndpointSlices can act as the source of truth for kube-proxy when it comes to
EndpointSlices can act as the source of truth for
{{< glossary_tooltip term_id="kube-proxy" text="kube-proxy" >}} when it comes to
how to route internal traffic. When enabled, they should provide a performance
improvement for services with large numbers of endpoints.
### Address Types
### Address types
EndpointSlices support three address types:
@ -87,55 +93,66 @@ EndpointSlices support three address types:
* IPv6
* FQDN (Fully Qualified Domain Name)
### Topology
### Topology information {#topology}
Each endpoint within an EndpointSlice can contain relevant topology information.
This is used to indicate where an endpoint is, containing information about the
corresponding Node, zone, and region. When the values are available, the
following Topology labels will be set by the EndpointSlice controller:
control plane sets the following Topology labels for EndpointSlices:
* `kubernetes.io/hostname` - The name of the Node this endpoint is on.
* `topology.kubernetes.io/zone` - The zone this endpoint is in.
* `topology.kubernetes.io/zone` - The zone this endpoint is in.
* `topology.kubernetes.io/region` - The region this endpoint is in.
The values of these labels are derived from resources associated with each
endpoint in a slice. The hostname label represents the value of the NodeName
field on the corresponding Pod. The zone and region labels represent the value
of the labels with the same names on the corresponding Node.
of the labels with the same names on the corresponding Node.
### Management
By default, EndpointSlices are created and managed by the EndpointSlice
controller. There are a variety of other use cases for EndpointSlices, such as
service mesh implementations, that could result in other entities or controllers
managing additional sets of EndpointSlices. To ensure that multiple entities can
manage EndpointSlices without interfering with each other, a
`endpointslice.kubernetes.io/managed-by` label is used to indicate the entity
managing an EndpointSlice. The EndpointSlice controller sets
`endpointslice-controller.k8s.io` as the value for this label on all
EndpointSlices it manages. Other entities managing EndpointSlices should also
set a unique value for this label.
Most often, the control plane (specifically, the endpoint slice
{{< glossary_tooltip text="controller" term_id="controller" >}}) creates and
manages EndpointSlice objects. There are a variety of other use cases for
EndpointSlices, such as service mesh implementations, that could result in othe
rentities or controllers managing additional sets of EndpointSlices.
To ensure that multiple entities can manage EndpointSlices without interfering
with each other, Kubernetes defines the
{{< glossary_tooltip term_id="label" text="label" >}}
`endpointslice.kubernetes.io/managed-by`, which indicates the entity managing
an EndpointSlice.
The endpoint slice controller sets `endpointslice-controller.k8s.io` as the value
for this label on all EndpointSlices it manages. Other entities managing
EndpointSlices should also set a unique value for this label.
### Ownership
In most use cases, EndpointSlices will be owned by the Service that it tracks
endpoints for. This is indicated by an owner reference on each EndpointSlice as
well as a `kubernetes.io/service-name` label that enables simple lookups of all
EndpointSlices belonging to a Service.
In most use cases, EndpointSlices are owned by the Service that the endpoint
slice object tracks endpoints for. This ownership is indicated by an owner
reference on each EndpointSlice as well as a `kubernetes.io/service-name`
label that enables simple lookups of all EndpointSlices belonging to a Service.
## EndpointSlice Controller
### EndpointSlice mirroring
The EndpointSlice controller watches Services and Pods to ensure corresponding
EndpointSlices are up to date. The controller will manage EndpointSlices for
every Service with a selector specified. These will represent the IPs of Pods
matching the Service selector.
In some cases, applications create custom Endpoints resources. To ensure that
these applications do not need to concurrently write to both Endpoints and
EndpointSlice resources, the cluster's control plane mirrors most Endpoints
resources to corresponding EndpointSlices.
### Size of EndpointSlices
The control plane mirrors Endpoints resources unless:
By default, EndpointSlices are limited to a size of 100 endpoints each. You can
configure this with the `--max-endpoints-per-slice` {{< glossary_tooltip
text="kube-controller-manager" term_id="kube-controller-manager" >}} flag up to
a maximum of 1000.
* the Endpoints resource has a `endpointslice.kubernetes.io/skip-mirror` label
set to `true`.
* the Endpoints resource has a `control-plane.alpha.kubernetes.io/leader`
annotation.
* the corresponding Service resource does not exist.
* the corresponding Service resource has a non-nil selector.
Individual Endpoints resources may translate into multiple EndpointSlices. This
will occur if an Endpoints resource has multiple subsets or includes endpoints
with multiple IP families (IPv4 and IPv6). A maximum of 1000 addresses per
subset will be mirrored to EndpointSlices.
### Distribution of EndpointSlices
@ -145,8 +162,8 @@ different target port numbers for the same named port, requiring different
EndpointSlices. This is similar to the logic behind how subsets are grouped
with Endpoints.
The controller tries to fill EndpointSlices as full as possible, but does not
actively rebalance them. The logic of the controller is fairly straightforward:
The control plane tries to fill EndpointSlices as full as possible, but does not
actively rebalance them. The logic is fairly straightforward:
1. Iterate through existing EndpointSlices, remove endpoints that are no longer
desired and update matching endpoints that have changed.
@ -154,7 +171,7 @@ actively rebalance them. The logic of the controller is fairly straightforward:
fill them up with any new endpoints needed.
3. If there's still new endpoints left to add, try to fit them into a previously
unchanged slice and/or create new ones.
Importantly, the third step prioritizes limiting EndpointSlice updates over a
perfectly full distribution of EndpointSlices. As an example, if there are 10
new endpoints to add and 2 EndpointSlices with room for 5 more endpoints each,
@ -172,15 +189,20 @@ In practice, this less than ideal distribution should be rare. Most changes
processed by the EndpointSlice controller will be small enough to fit in an
existing EndpointSlice, and if not, a new EndpointSlice is likely going to be
necessary soon anyway. Rolling updates of Deployments also provide a natural
repacking of EndpointSlices with all pods and their corresponding endpoints
repacking of EndpointSlices with all Pods and their corresponding endpoints
getting replaced.
### Duplicate endpoints
Due to the nature of EndpointSlice changes, endpoints may be represented in more
than one EndpointSlice at the same time. This naturally occurs as changes to
different EndpointSlice objects can arrive at the Kubernetes client watch/cache
at different times. Implementations using EndpointSlice must be able to have the
endpoint appear in more than one slice. A reference implementation of how to
perform endpoint deduplication can be found in the `EndpointSliceCache`
implementation in `kube-proxy`.
## {{% heading "whatsnext" %}}
* [Enabling EndpointSlices](/docs/tasks/administer-cluster/enabling-endpointslices)
* Learn about [Enabling EndpointSlices](/docs/tasks/administer-cluster/enabling-endpointslices)
* Read [Connecting Applications with Services](/docs/concepts/services-networking/connect-applications-service/)

2
content/en/docs/concepts/services-networking/ingress-controllers.md
View File

@ -26,7 +26,7 @@ Kubernetes as a project currently supports and maintains [GCE](https://git.k8s.i
* [Ambassador](https://www.getambassador.io/) API Gateway is an [Envoy](https://www.envoyproxy.io) based ingress
controller with [community](https://www.getambassador.io/docs) or
[commercial](https://www.getambassador.io/pro/) support from [Datawire](https://www.datawire.io/).
* [AppsCode Inc.](https://appscode.com) offers support and maintenance for the most widely used [HAProxy](http://www.haproxy.org/) based ingress controller [Voyager](https://appscode.com/products/voyager).
* [AppsCode Inc.](https://appscode.com) offers support and maintenance for the most widely used [HAProxy](https://www.haproxy.org/) based ingress controller [Voyager](https://appscode.com/products/voyager).
* [AWS ALB Ingress Controller](https://github.com/kubernetes-sigs/aws-alb-ingress-controller) enables ingress using the [AWS Application Load Balancer](https://aws.amazon.com/elasticloadbalancing/).
* [Contour](https://projectcontour.io/) is an [Envoy](https://www.envoyproxy.io/) based ingress controller
provided and supported by VMware.

304
content/en/docs/concepts/services-networking/ingress.md
View File

@ -7,7 +7,7 @@ weight: 40
---
<!-- overview -->
{{< feature-state for_k8s_version="v1.1" state="beta" >}}
{{< feature-state for_k8s_version="v1.19" state="stable" >}}
{{< glossary_definition term_id="ingress" length="all" >}}
@ -25,7 +25,7 @@ For clarity, this guide defines the following terms:
## What is Ingress?
[Ingress](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#ingress-v1beta1-networking-k8s-io) exposes HTTP and HTTPS routes from outside the cluster to
[Ingress](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#ingress-v1-networking-k8s-io) exposes HTTP and HTTPS routes from outside the cluster to
{{< link text="services" url="/docs/concepts/services-networking/service/" >}} within the cluster.
Traffic routing is controlled by rules defined on the Ingress resource.
@ -57,27 +57,11 @@ controllers operate slightly differently.
Make sure you review your Ingress controller's documentation to understand the caveats of choosing it.
{{< /note >}}
## The Ingress Resource
## The Ingress resource
A minimal Ingress resource example:
```yaml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: test-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- http:
paths:
- path: /testpath
pathType: Prefix
backend:
serviceName: test
servicePort: 80
```
{{< codenew file="service/networking/minimal-ingress.yaml" >}}
As with all other Kubernetes resources, an Ingress needs `apiVersion`, `kind`, and `metadata` fields.
The name of an Ingress object must be a valid
@ -100,75 +84,134 @@ Each HTTP rule contains the following information:
* An optional host. In this example, no host is specified, so the rule applies to all inbound
HTTP traffic through the IP address specified. If a host is provided (for example,
foo.bar.com), the rules apply to that host.
* A list of paths (for example, `/testpath`), each of which has an associated backend defined with a `serviceName`
and `servicePort`. Both the host and path must match the content of an incoming request before the
load balancer directs traffic to the referenced Service.
* A list of paths (for example, `/testpath`), each of which has an associated
backend defined with a `service.name` and a `service.port.name` or
`service.port.number`. Both the host and path must match the content of an
incoming request before the load balancer directs traffic to the referenced
Service.
* A backend is a combination of Service and port names as described in the
[Service doc](/docs/concepts/services-networking/service/). HTTP (and HTTPS) requests to the
[Service doc](/docs/concepts/services-networking/service/) or a [custom resource backend](#resource-backend) by way of a {{< glossary_tooltip term_id="CustomResourceDefinition" text="CRD" >}}. HTTP (and HTTPS) requests to the
Ingress that matches the host and path of the rule are sent to the listed backend.
A default backend is often configured in an Ingress controller to service any requests that do not
A `defaultBackend` is often configured in an Ingress controller to service any requests that do not
match a path in the spec.
### Default Backend
### DefaultBackend {#default-backend}
An Ingress with no rules sends all traffic to a single default backend. The default
backend is typically a configuration option of the [Ingress controller](/docs/concepts/services-networking/ingress-controllers) and is not specified in your Ingress resources.
An Ingress with no rules sends all traffic to a single default backend. The `defaultBackend` is conventionally a configuration option
of the [Ingress controller](/docs/concepts/services-networking/ingress-controllers) and is not specified in your Ingress resources.
If none of the hosts or paths match the HTTP request in the Ingress objects, the traffic is
routed to your default backend.
### Path Types
### Resource backends {#resource-backend}
Each path in an Ingress has a corresponding path type. There are three supported
path types:
A `Resource` backend is an ObjectRef to another Kubernetes resource within the
same namespace of the Ingress object. A `Resource` is a mutually exclusive
setting with Service, and will fail validation if both are specified. A common
usage for a `Resource` backend is to ingress data to an object storage backend
with static assets.
* _`ImplementationSpecific`_ (default): With this path type, matching is up to
the IngressClass. Implementations can treat this as a separate `pathType` or
treat it identically to `Prefix` or `Exact` path types.
{{< codenew file="service/networking/ingress-resource-backend.yaml" >}}
* _`Exact`_: Matches the URL path exactly and with case sensitivity.
After creating the Ingress above, you can view it with the following command:
* _`Prefix`_: Matches based on a URL path prefix split by `/`. Matching is case
```bash
kubectl describe ingress ingress-resource-backend
```
```
Name: ingress-resource-backend
Namespace: default
Address:
Default backend: APIGroup: k8s.example.com, Kind: StorageBucket, Name: static-assets
Rules:
Host Path Backends
---- ---- --------
*
/icons APIGroup: k8s.example.com, Kind: StorageBucket, Name: icon-assets
Annotations: <none>
Events: <none>
```
### Path types
Each path in an Ingress is required to have a corresponding path type. Paths
that do not include an explicit `pathType` will fail validation. There are three
supported path types:
* `ImplementationSpecific`: With this path type, matching is up to the
IngressClass. Implementations can treat this as a separate `pathType` or treat
it identically to `Prefix` or `Exact` path types.
* `Exact`: Matches the URL path exactly and with case sensitivity.
* `Prefix`: Matches based on a URL path prefix split by `/`. Matching is case
sensitive and done on a path element by element basis. A path element refers
to the list of labels in the path split by the `/` separator. A request is a
match for path _p_ if every _p_ is an element-wise prefix of _p_ of the
request path.
{{< note >}}
If the last element of the path is a substring of the last element in request path, it is not a match (for example: `/foo/bar` matches`/foo/bar/baz`, but does not match `/foo/barbaz`).
{{< /note >}}
#### Multiple Matches
{{< note >}} If the last element of the path is a substring of the last
element in request path, it is not a match (for example: `/foo/bar`
matches`/foo/bar/baz`, but does not match `/foo/barbaz`). {{< /note >}}
### Examples
| Kind | Path(s) | Request path(s) | Matches? |
|--------|---------------------------------|-------------------------------|------------------------------------|
| Prefix | `/` | (all paths) | Yes |
| Exact | `/foo` | `/foo` | Yes |
| Exact | `/foo` | `/bar` | No |
| Exact | `/foo` | `/foo/` | No |
| Exact | `/foo/` | `/foo` | No |
| Prefix | `/foo` | `/foo`, `/foo/` | Yes |
| Prefix | `/foo/` | `/foo`, `/foo/` | Yes |
| Prefix | `/aaa/bb` | `/aaa/bbb` | No |
| Prefix | `/aaa/bbb` | `/aaa/bbb` | Yes |
| Prefix | `/aaa/bbb/` | `/aaa/bbb` | Yes, ignores trailing slash |
| Prefix | `/aaa/bbb` | `/aaa/bbb/` | Yes, matches trailing slash |
| Prefix | `/aaa/bbb` | `/aaa/bbb/ccc` | Yes, matches subpath |
| Prefix | `/aaa/bbb` | `/aaa/bbbxyz` | No, does not match string prefix |
| Prefix | `/`, `/aaa` | `/aaa/ccc` | Yes, matches `/aaa` prefix |
| Prefix | `/`, `/aaa`, `/aaa/bbb` | `/aaa/bbb` | Yes, matches `/aaa/bbb` prefix |
| Prefix | `/`, `/aaa`, `/aaa/bbb` | `/ccc` | Yes, matches `/` prefix |
| Prefix | `/aaa` | `/ccc` | No, uses default backend |
| Mixed | `/foo` (Prefix), `/foo` (Exact) | `/foo` | Yes, prefers Exact |
#### Multiple matches
In some cases, multiple paths within an Ingress will match a request. In those
cases precedence will be given first to the longest matching path. If two paths
are still equally matched, precedence will be given to paths with an exact path
type over prefix path type.
## Ingress Class
## Hostname wildcards
Hosts can be precise matches (for example “`foo.bar.com`”) or a wildcard (for
example “`*.foo.com`”). Precise matches require that the HTTP `host` header
matches the `host` field. Wildcard matches require the HTTP `host` header is
equal to the suffix of the wildcard rule.
| Host | Host header | Match? |
| ----------- |-------------------| --------------------------------------------------|
| `*.foo.com` | `bar.foo.com` | Matches based on shared suffix |
| `*.foo.com` | `baz.bar.foo.com` | No match, wildcard only covers a single DNS label |
| `*.foo.com` | `foo.com` | No match, wildcard only covers a single DNS label |
{{< codenew file="service/networking/ingress-wildcard-host.yaml" >}}
## Ingress class
Ingresses can be implemented by different controllers, often with different
configuration. Each Ingress should specify a class, a reference to an
IngressClass resource that contains additional configuration including the name
of the controller that should implement the class.
```yaml
apiVersion: networking.k8s.io/v1beta1
kind: IngressClass
metadata:
name: external-lb
spec:
controller: example.com/ingress-controller
parameters:
apiGroup: k8s.example.com/v1alpha
kind: IngressParameters
name: external-lb
```
{{< codenew file="service/networking/external-lb.yaml" >}}
IngressClass resources contain an optional parameters field. This can be used to
reference additional configuration for this class.
### Deprecated Annotation
### Deprecated annotation
Before the IngressClass resource and `ingressClassName` field were added in
Kubernetes 1.18, Ingress classes were specified with a
@ -181,7 +224,7 @@ used to reference the name of the Ingress controller that should implement the
Ingress, the field is a reference to an IngressClass resource that contains
additional Ingress configuration, including the name of the Ingress controller.
### Default Ingress Class
### Default IngressClass {#default-ingress-class}
You can mark a particular IngressClass as default for your cluster. Setting the
`ingressclass.kubernetes.io/is-default-class` annotation to `true` on an
@ -197,24 +240,24 @@ IngressClasses are marked as default in your cluster.
## Types of Ingress
### Single Service Ingress
### Ingress backed by a single Service {#single-service-ingress}
There are existing Kubernetes concepts that allow you to expose a single Service
(see [alternatives](#alternatives)). You can also do this with an Ingress by specifying a
*default backend* with no rules.
{{< codenew file="service/networking/ingress.yaml" >}}
{{< codenew file="service/networking/test-ingress.yaml" >}}
If you create it using `kubectl apply -f` you should be able to view the state
of the Ingress you just added:
```shell
```bash
kubectl get ingress test-ingress
```
```
NAME HOSTS ADDRESS PORTS AGE
test-ingress * 203.0.113.123 80 59s
NAME CLASS HOSTS ADDRESS PORTS AGE
test-ingress external-lb * 203.0.113.123 80 59s
```
Where `203.0.113.123` is the IP allocated by the Ingress controller to satisfy
@ -231,34 +274,14 @@ A fanout configuration routes traffic from a single IP address to more than one
based on the HTTP URI being requested. An Ingress allows you to keep the number of load balancers
down to a minimum. For example, a setup like:
```none
```
foo.bar.com -> 178.91.123.132 -> / foo service1:4200
/ bar service2:8080
```
would require an Ingress such as:
```yaml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: simple-fanout-example
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host: foo.bar.com
http:
paths:
- path: /foo
backend:
serviceName: service1
servicePort: 4200
- path: /bar
backend:
serviceName: service2
servicePort: 8080
```
{{< codenew file="service/networking/simple-fanout-example.yaml" >}}
When you create the Ingress with `kubectl apply -f`:
@ -277,8 +300,6 @@ Rules:
foo.bar.com
/foo service1:4200 (10.8.0.90:4200)
/bar service2:8080 (10.8.0.91:8080)
Annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
@ -291,7 +312,7 @@ When it has done so, you can see the address of the load balancer at the
Address field.
{{< note >}}
Depending on the [Ingress controller](/docs/concepts/services-networking/ingress-controllers)
Depending on the [Ingress controller](/docs/concepts/services-networking/ingress-controllers/)
you are using, you may need to create a default-http-backend
[Service](/docs/concepts/services-networking/service/).
{{< /note >}}
@ -309,68 +330,26 @@ bar.foo.com --| |-> bar.foo.com service2:80
The following Ingress tells the backing load balancer to route requests based on
the [Host header](https://tools.ietf.org/html/rfc7230#section-5.4).
```yaml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: name-virtual-host-ingress
spec:
rules:
- host: foo.bar.com
http:
paths:
- backend:
serviceName: service1
servicePort: 80
- host: bar.foo.com
http:
paths:
- backend:
serviceName: service2
servicePort: 80
```
{{< codenew file="service/networking/name-virtual-host-ingress.yaml" >}}
If you create an Ingress resource without any hosts defined in the rules, then any
web traffic to the IP address of your Ingress controller can be matched without a name based
virtual host being required.
For example, the following Ingress resource will route traffic
For example, the following Ingress routes traffic
requested for `first.bar.com` to `service1`, `second.foo.com` to `service2`, and any traffic
to the IP address without a hostname defined in request (that is, without a request header being
presented) to `service3`.
```yaml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: name-virtual-host-ingress
spec:
rules:
- host: first.bar.com
http:
paths:
- backend:
serviceName: service1
servicePort: 80
- host: second.foo.com
http:
paths:
- backend:
serviceName: service2
servicePort: 80
- http:
paths:
- backend:
serviceName: service3
servicePort: 80
```
{{< codenew file="service/networking/name-virtual-host-ingress-no-third-host.yaml" >}}
### TLS
You can secure an Ingress by specifying a {{< glossary_tooltip term_id="secret" >}}
that contains a TLS private key and certificate. Currently the Ingress only
supports a single TLS port, 443, and assumes TLS termination. If the TLS
configuration section in an Ingress specifies different hosts, they are
that contains a TLS private key and certificate. The Ingress resource only
supports a single TLS port, 443, and assumes TLS termination at the ingress point
(traffic to the Service and its Pods is in plaintext).
If the TLS configuration section in an Ingress specifies different hosts, they are
multiplexed on the same port according to the hostname specified through the
SNI TLS extension (provided the Ingress controller supports SNI). The TLS secret
must contain keys named `tls.crt` and `tls.key` that contain the certificate
@ -393,25 +372,7 @@ secure the channel from the client to the load balancer using TLS. You need to m
sure the TLS secret you created came from a certificate that contains a Common
Name (CN), also known as a Fully Qualified Domain Name (FQDN) for `sslexample.foo.com`.
```yaml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: tls-example-ingress
spec:
tls:
- hosts:
- sslexample.foo.com
secretName: testsecret-tls
rules:
- host: sslexample.foo.com
http:
paths:
- path: /
backend:
serviceName: service1
servicePort: 80
```
{{< codenew file="service/networking/tls-example-ingress.yaml" >}}
{{< note >}}
There is a gap between TLS features supported by various Ingress
@ -421,7 +382,7 @@ controllers. Please refer to documentation on
platform specific Ingress controller to understand how TLS works in your environment.
{{< /note >}}
### Loadbalancing
### Load balancing {#load-balancing}
An Ingress controller is bootstrapped with some load balancing policy settings
that it applies to all Ingress, such as the load balancing algorithm, backend
@ -432,10 +393,10 @@ a Service.
It's also worth noting that even though health checks are not exposed directly
through the Ingress, there exist parallel concepts in Kubernetes such as
[readiness probes](/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/)
[readiness probes](/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/)
that allow you to achieve the same end result. Please review the controller
specific documentation to see how they handle health checks (
[nginx](https://git.k8s.io/ingress-nginx/README.md),
specific documentation to see how they handle health checks (for example:
[nginx](https://git.k8s.io/ingress-nginx/README.md), or
[GCE](https://git.k8s.io/ingress-gce/README.md#health-checks)).
## Updating an Ingress
@ -478,16 +439,22 @@ spec:
http:
paths:
- backend:
serviceName: service1
servicePort: 80
service:
name: service1
port:
number: 80
path: /foo
pathType: Prefix
- host: bar.baz.com
http:
paths:
- backend:
serviceName: service2
servicePort: 80
service:
name: service2
port:
number: 80
path: /foo
pathType: Prefix
..
```
@ -525,16 +492,9 @@ You can achieve the same outcome by invoking `kubectl replace -f` on a modified
## Failing across availability zones
Techniques for spreading traffic across failure domains differs between cloud providers.
Please check the documentation of the relevant [Ingress controller](/docs/concepts/services-networking/ingress-controllers) for details. You can also refer to the [federation documentation](https://github.com/kubernetes-sigs/federation-v2)
Please check the documentation of the relevant [Ingress controller](/docs/concepts/services-networking/ingress-controllers) for details.
for details on deploying Ingress in a federated cluster.
## Future Work
Track [SIG Network](https://github.com/kubernetes/community/tree/master/sig-network)
for more details on the evolution of Ingress and related resources. You may also track the
[Ingress repository](https://github.com/kubernetes/ingress/tree/master) for more details on the
evolution of various Ingress controllers.
## Alternatives
You can expose a Service in multiple ways that don't directly involve the Ingress resource:
@ -547,6 +507,6 @@ You can expose a Service in multiple ways that don't directly involve the Ingres
## {{% heading "whatsnext" %}}
* Learn about the [Ingress API](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#ingress-v1beta1-networking-k8s-io)
* Learn about [Ingress Controllers](/docs/concepts/services-networking/ingress-controllers/)
* [Set up Ingress on Minikube with the NGINX Controller](/docs/tasks/access-application-cluster/ingress-minikube)
* Learn about [Ingress controllers](/docs/concepts/services-networking/ingress-controllers/)
* [Set up Ingress on Minikube with the NGINX Controller](/docs/tasks/access-application-cluster/ingress-minikube/)

34
content/en/docs/concepts/services-networking/network-policies.md
View File

@ -8,14 +8,19 @@ content_type: concept
weight: 50
---
{{< toc >}}
<!-- overview -->
A network policy is a specification of how groups of {{< glossary_tooltip text="pods" term_id="pod">}} are allowed to communicate with each other and other network endpoints.
NetworkPolicy resources use {{< glossary_tooltip text="labels" term_id="label">}} to select pods and define rules which specify what traffic is allowed to the selected pods.
If you want to control traffic flow at the IP address or port level (OSI layer 3 or 4), then you might consider using Kubernetes NetworkPolicies for particular applications in your cluster. NetworkPolicies are an application-centric construct which allow you to specify how a {{< glossary_tooltip text="pod" term_id="pod">}} is allowed to communicate with various network "entities" (we use the word "entity" here to avoid overloading the more common terms such as "endpoints" and "services", which have specific Kubernetes connotations) over the network.
The entities that a Pod can communicate with are identified through a combination of the following 3 identifiers:
1. Other pods that are allowed (exception: a pod cannot block access to itself)
2. Namespaces that are allowed
3. IP blocks (exception: traffic to and from the node where a Pod is running is always allowed, regardless of the IP address of the Pod or the node)
When defining a pod- or namespace- based NetworkPolicy, you use a {{< glossary_tooltip text="selector" term_id="selector">}} to specify what traffic is allowed to and from the Pod(s) that match the selector.
Meanwhile, when IP based NetworkPolicies are created, we define policies based on IP blocks (CIDR ranges).
<!-- body -->
## Prerequisites
@ -96,7 +101,7 @@ __egress__: Each NetworkPolicy may include a list of allowed `egress` rules. Ea
So, the example NetworkPolicy:
1. isolates "role=db" pods in the "default" namespace for both ingress and egress traffic (if they weren't already isolated)
2. (Ingress rules) allows connections to all pods in the “default” namespace with the label “role=db” on TCP port 6379 from:
2. (Ingress rules) allows connections to all pods in the "default" namespace with the label "role=db" on TCP port 6379 from:
* any pod in the "default" namespace with the label "role=frontend"
* any pod in a namespace with the label "project=myproject"
@ -205,17 +210,30 @@ This ensures that even pods that aren't selected by any other NetworkPolicy will
## SCTP support
{{< feature-state for_k8s_version="v1.12" state="alpha" >}}
{{< feature-state for_k8s_version="v1.19" state="beta" >}}
To use this feature, you (or your cluster administrator) will need to enable the `SCTPSupport` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/) for the API server with `--feature-gates=SCTPSupport=true,…`.
As a beta feature, this is enabled by default. To disable SCTP at a cluster level, you (or your cluster administrator) will need to disable the `SCTPSupport` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/) for the API server with `--feature-gates=SCTPSupport=false,…`.
When the feature gate is enabled, you can set the `protocol` field of a NetworkPolicy to `SCTP`.
{{< note >}}
You must be using a {{< glossary_tooltip text="CNI" term_id="cni" >}} plugin that supports SCTP protocol NetworkPolicies.
{{< /note >}}
# What you CAN'T do with network policy's (at least, not yet)
As of Kubernetes 1.20, the following functionality does not exist in the NetworkPolicy API, but you might be able to implement workarounds using Operating System components (such as SELinux, OpenVSwitch, IPTables, and so on) or Layer 7 technologies (Ingress controllers, Service Mesh implementations) or admission controllers. In case you are new to network security in Kubernetes, its worth noting that the following User Stories cannot (yet) be implemented using the NetworkPolicy API. Some (but not all) of these user stories are actively being discussed for future releases of the NetworkPolicy API.
- Forcing internal cluster traffic to go through a common gateway (this might be best served with a service mesh or other proxy).
- Anything TLS related (use a service mesh or ingress controller for this).
- Node specific policies (you can use CIDR notation for these, but you cannot target nodes by their Kubernetes identities specifically).
- Targeting of namespaces or services by name (you can, however, target pods or namespaces by their{{< glossary_tooltip text="labels" term_id="label" >}}, which is often a viable workaround).
- Creation or management of "Policy requests" that are fulfilled by a third party.
- Default policies which are applied to all namespaces or pods (there are some third party Kubernetes distributions and projects which can do this).
- Advanced policy querying and reachability tooling.
- The ability to target ranges of Ports in a single policy declaration.
- The ability to log network security events (for example connections that are blocked or accepted).
- The ability to explicitly deny policies (currently the model for NetworkPolicies are deny by default, with only the ability to add allow rules).
- The ability to prevent loopback or incoming host traffic (Pods cannot currently block localhost access, nor do they have the ability to block access from their resident node).
## {{% heading "whatsnext" %}}
@ -223,5 +241,3 @@ You must be using a {{< glossary_tooltip text="CNI" term_id="cni" >}} plugin tha
- See the [Declare Network Policy](/docs/tasks/administer-cluster/declare-network-policy/)
walkthrough for further examples.
- See more [recipes](https://github.com/ahmetb/kubernetes-network-policy-recipes) for common scenarios enabled by the NetworkPolicy resource.

64
content/en/docs/concepts/services-networking/service.md
View File

@ -20,8 +20,6 @@ With Kubernetes you don't need to modify your application to use an unfamiliar s
Kubernetes gives Pods their own IP addresses and a single DNS name for a set of Pods,
and can load-balance across them.
<!-- body -->
## Motivation
@ -35,8 +33,8 @@ Each Pod gets its own IP address, however in a Deployment, the set of Pods
running in one moment in time could be different from
the set of Pods running that application a moment later.
This leads to a problem: if some set of Pods (call them “backends”) provides
functionality to other Pods (call them “frontends”) inside your cluster,
This leads to a problem: if some set of Pods (call them "backends") provides
functionality to other Pods (call them "frontends") inside your cluster,
how do the frontends find out and keep track of which IP address to connect
to, so that the frontend can use the backend part of the workload?
@ -93,7 +91,7 @@ spec:
targetPort: 9376
```
This specification creates a new Service object named “my-service”, which
This specification creates a new Service object named "my-service", which
targets TCP port 9376 on any Pod with the `app=MyApp` label.
Kubernetes assigns this Service an IP address (sometimes called the "cluster IP"),
@ -102,7 +100,7 @@ which is used by the Service proxies
The controller for the Service selector continuously scans for Pods that
match its selector, and then POSTs any updates to an Endpoint object
also named “my-service”.
also named "my-service".
{{< note >}}
A Service can map _any_ incoming `port` to a `targetPort`. By default and
@ -204,14 +202,11 @@ described in detail in [EndpointSlices](/docs/concepts/services-networking/endpo
### Application protocol
{{< feature-state for_k8s_version="v1.18" state="alpha" >}}
{{< feature-state for_k8s_version="v1.19" state="beta" >}}
The AppProtocol field provides a way to specify an application protocol to be
used for each Service port.
As an alpha feature, this field is not enabled by default. To use this field,
enable the `ServiceAppProtocol` [feature
gate](/docs/reference/command-line-tools-reference/feature-gates/).
used for each Service port. The value of this field is mirrored by corresponding
Endpoints and EndpointSlice resources.
## Virtual IPs and service proxies
@ -321,7 +316,7 @@ falls back to running in iptables proxy mode.
![Services overview diagram for IPVS proxy](/images/docs/services-ipvs-overview.svg)
In these proxy models, the traffic bound for the Service’s IP:Port is
In these proxy models, the traffic bound for the Service's IP:Port is
proxied to an appropriate backend without the clients knowing anything
about Kubernetes or Services or Pods.
@ -390,7 +385,7 @@ variables and DNS.
When a Pod is run on a Node, the kubelet adds a set of environment variables
for each active Service. It supports both [Docker links
compatible](https://docs.docker.com/userguide/dockerlinks/) variables (see
[makeLinkVariables](http://releases.k8s.io/{{< param "githubbranch" >}}/pkg/kubelet/envvars/envvars.go#L49))
[makeLinkVariables](https://releases.k8s.io/{{< param "githubbranch" >}}/pkg/kubelet/envvars/envvars.go#L49))
and simpler `{SVCNAME}_SERVICE_HOST` and `{SVCNAME}_SERVICE_PORT` variables,
where the Service name is upper-cased and dashes are converted to underscores.
@ -449,7 +444,7 @@ You can find more information about `ExternalName` resolution in
## Headless Services
Sometimes you don't need load-balancing and a single Service IP. In
this case, you can create what are termed “headless” Services, by explicitly
this case, you can create what are termed "headless" Services, by explicitly
specifying `"None"` for the cluster IP (`.spec.clusterIP`).
You can use a headless Service to interface with other service discovery mechanisms,
@ -687,7 +682,7 @@ metadata:
```yaml
[...]
metadata:
annotations:
annotations:
service.kubernetes.io/qcloud-loadbalancer-internal-subnetid: subnet-xxxxx
[...]
```
@ -696,7 +691,7 @@ metadata:
```yaml
[...]
metadata:
annotations:
annotations:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type: "intranet"
[...]
```
@ -754,7 +749,7 @@ In the above example, if the Service contained three ports, `80`, `443`, and
`8443`, then `443` and `8443` would use the SSL certificate, but `80` would just
be proxied HTTP.
From Kubernetes v1.9 onwards you can use [predefined AWS SSL policies](http://docs.aws.amazon.com/elasticloadbalancing/latest/classic/elb-security-policy-table.html) with HTTPS or SSL listeners for your Services.
From Kubernetes v1.9 onwards you can use [predefined AWS SSL policies](https://docs.aws.amazon.com/elasticloadbalancing/latest/classic/elb-security-policy-table.html) with HTTPS or SSL listeners for your Services.
To see which policies are available for use, you can use the `aws` command line tool:
```bash
@ -866,6 +861,7 @@ There are other annotations to manage Classic Elastic Load Balancers that are de
service.beta.kubernetes.io/aws-load-balancer-healthcheck-interval: "20"
# The approximate interval, in seconds, between health checks of an
# individual instance. Defaults to 10, must be between 5 and 300
service.beta.kubernetes.io/aws-load-balancer-healthcheck-timeout: "5"
# The amount of time, in seconds, during which no response means a failed
# health check. This value must be less than the service.beta.kubernetes.io/aws-load-balancer-healthcheck-interval
@ -873,6 +869,10 @@ There are other annotations to manage Classic Elastic Load Balancers that are de
service.beta.kubernetes.io/aws-load-balancer-extra-security-groups: "sg-53fae93f,sg-42efd82e"
# A list of additional security groups to be added to the ELB
service.beta.kubernetes.io/aws-load-balancer-target-node-labels: "ingress-gw,gw-name=public-api"
# A comma separated list of key-value pairs which are used
# to select the target nodes for the load balancer
```
#### Network Load Balancer support on AWS {#aws-nlb-support}
@ -889,7 +889,7 @@ To use a Network Load Balancer on AWS, use the annotation `service.beta.kubernet
```
{{< note >}}
NLB only works with certain instance classes; see the [AWS documentation](http://docs.aws.amazon.com/elasticloadbalancing/latest/network/target-group-register-targets.html#register-deregister-targets)
NLB only works with certain instance classes; see the [AWS documentation](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/target-group-register-targets.html#register-deregister-targets)
on Elastic Load Balancing for a list of supported instance types.
{{< /note >}}
@ -950,25 +950,25 @@ There are other annotations for managing Cloud Load Balancers on TKE as shown be
# ID of an existing load balancer
service.kubernetes.io/tke-existed-lbid：lb-6swtxxxx
# Custom parameters for the load balancer (LB), does not support modification of LB type yet
service.kubernetes.io/service.extensiveParameters: ""
# Custom parameters for the LB listener
# Custom parameters for the LB listener
service.kubernetes.io/service.listenerParameters: ""
# Specifies the type of Load balancer;
# valid values: classic (Classic Cloud Load Balancer) or application (Application Cloud Load Balancer)
service.kubernetes.io/loadbalance-type: xxxxx
# Specifies the public network bandwidth billing method;
# Specifies the public network bandwidth billing method;
# valid values: TRAFFIC_POSTPAID_BY_HOUR(bill-by-traffic) and BANDWIDTH_POSTPAID_BY_HOUR (bill-by-bandwidth).
service.kubernetes.io/qcloud-loadbalancer-internet-charge-type: xxxxxx
# Specifies the bandwidth value (value range: [1,2000] Mbps).
service.kubernetes.io/qcloud-loadbalancer-internet-max-bandwidth-out: "10"
# When this annotation is set，the loadbalancers will only register nodes
# When this annotation is set，the loadbalancers will only register nodes
# with pod running on it, otherwise all nodes will be registered.
service.kubernetes.io/local-svc-only-bind-node-with-pod: true
```
@ -1046,9 +1046,9 @@ spec:
## Shortcomings
Using the userspace proxy for VIPs, work at small to medium scale, but will
not scale to very large clusters with thousands of Services. The [original
design proposal for portals](http://issue.k8s.io/1107) has more details on
this.
not scale to very large clusters with thousands of Services. The
[original design proposal for portals](https://github.com/kubernetes/kubernetes/issues/1107)
has more details on this.
Using the userspace proxy obscures the source IP address of a packet accessing
a Service.
@ -1117,7 +1117,7 @@ connections on it.
When a client connects to the Service's virtual IP address, the iptables
rule kicks in, and redirects the packets to the proxy's own port.
The “Service proxy” chooses a backend, and starts proxying traffic from the client to the backend.
The "Service proxy" chooses a backend, and starts proxying traffic from the client to the backend.
This means that Service owners can choose any port they want without risk of
collision. Clients can simply connect to an IP and port, without being aware
@ -1193,11 +1193,11 @@ followed by the data from the client.
### SCTP
{{< feature-state for_k8s_version="v1.12" state="alpha" >}}
{{< feature-state for_k8s_version="v1.19" state="beta" >}}
Kubernetes supports SCTP as a `protocol` value in Service, Endpoint, NetworkPolicy and Pod definitions as an alpha feature. To enable this feature, the cluster administrator needs to enable the `SCTPSupport` feature gate on the apiserver, for example, `--feature-gates=SCTPSupport=true,…`.
Kubernetes supports SCTP as a `protocol` value in Service, Endpoints, EndpointSlice, NetworkPolicy and Pod definitions. As a beta feature, this is enabled by default. To disable SCTP at a cluster level, you (or your cluster administrator) will need to disable the `SCTPSupport` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/) for the API server with `--feature-gates=SCTPSupport=false,…`.
When the feature gate is enabled, you can set the `protocol` field of a Service, Endpoint, NetworkPolicy or Pod to `SCTP`. Kubernetes sets up the network accordingly for the SCTP associations, just like it does for TCP connections.
When the feature gate is enabled, you can set the `protocol` field of a Service, Endpoints, EndpointSlice, NetworkPolicy or Pod to `SCTP`. Kubernetes sets up the network accordingly for the SCTP associations, just like it does for TCP connections.
#### Warnings {#caveat-sctp-overview}

6
content/en/docs/concepts/storage/dynamic-provisioning.md
View File

@ -33,7 +33,7 @@ from the API group `storage.k8s.io`. A cluster administrator can define as many
that provisioner when provisioning.
A cluster administrator can define and expose multiple flavors of storage (from
the same or different storage systems) within a cluster, each with a custom set
of parameters. This design also ensures that end users don’t have to worry
of parameters. This design also ensures that end users don't have to worry
about the complexity and nuances of how storage is provisioned, but still
have the ability to select from multiple storage options.
@ -85,8 +85,8 @@ is deprecated since v1.6. Users now can and should instead use the
this field must match the name of a `StorageClass` configured by the
administrator (see [below](#enabling-dynamic-provisioning)).
To select the “fast” storage class, for example, a user would create the
following `PersistentVolumeClaim`:
To select the "fast" storage class, for example, a user would create the
following PersistentVolumeClaim:
```yaml
apiVersion: v1

273
content/en/docs/concepts/storage/ephemeral-volumes.md Normal file
View File

@ -0,0 +1,273 @@
---
reviewers:
- jsafrane
- saad-ali
- msau42
- xing-yang
- pohly
title: Ephemeral Volumes
content_type: concept
weight: 50
---
<!-- overview -->
This document describes _ephemeral volumes_ in Kubernetes. Familiarity
with [volumes](/docs/concepts/storage/volumes/) is suggested, in
particular PersistentVolumeClaim and PersistentVolume.
<!-- body -->
Some application need additional storage but don't care whether that
data is stored persistently across restarts. For example, caching
services are often limited by memory size and can move infrequently
used data into storage that is slower than memory with little impact
on overall performance.
Other applications expect some read-only input data to be present in
files, like configuration data or secret keys.
_Ephemeral volumes_ are designed for these use cases. Because volumes
follow the Pod's lifetime and get created and deleted along with the
Pod, Pods can be stopped and restarted without being limited to where
some persistent volume is available.
Ephemeral volumes are specified _inline_ in the Pod spec, which
simplifies application deployment and management.
### Types of ephemeral volumes
Kubernetes supports several different kinds of ephemeral volumes for
different purposes:
- [emptyDir](/docs/concepts/volumes/#emptydir): empty at Pod startup,
with storage coming locally from the kubelet base directory (usually
the root disk) or RAM
- [configMap](/docs/concepts/volumes/#configmap),
[downwardAPI](/docs/concepts/volumes/#downwardapi),
[secret](/docs/concepts/storage/volumes/#secret): inject different
kinds of Kubernetes data into a Pod
- [CSI ephemeral
volumes](docs/concepts/storage/volumes/#csi-ephemeral-volumes):
similar to the previous volume kinds, but provided by special [CSI
drivers](https://github.com/container-storage-interface/spec/blob/master/spec.md)
which specifically [support this feature](https://kubernetes-csi.github.io/docs/drivers.html)
- [generic ephemeral volumes](#generic-ephemeral-volumes), which
can be provided by all storage drivers that also support persistent volumes
`emptyDir`, `configMap`, `downwardAPI`, `secret` are provided as
[local ephemeral
storage](/docs/concepts/configuration/manage-resources-containers/#local-ephemeral-storage).
They are managed by kubelet on each node.
CSI ephemeral volumes *must* be provided by third-party CSI storage
drivers.
Generic ephemeral volumes *can* be provided by third-party CSI storage
drivers, but also by any other storage driver that supports dynamic
provisioning. Some CSI drivers are written specifically for CSI
ephemeral volumes and do not support dynamic provisioning: those then
cannot be used for generic ephemeral volumes.
The advantage of using third-party drivers is that they can offer
functionality that Kubernetes itself does not support, for example
storage with different performance characteristics than the disk that
is managed by kubelet, or injecting different data.
### CSI ephemeral volumes
{{< feature-state for_k8s_version="v1.16" state="beta" >}}
This feature requires the `CSIInlineVolume` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/) to be enabled. It
is enabled by default starting with Kubernetes 1.16.
{{< note >}}
CSI ephemeral volumes are only supported by a subset of CSI drivers.
The Kubernetes CSI [Drivers list](https://kubernetes-csi.github.io/docs/drivers.html)
shows which drivers support ephemeral volumes.
{{< /note >}}
Conceptually, CSI ephemeral volumes are similar to `configMap`,
`downwardAPI` and `secret` volume types: the storage is managed locally on each
node and is created together with other local resources after a Pod has been
scheduled onto a node. Kubernetes has no concept of rescheduling Pods
anymore at this stage. Volume creation has to be unlikely to fail,
otherwise Pod startup gets stuck. In particular, [storage capacity
aware Pod scheduling](/docs/concepts/storage-capacity/) is *not*
supported for these volumes. They are currently also not covered by
the storage resource usage limits of a Pod, because that is something
that kubelet can only enforce for storage that it manages itself.
Here's an example manifest for a Pod that uses CSI ephemeral storage:
```yaml
kind: Pod
apiVersion: v1
metadata:
name: my-csi-app
spec:
containers:
- name: my-frontend
image: busybox
volumeMounts:
- mountPath: "/data"
name: my-csi-inline-vol
command: [ "sleep", "1000000" ]
volumes:
- name: my-csi-inline-vol
csi:
driver: inline.storage.kubernetes.io
volumeAttributes:
foo: bar
```
The `volumeAttributes` determine what volume is prepared by the
driver. These attributes are specific to each driver and not
standardized. See the documentation of each CSI driver for further
instructions.
As a cluster administrator, you can use a [PodSecurityPolicy](/docs/concepts/policy/pod-security-policy/) to control which CSI drivers can be used in a Pod, specified with the
[`allowedCSIDrivers` field](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#podsecuritypolicyspec-v1beta1-policy).
### Generic ephemeral volumes
{{< feature-state for_k8s_version="v1.19" state="alpha" >}}
This feature requires the `GenericEphemeralVolume` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/) to be
enabled. Because this is an alpha feature, it is disabled by default.
Generic ephemeral volumes are similar to `emptyDir` volumes, just more
flexible:
- Storage can be local or network-attached.
- Volumes can have a fixed size that Pods are not able to exceed.
- Volumes may have some initial data, depending on the driver and
parameters.
- Typical operations on volumes are supported assuming that the driver
supports them, including
([snapshotting](/docs/concepts/storage/volume-snapshots/),
[cloning](/docs/concepts/storage/volume-pvc-datasource/),
[resizing](/docs/concepts/storage/persistent-volumes/#expanding-persistent-volumes-claims),
and [storage capacity tracking](/docs/concepts/storage-capacity/).
Example:
```yaml
kind: Pod
apiVersion: v1
metadata:
name: my-app
spec:
containers:
- name: my-frontend
image: busybox
volumeMounts:
- mountPath: "/scratch"
name: scratch-volume
command: [ "sleep", "1000000" ]
volumes:
- name: scratch-volume
ephemeral:
volumeClaimTemplate:
metadata:
labels:
type: my-frontend-volume
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: "scratch-storage-class"
resources:
requests:
storage: 1Gi
```
### Lifecycle and PersistentVolumeClaim
The key design idea is that the [parameters for a
volume claim](/docs/reference/generated/kubernetes-api/#ephemeralvolumesource-v1alpha1-core)
are allowed inside a volume source of the Pod. Labels, annotations and
the whole set of fields for a PersistentVolumeClaim are supported. When such a Pod gets
created, the ephemeral volume controller then creates an actual PersistentVolumeClaim
object in the same namespace as the Pod and ensures that the PersistentVolumeClaim
gets deleted when the Pod gets deleted.
That triggers volume binding and/or provisioning, either immediately if
the {{< glossary_tooltip text="StorageClass" term_id="storage-class" >}} uses immediate volume binding or when the Pod is
tentatively scheduled onto a node (`WaitForFirstConsumer` volume
binding mode). The latter is recommended for generic ephemeral volumes
because then the scheduler is free to choose a suitable node for
the Pod. With immediate binding, the scheduler is forced to select a node that has
access to the volume once it is available.
In terms of [resource ownership](/docs/concepts/workloads/controllers/garbage-collection/#owners-and-dependents),
a Pod that has generic ephemeral storage is the owner of the PersistentVolumeClaim(s)
that provide that ephemeral storage. When the Pod is deleted,
the Kubernetes garbage collector deletes the PVC, which then usually
triggers deletion of the volume because the default reclaim policy of
storage classes is to delete volumes. You can create quasi-ephemeral local storage
using a StorageClass with a reclaim policy of `retain`: the storage outlives the Pod,
and in this case you need to ensure that volume clean up happens separately.
While these PVCs exist, they can be used like any other PVC. In
particular, they can be referenced as data source in volume cloning or
snapshotting. The PVC object also holds the current status of the
volume.
### PersistentVolumeClaim naming
Naming of the automatically created PVCs is deterministic: the name is
a combination of Pod name and volume name, with a hyphen (`-`) in the
middle. In the example above, the PVC name will be
`my-app-scratch-volume`. This deterministic naming makes it easier to
interact with the PVC because one does not have to search for it once
the Pod name and volume name are known.
The deterministic naming also introduces a potential conflict between different
Pods (a Pod "pod-a" with volume "scratch" and another Pod with name
"pod" and volume "a-scratch" both end up with the same PVC name
"pod-a-scratch") and between Pods and manually created PVCs.
Such conflicts are detected: a PVC is only used for an ephemeral
volume if it was created for the Pod. This check is based on the
ownership relationship. An existing PVC is not overwritten or
modified. But this does not resolve the conflict because without the
right PVC, the Pod cannot start.
{{< caution >}}
Take care when naming Pods and volumes inside the
same namespace, so that these conflicts can't occur.
{{< /caution >}}
### Security
Enabling the GenericEphemeralVolume feature allows users to create
PVCs indirectly if they can create Pods, even if they do not have
permission to create PVCs directly. Cluster administrators must be
aware of this. If this does not fit their security model, they have
two choices:
- Explicitly disable the feature through the feature gate, to avoid
being surprised when some future Kubernetes version enables it
by default.
- Use a [Pod Security
Policy](/docs/concepts/policy/pod-security-policy/) where the
`volumes` list does not contain the `ephemeral` volume type.
The normal namespace quota for PVCs in a namespace still applies, so
even if users are allowed to use this new mechanism, they cannot use
it to circumvent other policies.
## {{% heading "whatsnext" %}}
### Ephemeral volumes managed by kubelet
See [local ephemeral storage](/docs/concepts/configuration/manage-resources-containers/#local-ephemeral-storage).
### CSI ephemeral volumes
- For more information on the design, see the [Ephemeral Inline CSI
volumes KEP](https://github.com/kubernetes/enhancements/blob/ad6021b3d61a49040a3f835e12c8bb5424db2bbb/keps/sig-storage/20190122-csi-inline-volumes.md).
- For more information on further development of this feature, see the [enhancement tracking issue #596](https://github.com/kubernetes/enhancements/issues/596).
### Generic ephemeral volumes
- For more information on the design, see the
[Generic ephemeral inline volumes KEP](https://github.com/kubernetes/enhancements/blob/master/keps/sig-storage/1698-generic-ephemeral-volumes/README.md).
- For more information on further development of this feature, see the [enhancement tracking issue #1698](https://github.com/kubernetes/enhancements/issues/1698).

19
content/en/docs/concepts/storage/persistent-volumes.md
View File

@ -19,9 +19,6 @@ weight: 20
This document describes the current state of _persistent volumes_ in Kubernetes. Familiarity with [volumes](/docs/concepts/storage/volumes/) is suggested.
<!-- body -->
## Introduction
@ -148,7 +145,11 @@ The `Recycle` reclaim policy is deprecated. Instead, the recommended approach is
If supported by the underlying volume plugin, the `Recycle` reclaim policy performs a basic scrub (`rm -rf /thevolume/*`) on the volume and makes it available again for a new claim.
However, an administrator can configure a custom recycler Pod template using the Kubernetes controller manager command line arguments as described [here](/docs/admin/kube-controller-manager/). The custom recycler Pod template must contain a `volumes` specification, as shown in the example below:
However, an administrator can configure a custom recycler Pod template using
the Kubernetes controller manager command line arguments as described in the
[reference](/docs/reference/command-line-tools-reference/kube-controller-manager/).
The custom recycler Pod template must contain a `volumes` specification, as
shown in the example below:
```yaml
apiVersion: v1
@ -253,6 +254,16 @@ FlexVolume resize is possible only when the underlying driver supports resize.
Expanding EBS volumes is a time-consuming operation. Also, there is a per-volume quota of one modification every 6 hours.
{{< /note >}}
#### Recovering from Failure when Expanding Volumes
If expanding underlying storage fails, the cluster administrator can manually recover the Persistent Volume Claim (PVC) state and cancel the resize requests. Otherwise, the resize requests are continuously retried by the controller without administrator intervention.
1. Mark the PersistentVolume(PV) that is bound to the PersistentVolumeClaim(PVC) with `Retain` reclaim policy.
2. Delete the PVC. Since PV has `Retain` reclaim policy - we will not lose any data when we recreate the PVC.
3. Delete the `claimRef` entry from PV specs, so as new PVC can bind to it. This should make the PV `Available`.
4. Re-create the PVC with smaller size than PV and set `volumeName` field of the PVC to the name of the PV. This should bind new PVC to existing PV.
5. Don't forget to restore the reclaim policy of the PV.
## Types of Persistent Volumes

140
content/en/docs/concepts/storage/storage-capacity.md Normal file
View File

@ -0,0 +1,140 @@
---
reviewers:
- jsafrane
- saad-ali
- msau42
- xing-yang
- pohly
title: Storage Capacity
content_type: concept
weight: 45
---
<!-- overview -->
Storage capacity is limited and may vary depending on the node on
which a pod runs: network-attached storage might not be accessible by
all nodes, or storage is local to a node to begin with.
{{< feature-state for_k8s_version="v1.19" state="alpha" >}}
This page describes how Kubernetes keeps track of storage capacity and
how the scheduler uses that information to schedule Pods onto nodes
that have access to enough storage capacity for the remaining missing
volumes. Without storage capacity tracking, the scheduler may choose a
node that doesn't have enough capacity to provision a volume and
multiple scheduling retries will be needed.
Tracking storage capacity is supported for {{< glossary_tooltip
text="Container Storage Interface" term_id="csi" >}} (CSI) drivers and
[needs to be enabled](#enabling-storage-capacity-tracking) when installing a CSI driver.
<!-- body -->
## API
There are two API extensions for this feature:
- [CSIStorageCapacity](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#csistoragecapacity-v1alpha1-storage-k8s-io) objects:
these get produced by a CSI driver in the namespace
where the driver is installed. Each object contains capacity
information for one storage class and defines which nodes have
access to that storage.
- [The `CSIDriverSpec.StorageCapacity` field](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#csidriverspec-v1-storage-k8s-io):
when set to `true`, the Kubernetes scheduler will consider storage
capacity for volumes that use the CSI driver.
## Scheduling
Storage capacity information is used by the Kubernetes scheduler if:
- the `CSIStorageCapacity` feature gate is true,
- a Pod uses a volume that has not been created yet,
- that volume uses a {{< glossary_tooltip text="StorageClass" term_id="storage-class" >}} which references a CSI driver and
uses `WaitForFirstConsumer` [volume binding
mode](/docs/concepts/storage/storage-classes/#volume-binding-mode),
and
- the `CSIDriver` object for the driver has `StorageCapacity` set to
true.
In that case, the scheduler only considers nodes for the Pod which
have enough storage available to them. This check is very
simplistic and only compares the size of the volume against the
capacity listed in `CSIStorageCapacity` objects with a topology that
includes the node.
For volumes with `Immediate` volume binding mode, the storage driver
decides where to create the volume, independently of Pods that will
use the volume. The scheduler then schedules Pods onto nodes where the
volume is available after the volume has been created.
For [CSI ephemeral volumes](/docs/concepts/storage/volumes/#csi),
scheduling always happens without considering storage capacity. This
is based on the assumption that this volume type is only used by
special CSI drivers which are local to a node and do not need
significant resources there.
## Rescheduling
When a node has been selected for a Pod with `WaitForFirstConsumer`
volumes, that decision is still tentative. The next step is that the
CSI storage driver gets asked to create the volume with a hint that the
volume is supposed to be available on the selected node.
Because Kubernetes might have chosen a node based on out-dated
capacity information, it is possible that the volume cannot really be
created. The node selection is then reset and the Kubernetes scheduler
tries again to find a node for the Pod.
## Limitations
Storage capacity tracking increases the chance that scheduling works
on the first try, but cannot guarantee this because the scheduler has
to decide based on potentially out-dated information. Usually, the
same retry mechanism as for scheduling without any storage capacity
information handles scheduling failures.
One situation where scheduling can fail permanently is when a Pod uses
multiple volumes: one volume might have been created already in a
topology segment which then does not have enough capacity left for
another volume. Manual intervention is necessary to recover from this,
for example by increasing capacity or deleting the volume that was
already created. [Further
work](https://github.com/kubernetes/enhancements/pull/1703) is needed
to handle this automatically.
## Enabling storage capacity tracking
Storage capacity tracking is an *alpha feature* and only enabled when
the `CSIStorageCapacity` [feature
gate](/docs/reference/command-line-tools-reference/feature-gates/) and
the `storage.k8s.io/v1alpha1` {{< glossary_tooltip text="API group" term_id="api-group" >}} are enabled. For details on
that, see the `--feature-gates` and `--runtime-config` [kube-apiserver
parameters](/docs/reference/command-line-tools-reference/kube-apiserver/).
A quick check
whether a Kubernetes cluster supports the feature is to list
CSIStorageCapacity objects with:
```shell
kubectl get csistoragecapacities --all-namespaces
```
If your cluster supports CSIStorageCapacity, the response is either a list of CSIStorageCapacity objects or:
```
No resources found
```
If not supported, this error is printed instead:
```
error: the server doesn't have a resource type "csistoragecapacities"
```
In addition to enabling the feature in the cluster, a CSI
driver also has to
support it. Please refer to the driver's documentation for
details.
## {{% heading "whatsnext" %}}
- For more information on the design, see the
[Storage Capacity Constraints for Pod Scheduling KEP](https://github.com/kubernetes/enhancements/blob/master/keps/sig-storage/1472-storage-capacity-tracking/README.md).
- For more information on further development of this feature, see the [enhancement tracking issue #1472](https://github.com/kubernetes/enhancements/issues/1472).
- Learn about [Kubernetes Scheduler](/docs/concepts/scheduling-eviction/kube-scheduler/)

10
content/en/docs/concepts/storage/storage-classes.md
View File

@ -15,8 +15,6 @@ This document describes the concept of a StorageClass in Kubernetes. Familiarity
with [volumes](/docs/concepts/storage/volumes/) and
[persistent volumes](/docs/concepts/storage/persistent-volumes) is suggested.
<!-- body -->
## Introduction
@ -168,11 +166,11 @@ A cluster administrator can address this issue by specifying the `WaitForFirstCo
will delay the binding and provisioning of a PersistentVolume until a Pod using the PersistentVolumeClaim is created.
PersistentVolumes will be selected or provisioned conforming to the topology that is
specified by the Pod's scheduling constraints. These include, but are not limited to, [resource
requirements](/docs/concepts/configuration/manage-compute-resources-container),
requirements](/docs/concepts/configuration/manage-resources-containers/),
[node selectors](/docs/concepts/scheduling-eviction/assign-pod-node/#nodeselector),
[pod affinity and
anti-affinity](/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity),
and [taints and tolerations](/docs/concepts/configuration/taint-and-toleration).
and [taints and tolerations](/docs/concepts/scheduling-eviction/taint-and-toleration).
The following plugins support `WaitForFirstConsumer` with dynamic provisioning:
@ -244,7 +242,7 @@ parameters:
```
* `type`: `io1`, `gp2`, `sc1`, `st1`. See
[AWS docs](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSVolumeTypes.html)
[AWS docs](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSVolumeTypes.html)
for details. Default: `gp2`.
* `zone` (Deprecated): AWS zone. If neither `zone` nor `zones` is specified, volumes are
generally round-robin-ed across all active zones where Kubernetes cluster
@ -256,7 +254,7 @@ parameters:
* `iopsPerGB`: only for `io1` volumes. I/O operations per second per GiB. AWS
volume plugin multiplies this with size of requested volume to compute IOPS
of the volume and caps it at 20 000 IOPS (maximum supported by AWS, see
[AWS docs](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSVolumeTypes.html).
[AWS docs](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSVolumeTypes.html).
A string is expected here, i.e. `"10"`, not `10`.
* `fsType`: fsType that is supported by kubernetes. Default: `"ext4"`.
* `encrypted`: denotes whether the EBS volume should be encrypted or not.

4
content/en/docs/concepts/storage/volume-snapshots.md
View File

@ -94,14 +94,14 @@ using the attribute `volumeSnapshotClassName`. If nothing is set, then the defau
For pre-provisioned snapshots, you need to specify a `volumeSnapshotContentName` as the source for the snapshot as shown in the following example. The `volumeSnapshotContentName` source field is required for pre-provisioned snapshots.
```
```yaml
apiVersion: snapshot.storage.k8s.io/v1beta1
kind: VolumeSnapshot
metadata:
name: test-snapshot
spec:
source:
volumeSnapshotContentName: test-content
volumeSnapshotContentName: test-content
```
## Volume Snapshot Contents

104
content/en/docs/concepts/storage/volumes.md
View File

@ -18,10 +18,7 @@ Container starts with a clean state. Second, when running Containers together
in a `Pod` it is often necessary to share files between those Containers. The
Kubernetes `Volume` abstraction solves both of these problems.
Familiarity with [Pods](/docs/user-guide/pods) is suggested.
Familiarity with [Pods](/docs/concepts/workloads/pods/pod/) is suggested.
<!-- body -->
@ -100,7 +97,7 @@ We welcome additional contributions.
### awsElasticBlockStore {#awselasticblockstore}
An `awsElasticBlockStore` volume mounts an Amazon Web Services (AWS) [EBS
Volume](http://aws.amazon.com/ebs/) into your Pod. Unlike
Volume](https://aws.amazon.com/ebs/) into your Pod. Unlike
`emptyDir`, which is erased when a Pod is removed, the contents of an EBS
volume are preserved and the volume is merely unmounted. This means that an
EBS volume can be pre-populated with data, and that data can be "handed off"
@ -173,14 +170,14 @@ More details can be found [here](https://github.com/kubernetes/examples/tree/{{<
#### CSI Migration
{{< feature-state for_k8s_version="v1.15" state="alpha" >}}
{{< feature-state for_k8s_version="v1.19" state="beta" >}}
The CSI Migration feature for azureDisk, when enabled, shims all plugin operations
from the existing in-tree plugin to the `disk.csi.azure.com` Container
Storage Interface (CSI) Driver. In order to use this feature, the [Azure Disk CSI
Driver](https://github.com/kubernetes-sigs/azuredisk-csi-driver)
must be installed on the cluster and the `CSIMigration` and `CSIMigrationAzureDisk`
Alpha features must be enabled.
features must be enabled.
### azureFile {#azurefile}
@ -401,8 +398,8 @@ See the [Flocker example](https://github.com/kubernetes/examples/tree/{{< param
### gcePersistentDisk {#gcepersistentdisk}
A `gcePersistentDisk` volume mounts a Google Compute Engine (GCE) [Persistent
Disk](http://cloud.google.com/compute/docs/disks) into your Pod. Unlike
A `gcePersistentDisk` volume mounts a Google Compute Engine (GCE)
[Persistent Disk](https://cloud.google.com/compute/docs/disks) into your Pod. Unlike
`emptyDir`, which is erased when a Pod is removed, the contents of a PD are
preserved and the volume is merely unmounted. This means that a PD can be
pre-populated with data, and that data can be "handed off" between Pods.
@ -537,7 +534,7 @@ spec:
### glusterfs {#glusterfs}
A `glusterfs` volume allows a [Glusterfs](http://www.gluster.org) (an open
A `glusterfs` volume allows a [Glusterfs](https://www.gluster.org) (an open
source networked filesystem) volume to be mounted into your Pod. Unlike
`emptyDir`, which is erased when a Pod is removed, the contents of a
`glusterfs` volume are preserved and the volume is merely unmounted. This
@ -589,7 +586,7 @@ Watch out when using this type of volume, because:
able to account for resources used by a `hostPath`
* the files or directories created on the underlying hosts are only writable by root. You
either need to run your process as root in a
[privileged Container](/docs/user-guide/security-context) or modify the file
[privileged Container](/docs/tasks/configure-pod-container/security-context/) or modify the file
permissions on the host to be able to write to a `hostPath` volume
#### Example Pod
@ -952,7 +949,7 @@ More details and examples can be found [here](https://github.com/kubernetes/exam
### quobyte {#quobyte}
A `quobyte` volume allows an existing [Quobyte](http://www.quobyte.com) volume to
A `quobyte` volume allows an existing [Quobyte](https://www.quobyte.com) volume to
be mounted into your Pod.
{{< caution >}}
@ -966,8 +963,8 @@ GitHub project has [instructions](https://github.com/quobyte/quobyte-csi#quobyte
### rbd {#rbd}
An `rbd` volume allows a [Rados Block
Device](http://ceph.com/docs/master/rbd/rbd/) volume to be mounted into your
An `rbd` volume allows a
[Rados Block Device](https://ceph.com/docs/master/rbd/rbd/) volume to be mounted into your
Pod. Unlike `emptyDir`, which is erased when a Pod is removed, the contents of
a `rbd` volume are preserved and the volume is merely unmounted. This
means that a RBD volume can be pre-populated with data, and that data can
@ -1044,7 +1041,7 @@ A Container using a Secret as a [subPath](#using-subpath) volume mount will not
receive Secret updates.
{{< /note >}}
Secrets are described in more detail [here](/docs/user-guide/secrets).
Secrets are described in more detail [here](/docs/concepts/configuration/secret/).
### storageOS {#storageos}
@ -1160,6 +1157,37 @@ spec:
More examples can be found [here](https://github.com/kubernetes/examples/tree/master/staging/volumes/vsphere).
#### CSI migration
{{< feature-state for_k8s_version="v1.19" state="beta" >}}
The CSI Migration feature for vsphereVolume, when enabled, shims all plugin operations
from the existing in-tree plugin to the `csi.vsphere.vmware.com` {{< glossary_tooltip text="CSI" term_id="csi" >}} driver. In order to use this feature, the [vSphere CSI
Driver](https://github.com/kubernetes-sigs/vsphere-csi-driver)
must be installed on the cluster and the `CSIMigration` and `CSIMigrationvSphere`
[feature gates](/docs/reference/command-line-tools-reference/feature-gates/) must be enabled.
This also requires minimum vSphere vCenter/ESXi Version to be 7.0u1 and minimum HW Version to be VM version 15.
{{< note >}}
The following StorageClass parameters from the built-in vsphereVolume plugin are not supported by the vSphere CSI driver:
* `diskformat`
* `hostfailurestotolerate`
* `forceprovisioning`
* `cachereservation`
* `diskstripes`
* `objectspacereservation`
* `iopslimit`
Existing volumes created using these parameters will be migrated to the vSphere CSI driver, but new volumes created by the vSphere CSI driver will not be honoring these parameters.
{{< /note >}}
#### CSI Migration Complete
{{< feature-state for_k8s_version="v1.19" state="beta" >}}
To turn off the vsphereVolume plugin from being loaded by controller manager and kubelet, you need to set this feature flag to true. This requires `csi.vsphere.vmware.com` {{< glossary_tooltip text="CSI" term_id="csi" >}} driver being installed on all worker nodes.
## Using subPath
@ -1203,7 +1231,6 @@ spec:
Use the `subPathExpr` field to construct `subPath` directory names from Downward API environment variables.
This feature requires the `VolumeSubpathEnvExpansion` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/) to be enabled. It is enabled by default starting with Kubernetes 1.15.
The `subPath` and `subPathExpr` properties are mutually exclusive.
In this example, a Pod uses `subPathExpr` to create a directory `pod1` within the hostPath volume `/var/log/pods`, using the pod name from the Downward API. The host directory `/var/log/pods/pod1` is mounted at `/logs` in the container.
@ -1244,11 +1271,12 @@ medium of the filesystem holding the kubelet root dir (typically
Pods.
In the future, we expect that `emptyDir` and `hostPath` volumes will be able to
request a certain amount of space using a [resource](/docs/user-guide/compute-resources)
request a certain amount of space using a [resource](/docs/concepts/configuration/manage-resources-containers/)
specification, and to select the type of media to use, for clusters that have
several media types.
## Out-of-Tree Volume Plugins
The Out-of-tree volume plugins include the Container Storage Interface (CSI)
and FlexVolume. They enable storage vendors to create custom storage plugins
without adding them to the Kubernetes repository.
@ -1292,8 +1320,11 @@ Once a CSI compatible volume driver is deployed on a Kubernetes cluster, users
may use the `csi` volume type to attach, mount, etc. the volumes exposed by the
CSI driver.
The `csi` volume type does not support direct reference from Pod and may only be
referenced in a Pod via a `PersistentVolumeClaim` object.
A `csi` volume can be used in a pod in three different ways:
- through a reference to a [`persistentVolumeClaim`](#persistentvolumeclaim)
- with a [generic ephemeral volume](/docs/concepts/storage/ephemeral-volumes/#generic-ephemeral-volume) (alpha feature)
- with a [CSI ephemeral volume](/docs/concepts/storage/ephemeral-volumes/#csi-ephemeral-volume) if the driver
supports that (beta feature)
The following fields are available to storage administrators to configure a CSI
persistent volume:
@ -1356,37 +1387,14 @@ as usual, without any CSI specific changes.
{{< feature-state for_k8s_version="v1.16" state="beta" >}}
This feature allows CSI volumes to be directly embedded in the Pod specification instead of a PersistentVolume. Volumes specified in this way are ephemeral and do not persist across Pod restarts.
You can directly configure CSI volumes within the Pod
specification. Volumes specified in this way are ephemeral and do not
persist across Pod restarts. See [Ephemeral
Volumes](/docs/concepts/storage/ephemeral-volumes/#csi-ephemeral-volume)
for more information.
Example:
#### {{% heading "whatsnext" %}}
```yaml
kind: Pod
apiVersion: v1
metadata:
name: my-csi-app
spec:
containers:
- name: my-frontend
image: busybox
volumeMounts:
- mountPath: "/data"
name: my-csi-inline-vol
command: [ "sleep", "1000000" ]
volumes:
- name: my-csi-inline-vol
csi:
driver: inline.storage.kubernetes.io
volumeAttributes:
foo: bar
```
This feature requires CSIInlineVolume feature gate to be enabled. It
is enabled by default starting with Kubernetes 1.16.
CSI ephemeral volumes are only supported by a subset of CSI drivers. Please see the list of CSI drivers [here](https://kubernetes-csi.github.io/docs/drivers.html).
# Developer resources
For more information on how to develop a CSI driver, refer to the [kubernetes-csi
documentation](https://kubernetes-csi.github.io/docs/)

14
content/en/docs/concepts/workloads/controllers/daemonset.md
View File

@ -7,7 +7,7 @@ reviewers:
- kow3ns
title: DaemonSet
content_type: concept
weight: 50
weight: 40
---
<!-- overview -->
@ -26,9 +26,6 @@ In a simple case, one DaemonSet, covering all nodes, would be used for each type
A more complex setup might use multiple DaemonSets for a single type of daemon, but with
different flags and/or different memory and cpu requests for different hardware types.
<!-- body -->
## Writing a DaemonSet Spec
@ -48,7 +45,8 @@ kubectl apply -f https://k8s.io/examples/controllers/daemonset.yaml
### Required Fields
As with all other Kubernetes config, a DaemonSet needs `apiVersion`, `kind`, and `metadata` fields. For
general information about working with config files, see [deploying applications](/docs/user-guide/deploying-applications/),
general information about working with config files, see
[running stateless applications](/docs/tasks/run-application/run-stateless-application-deployment/),
[configuring containers](/docs/tasks/), and [object management using kubectl](/docs/concepts/overview/working-with-objects/object-management/) documents.
The name of a DaemonSet object must be a valid
@ -71,7 +69,7 @@ A Pod Template in a DaemonSet must have a [`RestartPolicy`](/docs/concepts/workl
### Pod Selector
The `.spec.selector` field is a pod selector. It works the same as the `.spec.selector` of
a [Job](/docs/concepts/jobs/run-to-completion-finite-workloads/).
a [Job](/docs/concepts/workloads/controllers/job/).
As of Kubernetes 1.8, you must specify a pod selector that matches the labels of the
`.spec.template`. The pod selector will no longer be defaulted when left empty. Selector
@ -147,7 +145,7 @@ automatically to DaemonSet Pods. The default scheduler ignores
### Taints and Tolerations
Although Daemon Pods respect
[taints and tolerations](/docs/concepts/configuration/taint-and-toleration),
[taints and tolerations](/docs/concepts/scheduling-eviction/taint-and-toleration/),
the following tolerations are added to DaemonSet Pods automatically according to
the related features.
@ -213,7 +211,7 @@ use a DaemonSet rather than creating individual Pods.
### Static Pods
It is possible to create Pods by writing a file to a certain directory watched by Kubelet. These
are called [static pods](/docs/concepts/cluster-administration/static-pod/).
are called [static pods](/docs/tasks/configure-pod-container/static-pod/).
Unlike DaemonSet, static Pods cannot be managed with kubectl
or other Kubernetes API clients. Static Pods do not depend on the apiserver, making them useful
in cluster bootstrapping cases. Also, static Pods may be deprecated in the future.

18
content/en/docs/concepts/workloads/controllers/deployment.md
View File

@ -8,7 +8,7 @@ feature:
Kubernetes progressively rolls out changes to your application or its configuration, while monitoring application health to ensure it doesn't kill all your instances at the same time. If something goes wrong, Kubernetes will rollback the change for you. Take advantage of a growing ecosystem of deployment solutions.
content_type: concept
weight: 30
weight: 10
---
<!-- overview -->
@ -22,7 +22,6 @@ You describe a _desired state_ in a Deployment, and the Deployment {{< glossary_
Do not manage ReplicaSets owned by a Deployment. Consider opening an issue in the main Kubernetes repository if your use case is not covered below.
{{< /note >}}
<!-- body -->
## Use Case
@ -85,7 +84,7 @@ Follow the steps given below to create the above Deployment:
2. Run `kubectl get deployments` to check if the Deployment was created.
If the Deployment is still being created, the output is similar to the following:
```shell
```
NAME READY UP-TO-DATE AVAILABLE AGE
nginx-deployment 0/3 0 0 1s
```
@ -101,21 +100,21 @@ Follow the steps given below to create the above Deployment:
3. To see the Deployment rollout status, run `kubectl rollout status deployment.v1.apps/nginx-deployment`.
The output is similar to:
```shell
```
Waiting for rollout to finish: 2 out of 3 new replicas have been updated...
deployment.apps/nginx-deployment successfully rolled out
```
4. Run the `kubectl get deployments` again a few seconds later.
The output is similar to this:
```shell
```
NAME READY UP-TO-DATE AVAILABLE AGE
nginx-deployment 3/3 3 3 18s
```
Notice that the Deployment has created all three replicas, and all replicas are up-to-date (they contain the latest Pod template) and available.
5. To see the ReplicaSet (`rs`) created by the Deployment, run `kubectl get rs`. The output is similar to this:
```shell
```
NAME DESIRED CURRENT READY AGE
nginx-deployment-75675f5897 3 3 3 18s
```
@ -132,7 +131,7 @@ Follow the steps given below to create the above Deployment:
6. To see the labels automatically generated for each Pod, run `kubectl get pods --show-labels`.
The output is similar to:
```shell
```
NAME READY STATUS RESTARTS AGE LABELS
nginx-deployment-75675f5897-7ci7o 1/1 Running 0 18s app=nginx,pod-template-hash=3123191453
nginx-deployment-75675f5897-kzszj 1/1 Running 0 18s app=nginx,pod-template-hash=3123191453
@ -1040,7 +1039,8 @@ can create multiple Deployments, one for each release, following the canary patt
## Writing a Deployment Spec
As with all other Kubernetes configs, a Deployment needs `.apiVersion`, `.kind`, and `.metadata` fields.
For general information about working with config files, see [deploying applications](/docs/tutorials/stateless-application/run-stateless-application-deployment/),
For general information about working with config files, see
[deploying applications](/docs/tasks/run-application/run-stateless-application-deployment/),
configuring containers, and [using kubectl to manage resources](/docs/concepts/overview/working-with-objects/object-management/) documents.
The name of a Deployment object must be a valid
[DNS subdomain name](/docs/concepts/overview/working-with-objects/names#dns-subdomain-names).
@ -1054,7 +1054,7 @@ The `.spec.template` and `.spec.selector` are the only required field of the `.s
The `.spec.template` is a [Pod template](/docs/concepts/workloads/pods/#pod-templates). It has exactly the same schema as a {{< glossary_tooltip text="Pod" term_id="pod" >}}, except it is nested and does not have an `apiVersion` or `kind`.
In addition to required fields for a Pod, a Pod template in a Deployment must specify appropriate
labels and an appropriate restart policy. For labels, make sure not to overlap with other controllers. See [selector](#selector)).
labels and an appropriate restart policy. For labels, make sure not to overlap with other controllers. See [selector](#selector).
Only a [`.spec.template.spec.restartPolicy`](/docs/concepts/workloads/pods/pod-lifecycle/#restart-policy) equal to `Always` is
allowed, which is the default if not specified.

2
content/en/docs/concepts/workloads/controllers/garbage-collection.md
View File

@ -1,7 +1,7 @@
---
title: Garbage Collection
content_type: concept
weight: 70
weight: 60
---
<!-- overview -->

8
content/en/docs/concepts/workloads/controllers/job.md
View File

@ -8,7 +8,7 @@ feature:
title: Batch execution
description: >
In addition to services, Kubernetes can manage your batch and CI workloads, replacing containers that fail, if desired.
weight: 60
weight: 50
---
<!-- overview -->
@ -24,9 +24,6 @@ due to a node hardware failure or a node reboot).
You can also use a Job to run multiple Pods in parallel.
<!-- body -->
## Running an example Job
@ -122,6 +119,7 @@ A Job also needs a [`.spec` section](https://git.k8s.io/community/contributors/d
The `.spec.template` is the only required field of the `.spec`.
The `.spec.template` is a [pod template](/docs/concepts/workloads/pods/#pod-templates). It has exactly the same schema as a {{< glossary_tooltip text="Pod" term_id="pod" >}}, except it is nested and does not have an `apiVersion` or `kind`.
In addition to required fields for a Pod, a pod template in a Job must specify appropriate
@ -450,7 +448,7 @@ requires only a single Pod.
### Replication Controller
Jobs are complementary to [Replication Controllers](/docs/user-guide/replication-controller).
Jobs are complementary to [Replication Controllers](/docs/concepts/workloads/controllers/replicationcontroller/).
A Replication Controller manages Pods which are not expected to terminate (e.g. web servers), and a Job
manages Pods that are expected to terminate (e.g. batch tasks).

11
content/en/docs/concepts/workloads/controllers/replicaset.md
View File

@ -5,7 +5,7 @@ reviewers:
- madhusudancs
title: ReplicaSet
content_type: concept
weight: 10
weight: 20
---
<!-- overview -->
@ -245,9 +245,10 @@ For the template's [restart policy](/docs/concepts/workloads/Pods/pod-lifecycle/
The `.spec.selector` field is a [label selector](/docs/concepts/overview/working-with-objects/labels/). As discussed
[earlier](#how-a-replicaset-works) these are the labels used to identify potential Pods to acquire. In our
`frontend.yaml` example, the selector was:
```shell
```yaml
matchLabels:
tier: frontend
tier: frontend
```
In the ReplicaSet, `.spec.template.metadata.labels` must match `spec.selector`, or it will
@ -295,7 +296,7 @@ curl -X DELETE 'localhost:8080/apis/apps/v1/namespaces/default/replicasets/fron
Once the original is deleted, you can create a new ReplicaSet to replace it. As long
as the old and new `.spec.selector` are the same, then the new one will adopt the old Pods.
However, it will not make any effort to make existing Pods match a new, different pod template.
To update Pods to a new spec in a controlled way, use a
To update Pods to a new spec in a controlled way, use a
[Deployment](/docs/concepts/workloads/controllers/deployment/#creating-a-deployment), as ReplicaSets do not support a rolling update directly.
### Isolating Pods from a ReplicaSet
@ -340,7 +341,7 @@ kubectl autoscale rs frontend --max=10 --min=3 --cpu-percent=50
[`Deployment`](/docs/concepts/workloads/controllers/deployment/) is an object which can own ReplicaSets and update
them and their Pods via declarative, server-side rolling updates.
While ReplicaSets can be used independently, today they're mainly used by Deployments as a mechanism to orchestrate Pod
creation, deletion and updates. When you use Deployments you don’t have to worry about managing the ReplicaSets that
creation, deletion and updates. When you use Deployments you don't have to worry about managing the ReplicaSets that
they create. Deployments own and manage their ReplicaSets.
As such, it is recommended to use Deployments when you want ReplicaSets.

23
content/en/docs/concepts/workloads/controllers/replicationcontroller.md
View File

@ -10,7 +10,7 @@ feature:
Restarts containers that fail, replaces and reschedules containers when nodes die, kills containers that don't respond to your user-defined health check, and doesn't advertise them to clients until they are ready to serve.
content_type: concept
weight: 20
weight: 90
---
<!-- overview -->
@ -23,9 +23,6 @@ A _ReplicationController_ ensures that a specified number of pod replicas are ru
time. In other words, a ReplicationController makes sure that a pod or a homogeneous set of pods is
always up and available.
<!-- body -->
## How a ReplicationController Works
@ -134,7 +131,7 @@ labels and an appropriate restart policy. For labels, make sure not to overlap w
Only a [`.spec.template.spec.restartPolicy`](/docs/concepts/workloads/pods/pod-lifecycle/#restart-policy) equal to `Always` is allowed, which is the default if not specified.
For local container restarts, ReplicationControllers delegate to an agent on the node,
for example the [Kubelet](/docs/admin/kubelet/) or Docker.
for example the [Kubelet](/docs/reference/command-line-tools-reference/kubelet/) or Docker.
### Labels on the ReplicationController
@ -214,7 +211,7 @@ The ReplicationController makes it easy to scale the number of replicas up or do
The ReplicationController is designed to facilitate rolling updates to a service by replacing pods one-by-one.
As explained in [#1353](http://issue.k8s.io/1353), the recommended approach is to create a new ReplicationController with 1 replica, scale the new (+1) and old (-1) controllers one by one, and then delete the old controller after it reaches 0 replicas. This predictably updates the set of pods regardless of unexpected failures.
As explained in [#1353](https://issue.k8s.io/1353), the recommended approach is to create a new ReplicationController with 1 replica, scale the new (+1) and old (-1) controllers one by one, and then delete the old controller after it reaches 0 replicas. This predictably updates the set of pods regardless of unexpected failures.
Ideally, the rolling update controller would take application readiness into account, and would ensure that a sufficient number of pods were productively serving at any given time.
@ -239,11 +236,11 @@ Pods created by a ReplicationController are intended to be fungible and semantic
## Responsibilities of the ReplicationController
The ReplicationController simply ensures that the desired number of pods matches its label selector and are operational. Currently, only terminated pods are excluded from its count. In the future, [readiness](http://issue.k8s.io/620) and other information available from the system may be taken into account, we may add more controls over the replacement policy, and we plan to emit events that could be used by external clients to implement arbitrarily sophisticated replacement and/or scale-down policies.
The ReplicationController simply ensures that the desired number of pods matches its label selector and are operational. Currently, only terminated pods are excluded from its count. In the future, [readiness](https://issue.k8s.io/620) and other information available from the system may be taken into account, we may add more controls over the replacement policy, and we plan to emit events that could be used by external clients to implement arbitrarily sophisticated replacement and/or scale-down policies.
The ReplicationController is forever constrained to this narrow responsibility. It itself will not perform readiness nor liveness probes. Rather than performing auto-scaling, it is intended to be controlled by an external auto-scaler (as discussed in [#492](http://issue.k8s.io/492)), which would change its `replicas` field. We will not add scheduling policies (for example, [spreading](http://issue.k8s.io/367#issuecomment-48428019)) to the ReplicationController. Nor should it verify that the pods controlled match the currently specified template, as that would obstruct auto-sizing and other automated processes. Similarly, completion deadlines, ordering dependencies, configuration expansion, and other features belong elsewhere. We even plan to factor out the mechanism for bulk pod creation ([#170](http://issue.k8s.io/170)).
The ReplicationController is forever constrained to this narrow responsibility. It itself will not perform readiness nor liveness probes. Rather than performing auto-scaling, it is intended to be controlled by an external auto-scaler (as discussed in [#492](https://issue.k8s.io/492)), which would change its `replicas` field. We will not add scheduling policies (for example, [spreading](https://issue.k8s.io/367#issuecomment-48428019)) to the ReplicationController. Nor should it verify that the pods controlled match the currently specified template, as that would obstruct auto-sizing and other automated processes. Similarly, completion deadlines, ordering dependencies, configuration expansion, and other features belong elsewhere. We even plan to factor out the mechanism for bulk pod creation ([#170](https://issue.k8s.io/170)).
The ReplicationController is intended to be a composable building-block primitive. We expect higher-level APIs and/or tools to be built on top of it and other complementary primitives for user convenience in the future. The "macro" operations currently supported by kubectl (run, scale) are proof-of-concept examples of this. For instance, we could imagine something like [Asgard](http://techblog.netflix.com/2012/06/asgard-web-based-cloud-management-and.html) managing ReplicationControllers, auto-scalers, services, scheduling policies, canaries, etc.
The ReplicationController is intended to be a composable building-block primitive. We expect higher-level APIs and/or tools to be built on top of it and other complementary primitives for user convenience in the future. The "macro" operations currently supported by kubectl (run, scale) are proof-of-concept examples of this. For instance, we could imagine something like [Asgard](https://techblog.netflix.com/2012/06/asgard-web-based-cloud-management-and.html) managing ReplicationControllers, auto-scalers, services, scheduling policies, canaries, etc.
## API Object
@ -257,8 +254,8 @@ API object can be found at:
### ReplicaSet
[`ReplicaSet`](/docs/concepts/workloads/controllers/replicaset/) is the next-generation ReplicationController that supports the new [set-based label selector](/docs/concepts/overview/working-with-objects/labels/#set-based-requirement).
It’s mainly used by [`Deployment`](/docs/concepts/workloads/controllers/deployment/) as a mechanism to orchestrate pod creation, deletion and updates.
Note that we recommend using Deployments instead of directly using Replica Sets, unless you require custom update orchestration or don’t require updates at all.
It's mainly used by [Deployment](/docs/concepts/workloads/controllers/deployment/) as a mechanism to orchestrate pod creation, deletion and updates.
Note that we recommend using Deployments instead of directly using Replica Sets, unless you require custom update orchestration or don't require updates at all.
### Deployment (Recommended)
@ -271,7 +268,7 @@ Unlike in the case where a user directly created pods, a ReplicationController r
### Job
Use a [`Job`](/docs/concepts/jobs/run-to-completion-finite-workloads/) instead of a ReplicationController for pods that are expected to terminate on their own
Use a [`Job`](/docs/concepts/workloads/controllers/job/) instead of a ReplicationController for pods that are expected to terminate on their own
(that is, batch jobs).
### DaemonSet
@ -283,6 +280,6 @@ safe to terminate when the machine is otherwise ready to be rebooted/shutdown.
## For more information
Read [Run Stateless AP Replication Controller](/docs/tutorials/stateless-application/run-stateless-ap-replication-controller/).
Read [Run Stateless Application Deployment](/docs/tasks/run-application/run-stateless-application-deployment/).

4
content/en/docs/concepts/workloads/controllers/statefulset.md
View File

@ -8,7 +8,7 @@ reviewers:
- smarterclayton
title: StatefulSets
content_type: concept
weight: 40
weight: 30
---
<!-- overview -->
@ -200,7 +200,7 @@ The StatefulSet should not specify a `pod.Spec.TerminationGracePeriodSeconds` of
When the nginx example above is created, three Pods will be deployed in the order
web-0, web-1, web-2. web-1 will not be deployed before web-0 is
[Running and Ready](/docs/user-guide/pod-states/), and web-2 will not be deployed until
[Running and Ready](/docs/concepts/workloads/pods/pod-lifecycle/), and web-2 will not be deployed until
web-1 is Running and Ready. If web-0 should fail, after web-1 is Running and Ready, but before
web-2 is launched, web-2 will not be launched until web-0 is successfully relaunched and
becomes Running and Ready.

12
content/en/docs/concepts/workloads/controllers/ttlafterfinished.md
View File

@ -20,12 +20,6 @@ Alpha Disclaimer: this feature is currently alpha, and can be enabled with both
[feature gate](/docs/reference/command-line-tools-reference/feature-gates/)
`TTLAfterFinished`.
<!-- body -->
## TTL Controller
@ -82,9 +76,7 @@ very small. Please be aware of this risk when setting a non-zero TTL.
## {{% heading "whatsnext" %}}
* [Clean up Jobs automatically](/docs/concepts/workloads/controllers/job/#clean-up-finished-jobs-automatically)
[Clean up Jobs automatically](/docs/concepts/workloads/controllers/jobs-run-to-completion/#clean-up-finished-jobs-automatically)
[Design doc](https://github.com/kubernetes/enhancements/blob/master/keps/sig-apps/0026-ttl-after-finish.md)
* [Design doc](https://github.com/kubernetes/enhancements/blob/master/keps/sig-apps/0026-ttl-after-finish.md)

12
content/en/docs/concepts/workloads/pods/_index.md
View File

@ -19,7 +19,7 @@ A _Pod_ (as in a pod of whales or pea pod) is a group of one or more
for how to run the containers. A Pod's contents are always co-located and
co-scheduled, and run in a shared context. A Pod models an
application-specific "logical host": it contains one or more application
containers which are relatively tightly coupled.
containers which are relatively tightly coupled.
In non-cloud contexts, applications executed on the same physical or virtual machine are analogous to cloud applications executed on the same logical host.
As well as application containers, a Pod can contain
@ -51,7 +51,7 @@ with shared namespaces and shared filesystem volumes.
Usually you don't need to create Pods directly, even singleton Pods. Instead, create them using workload resources such as {{< glossary_tooltip text="Deployment"
term_id="deployment" >}} or {{< glossary_tooltip text="Job" term_id="job" >}}.
If your Pods need to track state, consider the
If your Pods need to track state, consider the
{{< glossary_tooltip text="StatefulSet" term_id="statefulset" >}} resource.
Pods in a Kubernetes cluster are used in two main ways:
@ -65,7 +65,7 @@ Pods in a Kubernetes cluster are used in two main ways:
tightly coupled and need to share resources. These co-located containers
form a single cohesive unit of service—for example, one container serving data
stored in a shared volume to the public, while a separate _sidecar_ container
refreshes or updates those files.
refreshes or updates those files.
The Pod wraps these containers, storage resources, and an ephemeral network
identity together as a single unit.
@ -190,7 +190,7 @@ changing existing code.
## Resource sharing and communication
Pods enable data sharing and communication among their constituent
containters.
containers.
### Storage in Pods {#pod-storage}
@ -257,8 +257,8 @@ but cannot be controlled from there.
* Lean about [RuntimeClass](/docs/concepts/containers/runtime-class/) and how you can use it to
configure different Pods with different container runtime configurations.
* Read about [Pod topology spread constraints](/docs/concepts/workloads/pods/pod-topology-spread-constraints/).
* Read about [PodDisruptionBudget](https://kubernetes.io/docs/concepts/workloads/pods/disruptions/) and how you can use it to manage application availability during disruptions.
* Pod is a top-level resource in the Kubernetes REST API.
* Read about [PodDisruptionBudget](/docs/concepts/workloads/pods/disruptions/) and how you can use it to manage application availability during disruptions.
* Pod is a top-level resource in the Kubernetes REST API.
The [Pod](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#pod-v1-core)
object definition describes the object in detail.
* [The Distributed System Toolkit: Patterns for Composite Containers](https://kubernetes.io/blog/2015/06/the-distributed-system-toolkit-patterns) explains common layouts for Pods with more than one container.

15
content/en/docs/concepts/workloads/pods/disruptions.md
View File

@ -16,7 +16,6 @@ what types of disruptions can happen to Pods.
It is also for cluster administrators who want to perform automated
cluster actions, like upgrading and autoscaling clusters.
<!-- body -->
## Voluntary and involuntary disruptions
@ -70,15 +69,15 @@ deleting deployments or pods bypasses Pod Disruption Budgets.
Here are some ways to mitigate involuntary disruptions:
- Ensure your pod [requests the resources](/docs/tasks/configure-pod-container/assign-cpu-ram-container) it needs.
- Ensure your pod [requests the resources](/docs/tasks/configure-pod-container/assign-memory-resource) it needs.
- Replicate your application if you need higher availability. (Learn about running replicated
[stateless](/docs/tasks/run-application/run-stateless-application-deployment/)
and [stateful](/docs/tasks/run-application/run-replicated-stateful-application/) applications.)
[stateless](/docs/tasks/run-application/run-stateless-application-deployment/)
and [stateful](/docs/tasks/run-application/run-replicated-stateful-application/) applications.)
- For even higher availability when running replicated applications,
spread applications across racks (using
[anti-affinity](/docs/user-guide/node-selection/#inter-pod-affinity-and-anti-affinity-beta-feature))
or across zones (if using a
[multi-zone cluster](/docs/setup/multiple-zones).)
spread applications across racks (using
[anti-affinity](/docs/user-guide/node-selection/#inter-pod-affinity-and-anti-affinity-beta-feature))
or across zones (if using a
[multi-zone cluster](/docs/setup/multiple-zones).)
The frequency of voluntary disruptions varies. On a basic Kubernetes cluster, there are
no voluntary disruptions at all. However, your cluster administrator or hosting provider

23
content/en/docs/concepts/workloads/pods/pod-lifecycle.md
View File

@ -45,7 +45,7 @@ higher-level abstraction, called a
managing the relatively disposable Pod instances.
A given Pod (as defined by a UID) is never "rescheduled" to a different node; instead,
that Pod can be replaced by a new, near-identical Pod, with even the same name i
that Pod can be replaced by a new, near-identical Pod, with even the same name if
desired, but with a different UID.
When something is said to have the same lifetime as a Pod, such as a
@ -78,7 +78,7 @@ Here are the possible values for `phase`:
Value | Description
:-----|:-----------
`Pending` | The Pod has been accepted by the Kubernetes cluster, but one or more of the containers has not been set up and made ready to run. This includes time a Pod spends waiting to bescheduled as well as the time spent downloading container images over the network.
`Pending` | The Pod has been accepted by the Kubernetes cluster, but one or more of the containers has not been set up and made ready to run. This includes time a Pod spends waiting to be scheduled as well as the time spent downloading container images over the network.
`Running` | The Pod has been bound to a node, and all of the containers have been created. At least one container is still running, or is in the process of starting or restarting.
`Succeeded` | All containers in the Pod have terminated in success, and will not be restarted.
`Failed` | All containers in the Pod have terminated, and at least one container has terminated in failure. That is, the container either exited with non-zero status or was terminated by the system.
@ -107,7 +107,7 @@ Each state has a specific meaning:
### `Waiting` {#container-state-waiting}
If a container is not in either the `Running` or `Terminated` state, it `Waiting`.
If a container is not in either the `Running` or `Terminated` state, it is `Waiting`.
A container in the `Waiting` state is still running the operations it requires in
order to complete start up: for example, pulling the container image from a container
image registry, or applying {{< glossary_tooltip text="Secret" term_id="secret" >}}
@ -118,15 +118,15 @@ a Reason field to summarize why the container is in that state.
### `Running` {#container-state-running}
The `Running` status indicates that a container is executing without issues. If there
was a `postStart` hook configured, it has already executed and executed. When you use
was a `postStart` hook configured, it has already executed and finished. When you use
`kubectl` to query a Pod with a container that is `Running`, you also see information
about when the container entered the `Running` state.
### `Terminated` {#container-state-terminated}
A container in the `Terminated` state has begin execution and has then either run to
completion or has failed for some reason. When you use `kubectl` to query a Pod with
a container that is `Terminated`, you see a reason, and exit code, and the start and
A container in the `Terminated` state began execution and then either ran to
completion or failed for some reason. When you use `kubectl` to query a Pod with
a container that is `Terminated`, you see a reason, an exit code, and the start and
finish time for that container's period of execution.
If a container has a `preStop` hook configured, that runs before the container enters
@ -164,7 +164,7 @@ Field name | Description
`lastProbeTime` | Timestamp of when the Pod condition was last probed.
`lastTransitionTime` | Timestamp for when the Pod last transitioned from one status to another.
`reason` | Machine-readable, UpperCamelCase text indicating the reason for the condition's last transition.
`messsage | Human-readable message indicating details about the last status transition.
`message` | Human-readable message indicating details about the last status transition.
### Pod readiness {#pod-readiness-gate}
@ -341,8 +341,8 @@ before the Pod is allowed to be forcefully killed. With that forceful shutdown t
place, the {{< glossary_tooltip text="kubelet" term_id="kubelet" >}} attempts graceful
shutdown.
Typically, the container runtime sends a a TERM signal is sent to the main process in each
container. Once the grace period has expired, the KILL signal is sent to any remainig
Typically, the container runtime sends a TERM signal to the main process in each
container. Once the grace period has expired, the KILL signal is sent to any remaining
processes, and the Pod is then deleted from the
{{< glossary_tooltip text="API Server" term_id="kube-apiserver" >}}. If the kubelet or the
container runtime's management service is restarted while waiting for processes to terminate, the
@ -392,7 +392,7 @@ An example flow:
### Forced Pod termination {#pod-termination-forced}
{{< caution >}}
Forced deletions can be potentially disruptiove for some workloads and their Pods.
Forced deletions can be potentially disruptive for some workloads and their Pods.
{{< /caution >}}
By default, all deletes are graceful within 30 seconds. The `kubectl delete` command supports
@ -442,4 +442,3 @@ This avoids a resource leak as Pods are created and terminated over time.
* For detailed information about Pod / Container status in the API, see [PodStatus](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#podstatus-v1-core)
and
[ContainerStatus](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#containerstatus-v1-core).

84
content/en/docs/concepts/workloads/pods/pod-topology-spread-constraints.md
View File

@ -6,8 +6,6 @@ weight: 40
<!-- overview -->
{{< feature-state for_k8s_version="v1.18" state="beta" >}}
You can use _topology spread constraints_ to control how {{< glossary_tooltip text="Pods" term_id="Pod" >}} are spread across your cluster among failure-domains such as regions, zones, nodes, and other user-defined topology domains. This can help to achieve high availability as well as efficient resource utilization.
@ -16,13 +14,6 @@ You can use _topology spread constraints_ to control how {{< glossary_tooltip te
## Prerequisites
### Enable Feature Gate
The `EvenPodsSpread` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/)
must be enabled for the
{{< glossary_tooltip text="API Server" term_id="kube-apiserver" >}} **and**
{{< glossary_tooltip text="scheduler" term_id="kube-scheduler" >}}.
### Node Labels
Topology spread constraints rely on node labels to identify the topology domain(s) that each Node is in. For example, a Node might have labels: `node=node1,zone=us-east-1a,region=us-east-1`
@ -53,7 +44,7 @@ Instead of manually applying labels, you can also reuse the [well-known labels](
### API
The field `pod.spec.topologySpreadConstraints` is introduced in 1.16 as below:
The API field `pod.spec.topologySpreadConstraints` is defined as below:
```
apiVersion: v1
@ -70,7 +61,15 @@ spec:
You can define one or multiple `topologySpreadConstraint` to instruct the kube-scheduler how to place each incoming Pod in relation to the existing Pods across your cluster. The fields are:
- **maxSkew** describes the degree to which Pods may be unevenly distributed. It's the maximum permitted difference between the number of matching Pods in any two topology domains of a given topology type. It must be greater than zero.
- **maxSkew** describes the degree to which Pods may be unevenly distributed.
It's the maximum permitted difference between the number of matching Pods in
any two topology domains of a given topology type. It must be greater than
zero. Its semantics differs according to the value of `whenUnsatisfiable`:
- when `whenUnsatisfiable` equals to "DoNotSchedule", `maxSkew` is the maximum
permitted difference between the number of matching pods in the target
topology and the global minimum.
- when `whenUnsatisfiable` equals to "ScheduleAnyway", scheduler gives higher
precedence to topologies that would help reduce the skew.
- **topologyKey** is the key of node labels. If two Nodes are labelled with this key and have identical values for that label, the scheduler treats both Nodes as being in the same topology. The scheduler tries to place a balanced number of Pods into each topology domain.
- **whenUnsatisfiable** indicates how to deal with a Pod if it doesn't satisfy the spread constraint:
- `DoNotSchedule` (default) tells the scheduler not to schedule it.
@ -97,7 +96,7 @@ If we want an incoming Pod to be evenly spread with existing Pods across zones,
{{< codenew file="pods/topology-spread-constraints/one-constraint.yaml" >}}
`topologyKey: zone` implies the even distribution will only be applied to the nodes which have label pair "zone:&lt;any value&gt;" present. `whenUnsatisfiable: DoNotSchedule` tells the scheduler to let it stay pending if the incoming Pod can’t satisfy the constraint.
`topologyKey: zone` implies the even distribution will only be applied to the nodes which have label pair "zone:&lt;any value&gt;" present. `whenUnsatisfiable: DoNotSchedule` tells the scheduler to let it stay pending if the incoming Pod can't satisfy the constraint.
If the scheduler placed this incoming Pod into "zoneA", the Pods distribution would become [3, 1], hence the actual skew is 2 (3 - 1) - which violates `maxSkew: 1`. In this example, the incoming Pod can only be placed onto "zoneB":
@ -115,7 +114,7 @@ You can tweak the Pod spec to meet various kinds of requirements:
- Change `maxSkew` to a bigger value like "2" so that the incoming Pod can be placed onto "zoneA" as well.
- Change `topologyKey` to "node" so as to distribute the Pods evenly across nodes instead of zones. In the above example, if `maxSkew` remains "1", the incoming Pod can only be placed onto "node4".
- Change `whenUnsatisfiable: DoNotSchedule` to `whenUnsatisfiable: ScheduleAnyway` to ensure the incoming Pod to be always schedulable (suppose other scheduling APIs are satisfied). However, it’s preferred to be placed onto the topology domain which has fewer matching Pods. (Be aware that this preferability is jointly normalized with other internal scheduling priorities like resource usage ratio, etc.)
- Change `whenUnsatisfiable: DoNotSchedule` to `whenUnsatisfiable: ScheduleAnyway` to ensure the incoming Pod to be always schedulable (suppose other scheduling APIs are satisfied). However, it's preferred to be placed onto the topology domain which has fewer matching Pods. (Be aware that this preferability is jointly normalized with other internal scheduling priorities like resource usage ratio, etc.)
### Example: Multiple TopologySpreadConstraints
@ -161,10 +160,10 @@ There are some implicit conventions worth noting here:
- Nodes without `topologySpreadConstraints[*].topologyKey` present will be bypassed. It implies that:
1. the Pods located on those nodes do not impact `maxSkew` calculation - in the above example, suppose "node1" does not have label "zone", then the 2 Pods will be disregarded, hence the incomingPod will be scheduled into "zoneA".
1. the Pods located on those nodes do not impact `maxSkew` calculation - in the above example, suppose "node1" does not have label "zone", then the 2 Pods will be disregarded, hence the incoming Pod will be scheduled into "zoneA".
2. the incoming Pod has no chances to be scheduled onto this kind of nodes - in the above example, suppose a "node5" carrying label `{zone-typo: zoneC}` joins the cluster, it will be bypassed due to the absence of label key "zone".
- Be aware of what will happen if the incomingPod’s `topologySpreadConstraints[*].labelSelector` doesn’t match its own labels. In the above example, if we remove the incoming Pod’s labels, it can still be placed onto "zoneB" since the constraints are still satisfied. However, after the placement, the degree of imbalance of the cluster remains unchanged - it’s still zoneA having 2 Pods which hold label {foo:bar}, and zoneB having 1 Pod which holds label {foo:bar}. So if this is not what you expect, we recommend the workload’s `topologySpreadConstraints[*].labelSelector` to match its own labels.
- Be aware of what will happen if the incomingPod's `topologySpreadConstraints[*].labelSelector` doesn't match its own labels. In the above example, if we remove the incoming Pod's labels, it can still be placed onto "zoneB" since the constraints are still satisfied. However, after the placement, the degree of imbalance of the cluster remains unchanged - it's still zoneA having 2 Pods which hold label {foo:bar}, and zoneB having 1 Pod which holds label {foo:bar}. So if this is not what you expect, we recommend the workload's `topologySpreadConstraints[*].labelSelector` to match its own labels.
- If the incoming Pod has `spec.nodeSelector` or `spec.affinity.nodeAffinity` defined, nodes not matching them will be bypassed.
@ -186,7 +185,7 @@ There are some implicit conventions worth noting here:
### Cluster-level default constraints
{{< feature-state for_k8s_version="v1.18" state="alpha" >}}
{{< feature-state for_k8s_version="v1.19" state="beta" >}}
It is possible to set default topology spread constraints for a cluster. Default
topology spread constraints are applied to a Pod if, and only if:
@ -195,7 +194,7 @@ topology spread constraints are applied to a Pod if, and only if:
- It belongs to a service, replication controller, replica set or stateful set.
Default constraints can be set as part of the `PodTopologySpread` plugin args
in a [scheduling profile](/docs/reference/scheduling/profiles).
in a [scheduling profile](/docs/reference/scheduling/config/#profiles).
The constraints are specified with the same [API above](#api), except that
`labelSelector` must be empty. The selectors are calculated from the services,
replication controllers, replica sets or stateful sets that the Pod belongs to.
@ -203,7 +202,7 @@ replication controllers, replica sets or stateful sets that the Pod belongs to.
An example configuration might look like follows:
```yaml
apiVersion: kubescheduler.config.k8s.io/v1alpha2
apiVersion: kubescheduler.config.k8s.io/v1beta1
kind: KubeSchedulerConfiguration
profiles:
@ -212,18 +211,49 @@ profiles:
args:
defaultConstraints:
- maxSkew: 1
topologyKey: failure-domain.beta.kubernetes.io/zone
topologyKey: topology.kubernetes.io/zone
whenUnsatisfiable: ScheduleAnyway
```
{{< note >}}
The score produced by default scheduling constraints might conflict with the
score produced by the
[`DefaultPodTopologySpread` plugin](/docs/reference/scheduling/profiles/#scheduling-plugins).
[`SelectorSpread` plugin](/docs/reference/scheduling/config/#scheduling-plugins).
It is recommended that you disable this plugin in the scheduling profile when
using default constraints for `PodTopologySpread`.
{{< /note >}}
#### Internal default constraints
{{< feature-state for_k8s_version="v1.19" state="alpha" >}}
When you enable the `DefaultPodTopologySpread` feature gate, the
legacy `SelectorSpread` plugin is disabled.
kube-scheduler uses the following default topology constraints for the
`PodTopologySpread` plugin configuration:
```yaml
defaultConstraints:
- maxSkew: 3
topologyKey: "kubernetes.io/hostname"
whenUnsatisfiable: ScheduleAnyway
- maxSkew: 5
topologyKey: "topology.kubernetes.io/zone"
whenUnsatisfiable: ScheduleAnyway
```
Also, the legacy `SelectorSpread` plugin, which provides an equivalent behavior,
is disabled.
{{< note >}}
If your nodes are not expected to have **both** `kubernetes.io/hostname` and
`topology.kubernetes.io/zone` labels set, define your own constraints
instead of using the Kubernetes defaults.
The `PodTopologySpread` plugin does not score the nodes that don't have
the topology keys specified in the spreading constraints.
{{< /note >}}
## Comparison with PodAffinity/PodAntiAffinity
In Kubernetes, directives related to "Affinity" control how Pods are
@ -234,15 +264,19 @@ scheduled - more packed or more scattered.
- For `PodAntiAffinity`, only one Pod can be scheduled into a
single topology domain.
The "EvenPodsSpread" feature provides flexible options to distribute Pods evenly across different
topology domains - to achieve high availability or cost-saving. This can also help on rolling update
workloads and scaling out replicas smoothly. See [Motivation](https://github.com/kubernetes/enhancements/tree/master/keps/sig-scheduling/895-pod-topology-spread#motivation) for more details.
For finer control, you can specify topology spread constraints to distribute
Pods across different topology domains - to achieve either high availability or
cost-saving. This can also help on rolling update workloads and scaling out
replicas smoothly. See
[Motivation](https://github.com/kubernetes/enhancements/tree/master/keps/sig-scheduling/895-pod-topology-spread#motivation)
for more details.
## Known Limitations
As of 1.18, at which this feature is Beta, there are some known limitations:
- Scaling down a Deployment may result in imbalanced Pods distribution.
- Pods matched on tainted nodes are respected. See [Issue 80921](https://github.com/kubernetes/kubernetes/issues/80921)
## {{% heading "whatsnext" %}}
- [Blog: Introducing PodTopologySpread](https://kubernetes.io/blog/2020/05/introducing-podtopologyspread/)
explains `maxSkew` in details, as well as bringing up some advanced usage examples.

2
content/en/docs/contribute/_index.md
View File

@ -53,7 +53,7 @@ roles and permissions.
- Read the [Contribution overview](/docs/contribute/new-content/overview/) to
learn about the different ways you can contribute.
- Check [kubernetes/website issues list](/https://github.com/kubernetes/website/issues/)
- Check [`kubernetes/website` issues list](https://github.com/kubernetes/website/issues/)
for issues that make good entry points.
- [Open a pull request using GitHub](/docs/contribute/new-content/open-a-pr/#changes-using-github)
to existing documentation and learn more about filing issues in GitHub.

4
content/en/docs/contribute/advanced.md
View File

@ -184,8 +184,8 @@ Begin and end meetings on time.
### Recording meetings on Zoom
When you’re ready to start the recording, click Record to Cloud.
When you're ready to start the recording, click Record to Cloud.
When you’re ready to stop recording, click Stop.
When you're ready to stop recording, click Stop.
The video uploads automatically to YouTube.

2
content/en/docs/contribute/generate-ref-docs/kubectl.md
View File

@ -127,7 +127,7 @@ Monitor your cherry-pick pull request until it is merged into the release branch
{{< note >}}
Proposing a cherry pick requires that you have permission to set a label and a
milestone in your pull request. If you don’t have those permissions, you will
milestone in your pull request. If you don't have those permissions, you will
need to work with someone who can set the label and milestone for you.
{{< /note >}}

2
content/en/docs/contribute/localization.md
View File

@ -75,7 +75,7 @@ For an example of adding a label, see the PR for adding the [Italian language la
Let Kubernetes SIG Docs know you're interested in creating a localization! Join the [SIG Docs Slack channel](https://kubernetes.slack.com/messages/C1J0BPD2M/). Other localization teams are happy to help you get started and answer any questions you have.
You can also create a Slack channel for your localization in the `kubernetes/community` repository. For an example of adding a Slack channel, see the PR for [adding channels for Indonesian and Portuguese](https://github.com/kubernetes/community/pull/3605).
You can also create a Slack channel for your localization in the `kubernetes/community` repository. For an example of adding a Slack channel, see the PR for [adding a channel for Persian](https://github.com/kubernetes/community/pull/4980).
## Minimum required content

2
content/en/docs/contribute/participate/_index.md
View File

@ -105,7 +105,7 @@ SIG Docs approvers. Here's how it works.
- Any Kubernetes member can add the `lgtm` label by adding a `/lgtm` comment.
- Only SIG Docs approvers can merge a pull request
by adding an `/approve` comment. Some approvers also perform additional
specific roles, such as [PR Wrangler](/docs/contribute/advanced#be-the-pr-wrangler-for-a-week) or
specific roles, such as [PR Wrangler](/docs/contribute/participate/pr-wranglers/) or
[SIG Docs chairperson](#sig-docs-chairperson).

17
content/en/docs/contribute/participate/pr-wranglers.md
View File

@ -20,7 +20,7 @@ Each day in a week-long shift as PR Wrangler:
- Review [open pull requests](https://github.com/kubernetes/website/pulls) for quality and adherence to the [Style](/docs/contribute/style/style-guide/) and [Content](/docs/contribute/style/content-guide/) guides.
- Start with the smallest PRs (`size/XS`) first, and end with the largest (`size/XXL`). Review as many PRs as you can.
- Make sure PR contributors sign the [CLA](https://github.com/kubernetes/community/blob/master/CLA.md).
- Use [this](https://github.com/zparnold/k8s-docs-pr-botherer) script to remind contributors that haven’t signed the CLA to do so.
- Use [this](https://github.com/zparnold/k8s-docs-pr-botherer) script to remind contributors that haven't signed the CLA to do so.
- Provide feedback on changes and ask for technical reviews from members of other SIGs.
- Provide inline suggestions on the PR for the proposed content changes.
- If you need to verify content, comment on the PR and request more details.
@ -47,6 +47,19 @@ These queries exclude localization PRs. All queries are against the main branch
- [Quick Wins](https://github.com/kubernetes/website/pulls?utf8=%E2%9C%93&q=is%3Apr+is%3Aopen+base%3Amaster+-label%3A%22do-not-merge%2Fwork-in-progress%22+-label%3A%22do-not-merge%2Fhold%22+label%3A%22cncf-cla%3A+yes%22+label%3A%22size%2FXS%22+label%3A%22language%2Fen%22): Lists PRs against the main branch with no clear blockers. (change "XS" in the size label as you work through the PRs [XS, S, M, L, XL, XXL]).
- [Not against the main branch](https://github.com/kubernetes/website/pulls?q=is%3Aopen+is%3Apr+label%3Alanguage%2Fen+-base%3Amaster): If the PR is against a `dev-` branch, it's for an upcoming release. Assign the [docs release manager](https://github.com/kubernetes/sig-release/tree/master/release-team#kubernetes-release-team-roles) using: `/assign @<manager's_github-username>`. If the PR is against an old branch, help the author figure out whether it's targeted against the best branch.
### Helpful Prow commands for wranglers
```
# add English label
/language en
# add squash label to PR if more than one commit
/label tide/merge-method-squash
# retitle a PR via Prow (such as a work-in-progress [WIP] or better detail of PR)
/retitle [WIP] <TITLE>
```
### When to close Pull Requests
Reviews and approvals are one tool to keep our PR queue short and current. Another tool is closure.
@ -66,4 +79,4 @@ To close a pull request, leave a `/close` comment on the PR.
The [`fejta-bot`](https://github.com/fejta-bot) bot marks issues as stale after 90 days of inactivity. After 30 more days it marks issues as rotten and closes them. PR wranglers should close issues after 14-30 days of inactivity.
{{< /note >}}
{{< /note >}}

Some files were not shown because too many files have changed in this diff Show More