1077 lines
38 KiB
Markdown
1077 lines
38 KiB
Markdown
---
|
|
title: Declarative Management of Kubernetes Objects Using Configuration Files
|
|
content_type: task
|
|
weight: 10
|
|
---
|
|
|
|
<!-- overview -->
|
|
Kubernetes objects can be created, updated, and deleted by storing multiple
|
|
object configuration files in a directory and using `kubectl apply` to
|
|
recursively create and update those objects as needed. This method
|
|
retains writes made to live objects without merging the changes
|
|
back into the object configuration files. `kubectl diff` also gives you a
|
|
preview of what changes `apply` will make.
|
|
|
|
## {{% heading "prerequisites" %}}
|
|
|
|
Install [`kubectl`](/docs/tasks/tools/).
|
|
|
|
{{< include "task-tutorial-prereqs.md" >}} {{< version-check >}}
|
|
|
|
<!-- steps -->
|
|
|
|
## Trade-offs
|
|
|
|
The `kubectl` tool supports three kinds of object management:
|
|
|
|
* Imperative commands
|
|
* Imperative object configuration
|
|
* Declarative object configuration
|
|
|
|
See [Kubernetes Object Management](/docs/concepts/overview/working-with-objects/object-management/)
|
|
for a discussion of the advantages and disadvantage of each kind of object management.
|
|
|
|
## Overview
|
|
|
|
Declarative object configuration requires a firm understanding of
|
|
the Kubernetes object definitions and configuration. Read and complete
|
|
the following documents if you have not already:
|
|
|
|
* [Managing Kubernetes Objects Using Imperative Commands](/docs/tasks/manage-kubernetes-objects/imperative-command/)
|
|
* [Imperative Management of Kubernetes Objects Using Configuration Files](/docs/tasks/manage-kubernetes-objects/imperative-config/)
|
|
|
|
Following are definitions for terms used in this document:
|
|
|
|
- *object configuration file / configuration file*: A file that defines the
|
|
configuration for a Kubernetes object. This topic shows how to pass configuration
|
|
files to `kubectl apply`. Configuration files are typically stored in source control, such as Git.
|
|
- *live object configuration / live configuration*: The live configuration
|
|
values of an object, as observed by the Kubernetes cluster. These are kept in the Kubernetes
|
|
cluster storage, typically etcd.
|
|
- *declarative configuration writer / declarative writer*: A person or software component
|
|
that makes updates to a live object. The live writers referred to in this topic make changes
|
|
to object configuration files and run `kubectl apply` to write the changes.
|
|
|
|
## How to create objects
|
|
|
|
Use `kubectl apply` to create all objects, except those that already exist,
|
|
defined by configuration files in a specified directory:
|
|
|
|
```shell
|
|
kubectl apply -f <directory>
|
|
```
|
|
|
|
This sets the `kubectl.kubernetes.io/last-applied-configuration: '{...}'`
|
|
annotation on each object. The annotation contains the contents of the object
|
|
configuration file that was used to create the object.
|
|
|
|
{{< note >}}
|
|
Add the `-R` flag to recursively process directories.
|
|
{{< /note >}}
|
|
|
|
Here's an example of an object configuration file:
|
|
|
|
{{% code_sample file="application/simple_deployment.yaml" %}}
|
|
|
|
Run `kubectl diff` to print the object that will be created:
|
|
|
|
```shell
|
|
kubectl diff -f https://k8s.io/examples/application/simple_deployment.yaml
|
|
```
|
|
|
|
{{< note >}}
|
|
`diff` uses [server-side dry-run](/docs/reference/using-api/api-concepts/#dry-run),
|
|
which needs to be enabled on `kube-apiserver`.
|
|
|
|
Since `diff` performs a server-side apply request in dry-run mode,
|
|
it requires granting `PATCH`, `CREATE`, and `UPDATE` permissions.
|
|
See [Dry-Run Authorization](/docs/reference/using-api/api-concepts#dry-run-authorization)
|
|
for details.
|
|
|
|
{{< /note >}}
|
|
|
|
Create the object using `kubectl apply`:
|
|
|
|
```shell
|
|
kubectl apply -f https://k8s.io/examples/application/simple_deployment.yaml
|
|
```
|
|
|
|
Print the live configuration using `kubectl get`:
|
|
|
|
```shell
|
|
kubectl get -f https://k8s.io/examples/application/simple_deployment.yaml -o yaml
|
|
```
|
|
|
|
The output shows that the `kubectl.kubernetes.io/last-applied-configuration` annotation
|
|
was written to the live configuration, and it matches the configuration file:
|
|
|
|
```yaml
|
|
kind: Deployment
|
|
metadata:
|
|
annotations:
|
|
# ...
|
|
# This is the json representation of simple_deployment.yaml
|
|
# It was written by kubectl apply when the object was created
|
|
kubectl.kubernetes.io/last-applied-configuration: |
|
|
{"apiVersion":"apps/v1","kind":"Deployment",
|
|
"metadata":{"annotations":{},"name":"nginx-deployment","namespace":"default"},
|
|
"spec":{"minReadySeconds":5,"selector":{"matchLabels":{"app":nginx}},"template":{"metadata":{"labels":{"app":"nginx"}},
|
|
"spec":{"containers":[{"image":"nginx:1.14.2","name":"nginx",
|
|
"ports":[{"containerPort":80}]}]}}}}
|
|
# ...
|
|
spec:
|
|
# ...
|
|
minReadySeconds: 5
|
|
selector:
|
|
matchLabels:
|
|
# ...
|
|
app: nginx
|
|
template:
|
|
metadata:
|
|
# ...
|
|
labels:
|
|
app: nginx
|
|
spec:
|
|
containers:
|
|
- image: nginx:1.14.2
|
|
# ...
|
|
name: nginx
|
|
ports:
|
|
- containerPort: 80
|
|
# ...
|
|
# ...
|
|
# ...
|
|
# ...
|
|
```
|
|
|
|
## How to update objects
|
|
|
|
You can also use `kubectl apply` to update all objects defined in a directory, even
|
|
if those objects already exist. This approach accomplishes the following:
|
|
|
|
1. Sets fields that appear in the configuration file in the live configuration.
|
|
2. Clears fields removed from the configuration file in the live configuration.
|
|
|
|
```shell
|
|
kubectl diff -f <directory>
|
|
kubectl apply -f <directory>
|
|
```
|
|
|
|
{{< note >}}
|
|
Add the `-R` flag to recursively process directories.
|
|
{{< /note >}}
|
|
|
|
Here's an example configuration file:
|
|
|
|
{{% code_sample file="application/simple_deployment.yaml" %}}
|
|
|
|
Create the object using `kubectl apply`:
|
|
|
|
```shell
|
|
kubectl apply -f https://k8s.io/examples/application/simple_deployment.yaml
|
|
```
|
|
|
|
{{< note >}}
|
|
For purposes of illustration, the preceding command refers to a single
|
|
configuration file instead of a directory.
|
|
{{< /note >}}
|
|
|
|
Print the live configuration using `kubectl get`:
|
|
|
|
```shell
|
|
kubectl get -f https://k8s.io/examples/application/simple_deployment.yaml -o yaml
|
|
```
|
|
|
|
The output shows that the `kubectl.kubernetes.io/last-applied-configuration` annotation
|
|
was written to the live configuration, and it matches the configuration file:
|
|
|
|
```yaml
|
|
kind: Deployment
|
|
metadata:
|
|
annotations:
|
|
# ...
|
|
# This is the json representation of simple_deployment.yaml
|
|
# It was written by kubectl apply when the object was created
|
|
kubectl.kubernetes.io/last-applied-configuration: |
|
|
{"apiVersion":"apps/v1","kind":"Deployment",
|
|
"metadata":{"annotations":{},"name":"nginx-deployment","namespace":"default"},
|
|
"spec":{"minReadySeconds":5,"selector":{"matchLabels":{"app":nginx}},"template":{"metadata":{"labels":{"app":"nginx"}},
|
|
"spec":{"containers":[{"image":"nginx:1.14.2","name":"nginx",
|
|
"ports":[{"containerPort":80}]}]}}}}
|
|
# ...
|
|
spec:
|
|
# ...
|
|
minReadySeconds: 5
|
|
selector:
|
|
matchLabels:
|
|
# ...
|
|
app: nginx
|
|
template:
|
|
metadata:
|
|
# ...
|
|
labels:
|
|
app: nginx
|
|
spec:
|
|
containers:
|
|
- image: nginx:1.14.2
|
|
# ...
|
|
name: nginx
|
|
ports:
|
|
- containerPort: 80
|
|
# ...
|
|
# ...
|
|
# ...
|
|
# ...
|
|
```
|
|
|
|
Directly update the `replicas` field in the live configuration by using `kubectl scale`.
|
|
This does not use `kubectl apply`:
|
|
|
|
```shell
|
|
kubectl scale deployment/nginx-deployment --replicas=2
|
|
```
|
|
|
|
Print the live configuration using `kubectl get`:
|
|
|
|
```shell
|
|
kubectl get deployment nginx-deployment -o yaml
|
|
```
|
|
|
|
The output shows that the `replicas` field has been set to 2, and the `last-applied-configuration`
|
|
annotation does not contain a `replicas` field:
|
|
|
|
```yaml
|
|
apiVersion: apps/v1
|
|
kind: Deployment
|
|
metadata:
|
|
annotations:
|
|
# ...
|
|
# note that the annotation does not contain replicas
|
|
# because it was not updated through apply
|
|
kubectl.kubernetes.io/last-applied-configuration: |
|
|
{"apiVersion":"apps/v1","kind":"Deployment",
|
|
"metadata":{"annotations":{},"name":"nginx-deployment","namespace":"default"},
|
|
"spec":{"minReadySeconds":5,"selector":{"matchLabels":{"app":nginx}},"template":{"metadata":{"labels":{"app":"nginx"}},
|
|
"spec":{"containers":[{"image":"nginx:1.14.2","name":"nginx",
|
|
"ports":[{"containerPort":80}]}]}}}}
|
|
# ...
|
|
spec:
|
|
replicas: 2 # written by scale
|
|
# ...
|
|
minReadySeconds: 5
|
|
selector:
|
|
matchLabels:
|
|
# ...
|
|
app: nginx
|
|
template:
|
|
metadata:
|
|
# ...
|
|
labels:
|
|
app: nginx
|
|
spec:
|
|
containers:
|
|
- image: nginx:1.14.2
|
|
# ...
|
|
name: nginx
|
|
ports:
|
|
- containerPort: 80
|
|
# ...
|
|
```
|
|
|
|
Update the `simple_deployment.yaml` configuration file to change the image from
|
|
`nginx:1.14.2` to `nginx:1.16.1`, and delete the `minReadySeconds` field:
|
|
|
|
{{% code_sample file="application/update_deployment.yaml" %}}
|
|
|
|
Apply the changes made to the configuration file:
|
|
|
|
```shell
|
|
kubectl diff -f https://k8s.io/examples/application/update_deployment.yaml
|
|
kubectl apply -f https://k8s.io/examples/application/update_deployment.yaml
|
|
```
|
|
|
|
Print the live configuration using `kubectl get`:
|
|
|
|
```shell
|
|
kubectl get -f https://k8s.io/examples/application/update_deployment.yaml -o yaml
|
|
```
|
|
|
|
The output shows the following changes to the live configuration:
|
|
|
|
* The `replicas` field retains the value of 2 set by `kubectl scale`.
|
|
This is possible because it is omitted from the configuration file.
|
|
* The `image` field has been updated to `nginx:1.16.1` from `nginx:1.14.2`.
|
|
* The `last-applied-configuration` annotation has been updated with the new image.
|
|
* The `minReadySeconds` field has been cleared.
|
|
* The `last-applied-configuration` annotation no longer contains the `minReadySeconds` field.
|
|
|
|
```yaml
|
|
apiVersion: apps/v1
|
|
kind: Deployment
|
|
metadata:
|
|
annotations:
|
|
# ...
|
|
# The annotation contains the updated image to nginx 1.16.1,
|
|
# but does not contain the updated replicas to 2
|
|
kubectl.kubernetes.io/last-applied-configuration: |
|
|
{"apiVersion":"apps/v1","kind":"Deployment",
|
|
"metadata":{"annotations":{},"name":"nginx-deployment","namespace":"default"},
|
|
"spec":{"selector":{"matchLabels":{"app":nginx}},"template":{"metadata":{"labels":{"app":"nginx"}},
|
|
"spec":{"containers":[{"image":"nginx:1.16.1","name":"nginx",
|
|
"ports":[{"containerPort":80}]}]}}}}
|
|
# ...
|
|
spec:
|
|
replicas: 2 # Set by `kubectl scale`. Ignored by `kubectl apply`.
|
|
# minReadySeconds cleared by `kubectl apply`
|
|
# ...
|
|
selector:
|
|
matchLabels:
|
|
# ...
|
|
app: nginx
|
|
template:
|
|
metadata:
|
|
# ...
|
|
labels:
|
|
app: nginx
|
|
spec:
|
|
containers:
|
|
- image: nginx:1.16.1 # Set by `kubectl apply`
|
|
# ...
|
|
name: nginx
|
|
ports:
|
|
- containerPort: 80
|
|
# ...
|
|
# ...
|
|
# ...
|
|
# ...
|
|
```
|
|
|
|
{{< warning >}}
|
|
Mixing `kubectl apply` with the imperative object configuration commands
|
|
`create` and `replace` is not supported. This is because `create`
|
|
and `replace` do not retain the `kubectl.kubernetes.io/last-applied-configuration`
|
|
that `kubectl apply` uses to compute updates.
|
|
{{< /warning >}}
|
|
|
|
## How to delete objects
|
|
|
|
There are two approaches to delete objects managed by `kubectl apply`.
|
|
|
|
### Recommended: `kubectl delete -f <filename>`
|
|
|
|
Manually deleting objects using the imperative command is the recommended
|
|
approach, as it is more explicit about what is being deleted, and less likely
|
|
to result in the user deleting something unintentionally:
|
|
|
|
```shell
|
|
kubectl delete -f <filename>
|
|
```
|
|
|
|
### Alternative: `kubectl apply -f <directory> --prune`
|
|
|
|
As an alternative to `kubectl delete`, you can use `kubectl apply` to identify objects to be deleted after
|
|
their manifests have been removed from a directory in the local filesystem.
|
|
|
|
In Kubernetes {{< skew currentVersion >}}, there are two pruning modes available in kubectl apply:
|
|
|
|
- Allowlist-based pruning: This mode has existed since kubectl v1.5 but is still
|
|
in alpha due to usability, correctness and performance issues with its design.
|
|
The ApplySet-based mode is designed to replace it.
|
|
- ApplySet-based pruning: An _apply set_ is a server-side object (by default, a Secret)
|
|
that kubectl can use to accurately and efficiently track set membership across **apply**
|
|
operations. This mode was introduced in alpha in kubectl v1.27 as a replacement for allowlist-based pruning.
|
|
|
|
{{< tabs name="kubectl_apply_prune" >}}
|
|
{{% tab name="Allow list" %}}
|
|
|
|
{{< feature-state for_k8s_version="v1.5" state="alpha" >}}
|
|
|
|
{{< warning >}}
|
|
Take care when using `--prune` with `kubectl apply` in allow list mode. Which
|
|
objects are pruned depends on the values of the `--prune-allowlist`, `--selector`
|
|
and `--namespace` flags, and relies on dynamic discovery of the objects in scope.
|
|
Especially if flag values are changed between invocations, this can lead to objects
|
|
being unexpectedly deleted or retained.
|
|
{{< /warning >}}
|
|
|
|
To use allowlist-based pruning, add the following flags to your `kubectl apply` invocation:
|
|
|
|
- `--prune`: Delete previously applied objects that are not in the set passed to the current invocation.
|
|
- `--prune-allowlist`: A list of group-version-kinds (GVKs) to consider for pruning.
|
|
This flag is optional but strongly encouraged, as its default value is a partial
|
|
list of both namespaced and cluster-scoped types, which can lead to surprising results.
|
|
- `--selector/-l`: Use a label selector to constrain the set of objects selected
|
|
for pruning. This flag is optional but strongly encouraged.
|
|
- `--all`: use instead of `--selector/-l` to explicitly select all previously
|
|
applied objects of the allowlisted types.
|
|
|
|
Allowlist-based pruning queries the API server for all objects of the allowlisted GVKs that match the given labels (if any), and attempts to match the returned live object configurations against the object
|
|
manifest files. If an object matches the query, and it does not have a
|
|
manifest in the directory, and it has a `kubectl.kubernetes.io/last-applied-configuration` annotation,
|
|
it is deleted.
|
|
|
|
```shell
|
|
kubectl apply -f <directory> --prune -l <labels> --prune-allowlist=<gvk-list>
|
|
```
|
|
|
|
{{< warning >}}
|
|
Apply with prune should only be run against the root directory
|
|
containing the object manifests. Running against sub-directories
|
|
can cause objects to be unintentionally deleted if they were previously applied,
|
|
have the labels given (if any), and do not appear in the subdirectory.
|
|
{{< /warning >}}
|
|
|
|
{{% /tab %}}
|
|
|
|
{{% tab name="Apply set" %}}
|
|
|
|
{{< feature-state for_k8s_version="v1.27" state="alpha" >}}
|
|
|
|
{{< caution >}}
|
|
`kubectl apply --prune --applyset` is in alpha, and backwards incompatible
|
|
changes might be introduced in subsequent releases.
|
|
{{< /caution >}}
|
|
|
|
To use ApplySet-based pruning, set the `KUBECTL_APPLYSET=true` environment variable,
|
|
and add the following flags to your `kubectl apply` invocation:
|
|
- `--prune`: Delete previously applied objects that are not in the set passed
|
|
to the current invocation.
|
|
- `--applyset`: The name of an object that kubectl can use to accurately and
|
|
efficiently track set membership across `apply` operations.
|
|
|
|
```shell
|
|
KUBECTL_APPLYSET=true kubectl apply -f <directory> --prune --applyset=<name>
|
|
```
|
|
|
|
By default, the type of the ApplySet parent object used is a Secret. However,
|
|
ConfigMaps can also be used in the format: `--applyset=configmaps/<name>`.
|
|
When using a Secret or ConfigMap, kubectl will create the object if it does not already exist.
|
|
|
|
It is also possible to use custom resources as ApplySet parent objects. To enable
|
|
this, label the Custom Resource Definition (CRD) that defines the resource you want
|
|
to use with the following: `applyset.kubernetes.io/is-parent-type: true`. Then, create
|
|
the object you want to use as an ApplySet parent (kubectl does not do this automatically
|
|
for custom resources). Finally, refer to that object in the applyset flag as follows:
|
|
`--applyset=<resource>.<group>/<name>` (for example, `widgets.custom.example.com/widget-name`).
|
|
|
|
With ApplySet-based pruning, kubectl adds the `applyset.kubernetes.io/part-of=<parentID>`
|
|
label to each object in the set before they are sent to the server. For performance reasons,
|
|
it also collects the list of resource types and namespaces that the set contains and adds
|
|
these in annotations on the live parent object. Finally, at the end of the apply operation,
|
|
it queries the API server for objects of those types in those namespaces
|
|
(or in the cluster scope, as applicable) that belong to the set, as defined by the
|
|
`applyset.kubernetes.io/part-of=<parentID>` label.
|
|
|
|
Caveats and restrictions:
|
|
|
|
- Each object may be a member of at most one set.
|
|
- The `--namespace` flag is required when using any namespaced parent, including
|
|
the default Secret. This means that ApplySets spanning multiple namespaces must
|
|
use a cluster-scoped custom resource as the parent object.
|
|
- To safely use ApplySet-based pruning with multiple directories,
|
|
use a unique ApplySet name for each.
|
|
|
|
{{% /tab %}}
|
|
|
|
{{< /tabs >}}
|
|
|
|
## How to view an object
|
|
|
|
You can use `kubectl get` with `-o yaml` to view the configuration of a live object:
|
|
|
|
```shell
|
|
kubectl get -f <filename|url> -o yaml
|
|
```
|
|
|
|
## How apply calculates differences and merges changes
|
|
|
|
{{< caution >}}
|
|
A *patch* is an update operation that is scoped to specific fields of an object
|
|
instead of the entire object. This enables updating only a specific set of fields
|
|
on an object without reading the object first.
|
|
{{< /caution >}}
|
|
|
|
When `kubectl apply` updates the live configuration for an object,
|
|
it does so by sending a patch request to the API server. The
|
|
patch defines updates scoped to specific fields of the live object
|
|
configuration. The `kubectl apply` command calculates this patch request
|
|
using the configuration file, the live configuration, and the
|
|
`last-applied-configuration` annotation stored in the live configuration.
|
|
|
|
### Merge patch calculation
|
|
|
|
The `kubectl apply` command writes the contents of the configuration file to the
|
|
`kubectl.kubernetes.io/last-applied-configuration` annotation. This
|
|
is used to identify fields that have been removed from the configuration
|
|
file and need to be cleared from the live configuration. Here are the steps used
|
|
to calculate which fields should be deleted or set:
|
|
|
|
1. Calculate the fields to delete. These are the fields present in
|
|
`last-applied-configuration` and missing from the configuration file.
|
|
2. Calculate the fields to add or set. These are the fields present in
|
|
the configuration file whose values don't match the live configuration.
|
|
|
|
Here's an example. Suppose this is the configuration file for a Deployment object:
|
|
|
|
{{% code_sample file="application/update_deployment.yaml" %}}
|
|
|
|
Also, suppose this is the live configuration for the same Deployment object:
|
|
|
|
```yaml
|
|
apiVersion: apps/v1
|
|
kind: Deployment
|
|
metadata:
|
|
annotations:
|
|
# ...
|
|
# note that the annotation does not contain replicas
|
|
# because it was not updated through apply
|
|
kubectl.kubernetes.io/last-applied-configuration: |
|
|
{"apiVersion":"apps/v1","kind":"Deployment",
|
|
"metadata":{"annotations":{},"name":"nginx-deployment","namespace":"default"},
|
|
"spec":{"minReadySeconds":5,"selector":{"matchLabels":{"app":nginx}},"template":{"metadata":{"labels":{"app":"nginx"}},
|
|
"spec":{"containers":[{"image":"nginx:1.14.2","name":"nginx",
|
|
"ports":[{"containerPort":80}]}]}}}}
|
|
# ...
|
|
spec:
|
|
replicas: 2 # written by scale
|
|
# ...
|
|
minReadySeconds: 5
|
|
selector:
|
|
matchLabels:
|
|
# ...
|
|
app: nginx
|
|
template:
|
|
metadata:
|
|
# ...
|
|
labels:
|
|
app: nginx
|
|
spec:
|
|
containers:
|
|
- image: nginx:1.14.2
|
|
# ...
|
|
name: nginx
|
|
ports:
|
|
- containerPort: 80
|
|
# ...
|
|
```
|
|
|
|
Here are the merge calculations that would be performed by `kubectl apply`:
|
|
|
|
1. Calculate the fields to delete by reading values from
|
|
`last-applied-configuration` and comparing them to values in the
|
|
configuration file.
|
|
Clear fields explicitly set to null in the local object configuration file
|
|
regardless of whether they appear in the `last-applied-configuration`.
|
|
In this example, `minReadySeconds` appears in the
|
|
`last-applied-configuration` annotation, but does not appear in the configuration file.
|
|
**Action:** Clear `minReadySeconds` from the live configuration.
|
|
2. Calculate the fields to set by reading values from the configuration
|
|
file and comparing them to values in the live configuration. In this example,
|
|
the value of `image` in the configuration file does not match
|
|
the value in the live configuration. **Action:** Set the value of `image` in the live configuration.
|
|
3. Set the `last-applied-configuration` annotation to match the value
|
|
of the configuration file.
|
|
4. Merge the results from 1, 2, 3 into a single patch request to the API server.
|
|
|
|
Here is the live configuration that is the result of the merge:
|
|
|
|
```yaml
|
|
apiVersion: apps/v1
|
|
kind: Deployment
|
|
metadata:
|
|
annotations:
|
|
# ...
|
|
# The annotation contains the updated image to nginx 1.16.1,
|
|
# but does not contain the updated replicas to 2
|
|
kubectl.kubernetes.io/last-applied-configuration: |
|
|
{"apiVersion":"apps/v1","kind":"Deployment",
|
|
"metadata":{"annotations":{},"name":"nginx-deployment","namespace":"default"},
|
|
"spec":{"selector":{"matchLabels":{"app":nginx}},"template":{"metadata":{"labels":{"app":"nginx"}},
|
|
"spec":{"containers":[{"image":"nginx:1.16.1","name":"nginx",
|
|
"ports":[{"containerPort":80}]}]}}}}
|
|
# ...
|
|
spec:
|
|
selector:
|
|
matchLabels:
|
|
# ...
|
|
app: nginx
|
|
replicas: 2 # Set by `kubectl scale`. Ignored by `kubectl apply`.
|
|
# minReadySeconds cleared by `kubectl apply`
|
|
# ...
|
|
template:
|
|
metadata:
|
|
# ...
|
|
labels:
|
|
app: nginx
|
|
spec:
|
|
containers:
|
|
- image: nginx:1.16.1 # Set by `kubectl apply`
|
|
# ...
|
|
name: nginx
|
|
ports:
|
|
- containerPort: 80
|
|
# ...
|
|
# ...
|
|
# ...
|
|
# ...
|
|
```
|
|
|
|
### How different types of fields are merged
|
|
|
|
How a particular field in a configuration file is merged with
|
|
the live configuration depends on the
|
|
type of the field. There are several types of fields:
|
|
|
|
- *primitive*: A field of type string, integer, or boolean.
|
|
For example, `image` and `replicas` are primitive fields. **Action:** Replace.
|
|
|
|
- *map*, also called *object*: A field of type map or a complex type that contains subfields. For example, `labels`,
|
|
`annotations`,`spec` and `metadata` are all maps. **Action:** Merge elements or subfields.
|
|
|
|
- *list*: A field containing a list of items that can be either primitive types or maps.
|
|
For example, `containers`, `ports`, and `args` are lists. **Action:** Varies.
|
|
|
|
When `kubectl apply` updates a map or list field, it typically does
|
|
not replace the entire field, but instead updates the individual subelements.
|
|
For instance, when merging the `spec` on a Deployment, the entire `spec` is
|
|
not replaced. Instead the subfields of `spec`, such as `replicas`, are compared
|
|
and merged.
|
|
|
|
### Merging changes to primitive fields
|
|
|
|
Primitive fields are replaced or cleared.
|
|
|
|
{{< note >}}
|
|
`-` is used for "not applicable" because the value is not used.
|
|
{{< /note >}}
|
|
|
|
| Field in object configuration file | Field in live object configuration | Field in last-applied-configuration | Action |
|
|
|-------------------------------------|------------------------------------|-------------------------------------|-------------------------------------------|
|
|
| Yes | Yes | - | Set live to configuration file value. |
|
|
| Yes | No | - | Set live to local configuration. |
|
|
| No | - | Yes | Clear from live configuration. |
|
|
| No | - | No | Do nothing. Keep live value. |
|
|
|
|
### Merging changes to map fields
|
|
|
|
Fields that represent maps are merged by comparing each of the subfields or elements of the map:
|
|
|
|
{{< note >}}
|
|
`-` is used for "not applicable" because the value is not used.
|
|
{{< /note >}}
|
|
|
|
| Key in object configuration file | Key in live object configuration | Field in last-applied-configuration | Action |
|
|
|-------------------------------------|------------------------------------|-------------------------------------|----------------------------------|
|
|
| Yes | Yes | - | Compare sub fields values. |
|
|
| Yes | No | - | Set live to local configuration. |
|
|
| No | - | Yes | Delete from live configuration. |
|
|
| No | - | No | Do nothing. Keep live value. |
|
|
|
|
### Merging changes for fields of type list
|
|
|
|
Merging changes to a list uses one of three strategies:
|
|
|
|
* Replace the list if all its elements are primitives.
|
|
* Merge individual elements in a list of complex elements.
|
|
* Merge a list of primitive elements.
|
|
|
|
The choice of strategy is made on a per-field basis.
|
|
|
|
#### Replace the list if all its elements are primitives
|
|
|
|
Treat the list the same as a primitive field. Replace or delete the
|
|
entire list. This preserves ordering.
|
|
|
|
**Example:** Use `kubectl apply` to update the `args` field of a Container in a Pod. This sets
|
|
the value of `args` in the live configuration to the value in the configuration file.
|
|
Any `args` elements that had previously been added to the live configuration are lost.
|
|
The order of the `args` elements defined in the configuration file is
|
|
retained in the live configuration.
|
|
|
|
```yaml
|
|
# last-applied-configuration value
|
|
args: ["a", "b"]
|
|
|
|
# configuration file value
|
|
args: ["a", "c"]
|
|
|
|
# live configuration
|
|
args: ["a", "b", "d"]
|
|
|
|
# result after merge
|
|
args: ["a", "c"]
|
|
```
|
|
|
|
**Explanation:** The merge used the configuration file value as the new list value.
|
|
|
|
#### Merge individual elements of a list of complex elements:
|
|
|
|
Treat the list as a map, and treat a specific field of each element as a key.
|
|
Add, delete, or update individual elements. This does not preserve ordering.
|
|
|
|
This merge strategy uses a special tag on each field called a `patchMergeKey`. The
|
|
`patchMergeKey` is defined for each field in the Kubernetes source code:
|
|
[types.go](https://github.com/kubernetes/api/blob/d04500c8c3dda9c980b668c57abc2ca61efcf5c4/core/v1/types.go#L2747)
|
|
When merging a list of maps, the field specified as the `patchMergeKey` for a given element
|
|
is used like a map key for that element.
|
|
|
|
**Example:** Use `kubectl apply` to update the `containers` field of a PodSpec.
|
|
This merges the list as though it was a map where each element is keyed
|
|
by `name`.
|
|
|
|
```yaml
|
|
# last-applied-configuration value
|
|
containers:
|
|
- name: nginx
|
|
image: nginx:1.16
|
|
- name: nginx-helper-a # key: nginx-helper-a; will be deleted in result
|
|
image: helper:1.3
|
|
- name: nginx-helper-b # key: nginx-helper-b; will be retained
|
|
image: helper:1.3
|
|
|
|
# configuration file value
|
|
containers:
|
|
- name: nginx
|
|
image: nginx:1.16
|
|
- name: nginx-helper-b
|
|
image: helper:1.3
|
|
- name: nginx-helper-c # key: nginx-helper-c; will be added in result
|
|
image: helper:1.3
|
|
|
|
# live configuration
|
|
containers:
|
|
- name: nginx
|
|
image: nginx:1.16
|
|
- name: nginx-helper-a
|
|
image: helper:1.3
|
|
- name: nginx-helper-b
|
|
image: helper:1.3
|
|
args: ["run"] # Field will be retained
|
|
- name: nginx-helper-d # key: nginx-helper-d; will be retained
|
|
image: helper:1.3
|
|
|
|
# result after merge
|
|
containers:
|
|
- name: nginx
|
|
image: nginx:1.16
|
|
# Element nginx-helper-a was deleted
|
|
- name: nginx-helper-b
|
|
image: helper:1.3
|
|
args: ["run"] # Field was retained
|
|
- name: nginx-helper-c # Element was added
|
|
image: helper:1.3
|
|
- name: nginx-helper-d # Element was ignored
|
|
image: helper:1.3
|
|
```
|
|
|
|
**Explanation:**
|
|
|
|
- The container named "nginx-helper-a" was deleted because no container
|
|
named "nginx-helper-a" appeared in the configuration file.
|
|
- The container named "nginx-helper-b" retained the changes to `args`
|
|
in the live configuration. `kubectl apply` was able to identify
|
|
that "nginx-helper-b" in the live configuration was the same
|
|
"nginx-helper-b" as in the configuration file, even though their fields
|
|
had different values (no `args` in the configuration file). This is
|
|
because the `patchMergeKey` field value (name) was identical in both.
|
|
- The container named "nginx-helper-c" was added because no container
|
|
with that name appeared in the live configuration, but one with
|
|
that name appeared in the configuration file.
|
|
- The container named "nginx-helper-d" was retained because
|
|
no element with that name appeared in the last-applied-configuration.
|
|
|
|
#### Merge a list of primitive elements
|
|
|
|
As of Kubernetes 1.5, merging lists of primitive elements is not supported.
|
|
|
|
{{< note >}}
|
|
Which of the above strategies is chosen for a given field is controlled by
|
|
the `patchStrategy` tag in [types.go](https://github.com/kubernetes/api/blob/d04500c8c3dda9c980b668c57abc2ca61efcf5c4/core/v1/types.go#L2748)
|
|
If no `patchStrategy` is specified for a field of type list, then
|
|
the list is replaced.
|
|
{{< /note >}}
|
|
|
|
{{< comment >}}
|
|
TODO(pwittrock): Uncomment this for 1.6
|
|
|
|
- Treat the list as a set of primitives. Replace or delete individual
|
|
elements. Does not preserve ordering. Does not preserve duplicates.
|
|
|
|
**Example:** Using apply to update the `finalizers` field of ObjectMeta
|
|
keeps elements added to the live configuration. Ordering of finalizers
|
|
is lost.
|
|
{{< /comment >}}
|
|
|
|
## Default field values
|
|
|
|
The API server sets certain fields to default values in the live configuration if they are
|
|
not specified when the object is created.
|
|
|
|
Here's a configuration file for a Deployment. The file does not specify `strategy`:
|
|
|
|
{{% code_sample file="application/simple_deployment.yaml" %}}
|
|
|
|
Create the object using `kubectl apply`:
|
|
|
|
```shell
|
|
kubectl apply -f https://k8s.io/examples/application/simple_deployment.yaml
|
|
```
|
|
|
|
Print the live configuration using `kubectl get`:
|
|
|
|
```shell
|
|
kubectl get -f https://k8s.io/examples/application/simple_deployment.yaml -o yaml
|
|
```
|
|
|
|
The output shows that the API server set several fields to default values in the live
|
|
configuration. These fields were not specified in the configuration file.
|
|
|
|
```yaml
|
|
apiVersion: apps/v1
|
|
kind: Deployment
|
|
# ...
|
|
spec:
|
|
selector:
|
|
matchLabels:
|
|
app: nginx
|
|
minReadySeconds: 5
|
|
replicas: 1 # defaulted by apiserver
|
|
strategy:
|
|
rollingUpdate: # defaulted by apiserver - derived from strategy.type
|
|
maxSurge: 1
|
|
maxUnavailable: 1
|
|
type: RollingUpdate # defaulted by apiserver
|
|
template:
|
|
metadata:
|
|
creationTimestamp: null
|
|
labels:
|
|
app: nginx
|
|
spec:
|
|
containers:
|
|
- image: nginx:1.14.2
|
|
imagePullPolicy: IfNotPresent # defaulted by apiserver
|
|
name: nginx
|
|
ports:
|
|
- containerPort: 80
|
|
protocol: TCP # defaulted by apiserver
|
|
resources: {} # defaulted by apiserver
|
|
terminationMessagePath: /dev/termination-log # defaulted by apiserver
|
|
dnsPolicy: ClusterFirst # defaulted by apiserver
|
|
restartPolicy: Always # defaulted by apiserver
|
|
securityContext: {} # defaulted by apiserver
|
|
terminationGracePeriodSeconds: 30 # defaulted by apiserver
|
|
# ...
|
|
```
|
|
|
|
In a patch request, defaulted fields are not re-defaulted unless they are explicitly cleared
|
|
as part of a patch request. This can cause unexpected behavior for
|
|
fields that are defaulted based
|
|
on the values of other fields. When the other fields are later changed,
|
|
the values defaulted from them will not be updated unless they are
|
|
explicitly cleared.
|
|
|
|
For this reason, it is recommended that certain fields defaulted
|
|
by the server are explicitly defined in the configuration file, even
|
|
if the desired values match the server defaults. This makes it
|
|
easier to recognize conflicting values that will not be re-defaulted
|
|
by the server.
|
|
|
|
**Example:**
|
|
|
|
```yaml
|
|
# last-applied-configuration
|
|
spec:
|
|
template:
|
|
metadata:
|
|
labels:
|
|
app: nginx
|
|
spec:
|
|
containers:
|
|
- name: nginx
|
|
image: nginx:1.14.2
|
|
ports:
|
|
- containerPort: 80
|
|
|
|
# configuration file
|
|
spec:
|
|
strategy:
|
|
type: Recreate # updated value
|
|
template:
|
|
metadata:
|
|
labels:
|
|
app: nginx
|
|
spec:
|
|
containers:
|
|
- name: nginx
|
|
image: nginx:1.14.2
|
|
ports:
|
|
- containerPort: 80
|
|
|
|
# live configuration
|
|
spec:
|
|
strategy:
|
|
type: RollingUpdate # defaulted value
|
|
rollingUpdate: # defaulted value derived from type
|
|
maxSurge : 1
|
|
maxUnavailable: 1
|
|
template:
|
|
metadata:
|
|
labels:
|
|
app: nginx
|
|
spec:
|
|
containers:
|
|
- name: nginx
|
|
image: nginx:1.14.2
|
|
ports:
|
|
- containerPort: 80
|
|
|
|
# result after merge - ERROR!
|
|
spec:
|
|
strategy:
|
|
type: Recreate # updated value: incompatible with rollingUpdate
|
|
rollingUpdate: # defaulted value: incompatible with "type: Recreate"
|
|
maxSurge : 1
|
|
maxUnavailable: 1
|
|
template:
|
|
metadata:
|
|
labels:
|
|
app: nginx
|
|
spec:
|
|
containers:
|
|
- name: nginx
|
|
image: nginx:1.14.2
|
|
ports:
|
|
- containerPort: 80
|
|
```
|
|
|
|
**Explanation:**
|
|
|
|
1. The user creates a Deployment without defining `strategy.type`.
|
|
2. The server defaults `strategy.type` to `RollingUpdate` and defaults the
|
|
`strategy.rollingUpdate` values.
|
|
3. The user changes `strategy.type` to `Recreate`. The `strategy.rollingUpdate`
|
|
values remain at their defaulted values, though the server expects them to be cleared.
|
|
If the `strategy.rollingUpdate` values had been defined initially in the configuration file,
|
|
it would have been more clear that they needed to be deleted.
|
|
4. Apply fails because `strategy.rollingUpdate` is not cleared. The `strategy.rollingupdate`
|
|
field cannot be defined with a `strategy.type` of `Recreate`.
|
|
|
|
Recommendation: These fields should be explicitly defined in the object configuration file:
|
|
|
|
- Selectors and PodTemplate labels on workloads, such as Deployment, StatefulSet, Job, DaemonSet,
|
|
ReplicaSet, and ReplicationController
|
|
- Deployment rollout strategy
|
|
|
|
### How to clear server-defaulted fields or fields set by other writers
|
|
|
|
Fields that do not appear in the configuration file can be cleared by
|
|
setting their values to `null` and then applying the configuration file.
|
|
For fields defaulted by the server, this triggers re-defaulting
|
|
the values.
|
|
|
|
## How to change ownership of a field between the configuration file and direct imperative writers
|
|
|
|
These are the only methods you should use to change an individual object field:
|
|
|
|
- Use `kubectl apply`.
|
|
- Write directly to the live configuration without modifying the configuration file:
|
|
for example, use `kubectl scale`.
|
|
|
|
### Changing the owner from a direct imperative writer to a configuration file
|
|
|
|
Add the field to the configuration file. For the field, discontinue direct updates to
|
|
the live configuration that do not go through `kubectl apply`.
|
|
|
|
### Changing the owner from a configuration file to a direct imperative writer
|
|
|
|
As of Kubernetes 1.5, changing ownership of a field from a configuration file to
|
|
an imperative writer requires manual steps:
|
|
|
|
- Remove the field from the configuration file.
|
|
- Remove the field from the `kubectl.kubernetes.io/last-applied-configuration` annotation on the live object.
|
|
|
|
## Changing management methods
|
|
|
|
Kubernetes objects should be managed using only one method at a time.
|
|
Switching from one method to another is possible, but is a manual process.
|
|
|
|
{{< note >}}
|
|
It is OK to use imperative deletion with declarative management.
|
|
{{< /note >}}
|
|
|
|
{{< comment >}}
|
|
TODO(pwittrock): We need to make using imperative commands with
|
|
declarative object configuration work so that it doesn't write the
|
|
fields to the annotation, and instead. Then add this bullet point.
|
|
|
|
- using imperative commands with declarative configuration to manage where each manages different fields.
|
|
{{< /comment >}}
|
|
|
|
### Migrating from imperative command management to declarative object configuration
|
|
|
|
Migrating from imperative command management to declarative object
|
|
configuration involves several manual steps:
|
|
|
|
1. Export the live object to a local configuration file:
|
|
|
|
```shell
|
|
kubectl get <kind>/<name> -o yaml > <kind>_<name>.yaml
|
|
```
|
|
|
|
1. Manually remove the `status` field from the configuration file.
|
|
|
|
{{< note >}}
|
|
This step is optional, as `kubectl apply` does not update the status field
|
|
even if it is present in the configuration file.
|
|
{{< /note >}}
|
|
|
|
1. Set the `kubectl.kubernetes.io/last-applied-configuration` annotation on the object:
|
|
|
|
```shell
|
|
kubectl replace --save-config -f <kind>_<name>.yaml
|
|
```
|
|
|
|
1. Change processes to use `kubectl apply` for managing the object exclusively.
|
|
|
|
{{< comment >}}
|
|
TODO(pwittrock): Why doesn't export remove the status field? Seems like it should.
|
|
{{< /comment >}}
|
|
|
|
### Migrating from imperative object configuration to declarative object configuration
|
|
|
|
1. Set the `kubectl.kubernetes.io/last-applied-configuration` annotation on the object:
|
|
|
|
```shell
|
|
kubectl replace --save-config -f <kind>_<name>.yaml
|
|
```
|
|
|
|
1. Change processes to use `kubectl apply` for managing the object exclusively.
|
|
|
|
## Defining controller selectors and PodTemplate labels
|
|
|
|
{{< warning >}}
|
|
Updating selectors on controllers is strongly discouraged.
|
|
{{< /warning >}}
|
|
|
|
The recommended approach is to define a single, immutable PodTemplate label
|
|
used only by the controller selector with no other semantic meaning.
|
|
|
|
**Example:**
|
|
|
|
```yaml
|
|
selector:
|
|
matchLabels:
|
|
controller-selector: "apps/v1/deployment/nginx"
|
|
template:
|
|
metadata:
|
|
labels:
|
|
controller-selector: "apps/v1/deployment/nginx"
|
|
```
|
|
|
|
## {{% heading "whatsnext" %}}
|
|
|
|
* [Managing Kubernetes Objects Using Imperative Commands](/docs/tasks/manage-kubernetes-objects/imperative-command/)
|
|
* [Imperative Management of Kubernetes Objects Using Configuration Files](/docs/tasks/manage-kubernetes-objects/imperative-config/)
|
|
* [Kubectl Command Reference](/docs/reference/generated/kubectl/kubectl-commands/)
|
|
* [Kubernetes API Reference](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/)
|