fluxcd
/
kustomize-controller
mirror of https://github.com/fluxcd/kustomize-controller.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Add health checks and dependencies to API spec

This commit is contained in:
stefanprodan 2020-04-19 22:54:44 +03:00
parent 759fd5d6f5
commit 91fa7b856d
2 changed files with 259 additions and 78 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

81
README.md
View File

@ -2,18 +2,22 @@
[![e2e](https://github.com/fluxcd/kustomize-controller/workflows/e2e/badge.svg)](https://github.com/fluxcd/kustomize-controller/actions) [![e2e](https://github.com/fluxcd/kustomize-controller/workflows/e2e/badge.svg)](https://github.com/fluxcd/kustomize-controller/actions)
The kustomize-controller is a Kubernetes operator that applies [kustomizations](docs/spec/v1alpha1/README.md) in-cluster. The kustomize-controller is a continuous delivery (CD) tool for Kubernetes.
The controller runs CD pipelines inside the cluster for workloads and infrastructure manifests
coming from source control systems that are generated with Kustomize.
![overview](docs/diagrams/fluxcd-kustomize-source-controllers.png) ![overview](docs/diagrams/fluxcd-kustomize-source-controllers.png)
Features: Features:
* watches for `Kustomization` objects * watches for [Kustomization](docs/spec/v1alpha1/README.md) objects
* fetches artifacts produced by [source-controller](https://github.com/fluxcd/source-controller) from `Source` objects * fetches artifacts produced by [source-controller](https://github.com/fluxcd/source-controller) from `Source` objects
* watches `Source` objects for revision changes * watches `Source` objects for revision changes
* builds the kustomization using the latest fetched artifact * generates Kubernetes manifests with kustomize build
* validates the Kubernetes objects with client-side or APIServer dry-run * validates the build output with client-side or APIServer dry-run
* applies the resulting Kubernetes manifests on the cluster * applies the generated manifests on the cluster
* prunes the Kubernetes objects removed from source based on a label selector * prunes the Kubernetes objects removed from source
* checks the health of the deployed workloads
* runs `Kustomizations` in a specific order, taking into account the depends-on relationship
## Usage ## Usage
@ -73,7 +77,7 @@ kubectl annotate --overwrite gitrepository/podinfo source.fluxcd.io/syncAt="$(da
### Define a kustomization ### Define a kustomization
Create a [kustomization](docs/spec/v1alpha1/README.md) object that uses the git repository defined above: Create a kustomization object that uses the git repository defined above:
```yaml ```yaml
apiVersion: kustomize.fluxcd.io/v1alpha1 apiVersion: kustomize.fluxcd.io/v1alpha1
@ -88,18 +92,23 @@ spec:
kind: GitRepository kind: GitRepository
name: podinfo name: podinfo
validation: client validation: client
healthChecks:
- kind: Deployment
name: podinfo
namespace: dev
timeout: 80s
``` ```
With `spec.interval` we tell the controller how often it should reconcile the cluster state. A detailed explanation of the Kustomization object and its fields
With `spec.path` we tell the controller where to look for the `kustomization.yaml` file. can be found in the [specification doc](docs/spec/v1alpha1/README.md).
With `spec.prune` we configure [garbage collection](docs/spec/v1alpha1/README.md#garbage-collection).
With `spec.validation` we instruct the controller to validate the Kubernetes objects before applying them in-cluster.
When setting the validation to `server`, the controller will perform an
[APIServer dry-run](https://kubernetes.io/blog/2019/01/14/apiserver-dry-run-and-kubectl-diff/)
(requires Kubernetes >= 1.16).
Save the above file and apply it on the cluster. Based on the above definition, the kustomize-controller fetches the Git repository content from source-controller,
You can wait for the kustomize controller to apply the manifest corresponding to the dev overlay with: generates Kubernetes manifests by running kustomize build inside `./overlays/dev/`,
and validates them with a dry-run apply. If the manifests pass validation, the controller will apply them
on the cluster and starts the health assessment of the deployed workload. If the health checks are passing, the
Kustomization object status transitions to a ready state.
You can wait for the kustomize controller to complete the deployment with:
```bash ```bash
kubectl wait kustomization/podinfo-dev --for=condition=ready kubectl wait kustomization/podinfo-dev --for=condition=ready
@ -155,6 +164,46 @@ status:
} }
``` ```
### Define the order for kustomizations
When defining a kustomization, you may need to make sure other kustomizations have been
successfully applied beforehand. A kustomization can specify a list of dependencies with `spec.dependsOn`.
When combined with health assessment, a kustomization will run after all its dependencies health checks are passing.
For example, a service mesh proxy injector should be running before deploying applications inside the mesh:
```yaml
apiVersion: kustomize.fluxcd.io/v1alpha1
kind: Kustomization
metadata:
name: istio
spec:
interval: 10m
path: "./profiles/default/"
sourceRef:
kind: GitRepository
name: istio
healthChecks:
- kind: Deployment
name: istiod
namespace: istio-system
timeout: 2m
---
apiVersion: kustomize.fluxcd.io/v1alpha1
kind: Kustomization
metadata:
name: podinfo-dev
spec:
dependsOn:
- istio
interval: 5m
path: "./overlays/dev/"
prune: "env=dev"
sourceRef:
kind: GitRepository
name: podinfo
```
### Deploy releases to production ### Deploy releases to production
For production deployments, instead of synchronizing with a branch you can use a semver range to target stable releases: For production deployments, instead of synchronizing with a branch you can use a semver range to target stable releases:

256
docs/spec/v1alpha1/README.md
View File

@ -1,14 +1,22 @@
# Kustomization API # kustomize.fluxcd.io/v1alpha1
This is the v1alpha1 API specification for defining continuous delivery pipelines
of Kubernetes objects generated with Kustomize.
## Specification ## Specification
A **kustomization** object defines the source of Kubernetes manifests by referencing an object A **kustomization** object defines the source of Kubernetes manifests by referencing an object
managed by [source-controller](https://github.com/fluxcd/source-controller), managed by [source-controller](https://github.com/fluxcd/source-controller),
the path to the kustomization file within that source, the path to the kustomization file within that source,
and the interval at which the kustomization is applied on the cluster. and the interval at which the kustomize build output is applied on the cluster.
```go ```go
type KustomizationSpec struct { type KustomizationSpec struct {
// A list of kustomization that must be ready before this
// kustomization can be applied.
// +optional
DependsOn []string `json:"dependsOn,omitempty"`
// The interval at which to apply the kustomization. // The interval at which to apply the kustomization.
// +required // +required
Interval metav1.Duration `json:"interval"` Interval metav1.Duration `json:"interval"`
@ -23,6 +31,11 @@ type KustomizationSpec struct {
// +optional // +optional
Prune string `json:"prune,omitempty"` Prune string `json:"prune,omitempty"`
// A list of workloads (Deployments, DaemonSets and StatefulSets)
// to be included in the health assessment.
// +optional
HealthChecks []WorkloadReference `json:"healthChecks,omitempty"`
// Reference of the source where the kustomization file is. // Reference of the source where the kustomization file is.
// +required // +required
SourceRef corev1.TypedLocalObjectReference `json:"sourceRef"` SourceRef corev1.TypedLocalObjectReference `json:"sourceRef"`
@ -32,6 +45,11 @@ type KustomizationSpec struct {
// +optional // +optional
Suspend bool `json:"suspend,omitempty"` Suspend bool `json:"suspend,omitempty"`
// Timeout for validation, apply and health checking operations.
// Defaults to 'Interval' duration.
// +optional
Timeout *metav1.Duration `json:"timeout,omitempty"`
// Validate the Kubernetes objects before applying them on the cluster. // Validate the Kubernetes objects before applying them on the cluster.
// The validation strategy can be 'client' (local dry-run) or 'server' (APIServer dry-run). // The validation strategy can be 'client' (local dry-run) or 'server' (APIServer dry-run).
// +kubebuilder:validation:Enum=client;server // +kubebuilder:validation:Enum=client;server
@ -40,63 +58,7 @@ type KustomizationSpec struct {
} }
``` ```
### Kustomization execution The status sub-resource describes the result of the last kustomization execution:
The kustomization `spec.interval` tells the controller at which interval to fetch the Kubernetes manifest for the source,
build the kustomization and apply it on the cluster.
The interval time units are `s`, `m` and `h` e.g. `interval: 5m`, the minimum value should be over 60 seconds.
The kustomization execution can be suspended by setting `spec.susped` to `true`.
The controller can be told to execute the kustomization outside of the specified interval
by annotating the kustomization object with:
```go
const (
// SyncAtAnnotation is the annotation used for triggering a
// sync outside of the specified schedule.
SyncAtAnnotation string = "kustomize.fluxcd.io/syncAt"
)
```
On-demand execution example:
```bash
kubectl annotate --overwrite kustomization/podinfo kustomize.fluxcd.io/syncAt="$(date +%s)"
```
### Kustomization source
The kustomization `spec.sourceRef` is a reference to an object managed by
[source-controller](https://github.com/fluxcd/source-controller). When the source
[revision](https://github.com/fluxcd/source-controller/blob/master/docs/spec/v1alpha1/common.md#source-status)
changes, it creates a Kubernetes event that triggers a kustomization apply outside of the specified interval.
Source supported types:
* [GitRepository](https://github.com/fluxcd/source-controller/blob/master/docs/spec/v1alpha1/gitrepositories.md)
### Garbage collection
Garbage collection means that the Kubernetes objects that were previously applied on the cluster
but are missing from the current apply, will be removed. Garbage collection is also performed when a Kustomization
object is deleted, triggering a removal of all Kubernetes objects previously applied on the cluster.
When garbage collection is enabled, all Kubernetes objects must have a common label that matches the `spec.prune`
[label selector](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/).
For example, `prune: env=dev,app=frontend` requires a `kustomization.yaml` with `commonLabels`:
```yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
commonLabels:
env: dev
app: frontend
```
## Status
The status sub-resource describes the result of the last kustomization execution.
```go ```go
type KustomizationStatus struct { type KustomizationStatus struct {
@ -139,11 +101,181 @@ const (
) )
``` ```
Wait for condition example: ## Kustomization source
```bash The kustomization `spec.sourceRef` is a reference to an object managed by
kubectl wait kustomization/podinfo --for=condition=ready --timeout=1m [source-controller](https://github.com/fluxcd/source-controller). When the source
[revision](https://github.com/fluxcd/source-controller/blob/master/docs/spec/v1alpha1/common.md#source-status)
changes, it generates a Kubernetes event that triggers a kustomize build and apply.
Source supported types:
* [GitRepository](https://github.com/fluxcd/source-controller/blob/master/docs/spec/v1alpha1/gitrepositories.md)
> **Note** that the source must contain the kustomization.yaml and all the Kubernetes manifests and configuration files
> referenced in the kustomization.yaml.
## Kustomization execution
The kustomization `spec.interval` tells the controller at which interval to fetch the
Kubernetes manifest for the source, build the kustomization and apply it on the cluster.
The interval time units are `s`, `m` and `h` e.g. `interval: 5m`, the minimum value should be over 60 seconds.
The kustomization execution can be suspended by setting `spec.susped` to `true`.
The controller can be told to execute the kustomization outside of the specified interval
by annotating the kustomization object with:
```go
const (
// SyncAtAnnotation is the annotation used for triggering a
// sync outside of the specified schedule.
SyncAtAnnotation string = "kustomize.fluxcd.io/syncAt"
)
``` ```
On-demand execution example:
```bash
kubectl annotate --overwrite kustomization/podinfo kustomize.fluxcd.io/syncAt="$(date +%s)"
```
## Garbage collection
Garbage collection means that the Kubernetes objects that were previously applied on the cluster
but are missing from the current apply, are removed from cluster automatically.
Garbage collection is also performed when a Kustomization object is deleted,
triggering a removal of all Kubernetes objects previously applied on the cluster.
Tpo enable garbage collection, all Kubernetes objects must have a common label that matches the `spec.prune`
[label selector](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/).
For example, `prune: env=dev,app=frontend` requires a `kustomization.yaml` with `commonLabels`:
```yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
commonLabels:
env: dev
app: frontend
```
> **Note** that each kustomization must have a unique combination of labels values, otherwise the
> garbage collection will remove resources outside of the kustomization scope.
## Health assessment
A kustomization can contain a series of health checks used to determine the
[rollout status](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#deployment-status)
of the deployed workloads. A health check entry can reference one of the following Kubernetes types:
Deployment, DaemonSet or StatefulSet.
Assuming the kustomization source contains a Kubernetes Deployment named `backend`,
a health check can be defined as follows:
```yaml
apiVersion: kustomize.fluxcd.io/v1alpha1
kind: Kustomization
metadata:
name: backend
spec:
interval: 5m
path: "./webapp/backend/"
prune: "component=backend"
sourceRef:
kind: GitRepository
name: webapp
healthChecks:
- kind: Deployment
name: backend
namespace: dev
timeout: 2m
```
After applying the kustomize build output, the controller verifies if the rollout completed successfully.
If the deployment was successful, the kustomization ready condition is marked as `true`,
if the rollout failed, or if it takes more than the specified timeout to complete, then the
kustomization ready condition is set to `false`. If the deployment becomes healthy on the next
execution, then the kustomization is marked as ready.
## Kustomization dependencies
When running a kustomization, you may need to make sure other resources exist before the
workloads defined in your kustomization are deployed.
For example, a namespace must exist before applying resources to it.
With `spec.dependsOn` you can specify that the execution of a kustomization follows another.
When you add `dependsOn` entries to a kustomization, that kustomization is applied
only after all of its dependencies are ready. The readiness state of a kustomization is determined by
its last apply status condition.
Assuming two kustomizations: a `common` one, that contains a namespace definition and a `backend` one, that
contains the workload to be deployed in that namespace. You can instruct the controller to apply the `common`
kustomization first with:
```yaml
apiVersion: kustomize.fluxcd.io/v1alpha1
kind: Kustomization
metadata:
name: common
spec:
interval: 5m
path: "./webapp/common/"
prune: "part-of=webapp"
sourceRef:
kind: GitRepository
name: webapp
---
apiVersion: kustomize.fluxcd.io/v1alpha1
kind: Kustomization
metadata:
name: backend
spec:
dependsOn:
- common
interval: 5m
path: "./webapp/backend/"
prune: "part-of=webapp,component=backend"
sourceRef:
kind: GitRepository
name: webapp
```
When combined with health assessment, a kustomization will run after all its dependencies health checks are passing.
For example, a service mesh proxy injector should be running before deploying applications inside the mesh.
```yaml
apiVersion: kustomize.fluxcd.io/v1alpha1
kind: Kustomization
metadata:
name: istio
spec:
interval: 5m
path: "./profiles/default/"
sourceRef:
kind: GitRepository
name: istio
healthChecks:
- kind: Deployment
name: istiod
namespace: istio-system
timeout: 2m
---
apiVersion: kustomize.fluxcd.io/v1alpha1
kind: Kustomization
metadata:
name: backend
spec:
dependsOn:
- common
- istio
interval: 5m
path: "./webapp/backend/"
prune: "part-of=webapp,component=backend"
sourceRef:
kind: GitRepository
name: webapp
```
> **Note** that circular dependencies between kustomizations must be avoided, otherwise the
> interdependent kustomizations will never be applied on the cluster.