fluxcd
/
cli-utils
mirror of https://github.com/fluxcd/cli-utils.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

chore: Update readme with features and packages

This commit is contained in:
Karl Isenberg 2022-06-09 19:08:49 -07:00
parent b00a8f8d45
commit 596a6f552d
1 changed files with 181 additions and 30 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

211
README.md
View File

@ -1,27 +1,38 @@
# cli-utils # cli-utils
The cli-utils repository contains an actuation library, which wraps `kubectl apply` code. `cli-utils` is a collection of Go libraries designed to facilitate bulk
This library allows importers to easily execute `kubectl apply`, while also actuation of Kubernetes resource objects by wraping and enahancing
addressing several deficiencies of the current implementation of `kubectl apply`. `kubectl apply` with a more user friendly abstraction.
The library enhances `kubectl apply` in the following ways:
1. **Pruning**: adds new, experimental automatic object deletion functionality. While the name incidates a focus on CLI utilities, the project has evolved to
2. **Sorting**: adds optional resource sorting functionality to apply or delete objects encompass a broader scope, including CLI use and server-side use in GitOps
in a particular order. controllers.
3. **Apply Time Mutation**: adds optional functionality to dynamically substitute fields
from one resource config into another.
TODO(seans): Add examples of API, once we've achieved an alpha API. ## Features
1. **Pruning**
1. **Status Interpretation**
1. **Status Lookup**
1. **Diff & Preview**
1. **Waiting for Reconciliation**
1. **Resource Ordering**
1. **Explicit Dependency Ordering**
1. **Implicit Dependency Ordering**
1. **Apply Time Mutation**
1. **CLI Printers**
### Pruning ### Pruning
The Applier automatically deletes objects that were previously applied and then
removed from the input set on a subsequent apply.
The current implementation of `kubectl apply --prune` uses labels to identify the The current implementation of `kubectl apply --prune` uses labels to identify the
set of previously applied objects in the prune set calculation. But the use of labels set of previously applied objects in the prune set calculation. But the use of labels
has significant downsides. The current `kubectl apply --prune` implemenation is alpha, has significant downsides. The current `kubectl apply --prune` implemenation is alpha,
and it is improbable that it will graduate to beta. This library attempts to address and it is improbable that it will graduate to beta. `cli-utils` attempts to address
the current `kubectl apply --prune` deficiencies by storing the set of previously the current `kubectl apply --prune` deficiencies by storing the set of previously
applied objects in an **inventory** object which is applied to the cluster. The applied objects in an **inventory** object which is applied to the cluster. The
**inventory** object is a `ConfigMap` with the `inventory-id` label, and references reference implimentation uses a `ConfigMap` as an **inventory** object, and references
to the applied objects are stored in the `data` section of the `ConfigMap`. to the applied objects are stored in the `data` section of the `ConfigMap`.
The following example illustrates a `ConfigMap` resource used as an inventory object: The following example illustrates a `ConfigMap` resource used as an inventory object:
@ -47,15 +58,72 @@ metadata:
cli-utils.sigs.k8s.io/inventory-id: 46d8946c-c1fa-4e1d-9357-b37fb9bae25f cli-utils.sigs.k8s.io/inventory-id: 46d8946c-c1fa-4e1d-9357-b37fb9bae25f
``` ```
### Apply Sort Ordering ### Status Interpretation
Adding an optional `config.kubernetes.io/depends-on: <OBJECT>` annotation to a The `kstatus` library can be used to read an object's current status and interpret
resource config provides apply ordering functionality. After manually specifying whether that object has be reconciled (aka Current) or not, including whether it
the dependency relationship among applied resources with this annotation, the is expected to never reconcile (aka Failed).
library will sort the resources and apply/prune them in the correct order.
Importantly, the library will wait for an object to reconcile successfully within ### Status Lookup
the cluster before applying dependent resources. Prune (deletion) ordering is
the opposite of apply ordering. In addition to performing interpritation of status from an object in-memory,
`cli-utils` can also be used to query status from the server, allowing you to
retrieve the status of previously or concurrently applied objects.
### Diff & Preview
`cli-utils` can be used to compare local object manifests with remote objects
from the server. These can be compared locally with diff or remotely with
preview (aka dry-run). This can be useful for discovering drift or previewing
which changes would be made, if the loal manifests were applied.
### Waiting for Reconciliation
The Applier automatically watches applied and deleted objects and tracks their
status, blocking until the objects have reconciled, failed, or been fully
deleted.
This functionality is similar to `kubectl delete <resource> <name> --wait`, in
that is waits for all finalizers to complete, except it also works for creates
and updates.
While there is a `kubectl apply <resource> <name> --wait`, it only waits for
deletes when combined with `--prune`. `cli-utils` provides an alternative that
works for all spec changes, waiting for reconciliation, the convergence of
status to the desired specification. After reconciliation, it is expected that
the object has reached a steady state until the specification is changed again.
### Resource Ordering
The Applier and Destroyer use resource type to determine which order to apply
and delete objects.
In contrast, when using `kubectl apply`, the objects are applied in alphanumeric
order of their file names, and top to bottom in each file. With `cli-utils`,
this manual sorting is unnecessary for many common use cases.
### Explicit Dependency Ordering
While resource ordering provides a smart default user experience, sometimes
resource type alone is not enough to determine desired ordering. In these cases,
the user can use explicit dependency ordering by adding a
`config.kubernetes.io/depends-on: <OBJECT_REFERENCE>` annotation to an object.
The Applier and Destroyer use these explicit dependency directives to build a
dependency tree and flatten it for determining apply ordering. When deleting,
the order is reversed, ensuring that dependencies are not deleted before the
objects that depend on them (aka dependents).
In addition to ordering the applies and deletes, dependency ordering also waits
for dependency reconciliation when applying and deletion finalization when
deleting. This ensures that dependencies are not just applied first, but have
reconciled before their dependents are applied. Likewise, dependents are not
just deleted first, but have completed finalization before their dependencies
are deleted.
Also, because dependency ordering is enforced during actuation, a dependency
cannot be pruned by the Applier unless all its dependents are also deleted. This
prevents accidental premature deletion of objects that are still in active use.
In the following example, the `config.kubernetes.io/depends-on` annotation In the following example, the `config.kubernetes.io/depends-on` annotation
identifies that `pod-c` must be successfully applied prior to `pod-a` identifies that `pod-c` must be successfully applied prior to `pod-a`
@ -74,19 +142,46 @@ spec:
image: k8s.gcr.io/pause:2.0 image: k8s.gcr.io/pause:2.0
``` ```
### Implicit Dependency Ordering
In addition to being able to specify explicit dependencies, `cli-utils`
automatically detects some implicit dependencies.
Implicit dependencies include:
1. Namespace-scoped resource objects depend on their Namespace.
2. Custom resource objects depend on their Custom Resource Definition
Like resource ordering, implicit dependency ordering improves the apply and
delete experience to reduce the need to manually specify ordering for many
common use cases. This allows more objects to be applied together all at once,
with less manual orchestration.
### Apply-Time Mutation ### Apply-Time Mutation
**apply-time mutation** functionality allows library users to dynamically fill in The Applier can dynamically modify objects before applying them, performing
resource field values from one object into another, even though they are applied field value substitution using input(s) from dependency fields.
at the same time. By adding a `config.kubernetes.io/apply-time-mutation` annotation,
a resource specifies the field in another object as well as the location for the
local field subsitution. For example, if an object's IP address is set during
actuation, another object applied at the same time can reference that IP address.
This functionality leverages the previously described **Apply Sort Ordering** to
ensure the source resource field is populated before applying the target resource.
In the following example, `pod-a` will substitute the IP address/port from the This allows for applying objects together in a set that you would otherwise need
source `pod-b` into the `pod-a` SERVICE_HOST environment variable: to seperate into multiple sets, with manual modifications between applies.
Apply-Time Mutation is configured using the
`config.kubernetes.io/apply-time-mutation` annotation on the target object to be
modified. The annotation may specify one or more substitutions. Each
substitution inncludes a source object, and source field path, and a parget
field path, with an optional token.
If the token is specified, the token is
replaced in the target field value string with the source field value. If the
token is not specified, the whole target field value is replaced with the
source field value. This alternatively allows either templated interpretation or
type preservation.
The source and target field paths are specified using JSONPath, allowing for
robust navigation of complex resource field hierarchies using a familiar syntax.
In the following example, `pod-a` will substitute the IP address and port from
the spec and status of the source `pod-b` into the spec of the target `pod-a`:
```yaml ```yaml
kind: Pod kind: Pod
@ -119,6 +214,62 @@ spec:
value: "${pob-b-ip}:${pob-b-port}" value: "${pob-b-ip}:${pob-b-port}"
``` ```
The primary reason to do this with Apply-Time Mutation, instead of client-side
manifest templating is that the pod IP is populated by a controller at runtime
during reconciliation, and is not known before applying.
That said, this is a toy example using built-in types. For pods, you probably
actually want to use DNS for service discovery instead.
Most use cases for Apply-Time Mutation are actually using custom resources, as a
temporary alternative to building higher level abstractions, modifying
interfaces, or creating dependencies between otherwise independent interfaces.
### CLI Printers
Since the original intent of `cli-utils` was to contain common code for CLIs,
and end-to-end testing requires a reference implimentation, a few printers are
included to translate from the primary event stream into STDOUT text:
1. **Event Printer**: The event printer just prints text to STDOT whenever an
event is recieved.
1. **JSON Printer**: The JSON printer converts events into a JSON string per
line, intended for automated interpretation by machine.
1. **Table Printer**: The table printer writes and updates in-place a table
with one object per line, intended for human consumption.
## Packages
├── **cmd**: the kapply CLI command
├── **examples**: examples that serve as additional end-to-end tests using mdrip
├── **hack**: hacky scripts used by make
├── **pkg**
│   ├── **apis**: API resources that satisfy the kubernetes Object interface
│   ├── **apply**: bulk applier and destroyer
│   ├── **common**: placeholder for common tools that should probably have their own package
│   ├── **config**: inventory config bootstrapping
│   ├── **errors**: error printing
│   ├── **flowcontrol**: flow control enablement discovery
│   ├── **inventory**: inventory resource reference implimentation
│   ├── **jsonpath**: utility for using jsonpath to read & write Unstructured object fields
│   ├── **kstatus**: object status event watcher with ability to reduce status to a single enum
│   ├── **manifestreader**: bolk resource object manifest reading and parsing
│   ├── **multierror**: error composition
│   ├── **object**: library for dealing with Unstructured objects
│   ├── **ordering**: sort functionality for objects
│   ├── **print**: CLI output
│   ├── **printers**: CLI output
│   └── **testutil**: utility for facilitating testing
├── **release**: goreleaser config
├── **scripts**: scripts used by make
└── **test**: end-to-end and stress tests
## kapply
To facilitate testing, this repository includes a reference CLI called `kapply`.
The `kapply` tool is not intended for direct consumer use, but may be useful
when trying to determine how to best utilize the `cli-utils` library packages.
## Community, discussion, contribution, and support ## Community, discussion, contribution, and support
Learn how to engage with the Kubernetes community on the [community page](http://kubernetes.io/community/). Learn how to engage with the Kubernetes community on the [community page](http://kubernetes.io/community/).