crossplane
/
docs
mirror of https://github.com/crossplane/docs.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
docs/content/v1.12/concepts/composition.md

1380 lines
54 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

---
title: Composite Resources
weight: 103
---
Crossplane Composite Resources are opinionated Kubernetes Custom Resources that
are _composed_ of [Managed Resources][managed-resources]. We often call them XRs
for short.
![Diagram of claims, XRs, and Managed Resources][xrs-and-mrs]
Composite Resources are designed to let you build your own platform with your
own opinionated concepts and APIs without needing to write a Kubernetes
controller from scratch. Instead, you define the schema of your XR and teach
Crossplane which Managed Resources it should compose (i.e. create) when someone
creates the XR you defined.
If you're already familiar with Composite Resources and looking for a detailed
configuration reference or some tips, tricks, and troubleshooting information,
try the [Composition Reference][xr-ref].
Below is an example of a Composite Resource:
```yaml
apiVersion: database.example.org/v1alpha1
kind: XPostgreSQLInstance
metadata:
name: my-db
spec:
parameters:
storageGB: 20
compositionRef:
name: production
writeConnectionSecretToRef:
namespace: crossplane-system
name: my-db-connection-details
```
You define your own XRs, so they can be of whatever API version and kind you
like, and contain whatever spec and status fields you need.
## How It Works
The first step towards using Composite Resources is configuring Crossplane so
that it knows what XRs you'd like to exist, and what to do when someone creates
one of those XRs. This is done using a `CompositeResourceDefinition` (XRD)
resource and one or more `Composition` resources.
Once you've configured Crossplane with the details of your new XR you can either
create one directly, or use a _claim_. Typically only the folks responsible for
configuring Crossplane (often a platform or SRE team) have permission to create
XRs directly. Everyone else manages XRs via a lightweight proxy resource called
a Composite Resource Claim (or claim for short). More on that later.
![Diagram combining all Composition concepts][how-it-works]
> If you're coming from the Terraform world you can think of an XRD as similar
> to the `variable` blocks of a Terraform module, while the `Composition` is
> the rest of the module's HCL code that describes how to use those variables to
> create a bunch of resources. In this analogy the XR or claim is a little like
> a `tfvars` file providing inputs to the module.
### Defining Composite Resources
A `CompositeResourceDefinition` (or XRD) defines the type and schema of your XR.
It lets Crossplane know that you want a particular kind of XR to exist, and what
fields that XR should have. An XRD is a little like a `CustomResourceDefinition`
(CRD), but slightly more opinionated. Writing an XRD is mostly a matter of
specifying an OpenAPI ["structural schema"][crd-docs].
The XRD that defines the `XPostgreSQLInstance` XR above would look like this:
```yaml
apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
name: xpostgresqlinstances.database.example.org
spec:
group: database.example.org
names:
kind: XPostgreSQLInstance
plural: xpostgresqlinstances
claimNames:
kind: PostgreSQLInstance
plural: postgresqlinstances
versions:
- name: v1alpha1
served: true
referenceable: true
schema:
openAPIV3Schema:
type: object
properties:
spec:
type: object
properties:
parameters:
type: object
properties:
storageGB:
type: integer
required:
- storageGB
required:
- parameters
```
You might notice that the `XPostgreSQLInstance` example above has some fields
that don't appear in the XRD, like the `writeConnectionSecretToRef` and
`compositionRef` fields. This is because Crossplane automatically injects some
standard Crossplane Resource Model (XRM) fields into all XRs.
### Configuring Composition
A `Composition` lets Crossplane know what to do when someone creates a Composite
Resource. Each `Composition` creates a link between an XR and a set of one or
more Managed Resources - when the XR is created, updated, or deleted the set of
Managed Resources are created, updated or deleted accordingly.
You can add multiple Compositions for each XRD, and choose which should be used
when XRs are created. This allows a Composition to act like a class of service -
for example you could configure one Composition for each environment you
support, such as production, staging, and development.
A basic `Composition` for the above `XPostgreSQLInstance` might look like this:
```yaml
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
name: example
labels:
crossplane.io/xrd: xpostgresqlinstances.database.example.org
provider: gcp
spec:
writeConnectionSecretsToNamespace: crossplane-system
compositeTypeRef:
apiVersion: database.example.org/v1alpha1
kind: XPostgreSQLInstance
resources:
- name: cloudsqlinstance
base:
apiVersion: database.gcp.crossplane.io/v1beta1
kind: CloudSQLInstance
spec:
forProvider:
databaseVersion: POSTGRES_12
region: us-central1
settings:
tier: db-custom-1-3840
dataDiskType: PD_SSD
ipConfiguration:
ipv4Enabled: true
authorizedNetworks:
- value: "0.0.0.0/0"
patches:
- type: FromCompositeFieldPath
fromFieldPath: spec.parameters.storageGB
toFieldPath: spec.forProvider.settings.dataDiskSizeGb
```
The above `Composition` tells Crossplane that when someone creates an
`XPostgreSQLInstance` XR Crossplane should create a `CloudSQLInstance` in
response. The `storageGB` field of the `XPostgreSQLInstance` should be used to
configure the `dataDiskSizeGb` field of the `CloudSQLInstance`. This is only a
small subset of the functionality a `Composition` enables - take a look at the
[reference page][xr-ref] to learn more.
> We almost always talk about XRs composing Managed Resources, but actually an
> XR can also compose other XRs to allow nested layers of abstraction. XRs don't
> support composing arbitrary Kubernetes resources (e.g. Deployments, operators,
> etc) directly but you can do so using our [Kubernetes][provider-kubernetes]
> and [Helm][provider-helm] providers.
### Claiming Composite Resources
Crossplane uses Composite Resource Claims (or just claims, for short) to allow
application operators to provision and manage XRs. When we talk about using XRs
it's typically implied that the XR is being used via a claim. Claims are almost
identical to their corresponding XRs. It helps to think of a claim as an
application team’s interface to an XR. You could also think of claims as the
public (app team) facing part of the opinionated platform API, while XRs are the
private (platform team) facing part.
A claim for the `XPostgreSQLInstance` XR above would look like this:
```yaml
apiVersion: database.example.org/v1alpha1
kind: PostgreSQLInstance
metadata:
namespace: default
name: my-db
spec:
parameters:
storageGB: 20
compositionRef:
name: production
writeConnectionSecretToRef:
name: my-db-connection-details
```
There are three key differences between an XR and a claim:
1. Claims are namespaced, while XRs (and Managed Resources) are cluster scoped.
1. Claims are of a different `kind` than the XR - by convention the XR's `kind`
without the proceeding `X`. For example a `PostgreSQLInstance` claims an
`XPostgreSQLInstance`.
1. An active claim contains a reference to its corresponding XR, while an XR
contains both a reference to the claim an array of references to the managed
resources it composes.
Not all XRs offer a claim - doing so is optional. See the XRD section of the
[Composition reference][xr-ref] to learn how to offer a claim.
![Diagram showing the relationship between claims and XRs][claims-and-xrs]
Claims may seem a little superfluous at first, but they enable some handy
scenarios, including:
- **Private XRs.** Sometimes a platform team might not want a type of XR to be
directly consumed by their application teams. For example because the XR
represents 'supporting' infrastructure - consider the above VPC `XNetwork` XR. App
teams might create `PostgreSQLInstance` claims that _reference_ (i.e. consume)
an `XNetwork`, but they shouldn't be _creating their own_. Similarly, some
kinds of XR might be intended only for 'nested' use - intended only to be
composed by other XRs.
- **Global XRs**. Not all infrastructure is conceptually namespaced. Say your
organisation uses team scoped namespaces. A `PostgreSQLInstance` that belongs
to Team A should probably be part of the `team-a` namespace - you'd represent
this by creating a `PostgreSQLInstance` claim in that namespace. On the other
hand the `XNetwork` XR we mentioned previously could be referenced (i.e. used)
by XRs from many different namespaces - it doesn't exist to serve a particular
team.
- **Pre-provisioned XRs**. Finally, separating claims from XRs allows a platform
team to pre-provision certain kinds of XR. Typically an XR is created
on-demand in response to the creation of a claim, but it's also possible for a
claim to instead request an existing XR. This can allow application teams to
instantly claim infrastructure like database instances that would otherwise
take minutes to provision on-demand.
This reference provides detailed examples of defining, configuring, and using
Composite Resources in Crossplane. You can also refer to Crossplane's [API
documentation][api-docs] for more details. If you're looking for a more general
overview of Composite Resources and Composition in Crossplane, try the
[Composite Resources][xr-concepts] page under Concepts.
## Composite Resources and Claims
The type and most of the schema of Composite Resources and claims are largely of
your own choosing, but there is some common 'machinery' injected into them.
Here's a hypothetical XR that doesn't have any user-defined fields and thus only
includes the automatically injected Crossplane machinery:
```yaml
apiVersion: database.example.org/v1alpha1
kind: XPostgreSQLInstance
metadata:
# This XR was created automatically by a claim, so its name is derived from
# the claim's name.
name: my-db-mfd1b
annotations:
# The external name annotation has special meaning in Crossplane. When a
# claim creates an XR its external name will automatically be propagated to
# the XR. Whether and how the external name is propagated to the resources
# the XR composes is up to its Composition.
crossplane.io/external-name: production-db-0
spec:
# XRs have a reference to the claim that created them (or, if the XR was
# pre-provisioned, to the claim that later claimed them).
claimRef:
apiVersion: database.example.org/v1alpha1
kind: PostgreSQLInstance
name: my-db
# The compositeDeletePolicy specifies the propagation policy that will be used by Crossplane
# when deleting the Composite Resource that is associated with the Claim. The default
# value is Background, which causes the Composite resource to be deleted using
# the kubernetes default propagation policy of Background, and all associated
# resources will be deleted simultaneously. The other value for this field is Foreground,
# which will cause the Composite resource to be deleted using Foreground Cascading Deletion.
# Kubernetes will add a foregroundDeletion finalizer to all of the resources in the
# dependency graph, and they will be deleted starting with the edge or leaf nodes and
# working back towards the root Composite. See https://kubernetes.io/docs/concepts/architecture/garbage-collection/#cascading-deletion
# for more information on cascading deletion.
compositeDeletePolicy: Background
# The compositionRef specifies which Composition this XR will use to compose
# resources when it is created, updated, or deleted. This can be omitted and
# will be set automatically if the XRD has a default or enforced composition
# reference, or if the below composition selector is set.
compositionRef:
name: production-us-east
# The compositionSelector allows you to match a Composition by labels rather
# than naming one explicitly. It is used to set the compositionRef if none is
# specified explicitly.
compositionSelector:
matchLabels:
environment: production
region: us-east
provider: gcp
# The environment is an in-memory object that can be patched from / to during
# rendering.
# The environment is composed by merging the 'data' of all EnvironmentConfigs
# referenced below. It is disposed after every reconcile.
# NOTE: EnvironmentConfigs are an alpha feature and need to be enabled with
# the '--enable-environment-configs' flag on startup.
environment:
# EnvironmentConfigs is a list of object references that is made up of
# name references and label selectors
environmentConfigs:
- type: Reference # this is the default
ref:
name: example-environment
- type: Selector
selector:
- key: stage
type: FromCompositeFieldPath # this is the default
valueFromFieldPath: spec.parameters.stage
- key: provider
type: Value
value: "gcp"
# The resourceRefs array contains references to all of the resources of which
# this XR is composed. Despite being in spec this field isn't intended to be
# configured by humans - Crossplane will take care of keeping it updated.
resourceRefs:
- apiVersion: database.gcp.crossplane.io/v1beta1
kind: CloudSQLInstance
name: my-db-mfd1b-md9ab
# The writeConnectionSecretToRef field specifies a Kubernetes Secret that this
# XR should write its connection details (if any) to.
writeConnectionSecretToRef:
namespace: crossplane-system
name: my-db-connection-details
status:
# An XR's 'Ready' condition will become True when all of the resources it
# composes are deemed ready. Refer to the Composition 'readinessChecks' field
# for more information.
conditions:
- type: Ready
statue: "True"
reason: Available
lastTransitionTime: 2021-10-02T07:20:50.52Z
# The last time the XR published its connection details to a Secret.
connectionDetails:
lastPublishedTime: 2021-10-02T07:20:51.24Z
```
Similarly, here's an example of the claim that corresponds to the above XR:
```yaml
apiVersion: database.example.org/v1alpha1
kind: PostgreSQLInstance
metadata:
# Claims are namespaced, unlike XRs.
namespace: default
name: my-db
annotations:
# The external name annotation has special meaning in Crossplane. When a
# claim creates an XR its external name will automatically be propagated to
# the XR. Whether and how the external name is propagated to the resources
# the XR composes is up to its Composition.
crossplane.io/external-name: production-db-0
spec:
# The resourceRef field references the XR this claim corresponds to. You can
# either set it to an existing (compatible) XR that you'd like to claim or
# (the more common approach) leave it blank and let Crossplane automatically
# create and reference an XR for you.
resourceRef:
apiVersion: database.example.org/v1alpha1
kind: XPostgreSQLInstance
name: my-db-mfd1b
# A claim's compositionRef and compositionSelector work the same way as an XR.
compositionRef:
name: production-us-east
compositionSelector:
matchLabels:
environment: production
region: us-east
provider: gcp
# A claim's writeConnectionSecretToRef mostly works the same way as an XR's.
# The one difference is that the Secret is always written to the namespace of
# the claim.
writeConnectionSecretToRef:
name: my-db-connection-details
status:
# A claim's 'Ready' condition will become True when its XR's 'Ready' condition
# becomes True.
conditions:
- type: Ready
statue: "True"
reason: Available
lastTransitionTime: 2021-10-02T07:20:50.52Z
# The last time the claim published its connection details to a Secret.
connectionDetails:
lastPublishedTime: 2021-10-02T07:20:51.24Z
```
> If your XR or claim isn't working as you'd expect you can try running `kubectl
> describe` against it for details - pay particular attention to any events and
> status conditions. You may need to follow the references from claim to XR to
> composed resources to find out what's happening.
## CompositeResourceDefinitions
Below is an example `CompositeResourceDefinition` that includes all configurable
fields.
```yaml
apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
# XRDs must be named '<plural>.<group>', per the plural and group names below.
name: xpostgresqlinstances.example.org
spec:
# This XRD defines an XR in the 'example.org' API group.
group: example.org
# The kind of this XR will be 'XPostgreSQLInstance`. You may also optionally
# specify a singular name and a listKind.
names:
kind: XPostgreSQLInstance
plural: xpostgresqlinstances
# This type of XR offers a claim. Omit claimNames if you don't want to do so.
# The claimNames must be different from the names above - a common convention
# is that names are prefixed with 'X' while claim names are not. This lets app
# team members think of creating a claim as (e.g.) 'creating a
# PostgreSQLInstance'.
claimNames:
kind: PostgreSQLInstance
plural: postgresqlinstances
# Each type of XR can declare any keys they write to their connection secret
# which will act as a filter during aggregation of the connection secret from
# composed resources. It's recommended to provide the set of keys here so that
# consumers of claims and XRs can see what to expect in the connection secret.
# If no key is given, then all keys in the aggregated connection secret will
# be written to the connection secret of the XR.
connectionSecretKeys:
- hostname
# Each type of XR may specify a default Composite Delete Policy to be used
# when the Claim has no compositeDeletePolicy. The valid values are Background
# and Foreground, and the default is Background. See the description of the
# compositeDeletePolicy parameter for more information.
defaultCompositeDeletePolicy: Background
# Each type of XR may specify a default Composition to be used when none is
# specified (e.g. when the XR has no compositionRef or selector). A similar
# enforceCompositionRef field also exists to allow XRs to enforce a specific
# Composition that should always be used.
defaultCompositionRef:
name: example
# Each type of XR may specify a default Composition Update Policy to be used
# when the Claim has no compositionUpdatePolicy. The valid values are Automatic
# and Manual and the default is Automatic.
defaultCompositionUpdatePolicy: Automatic
# Each type of XR may be served at different versions - e.g. v1alpha1, v1beta1
# and v1 - simultaneously. Currently Crossplane requires that all versions
# have an identical schema, so this is mostly useful to 'promote' a type of XR
# from alpha to beta to production ready.
versions:
- name: v1alpha1
# Served specifies that XRs should be served at this version. It can be set
# to false to temporarily disable a version, for example to test whether
# doing so breaks anything before a version is removed wholesale.
served: true
# Referenceable denotes the version of a type of XR that Compositions may
# use. Only one version may be referenceable.
referenceable: true
# Schema is an OpenAPI schema just like the one used by Kubernetes CRDs. It
# determines what fields your XR and claim will have. Note that Crossplane
# will automatically extend with some additional Crossplane machinery.
schema:
openAPIV3Schema:
type: object
properties:
spec:
type: object
properties:
parameters:
type: object
properties:
storageGB:
type: integer
required:
- storageGB
required:
- parameters
status:
type: object
properties:
address:
description: Address of this MySQL server.
type: string
```
Take a look at the Kubernetes [CRD documentation][crd-docs] for a more detailed
guide to writing OpenAPI schemas. Note that the following fields are reserved
for Crossplane machinery, and will be ignored if your schema includes them:
* `spec.resourceRef`
* `spec.resourceRefs`
* `spec.claimRef`
* `spec.writeConnectionSecretToRef`
* `status.conditions`
* `status.connectionDetails`
> If your `CompositeResourceDefinition` isn't working as you'd expect you can
> try running `kubectl describe xrd` for details - pay particular attention to
> any events and status conditions.
## Compositions
You'll encounter a lot of 'field paths' when reading or writing a `Composition`.
Field paths reference a field within a Kubernetes object via a simple string
'path'. [API conventions][field-paths] describe the syntax as:
> Standard JavaScript syntax for accessing that field, assuming the JSON object
> was transformed into a JavaScript object, without the leading dot, such as
> `metadata.name`.
Valid field paths include:
* `metadata.name` - The `name` field of the `metadata` object.
* `spec.containers[0].name` - The `name` field of the 0th `containers` element.
* `data[.config.yml]` - The `.config.yml` field of the `data` object.
* `apiVersion` - The `apiVersion` field of the root object.
While the following are invalid:
* `.metadata.name` - Leading period.
* `metadata..name` - Double period.
* `metadata.name.` - Trailing period.
* `spec.containers[]` - Empty brackets.
* `spec.containers.[0].name` - Period before open bracket.
Below is a detailed example of a `Composition`. While detailed, this example
doesn't include every patch, transform, connection detail, and readiness check
type. Keep reading below to discover those.
```yaml
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
name: example
labels:
# An optional convention is to include a label of the XRD. This allows
# easy discovery of compatible Compositions.
crossplane.io/xrd: xpostgresqlinstances.database.example.org
# The following label marks this Composition for GCP. This label can
# be used in 'compositionSelector' in an XR or Claim.
provider: gcp
spec:
# Each Composition must declare that it is compatible with a particular type
# of Composite Resource using its 'compositeTypeRef' field. The referenced
# version must be marked 'referenceable' in the XRD that defines the XR.
compositeTypeRef:
apiVersion: database.example.org/v1alpha1
kind: XPostgreSQLInstance
# When an XR is created in response to a claim Crossplane needs to know where
# it should create the XR's connection secret. This is configured using the
# 'writeConnectionSecretsToNamespace' field.
writeConnectionSecretsToNamespace: crossplane-system
# Each Composition must specify at least one composed resource template. In
# this case the Composition tells Crossplane that it should create, update, or
# delete a CloudSQLInstance whenever someone creates, updates, or deletes an
# XPostgresSQLInstance.
resources:
# It's good practice to provide a unique name for each entry. Note that
# this identifies the resources entry within the Composition - it's not
# the name the CloudSQLInstance. The 'name' field will be required in a
# future version of this API.
- name: cloudsqlinstance
# The 'base' template for the CloudSQLInstance Crossplane will create.
# You can use the base template to specify fields that never change, or
# default values for fields that may optionally be patched over. Bases must
# be a valid Crossplane resource - a Managed Resource, Composite Resource,
# or a ProviderConfig.
base:
apiVersion: database.gcp.crossplane.io/v1beta1
kind: CloudSQLInstance
spec:
forProvider:
databaseVersion: POSTGRES_12
region: us-central1
settings:
dataDiskType: PD_SSD
ipConfiguration:
ipv4Enabled: true
authorizedNetworks:
- value: "0.0.0.0/0"
# Each resource can optionally specify a set of 'patches' that copy fields
# from (or to) the XR.
patches:
# FromCompositeFieldPath is the default when 'type' is omitted, but it's
# good practice to always include the type for readability.
- type: FromCompositeFieldPath
fromFieldPath: spec.parameters.size
toFieldPath: spec.forProvider.settings.tier
# Each patch can optionally specify one or more 'transforms', which
# transform the 'from' field's value before applying it to the 'to' field.
# Transforms are applied in the order they are specified; each transform's
# output is passed to the following transform's input.
transforms:
- type: map
map:
medium: db-custom-1-3840
policy:
# By default a patch from a field path that does not exist is simply
# skipped until it does. Use the 'Required' policy to instead block and
# return an error when the field path does not exist.
fromFieldPath: Required
# You can patch entire objects or arrays from one resource to another.
# By default the 'to' object or array will be overwritten, not merged.
# Use the 'mergeOptions' field to override this behaviour. Note that
# these fields accidentally leak Go terminology - 'slice' means 'array'.
# 'map' means 'map' in YAML or 'object' in JSON.
mergeOptions:
appendSlice: true
keepMapValues: true
# You can include connection details to propagate from this CloudSQLInstance
# up to the XPostgreSQLInstance XR (and then on to the PostgreSQLInstance
# claim). Remember that your XRD must declare which connection secret keys
# it supports.
connectionDetails:
- name: hostname
fromConnectionSecretKey: hostname
# By default an XR's 'Ready' status condition will become True when the
# 'Ready' status conditions of all of its composed resources become true.
# You can optionally specify custom readiness checks to override this.
readinessChecks:
- type: None
# If you find yourself repeating patches a lot you can group them as a named
# 'patch set' then use a PatchSet type patch to reference them.
patchSets:
- name: metadata
patches:
- type: FromCompositeFieldPath
# When both field paths are the same you can omit the 'toFieldPath' and it
# will default to the 'fromFieldPath'.
fromFieldPath: metadata.labels[some-important-label]
```
### Pause Annotation
There is an annotation named `crossplane.io/paused` that you can use on
Composite Resources and Composite Resource Claims to temporarily pause
reconciliations of their respective controllers on them. An example
for a Composite Resource Claim is as follows:
```yaml
apiVersion: test.com/v1alpha1
kind: MyResource
metadata:
annotations:
crossplane.io/paused: "true"
namespace: upbound-system
name: my-resource
spec:
parameters:
tagValue: demo-test
compositionRef:
name: example
```
where `MyResource` is a Composite Resource Claim kind.
When a Composite Resource or a Claim has the `crossplane.io/paused` annotation
with its value set to `true`, the Composite Resource controller or the Claim
controller pauses reconciliations on the resource until
the annotation is removed or its value set to something other than `true`.
Before temporarily pausing reconciliations, an event with the type `Synced`,
the status `False`, and the reason `ReconcilePaused` is emitted
on the resource.
Please also note that annotations on a Composite Resource Claim are propagated
to the associated Composite Resource but when the
`crossplane.io/paused: "true"` annotation is added to a Claim, because
reconciliations on the Claim are now paused, this newly added annotation
will not be propagated. However, whenever the annotation's value is set to a
non-`true` value, reconciliations on the Claim will now resume, and thus the
annotation will now be propagated to the associated Composite Resource
with a non-`true` value. An implication of the described behavior is that
pausing reconciliations on the Claim will not inherently pause reconciliations
on the associated Composite Resource.
### Patch Types
You can use the following types of patch in a `Composition`:
`FromCompositeFieldPath`. The default if the `type` is omitted. This type
patches from a field within the XR to a field within the composed resource. It's
commonly used to expose a composed resource spec field as an XR spec field.
```yaml
# Patch from the XR's spec.parameters.size field to the composed resource's
# spec.forProvider.settings.tier field.
- type: FromCompositeFieldPath
fromFieldPath: spec.parameters.size
toFieldPath: spec.forProvider.settings.tier
```
`ToCompositeFieldPath`. The inverse of `FromCompositeFieldPath`. This type
patches from a field within the composed resource to a field within the XR. It's
commonly used to derive an XR status field from a composed resource status
field.
```yaml
# Patch from the composed resource's status.atProvider.zone field to the XR's
# status.zone field.
- type: ToCompositeFieldPath
fromFieldPath: status.atProvider.zone
toFieldPath: status.zone
```
`FromCompositeFieldPath` and `ToCompositeFieldPath` patches can also take a wildcarded
field path in the `toFieldPath` parameter and patch each array element in the `toFieldPath`
with the singular value provided in the `fromFieldPath`.
```yaml
# Patch from the XR's spec.parameters.allowedIPs to the CIDRBlock elements
# inside the array spec.forProvider.firewallRules on the composed resource.
resources:
- name: exampleFirewall
base:
apiVersion: firewall.example.crossplane.io/v1beta1
kind: Firewall
spec:
forProvider:
firewallRules:
- Action: "Allow"
Destination: "example1"
CIDRBlock: ""
- Action: "Allow"
Destination: "example2"
CIDRBlock: ""
- type: FromCompositeFieldPath
fromFieldPath: spec.parameters.allowedIP
toFieldPath: spec.forProvider.firewallRules[*].CIDRBlock
```
`FromEnvironmentFieldPath`. This type patches from a field within the in-memory
environment to a field within the composed resource. It's commonly used to
expose a composed resource spec field as an XR spec field.
Note that EnvironmentConfigs are an alpha feature and need to be enabled with
the `--enable-environment-configs` flag on startup.
```yaml
# Patch from the environment's tier.name field to the composed resource's
# spec.forProvider.settings.tier field.
- type: FromEnvironmentFieldPath
fromFieldPath: tier.name
toFieldPath: spec.forProvider.settings.tier
```
`ToEnvironmentFieldPath`. This type patches from a composed field to the
in-memory environment. Note that, unlike `ToCompositeFieldPath` patches, this
is executed before the composed resource is applied on the cluster which means
that the `status` is not available.
Note that EnvironmentConfigs are an alpha feature and need to be enabled with
the `--enable-environment-configs` flag on startup.
```yaml
# Patch from the environment's tier.name field to the composed resource's
# spec.forProvider.settings.tier field.
- type: ToEnvironmentFieldPath
fromFieldPath: spec.forProvider.settings.tier
toFieldPath: tier.name
```
Note that the field to be patched requires some initial value to be set.
`CombineFromComposite`. Combines multiple fields from the XR to produce one
composed resource field.
```yaml
# Patch from the XR's spec.parameters.location field and the
# metadata.labels[crossplane.io/claim-name] label to the composed
# resource's spec.forProvider.administratorLogin field.
- type: CombineFromComposite
combine:
# The patch will only be applied when all variables have non-zero values.
variables:
- fromFieldPath: spec.parameters.location
- fromFieldPath: metadata.labels[crossplane.io/claim-name]
strategy: string
string:
fmt: "%s-%s"
toFieldPath: spec.forProvider.administratorLogin
# By default Crossplane will skip the patch until all of the variables to be
# combined have values. Set the fromFieldPath policy to 'Required' to instead
# abort composition and return an error if a variable has no value.
policy:
fromFieldPath: Required
```
`CombineFromEnvironment`. Combines multiple fields from the in-memory
environment to produce one composed resource field.
Note that EnvironmentConfigs are an alpha feature and need to be enabled with
the `--enable-environment-configs` flag on startup.
```yaml
# Patch from the environments's location field and region to the composed
# resource's spec.forProvider.administratorLogin field.
- type: CombineFromEnvironment
combine:
# The patch will only be applied when all variables have non-zero values.
variables:
- fromFieldPath: location
- fromFieldPath: region
strategy: string
string:
fmt: "%s-%s"
toFieldPath: spec.forProvider.administratorLogin
```
At the time of writing only the `string` combine strategy is supported. It uses
[Go string formatting][pkg/fmt] to combine values, so if the XR's location was
`us-west` and its claim name was `db` the composed resource's administratorLogin
would be set to `us-west-db`.
`CombineToComposite` is the inverse of `CombineFromComposite`.
```yaml
# Patch from the composed resource's spec.parameters.administratorLogin and
# status.atProvider.fullyQualifiedDomainName fields back to the XR's
# status.adminDSN field.
- type: CombineToComposite
combine:
variables:
- fromFieldPath: spec.parameters.administratorLogin
- fromFieldPath: status.atProvider.fullyQualifiedDomainName
strategy: string
# Here, our administratorLogin parameter and fullyQualifiedDomainName
# status are formatted to a single output string representing a DSN.
string:
fmt: "mysql://%s@%s:3306/my-database-name"
toFieldPath: status.adminDSN
```
`CombineToEnvironment` is the inverse of `CombineFromEnvironment`.
Note that EnvironmentConfigs are an alpha feature and need to be enabled with
the `--enable-environment-configs` flag on startup.
```yaml
# Patch from the composed resource's spec.parameters.administratorLogin and
# spec.forProvider.domainName fields back to the environment's adminDSN field.
- type: CombineToEnvironment
combine:
variables:
- fromFieldPath: spec.parameters.administratorLogin
- fromFieldPath: spec.forProvider.domainName
strategy: string
# Here, our administratorLogin parameter and fullyQualifiedDomainName
# status are formatted to a single output string representing a DSN.
string:
fmt: "mysql://%s@%s:3306/my-database-name"
toFieldPath: adminDSN
```
`PatchSet`. References a named set of patches defined in the `spec.patchSets`
array of a `Composition`.
```yaml
# This is equivalent to specifying all of the patches included in the 'metadata'
# PatchSet.
- type: PatchSet
patchSetName: metadata
```
The `patchSets` array may not contain patches of `type: PatchSet`. The
`transforms` and `patchPolicy` fields are ignored by `type: PatchSet`.
### Transform Types
You can use the following types of transform on a value being patched:
`map`. Transforms values using a map.
```yaml
# If the value of the 'from' field is 'us-west', the value of the 'to' field
# will be set to 'West US'.
- type: map
map:
us-west: West US
us-east: East US
au-east: Australia East
```
`match`. A more complex version of `map` that can match different kinds of
patterns. It should be used if more advanced pattern matchings than a simple
string equality check are required.
The result of the first matching pattern is used as the output of this
transform.
If no pattern matches, you can either fallback to a given `fallbackValue` or
fallback to the input value by setting the `fallbackTo` field to `Input`.
```yaml
# In the example below, if the value in the 'from' field is 'us-west', the
# value in the 'to' field will be set to 'West US'.
# If the value in the 'from' field is 'eu-west', the value in the 'to' field
# will be set to 'Unknown' because no pattern matches.
- type: match
match:
patterns:
- type: literal # Not needed. This is the default.
literal: us-west
result: West US
- type: regexp
regexp: '^af-.*'
result: Somewhere in Africa
fallbackTo: Value # Not needed. This is the default.
fallbackValue: Unknown
# If fallbackTo is set to Input, the output will be the input value if no
# pattern matches.
# In the example below, if the value in the 'from' field is 'us-west', the
# value in the 'to' field will be set to 'West US'.
# If the value in the 'from' field is 'eu-west', the value in the 'to' field
# will be set to 'eu-west' because no pattern matches.
- type: match
match:
patterns:
- type: literal
literal: us-west
result: West US
- type: regexp
regexp: '^af-.*'
result: Somewhere in Africa
fallbackTo: Input
```
`math`. Transforms values using math. The input value must be an integer.
* math transform type `Multiply`, multiplies the input by the given value.
* math transform type `ClampMin`, sets a minimum value for the output.
* math transform type `ClampMax`, sets a maximum value for the output.
```yaml
# If you omit the field type, by default type is set to `Multiply`
# If the value of the 'from' field is 2, the value of the 'to' field will be set
# to 4.
- type: math
math:
multiply: 2
# This is the same as above
# If the value of the 'from' field is 2, the value of the 'to' field will be set
# to 4.
- type: math
math:
type: Multiply
multiply: 2
# If the value of the 'from' field is 3, the value of the 'to' field will
# be set to 4.
- type: math
math:
type: ClampMin
clampMin: 4
# If the value of the 'from' field is 3, the value of the 'to' field will
# be set to 2.
- type: math
math:
type: ClampMax
clampMax: 2
```
`string`. Transforms string values.
* string transform type `Format`, Currently only Go style fmt is supported. [Go style `fmt`][pkg/fmt] is supported.
* string transform type `Convert`, accepts one of `ToUpper`, `ToLower`, `ToBase64`, `FromBase64`, `ToJson`, `ToSha1`, `ToSha256`, `ToSha512`.
* string transform type `TrimPrefix`, accepts a string to be trimmed from the beginning of the input.
* string transform type `TrimSuffix`, accepts a string to be trimmed from the end of the input.
* string transform type `Regexp`, accepts a string for regexp to be applied to.
```yaml
# If you omit the field type, by default type is set to `Format`
# If the value of the 'from' field is 'hello', the value of the 'to' field will
# be set to 'hello-world'.
- type: string
string:
fmt: "%s-world"
# This is the same as above
# the value of the 'to' field will be set to 'hello-world'.
- type: string
string:
type: Format
fmt: "%s-world"
# If the value of the 'from' field is 'hello', the value of the 'to' field will
# be set to 'HELLO'.
- type: string
string:
type: Convert
convert: ToUpper
# If the value of the 'from' field is 'Hello', the value of the 'to' field will
# be set to 'hello'.
- type: string
string:
type: Convert
convert: ToLower
# If the value of the 'from' field is 'Hello', the value of the 'to' field will
# be set to 'SGVsbG8='.
- type: string
string:
type: Convert
convert: ToBase64
# If the value of the 'from' field is 'SGVsbG8=', the value of the 'to' field will
# be set to 'Hello'.
- type: string
string:
type: Convert
convert: FromBase64
# If the value of the 'from' field is not nil, the value of the 'to' field will be
# set to raw JSON representation of the 'from' field.
- type: string
string:
type: Convert
convert: ToJson
# The output will be the hash of the JSON representation of the 'from' field.
- type: string
string:
type: Convert
convert: ToSha1 # alternatives: 'ToSha256' or 'ToSha512'
# If the value of the 'from' field is https://crossplane.io, the value of the 'to' field will
# be set to crossplane.io
- type: string
string:
type: TrimPrefix
trim: 'https://'
# If the value of the 'from' field is my-string-test, the value of the 'to' field will
# be set to my-string
- type: string
string:
type: TrimSuffix
trim: '-test'
# If the value of the 'from' field is 'arn:aws:iam::42:example, the value of the
# 'to' field will be set to "42". Note that the 'to' field is always a string.
- type: string
string:
type: Regexp
regexp:
match: 'arn:aws:iam::(\d+):.*'
group: 1 # Optional capture group. Omit to match the entire regexp.
```
`convert`. Transforms values of one type to another, for example from a string
to an integer. The following values are supported by the `from` and `to` fields:
* `string`
* `bool`
* `int`
* `int64`
* `float64`
The strings 1, t, T, TRUE, true, and True are considered 'true', while 0, f, F,
FALSE, false, and False are considered 'false'. The integer 1 and float 1.0 are
considered true, while all other values are considered false. Similarly, boolean
true converts to integer 1 and float 1.0, while false converts to 0 and 0.0.
```yaml
# If the value to be converted is "1" (a string), the value of the 'toType'
# field will be set to 1 (an integer).
- type: convert
convert:
toType: int
```
Converting `string` to `float64` additionally supports parsing string in
[K8s quantity format](https://pkg.go.dev/k8s.io/apimachinery/pkg/api/resource#Quantity),
such as `1000m` or `500 Mi`:
```yaml
- type: convert
convert:
toType: float64
format: quantity
```
### Connection Details
Connection details secret of XR is an aggregated sum of the connection details
of the composed resources. It's recommended that the author of XRD specify
exactly which keys will be allowed in the XR connection secret by listing them
in `spec.connectionSecretKeys` so that consumers of claims and XRs can see what
they can expect in the connection details secret.
If `spec.connectionSecretKeys` is empty, then all keys of the aggregated connection
details secret will be propagated.
You can derive the following types of connection details from a composed
resource to be aggregated:
`FromConnectionSecretKey`. Derives an XR connection detail from a connection
secret key of a composed resource.
```yaml
# Derive the XR's 'user' connection detail from the 'username' key of the
# composed resource's connection secret.
- type: FromConnectionSecretKey
name: user
fromConnectionSecretKey: username
```
`FromFieldPath`. Derives an XR connection detail from a field path within the
composed resource.
```yaml
# Derive the XR's 'user' connection detail from the 'adminUser' status field of
# the composed resource.
- type: FromFieldPath
name: user
fromFieldPath: status.atProvider.adminUser
```
`FromValue`. Derives an XR connection detail from a fixed value.
```yaml
# Always sets the XR's 'user' connection detail to 'admin'.
- type: FromValue
name: user
value: admin
```
### Readiness Checks
Crossplane can use the following types of readiness check to determine whether a
composed resource is ready (and therefore whether the XR and claim should be
considered ready). Specify multiple readiness checks if multiple conditions must
be met for a composed resource to be considered ready.
> Note that if you don't specify any readiness checks Crossplane will consider
> the composed resource to be ready when its 'Ready' status condition becomes
> 'True'.
`MatchString`. Considers the composed resource to be ready when the value of a
field within that resource matches a specified string.
```yaml
# The composed resource will be considered ready when the 'state' status field
# matches the string 'Online'.
- type: MatchString
fieldPath: status.atProvider.state
matchString: "Online"
```
`MatchInteger`. Considers the composed resource to be ready when the value of a
field within that resource matches a specified integer.
```yaml
# The composed resource will be considered ready when the 'state' status field
# matches the integer 4.
- type: MatchInteger
fieldPath: status.atProvider.state
matchInteger: 4
```
`NonEmpty`. Considers the composed resource to be ready when a field exists in
the composed resource. The name of this check can be a little confusing in that
a field that exists with a zero value (e.g. an empty string or zero integer) is
not considered to be 'empty', and thus will pass the readiness check.
```yaml
# The composed resource will be considered ready if and when 'online' status
# field exists.
- type: NonEmpty
fieldPath: status.atProvider.online
```
`None`. Considers the composed resource to be ready as soon as it exists.
### Composition validation
Crossplane uses a `Validating Webhook` to inform users of any potential
errors in a `Composition`. By default webhooks only perform
`logical checks`. `logical checks` enforce requirements that
aren't explicitly defined in the schema but Crossplane assumes to hold at runtime.
#### Experimental validation with schemas
Enable experimental schema-aware validation in Crossplane
through the `--enable-composition-webhook-schema-validation` feature flag. This
enables Composition validation against available schemas in the cluster.
For example, ensuring that `fieldPaths` are valid and source and destination
types match taking into account provided transforms too.
The `crossplane.io/composition-validation-mode` annotation on the Composition
allows setting one of two modes for schema validation:
- `loose` (default): Validates Compositions against required schemas. If a
required schema is missing, schema validation stops, emits a warning and
falls back to `logical checks` only.
- `strict`: Validates Compositions against required schemas, and rejects them
when finding errors. Rejects any Compositions missing required schemas.
See the [Composition Validating Webhook design document][validation-design-doc]
for more information about future development around schema-aware validation.
#### Disabling webhooks
Crossplane enables webhooks by default. Turn off webhooks by
`webhooks.enabled` to `false` in the provided Helm Chart.
### Missing Functionality
You might find while reading through this reference that Crossplane is missing
some functionality you need to compose resources. If that's the case, please
[raise an issue] with as much detail **about your use case** as possible. Please
understand that the Crossplane maintainers are growing the feature set of the
`Composition` type conservatively. We highly value the input of our users and
community, but we also feel it's critical to avoid bloat and complexity. We
therefore wish to carefully consider each new addition. We feel some features
may be better suited for a real, expressive programming language and intend to
build an alternative to the `Composition` type as it's documented here per
[this proposal][issue-2524].
## Tips, Tricks, and Troubleshooting
In this section we'll cover some common tips, tricks, and troubleshooting steps
for working with Composite Resources. If you're trying to track down why your
Composite Resources aren't working the [Troubleshooting][trouble-ref] page also
has some useful information.
### Troubleshooting Claims and XRs
Crossplane relies heavily on status conditions and events for troubleshooting.
You can see both using `kubectl describe` - for example:
```console
# Describe the PostgreSQLInstance claim named my-db
kubectl describe postgresqlinstance.database.example.org my-db
```
Per Kubernetes convention, Crossplane keeps errors close to the place they
happen. This means that if your claim is not becoming ready due to an issue with
your `Composition` or with a composed resource you'll need to "follow the
references" to find out why. Your claim will only tell you that the XR is not
yet ready.
To follow the references:
1. Find your XR by running `kubectl describe` on your claim and looking for its
"Resource Ref" (aka `spec.resourceRef`).
1. Run `kubectl describe` on your XR. This is where you'll find out about issues
with the `Composition` you're using, if any.
1. If there are no issues but your XR doesn't seem to be becoming ready, take a
look for the "Resource Refs" (or `spec.resourceRefs`) to find your composed
resources.
1. Run `kubectl describe` on each referenced composed resource to determine
whether it is ready and what issues, if any, it is encountering.
### Composite Resource Connection Secrets
Claim and Composite Resource connection secrets are often derived from the
connection secrets of the managed resources they compose. This is a common
source of confusion because several things need to align for it to work:
1. The **claim** must specify the secret where the aggregated connection details
should be written
* This is the `spec.writeConnectionSecretToRef` field in a claim
* If creating a composite resource directly (without a claim) then this same
field must be set on your composite resource instead
1. The **composite resource definition** must state which connection details to
aggregate from its children to publish to the claim
* This is the `spec.connectionSecretKeys` field in a
`CompositeResourceDefinition`
1. The **composition** must define where to write its aggregated connection
details
* This is the `spec.writeConnectionSecretsToNamespace` field in the
`Composition`
1. Each child **composed resource** must define the connection details it
publishes and where to write them
* These are the `connectionDetails` and
`base.spec.writeConnectionSecretToRef` fields of the composed resources
Finally, you can't currently edit a XRD's supported connection details. The
XRD's `spec.connectionSecretKeys` is effectively immutable. This may change in
future per [this issue][issue-2024]
### Claiming an Existing Composite Resource
Most people create Composite Resources using a claim, but you can actually claim
an existing Composite Resource as long as its a type of XR that offers a claim
and no one else has already claimed it. To do so:
1. Set the `spec.resourceRef` of your claim to reference the existing XR.
```yaml
apiVersion: database.example.org/v1alpha1
kind: PostgreSQLInstance
metadata:
name: example
namespace: default
spec:
resourceRef:
apiVersion: database.example.org/v1alpha1
kind: XPostgreSQLInstance
name: example-d4lmv
```
1. Make sure the rest of your claim's spec fields match the XR's.
If your claim's spec fields don't match the XR's Crossplane will still claim it
but will then try to update the XR's spec fields to match the claim's.
### Influencing External Names
The `crossplane.io/external-name` annotation has special meaning to Crossplane
managed resources - it specifies the name (or identifier) of the resource in the
external system, for example the actual name of a `CloudSQLInstance` in the GCP
API. Some managed resources don't let you specify an external name - in those
cases Crossplane will set it for you to whatever the external system requires.
If you add the `crossplane.io/external-name` annotation to a claim Crossplane
will automatically propagate it when it creates an XR. It's good practice to
have your `Composition` further propagate the annotation to one or more composed
resources, but it's not required.
### Mixing and Matching Providers
Crossplane has providers for many things in addition to the big clouds. Take a
look at the [Upbound Marketplace][upbound-marketplace] to find many of them.
Keep in mind that you can mix and match managed resources from different
providers within a `Composition` to create Composite Resources. For example you
might use provider-aws and provider-sql to create an XR that provisions an
`RDSInstance` then creates an SQL `Database` and `User`, or provider-gcp and
provider-helm to create a `GKECluster` and deploy a Helm Chart `Release` to it.
Often when mixing and matching providers you'll need to compose a
`ProviderConfig` for one provider that loads credentials from the connection
secret of a managed resource from another provider. Sometimes you may need to
use an intermediary XR to mutate the connection details to suit your needs.
[This example][helm-and-gcp] from provider-helm demonstrates using a GKE cluster
connection secret as Helm `ProviderConfig` credentials.
### Patching From One Composed Resource to Another or Itself
It's not possible to patch _directly_ from one composed resource to another -
i.e. from one entry in the `spec.resources` array of a `Composition` to another.
It is however possible to achieve this by using the XR as an intermediary. To do
so:
1. Use a `ToCompositeFieldPath` patch to patch from your source composed
resource to the XR. Typically you'll want to patch to a status field or an
annotation.
1. Use a `FromCompositeFieldPath` patch to patch from the 'intermediary' field
you patched to in step 1 to a field on the destination composed resource.
Note that the source and the target composed resource can be the same.
[crd-docs]: https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/
[raise an issue]: https://github.com/crossplane/crossplane/issues/new?assignees=&labels=enhancement&template=feature_request.md
[issue-2524]: https://github.com/crossplane/crossplane/issues/2524
[field-paths]: https://github.com/kubernetes/community/blob/61f3d0/contributors/devel/sig-architecture/api-conventions.md#selecting-fields
[pkg/fmt]: https://pkg.go.dev/fmt
[upbound-marketplace]: https://marketplace.upbound.io
[helm-and-gcp]: https://github.com/crossplane-contrib/provider-helm/blob/2dcbdd0/examples/in-composition/composition.yaml
[issue-2024]: https://github.com/crossplane/crossplane/issues/2024
[xrs-and-mrs]: /media/composition-xrs-and-mrs.svg
[how-it-works]: /media/composition-how-it-works.svg
[provider-kubernetes]: https://marketplace.upbound.io/providers/crossplane-contrib/provider-kubernetes
[provider-helm]: https://marketplace.upbound.io/providers/crossplane-contrib/provider-helm/
[claims-and-xrs]: /media/composition-claims-and-xrs.svg
[xr-ref]: {{<ref "#compositions" >}}
[managed-resources]: {{<ref "managed-resources" >}}
[validation-design-doc]: https://github.com/crossplane/crossplane/blob/master/design/design-doc-composition-validating-webhook.md
Reference in New Issue View Git Blame Copy Permalink