--- title: Composite Resources weight: 103 --- Crossplane Composite Resources are opinionated Kubernetes Custom Resources that are _composed_ of [Managed Resources][managed-resources]. We often call them XRs for short. ![Diagram of claims, XRs, and Managed Resources][xrs-and-mrs] Composite Resources are designed to let you build your own platform with your own opinionated concepts and APIs without needing to write a Kubernetes controller from scratch. Instead, you define the schema of your XR and teach Crossplane which Managed Resources it should compose (i.e. create) when someone creates the XR you defined. If you're already familiar with Composite Resources and looking for a detailed configuration reference or some tips, tricks, and troubleshooting information, try the [Composition Reference][xr-ref]. Below is an example of a Composite Resource: ```yaml apiVersion: database.example.org/v1alpha1 kind: XPostgreSQLInstance metadata: name: my-db spec: parameters: storageGB: 20 compositionRef: name: production writeConnectionSecretToRef: namespace: crossplane-system name: my-db-connection-details ``` You define your own XRs, so they can be of whatever API version and kind you like, and contain whatever spec and status fields you need. ## How It Works The first step towards using Composite Resources is configuring Crossplane so that it knows what XRs you'd like to exist, and what to do when someone creates one of those XRs. This is done using a `CompositeResourceDefinition` (XRD) resource and one or more `Composition` resources. Once you've configured Crossplane with the details of your new XR you can either create one directly, or use a _claim_. Typically only the folks responsible for configuring Crossplane (often a platform or SRE team) have permission to create XRs directly. Everyone else manages XRs via a lightweight proxy resource called a Composite Resource Claim (or claim for short). More on that later. ![Diagram combining all Composition concepts][how-it-works] > If you're coming from the Terraform world you can think of an XRD as similar > to the `variable` blocks of a Terraform module, while the `Composition` is > the rest of the module's HCL code that describes how to use those variables to > create a bunch of resources. In this analogy the XR or claim is a little like > a `tfvars` file providing inputs to the module. ### Defining Composite Resources A `CompositeResourceDefinition` (or XRD) defines the type and schema of your XR. It lets Crossplane know that you want a particular kind of XR to exist, and what fields that XR should have. An XRD is a little like a `CustomResourceDefinition` (CRD), but slightly more opinionated. Writing an XRD is mostly a matter of specifying an OpenAPI ["structural schema"][crd-docs]. The XRD that defines the `XPostgreSQLInstance` XR above would look like this: ```yaml apiVersion: apiextensions.crossplane.io/v1 kind: CompositeResourceDefinition metadata: name: xpostgresqlinstances.database.example.org spec: group: database.example.org names: kind: XPostgreSQLInstance plural: xpostgresqlinstances claimNames: kind: PostgreSQLInstance plural: postgresqlinstances versions: - name: v1alpha1 served: true referenceable: true schema: openAPIV3Schema: type: object properties: spec: type: object properties: parameters: type: object properties: storageGB: type: integer required: - storageGB required: - parameters ``` You might notice that the `XPostgreSQLInstance` example above has some fields that don't appear in the XRD, like the `writeConnectionSecretToRef` and `compositionRef` fields. This is because Crossplane automatically injects some standard Crossplane Resource Model (XRM) fields into all XRs. ### Configuring Composition A `Composition` lets Crossplane know what to do when someone creates a Composite Resource. Each `Composition` creates a link between an XR and a set of one or more Managed Resources - when the XR is created, updated, or deleted the set of Managed Resources are created, updated or deleted accordingly. You can add multiple Compositions for each XRD, and choose which should be used when XRs are created. This allows a Composition to act like a class of service - for example you could configure one Composition for each environment you support, such as production, staging, and development. A basic `Composition` for the above `XPostgreSQLInstance` might look like this: ```yaml apiVersion: apiextensions.crossplane.io/v1 kind: Composition metadata: name: example labels: crossplane.io/xrd: xpostgresqlinstances.database.example.org provider: gcp spec: writeConnectionSecretsToNamespace: crossplane-system compositeTypeRef: apiVersion: database.example.org/v1alpha1 kind: XPostgreSQLInstance resources: - name: cloudsqlinstance base: apiVersion: database.gcp.crossplane.io/v1beta1 kind: CloudSQLInstance spec: forProvider: databaseVersion: POSTGRES_12 region: us-central1 settings: tier: db-custom-1-3840 dataDiskType: PD_SSD ipConfiguration: ipv4Enabled: true authorizedNetworks: - value: "0.0.0.0/0" patches: - type: FromCompositeFieldPath fromFieldPath: spec.parameters.storageGB toFieldPath: spec.forProvider.settings.dataDiskSizeGb ``` The above `Composition` tells Crossplane that when someone creates an `XPostgreSQLInstance` XR Crossplane should create a `CloudSQLInstance` in response. The `storageGB` field of the `XPostgreSQLInstance` should be used to configure the `dataDiskSizeGb` field of the `CloudSQLInstance`. This is only a small subset of the functionality a `Composition` enables - take a look at the [reference page][xr-ref] to learn more. > We almost always talk about XRs composing Managed Resources, but actually an > XR can also compose other XRs to allow nested layers of abstraction. XRs don't > support composing arbitrary Kubernetes resources (e.g. Deployments, operators, > etc) directly but you can do so using our [Kubernetes][provider-kubernetes] > and [Helm][provider-helm] providers. ### Claiming Composite Resources Crossplane uses Composite Resource Claims (or just claims, for short) to allow application operators to provision and manage XRs. When we talk about using XRs it's typically implied that the XR is being used via a claim. Claims are almost identical to their corresponding XRs. It helps to think of a claim as an application team’s interface to an XR. You could also think of claims as the public (app team) facing part of the opinionated platform API, while XRs are the private (platform team) facing part. A claim for the `XPostgreSQLInstance` XR above would look like this: ```yaml apiVersion: database.example.org/v1alpha1 kind: PostgreSQLInstance metadata: namespace: default name: my-db spec: parameters: storageGB: 20 compositionRef: name: production writeConnectionSecretToRef: name: my-db-connection-details ``` There are three key differences between an XR and a claim: 1. Claims are namespaced, while XRs (and Managed Resources) are cluster scoped. 1. Claims are of a different `kind` than the XR - by convention the XR's `kind` without the proceeding `X`. For example a `PostgreSQLInstance` claims an `XPostgreSQLInstance`. 1. An active claim contains a reference to its corresponding XR, while an XR contains both a reference to the claim an array of references to the managed resources it composes. Not all XRs offer a claim - doing so is optional. See the XRD section of the [Composition reference][xr-ref] to learn how to offer a claim. ![Diagram showing the relationship between claims and XRs][claims-and-xrs] Claims may seem a little superfluous at first, but they enable some handy scenarios, including: - **Private XRs.** Sometimes a platform team might not want a type of XR to be directly consumed by their application teams. For example because the XR represents 'supporting' infrastructure - consider the above VPC `XNetwork` XR. App teams might create `PostgreSQLInstance` claims that _reference_ (i.e. consume) an `XNetwork`, but they shouldn't be _creating their own_. Similarly, some kinds of XR might be intended only for 'nested' use - intended only to be composed by other XRs. - **Global XRs**. Not all infrastructure is conceptually namespaced. Say your organisation uses team scoped namespaces. A `PostgreSQLInstance` that belongs to Team A should probably be part of the `team-a` namespace - you'd represent this by creating a `PostgreSQLInstance` claim in that namespace. On the other hand the `XNetwork` XR we mentioned previously could be referenced (i.e. used) by XRs from many different namespaces - it doesn't exist to serve a particular team. - **Pre-provisioned XRs**. Finally, separating claims from XRs allows a platform team to pre-provision certain kinds of XR. Typically an XR is created on-demand in response to the creation of a claim, but it's also possible for a claim to instead request an existing XR. This can allow application teams to instantly claim infrastructure like database instances that would otherwise take minutes to provision on-demand. This reference provides detailed examples of defining, configuring, and using Composite Resources in Crossplane. You can also refer to Crossplane's [API documentation][api-docs] for more details. If you're looking for a more general overview of Composite Resources and Composition in Crossplane, try the [Composite Resources][xr-concepts] page under Concepts. ## Composite Resources and Claims The type and most of the schema of Composite Resources and claims are largely of your own choosing, but there is some common 'machinery' injected into them. Here's a hypothetical XR that doesn't have any user-defined fields and thus only includes the automatically injected Crossplane machinery: ```yaml apiVersion: database.example.org/v1alpha1 kind: XPostgreSQLInstance metadata: # This XR was created automatically by a claim, so its name is derived from # the claim's name. name: my-db-mfd1b annotations: # The external name annotation has special meaning in Crossplane. When a # claim creates an XR its external name will automatically be propagated to # the XR. Whether and how the external name is propagated to the resources # the XR composes is up to its Composition. crossplane.io/external-name: production-db-0 spec: # XRs have a reference to the claim that created them (or, if the XR was # pre-provisioned, to the claim that later claimed them). claimRef: apiVersion: database.example.org/v1alpha1 kind: PostgreSQLInstance name: my-db # The compositeDeletePolicy specifies the propagation policy that will be used by Crossplane # when deleting the Composite Resource that is associated with the Claim. The default # value is Background, which causes the Composite resource to be deleted using # the kubernetes default propagation policy of Background, and all associated # resources will be deleted simultaneously. The other value for this field is Foreground, # which will cause the Composite resource to be deleted using Foreground Cascading Deletion. # Kubernetes will add a foregroundDeletion finalizer to all of the resources in the # dependency graph, and they will be deleted starting with the edge or leaf nodes and # working back towards the root Composite. See https://kubernetes.io/docs/concepts/architecture/garbage-collection/#cascading-deletion # for more information on cascading deletion. compositeDeletePolicy: Background # The compositionRef specifies which Composition this XR will use to compose # resources when it is created, updated, or deleted. This can be omitted and # will be set automatically if the XRD has a default or enforced composition # reference, or if the below composition selector is set. compositionRef: name: production-us-east # The compositionSelector allows you to match a Composition by labels rather # than naming one explicitly. It is used to set the compositionRef if none is # specified explicitly. compositionSelector: matchLabels: environment: production region: us-east provider: gcp # The environment is an in-memory object that can be patched from / to during # rendering. # The environment is composed by merging the 'data' of all EnvironmentConfigs # referenced below. It is disposed after every reconcile. # NOTE: EnvironmentConfigs are an alpha feature and need to be enabled with # the '--enable-environment-configs' flag on startup. environment: # EnvironmentConfigs is a list of object references that is made up of # name references and label selectors environmentConfigs: - type: Reference # this is the default ref: name: example-environment - type: Selector selector: - key: stage type: FromCompositeFieldPath # this is the default valueFromFieldPath: spec.parameters.stage - key: provider type: Value value: "gcp" # The resourceRefs array contains references to all of the resources of which # this XR is composed. Despite being in spec this field isn't intended to be # configured by humans - Crossplane will take care of keeping it updated. resourceRefs: - apiVersion: database.gcp.crossplane.io/v1beta1 kind: CloudSQLInstance name: my-db-mfd1b-md9ab # The writeConnectionSecretToRef field specifies a Kubernetes Secret that this # XR should write its connection details (if any) to. writeConnectionSecretToRef: namespace: crossplane-system name: my-db-connection-details status: # An XR's 'Ready' condition will become True when all of the resources it # composes are deemed ready. Refer to the Composition 'readinessChecks' field # for more information. conditions: - type: Ready statue: "True" reason: Available lastTransitionTime: 2021-10-02T07:20:50.52Z # The last time the XR published its connection details to a Secret. connectionDetails: lastPublishedTime: 2021-10-02T07:20:51.24Z ``` Similarly, here's an example of the claim that corresponds to the above XR: ```yaml apiVersion: database.example.org/v1alpha1 kind: PostgreSQLInstance metadata: # Claims are namespaced, unlike XRs. namespace: default name: my-db annotations: # The external name annotation has special meaning in Crossplane. When a # claim creates an XR its external name will automatically be propagated to # the XR. Whether and how the external name is propagated to the resources # the XR composes is up to its Composition. crossplane.io/external-name: production-db-0 spec: # The resourceRef field references the XR this claim corresponds to. You can # either set it to an existing (compatible) XR that you'd like to claim or # (the more common approach) leave it blank and let Crossplane automatically # create and reference an XR for you. resourceRef: apiVersion: database.example.org/v1alpha1 kind: XPostgreSQLInstance name: my-db-mfd1b # A claim's compositionRef and compositionSelector work the same way as an XR. compositionRef: name: production-us-east compositionSelector: matchLabels: environment: production region: us-east provider: gcp # A claim's writeConnectionSecretToRef mostly works the same way as an XR's. # The one difference is that the Secret is always written to the namespace of # the claim. writeConnectionSecretToRef: name: my-db-connection-details status: # A claim's 'Ready' condition will become True when its XR's 'Ready' condition # becomes True. conditions: - type: Ready statue: "True" reason: Available lastTransitionTime: 2021-10-02T07:20:50.52Z # The last time the claim published its connection details to a Secret. connectionDetails: lastPublishedTime: 2021-10-02T07:20:51.24Z ``` > If your XR or claim isn't working as you'd expect you can try running `kubectl > describe` against it for details - pay particular attention to any events and > status conditions. You may need to follow the references from claim to XR to > composed resources to find out what's happening. ## CompositeResourceDefinitions Below is an example `CompositeResourceDefinition` that includes all configurable fields. ```yaml apiVersion: apiextensions.crossplane.io/v1 kind: CompositeResourceDefinition metadata: # XRDs must be named '.', per the plural and group names below. name: xpostgresqlinstances.example.org spec: # This XRD defines an XR in the 'example.org' API group. group: example.org # The kind of this XR will be 'XPostgreSQLInstance`. You may also optionally # specify a singular name and a listKind. names: kind: XPostgreSQLInstance plural: xpostgresqlinstances # This type of XR offers a claim. Omit claimNames if you don't want to do so. # The claimNames must be different from the names above - a common convention # is that names are prefixed with 'X' while claim names are not. This lets app # team members think of creating a claim as (e.g.) 'creating a # PostgreSQLInstance'. claimNames: kind: PostgreSQLInstance plural: postgresqlinstances # Each type of XR can declare any keys they write to their connection secret # which will act as a filter during aggregation of the connection secret from # composed resources. It's recommended to provide the set of keys here so that # consumers of claims and XRs can see what to expect in the connection secret. # If no key is given, then all keys in the aggregated connection secret will # be written to the connection secret of the XR. connectionSecretKeys: - hostname # Each type of XR may specify a default Composite Delete Policy to be used # when the Claim has no compositeDeletePolicy. The valid values are Background # and Foreground, and the default is Background. See the description of the # compositeDeletePolicy parameter for more information. defaultCompositeDeletePolicy: Background # Each type of XR may specify a default Composition to be used when none is # specified (e.g. when the XR has no compositionRef or selector). A similar # enforceCompositionRef field also exists to allow XRs to enforce a specific # Composition that should always be used. defaultCompositionRef: name: example # Each type of XR may specify a default Composition Update Policy to be used # when the Claim has no compositionUpdatePolicy. The valid values are Automatic # and Manual and the default is Automatic. defaultCompositionUpdatePolicy: Automatic # Each type of XR may be served at different versions - e.g. v1alpha1, v1beta1 # and v1 - simultaneously. Currently Crossplane requires that all versions # have an identical schema, so this is mostly useful to 'promote' a type of XR # from alpha to beta to production ready. versions: - name: v1alpha1 # Served specifies that XRs should be served at this version. It can be set # to false to temporarily disable a version, for example to test whether # doing so breaks anything before a version is removed wholesale. served: true # Referenceable denotes the version of a type of XR that Compositions may # use. Only one version may be referenceable. referenceable: true # Schema is an OpenAPI schema just like the one used by Kubernetes CRDs. It # determines what fields your XR and claim will have. Note that Crossplane # will automatically extend with some additional Crossplane machinery. schema: openAPIV3Schema: type: object properties: spec: type: object properties: parameters: type: object properties: storageGB: type: integer required: - storageGB required: - parameters status: type: object properties: address: description: Address of this MySQL server. type: string ``` Take a look at the Kubernetes [CRD documentation][crd-docs] for a more detailed guide to writing OpenAPI schemas. Note that the following fields are reserved for Crossplane machinery, and will be ignored if your schema includes them: * `spec.resourceRef` * `spec.resourceRefs` * `spec.claimRef` * `spec.writeConnectionSecretToRef` * `status.conditions` * `status.connectionDetails` > If your `CompositeResourceDefinition` isn't working as you'd expect you can > try running `kubectl describe xrd` for details - pay particular attention to > any events and status conditions. ## Compositions You'll encounter a lot of 'field paths' when reading or writing a `Composition`. Field paths reference a field within a Kubernetes object via a simple string 'path'. [API conventions][field-paths] describe the syntax as: > Standard JavaScript syntax for accessing that field, assuming the JSON object > was transformed into a JavaScript object, without the leading dot, such as > `metadata.name`. Valid field paths include: * `metadata.name` - The `name` field of the `metadata` object. * `spec.containers[0].name` - The `name` field of the 0th `containers` element. * `data[.config.yml]` - The `.config.yml` field of the `data` object. * `apiVersion` - The `apiVersion` field of the root object. While the following are invalid: * `.metadata.name` - Leading period. * `metadata..name` - Double period. * `metadata.name.` - Trailing period. * `spec.containers[]` - Empty brackets. * `spec.containers.[0].name` - Period before open bracket. Below is a detailed example of a `Composition`. While detailed, this example doesn't include every patch, transform, connection detail, and readiness check type. Keep reading below to discover those. ```yaml apiVersion: apiextensions.crossplane.io/v1 kind: Composition metadata: name: example labels: # An optional convention is to include a label of the XRD. This allows # easy discovery of compatible Compositions. crossplane.io/xrd: xpostgresqlinstances.database.example.org # The following label marks this Composition for GCP. This label can # be used in 'compositionSelector' in an XR or Claim. provider: gcp spec: # Each Composition must declare that it is compatible with a particular type # of Composite Resource using its 'compositeTypeRef' field. The referenced # version must be marked 'referenceable' in the XRD that defines the XR. compositeTypeRef: apiVersion: database.example.org/v1alpha1 kind: XPostgreSQLInstance # When an XR is created in response to a claim Crossplane needs to know where # it should create the XR's connection secret. This is configured using the # 'writeConnectionSecretsToNamespace' field. writeConnectionSecretsToNamespace: crossplane-system # Each Composition must specify at least one composed resource template. In # this case the Composition tells Crossplane that it should create, update, or # delete a CloudSQLInstance whenever someone creates, updates, or deletes an # XPostgresSQLInstance. resources: # It's good practice to provide a unique name for each entry. Note that # this identifies the resources entry within the Composition - it's not # the name the CloudSQLInstance. The 'name' field will be required in a # future version of this API. - name: cloudsqlinstance # The 'base' template for the CloudSQLInstance Crossplane will create. # You can use the base template to specify fields that never change, or # default values for fields that may optionally be patched over. Bases must # be a valid Crossplane resource - a Managed Resource, Composite Resource, # or a ProviderConfig. base: apiVersion: database.gcp.crossplane.io/v1beta1 kind: CloudSQLInstance spec: forProvider: databaseVersion: POSTGRES_12 region: us-central1 settings: dataDiskType: PD_SSD ipConfiguration: ipv4Enabled: true authorizedNetworks: - value: "0.0.0.0/0" # Each resource can optionally specify a set of 'patches' that copy fields # from (or to) the XR. patches: # FromCompositeFieldPath is the default when 'type' is omitted, but it's # good practice to always include the type for readability. - type: FromCompositeFieldPath fromFieldPath: spec.parameters.size toFieldPath: spec.forProvider.settings.tier # Each patch can optionally specify one or more 'transforms', which # transform the 'from' field's value before applying it to the 'to' field. # Transforms are applied in the order they are specified; each transform's # output is passed to the following transform's input. transforms: - type: map map: medium: db-custom-1-3840 policy: # By default a patch from a field path that does not exist is simply # skipped until it does. Use the 'Required' policy to instead block and # return an error when the field path does not exist. fromFieldPath: Required # You can patch entire objects or arrays from one resource to another. # By default the 'to' object or array will be overwritten, not merged. # Use the 'mergeOptions' field to override this behaviour. Note that # these fields accidentally leak Go terminology - 'slice' means 'array'. # 'map' means 'map' in YAML or 'object' in JSON. mergeOptions: appendSlice: true keepMapValues: true # You can include connection details to propagate from this CloudSQLInstance # up to the XPostgreSQLInstance XR (and then on to the PostgreSQLInstance # claim). Remember that your XRD must declare which connection secret keys # it supports. connectionDetails: - name: hostname fromConnectionSecretKey: hostname # By default an XR's 'Ready' status condition will become True when the # 'Ready' status conditions of all of its composed resources become true. # You can optionally specify custom readiness checks to override this. readinessChecks: - type: None # If you find yourself repeating patches a lot you can group them as a named # 'patch set' then use a PatchSet type patch to reference them. patchSets: - name: metadata patches: - type: FromCompositeFieldPath # When both field paths are the same you can omit the 'toFieldPath' and it # will default to the 'fromFieldPath'. fromFieldPath: metadata.labels[some-important-label] ``` ### Pause Annotation There is an annotation named `crossplane.io/paused` that you can use on Composite Resources and Composite Resource Claims to temporarily pause reconciliations of their respective controllers on them. An example for a Composite Resource Claim is as follows: ```yaml apiVersion: test.com/v1alpha1 kind: MyResource metadata: annotations: crossplane.io/paused: "true" namespace: upbound-system name: my-resource spec: parameters: tagValue: demo-test compositionRef: name: example ``` where `MyResource` is a Composite Resource Claim kind. When a Composite Resource or a Claim has the `crossplane.io/paused` annotation with its value set to `true`, the Composite Resource controller or the Claim controller pauses reconciliations on the resource until the annotation is removed or its value set to something other than `true`. Before temporarily pausing reconciliations, an event with the type `Synced`, the status `False`, and the reason `ReconcilePaused` is emitted on the resource. Please also note that annotations on a Composite Resource Claim are propagated to the associated Composite Resource but when the `crossplane.io/paused: "true"` annotation is added to a Claim, because reconciliations on the Claim are now paused, this newly added annotation will not be propagated. However, whenever the annotation's value is set to a non-`true` value, reconciliations on the Claim will now resume, and thus the annotation will now be propagated to the associated Composite Resource with a non-`true` value. An implication of the described behavior is that pausing reconciliations on the Claim will not inherently pause reconciliations on the associated Composite Resource. ### Patch Types You can use the following types of patch in a `Composition`: `FromCompositeFieldPath`. The default if the `type` is omitted. This type patches from a field within the XR to a field within the composed resource. It's commonly used to expose a composed resource spec field as an XR spec field. ```yaml # Patch from the XR's spec.parameters.size field to the composed resource's # spec.forProvider.settings.tier field. - type: FromCompositeFieldPath fromFieldPath: spec.parameters.size toFieldPath: spec.forProvider.settings.tier ``` `ToCompositeFieldPath`. The inverse of `FromCompositeFieldPath`. This type patches from a field within the composed resource to a field within the XR. It's commonly used to derive an XR status field from a composed resource status field. ```yaml # Patch from the composed resource's status.atProvider.zone field to the XR's # status.zone field. - type: ToCompositeFieldPath fromFieldPath: status.atProvider.zone toFieldPath: status.zone ``` `FromCompositeFieldPath` and `ToCompositeFieldPath` patches can also take a wildcarded field path in the `toFieldPath` parameter and patch each array element in the `toFieldPath` with the singular value provided in the `fromFieldPath`. ```yaml # Patch from the XR's spec.parameters.allowedIPs to the CIDRBlock elements # inside the array spec.forProvider.firewallRules on the composed resource. resources: - name: exampleFirewall base: apiVersion: firewall.example.crossplane.io/v1beta1 kind: Firewall spec: forProvider: firewallRules: - Action: "Allow" Destination: "example1" CIDRBlock: "" - Action: "Allow" Destination: "example2" CIDRBlock: "" - type: FromCompositeFieldPath fromFieldPath: spec.parameters.allowedIP toFieldPath: spec.forProvider.firewallRules[*].CIDRBlock ``` `FromEnvironmentFieldPath`. This type patches from a field within the in-memory environment to a field within the composed resource. It's commonly used to expose a composed resource spec field as an XR spec field. Note that EnvironmentConfigs are an alpha feature and need to be enabled with the `--enable-environment-configs` flag on startup. ```yaml # Patch from the environment's tier.name field to the composed resource's # spec.forProvider.settings.tier field. - type: FromEnvironmentFieldPath fromFieldPath: tier.name toFieldPath: spec.forProvider.settings.tier ``` `ToEnvironmentFieldPath`. This type patches from a composed field to the in-memory environment. Note that, unlike `ToCompositeFieldPath` patches, this is executed before the composed resource is applied on the cluster which means that the `status` is not available. Note that EnvironmentConfigs are an alpha feature and need to be enabled with the `--enable-environment-configs` flag on startup. ```yaml # Patch from the environment's tier.name field to the composed resource's # spec.forProvider.settings.tier field. - type: ToEnvironmentFieldPath fromFieldPath: spec.forProvider.settings.tier toFieldPath: tier.name ``` Note that the field to be patched requires some initial value to be set. `CombineFromComposite`. Combines multiple fields from the XR to produce one composed resource field. ```yaml # Patch from the XR's spec.parameters.location field and the # metadata.labels[crossplane.io/claim-name] label to the composed # resource's spec.forProvider.administratorLogin field. - type: CombineFromComposite combine: # The patch will only be applied when all variables have non-zero values. variables: - fromFieldPath: spec.parameters.location - fromFieldPath: metadata.labels[crossplane.io/claim-name] strategy: string string: fmt: "%s-%s" toFieldPath: spec.forProvider.administratorLogin # By default Crossplane will skip the patch until all of the variables to be # combined have values. Set the fromFieldPath policy to 'Required' to instead # abort composition and return an error if a variable has no value. policy: fromFieldPath: Required ``` `CombineFromEnvironment`. Combines multiple fields from the in-memory environment to produce one composed resource field. Note that EnvironmentConfigs are an alpha feature and need to be enabled with the `--enable-environment-configs` flag on startup. ```yaml # Patch from the environments's location field and region to the composed # resource's spec.forProvider.administratorLogin field. - type: CombineFromEnvironment combine: # The patch will only be applied when all variables have non-zero values. variables: - fromFieldPath: location - fromFieldPath: region strategy: string string: fmt: "%s-%s" toFieldPath: spec.forProvider.administratorLogin ``` At the time of writing only the `string` combine strategy is supported. It uses [Go string formatting][pkg/fmt] to combine values, so if the XR's location was `us-west` and its claim name was `db` the composed resource's administratorLogin would be set to `us-west-db`. `CombineToComposite` is the inverse of `CombineFromComposite`. ```yaml # Patch from the composed resource's spec.parameters.administratorLogin and # status.atProvider.fullyQualifiedDomainName fields back to the XR's # status.adminDSN field. - type: CombineToComposite combine: variables: - fromFieldPath: spec.parameters.administratorLogin - fromFieldPath: status.atProvider.fullyQualifiedDomainName strategy: string # Here, our administratorLogin parameter and fullyQualifiedDomainName # status are formatted to a single output string representing a DSN. string: fmt: "mysql://%s@%s:3306/my-database-name" toFieldPath: status.adminDSN ``` `CombineToEnvironment` is the inverse of `CombineFromEnvironment`. Note that EnvironmentConfigs are an alpha feature and need to be enabled with the `--enable-environment-configs` flag on startup. ```yaml # Patch from the composed resource's spec.parameters.administratorLogin and # spec.forProvider.domainName fields back to the environment's adminDSN field. - type: CombineToEnvironment combine: variables: - fromFieldPath: spec.parameters.administratorLogin - fromFieldPath: spec.forProvider.domainName strategy: string # Here, our administratorLogin parameter and fullyQualifiedDomainName # status are formatted to a single output string representing a DSN. string: fmt: "mysql://%s@%s:3306/my-database-name" toFieldPath: adminDSN ``` `PatchSet`. References a named set of patches defined in the `spec.patchSets` array of a `Composition`. ```yaml # This is equivalent to specifying all of the patches included in the 'metadata' # PatchSet. - type: PatchSet patchSetName: metadata ``` The `patchSets` array may not contain patches of `type: PatchSet`. The `transforms` and `patchPolicy` fields are ignored by `type: PatchSet`. ### Transform Types You can use the following types of transform on a value being patched: `map`. Transforms values using a map. ```yaml # If the value of the 'from' field is 'us-west', the value of the 'to' field # will be set to 'West US'. - type: map map: us-west: West US us-east: East US au-east: Australia East ``` `match`. A more complex version of `map` that can match different kinds of patterns. It should be used if more advanced pattern matchings than a simple string equality check are required. The result of the first matching pattern is used as the output of this transform. If no pattern matches, you can either fallback to a given `fallbackValue` or fallback to the input value by setting the `fallbackTo` field to `Input`. ```yaml # In the example below, if the value in the 'from' field is 'us-west', the # value in the 'to' field will be set to 'West US'. # If the value in the 'from' field is 'eu-west', the value in the 'to' field # will be set to 'Unknown' because no pattern matches. - type: match match: patterns: - type: literal # Not needed. This is the default. literal: us-west result: West US - type: regexp regexp: '^af-.*' result: Somewhere in Africa fallbackTo: Value # Not needed. This is the default. fallbackValue: Unknown # If fallbackTo is set to Input, the output will be the input value if no # pattern matches. # In the example below, if the value in the 'from' field is 'us-west', the # value in the 'to' field will be set to 'West US'. # If the value in the 'from' field is 'eu-west', the value in the 'to' field # will be set to 'eu-west' because no pattern matches. - type: match match: patterns: - type: literal literal: us-west result: West US - type: regexp regexp: '^af-.*' result: Somewhere in Africa fallbackTo: Input ``` `math`. Transforms values using math. The input value must be an integer. * math transform type `Multiply`, multiplies the input by the given value. * math transform type `ClampMin`, sets a minimum value for the output. * math transform type `ClampMax`, sets a maximum value for the output. ```yaml # If you omit the field type, by default type is set to `Multiply` # If the value of the 'from' field is 2, the value of the 'to' field will be set # to 4. - type: math math: multiply: 2 # This is the same as above # If the value of the 'from' field is 2, the value of the 'to' field will be set # to 4. - type: math math: type: Multiply multiply: 2 # If the value of the 'from' field is 3, the value of the 'to' field will # be set to 4. - type: math math: type: ClampMin clampMin: 4 # If the value of the 'from' field is 3, the value of the 'to' field will # be set to 2. - type: math math: type: ClampMax clampMax: 2 ``` `string`. Transforms string values. * string transform type `Format`, Currently only Go style fmt is supported. [Go style `fmt`][pkg/fmt] is supported. * string transform type `Convert`, accepts one of `ToUpper`, `ToLower`, `ToBase64`, `FromBase64`, `ToJson`, `ToSha1`, `ToSha256`, `ToSha512`. * string transform type `TrimPrefix`, accepts a string to be trimmed from the beginning of the input. * string transform type `TrimSuffix`, accepts a string to be trimmed from the end of the input. * string transform type `Regexp`, accepts a string for regexp to be applied to. ```yaml # If you omit the field type, by default type is set to `Format` # If the value of the 'from' field is 'hello', the value of the 'to' field will # be set to 'hello-world'. - type: string string: fmt: "%s-world" # This is the same as above # the value of the 'to' field will be set to 'hello-world'. - type: string string: type: Format fmt: "%s-world" # If the value of the 'from' field is 'hello', the value of the 'to' field will # be set to 'HELLO'. - type: string string: type: Convert convert: ToUpper # If the value of the 'from' field is 'Hello', the value of the 'to' field will # be set to 'hello'. - type: string string: type: Convert convert: ToLower # If the value of the 'from' field is 'Hello', the value of the 'to' field will # be set to 'SGVsbG8='. - type: string string: type: Convert convert: ToBase64 # If the value of the 'from' field is 'SGVsbG8=', the value of the 'to' field will # be set to 'Hello'. - type: string string: type: Convert convert: FromBase64 # If the value of the 'from' field is not nil, the value of the 'to' field will be # set to raw JSON representation of the 'from' field. - type: string string: type: Convert convert: ToJson # The output will be the hash of the JSON representation of the 'from' field. - type: string string: type: Convert convert: ToSha1 # alternatives: 'ToSha256' or 'ToSha512' # If the value of the 'from' field is https://crossplane.io, the value of the 'to' field will # be set to crossplane.io - type: string string: type: TrimPrefix trim: 'https://' # If the value of the 'from' field is my-string-test, the value of the 'to' field will # be set to my-string - type: string string: type: TrimSuffix trim: '-test' # If the value of the 'from' field is 'arn:aws:iam::42:example, the value of the # 'to' field will be set to "42". Note that the 'to' field is always a string. - type: string string: type: Regexp regexp: match: 'arn:aws:iam::(\d+):.*' group: 1 # Optional capture group. Omit to match the entire regexp. ``` `convert`. Transforms values of one type to another, for example from a string to an integer. The following values are supported by the `from` and `to` fields: * `string` * `bool` * `int` * `int64` * `float64` The strings 1, t, T, TRUE, true, and True are considered 'true', while 0, f, F, FALSE, false, and False are considered 'false'. The integer 1 and float 1.0 are considered true, while all other values are considered false. Similarly, boolean true converts to integer 1 and float 1.0, while false converts to 0 and 0.0. ```yaml # If the value to be converted is "1" (a string), the value of the 'toType' # field will be set to 1 (an integer). - type: convert convert: toType: int ``` Converting `string` to `float64` additionally supports parsing string in [K8s quantity format](https://pkg.go.dev/k8s.io/apimachinery/pkg/api/resource#Quantity), such as `1000m` or `500 Mi`: ```yaml - type: convert convert: toType: float64 format: quantity ``` ### Connection Details Connection details secret of XR is an aggregated sum of the connection details of the composed resources. It's recommended that the author of XRD specify exactly which keys will be allowed in the XR connection secret by listing them in `spec.connectionSecretKeys` so that consumers of claims and XRs can see what they can expect in the connection details secret. If `spec.connectionSecretKeys` is empty, then all keys of the aggregated connection details secret will be propagated. You can derive the following types of connection details from a composed resource to be aggregated: `FromConnectionSecretKey`. Derives an XR connection detail from a connection secret key of a composed resource. ```yaml # Derive the XR's 'user' connection detail from the 'username' key of the # composed resource's connection secret. - type: FromConnectionSecretKey name: user fromConnectionSecretKey: username ``` `FromFieldPath`. Derives an XR connection detail from a field path within the composed resource. ```yaml # Derive the XR's 'user' connection detail from the 'adminUser' status field of # the composed resource. - type: FromFieldPath name: user fromFieldPath: status.atProvider.adminUser ``` `FromValue`. Derives an XR connection detail from a fixed value. ```yaml # Always sets the XR's 'user' connection detail to 'admin'. - type: FromValue name: user value: admin ``` ### Readiness Checks Crossplane can use the following types of readiness check to determine whether a composed resource is ready (and therefore whether the XR and claim should be considered ready). Specify multiple readiness checks if multiple conditions must be met for a composed resource to be considered ready. > Note that if you don't specify any readiness checks Crossplane will consider > the composed resource to be ready when its 'Ready' status condition becomes > 'True'. `MatchString`. Considers the composed resource to be ready when the value of a field within that resource matches a specified string. ```yaml # The composed resource will be considered ready when the 'state' status field # matches the string 'Online'. - type: MatchString fieldPath: status.atProvider.state matchString: "Online" ``` `MatchInteger`. Considers the composed resource to be ready when the value of a field within that resource matches a specified integer. ```yaml # The composed resource will be considered ready when the 'state' status field # matches the integer 4. - type: MatchInteger fieldPath: status.atProvider.state matchInteger: 4 ``` `NonEmpty`. Considers the composed resource to be ready when a field exists in the composed resource. The name of this check can be a little confusing in that a field that exists with a zero value (e.g. an empty string or zero integer) is not considered to be 'empty', and thus will pass the readiness check. ```yaml # The composed resource will be considered ready if and when 'online' status # field exists. - type: NonEmpty fieldPath: status.atProvider.online ``` `None`. Considers the composed resource to be ready as soon as it exists. ### Composition validation Crossplane uses a `Validating Webhook` to inform users of any potential errors in a `Composition`. By default webhooks only perform `logical checks`. `logical checks` enforce requirements that aren't explicitly defined in the schema but Crossplane assumes to hold at runtime. #### Experimental validation with schemas Enable experimental schema-aware validation in Crossplane through the `--enable-composition-webhook-schema-validation` feature flag. This enables Composition validation against available schemas in the cluster. For example, ensuring that `fieldPaths` are valid and source and destination types match taking into account provided transforms too. The `crossplane.io/composition-validation-mode` annotation on the Composition allows setting one of two modes for schema validation: - `loose` (default): Validates Compositions against required schemas. If a required schema is missing, schema validation stops, emits a warning and falls back to `logical checks` only. - `strict`: Validates Compositions against required schemas, and rejects them when finding errors. Rejects any Compositions missing required schemas. See the [Composition Validating Webhook design document][validation-design-doc] for more information about future development around schema-aware validation. #### Disabling webhooks Crossplane enables webhooks by default. Turn off webhooks by `webhooks.enabled` to `false` in the provided Helm Chart. ### Missing Functionality You might find while reading through this reference that Crossplane is missing some functionality you need to compose resources. If that's the case, please [raise an issue] with as much detail **about your use case** as possible. Please understand that the Crossplane maintainers are growing the feature set of the `Composition` type conservatively. We highly value the input of our users and community, but we also feel it's critical to avoid bloat and complexity. We therefore wish to carefully consider each new addition. We feel some features may be better suited for a real, expressive programming language and intend to build an alternative to the `Composition` type as it's documented here per [this proposal][issue-2524]. ## Tips, Tricks, and Troubleshooting In this section we'll cover some common tips, tricks, and troubleshooting steps for working with Composite Resources. If you're trying to track down why your Composite Resources aren't working the [Troubleshooting][trouble-ref] page also has some useful information. ### Troubleshooting Claims and XRs Crossplane relies heavily on status conditions and events for troubleshooting. You can see both using `kubectl describe` - for example: ```console # Describe the PostgreSQLInstance claim named my-db kubectl describe postgresqlinstance.database.example.org my-db ``` Per Kubernetes convention, Crossplane keeps errors close to the place they happen. This means that if your claim is not becoming ready due to an issue with your `Composition` or with a composed resource you'll need to "follow the references" to find out why. Your claim will only tell you that the XR is not yet ready. To follow the references: 1. Find your XR by running `kubectl describe` on your claim and looking for its "Resource Ref" (aka `spec.resourceRef`). 1. Run `kubectl describe` on your XR. This is where you'll find out about issues with the `Composition` you're using, if any. 1. If there are no issues but your XR doesn't seem to be becoming ready, take a look for the "Resource Refs" (or `spec.resourceRefs`) to find your composed resources. 1. Run `kubectl describe` on each referenced composed resource to determine whether it is ready and what issues, if any, it is encountering. ### Composite Resource Connection Secrets Claim and Composite Resource connection secrets are often derived from the connection secrets of the managed resources they compose. This is a common source of confusion because several things need to align for it to work: 1. The **claim** must specify the secret where the aggregated connection details should be written * This is the `spec.writeConnectionSecretToRef` field in a claim * If creating a composite resource directly (without a claim) then this same field must be set on your composite resource instead 1. The **composite resource definition** must state which connection details to aggregate from its children to publish to the claim * This is the `spec.connectionSecretKeys` field in a `CompositeResourceDefinition` 1. The **composition** must define where to write its aggregated connection details * This is the `spec.writeConnectionSecretsToNamespace` field in the `Composition` 1. Each child **composed resource** must define the connection details it publishes and where to write them * These are the `connectionDetails` and `base.spec.writeConnectionSecretToRef` fields of the composed resources Finally, you can't currently edit a XRD's supported connection details. The XRD's `spec.connectionSecretKeys` is effectively immutable. This may change in future per [this issue][issue-2024] ### Claiming an Existing Composite Resource Most people create Composite Resources using a claim, but you can actually claim an existing Composite Resource as long as its a type of XR that offers a claim and no one else has already claimed it. To do so: 1. Set the `spec.resourceRef` of your claim to reference the existing XR. ```yaml apiVersion: database.example.org/v1alpha1 kind: PostgreSQLInstance metadata: name: example namespace: default spec: resourceRef: apiVersion: database.example.org/v1alpha1 kind: XPostgreSQLInstance name: example-d4lmv ``` 1. Make sure the rest of your claim's spec fields match the XR's. If your claim's spec fields don't match the XR's Crossplane will still claim it but will then try to update the XR's spec fields to match the claim's. ### Influencing External Names The `crossplane.io/external-name` annotation has special meaning to Crossplane managed resources - it specifies the name (or identifier) of the resource in the external system, for example the actual name of a `CloudSQLInstance` in the GCP API. Some managed resources don't let you specify an external name - in those cases Crossplane will set it for you to whatever the external system requires. If you add the `crossplane.io/external-name` annotation to a claim Crossplane will automatically propagate it when it creates an XR. It's good practice to have your `Composition` further propagate the annotation to one or more composed resources, but it's not required. ### Mixing and Matching Providers Crossplane has providers for many things in addition to the big clouds. Take a look at the [Upbound Marketplace][upbound-marketplace] to find many of them. Keep in mind that you can mix and match managed resources from different providers within a `Composition` to create Composite Resources. For example you might use provider-aws and provider-sql to create an XR that provisions an `RDSInstance` then creates an SQL `Database` and `User`, or provider-gcp and provider-helm to create a `GKECluster` and deploy a Helm Chart `Release` to it. Often when mixing and matching providers you'll need to compose a `ProviderConfig` for one provider that loads credentials from the connection secret of a managed resource from another provider. Sometimes you may need to use an intermediary XR to mutate the connection details to suit your needs. [This example][helm-and-gcp] from provider-helm demonstrates using a GKE cluster connection secret as Helm `ProviderConfig` credentials. ### Patching From One Composed Resource to Another or Itself It's not possible to patch _directly_ from one composed resource to another - i.e. from one entry in the `spec.resources` array of a `Composition` to another. It is however possible to achieve this by using the XR as an intermediary. To do so: 1. Use a `ToCompositeFieldPath` patch to patch from your source composed resource to the XR. Typically you'll want to patch to a status field or an annotation. 1. Use a `FromCompositeFieldPath` patch to patch from the 'intermediary' field you patched to in step 1 to a field on the destination composed resource. Note that the source and the target composed resource can be the same. [crd-docs]: https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/ [raise an issue]: https://github.com/crossplane/crossplane/issues/new?assignees=&labels=enhancement&template=feature_request.md [issue-2524]: https://github.com/crossplane/crossplane/issues/2524 [field-paths]: https://github.com/kubernetes/community/blob/61f3d0/contributors/devel/sig-architecture/api-conventions.md#selecting-fields [pkg/fmt]: https://pkg.go.dev/fmt [upbound-marketplace]: https://marketplace.upbound.io [helm-and-gcp]: https://github.com/crossplane-contrib/provider-helm/blob/2dcbdd0/examples/in-composition/composition.yaml [issue-2024]: https://github.com/crossplane/crossplane/issues/2024 [xrs-and-mrs]: /media/composition-xrs-and-mrs.svg [how-it-works]: /media/composition-how-it-works.svg [provider-kubernetes]: https://marketplace.upbound.io/providers/crossplane-contrib/provider-kubernetes [provider-helm]: https://marketplace.upbound.io/providers/crossplane-contrib/provider-helm/ [claims-and-xrs]: /media/composition-claims-and-xrs.svg [xr-ref]: {{}} [managed-resources]: {{}} [validation-design-doc]: https://github.com/crossplane/crossplane/blob/master/design/design-doc-composition-validating-webhook.md