docs/content/v1.12/concepts/composition.md

title	weight
Composite Resources	103

Crossplane Composite Resources are opinionated Kubernetes Custom Resources that are composed of [Managed Resources][managed-resources]. We often call them XRs for short.

Composite Resources are designed to let you build your own platform with your own opinionated concepts and APIs without needing to write a Kubernetes controller from scratch. Instead, you define the schema of your XR and teach Crossplane which Managed Resources it should compose (i.e. create) when someone creates the XR you defined.

If you're already familiar with Composite Resources and looking for a detailed configuration reference or some tips, tricks, and troubleshooting information, try the [Composition Reference][xr-ref].

Below is an example of a Composite Resource:

apiVersion: database.example.org/v1alpha1
kind: XPostgreSQLInstance
metadata:
  name: my-db
spec:
  parameters:
    storageGB: 20
  compositionRef:
    name: production
  writeConnectionSecretToRef:
    namespace: crossplane-system
    name: my-db-connection-details

You define your own XRs, so they can be of whatever API version and kind you like, and contain whatever spec and status fields you need.

How It Works

The first step towards using Composite Resources is configuring Crossplane so that it knows what XRs you'd like to exist, and what to do when someone creates one of those XRs. This is done using a CompositeResourceDefinition (XRD) resource and one or more Composition resources.

Once you've configured Crossplane with the details of your new XR you can either create one directly, or use a claim. Typically only the folks responsible for configuring Crossplane (often a platform or SRE team) have permission to create XRs directly. Everyone else manages XRs via a lightweight proxy resource called a Composite Resource Claim (or claim for short). More on that later.

If you're coming from the Terraform world you can think of an XRD as similar to the variable blocks of a Terraform module, while the Composition is the rest of the module's HCL code that describes how to use those variables to create a bunch of resources. In this analogy the XR or claim is a little like a tfvars file providing inputs to the module.

Defining Composite Resources

A CompositeResourceDefinition (or XRD) defines the type and schema of your XR. It lets Crossplane know that you want a particular kind of XR to exist, and what fields that XR should have. An XRD is a little like a CustomResourceDefinition (CRD), but slightly more opinionated. Writing an XRD is mostly a matter of specifying an OpenAPI "structural schema".

The XRD that defines the XPostgreSQLInstance XR above would look like this:

apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
  name: xpostgresqlinstances.database.example.org
spec:
  group: database.example.org
  names:
    kind: XPostgreSQLInstance
    plural: xpostgresqlinstances
  claimNames:
    kind: PostgreSQLInstance
    plural: postgresqlinstances
  versions:
  - name: v1alpha1
    served: true
    referenceable: true
    schema:
      openAPIV3Schema:
        type: object
        properties:
          spec:
            type: object
            properties:
              parameters:
                type: object
                properties:
                  storageGB:
                    type: integer
                required:
                - storageGB
            required:
            - parameters

You might notice that the XPostgreSQLInstance example above has some fields that don't appear in the XRD, like the writeConnectionSecretToRef and compositionRef fields. This is because Crossplane automatically injects some standard Crossplane Resource Model (XRM) fields into all XRs.

Configuring Composition

A Composition lets Crossplane know what to do when someone creates a Composite Resource. Each Composition creates a link between an XR and a set of one or more Managed Resources - when the XR is created, updated, or deleted the set of Managed Resources are created, updated or deleted accordingly.

You can add multiple Compositions for each XRD, and choose which should be used when XRs are created. This allows a Composition to act like a class of service - for example you could configure one Composition for each environment you support, such as production, staging, and development.

A basic Composition for the above XPostgreSQLInstance might look like this:

apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: example
  labels:
    crossplane.io/xrd: xpostgresqlinstances.database.example.org
    provider: gcp
spec:
  writeConnectionSecretsToNamespace: crossplane-system
  compositeTypeRef:
    apiVersion: database.example.org/v1alpha1
    kind: XPostgreSQLInstance
  resources:
  - name: cloudsqlinstance
    base:
      apiVersion: database.gcp.crossplane.io/v1beta1
      kind: CloudSQLInstance
      spec:
        forProvider:
          databaseVersion: POSTGRES_12
          region: us-central1
          settings:
            tier: db-custom-1-3840
            dataDiskType: PD_SSD
            ipConfiguration:
              ipv4Enabled: true
              authorizedNetworks:
                - value: "0.0.0.0/0"
    patches:
    - type: FromCompositeFieldPath
      fromFieldPath: spec.parameters.storageGB
      toFieldPath: spec.forProvider.settings.dataDiskSizeGb

The above Composition tells Crossplane that when someone creates an XPostgreSQLInstance XR Crossplane should create a CloudSQLInstance in response. The storageGB field of the XPostgreSQLInstance should be used to configure the dataDiskSizeGb field of the CloudSQLInstance. This is only a small subset of the functionality a Composition enables - take a look at the [reference page][xr-ref] to learn more.

We almost always talk about XRs composing Managed Resources, but actually an XR can also compose other XRs to allow nested layers of abstraction. XRs don't support composing arbitrary Kubernetes resources (e.g. Deployments, operators, etc) directly but you can do so using our Kubernetes and Helm providers.

Claiming Composite Resources

Crossplane uses Composite Resource Claims (or just claims, for short) to allow application operators to provision and manage XRs. When we talk about using XRs it's typically implied that the XR is being used via a claim. Claims are almost identical to their corresponding XRs. It helps to think of a claim as an application team’s interface to an XR. You could also think of claims as the public (app team) facing part of the opinionated platform API, while XRs are the private (platform team) facing part.

A claim for the XPostgreSQLInstance XR above would look like this:

apiVersion: database.example.org/v1alpha1
kind: PostgreSQLInstance
metadata:
  namespace: default
  name: my-db
spec:
  parameters:
    storageGB: 20
  compositionRef:
    name: production
  writeConnectionSecretToRef:
    name: my-db-connection-details

There are three key differences between an XR and a claim:

1. Claims are namespaced, while XRs (and Managed Resources) are cluster scoped.
2. Claims are of a different kind than the XR - by convention the XR's kind without the proceeding X. For example a PostgreSQLInstance claims an XPostgreSQLInstance.
3. An active claim contains a reference to its corresponding XR, while an XR contains both a reference to the claim an array of references to the managed resources it composes.

Not all XRs offer a claim - doing so is optional. See the XRD section of the [Composition reference][xr-ref] to learn how to offer a claim.

Claims may seem a little superfluous at first, but they enable some handy scenarios, including:

• Private XRs. Sometimes a platform team might not want a type of XR to be directly consumed by their application teams. For example because the XR represents 'supporting' infrastructure - consider the above VPC XNetwork XR. App teams might create PostgreSQLInstance claims that reference (i.e. consume) an XNetwork, but they shouldn't be creating their own. Similarly, some kinds of XR might be intended only for 'nested' use - intended only to be composed by other XRs.

• Global XRs. Not all infrastructure is conceptually namespaced. Say your organisation uses team scoped namespaces. A PostgreSQLInstance that belongs to Team A should probably be part of the team-a namespace - you'd represent this by creating a PostgreSQLInstance claim in that namespace. On the other hand the XNetwork XR we mentioned previously could be referenced (i.e. used) by XRs from many different namespaces - it doesn't exist to serve a particular team.

• Pre-provisioned XRs. Finally, separating claims from XRs allows a platform team to pre-provision certain kinds of XR. Typically an XR is created on-demand in response to the creation of a claim, but it's also possible for a claim to instead request an existing XR. This can allow application teams to instantly claim infrastructure like database instances that would otherwise take minutes to provision on-demand.

This reference provides detailed examples of defining, configuring, and using Composite Resources in Crossplane. You can also refer to Crossplane's [API documentation][api-docs] for more details. If you're looking for a more general overview of Composite Resources and Composition in Crossplane, try the [Composite Resources][xr-concepts] page under Concepts.

Composite Resources and Claims

The type and most of the schema of Composite Resources and claims are largely of your own choosing, but there is some common 'machinery' injected into them. Here's a hypothetical XR that doesn't have any user-defined fields and thus only includes the automatically injected Crossplane machinery:

apiVersion: database.example.org/v1alpha1
kind: XPostgreSQLInstance
metadata:
  # This XR was created automatically by a claim, so its name is derived from
  # the claim's name.
  name: my-db-mfd1b
  annotations:
    # The external name annotation has special meaning in Crossplane. When a
    # claim creates an XR its external name will automatically be propagated to
    # the XR. Whether and how the external name is propagated to the resources
    # the XR composes is up to its Composition.
    crossplane.io/external-name: production-db-0
spec:
  # XRs have a reference to the claim that created them (or, if the XR was
  # pre-provisioned, to the claim that later claimed them).
  claimRef:
    apiVersion: database.example.org/v1alpha1
    kind: PostgreSQLInstance
    name: my-db
  # The compositeDeletePolicy specifies the propagation policy that will be used by Crossplane
  # when deleting the Composite Resource that is associated with the Claim.  The default
  # value is Background, which causes the Composite resource to be deleted using
  # the kubernetes default propagation policy of Background, and all associated
  # resources will be deleted simultaneously.  The other value for this field is Foreground,
  # which will cause the Composite resource to be deleted using Foreground Cascading Deletion.
  # Kubernetes will add a foregroundDeletion finalizer to all of the resources in the
  # dependency graph, and they will be deleted starting with the edge or leaf nodes and
  # working back towards the root Composite.  See https://kubernetes.io/docs/concepts/architecture/garbage-collection/#cascading-deletion
  # for more information on cascading deletion.
  compositeDeletePolicy: Background
  # The compositionRef specifies which Composition this XR will use to compose
  # resources when it is created, updated, or deleted. This can be omitted and
  # will be set automatically if the XRD has a default or enforced composition
  # reference, or if the below composition selector is set.
  compositionRef:
    name: production-us-east
  # The compositionSelector allows you to match a Composition by labels rather
  # than naming one explicitly. It is used to set the compositionRef if none is
  # specified explicitly.
  compositionSelector:
    matchLabels:
      environment: production
      region: us-east
      provider: gcp
  # The environment is an in-memory object that can be patched from / to during
  # rendering.
  # The environment is composed by merging the 'data' of all EnvironmentConfigs
  # referenced below. It is disposed after every reconcile.
  # NOTE: EnvironmentConfigs are an alpha feature and need to be enabled with
  #       the '--enable-environment-configs' flag on startup.
  environment:
    # EnvironmentConfigs is a list of object references that is made up of
    # name references and label selectors
    environmentConfigs:
      - type: Reference # this is the default
        ref:
          name: example-environment
      - type: Selector
        selector:
          - key: stage
            type: FromCompositeFieldPath # this is the default
            valueFromFieldPath: spec.parameters.stage
          - key: provider
            type: Value
            value: "gcp"
  # The resourceRefs array contains references to all of the resources of which
  # this XR is composed. Despite being in spec this field isn't intended to be
  # configured by humans - Crossplane will take care of keeping it updated.
  resourceRefs:
  - apiVersion: database.gcp.crossplane.io/v1beta1
    kind: CloudSQLInstance
    name: my-db-mfd1b-md9ab
  # The writeConnectionSecretToRef field specifies a Kubernetes Secret that this
  # XR should write its connection details (if any) to.
  writeConnectionSecretToRef:
    namespace: crossplane-system
    name: my-db-connection-details
status:
  # An XR's 'Ready' condition will become True when all of the resources it
  # composes are deemed ready. Refer to the Composition 'readinessChecks' field
  # for more information.
  conditions:
  - type: Ready
    statue: "True"
    reason: Available
    lastTransitionTime: 2021-10-02T07:20:50.52Z
  # The last time the XR published its connection details to a Secret.
  connectionDetails:
    lastPublishedTime: 2021-10-02T07:20:51.24Z

Similarly, here's an example of the claim that corresponds to the above XR:

apiVersion: database.example.org/v1alpha1
kind: PostgreSQLInstance
metadata:
  # Claims are namespaced, unlike XRs.
  namespace: default
  name: my-db
  annotations:
    # The external name annotation has special meaning in Crossplane. When a
    # claim creates an XR its external name will automatically be propagated to
    # the XR. Whether and how the external name is propagated to the resources
    # the XR composes is up to its Composition.
    crossplane.io/external-name: production-db-0
spec:
  # The resourceRef field references the XR this claim corresponds to. You can
  # either set it to an existing (compatible) XR that you'd like to claim or
  # (the more common approach) leave it blank and let Crossplane automatically
  # create and reference an XR for you.
  resourceRef:
    apiVersion: database.example.org/v1alpha1
    kind: XPostgreSQLInstance
    name: my-db-mfd1b
  # A claim's compositionRef and compositionSelector work the same way as an XR.
  compositionRef:
    name: production-us-east
  compositionSelector:
    matchLabels:
      environment: production
      region: us-east
      provider: gcp
  # A claim's writeConnectionSecretToRef mostly works the same way as an XR's.
  # The one difference is that the Secret is always written to the namespace of
  # the claim.
  writeConnectionSecretToRef:
    name: my-db-connection-details
status:
  # A claim's 'Ready' condition will become True when its XR's 'Ready' condition
  # becomes True.
  conditions:
  - type: Ready
    statue: "True"
    reason: Available
    lastTransitionTime: 2021-10-02T07:20:50.52Z
  # The last time the claim published its connection details to a Secret.
  connectionDetails:
    lastPublishedTime: 2021-10-02T07:20:51.24Z

If your XR or claim isn't working as you'd expect you can try running kubectl describe against it for details - pay particular attention to any events and status conditions. You may need to follow the references from claim to XR to composed resources to find out what's happening.

CompositeResourceDefinitions

Below is an example CompositeResourceDefinition that includes all configurable fields.

apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
  # XRDs must be named '<plural>.<group>', per the plural and group names below.
  name: xpostgresqlinstances.example.org
spec:
  # This XRD defines an XR in the 'example.org' API group.
  group: example.org
  # The kind of this XR will be 'XPostgreSQLInstance`. You may also optionally
  # specify a singular name and a listKind.
  names:
    kind: XPostgreSQLInstance
    plural: xpostgresqlinstances
  # This type of XR offers a claim. Omit claimNames if you don't want to do so.
  # The claimNames must be different from the names above - a common convention
  # is that names are prefixed with 'X' while claim names are not. This lets app
  # team members think of creating a claim as (e.g.) 'creating a
  # PostgreSQLInstance'.
  claimNames:
    kind: PostgreSQLInstance
    plural: postgresqlinstances
  # Each type of XR can declare any keys they write to their connection secret
  # which will act as a filter during aggregation of the connection secret from
  # composed resources. It's recommended to provide the set of keys here so that
  # consumers of claims and XRs can see what to expect in the connection secret.
  # If no key is given, then all keys in the aggregated connection secret will
  # be written to the connection secret of the XR.
  connectionSecretKeys:
  - hostname
  # Each type of XR may specify a default Composite Delete Policy to be used
  # when the Claim has no compositeDeletePolicy.  The valid values are Background
  # and Foreground, and the default is Background.  See the description of the
  # compositeDeletePolicy parameter for more information.
  defaultCompositeDeletePolicy: Background
  # Each type of XR may specify a default Composition to be used when none is
  # specified (e.g. when the XR has no compositionRef or selector). A similar
  # enforceCompositionRef field also exists to allow XRs to enforce a specific
  # Composition that should always be used.
  defaultCompositionRef:
    name: example
  # Each type of XR may specify a default Composition Update Policy to be used
  # when the Claim has no compositionUpdatePolicy.  The valid values are Automatic
  # and Manual and the default is Automatic.
  defaultCompositionUpdatePolicy: Automatic
  # Each type of XR may be served at different versions - e.g. v1alpha1, v1beta1
  # and v1 - simultaneously. Currently Crossplane requires that all versions
  # have an identical schema, so this is mostly useful to 'promote' a type of XR
  # from alpha to beta to production ready.
  versions:
  - name: v1alpha1
    # Served specifies that XRs should be served at this version. It can be set
    # to false to temporarily disable a version, for example to test whether
    # doing so breaks anything before a version is removed wholesale.
    served: true
    # Referenceable denotes the version of a type of XR that Compositions may
    # use. Only one version may be referenceable.
    referenceable: true
    # Schema is an OpenAPI schema just like the one used by Kubernetes CRDs. It
    # determines what fields your XR and claim will have. Note that Crossplane
    # will automatically extend with some additional Crossplane machinery.
    schema:
      openAPIV3Schema:
        type: object
        properties:
          spec:
            type: object
            properties:
              parameters:
                type: object
                properties:
                  storageGB:
                    type: integer
                required:
                - storageGB
            required:
            - parameters
          status:
            type: object
            properties:
              address:
                description: Address of this MySQL server.
                type: string

Take a look at the Kubernetes CRD documentation for a more detailed guide to writing OpenAPI schemas. Note that the following fields are reserved for Crossplane machinery, and will be ignored if your schema includes them:

• spec.resourceRef
• spec.resourceRefs
• spec.claimRef
• spec.writeConnectionSecretToRef
• status.conditions
• status.connectionDetails

If your CompositeResourceDefinition isn't working as you'd expect you can try running kubectl describe xrd for details - pay particular attention to any events and status conditions.

Compositions

You'll encounter a lot of 'field paths' when reading or writing a Composition. Field paths reference a field within a Kubernetes object via a simple string 'path'. API conventions describe the syntax as:

Standard JavaScript syntax for accessing that field, assuming the JSON object was transformed into a JavaScript object, without the leading dot, such as metadata.name.

Valid field paths include:

• metadata.name - The name field of the metadata object.
• spec.containers[0].name - The name field of the 0th containers element.
• data[.config.yml] - The .config.yml field of the data object.
• apiVersion - The apiVersion field of the root object.

While the following are invalid:

• .metadata.name - Leading period.
• metadata..name - Double period.
• metadata.name. - Trailing period.
• spec.containers[] - Empty brackets.
• spec.containers.[0].name - Period before open bracket.

Below is a detailed example of a Composition. While detailed, this example doesn't include every patch, transform, connection detail, and readiness check type. Keep reading below to discover those.

apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: example
  labels:
    # An optional convention is to include a label of the XRD. This allows
    # easy discovery of compatible Compositions.
    crossplane.io/xrd: xpostgresqlinstances.database.example.org
    # The following label marks this Composition for GCP. This label can 
    # be used in 'compositionSelector' in an XR or Claim.
    provider: gcp
spec:

  # Each Composition must declare that it is compatible with a particular type
  # of Composite Resource using its 'compositeTypeRef' field. The referenced
  # version must be marked 'referenceable' in the XRD that defines the XR.
  compositeTypeRef:
    apiVersion: database.example.org/v1alpha1
    kind: XPostgreSQLInstance

  # When an XR is created in response to a claim Crossplane needs to know where
  # it should create the XR's connection secret. This is configured using the
  # 'writeConnectionSecretsToNamespace' field.
  writeConnectionSecretsToNamespace: crossplane-system

  # Each Composition must specify at least one composed resource template. In
  # this case the Composition tells Crossplane that it should create, update, or
  # delete a CloudSQLInstance whenever someone creates, updates, or deletes an
  # XPostgresSQLInstance.
  resources:

    # It's good practice to provide a unique name for each entry. Note that
    # this identifies the resources entry within the Composition - it's not
    # the name the CloudSQLInstance. The 'name' field will be required in a
    # future version of this API.
  - name: cloudsqlinstance

    # The 'base' template for the CloudSQLInstance Crossplane will create.
    # You can use the base template to specify fields that never change, or
    # default values for fields that may optionally be patched over. Bases must
    # be a valid Crossplane resource - a Managed Resource, Composite Resource,
    # or a ProviderConfig.
    base:
      apiVersion: database.gcp.crossplane.io/v1beta1
      kind: CloudSQLInstance
      spec:
        forProvider:
          databaseVersion: POSTGRES_12
          region: us-central1
          settings:
            dataDiskType: PD_SSD
            ipConfiguration:
              ipv4Enabled: true
              authorizedNetworks:
                - value: "0.0.0.0/0"
      
    # Each resource can optionally specify a set of 'patches' that copy fields
    # from (or to) the XR.
    patches:
      # FromCompositeFieldPath is the default when 'type' is omitted, but it's
      # good practice to always include the type for readability.
    - type: FromCompositeFieldPath
      fromFieldPath: spec.parameters.size
      toFieldPath: spec.forProvider.settings.tier

      # Each patch can optionally specify one or more 'transforms', which
      # transform the 'from' field's value before applying it to the 'to' field.
      # Transforms are applied in the order they are specified; each transform's
      # output is passed to the following transform's input.
      transforms:
      - type: map
        map:
          medium: db-custom-1-3840

      policy:
        # By default a patch from a field path that does not exist is simply
        # skipped until it does. Use the 'Required' policy to instead block and
        # return an error when the field path does not exist.
        fromFieldPath: Required

        # You can patch entire objects or arrays from one resource to another.
        # By default the 'to' object or array will be overwritten, not merged.
        # Use the 'mergeOptions' field to override this behaviour. Note that
        # these fields accidentally leak Go terminology - 'slice' means 'array'.
        # 'map' means 'map' in YAML or 'object' in JSON.
        mergeOptions:
          appendSlice: true
          keepMapValues: true
    
    # You can include connection details to propagate from this CloudSQLInstance
    # up to the XPostgreSQLInstance XR (and then on to the PostgreSQLInstance
    # claim). Remember that your XRD must declare which connection secret keys
    # it supports.
    connectionDetails:
    - name: hostname
      fromConnectionSecretKey: hostname
    
    # By default an XR's 'Ready' status condition will become True when the
    # 'Ready' status conditions of all of its composed resources become true.
    # You can optionally specify custom readiness checks to override this.
    readinessChecks:
    - type: None

    
  # If you find yourself repeating patches a lot you can group them as a named
  # 'patch set' then use a PatchSet type patch to reference them.
  patchSets:
  - name: metadata
    patches:
    - type: FromCompositeFieldPath
      # When both field paths are the same you can omit the 'toFieldPath' and it
      # will default to the 'fromFieldPath'.
      fromFieldPath: metadata.labels[some-important-label]

Pause Annotation

There is an annotation named crossplane.io/paused that you can use on Composite Resources and Composite Resource Claims to temporarily pause reconciliations of their respective controllers on them. An example for a Composite Resource Claim is as follows:

apiVersion: test.com/v1alpha1
kind: MyResource
metadata:
  annotations:
     crossplane.io/paused: "true"
  namespace: upbound-system
  name: my-resource
spec:
  parameters:
    tagValue: demo-test
  compositionRef:
    name: example

where MyResource is a Composite Resource Claim kind. When a Composite Resource or a Claim has the crossplane.io/paused annotation with its value set to true, the Composite Resource controller or the Claim controller pauses reconciliations on the resource until the annotation is removed or its value set to something other than true. Before temporarily pausing reconciliations, an event with the type Synced, the status False, and the reason ReconcilePaused is emitted on the resource. Please also note that annotations on a Composite Resource Claim are propagated to the associated Composite Resource but when the crossplane.io/paused: "true" annotation is added to a Claim, because reconciliations on the Claim are now paused, this newly added annotation will not be propagated. However, whenever the annotation's value is set to a non-true value, reconciliations on the Claim will now resume, and thus the annotation will now be propagated to the associated Composite Resource with a non-true value. An implication of the described behavior is that pausing reconciliations on the Claim will not inherently pause reconciliations on the associated Composite Resource.

Patch Types

You can use the following types of patch in a Composition:

FromCompositeFieldPath. The default if the type is omitted. This type patches from a field within the XR to a field within the composed resource. It's commonly used to expose a composed resource spec field as an XR spec field.

# Patch from the XR's spec.parameters.size field to the composed resource's
# spec.forProvider.settings.tier field.
- type: FromCompositeFieldPath
  fromFieldPath: spec.parameters.size
  toFieldPath: spec.forProvider.settings.tier

ToCompositeFieldPath. The inverse of FromCompositeFieldPath. This type patches from a field within the composed resource to a field within the XR. It's commonly used to derive an XR status field from a composed resource status field.

# Patch from the composed resource's status.atProvider.zone field to the XR's
# status.zone field.
- type: ToCompositeFieldPath
  fromFieldPath: status.atProvider.zone
  toFieldPath: status.zone

FromCompositeFieldPath and ToCompositeFieldPath patches can also take a wildcarded field path in the toFieldPath parameter and patch each array element in the toFieldPath with the singular value provided in the fromFieldPath.

# Patch from the XR's spec.parameters.allowedIPs to the CIDRBlock elements
# inside the array spec.forProvider.firewallRules on the composed resource.
resources:
- name: exampleFirewall
  base:
    apiVersion: firewall.example.crossplane.io/v1beta1
    kind: Firewall
    spec:
      forProvider:
        firewallRules:
        - Action: "Allow"
          Destination: "example1"
          CIDRBlock: ""
        - Action: "Allow"
          Destination: "example2"
          CIDRBlock: ""
- type: FromCompositeFieldPath
  fromFieldPath: spec.parameters.allowedIP
  toFieldPath: spec.forProvider.firewallRules[*].CIDRBlock

FromEnvironmentFieldPath. This type patches from a field within the in-memory environment to a field within the composed resource. It's commonly used to expose a composed resource spec field as an XR spec field. Note that EnvironmentConfigs are an alpha feature and need to be enabled with the --enable-environment-configs flag on startup.

# Patch from the environment's tier.name field to the composed resource's
# spec.forProvider.settings.tier field.
- type: FromEnvironmentFieldPath
  fromFieldPath: tier.name
  toFieldPath: spec.forProvider.settings.tier

ToEnvironmentFieldPath. This type patches from a composed field to the in-memory environment. Note that, unlike ToCompositeFieldPath patches, this is executed before the composed resource is applied on the cluster which means that the status is not available. Note that EnvironmentConfigs are an alpha feature and need to be enabled with the --enable-environment-configs flag on startup.

# Patch from the environment's tier.name field to the composed resource's
# spec.forProvider.settings.tier field.
- type: ToEnvironmentFieldPath
  fromFieldPath: spec.forProvider.settings.tier
  toFieldPath: tier.name

Note that the field to be patched requires some initial value to be set.

CombineFromComposite. Combines multiple fields from the XR to produce one composed resource field.

# Patch from the XR's spec.parameters.location field and the
# metadata.labels[crossplane.io/claim-name] label to the composed
# resource's spec.forProvider.administratorLogin field.
- type: CombineFromComposite
  combine:
    # The patch will only be applied when all variables have non-zero values.
    variables:
    - fromFieldPath: spec.parameters.location
    - fromFieldPath: metadata.labels[crossplane.io/claim-name]
    strategy: string
    string:
      fmt: "%s-%s"
  toFieldPath: spec.forProvider.administratorLogin
  # By default Crossplane will skip the patch until all of the variables to be
  # combined have values. Set the fromFieldPath policy to 'Required' to instead
  # abort composition and return an error if a variable has no value.
  policy:
    fromFieldPath: Required

CombineFromEnvironment. Combines multiple fields from the in-memory environment to produce one composed resource field. Note that EnvironmentConfigs are an alpha feature and need to be enabled with the --enable-environment-configs flag on startup.

# Patch from the environments's location field and region to the composed
# resource's spec.forProvider.administratorLogin field.
- type: CombineFromEnvironment
  combine:
    # The patch will only be applied when all variables have non-zero values.
    variables:
    - fromFieldPath: location
    - fromFieldPath: region
    strategy: string
    string:
      fmt: "%s-%s"
  toFieldPath: spec.forProvider.administratorLogin

At the time of writing only the string combine strategy is supported. It uses Go string formatting to combine values, so if the XR's location was us-west and its claim name was db the composed resource's administratorLogin would be set to us-west-db.

CombineToComposite is the inverse of CombineFromComposite.

# Patch from the composed resource's spec.parameters.administratorLogin and
# status.atProvider.fullyQualifiedDomainName fields back to the XR's
# status.adminDSN field.
- type: CombineToComposite
  combine:
    variables:
      - fromFieldPath: spec.parameters.administratorLogin
      - fromFieldPath: status.atProvider.fullyQualifiedDomainName
    strategy: string
    # Here, our administratorLogin parameter and fullyQualifiedDomainName
    # status are formatted to a single output string representing a DSN.
    string:
      fmt: "mysql://%s@%s:3306/my-database-name"
  toFieldPath: status.adminDSN

CombineToEnvironment is the inverse of CombineFromEnvironment. Note that EnvironmentConfigs are an alpha feature and need to be enabled with the --enable-environment-configs flag on startup.

# Patch from the composed resource's spec.parameters.administratorLogin and
# spec.forProvider.domainName fields back to the environment's adminDSN field.
- type: CombineToEnvironment
  combine:
    variables:
      - fromFieldPath: spec.parameters.administratorLogin
      - fromFieldPath: spec.forProvider.domainName
    strategy: string
    # Here, our administratorLogin parameter and fullyQualifiedDomainName
    # status are formatted to a single output string representing a DSN.
    string:
      fmt: "mysql://%s@%s:3306/my-database-name"
  toFieldPath: adminDSN

PatchSet. References a named set of patches defined in the spec.patchSets array of a Composition.

# This is equivalent to specifying all of the patches included in the 'metadata'
# PatchSet.
- type: PatchSet
  patchSetName: metadata

The patchSets array may not contain patches of type: PatchSet. The transforms and patchPolicy fields are ignored by type: PatchSet.

Transform Types

You can use the following types of transform on a value being patched:

map. Transforms values using a map.

# If the value of the 'from' field is 'us-west', the value of the 'to' field
# will be set to 'West US'.
- type: map
  map:
    us-west: West US
    us-east: East US
    au-east: Australia East

match. A more complex version of map that can match different kinds of patterns. It should be used if more advanced pattern matchings than a simple string equality check are required. The result of the first matching pattern is used as the output of this transform. If no pattern matches, you can either fallback to a given fallbackValue or fallback to the input value by setting the fallbackTo field to Input.

# In the example below, if the value in the 'from' field is 'us-west', the
# value in the 'to' field will be set to 'West US'.
# If the value in the 'from' field is 'eu-west', the value in the 'to' field
# will be set to 'Unknown' because no pattern matches.
- type: match
  match:
    patterns:
      - type: literal # Not needed. This is the default.
        literal: us-west
        result: West US
      - type: regexp
        regexp: '^af-.*'
        result: Somewhere in Africa
    fallbackTo: Value # Not needed. This is the default.
    fallbackValue: Unknown

# If fallbackTo is set to Input, the output will be the input value if no
# pattern matches.
# In the example below, if the value in the 'from' field is 'us-west', the
# value in the 'to' field will be set to 'West US'.
# If the value in the 'from' field is 'eu-west', the value in the 'to' field
# will be set to 'eu-west' because no pattern matches.
- type: match
  match:
    patterns:
      - type: literal
        literal: us-west
        result: West US
      - type: regexp
        regexp: '^af-.*'
        result: Somewhere in Africa
    fallbackTo: Input

math. Transforms values using math. The input value must be an integer.

• math transform type Multiply, multiplies the input by the given value.
• math transform type ClampMin, sets a minimum value for the output.
• math transform type ClampMax, sets a maximum value for the output.

# If you omit the field type, by default type is set to `Multiply`
# If the value of the 'from' field is 2, the value of the 'to' field will be set
# to 4.
- type: math
  math:
    multiply: 2
    
# This is the same as above
# If the value of the 'from' field is 2, the value of the 'to' field will be set
# to 4.
- type: math
  math:
    type: Multiply
    multiply: 2

# If the value of the 'from' field is 3, the value of the 'to' field will
# be set to 4.
- type: math
  math:
    type: ClampMin
    clampMin: 4

# If the value of the 'from' field is 3, the value of the 'to' field will
# be set to 2.
- type: math
  math:
    type: ClampMax
    clampMax: 2

string. Transforms string values.

• string transform type Format, Currently only Go style fmt is supported. Go style fmt is supported.
• string transform type Convert, accepts one of ToUpper, ToLower, ToBase64, FromBase64, ToJson, ToSha1, ToSha256, ToSha512.
• string transform type TrimPrefix, accepts a string to be trimmed from the beginning of the input.
• string transform type TrimSuffix, accepts a string to be trimmed from the end of the input.
• string transform type Regexp, accepts a string for regexp to be applied to.

# If you omit the field type, by default type is set to `Format` 
# If the value of the 'from' field is 'hello', the value of the 'to' field will
# be set to 'hello-world'.
- type: string
  string:
    fmt: "%s-world"

# This is the same as above
# the value of the 'to' field will be set to 'hello-world'.
- type: string
  string:
    type: Format
    fmt: "%s-world"

# If the value of the 'from' field is 'hello', the value of the 'to' field will
# be set to 'HELLO'.
- type: string
  string:
    type: Convert
    convert: ToUpper

# If the value of the 'from' field is 'Hello', the value of the 'to' field will
# be set to 'hello'.
- type: string
  string:
    type: Convert
    convert: ToLower

# If the value of the 'from' field is 'Hello', the value of the 'to' field will
# be set to 'SGVsbG8='.
- type: string
  string:
     type: Convert
     convert: ToBase64

# If the value of the 'from' field is 'SGVsbG8=', the value of the 'to' field will
# be set to 'Hello'.
- type: string
  string:
     type: Convert
     convert: FromBase64

# If the value of the 'from' field is not nil, the value of the 'to' field will be
# set to raw JSON representation of the 'from' field.
- type: string
  string:
     type: Convert
     convert: ToJson

# The output will be the hash of the JSON representation of the 'from' field.
- type: string
  string:
    type: Convert
    convert: ToSha1 # alternatives: 'ToSha256' or 'ToSha512'

# If the value of the 'from' field is https://crossplane.io, the value of the 'to' field will
# be set to crossplane.io
- type: string
  string:
    type: TrimPrefix
    trim: 'https://'

# If the value of the 'from' field is my-string-test, the value of the 'to' field will
# be set to my-string
- type: string
  string:
     type: TrimSuffix
     trim: '-test'

# If the value of the 'from' field is 'arn:aws:iam::42:example, the value of the
# 'to' field will be set to "42". Note that the 'to' field is always a string. 
- type: string
  string:
     type: Regexp
     regexp:
      match: 'arn:aws:iam::(\d+):.*'
      group: 1  # Optional capture group. Omit to match the entire regexp.

convert. Transforms values of one type to another, for example from a string to an integer. The following values are supported by the from and to fields:

• string
• bool
• int
• int64
• float64

The strings 1, t, T, TRUE, true, and True are considered 'true', while 0, f, F, FALSE, false, and False are considered 'false'. The integer 1 and float 1.0 are considered true, while all other values are considered false. Similarly, boolean true converts to integer 1 and float 1.0, while false converts to 0 and 0.0.

# If the value to be converted is "1" (a string), the value of the 'toType'
# field will be set to 1 (an integer).
- type: convert
  convert:
   toType: int

Converting string to float64 additionally supports parsing string in K8s quantity format, such as 1000m or 500 Mi:

- type: convert
  convert:
   toType: float64
   format: quantity

Connection Details

Connection details secret of XR is an aggregated sum of the connection details of the composed resources. It's recommended that the author of XRD specify exactly which keys will be allowed in the XR connection secret by listing them in spec.connectionSecretKeys so that consumers of claims and XRs can see what they can expect in the connection details secret.

If spec.connectionSecretKeys is empty, then all keys of the aggregated connection details secret will be propagated.

You can derive the following types of connection details from a composed resource to be aggregated:

FromConnectionSecretKey. Derives an XR connection detail from a connection secret key of a composed resource.

# Derive the XR's 'user' connection detail from the 'username' key of the
# composed resource's connection secret.
- type: FromConnectionSecretKey
  name: user
  fromConnectionSecretKey: username

FromFieldPath. Derives an XR connection detail from a field path within the composed resource.

# Derive the XR's 'user' connection detail from the 'adminUser' status field of
# the composed resource.
- type: FromFieldPath
  name: user
  fromFieldPath: status.atProvider.adminUser

FromValue. Derives an XR connection detail from a fixed value.

# Always sets the XR's 'user' connection detail to 'admin'.
- type: FromValue
  name: user
  value: admin

Readiness Checks

Crossplane can use the following types of readiness check to determine whether a composed resource is ready (and therefore whether the XR and claim should be considered ready). Specify multiple readiness checks if multiple conditions must be met for a composed resource to be considered ready.

Note that if you don't specify any readiness checks Crossplane will consider the composed resource to be ready when its 'Ready' status condition becomes 'True'.

MatchString. Considers the composed resource to be ready when the value of a field within that resource matches a specified string.

# The composed resource will be considered ready when the 'state' status field
# matches the string 'Online'.
- type: MatchString
  fieldPath: status.atProvider.state
  matchString: "Online"

MatchInteger. Considers the composed resource to be ready when the value of a field within that resource matches a specified integer.

# The composed resource will be considered ready when the 'state' status field
# matches the integer 4.
- type: MatchInteger
  fieldPath: status.atProvider.state
  matchInteger: 4

NonEmpty. Considers the composed resource to be ready when a field exists in the composed resource. The name of this check can be a little confusing in that a field that exists with a zero value (e.g. an empty string or zero integer) is not considered to be 'empty', and thus will pass the readiness check.

# The composed resource will be considered ready if and when 'online' status
# field  exists.
- type: NonEmpty
  fieldPath: status.atProvider.online

None. Considers the composed resource to be ready as soon as it exists.

Composition validation

Crossplane uses a Validating Webhook to inform users of any potential errors in a Composition. By default webhooks only perform logical checks. logical checks enforce requirements that aren't explicitly defined in the schema but Crossplane assumes to hold at runtime.

Experimental validation with schemas

Enable experimental schema-aware validation in Crossplane through the --enable-composition-webhook-schema-validation feature flag. This enables Composition validation against available schemas in the cluster. For example, ensuring that fieldPaths are valid and source and destination types match taking into account provided transforms too.

The crossplane.io/composition-validation-mode annotation on the Composition allows setting one of two modes for schema validation:

• loose (default): Validates Compositions against required schemas. If a required schema is missing, schema validation stops, emits a warning and falls back to logical checks only.
• strict: Validates Compositions against required schemas, and rejects them when finding errors. Rejects any Compositions missing required schemas.

See the [Composition Validating Webhook design document][validation-design-doc] for more information about future development around schema-aware validation.

Disabling webhooks

Crossplane enables webhooks by default. Turn off webhooks by webhooks.enabled to false in the provided Helm Chart.

Missing Functionality

You might find while reading through this reference that Crossplane is missing some functionality you need to compose resources. If that's the case, please raise an issue with as much detail about your use case as possible. Please understand that the Crossplane maintainers are growing the feature set of the Composition type conservatively. We highly value the input of our users and community, but we also feel it's critical to avoid bloat and complexity. We therefore wish to carefully consider each new addition. We feel some features may be better suited for a real, expressive programming language and intend to build an alternative to the Composition type as it's documented here per this proposal.

Tips, Tricks, and Troubleshooting

In this section we'll cover some common tips, tricks, and troubleshooting steps for working with Composite Resources. If you're trying to track down why your Composite Resources aren't working the [Troubleshooting][trouble-ref] page also has some useful information.

Troubleshooting Claims and XRs

Crossplane relies heavily on status conditions and events for troubleshooting. You can see both using kubectl describe - for example:

# Describe the PostgreSQLInstance claim named my-db
kubectl describe postgresqlinstance.database.example.org my-db

Per Kubernetes convention, Crossplane keeps errors close to the place they happen. This means that if your claim is not becoming ready due to an issue with your Composition or with a composed resource you'll need to "follow the references" to find out why. Your claim will only tell you that the XR is not yet ready.

To follow the references:

1. Find your XR by running kubectl describe on your claim and looking for its "Resource Ref" (aka spec.resourceRef).
2. Run kubectl describe on your XR. This is where you'll find out about issues with the Composition you're using, if any.
3. If there are no issues but your XR doesn't seem to be becoming ready, take a look for the "Resource Refs" (or spec.resourceRefs) to find your composed resources.
4. Run kubectl describe on each referenced composed resource to determine whether it is ready and what issues, if any, it is encountering.

Composite Resource Connection Secrets

Claim and Composite Resource connection secrets are often derived from the connection secrets of the managed resources they compose. This is a common source of confusion because several things need to align for it to work:

1. The claim must specify the secret where the aggregated connection details should be written
   • This is the spec.writeConnectionSecretToRef field in a claim
   • If creating a composite resource directly (without a claim) then this same field must be set on your composite resource instead
2. The composite resource definition must state which connection details to aggregate from its children to publish to the claim
   • This is the spec.connectionSecretKeys field in a CompositeResourceDefinition
3. The composition must define where to write its aggregated connection details
   • This is the spec.writeConnectionSecretsToNamespace field in the Composition
4. Each child composed resource must define the connection details it publishes and where to write them
   • These are the connectionDetails and base.spec.writeConnectionSecretToRef fields of the composed resources

Finally, you can't currently edit a XRD's supported connection details. The XRD's spec.connectionSecretKeys is effectively immutable. This may change in future per this issue

Claiming an Existing Composite Resource

Most people create Composite Resources using a claim, but you can actually claim an existing Composite Resource as long as its a type of XR that offers a claim and no one else has already claimed it. To do so:

1. Set the spec.resourceRef of your claim to reference the existing XR.

   apiVersion: database.example.org/v1alpha1
   kind: PostgreSQLInstance
   metadata:
     name: example
     namespace: default
   spec:
     resourceRef:
       apiVersion: database.example.org/v1alpha1
       kind: XPostgreSQLInstance
       name: example-d4lmv
   

2. Make sure the rest of your claim's spec fields match the XR's.

If your claim's spec fields don't match the XR's Crossplane will still claim it but will then try to update the XR's spec fields to match the claim's.

Influencing External Names

The crossplane.io/external-name annotation has special meaning to Crossplane managed resources - it specifies the name (or identifier) of the resource in the external system, for example the actual name of a CloudSQLInstance in the GCP API. Some managed resources don't let you specify an external name - in those cases Crossplane will set it for you to whatever the external system requires.

If you add the crossplane.io/external-name annotation to a claim Crossplane will automatically propagate it when it creates an XR. It's good practice to have your Composition further propagate the annotation to one or more composed resources, but it's not required.

Mixing and Matching Providers

Crossplane has providers for many things in addition to the big clouds. Take a look at the Upbound Marketplace to find many of them. Keep in mind that you can mix and match managed resources from different providers within a Composition to create Composite Resources. For example you might use provider-aws and provider-sql to create an XR that provisions an RDSInstance then creates an SQL Database and User, or provider-gcp and provider-helm to create a GKECluster and deploy a Helm Chart Release to it.

Often when mixing and matching providers you'll need to compose a ProviderConfig for one provider that loads credentials from the connection secret of a managed resource from another provider. Sometimes you may need to use an intermediary XR to mutate the connection details to suit your needs. This example from provider-helm demonstrates using a GKE cluster connection secret as Helm ProviderConfig credentials.

Patching From One Composed Resource to Another or Itself

It's not possible to patch directly from one composed resource to another - i.e. from one entry in the spec.resources array of a Composition to another. It is however possible to achieve this by using the XR as an intermediary. To do so:

1. Use a ToCompositeFieldPath patch to patch from your source composed resource to the XR. Typically you'll want to patch to a status field or an annotation.
2. Use a FromCompositeFieldPath patch to patch from the 'intermediary' field you patched to in step 1 to a field on the destination composed resource.

Note that the source and the target composed resource can be the same.

[xr-ref]: {{<ref "#compositions" >}} [managed-resources]: {{<ref "managed-resources" >}} [validation-design-doc]: https://github.com/crossplane/crossplane/blob/master/design/design-doc-composition-validating-webhook.md