--- title: GCP Quickstart Part 2 weight: 120 tocHidden: true --- {{< hint "important" >}} This guide is part 2 of a series. Follow **[part 1]({{}})** to install Crossplane and connect your Kubernetes cluster to GCP. **[Part 3]({{}})** covers patching _composite resources_ and using Crossplane _packages_. {{< /hint >}} This section creates a _[Composition](#create-a-composition)_, _[Composite Resource Definition](#define-a-composite-resource)_ and a _[Claim](#create-a-claim)_ to create a custom Kubernetes API to create GCP resources. This custom API is a _Composite Resource_ (XR) API. ## Prerequisites * Complete [quickstart part 1]({{}}) connecting Kubernetes to GCP. * a GCP account with permissions to create a GCP [storage bucket](https://cloud.google.com/storage) and a [Pub/Sub topic](https://cloud.google.com/pubsub). {{}} 1. Add the Crossplane Helm repository and install Crossplane. ```shell helm repo add \ crossplane-stable https://charts.crossplane.io/stable helm repo update && helm install crossplane \ crossplane-stable/crossplane \ --namespace crossplane-system \ --create-namespace ``` 2. When the Crossplane pods finish installing and are ready, apply the GCP Provider. ```yaml {label="provider",copy-lines="all"} cat <}} The [GCP documentation](https://cloud.google.com/iam/docs/creating-managing-service-account-keys) provides information on how to generate a service account JSON file. {{< /hint >}} 4. Create a Kubernetes secret from the GCP JSON file ```shell {label="kube-create-secret",copy-lines="all"} kubectl create secret \ generic gcp-secret \ -n crossplane-system \ --from-file=creds=./gcp-credentials.json ``` 5. Create a _ProviderConfig_ Include your {{< hover label="providerconfig" line="7" >}}GCP project ID{{< /hover >}} in the _ProviderConfig_ settings. {{< hint type="warning" >}} Find your GCP project ID from the `project_id` field of the `gcp-credentials.json` file. {{< /hint >}} {{< editCode >}} ```yaml {label="providerconfig",copy-lines="all"} cat <$@ credentials: source: Secret secretRef: namespace: crossplane-system name: gcp-secret key: creds EOF ``` {{< /editCode >}} {{}} ## Create a composition [Part 1]({{}}) created a single _managed resource_. A _Composition_ is a template to create one or more _managed resources_ at the same time. This sample _composition_ creates a Pub/Sub instance and associated GCP storage bucket. {{< hint "note" >}} This example comes from part of the GCP [Stream messages from Pub/Sub by using Dataflow](https://cloud.google.com/pubsub/docs/stream-messages-dataflow) guide. {{< /hint >}} To create a _composition_, first define each individual managed resource. ### Create a storage bucket object Define a `bucket` resource using the configuration from the previous section: {{< hint "note" >}} Don't apply this configuration. This YAML is part of a larger definition. {{< /hint >}} ```yaml apiVersion: storage.gcp.upbound.io/v1beta1 kind: Bucket metadata: name: crossplane-quickstart-bucket spec: forProvider: location: US providerConfigRef: name: default ``` ### Create a Pub/Sub topic resource Next, define a Pub/Sub `topic` resource. {{< hint "tip" >}} The [Upbound Marketplace](https://marketplace.upbound.io/) provides [schema documentation](https://marketplace.upbound.io/providers/upbound/provider-gcp/v0.28.0/resources/pubsub.gcp.upbound.io/Topic/v1beta1) for a `topic` resource. {{< /hint >}} The _GCP Provider_ defines the {{}}apiVersion{{}} and {{}}kind{{}}. A Pub/Sub topic doesn't have requirements but using {{}}messageStoragePolicy.allowedPersistenceRegions{{< /hover >}} can keep messages stored in the same location as the storage bucket. ```yaml {label="topicMR"} apiVersion: pubsub.gcp.upbound.io/v1beta1 kind: Topic metadata: name: crossplane-quickstart-topic spec: forProvider: messageStoragePolicy: - allowedPersistenceRegions: - "us-central1" ``` {{< hint "note" >}} Pub/Sub topic specifics are beyond the scope of this guide. Read the GCP [Pub/Sub API reference](https://cloud.google.com/compute/docs/apis) for more information. {{}} ### Create the composition object The _composition_ combines the two resource definitions. A {{}}Composition{{}} comes from the {{}}Crossplane{{}} API resources. Create any {{}}name{{}} for this _composition_. ```yaml {label="compName"} apiVersion: apiextensions.crossplane.io/v1 kind: Composition metadata: name: topic-with-bucket ``` Add the resources to the {{}}spec.resources{{}} section of the _composition_. Give each resource a {{}}name{{}} and put the resource definition under the {{}}base{{}} key. {{< hint "note" >}} Don't include resource `metadata` under the {{}}base{{}} key. {{< /hint >}} ```yaml {label="specResources"} apiVersion: apiextensions.crossplane.io/v1 kind: Composition metadata: name: topic-with-bucket spec: resources: - name: crossplane-quickstart-bucket base: apiVersion: storage.gcp.upbound.io/v1beta1 kind: Bucket spec: forProvider: location: US - name: crossplane-quickstart-topic base: apiVersion: pubsub.gcp.upbound.io/v1beta1 kind: Topic spec: forProvider: messageStoragePolicy: - allowedPersistenceRegions: - "us-central1" ``` _Compositions_ are a template for generating resources. A _composite resource_ actually creates the resources. A _composition_ defines which _composite resources_ can use this template. _Compositions_ do this with the {{}}spec.compositeTypeRef{{}} definition. {{< hint "tip" >}} Crossplane recommends prefacing the `kind` with an `X` to show it's a Composition. {{< /hint >}} ```yaml {label="compRef"} apiVersion: apiextensions.crossplane.io/v1 kind: Composition metadata: name: topic-with-bucket spec: compositeTypeRef: apiVersion: custom-api.example.org/v1alpha1 kind: XTopicBucket resources: # Removed for Brevity ``` A _composite resource_ is actually a custom Kubernetes API type you define. The platform team controls the kind, API endpoint and version. With this {{}}spec.compositeTypeRef{{}} Crossplane allows _composite resources_ from the API group {{}}custom-api.example.org{{}} that are of {{}}kind: XTopicBucket{{}} to use this template to create resources. No other API group or kind can use this template. ### Apply the composition Apply the full _Composition_ to your Kubernetes cluster. ```yaml {copy-lines="all"} cat <}} _Composite resource definitions_ are also called `XRDs` for short. {{< /hint >}} Just like a _composition_ the {{}}composite resource definition{{}} is part of the {{}}Crossplane{{}} API group. The _XRD_ {{}}name{{}} is the new API endpoint. {{< hint "tip" >}} Crossplane recommends using a plural name for the _XRD_ {{}}name{{}}. {{< /hint >}} ```yaml {label="xrdName"} apiVersion: apiextensions.crossplane.io/v1 kind: CompositeResourceDefinition metadata: name: xtopicbuckets.custom-api.example.org ``` The _XRD's_ {{}}spec{{}} defines the new custom API. ### Define the API endpoint and kind First, define the new API {{}}group{{}}. Next, create the API {{}}kind{{}} and {{}}plural{{}}. ```yaml {label="xrdGroup"} apiVersion: apiextensions.crossplane.io/v1 kind: CompositeResourceDefinition metadata: name: xtopicbuckets.custom-api.example.org spec: group: custom-api.example.org names: kind: XTopicBucket plural: xtopicbuckets ``` {{}} The _XRD_ {{}}group{{}} matches the _composition_ {{}}apiVersion{{}} and the _XRD_ {{}}kind{{}} matches the _composition_ {{}}compositeTypeRef.kind{{}}. ```yaml {label="noteComp"} kind: Composition # Removed for brevity spec: compositeTypeRef: apiVersion: custom-api.example.org/v1alpha1 kind: XTopicBucket ``` {{< /hint >}} ### Set the API version In Kubernetes, all API endpoints have a version to show the stability of the API and track revisions. Apply a version to the _XRD_ with a {{}}versions.name{{}}. This matches the {{}}compositeTypeRef.apiVersion{{}} _XRDs_ require both {{}}versions.served{{}} and {{}}versions.referenceable{{}}. ```yaml {label="xrdVersion"} apiVersion: apiextensions.crossplane.io/v1 kind: CompositeResourceDefinition metadata: name: xtopicbuckets.custom-api.example.org spec: group: custom-api.example.org names: kind: XTopicBucket plural: xtopicbuckets versions: - name: v1alpha1 served: true referenceable: true ``` {{}} For more information on defining versions in Kubernetes read the [API versioning](https://kubernetes.io/docs/reference/using-api/#api-versioning) section of the Kubernetes documentation. {{< /hint >}} ### Create the API schema With an API endpoint named, now define the API schema, or what's allowed inside the `spec` of the new Kubernetes object. {{< hint "note" >}} _XRDs_ follow the Kubernetes [_custom resource definition_ rules for schemas](https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/#specifying-a-structural-schema). {{}} Place the API {{< hover label="xrdSchema" line="8" >}}schema{{}} under the {{< hover label="xrdSchema" line="7" >}}version.name{{}} The _XRD_ type defines the next lines. They're always the same. {{< hover label="xrdSchema" line="9" >}}openAPIV3Schema{{}} specifies how the schema gets validated. Next, the entire API is an {{< hover label="xrdSchema" line="10" >}}object{{}} with a {{< hover label="xrdSchema" line="11" >}}property{{}} of {{< hover label="xrdSchema" line="12" >}}spec{{}}. The {{< hover label="xrdSchema" line="12" >}}spec{{}} is also an {{< hover label="xrdSchema" line="13" >}}object{{}} with {{< hover label="xrdSchema" line="14" >}}properties{{}}. ```yaml {label="xrdSchema"} apiVersion: apiextensions.crossplane.io/v1 kind: CompositeResourceDefinition # Removed for brevity spec: # Removed for brevity versions: - name: v1alpha1 schema: openAPIV3Schema: type: object properties: spec: type: object properties: ``` {{< hint "tip" >}} For more information on the values allowed in a _composite resource definition_ view its schema with `kubectl explain xrd` {{< /hint >}} Now, define the custom API. Your custom API continues under the last {{}}properties{{}} definition in the previous example. This custom API has one setting: * {{}}location{{}} - where to deploy the resources, a choice of "EU" or "US." Users can't change any other settings of the storage bucket or Pub/Sub topic. The{{}}location{{}} is a {{}}string{{}} and matches the regular expression that's {{}}oneOf{{}} {{}}EU{{}} or {{}}US{{}}. This API requires the setting {{}}location{{}}. ```yaml {label="customAPI"} # Removed for brevity # schema.openAPIV3Schema.type.properties.spec properties: location: type: string oneOf: - pattern: '^EU$' - pattern: '^US$' required: - location ``` ### Enable claims to the API Tell this _XRD_ to offer a _claim_ by defining the _claim_ API endpoint under the _XRD_ {{}}spec{{< /hover >}}. {{< hint "tip" >}} Crossplane recommends a _Claim_ {{}}kind{{}} match the _Composite Resource Definition_ (XRD) {{}}kind{{}}, without the preceding `X`. {{< /hint >}} ```yaml {label="XRDclaim"} apiVersion: apiextensions.crossplane.io/v1 kind: CompositeResourceDefinition # Removed for brevity spec: # Removed for brevity names: kind: XTopicBucket plural: xtopicbuckets claimNames: kind: TopicBucket plural: topicbuckets ``` {{}} The [Claims](#create-a-claim) section later in this guide discusses _claims_. {{< /hint >}} ### Apply the composite resource definition Apply the complete _XRD_ to your Kubernetes cluster. ```yaml {copy-lines="all"} cat <}}group{{}} becomes the _composite resource_ {{}}apiVersion{{}}. The _XRD_ {{}}kind{{}} is the _composite resource_ {{}}kind{{}} The _XRD_ API {{}}spec{{}} defines the _composite resource_ {{}}spec{{}}. The _XRD_ {{}}properties{{}} section defines the options for the _composite resource_ {{}}spec{{}}. The one option is {{}}location{{}} and it can be either {{}}EU{{}} or {{}}US{{}}. This _composite resource_ uses {{}}location: US{{}}. ### Apply the composite resource Apply the composite resource to the Kubernetes cluster. ```yaml {label="xr", copy-lines="all"} cat <}} Use `kubectl get ` to view a specific `kind` of _composite resource_. View all _composite resources_ with `kubectl get composite`. {{< /hint >}} ```shell {copy-lines="1"} kubectl get XTopicBucket NAME SYNCED READY COMPOSITION AGE my-composite-resource True True topic-with-bucket 2m3s ``` Both `SYNCED` and `READY` are `True` when Crossplane created the GCP resources. Now look at the GCP storage `bucket` and Pub/Sub `topic` _managed resources_ with `kubectl get bucket` and `kubectl get topic`. ```shell {copy-lines="1"} kubectl get bucket NAME READY SYNCED EXTERNAL-NAME AGE my-composite-resource-m6lbx True True my-composite-resource-m6lbx 4m34s ``` ```shell {copy-lines="1"} kubectl get topics NAME READY SYNCED EXTERNAL-NAME AGE my-composite-resource-88vzp True True my-composite-resource-88vzp 4m48s ``` The _composite resource_ automatically generated both _managed resources_. Using `kubectl describe` on a _managed resource_ shows the `Owner References` is the _composite resource_. ```yaml {copy-lines="1"} kubectl describe bucket | grep "Owner References" -A5 Owner References: API Version: custom-api.example.org/v1alpha1 Block Owner Deletion: true Controller: true Kind: XTopicBucket Name: my-composite-resource ``` Each _composite resource_ creates and owns a unique set of _managed resources_. If you create a second _composite resource_ Crossplane creates a new storage `bucket` and Pub/Sub `topic`. ```yaml {label="xr", copy-lines="all"} cat <}} Delete a specific _composite resource_ with `kubectl delete ` or `kubectl delete composite `. {{< /hint >}} Delete the second composition ```shell kubectl delete XTopicBucket my-second-composite-resource ``` {{}} There may a delay in deleting the _managed resources_. Crossplane is making API calls to GCP and waits for GCP to confirm they deleted the resources before updating the state in Kubernetes. {{}} Now a single bucket and topic exist. ```shell {copy-lines="1"} kubectl get bucket NAME READY SYNCED EXTERNAL-NAME AGE my-composite-resource-m6lbx True True my-composite-resource-m6lbx 11m ``` ```shell {copy-lines="1"} kubectl get topic NAME READY SYNCED EXTERNAL-NAME AGE my-composite-resource-88vzp True True my-composite-resource-88vzp 11m ``` Delete the other _composite resource_ to remove the last `bucket` and `table` _managed resources_. ```shell kubectl delete xtopicbucket my-composite-resource ``` _Composite resources_ are great for creating one or more related resources against a template, but all _composite resources_ exist at the Kubernetes "cluster level." There's no isolation between _composite resources_. Crossplane uses _claims_ to create resources with namespace isolation. ## Create a claim _Claims_, just like _composite resources_ use the custom API defined in the _XRD_. Unlike a _composite resource_, Crossplane can create _claims_ in a namespace. ### Create a new Kubernetes namespace Create a new namespace with `kubectl create namespace`. ```shell kubectl create namespace test ``` Look at the _XRD_ to see the parameters for the _claim_. A _claim_ uses the same {{}}group{{}} a _composite resource_ uses but a different {{}}kind{{}}. ```yaml {label="XRDclaim2"} apiVersion: apiextensions.crossplane.io/v1 kind: CompositeResourceDefinition # Removed for brevity spec: # Removed for brevity group: custom-api.example.org claimNames: kind: TopicBucket plural: topicbuckets ``` Like the _composite resource_, create a new object with the {{}}custom-api.example.org{{}} API endpoint. The _XRD_ {{}}claimNames.kind{{}} defines the {{}}kind{{}}. The {{}}spec{{}} uses the same API options as the _composite resource_. ### Apply the claim Apply the _claim_ to your Kubernetes cluster. ```yaml {label="claim", copy-lines="all"} cat <}} View claims with `kubectl get ` or use `kubectl get claim` to view all _claims_. {{}} ```shell {copy-lines="1"} kubectl get TopicBucket -n test NAME SYNCED READY CONNECTION-SECRET AGE claimed-topic-with-bucket True True 4m37s ``` When Crossplane creates a _claim_, a unique _composite resource_ is also created. View the new _composite resource_ with `kubectl get xtopicbucket`. ```shell {copy-lines="1"} kubectl get xtopicbucket NAME SYNCED READY COMPOSITION AGE claimed-topic-with-bucket-7k2lj True True topic-with-bucket 4m58s ``` The _composite resource_ exists at the "cluster scope" while the _claim_ exists at the "namespace scope." Create a second namespace and a second claim. ```shell {copy-lines="all"} kubectl create namespace test2 cat <}}) of this guide covers _composition patches_ and making all this configuration portable in Crossplane _packages_. ## Next steps * **[Continue to part 3]({{< ref "provider-gcp-part-3">}})** to create a learn about _patching_ resources and creating Crossplane _packages_. * Explore GCP resources that Crossplane can configure in the [Provider CRD reference](https://marketplace.upbound.io/providers/upbound/provider-family-gcp/). * Join the [Crossplane Slack](https://slack.crossplane.io/) and connect with Crossplane users and contributors.