GCP Quickstart Update (#369)

This commit is contained in:
Pete Lumbis 2023-03-13 10:10:50 -04:00 committed by GitHub
parent 3f4cb79be4
commit 686070b8b4
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
6 changed files with 2732 additions and 68 deletions

View File

@ -0,0 +1,967 @@
---
title: GCP Quickstart Part 2
weight: 120
tocHidden: true
---
{{< hint "important" >}}
This guide is part 2 of a series. Follow **[part 1]({{<ref "provider-gcp" >}})**
to install Crossplane and connect your Kubernetes cluster to GCP.
**[Part 3]({{<ref "provider-gcp-part-3">}})** covers patching
_composite resources_ and using Crossplane _packages_.
{{< /hint >}}
This section creates a _[Composition](#create-a-composition)_,
_[Composite Resource Definition](#define-a-composite-resource)_ and a
_[Claim](#create-a-claim)_
to create a custom Kubernetes API to create GCP resources. This custom API is a
_Composite Resource_ (XR) API.
## Prerequisites
* Complete [quickstart part 1]({{<ref "provider-gcp" >}}) connecting Kubernetes
to GCP.
* a GCP account with permissions to create a GCP
[storage bucket](https://cloud.google.com/storage) and a
[Pub/Sub topic](https://cloud.google.com/pubsub).
{{<expand "Skip part 1 and just get started" >}}
1. Add the Crossplane Helm repository and install Crossplane.
```shell
helm repo add \
crossplane-stable https://charts.crossplane.io/stable
helm repo update
&&
helm install crossplane \
crossplane-stable/crossplane \
--namespace crossplane-system \
--create-namespace
```
2. When the Crossplane pods finish installing and are ready, apply the GCP Provider.
```yaml {label="provider",copy-lines="all"}
cat <<EOF | kubectl apply -f -
apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
name: upbound-provider-gcp
spec:
package: xpkg.upbound.io/upbound/provider-gcp:v0.28.0
EOF
```
3. Create a file called `gcp-credentials.json` with your GCP service account
JSON file.
{{< hint type="tip" >}}
The
[GCP documentation](https://cloud.google.com/iam/docs/creating-managing-service-account-keys)
provides information on how to generate a service account JSON file.
{{< /hint >}}
4. Create a Kubernetes secret from the GCP JSON file
```shell {label="kube-create-secret",copy-lines="all"}
kubectl create secret \
generic gcp-secret \
-n crossplane-system \
--from-file=creds=./gcp-credentials.json
```
5. Create a _ProviderConfig_
Include your
{{< hover label="providerconfig" line="7" >}}GCP project ID{{< /hover >}} in the
_ProviderConfig_ settings.
{{< hint type="warning" >}}
Find your GCP project ID from the `project_id` field of the
`gcp-credentials.json` file.
{{< /hint >}}
{{< editCode >}}
```yaml {label="providerconfig",copy-lines="all"}
cat <<EOF | kubectl apply -f -
apiVersion: gcp.upbound.io/v1beta1
kind: ProviderConfig
metadata:
name: default
spec:
projectID: $$<PROJECT_ID>$$
credentials:
source: Secret
secretRef:
namespace: crossplane-system
name: gcp-secret
key: creds
EOF
```
{{< /editCode >}}
{{</expand >}}
## Create a composition
[Part 1]({{<ref "provider-gcp" >}}) created a single _managed resource_.
A _Composition_ is a template to create one or more _managed resources_ at the
same time.
This sample _composition_ creates a Pub/Sub instance and associated GCP storage
bucket.
{{< hint "note" >}}
This example comes from part of the GCP
[Stream messages from Pub/Sub by using Dataflow](https://cloud.google.com/pubsub/docs/stream-messages-dataflow)
guide.
{{< /hint >}}
To create a _composition_, first define each individual managed resource.
### Create a storage bucket object
Define a `bucket` resource using the configuration from the previous section:
{{< hint "note" >}}
Don't apply this configuration. This YAML is part of a larger
definition.
{{< /hint >}}
```yaml
apiVersion: storage.gcp.upbound.io/v1beta1
kind: Bucket
metadata:
name: crossplane-quickstart-bucket
spec:
forProvider:
location: US
providerConfigRef:
name: default
```
### Create a Pub/Sub topic resource
Next, define a Pub/Sub `topic` resource.
{{< hint "tip" >}}
The [Upbound Marketplace](https://marketplace.upbound.io/) provides
[schema documentation](https://marketplace.upbound.io/providers/upbound/provider-gcp/v0.28.0/resources/pubsub.gcp.upbound.io/Topic/v1beta1)
for a `topic` resource.
{{< /hint >}}
The _GCP Provider_ defines the
{{<hover line="1" label="topicMR">}}apiVersion{{</hover>}}
and
{{<hover line="2" label="topicMR">}}kind{{</hover>}}.
A Pub/Sub topic doesn't have requirements but using
{{<hover line="8" label="topicMR">}}messageStoragePolicy.allowedPersistenceRegions{{< /hover >}}
can keep messages stored in the same location as the storage bucket.
```yaml {label="topicMR"}
apiVersion: pubsub.gcp.upbound.io/v1beta1
kind: Topic
metadata:
name: crossplane-quickstart-topic
spec:
forProvider:
messageStoragePolicy:
- allowedPersistenceRegions:
- "us-central1"
```
{{< hint "note" >}}
Pub/Sub topic specifics are beyond the scope of this guide. Read the GCP
[Pub/Sub API reference](https://cloud.google.com/compute/docs/apis)
for more information.
{{</hint >}}
### Create the composition object
The _composition_ combines the two resource definitions.
A
{{<hover label="compName" line="2">}}Composition{{</ hover>}} comes from the
{{<hover label="compName" line="1">}}Crossplane{{</ hover>}}
API resources.
Create any {{<hover label="compName" line="4">}}name{{</ hover>}} for this _composition_.
```yaml {label="compName"}
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
name: topic-with-bucket
```
Add the resources to the
{{<hover label="specResources" line="5">}}spec.resources{{</ hover>}}
section of the _composition_.
Give each resource a
{{<hover label="specResources" line="7">}}name{{</ hover>}}
and put the resource definition under the
{{<hover label="specResources" line="8">}}base{{</ hover>}}
key.
{{< hint "note" >}}
Don't include resource `metadata` under the
{{<hover label="specResources" line="8">}}base{{</ hover>}} key.
{{< /hint >}}
```yaml {label="specResources"}
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
name: topic-with-bucket
spec:
resources:
- name: crossplane-quickstart-bucket
base:
apiVersion: storage.gcp.upbound.io/v1beta1
kind: Bucket
spec:
forProvider:
location: US
- name: crossplane-quickstart-topic
base:
apiVersion: pubsub.gcp.upbound.io/v1beta1
kind: Topic
spec:
forProvider:
messageStoragePolicy:
- allowedPersistenceRegions:
- "us-central1"
```
_Compositions_ are a template for generating resources. A
_composite resource_ actually creates the resources.
A _composition_ defines which _composite resources_ can use this
template.
_Compositions_ do this with the
{{<hover label="compRef" line="6">}}spec.compositeTypeRef{{</ hover>}}
definition.
{{< hint "tip" >}}
Crossplane recommends prefacing the `kind` with an `X` to show it's a
Composition.
{{< /hint >}}
```yaml {label="compRef"}
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
name: topic-with-bucket
spec:
compositeTypeRef:
apiVersion: custom-api.example.org/v1alpha1
kind: XTopicBucket
resources:
# Removed for Brevity
```
A _composite resource_ is actually a custom Kubernetes API type you define. The
platform team controls the kind, API endpoint and version.
<!-- vale gitlab.SentenceLength = NO -->
<!-- Length is because of shortcodes, ignore -->
With this {{<hover label="compRef" line="6">}}spec.compositeTypeRef{{</ hover>}}
Crossplane allows _composite resources_ from the API group
{{<hover label="compRef" line="7">}}custom-api.example.org{{</ hover>}}
that are of
{{<hover label="compRef" line="8">}}kind: XTopicBucket{{</ hover>}}
to use this template to create resources. No other API group or kind can use
this template.
<!-- vale gitlab.SentenceLength = YES -->
### Apply the composition
Apply the full _Composition_ to your Kubernetes cluster.
```yaml {copy-lines="all"}
cat <<EOF | kubectl apply -f -
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
name: topic-with-bucket
spec:
compositeTypeRef:
apiVersion: custom-api.example.org/v1alpha1
kind: XTopicBucket
resources:
- name: crossplane-quickstart-bucket
base:
apiVersion: storage.gcp.upbound.io/v1beta1
kind: Bucket
spec:
forProvider:
location: US
- name: crossplane-quickstart-topic
base:
apiVersion: pubsub.gcp.upbound.io/v1beta1
kind: Topic
spec:
forProvider:
messageStoragePolicy:
- allowedPersistenceRegions:
- "us-central1"
EOF
```
Confirm the _composition_ exists with `kubectl get composition`
```shell {copy-lines="1"}
kubectl get composition
NAME AGE
topic-with-bucket 8s
```
## Define a composite resource
The _composition_ that was just created limited which _composite resources_ can
use that template.
A _composite resource_ is a custom API defined by the platform team.
A _composite resource definition_ defines the schema for a _composite resource_.
A _composite resource definition_ installs the custom API type into Kubernetes
and defines what `spec` keys and values are valid when calling this new custom API.
Before creating a _composite resource_ Crossplane requires a _composite resource definition_.
{{< hint "tip" >}}
_Composite resource definitions_ are also called `XRDs` for short.
{{< /hint >}}
Just like a _composition_ the
{{<hover label="xrdName" line="2" >}}composite resource definition{{</hover>}}
is part of the
{{<hover label="xrdName" line="1" >}}Crossplane{{</hover>}}
API group.
The _XRD_ {{<hover label="xrdName" line="4" >}}name{{</hover>}} is the new
API endpoint.
{{< hint "tip" >}}
Crossplane recommends using a plural name for the _XRD_
{{<hover label="xrdName" line="4" >}}name{{</hover>}}.
{{< /hint >}}
```yaml {label="xrdName"}
apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
name: xtopicbuckets.custom-api.example.org
```
The _XRD's_
{{<hover label="xrdGroup" line="5" >}}spec{{</hover>}} defines the new custom
API.
### Define the API endpoint and kind
First, define the new API
{{<hover label="xrdGroup" line="6" >}}group{{</hover>}}.
Next, create the API {{<hover label="xrdGroup" line="8" >}}kind{{</hover>}} and
{{<hover label="xrdGroup" line="9" >}}plural{{</hover>}}.
```yaml {label="xrdGroup"}
apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
name: xtopicbuckets.custom-api.example.org
spec:
group: custom-api.example.org
names:
kind: XTopicBucket
plural: xtopicbuckets
```
{{<hint "note" >}}
The _XRD_ {{<hover label="xrdGroup" line="6" >}}group{{</hover>}} matches the _composition_ {{<hover label="noteComp"
line="5">}}apiVersion{{</hover>}} and the
_XRD_ {{<hover label="xrdGroup" line="8" >}}kind{{</hover>}} matches the _composition_
{{<hover label="noteComp" line="6">}}compositeTypeRef.kind{{</hover>}}.
```yaml {label="noteComp"}
kind: Composition
# Removed for brevity
spec:
compositeTypeRef:
apiVersion: custom-api.example.org/v1alpha1
kind: XTopicBucket
```
{{< /hint >}}
### Set the API version
In Kubernetes, all API endpoints have a version to show the stability of the API
and track revisions.
Apply a version to the _XRD_ with a
{{<hover label="xrdVersion" line="11">}}versions.name{{</hover>}}.
This matches the
{{<hover label="noteComp"line="5">}}compositeTypeRef.apiVersion{{</hover>}}
_XRDs_ require both
{{<hover label="xrdVersion" line="12">}}versions.served{{</hover>}}
and
{{<hover label="xrdVersion" line="13">}}versions.referenceable{{</hover>}}.
```yaml {label="xrdVersion"}
apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
name: xtopicbuckets.custom-api.example.org
spec:
group: custom-api.example.org
names:
kind: XTopicBucket
plural: xtopicbuckets
versions:
- name: v1alpha1
served: true
referenceable: true
```
{{<hint "note" >}}
For more information on defining versions in Kubernetes read the
[API versioning](https://kubernetes.io/docs/reference/using-api/#api-versioning)
section of the Kubernetes documentation.
{{< /hint >}}
### Create the API schema
With an API endpoint named, now define the API schema, or what's allowed
inside the `spec` of the new Kubernetes object.
{{< hint "note" >}}
_XRDs_ follow the Kubernetes
[_custom resource definition_ rules for schemas](https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/#specifying-a-structural-schema).
{{</hint >}}
Place the API
{{< hover label="xrdSchema" line="8" >}}schema{{</hover>}}
under the
{{< hover label="xrdSchema" line="7" >}}version.name{{</hover>}}
The _XRD_ type defines the next lines. They're always the same.
<!-- vale write-good.TooWordy = NO -->
<!-- allow "validate" -->
{{< hover label="xrdSchema" line="9" >}}openAPIV3Schema{{</hover>}} specifies
how the schema gets validated.
<!-- vale write-good.TooWordy = YES -->
Next, the entire API is an
{{< hover label="xrdSchema" line="10" >}}object{{</hover>}}
with a
{{< hover label="xrdSchema" line="11" >}}property{{</hover>}} of
{{< hover label="xrdSchema" line="12" >}}spec{{</hover>}}.
The
{{< hover label="xrdSchema" line="12" >}}spec{{</hover>}} is also an
{{< hover label="xrdSchema" line="13" >}}object{{</hover>}} with
{{< hover label="xrdSchema" line="14" >}}properties{{</hover>}}.
```yaml {label="xrdSchema"}
apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
# Removed for brevity
spec:
# Removed for brevity
versions:
- name: v1alpha1
schema:
openAPIV3Schema:
type: object
properties:
spec:
type: object
properties:
```
{{< hint "tip" >}}
For more information on the values allowed in a _composite resource definition_ view its schema with
`kubectl explain xrd`
{{< /hint >}}
Now, define the custom API. Your custom API continues under the last
{{<hover label="xrdSchema" line="14">}}properties{{</hover>}} definition in the
previous example.
This custom API has one setting:
<!-- vale Google.We = NO -->
* {{<hover label="customAPI" line="4" >}}location{{</hover >}} - where to deploy
the resources, a choice of "EU" or "US."
Users can't change any other settings of the storage bucket or Pub/Sub topic.
The{{<hover label="customAPI" line="4" >}}location{{</hover >}}
is a {{<hover label="customAPI" line="5" >}}string{{</hover >}}
and matches the regular expression that's
{{<hover label="customAPI" line="6" >}}oneOf{{</hover >}}
{{<hover label="customAPI" line="7" >}}EU{{</hover >}}
or
{{<hover label="customAPI" line="8" >}}US{{</hover >}}.
This API requires the setting
{{<hover label="customAPI" line="10" >}}location{{</hover >}}.
```yaml {label="customAPI"}
# Removed for brevity
# schema.openAPIV3Schema.type.properties.spec
properties:
location:
type: string
oneOf:
- pattern: '^EU$'
- pattern: '^US$'
required:
- location
```
### Enable claims to the API
Tell this _XRD_ to offer a _claim_ by defining the _claim_ API endpoint under
the _XRD_ {{<hover label="XRDclaim" line="4">}}spec{{< /hover >}}.
{{< hint "tip" >}}
Crossplane recommends a _Claim_
{{<hover label="XRDclaim" line="10" >}}kind{{</hover>}} match the
_Composite Resource Definition_ (XRD)
{{<hover label="XRDclaim" line="7" >}}kind{{</ hover>}},
without the preceding `X`.
{{< /hint >}}
```yaml {label="XRDclaim"}
apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
# Removed for brevity
spec:
# Removed for brevity
names:
kind: XTopicBucket
plural: xtopicbuckets
claimNames:
kind: TopicBucket
plural: topicbuckets
```
{{<hint "note" >}}
The [Claims](#create-a-claim) section later in this guide discusses _claims_.
{{< /hint >}}
### Apply the composite resource definition
Apply the complete _XRD_ to your Kubernetes cluster.
```yaml {copy-lines="all"}
cat <<EOF | kubectl apply -f -
apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
name: xtopicbuckets.custom-api.example.org
spec:
group: custom-api.example.org
names:
kind: XTopicBucket
plural: xtopicbuckets
versions:
- name: v1alpha1
served: true
referenceable: true
schema:
openAPIV3Schema:
type: object
properties:
spec:
type: object
properties:
location:
type: string
oneOf:
- pattern: '^EU$'
- pattern: '^US$'
required:
- location
claimNames:
kind: TopicBucket
plural: topicbuckets
EOF
```
Verify Kubernetes created the XRD with `kubectl get xrd`
```shell {copy-lines="1",label="getXRD"}
kubectl get xrd
NAME ESTABLISHED OFFERED AGE
xtopicbuckets.custom-api.example.org True True 9s
```
## Create a composite resource
Creating an _XRD_ allows the creation _composite resources_.
A _composite resource_ uses the custom API created in the _XRD_.
The _XRD_ maps the _composite resource_ values to the _composition_ template and
creates new _managed resources_.
Looking at part of the _XRD_:
```yaml {label="xrdSnip"}
apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
# Removed for brevity
spec:
group: custom-api.example.org
names:
kind: XTopicBucket
# Removed for brevity
spec:
type: object
properties:
location:
type: string
oneOf:
- pattern: '^EU$'
- pattern: '^US$'
```
The _XRD_ {{<hover label="xrdSnip" line="5">}}group{{</hover>}}
becomes the _composite resource_
{{<hover label="xr" line="2">}}apiVersion{{</hover>}}.
The _XRD_ {{<hover label="xrdSnip" line="7">}}kind{{</hover>}}
is the _composite resource_
{{<hover label="xr" line="3">}}kind{{</hover>}}
The _XRD_ API {{<hover label="xrdSnip" line="9">}}spec{{</hover>}} defines the
_composite resource_ {{<hover label="xr" line="6">}}spec{{</hover>}}.
The _XRD_ {{<hover label="xrdSnip" line="11">}}properties{{</hover>}} section
defines the options for the _composite resource_
{{<hover label="xr" line="6">}}spec{{</hover>}}.
The one option is {{<hover label="xrdSnip" line="12">}}location{{</hover>}}
and it can be either {{<hover label="xrdSnip" line="15">}}EU{{</hover>}} or
{{<hover label="xrdSnip" line="16">}}US{{</hover>}}.
This _composite resource_ uses
{{<hover label="xr" line="7">}}location: US{{</hover>}}.
<!-- vale Google.We = YES -->
### Apply the composite resource
Apply the composite resource to the Kubernetes cluster.
```yaml {label="xr", copy-lines="all"}
cat <<EOF | kubectl apply -f -
apiVersion: custom-api.example.org/v1alpha1
kind: XTopicBucket
metadata:
name: my-composite-resource
spec:
location: "US"
EOF
```
### Verify the composite resource
Verify Crossplane created the _composite resource_ with `kubectl get xdatasetwithbucket`
{{<hint "tip" >}}
Use `kubectl get <composite resource kind>` to view a specific `kind` of _composite resource_.
View all _composite resources_ with `kubectl get composite`.
{{< /hint >}}
```shell {copy-lines="1"}
kubectl get XTopicBucket
NAME SYNCED READY COMPOSITION AGE
my-composite-resource True True topic-with-bucket 2m3s
```
Both `SYNCED` and `READY` are `True` when Crossplane created the GCP resources.
Now look at the GCP storage `bucket` and Pub/Sub `topic` _managed resources_
with `kubectl get bucket` and `kubectl get topic`.
```shell {copy-lines="1"}
kubectl get bucket
NAME READY SYNCED EXTERNAL-NAME AGE
my-composite-resource-m6lbx True True my-composite-resource-m6lbx 4m34s
```
```shell {copy-lines="1"}
kubectl get topics
NAME READY SYNCED EXTERNAL-NAME AGE
my-composite-resource-88vzp True True my-composite-resource-88vzp 4m48s
```
The _composite resource_ automatically generated both _managed resources_.
Using `kubectl describe` on a _managed resource_ shows the `Owner References` is
the _composite resource_.
```yaml {copy-lines="1"}
kubectl describe bucket | grep "Owner References" -A5
Owner References:
API Version: custom-api.example.org/v1alpha1
Block Owner Deletion: true
Controller: true
Kind: XTopicBucket
Name: my-composite-resource
```
Each _composite resource_ creates and owns a unique set of _managed resources_.
If you create a second _composite resource_ Crossplane creates a new storage
`bucket` and Pub/Sub `topic`.
```yaml {label="xr", copy-lines="all"}
cat <<EOF | kubectl apply -f -
apiVersion: custom-api.example.org/v1alpha1
kind: XTopicBucket
metadata:
name: my-second-composite-resource
spec:
location: "US"
EOF
```
Again, use `kubectl get XTopicBucket` to view both _composite resources_.
```shell {copy-lines="1"}
kubectl get XTopicBucket
NAME SYNCED READY COMPOSITION AGE
my-composite-resource True True topic-with-bucket 8m41s
my-second-composite-resource True True topic-with-bucket 2m4s
```
And see there are two `bucket` and two `topic` _managed resources_.
```shell {copy-lines="1"}
kubectl get bucket
NAME READY SYNCED EXTERNAL-NAME AGE
my-composite-resource-m6lbx True True my-composite-resource-m6lbx 9m18s
my-second-composite-resource-rkhbd True True my-second-composite-resource-rkhbd 2m41s
```
```shell {copy-lines="1"}
kubectl get topic
NAME READY SYNCED EXTERNAL-NAME AGE
my-composite-resource-88vzp True True my-composite-resource-88vzp 9m31s
my-second-composite-resource-4wv89 True True my-second-composite-resource-4wv89 2m54s
```
### Delete the composite resources
Because the _composite resource_ is the `Owner` of the _managed resources_, when
Crossplane deletes the _composite resource_, it also deletes the _managed resources_ automatically.
Delete the new _composite resource_ with `kubectl delete XTopicBucket`.
{{<hint "tip" >}}
Delete a specific _composite resource_ with
`kubectl delete <composite kind> <name>` or
`kubectl delete composite <name>`.
{{< /hint >}}
Delete the second composition
```shell
kubectl delete XTopicBucket my-second-composite-resource
```
{{<hint "note">}}
There may a delay in deleting the _managed resources_. Crossplane is making API
calls to GCP and waits for GCP to confirm they deleted the resources before
updating the state in Kubernetes.
{{</hint >}}
Now a single bucket and topic exist.
```shell {copy-lines="1"}
kubectl get bucket
NAME READY SYNCED EXTERNAL-NAME AGE
my-composite-resource-m6lbx True True my-composite-resource-m6lbx 11m
```
```shell {copy-lines="1"}
kubectl get topic
NAME READY SYNCED EXTERNAL-NAME AGE
my-composite-resource-88vzp True True my-composite-resource-88vzp 11m
```
Delete the other _composite resource_ to remove the last `bucket` and `table`
_managed resources_.
```shell
kubectl delete xtopicbucket my-composite-resource
```
_Composite resources_ are great for creating one or more related resources against
a template, but all _composite resources_ exist at the Kubernetes "cluster
level." There's no isolation between _composite resources_. Crossplane uses
_claims_ to create resources with namespace isolation.
## Create a claim
_Claims_, just like _composite resources_ use the custom API defined in the
_XRD_. Unlike a _composite resource_, Crossplane can create _claims_ in a
namespace.
### Create a new Kubernetes namespace
Create a new namespace with `kubectl create namespace`.
```shell
kubectl create namespace test
```
Look at the _XRD_ to see the parameters for the _claim_.
A _claim_ uses the same {{<hover label="XRDclaim2" line="6" >}}group{{</hover>}}
a _composite resource_ uses but a different
{{<hover label="XRDclaim2" line="8" >}}kind{{</hover>}}.
```yaml {label="XRDclaim2"}
apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
# Removed for brevity
spec:
# Removed for brevity
group: custom-api.example.org
claimNames:
kind: TopicBucket
plural: topicbuckets
```
Like the _composite resource_, create a new object with the
{{<hover label="claim" line="2" >}}custom-api.example.org{{</hover>}} API
endpoint.
The _XRD_
{{<hover label="XRDclaim2" line="8" >}}claimNames.kind{{</hover>}} defines the
{{<hover label="claim" line="3" >}}kind{{</hover>}}.
The {{<hover label="claim" line="7" >}}spec{{</hover>}} uses the same
API options as the _composite resource_.
### Apply the claim
Apply the _claim_ to your Kubernetes cluster.
```yaml {label="claim", copy-lines="all"}
cat <<EOF | kubectl apply -f -
apiVersion: custom-api.example.org/v1alpha1
kind: TopicBucket
metadata:
name: claimed-topic-with-bucket
namespace: test
spec:
location: "US"
EOF
```
### Verify the claim
Verify Crossplane created the _claim_ with `kubectl get TopicBucket` in the `test`
namespace.
{{<hint "tip" >}}
View claims with `kubectl get <kind>` or use `kubectl get claim` to view all
_claims_.
{{</hint >}}
```shell {copy-lines="1"}
kubectl get TopicBucket -n test
NAME SYNCED READY CONNECTION-SECRET AGE
claimed-topic-with-bucket True True 4m37s
```
When Crossplane creates a _claim_, a unique _composite resource_ is also
created. View the new _composite resource_ with `kubectl get xtopicbucket`.
```shell {copy-lines="1"}
kubectl get xtopicbucket
NAME SYNCED READY COMPOSITION AGE
claimed-topic-with-bucket-7k2lj True True topic-with-bucket 4m58s
```
The _composite resource_ exists at the "cluster scope" while the _claim_ exists
at the "namespace scope."
Create a second namespace and a second claim.
```shell {copy-lines="all"}
kubectl create namespace test2
cat <<EOF | kubectl apply -f -
apiVersion: custom-api.example.org/v1alpha1
kind: TopicBucket
metadata:
name: second-claimed-topic-with-bucket
namespace: test2
spec:
location: "US"
EOF
```
View the _claims_ in all namespaces with `kubectl get topicbucket -A`
```shell {copy-lines="1"}
kubectl get topicbucket -A
NAMESPACE NAME SYNCED READY CONNECTION-SECRET AGE
test claimed-topic-with-bucket True True 8m48s
test2 second-claimed-topic-with-bucket True True 2m24s
```
Now look at the _composite resources_ at the cluster scope.
```shell {copy-lines="1"}
kubectl get xtopicbucket
NAME SYNCED READY COMPOSITION AGE
claimed-topic-with-bucket-7k2lj True True topic-with-bucket 9m11s
second-claimed-topic-with-bucket-d5x58 True True topic-with-bucket 2m47s
```
Crossplane created a second _composite resource_ for the second _claim_.
Looking at the GCP storage `bucket` and Pub/Sub `topic` shows two of each
resource, one for each claim.
```shell {copy-lines="1"}
kubectl get bucket
NAME READY SYNCED EXTERNAL-NAME AGE
claimed-topic-with-bucket-7k2lj-qf2m6 True True claimed-topic-with-bucket-7k2lj-qf2m6 9m46s
second-claimed-topic-with-bucket-d5x58-drlxr True True second-claimed-topic-with-bucket-d5x58-drlxr 3m22s
```
```shell {copy-lines="1"}
kubectl get topic
NAME READY SYNCED EXTERNAL-NAME AGE
claimed-topic-with-bucket-7k2lj-8xn7t True True claimed-topic-with-bucket-7k2lj-8xn7t 9m59s
second-claimed-topic-with-bucket-d5x58-ctkrp True True second-claimed-topic-with-bucket-d5x58-ctkrp 3m35s
```
### Delete the claims
Removing the _claims_ removes the _composite resources_ and the associated
_managed resources_.
```shell
kubectl delete topicbucket claimed-topic-with-bucket -n test
kubectl delete topicbucket second-claimed-topic-with-bucket -n test2
```
Verify Crossplane removed all the _managed resources_.
```shell
kubectl get bucket
No resources found
```
```shell
kubectl get table
No resources found
```
Claims are powerful tools to give users resources in their own isolated
namespace. But these examples haven't shown how the custom API can change
the settings defined in the _composition_. This _composition patching_ applies
the API settings when creating resources.
[Part 3]({{< ref "provider-gcp-part-3">}}) of this guide covers
_composition patches_ and making all this configuration portable in Crossplane
_packages_.
## Next steps
* **[Continue to part 3]({{< ref "provider-gcp-part-3">}})** to create a learn
about _patching_ resources and creating Crossplane _packages_.
* Explore GCP resources that Crossplane can configure in the
[Provider CRD reference](https://marketplace.upbound.io/providers/upbound/provider-gcp/latest/crds).
* Join the [Crossplane Slack](https://slack.crossplane.io/) and connect with Crossplane users and contributors.

View File

@ -0,0 +1,659 @@
---
title: GCP Quickstart Part 3
weight: 120
tocHidden: true
---
{{< hint "important" >}}
This guide is part 3 of a series.
Follow **[part 1]({{<ref "provider-gcp" >}})**
to install Crossplane and connect your Kubernetes cluster to GCP.
Follow **[part 2]({{<ref "provider-gcp-part-2" >}})** to create a _composition_,
_custom resource definition_ and a _claim_.
{{< /hint >}}
[Part 2]({{<ref "provider-gcp-part-2" >}}) created a _composite resource
definition_ to define the schema of the custom API. Users create a _claim_ to
use the custom API and apply their options. Part 2 didn't show how the options
set in a _claim_ change or get applied the associated _composite resources_.
## Prerequisites
* Complete quickstart [part 1]({{<ref "provider-gcp" >}}) and [Part 2]({{<ref
"provider-gcp-part-2" >}}) to install Crossplane and the quickstart
configurations.
{{<expand "Skip parts 1 and 2 and just get started" >}}
1. Add the Crossplane Helm repository and install Crossplane.
```shell
helm repo add \
crossplane-stable https://charts.crossplane.io/stable
helm repo update
&&
helm install crossplane \
crossplane-stable/crossplane \
--namespace crossplane-system \
--create-namespace
```
2. When the Crossplane pods finish installing and are ready, apply the GCP Provider.
```yaml {label="provider",copy-lines="all"}
cat <<EOF | kubectl apply -f -
apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
name: upbound-provider-gcp
spec:
package: xpkg.upbound.io/upbound/provider-gcp:v0.28.0
EOF
```
3. Create a file called `gcp-credentials.json` with your GCP service account
JSON file.
{{< hint type="tip" >}}
The
[GCP documentation](https://cloud.google.com/iam/docs/creating-managing-service-account-keys)
provides information on how to generate a service account JSON file.
{{< /hint >}}
4. Create a Kubernetes secret from the GCP JSON file
```shell {label="kube-create-secret",copy-lines="all"}
kubectl create secret \
generic gcp-secret \
-n crossplane-system \
--from-file=creds=./gcp-credentials.json
```
5. Create a _ProviderConfig_
Include your
{{< hover label="providerconfig" line="7" >}}GCP project ID{{< /hover >}} in the
_ProviderConfig_ settings.
{{< hint type="warning" >}}
Find your GCP project ID from the `project_id` field of the
`gcp-credentials.json` file.
{{< /hint >}}
{{< editCode >}}
```yaml {label="providerconfig",copy-lines="all"}
cat <<EOF | kubectl apply -f -
apiVersion: gcp.upbound.io/v1beta1
kind: ProviderConfig
metadata:
name: default
spec:
projectID: $$<PROJECT_ID>$$
credentials:
source: Secret
secretRef:
namespace: crossplane-system
name: gcp-secret
key: creds
EOF
```
{{< /editCode >}}
6. Create a _composition_
```yaml {copy-lines="all"}
cat <<EOF | kubectl apply -f -
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
name: topic-with-bucket
spec:
compositeTypeRef:
apiVersion: custom-api.example.org/v1alpha1
kind: XTopicBucket
resources:
- name: crossplane-quickstart-bucket
base:
apiVersion: storage.gcp.upbound.io/v1beta1
kind: Bucket
spec:
forProvider:
location: US
- name: crossplane-quickstart-topic
base:
apiVersion: pubsub.gcp.upbound.io/v1beta1
kind: Topic
spec:
forProvider:
messageStoragePolicy:
- allowedPersistenceRegions:
- "us-central1"
EOF
```
7. Create a _composite resource definition_
```yaml {copy-lines="all"}
cat <<EOF | kubectl apply -f -
apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
name: xtopicbuckets.custom-api.example.org
spec:
group: custom-api.example.org
names:
kind: XTopicBucket
plural: xtopicbuckets
versions:
- name: v1alpha1
served: true
referenceable: true
schema:
openAPIV3Schema:
type: object
properties:
spec:
type: object
properties:
location:
type: string
oneOf:
- pattern: '^EU$'
- pattern: '^US$'
required:
- location
claimNames:
kind: TopicBucket
plural: topicbuckets
EOF
```
8. Create a new namespace
```shell
kubectl create namespace test
```
{{</expand >}}
## Enable composition patches
In a _composition_, `patches` map fields in the custom API to fields inside the
_managed resources_.
The example _composition_ has two _managed resources_, a
{{<hover label="compResources" line="8">}}bucket{{</hover>}} and a
{{<hover label="compResources" line="15">}}topic{{</hover>}}.
```yaml {label="compResources"}
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
# Removed for Brevity
resources:
- name: crossplane-quickstart-bucket
base:
apiVersion: storage.gcp.upbound.io/v1beta1
kind: Bucket
spec:
forProvider:
location: US
- name: crossplane-quickstart-topic
base:
apiVersion: pubsub.gcp.upbound.io/v1beta1
kind: Topic
spec:
forProvider:
messageStoragePolicy:
- allowedPersistenceRegions:
- "us-central1"
```
<!-- vale Google.We = NO -->
The custom API defined a single option,
{{<hover label="xrdSnip" line="12">}}location{{</hover>}}. A
{{<hover label="xrdSnip" line="12">}}location{{</hover>}} can be either
{{<hover label="xrdSnip" line="15">}}EU{{</hover>}} or
{{<hover label="xrdSnip" line="16">}}US{{</hover>}}.
<!-- vale Google.We = YES -->
```yaml {label="xrdSnip"}
apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
# Removed for brevity
spec:
group: custom-api.example.org
names:
kind: XDatabase
# Removed for brevity
spec:
type: object
properties:
location:
type: string
oneOf:
- pattern: '^EU$'
- pattern: '^US$'
```
Creating a _composition_ `patch` allows Crossplane to update the settings of a
_composite resource_. Patches apply to an individual _managed resource_
inside the _composition_.
A {{<hover label="patch" line="14">}}patch{{</hover>}} has a
{{<hover label="patch" line="15">}}fromField{{</hover>}} and a
{{<hover label="patch" line="16">}}toField{{</hover>}} specifying which value
_from_ the custom API should apply _to_ a field in the _managed resource_.
Patches can create a
{{<hover label="patch" line="17">}}transform{{</hover>}} to change the _from_
field before it's applied.
The transform
{{<hover label="patch" line="18">}}type{{</hover>}} is what kind of change to
make on the _from_ field. Types of changes could include appending a string,
preforming a math operation or mapping one value to another.
Applying a {{<hover label="patch" line="14">}}patch{{</hover>}} to the
{{<hover label="patch" line="8">}}Topic{{</hover>}} uses the custom API
{{<hover label="patch" line="15">}}spec.location{{</hover>}} field to use as the
_managed resource_
{{<hover label="patch" line="12">}}allowedPersistenceRegions{{</hover>}} value.
<!-- vale Google.We = NO -->
The custom API value "EU" is
{{<hover label="patch" line="20">}}mapped{{</hover>}} to the value
"europe-central2" and "US" is
{{<hover label="patch" line="21">}}mapped{{</hover>}} to the value
"us-central1."
<!-- vale Google.We = YES -->
```yaml {label="patch"}
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
# Removed for Brevity
resources:
- name: crossplane-quickstart-topic
base:
apiVersion: pubsub.gcp.upbound.io/v1beta1
kind: Topic
spec:
forProvider:
messageStoragePolicy:
- allowedPersistenceRegions:
- "us-central1"
patches:
- fromFieldPath: "spec.location"
toFieldPath: "spec.forProvider.messageStoragePolicy[*].allowedPersistenceRegions[*]"
transforms:
- type: map
map:
EU: "europe-central2"
US: "us-central1"
```
<!-- vale Google.We = NO -->
Patching is a powerful tool enabling simpler or abstracted APIs. Developers
don't need to know the specific GCP region, just the abstracted
option of "EU" or "US."
<!-- vale Google.We = YES -->
### Apply the updated composition
Apply a similar `patch` to the `Bucket` _managed resource_ and apply the updated
_composition_.
```yaml
cat <<EOF | kubectl apply -f -
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
name: topic-with-bucket
spec:
compositeTypeRef:
apiVersion: custom-api.example.org/v1alpha1
kind: XTopicBucket
resources:
- name: crossplane-quickstart-bucket
base:
apiVersion: storage.gcp.upbound.io/v1beta1
kind: Bucket
spec:
forProvider:
location: "US"
patches:
- fromFieldPath: "spec.location"
toFieldPath: "spec.forProvider.location"
transforms:
- type: map
map:
EU: "EU"
US: "US"
- name: crossplane-quickstart-topic
base:
apiVersion: pubsub.gcp.upbound.io/v1beta1
kind: Topic
spec:
forProvider:
messageStoragePolicy:
- allowedPersistenceRegions:
- "us-central1"
patches:
- fromFieldPath: "spec.location"
toFieldPath: "spec.forProvider.messageStoragePolicy[*].allowedPersistenceRegions[*]"
transforms:
- type: map
map:
EU: "europe-central2"
US: "us-central1"
EOF
```
### Create a claim
Create a new _claim_ and set the
{{<hover label="claim" line="8" >}}location{{</hover >}} to "EU."
```yaml {label="claim"}
cat <<EOF | kubectl apply -f -
apiVersion: custom-api.example.org/v1alpha1
kind: TopicBucket
metadata:
name: claimed-eu-topic-with-bucket
namespace: test
spec:
location: "EU"
EOF
```
View the _claim_ with `kubectl get claim`
```shell {copy-lines="1"}
kubectl get TopicBucket -n test
NAME SYNCED READY CONNECTION-SECRET AGE
claimed-eu-topic-with-bucket True True 2m26s
```
The claim reports `SYNCED` and `READY` as `True` after Crossplane creates
all the _managed resources_.
Describe the `Topic` resource to see the GCP location is
{{< hover label="topicLocation" line="5">}}europe-central2{{< /hover >}}.
```shell {copy-lines="1",label="topicLocation"}
kubectl describe topic | grep "For Provider" -A3
For Provider:
Message Storage Policy:
Allowed Persistence Regions:
europe-central2
```
Describe the `Bucket` resource to see the GCP location is also set to
{{<hover label="bucketLocation" line="3">}}EU{{</hover>}}.
```shell {copy-lines="1",label="bucketLocation"}
kubectl describe bucket | grep "For Provider" -A1
For Provider:
Location: EU
```
<!-- vale Google.We = NO -->
Using {{<hover label="claim" line="8" >}}location: "EU"{{</hover >}} in the
claim patches the _composite resource_, updating the `Topic` GCP region from
`us-central1` to `europe-central-2` and the `Bucket` from GCP region `US` to GCP
region `EU`.
The developer creating the claim doesn't need to know which specific GCP region
or the naming conventions. Using the abstract API options of "EU" or "US" the
developer places their resources in the desired location.
In this example, patching also allows platform teams to ensure all resources are
in the same location.
<!-- vale Google.We = YES -->
Deleting the claim removes the _managed resources_.
{{<hint "note" >}}
The _managed resources_ take up to 5 minutes to delete.
{{< /hint >}}
```shell
kubectl delete TopicBucket claimed-eu-topic-with-bucket -n test
```
## Create a Crossplane configuration package
Creating a configuration package makes your Crossplane custom APIs portable
and versioned.
Crossplane _configuration packages_ allow users to combine their _custom
resource definition_ and _composition_ files into an OCI image.
{{< hint "note" >}}
The [Open Container Initiative](https://opencontainers.org/faq/)
defines the OCI image standard.
An OCI images is a standard way to package data.
{{< /hint >}}
You can host configuration packages in image registries like
[Docker Hub](https://hub.docker.com/) or the
[Upbound Marketplace](https://marketplace.upbound.io/).
Crossplane can download and install configuration packages into a Kubernetes
cluster.
Building and installing configuration packages requires an OCI image compatible
tool.
{{< hint "note" >}}
You can use any software that builds OCI images. This includes
[Docker](https://www.docker.com/) or
[Upbound's Up command-line tool](https://github.com/upbound/up)
{{< /hint >}}
A configuration package includes three files:
* `crossplane.yaml` defines the metadata of the package.
* `definition.yaml` is the _composite resource definition_ for the package.
* `composition.yaml` is the _composition_ template for the package.
<!-- vale gitlab.Substitutions = NO -->
<!-- yaml is in the filename -->
### Create a crossplane.yaml file
<!-- vale gitlab.Substitutions = YES -->
Configuration packages describe their contents and requirements with a
`crossplane.yaml` file.
The `crossplane.yaml` file lists the required Crossplane _providers_ and their
compatible versions as well as the required Crossplane version.
The Crossplane
{{<hover label="xpyaml" line="1" >}}meta.pkg{{</hover>}} API defines the schema
for a
{{<hover label="xpyaml" line="2" >}}Configuration{{</hover>}}.
Inside the {{<hover label="xpyaml" line="5" >}}spec{{</hover>}} define the
required Crossplane
{{<hover label="xpyaml" line="7" >}}version{{</hover>}}.
The {{<hover label="xpyaml" line="8" >}}dependsOn{{</hover>}} section lists the
dependencies for a package.
This package lists the Upbound
{{<hover label="xpyaml" line="9" >}}provider-gcp{{</hover>}}
version {{<hover label="xpyaml" line="10" >}}0.28.0{{</hover>}} or later as a
dependency.
{{<hint "tip" >}}
Crossplane automatically installs dependencies. Dependencies can include other
configuration packages.
{{< /hint >}}
```yaml {label="xpyaml"}
apiVersion: meta.pkg.crossplane.io/v1
kind: Configuration
metadata:
name: crossplane-gcp-quickstart
spec:
crossplane:
version: ">=v1.11.0"
dependsOn:
- provider: xpkg.upbound.io/upbound/provider-gcp
version: ">=v0.28.0"
```
Create a new directory and save the `crossplane.yaml` file.
```yaml {copy-lines="all"}
mkdir crossplane-gcp-quickstart
cat <<EOF > crossplane-gcp-quickstart/crossplane.yaml
apiVersion: meta.pkg.crossplane.io/v1
kind: Configuration
metadata:
name: crossplane-gcp-quickstart
spec:
crossplane:
version: ">=v1.11.0"
dependsOn:
- provider: xpkg.upbound.io/upbound/provider-gcp
version: ">=v0.28.0"
EOF
```
<!-- vale gitlab.Substitutions = NO -->
<!-- yaml is in the filename -->
### Create a definition.yaml file
<!-- vale gitlab.Substitutions = YES -->
A configuration package requires a _composite resource definition_ (XRD) to define the
custom API.
Save the _XRD_ as `definition.yaml` in the same directory as the
`crossplane.yaml` file.
```yaml {copy-lines="all"}
cat <<EOF > crossplane-gcp-quickstart/definition.yaml
apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
name: xtopicbuckets.custom-api.example.org
spec:
group: custom-api.example.org
names:
kind: XTopicBucket
plural: xtopicbuckets
versions:
- name: v1alpha1
served: true
referenceable: true
schema:
openAPIV3Schema:
type: object
properties:
spec:
type: object
properties:
location:
type: string
oneOf:
- pattern: '^EU$'
- pattern: '^US$'
required:
- location
claimNames:
kind: TopicBucket
plural: topicbuckets
EOF
```
<!-- vale gitlab.Substitutions = NO -->
<!-- yaml is in the filename -->
### Create a composition.yaml file
<!-- vale gitlab.Substitutions = YES -->
The _composition_ template creates the _managed resources_ and allows _patches_
to customize the _managed resources_.
Copy the _composition_ into the `composition.yaml` file in the same directory as
`crossplane.yaml`.
```yaml
cat <<EOF > crossplane-gcp-quickstart/composition.yaml
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
name: topic-with-bucket
spec:
compositeTypeRef:
apiVersion: custom-api.example.org/v1alpha1
kind: XTopicBucket
resources:
- name: crossplane-quickstart-bucket
base:
apiVersion: storage.gcp.upbound.io/v1beta1
kind: Bucket
spec:
forProvider:
location: "US"
patches:
- fromFieldPath: "spec.location"
toFieldPath: "spec.forProvider.location"
transforms:
- type: map
map:
EU: "EU"
US: "US"
- name: crossplane-quickstart-topic
base:
apiVersion: pubsub.gcp.upbound.io/v1beta1
kind: Topic
spec:
forProvider:
messageStoragePolicy:
- allowedPersistenceRegions:
- "us-central1"
patches:
- fromFieldPath: "spec.location"
toFieldPath: "spec.forProvider.messageStoragePolicy[*].allowedPersistenceRegions[*]"
transforms:
- type: map
map:
EU: "europe-central2"
US: "us-central1"
EOF
```
### Install the Crossplane command-line
To build a configuration package install the Crossplane Kubernetes command-line
extension.
```shell
wget "https://raw.githubusercontent.com/crossplane/crossplane/master/install.sh"
chmod +x install.sh
./install.sh
sudo mv kubectl-crossplane /usr/bin
```
Verify the Crossplane command-line installed with `kubectl crossplane --help`
```shell
kubectl crossplane --help
Usage: kubectl crossplane <command>
A command line tool for interacting with Crossplane.
Flags:
-h, --help Show context-sensitive help.
-v, --version Print version and quit.
--verbose Print verbose logging statements.
# Ouptut removed for brevity
```
### Build a configuration package
Use the `kubectl crossplane` command to create an `.xpkg` file containing the
custom APIs and Crossplane configuration.
```shell
kubectl crossplane build configuration -f crossplane-gcp-quickstart/ --name="crossplane-gcp-quickstart"
```
Now an `.xpkg` OCI image is inside the `crossplane-gcp-quickstart` directory.
```shell
ls crossplane-gcp-quickstart/
composition.yaml crossplane-gcp-quickstart.xpkg crossplane.yaml definition.yaml
```
## Next steps
* Explore GCP resources that Crossplane can configure in the [Provider CRD reference](https://marketplace.upbound.io/providers/upbound/provider-gcp/latest/crds).
* Join the [Crossplane Slack](https://slack.crossplane.io/) and connect with Crossplane users and contributors.
* Read more about [Crossplane concepts]({{<ref "../concepts" >}})

File diff suppressed because it is too large Load Diff

View File

@ -4,6 +4,8 @@ Bootstrap
CRDs
Crossplane
Crossplane's
Dataflow
Datastore
editCode
Env
GCP's

View File

@ -8,19 +8,24 @@ indicators:
- ':'
exceptions:
- Azure
- BigQuery
- CLI
- Code
- Cosmos
- Crossplane
- Docker
- Emmet
- Hugo
- gRPC
- Helm
- I
- Kubernetes
- Linux
- macOS
- Marketplace
- MongoDB
- ProviderConfig
- Pub/Sub
- REPL
- Studio
- TypeScript

View File

@ -5,3 +5,7 @@ level: warning
scope: heading
tokens:
- '[A-Z]{2,4}'
exceptions:
- API
- GCP
- JSON