28 KiB
title | weight | description |
---|---|---|
Write a Composition Function in Go | 80 | Composition functions allow you to template resources using Go |
Composition functions (or just functions, for short) are custom programs that template Crossplane resources. Crossplane calls composition functions to determine what resources it should create when you create a composite resource (XR). Read the [concepts]({{<ref "../concepts/compositions" >}}) page to learn more about composition functions.
You can write a function to template resources using a general purpose programming language. Using a general purpose programming language allows a function to use advanced logic to template resources, like loops and conditionals. This guide explains how to write a composition function in Go.
{{< hint "important" >}} It helps to be familiar with [how composition functions work]({{<ref "../concepts/compositions#how-composition-functions-work" >}}) before following this guide. {{< /hint >}}
Understand the steps
This guide covers writing a composition function for an {{}}XBuckets{{}} composite resource (XR).
apiVersion: example.crossplane.io/v1
kind: XBuckets
metadata:
name: example-buckets
spec:
region: us-east-2
names:
- crossplane-functions-example-a
- crossplane-functions-example-b
- crossplane-functions-example-c
An XBuckets
XR has a region and an array of bucket names. The function will
create an Amazon Web Services (AWS) S3 bucket for each entry in the names array.
To write a function in Go:
- Install the tools you need to write the function
- Initialize the function from a template
- Edit the template to add the function's logic
- Test the function end-to-end
- Build and push the function to a package repository
This guide covers each of these steps in detail.
Install the tools you need to write the function
To write a function in Go you need:
- Go v1.23 or newer. The guide uses Go v1.23.
- Docker Engine. This guide uses Engine v24.
- The [Crossplane CLI]({{<ref "../cli" >}}) v1.17 or newer. This guide uses Crossplane CLI v1.17.
{{<hint "note">}} You don't need access to a Kubernetes cluster or a Crossplane control plane to build or test a composition function. {{}}
Initialize the function from a template
Use the crossplane xpkg init
command to initialize a new function. When
you run this command it initializes your function using
a GitHub repository
as a template.
crossplane xpkg init function-xbuckets function-template-go -d function-xbuckets
Initialized package "function-xbuckets" in directory "/home/negz/control/negz/function-xbuckets" from https://github.com/crossplane/function-template-go/tree/91a1a5eed21964ff98966d72cc6db6f089ad63f4 (main)
To get started:
1. Replace `function-template-go` with your function in `go.mod`,
`package/crossplane.yaml`, and any Go imports. (You can also do this
automatically by running the `./init.sh <function-name>` script.)
2. Update `input/v1beta1/` to reflect your desired input (and run `go generate`)
3. Add your logic to `RunFunction` in `fn.go`
4. Add tests for your logic in `fn_test.go`
5. Update `README.md`, to be about your function!
Found init.sh script!
Do you want to run it? [y]es/[n]o/[v]iew: y
Function function-xbuckets has been initialised successfully
The crossplane xpkg init
command creates a directory named
function-xbuckets
. When you run the command the new directory should look like
this:
ls function-xbuckets
Dockerfile LICENSE NOTES.txt README.md example fn.go fn_test.go go.mod go.sum init.sh input main.go package renovate.json
The fn.go
file is where you add the function's code. It's useful to know about
some other files in the template:
main.go
runs the function. You don't need to editmain.go
.Dockerfile
builds the function runtime. You don't need to editDockerfile
.- The
input
directory defines the function's input type. - The
package
directory contains metadata used to build the function package.
{{<hint "tip">}}
Starting with v1.15 of the Crossplane CLI, crossplane xpkg init
gives you the
option of running an initialization script to automate tasks like replacing the
template name with the new function's name.
{{}}
You must make some changes before you start adding code:
- Edit
package/crossplane.yaml
to change the package's name. - Edit
go.mod
to change the Go module's name.
Name your package function-xbuckets
.
The name of your module depends on where you want to keep your function code. If
you push Go code to GitHub, you can use your GitHub username. For example
module github.com/negz/function-xbuckets
.
The function in this guide doesn't use an input type. For this function you
should delete the input
and package/input
directories.
The input
directory defines a Go struct that a function can use to take input,
using the input
field from a Composition. The
[composition functions]({{<ref "../concepts/compositions/#function-input" >}})
documentation explains how to pass an input to a composition function.
The package/input
directory contains an OpenAPI schema generated from the
structs in the input
directory.
{{<hint "tip">}} If you're writing a function that uses an input, edit the input to meet your function's requirements.
Change the input's kind and API group. Don't use Input
and
template.fn.crossplane.io
. Instead use something meaningful to your function.
When you edit files under the input
directory you must update some generated
files by running go generate
. See input/generate.go
for details.
go generate ./...
{{}}
Edit the template to add the function's logic
You add your function's logic to the
{{}}RunFunction{{}}
method in fn.go
. When you first open the file it contains a "hello world"
function.
func (f *Function) RunFunction(_ context.Context, req *fnv1.RunFunctionRequest) (*fnv1.RunFunctionResponse, error) {
f.log.Info("Running Function", "tag", req.GetMeta().GetTag())
rsp := response.To(req, response.DefaultTTL)
in := &v1beta1.Input{}
if err := request.GetInput(req, in); err != nil {
response.Fatal(rsp, errors.Wrapf(err, "cannot get Function input from %T", req))
return rsp, nil
}
response.Normalf(rsp, "I was run with input %q", in.Example)
return rsp, nil
}
All Go composition functions have a RunFunction
method. Crossplane passes
everything the function needs to run in a
{{}}RunFunctionRequest{{}} struct.
The function tells Crossplane what resources it should compose by returning a {{}}RunFunctionResponse{{}} struct.
{{<hint "tip">}}
Crossplane generates the RunFunctionRequest
and RunFunctionResponse
structs
using Protocol Buffers. You can find detailed schemas for
RunFunctionRequest
and RunFunctionResponse
in the
Buf Schema Registry.
{{}}
Edit the RunFunction
method to replace it with this code.
func (f *Function) RunFunction(_ context.Context, req *fnv1.RunFunctionRequest) (*fnv1.RunFunctionResponse, error) {
rsp := response.To(req, response.DefaultTTL)
xr, err := request.GetObservedCompositeResource(req)
if err != nil {
response.Fatal(rsp, errors.Wrapf(err, "cannot get observed composite resource from %T", req))
return rsp, nil
}
region, err := xr.Resource.GetString("spec.region")
if err != nil {
response.Fatal(rsp, errors.Wrapf(err, "cannot read spec.region field of %s", xr.Resource.GetKind()))
return rsp, nil
}
names, err := xr.Resource.GetStringArray("spec.names")
if err != nil {
response.Fatal(rsp, errors.Wrapf(err, "cannot read spec.names field of %s", xr.Resource.GetKind()))
return rsp, nil
}
desired, err := request.GetDesiredComposedResources(req)
if err != nil {
response.Fatal(rsp, errors.Wrapf(err, "cannot get desired resources from %T", req))
return rsp, nil
}
_ = v1beta1.AddToScheme(composed.Scheme)
for _, name := range names {
b := &v1beta1.Bucket{
ObjectMeta: metav1.ObjectMeta{
Annotations: map[string]string{
"crossplane.io/external-name": name,
},
},
Spec: v1beta1.BucketSpec{
ForProvider: v1beta1.BucketParameters{
Region: ptr.To[string](region),
},
},
}
cd, err := composed.From(b)
if err != nil {
response.Fatal(rsp, errors.Wrapf(err, "cannot convert %T to %T", b, &composed.Unstructured{}))
return rsp, nil
}
desired[resource.Name("xbuckets-"+name)] = &resource.DesiredComposed{Resource: cd}
}
if err := response.SetDesiredComposedResources(rsp, desired); err != nil {
response.Fatal(rsp, errors.Wrapf(err, "cannot set desired composed resources in %T", rsp))
return rsp, nil
}
return rsp, nil
}
Expand the below block to view the full fn.go
, including imports and
commentary explaining the function's logic.
{{<expand "The full fn.go file" >}}
package main
import (
"context"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/utils/ptr"
"github.com/upbound/provider-aws/apis/s3/v1beta1"
"github.com/crossplane/function-sdk-go/errors"
"github.com/crossplane/function-sdk-go/logging"
fnv1 "github.com/crossplane/function-sdk-go/proto/v1"
"github.com/crossplane/function-sdk-go/request"
"github.com/crossplane/function-sdk-go/resource"
"github.com/crossplane/function-sdk-go/resource/composed"
"github.com/crossplane/function-sdk-go/response"
)
// Function returns whatever response you ask it to.
type Function struct {
fnv1.UnimplementedFunctionRunnerServiceServer
log logging.Logger
}
// RunFunction observes an XBuckets composite resource (XR). It adds an S3
// bucket to the desired state for every entry in the XR's spec.names array.
func (f *Function) RunFunction(_ context.Context, req *fnv1.RunFunctionRequest) (*fnv1.RunFunctionResponse, error) {
f.log.Info("Running Function", "tag", req.GetMeta().GetTag())
// Create a response to the request. This copies the desired state and
// pipeline context from the request to the response.
rsp := response.To(req, response.DefaultTTL)
// Read the observed XR from the request. Most functions use the observed XR
// to add desired managed resources.
xr, err := request.GetObservedCompositeResource(req)
if err != nil {
// You can set a custom status condition on the claim. This
// allows you to communicate with the user.
response.ConditionFalse(rsp, "FunctionSuccess", "InternalError").
WithMessage("Something went wrong.").
TargetCompositeAndClaim()
// You can emit an event regarding the claim. This allows you to
// communicate with the user. Note that events should be used
// sparingly and are subject to throttling
response.Warning(rsp, errors.New("something went wrong")).
TargetCompositeAndClaim()
// If the function can't read the XR, the request is malformed. This
// should never happen. The function returns a fatal result. This tells
// Crossplane to stop running functions and return an error.
response.Fatal(rsp, errors.Wrapf(err, "cannot get observed composite resource from %T", req))
return rsp, nil
}
// Create an updated logger with useful information about the XR.
log := f.log.WithValues(
"xr-version", xr.Resource.GetAPIVersion(),
"xr-kind", xr.Resource.GetKind(),
"xr-name", xr.Resource.GetName(),
)
// Get the region from the XR. The XR has getter methods like GetString,
// GetBool, etc. You can use them to get values by their field path.
region, err := xr.Resource.GetString("spec.region")
if err != nil {
response.Fatal(rsp, errors.Wrapf(err, "cannot read spec.region field of %s", xr.Resource.GetKind()))
return rsp, nil
}
// Get the array of bucket names from the XR.
names, err := xr.Resource.GetStringArray("spec.names")
if err != nil {
response.Fatal(rsp, errors.Wrapf(err, "cannot read spec.names field of %s", xr.Resource.GetKind()))
return rsp, nil
}
// Get all desired composed resources from the request. The function will
// update this map of resources, then save it. This get, update, set pattern
// ensures the function keeps any resources added by other functions.
desired, err := request.GetDesiredComposedResources(req)
if err != nil {
response.Fatal(rsp, errors.Wrapf(err, "cannot get desired resources from %T", req))
return rsp, nil
}
// Add v1beta1 types (including Bucket) to the composed resource scheme.
// composed.From uses this to automatically set apiVersion and kind.
_ = v1beta1.AddToScheme(composed.Scheme)
// Add a desired S3 bucket for each name.
for _, name := range names {
// One advantage of writing a function in Go is strong typing. The
// function can import and use managed resource types from the provider.
b := &v1beta1.Bucket{
ObjectMeta: metav1.ObjectMeta{
// Set the external name annotation to the desired bucket name.
// This controls what the bucket will be named in AWS.
Annotations: map[string]string{
"crossplane.io/external-name": name,
},
},
Spec: v1beta1.BucketSpec{
ForProvider: v1beta1.BucketParameters{
// Set the bucket's region to the value read from the XR.
Region: ptr.To[string](region),
},
},
}
// Convert the bucket to the unstructured resource data format the SDK
// uses to store desired composed resources.
cd, err := composed.From(b)
if err != nil {
response.Fatal(rsp, errors.Wrapf(err, "cannot convert %T to %T", b, &composed.Unstructured{}))
return rsp, nil
}
// Add the bucket to the map of desired composed resources. It's
// important that the function adds the same bucket every time it's
// called. It's also important that the bucket is added with the same
// resource.Name every time it's called. The function prefixes the name
// with "xbuckets-" to avoid collisions with any other composed
// resources that might be in the desired resources map.
desired[resource.Name("xbuckets-"+name)] = &resource.DesiredComposed{Resource: cd}
}
// Finally, save the updated desired composed resources to the response.
if err := response.SetDesiredComposedResources(rsp, desired); err != nil {
response.Fatal(rsp, errors.Wrapf(err, "cannot set desired composed resources in %T", rsp))
return rsp, nil
}
// Log what the function did. This will only appear in the function's pod
// logs. A function can use response.Normal and response.Warning to emit
// Kubernetes events associated with the XR it's operating on.
log.Info("Added desired buckets", "region", region, "count", len(names))
// You can set a custom status condition on the claim. This allows you
// to communicate with the user.
response.ConditionTrue(rsp, "FunctionSuccess", "Success").
TargetCompositeAndClaim()
return rsp, nil
}
{{}}
This code:
- Gets the observed composite resource from the
RunFunctionRequest
. - Gets the region and bucket names from the observed composite resource.
- Adds one desired S3 bucket for each bucket name.
- Returns the desired S3 buckets in a
RunFunctionResponse
.
The code uses the v1beta1.Bucket
type from the AWS S3 provider.
One advantage of writing a function in Go is that you can compose resources
using the same strongly typed structs Crossplane uses in its providers.
You must get the AWS Provider Go module to use this type:
go get github.com/upbound/provider-aws@v1.14.0
Crossplane provides a
software development kit (SDK)
for writing composition functions in Go. This function uses
utilities from the SDK. In particular the request
and response
packages make
working with the RunFunctionRequest
and RunFunctionResponse
types easier.
{{<hint "tip">}} Read the Go package documentation for the SDK. {{}}
Test the function end-to-end
Test your function by adding unit tests, and by using the crossplane render
command.
Go has rich support for unit testing. When you initialize a function from the
template it adds some unit tests to fn_test.go
. These tests follow Go's
recommendations. They use only
pkg/testing
from the Go standard library and
google/go-cmp
.
To add test cases, update the cases
map in TestRunFunction
. Expand the below
block to view the full fn_test.go
file for the function.
{{<expand "The full fn_test.go file" >}}
package main
import (
"context"
"testing"
"time"
"github.com/google/go-cmp/cmp"
"github.com/google/go-cmp/cmp/cmpopts"
"google.golang.org/protobuf/testing/protocmp"
"google.golang.org/protobuf/types/known/durationpb"
"github.com/crossplane/crossplane-runtime/pkg/logging"
fnv1 "github.com/crossplane/function-sdk-go/proto/v1"
"github.com/crossplane/function-sdk-go/resource"
)
func TestRunFunction(t *testing.T) {
type args struct {
ctx context.Context
req *fnv1.RunFunctionRequest
}
type want struct {
rsp *fnv1.RunFunctionResponse
err error
}
cases := map[string]struct {
reason string
args args
want want
}{
"AddTwoBuckets": {
reason: "The Function should add two buckets to the desired composed resources",
args: args{
req: &fnv1.RunFunctionRequest{
Observed: &fnv1.State{
Composite: &fnv1.Resource{
// MustStructJSON is a handy way to provide mock
// resources.
Resource: resource.MustStructJSON(`{
"apiVersion": "example.crossplane.io/v1alpha1",
"kind": "XBuckets",
"metadata": {
"name": "test"
},
"spec": {
"region": "us-east-2",
"names": [
"test-bucket-a",
"test-bucket-b"
]
}
}`),
},
},
},
},
want: want{
rsp: &fnv1.RunFunctionResponse{
Meta: &fnv1.ResponseMeta{Ttl: durationpb.New(60 * time.Second)},
Desired: &fnv1.State{
Resources: map[string]*fnv1.Resource{
"xbuckets-test-bucket-a": {Resource: resource.MustStructJSON(`{
"apiVersion": "s3.aws.upbound.io/v1beta1",
"kind": "Bucket",
"metadata": {
"annotations": {
"crossplane.io/external-name": "test-bucket-a"
}
},
"spec": {
"forProvider": {
"region": "us-east-2"
}
},
"status": {
"observedGeneration": 0
}
}`)},
"xbuckets-test-bucket-b": {Resource: resource.MustStructJSON(`{
"apiVersion": "s3.aws.upbound.io/v1beta1",
"kind": "Bucket",
"metadata": {
"annotations": {
"crossplane.io/external-name": "test-bucket-b"
}
},
"spec": {
"forProvider": {
"region": "us-east-2"
}
},
"status": {
"observedGeneration": 0
}
}`)},
},
},
Conditions: []*fnv1.Condition{
{
Type: "FunctionSuccess",
Status: fnv1.Status_STATUS_CONDITION_TRUE,
Reason: "Success",
Target: fnv1.Target_TARGET_COMPOSITE_AND_CLAIM.Enum(),
},
},
},
},
},
}
for name, tc := range cases {
t.Run(name, func(t *testing.T) {
f := &Function{log: logging.NewNopLogger()}
rsp, err := f.RunFunction(tc.args.ctx, tc.args.req)
if diff := cmp.Diff(tc.want.rsp, rsp, protocmp.Transform()); diff != "" {
t.Errorf("%s\nf.RunFunction(...): -want rsp, +got rsp:\n%s", tc.reason, diff)
}
if diff := cmp.Diff(tc.want.err, err, cmpopts.EquateErrors()); diff != "" {
t.Errorf("%s\nf.RunFunction(...): -want err, +got err:\n%s", tc.reason, diff)
}
})
}
}
{{}}
Run the unit tests using the go test
command:
go test -v -cover .
=== RUN TestRunFunction
=== RUN TestRunFunction/AddTwoBuckets
--- PASS: TestRunFunction (0.00s)
--- PASS: TestRunFunction/AddTwoBuckets (0.00s)
PASS
coverage: 52.6% of statements
ok github.com/negz/function-xbuckets 0.016s coverage: 52.6% of statements
You can preview the output of a Composition that uses this function using the Crossplane CLI. You don't need a Crossplane control plane to do this.
Under function-xbuckets
, there is a directory named example
with Composite
Resource, Composition and Function YAML files.
Expand the following block to see example files.
{{<expand "The xr.yaml, composition.yaml and function.yaml files">}}
You can recreate the output below using by running crossplane render
with
these files.
The xr.yaml
file contains the composite resource to render:
apiVersion: example.crossplane.io/v1
kind: XBuckets
metadata:
name: example-buckets
spec:
region: us-east-2
names:
- crossplane-functions-example-a
- crossplane-functions-example-b
- crossplane-functions-example-c
The composition.yaml
file contains the Composition to use to render the
composite resource:
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
name: create-buckets
spec:
compositeTypeRef:
apiVersion: example.crossplane.io/v1
kind: XBuckets
mode: Pipeline
pipeline:
- step: create-buckets
functionRef:
name: function-xbuckets
The functions.yaml
file contains the Functions the Composition references in
its pipeline steps:
apiVersion: pkg.crossplane.io/v1
kind: Function
metadata:
name: function-xbuckets
annotations:
render.crossplane.io/runtime: Development
spec:
# The CLI ignores this package when using the Development runtime.
# You can set it to any value.
package: xpkg.crossplane.io/negz/function-xbuckets:v0.1.0
{{}}
The Function in functions.yaml
uses the
{{}}Development{{}}
runtime. This tells crossplane render
that your function is running
locally. It connects to your locally running function instead of using Docker to
pull and run the function.
apiVersion: pkg.crossplane.io/v1
kind: Function
metadata:
name: function-xbuckets
annotations:
render.crossplane.io/runtime: Development
Use go run
to run your function locally.
go run . --insecure --debug
{{<hint "warning">}} The {{}}insecure{{}} flag tells the function to run without encryption or authentication. Only use it during testing and development. {{}}
In a separate terminal, run crossplane render
.
crossplane render xr.yaml composition.yaml functions.yaml
This command calls your function. In the terminal where your function is running you should now see log output:
go run . --insecure --debug
2023-10-31T16:17:32.158-0700 INFO function-xbuckets/fn.go:29 Running Function {"tag": ""}
2023-10-31T16:17:32.159-0700 INFO function-xbuckets/fn.go:125 Added desired buckets {"xr-version": "example.crossplane.io/v1", "xr-kind": "XBuckets", "xr-name": "example-buckets", "region": "us-east-2", "count": 3}
The crossplane render
command prints the desired resources the function
returns.
---
apiVersion: example.crossplane.io/v1
kind: XBuckets
metadata:
name: example-buckets
---
apiVersion: s3.aws.upbound.io/v1beta1
kind: Bucket
metadata:
annotations:
crossplane.io/composition-resource-name: xbuckets-crossplane-functions-example-b
crossplane.io/external-name: crossplane-functions-example-b
generateName: example-buckets-
labels:
crossplane.io/composite: example-buckets
ownerReferences:
# Omitted for brevity
spec:
forProvider:
region: us-east-2
---
apiVersion: s3.aws.upbound.io/v1beta1
kind: Bucket
metadata:
annotations:
crossplane.io/composition-resource-name: xbuckets-crossplane-functions-example-c
crossplane.io/external-name: crossplane-functions-example-c
generateName: example-buckets-
labels:
crossplane.io/composite: example-buckets
ownerReferences:
# Omitted for brevity
spec:
forProvider:
region: us-east-2
---
apiVersion: s3.aws.upbound.io/v1beta1
kind: Bucket
metadata:
annotations:
crossplane.io/composition-resource-name: xbuckets-crossplane-functions-example-a
crossplane.io/external-name: crossplane-functions-example-a
generateName: example-buckets-
labels:
crossplane.io/composite: example-buckets
ownerReferences:
# Omitted for brevity
spec:
forProvider:
region: us-east-2
{{<hint "tip">}} Read the composition functions documentation to learn more about [testing composition functions]({{< ref "../concepts/compositions#test-a-composition" >}}). {{}}
Build and push the function to a package registry
You build a function in two stages. First you build the function's runtime. This
is the Open Container Initiative (OCI) image Crossplane uses to run your
function. You then embed that runtime in a package, and push it to a package
registry. The Crossplane CLI uses xpkg.crossplane.io
as its default package
registry.
A function supports a single platform, like linux/amd64
, by default. You can
support multiple platforms by building a runtime and package for each platform,
then pushing all the packages to a single tag in the registry.
Pushing your function to a registry allows you to use your function in a Crossplane control plane. See the [composition functions documentation]({{<ref "../concepts/compositions" >}}) to learn how to use a function in a control plane.
Use Docker to build a runtime for each platform.
docker build . --quiet --platform=linux/amd64 --tag runtime-amd64
sha256:fdf40374cc6f0b46191499fbc1dbbb05ddb76aca854f69f2912e580cfe624b4b
docker build . --quiet --platform=linux/arm64 --tag runtime-arm64
sha256:cb015ceabf46d2a55ccaeebb11db5659a2fb5e93de36713364efcf6d699069af
{{<hint "tip">}}
You can use whatever tag you want. There's no need to push the runtime images to
a registry. The tag is only used to tell crossplane xpkg build
what runtime to
embed.
{{}}
Use the Crossplane CLI to build a package for each platform. Each package embeds a runtime image.
The {{}}--package-root{{}} flag specifies
the package
directory, which contains crossplane.yaml
. This includes
metadata about the package.
The {{}}--embed-runtime-image{{}} flag specifies the runtime image tag built using Docker.
The {{}}--package-file{{}} flag specifies
specifies where to write the package file to disk. Crossplane package files use
the extension .xpkg
.
crossplane xpkg build \
--package-root=package \
--embed-runtime-image=runtime-amd64 \
--package-file=function-amd64.xpkg
crossplane xpkg build \
--package-root=package \
--embed-runtime-image=runtime-arm64 \
--package-file=function-arm64.xpkg
{{<hint "tip">}} Crossplane packages are special OCI images. Read more about packages in the [packages documentation]({{< ref "../concepts/packages" >}}). {{}}
Push both package files to a registry. Pushing both files to one tag in the
registry creates a
multi-platform
package that runs on both linux/arm64
and linux/amd64
hosts.
crossplane xpkg push \
--package-files=function-amd64.xpkg,function-arm64.xpkg \
negz/function-xbuckets:v0.1.0
{{<hint "tip">}}
If you push the function to a GitHub repository the template automatically sets
up continuous integration (CI) using
GitHub Actions. The CI workflow will
lint, test, and build your function. You can see how the template configures CI
by reading .github/workflows/ci.yaml
.
{{}}