Clean up create event source docs (#3365)

* Clean up create event source docs

* moved files

* updates
This commit is contained in:
Ashleigh Brennan 2021-03-30 16:09:21 -05:00 committed by GitHub
parent cc325fa475
commit fc79242c37
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
18 changed files with 348 additions and 365 deletions

View File

@ -1,299 +0,0 @@
# Introduction
As stated in [tutorial on writing a Source with a Receive Adapter](../writing-event-source/README.md), there are multiple ways to
create event sources. The way in that tutorial is to create an independent event source that has its own CRD.
This tutorial provides a simpler mechanism to build an event source in Javascript and use it with
[ContainerSource](../../../eventing/sources/README.md#meta-sources) and / or the [SinkBinding](../../../eventing/sources/README.md#meta-sources).
[ContainerSource](../../../eventing/sources/README.md#meta-sources) is an easy way to turn any dispatcher container into an Event Source.
Similarly, another option is using [SinkBinding](../../../eventing/sources/README.md#meta-sources)
which provides a framework for injecting environment variables into any Kubernetes resource which has a `spec.template` that looks like a Pod (aka PodSpecable).
SinkBinding is a newer concept and it should be preferred over ContainerSource.
Code for this tutorial is available [here](https://github.com/knative/docs/tree/main/docs/eventing/samples/writing-event-source-easy-way).
# Bootstrapping
Create the project and add the dependencies:
```bash
npm init
npm install cloudevents-sdk@2.0.1 --save
```
Please note that because of a [bug](https://github.com/cloudevents/sdk-javascript/issues/191), you will
need at least `2.0.1` version of the Javascript SDK.
## Making use of ContainerSource
`ContainerSource` and `SinkBinding` both work by injecting environment variables to an application.
Injected environment variables at minimum contain the URL of a sink that will receive events.
Following example emits an event to the sink every 1000 milliseconds.
The sink URL to post the events will be made available to the application via the `K_SINK` environment variable by `ContainerSource`.
```javascript
// File - index.js
const { CloudEvent, HTTPEmitter } = require("cloudevents-sdk");
let sinkUrl = process.env['K_SINK'];
console.log("Sink URL is " + sinkUrl);
let emitter = new HTTPEmitter({
url: sinkUrl
});
let eventIndex = 0;
setInterval(function () {
console.log("Emitting event #" + ++eventIndex);
let myevent = new CloudEvent({
source: "urn:event:from:my-api/resource/123",
type: "your.event.source.type",
id: "your-event-id",
dataContentType: "application/json",
data: {"hello": "World " + eventIndex},
});
// Emit the event
emitter.send(myevent)
.then(response => {
// Treat the response
console.log("Event posted successfully");
console.log(response.data);
})
.catch(err => {
// Deal with errors
console.log("Error during event post");
console.error(err);
});
}, 1000);
```
```dockerfile
# File - Dockerfile
FROM node:10
WORKDIR /usr/src/app
COPY package*.json ./
RUN npm install
COPY . .
EXPOSE 8080
CMD [ "node", "index.js" ]
```
Build and push the image:
```bash
docker build . -t path/to/image/registry/node-knative-heartbeat-source:v1
docker push path/to/image/registry/node-knative-heartbeat-source:v1
```
Create the event display service which simply logs any cloudevents posted to it.
```bash
cat <<EOS |kubectl apply -f -
---
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
name: event-display
spec:
template:
spec:
containers:
- image: docker.io/aliok/event_display-864884f202126ec3150c5fcef437d90c@sha256:93cb4dcda8fee80a1f68662ae6bf20301471b046ede628f3c3f94f39752fbe08
EOS
```
Create the `ContainerSource`:
```bash
cat <<EOS |kubectl apply -f -
---
apiVersion: sources.knative.dev/v1
kind: ContainerSource
metadata:
name: test-heartbeats
spec:
template:
spec:
containers:
- image: path/to/image/registry/node-knative-heartbeat-source:v1
name: heartbeats
sink:
ref:
apiVersion: serving.knative.dev/v1
kind: Service
name: event-display
EOS
```
Check the logs of the event display service. You will see a new message is pushed every second:
```bash
$ kubectl logs -l serving.knative.dev/service=event-display -c user-container
☁️ cloudevents.Event
Validation: valid
Context Attributes,
specversion: 1.0
type: your.event.source.type
source: urn:event:from:your-api/resource/123
id: your-event-id
datacontenttype: application/json
Data,
{
"hello": "World 1"
}
```
If you are interested in seeing what is injected into the event source as a `K_SINK`, you can check the logs:
```bash
$ kubectl logs test-heartbeats-deployment-7575c888c7-85w5t
Sink URL is http://event-display.default.svc.cluster.local
Emitting event #1
Emitting event #2
Event posted successfully
Event posted successfully
```
Please note that the example code above is using _Binary_ mode for CloudEvents.
Simply change
```javascript
let binding = new v1.BinaryHTTPEmitter(config);
```
with
```javascript
let binding = new v1.StructuredHTTPEmitter(config);
```
to employ structured mode.
However, binary mode should be used in most of the cases as:
- It is faster in terms of serialization and deserialization
- It works better with cloudevents-aware proxies (like Knative Channels) can simply check the header instead of parsing the payload
## Making use of SinkBinding
`SinkBinding` is a more powerful way of making any Kubernetes resource an event source.
`ContainerSource` will create the container for your event source's image and it will be `ContainerSource` responsibility to manage the container.
`SinkBinding` though, will not create any containers. It will inject the sink information to the already existing Kubernetes resources.
This is a more flexible approach as you can use any Kubernetes `PodSpecable` as an event source, such as `Deployment`, `Job`, `Knative Service`, `DaemonSet` etc.
We don't need any code changes in our source for making it work with `SinkBinding`.
Create the event display as in the section before:
```bash
cat <<EOS |kubectl apply -f -
---
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
name: event-display
spec:
template:
spec:
containers:
- image: docker.io/aliok/event_display-864884f202126ec3150c5fcef437d90c@sha256:93cb4dcda8fee80a1f68662ae6bf20301471b046ede628f3c3f94f39752fbe08
EOS
```
Create a Kubernetes deployment that runs the event source:
```bash
cat <<EOS |kubectl apply -f -
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: node-heartbeats-deployment
labels:
app: node-heartbeats
spec:
replicas: 2
selector:
matchLabels:
app: node-heartbeats
template:
metadata:
labels:
app: node-heartbeats
spec:
containers:
- name: node-heartbeats
image: path/to/image/registry/node-knative-heartbeat-source:v1
ports:
- containerPort: 8080
EOS
```
As the `SinkBinding` is not created yet, `K_SINK` environment variable is not yet injected and the event source will complain about that.
```
$ kubectl logs node-heartbeats-deployment-9ffbb644b-llkzk
Sink URL is undefined
Emitting event #1
Error during event post
TypeError [ERR_INVALID_ARG_TYPE]: The "url" argument must be of type string. Received type undefined
```
Create the `SinkBinding`:
```bash
cat <<EOS |kubectl apply -f -
---
apiVersion: sources.knative.dev/v1
kind: SinkBinding
metadata:
name: bind-node-heartbeat
spec:
subject:
apiVersion: apps/v1
kind: Deployment
selector:
matchLabels:
app: node-heartbeats
sink:
ref:
apiVersion: serving.knative.dev/v1
kind: Service
name: event-display
EOS
```
You will see the pods are recreated and this time the `K_SINK` environment variable is injected.
Also note that since the `replicas` is set to 2, there will be 2 pods that are posting events to the sink.
```bash
$ kubectl logs event-display-dpplv-deployment-67c9949cf9-bvjvk -c user-container
☁️ cloudevents.Event
Validation: valid
Context Attributes,
specversion: 1.0
type: your.event.source.type
source: urn:event:from:your-api/resource/123
id: your-event-id
datacontenttype: application/json
Data,
{
"hello": "World 1"
}
☁️ cloudevents.Event
Validation: valid
Context Attributes,
specversion: 1.0
type: your.event.source.type
source: urn:event:from:your-api/resource/123
id: your-event-id
datacontenttype: application/json
Data,
{
"hello": "World 1"
}
```

View File

@ -1,8 +0,0 @@
---
title: "Writing an Event Source using the easy way"
linkTitle: "Event Source - easy way"
weight: 10
type: "docs"
---
{{% readfile file="README.md" %}}

View File

@ -1,50 +0,0 @@
## Introduction
This tutorial will walk you though writing a new event source for Knative
Eventing using a sample repository and explaining the key concepts used
throughout each component.
After completing the tutorial, you'll have a basic event source controller as
well as receive adapter, which events can be viewed through a basic
`event_display` Knative Service.
Just want to see the code? The reference project is
[https://github.com/knative-sandbox/sample-source](https://github.com/knative-sandbox/sample-source).
[Knative Sources](../../sources/#knative-sources) can be used as a reference.
## Other ways
With the approach in this tutorial, you will create a CRD and a controller for the event source which makes it reusable.
You can also write your own event source using a [ContainerSource](../../../eventing/sources/README.md#meta-sources) which
is an easy way to turn any dispatcher container into an Event Source. Similarly, another option is using [SinkBinding](../../../eventing/sources/README.md#meta-sources)
which provides a framework for injecting environment variables into any Kubernetes resource which has a `spec.template` that looks like a Pod (aka PodSpecable).
## Target Audience
The target audience is already familiar with Kubernetes and Go development and
wants to develop a new event source, importing their custom events via Knative
Eventing into the Knative system.
## Before You Begin
You'll need these tools installed:
- git
- golang
- [ko](https://github.com/google/ko/) (optional)
- [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/) (optional)
- [minikube](https://github.com/kubernetes/minikube) (optional)
You're encouraged to clone the [sample source](https://github.com/knative-sandbox/sample-source) and make changes there.
## Steps
1. [Separation of Concerns](./01-theory.md)
2. [API Definition](./02-lifecycle-and-types.md)
3. [Controller](./03-controller.md)
4. [Reconciler](./04-reconciler.md)
5. [Receive Adapter](./05-receive-adapter.md)
6. [Example YAML](./06-yaml.md)
7. [Moving the event source to the `knative-sandbox` organization](./07-knative-sandbox.md)

View File

@ -1,7 +0,0 @@
---
title: "Writing an Event Source"
weight: 60
type: "docs"
---
{{% readfile file="README.md" %}}

View File

@ -0,0 +1,10 @@
---
title: "Creating an event source"
weight: 100
type: "docs"
---
You can create your own event source for use with Knative Eventing components by using the following methods:
- Build an event source in Javascript, and implement it using a ContainerSource or SinkBinding.
- By creating your own event source controller, receiver adapter, and custom resource definition (CRD).

View File

@ -0,0 +1,281 @@
---
title: "Writing an event source using Javascript"
weight: 10
type: "docs"
showlandingtoc: "false"
aliases:
- /docs/eventing/samples/writing-event-source-easy-way
---
This tutorial provides instructions to build an event source in Javascript and implement it with a ContainerSource or SinkBinding.
- Using a [ContainerSource](../../containersource) is a simple way to turn any dispatcher container into a Knative event source.
- Using [SinkBinding](../../sinkbinding) provides a framework for injecting environment variables into any Kubernetes resource that has a `spec.template` and is [PodSpecable](https://pkg.go.dev/knative.dev/pkg/apis/duck/v1#PodSpecable).
ContainerSource and SinkBinding both work by injecting environment variables to an application. Injected environment variables at minimum contain the URL of a sink that will receive events.
## Bootstrapping
Create the project and add the dependencies:
```bash
npm init
npm install cloudevents-sdk@2.0.1 --save
```
**NOTE:** Due to this [bug](https://github.com/cloudevents/sdk-javascript/issues/191), you must use version 2.0.1 of the Javascript SDK or newer.
## Using ContainerSource
A ContainerSource creates a container for your event source image and manages this container.
The sink URL to post the events will be made available to the application through the `K_SINK` environment variable by the ContainerSource.
### Example
The following example event source emits an event to the sink every 1000 milliseconds:
```javascript
// File - index.js
const { CloudEvent, HTTPEmitter } = require("cloudevents-sdk");
let sinkUrl = process.env['K_SINK'];
console.log("Sink URL is " + sinkUrl);
let emitter = new HTTPEmitter({
url: sinkUrl
});
let eventIndex = 0;
setInterval(function () {
console.log("Emitting event #" + ++eventIndex);
let myevent = new CloudEvent({
source: "urn:event:from:my-api/resource/123",
type: "your.event.source.type",
id: "your-event-id",
dataContentType: "application/json",
data: {"hello": "World " + eventIndex},
});
// Emit the event
emitter.send(myevent)
.then(response => {
// Treat the response
console.log("Event posted successfully");
console.log(response.data);
})
.catch(err => {
// Deal with errors
console.log("Error during event post");
console.error(err);
});
}, 1000);
```
```dockerfile
# File - Dockerfile
FROM node:10
WORKDIR /usr/src/app
COPY package*.json ./
RUN npm install
COPY . .
EXPOSE 8080
CMD [ "node", "index.js" ]
```
The example code uses _Binary_ mode for CloudEvents. To employ structured code, change `let binding = new v1.BinaryHTTPEmitter(config);` to `let binding = new v1.StructuredHTTPEmitter(config);`.
Binary mode is used in most cases because:
- It is faster in terms of serialization and deserialization.
- It works better with CloudEvent-aware proxies, such as Knative Channels, and can simply check the header instead of parsing the payload.
### Procedure
1. Build and push the image:
```bash
docker build . -t path/to/image/registry/node-knative-heartbeat-source:v1
docker push path/to/image/registry/node-knative-heartbeat-source:v1
```
2. Create the event display service which logs any CloudEvents posted to it:
```yaml
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
name: event-display
spec:
template:
spec:
containers:
- image: docker.io/aliok/event_display-864884f202126ec3150c5fcef437d90c@sha256:93cb4dcda8fee80a1f68662ae6bf20301471b046ede628f3c3f94f39752fbe08
```
3. Create the ContainerSource object:
```yaml
apiVersion: sources.knative.dev/v1
kind: ContainerSource
metadata:
name: test-heartbeats
spec:
template:
spec:
containers:
- image: path/to/image/registry/node-knative-heartbeat-source:v1
name: heartbeats
sink:
ref:
apiVersion: serving.knative.dev/v1
kind: Service
name: event-display
```
4. Check the logs of the event display service. You will see a new message is pushed every second:
```bash
$ kubectl logs -l serving.knative.dev/service=event-display -c user-container
☁️ cloudevents.Event
Validation: valid
Context Attributes,
specversion: 1.0
type: your.event.source.type
source: urn:event:from:your-api/resource/123
id: your-event-id
datacontenttype: application/json
Data,
{
"hello": "World 1"
}
```
5. Optional: If you are interested in seeing what is injected into the event source as a `K_SINK`, you can check the logs:
```bash
$ kubectl logs test-heartbeats-deployment-7575c888c7-85w5t
Sink URL is http://event-display.default.svc.cluster.local
Emitting event #1
Emitting event #2
Event posted successfully
Event posted successfully
```
## Using SinkBinding
SinkBinding does not create any containers. It injects the sink information to an already existing Kubernetes resources. This is a flexible approach as you can use any Kubernetes PodSpecable object as an event source, such as Deployment, Job, or Knative services.
### Procedure
1. Create an event display service:
```yaml
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
name: event-display
spec:
template:
spec:
containers:
- image: docker.io/aliok/event_display-864884f202126ec3150c5fcef437d90c@sha256:93cb4dcda8fee80a1f68662ae6bf20301471b046ede628f3c3f94f39752fbe08
```
2. Create a Kubernetes deployment that runs the event source:
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: node-heartbeats-deployment
labels:
app: node-heartbeats
spec:
replicas: 2
selector:
matchLabels:
app: node-heartbeats
template:
metadata:
labels:
app: node-heartbeats
spec:
containers:
- name: node-heartbeats
image: path/to/image/registry/node-knative-heartbeat-source:v1
ports:
- containerPort: 8080
```
3. Because the SinkBinding has not yet been created, you will see an error message, because the `K_SINK` environment variable is not yet injected:
```bash
$ kubectl logs node-heartbeats-deployment-9ffbb644b-llkzk
Sink URL is undefined
Emitting event #1
Error during event post
TypeError [ERR_INVALID_ARG_TYPE]: The "url" argument must be of type string. Received type undefined
```
4. Create the SinkBinding object:
```yaml
apiVersion: sources.knative.dev/v1
kind: SinkBinding
metadata:
name: bind-node-heartbeat
spec:
subject:
apiVersion: apps/v1
kind: Deployment
selector:
matchLabels:
app: node-heartbeats
sink:
ref:
apiVersion: serving.knative.dev/v1
kind: Service
name: event-display
```
You will see the pods are recreated and this time the `K_SINK` environment variable is injected.
Also note that since the `replicas` is set to 2, there will be 2 pods that are posting events to the sink.
```bash
$ kubectl logs event-display-dpplv-deployment-67c9949cf9-bvjvk -c user-container
☁️ cloudevents.Event
Validation: valid
Context Attributes,
specversion: 1.0
type: your.event.source.type
source: urn:event:from:your-api/resource/123
id: your-event-id
datacontenttype: application/json
Data,
{
"hello": "World 1"
}
☁️ cloudevents.Event
Validation: valid
Context Attributes,
specversion: 1.0
type: your.event.source.type
source: urn:event:from:your-api/resource/123
id: your-event-id
datacontenttype: application/json
Data,
{
"hello": "World 1"
}
```

View File

@ -3,6 +3,8 @@ title: "Design and Theory Behind an Event Source"
linkTitle: "Design of an Event Source"
weight: 10
type: "docs"
aliases:
- /docs/eventing/samples/writing-event-source/01-theory
---
# Topics
@ -103,7 +105,7 @@ ${KNATIVE_CODEGEN_PKG}/hack/generate-knative.sh "injection" \
File Layout & Hierarchy:
* `cmd/controller/main.go` - Pass sources NewController implementation to the shared main
* `cmd/receive_adapter/main.go` - Translate resource variables to underlying adapter struct (to eventually be passed into the serverless system)
* `cmd/receive_adapter/main.go` - Translate resource variables to underlying adapter struct (to eventually be passed into the serverless system)
* `pkg/reconciler/sample/controller.go` - NewController implementation to pass to sharedmain
* `pkg/reconciler/sample/samplesource.go` - reconciliation functions for the receive adapter
* `pkg/apis/samples/VERSION/samplesource_types.go` - schema for the underlying api types (variables to be defined in the resource yaml)

View File

@ -3,6 +3,8 @@ title: "Sample Source Lifecycle and Types"
linkTitle: "Lifecycle and Types"
weight: 20
type: "docs"
aliases:
- /docs/eventing/samples/writing-event-source/02-lifecycle-and-types
---
## API Definition

View File

@ -3,6 +3,8 @@ title: "Controller Implementation and Design"
linkTitle: "Controller Implemetation"
weight: 30
type: "docs"
aliases:
- /docs/eventing/samples/writing-event-source/03-controller
---
# Controller Implementation

View File

@ -3,6 +3,8 @@ title: "Reconciler Implementation and Design"
linkTitle: "Reconciler Implementation"
weight: 40
type: "docs"
aliases:
- /docs/eventing/samples/writing-event-source/04-reconciler
---
## Reconciler Functionality

View File

@ -3,6 +3,8 @@ title: "Receive Adapter Implementation and Design"
linkTitle: "Receive Adapter Implementation"
weight: 50
type: "docs"
aliases:
- /docs/eventing/samples/writing-event-source/05-receive-adapter
---
## Receive Adapter cmd

View File

@ -3,6 +3,8 @@ title: "Publishing to your Kubernetes cluster"
linkTitle: "Publishing to your cluster"
weight: 60
type: "docs"
aliases:
- /docs/eventing/samples/writing-event-source/06-yaml
---
## Run the sample source locally

View File

@ -3,6 +3,8 @@ title: "Moving the event source to knative-sandbox"
linkTitle: "Moving to knative-sandbox"
weight: 70
type: "docs"
aliases:
- /docs/eventing/samples/writing-event-source/07-knative-sandbox
---
If you would like to move your source over to the [`knative-sandbox`](https://github.com/knative-sandbox) organization follow the instructions to [create a sandbox repository](https://knative.dev/community/contributing/mechanics/creating-a-sandbox-repo/).

View File

@ -0,0 +1,42 @@
---
title: "Creating an event source by using the sample event source"
linkTitle: "Using the sample event source"
weight: 60
type: "docs"
showlandingtoc: "false"
aliases:
- /docs/eventing/samples/writing-event-source
---
This guide explains how to create your own event source for Knative
Eventing by using a [sample repository](https://github.com/knative-sandbox/sample-source), and explains the key concepts behind each required component. Documentation for the default [Knative event sources](../../sources/) can be used as an additional reference.
After completing the provided tutorial, you will have created a basic event source controller and a receive adapter. Events can be viewed by using the `event_display` Knative service.
<!--TODO: Provide links to docs about what the event source controller and receiver adapter are-->
<!-- Is Go required? Is this for all Knative development or just event source creation?-->
## Prerequisites
- You are familiar with Kubernetes and Go development.
- You have installed Git.
- You have installed Go.
- Clone the [sample source](https://github.com/knative-sandbox/sample-source). <!--optional?-->
<!-- add links, versions if required-->
<!---TODO: decide...Maybe don't list these if they're optional, unless they're called out in a procedure-->
Optional:
- Install the [ko](https://github.com/google/ko/) CLI tool.
- Install the [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/) CLI tool.
- Set up [minikube](https://github.com/kubernetes/minikube).
## Steps
1. [Separation of Concerns](./01-theory)
2. [API Definition](./02-lifecycle-and-types)
3. [Controller](./03-controller)
4. [Reconciler](./04-reconciler)
5. [Receive Adapter](./05-receive-adapter)
6. [Example YAML](./06-yaml)
7. [Moving the event source to the `knative-sandbox` organization](./07-knative-sandbox)