cncf
/
wg-serverless
mirror of https://github.com/cncf/wg-serverless.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

New State: Callback State (#174) 

* initial add

* adding schema def and fixes

* finishing description and add example

* fix small typo

* update to image after media dir restructure

* updated to add state description in main states table

* fixing event state title format

* fixing typo

* removing extra white space
This commit is contained in:
Tihomir Surdilovic 2020-03-13 13:13:11 -04:00 committed by GitHub
parent c0850d5da6
commit 570feff8e5
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
4 changed files with 539 additions and 18 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

261
workflow/spec/examples.md
View File

@ -5,14 +5,15 @@
- [Hello World](#Hello-World-Example)
- [Greeting](#Greeting-Example)
- [Event-based greeting](#Event-Based-Greeting-Example)
- [Solving Math Problems (ForEach)](#Solving-Math-Problems-Example)
- [Solving Math Problems (ForEach state)](#Solving-Math-Problems-Example)
- [Parallel Execution](#Parallel-Execution-Example)
- [Applicant Request Decision (Switch + SubFlow)](#Applicant-Request-Decision-Example)
- [Applicant Request Decision (Switch + SubFlow states)](#Applicant-Request-Decision-Example)
- [Provision Orders (Error Handling)](#Provision-Orders-Example)
- [Monitor Job for completion (Polling)](#Monitor-Job-Example)
- [Send CloudEvent on Workflow Completion](#Send-CloudEvent-On-Workfow-Completion-Example)
- [Monitor Patient Vital Signs](#Monitor-Patient-Vital-Signs-Example)
- [Finalize College Application](#Finalize-College-Application-Example)
- [Monitor Patient Vital Signs (Event state)](#Monitor-Patient-Vital-Signs-Example)
- [Finalize College Application (Event state)](#Finalize-College-Application-Example)
- [Perform Customer Credit Check (Callback state)](#Perform-Customer-Credit-Check-Example)
### Hello World Example
@ -1846,3 +1847,255 @@ states:
<p align="center">
<img src="media/examples/example-finalizecollegeapplication.png" with="400px" height="400px" alt="Finalize College Application Example"/>
</p>
### Perform Customer Credit Check Example
#### Description
In this example our serverless workflow needs to integrate with an external microservice to perform
a credit check. We assume that this external microservice notifies a human actor which has to make
the approval decision based on customer information. Once this decision is made the service emits a CloudEvent which
includes the decision information as part of its payload.
The workflow waits for this callback event and then triggers workflow transitions based on the
credit check decision results.
The workflow data input is assumed to be:
```json
{
"customer": {
"id": "customer123",
"name": "John Doe",
"SSN": 123456,
"yearlyIncome": 50000,
"address": "123 MyLane, MyCity, MyCountry",
"employer": "MyCompany"
}
}
```
The callback event that our workflow will wait on is assumed to have the following formats.
For approved credit check, for example:
```json
{
"specversion" : "1.0",
"type" : "creditCheckCompleteType",
"datacontenttype" : "application/json",
...
"data": {
"creditCheck": [
{
"id": "customer123",
"score": 700,
"decision": "Approved",
"reason": "Good credit score"
}
]
}
}
```
And for denied credit check, for example:
```json
{
"specversion" : "1.0",
"type" : "creditCheckCompleteType",
"datacontenttype" : "application/json",
...
"data": {
"creditCheck": [
{
"id": "customer123",
"score": 580,
"decision": "Denied",
"reason": "Low credit score. Recent late payments"
}
]
}
}
```
#### Workflow Definition
<table>
<tr>
<th>JSON</th>
<th>YAML</th>
</tr>
<tr>
<td valign="top">
```json
{
"id": "customercreditcheck",
"version": "1.0",
"name": "Customer Credit Check Workflow",
"description": "Perform Customer Credit Check",
"startsAt": "CheckCredit",
"functions": [
{
"name": "callCreditCheckMicroservice",
"resource": "creditCheckResource",
"type": "microservice"
},
{
"name": "sendRejectionEmailFunction",
"resource": "rejectEmailResource"
}
],
"events": [
{
"name": "CreditCheckCompletedEvent",
"type": "creditCheckCompleteType",
"source": "creditCheckSource",
"correlationToken": "customerId"
}
],
"states": [
{
"name": "CheckCredit",
"type": "CALLBACK",
"action": {
"functionRef": {
"refName": "callCreditCheckMicroservice",
"parameters": {
"customer": "$.customer"
}
}
},
"eventRef": "CreditCheckCompletedEvent",
"timeout": "PT15M",
"transition": {
"nextState": "EvaluateDecision"
}
},
{
"name": "EvaluateDecision",
"type": "SWITCH",
"choices": [
{
"path": "$.creditCheck.decision",
"value": "Approved",
"operator": "Equals",
"transition": {
"nextState": "StartApplication"
}
},
{
"path": "$.creditCheck.decision",
"value": "Denied",
"operator": "Equals",
"transition": {
"nextState": "RejectApplication"
}
}
],
"default": {
"nextState": "RejectApplication"
}
},
{
"name": "StartApplication",
"type": "SUBFLOW",
"workflowId": "startApplicationWorkflowId",
"end": {
"type": "DEFAULT"
}
},
{
"name": "RejectApplication",
"type": "OPERATION",
"actionMode": "SEQUENTIAL",
"actions": [
{
"functionRef": {
"refName": "sendRejectionEmailFunction",
"parameters": {
"applicant": "$.customer"
}
}
}
],
"end": {
"type": "DEFAULT"
}
}
]
}
```
</td>
<td valign="top">
```yaml
id: customercreditcheck
version: '1.0'
name: Customer Credit Check Workflow
description: Perform Customer Credit Check
startsAt: CheckCredit
functions:
- name: callCreditCheckMicroservice
resource: creditCheckResource
type: microservice
- name: sendRejectionEmailFunction
resource: rejectEmailResource
events:
- name: CreditCheckCompletedEvent
type: creditCheckCompleteType
source: creditCheckSource
correlationToken: customerId
states:
- name: CheckCredit
type: CALLBACK
action:
functionRef:
refName: callCreditCheckMicroservice
parameters:
customer: "$.customer"
eventRef: CreditCheckCompletedEvent
timeout: PT15M
transition:
nextState: EvaluateDecision
- name: EvaluateDecision
type: SWITCH
choices:
- path: "$.creditCheck.decision"
value: Approved
operator: Equals
transition:
nextState: StartApplication
- path: "$.creditCheck.decision"
value: Denied
operator: Equals
transition:
nextState: RejectApplication
default:
nextState: RejectApplication
- name: StartApplication
type: SUBFLOW
workflowId: startApplicationWorkflowId
end:
type: DEFAULT
- name: RejectApplication
type: OPERATION
actionMode: SEQUENTIAL
actions:
- functionRef:
refName: sendRejectionEmailFunction
parameters:
applicant: "$.customer"
end:
type: DEFAULT
```
</td>
</tr>
</table>
#### Workflow Diagram
<p align="center">
<img src="media/examples/example-customercreditcheck.png" with="400px" height="400px" alt="Perform Customer Credit Check Example"/>
</p>

BIN
workflow/spec/media/examples/example-customercreditcheck.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 267 KiB

108
workflow/spec/schema/serverless-workflow-schema.json
View File

@ -97,6 +97,10 @@
{
"title": "ForEach State",
"$ref": "#/definitions/foreachstate"
},
{
"title": "Callback State",
"$ref": "#/definitions/callbackstate"
}
]
}
@ -335,6 +339,10 @@
{
"title": "ForEach State",
"$ref": "#/definitions/foreachstate"
},
{
"title": "Callback State",
"$ref": "#/definitions/callbackstate"
}
]
}
@ -1016,6 +1024,10 @@
{
"title": "ForEach State",
"$ref": "#/definitions/foreachstate"
},
{
"title": "Callback State",
"$ref": "#/definitions/callbackstate"
}
]
}
@ -1079,6 +1091,102 @@
}
]
},
"callbackstate": {
"type": "object",
"description": "This state performs an action, then waits for the callback event that denotes completion of the action.",
"properties": {
"id": {
"type": "string",
"description": "Unique state id",
"minLength": 1
},
"name": {
"type": "string",
"description": "State name"
},
"type": {
"type" : "string",
"enum": ["CALLBACK"],
"description": "State type"
},
"action": {
"description": "Defines the action to be executed",
"$ref": "#/definitions/action"
},
"eventRef": {
"type" : "string",
"description": "References an unique callback event name in the defined workflow events"
},
"timeout": {
"type": "string",
"description": "Time period to wait for incoming events (ISO 8601 format)"
},
"eventDataFilter": {
"description": "Callback event data filter definition",
"$ref": "#/definitions/eventdatafilter"
},
"stateDataFilter": {
"description": "State data filter definition",
"$ref": "#/definitions/statedatafilter"
},
"retry": {
"type": "array",
"description": "States retry definitions",
"items": {
"type": "object",
"$ref": "#/definitions/retry"
}
},
"onError": {
"type": "array",
"description": "States error handling definitions",
"items": {
"type": "object",
"$ref": "#/definitions/error"
}
},
"dataInputSchema": {
"type": "string",
"format": "uri",
"description": "URI to JSON Schema that state data input adheres to"
},
"dataOutputSchema": {
"type": "string",
"format": "uri",
"description": "URI to JSON Schema that state data output adheres to"
},
"transition": {
"description": "Next transition of the workflow after all the actions have been performed",
"$ref": "#/definitions/transition"
},
"end": {
"$ref": "#/definitions/end",
"description": "State end definition"
}
},
"oneOf": [
{
"required": [
"name",
"type",
"action",
"eventRef",
"timeout",
"end"
]
},
{
"required": [
"name",
"type",
"action",
"eventRef",
"timeout",
"transition"
]
}
]
},
"defaultchoice": {
"type": "object",
"description": "Default Choice",

188
workflow/spec/spec.md
View File

@ -89,9 +89,10 @@ Serverless Workflow allows users to:
2. Define which functions are executed in each step.
3. Define which event or combination of events trigger function execution.
4. Define function execution behavior (sequential, parallel, etc).
5. Specify information filtering throughout the execution of the serverless workflow.
6. Define error conditions with retries.
7. If a function is triggered by two or more events, define what label/key should be used to correlate those events to the same serverless workflow instance.
5. Specify manual decision steps during workflow execution.
6. Specify information filtering throughout the execution of the serverless workflow.
7. Define error conditions with retries.
8. Correlate trigger events with workflow instances.
Following diagram illustrates functional flow that involves states, events and functions. It shows that
incoming events can trigger function calls during flow execution.
@ -104,8 +105,8 @@ incoming events can trigger function calls during flow execution.
Following sections provide detailed descriptions of the Serverless Workflow Model. For each part of the model we provide:
- Parameter description in table format
- [JSON Schema](https://json-schema.org/) definition
- Parameter description in table format.
- [JSON Schema](https://json-schema.org/) definition.
### Workflow Model
@ -238,6 +239,10 @@ As mentioned, implementation compliance is based on the workflow definition lang
{
"title": "ForEach State",
"$ref": "#/definitions/foreachstate"
},
{
"title": "Callback State",
"$ref": "#/definitions/callbackstate"
}
]
}
@ -428,8 +433,9 @@ States define building blocks of the Serverless Workflow. The specification defi
| **[SubFlow](#SubFlow-State)** | Represents the invocation of another workflow from within a workflow | no | yes | no | yes | no | no |
| **[Relay](#Relay-State)** | Relay state data input to output | no | yes | no | yes | no | no |
| **[ForEach](#ForEach-State)** | Parallel execution of states for each element of a data array | no | yes | no | yes (includes retries) | yes | no |
| **[Callback](#Callback-State)** | Manual decision step. Executes a function and waits for callback event that indicates completion of the manual decision.| yes | yes | yes (including retries) | yes | no | no |
Following is a detailed decription of each of the defined states.
Following is a detailed decription of each of the defined states:
#### Event State
@ -597,7 +603,7 @@ have not been received during this time, the state should transition to the next
"properties": {
"eventRefs": {
"type" : "array",
"description": "References one or more unique event names in the defined workflow"
"description": "References one or more unique event names in the defined workflow events"
},
"actionMode": {
"type" : "string",
@ -687,7 +693,6 @@ instance in case it is an end state without performing any actions.
| Parameter | Description | Type | Required |
| --- | --- | --- | --- |
| [functionRef](#FunctionRef-Definition) | References a reusable function definition to be invoked | object | yes |
| timeout | Max amount of time (ISO 8601 format) to wait for the completion of the function's execution. For example: "PT15M" (wait 15 minutes), or "P2DT3H4M" (wait 2 days, 3 hours and 4 minutes) | string | no |
| [actionDataFilter](#action-data-filter) | Action data filter definition | object | no |
<details><summary><strong>Click to view JSON Schema</strong></summary>
@ -701,10 +706,6 @@ instance in case it is an end state without performing any actions.
"$ref": "#/definitions/functionref",
"description": "References a reusable function definition to be invoked"
},
"timeout": {
"type": "string",
"description": "Specifies the maximum amount of time (ISO 8601 format) to wait for the completion of the function's execution. The function timer is started when the request is sent to the invoked function"
},
"actionDataFilter": {
"$ref": "#/definitions/actiondatafilter"
}
@ -1570,6 +1571,10 @@ Branches contain one or more states. Each branch must define a starting state vi
{
"title": "ForEach State",
"$ref": "#/definitions/foreachstate"
},
{
"title": "Callback State",
"$ref": "#/definitions/callbackstate"
}
]
}
@ -1696,7 +1701,7 @@ Parallel state must wait for all branches which have this property set to "true"
</details>
It is often the case that you want to group your workflows into small, reusable logical units that perform certain needed functionality.
Even though you can use the Event state to call an externally deployed services (via function), at times
Even though you can use the Event or Callback states to call externally deployed services (via function), at times
there is a need to include/inject another serverless workflow (from classpath/local file system etc, depending on the implementation logic).
In that case you would use the SubFlow State.
It also allows users to model their workflows with reusability and logical grouping in mind.
@ -2062,6 +2067,10 @@ This allows you to test if your workflow behaves properly for cases when there a
{
"title": "ForEach State",
"$ref": "#/definitions/foreachstate"
},
{
"title": "Callback State",
"$ref": "#/definitions/callbackstate"
}
]
}
@ -2327,6 +2336,152 @@ Once iterations over the completed orders complete, workflow execution finishes
So in this example, our ForEach state will send two confirmation emails, one for each of the completed orders
defined in the orders array of its data input.
#### Callback State
| Parameter | Description | Type | Required |
| --- | --- | --- | --- |
| id | Unique state id | string | no |
| name | State name | string | yes |
| type | State type | string | yes |
| [action](#Action-Definition) | Defines the action to be executed. | object | yes |
| eventRef | References an unique callback event name in the defined workflow [events](#Event-Definition) | string | yes |
| [timeout](#eventstate-timeout) | Time period to wait from when action is executed until the callback event is received (ISO 8601 format). For example: "PT15M" (wait 15 minutes), or "P2DT3H4M" (wait 2 days, 3 hours and 4 minutes)| string | yes |
| [eventDataFilter](#event-data-filter) | Callback event data filter definition | object | no |
| [stateDataFilter](#state-data-filter) | State data filter definition | object | no |
| [retry](#workflow-retrying) | States retry definitions | array | no |
| [onError](#Workflow-Error-Handling) | States error handling definitions | array | no |
| [dataInputSchema](#Information-Passing-Between-States) | URI to JSON Schema that state data input adheres to | string | no |
| [dataOutputSchema](#Information-Passing-Between-States) | URI to JSON Schema that state data output adheres to | string | no |
| [transition](#Transitions) | Next transition of the workflow after callback event has been received | string | yes |
| [end](#End-Definition) | Is this state an end state | object | no |
<details><summary><strong>Click to view JSON Schema</strong></summary>
<p>
```json
{
"type": "object",
"description": "This state performs an action, then waits for the callback event that denotes completion of the action.",
"properties": {
"id": {
"type": "string",
"description": "Unique state id",
"minLength": 1
},
"name": {
"type": "string",
"description": "State name"
},
"type": {
"type" : "string",
"enum": ["CALLBACK"],
"description": "State type"
},
"action": {
"description": "Defines the action to be executed",
"$ref": "#/definitions/callbackaction"
},
"eventRef": {
"type" : "string",
"description": "References an unique callback event name in the defined workflow events"
},
"timeout": {
"type": "string",
"description": "Time period to wait for incoming events (ISO 8601 format)"
},
"eventDataFilter": {
"description": "Callback event data filter definition",
"$ref": "#/definitions/eventdatafilter"
},
"stateDataFilter": {
"description": "State data filter definition",
"$ref": "#/definitions/statedatafilter"
},
"retry": {
"type": "array",
"description": "States retry definitions",
"items": {
"type": "object",
"$ref": "#/definitions/retry"
}
},
"onError": {
"type": "array",
"description": "States error handling definitions",
"items": {
"type": "object",
"$ref": "#/definitions/error"
}
},
"dataInputSchema": {
"type": "string",
"format": "uri",
"description": "URI to JSON Schema that state data input adheres to"
},
"dataOutputSchema": {
"type": "string",
"format": "uri",
"description": "URI to JSON Schema that state data output adheres to"
},
"transition": {
"description": "Next transition of the workflow after all the actions have been performed",
"$ref": "#/definitions/transition"
},
"end": {
"$ref": "#/definitions/end",
"description": "State end definition"
}
},
"oneOf": [
{
"required": [
"name",
"type",
"action",
"eventRef",
"timeout",
"end"
]
},
{
"required": [
"name",
"type",
"action",
"eventRef",
"timeout",
"transition"
]
}
]
}
```
</p>
</details>
Serverless orchestration can at times require manual steps/decisions to be made. While some work performed
in a serverless workflow can be executed automatically, some decisions must involve manual steps (human decisions for example.
Callback state allows you to explicitly model manual decision steps during workflow execution.
The action property defines a function call which triggers an external activity/service. Once the action executes,
the callback state will wait for a CloudEvent (defined via the eventRef property) which indicates the completion
of the manual decision by the called service.
Note that the called decision services is responsible for emitting the callback CloudEvent indicating the completion of the
decision and including the decision results as part of the event payload. This event must be correlated to the
workflow instance using the callback events context attribute defined in the correlationToken parameter of the
referenced [Event Definition](#Event-Definition).
Once the completion (callback) event is received, the callback state completes its execution and transitions to the next
defined workflow state or completes workflow execution in case it is an end state.
The callback event payload is merged with the callback state data and can be filtered via the eventDataFilter definition.
The callback state timeout parameter defines a time period from the action execution until the callback event should be received.
If the defined callback event has not been received during this time period, the state should transition to the next state or end workflow execution (if it is an end state).
#### End Definition
| Parameter | Description | Type | Required |
@ -2640,6 +2795,8 @@ or be passed as data output to transition states.
<img src="media/spec/eventdatamerged.png" with="500px" height="300px" alt="Event data merged with state data input"/>
</p>
Similarly for Callback states, the callback event data is merged with the data input of the Callback state.
#### Information Passing Between States
States in Serverless workflow can receive data (data input) as well as produce a data result (data output). The states data input is
@ -2673,7 +2830,7 @@ There are several types of data filters defined:
- [Event Data Filter](#event-data-filter)
- [Error Data Filter](#error-data-filter)
All states can define state and error data filters. States which can consume events ([Event states](#Event-State)) can define event data filters, and states
All states can define state and error data filters. States which can consume events ([Event states](#Event-State), [Callback states](#Callback-State)) can define event data filters, and states
that can perform actions ([Event states](#Event-State), [Operation states](#Operation-State)) can define action data filters for each of the
actions they perform.
@ -2871,6 +3028,9 @@ Here is an example using an even filter:
<img src="media/spec/event-data-filter-example1.png" with="300px" height="400px" alt="Event Data Filter Example"/>
</p>
Similarly the consumed callback CloudEvent in [Callback states](#Callback-State) can be filered using
an event filter.
#### <a name="error-data-filter"></a> State information filtering - Error Data Filter
| Parameter | Description | Type | Required |