mirror of https://github.com/dapr/docs.git
Merge branch 'v1.0-rc3' into issue-1089
This commit is contained in:
commit
dd3bd72cb3
|
@ -0,0 +1,72 @@
|
||||||
|
---
|
||||||
|
type: docs
|
||||||
|
title: "Message Time-To-Live"
|
||||||
|
linkTitle: "Message TTL"
|
||||||
|
weight: 6000
|
||||||
|
description: "Use time-to-live in Pub/Sub messages."
|
||||||
|
---
|
||||||
|
|
||||||
|
## Introduction
|
||||||
|
|
||||||
|
Dapr enables per-message time-to-live (TTL). This means that applications can set time-to-live per message, and subscribers will not receive those messages after expiration.
|
||||||
|
|
||||||
|
All Dapr [pub/sub components]({{< ref supported-pubsub >}}) are compatible with message TTL, as Dapr handles the TTL logic within the runtime. Simply set the `ttlInSeconds` metadata when publishing a message.
|
||||||
|
|
||||||
|
In some components, such as Kafka, time-to-live can be configured in the topic via `retention.ms` as per [documentation](https://kafka.apache.org/documentation/#topicconfigs_retention.ms). With message TTL in Dapr, applications using Kafka can now set time-to-live per message in addition to per topic.
|
||||||
|
|
||||||
|
## Native message TTL support
|
||||||
|
|
||||||
|
When message time-to-live has native support in the component, Dapr will simply forward the time-to-live configuration without adding extra logic, keeping predictable behavior. This is helpful when the expired messages are handled differently by the component - like in Azure Service Bus, where expired messages are stored in the dead letter queue and not simply deleted.
|
||||||
|
|
||||||
|
### Supported components
|
||||||
|
|
||||||
|
#### Azure Service Bus
|
||||||
|
|
||||||
|
Azure Service Bus supports [entity level time-to-live]((https://docs.microsoft.com/en-us/azure/service-bus-messaging/message-expiration)). It means that messages have a default time-to-live but can also be set with a shorter timespan at publishing time. Dapr will propagate the time-to-live metadata for the message and let Azure Service Bus handle expiration directly.
|
||||||
|
|
||||||
|
## Non-Dapr subscribers
|
||||||
|
|
||||||
|
If messages are consumed by subscribers without Dapr, expired messages are not automatically dropped, as expiration is handled by the Dapr runtime when a Dapr application receives a message. However, subscribers can programmatically drop expired messages by adding logic to handle the `expiration` attribute in the cloud event, which follows the [RFC3339](https://tools.ietf.org/html/rfc3339) format.
|
||||||
|
|
||||||
|
When non-Dapr subscribers use components such as Azure Service Bus, which natively handle message TTL, they will not receive expired messages. No extra logic is needed.
|
||||||
|
|
||||||
|
## Example
|
||||||
|
|
||||||
|
Message TTL can be set in the metadata as part of the publishing request:
|
||||||
|
|
||||||
|
{{< tabs curl "Python SDK">}}
|
||||||
|
|
||||||
|
{{% codetab %}}
|
||||||
|
```bash
|
||||||
|
curl -X "POST" http://localhost:3500/v1.0/publish/pubsub/TOPIC_A?metadata.ttlInSeconds=120 -H "Content-Type: application/json" -d '{"order-number": "345"}'
|
||||||
|
```
|
||||||
|
{{% /codetab %}}
|
||||||
|
|
||||||
|
{{% codetab %}}
|
||||||
|
```python
|
||||||
|
from dapr.clients import DaprClient
|
||||||
|
|
||||||
|
with DaprClient() as d:
|
||||||
|
req_data = {
|
||||||
|
'order-number': '345'
|
||||||
|
}
|
||||||
|
# Create a typed message with content type and body
|
||||||
|
resp = d.publish_event(
|
||||||
|
pubsub_name='pubsub',
|
||||||
|
topic='TOPIC_A',
|
||||||
|
data=json.dumps(req_data),
|
||||||
|
metadata=(
|
||||||
|
('ttlInSeconds', '120')
|
||||||
|
)
|
||||||
|
)
|
||||||
|
# Print the request
|
||||||
|
print(req_data, flush=True)
|
||||||
|
```
|
||||||
|
{{% /codetab %}}
|
||||||
|
|
||||||
|
{{< /tabs >}}
|
||||||
|
|
||||||
|
See [this guide]({{< ref pubsub_api.md >}}) for a full reference on the pub/sub API.
|
||||||
|
|
||||||
|
## Related links
|
||||||
|
- [Pub/sub API reference]({{< ref pubsub_api.md >}})
|
|
@ -8,7 +8,7 @@ description: "Use Dapr Pub/Sub with multiple namespaces"
|
||||||
|
|
||||||
In some scenarios, applications can be spread across namespaces and share a queue or topic via PubSub. In this case, the PubSub component must be provisioned on each namespace.
|
In some scenarios, applications can be spread across namespaces and share a queue or topic via PubSub. In this case, the PubSub component must be provisioned on each namespace.
|
||||||
|
|
||||||
In this example, we will use the [PubSub sample](https://github.com/dapr/quickstarts/tree/master/pub-sub). Redis installation and the subscribers will be in `namespace-a` while the publisher UI will be on `namespace-b`. This solution should also work if Redis was installed on another namespace or if we used a managed cloud service like Azure ServiceBus.
|
This example uses the [PubSub sample](https://github.com/dapr/quickstarts/tree/master/pub-sub). The Redis installation and the subscribers are in `namespace-a` while the publisher UI is in `namespace-b`. This solution should also work if Redis is installed on another namespace or if you use a managed cloud service like Azure ServiceBus.
|
||||||
|
|
||||||
The table below shows which resources are deployed to which namespaces:
|
The table below shows which resources are deployed to which namespaces:
|
||||||
|
|
||||||
|
@ -43,7 +43,6 @@ apiVersion: dapr.io/v1alpha1
|
||||||
kind: Component
|
kind: Component
|
||||||
metadata:
|
metadata:
|
||||||
name: pubsub
|
name: pubsub
|
||||||
namespace: default
|
|
||||||
spec:
|
spec:
|
||||||
type: pubsub.redis
|
type: pubsub.redis
|
||||||
version: v1
|
version: v1
|
||||||
|
|
|
@ -3,6 +3,6 @@ type: docs
|
||||||
title: "Supported pub/sub components"
|
title: "Supported pub/sub components"
|
||||||
linkTitle: "Supported pub/sub"
|
linkTitle: "Supported pub/sub"
|
||||||
weight: 30000
|
weight: 30000
|
||||||
description: List of all the supported external bindings that can interface with Dapr
|
description: List of all the supported external pubsub brokers that can interface with Dapr
|
||||||
simple_list: true
|
simple_list: true
|
||||||
---
|
---
|
|
@ -40,8 +40,8 @@ The above example uses secrets as plain strings. It is recommended to use a secr
|
||||||
|--------------------|:--------:|---------|---------|
|
|--------------------|:--------:|---------|---------|
|
||||||
| brokers | Y | Comma separated list of kafka brokers | `localhost:9092`, `dapr-kafka.myapp.svc.cluster.local:9092`
|
| brokers | Y | Comma separated list of kafka brokers | `localhost:9092`, `dapr-kafka.myapp.svc.cluster.local:9092`
|
||||||
| authRequired | N | Enable authentication on the Kafka broker. Defaults to `"false"`. |`"true"`, `"false"`
|
| authRequired | N | Enable authentication on the Kafka broker. Defaults to `"false"`. |`"true"`, `"false"`
|
||||||
| saslUsername | N | Username used for authentication. Only required if authRequired is set to true | `"adminuser"`
|
| saslUsername | N | Username used for authentication. Only required if authRequired is set to true. | `"adminuser"`
|
||||||
| saslPassword | N | Password used for authentication. Can be `secretKeyRef` to use a secret reference. Only required if authRequired is set to true | `""`, `"KeFg23!"`
|
| saslPassword | N | Password used for authentication. Can be `secretKeyRef` to use a secret reference. Only required if authRequired is set to true. Can be `secretKeyRef` to use a [secret reference]({{< ref component-secrets.md >}}) | `""`, `"KeFg23!"`
|
||||||
|
|
||||||
|
|
||||||
## Create a Kafka instance
|
## Create a Kafka instance
|
||||||
|
@ -61,5 +61,5 @@ To run Kafka on Kubernetes, you can use the [Helm Chart](https://github.com/helm
|
||||||
|
|
||||||
## Related links
|
## Related links
|
||||||
- [Basic schema for a Dapr component]({{< ref component-schema >}})
|
- [Basic schema for a Dapr component]({{< ref component-schema >}})
|
||||||
- Read [this topic]({{< ref "howto-publish-subscribe.md##step-1-setup-the-pubsub-component" >}}) for instructions on configuring pub/sub components
|
- Read [this guide]({{< ref "howto-publish-subscribe.md##step-1-setup-the-pubsub-component" >}}) for instructions on configuring pub/sub components
|
||||||
- [Pub/Sub building block]({{< ref pubsub >}})
|
- [Pub/Sub building block]({{< ref pubsub >}})
|
|
@ -5,9 +5,56 @@ linkTitle: "AWS SNS/SQS"
|
||||||
description: "Detailed documentation on the AWS SNS/SQS pubsub component"
|
description: "Detailed documentation on the AWS SNS/SQS pubsub component"
|
||||||
---
|
---
|
||||||
|
|
||||||
This article describes configuring Dapr to use AWS SNS/SQS for pub/sub on local and Kubernetes environments.
|
## Component format
|
||||||
|
To setup AWS SNS/SQS for pub/sub, you create a component of type `pubsub.snssqs`. See [this guide]({{< ref "howto-publish-subscribe.md#step-1-setup-the-pubsub-component" >}}) on how to create and apply a pubsub configuration.
|
||||||
|
|
||||||
## Setup SNS/SQS
|
```yaml
|
||||||
|
apiVersion: dapr.io/v1alpha1
|
||||||
|
kind: Component
|
||||||
|
metadata:
|
||||||
|
name: snssqs-pubsub
|
||||||
|
namespace: default
|
||||||
|
spec:
|
||||||
|
type: pubsub.snssqs
|
||||||
|
version: v1
|
||||||
|
metadata:
|
||||||
|
- name: accessKey
|
||||||
|
value: "AKIAIOSFODNN7EXAMPLE"
|
||||||
|
- name: secretKey
|
||||||
|
value: "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
|
||||||
|
- name: region
|
||||||
|
value: "us-east-1"
|
||||||
|
- name: sessionToken
|
||||||
|
value: "TOKEN"
|
||||||
|
- name: messageVisibilityTimeout
|
||||||
|
value: 10
|
||||||
|
- name: messageRetryLimit
|
||||||
|
value: 10
|
||||||
|
- name: messageWaitTimeSeconds
|
||||||
|
value: 1
|
||||||
|
- name: messageMaxNumber
|
||||||
|
value: 10
|
||||||
|
```
|
||||||
|
|
||||||
|
{{% alert title="Warning" color="warning" %}}
|
||||||
|
The above example uses secrets as plain strings. It is recommended to use a secret store for the secrets as described [here]({{< ref component-secrets.md >}}).
|
||||||
|
{{% /alert %}}
|
||||||
|
|
||||||
|
## Spec metadata fields
|
||||||
|
|
||||||
|
| Field | Required | Details | Example |
|
||||||
|
|--------------------|:--------:|---------|---------|
|
||||||
|
| accessKey | Y | ID of the AWS account with appropriate permissions to SNS and SQS. Can be `secretKeyRef` to use a secret reference | `"AKIAIOSFODNN7EXAMPLE"`
|
||||||
|
| secretKey | Y | Secret for the AWS user. Can be `secretKeyRef` to use a secret reference |`"wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"`
|
||||||
|
| region | Y | The AWS region to the instance. See this page for valid regions: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.RegionsAndAvailabilityZones.html. Ensure that SNS and SQS are available in that region.| `"us-east-1"`
|
||||||
|
| endpoint | N |AWS endpoint for the component to use. Only used for local development. The `endpoint` is unncessary when running against production AWS | `"http://localhost:4566"`
|
||||||
|
| sessionToken | N |AWS session token to use. A session token is only required if you are using temporary security credentials. | `"TOKEN"`
|
||||||
|
| messageVisibilityTimeout | N |Amount of time in seconds that a message is hidden from receive requests after it is sent to a subscriber. Default: `10` | `10`
|
||||||
|
| messageRetryLimit | N |Number of times to resend a message after processing of that message fails before removing that message from the queue. Default: `10` | `10`
|
||||||
|
| messageWaitTimeSeconds | N |amount of time to await receipt of a message before making another request. Default: `1` | `1`
|
||||||
|
| messageMaxNumber | N |maximum number of messages to receive from the queue at a time. Default: `10`, Maximum: `10` | `10`
|
||||||
|
|
||||||
|
## Create an SNS/SQS instance
|
||||||
|
|
||||||
{{< tabs "Self-Hosted" "Kubernetes" "AWS" >}}
|
{{< tabs "Self-Hosted" "Kubernetes" "AWS" >}}
|
||||||
|
|
||||||
|
@ -23,7 +70,7 @@ See [Authenticating to AWS]({{< ref authenticating-aws.md >}}) for information a
|
||||||
apiVersion: dapr.io/v1alpha1
|
apiVersion: dapr.io/v1alpha1
|
||||||
kind: Component
|
kind: Component
|
||||||
metadata:
|
metadata:
|
||||||
name: messagebus
|
name: snssqs-pubsub
|
||||||
spec:
|
spec:
|
||||||
type: pubsub.snssqs
|
type: pubsub.snssqs
|
||||||
version: v1
|
version: v1
|
||||||
|
@ -31,7 +78,7 @@ spec:
|
||||||
- name: endpoint
|
- name: endpoint
|
||||||
value: http://localhost:4566
|
value: http://localhost:4566
|
||||||
# Use us-east-1 for localstack
|
# Use us-east-1 for localstack
|
||||||
- name: awsRegion
|
- name: region
|
||||||
value: us-east-1
|
value: us-east-1
|
||||||
```
|
```
|
||||||
{{% /codetab %}}
|
{{% /codetab %}}
|
||||||
|
@ -45,6 +92,7 @@ apiVersion: apps/v1
|
||||||
kind: Deployment
|
kind: Deployment
|
||||||
metadata:
|
metadata:
|
||||||
name: localstack
|
name: localstack
|
||||||
|
namespace: default
|
||||||
spec:
|
spec:
|
||||||
# using the selector, we will expose the running deployments
|
# using the selector, we will expose the running deployments
|
||||||
# this is how Kubernetes knows, that a given service belongs to a deployment
|
# this is how Kubernetes knows, that a given service belongs to a deployment
|
||||||
|
@ -84,51 +132,15 @@ spec:
|
||||||
|
|
||||||
{{% codetab %}}
|
{{% codetab %}}
|
||||||
In order to run in AWS, you should create an IAM user with permissions to the SNS and SQS services.
|
In order to run in AWS, you should create an IAM user with permissions to the SNS and SQS services.
|
||||||
Use the account ID and account secret and plug them into the `awsAccountID` and `awsAccountSecret`
|
Use the `AWS account ID` and `AWS account secret` and plug them into the `accessKey` and `secretKey` in the component metadata using Kubernetes secrets and `secretKeyRef`.
|
||||||
in the component metadata using kubernetes secrets.
|
|
||||||
{{% /codetab %}}
|
{{% /codetab %}}
|
||||||
|
|
||||||
{{< /tabs >}}
|
{{< /tabs >}}
|
||||||
|
|
||||||
## Create a Dapr component
|
|
||||||
|
|
||||||
The next step is to create a Dapr component for SNS/SQS.
|
|
||||||
|
|
||||||
Create the following YAML file named `snssqs.yaml`:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
apiVersion: dapr.io/v1alpha1
|
|
||||||
kind: Component
|
|
||||||
metadata:
|
|
||||||
name: <NAME>
|
|
||||||
namespace: <NAMESPACE>
|
|
||||||
spec:
|
|
||||||
type: pubsub.snssqs
|
|
||||||
version: v1
|
|
||||||
metadata:
|
|
||||||
# ID of the AWS account with appropriate permissions to SNS and SQS
|
|
||||||
- name: accessKey
|
|
||||||
value: **********
|
|
||||||
# Secret for the AWS user
|
|
||||||
- name: secretKey
|
|
||||||
value: **********
|
|
||||||
# The AWS region you want to operate in.
|
|
||||||
# See this page for valid regions: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.RegionsAndAvailabilityZones.html
|
|
||||||
# Make sure that SNS and SQS are available in that region.
|
|
||||||
- name: region
|
|
||||||
value: us-east-1
|
|
||||||
```
|
|
||||||
|
|
||||||
{{% alert title="Warning" color="warning" %}}
|
|
||||||
The above example uses secrets as plain strings. It is recommended to use a secret store for the secrets as described [here]({{< ref component-secrets.md >}}).
|
|
||||||
{{% /alert %}}
|
|
||||||
|
|
||||||
## Apply the configuration
|
|
||||||
|
|
||||||
Visit [this guide]({{< ref "howto-publish-subscribe.md#step-2-publish-a-topic" >}}) for instructions on configuring pub/sub components.
|
|
||||||
|
|
||||||
## Related links
|
## Related links
|
||||||
|
- [Basic schema for a Dapr component]({{< ref component-schema >}})
|
||||||
- [Pub/Sub building block]({{< ref pubsub >}})
|
- [Pub/Sub building block]({{< ref pubsub >}})
|
||||||
|
- Read [this guide]({{< ref "howto-publish-subscribe.md#step-2-publish-a-topic" >}}) for instructions on configuring pub/sub components
|
||||||
- [AWS SQS as subscriber to SNS](https://docs.aws.amazon.com/sns/latest/dg/sns-sqs-as-subscriber.html)
|
- [AWS SQS as subscriber to SNS](https://docs.aws.amazon.com/sns/latest/dg/sns-sqs-as-subscriber.html)
|
||||||
- [AWS SNS API refernce](https://docs.aws.amazon.com/sns/latest/api/Welcome.html)
|
- [AWS SNS API refernce](https://docs.aws.amazon.com/sns/latest/api/Welcome.html)
|
||||||
- [AWS SQS API refernce](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/Welcome.html)
|
- [AWS SQS API refernce](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/Welcome.html)
|
||||||
|
|
|
@ -36,9 +36,9 @@ The above example uses secrets as plain strings. It is recommended to use a secr
|
||||||
|
|
||||||
| Field | Required | Details | Example |
|
| Field | Required | Details | Example |
|
||||||
|--------------------|:--------:|---------|---------|
|
|--------------------|:--------:|---------|---------|
|
||||||
| connectionString | Y | Connection-string for the Event Hubs | `Endpoint=sb://{EventHubNamespace}.servicebus.windows.net/;SharedAccessKeyName={PolicyName};SharedAccessKey={Key};EntityPath={EventHub}`
|
| connectionString | Y | Connection-string for the Event Hubs | `"Endpoint=sb://{EventHubNamespace}.servicebus.windows.net/;SharedAccessKeyName={PolicyName};SharedAccessKey={Key};EntityPath={EventHub}"`
|
||||||
| storageAccountName | Y | Storage account name to use for the EventProcessorHost |`"myeventhubstorage"`
|
| storageAccountName | Y | Storage account name to use for the EventProcessorHost |`"myeventhubstorage"`
|
||||||
| storageAccountKey | Y | Storage account key to use for the EventProcessorHost | `"112233445566778899"`
|
| storageAccountKey | Y | Storage account key to use for the EventProcessorHost. Can be `secretKeyRef` to use a secret reference | `"112233445566778899"`
|
||||||
| storageContainerName | Y | Storage container name for the storage account name. | `"myeventhubstoragecontainer"`
|
| storageContainerName | Y | Storage container name for the storage account name. | `"myeventhubstoragecontainer"`
|
||||||
|
|
||||||
|
|
||||||
|
@ -56,8 +56,7 @@ For example, a Dapr app running on Kubernetes with `dapr.io/app-id: "myapp"` wil
|
||||||
|
|
||||||
Note: Dapr passes the name of the Consumer group to the EventHub and so this is not supplied in the metadata.
|
Note: Dapr passes the name of the Consumer group to the EventHub and so this is not supplied in the metadata.
|
||||||
|
|
||||||
|
|
||||||
## Related links
|
## Related links
|
||||||
- [Basic schema for a Dapr component]({{< ref component-schema >}})
|
- [Basic schema for a Dapr component]({{< ref component-schema >}})
|
||||||
- Read [this topic]({{< ref "howto-publish-subscribe.md#step-2-publish-a-topic" >}}) for instructions on configuring pub/sub components
|
- Read [this guide]({{< ref "howto-publish-subscribe.md#step-2-publish-a-topic" >}}) for instructions on configuring pub/sub components
|
||||||
- [Pub/Sub building block]({{< ref pubsub >}})
|
- [Pub/Sub building block]({{< ref pubsub >}})
|
||||||
|
|
|
@ -5,52 +5,45 @@ linkTitle: "Azure Service Bus"
|
||||||
description: "Detailed documentation on the Azure Service Bus pubsub component"
|
description: "Detailed documentation on the Azure Service Bus pubsub component"
|
||||||
---
|
---
|
||||||
|
|
||||||
## Setup Azure Service Bus
|
## Component format
|
||||||
|
To setup Azure Event Hubs pubsub create a component of type `pubsub.azure.servicebus`. See [this guide]({{< ref "howto-publish-subscribe.md#step-1-setup-the-pubsub-component" >}}) on how to create and apply a pubsub configuration.
|
||||||
Follow the instructions [here](https://docs.microsoft.com/en-us/azure/service-bus-messaging/service-bus-quickstart-topics-subscriptions-portal) on setting up Azure Service Bus Topics.
|
|
||||||
|
|
||||||
## Create a Dapr component
|
|
||||||
|
|
||||||
The next step is to create a Dapr component for Azure Service Bus.
|
|
||||||
|
|
||||||
Create the following YAML file named `azuresb.yaml`:
|
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
apiVersion: dapr.io/v1alpha1
|
apiVersion: dapr.io/v1alpha1
|
||||||
kind: Component
|
kind: Component
|
||||||
metadata:
|
metadata:
|
||||||
name: <NAME>
|
name: servicebus-pubsub
|
||||||
namespace: <NAMESPACE>
|
namespace: default
|
||||||
spec:
|
spec:
|
||||||
type: pubsub.azure.servicebus
|
type: pubsub.azure.servicebus
|
||||||
version: v1
|
version: v1
|
||||||
metadata:
|
metadata:
|
||||||
- name: connectionString
|
- name: connectionString # Required
|
||||||
value: <REPLACE-WITH-CONNECTION-STRING> # Required.
|
value: "Endpoint=sb://{ServiceBusNamespace}.servicebus.windows.net/;SharedAccessKeyName={PolicyName};SharedAccessKey={Key};EntityPath={ServiceBus}"
|
||||||
- name: timeoutInSec
|
- name: timeoutInSec # Optional
|
||||||
value: <REPLACE-WITH-TIMEOUT-IN-SEC> # Optional. Default: "60". Timeout for sending messages and management operations.
|
value: 60
|
||||||
- name: handlerTimeoutInSec
|
- name: handlerTimeoutInSec # Optional
|
||||||
value: <REPLACE-WITH-HANDLER-TIMEOUT-IN-SEC> # Optional. Default: "60". Timeout for invoking app handler.
|
value: 60
|
||||||
- name: disableEntityManagement
|
- name: disableEntityManagement # Optional
|
||||||
value: <REPLACE-WITH-DISABLE-ENTITY-MANAGEMENT> # Optional. Default: false. When set to true, topics and subscriptions do not get created automatically.
|
value: "false"
|
||||||
- name: maxDeliveryCount
|
- name: maxDeliveryCount # Optional
|
||||||
value: <REPLACE-WITH-MAX-DELIVERY-COUNT> # Optional. Defines the number of attempts the server will make to deliver a message.
|
value: 3
|
||||||
- name: lockDurationInSec
|
- name: lockDurationInSec # Optional
|
||||||
value: <REPLACE-WITH-LOCK-DURATION-IN-SEC> # Optional. Defines the length in seconds that a message will be locked for before expiring.
|
value: 60
|
||||||
- name: lockRenewalInSec
|
- name: lockRenewalInSec # Optional
|
||||||
value: <REPLACE-WITH-LOCK-RENEWAL-IN-SEC> # Optional. Default: "20". Defines the frequency at which buffered message locks will be renewed.
|
value: 20
|
||||||
- name: maxActiveMessages
|
- name: maxActiveMessages # Optional
|
||||||
value: <REPLACE-WITH-MAX-ACTIVE-MESSAGES> # Optional. Default: "10000". Defines the maximum number of messages to be buffered or processing at once.
|
value: 2000
|
||||||
- name: maxActiveMessagesRecoveryInSec
|
- name: maxActiveMessagesRecoveryInSec # Optional
|
||||||
value: <REPLACE-WITH-MAX-ACTIVE-MESSAGES-RECOVERY-IN-SEC> # Optional. Default: "2". Defines the number of seconds to wait once the maximum active message limit is reached.
|
value: 2
|
||||||
- name: maxConcurrentHandlers
|
- name: maxConcurrentHandlers # Optional
|
||||||
value: <REPLACE-WITH-MAX-CONCURRENT-HANDLERS> # Optional. Defines the maximum number of concurrent message handlers
|
value: 10
|
||||||
- name: prefetchCount
|
- name: prefetchCount # Optional
|
||||||
value: <REPLACE-WITH-PREFETCH-COUNT> # Optional. Defines the number of prefetched messages (use for high throughput / low latency scenarios)
|
value: 5
|
||||||
- name: defaultMessageTimeToLiveInSec
|
- name: defaultMessageTimeToLiveInSec # Optional
|
||||||
value: <REPLACE-WITH-MESSAGE-TIME-TO-LIVE-IN-SEC> # Optional.
|
value: 10
|
||||||
- name: autoDeleteOnIdleInSec
|
- name: autoDeleteOnIdleInSec # Optional
|
||||||
value: <REPLACE-WITH-AUTO-DELETE-ON-IDLE-IN-SEC> # Optional.
|
value: 10
|
||||||
```
|
```
|
||||||
|
|
||||||
> __NOTE:__ The above settings are shared across all topics that use this component.
|
> __NOTE:__ The above settings are shared across all topics that use this component.
|
||||||
|
@ -59,9 +52,29 @@ spec:
|
||||||
The above example uses secrets as plain strings. It is recommended to use a secret store for the secrets as described [here]({{< ref component-secrets.md >}}).
|
The above example uses secrets as plain strings. It is recommended to use a secret store for the secrets as described [here]({{< ref component-secrets.md >}}).
|
||||||
{{% /alert %}}
|
{{% /alert %}}
|
||||||
|
|
||||||
## Apply the configuration
|
## Spec metadata fields
|
||||||
|
|
||||||
Visit [this guide]({{< ref "howto-publish-subscribe.md#step-2-publish-a-topic" >}}) for instructions on configuring pub/sub components.
|
| Field | Required | Details | Example |
|
||||||
|
|--------------------|:--------:|---------|---------|
|
||||||
|
| connectionString | Y | Connection-string for the Event Hubs | "`Endpoint=sb://{ServiceBusNamespace}.servicebus.windows.net/;SharedAccessKeyName={PolicyName};SharedAccessKey={Key};EntityPath={ServiceBus}`"
|
||||||
|
| timeoutInSec | N | Timeout for sending messages and management operations. Default: `60` |`30`
|
||||||
|
| handlerTimeoutInSec| N | Timeout for invoking app handler. # Optional. Default: `60` | `30`
|
||||||
|
| disableEntityManagement | N | When set to true, topics and subscriptions do not get created automatically. Default: `"false"` | `"true"`, `"false"`
|
||||||
|
| maxDeliveryCount | N |Defines the number of attempts the server will make to deliver a message. Default set by server| `10`
|
||||||
|
| lockDurationInSec | N |Defines the length in seconds that a message will be locked for before expiring. Default set by server | `30`
|
||||||
|
| lockRenewalInSec | N |Defines the frequency at which buffered message locks will be renewed. Default: `20`. | `20`
|
||||||
|
| maxActiveMessages | N |Defines the maximum number of messages to be buffered or processing at once. Default: `10000` | `2000`
|
||||||
|
| maxActiveMessagesRecoveryInSec | N |Defines the number of seconds to wait once the maximum active message limit is reached. Default: `2` | `10`
|
||||||
|
| maxConcurrentHandlers | N |Defines the maximum number of concurrent message handlers | `10`
|
||||||
|
| prefetchCount | N |Defines the number of prefetched messages (use for high throughput / low latency scenarios)| `5`
|
||||||
|
| defaultMessageTimeToLiveInSec | N |Default message time to live. | `10`
|
||||||
|
| autoDeleteOnIdleInSec | N |Time in seconds to wait before auto deleting messages. | `10`
|
||||||
|
|
||||||
|
## Create an Azure Service Bus
|
||||||
|
|
||||||
|
Follow the instructions [here](https://docs.microsoft.com/en-us/azure/service-bus-messaging/service-bus-quickstart-topics-subscriptions-portal) on setting up Azure Service Bus Topics.
|
||||||
|
|
||||||
## Related links
|
## Related links
|
||||||
|
- [Basic schema for a Dapr component]({{< ref component-schema >}})
|
||||||
- [Pub/Sub building block]({{< ref pubsub >}})
|
- [Pub/Sub building block]({{< ref pubsub >}})
|
||||||
|
- Read [this guide]({{< ref "howto-publish-subscribe.md#step-2-publish-a-topic" >}}) for instructions on configuring pub/sub components
|
||||||
|
|
|
@ -5,28 +5,20 @@ linkTitle: "GCP Pub/Sub"
|
||||||
description: "Detailed documentation on the GCP Pub/Sub component"
|
description: "Detailed documentation on the GCP Pub/Sub component"
|
||||||
---
|
---
|
||||||
|
|
||||||
## Setup GCP Pub/Sub
|
|
||||||
|
|
||||||
Follow the instructions [here](https://cloud.google.com/pubsub/docs/quickstart-console) on setting up Google Cloud Pub/Sub system.
|
|
||||||
|
|
||||||
## Create a Dapr component
|
## Create a Dapr component
|
||||||
|
|
||||||
The next step is to create a Dapr component for Google Cloud Pub/Sub
|
To setup GCP pubsub create a component of type `pubsub.gcp.pubsub`. See [this guide]({{< ref "howto-publish-subscribe.md#step-1-setup-the-pubsub-component" >}}) on how to create and apply a pubsub configuration
|
||||||
|
|
||||||
Create the following YAML file named `messagebus.yaml`:
|
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
apiVersion: dapr.io/v1alpha1
|
apiVersion: dapr.io/v1alpha1
|
||||||
kind: Component
|
kind: Component
|
||||||
metadata:
|
metadata:
|
||||||
name: <NAME>
|
name: gcp-pubsub
|
||||||
namespace: <NAMESPACE>
|
namespace: default
|
||||||
spec:
|
spec:
|
||||||
type: pubsub.gcp.pubsub
|
type: pubsub.gcp.pubsub
|
||||||
version: v1
|
version: v1
|
||||||
metadata:
|
metadata:
|
||||||
- name: topic
|
|
||||||
value: <TOPIC_NAME>
|
|
||||||
- name: type
|
- name: type
|
||||||
value: service_account
|
value: service_account
|
||||||
- name: project_id
|
- name: project_id
|
||||||
|
@ -44,33 +36,38 @@ spec:
|
||||||
- name: auth_provider_x509_cert_url
|
- name: auth_provider_x509_cert_url
|
||||||
value: https://www.googleapis.com/oauth2/v1/certs
|
value: https://www.googleapis.com/oauth2/v1/certs
|
||||||
- name: client_x509_cert_url
|
- name: client_x509_cert_url
|
||||||
value: https://www.googleapis.com/robot/v1/metadata/x509/<PROJECT_NAME>.iam.gserviceaccount.com
|
value: https://www.googleapis.com/robot/v1/metadata/x509/<PROJECT_NAME>.iam.gserviceaccount.com #replace PROJECT_NAME
|
||||||
- name: private_key
|
- name: private_key
|
||||||
value: <PRIVATE_KEY> # replace x509 cert here
|
value: <PRIVATE_KEY> # replace x509 cert
|
||||||
- name: disableEntityManagement
|
- name: disableEntityManagement
|
||||||
value: <REPLACE-WITH-DISABLE-ENTITY-MANAGEMENT> # Optional. Default: false. When set to true, topics and subscriptions do not get created automatically.
|
value: "false"
|
||||||
```
|
```
|
||||||
|
|
||||||
- `topic` is the Pub/Sub topic name.
|
## Spec metadata fields
|
||||||
- `type` is the GCP credentials type.
|
|
||||||
- `project_id` is the GCP project id.
|
| Field | Required | Details | Example |
|
||||||
- `private_key_id` is the GCP private key id.
|
|--------------------|:--------:|---------|---------|
|
||||||
- `client_email` is the GCP client email.
|
| type | Y | GCP credentials type | `service_account`
|
||||||
- `client_id` is the GCP client id.
|
| project_id | Y | GCP project id| `projectId`
|
||||||
- `auth_uri` is Google account OAuth endpoint.
|
| private_key_id | Y | GCP private key id | `"privateKeyId"`
|
||||||
- `token_uri` is Google account token uri.
|
| private_key | Y | GCP credentials private key. Replace with x509 cert | `12345-12345`
|
||||||
- `auth_provider_x509_cert_url` is the GCP credentials cert url.
|
| client_email | Y | GCP client email | `"client@email.com"`
|
||||||
- `client_x509_cert_url` is the GCP credentials project x509 cert url.
|
| client_id | Y | GCP client id | `0123456789-0123456789`
|
||||||
- `private_key` is the GCP credentials private key.
|
| auth_uri | Y | Google account OAuth endpoint | `https://accounts.google.com/o/oauth2/auth`
|
||||||
- `disableEntityManagement` Optional. Default: false. When set to true, topics and subscriptions do not get created automatically.
|
| token_uri | Y | Google account token uri | `https://oauth2.googleapis.com/token`
|
||||||
|
| auth_provider_x509_cert_url | Y | GCP credentials cert url | `https://www.googleapis.com/oauth2/v1/certs`
|
||||||
|
| client_x509_cert_url | Y | GCP credentials project x509 cert url | `https://www.googleapis.com/robot/v1/metadata/x509/<PROJECT_NAME>.iam.gserviceaccount.com`
|
||||||
|
| disableEntityManagement | N | When set to `"true"`, topics and subscriptions do not get created automatically. Default: `"false"` | `"true"`, `"false"`
|
||||||
|
|
||||||
{{% alert title="Warning" color="warning" %}}
|
{{% alert title="Warning" color="warning" %}}
|
||||||
The above example uses secrets as plain strings. It is recommended to use a secret store for the secrets as described [here]({{< ref component-secrets.md >}}).
|
The above example uses secrets as plain strings. It is recommended to use a secret store for the secrets as described [here]({{< ref component-secrets.md >}}).
|
||||||
{{% /alert %}}
|
{{% /alert %}}
|
||||||
|
|
||||||
## Apply the configuration
|
## Create a GCP Pub/Sub
|
||||||
|
|
||||||
Visit [this guide]({{< ref "howto-publish-subscribe.md#step-2-publish-a-topic" >}}) for instructions on configuring pub/sub components.
|
Follow the instructions [here](https://cloud.google.com/pubsub/docs/quickstart-console) on setting up Google Cloud Pub/Sub system.
|
||||||
|
|
||||||
## Related links
|
## Related links
|
||||||
|
- [Basic schema for a Dapr component]({{< ref component-schema >}})
|
||||||
|
- Read [this guide]({{< ref "howto-publish-subscribe.md#step-2-publish-a-topic" >}}) for instructions on configuring pub/sub components
|
||||||
- [Pub/Sub building block]({{< ref pubsub >}})
|
- [Pub/Sub building block]({{< ref pubsub >}})
|
|
@ -5,7 +5,35 @@ linkTitle: "Hazelcast"
|
||||||
description: "Detailed documentation on the Hazelcast pubsub component"
|
description: "Detailed documentation on the Hazelcast pubsub component"
|
||||||
---
|
---
|
||||||
|
|
||||||
## Setup Hazelcast
|
## Component format
|
||||||
|
To setup hazelcast pubsub create a component of type `pubsub.hazelcast`. See [this guide]({{< ref "howto-publish-subscribe.md#step-1-setup-the-pubsub-component" >}}) on how to create and apply a pubsub configuration.
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
apiVersion: dapr.io/v1alpha1
|
||||||
|
kind: Component
|
||||||
|
metadata:
|
||||||
|
name: hazelcast-pubsub
|
||||||
|
namespace: default
|
||||||
|
spec:
|
||||||
|
type: pubsub.hazelcast
|
||||||
|
version: v1
|
||||||
|
metadata:
|
||||||
|
- name: hazelcastServers
|
||||||
|
value: "hazelcast:3000,hazelcast2:3000"
|
||||||
|
```
|
||||||
|
|
||||||
|
{{% alert title="Warning" color="warning" %}}
|
||||||
|
The above example uses secrets as plain strings. It is recommended to use a secret store for the secrets as described [here]({{< ref component-secrets.md >}}).
|
||||||
|
{{% /alert %}}
|
||||||
|
|
||||||
|
## Spec metadata fields
|
||||||
|
|
||||||
|
| Field | Required | Details | Example |
|
||||||
|
|--------------------|:--------:|---------|---------|
|
||||||
|
| connectionString | Y | A comma delimited string of servers. Example: "hazelcast:3000,hazelcast2:3000" | `"hazelcast:3000,hazelcast2:3000"`
|
||||||
|
|
||||||
|
|
||||||
|
## Create a Hazelcast instance
|
||||||
|
|
||||||
{{< tabs "Self-Hosted" "Kubernetes">}}
|
{{< tabs "Self-Hosted" "Kubernetes">}}
|
||||||
|
|
||||||
|
@ -25,33 +53,7 @@ The easiest way to install Hazelcast on Kubernetes is by using the [Helm chart](
|
||||||
|
|
||||||
{{< /tabs >}}
|
{{< /tabs >}}
|
||||||
|
|
||||||
## Create a Dapr component
|
|
||||||
|
|
||||||
The next step is to create a Dapr component for Hazelcast.
|
|
||||||
|
|
||||||
Create the following YAML file named `hazelcast.yaml`:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
apiVersion: dapr.io/v1alpha1
|
|
||||||
kind: Component
|
|
||||||
metadata:
|
|
||||||
name: <NAME>
|
|
||||||
namespace: <NAMESPACE>
|
|
||||||
spec:
|
|
||||||
type: pubsub.hazelcast
|
|
||||||
version: v1
|
|
||||||
metadata:
|
|
||||||
- name: hazelcastServers
|
|
||||||
value: <REPLACE-WITH-HOSTS> # Required. A comma delimited string of servers. Example: "hazelcast:3000,hazelcast2:3000"
|
|
||||||
```
|
|
||||||
|
|
||||||
{{% alert title="Warning" color="warning" %}}
|
|
||||||
The above example uses secrets as plain strings. It is recommended to use a secret store for the secrets as described [here]({{< ref component-secrets.md >}}).
|
|
||||||
{{% /alert %}}
|
|
||||||
|
|
||||||
## Apply the configuration
|
|
||||||
|
|
||||||
Visit [this guide]({{< ref "howto-publish-subscribe.md#step-2-publish-a-topic" >}}) for instructions on configuring pub/sub components.
|
|
||||||
|
|
||||||
## Related links
|
## Related links
|
||||||
|
- [Basic schema for a Dapr component]({{< ref component-schema >}})
|
||||||
|
- Read [this guide]({{< ref "howto-publish-subscribe.md#step-2-publish-a-topic" >}}) for instructions on configuring pub/sub components
|
||||||
- [Pub/Sub building block]({{< ref pubsub >}})
|
- [Pub/Sub building block]({{< ref pubsub >}})
|
|
@ -5,7 +5,78 @@ linkTitle: "MQTT"
|
||||||
description: "Detailed documentation on the MQTT pubsub component"
|
description: "Detailed documentation on the MQTT pubsub component"
|
||||||
---
|
---
|
||||||
|
|
||||||
## Setup MQTT
|
## Component format
|
||||||
|
|
||||||
|
To setup MQTT pubsub create a component of type `pubsub.mqtt`. See [this guide]({{< ref "howto-publish-subscribe.md#step-1-setup-the-pubsub-component" >}}) on how to create and apply a pubsub configuration
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
apiVersion: dapr.io/v1alpha1
|
||||||
|
kind: Component
|
||||||
|
metadata:
|
||||||
|
name: mqtt-pubsub
|
||||||
|
namespace: default
|
||||||
|
spec:
|
||||||
|
type: pubsub.mqtt
|
||||||
|
version: v1
|
||||||
|
metadata:
|
||||||
|
- name: url
|
||||||
|
value: "tcp://[username][:password]@host.domain[:port]"
|
||||||
|
- name: qos
|
||||||
|
value: 1
|
||||||
|
- name: retain
|
||||||
|
value: "false"
|
||||||
|
- name: cleanSession
|
||||||
|
value: "false"
|
||||||
|
```
|
||||||
|
## Spec metadata fields
|
||||||
|
|
||||||
|
| Field | Required | Details | Example |
|
||||||
|
|--------------------|:--------:|---------|---------|
|
||||||
|
| url | Y | Address of the MQTT broker | Use `**tcp://**` scheme for non-TLS communication. Use`**tcps://**` scheme for TLS communication. <br> "tcp://[username][:password]@host.domain[:port]"
|
||||||
|
| qos | N | Indicates the Quality of Service Level (QoS) of the message. Default 0|`1`
|
||||||
|
| retain | N | Defines whether the message is saved by the broker as the last known good value for a specified topic. Default `"false"` | `"true"`, `"false"`
|
||||||
|
| cleanSession | N | will set the "clean session" in the connect message when client connects to an MQTT broker. Default `"true"` | `"true"`, `"false"`
|
||||||
|
| caCert | Required for using TLS | Certificate authority certificate. Can be `secretKeyRef` to use a secret reference | `0123456789-0123456789`
|
||||||
|
| clientCert | Required for using TLS | Client certificate. Can be `secretKeyRef` to use a secret reference | `0123456789-0123456789`
|
||||||
|
| clientKey | Required for using TLS | Client key. Can be `secretKeyRef` to use a secret reference | `012345`
|
||||||
|
|
||||||
|
|
||||||
|
### Communication using TLS
|
||||||
|
To configure communication using TLS, ensure mosquitto broker is configured to support certificates.
|
||||||
|
Pre-requisite includes `certficate authority certificate`, `ca issued client certificate`, `client private key`.
|
||||||
|
Here is an example.
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
apiVersion: dapr.io/v1alpha1
|
||||||
|
kind: Component
|
||||||
|
metadata:
|
||||||
|
name: mqtt-pubsub
|
||||||
|
namespace: default
|
||||||
|
spec:
|
||||||
|
type: pubsub.mqtt
|
||||||
|
metadata:
|
||||||
|
- name: url
|
||||||
|
value: "tcps://host.domain[:port]"
|
||||||
|
- name: qos
|
||||||
|
value: 1
|
||||||
|
- name: retain
|
||||||
|
value: "false"
|
||||||
|
- name: cleanSession
|
||||||
|
value: "false"
|
||||||
|
- name: caCert
|
||||||
|
value: ''
|
||||||
|
- name: clientCert
|
||||||
|
value: ''
|
||||||
|
- name: clientKey
|
||||||
|
value: ''
|
||||||
|
```
|
||||||
|
|
||||||
|
{{% alert title="Warning" color="warning" %}}
|
||||||
|
The above example uses secrets as plain strings. It is recommended to use a secret store for the secrets as described [here]({{< ref component-secrets.md >}}).
|
||||||
|
{{% /alert %}}
|
||||||
|
|
||||||
|
|
||||||
|
## Create a MQTT broker
|
||||||
|
|
||||||
{{< tabs "Self-Hosted" "Kubernetes">}}
|
{{< tabs "Self-Hosted" "Kubernetes">}}
|
||||||
|
|
||||||
|
@ -75,68 +146,7 @@ You can then interact with the server using the client port: `tcp://mqtt-broker.
|
||||||
|
|
||||||
{{< /tabs >}}
|
{{< /tabs >}}
|
||||||
|
|
||||||
## Create a Dapr component
|
|
||||||
|
|
||||||
The next step is to create a Dapr component for MQTT.
|
|
||||||
|
|
||||||
Create the following yaml file named `mqtt.yaml`
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
apiVersion: dapr.io/v1alpha1
|
|
||||||
kind: Component
|
|
||||||
metadata:
|
|
||||||
name: <NAME>
|
|
||||||
namespace: <NAMESPACE>
|
|
||||||
spec:
|
|
||||||
type: pubsub.mqtt
|
|
||||||
version: v1
|
|
||||||
metadata:
|
|
||||||
- name: url
|
|
||||||
value: "tcp://[username][:password]@host.domain[:port]"
|
|
||||||
- name: qos
|
|
||||||
value: 1
|
|
||||||
- name: retain
|
|
||||||
value: "false"
|
|
||||||
- name: cleanSession
|
|
||||||
value: "false"
|
|
||||||
```
|
|
||||||
|
|
||||||
To configure communication using TLS, ensure mosquitto broker is configured to support certificates.
|
|
||||||
Pre-requisite includes `certficate authority certificate`, `ca issued client certificate`, `client private key`.
|
|
||||||
Make following additional changes to mqtt pubsub components for supporting TLS.
|
|
||||||
```yaml
|
|
||||||
...
|
|
||||||
spec:
|
|
||||||
type: pubsub.mqtt
|
|
||||||
metadata:
|
|
||||||
- name: url
|
|
||||||
value: "tcps://host.domain[:port]"
|
|
||||||
- name: caCert
|
|
||||||
value: ''
|
|
||||||
- name: clientCert
|
|
||||||
value: ''
|
|
||||||
- name: clientKey
|
|
||||||
value: ''
|
|
||||||
```
|
|
||||||
|
|
||||||
Where:
|
|
||||||
- **url** (required) is the address of the MQTT broker.
|
|
||||||
- - use **tcp://** scheme for non-TLS communication.
|
|
||||||
- - use **tcps://** scheme for TLS communication.
|
|
||||||
- **qos** (optional) indicates the Quality of Service Level (QoS) of the message. (Default 0)
|
|
||||||
- **retain** (optional) defines whether the message is saved by the broker as the last known good value for a specified topic. (Default false)
|
|
||||||
- **cleanSession** (optional) will set the "clean session" in the connect message when client connects to an MQTT broker . (Default true)
|
|
||||||
- **caCert** (required for using TLS) is the certificate authority certificate.
|
|
||||||
- **clientCert** (required for using TLS) is the client certificate.
|
|
||||||
- **clientKey** (required for using TLS) is the client key.
|
|
||||||
|
|
||||||
{{% alert title="Warning" color="warning" %}}
|
|
||||||
The above example uses secrets as plain strings. It is recommended to use a secret store for the secrets as described [here]({{< ref component-secrets.md >}}).
|
|
||||||
{{% /alert %}}
|
|
||||||
|
|
||||||
## Apply the configuration
|
|
||||||
|
|
||||||
Visit [this guide]({{< ref "howto-publish-subscribe.md#step-2-publish-a-topic" >}}) for instructions on configuring pub/sub components.
|
|
||||||
|
|
||||||
## Related links
|
## Related links
|
||||||
|
- [Basic schema for a Dapr component]({{< ref component-schema >}})
|
||||||
|
- Read [this guide]({{< ref "howto-publish-subscribe.md#step-2-publish-a-topic" >}}) for instructions on configuring pub/sub components
|
||||||
- [Pub/Sub building block]({{< ref pubsub >}})
|
- [Pub/Sub building block]({{< ref pubsub >}})
|
|
@ -5,7 +5,61 @@ linkTitle: "NATS streaming"
|
||||||
description: "Detailed documentation on the NATS pubsub component"
|
description: "Detailed documentation on the NATS pubsub component"
|
||||||
---
|
---
|
||||||
|
|
||||||
## Setup NATS
|
## Component format
|
||||||
|
To setup NATS streaming pubsub create a component of type `pubsub.natsstreaming`. See [this guide]({{< ref "howto-publish-subscribe.md#step-1-setup-the-pubsub-component" >}}) on how to create and apply a pubsub configuration.
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
apiVersion: dapr.io/v1alpha1
|
||||||
|
kind: Component
|
||||||
|
metadata:
|
||||||
|
name: natsstreaming-pubsub
|
||||||
|
namespace: default
|
||||||
|
spec:
|
||||||
|
type: pubsub.natsstreaming
|
||||||
|
version: v1
|
||||||
|
metadata:
|
||||||
|
- name: natsURL
|
||||||
|
value: "nats://localhost:4222"
|
||||||
|
- name: natsStreamingClusterID
|
||||||
|
value: "clusterId"
|
||||||
|
# below are subscription configuration.
|
||||||
|
- name: subscriptionType
|
||||||
|
value: <REPLACE-WITH-SUBSCRIPTION-TYPE> # Required. Allowed values: topic, queue.
|
||||||
|
- name: deliverNew
|
||||||
|
value: true
|
||||||
|
# - name: ackWaitTime
|
||||||
|
# value: "" # Optional. See: https://docs.nats.io/developing-with-nats-streaming/acks#acknowledgements
|
||||||
|
# - name: maxInFlight
|
||||||
|
# value: "" # Optional. See: https://docs.nats.io/developing-with-nats-streaming/acks#acknowledgements
|
||||||
|
# - name: durableSubscriptionName
|
||||||
|
# value: ""
|
||||||
|
# following subscription options - only one can be used
|
||||||
|
# - name: startAtSequence
|
||||||
|
# value: 1
|
||||||
|
# - name: startWithLastReceived
|
||||||
|
# value: false
|
||||||
|
# - name: deliverAll
|
||||||
|
# value: false
|
||||||
|
# - name: startAtTimeDelta
|
||||||
|
# value: ""
|
||||||
|
# - name: startAtTime
|
||||||
|
# value: ""
|
||||||
|
# - name: startAtTimeFormat
|
||||||
|
# value: ""
|
||||||
|
```
|
||||||
|
|
||||||
|
{{% alert title="Warning" color="warning" %}}
|
||||||
|
The above example uses secrets as plain strings. It is recommended to use a secret store for the secrets as described [here]({{< ref component-secrets.md >}}).
|
||||||
|
{{% /alert %}}
|
||||||
|
|
||||||
|
## Spec metadata fields
|
||||||
|
|
||||||
|
| Field | Required | Details | Example |
|
||||||
|
|--------------------|:--------:|---------|---------|
|
||||||
|
| natsURL | Y | NATS server address URL | "`nats://localhost:4222`"
|
||||||
|
| natsStreamingClusterID | Y | NATS cluster ID |`"clusterId"`
|
||||||
|
|
||||||
|
## Create a NATS server
|
||||||
|
|
||||||
{{< tabs "Self-Hosted" "Kubernetes">}}
|
{{< tabs "Self-Hosted" "Kubernetes">}}
|
||||||
|
|
||||||
|
@ -40,59 +94,7 @@ For example, if installing using the example above, the NATS Streaming address w
|
||||||
|
|
||||||
{{< /tabs >}}
|
{{< /tabs >}}
|
||||||
|
|
||||||
## Create a Dapr component
|
|
||||||
|
|
||||||
The next step is to create a Dapr component for NATS-Streaming.
|
|
||||||
|
|
||||||
Create the following YAML file named `nats-stan.yaml`:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
apiVersion: dapr.io/v1alpha1
|
|
||||||
kind: Component
|
|
||||||
metadata:
|
|
||||||
name: <NAME>
|
|
||||||
namespace: <NAMESPACE>
|
|
||||||
spec:
|
|
||||||
type: pubsub.natsstreaming
|
|
||||||
version: v1
|
|
||||||
metadata:
|
|
||||||
- name: natsURL
|
|
||||||
value: <REPLACE-WITH-NATS-SERVER-ADDRESS> # Required. example nats://localhost:4222
|
|
||||||
- name: natsStreamingClusterID
|
|
||||||
value: <REPLACE-WITH-NATS-CLUSTERID> # Required.
|
|
||||||
# blow are subscription configuration.
|
|
||||||
- name: subscriptionType
|
|
||||||
value: <REPLACE-WITH-SUBSCRIPTION-TYPE> # Required. Allowed values: topic, queue.
|
|
||||||
- name: deliverNew
|
|
||||||
value: true
|
|
||||||
# - name: ackWaitTime
|
|
||||||
# value: "" # Optional. See: https://docs.nats.io/developing-with-nats-streaming/acks#acknowledgements
|
|
||||||
# - name: maxInFlight
|
|
||||||
# value: "" # Optional. See: https://docs.nats.io/developing-with-nats-streaming/acks#acknowledgements
|
|
||||||
# - name: durableSubscriptionName
|
|
||||||
# value: ""
|
|
||||||
# following subscription options - only one can be used
|
|
||||||
# - name: startAtSequence
|
|
||||||
# value: 1
|
|
||||||
# - name: startWithLastReceived
|
|
||||||
# value: false
|
|
||||||
# - name: deliverAll
|
|
||||||
# value: false
|
|
||||||
# - name: startAtTimeDelta
|
|
||||||
# value: ""
|
|
||||||
# - name: startAtTime
|
|
||||||
# value: ""
|
|
||||||
# - name: startAtTimeFormat
|
|
||||||
# value: ""
|
|
||||||
```
|
|
||||||
|
|
||||||
{{% alert title="Warning" color="warning" %}}
|
|
||||||
The above example uses secrets as plain strings. It is recommended to use a secret store for the secrets as described [here]({{< ref component-secrets.md >}}).
|
|
||||||
{{% /alert %}}
|
|
||||||
|
|
||||||
## Apply the configuration
|
|
||||||
|
|
||||||
Visit [this guide]({{< ref "howto-publish-subscribe.md#step-2-publish-a-topic" >}}) for instructions on configuring pub/sub components.
|
|
||||||
|
|
||||||
## Related links
|
## Related links
|
||||||
|
- [Basic schema for a Dapr component]({{< ref component-schema >}})
|
||||||
|
- Read [this guide]({{< ref "howto-publish-subscribe.md#step-2-publish-a-topic" >}}) for instructions on configuring pub/sub components
|
||||||
- [Pub/Sub building block]({{< ref pubsub >}})
|
- [Pub/Sub building block]({{< ref pubsub >}})
|
||||||
|
|
|
@ -5,7 +5,34 @@ linkTitle: "Pulsar"
|
||||||
description: "Detailed documentation on the Pulsar pubsub component"
|
description: "Detailed documentation on the Pulsar pubsub component"
|
||||||
---
|
---
|
||||||
|
|
||||||
## Setup Pulsar
|
## Component format
|
||||||
|
To setup Pulsar pubsub create a component of type `pubsub.pulsar`. See [this guide]({{< ref "howto-publish-subscribe.md#step-1-setup-the-pubsub-component" >}}) on how to create and apply a pubsub configuration.
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
apiVersion: dapr.io/v1alpha1
|
||||||
|
kind: Component
|
||||||
|
metadata:
|
||||||
|
name: pulsar-pubsub
|
||||||
|
namespace: default
|
||||||
|
spec:
|
||||||
|
type: pubsub.pulsar
|
||||||
|
version: v1
|
||||||
|
metadata:
|
||||||
|
- name: host
|
||||||
|
value: "localhost:6650"
|
||||||
|
- name: enableTLS
|
||||||
|
value: "false"
|
||||||
|
|
||||||
|
```
|
||||||
|
## Spec metadata fields
|
||||||
|
|
||||||
|
| Field | Required | Details | Example |
|
||||||
|
|--------------------|:--------:|---------|---------|
|
||||||
|
| host | Y | Address of the Pulsar broker. Default is `"localhost:6650"` | `"localhost:6650"`
|
||||||
|
| enableTLS | Y | Enable TLS. Default: `"false"` | `"true"`, `"false"`
|
||||||
|
|
||||||
|
|
||||||
|
## Create a Pulsar instance
|
||||||
|
|
||||||
{{< tabs "Self-Hosted" "Kubernetes">}}
|
{{< tabs "Self-Hosted" "Kubernetes">}}
|
||||||
|
|
||||||
|
@ -23,37 +50,12 @@ docker run -it \
|
||||||
{{% /codetab %}}
|
{{% /codetab %}}
|
||||||
|
|
||||||
{{% codetab %}}
|
{{% codetab %}}
|
||||||
Please refer to the following [Helm chart](https://pulsar.apache.org/docs/en/kubernetes-helm/) Documentation.
|
Refer to the following [Helm chart](https://pulsar.apache.org/docs/en/kubernetes-helm/) Documentation.
|
||||||
{{% /codetab %}}
|
{{% /codetab %}}
|
||||||
|
|
||||||
{{< /tabs >}}
|
{{< /tabs >}}
|
||||||
|
|
||||||
## Create a Dapr component
|
|
||||||
|
|
||||||
The next step is to create a Dapr component for Pulsar.
|
|
||||||
|
|
||||||
Create the following YAML file named pulsar.yaml:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
apiVersion: dapr.io/v1alpha1
|
|
||||||
kind: Component
|
|
||||||
metadata:
|
|
||||||
name: <NAME>
|
|
||||||
namespace: <NAMESPACE>
|
|
||||||
spec:
|
|
||||||
type: pubsub.pulsar
|
|
||||||
version: v1
|
|
||||||
metadata:
|
|
||||||
- name: host
|
|
||||||
value: <REPLACE WITH PULSAR URL> #default is localhost:6650
|
|
||||||
- name: enableTLS
|
|
||||||
value: <TRUE/FALSE>
|
|
||||||
|
|
||||||
```
|
|
||||||
|
|
||||||
## Apply the configuration
|
|
||||||
|
|
||||||
Visit [this guide]({{< ref "howto-publish-subscribe.md#step-2-publish-a-topic" >}}) for instructions on configuring pub/sub components.
|
|
||||||
|
|
||||||
## Related links
|
## Related links
|
||||||
|
- [Basic schema for a Dapr component]({{< ref component-schema >}})
|
||||||
|
- Read [this guide]({{< ref "howto-publish-subscribe.md#step-2-publish-a-topic" >}}) for instructions on configuring pub/sub components
|
||||||
- [Pub/Sub building block]({{< ref pubsub >}})
|
- [Pub/Sub building block]({{< ref pubsub >}})
|
|
@ -5,7 +5,57 @@ linkTitle: "RabbitMQ"
|
||||||
description: "Detailed documentation on the RabbitMQ pubsub component"
|
description: "Detailed documentation on the RabbitMQ pubsub component"
|
||||||
---
|
---
|
||||||
|
|
||||||
## Setup RabbitMQ
|
## Component format
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
apiVersion: dapr.io/v1alpha1
|
||||||
|
kind: Component
|
||||||
|
metadata:
|
||||||
|
name: rabbitmq-pubsub
|
||||||
|
namespace: default
|
||||||
|
spec:
|
||||||
|
type: pubsub.rabbitmq
|
||||||
|
version: v1
|
||||||
|
metadata:
|
||||||
|
- name: host
|
||||||
|
value: "amqp://localhost:5672"
|
||||||
|
- name: durable
|
||||||
|
value: "false"
|
||||||
|
- name: deletedWhenUnused
|
||||||
|
value: "false"
|
||||||
|
- name: autoAck
|
||||||
|
value: "false"
|
||||||
|
- name: deliveryMode
|
||||||
|
value: "0"
|
||||||
|
- name: requeueInFailure
|
||||||
|
value: "false"
|
||||||
|
- name: prefetchCount
|
||||||
|
value: "0"
|
||||||
|
- name: reconnectWait
|
||||||
|
value: "0"
|
||||||
|
- name: concurrency
|
||||||
|
value: parallel
|
||||||
|
```
|
||||||
|
{{% alert title="Warning" color="warning" %}}
|
||||||
|
The above example uses secrets as plain strings. It is recommended to use a secret store for the secrets as described [here]({{< ref component-secrets.md >}}).
|
||||||
|
{{% /alert %}}
|
||||||
|
|
||||||
|
## Spec metadata fields
|
||||||
|
|
||||||
|
| Field | Required | Details | Example |
|
||||||
|
|--------------------|:--------:|---------|---------|
|
||||||
|
| host | Y | Connection-string for the rabbitmq host | `amqp://user:pass@localhost:5672`
|
||||||
|
| durable | N | Whether or not to use [durable](https://www.rabbitmq.com/queues.html#durability) queues. Defaults to `"false"` | `"true"`, `"false"`
|
||||||
|
| deletedWhenUnused | N | Whether or not the queue sohuld be configured to [auto-delete](https://www.rabbitmq.com/queues.html) Defaults to `"false"` | `"true"`, `"false"`
|
||||||
|
| autoAck | N | Whether or not the queue consumer should [auto-ack](https://www.rabbitmq.com/confirms.html) messages. Defaults to `"false"` | `"true"`, `"false"`
|
||||||
|
| deliveryMode | N | Persistence mode when publishing messages. Defaults to `"0"`. RabbitMQ treats `"2"` as persistent, all other numbers as non-persistent | `"0"`, `"2"`
|
||||||
|
| requeueInFailure | N | Whether or not to requeue when sending a [negative acknolwedgement](https://www.rabbitmq.com/nack.html) in case of a failure. Defaults to `"false"` | `"true"`, `"false"`
|
||||||
|
| prefetchCount | N | Number of messages to [prefecth](https://www.rabbitmq.com/consumer-prefetch.html). Consider changing this to a non-zero value for production environments. Defaults to `"0"`, which means that all available messages will be pre-fetched. | `"2"`
|
||||||
|
| reconnectWait | N | How long to wait (in seconds) before reconnecting if a connection failure occurs | `"0"`
|
||||||
|
| concurrency | N | `paralell` is the default, and allows processing multiple messages in paralell (limited by the `app-max-concurrency` annotation, if configured). Set to `single` to disable paralell processing. In most situations there's no reason to change this. | `paralell`, `single`
|
||||||
|
|
||||||
|
|
||||||
|
## Create a RabbitMQ server
|
||||||
|
|
||||||
{{< tabs "Self-Hosted" "Kubernetes" >}}
|
{{< tabs "Self-Hosted" "Kubernetes" >}}
|
||||||
|
|
||||||
|
@ -38,45 +88,8 @@ For example, if installing using the example above, the RabbitMQ server client a
|
||||||
|
|
||||||
{{< /tabs >}}
|
{{< /tabs >}}
|
||||||
|
|
||||||
## Create a Dapr component
|
|
||||||
|
|
||||||
The next step is to create a Dapr component for RabbitMQ.
|
|
||||||
|
|
||||||
Create the following YAML file named `rabbitmq.yaml`:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
apiVersion: dapr.io/v1alpha1
|
|
||||||
kind: Component
|
|
||||||
metadata:
|
|
||||||
name: <NAME>
|
|
||||||
namespace: <NAMESPACE>
|
|
||||||
spec:
|
|
||||||
type: pubsub.rabbitmq
|
|
||||||
version: v1
|
|
||||||
metadata:
|
|
||||||
- name: host
|
|
||||||
value: <REPLACE-WITH-HOST> # Required. Example: "amqp://rabbitmq.default.svc.cluster.local:5672", "amqp://localhost:5672"
|
|
||||||
- name: durable
|
|
||||||
value: <REPLACE-WITH-DURABLE> # Optional. Default: "false"
|
|
||||||
- name: deletedWhenUnused
|
|
||||||
value: <REPLACE-WITH-DELETE-WHEN-UNUSED> # Optional. Default: "false"
|
|
||||||
- name: autoAck
|
|
||||||
value: <REPLACE-WITH-AUTO-ACK> # Optional. Default: "false"
|
|
||||||
- name: deliveryMode
|
|
||||||
value: <REPLACE-WITH-DELIVERY-MODE> # Optional. Default: "0". Values between 0 - 2.
|
|
||||||
- name: requeueInFailure
|
|
||||||
value: <REPLACE-WITH-REQUEUE-IN-FAILURE> # Optional. Default: "false".
|
|
||||||
- name: concurrencyMode
|
|
||||||
value: <REPLACE-WITH-CONCURRENCY-MODE> # Optional. Default: "parallel". Values: "single", "parallel".
|
|
||||||
```
|
|
||||||
|
|
||||||
{{% alert title="Warning" color="warning" %}}
|
|
||||||
The above example uses secrets as plain strings. It is recommended to use a secret store for the secrets as described [here]({{< ref component-secrets.md >}}).
|
|
||||||
{{% /alert %}}
|
|
||||||
|
|
||||||
## Apply the configuration
|
|
||||||
|
|
||||||
Visit [this guide]({{< ref "howto-publish-subscribe.md#step-2-publish-a-topic" >}}) for instructions on configuring pub/sub components.
|
|
||||||
|
|
||||||
## Related links
|
## Related links
|
||||||
|
- [Basic schema for a Dapr component]({{< ref component-schema >}}) in the Related links section
|
||||||
|
- Read [this guide]({{< ref "howto-publish-subscribe.md#step-2-publish-a-topic" >}}) for instructions on configuring pub/sub components
|
||||||
- [Pub/Sub building block]({{< ref pubsub >}})
|
- [Pub/Sub building block]({{< ref pubsub >}})
|
||||||
|
|
|
@ -106,5 +106,5 @@ The Dapr CLI automatically deploys a local redis instance in self hosted mode as
|
||||||
|
|
||||||
## Related links
|
## Related links
|
||||||
- [Basic schema for a Dapr component]({{< ref component-schema >}})
|
- [Basic schema for a Dapr component]({{< ref component-schema >}})
|
||||||
- Read [this topic]({{< ref "howto-publish-subscribe.md#step-2-publish-a-topic" >}}) for instructions on configuring pub/sub components
|
- Read [this guide]({{< ref "howto-publish-subscribe.md#step-2-publish-a-topic" >}}) for instructions on configuring pub/sub components
|
||||||
- [Pub/Sub building block]({{< ref pubsub >}})
|
- [Pub/Sub building block]({{< ref pubsub >}})
|
|
@ -14,7 +14,7 @@ Dapr guarantees at least once semantics for this endpoint.
|
||||||
### HTTP Request
|
### HTTP Request
|
||||||
|
|
||||||
```
|
```
|
||||||
POST http://localhost:<daprPort>/v1.0/publish/<pubsubname>/<topic>
|
POST http://localhost:<daprPort>/v1.0/publish/<pubsubname>/<topic>[?<metadata>]
|
||||||
```
|
```
|
||||||
|
|
||||||
### HTTP Response codes
|
### HTTP Response codes
|
||||||
|
@ -31,8 +31,9 @@ Code | Description
|
||||||
Parameter | Description
|
Parameter | Description
|
||||||
--------- | -----------
|
--------- | -----------
|
||||||
daprPort | the Dapr port
|
daprPort | the Dapr port
|
||||||
pubsubname | the name of pubsub component.
|
pubsubname | the name of pubsub component
|
||||||
topic | the name of the topic
|
topic | the name of the topic
|
||||||
|
metadata | query parameters for metadata as described below
|
||||||
|
|
||||||
> Note, all URL parameters are case-sensitive.
|
> Note, all URL parameters are case-sensitive.
|
||||||
|
|
||||||
|
@ -44,6 +45,16 @@ curl -X POST http://localhost:3500/v1.0/publish/pubsubName/deathStarStatus \
|
||||||
}'
|
}'
|
||||||
```
|
```
|
||||||
|
|
||||||
|
#### Metadata
|
||||||
|
|
||||||
|
Metadata can be sent via query parameters in the request's URL. It must be prefixed with `metadata.` as shown below.
|
||||||
|
|
||||||
|
Parameter | Description
|
||||||
|
--------- | -----------
|
||||||
|
metadata.ttlInSeconds | the number of seconds for the message to expire as [described here]({{< ref pubsub-message-ttl.md >}})
|
||||||
|
|
||||||
|
> Additional metadata parameters are available based on each pubsub component.
|
||||||
|
|
||||||
## Optional Application (User Code) Routes
|
## Optional Application (User Code) Routes
|
||||||
|
|
||||||
### Provide a route for Dapr to discover topic subscriptions
|
### Provide a route for Dapr to discover topic subscriptions
|
||||||
|
|
Loading…
Reference in New Issue