Merge commit 'refs/pull/3005/head' of https://github.com/dapr/docs into v1.10

This commit is contained in:
Deepanshu Agarwal 2023-01-20 15:20:57 +05:30
commit 22a3c40890
6 changed files with 264 additions and 0 deletions

View File

@ -0,0 +1,163 @@
---
type: docs
title: "Send and receive messages in bulk"
linkTitle: "Send and receive messages in bulk"
weight: 7100
description: "Learn how to use the bulk publish and subscribe APIs in Dapr."
---
{{% alert title="alpha" color="warning" %}}
The bulk publish and subscribe APIs are in **alpha** stage.
{{% /alert %}}
With the bulk publish and subscribe APIs, you can send and receive multiple messages in a single request.
## Native bulk publish and subscribe support
When a pub/sub component supports the bulk publish API natively, Dapr also publishes messages to the underlying pub/sub component in bulk.
Otherwise, Dapr falls back to sending messages one by one to the underlying pub/sub component. This is still more efficient than using the regular publish API, because applications can still send multiple messages in a single request to Dapr.
## Supported components
Refer to the [component reference]({{< ref supported-pubsub >}}) to see which components support the bulk publish API natively.
## Publishing messages in bulk
The bulk publish API allows you to publish multiple messages to a topic in a single request. If any of the messages fail to publish, the bulk publish operation returns a list of failed messages. Note, the bulk publish operation does not guarantee the order of messages.
### Example
{{< tabs Java Javascript "HTTP API (Bash)" "HTTP API (PowerShell)" >}}
{{% codetab %}}
```java
import io.dapr.client.DaprClientBuilder;
import io.dapr.client.DaprPreviewClient;
import io.dapr.client.domain.BulkPublishResponse;
import io.dapr.client.domain.BulkPublishResponseFailedEntry;
import java.util.ArrayList;
import java.util.List;
class BulkPublisher {
public void publishMessages() {
try (DaprPreviewClient client = (new DaprClientBuilder()).buildPreviewClient()) {
// Create a list of messages to publish
List<String> messages = new ArrayList<>();
for (int i = 0; i < 10; i++) {
String message = String.format("This is message #%d", i);
messages.add(message);
}
// Publish list of messages using the bulk publish API
BulkPublishResponse<String> res = client.publishEvents(PUBSUB_NAME, TOPIC_NAME, "text/plain", messages).block();
}
}
}
```
{{% /codetab %}}
{{% codetab %}}
```typescript
import { DaprClient } from "@dapr/dapr";
const pubSubName = "my-pubsub-name";
const topic = "topic-a";
async function start() {
const client = new DaprClient();
// Publish multiple messages to a topic.
await client.pubsub.publishBulk(pubSubName, topic, ["message 1", "message 2", "message 3"]);
// Publish multiple messages to a topic with explicit bulk publish messages.
const bulkPublishMessages = [
{
entryID: "entry-1",
contentType: "application/json",
event: { hello: "foo message 1" },
},
{
entryID: "entry-2",
contentType: "application/cloudevents+json",
event: {
specversion: "1.0",
source: "/some/source",
type: "example",
id: "1234",
data: "foo message 2",
datacontenttype: "text/plain"
},
},
{
entryID: "entry-3",
contentType: "text/plain",
event: "foo message 3",
},
];
await client.pubsub.publishBulk(pubSubName, topic, bulkPublishMessages);
}
start().catch((e) => {
console.error(e);
process.exit(1);
});
```
{{% /codetab %}}
{{% codetab %}}
```bash
curl -X POST http://localhost:3500/v1.0-alpha1/publish/bulk/my-pubsub-name/topic-a \
-H 'Content-Type: application/json' \
-d '[
{
"entryId": "ae6bf7c6-4af2-11ed-b878-0242ac120002",
"event": "first",
"contentType": "text/plain"
},
{
"entryId": "b1f40bd6-4af2-11ed-b878-0242ac120002",
"event": {
"message": "second"
},
"contentType": "application/json"
},
]'
```
{{% /codetab %}}
{{% codetab %}}
```powershell
Invoke-RestMethod -Method Post -ContentType 'application/json' -Uri 'http://localhost:3500/v1.0-alpha1/publish/bulk/my-pubsub-name/topic-a' `
-Body '[
{
"entryId": "ae6bf7c6-4af2-11ed-b878-0242ac120002",
"event": "first",
"contentType": "text/plain"
},
{
"entryId": "b1f40bd6-4af2-11ed-b878-0242ac120002",
"event": {
"message": "second"
},
"contentType": "application/json"
},
]'
```
{{% /codetab %}}
{{< /tabs >}}
## Related links
- List of [supported pub/sub components]({{< ref supported-pubsub >}})
- Read the [API reference]({{< ref pubsub_api.md >}})

View File

@ -119,6 +119,10 @@ By default, all topic messages associated with an instance of a pub/sub componen
Dapr can set a timeout message on a per-message basis, meaning that if the message is not read from the pub/sub component, then the message is discarded. This timeout message prevents a build up of unread messages. If a message has been in the queue longer than the configured TTL, it is marked as dead. For more information, read [pub/sub message TTL]({{< ref pubsub-message-ttl.md >}}).
### Send and receive messages in bulk
Dapr supports sending and receiving multiple messages in a single request. This is useful for applications that require a high throughput. For more information, read [pub/sub bulk messages]({{< ref pubsub-bulk.md >}}).
## Try out pub/sub
### Quickstarts and tutorials

View File

@ -64,6 +64,90 @@ Parameter | Description
> Additional metadata parameters are available based on each pubsub component.
## Publish multiple messages to a given topic
This endpoint lets you publish multiple messages to consumers who are listening on a `topic`.
### HTTP Request
```
POST http://localhost:<daprPort>/v1.0-alpha1/publish/bulk/<pubsubname>/<topic>[?<metadata>]
```
The request body should contain a JSON array of entries with:
- Unique entry IDs
- The event to publish
- The content type of the event
If the content type for an event is not `application/cloudevents+json`, it is auto-wrapped as a CloudEvent (unless `metadata.rawPayload` is set to `true`).
Example:
```bash
curl -X POST http://localhost:3500/v1.0-alpha1/publish/bulk/pubsubName/deathStarStatus \
-H 'Content-Type: application/json' \
-d '[
{
"entryId": "ae6bf7c6-4af2-11ed-b878-0242ac120002",
"event": "first",
"contentType": "text/plain"
},
{
"entryId": "b1f40bd6-4af2-11ed-b878-0242ac120002",
"event": {
"message": "second"
},
"contentType": "application/json"
},
]'
```
### Headers
The `Content-Type` header should always be set to `application/json`.
### URL Parameters
|**Parameter**|**Description**|
|--|--|
|`daprPort`|The Dapr port|
|`pubsubname`|The name of pub/sub component|
|`topic`|The name of the topic|
|`metadata`|Query parameters for [metadata]({{< ref "pubsub_api.md#metadata" >}})|
### Metadata
Metadata can be sent via query parameters in the request's URL. It must be prefixed with `metadata.`, as shown in the table below.
|**Parameter**|**Description**|
|--|--|
|`metadata.rawPayload`|Boolean to determine if Dapr should publish the messages without wrapping them as CloudEvent.|
|`metadata.maxBulkPubBytes`|Maximum bytes to publish in a bulk publish request.|
#### HTTP Response
|**HTTP Status**|**Description**|
|--|--|
|204|All messages delivered|
|400|Pub/sub does not exist|
|403|Forbidden by access controls|
|500|At least one message failed to be delivered|
The response body is a JSON containing a list of failed entries. Example:
```json
{
"failedEntries": [
{
"entryId": "ae6bf7c6-4af2-11ed-b878-0242ac120002",
"error": "error message"
},
],
"errorCode": "ERR_PUBSUB_PUBLISH_MESSAGE"
}
```
## Optional Application (User Code) Routes
### Provide a route for Dapr to discover topic subscriptions

View File

@ -3,8 +3,14 @@
state: Stable
version: v1
since: "1.8"
features:
bulkPublish: true
bulkSubscribe: false
- component: Azure Service Bus
link: setup-azure-servicebus
state: Stable
version: v1
since: "1.0"
features:
bulkPublish: true
bulkSubscribe: true

View File

@ -13,6 +13,9 @@
state: Stable
version: v1
since: "1.5"
features:
bulkPublish: true
bulkSubscribe: true
- component: Redis Streams
link: setup-redis-pubsub
state: Stable

View File

@ -10,6 +10,8 @@
<table width="100%">
<tr>
<th>Component</th>
<th>Bulk Publish</th>
<th>Bulk Subscribe</th>
<th>Status</th>
<th>Component version</th>
<th>Since runtime version</th>
@ -19,6 +21,8 @@
<td><a href="/reference/components-reference/supported-pubsub/{{ .link }}/" }}>{{ .component
}}</a>
</td>
<td align="center">{{ if .features.bulkPublish }}✅{{else}}<img src="/images/emptybox.png">{{ end }}</td>
<td align="center">{{ if .features.bulkSubscribe }}✅{{else}}<img src="/images/emptybox.png">{{ end }}</td>
<td>{{ .state }}</td>
<td>{{ .version }}</td>
<td>{{ .since }}</td>