dapr
/
docs
mirror of https://github.com/dapr/docs.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Kafka bulk support documentation (#3071) 

* add initial docs for kafka bulk support

Signed-off-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com>

* address review comments

Signed-off-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com>

---------

Signed-off-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com>
This commit is contained in:
Mukundan Sundararajan 2023-02-08 09:30:12 +05:30 committed by GitHub
parent b0bf194c81
commit efcb297f17
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 18 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

23
daprdocs/content/en/reference/components-reference/supported-pubsub/setup-apache-kafka.md
View File

@ -111,7 +111,7 @@ spec:
value: 200ms
- name: version # Optional.
value: 0.10.2.0
- name: disableTls
- name: disableTls
value: "true"
```
@ -201,13 +201,13 @@ spec:
#### OAuth2 or OpenID Connect
Setting `authType` to `oidc` enables SASL authentication via the **OAUTHBEARER** mechanism. This supports specifying a bearer token from an external OAuth2 or [OIDC](https://en.wikipedia.org/wiki/OpenID) identity provider. Currently, only the **client_credentials** grant is supported.
Setting `authType` to `oidc` enables SASL authentication via the **OAUTHBEARER** mechanism. This supports specifying a bearer token from an external OAuth2 or [OIDC](https://en.wikipedia.org/wiki/OpenID) identity provider. Currently, only the **client_credentials** grant is supported.
Configure `oidcTokenEndpoint` to the full URL for the identity provider access token endpoint.
Configure `oidcTokenEndpoint` to the full URL for the identity provider access token endpoint.
Set `oidcClientID` and `oidcClientSecret` to the client credentials provisioned in the identity provider.
Set `oidcClientID` and `oidcClientSecret` to the client credentials provisioned in the identity provider.
If `caCert` is specified in the component configuration, the certificate is appended to the system CA trust for verifying the identity provider certificate. Similarly, if `skipVerify` is specified in the component configuration, verification will also be skipped when accessing the identity provider.
If `caCert` is specified in the component configuration, the certificate is appended to the system CA trust for verifying the identity provider certificate. Similarly, if `skipVerify` is specified in the component configuration, verification will also be skipped when accessing the identity provider.
By default, the only scope requested for the token is `openid`; it is **highly** recommended that additional scopes be specified via `oidcScopes` in a comma-separated list and validated by the Kafka broker. If additional scopes are not used to narrow the validity of the access token,
a compromised Kafka broker could replay the token to access other services as the Dapr clientID.
@ -296,6 +296,19 @@ auth:
secretStore: <SECRET_STORE_NAME>
```
## Sending and receiving multiple messages
Apache Kafka component supports sending and receiving multiple messages in a single operation using the bulk Pub/sub API.
### Configuring bulk subscribe
When subscribing to a topic, you can configure `bulkSubscribe` options. Refer to [Subscription methods]({{< ref subscription-methods >}}) for more details. Learn more about [the bulk subscribe API]({{< ref pubsub-bulk.md >}}).
| Configuration | Default |
|----------|---------|
| `maxBulkAwaitDurationMs` | `10000` (10s) |
| `maxBulkSubCount` | `80` |
## Per-call metadata fields
### Partition Key