dapr
/
docs
mirror of https://github.com/dapr/docs.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #1656 from CodeMonkeyLeet/update_kafka_component_specs 

Update Kafka component specs
This commit is contained in:
Ori Zohar 2021-07-16 13:17:36 -07:00 committed by GitHub
parent 064571439f 1de5dccdef
commit e633d5db05
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 51 additions and 52 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

56
daprdocs/content/en/reference/components-reference/supported-bindings/kafka.md
View File

@ -9,53 +9,50 @@ aliases:
## Component format ## Component format
To setup Kafka binding create a component of type `bindings.kafka`. See [this guide]({{< ref "howto-bindings.md#1-create-a-binding" >}}) on how to create and apply a binding configuration. To setup Kafka binding create a component of type `bindings.kafka`. See [this guide]({{< ref "howto-bindings.md#1-create-a-binding" >}}) on how to create and apply a binding configuration. For details on using `secretKeyRef`, see the guide on [how to reference secrets in components]({{< ref component-secrets.md >}}).
```yaml ```yaml
apiVersion: dapr.io/v1alpha1 apiVersion: dapr.io/v1alpha1
kind: Component kind: Component
metadata: metadata:
name: <NAME> name: kafka-binding
namespace: <NAMESPACE> namespace: default
spec: spec:
type: bindings.kafka type: bindings.kafka
version: v1 version: v1
metadata: metadata:
- name: topics # Optional. in use for input bindings - name: topics # Optional. Used for input bindings.
value: topic1,topic2 value: "topic1,topic2"
- name: brokers - name: brokers # Required.
value: localhost:9092,localhost:9093 value: "localhost:9092,localhost:9093"
- name: consumerGroup - name: consumerGroup # Optional. Used for input bindings.
value: group1 value: "group1"
- name: publishTopic # Optional. in use for output bindings - name: publishTopic # Optional. Used for output bindings.
value: topic3 value: "topic3"
- name: authRequired # Required. default: "true" - name: authRequired # Required.
value: "false" value: "true"
- name: saslUsername # Optional. - name: saslUsername # Required if authRequired is `true`.
value: "user" value: "user"
- name: saslPassword # Optional. - name: saslPassword # Required if authRequired is `true`.
value: "password" secretKeyRef:
name: kafka-secrets
key: saslPasswordSecret
- name: maxMessageBytes # Optional. - name: maxMessageBytes # Optional.
value: 1024 value: 1024
``` ```
{{% alert title="Warning" color="warning" %}}
The above example uses secrets as plain strings. It is recommended to use a secret store for the secrets as described [here]({{< ref component-secrets.md >}}).
{{% /alert %}}
## Spec metadata fields ## Spec metadata fields
| Field | Required | Binding support | Details | Example | | Field | Required | Binding support | Details | Example |
|--------------------|:--------:|------------|-----|---------| |--------------------|:--------:|------------|-----|---------|
| topics | N | Input | A comma separated string of topics | `"mytopic1,topic2"` | | topics | N | Input | A comma-separated string of topics. | `"mytopic1,topic2"` |
| brokers | Y | Input/Output | A comma separated string of kafka brokers | `"localhost:9092,localhost:9093"` | | brokers | Y | Input/Output | A comma-separated string of Kafka brokers. | `"localhost:9092,dapr-kafka.myapp.svc.cluster.local:9093"` |
| consumerGroup | N | Input | A kafka consumer group to listen on | `"group1"` | | consumerGroup | N | Input | A kafka consumer group to listen on. Each record published to a topic is delivered to one consumer within each consumer group subscribed to the topic. | `"group1"` |
| publishTopic | Y | Output | The topic to publish to | `"mytopic"` | | publishTopic | Y | Output | The topic to publish to. | `"mytopic"` |
| authRequired | Y | Input/Output | Determines whether to use SASL authentication or not. Defaults to `"true"` | `"true"`, `"false"` | | authRequired | Y | Input/Output | Enable [SASL](https://en.wikipedia.org/wiki/Simple_Authentication_and_Security_Layer) authentication with the Kafka brokers. | `"true"`, `"false"` |
| saslUsername | N | Input/Output | The SASL username for authentication. Only used if `authRequired` is set to - `"true"` | `"user"` | | saslUsername | N | Input/Output | The SASL username used for authentication. Only required if `authRequired` is set to `"true"`. | `"adminuser"` |
| saslPassword | N | Input/Output | The SASL password for authentication. Only used if `authRequired` is set to - `"true"` | `"password"` | | saslPassword | N | Input/Output | The SASL password used for authentication. Can be `secretKeyRef` to use a [secret reference]({{< ref component-secrets.md >}}). Only required if `authRequired` is set to `"true"`. | `""`, `"KeFg23!"` |
| maxMessageBytes | N | Input/Output | The maximum size allowed for a single Kafka message. Defaults to 1024 | `2048` | | maxMessageBytes | N | Input/Output | The maximum size in bytes allowed for a single Kafka message. Defaults to 1024. | `2048` |
## Binding support ## Binding support
@ -87,7 +84,6 @@ curl -X POST http://localhost:3500/v1.0/bindings/myKafka \
}' }'
``` ```
## Related links ## Related links
- [Basic schema for a Dapr component]({{< ref component-schema >}}) - [Basic schema for a Dapr component]({{< ref component-schema >}})

47
daprdocs/content/en/reference/components-reference/supported-pubsub/setup-apache-kafka.md
View File

@ -9,7 +9,7 @@ aliases:
## Component format ## Component format
To setup Apache Kafka pubsub create a component of type `pubsub.kafka`. See [this guide]({{< ref "howto-publish-subscribe.md#step-1-setup-the-pubsub-component" >}}) on how to create and apply a pubsub configuration. To setup Apache Kafka pubsub create a component of type `pubsub.kafka`. See [this guide]({{< ref "howto-publish-subscribe.md#step-1-setup-the-pubsub-component" >}}) on how to create and apply a pubsub configuration. For details on using `secretKeyRef`, see the guide on [how to reference secrets in components]({{< ref component-secrets.md >}}).
```yaml ```yaml
apiVersion: dapr.io/v1alpha1 apiVersion: dapr.io/v1alpha1
@ -21,32 +21,35 @@ spec:
type: pubsub.kafka type: pubsub.kafka
version: v1 version: v1
metadata: metadata:
# Kafka broker connection setting - name: brokers # Required. Kafka broker connection setting
- name: brokers value: "dapr-kafka.myapp.svc.cluster.local:9092"
value: "dapr-kafka.myapp.svc.cluster.local:9092" - name: consumerGroup # Optional. Used for input bindings.
- name: authRequired value: "group1"
value: "true" - name: clientID # Optional. Used as client tracing ID by Kafka brokers.
- name: saslUsername value: "my-dapr-app-id"
value: "adminuser" - name: authRequired # Required.
- name: saslPassword value: "true"
value: "KeFg23!" - name: saslUsername # Required if authRequired is `true`.
- name: maxMessageBytes value: "adminuser"
value: 1024 - name: saslPassword # Required if authRequired is `true`.
secretKeyRef:
name: kafka-secrets
key: saslPasswordSecret
- name: maxMessageBytes # Optional.
value: 1024
``` ```
{{% alert title="Warning" color="warning" %}}
The above example uses secrets as plain strings. It is recommended to use a secret store for the secrets as described [here]({{< ref component-secrets.md >}}).
{{% /alert %}}
## Spec metadata fields ## Spec metadata fields
| Field | Required | Details | Example | | Field | Required | Details | Example |
|--------------------|:--------:|---------|---------| |--------------------|:--------:|---------|---------|
| brokers | Y | Comma separated list of kafka brokers | `localhost:9092`, `dapr-kafka.myapp.svc.cluster.local:9092` | brokers | Y | A comma-separated list of Kafka brokers. | `"localhost:9092,dapr-kafka.myapp.svc.cluster.local:9093"`
| authRequired | N | Enable authentication on the Kafka broker. Defaults to `"false"`. |`"true"`, `"false"` | consumerGroup | N | A kafka consumer group to listen on. Each record published to a topic is delivered to one consumer within each consumer group subscribed to the topic. | `"group1"`
| saslUsername | N | Username used for authentication. Only required if authRequired is set to true. | `"adminuser"` | clientID | N | A user-provided string sent with every request to the Kafka brokers for logging, debugging, and auditing purposes. Defaults to `"sarama"`. | `"my-dapr-app"`
| saslPassword | N | Password used for authentication. Can be `secretKeyRef` to use a secret reference. Only required if authRequired is set to true. Can be `secretKeyRef` to use a [secret reference]({{< ref component-secrets.md >}}) | `""`, `"KeFg23!"` | authRequired | Y | Enable [SASL](https://en.wikipedia.org/wiki/Simple_Authentication_and_Security_Layer) authentication with the Kafka brokers. | `"true"`, `"false"`
| maxMessageBytes | N | The maximum message size allowed for a single Kafka message. Default is 1024. | `2048` | saslUsername | N | The SASL username used for authentication. Only required if `authRequired` is set to `"true"`. | `"adminuser"`
| saslPassword | N | The SASL password used for authentication. Can be `secretKeyRef` to use a [secret reference]({{< ref component-secrets.md >}}). Only required if `authRequired` is set to `"true"`. | `""`, `"KeFg23!"`
| maxMessageBytes | N | The maximum size in bytes allowed for a single Kafka message. Defaults to 1024. | `2048`
## Per-call metadata fields ## Per-call metadata fields
@ -69,6 +72,7 @@ curl -X POST http://localhost:3500/v1.0/publish/myKafka/myTopic?metadata.partiti
``` ```
## Create a Kafka instance ## Create a Kafka instance
{{< tabs "Self-Hosted" "Kubernetes">}} {{< tabs "Self-Hosted" "Kubernetes">}}
{{% codetab %}} {{% codetab %}}
@ -82,7 +86,6 @@ To run Kafka on Kubernetes, you can use any Kafka operator, such as [Strimzi](ht
{{< /tabs >}} {{< /tabs >}}
## Related links ## Related links
- [Basic schema for a Dapr component]({{< ref component-schema >}}) - [Basic schema for a Dapr component]({{< ref component-schema >}})
- Read [this guide]({{< ref "howto-publish-subscribe.md##step-1-setup-the-pubsub-component" >}}) for instructions on configuring pub/sub components - Read [this guide]({{< ref "howto-publish-subscribe.md##step-1-setup-the-pubsub-component" >}}) for instructions on configuring pub/sub components