mirror of https://github.com/knative/docs.git
Docs for KafkaNamespaced broker (#5389)
* Docs for KafkaNamespaced broker * Some basic fixes * Remove some trailing whitespace * Update docs/eventing/brokers/broker-types/kafka-broker/README.md Co-authored-by: Pierangelo Di Pilato <pierangelodipilato@gmail.com> * Update docs/eventing/brokers/broker-types/kafka-broker/README.md Co-authored-by: Pierangelo Di Pilato <pierangelodipilato@gmail.com> * Address comments * Address comments * Address comments * Address comments --------- Co-authored-by: Pierangelo Di Pilato <pierangelodipilato@gmail.com>
This commit is contained in:
Ali Ok
GitHub
parent
c7de3c428c
commit
7d1c207577
|
|
@ -16,7 +16,7 @@ The following is a list of brokers provided by the community or vendors:
|
|||
|
||||
### Apache Kafka broker
|
||||
|
||||
For more information, see [Apache Kafka Broker](kafka-broker/README.md).
|
||||
This broker implementation uses [Apache Kafka](https://kafka.apache.org/) as its backing technology. For more information, see [Apache Kafka Broker](kafka-broker/README.md).
|
||||
|
||||
### RabbitMQ broker
|
||||
|
||||
|
|
|
|||
|
|
@ -10,6 +10,7 @@ Notable features are:
|
|||
- [Extensively configurable](#kafka-producer-and-consumer-configurations)
|
||||
- Ordered delivery of events based on [CloudEvents partitioning extension](https://github.com/cloudevents/spec/blob/v1.0.2/cloudevents/extensions/partitioning.md)
|
||||
- Support any Kafka version, see [compatibility matrix](https://cwiki.apache.org/confluence/display/KAFKA/Compatibility+Matrix)
|
||||
- Supports 2 [data plane modes](#data-plane-isolation-vs-shared-data-plane): data plane isolation per-namespace or shared data plane
|
||||
|
||||
The Knative Kafka Broker stores incoming CloudEvents as Kafka records, using the [binary content mode](https://github.com/cloudevents/spec/blob/v1.0.2/cloudevents/bindings/kafka-protocol-binding.md#32-binary-content-mode), because it is more efficient due to its optimizations for transport or routing, as well avoid JSON parsing. Using `binary content mode` means all CloudEvent attributes and extensions are mapped as [headers on the Kafka record](https://github.com/cloudevents/spec/blob/v1.0.2/cloudevents/bindings/kafka-protocol-binding.md#323-metadata-headers), while the `data` of the CloudEvent corresponds to the actual value of the Kafka record. This is another benefit of using `binary content mode` over `structured content mode` as it is less _obstructive_ and therefore compatible with systems that do not understand CloudEvents.
|
||||
|
||||
|
|
@ -386,6 +387,65 @@ The supported consumer delivery guarantees are:
|
|||
|
||||
The `unordered` delivery is the default ordering guarantee.
|
||||
|
||||
### Additional information
|
||||
## Data plane Isolation vs Shared Data plane
|
||||
|
||||
Knative Kafka Broker implementation has 2 planes: control plane and data plane. Control plane consists of controllers that talk to Kubernetes API, watch for custom objects and manage the data plane.
|
||||
|
||||
Data plane is the collection of components that listen for incoming events, talk to Apache Kafka and also sends events to the event sinks. This is where the events flow. Knative Kafka Broker data plane consists of `kafka-broker-receiver` and `kafka-broker-dispatcher` deployments.
|
||||
|
||||
When using the Broker class `Kafka`, the Knative Kafka Broker uses a shared data plane. That means, `kafka-broker-receiver` and `kafka-broker-dispatcher` deployments in `knative-eventing` namespace is used for all Kafka Brokers in the cluster.
|
||||
|
||||
However, when `KafkaNamespaced` is set as the Broker class, Kafka broker controller creates a new data plane for each namespace that there is a broker exists. This data plane is used by all `KafkaNamespaced` brokers in that namespace.
|
||||
|
||||
That provides isolation between the data planes, which means that the `kafka-broker-receiver` and `kafka-broker-dispatcher` deployments in the user namespace are only used for the broker in that namespace.
|
||||
|
||||
!!! note
|
||||
As a consequence of separate data planes, this security feature creates more deployments and uses more resources. Unless you have such isolation requirements, it is recommended to go with *regular* Broker with `Kafka` class.
|
||||
|
||||
To create a `KafkaNamespaced` broker, you must set the `eventing.knative.dev/broker.class` annotation to `KafkaNamespaced`:
|
||||
|
||||
```yaml
|
||||
apiVersion: eventing.knative.dev/v1
|
||||
kind: Broker
|
||||
metadata:
|
||||
annotations:
|
||||
# case-sensitive
|
||||
eventing.knative.dev/broker.class: KafkaNamespaced
|
||||
name: default
|
||||
namespace: my-namespace
|
||||
spec:
|
||||
config:
|
||||
# the referenced `configmap` must be in the same namespace with the `Broker` object, in this case `my-namespace`
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
name: my-config
|
||||
# namespace: my-namespace # no need to define, defaults to Broker's namespace
|
||||
```
|
||||
|
||||
!!! note
|
||||
The `configmap` that is specified in `spec.config` **must** be in the same namespace with the `Broker` object:
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: my-config
|
||||
namespace: my-namespace
|
||||
data:
|
||||
...
|
||||
```
|
||||
|
||||
Upon the creation of the first `Broker` with `KafkaNamespaced` class, the `kafka-broker-receiver` and `kafka-broker-dispatcher` deployments are created in the namespace. After that, all the brokers with `KafkaNamespaced` class in the same namespace use the same data plane. When there are no brokers of `KafkaNamespaced` class in the namespace, the data plane in the namespace will be deleted.
|
||||
|
||||
### Configuring `KafkaNamespaced` brokers
|
||||
|
||||
All the configuration mechanisms that are available for the `Kafka` Broker class are also available for the brokers with `KafkaNamespaced` class with these exceptions:
|
||||
|
||||
* [Above](#kafka-producer-and-consumer-configurations) it is described how producer and consumer configurations is done by modifying the `config-kafka-broker-data-plane` configmap in the `knative-eventing` namespace. Since Kafka Broker controller propagates this configmap into the user namespace, currently there is no way to configure producer and consumer configurations per namespace. Any value set in the `config-kafka-broker-data-plane` `ConfigMap` in the `knative-eventing` namespace will be also used in the user namespace.
|
||||
* Because of the same propagation, it is also not possible to configure consumer offsets commit interval per namespace.
|
||||
* A few more configmaps are propagated: `config-tracing` and `kafka-config-logging`. This means, tracing and logging are also not configurable per namespace.
|
||||
* Similarly, the data plane deployments are propagated from the `knative-eventing` namespace to the user namespace. This means that the data plane deployments are not configurable per namespace and will be identical to the ones in the `knative-eventing` namespace.
|
||||
|
||||
## Additional information
|
||||
|
||||
- To report a bug or request a feature, open an issue in the [eventing-kafka-broker repository](https://github.com/knative-sandbox/eventing-kafka-broker).
|
||||
|
|
|
|||
Loading…
Reference in New Issue