From b224673e6008983129842d363a3542cc4637f27d Mon Sep 17 00:00:00 2001 From: Knative Prow Robot Date: Mon, 13 Dec 2021 08:38:19 -0800 Subject: [PATCH] [release-1.0] Document Kafka Broker commit offset interval (#4560) * Document Kafka Broker commit offset interval Signed-off-by: Pierangelo Di Pilato * Apply review suggestion Signed-off-by: Pierangelo Di Pilato Co-authored-by: Pierangelo Di Pilato --- docs/eventing/broker/kafka-broker/README.md | 35 +++++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/docs/eventing/broker/kafka-broker/README.md b/docs/eventing/broker/kafka-broker/README.md index bd89c5a8f..743b07b42 100644 --- a/docs/eventing/broker/kafka-broker/README.md +++ b/docs/eventing/broker/kafka-broker/README.md @@ -224,6 +224,41 @@ kubectl create secret --namespace generic \ _NOTE: `ca.crt` can be omitted to fallback to use system's root CA set._ +## Consumer Offsets Commit Interval + +Kafka consumers keep track of the last successfully sent events by committing offsets. + +Knative Kafka Broker commits the offset every `auto.commit.interval.ms` milliseconds. + +!!! note + To prevent negative impacts to performance, it is not recommended committing + offsets every time an event is successfully sent to a subscriber. + +The interval can be changed by changing the `config-kafka-broker-data-plane` `ConfigMap` +in the `knative-eventing` namespace by modifying the parameter `auto.commit.interval.ms` as follows: + +```yaml + +apiVersion: v1 +kind: ConfigMap +metadata: + name: config-kafka-broker-data-plane + namespace: knative-eventing +data: + # Some configurations omitted ... + config-kafka-broker-consumer.properties: | + # Some configurations omitted ... + + # Commit the offset every 5000 millisecods (5 seconds) + auto.commit.interval.ms=5000 +``` + +!!! note + Knative Kafka Broker guarantees at least once delivery, which means that your applications may + receive duplicate events. A higher commit interval means that there is a higher probability of + receiving duplicate events, because when a Consumer restarts, it restarts from the last + committed offset. + ## Kafka Producer and Consumer configurations Knative exposes all available Kafka producer and consumer configurations that can be modified to suit your workloads.