dapr
/
components-contrib
mirror of https://github.com/dapr/components-contrib.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
components-contrib/tests/certification/pubsub/kafka
History
Bernd Verst ede060959a
Refactors all state store to standardize metadata parsing (#2257) 		2022-11-05 15:36:44 -07:00
..
components Refactory kafka binding to reuse the kafka common code extracting from kafka pubsub component (#1696) 2022-05-10 20:07:23 -07:00
oauth-config Add OAuth2/OIDC support to Kafka pubsub (#1229) 2021-12-28 10:34:28 -08:00
strimzi-broker-certs Add OAuth2/OIDC support to Kafka pubsub (#1229) 2021-12-28 10:34:28 -08:00
strimzi-ca-certs Add OAuth2/OIDC support to Kafka pubsub (#1229) 2021-12-28 10:34:28 -08:00
strimzi-client-ca Add OAuth2/OIDC support to Kafka pubsub (#1229) 2021-12-28 10:34:28 -08:00
strimzi-kafka1-config Add OAuth2/OIDC support to Kafka pubsub (#1229) 2021-12-28 10:34:28 -08:00
strimzi-kafka2-config Add OAuth2/OIDC support to Kafka pubsub (#1229) 2021-12-28 10:34:28 -08:00
strimzi-kafka3-config Add OAuth2/OIDC support to Kafka pubsub (#1229) 2021-12-28 10:34:28 -08:00
strimzi-listener-certs Add OAuth2/OIDC support to Kafka pubsub (#1229) 2021-12-28 10:34:28 -08:00
README.md Kafka certification: Adding more consumers and verifying messages are consumed (#1262) 2021-11-02 13:51:15 -07:00
config.yaml Initial certification testing "framework" (#1204) 2021-10-20 10:57:47 -07:00
docker-compose.yml Add OAuth2/OIDC support to Kafka pubsub (#1229) 2021-12-28 10:34:28 -08:00
go.mod Refactors all state store to standardize metadata parsing (#2257) 2022-11-05 15:36:44 -07:00
go.sum Refactors all state store to standardize metadata parsing (#2257) 2022-11-05 15:36:44 -07:00
kafka_test.go Components' init functions should return interfaces (#1997) 2022-08-25 18:27:45 -07:00

README.md

Kafka certifcation testing

This project aims to test the Kafka Pub/Sub component under various conditions.

Test plan

Basic tests

  • Bring up a 3-node Kafka cluster
    • Configured to have 10+ partitions per topic
  • Start 1 sidecar/application
    • Test: Send 1000+ unique messages with keys set
    • App: Simulate periodic errors
    • Component: Retries on error
    • App: Observes successful messages
    • Test: Confirms that all expected messages were received (in order)

Kafka infra tests

  • Start a constant flow of publishing and subscribing
    • Test: Keeps count of total sent/received
  • Start a second sidecar/application using a different consumer group
    • Test: Publishes a specific amount of messages
    • Each consumer group should receive all messages
  • Start third consumer with a matching consumer group
    • Test: Publishes a specific amount of messages
    • Component: Between the each of the consumers in the group, all messages should be consumed, but not necessarily in order.
  • Stop 1 broker node so that 2 of 3 are active
    • The 2 applications should handle the server rebalance
  • Stop another broker so that 1 of 3 are active (loss of quorum)
    • Test: Begins trying to reconnect & publish
    • Component: Begins trying to reconnect & re-subscribe
  • Stop the last broker so that 0 of 3 are active (complete outage)
    • Same as reconnection behavior above
  • Restart both brokers so that 3 of 3 are active
    • Test & Component: Reconnect
    • Count of total sent should equal total received
  • Stop consumer with >1 sidecar subscribed
    • Test: Publishes messages in the background
    • Component: Handles a consumer rebalance

Network tests

  • Simulate network interruption
    • Test: Begins trying to reconnect & publish
    • Component: Begins trying to reconnect & re-subscribe

Data integrity tests

  • TODO Start a new sidecar/application
  • TODO Verify cloud events
    • TODO Publish various cloud events
    • TODO App receives Kafka messages and verifies their binary encoding
  • TODO Verify raw events
    • TODO Publish various raw events
    • TODO App receives Kafka messages and verifies their binary encoding