diff --git a/daprdocs/content/en/developing-applications/building-blocks/pubsub/howto-namespace.md b/daprdocs/content/en/developing-applications/building-blocks/pubsub/howto-namespace.md new file mode 100644 index 000000000..4e86fdbf2 --- /dev/null +++ b/daprdocs/content/en/developing-applications/building-blocks/pubsub/howto-namespace.md @@ -0,0 +1,74 @@ +--- +type: docs +title: "How to: Set up pub/sub namespace consumer groups" +linkTitle: "How to: Namespace consumer groups" +weight: 5000 +description: "Learn how to use the metadata-based namespace consumer group in your component" +--- + +You've set up [Dapr's pub/sub API building block]({{< ref pubsub-overview >}}), and your applications are publishing and subscribing to topics smoothly, using a centralized message broker. What if you'd like to perform simple A/B testing, blue/green deployments, or even canary deployments for your applications? Even with using Dapr, this can prove difficult. + +Dapr solves multi-tenancy at-scale with its pub/sub namespace consumer groups construct. + +## Without namespace consumer groups + +Let's say you have a Kubernetes cluster, with two applications (App1 and App2) deployed to the same namespace (namespace-a). App2 publishes to a topic called `order`, while App1 subscribes to the topic called `order`. This will create two consumer groups, named after your applications (App1 and App2). + +Diagram showing basic pubsub process. + +In order to perform simple testing and deployments while using a centralized message broker, you create another namespace with two applications of the same `app-id`, App1 and App2. + +Dapr creates consumer groups using the `app-id` of individual applications, so the consumer group names will remain App1 and App2. + +Diagram showing complications around multi-tenancy without Dapr namespace consumer groups. + +To avoid this, you'd then need to have something "creep" into your code to change the `app-id`, depending on the namespace on which you're running. This workaround is cumbersome and a significant painpoint. + +## With namespace consumer groups + +Not only can Dapr allow you to change the behavior of a consumer group with a consumerID for your UUID and pod names, Dapr also provides a **namespace construct** that lives in the pub/sub component metadata. For example, using Redis as your message broker: + +```yaml +apiVersion: dapr.io/v1alpha1 +kind: Component +metadata: + name: pubsub +spec: + type: pubsub.redis + version: v1 + metadata: + - name: redisHost + value: localhost:6379 + - name: redisPassword + value: "" + - name: consumerID + value: "{namespace}" +``` + +By configuring `consumerID` with the `{namespace}` value, you'll be able to use the same `app-id` with the same topics from different namespaces. + +Diagram showing how namespace consumer groups help with multi-tenancy. + +In the diagram above, you have two namespaces, each with applications of the same `app-id`, publishing and subscribing to the same centralized message broker `orders`. This time, however, Dapr has created consumer group names prefixed with the namespace in which they're running. + +Without you needing to change your code/`app-id`, the namespace consumer group allows you to: +- Add more namespaces +- Keep the same topics +- Keep the same `app-id` across namespaces +- Have your entire deployment pipeline remain intact + +Simply include the `"{namespace}"` consumer group construct in your component metadata. You don't need to encode the namespace in the metadata. Dapr understands the namespace it is running in and completes the namespace value for you, like a dynamic metadata value injected by the runtime. + +{{% alert title="Note" color="primary" %}} +If you add the namespace consumer group to your metadata afterwards, Dapr updates everything for you. This means that you can add namespace metadata value to existing pub/sub deployments. +{{% /alert %}} + +## Demo + +Watch [this video for an overview on pub/sub multi-tenancy](https://youtu.be/eK463jugo0c?t=1188): + + + +## Next steps + +- Learn more about configuring Pub/Sub components with multiple namespaces [pub/sub namespaces]({{< ref pubsub-namespaces >}}). \ No newline at end of file diff --git a/daprdocs/content/en/developing-applications/building-blocks/pubsub/pubsub-message-ttl.md b/daprdocs/content/en/developing-applications/building-blocks/pubsub/pubsub-message-ttl.md index d6029842c..915661050 100644 --- a/daprdocs/content/en/developing-applications/building-blocks/pubsub/pubsub-message-ttl.md +++ b/daprdocs/content/en/developing-applications/building-blocks/pubsub/pubsub-message-ttl.md @@ -2,7 +2,7 @@ type: docs title: "Message Time-to-Live (TTL)" linkTitle: "Message TTL" -weight: 6000 +weight: 7000 description: "Use time-to-live in pub/sub messages." --- diff --git a/daprdocs/content/en/developing-applications/building-blocks/pubsub/pubsub-overview.md b/daprdocs/content/en/developing-applications/building-blocks/pubsub/pubsub-overview.md index d7d28f3b7..2b90adda6 100644 --- a/daprdocs/content/en/developing-applications/building-blocks/pubsub/pubsub-overview.md +++ b/daprdocs/content/en/developing-applications/building-blocks/pubsub/pubsub-overview.md @@ -32,7 +32,7 @@ When using pub/sub in Dapr: 1. The pub/sub building block makes calls into a Dapr pub/sub component that encapsulates a specific message broker. 1. To receive messages on a topic, Dapr subscribes to the pub/sub component on behalf of your service with a topic and delivers the messages to an endpoint on your service when they arrive. -In the diagram below, a "shipping" service and an "email" service have both subscribed to topics published by a "cart" service. Each service loads pub/sub component configuration files that point to the same pub/sub message bus component; for example: Redis Streams, NATS Streaming, Azure Service Bus, or GCP pub/sub. +In the diagram below, a "shipping" service and an "email" service have both subscribed to topics published by a "cart" service. Each service loads pub/sub component configuration files that point to the same pub/sub message broker component; for example: Redis Streams, NATS Streaming, Azure Service Bus, or GCP pub/sub.

@@ -94,6 +94,10 @@ For more information on message routing, read [Dapr pub/sub API reference]({{< r Sometimes, messages can't be processed because of a variety of possible issues, such as erroneous conditions within the producer or consumer application or an unexpected state change that causes an issue with your application code. Dapr allows developers to set dead letter topics to deal with messages that cannot be delivered to an application. This feature is available on all pub/sub components and prevents consumer applications from endlessly retrying a failed message. For more information, read about [dead letter topics]({{< ref "pubsub-deadletter.md">}}) +### Namespace consumer groups + +Dapr solves multi-tenancy at-scale with [namespaces for consumer groups]({{< ref howto-namespace >}}). Simply include the `"{namespace}"` value in your component metadata for consumer groups to allow multiple namespaces with applications of the same `app-id` to publish and subscribe to the same message broker. + ### At-least-once guarantee Dapr guarantees at-least-once semantics for message delivery. When an application publishes a message to a topic using the pub/sub API, Dapr ensures the message is delivered *at least once* to every subscriber. diff --git a/daprdocs/content/en/developing-applications/building-blocks/pubsub/pubsub-scopes.md b/daprdocs/content/en/developing-applications/building-blocks/pubsub/pubsub-scopes.md index f2dd790c6..0c830f0a8 100644 --- a/daprdocs/content/en/developing-applications/building-blocks/pubsub/pubsub-scopes.md +++ b/daprdocs/content/en/developing-applications/building-blocks/pubsub/pubsub-scopes.md @@ -2,7 +2,7 @@ type: docs title: "Scope Pub/sub topic access" linkTitle: "Scope topic access" -weight: 5000 +weight: 6000 description: "Use scopes to limit pub/sub topics to specific applications" --- diff --git a/daprdocs/content/en/developing-applications/building-blocks/service-invocation/service-invocation-namespaces.md b/daprdocs/content/en/developing-applications/building-blocks/service-invocation/service-invocation-namespaces.md index ce59ef80a..6ee966558 100644 --- a/daprdocs/content/en/developing-applications/building-blocks/service-invocation/service-invocation-namespaces.md +++ b/daprdocs/content/en/developing-applications/building-blocks/service-invocation/service-invocation-namespaces.md @@ -1,7 +1,7 @@ --- type: docs -title: "Service invocation across namespaces" -linkTitle: "Service invocation namespaces" +title: "How to: Service invocation across namespaces" +linkTitle: "How to: Service invocation namespaces" weight: 1000 description: "Call between services deployed to different namespaces" --- diff --git a/daprdocs/static/images/howto-namespace/basic-pubsub.png b/daprdocs/static/images/howto-namespace/basic-pubsub.png new file mode 100644 index 000000000..4f56b0bcb Binary files /dev/null and b/daprdocs/static/images/howto-namespace/basic-pubsub.png differ diff --git a/daprdocs/static/images/howto-namespace/with-namespace.png b/daprdocs/static/images/howto-namespace/with-namespace.png new file mode 100644 index 000000000..f736b47f9 Binary files /dev/null and b/daprdocs/static/images/howto-namespace/with-namespace.png differ diff --git a/daprdocs/static/images/howto-namespace/without-namespace.png b/daprdocs/static/images/howto-namespace/without-namespace.png new file mode 100644 index 000000000..e90346cf0 Binary files /dev/null and b/daprdocs/static/images/howto-namespace/without-namespace.png differ diff --git a/sdkdocs/js b/sdkdocs/js index be0277d81..8bbfdbdcb 160000 --- a/sdkdocs/js +++ b/sdkdocs/js @@ -1 +1 @@ -Subproject commit be0277d815e5dd4e107ddffd1ca21cc4502806a5 +Subproject commit 8bbfdbdcb536faaa338db9fd05bc73fd746c01b9