Updated all the pubsub topics + a few others (#1140)

* Updated all the pubsub topics + otherss

* Changes for Issue #1093

* Pushing updates from review feedback

* Fixing up the Publish CLI example

* Fixing reference

* Fixing having a single place for the Dapr side car

Co-authored-by: Yaron Schneider <yaronsc@microsoft.com>
Co-authored-by: Aaron Crawfis <Aaron.Crawfis@microsoft.com>
Co-authored-by: Ori Zohar <orzohar@microsoft.com>
This commit is contained in:
Mark Fussell 2021-02-02 21:40:06 -08:00 committed by GitHub
parent 706487a474
commit 08d86488bf
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
15 changed files with 257 additions and 80 deletions

View File

@ -12,9 +12,13 @@ Pub/Sub is a common pattern in a distributed system with many services that want
Using Pub/Sub, you can enable scenarios where event consumers are decoupled from event producers.
Dapr provides an extensible Pub/Sub system with At-Least-Once guarantees, allowing developers to publish and subscribe to topics.
Dapr provides different implementation of the underlying system, and allows operators to bring in their preferred infrastructure, for example Redis Streams, Kafka, etc.
Dapr provides components for pub/sub, that enable operators to use their preferred infrastructure, for example Redis Streams, Kafka, etc.
## Step 1: Setup the Pub/Sub component
The following example creates applications to publish and subscribe to a topic called `deathStarStatus`.
<img src="/images/pubsub-publish-subscribe-example.png" width=1000>
<br></br>
The first step is to setup the Pub/Sub component:
@ -70,10 +74,11 @@ spec:
Dapr allows two methods by which you can subscribe to topics:
- **Declaratively**, where subscriptions are are defined in an external file.
- **Programmatically**, where subscriptions are defined in user code
- **Programmatically**, where subscriptions are defined in user code.
{{% alert title="Note" color="primary" %}}
Both declarative and programmatic approaches support the same features. The declarative approach removes the Dapr dependancy from the user code and allows for the use of an existing application to subscribe to topics. The programmatic approach implements the subscription in user code.
Both declarative and programmatic approaches support the same features. The declarative approach removes the Dapr dependency from your code and allows, for example, existing applications to subscribe to topics, without having to change code. The programmatic approach implements the subscription in your code.
{{% /alert %}}
### Declarative subscriptions
@ -102,9 +107,9 @@ Set the component with:
{{< tabs "Self-Hosted (CLI)" Kubernetes>}}
{{% codetab %}}
Place the CRD in your `./components` directory. When Dapr starts up, it will load subscriptions along with components.
Place the CRD in your `./components` directory. When Dapr starts up, it loads subscriptions along with components.
*Note: By default, Dapr loads components from `$HOME/.dapr/components` on MacOS/Linux and `%USERPROFILE%\.dapr\components` on Windows.*
Note: By default, Dapr loads components from `$HOME/.dapr/components` on MacOS/Linux and `%USERPROFILE%\.dapr\components` on Windows.
You can also override the default directory by pointing the Dapr CLI to a components path:
@ -193,10 +198,10 @@ dapr --app-id app2 --app-port 3000 run node app2.js
### Programmatic subscriptions
To subscribe to topics, start a web server in the programming language of your choice and listen on the following `GET` endpoint: `/dapr/subscribe`.
The Dapr instance will call into your app at startup and expect a JSON response for the topic subscriptions with:
- `pubsubname`: Which pub/sub component Dapr should use
- `topic`: Which topic to subscribe to
- `route`: Which endpoint for Dapr to call on when a message comes to that topic
The Dapr instance calls into your app at startup and expect a JSON response for the topic subscriptions with:
- `pubsubname`: Which pub/sub component Dapr should use.
- `topic`: Which topic to subscribe to.
- `route`: Which endpoint for Dapr to call on when a message comes to that topic.
#### Example
@ -279,21 +284,25 @@ The `/dsstatus` endpoint matches the `route` defined in the subscriptions and th
## Step 3: Publish a topic
To publish a message to a topic, invoke the following endpoint on a Dapr instance:
To publish a topic you need to run an instance of a Dapr sidecar to use the pubsub Redis component. You can use the default Redis component installed into your local environment.
Start an instance of Dapr with an app-id called `testpubsub`:
```bash
dapr run --app-id testpubsub --dapr-http-port 3500
```
{{< tabs "Dapr CLI" "HTTP API (Bash)" "HTTP API (PowerShell)">}}
{{% codetab %}}
Then publish a message to the `deathStarStatus` topic:
```bash
dapr publish --pubsub pubsub --topic deathStarStatus --data '{"status": "completed"}'
dapr publish --publish-app-id testpubapp --pubsub pubsub --topic deathStarStatus --data '{"status": "completed"}'
```
{{% /codetab %}}
{{% codetab %}}
Begin by ensuring a Dapr sidecar is running:
```bash
dapr --app-id myapp --port 3500 run
```
Then publish a message to the `deathStarStatus` topic:
```bash
curl -X POST http://localhost:3500/v1.0/publish/pubsub/deathStarStatus -H "Content-Type: application/json" -d '{"status": "completed"}'
@ -301,10 +310,6 @@ curl -X POST http://localhost:3500/v1.0/publish/pubsub/deathStarStatus -H "Conte
{{% /codetab %}}
{{% codetab %}}
Begin by ensuring a Dapr sidecar is running:
```bash
dapr --app-id myapp --port 3500 run
```
Then publish a message to the `deathStarStatus` topic:
```powershell
Invoke-RestMethod -Method Post -ContentType 'application/json' -Body '{"status": "completed"}' -Uri 'http://localhost:3500/v1.0/publish/pubsub/deathStarStatus'
@ -342,7 +347,43 @@ app.post('/dsstatus', (req, res) => {
{{< /tabs >}}
## (Optional) Step 5: Publishing a topic with code
{{< tabs Node>}}
{{% codetab %}}
If you prefer publishing a topic using code, here is an example.
```javascript
const express = require('express');
const path = require('path');
const request = require('request');
const bodyParser = require('body-parser');
const app = express();
app.use(bodyParser.json());
const daprPort = process.env.DAPR_HTTP_PORT || 3500;
const daprUrl = `http://localhost:${daprPort}/v1.0`;
const port = 8080;
const pubsubName = 'pubsub';
app.post('/publish', (req, res) => {
console.log("Publishing: ", req.body);
const publishUrl = `${daprUrl}/publish/${pubsubName}/deathStarStatus`;
request( { uri: publishUrl, method: 'POST', json: req.body } );
res.sendStatus(200);
});
app.listen(process.env.PORT || port, () => console.log(`Listening on port ${port}!`));
```
{{% /codetab %}}
{{< /tabs >}}
## Next steps
- [Scope access to your pub/sub topics]({{< ref pubsub-scopes.md >}})
- [Pub/Sub quickstart](https://github.com/dapr/quickstarts/tree/master/pub-sub)
- [Pub/sub components]({{< ref setup-pubsub >}})
- Try the [Pub/Sub quickstart sample](https://github.com/dapr/quickstarts/tree/master/pub-sub)
- Learn about [topic scoping]({{< ref pubsub-scopes.md >}})
- Learn about [message time-to-live]({{< ref pubsub-message-ttl.md >}})
- Learn [how to configure Pub/Sub components with multiple namespaces]({{< ref pubsub-namespaces.md >}})
- List of [pub/sub components]({{< ref setup-pubsub >}})
- Read the [API reference]({{< ref pubsub_api.md >}})

View File

@ -1,6 +1,6 @@
---
type: docs
title: "Message Time-To-Live"
title: "Message Time-to-Live (TTL)"
linkTitle: "Message TTL"
weight: 6000
description: "Use time-to-live in Pub/Sub messages."
@ -8,7 +8,7 @@ description: "Use time-to-live in Pub/Sub messages."
## Introduction
Dapr enables per-message time-to-live (TTL). This means that applications can set time-to-live per message, and subscribers will not receive those messages after expiration.
Dapr enables per-message time-to-live (TTL). This means that applications can set time-to-live per message, and subscribers do not receive those messages after expiration.
All Dapr [pub/sub components]({{< ref supported-pubsub >}}) are compatible with message TTL, as Dapr handles the TTL logic within the runtime. Simply set the `ttlInSeconds` metadata when publishing a message.
@ -16,19 +16,19 @@ In some components, such as Kafka, time-to-live can be configured in the topic v
## Native message TTL support
When message time-to-live has native support in the component, Dapr will simply forward the time-to-live configuration without adding extra logic, keeping predictable behavior. This is helpful when the expired messages are handled differently by the component - like in Azure Service Bus, where expired messages are stored in the dead letter queue and not simply deleted.
When message time-to-live has native support in the pub/sub component, Dapr simply forwards the time-to-live configuration without adding any extra logic, keeping predictable behavior. This is helpful when the expired messages are handled differently by the component. For example, with Azure Service Bus, where expired messages are stored in the dead letter queue and are not simply deleted.
### Supported components
#### Azure Service Bus
Azure Service Bus supports [entity level time-to-live]((https://docs.microsoft.com/en-us/azure/service-bus-messaging/message-expiration)). It means that messages have a default time-to-live but can also be set with a shorter timespan at publishing time. Dapr will propagate the time-to-live metadata for the message and let Azure Service Bus handle expiration directly.
Azure Service Bus supports [entity level time-to-live]((https://docs.microsoft.com/en-us/azure/service-bus-messaging/message-expiration)). This means that messages have a default time-to-live but can also be set with a shorter timespan at publishing time. Dapr propagates the time-to-live metadata for the message and lets Azure Service Bus handle the expiration directly.
## Non-Dapr subscribers
If messages are consumed by subscribers without Dapr, expired messages are not automatically dropped, as expiration is handled by the Dapr runtime when a Dapr application receives a message. However, subscribers can programmatically drop expired messages by adding logic to handle the `expiration` attribute in the cloud event, which follows the [RFC3339](https://tools.ietf.org/html/rfc3339) format.
If messages are consumed by subscribers not using Dapr, the expired messages are not automatically dropped, as expiration is handled by the Dapr runtime when a Dapr sidecar receives a message. However, subscribers can programmatically drop expired messages by adding logic to handle the `expiration` attribute in the cloud event, which follows the [RFC3339](https://tools.ietf.org/html/rfc3339) format.
When non-Dapr subscribers use components such as Azure Service Bus, which natively handle message TTL, they will not receive expired messages. No extra logic is needed.
When non-Dapr subscribers use components such as Azure Service Bus, which natively handle message TTL, they do not receive expired messages. Here, no extra logic is needed.
## Example
@ -66,7 +66,11 @@ with DaprClient() as d:
{{< /tabs >}}
See [this guide]({{< ref pubsub_api.md >}}) for a full reference on the pub/sub API.
See [this guide]({{< ref pubsub_api.md >}}) for a reference on the pub/sub API.
## Related links
- [Pub/sub API reference]({{< ref pubsub_api.md >}})
- Learn about [topic scoping]({{< ref pubsub-scopes.md >}})
- Learn [how to configure Pub/Sub components with multiple namespaces]({{< ref pubsub-namespaces.md >}})
- List of [pub/sub components]({{< ref supported-pubsub >}})
- Read the [API reference]({{< ref pubsub_api.md >}})

View File

@ -8,19 +8,36 @@ description: "Overview of the Dapr Pub/Sub building block"
## Introduction
The [publish/subscribe pattern](https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern) allows microservices to communicate with each other using messages. The **producer** sends messages to a **topic** without knowledge of what application will receive them. Similarly, a **consumer** will subscribe to the topic and receive its messages without any knowledge of what application produced these messages. This pattern is especially useful when you need to decouple microservices from one another.
The [publish/subscribe pattern](https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern) allows microservices to communicate with each other using messages. The **producer or publisher** sends messages to a **topic** without knowledge of what application will receive them. This involves writing them to an input channel. Similarly, a **consumer or subscriber** subscribes to the topic and receive its messages without any knowledge of what service produced these messages. This involves receiving messages from an output channel. An intermediary message broker is responsible for copying each message from an input channel to an output channels for all subscribers interested in that message. This pattern is especially useful when you need to decouple microservices from one another.
The publish/subscribe API in Dapr provides an at-least-once guarantee and integrates with various message brokers and queuing systems. The specific implementation to your application is pluggable and configured externally at runtime. This approach removes the dependency from your application and, as a result, makes your application more portable. The complete list of available publish/subscribe implementations is available [here]({{< ref supported-pubsub >}}).
The publish/subscribe API in Dapr provides an at-least-once guarantee and integrates with various message brokers and queuing systems. The specific implementation used by your service is pluggable and configured as a Dapr pub/sub component at runtime. This approach removes the dependency from your service and, as a result, makes your service more portable and flexible to changes.
The complete list of Dapr pub/sub components is [here]({{< ref supported-pubsub >}}).
<img src="/images/pubsub-overview-pattern.png" width=1000>
<br></br>
The Dapr pub/sub building block provides a platform-agnostic API to send and receive messages. Your services publish messages to a named topic and also subscribe to a topic to consume the messages.
The service makes a network call to a Dapr pub/sub building block, exposed as a sidecar. This building block then makes calls into a Dapr pub/sub component that encapsulates a specific message broker product. To receive topics, Dapr subscribes to the Dapr pub/sub component on behalf of your service and delivers the messages to an endpoint when they arrive.
The diagram below shows an example of a "shipping" service and an "email" service that have both subscribed to topics that are published by the "cart" service. Each service loads pub/sub component configuration files that point to the same pub/sub message bus component, for example Redis Streams, NATS Streaming, Azure Service Bus, or GCP Pub/Sub.
<img src="/images/pubsub-overview-components.png" width=1000>
<br></br>
The diagram below has the same services, however this time showing the Dapr publish API that sends an "order" topic and order endpoints on the subscribing services that these topic messages are delivered posted to by Dapr.
<img src="/images/pubsub-overview-publish-API.png" width=1000>
<br></br>
## Features
The pub/sub building block provides several features to your application.
### Publish/Subscribe API
### Cloud Events message format
The publish/subscribe API is located in the [API reference]({{< ref pubsub_api.md >}}).
### Message Format
To enable message routing and to provide additional context with each message Dapr uses the [CloudEvents 1.0 specification](https://github.com/cloudevents/spec/tree/v1.0) as its message format. Any message sent by an application to a topic using Dapr will automatically be "wrapped" in Cloud Events envelope, using `Content-Type` header value for `datacontenttype` attribute.
To enable message routing and to provide additional context with each message, Dapr uses the [CloudEvents 1.0 specification](https://github.com/cloudevents/spec/tree/v1.0) as its message format. Any message sent by an application to a topic using Dapr is automatically "wrapped" in a Cloud Events envelope, using `Content-Type` header value for `datacontenttype` attribute.
Dapr implements the following Cloud Events fields:
@ -45,30 +62,54 @@ The following example shows an XML content in CloudEvent v1.0 serialized as JSON
}
```
> Starting with v0.9 release, Dapr no longer wraps published content into CloudEvent if the published payload is already in the CloudEvent format.
### Message subscription
### Message Subscription
Dapr applications can subscribe to published topics. Dapr allows two methods by which your applications can subscribe to topics:
Dapr allows two methods by which you can subscribe to topics: **declarative**, where a subscription is defined in an external file, and **programmatic**, where a subscription is defined in the user code. For more information see Dapr's documentation on [subscribing to a topic](https://docs.dapr.io/developing-applications/building-blocks/pubsub/howto-publish-subscribe/#step-2-subscribe-to-topics).
- **Declarative**, where a subscription is defined in an external file,
- **Programmatic**, where a subscription is defined in the user code.
Both declarative and programmatic approaches support the same features. The declarative approach removes the Dapr dependency from your code and allows for existing applications to subscribe to topics, without having to change code. The programmatic approach implements the subscription in your code.
### Message Delivery
For more information read [How-To: Publish a message and subscribe to a topic]({{< ref howto-publish-subscribe >}}).
In principle, Dapr considers message successfully delivered when the subscriber responds with a non-error response after processing the message. For more granular control, Dapr's publish/subscribe API also provides explicit statuses, defined in the response payload, which the subscriber can use to indicate the specific handling instructions to Dapr (e.g. `RETRY` or `DROP`). For more information message routing see [Dapr publish/subscribe API documentation]({{< ref "pubsub_api.md#provide-routes-for-dapr-to-deliver-topic-events" >}})
### At-Least-Once guarantee
### Message delivery
Dapr guarantees at-least-once semantics for message delivery. That means that when an application publishes a message to a topic using the publish/subscribe API, Dapr ensures that this message will be delivered at least once to every subscriber.
In principle, Dapr considers message successfully delivered when the subscriber responds with a non-error response after processing the message. For more granular control, Dapr's publish/subscribe API also provides explicit statuses, defined in the response payload, which the subscriber can use to indicate the specific handling instructions to Dapr (e.g. `RETRY` or `DROP`). For more information on message routing read [Dapr publish/subscribe API documentation]({{< ref "pubsub_api.md#provide-routes-for-dapr-to-deliver-topic-events" >}})
### Consumer groups and multiple instances
### At-least-once guarantee
The burden of dealing with concepts like consumer groups and multiple application instances using a single consumer group is all handled automatically by Dapr. When multiple instances of the same application (same IDs) subscribe to a topic, Dapr delivers each message to only one instance of that application. Similarly, if two different applications (different IDs) subscribe to the same topic, Dapr will deliver each message to only one instance of each application.
Dapr guarantees "At-Least-Once" semantics for message delivery. That means that when an application publishes a message to a topic using the publish/subscribe API, Dapr ensures that this message will be delivered at least once to every subscriber.
### Consumer groups and competing consumers pattern
The burden of dealing with concepts like consumer groups and multiple application instances using a single consumer group is all handled automatically by Dapr. When multiple instances of the same application (running same app-IDs) subscribe to a topic, Dapr delivers each message to *only one instance of **that** application*. This is commonly known as the competing consumers pattern and is illustrated in the diagram below.
<img src="/images/pubsub-overview-pattern-competing-consumers.png" width=1000>
<br></br>
Similarly, if two different applications (different app-IDs) subscribe to the same topic, Dapr deliver each message to *only one instance of **each** application*.
### Topic scoping
By default, all topics backing the Dapr publish/subscribe component (e.g. Kafka, Redis, RabbitMQ) are available to every application configured with that component. To limit which application can publish or subscribe to topics, Dapr provides topic scoping. See [publish/subscribe topic scoping]({{< ref pubsub-scopes.md >}}) for more information.
By default, all topics backing the Dapr pub/sub component (e.g. Kafka, Redis Stream, RabbitMQ) are available to every application configured with that component. To limit which application can publish or subscribe to topics, Dapr provides topic scoping. This enables to you say which topics an application is allowed to published and which topics an application is allowed to subscribed to. For more information read [publish/subscribe topic scoping]({{< ref pubsub-scopes.md >}}).
### Message Time-to-Live (TTL)
Dapr can set a timeout message on a per message basis, meaning that if the message is not read from the pub/sub component, then the message is discarded. This is to prevent the build up of messages that are not read. A message that has been in the queue for longer than the configured TTL is said to be dead. For more information read [publish/subscribe message time-to-live]({{< ref pubsub-message-ttl.md >}}).
- Note: Message TTL can also be set for a given queue at the time of component creation. Look at the specific characteristic of the component that you are using.
### Publish/Subscribe API
The publish/subscribe API is located in the [API reference]({{< ref pubsub_api.md >}}).
## Next steps
- Read the How-To guide on [publishing and subscribing]({{< ref howto-publish-subscribe.md >}})
- Learn about [Pub/Sub scopes]({{< ref pubsub-scopes.md >}})
- Try the [Pub/Sub quickstart sample](https://github.com/dapr/quickstarts/tree/master/pub-sub)
- Read the [guide on publishing and subscribing]({{< ref howto-publish-subscribe.md >}})
- Learn about [topic scoping]({{< ref pubsub-scopes.md >}})
- Learn about [message time-to-live]({{< ref pubsub-message-ttl.md >}})
- Learn [how to configure Pub/Sub components with multiple namespaces]({{< ref pubsub-namespaces.md >}})
- List of [pub/sub components]({{< ref supported-pubsub >}})
- Read the [API reference]({{< ref pubsub_api.md >}})

View File

@ -158,4 +158,11 @@ The table below shows which application is allowed to subscribe to the topics:
## Demo
<iframe width="560" height="315" src="https://www.youtube.com/embed/7VdWBBGcbHQ?start=513" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
<iframe width="560" height="315" src="https://www.youtube.com/embed/7VdWBBGcbHQ?start=513" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
## Related links
- Learn [how to configure Pub/Sub components with multiple namespaces]({{< ref pubsub-namespaces.md >}})
- Learn about [message time-to-live]({{< ref pubsub-message-ttl.md >}})
- List of [pub/sub components]({{< ref supported-pubsub >}})
- Read the [API reference]({{< ref pubsub_api.md >}})

View File

@ -8,11 +8,49 @@ description: "Limit component access to particular Dapr instances"
Dapr components are namespaced (separate from the Kubernetes namespace concept), meaning a Dapr runtime instance can only access components that have been deployed to the same namespace.
When Dapr runs, it matches it's own configured namespace with the namespace of the components that it loads and initializes only the ones matching its namespaces. All other components in a different namespace are not loaded.
## Namespaces
Namespaces can be used to limit component access to particular Dapr instances.
### Example of component namespacing in Kubernetes
{{< tabs "Self-Hosted" "Kubernetes">}}
{{% codetab %}}
In self hosted mode, a developer can specify the namespace to a Dapr instance by setting the `NAMESPACE` environment variable.
If the `NAMESPACE` environment variable is set, Dapr does not load any component that does not specify the same namespace in its metadata.
For example given this component in the `production` namespace
```yaml
apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
name: statestore
namespace: production
spec:
type: state.redis
version: v1
metadata:
- name: redisHost
value: redis-master:6379
```
To tell Dapr which namespace it is deployed to, set the environment variable:
MacOS/Linux:
```bash
export NAMESPACE=production
# run Dapr as usual
```
Windows:
```powershell
setx NAMESPACE "production"
# run Dapr as usual
```
{{% /codetab %}}
{{% codetab %}}
Let's consider the following component in Kubernetes:
```yaml
@ -30,32 +68,30 @@ spec:
```
In this example, the Redis component is only accessible to Dapr instances running inside the `production` namespace.
{{% /codetab %}}
### Example of component namespacing in self-hosted mode
{{< /tabs >}}
In self hosted mode, a developer can specify the namespace to a Dapr instance by setting the `NAMESPACE` environment variable.
If the `NAMESPACE` environment variable is set, Dapr will not load any component that does not specify the same namespace in its metadata.
## Using namespaces with service invocation
Considering the example above, to tell Dapr which namespace it is deployed to, set the environment variable:
MacOS/Linux:
When using service invocation an application in a namespace you have to qualify it with the namespace. For example calling the `ping` method on `myapp` which is scoped to the `production` namespace would be like this.
```bash
export NAMESPACE=production
# run Dapr as usual
https://localhost:3500/v1.0/invoke/myapp.production/method/ping
```
Windows:
Or using a curl command from an external DNS address, in this case `api.demo.dapr.team` would be like this.
```powershell
setx NAMESPACE "production"
# run Dapr as usual
MacOS/Linux:
```
curl -i -d '{ "message": "hello" }' \
-H "Content-type: application/json" \
-H "dapr-api-token: ${API_TOKEN}" \
https://api.demo.dapr.team/v1.0/invoke/myapp.production/method/ping
```
When Dapr runs, it matches it's own configured namespace with the namespace of the components that it loads and initializes only the the one matching its namespaces. All other components in a different namespace are not loaded.
## Using namespaces with pub/sub
Read [Pub/Sub and namespaces]({{< ref "component-scopes.md" >}}) for more information on scoping components.
## Application access to components with scopes
@ -84,3 +120,9 @@ scopes:
## Example
<iframe width="560" height="315" src="https://www.youtube.com/embed/8W-iBDNvCUM?start=1763" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
## Related links
- [Configure Pub/Sub components with multiple namespaces]({{< ref "pubsub-namespaces.md" >}})
- [Use secret scoping]({{< ref "secrets-scopes.md" >}})
- [Limit the secrets that can be read from secret stores]({{< ref "secret-scope.md" >}})

View File

@ -96,6 +96,11 @@ The following example shows you how to create a Kubernetes secret to hold the co
```bash
kubectl apply -f ./eventhubs.yaml
```
## Scoping access to secrets
Dapr can restrict access to secrets in a secret store using its configuration. Read [How To: Use secret scoping]({{< ref "secrets-scopes.md" >}}) and [How-To: Limit the secrets that can be read from secret stores]({{< ref "secret-scope.md" >}}) for more information. This is the recommended way to limit access to secrets using Dapr.
## Kubernetes permissions
### Default namespace
@ -138,3 +143,8 @@ These resources grant Dapr permissions to get secrets from the Kubernetes secret
{{% alert title="Note" color="warning" %}}
In production scenario to limit Dapr's access to certain secret resources alone, you can use the `resourceNames` field. See this [link](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#referring-to-resources) for further explanation.
{{% /alert %}}
## Related links
- [Use secret scoping]({{< ref "secrets-scopes.md" >}})
- [Limit the secrets that can be read from secret stores]({{< ref "secret-scope.md" >}})

View File

@ -1,14 +1,23 @@
---
type: docs
title: "Pub/Sub and namespaces"
linkTitle: "Kubernetes namespaces"
title: "HowTo: Configure Pub/Sub components with multiple namespaces"
linkTitle: "Multiple namespaces"
weight: 20000
description: "Use Dapr Pub/Sub with multiple namespaces"
---
In some scenarios, applications can be spread across namespaces and share a queue or topic via PubSub. In this case, the PubSub component must be provisioned on each namespace.
This example uses the [PubSub sample](https://github.com/dapr/quickstarts/tree/master/pub-sub). The Redis installation and the subscribers are in `namespace-a` while the publisher UI is in `namespace-b`. This solution should also work if Redis is installed on another namespace or if you use a managed cloud service like Azure ServiceBus.
{{% alert title="Note" color="primary" %}}
Namespaces are a Dapr concept used for scoping applications and components. This example uses Kubernetes namespaces, however the Dapr component namespace scoping can be used on any supported platform. Read [How-To: Scope components to one or more applications]({{< ref "component-scopes.md" >}}) for more information on scoping components.
{{% /alert %}}
This example uses the [PubSub sample](https://github.com/dapr/quickstarts/tree/master/pub-sub). The Redis installation and the subscribers are in `namespace-a` while the publisher UI is in `namespace-b`. This solution will also work if Redis is installed on another namespace or if you use a managed cloud service like Azure ServiceBus, AWS SNS/SQS or GCP PubSub.
This is a diagram of the example using namespaces.
<img src="/images/pubsub-multiple-namespaces.png" width=1000>
<br></br>
The table below shows which resources are deployed to which namespaces:
@ -23,8 +32,8 @@ The table below shows which resources are deployed to which namespaces:
## Pre-requisites
* [Dapr installed](https://github.com/dapr/docs/blob/master/getting-started/environment-setup.md) on any namespace since Dapr works at the cluster level.
* Checkout and cd into directory for [PubSub sample](https://github.com/dapr/quickstarts/tree/master/pub-sub).
* [Dapr installed on Kubernetes]({{< ref "kubernetes-deploy.md" >}}) in any namespace since Dapr works at the cluster level.
* Checkout and cd into the directory for [PubSub quickstart](https://github.com/dapr/quickstarts/tree/master/pub-sub).
## Setup `namespace-a`
@ -110,4 +119,10 @@ kubectl delete -f deploy/redis.yaml --namespace namespace-b
kubectl config set-context --current --namespace=default
kubectl delete namespace namespace-a
kubectl delete namespace namespace-b
```
```
## Related links
- [Scope components to one or more applications]({{< ref "component-scopes.md" >}})
- [Use secret scoping]({{< ref "secrets-scopes.md" >}})
- [Limit the secrets that can be read from secret stores]({{< ref "secret-scope.md" >}})

View File

@ -2,14 +2,16 @@
type: docs
title: "Overview"
linkTitle: "Overview"
description: "General overview on set up of message brokers for Dapr Pub/Sub"
description: "Overview on setting up of pub/sub components for Dapr"
weight: 10000
type: docs
---
Dapr integrates with pub/sub message buses to provide apps with the ability to create event-driven, loosely coupled architectures where producers send events to consumers via topics. It supports the configuration of multiple, named, pub/sub components *per application*. Each pub/sub component has a name and this name is used when publishing a message topic
Dapr integrates with pub/sub message buses to provide applications with the ability to create event-driven, loosely coupled architectures where producers send events to consumers via topics.
Pub/Sub message buses are extensible and can be found in the [components-contrib repo](https://github.com/dapr/components-contrib).
Dapr supports the configuration of multiple, named, pub/sub components *per application*. Each pub/sub component has a name and this name is used when publishing a message topic. Read the [API reference]({{< ref pubsub_api.md >}}) for details on how to publish and subscribe to topics.
Pub/sub components are extensible. A list of support pub/sub components is [here]({{< ref supported-pubsub >}}) and the implementations can be found in the [components-contrib repo](https://github.com/dapr/components-contrib).
A pub/sub in Dapr is described using a `Component` file:
@ -30,11 +32,18 @@ spec:
...
```
The type of pub/sub is determined by the `type` field, and things like connection strings and other metadata are put in the `.metadata` section.
The type of pub/sub is determined by the `type` field, and properties such as connection strings and other metadata are put in the `.metadata` section.
Even though you can put plain text secrets in there, it is recommended you use a [secret store]({{< ref component-secrets.md >}}) using a `secretKeyRef`
Visit [this guide]({{< ref "howto-publish-subscribe.md#step-2-publish-a-topic" >}}) for instructions on configuring pub/sub components.
Visit [this guide]({{< ref "howto-publish-subscribe.md#step-2-publish-a-topic" >}}) for instructions on configuring and using pub/sub components.
## Related links
- [Supported pub/sub components]({{< ref supported-pubsub >}})
- Overview of the Dapr [Pub/Sub building block]({{< ref pubsub-overview.md >}})
- Try the [Pub/Sub quickstart sample](https://github.com/dapr/quickstarts/tree/master/pub-sub)
- Read the [guide on publishing and subscribing]({{< ref howto-publish-subscribe.md >}})
- Learn about [topic scoping]({{< ref pubsub-scopes.md >}})
- Learn about [message time-to-live]({{< ref pubsub-message-ttl.md >}})
- Learn [how to configure Pub/Sub components with multiple namespaces]({{< ref pubsub-namespaces.md >}})
- List of [pub/sub components]({{< ref supported-pubsub >}})
- Read the [API reference]({{< ref pubsub_api.md >}})

View File

@ -195,3 +195,11 @@ The following example shows how to set the Host IP env var to `127.0.0.1`:
```bash
export DAPR_HOST_IP=127.0.0.1
```
## None of my components are getting loaded when my application starts. I keep getting "Error component X cannot be found"
This is usually due to one of the following issues
- You may have defined the `NAMESPACE` environment variable locally or deployed your components into a different namespace in Kubernetes. Check which namespace your app and the components are deployed to. Read [scoping components to one or more applications]({{< ref "component-scopes.md" >}}) for more information.
- You may have not provided a `--components-path` with the Dapr `run` commands or not placed your components into the default components folder for your OS. Read [define a component]({{< ref "get-started-component.md" >}}) for more information.
- You may have a syntax issue in component YAML file. Check your component YAML with the component [YAML samples]({{< ref "components.md" >}}).

Binary file not shown.

After

Width:  |  Height:  |  Size: 54 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 81 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 28 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 45 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 88 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 42 KiB