From 08939fc6271e8636449b78a3533a4a0ff87083fd Mon Sep 17 00:00:00 2001 From: Farshad Davoudi Date: Fri, 5 Jan 2024 20:18:00 +0330 Subject: [PATCH 1/4] Update multi-app-template.md (#3943) Fix some mistakes in the doc and try to make it better and more clear Signed-off-by: Farshad Davoudi --- .../multi-app-dapr-run/multi-app-template.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/daprdocs/content/en/developing-applications/local-development/multi-app-dapr-run/multi-app-template.md b/daprdocs/content/en/developing-applications/local-development/multi-app-dapr-run/multi-app-template.md index 2d85efa11..e5c526da4 100644 --- a/daprdocs/content/en/developing-applications/local-development/multi-app-dapr-run/multi-app-template.md +++ b/daprdocs/content/en/developing-applications/local-development/multi-app-dapr-run/multi-app-template.md @@ -162,9 +162,9 @@ apps: The following rules apply for all the paths present in the template file: - If the path is absolute, it is used as is. - - All relative paths under command section should be provided relative to the template file path. + - All relative paths under common section should be provided relative to the template file path. - `appDirPath` under apps section should be provided relative to the template file path. - - All relative paths under app section should be provided relative to the `appDirPath`. + - All other relative paths under apps section should be provided relative to the `appDirPath`. {{% /codetab %}} From 20494df4ec40006b210a18097f5d495f48cb13ab Mon Sep 17 00:00:00 2001 From: Simon Headley Date: Fri, 5 Jan 2024 19:05:14 +0200 Subject: [PATCH 2/4] Specifying the behaviour of the ConsumerID field when configuring Kafka pub/sub (#3935) Signed-off-by: Simon Headley Co-authored-by: Yaron Schneider --- .../supported-pubsub/setup-apache-kafka.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-apache-kafka.md b/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-apache-kafka.md index 05ff83520..4b8aacfe5 100644 --- a/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-apache-kafka.md +++ b/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-apache-kafka.md @@ -58,8 +58,8 @@ spec: | Field | Required | Details | Example | |--------------------|:--------:|---------|---------| | brokers | Y | A comma-separated list of Kafka brokers. | `"localhost:9092,dapr-kafka.myapp.svc.cluster.local:9093"` -| consumerGroup | N | A kafka consumer group to listen on. Each record published to a topic is delivered to one consumer within each consumer group subscribed to the topic. | `"group1"` -| consumerID | N | Consumer ID (consumer tag) organizes one or more consumers into a group. Consumers with the same consumer ID work as one virtual consumer; for example, a message is processed only once by one of the consumers in the group. If the `consumerID` is not provided, the Dapr runtime set it to the Dapr application ID (`appID`) value. | `"channel1"` +| consumerGroup | N | A kafka consumer group to listen on. Each record published to a topic is delivered to one consumer within each consumer group subscribed to the topic. If a value for `consumerGroup` is provided, any value for `consumerID` is ignored - a combination of the consumer group and a random unique identifier will be set for the `consumerID` instead. | `"group1"` +| consumerID | N | Consumer ID (consumer tag) organizes one or more consumers into a group. Consumers with the same consumer ID work as one virtual consumer; for example, a message is processed only once by one of the consumers in the group. If the `consumerID` is not provided, the Dapr runtime set it to the Dapr application ID (`appID`) value. If a value for `consumerGroup` is provided, any value for `consumerID` is ignored - a combination of the consumer group and a random unique identifier will be set for the `consumerID` instead. | `"channel1"` | clientID | N | A user-provided string sent with every request to the Kafka brokers for logging, debugging, and auditing purposes. Defaults to `"namespace.appID"` for Kubernetes mode or `"appID"` for Self-Hosted mode. | `"my-namespace.my-dapr-app"`, `"my-dapr-app"` | authRequired | N | *Deprecated* Enable [SASL](https://en.wikipedia.org/wiki/Simple_Authentication_and_Security_Layer) authentication with the Kafka brokers. | `"true"`, `"false"` | authType | Y | Configure or disable authentication. Supported values: `none`, `password`, `mtls`, or `oidc` | `"password"`, `"none"` From 174c51012b3d3a2a75946f344067db027d647e17 Mon Sep 17 00:00:00 2001 From: Hannah Hunter <94493363+hhunter-ms@users.noreply.github.com> Date: Fri, 5 Jan 2024 20:42:38 -0500 Subject: [PATCH 3/4] [AWS SSM parameter store] Add prefix option (#3920) * add prefix to spec Signed-off-by: Hannah Hunter * add prefix to example Signed-off-by: Hannah Hunter --------- Signed-off-by: Hannah Hunter Co-authored-by: Mark Fussell --- .../supported-secret-stores/aws-parameter-store.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/daprdocs/content/en/reference/components-reference/supported-secret-stores/aws-parameter-store.md b/daprdocs/content/en/reference/components-reference/supported-secret-stores/aws-parameter-store.md index eab410fe1..bc951b50b 100644 --- a/daprdocs/content/en/reference/components-reference/supported-secret-stores/aws-parameter-store.md +++ b/daprdocs/content/en/reference/components-reference/supported-secret-stores/aws-parameter-store.md @@ -30,6 +30,8 @@ spec: value: "[aws_secret_key]" - name: sessionToken value: "[aws_session_token]" + - name: prefix + value: "[secret_name]" ``` {{% alert title="Warning" color="warning" %}} The above example uses secrets as plain strings. It is recommended to use a local secret store such as [Kubernetes secret store]({{< ref kubernetes-secret-store.md >}}) or a [local file]({{< ref file-secret-store.md >}}) to bootstrap secure key storage. @@ -43,6 +45,7 @@ The above example uses secrets as plain strings. It is recommended to use a loca | accessKey | Y | The AWS Access Key to access this resource | `"key"` | | secretKey | Y | The AWS Secret Access Key to access this resource | `"secretAccessKey"` | | sessionToken | N | The AWS session token to use | `"sessionToken"` | +| prefix | N | Allows you to specify more than one SSM parameter store secret store component. | `"prefix"` | {{% alert title="Important" color="warning" %}} When running the Dapr sidecar (daprd) with your application on EKS (AWS Kubernetes), if you're using a node/pod that has already been attached to an IAM policy defining access to AWS resources, you **must not** provide AWS access-key, secret-key, and tokens in the definition of the component spec you're using. From 5c132e6e3d5f61d44286977184d0ff65300b8ac9 Mon Sep 17 00:00:00 2001 From: Simon Headley Date: Tue, 9 Jan 2024 08:02:59 +0200 Subject: [PATCH 4/4] =?UTF-8?q?Adding=20write=20performance=20optimization?= =?UTF-8?q?=20section=20to=20Cosmos=20DB=20state=20stor=E2=80=A6=20(#3934)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Adding write performance optimization section to Cosmos DB state store documentation Signed-off-by: Simon Headley * Applying suggestions as per PR review Signed-off-by: Simon Headley * Switching from Markdown note to {{% alert %}} {$ /alert $}} as per PR review Signed-off-by: Simon Headley * Putting example paragraph in a new line as per review Signed-off-by: Simon Headley --------- Signed-off-by: Simon Headley Co-authored-by: Mark Fussell --- .../setup-azure-cosmosdb.md | 36 +++++++++++++++++++ 1 file changed, 36 insertions(+) diff --git a/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-azure-cosmosdb.md b/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-azure-cosmosdb.md index 0d636a452..d7ee723ea 100644 --- a/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-azure-cosmosdb.md +++ b/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-azure-cosmosdb.md @@ -172,6 +172,42 @@ az cosmosdb sql role assignment create \ --role-definition-id "$ROLE_ID" ``` +## Optimizing Cosmos DB for bulk operation write performance + +If you are building a system that only ever reads data from Cosmos DB via key (`id`), which is the default Dapr behavior when using the state management API or actors, there are ways you can optimize Cosmos DB for improved write speeds. This is done by excluding all paths from indexing. By default, Cosmos DB indexes all fields inside of a document. On systems that are write-heavy and run little-to-no queries on values within a document, this indexing policy slows down the time it takes to write or update a document in Cosmos DB. This is exacerbated in high-volume systems. + +For example, the default Terraform definition for a Cosmos SQL container indexing reads as follows: + +```tf +indexing_policy { + indexing_mode = "consistent" + + included_path { + path = "/*" + } +} +``` + +It is possible to force Cosmos DB to only index the `id` and `partitionKey` fields by excluding all other fields from indexing. This can be done by updating the above to read as follows: + +```tf +indexing_policy { + # This could also be set to "none" if you are using the container purely as a key-value store. This may be applicable if your container is only going to be used as a distributed cache. + indexing_mode = "consistent" + + # Note that included_path has been replaced with excluded_path + excluded_path { + path = "/*" + } +} +``` + +{{% alert title="Note" color="primary" %}} + +This optimization comes at the cost of queries against fields inside of documents within the state store. This would likely impact any stored procedures or SQL queries defined and executed. It is only recommended that this optimization be applied only if you are using the Dapr State Management API or Dapr Actors to interact with Cosmos DB. + +{{% /alert %}} + ## Related links - [Basic schema for a Dapr component]({{< ref component-schema >}})