From 44baf48b7a9f794ac2da8946ef30bfcabec4fbda Mon Sep 17 00:00:00 2001 From: Josh van Leeuwen Date: Fri, 2 Jun 2023 06:18:17 +0100 Subject: [PATCH] Adds TTL documentation to CosmosDB and MySQL (#3471) * Adds TTL documentation to CosmosDB and MySQL Signed-off-by: joshvanl * Remove Microsoft link localization form the CosmosDB MySQL reference docs Signed-off-by: joshvanl * Update daprdocs/content/en/reference/components-reference/supported-state-stores/setup-azure-cosmosdb.md Co-authored-by: Mark Fussell Signed-off-by: Josh van Leeuwen * Update daprdocs/content/en/reference/components-reference/supported-state-stores/setup-mysql.md Co-authored-by: Mark Fussell Signed-off-by: Josh van Leeuwen --------- Signed-off-by: joshvanl Signed-off-by: Josh van Leeuwen Co-authored-by: Mark Fussell --- .../supported-state-stores/setup-azure-cosmosdb.md | 4 ++++ .../supported-state-stores/setup-mysql.md | 12 ++++++++++++ .../supported-state-stores/setup-postgresql.md | 2 +- 3 files changed, 17 insertions(+), 1 deletion(-) diff --git a/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-azure-cosmosdb.md b/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-azure-cosmosdb.md index 2c7a4651c..a1f4f59b9 100644 --- a/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-azure-cosmosdb.md +++ b/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-azure-cosmosdb.md @@ -70,6 +70,10 @@ In order to setup Cosmos DB as a state store, you need the following properties - **Database**: The name of the database - **Collection**: The name of the collection (or container) +### TTLs and cleanups + +This state store supports [Time-To-Live (TTL)]({{< ref state-store-ttl.md >}}) for records stored with Dapr. When storing data using Dapr, you can set the `ttlInSeconds` metadata property to override the default TTL on the CosmodDB container, indicating when the data should be considered "expired". Note that this value _only_ takes effect if the container's `DefaultTimeToLive` field has a non-NULL value. See the [CosmosDB documentation](https://docs.microsoft.com/azure/cosmos-db/nosql/time-to-live) for more information. + ## Best Practices for Production Use Azure Cosmos DB shares a strict metadata request rate limit across all databases in a single Azure Cosmos DB account. New connections to Azure Cosmos DB assume a large percentage of the allowable request rate limit. (See the [Cosmos DB documentation](https://docs.microsoft.com/azure/cosmos-db/sql/troubleshoot-request-rate-too-large#recommended-solution-3)) diff --git a/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-mysql.md b/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-mysql.md index 62c3a430f..65000b4d1 100644 --- a/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-mysql.md +++ b/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-mysql.md @@ -58,6 +58,7 @@ If you wish to use MySQL as an actor store, append the following to the yaml. | `timeoutInSeconds` | N | Timeout for all database operations. Defaults to `20` | `30` | | `pemPath` | N | Full path to the PEM file to use for [enforced SSL Connection](#enforced-ssl-connection) required if pemContents is not provided. Cannot be used in K8s environment | `"/path/to/file.pem"`, `"C:\path\to\file.pem"` | | `pemContents` | N | Contents of PEM file to use for [enforced SSL Connection](#enforced-ssl-connection) required if pemPath is not provided. Can be used in K8s environment | `"pem value"` | +| `cleanupIntervalInSeconds` | N | Interval, in seconds, to clean up rows with an expired TTL. Default: `3600` (that is 1 hour). Setting this to values <=0 disables the periodic cleanup. | `1800`, `-1` ## Setup MySQL @@ -134,6 +135,17 @@ Replace the `` value with your connection string. The connect If your server requires SSL your connection string must end with `&tls=custom` for example, `":@tcp(:3306)/?allowNativePasswords=true&tls=custom"`. You must replace the `` with a full path to the PEM file. The connection to MySQL will require a minimum TLS version of 1.2. +### TTLs and cleanups + +This state store supports [Time-To-Live (TTL)]({{< ref state-store-ttl.md >}}) for records stored with Dapr. When storing data using Dapr, you can set the `ttlInSeconds` metadata property to indicate when the data should be considered "expired". + +Because MySQL doesn't have built-in support for TTLs, this is implemented in Dapr by adding a column in the state table indicating when the data is to be considered "expired". Records that are "expired" are not returned to the caller, even if they're still physically stored in the database. A background "garbage collector" periodically scans the state table for expired rows and deletes them. + +The interval at which the deletion of expired records happens is set with the `cleanupIntervalInSeconds` metadata property, which defaults to 3600 seconds (that is, 1 hour). + +- Longer intervals require less frequent scans for expired rows, but can require storing expired records for longer, potentially requiring more storage space. If you plan to store many records in your state table, with short TTLs, consider setting `cleanupIntervalInSeconds` to a smaller value, for example `300` (300 seconds, or 5 minutes). +- If you do not plan to use TTLs with Dapr and the MySQL state store, you should consider setting `cleanupIntervalInSeconds` to a value <= 0 (e.g. `0` or `-1`) to disable the periodic cleanup and reduce the load on the database. + ## Related links - [Basic schema for a Dapr component]({{< ref component-schema >}}) - Read [this guide]({{< ref "howto-get-save-state.md#step-2-save-and-retrieve-a-single-state" >}}) for instructions on configuring state store components diff --git a/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-postgresql.md b/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-postgresql.md index d8472e840..6e1bfad92 100644 --- a/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-postgresql.md +++ b/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-postgresql.md @@ -92,7 +92,7 @@ Either the default "postgres" database can be used, or create a new database for ### TTLs and cleanups -This state store supports [Time-To-Live (TTL)](https://docs.dapr.io/developing-applications/building-blocks/state-management/state-store-ttl/) for records stored with Dapr. When storing data using Dapr, you can set the `ttlInSeconds` metadata property to indicate after how many seconds the data should be considered "expired". +This state store supports [Time-To-Live (TTL)]({{< ref state-store-ttl.md >}}) for records stored with Dapr. When storing data using Dapr, you can set the `ttlInSeconds` metadata property to indicate after how many seconds the data should be considered "expired". Because PostgreSQL doesn't have built-in support for TTLs, this is implemented in Dapr by adding a column in the state table indicating when the data is to be considered "expired". Records that are "expired" are not returned to the caller, even if they're still physically stored in the database. A background "garbage collector" periodically scans the state table for expired rows and deletes them.