From f0765662a0849c6aa95c326434e82db16a7decfb Mon Sep 17 00:00:00 2001 From: "Alessandro (Ale) Segala" <43508+ItalyPaleAle@users.noreply.github.com> Date: Wed, 11 Oct 2023 14:45:13 -0700 Subject: [PATCH 1/2] Update etcd version in table (#3809) * Update etcd version in table Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Update daprdocs/data/components/state_stores/generic.yaml Co-authored-by: Mark Fussell Signed-off-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com> --------- Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> Signed-off-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com> Co-authored-by: Mark Fussell --- daprdocs/data/components/state_stores/generic.yaml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/daprdocs/data/components/state_stores/generic.yaml b/daprdocs/data/components/state_stores/generic.yaml index a4ffcd52c..e0b685b64 100644 --- a/daprdocs/data/components/state_stores/generic.yaml +++ b/daprdocs/data/components/state_stores/generic.yaml @@ -45,8 +45,8 @@ - component: etcd link: setup-etcd state: Beta - version: v1 - since: "1.11" + version: v2 + since: "1.12" features: crud: true transactions: true From 74cf5ce84a4164a6defc2812a61f7a97783acd00 Mon Sep 17 00:00:00 2001 From: "Alessandro (Ale) Segala" <43508+ItalyPaleAle@users.noreply.github.com> Date: Wed, 11 Oct 2023 15:05:59 -0700 Subject: [PATCH 2/2] Updated docs for postgres and mysql components (#3811) * Updated docs for postgres and mysql components Fixes #3602 Also includes docs for https://github.com/dapr/components-contrib/pull/2975 Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> * Update mysql.md * Update postgresql.md * Update daprdocs/content/en/reference/components-reference/supported-bindings/mysql.md Signed-off-by: Mark Fussell * Update postgresql.md * Update mysql.md --------- Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> Signed-off-by: Mark Fussell Co-authored-by: Artur Souza Co-authored-by: Mark Fussell --- .../supported-bindings/mysql.md | 44 ++++- .../supported-bindings/postgresql.md | 79 ++++++-- .../postgresql-configuration-store.md | 183 ++++++++++++------ .../setup-postgresql.md | 91 +++++---- 4 files changed, 274 insertions(+), 123 deletions(-) diff --git a/daprdocs/content/en/reference/components-reference/supported-bindings/mysql.md b/daprdocs/content/en/reference/components-reference/supported-bindings/mysql.md index d03dcfcab..3c44b53a8 100644 --- a/daprdocs/content/en/reference/components-reference/supported-bindings/mysql.md +++ b/daprdocs/content/en/reference/components-reference/supported-bindings/mysql.md @@ -60,16 +60,23 @@ Note that you can not use secret just for username/password. If you use secret, ### SSL connection If your server requires SSL your connection string must end of `&tls=custom` for example: + ```bash ":@tcp(:3306)/?allowNativePasswords=true&tls=custom" ``` - You must replace the `` with a full path to the PEM file. If you are using [MySQL on Azure](http://bit.ly/AzureMySQLSSL) see the Azure [documentation on SSL database connections](http://bit.ly/MySQLSSL), for information on how to download the required certificate. The connection to MySQL will require a minimum TLS version of 1.2. -Also note that by default [MySQL go driver](https://github.com/go-sql-driver/mysql) only supports one SQL statement per query/command. +> You must replace the `` with a full path to the PEM file. If you are using Azure Database for MySQL see the Azure [documentation on SSL database connections](https://learn.microsoft.com/azure/mysql/single-server/how-to-configure-ssl), for information on how to download the required certificate. The connection to MySQL requires a minimum TLS version of 1.2. + +### Multiple statements + +By default, the [MySQL Go driver](https://github.com/go-sql-driver/mysql) only supports one SQL statement per query/command. + To allow multiple statements in one query you need to add `multiStatements=true` to a query string, for example: + ```bash ":@tcp(:3306)/?multiStatements=true" ``` + While this allows batch queries, it also greatly increases the risk of SQL injections. Only the result of the first query is returned, all other results are silently discarded. @@ -81,17 +88,35 @@ This component supports **output binding** with the following operations: - `query` - `close` +### Parametrized queries + +This binding supports parametrized queries, which allow separating the SQL query itself from user-supplied values. The usage of parametrized queries is **strongly recommended** for security reasons, as they prevent [SQL Injection attacks](https://owasp.org/www-community/attacks/SQL_Injection). + +For example: + +```sql +-- ❌ WRONG! Includes values in the query and is vulnerable to SQL Injection attacks. +SELECT * FROM mytable WHERE user_key = 'something'; + +-- ✅ GOOD! Uses parametrized queries. +-- This will be executed with parameters ["something"] +SELECT * FROM mytable WHERE user_key = ?; +``` + ### exec The `exec` operation can be used for DDL operations (like table creation), as well as `INSERT`, `UPDATE`, `DELETE` operations which return only metadata (e.g. number of affected rows). +The `params` property is a string containing a JSON-encoded array of parameters. + **Request** ```json { "operation": "exec", "metadata": { - "sql": "INSERT INTO foo (id, c1, ts) VALUES (1, 'demo', '2020-09-24T11:45:05Z07:00')" + "sql": "INSERT INTO foo (id, c1, ts) VALUES (?, ?, ?)", + "params": "[1, \"demo\", \"2020-09-24T11:45:05Z07:00\"]" } } ``` @@ -106,7 +131,7 @@ The `exec` operation can be used for DDL operations (like table creation), as we "start-time": "2020-09-24T11:13:46.405097Z", "end-time": "2020-09-24T11:13:46.414519Z", "rows-affected": "1", - "sql": "INSERT INTO foo (id, c1, ts) VALUES (1, 'demo', '2020-09-24T11:45:05Z07:00')" + "sql": "INSERT INTO foo (id, c1, ts) VALUES (?, ?, ?)" } } ``` @@ -115,13 +140,16 @@ The `exec` operation can be used for DDL operations (like table creation), as we The `query` operation is used for `SELECT` statements, which returns the metadata along with data in a form of an array of row values. +The `params` property is a string containing a JSON-encoded array of parameters. + **Request** ```json { "operation": "query", "metadata": { - "sql": "SELECT * FROM foo WHERE id < 3" + "sql": "SELECT * FROM foo WHERE id < $1", + "params": "[3]" } } ``` @@ -135,7 +163,7 @@ The `query` operation is used for `SELECT` statements, which returns the metadat "duration": "432µs", "start-time": "2020-09-24T11:13:46.405097Z", "end-time": "2020-09-24T11:13:46.420566Z", - "sql": "SELECT * FROM foo WHERE id < 3" + "sql": "SELECT * FROM foo WHERE id < ?" }, "data": [ {column_name: value, column_name: value, ...}, @@ -150,7 +178,7 @@ or numbers (language specific data type) ### close -Finally, the `close` operation can be used to explicitly close the DB connection and return it to the pool. This operation doesn't have any response. +The `close` operation can be used to explicitly close the DB connection and return it to the pool. This operation doesn't have any response. **Request** @@ -160,8 +188,6 @@ Finally, the `close` operation can be used to explicitly close the DB connection } ``` -> Note, the MySQL binding itself doesn't prevent SQL injection, like with any database application, validate the input before executing query. - ## Related links - [Basic schema for a Dapr component]({{< ref component-schema >}}) diff --git a/daprdocs/content/en/reference/components-reference/supported-bindings/postgresql.md b/daprdocs/content/en/reference/components-reference/supported-bindings/postgresql.md index 31c9f230c..cfaf92ad3 100644 --- a/daprdocs/content/en/reference/components-reference/supported-bindings/postgresql.md +++ b/daprdocs/content/en/reference/components-reference/supported-bindings/postgresql.md @@ -22,10 +22,11 @@ spec: type: bindings.postgresql version: v1 metadata: - - name: url # Required - value: "" - - name: direction - value: "" + # Connection string + - name: connectionString + value: "" + - name: direction + value: "" ``` {{% alert title="Warning" color="warning" %}} @@ -34,25 +35,48 @@ The above example uses secrets as plain strings. It is recommended to use a secr ## Spec metadata fields -| Field | Required | Binding support | Details | Example | -|--------------------|:--------:|------------|-----|---------| -| `url` | Y | Output | PostgreSQL connection string See [here](#url-format) for more details | `"user=dapr password=secret host=dapr.example.com port=5432 dbname=dapr sslmode=verify-ca"` | -| `direction` | N | Output | The direction of the binding | `"output"` | +### Authenticate using a connection string + +The following metadata options are **required** to authenticate using a PostgreSQL connection string. + +| Field | Required | Details | Example | +|--------|:--------:|---------|---------| +| `connectionString` | Y | The connection string for the PostgreSQL database. See the PostgreSQL [documentation on database connections](https://www.postgresql.org/docs/current/libpq-connect.html) for information on how to define a connection string. | `"host=localhost user=postgres password=example port=5432 connect_timeout=10 database=my_db"` + +### Authenticate using Azure AD + +Authenticating with Azure AD is supported with Azure Database for PostgreSQL. All authentication methods supported by Dapr can be used, including client credentials ("service principal") and Managed Identity. + +| Field | Required | Details | Example | +|--------|:--------:|---------|---------| +| `useAzureAD` | Y | Must be set to `true` to enable the component to retrieve access tokens from Azure AD. | `"true"` | +| `connectionString` | Y | The connection string for the PostgreSQL database.
This must contain the user, which corresponds to the name of the user created inside PostgreSQL that maps to the Azure AD identity; this is often the name of the corresponding principal (e.g. the name of the Azure AD application). This connection string should not contain any password. | `"host=mydb.postgres.database.azure.com user=myapplication port=5432 database=my_db sslmode=require"` | +| `azureTenantId` | N | ID of the Azure AD tenant | `"cd4b2887-304c-…"` | +| `azureClientId` | N | Client ID (application ID) | `"c7dd251f-811f-…"` | +| `azureClientSecret` | N | Client secret (application password) | `"Ecy3X…"` | + +### Other metadata options + +| Field | Required | Binding support |Details | Example | +|--------------------|:--------:|-----|---|---------| +| `maxConns` | N | Output | Maximum number of connections pooled by this component. Set to 0 or lower to use the default value, which is the greater of 4 or the number of CPUs. | `"4"` +| `connectionMaxIdleTime` | N | Output | Max idle time before unused connections are automatically closed in the connection pool. By default, there's no value and this is left to the database driver to choose. | `"5m"` +| `queryExecMode` | N | Output | Controls the default mode for executing queries. By default Dapr uses the extended protocol and automatically prepares and caches prepared statements. However, this may be incompatible with proxies such as PGBouncer. In this case it may be preferrable to use `exec` or `simple_protocol`. | `"simple_protocol"` ### URL format -The PostgreSQL binding uses [pgx connection pool](https://github.com/jackc/pgx) internally so the `url` parameter can be any valid connection string, either in a `DSN` or `URL` format: +The PostgreSQL binding uses [pgx connection pool](https://github.com/jackc/pgx) internally so the `connectionString` parameter can be any valid connection string, either in a `DSN` or `URL` format: **Example DSN** ```shell -user=dapr password=secret host=dapr.example.com port=5432 dbname=dapr sslmode=verify-ca +user=dapr password=secret host=dapr.example.com port=5432 dbname=my_dapr sslmode=verify-ca ``` **Example URL** ```shell -postgres://dapr:secret@dapr.example.com:5432/dapr?sslmode=verify-ca +postgres://dapr:secret@dapr.example.com:5432/my_dapr?sslmode=verify-ca ``` Both methods also support connection pool configuration variables: @@ -72,17 +96,35 @@ This component supports **output binding** with the following operations: - `query` - `close` +### Parametrized queries + +This binding supports parametrized queries, which allow separating the SQL query itself from user-supplied values. The usage of parametrized queries is **strongly recommended** for security reasons, as they prevent [SQL Injection attacks](https://owasp.org/www-community/attacks/SQL_Injection). + +For example: + +```sql +-- ❌ WRONG! Includes values in the query and is vulnerable to SQL Injection attacks. +SELECT * FROM mytable WHERE user_key = 'something'; + +-- ✅ GOOD! Uses parametrized queries. +-- This will be executed with parameters ["something"] +SELECT * FROM mytable WHERE user_key = $1; +``` + ### exec The `exec` operation can be used for DDL operations (like table creation), as well as `INSERT`, `UPDATE`, `DELETE` operations which return only metadata (e.g. number of affected rows). +The `params` property is a string containing a JSON-encoded array of parameters. + **Request** ```json { "operation": "exec", "metadata": { - "sql": "INSERT INTO foo (id, c1, ts) VALUES (1, 'demo', '2020-09-24T11:45:05Z07:00')" + "sql": "INSERT INTO foo (id, c1, ts) VALUES ($1, $2, $3)", + "params": "[1, \"demo\", \"2020-09-24T11:45:05Z07:00\"]" } } ``` @@ -97,7 +139,7 @@ The `exec` operation can be used for DDL operations (like table creation), as we "start-time": "2020-09-24T11:13:46.405097Z", "end-time": "2020-09-24T11:13:46.414519Z", "rows-affected": "1", - "sql": "INSERT INTO foo (id, c1, ts) VALUES (1, 'demo', '2020-09-24T11:45:05Z07:00')" + "sql": "INSERT INTO foo (id, c1, ts) VALUES ($1, $2, $3)" } } ``` @@ -106,13 +148,16 @@ The `exec` operation can be used for DDL operations (like table creation), as we The `query` operation is used for `SELECT` statements, which returns the metadata along with data in a form of an array of row values. +The `params` property is a string containing a JSON-encoded array of parameters. + **Request** ```json { "operation": "query", "metadata": { - "sql": "SELECT * FROM foo WHERE id < 3" + "sql": "SELECT * FROM foo WHERE id < $1", + "params": "[3]" } } ``` @@ -126,7 +171,7 @@ The `query` operation is used for `SELECT` statements, which returns the metadat "duration": "432µs", "start-time": "2020-09-24T11:13:46.405097Z", "end-time": "2020-09-24T11:13:46.420566Z", - "sql": "SELECT * FROM foo WHERE id < 3" + "sql": "SELECT * FROM foo WHERE id < $1" }, "data": "[ [0,\"test-0\",\"2020-09-24T04:13:46Z\"], @@ -138,7 +183,7 @@ The `query` operation is used for `SELECT` statements, which returns the metadat ### close -Finally, the `close` operation can be used to explicitly close the DB connection and return it to the pool. This operation doesn't have any response. +The `close` operation can be used to explicitly close the DB connection and return it to the pool. This operation doesn't have any response. **Request** @@ -148,8 +193,6 @@ Finally, the `close` operation can be used to explicitly close the DB connection } ``` -> Note, the PostgreSQL binding itself doesn't prevent SQL injection, like with any database application, validate the input before executing query. - ## Related links - [Basic schema for a Dapr component]({{< ref component-schema >}}) diff --git a/daprdocs/content/en/reference/components-reference/supported-configuration-stores/postgresql-configuration-store.md b/daprdocs/content/en/reference/components-reference/supported-configuration-stores/postgresql-configuration-store.md index 43b9820e0..15fa476ae 100644 --- a/daprdocs/content/en/reference/components-reference/supported-configuration-stores/postgresql-configuration-store.md +++ b/daprdocs/content/en/reference/components-reference/supported-configuration-stores/postgresql-configuration-store.md @@ -21,13 +21,36 @@ spec: type: configuration.postgresql version: v1 metadata: - - name: connectionString - value: "host=localhost user=postgres password=example port=5432 connect_timeout=10 database=config" - - name: table # name of the table which holds configuration information - value: "[your_configuration_table_name]" - - name: connMaxIdleTime # max timeout for connection - value : "15s" - + # Connection string + - name: connectionString + value: "host=localhost user=postgres password=example port=5432 connect_timeout=10 database=config" + # Name of the table which holds configuration information + - name: table + value: "[your_configuration_table_name]" + # Timeout for database operations, in seconds (optional) + #- name: timeoutInSeconds + # value: 20 + # Name of the table where to store the state (optional) + #- name: tableName + # value: "state" + # Name of the table where to store metadata used by Dapr (optional) + #- name: metadataTableName + # value: "dapr_metadata" + # Cleanup interval in seconds, to remove expired rows (optional) + #- name: cleanupIntervalInSeconds + # value: 3600 + # Maximum number of connections pooled by this component (optional) + #- name: maxConns + # value: 0 + # Max idle time for connections before they're closed (optional) + #- name: connectionMaxIdleTime + # value: 0 + # Controls the default mode for executing queries. (optional) + #- name: queryExecMode + # value: "" + # Uncomment this if you wish to use PostgreSQL as a state store for actors (optional) + #- name: actorStateStore + # value: "true" ``` {{% alert title="Warning" color="warning" %}} @@ -36,69 +59,101 @@ The above example uses secrets as plain strings. It is recommended to use a secr ## Spec metadata fields -| Field | Required | Details | Example | +### Authenticate using a connection string + +The following metadata options are **required** to authenticate using a PostgreSQL connection string. + +| Field | Required | Details | Example | +|--------|:--------:|---------|---------| +| `connectionString` | Y | The connection string for the PostgreSQL database. See the PostgreSQL [documentation on database connections](https://www.postgresql.org/docs/current/libpq-connect.html) for information on how to define a connection string. | `"host=localhost user=postgres password=example port=5432 connect_timeout=10 database=my_db"` + +### Authenticate using Azure AD + +Authenticating with Azure AD is supported with Azure Database for PostgreSQL. All authentication methods supported by Dapr can be used, including client credentials ("service principal") and Managed Identity. + +| Field | Required | Details | Example | +|--------|:--------:|---------|---------| +| `useAzureAD` | Y | Must be set to `true` to enable the component to retrieve access tokens from Azure AD. | `"true"` | +| `connectionString` | Y | The connection string for the PostgreSQL database.
This must contain the user, which corresponds to the name of the user created inside PostgreSQL that maps to the Azure AD identity; this is often the name of the corresponding principal (e.g. the name of the Azure AD application). This connection string should not contain any password. | `"host=mydb.postgres.database.azure.com user=myapplication port=5432 database=my_db sslmode=require"` | +| `azureTenantId` | N | ID of the Azure AD tenant | `"cd4b2887-304c-…"` | +| `azureClientId` | N | Client ID (application ID) | `"c7dd251f-811f-…"` | +| `azureClientSecret` | N | Client secret (application password) | `"Ecy3X…"` | + +### Other metadata options + +| Field | Required | Details | Example | |--------------------|:--------:|---------|---------| -| connectionString | Y | The connection string for PostgreSQL. Default pool_max_conns = 5 | `"host=localhost user=postgres password=example port=5432 connect_timeout=10 database=dapr_test pool_max_conns=10"` -| table | Y | Table name for configuration information, must be lowercased. | `configtable` +| `table` | Y | Table name for configuration information, must be lowercased. | `configtable` +| `maxConns` | N | Maximum number of connections pooled by this component. Set to 0 or lower to use the default value, which is the greater of 4 or the number of CPUs. | `"4"` +| `connectionMaxIdleTime` | N | Max idle time before unused connections are automatically closed in the connection pool. By default, there's no value and this is left to the database driver to choose. | `"5m"` +| `queryExecMode` | N | Controls the default mode for executing queries. By default Dapr uses the extended protocol and automatically prepares and caches prepared statements. However, this may be incompatible with proxies such as PGBouncer. In this case it may be preferrable to use `exec` or `simple_protocol`. | `"simple_protocol"` ## Set up PostgreSQL as Configuration Store -1. Start PostgreSQL Database -1. Connect to the PostgreSQL database and setup a configuration table with following schema - +1. Start the PostgreSQL Database +1. Connect to the PostgreSQL database and setup a configuration table with following schema: -| Field | Datatype | Nullable |Details | -|--------------------|:--------:|---------|---------| -| KEY | VARCHAR | N |Holds `"Key"` of the configuration attribute | -| VALUE | VARCHAR | N |Holds Value of the configuration attribute | -| VERSION | VARCHAR | N | Holds version of the configuration attribute -| METADATA | JSON | Y | Holds Metadata as JSON + | Field | Datatype | Nullable |Details | + |--------------------|:--------:|---------|---------| + | KEY | VARCHAR | N |Holds `"Key"` of the configuration attribute | + | VALUE | VARCHAR | N |Holds Value of the configuration attribute | + | VERSION | VARCHAR | N | Holds version of the configuration attribute | + | METADATA | JSON | Y | Holds Metadata as JSON | -```console -CREATE TABLE IF NOT EXISTS table_name ( - KEY VARCHAR NOT NULL, - VALUE VARCHAR NOT NULL, - VERSION VARCHAR NOT NULL, - METADATA JSON ); -``` -3. Create a TRIGGER on configuration table. An example function to create a TRIGGER is as follows - -```console -CREATE OR REPLACE FUNCTION configuration_event() RETURNS TRIGGER AS $$ - DECLARE - data json; - notification json; - - BEGIN + ```sql + CREATE TABLE IF NOT EXISTS table_name ( + KEY VARCHAR NOT NULL, + VALUE VARCHAR NOT NULL, + VERSION VARCHAR NOT NULL, + METADATA JSON + ); + ``` - IF (TG_OP = 'DELETE') THEN - data = row_to_json(OLD); - ELSE - data = row_to_json(NEW); - END IF; - - notification = json_build_object( - 'table',TG_TABLE_NAME, - 'action', TG_OP, - 'data', data); +3. Create a TRIGGER on configuration table. An example function to create a TRIGGER is as follows: + + ```sh + CREATE OR REPLACE FUNCTION configuration_event() RETURNS TRIGGER AS $$ + DECLARE + data json; + notification json; + + BEGIN + + IF (TG_OP = 'DELETE') THEN + data = row_to_json(OLD); + ELSE + data = row_to_json(NEW); + END IF; + + notification = json_build_object( + 'table',TG_TABLE_NAME, + 'action', TG_OP, + 'data', data); + PERFORM pg_notify('config',notification::text); + RETURN NULL; + END; + $$ LANGUAGE plpgsql; + ``` + +4. Create the trigger with data encapsulated in the field labeled as `data`: + + ```sql + notification = json_build_object( + 'table',TG_TABLE_NAME, + 'action', TG_OP, + 'data', data + ); + ``` - PERFORM pg_notify('config',notification::text); - RETURN NULL; - END; -$$ LANGUAGE plpgsql; -``` -4. Create the trigger with data encapsulated in the field labelled as `data` -```ps -notification = json_build_object( - 'table',TG_TABLE_NAME, - 'action', TG_OP, - 'data', data); -``` 5. The channel mentioned as attribute to `pg_notify` should be used when subscribing for configuration notifications 6. Since this is a generic created trigger, map this trigger to `configuration table` -```console -CREATE TRIGGER config -AFTER INSERT OR UPDATE OR DELETE ON configtable - FOR EACH ROW EXECUTE PROCEDURE notify_event(); -``` + + ```sql + CREATE TRIGGER config + AFTER INSERT OR UPDATE OR DELETE ON configtable + FOR EACH ROW EXECUTE PROCEDURE notify_event(); + ``` + 7. In the subscribe request add an additional metadata field with key as `pgNotifyChannel` and value should be set to same `channel name` mentioned in `pg_notify`. From the above example, it should be set to `config` {{% alert title="Note" color="primary" %}} @@ -106,12 +161,14 @@ When calling `subscribe` API, `metadata.pgNotifyChannel` should be used to speci Any number of keys can be added to a subscription request. Each subscription uses an exclusive database connection. It is strongly recommended to subscribe to multiple keys within a single subscription. This helps optimize the number of connections to the database. -Example of subscribe HTTP API - -```ps -curl --location --request GET 'http://:/configuration/mypostgresql/subscribe?key=&key=&metadata.pgNotifyChannel=' +Example of subscribe HTTP API: + +```sh +curl -l 'http://:/configuration/mypostgresql/subscribe?key=&key=&metadata.pgNotifyChannel=' ``` {{% /alert %}} ## Related links + - [Basic schema for a Dapr component]({{< ref component-schema >}}) - [Configuration building block]({{< ref configuration-api-overview >}}) diff --git a/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-postgresql.md b/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-postgresql.md index 6e1bfad92..0d5c68242 100644 --- a/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-postgresql.md +++ b/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-postgresql.md @@ -7,13 +7,7 @@ aliases: - "/operations/components/setup-state-store/supported-state-stores/setup-postgresql/" --- -This component allows using PostgreSQL (Postgres) as state store for Dapr. - -## Create a Dapr component - -Create a file called `postgresql.yaml`, paste the following and replace the `` value with your connection string. The connection string is a standard PostgreSQL connection string. For example, `"host=localhost user=postgres password=example port=5432 connect_timeout=10 database=dapr_test"`. See the PostgreSQL [documentation on database connections](https://www.postgresql.org/docs/current/libpq-connect.html) for information on how to define a connection string. - -If you want to also configure PostgreSQL to store actors, add the `actorStateStore` option as in the example below. +This component allows using PostgreSQL (Postgres) as state store for Dapr. See [this guide]({{< ref "howto-get-save-state.md#step-1-setup-a-state-store" >}}) on how to create and apply a state store configuration. ```yaml apiVersion: dapr.io/v1alpha1 @@ -24,42 +18,72 @@ spec: type: state.postgresql version: v1 metadata: - # Connection string - - name: connectionString - value: "" - # Timeout for database operations, in seconds (optional) - #- name: timeoutInSeconds - # value: 20 - # Name of the table where to store the state (optional) - #- name: tableName - # value: "state" - # Name of the table where to store metadata used by Dapr (optional) - #- name: metadataTableName - # value: "dapr_metadata" - # Cleanup interval in seconds, to remove expired rows (optional) - #- name: cleanupIntervalInSeconds - # value: 3600 - # Max idle time for connections before they're closed (optional) - #- name: connectionMaxIdleTime - # value: 0 - # Uncomment this if you wish to use PostgreSQL as a state store for actors (optional) - #- name: actorStateStore - # value: "true" + # Connection string + - name: connectionString + value: "" + # Timeout for database operations, in seconds (optional) + #- name: timeoutInSeconds + # value: 20 + # Name of the table where to store the state (optional) + #- name: tableName + # value: "state" + # Name of the table where to store metadata used by Dapr (optional) + #- name: metadataTableName + # value: "dapr_metadata" + # Cleanup interval in seconds, to remove expired rows (optional) + #- name: cleanupIntervalInSeconds + # value: 3600 + # Maximum number of connections pooled by this component (optional) + #- name: maxConns + # value: 0 + # Max idle time for connections before they're closed (optional) + #- name: connectionMaxIdleTime + # value: 0 + # Controls the default mode for executing queries. (optional) + #- name: queryExecMode + # value: "" + # Uncomment this if you wish to use PostgreSQL as a state store for actors (optional) + #- name: actorStateStore + # value: "true" ``` + {{% alert title="Warning" color="warning" %}} The above example uses secrets as plain strings. It is recommended to use a secret store for the secrets as described [here]({{< ref component-secrets.md >}}). {{% /alert %}} ## Spec metadata fields +### Authenticate using a connection string + +The following metadata options are **required** to authenticate using a PostgreSQL connection string. + +| Field | Required | Details | Example | +|--------|:--------:|---------|---------| +| `connectionString` | Y | The connection string for the PostgreSQL database. See the PostgreSQL [documentation on database connections](https://www.postgresql.org/docs/current/libpq-connect.html) for information on how to define a connection string. | `"host=localhost user=postgres password=example port=5432 connect_timeout=10 database=my_db"` + +### Authenticate using Azure AD + +Authenticating with Azure AD is supported with Azure Database for PostgreSQL. All authentication methods supported by Dapr can be used, including client credentials ("service principal") and Managed Identity. + +| Field | Required | Details | Example | +|--------|:--------:|---------|---------| +| `useAzureAD` | Y | Must be set to `true` to enable the component to retrieve access tokens from Azure AD. | `"true"` | +| `connectionString` | Y | The connection string for the PostgreSQL database.
This must contain the user, which corresponds to the name of the user created inside PostgreSQL that maps to the Azure AD identity; this is often the name of the corresponding principal (e.g. the name of the Azure AD application). This connection string should not contain any password. | `"host=mydb.postgres.database.azure.com user=myapplication port=5432 database=my_db sslmode=require"` | +| `azureTenantId` | N | ID of the Azure AD tenant | `"cd4b2887-304c-…"` | +| `azureClientId` | N | Client ID (application ID) | `"c7dd251f-811f-…"` | +| `azureClientSecret` | N | Client secret (application password) | `"Ecy3X…"` | + +### Other metadata options + | Field | Required | Details | Example | |--------------------|:--------:|---------|---------| -| `connectionString` | Y | The connection string for the PostgreSQL database | `"host=localhost user=postgres password=example port=5432 connect_timeout=10 database=dapr_test"` | `timeoutInSeconds` | N | Timeout, in seconds, for all database operations. Defaults to `20` | `30` | `tableName` | N | Name of the table where the data is stored. Defaults to `state`. Can optionally have the schema name as prefix, such as `public.state` | `"state"`, `"public.state"` | `metadataTableName` | N | Name of the table Dapr uses to store a few metadata properties. Defaults to `dapr_metadata`. Can optionally have the schema name as prefix, such as `public.dapr_metadata` | `"dapr_metadata"`, `"public.dapr_metadata"` | `cleanupIntervalInSeconds` | N | Interval, in seconds, to clean up rows with an expired TTL. Default: `3600` (i.e. 1 hour). Setting this to values <=0 disables the periodic cleanup. | `1800`, `-1` +| `maxConns` | N | Maximum number of connections pooled by this component. Set to 0 or lower to use the default value, which is the greater of 4 or the number of CPUs. | `"4"` | `connectionMaxIdleTime` | N | Max idle time before unused connections are automatically closed in the connection pool. By default, there's no value and this is left to the database driver to choose. | `"5m"` +| `queryExecMode` | N | Controls the default mode for executing queries. By default Dapr uses the extended protocol and automatically prepares and caches prepared statements. However, this may be incompatible with proxies such as PGBouncer. In this case it may be preferrable to use `exec` or `simple_protocol`. | `"simple_protocol"` | `actorStateStore` | N | Consider this state store for actors. Defaults to `"false"` | `"true"`, `"false"` ## Setup PostgreSQL @@ -70,20 +94,21 @@ The above example uses secrets as plain strings. It is recommended to use a secr 1. Run an instance of PostgreSQL. You can run a local instance of PostgreSQL in Docker CE with the following command: - This example does not describe a production configuration because it sets the password in plain text and the user name is left as the PostgreSQL default of "postgres". - ```bash docker run -p 5432:5432 -e POSTGRES_PASSWORD=example postgres ``` + > This example does not describe a production configuration because it sets the password in plain text and the user name is left as the PostgreSQL default of "postgres". + 2. Create a database for state data. Either the default "postgres" database can be used, or create a new database for storing state data. To create a new database in PostgreSQL, run the following SQL command: - ```SQL - CREATE DATABASE dapr_test; + ```sql + CREATE DATABASE my_dapr; ``` + {{% /codetab %}} {{% /tabs %}}