From 8f08e681df96ed161e841f4a49b4e2014aa41554 Mon Sep 17 00:00:00 2001 From: Tiago Alves Macambira Date: Tue, 11 Oct 2022 16:34:02 -0700 Subject: [PATCH] Update SecretStore API documentation to account for Features feature. (#2863) * Update SecretStore API documentation to account for Features feature. Updates secret stores data/components and partial HTML fragment to account for this new feature, updating the table of supported secret stores. Updates Secret Store API page to refer to this feature in the secret retrieval API documentation. Updates HashiCorp Vault documentation to explicitly mention this feature is supported and used by HashiCorp Vault. Closes #2787 Signed-off-by: Tiago Alves Macambira * Improves documentation, kas k8s as a multi kv secret store. Signed-off-by: Tiago Alves Macambira * Removing TODO and small fix Signed-off-by: Tiago Alves Macambira * Minor fix Signed-off-by: Tiago Alves Macambira * Clarification about vaultValueType Signed-off-by: Tiago Alves Macambira * Improve Local File Secret Store documentation Signed-off-by: Tiago Alves Macambira * Tiny fixes Signed-off-by: Tiago Alves Macambira * Tiny fixes Signed-off-by: Tiago Alves Macambira * Apply suggestions from code review Applying suggestions from @greenie-msft Co-authored-by: greenie-msft <56556602+greenie-msft@users.noreply.github.com> Signed-off-by: Tiago Alves Macambira * Apply suggestions from code review Co-authored-by: greenie-msft <56556602+greenie-msft@users.noreply.github.com> Signed-off-by: Tiago Alves Macambira * Apply suggestions from code review Co-authored-by: Hannah Hunter <94493363+hhunter-ms@users.noreply.github.com> Signed-off-by: Tiago Alves Macambira * Apply suggestions from code review Co-authored-by: Mark Fussell Signed-off-by: Tiago Alves Macambira * Update daprdocs/content/en/reference/components-reference/supported-secret-stores/file-secret-store.md Signed-off-by: Tiago Alves Macambira * Apply suggestions from code review Signed-off-by: Tiago Alves Macambira * Update `multiValued` definition Makes its relation to the `multipleKeyValuesPerSecret` behavior explicit. Signed-off-by: Tiago Alves Macambira Signed-off-by: Tiago Alves Macambira Co-authored-by: greenie-msft <56556602+greenie-msft@users.noreply.github.com> Co-authored-by: Hannah Hunter <94493363+hhunter-ms@users.noreply.github.com> Co-authored-by: Mark Fussell --- .../content/en/reference/api/secrets_api.md | 10 ++- .../file-secret-store.md | 84 +++++++++++++++++-- .../hashicorp-vault.md | 35 +++++++- .../components/secret_stores/generic.yaml | 6 ++ .../partials/components/secret-stores.html | 2 + 5 files changed, 125 insertions(+), 12 deletions(-) diff --git a/daprdocs/content/en/reference/api/secrets_api.md b/daprdocs/content/en/reference/api/secrets_api.md index 84c39467d..eae6f3901 100644 --- a/daprdocs/content/en/reference/api/secrets_api.md +++ b/daprdocs/content/en/reference/api/secrets_api.md @@ -53,10 +53,12 @@ metadata.version_stage | version stage for the given secret key #### Response Body -If a secret store has support for multiple keys in a secret, a JSON payload is returned with the key names as fields and their respective values. +If a secret store has support for multiple key-values in a secret, a JSON payload is returned with the key names as fields and their respective values. In case of a secret store that only has name/value semantics, a JSON payload is returned with the name of the secret as the field and the value of the secret as the value. +[See the classification of secret stores]({{< ref supported-secret-stores.md >}}) that support multiple keys in a secret and name/value semantics. + ##### Response with multiple keys in a secret (eg. Kubernetes): ```shell @@ -70,7 +72,9 @@ curl http://localhost:3500/v1.0/secrets/kubernetes/db-secret } ``` -##### Response with no keys in a secret: +The above example demonstrates a response from a secret store with multiple keys in a secret. Note that the secret name (`db-secret`) **is not** returned as part of the result. + +##### Response from a secret store with name/value semantics: ```shell curl http://localhost:3500/v1.0/secrets/vault/db-secret @@ -82,6 +86,8 @@ curl http://localhost:3500/v1.0/secrets/vault/db-secret } ``` +The above example demonstrates a response from a secret store with name/value semantics. Compared to the result from a secret store with multiple keys in a secret, this result returns a single key-value pair, with the secret name (`db-secret`) returned as the key in the key-value pair. + #### Response Codes Code | Description diff --git a/daprdocs/content/en/reference/components-reference/supported-secret-stores/file-secret-store.md b/daprdocs/content/en/reference/components-reference/supported-secret-stores/file-secret-store.md index 625eb95a7..b856aa1c4 100644 --- a/daprdocs/content/en/reference/components-reference/supported-secret-stores/file-secret-store.md +++ b/daprdocs/content/en/reference/components-reference/supported-secret-stores/file-secret-store.md @@ -41,6 +41,7 @@ spec: | secretsFile | Y | The path to the file where secrets are stored | `"path/to/file.json"` | | nestedSeparator | N | Used by the store when flattening the JSON hierarchy to a map. Defaults to `":"` | `":"` | multiValued | N | Allows one level of multi-valued key/value pairs before flattening JSON hierarchy. Defaults to `"false"` | `"true"` | +| multiValued | N | `"true"` sets the `multipleKeyValuesPerSecret` behavior. Allows one level of multi-valued key/value pairs before flattening JSON hierarchy. Defaults to `"false"` | `"true"` | ## Setup JSON file to hold the secrets @@ -56,24 +57,65 @@ Given the following JSON loaded from `secretsFile`: } ``` -If `multiValued` is `"false"`, the store will load the file and create a map with the following key value pairs: +The flag `multiValued` determines whether the secret store presents a [name/value behavior or a multiple key-value per secret behavior]({{< ref "secrets_api.md#response-body" >}}). + +### Name/Value semantics + + +If `multiValued` is `false`, the store loads [the JSON file]({{< ref "#setup-json-file-to-hold-the-secrets" >}}) and create a map with the following key-value pairs: | flattened key | value | | --- | --- | -|"redisPassword" | "your redis password" | -|"connectionStrings:sql" | "your sql connection string" | -|"connectionStrings:mysql"| "your mysql connection string" | +|"redisPassword" | `"your redis password"` | +|"connectionStrings:sql" | `"your sql connection string"` | +|"connectionStrings:mysql"| `"your mysql connection string"` | -Use the flattened key (`connectionStrings:sql`) to access the secret. The following JSON map returned: +If the `multiValued` setting set to true, invoking a `GET` request on the key `connectionStrings` results in a 500 HTTP response and an error message. For example: + +```shell +$ curl http://localhost:3501/v1.0/secrets/local-secret-store/connectionStrings +``` +```json +{ + "errorCode": "ERR_SECRET_GET", + "message": "failed getting secret with key connectionStrings from secret store local-secret-store: secret connectionStrings not found" +} +``` + +This error is expected, since the `connectionStrings` key is not present, per the table above. + +However, requesting for flattened key `connectionStrings:sql` would result in a successful response, with the following: + +```shell +$ curl http://localhost:3501/v1.0/secrets/local-secret-store/connectionStrings:sql +``` ```json { "connectionStrings:sql": "your sql connection string" } ``` -If `multiValued` is `"true"`, you would instead use the top level key. In this example, `connectionStrings` would return the following map: +### Multiple key-values behavior +If `multiValued` is `true`, the secret store enables multiple key-value per secret behavior: +- Nested structures after the top level will be flattened. +- It parses the [same JSON file]({{< ref "#setup-json-file-to-hold-the-secrets" >}}) into this table: + +| key | value | +| --- | --- | +|"redisPassword" | `"your redis password"` | +|"connectionStrings" | `{"mysql":"your mysql connection string","sql":"your sql connection string"}` | + +Notice that in the above table: +- `connectionStrings` is now a JSON object with two keys: `mysql` and `sql`. +- The `connectionStrings:sql` and `connectionStrings:mysql` flattened keys from the [table mapped for name/value semantics]({{< ref "#namevalue-semantics" >}}) are missing. + +Invoking a `GET` request on the key `connectionStrings` now results in a successful HTTP response similar to the following: + +```shell +$ curl http://localhost:3501/v1.0/secrets/local-secret-store/connectionStrings +``` ```json { "sql": "your sql connection string", @@ -81,9 +123,21 @@ If `multiValued` is `"true"`, you would instead use the top level key. In this e } ``` -Nested structures after the top level will be flattened. In this example, `connectionStrings` would return the following map: +Meanwhile, requesting for the flattened key `connectionStrings:sql` would now return a 500 HTTP error response with the following: -JSON from `secretsFile`: +```json +{ + "errorCode": "ERR_SECRET_GET", + "message": "failed getting secret with key connectionStrings:sql from secret store local-secret-store: secret connectionStrings:sql not found" +} +``` + + +#### Handling deeper nesting levels + +Notice that, as stated in the [spec metadata fields table](#spec-metadata-fields), `multiValued` only handles a single nesting level. + +Let's say you have a local file secret store with `multiValued` enabled, pointing to a `secretsFile` with the following JSON content: ```json { @@ -96,9 +150,21 @@ JSON from `secretsFile`: } } ``` +The contents of key `mysql` under `connectionStrings` has a nesting level greater than 1 and would be flattened. -Response: +Here is how it would look in memory: +| key | value | +| --- | --- | +|"redisPassword" | `"your redis password"` | +|"connectionStrings" | `{ "mysql:username": "your mysql username", "mysql:password": "your mysql password" }` | + + +Once again, requesting for key `connectionStrings` results in a successful HTTP response but its contents, as shown in the table above, would be flattened: + +```shell +$ curl http://localhost:3501/v1.0/secrets/local-secret-store/connectionStrings +``` ```json { "mysql:username": "your mysql username", diff --git a/daprdocs/content/en/reference/components-reference/supported-secret-stores/hashicorp-vault.md b/daprdocs/content/en/reference/components-reference/supported-secret-stores/hashicorp-vault.md index ad13d0f2d..3c7a9bb4f 100644 --- a/daprdocs/content/en/reference/components-reference/supported-secret-stores/hashicorp-vault.md +++ b/daprdocs/content/en/reference/components-reference/supported-secret-stores/hashicorp-vault.md @@ -64,7 +64,7 @@ The above example uses secrets as plain strings. It is recommended to use a loca | vaultKVPrefix | N | The prefix in vault. Defaults to `"dapr"` | `"dapr"`, `"myprefix"` | | vaultKVUsePrefix | N | If false, vaultKVPrefix is forced to be empty. If the value is not given or set to true, vaultKVPrefix is used when accessing the vault. Setting it to false is needed to be able to use the BulkGetSecret method of the store. | `"true"`, `"false"` | | enginePath | N | The [engine](https://www.vaultproject.io/api-docs/secret/kv/kv-v2) path in vault. Defaults to `"secret"` | `"kv"`, `"any"` | -| vaultValueType | N | Vault value type. `map` means to parse the value into `map[string]string`, `text` means to use the value as a string. Defaults to `"map"` | `"map"`, `"text"` | +| vaultValueType | N | Vault value type. `map` means to parse the value into `map[string]string`, `text` means to use the value as a string. 'map' sets the `multipleKeyValuesPerSecret` behavior. `text' makes Vault behave as a secret store with name/value semantics. Defaults to `"map"` | `"map"`, `"text"` | ## Setup Hashicorp Vault instance @@ -79,6 +79,39 @@ For Kubernetes, you can use the Helm Chart: }} + + +## Multiple key-values per secret + +HashiCorp Vault supports multiple key-values in a secret. While this behavior is ultimately dependent on the underlying [secret engine](https://www.vaultproject.io/docs/secrets#secrets-engines) configured by `enginePath`, it may change the way you store and retrieve keys from Vault. For instance, multiple key-values in a secret is the behavior exposed in the `secret` engine, the default engine configured by the `enginePath` field. + +When retrieving secrets, a JSON payload is returned with the key names as fields and their respective values. + +Suppose you add a secret to your Vault setup as follows: + +```shell +vault kv put secret/dapr/mysecret firstKey=aValue secondKey=anotherValue thirdKey=yetAnotherDistinctValue +``` + +In the example above, the secret is named `mysecret` and it has 3 key-values under it. +Observe that the secret is created under a `dapr` prefix, as this is the default value for the `vaultKVPrefix` flag. +Retrieving it from Dapr would result in the following output: + +```shell +$ curl http://localhost:3501/v1.0/secrets/my-hashicorp-vault/mysecret +``` + +```json +{ + "firstKey": "aValue", + "secondKey": "anotherValue", + "thirdKey": "yetAnotherDistinctValue" +} +``` + +Notice that the name of the secret (`mysecret`) is not repeated in the result. + + ## Related links - [Secrets building block]({{< ref secrets >}}) - [How-To: Retrieve a secret]({{< ref "howto-secrets.md" >}}) diff --git a/daprdocs/data/components/secret_stores/generic.yaml b/daprdocs/data/components/secret_stores/generic.yaml index b0e90d33e..cb909965a 100644 --- a/daprdocs/data/components/secret_stores/generic.yaml +++ b/daprdocs/data/components/secret_stores/generic.yaml @@ -3,6 +3,8 @@ state: Stable version: v1 since: "1.9" + features: + multipleKeyValuesPerSecret: true - component: Local environment variables link: envvar-secret-store state: Stable @@ -13,8 +15,12 @@ state: Stable version: v1 since: "1.9" + features: + multipleKeyValuesPerSecret: true - component: Kubernetes secrets link: kubernetes-secret-store state: Stable version: v1 since: "1.0" + features: + multipleKeyValuesPerSecret: true diff --git a/daprdocs/layouts/partials/components/secret-stores.html b/daprdocs/layouts/partials/components/secret-stores.html index 18fe10234..2f7eecc24 100644 --- a/daprdocs/layouts/partials/components/secret-stores.html +++ b/daprdocs/layouts/partials/components/secret-stores.html @@ -20,12 +20,14 @@ + {{ range sort $components "component" }} +
ComponentMultiple Key-Values Per Secret Status Component version Since runtime version
{{ if .features.multipleKeyValuesPerSecret }}✅{{else}}{{ end }} {{ .component }}