From d32456cb3272e44f735377f7fe96b4041d020d9c Mon Sep 17 00:00:00 2001 From: Yaron Schneider Date: Wed, 16 Oct 2019 06:54:50 -0700 Subject: [PATCH] Add State Store setup (#141) * fix links * add state store setup --- README.md | 1 - .../troubleshooting/common_issues.md | 4 +- best-practices/troubleshooting/logs.md | 4 +- concepts/README.md | 4 +- concepts/components/redis.md | 52 ++++++------ howto/setup-state-store/README.md | 47 +++++++++++ howto/setup-state-store/setup-redis.md | 81 ++++++++++++++++++- .../supported-state-stores.md | 9 ++- reference/api/service_invocation.md | 2 +- 9 files changed, 168 insertions(+), 36 deletions(-) create mode 100644 howto/setup-state-store/README.md diff --git a/README.md b/README.md index 7240dbdec..de48860de 100644 --- a/README.md +++ b/README.md @@ -21,4 +21,3 @@ Welcome to the Dapr documentation repository. You can learn more about Dapr from - **[Getting Started with ASP.NET Core](https://github.com/dapr/dotnet-sdk/tree/master/samples/AspNetCore)** - Samples for developing ASP.NET applications using the Dapr .NET SDK > Note: Dapr is language agnostic and provides a [RESTful HTTP API](./reference/api/Readme.md) in addition to the protobuf clients. - diff --git a/best-practices/troubleshooting/common_issues.md b/best-practices/troubleshooting/common_issues.md index fa9a6f22b..fdc5bc0de 100644 --- a/best-practices/troubleshooting/common_issues.md +++ b/best-practices/troubleshooting/common_issues.md @@ -59,7 +59,7 @@ To check, use kubectl get a list of components: `kubectl get components` If there isn't a state store component, it means you need to set one up. -Visit [here](../components/redis.md) for more details. +Visit [here](../../howto/setup-state-store/setup-redis.md) for more details. If everything's set up correctly, make sure you got the credentials right. Search the Dapr runtime logs and look for any state store errors: @@ -75,7 +75,7 @@ To check, use kubectl get a list of components: `kubectl get components` If there isn't a pub-sub component, it means you need to set one up. -Visit [here](../components/redis.md#configuring-redis-for-pubsub) for more details. +Visit [here](../../howto/setup-pub-sub-message-broker/README.md) for more details. If everything's set up correctly, make sure you got the credentials right. Search the Dapr runtime logs and look for any pub-sub errors: diff --git a/best-practices/troubleshooting/logs.md b/best-practices/troubleshooting/logs.md index 929ed4c5b..c55718919 100644 --- a/best-practices/troubleshooting/logs.md +++ b/best-practices/troubleshooting/logs.md @@ -45,7 +45,7 @@ dapr run node myapp.js == DAPR == time="2019-09-05T12:26:43-07:00" level=info msg="starting Dapr Runtime -- version 0.3.0-alpha -- commit b6f2810-dirty" == DAPR == time="2019-09-05T12:26:43-07:00" level=info msg="log level set to: info" == DAPR == time="2019-09-05T12:26:43-07:00" level=info msg="standalone mode configured" -== DAPR == time="2019-09-05T12:26:43-07:00" level=info msg="action id: Trackgreat-Lancer" +== DAPR == time="2019-09-05T12:26:43-07:00" level=info msg="dapr id: Trackgreat-Lancer" == DAPR == time="2019-09-05T12:26:43-07:00" level=info msg="loaded component statestore (state.redis)" == DAPR == time="2019-09-05T12:26:43-07:00" level=info msg="loaded component messagebus (pubsub.redis)" == DAPR == 2019/09/05 12:26:43 redis: connecting to localhost:6379 @@ -111,7 +111,7 @@ Next, get the logs for the Dapr sidecar container: time="2019-09-04T02:52:27Z" level=info msg="starting Dapr Runtime -- version 0.3.0-alpha -- commit b6f2810-dirty" time="2019-09-04T02:52:27Z" level=info msg="log level set to: info" time="2019-09-04T02:52:27Z" level=info msg="kubernetes mode configured" -time="2019-09-04T02:52:27Z" level=info msg="action id: addapp" +time="2019-09-04T02:52:27Z" level=info msg="dapr id: addapp" time="2019-09-04T02:52:27Z" level=info msg="application protocol: http. waiting on port 6000" time="2019-09-04T02:52:27Z" level=info msg="application discovered on port 6000" time="2019-09-04T02:52:27Z" level=info msg="actor runtime started. actor idle timeout: 1h0m0s. actor scan interval: 30s" diff --git a/concepts/README.md b/concepts/README.md index 7f347a973..b86880298 100644 --- a/concepts/README.md +++ b/concepts/README.md @@ -14,7 +14,7 @@ This directory contains various Dapr concepts. The goal of these documents is to * **Components** - Dapr uses a modular design, in which functionalities are grouped and delivered by a number of *components*, such as [pub-sub](./components/redis.md) and [secrets](./components/secrets.md). Many of the components are pluggable so that you can swap out the default implemenation with your custom implementations. + Dapr uses a modular design, in which functionalities are grouped and delivered by a number of *components*, such as [pub-sub](./publish-subscribe-messaging/Readme.md) and [secrets](./state-management/state-management.md). Many of the components are pluggable so that you can swap out the default implemenation with your custom implementations. * [**Distributed Tracing**](./tracing-logging/tracing-logging.md) @@ -39,4 +39,4 @@ This directory contains various Dapr concepts. The goal of these documents is to ## Extensibility -* [Redis](./components/redis.md) \ No newline at end of file +* [Components Contrib](https://github.com/dapr/components-contrib) \ No newline at end of file diff --git a/concepts/components/redis.md b/concepts/components/redis.md index 13f8a69e5..d9913c26f 100644 --- a/concepts/components/redis.md +++ b/concepts/components/redis.md @@ -9,19 +9,6 @@ Dapr can use Redis in two ways: Dapr can use any Redis instance - containerized, running on your local dev machine, or a managed cloud service. If you already have a Redis store, move on to the [Configuration](#configuration) section. -### Creating an Azure Managed Redis Cache - -**Note**: this approach requires having an Azure Subscription. - -1. Open [this link](https://ms.portal.azure.com/#create/Microsoft.Cache) to start the Azure Redis Cache creation flow. Log in if necessary. -2. Fill out necessary information and **check the "Unblock port 6379" box**, which will allow us to persist state without SSL. -3. Click "Create" to kickoff deployment of your Redis instance. -4. Once your instance is created, you'll need to grab your access key. Navigate to "Access Keys" under "Settings" and copy your key. -5. Run `kubectl get svc` and copy the cluster IP of your `redis-master`. -6. Finally, we need to add our key and our host to a `redis.yaml` file that Dapr can apply to our cluster. If you're running a sample, you'll add the host and key to the provided `redis.yaml`. If you're creating a project from the ground up, you'll create a `redis.yaml` file as specified in [Configuration](#configuration). Set the `redisHost` key to `[IP FROM PREVIOUS STEP]:6379` and the `redisPassword` key to the key you copied in step 4. **Note:** In a production-grade application, follow [secret management](https://github.com/dapr/docs/blob/master/concepts/components/secrets.md) instructions to securely manage your secrets. - -> **NOTE:** Dapr pub-sub uses [Redis Streams](https://redis.io/topics/streams-intro) that was introduced by Redis 5.0, which isn't currently available on Azure Managed Redis Cache. Consequently, you can use Azure Managed Redis Cache only for state persistence. - ### Creating a Redis Cache in your Kubernetes Cluster using Helm We can use [Helm](https://helm.sh/) to quickly create a Redis instance in our Kubernetes cluster. This approach requires [Installing Helm](https://github.com/helm/helm#install). @@ -42,7 +29,22 @@ We can use [Helm](https://helm.sh/) to quickly create a Redis instance in our Ku redisPassword: "lhDOkwTlp0" ``` -### Other ways to Create a Redis Cache +### Creating an Azure Managed Redis Cache + +**Note**: this approach requires having an Azure Subscription. + +1. Open [this link](https://ms.portal.azure.com/#create/Microsoft.Cache) to start the Azure Redis Cache creation flow. Log in if necessary. +2. Fill out necessary information and **check the "Unblock port 6379" box**, which will allow us to persist state without SSL. +3. Click "Create" to kickoff deployment of your Redis instance. +4. Once your instance is created, you'll need to grab your access key. Navigate to "Access Keys" under "Settings" and copy your key. +5. Run `kubectl get svc` and copy the cluster IP of your `redis-master`. +6. Finally, we need to add our key and our host to a `redis.yaml` file that Dapr can apply to our cluster. If you're running a sample, you'll add the host and key to the provided `redis.yaml`. If you're creating a project from the ground up, you'll create a `redis.yaml` file as specified in [Configuration](#configuration). Set the `redisHost` key to `[IP FROM PREVIOUS STEP]:6379` and the `redisPassword` key to the key you copied in step 4. **Note:** In a production-grade application, follow [secret management](https://github.com/dapr/docs/blob/master/concepts/components/secrets.md) instructions to securely manage your secrets. + +> **NOTE:** Dapr pub-sub uses [Redis Streams](https://redis.io/topics/streams-intro) that was introduced by Redis 5.0, which isn't currently available on Azure Managed Redis Cache. Consequently, you can use Azure Managed Redis Cache only for state persistence. + + + +### Other ways to Create a Redis Database - [AWS Redis](https://aws.amazon.com/redis/) - [GCP Cloud MemoryStore](https://cloud.google.com/memorystore/) @@ -62,9 +64,11 @@ metadata: name: statestore spec: type: state.redis - connectionInfo: - redisHost: "YOUR_REDIS_HOST_HERE" - redisPassword: "YOUR_REDIS_KEY_HERE" + metadata: + - name: redisHost + value: + - name: redisPassword + value: ``` ### Configuring Redis for Pub/Sub @@ -75,12 +79,14 @@ Create a file called redis.yaml, and paste the following: apiVersion: dapr.io/v1alpha1 kind: Component metadata: - name: messagebus + name: statestore spec: - type: pubsub.redis - connectionInfo: - redisHost: "YOUR_REDIS_HOST_HERE" - redisPassword: "YOUR_REDIS_PASSWORD_HERE" + type: state.redis + metadata: + - name: redisHost + value: + - name: redisPassword + value: ``` ## Apply the configuration @@ -93,4 +99,4 @@ kubectl apply -f redis.yaml ### Standalone -By default the Dapr CLI creates a local Redis instance when you run `dapr init`. However, if you want to configure a different Redis instance, create a directory named `eventsources` in the root path of your Action binary and then copy your `redis.yaml` into that directory. +By default the Dapr CLI creates a local Redis instance when you run `dapr init`. However, if you want to configure a different Redis instance, create a directory named `components` in the root path of your Dapr binary and then copy your `redis.yaml` into that directory. diff --git a/howto/setup-state-store/README.md b/howto/setup-state-store/README.md new file mode 100644 index 000000000..13423be3a --- /dev/null +++ b/howto/setup-state-store/README.md @@ -0,0 +1,47 @@ +# Setup a Dapr state store + +Dapr integrates with existing databases to provide apps with state management capabilities for CRUD oprations, transactions and more. +Currently, Dapr supports the configuration of one state store per cluster. + +State stores are extensible and can be found in the [components-contrib repo](https://github.com/dapr/components-contrib). + +A state store in Dapr is described using a `Component` file: + +``` +apiVersion: dapr.io/v1alpha1 +kind: Component +metadata: + name: statestore +spec: + type: state. + metadata: + - name: + value: + - name: + value: +... +``` + +The type of database is determined by the `type` field, and things like connection strings and other metadata are put in the `.metadata` section. +Even though you can put plain text secrets in there, it is recommended you use a s[ecret store](../../concepts/components/secrets.md). + +## Running locally + +When running locally with the Dapr CLI, a component file for a Redis state store will be automatically created in a `components` directory in your current working directory. + +You can make changes to this file the way you see fit, whether to change connection values or replace it with a different store. + +## Running in Kubernetes + +Dapr uses a Kubernetes Operator to update the sidecars running in the cluster with different components. +To setup a state store to Kubernetes, use `kubectl` to apply the component file: + +``` +kubectl apply -f statestore.yaml +``` + +## Reference + +[Setup Redis](./setup-redis.md) +[Setup Azure CosmosDB](./setup-azure-cosmosdb.md) +[Supported State Stores](./supported-state-stores.md) \ No newline at end of file diff --git a/howto/setup-state-store/setup-redis.md b/howto/setup-state-store/setup-redis.md index 808829791..c73616d02 100644 --- a/howto/setup-state-store/setup-redis.md +++ b/howto/setup-state-store/setup-redis.md @@ -1,3 +1,80 @@ -# documentation +# Setup Redis -Content for this file to be added +## Creating a Redis Store + +Dapr can use any Redis instance - containerized, running on your local dev machine, or a managed cloud service. If you already have a Redis store, move on to the [Configuration](#configuration) section. + +### Creating a Redis Cache in your Kubernetes Cluster using Helm + +We can use [Helm](https://helm.sh/) to quickly create a Redis instance in our Kubernetes cluster. This approach requires [Installing Helm](https://github.com/helm/helm#install). + +1. Install Redis into your cluster: `helm install stable/redis --name redis --set image.tag=5.0.5-debian-9-r104`. Note that we're explicitly setting an image tag to get a version greater than 5, which is what Dapr' pub-sub functionality requires. If you're intending on using Redis as just a state store (and not for pub-sub), you do not have to set the image version. +2. Run `kubectl get pods` to see the Redis containers now running in your cluster. +3. Run `kubectl get svc` and copy the cluster IP of your `redis-master`. Add this IP as the `redisHost` in your redis.yaml file, followed by ":6379". For example: + ```yaml + redisHost: "10.0.125.130:6379" + ``` +4. Next, we'll get our Redis password, which is slightly different depending on the OS we're using: + - **Windows**: Run `kubectl get secret --namespace default redis -o jsonpath="{.data.redis-password}" > encoded.b64`, which will create a file with your encoded password. Next, run `certutil -decode encoded.b64 password.txt`, which will put your redis password in a text file called `password.txt`. Copy the password and delete the two files. + + - **Linux/MacOS**: Run `kubectl get secret --namespace default redis -o jsonpath="{.data.redis-password}" | base64 --decode` and copy the outputted password. + + Add this password as the `redisPassword` value in your redis.yaml file. For example: + ```yaml + redisPassword: "lhDOkwTlp0" + ``` + +### Creating an Azure Managed Redis Cache + +**Note**: this approach requires having an Azure Subscription. + +1. Open [this link](https://ms.portal.azure.com/#create/Microsoft.Cache) to start the Azure Redis Cache creation flow. Log in if necessary. +2. Fill out necessary information and **check the "Unblock port 6379" box**, which will allow us to persist state without SSL. +3. Click "Create" to kickoff deployment of your Redis instance. +4. Once your instance is created, you'll need to grab your access key. Navigate to "Access Keys" under "Settings" and copy your key. +5. Run `kubectl get svc` and copy the cluster IP of your `redis-master`. +6. Finally, we need to add our key and our host to a `redis.yaml` file that Dapr can apply to our cluster. If you're running a sample, you'll add the host and key to the provided `redis.yaml`. If you're creating a project from the ground up, you'll create a `redis.yaml` file as specified in [Configuration](#configuration). Set the `redisHost` key to `[IP FROM PREVIOUS STEP]:6379` and the `redisPassword` key to the key you copied in step 4. **Note:** In a production-grade application, follow [secret management](https://github.com/dapr/docs/blob/master/concepts/components/secrets.md) instructions to securely manage your secrets. + +> **NOTE:** Dapr pub-sub uses [Redis Streams](https://redis.io/topics/streams-intro) that was introduced by Redis 5.0, which isn't currently available on Azure Managed Redis Cache. Consequently, you can use Azure Managed Redis Cache only for state persistence. + + +### Other ways to create a Redis Database + +- [AWS Redis](https://aws.amazon.com/redis/) +- [GCP Cloud MemoryStore](https://cloud.google.com/memorystore/) + +## Configuration + +To setup Redis, you need to create a component for `state.redis`. +
+The following yaml files demonstrates how to define each. **Note:** yaml files below illustrate secret management in plain text. In a production-grade application, follow [secret management](../../concepts/components/secrets.md) instructions to securely manage your secrets. + +### Configuring Redis for State Persistence and Retrieval + +Create a file called redis.yaml, and paste the following: + +```yaml +apiVersion: dapr.io/v1alpha1 +kind: Component +metadata: + name: statestore +spec: + type: state.redis + metadata: + - name: redisHost + value: + - name: redisPassword + value: +``` + +## Apply the configuration + +### Kubernetes + +``` +kubectl apply -f redis.yaml +``` + +### Standalone + +By default the Dapr CLI creates a local Redis instance when you run `dapr init`. When you run an app using `dapr run`, the component file will automatically be created for you in a `components` dir in your current working directory. diff --git a/howto/setup-state-store/supported-state-stores.md b/howto/setup-state-store/supported-state-stores.md index 808829791..91a5729ae 100644 --- a/howto/setup-state-store/supported-state-stores.md +++ b/howto/setup-state-store/supported-state-stores.md @@ -1,3 +1,6 @@ -# documentation - -Content for this file to be added +# Supported state stores +
+| Name | CRUD | Transactional +| ------------- | -------|------ | +| Redis | V | V +| Azure CosmosDB | V | X | diff --git a/reference/api/service_invocation.md b/reference/api/service_invocation.md index 9972657d3..60d88c6f3 100644 --- a/reference/api/service_invocation.md +++ b/reference/api/service_invocation.md @@ -22,7 +22,7 @@ Code | Description Parameter | Description --------- | ----------- -id | the Action ID associated with the remote app +id | the App ID associated with the remote app method-name | the name of the method or url to invoke on the remote app ```shell