--- type: docs title: "AWS S3 binding spec" linkTitle: "AWS S3" description: "Detailed documentation on the AWS S3 binding component" aliases: - "/operations/components/setup-bindings/supported-bindings/s3/" --- ## Component format To setup an AWS S3 binding create a component of type `bindings.aws.s3`. This binding works with other S3-compatible services, such as Minio. See [this guide]({{< ref "howto-bindings.md#1-create-a-binding" >}}) on how to create and apply a binding configuration. See [Authenticating to AWS]({{< ref authenticating-aws.md >}}) for information about authentication-related attributes. ```yaml apiVersion: dapr.io/v1alpha1 kind: Component metadata: name: spec: type: bindings.aws.s3 version: v1 metadata: - name: bucket value: "mybucket" - name: region value: "us-west-2" - name: endpoint value: "s3.us-west-2.amazonaws.com" - name: accessKey value: "*****************" - name: secretKey value: "*****************" - name: sessionToken value: "mysession" - name: decodeBase64 value: "" - name: encodeBase64 value: "" - name: forcePathStyle value: "" - name: disableSSL value: "" - name: insecureSSL value: "" ``` {{% alert title="Warning" color="warning" %}} The above example uses secrets as plain strings. It is recommended to use a secret store for the secrets as described [here]({{< ref component-secrets.md >}}). {{% /alert %}} ## Spec metadata fields | Field | Required | Binding support | Details | Example | |--------------------|:--------:|------------|-----|---------| | `bucket` | Y | Output | The name of the S3 bucket to write to | `"bucket"` | | `region` | Y | Output | The specific AWS region | `"us-east-1"` | | `endpoint` | N | Output | The specific AWS endpoint | `"s3.us-east-1.amazonaws.com"` | | `accessKey` | Y | Output | The AWS Access Key to access this resource | `"key"` | | `secretKey` | Y | Output | The AWS Secret Access Key to access this resource | `"secretAccessKey"` | | `sessionToken` | N | Output | The AWS session token to use | `"sessionToken"` | | `forcePathStyle` | N | Output | Currently Amazon S3 SDK supports virtual hosted-style and path-style access. `"true"` is path-style format like `"https:////"`. `"false"` is hosted-style format like `"https://./"`. Defaults to `"false"` | `"true"`, `"false"` | | `decodeBase64` | N | Output | Configuration to decode base64 file content before saving to bucket storage. (In case of saving a file with binary content). `"true"` is the only allowed positive value. Other positive variations like `"True", "1"` are not acceptable. Defaults to `false` | `"true"`, `"false"` | | `encodeBase64` | N | Output | Configuration to encode base64 file content before return the content. (In case of opening a file with binary content). `"true"` is the only allowed positive value. Other positive variations like `"True", "1"` are not acceptable. Defaults to `"false"` | `"true"`, `"false"` | | `disableSSL` | N | Output | Allows to connect to non `https://` endpoints. Defaults to `"false"` | `"true"`, `"false"` | | `insecureSSL` | N | Output | When connecting to `https://` endpoints, accepts invalid or self-signed certificates. Defaults to `"false"` | `"true"`, `"false"` | {{% alert title="Important" color="warning" %}} When running the Dapr sidecar (daprd) with your application on EKS (AWS Kubernetes), if you're using a node/pod that has already been attached to an IAM policy defining access to AWS resources, you **must not** provide AWS access-key, secret-key, and tokens in the definition of the component spec you're using. {{% /alert %}} ### S3 Bucket Creation {{< tabs "Minio" "LocalStack" "AWS" >}} {{% codetab %}} ### Using with Minio [Minio](https://min.io/) is a service that exposes local storage as S3-compatible block storage, and it's a popular alternative to S3 especially in development environments. You can use the S3 binding with Minio too, with some configuration tweaks: 1. Set `endpoint` to the address of the Minio server, including protocol (`http://` or `https://`) and the optional port at the end. For example, `http://minio.local:9000` (the values depend on your environment). 2. `forcePathStyle` must be set to `true` 3. The value for `region` is not important; you can set it to `us-east-1`. 4. Depending on your environment, you may need to set `disableSSL` to `true` if you're connecting to Minio using a non-secure connection (using the `http://` protocol). If you are using a secure connection (`https://` protocol) but with a self-signed certificate, you may need to set `insecureSSL` to `true`. {{% /codetab %}} {{% codetab %}} For local development, the [LocalStack project](https://github.com/localstack/localstack) is used to integrate AWS S3. Follow [these instructions](https://github.com/localstack/localstack#running) to run LocalStack. To run LocalStack locally from the command line using Docker, use a `docker-compose.yaml` similar to the following: ```yaml version: "3.8" services: localstack: container_name: "cont-aws-s3" image: localstack/localstack:1.4.0 ports: - "127.0.0.1:4566:4566" environment: - DEBUG=1 - DOCKER_HOST=unix:///var/run/docker.sock volumes: - "/init-aws.sh:/etc/localstack/init/ready.d/init-aws.sh" # init hook - "${LOCALSTACK_VOLUME_DIR:-./volume}:/var/lib/localstack" - "/var/run/docker.sock:/var/run/docker.sock" ``` To use the S3 component, you need to use an existing bucket. The example above uses a [LocalStack Initialization Hook](https://docs.localstack.cloud/references/init-hooks/) to setup the bucket. To use LocalStack with your S3 binding, you need to provide the `endpoint` configuration in the component metadata. The `endpoint` is unnecessary when running against production AWS. ```yaml apiVersion: dapr.io/v1alpha1 kind: Component metadata: name: aws-s3 namespace: default spec: type: bindings.aws.s3 version: v1 metadata: - name: bucket value: conformance-test-docker - name: endpoint value: "http://localhost:4566" - name: accessKey value: "my-access" - name: secretKey value: "my-secret" - name: region value: "us-east-1" ``` {{% /codetab %}} {{% codetab %}} To use the S3 component, you need to use an existing bucket. Follow the [AWS documentation for creating a bucket](https://docs.aws.amazon.com/AmazonS3/latest/userguide/create-bucket-overview.html). {{% /codetab %}} {{< /tabs >}} ## Binding support This component supports **output binding** with the following operations: - `create` : [Create file](#create-file) - `get` : [Get file](#get-file) - `delete` : [Delete file](#delete-file) - `list`: [List file](#list-files) ### Create file To perform a create operation, invoke the AWS S3 binding with a `POST` method and the following JSON body: > Note: by default, a random UUID is generated. See below for Metadata support to set the name ```json { "operation": "create", "data": "YOUR_CONTENT" } ``` #### Share object with a presigned URL To presign an object with a specified time-to-live, use the `presignTTL` metadata key on a `create` request. Valid values for `presignTTL` are [Go duration strings](https://pkg.go.dev/maze.io/x/duration#:~:text=A%20duration%20string%20is%20a,w%22%2C%20%22y%22). {{< tabs Windows Linux >}} {{% codetab %}} ```bash curl -d "{ \"operation\": \"create\", \"data\": \"Hello World\", \"metadata\": { \"presignTTL\": \"15m\" } }" \ http://localhost:/v1.0/bindings/ ``` {{% /codetab %}} {{% codetab %}} ```bash curl -d '{ "operation": "create", "data": "Hello World", "metadata": { "presignTTL": "15m" } }' \ http://localhost:/v1.0/bindings/ ``` {{% /codetab %}} {{< /tabs >}} ##### Response The response body contains the following example JSON: ```json { "location":"https://.s3..amazonaws.com/", "versionID":"", "presignURL": "https://.s3..amazonaws.com/image.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAJJWZ7B6WCRGMKFGQ%2F20180210%2Feu-west-2%2Fs3%2Faws4_request&X-Amz-Date=20180210T171315Z&X-Amz-Expires=1800&X-Amz-Signature=12b74b0788aa036bc7c3d03b3f20c61f1f91cc9ad8873e3314255dc479a25351&X-Amz-SignedHeaders=host" } ``` #### Examples ##### Save text to a random generated UUID file {{< tabs Windows Linux >}} {{% codetab %}} On Windows, utilize cmd prompt (PowerShell has different escaping mechanism) ```bash curl -d "{ \"operation\": \"create\", \"data\": \"Hello World\" }" http://localhost:/v1.0/bindings/ ``` {{% /codetab %}} {{% codetab %}} ```bash curl -d '{ "operation": "create", "data": "Hello World" }' \ http://localhost:/v1.0/bindings/ ``` {{% /codetab %}} {{< /tabs >}} ##### Save text to a specific file {{< tabs Windows Linux >}} {{% codetab %}} ```bash curl -d "{ \"operation\": \"create\", \"data\": \"Hello World\", \"metadata\": { \"key\": \"my-test-file.txt\" } }" \ http://localhost:/v1.0/bindings/ ``` {{% /codetab %}} {{% codetab %}} ```bash curl -d '{ "operation": "create", "data": "Hello World", "metadata": { "key": "my-test-file.txt" } }' \ http://localhost:/v1.0/bindings/ ``` {{% /codetab %}} {{< /tabs >}} ##### Save a file to a object To upload a file, encode it as Base64 and let the Binding know to deserialize it: ```yaml apiVersion: dapr.io/v1alpha1 kind: Component metadata: name: spec: type: bindings.aws.s3 version: v1 metadata: - name: bucket value: mybucket - name: region value: us-west-2 - name: endpoint value: s3.us-west-2.amazonaws.com - name: accessKey value: ***************** - name: secretKey value: ***************** - name: sessionToken value: mysession - name: decodeBase64 value: - name: forcePathStyle value: ``` Then you can upload it as you would normally: {{< tabs Windows Linux >}} {{% codetab %}} ```bash curl -d "{ \"operation\": \"create\", \"data\": \"YOUR_BASE_64_CONTENT\", \"metadata\": { \"key\": \"my-test-file.jpg\" } }" http://localhost:/v1.0/bindings/ ``` {{% /codetab %}} {{% codetab %}} ```bash curl -d '{ "operation": "create", "data": "YOUR_BASE_64_CONTENT", "metadata": { "key": "my-test-file.jpg" } }' \ http://localhost:/v1.0/bindings/ ``` {{% /codetab %}} {{< /tabs >}} ##### Upload from file path To upload a file from a supplied path (relative or absolute), use the `filepath` metadata key on a `create` request that contains empty `data` fields. {{< tabs Windows Linux >}} {{% codetab %}} ```bash curl -d '{ \"operation\": \"create\", \"metadata\": { \"filePath\": \"my-test-file.txt\" }}' http://localhost:/v1.0/bindings/ ``` {{% /codetab %}} {{% codetab %}} ```bash curl -d '{ "operation": "create", "metadata": { "filePath": "my-test-file.txt" }}' \ http://localhost:/v1.0/bindings/ ``` {{% /codetab %}} {{< /tabs >}} #### Response The response body will contain the following JSON: ```json { "location":"https://.s3..amazonaws.com/", "versionID":"}} {{% codetab %}} ```bash curl -d "{ \"operation\": \"presign\", \"metadata\": { \"presignTTL\": \"15m\", \"key\": \"my-test-file.txt\" } }" \ http://localhost:/v1.0/bindings/ ``` {{% /codetab %}} {{% codetab %}} ```bash curl -d '{ "operation": "presign", "metadata": { "presignTTL": "15m", "key": "my-test-file.txt" } }' \ http://localhost:/v1.0/bindings/ ``` {{% /codetab %}} {{< /tabs >}} ##### Response The response body contains the following example JSON: ```json { "presignURL": "https://.s3..amazonaws.com/image.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAJJWZ7B6WCRGMKFGQ%2F20180210%2Feu-west-2%2Fs3%2Faws4_request&X-Amz-Date=20180210T171315Z&X-Amz-Expires=1800&X-Amz-Signature=12b74b0788aa036bc7c3d03b3f20c61f1f91cc9ad8873e3314255dc479a25351&X-Amz-SignedHeaders=host" } ``` ### Get object To perform a get file operation, invoke the AWS S3 binding with a `POST` method and the following JSON body: ```json { "operation": "get", "metadata": { "key": "my-test-file.txt" } } ``` The metadata parameters are: - `key` - the name of the object #### Example {{< tabs Windows Linux >}} {{% codetab %}} ```bash curl -d '{ \"operation\": \"get\", \"metadata\": { \"key\": \"my-test-file.txt\" }}' http://localhost:/v1.0/bindings/ ``` {{% /codetab %}} {{% codetab %}} ```bash curl -d '{ "operation": "get", "metadata": { "key": "my-test-file.txt" }}' \ http://localhost:/v1.0/bindings/ ``` {{% /codetab %}} {{< /tabs >}} #### Response The response body contains the value stored in the object. ### Delete object To perform a delete object operation, invoke the AWS S3 binding with a `POST` method and the following JSON body: ```json { "operation": "delete", "metadata": { "key": "my-test-file.txt" } } ``` The metadata parameters are: - `key` - the name of the object #### Examples ##### Delete object {{< tabs Windows Linux >}} {{% codetab %}} ```bash curl -d '{ \"operation\": \"delete\", \"metadata\": { \"key\": \"my-test-file.txt\" }}' http://localhost:/v1.0/bindings/ ``` {{% /codetab %}} {{% codetab %}} ```bash curl -d '{ "operation": "delete", "metadata": { "key": "my-test-file.txt" }}' \ http://localhost:/v1.0/bindings/ ``` {{% /codetab %}} {{< /tabs >}} #### Response An HTTP 204 (No Content) and empty body will be returned if successful. ### List objects To perform a list object operation, invoke the S3 binding with a `POST` method and the following JSON body: ```json { "operation": "list", "data": { "maxResults": 10, "prefix": "file", "marker": "hvlcCQFSOD5TD", "delimiter": "i0FvxAn2EOEL6" } } ``` The data parameters are: - `maxResults` - (optional) sets the maximum number of keys returned in the response. By default the action returns up to 1,000 key names. The response might contain fewer keys but will never contain more. - `prefix` - (optional) limits the response to keys that begin with the specified prefix. - `marker` - (optional) marker is where you want Amazon S3 to start listing from. Amazon S3 starts listing after this specified key. Marker can be any key in the bucket. The marker value may then be used in a subsequent call to request the next set of list items. - `delimiter` - (optional) A delimiter is a character you use to group keys. #### Response The response body contains the list of found objects. The list of objects will be returned as JSON array in the following form: ```json { "CommonPrefixes": null, "Contents": [ { "ETag": "\"7e94cc9b0f5226557b05a7c2565dd09f\"", "Key": "hpNdFUxruNuwm", "LastModified": "2021-08-16T06:44:14Z", "Owner": { "DisplayName": "owner name", "ID": "owner id" }, "Size": 6916, "StorageClass": "STANDARD" } ], "Delimiter": "", "EncodingType": null, "IsTruncated": true, "Marker": "hvlcCQFSOD5TD", "MaxKeys": 1, "Name": "mybucketdapr", "NextMarker": "hzaUPWjmvyi9W", "Prefix": "" } ``` ## Related links - [Basic schema for a Dapr component]({{< ref component-schema >}}) - [Bindings building block]({{< ref bindings >}}) - [How-To: Trigger application with input binding]({{< ref howto-triggers.md >}}) - [How-To: Use bindings to interface with external resources]({{< ref howto-bindings.md >}}) - [Bindings API reference]({{< ref bindings_api.md >}}) - [Authenticating to AWS]({{< ref authenticating-aws.md >}})