Commit Graph

169 Commits

Author SHA1 Message Date
Alessandro (Ale) Segala 147addebe9 Adding Cosmos DB stored procedure files in the source tree
These are not included in the compiled app and have no effect (for now). However, we need these tagged together with the release so we can use them in docs.
A separate commit (not slated for 1.7) will use Go embed to actually use these in code

Signed-off-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com>
2022-04-05 01:38:11 +00:00
Alessandro (Ale) Segala 9a28bb2f70 Added retry logic and back-off to Cosmos DB components
Signed-off-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com>
2022-03-24 22:32:05 +00:00
Alessandro (Ale) Segala 4551449f0c Workaround for Cosmos DB state store unable to launch when using AAD auth
Fixes #1603

Signed-off-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com>
2022-03-24 20:55:09 +00:00
Shubham Sharma db670d26b9
Fix transaction API for Azure Cosmos DB (#1518)
* Fix cosmos transaction API to respect ordering

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* refactor to an idiomatic way

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Update storedproc caller to pass list of operations

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Update storedproc to operate sequentially

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Add doc

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Update storedproc

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Update storedproc to not throw error on delete failure due to missing document

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

Co-authored-by: Looong Dai <long.dai@intel.com>
2022-03-08 21:13:55 -08:00
Dmitry Shmulevich 7fdebda09b
state store query API: remove preceding '.' in key names (#1500)
Signed-off-by: Dmitry Shmulevich <dmitry.shmulevich@gmail.com>
2022-02-11 15:00:43 -08:00
Dmitry Shmulevich eaecd4cb34
statestore query API: remove 'value' prefix from key names in query JSON object (#1491)
Signed-off-by: Dmitry Shmulevich <dmitry.shmulevich@gmail.com>
2022-02-09 17:17:47 -08:00
Jigar 9078f11985
Added ContentType to pubsub/binding/state request-response (#1376)
Signed-off-by: jigargandhi <jigarr.gandhi@gmail.com>

Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
Co-authored-by: Looong Dai <long.dai@intel.com>
Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
2022-01-28 10:17:04 -08:00
Gehhilfe 44199ca4dc
Fix #1351 Fail with useful error message when an invalid account name is used (#1403)
* state azure blobstorage: fail for invalid account name

Signed-off-by: Tim Burkert <burkert.tim@gmail.com>

* state azure blobstorage: Remove duplicated code and add error check for common entpoint

Signed-off-by: Tim Burkert <burkert.tim@gmail.com>

Co-authored-by: Looong Dai <long.dai@intel.com>
Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
2022-01-13 09:02:07 -08:00
Dmitry Shmulevich c7adb917f3
update license to Apache v2.0 (#1406) 2022-01-04 19:53:31 -08:00
Bernd Verst 175b09ea40
Adds retry on CosmosDB Init in case of TooManyRequests error (#1329)
* Adds retry on CosmosDB Init in case of TooManyRequests error

* Use backoff v4

* missed some permanent errors

* clean up go.mod

* fix error type casting

* Add constant for HTTP 429
2021-11-29 12:05:43 -08:00
Dmitry Shmulevich ad9ebba204
update package name (#1267) 2021-11-02 15:47:07 -07:00
Artur Souza c4303d55db
Revert "Merge pull request #1143 from jigargandhi/content-type" (#1239)
This reverts commit e373b95489, reversing
changes made to 8505c1a1db.
2021-10-27 11:41:25 -07:00
Long Dai 67185db54e
Merge branch 'master' into content-type 2021-10-26 09:35:23 +08:00
Bernd Verst aa4f949838 Adds Dapr User Agent for Cosmos DB components 2021-10-25 17:59:34 -07:00
jigargandhi a5bb76bcfa fixing newline lint issue 2021-10-25 15:27:32 +05:30
jigargandhi 54e2c1e01a Fix linting issues 2021-10-25 15:13:40 +05:30
jigargandhi a09c7ade7b Added test for blob storage 2021-10-22 18:02:01 +05:30
Jigar Gandhi 34c20b0c0f removed nil check 2021-10-21 18:32:48 +05:30
Jigar 1acb76ee1c
Merge branch 'master' into content-type 2021-10-21 12:10:08 +05:30
Dmitry Shmulevich f96a71e4d7
add metadata to query request/response (#1215) 2021-10-20 17:22:42 -07:00
Dmitry Shmulevich e369fec17e
Add StateStore query API (#1153)
Add StateStore query API implementation for MongoDB, CosmosDB
2021-10-18 13:55:26 -07:00
Bernd Verst ac85ba2a8e
Add dapr user agent for Azure Components (#1152)
* Add dapr useragent for Service Bus and EventHubs

* Add useragent for tablestorage

* send dapr useragent

* more user agents

* formatting
2021-09-27 17:04:27 -07:00
Bernd Verst 905cbc028d restructure code and add comments per code review 2021-09-22 15:51:15 -07:00
Bernd Verst 0bd6eb2e9a cover all concurrency + etag cases 2021-09-22 12:18:13 -07:00
Bernd Verst 0cd1c54ba0 Adds first-write conformance test 2021-09-21 16:33:04 -07:00
Bernd Verst 0e3dc3e79a fix linter warnings 2021-09-21 16:33:04 -07:00
Bernd Verst 90791a30fc TableStorage conformance tests pass 2021-09-21 16:33:04 -07:00
Long Dai 988fed05ab
ci: standard linter config (#1102)
* ci: standard linter config

Signed-off-by: Long <long.dai@intel.com>

* Update utils.go

* fix pulsar issue

Signed-off-by: Long <long.dai@intel.com>

Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
2021-09-21 15:21:59 -07:00
Jigar 4f27152ec9
Merge branch 'master' into content-type 2021-09-14 18:29:46 +05:30
jigargandhi ecfbbd58ad Added content type to SetRequest struct 2021-09-14 18:26:36 +05:30
Simon Leet 4f3581377e
Support DefaultIdentificationHydrator in cosmosdb state component (#1121)
Follow-up to #1104 to support AD Auth codepath as well.

`NewConfig()` and `NewConfigWithServicePrincipal()` both set the
`Config.IdentificationHydrator` to the `DefaultIdentificationHydrator`
in the underlying a8m/documentdb library. That implementation ends up
calling `reflect.Value.Elem.FieldByName()`, which requires that the
value passed to it is an interface or pointer, otherwise the Elem()
call fails to dereference which causes a panic.

cosmosdb.go passes the input to `UpsertDocument` by value today,
which is eventually passed to the `DefaultIdentificationHydrator`,
so changing the value passed to a pointer resolves the failure.

Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com>
2021-09-02 19:00:53 -07:00
Simon Leet 874d62f391
Patch regression in cosmosdb state component Init (#1119)
PR #1104 introduced a regression in cosmosdb state component `Init()`
by creating a `NewConfig()` which sets a new DefaultIdentificationHydrator,
which causes a panic later when invoked.

Patch this by restoring the configuration setting to use a struct with
`nil` `IdentificationHydrator`.
2021-09-02 10:50:10 -07:00
Simon Leet 52bac7be17
Fix parseErr return in blobstorage Init() (#1108)
Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com>
2021-09-01 13:05:32 -07:00
Alessandro (Ale) Segala a992cd19f0
Support Azure AD auth for Cosmos DB (#1104)
* Support Azure AD auth for Cosmos DB

* Fixed linting errors

* Tidying go.sum

* Removed the need for nolint:shadow

Co-authored-by: Simon Leet <31784195+CodeMonkeyLeet@users.noreply.github.com>
Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com>
2021-09-01 12:44:05 -07:00
Simon Leet ce0674fb48
Patch cosmosdb first-write without etag state test (#1069)
Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com>
2021-08-17 15:47:28 -07:00
Alessandro (Ale) Segala d0816e32a8
Updated to use common Azure auth logic (#972)
* Common Azure auth logic
- Currently implemented on secretstores/azure/keyvault and state/azure/blobstorage
- Supports Azure AD via service principal (client credentials, client certificate, MSI) - based on the previous authorizer for AKV
- Allows using other Azure clouds (China, Germany, etc)
- For Blob Storage state, supports using custom endpoints (like emulators like Azurite)

* Add environment variable aliases

* Address linter warnings

* another lint thing

* Fixed typo in method description

* Updated metadata key names so they're more consistent

* Fix test

* Some more linter things

Co-authored-by: Bernd Verst <me@bernd.dev>
Co-authored-by: Yaron Schneider <yaronsc@microsoft.com>
Co-authored-by: Bernd Verst <berndverst@users.noreply.github.com>
2021-08-10 15:06:24 -07:00
Bernd Verst 37309935ca
Adds support for TTL in CosmosDB State Store (#991)
* Adds support for TTL in CosmosDB State Store

* return explicit errors

Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
2021-07-13 19:38:19 -07:00
문찬용 b387fcdff3
add Ping operation to`store` interface. (#757)
* add `Ping` operation.

* add `Ping` implementation of GA components

* add context to an argument for Ping

* add `Ping` implementation of blobstorage

* fix error string for passing lint

* Update blobstorage.go

Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
2021-05-25 23:48:46 -07:00
Phil Kedy 688df9bda4
Removed dependency on dapr/dapr in favor of dapr/kit. Fixed go mod dependencies afterwards. Go 1.16 in go.mod. Removed accidental log dependency. (#807) 2021-04-19 12:06:10 -07:00
Daniel Beall a35a715f9c
Update state/blobstorage.go to add metadata. (#786)
Co-authored-by: Phil Kedy <phil.kedy@gmail.com>
2021-04-15 21:40:09 -07:00
Artur Souza 683246700e Add error check for required fields in cosmosdb. 2021-03-16 15:21:13 -07:00
Artur Souza 4f8e7f3358 Defaults to application/json for cosmosdb in init() 2021-03-16 15:10:40 -07:00
Artur Souza 0d44da776c Fix cosmosdb state store to handle content-type, with JSON as default. 2021-03-16 12:39:16 -07:00
Artur Souza 931048aa60
Add Feature list for state store. (#744)
* Add Feature list for state store.

* List Transactional as a feature for state store.

* Fixing comments about etag.
2021-03-05 14:55:23 -08:00
Phil Kedy a15c616bb0
Make state.GetResponse ETag to a *string (#733)
* Make state.GetResponse ETag to a *string

* Small fixes post merge of #716

Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
2021-03-03 13:28:59 -08:00
Artur Souza 2562908dee
Conf state trx + Change mongodb get result for key not found + Fix byte[] in cosmosdb (#680)
* Add state transaction conf test.

* Does not return error for key not found in mongodb.

* Rewrite state conf test.

* Handle []byte in cosmosdb state txn (#664)

Fixes: dapr/dotnet-sdk#579

The issue here is that we had missed the case of pre-marshaled bytes in
the state transaction code path, which would result in the encoding/json
package doing it's normal thing and base64 encoding the data as a JSON
string.

The fix is to use json.RawMessage to avoid that. Turns out this code can
be simplified a bit.

* Fix cosmosdb handling of byte[].

* Fix lint.

* Fix UT for cosmosdb.

* Remove config from state store conf test.

Co-authored-by: Ryan Nowak <nowakra@gmail.com>
2021-02-10 18:15:59 -08:00
Yaron Schneider 3ef025c604
change headers (#679) 2021-02-09 18:57:55 -08:00
Yaron Schneider 99256e6bec
Provide distinction between missing and empty etags (#600)
* provide distinction between missing and empty etags

* linter

* linter 2
2021-01-14 10:41:38 -08:00
Yaron Schneider f44673c311
add missing error checks for etag errors (#584) 2021-01-11 10:15:47 -08:00
Yaron Schneider 4d3e1be105
Treat etag errors across all state stores (#579)
* treat etag errors across all state stores

* linter

* linter
2021-01-08 18:59:42 -08:00
Sky/敖小剑 5dc3587f7d
Add bulk get in state store - Plan B (#532)
* add BulkGet method on state store

* add a bool value in bulk get method to tell dapr if this store supports bulk get or not

* add  BulkGetResponse for bulk get; rollback GetResponse to remove new added filed key

* update to use BulkGetResponse for bulk get

* fix lint issues

* fix lint issues

* fix lnit issues
2020-11-26 23:07:27 -08:00
halspang f647d9eedb
Upgrade golang lint to 1.31 (#490)
Upgrade golang lint to 1.31.0

https://github.com/dapr/components-contrib/issues/439
2020-10-12 12:54:15 -07:00
Mukundan Sundararajan d2ee3200cd
Properly handle serialization deserizliation for redis transaction api. (#460)
* Properly handle serialization deserizliation for redis, postgres transaction api.

* Convert marshaler to a utils function.

* Add license comment

* init error inline
2020-09-15 12:54:29 -07:00
Leon Mai 857d5652bd
Use metadata at top level instead of per request (#452)
Co-authored-by: Yaron Schneider <yaronsc@microsoft.com>
2020-09-04 09:56:27 -07:00
Young Bu Park c69d5ea2ca
Add metadata property to TransactionalRequest (#430)
* Add Metadata to transactional request

* Use new transactional request strcut
2020-08-12 09:30:37 -07:00
Shalabh Mohan Shrivastava 42668a8ae1
State Store Components changes to adhere runtime changes for removing retry options (#426)
* State Store Components changes to adhere runtime changes for removing state retry options

* resolving code review comments - common method to CheckRequestOptions
2020-08-11 10:50:31 -07:00
Sivamuthu Kumar 6085834d79
Add blob storage as state store (#348)
* Add blob storage as state store

* Updated comments for blob storage

Co-authored-by: Young Bu Park <youngp@microsoft.com>
Co-authored-by: Yaron Schneider <yaronsc@microsoft.com>
2020-07-22 08:49:20 -07:00
LMWF 1983d86849 merge 2020-07-14 13:59:29 -07:00
LMWF 15be14b9b3 cleaned up 2020-07-14 13:17:38 -07:00
LMWF 17d3d0b848 #377 - cosmos deserialization issue 2020-07-13 20:07:36 -07:00
Leon Mai a086a42f2b
Implement the Multi() interface method adding transaction support in … (#372)
* Implement the Multi() interface method adding transaction support in the cosmos db component.  The implementation
uses a stored procedure to do this.  The stored procedure is registered in Init().

* linter

* code review comments

* code review

* remove comment

* casing and de-dup

* use 'in' in query

Co-authored-by: Yaron Schneider <yaronsc@microsoft.com>
2020-06-30 13:32:41 -07:00
Mukundan Sundararajan 5c62707aba
Upgrade go version to 1.14.3 and linter version to 1.26.0 (#352)
* Upgrade to go 1.14.3 and linter version to 1.26.0
2020-05-29 15:26:33 -07:00
Young Bu Park ee97c3c56a
Use dapr logger (#238)
* Use dapr logger

* skip linter false alarm
2020-02-28 15:49:49 -08:00
REDDY PRASAD 5f3f850852
Add GCP Secret Manager (#199)
*  Add GCP Secret Manager

* Update tests

* Fix lint errors

* Update as per review

* Run go mod tidy

Co-authored-by: Yaron Schneider <yaronsc@microsoft.com>
2020-01-30 21:51:52 -08:00
Ivan Gavryliuk 84c4e6bf8e Azure Table Storage state (#197)
* initial scaffolding

* tidy go.mod

* adding delete functionality and tests

* updating Readme.md

* ETag concurrency support and github review fixes

* removing unused isConflictError

* ping build, some issue with CI timing out

Co-authored-by: Young Bu Park <youngp@microsoft.com>
2020-01-28 09:20:18 -08:00
Francisco Beltrao 1a37f892aa Move SQLServer state store from azure folder (#181) 2020-01-07 23:38:44 -08:00
Francisco Beltrao db8b6135d6 Rename SQLServer state ctor to existing pattern (#176) 2020-01-07 06:17:13 -08:00
Francisco Beltrao 70381d8312 Add sql server state (#158)
* Add sql server state

* Fix lint

* Disable docker based tests

* Rename store to SQLServer
Use env variable DAPR_TEST_SQL_CONNSTRING to enable integratoion test

* Add support to Azure SQL on integration test

Co-authored-by: Yaron Schneider <yaronsc@microsoft.com>
2020-01-06 09:14:13 -08:00
Shalabh Mohan Shrivastava bcd23db009
Changes to include Package names pubsub/secret stores/state to be consistent with company name (#137)
* [Commit - PubSub changes ]

Changing the names of the packages directory so to be consistent with name of the company .

It is to be noted that package name can not have hyphen "-" in the name.
So changing the directory names and putting under company names.

For example - servicebus package would be

dapr/components-contrib/pubsub/azure/servicebus

* Commit 2 - Changes for Secret Stores

* Commit 3 - for the state store changes
2019-11-22 12:46:42 -08:00