Commit Graph

115 Commits

Author SHA1 Message Date
Bernd Verst 5d7367c335 CosmosDB State: Use document enveloped, handle doc NotFound
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-09-09 00:24:48 +00:00
Bernd Verst f1a2dd75f5 Return CosmosDB State Features correctly
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-09-08 16:04:19 -07:00
Ryan Lettieri 8fd5822a5c Removing unncessary features from cosmosdb
Signed-off-by: Ryan Lettieri <ryanLettieri@microsoft.com>
2022-09-08 15:25:34 -07:00
Ryan Lettieri 9189ddc4dc Adressing review comments for cosmosdb statestore track2sdk
Signed-off-by: Ryan Lettieri <ryanLettieri@microsoft.com>
2022-09-08 15:25:34 -07:00
Ryan Lettieri e012a807b7 Upgrading cosmosdb statestore to track2SDK
Signed-off-by: Ryan Lettieri <ryanLettieri@microsoft.com>
2022-09-08 15:25:33 -07:00
Alessandro (Ale) Segala bcea284c7b
Components' init functions should return interfaces (#1997)
* Components' init functions should return interfaces

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* 💄

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* pin latest berndverst contrib fork

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Fixed component registration in tests

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Updated all cert tests

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-08-25 18:27:45 -07:00
Bernd Verst e87cd5e4cb
Go 1.19 support and linter fixes (#1975)
* Go 1.19 support and linter fixes

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Update workflows for Go1.19 and new linter version

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Remove unnecessary space in nolint directive

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* disable additional linters which aren't used because of Go generics

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* enable gosec linter again

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Update bindings/zeebe/command/publish_message_test.go

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Update bindings/output_binding.go

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Use prepared statement for mysql table creation

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Ping is not ping

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* c'mon linter

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Fix MySQL gosec issue

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* revert mysql to be fixed later

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-08-18 00:45:23 -07:00
Bernd Verst 1edd408b9b
Fix CosmosDB state store handling of nulls (#1974) 2022-08-17 21:20:25 -07:00
qiangmzsx 79a3cd990a
Removing the use of the ioutil package And Fix CVE-2021-42576 (#1954)
* fix CVE-2021-42576

Signed-off-by: qiangmzsx <qiangmzsx@gmail.com>

* Removing the use of the ioutil package

Signed-off-by: qiangmzsx <qiangmzsx@gmail.com>

* Update middleware/http/nethttpadaptor/nethttpadaptor.go

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

Signed-off-by: qiangmzsx <qiangmzsx@gmail.com>
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-08-10 13:45:00 -07:00
Alessandro (Ale) Segala a2f3a84b96
Moved authentication to be an internal pkg (#1855)
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-07-06 11:05:34 -07:00
Pravin Pushkar 3f85dd6702
adding query api to the feature list (#1775)
Signed-off-by: Pravin Pushkar <ppushkar@microsoft.com>

Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-06-07 08:38:02 -07:00
Alessandro (Ale) Segala e6c67356cd
Updated to Go 1.18 (#1697)
* Updated to Go 1.18

Signed-off-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com>

* Added go.work file
With Go 1.18, this allows gopls (the Go language server used for example in VS Code) to work inside test apps too.
See: https://go.dev/doc/tutorial/workspaces

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Removed go.work

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* 💄

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-05-06 13:44:45 -07:00
ItalyPaleAle 0bc35571a8 💄💄💄 #PickyLinter
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-04-08 11:45:39 -07:00
ItalyPaleAle 9a8aae1818 💄
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-04-08 11:42:30 -07:00
Alessandro (Ale) Segala 869d9712aa
Merge branch 'master' into cosmosdb-sproc-embed 2022-04-08 11:26:45 -07:00
Alessandro (Ale) Segala 661d1b38b5 💄
Signed-off-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com>
2022-04-05 21:28:00 +00:00
Alessandro (Ale) Segala 612f6e6ec3
Cosmos DB: Delete does not need to query for the doc first
This, which is possible now because of improvements in the upstream SDK, should speed up delete operations significantly (and reducing RU usage).
Also, this should fix potential concurrency issues.

Signed-off-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com>
2022-04-05 21:20:57 +00:00
Alessandro (Ale) Segala 407936745e
Yet another Cosmos DB auth fix
This is fixed by upgrading the upstream SDK
Also added a missing handling of throttling in the Delete method

Signed-off-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com>
2022-04-05 21:01:22 +00:00
Alessandro (Ale) Segala 559e68daf4 Use Go Embed to include the Cosmos DB stored procedures code in the compiled app
Signed-off-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com>
2022-04-05 01:46:26 +00:00
Alessandro (Ale) Segala fd7600cdde Added license headers
Signed-off-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com>
2022-04-05 01:44:44 +00:00
Alessandro (Ale) Segala 650fac6da6 Adding Cosmos DB stored procedure files in the source tree
These are not included in the compiled app and have no effect (for now). However, we need these tagged together with the release so we can use them in docs.
A separate commit (not slated for 1.7) will use Go embed to actually use these in code

Signed-off-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com>
2022-04-05 01:44:44 +00:00
Alessandro (Ale) Segala 4ae8f5e978 Added license headers
Signed-off-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com>
2022-04-05 01:41:44 +00:00
Alessandro (Ale) Segala 147addebe9 Adding Cosmos DB stored procedure files in the source tree
These are not included in the compiled app and have no effect (for now). However, we need these tagged together with the release so we can use them in docs.
A separate commit (not slated for 1.7) will use Go embed to actually use these in code

Signed-off-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com>
2022-04-05 01:38:11 +00:00
Alessandro (Ale) Segala 9a28bb2f70 Added retry logic and back-off to Cosmos DB components
Signed-off-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com>
2022-03-24 22:32:05 +00:00
Alessandro (Ale) Segala 4551449f0c Workaround for Cosmos DB state store unable to launch when using AAD auth
Fixes #1603

Signed-off-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com>
2022-03-24 20:55:09 +00:00
Shubham Sharma db670d26b9
Fix transaction API for Azure Cosmos DB (#1518)
* Fix cosmos transaction API to respect ordering

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* refactor to an idiomatic way

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Update storedproc caller to pass list of operations

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Update storedproc to operate sequentially

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Add doc

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Update storedproc

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Update storedproc to not throw error on delete failure due to missing document

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

Co-authored-by: Looong Dai <long.dai@intel.com>
2022-03-08 21:13:55 -08:00
Dmitry Shmulevich 7fdebda09b
state store query API: remove preceding '.' in key names (#1500)
Signed-off-by: Dmitry Shmulevich <dmitry.shmulevich@gmail.com>
2022-02-11 15:00:43 -08:00
Dmitry Shmulevich eaecd4cb34
statestore query API: remove 'value' prefix from key names in query JSON object (#1491)
Signed-off-by: Dmitry Shmulevich <dmitry.shmulevich@gmail.com>
2022-02-09 17:17:47 -08:00
Dmitry Shmulevich c7adb917f3
update license to Apache v2.0 (#1406) 2022-01-04 19:53:31 -08:00
Bernd Verst 175b09ea40
Adds retry on CosmosDB Init in case of TooManyRequests error (#1329)
* Adds retry on CosmosDB Init in case of TooManyRequests error

* Use backoff v4

* missed some permanent errors

* clean up go.mod

* fix error type casting

* Add constant for HTTP 429
2021-11-29 12:05:43 -08:00
Dmitry Shmulevich ad9ebba204
update package name (#1267) 2021-11-02 15:47:07 -07:00
Bernd Verst aa4f949838 Adds Dapr User Agent for Cosmos DB components 2021-10-25 17:59:34 -07:00
Dmitry Shmulevich f96a71e4d7
add metadata to query request/response (#1215) 2021-10-20 17:22:42 -07:00
Dmitry Shmulevich e369fec17e
Add StateStore query API (#1153)
Add StateStore query API implementation for MongoDB, CosmosDB
2021-10-18 13:55:26 -07:00
Long Dai 988fed05ab
ci: standard linter config (#1102)
* ci: standard linter config

Signed-off-by: Long <long.dai@intel.com>

* Update utils.go

* fix pulsar issue

Signed-off-by: Long <long.dai@intel.com>

Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
2021-09-21 15:21:59 -07:00
Simon Leet 4f3581377e
Support DefaultIdentificationHydrator in cosmosdb state component (#1121)
Follow-up to #1104 to support AD Auth codepath as well.

`NewConfig()` and `NewConfigWithServicePrincipal()` both set the
`Config.IdentificationHydrator` to the `DefaultIdentificationHydrator`
in the underlying a8m/documentdb library. That implementation ends up
calling `reflect.Value.Elem.FieldByName()`, which requires that the
value passed to it is an interface or pointer, otherwise the Elem()
call fails to dereference which causes a panic.

cosmosdb.go passes the input to `UpsertDocument` by value today,
which is eventually passed to the `DefaultIdentificationHydrator`,
so changing the value passed to a pointer resolves the failure.

Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com>
2021-09-02 19:00:53 -07:00
Simon Leet 874d62f391
Patch regression in cosmosdb state component Init (#1119)
PR #1104 introduced a regression in cosmosdb state component `Init()`
by creating a `NewConfig()` which sets a new DefaultIdentificationHydrator,
which causes a panic later when invoked.

Patch this by restoring the configuration setting to use a struct with
`nil` `IdentificationHydrator`.
2021-09-02 10:50:10 -07:00
Alessandro (Ale) Segala a992cd19f0
Support Azure AD auth for Cosmos DB (#1104)
* Support Azure AD auth for Cosmos DB

* Fixed linting errors

* Tidying go.sum

* Removed the need for nolint:shadow

Co-authored-by: Simon Leet <31784195+CodeMonkeyLeet@users.noreply.github.com>
Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com>
2021-09-01 12:44:05 -07:00
Simon Leet ce0674fb48
Patch cosmosdb first-write without etag state test (#1069)
Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com>
2021-08-17 15:47:28 -07:00
Bernd Verst 37309935ca
Adds support for TTL in CosmosDB State Store (#991)
* Adds support for TTL in CosmosDB State Store

* return explicit errors

Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
2021-07-13 19:38:19 -07:00
문찬용 b387fcdff3
add Ping operation to`store` interface. (#757)
* add `Ping` operation.

* add `Ping` implementation of GA components

* add context to an argument for Ping

* add `Ping` implementation of blobstorage

* fix error string for passing lint

* Update blobstorage.go

Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
2021-05-25 23:48:46 -07:00
Phil Kedy 688df9bda4
Removed dependency on dapr/dapr in favor of dapr/kit. Fixed go mod dependencies afterwards. Go 1.16 in go.mod. Removed accidental log dependency. (#807) 2021-04-19 12:06:10 -07:00
Artur Souza 683246700e Add error check for required fields in cosmosdb. 2021-03-16 15:21:13 -07:00
Artur Souza 4f8e7f3358 Defaults to application/json for cosmosdb in init() 2021-03-16 15:10:40 -07:00
Artur Souza 0d44da776c Fix cosmosdb state store to handle content-type, with JSON as default. 2021-03-16 12:39:16 -07:00
Artur Souza 931048aa60
Add Feature list for state store. (#744)
* Add Feature list for state store.

* List Transactional as a feature for state store.

* Fixing comments about etag.
2021-03-05 14:55:23 -08:00
Phil Kedy a15c616bb0
Make state.GetResponse ETag to a *string (#733)
* Make state.GetResponse ETag to a *string

* Small fixes post merge of #716

Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
2021-03-03 13:28:59 -08:00
Artur Souza 2562908dee
Conf state trx + Change mongodb get result for key not found + Fix byte[] in cosmosdb (#680)
* Add state transaction conf test.

* Does not return error for key not found in mongodb.

* Rewrite state conf test.

* Handle []byte in cosmosdb state txn (#664)

Fixes: dapr/dotnet-sdk#579

The issue here is that we had missed the case of pre-marshaled bytes in
the state transaction code path, which would result in the encoding/json
package doing it's normal thing and base64 encoding the data as a JSON
string.

The fix is to use json.RawMessage to avoid that. Turns out this code can
be simplified a bit.

* Fix cosmosdb handling of byte[].

* Fix lint.

* Fix UT for cosmosdb.

* Remove config from state store conf test.

Co-authored-by: Ryan Nowak <nowakra@gmail.com>
2021-02-10 18:15:59 -08:00
Yaron Schneider 3ef025c604
change headers (#679) 2021-02-09 18:57:55 -08:00
Yaron Schneider 99256e6bec
Provide distinction between missing and empty etags (#600)
* provide distinction between missing and empty etags

* linter

* linter 2
2021-01-14 10:41:38 -08:00
Yaron Schneider f44673c311
add missing error checks for etag errors (#584) 2021-01-11 10:15:47 -08:00
Yaron Schneider 4d3e1be105
Treat etag errors across all state stores (#579)
* treat etag errors across all state stores

* linter

* linter
2021-01-08 18:59:42 -08:00
Sky/敖小剑 5dc3587f7d
Add bulk get in state store - Plan B (#532)
* add BulkGet method on state store

* add a bool value in bulk get method to tell dapr if this store supports bulk get or not

* add  BulkGetResponse for bulk get; rollback GetResponse to remove new added filed key

* update to use BulkGetResponse for bulk get

* fix lint issues

* fix lint issues

* fix lnit issues
2020-11-26 23:07:27 -08:00
halspang f647d9eedb
Upgrade golang lint to 1.31 (#490)
Upgrade golang lint to 1.31.0

https://github.com/dapr/components-contrib/issues/439
2020-10-12 12:54:15 -07:00
Mukundan Sundararajan d2ee3200cd
Properly handle serialization deserizliation for redis transaction api. (#460)
* Properly handle serialization deserizliation for redis, postgres transaction api.

* Convert marshaler to a utils function.

* Add license comment

* init error inline
2020-09-15 12:54:29 -07:00
Leon Mai 857d5652bd
Use metadata at top level instead of per request (#452)
Co-authored-by: Yaron Schneider <yaronsc@microsoft.com>
2020-09-04 09:56:27 -07:00
Young Bu Park c69d5ea2ca
Add metadata property to TransactionalRequest (#430)
* Add Metadata to transactional request

* Use new transactional request strcut
2020-08-12 09:30:37 -07:00
Shalabh Mohan Shrivastava 42668a8ae1
State Store Components changes to adhere runtime changes for removing retry options (#426)
* State Store Components changes to adhere runtime changes for removing state retry options

* resolving code review comments - common method to CheckRequestOptions
2020-08-11 10:50:31 -07:00
LMWF 1983d86849 merge 2020-07-14 13:59:29 -07:00
LMWF 15be14b9b3 cleaned up 2020-07-14 13:17:38 -07:00
LMWF 17d3d0b848 #377 - cosmos deserialization issue 2020-07-13 20:07:36 -07:00
Leon Mai a086a42f2b
Implement the Multi() interface method adding transaction support in … (#372)
* Implement the Multi() interface method adding transaction support in the cosmos db component.  The implementation
uses a stored procedure to do this.  The stored procedure is registered in Init().

* linter

* code review comments

* code review

* remove comment

* casing and de-dup

* use 'in' in query

Co-authored-by: Yaron Schneider <yaronsc@microsoft.com>
2020-06-30 13:32:41 -07:00
Mukundan Sundararajan 5c62707aba
Upgrade go version to 1.14.3 and linter version to 1.26.0 (#352)
* Upgrade to go 1.14.3 and linter version to 1.26.0
2020-05-29 15:26:33 -07:00
Young Bu Park ee97c3c56a
Use dapr logger (#238)
* Use dapr logger

* skip linter false alarm
2020-02-28 15:49:49 -08:00
Shalabh Mohan Shrivastava bcd23db009
Changes to include Package names pubsub/secret stores/state to be consistent with company name (#137)
* [Commit - PubSub changes ]

Changing the names of the packages directory so to be consistent with name of the company .

It is to be noted that package name can not have hyphen "-" in the name.
So changing the directory names and putting under company names.

For example - servicebus package would be

dapr/components-contrib/pubsub/azure/servicebus

* Commit 2 - Changes for Secret Stores

* Commit 3 - for the state store changes
2019-11-22 12:46:42 -08:00