Commit Graph

54 Commits

Author SHA1 Message Date
Josh van Leeuwen 31ccb5f169
Conformance Tests: State- ` ttlExpireTime` (#2863)
Signed-off-by: joshvanl <me@joshvanl.dev>
Signed-off-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com>
Co-authored-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com>
2023-08-04 12:11:52 -07:00
Alessandro (Ale) Segala b10ce96b49
State stores: expose TTL as a feature (#2987)
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2023-08-04 09:38:22 -07:00
Josh van Leeuwen de01000c9b
ETCD State store v2 (#2932)
Signed-off-by: joshvanl <me@joshvanl.dev>
2023-06-26 16:08:48 +00:00
Alessandro (Ale) Segala abaf74150e
Conformance tests: always enable required operations (#2851)
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2023-05-22 19:43:20 +00:00
Alessandro (Ale) Segala 1f27fd0e8b
Add conformance tests for etags in Bulk operations (#2843)
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Signed-off-by: Artur Souza <artursouza.ms@outlook.com>
Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
2023-05-17 16:28:29 -07:00
Alessandro (Ale) Segala e30d140479
Remove native/transactional BulkSet/BulkDelete from state stores (#2834)
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2023-05-17 14:56:34 -07:00
Alessandro (Ale) Segala 4f94da95cf
Refactor state store bulk operations (#2747)
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2023-04-07 19:19:47 +00:00
Alessandro (Ale) Segala 33f9c6ecef
SQLite: defer cancel's in select statements (#2717)
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2023-03-30 16:17:59 -07:00
Deepanshu Agarwal d6ce7bb5c3
Add MySQL TTL and Cleanup (#2641)
Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Co-authored-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Co-authored-by: Bernd Verst <github@bernd.dev>
2023-03-30 15:45:57 -07:00
Alessandro (Ale) Segala 04d1e71ce7
State stores should always allow values that are empty strings (#2723)
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2023-03-29 23:08:12 +00:00
Alessandro (Ale) Segala 029040c6a9
Fix inconsistency with TTL and Etags in SQL databases + fix SQL Server cert tests (#2688)
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Signed-off-by: Bernd Verst <github@bernd.dev>
Co-authored-by: Bernd Verst <github@bernd.dev>
2023-03-25 02:34:30 +00:00
Alessandro (Ale) Segala 6389cc207c
Add BulkGet support for SQLite and Postgres state stores (#2704)
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Signed-off-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com>
Co-authored-by: Josh van Leeuwen <me@joshvanl.dev>
Co-authored-by: Bernd Verst <github@bernd.dev>
2023-03-25 02:18:43 +00:00
Alessandro (Ale) Segala bc52e25c85
Many fixes to Oracle Database state store - now beta (#2702)
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Co-authored-by: Bernd Verst <github@bernd.dev>
2023-03-25 00:18:35 +00:00
joshvanl db56a28581 Don't case extra allocations. Revert 70s state wait for ttl deletion
Signed-off-by: joshvanl <me@joshvanl.dev>
2023-02-17 17:46:15 +00:00
Alessandro (Ale) Segala 974d2acd4e
Merge branch 'master' into state-store-ttl-mongodb 2023-02-17 17:37:01 +00:00
Josh van Leeuwen d098e38d6a
Propagate context from caller to appropriate places in the code (#2474)
* Propagates contexts to callers where appropriate.

Signed-off-by: joshvanl <me@joshvanl.dev>

* Updates units tests with new func signature

Signed-off-by: joshvanl <me@joshvanl.dev>

* Fix linting errors

Signed-off-by: joshvanl <me@joshvanl.dev>

* Add atomic gate to alicloud rocketmq close channel.

Signed-off-by: joshvanl <me@joshvanl.dev>

* bindings/aws/kinesis use a separate ctx variable name

Signed-off-by: joshvanl <me@joshvanl.dev>

* binding/kafka: use atomic to prevent closing the channel twice

Signed-off-by: joshvanl <me@joshvanl.dev>

* bindings/mqtt3: use atomic bool to prevent close channel being closed multiple times

Signed-off-by: joshvanl <me@joshvanl.dev>

* bindings/mqtt3: use Background context for handle operations:w

Signed-off-by: joshvanl <me@joshvanl.dev>

* state/cocroachdb: add context to Ping()

Signed-off-by: joshvanl <me@joshvanl.dev>

* bindings/postgres: add comment explaining use of context.

Signed-off-by: joshvanl <me@joshvanl.dev>

* Adds comment header to health/pinger.go

Signed-off-by: joshvanl <me@joshvanl.dev>

* pubsub/aws/snssqs: add waitgroup to wait for all go routines to finish
and block on Close(). Shuts down the subscription if there are no topic
handlers.

Signed-off-by: joshvanl <me@joshvanl.dev>

* pubsub/mqtt3: add atomic bool to prevent multiple channel closes. Add
wait group to block close on all goroutines to finish.

Signed-off-by: joshvanl <me@joshvanl.dev>

* pubsub/rabbitmq: fixes race conditions, uses atomic to prevent multiple
closes, add wait group to block close on all goroutines

Signed-off-by: joshvanl <me@joshvanl.dev>

* pubsub/redis: revert ctx passed when it could be cancelled. Add wait
group wait when closing.

Signed-off-by: joshvanl <me@joshvanl.dev>

* state/postges: pass context in init, and wait group on close.

Signed-off-by: joshvanl <me@joshvanl.dev>

* Update all `Ping()` to `PingContext()` where possible.

Signed-off-by: joshvanl <me@joshvanl.dev>

* state/in-memory: add atomic bool to prevent closing channel multiple
times. Add wait group to block on close()

Signed-off-by: joshvanl <me@joshvanl.dev>

* state/mysql: don't use same ctx variable name

Signed-off-by: joshvanl <me@joshvanl.dev>

* Pass correct loop context to redis go routines

Signed-off-by: joshvanl <me@joshvanl.dev>

* Rename context when creating timeouts in state

Signed-off-by: joshvanl <me@joshvanl.dev>

* Remove state.Features() from requiring a context

Signed-off-by: joshvanl <me@joshvanl.dev>

* Revert wasm request handle Close func to be without context to
implement io.Closer interface. Add 5 second timeout. Add io.Closer
assertion in test.

Signed-off-by: joshvanl <me@joshvanl.dev>

* Remove superfluous go lint vet directive

Signed-off-by: joshvanl <me@joshvanl.dev>

* Change Configuration Init function to take context

Signed-off-by: joshvanl <me@joshvanl.dev>

* Updates input binding interface to include a `Close() error` function. `Close`
blocks until all resources have been released and go routines have returned.

Signed-off-by: joshvanl <me@joshvanl.dev>

* Change `Close() error` in input binding struct to `io.Closer` interface.

Signed-off-by: joshvanl <me@joshvanl.dev>

* Update go.mod files to point to dapr/dapr PR https://github.com/dapr/dapr/pull/5831

Signed-off-by: joshvanl <me@joshvanl.dev>

* pubsub/redis: watch closeCh to shutdown worker instead of init context.

Signed-off-by: joshvanl <me@joshvanl.dev>

* pubsub/aws/snssqs + bindings/kubemq: ensure closeCh is caught so Close
correctly returns

Signed-off-by: joshvanl <me@joshvanl.dev>

* Close kubemq binding client on close. Ensure kafka consumer channel
cannot be closed more than once.

Signed-off-by: joshvanl <me@joshvanl.dev>

* Tweaks

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Fixed cert tests

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* binding/mqtt3: add inline Background context instead of passing to
handleMessage

Signed-off-by: joshvanl <me@joshvanl.dev>

* pubsub/mqtt3: remove context from createSubscriberClientOptions

Signed-off-by: joshvanl <me@joshvanl.dev>

* pubsub/mqtt3: Remove `ResetConnection` func

Signed-off-by: joshvanl <me@joshvanl.dev>

* pubsub/kafka: Don't resubscribe if Subscribe is cancelled.

Signed-off-by: joshvanl <me@joshvanl.dev>

* binding/mqtt3: don't use context to control establishing connection

Signed-off-by: joshvanl <me@joshvanl.dev>

* bindings/mqtt3: Fix linting errors

Signed-off-by: joshvanl <me@joshvanl.dev>

---------

Signed-off-by: joshvanl <me@joshvanl.dev>
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Co-authored-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
2023-02-16 14:18:35 -08:00
joshvanl 7402ff5cf9 Add TTL support to mongodb state store
Signed-off-by: joshvanl <me@joshvanl.dev>
2023-02-15 16:43:27 +00:00
Alessandro (Ale) Segala ae48594e7e
Merge branch 'master' into postgres-pgx 2022-12-22 15:39:08 -08:00
ItalyPaleAle 80b1bb14b7 Changed postgres state store to use pgx instead of db/sql
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-12-20 22:45:38 +00:00
ItalyPaleAle 717bfcf23a Enabled conf tests for state.cloudflare.kv
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-12-14 00:34:09 +00:00
ItalyPaleAle a1c863d7fd Small fixes
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-12-13 10:11:36 -08:00
1046102779 2dd7433a59 feature: add context to state API
Signed-off-by: 1046102779 <seachen@tencent.com>
2022-12-08 15:56:51 +08:00
ItalyPaleAle 4d4b98d339 Add conformance test for TTL in state stores
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-11-22 04:33:47 +00:00
1046102779 08f665cb83 feature: add context to state API
Signed-off-by: 1046102779 <seachen@tencent.com>
2022-10-21 21:05:47 +08:00
1046102779 5a367b401a feature: add context to state API
Signed-off-by: 1046102779 <seachen@tencent.com>
2022-10-21 17:25:14 +08:00
Marcos Candeia 8964d8bf41
Define common metadata across components (#1994)
* Add base metadata for all components

Signed-off-by: Marcos Candeia <marrcooos@gmail.com>

* Fix lint issues on consul and mysql tests

Signed-off-by: Marcos Candeia <marrcooos@gmail.com>

* Pin candeia/dapr runtime version

Signed-off-by: Marcos Candeia <marrcooos@gmail.com>

* Add replace directive to use mcandeia/dapr on all go submodules

Signed-off-by: Marcos Candeia <marrcooos@gmail.com>

* Fixed runtime config instantiation

Signed-off-by: Marcos Candeia <marrcooos@gmail.com>

Signed-off-by: Marcos Candeia <marrcooos@gmail.com>
2022-08-25 10:10:53 -07:00
Bernd Verst e87cd5e4cb
Go 1.19 support and linter fixes (#1975)
* Go 1.19 support and linter fixes

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Update workflows for Go1.19 and new linter version

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Remove unnecessary space in nolint directive

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* disable additional linters which aren't used because of Go generics

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* enable gosec linter again

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Update bindings/zeebe/command/publish_message_test.go

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Update bindings/output_binding.go

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Use prepared statement for mysql table creation

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Ping is not ping

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* c'mon linter

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Fix MySQL gosec issue

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* revert mysql to be fixed later

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-08-18 00:45:23 -07:00
ItalyPaleAle 12d2e7852c Added more logging to state conformance tests
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-07-07 15:50:22 -07:00
deepanshuagarwal a0ef46265b
Adding Pinger interface and Removing mandatory nature of Ping fuction (#1784)
* add `Pinger` interface.

Signed-off-by: chanyong.moon <dev.chanyongmoon@gmail.com>

* Convert Ping fuction to optional

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Removing unrequired Ping implementations

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Addressing comments

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

Co-authored-by: chanyong.moon <dev.chanyongmoon@gmail.com>
2022-06-14 08:27:54 -07:00
Mukundan Sundararajan 22d8bb2ab9 unexport struct
Signed-off-by: Mukundan Sundararajan <msundar.ms@outlook.com>
2022-03-16 09:15:58 +05:30
Mukundan Sundararajan 9545d8204d
Merge branch 'master' into fix-mongodb-canonical 2022-03-16 09:06:40 +05:30
Mukundan Sundararajan 83914e8eed marshall mongodb bson into relaxed json
Signed-off-by: Mukundan Sundararajan <msundar.ms@outlook.com>
2022-03-16 02:21:53 +05:30
Shubham Sharma f4ad3bb5e3 Add conf test for transaction ordering
Signed-off-by: Shubham Sharma <shubhash@microsoft.com>
2022-03-03 12:47:01 +05:30
Dmitry Shmulevich f819b290f7
redis state store: implement state query API (#1488)
Signed-off-by: Dmitry Shmulevich <dmitry.shmulevich@gmail.com>
2022-03-02 10:51:09 -08:00
Dmitry Shmulevich 7fdebda09b
state store query API: remove preceding '.' in key names (#1500)
Signed-off-by: Dmitry Shmulevich <dmitry.shmulevich@gmail.com>
2022-02-11 15:00:43 -08:00
Dmitry Shmulevich 1cb659c0b8
update conformance tests (#1492)
Signed-off-by: Dmitry Shmulevich <dmitry.shmulevich@gmail.com>
2022-02-09 17:58:48 -08:00
Ricardo Corrêa 8297fc929b
Query StateStore API implementation for PostgreSQL StateStore (#1457)
* query state integration with dbaccess (postgresql)

Signed-off-by: Ricardo Corrêa <r.c.correa@outlook.com>

* query state integration with postgresql component

Signed-off-by: Ricardo Corrêa <r.c.correa@outlook.com>

* query state for postgresql

Signed-off-by: Ricardo Corrêa <r.c.correa@outlook.com>

* fix: linter error 'rowserrcheck'

Signed-off-by: Ricardo Corrêa <r.c.correa@outlook.com>

* query state integration with dbaccess (postgresql)

Signed-off-by: Ricardo Corrêa <r.c.correa@outlook.com>

* query state integration with postgresql component

Signed-off-by: Ricardo Corrêa <r.c.correa@outlook.com>

* fix: linter error 'rowserrcheck'

Signed-off-by: Ricardo Corrêa <r.c.correa@outlook.com>

* fix: linter error 'goimports -ed with -local github.com/dapr/'

Signed-off-by: Ricardo Corrêa <r.c.correa@outlook.com>

* performance improvementt in PosgreSQL Query API

Signed-off-by: Ricardo Corrêa <r.c.correa@outlook.com>

* added conformance test for PostgreSQL Query API

Signed-off-by: Ricardo Corrêa <r.c.correa@outlook.com>

Co-authored-by: Looong Dai <long.dai@intel.com>
Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
2022-02-02 18:14:18 -08:00
Dmitry Shmulevich c7adb917f3
update license to Apache v2.0 (#1406) 2022-01-04 19:53:31 -08:00
Dmitry Shmulevich 3c52b73608
fixed typo in Query JSON (#1344) 2021-11-29 09:44:28 -08:00
Ian Luo 117408f76d
add state conformance test for postgresql #1117 (#1118)
* add state conformance test for postgresql #1117

* make postgresql pass comformance test

* include postgresql to github's workflow

* add newline at end of file for Linux text file

* enhance return

* add newline at end of file for valid Linux text file

* fix lint issue

* handle byte[] in special way so that to stick to json type in DB

* use jsonb instead of json

* fix lint issue

* doesn't allow "", a special case

* enhance assertEquals

* remove unused 'expectedReadResponse' to fix lint issue

* go fmt to fix lint issue

* fix lint issue

Co-authored-by: Simon Leet <31784195+CodeMonkeyLeet@users.noreply.github.com>
Co-authored-by: Yaron Schneider <yaronsc@microsoft.com>
Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
Co-authored-by: Long Dai <long.dai@intel.com>
Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com>
2021-10-21 14:56:14 -07:00
Dmitry Shmulevich f96a71e4d7
add metadata to query request/response (#1215) 2021-10-20 17:22:42 -07:00
Dmitry Shmulevich e369fec17e
Add StateStore query API (#1153)
Add StateStore query API implementation for MongoDB, CosmosDB
2021-10-18 13:55:26 -07:00
Bernd Verst 059dc8c0d2
Conformance tests & bugfixes for SQLServer (#1078)
* enable sqlserver conformance test

* Update SQL Server conformance test setup

* Fix sqlserver bug introduced in Go 1.16

* sqlserver almost conformant

* Fix conformance tests and stored procedure

* all sqlserver tests passing

* Minor touchup

* lint

* Remove unnecessary condition

* Add conformance test to GitHub workflow

* remove env variable

* Update stored procedure

* Simplify error handling

* Update state/sqlserver/sqlserver.go

Co-authored-by: Simon Leet <31784195+CodeMonkeyLeet@users.noreply.github.com>

* Update state/sqlserver/sqlserver.go

Co-authored-by: Simon Leet <31784195+CodeMonkeyLeet@users.noreply.github.com>

Co-authored-by: Bernd Verst <me@bernd.dev>
Co-authored-by: Simon Leet <31784195+CodeMonkeyLeet@users.noreply.github.com>
Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com>
2021-08-19 17:47:56 -07:00
Long Dai 10a2e5ee85
state: correct FIRST_WRITE (#995)
* state: correct FIRST_WRITE

Signed-off-by: Long Dai <long0dai@foxmail.com>

* add a test case

Signed-off-by: Long <long0dai@foxmail.com>
2021-08-03 17:26:45 -07:00
문찬용 b387fcdff3
add Ping operation to`store` interface. (#757)
* add `Ping` operation.

* add `Ping` implementation of GA components

* add context to an argument for Ping

* add `Ping` implementation of blobstorage

* fix error string for passing lint

* Update blobstorage.go

Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
2021-05-25 23:48:46 -07:00
Artur Souza 931048aa60
Add Feature list for state store. (#744)
* Add Feature list for state store.

* List Transactional as a feature for state store.

* Fixing comments about etag.
2021-03-05 14:55:23 -08:00
Phil Kedy a15c616bb0
Make state.GetResponse ETag to a *string (#733)
* Make state.GetResponse ETag to a *string

* Small fixes post merge of #716

Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
2021-03-03 13:28:59 -08:00
halspang cb3c881d39
Add ETag operation to statestore conformance test (#716)
* Add ETag operation to statestore conformance test

ETags are used to enable concurrency guarantees and are part of the
statestore specification. Not all stores need to use them, however
if the store chooses to, it should be part of the certification
process. This commit adds a simple ETag test scenario that verifies
the basic behaviors of ETags.

https://github.com/dapr/components-contrib/issues/711

* Update tests.yml

* Error on mongodb when etag does not match on delete.

Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
2021-03-03 12:27:10 -08:00
Phil Kedy 715cc8d688
Refactored conformance tests to use a new config package (#712)
* Refactored the conformance tests to use a new config package
Removed dependency on k8s apimachinery sets

* Comment tweak

* Renamed kit package to internal
2021-03-01 13:03:39 -08:00
Artur Souza 2562908dee
Conf state trx + Change mongodb get result for key not found + Fix byte[] in cosmosdb (#680)
* Add state transaction conf test.

* Does not return error for key not found in mongodb.

* Rewrite state conf test.

* Handle []byte in cosmosdb state txn (#664)

Fixes: dapr/dotnet-sdk#579

The issue here is that we had missed the case of pre-marshaled bytes in
the state transaction code path, which would result in the encoding/json
package doing it's normal thing and base64 encoding the data as a JSON
string.

The fix is to use json.RawMessage to avoid that. Turns out this code can
be simplified a bit.

* Fix cosmosdb handling of byte[].

* Fix lint.

* Fix UT for cosmosdb.

* Remove config from state store conf test.

Co-authored-by: Ryan Nowak <nowakra@gmail.com>
2021-02-10 18:15:59 -08:00