* Improving testing - verify expected methods are invoked
Signed-off-by: lucasjellema <lucasjellema@gmail.com>
* Adding support for composite keys (mapped to directories in buckets)
Signed-off-by: lucasjellema <lucasjellema@gmail.com>
* Adding support for TTL - expired elements are not returned in Get
Signed-off-by: lucasjellema <lucasjellema@gmail.com>
* fixing linting issues
Signed-off-by: lucasjellema <lucasjellema@gmail.com>
* fixing linting issue
Signed-off-by: lucasjellema <lucasjellema@gmail.com>
* adding support for TTL == -1 (never expire)
Signed-off-by: lucasjellema <lucasjellema@gmail.com>
* fix linting issue
Signed-off-by: lucasjellema <lucasjellema@gmail.com>
* introducing Instance Principal authentication for OCI client
Signed-off-by: lucasjellema <lucasjellema@gmail.com>
* fix linting issue
Signed-off-by: lucasjellema <lucasjellema@gmail.com>
* adding support for oci config file based authentication for OCI bjectStorage state store
Signed-off-by: lucasjellema <lucasjellema@gmail.com>
* expand ~ to user home directory in config file path; fix linting findings
Signed-off-by: lucasjellema <lucasjellema@gmail.com>
* gofumpt file because of linting issue
Signed-off-by: lucasjellema <lucasjellema@gmail.com>
* Do not allow config file path to start with ~/
Signed-off-by: lucasjellema <lucasjellema@gmail.com>
* introducing integration test for oci objectstorage state store
Signed-off-by: lucasjellema <lucasjellema@gmail.com>
* Fixing linting issues in integration test
Signed-off-by: lucasjellema <lucasjellema@gmail.com>
* Fixing linting issues in integration test
Signed-off-by: lucasjellema <lucasjellema@gmail.com>
Co-authored-by: Looong Dai <long.dai@intel.com>
Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
* state azure blobstorage: fail for invalid account name
Signed-off-by: Tim Burkert <burkert.tim@gmail.com>
* state azure blobstorage: Remove duplicated code and add error check for common entpoint
Signed-off-by: Tim Burkert <burkert.tim@gmail.com>
Co-authored-by: Looong Dai <long.dai@intel.com>
Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
* Implement State Store for OCI ObjectStorage service
Signed-off-by: lucasjellema <lucasjellema@gmail.com>
* add go.mod and go.sum with dependencies on OCI SDK
Signed-off-by: lucasjellema <lucasjellema@gmail.com>
* Removed dependency in unit test on OCI account
Signed-off-by: lucasjellema <lucasjellema@gmail.com>
* Adding extensive unit test (coverage) using mock-OCI client
Signed-off-by: lucasjellema <lucasjellema@gmail.com>
* Correcting linting issues and review requests
Signed-off-by: lucasjellema <lucasjellema@gmail.com>
Co-authored-by: Looong Dai <long.dai@intel.com>
* - Introduce `fmt.Sprintf` where applicable
- Add conformance test for cassandra component.
Signed-off-by: mbimbij <joseph.mbimbi@gmail.com>
* Start cassandra via docker-compose for its conformance test
Signed-off-by: mbimbij <joseph.mbimbi@gmail.com>
* Apply `go fmt` on `tests/conformance/common.go` as part of PR correction
Signed-off-by: mbimbij <joseph.mbimbi@gmail.com>
* Adds retry on CosmosDB Init in case of TooManyRequests error
* Use backoff v4
* missed some permanent errors
* clean up go.mod
* fix error type casting
* Add constant for HTTP 429
* support json in mysql state #1168
* fmt, fix lint issue
* fix lint issue
* fix unit test failure
* add test against 'isbinary'
* fix lint issue
* enhance assertEquals
* remove unused 'expectedReadResponse' to fix lint issue
* go fmt to fix lint issue
* fix lint issue
Co-authored-by: Long Dai <long.dai@intel.com>
Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
* add state conformance test for postgresql #1117
* make postgresql pass comformance test
* include postgresql to github's workflow
* add newline at end of file for Linux text file
* enhance return
* add newline at end of file for valid Linux text file
* fix lint issue
* handle byte[] in special way so that to stick to json type in DB
* use jsonb instead of json
* fix lint issue
* doesn't allow "", a special case
* enhance assertEquals
* remove unused 'expectedReadResponse' to fix lint issue
* go fmt to fix lint issue
* fix lint issue
Co-authored-by: Simon Leet <31784195+CodeMonkeyLeet@users.noreply.github.com>
Co-authored-by: Yaron Schneider <yaronsc@microsoft.com>
Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
Co-authored-by: Long Dai <long.dai@intel.com>
Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com>
* state alicloud.tablestore
* some bug fix
* gofumpt
* add unit test
* gofumpt
* golangci-lint
* code refactor
* goimports
* goimports
Co-authored-by: Simon Leet <31784195+CodeMonkeyLeet@users.noreply.github.com>
Co-authored-by: Long Dai <long0dai@foxmail.com>
Co-authored-by: Long Dai <long.dai@intel.com>
Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
* Add mysql conformance test and fix mysql state to make it pass the test #1122
* remove unused code
* add newline at end of Linux text file
* enhance setValue and relevant unit tests
* fix lint issue
* fix lint issue
Follow-up to #1104 to support AD Auth codepath as well.
`NewConfig()` and `NewConfigWithServicePrincipal()` both set the
`Config.IdentificationHydrator` to the `DefaultIdentificationHydrator`
in the underlying a8m/documentdb library. That implementation ends up
calling `reflect.Value.Elem.FieldByName()`, which requires that the
value passed to it is an interface or pointer, otherwise the Elem()
call fails to dereference which causes a panic.
cosmosdb.go passes the input to `UpsertDocument` by value today,
which is eventually passed to the `DefaultIdentificationHydrator`,
so changing the value passed to a pointer resolves the failure.
Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com>
PR #1104 introduced a regression in cosmosdb state component `Init()`
by creating a `NewConfig()` which sets a new DefaultIdentificationHydrator,
which causes a panic later when invoked.
Patch this by restoring the configuration setting to use a struct with
`nil` `IdentificationHydrator`.
* Support Azure AD auth for Cosmos DB
* Fixed linting errors
* Tidying go.sum
* Removed the need for nolint:shadow
Co-authored-by: Simon Leet <31784195+CodeMonkeyLeet@users.noreply.github.com>
Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com>
* Common Azure auth logic
- Currently implemented on secretstores/azure/keyvault and state/azure/blobstorage
- Supports Azure AD via service principal (client credentials, client certificate, MSI) - based on the previous authorizer for AKV
- Allows using other Azure clouds (China, Germany, etc)
- For Blob Storage state, supports using custom endpoints (like emulators like Azurite)
* Add environment variable aliases
* Address linter warnings
* another lint thing
* Fixed typo in method description
* Updated metadata key names so they're more consistent
* Fix test
* Some more linter things
Co-authored-by: Bernd Verst <me@bernd.dev>
Co-authored-by: Yaron Schneider <yaronsc@microsoft.com>
Co-authored-by: Bernd Verst <berndverst@users.noreply.github.com>
* Reduced cyclomatic complexity of Multi
There was a lint warning on this function.
* Reduced cyclomatic complexity of Init
* Updated existing unit tests.
Added test for default table and database names.
And invalid database name.
* Got all Integration tests passing after refactor.
* All tests passing.
* Increasing code coverage
* Creating db in sqlserver state store is complete.
* Fixing gofumpt issues
Co-authored-by: Phil Kedy <phil.kedy@gmail.com>
* Adds support for TTL in Redis State Store
* return explicit errors
* Refactor Redis TTL
* reduce nestedness
* Changing if logic for ttl to fit lint
* Ignore lint for nestif
Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
* Adds support for TTL in Cassandra State Store
* Update cassandra.go
* Update cassandra_test.go
* Update cassandra.go
Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
* Move pubsub redis client to internal
* Add failover for common redis. Refactor state redis.
* refactor binding redis
* fix lint
* Refactor to setting partten
* fix db and lint
* fix lint
* for backward compatibility
* Using a type alias to handle decoding duration values of -1 for Redis.
* Linter fixes
Co-authored-by: Phil Kedy <phil.kedy@gmail.com>
* add `Ping` operation.
* add `Ping` implementation of GA components
* add context to an argument for Ping
* add `Ping` implementation of blobstorage
* fix error string for passing lint
* Update blobstorage.go
Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
resolve conflict
* master:
Increasing overall test timeout to allow for conformance tests to run (#834)
Improve SMTP binding performance (#782)
add redis cluter mode for pubsub (#823)
downgrade the verbosity of lock renewal errors from warn to debug. This error is expected when messages are processed before beingrenewed (#803)
Improve pulsar publisher performance. (#827)
optimze(state/redis): output detail redis error (#825)
Consul nameresolution component (#750)
Add DNS name resolver. Add unit tests. (#743)
Azure Service Bus: Changed log level of error on close to info (#830)
Removed dependency on dapr/dapr in favor of dapr/kit. Fixed go mod dependencies afterwards. Go 1.16 in go.mod. Removed accidental log dependency. (#807)
Update state/blobstorage.go to add metadata. (#786)
Azure Service Bus publishing retries when server is "too busy" (#821)
# Conflicts:
# bindings/redis/redis.go
# go.mod
# go.sum
# pubsub/redis/redis.go
* Add ETag operation to statestore conformance test
ETags are used to enable concurrency guarantees and are part of the
statestore specification. Not all stores need to use them, however
if the store chooses to, it should be part of the certification
process. This commit adds a simple ETag test scenario that verifies
the basic behaviors of ETags.
https://github.com/dapr/components-contrib/issues/711
* Update tests.yml
* Error on mongodb when etag does not match on delete.
Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
* Add state transaction conf test.
* Does not return error for key not found in mongodb.
* Rewrite state conf test.
* Handle []byte in cosmosdb state txn (#664)
Fixes: dapr/dotnet-sdk#579
The issue here is that we had missed the case of pre-marshaled bytes in
the state transaction code path, which would result in the encoding/json
package doing it's normal thing and base64 encoding the data as a JSON
string.
The fix is to use json.RawMessage to avoid that. Turns out this code can
be simplified a bit.
* Fix cosmosdb handling of byte[].
* Fix lint.
* Fix UT for cosmosdb.
* Remove config from state store conf test.
Co-authored-by: Ryan Nowak <nowakra@gmail.com>
* Adding MySQL State Component
* Fixing File is not `gofumpt`-ed (gofumpt)
I believe the issue was No empty lines before a simple error check
* Fixing final lint errors that only showed up in CI
* gofumpt-ed in test files.
* Changing error for affected rows.
Want to distinguish database errors from application errors.
* The database (schema) can now be created for you.
Co-authored-by: Yaron Schneider <yaronsc@microsoft.com>
* add BulkGet method on state store
* add a bool value in bulk get method to tell dapr if this store supports bulk get or not
* add BulkGetResponse for bulk get; rollback GetResponse to remove new added filed key
* update to use BulkGetResponse for bulk get
* fix lint issues
* fix lint issues
* fix lnit issues
* rethinkdb state store
* added actor state change binding
* updated db record
* reconnect
* passing lint
* updated binding repsonse keys
* rethinkdb load test
* lint validated tests
* updated comments
* updates TransactionalStateRequest
* parameterized table
* correct error message
Co-authored-by: Young Bu Park <youngp@microsoft.com>
* update logger form nats-io's to dapr's
* run go mod tidy and go mod delete the nats-io gnatsd
* update rabbitmq prefetch feature because want to set each instance the number of messages could hold in once until ack.
* update for parse int
* update parseMetadata of rabbitMQ to read prefetchCount by string.
* [update rabbitMQ binding parseMetadata] Make error clear
* update the redis failover metadata
* [in rabbitMQ metadata parse] fix the bug of golint
* [redis statestore failover feature] add new client logic.
* update redis metadata as advice.
maligned struct of size 80 bytes could be of size 72 bytes.
* update rabbitMQ with a new parseMetadata way.
* omit comparison to bool constant as golangci-lint.
* [typo] matser -> master
* Revert "Merge branch 'master' of https://github.com/dapr/components-contrib into dapr-master"
This reverts commit 9a91bccc4b, reversing
changes made to 7e0cb1fa2d.
* Revert "Revert "Merge branch 'master' of https://github.com/dapr/components-contrib into dapr-master""
This reverts commit 110ef82681.
* Revert "Merge branch 'master' of https://github.com/dapr/components-contrib into dapr-master"
This reverts commit 9a91bccc4b, reversing
changes made to 7e0cb1fa2d.
* solve the conflct
* merge from the master to accept the change
* remove the defaultFailover as the reviewed.
* move newclient to function
* Update state/redis/redis.go
Co-authored-by: Nghia Tran <tcnghia@gmail.com>
* update comment to /* #nosec */
Co-authored-by: Yaron Schneider <yaronsc@microsoft.com>
Co-authored-by: Nghia Tran <tcnghia@gmail.com>
* State Store Components changes to adhere runtime changes for removing state retry options
* resolving code review comments - common method to CheckRequestOptions
* Add blob storage as state store
* Updated comments for blob storage
Co-authored-by: Young Bu Park <youngp@microsoft.com>
Co-authored-by: Yaron Schneider <yaronsc@microsoft.com>
* Added postgres state files
* Fix SQL Server integration tests. (#324)
* Initial unit and int tests for postgres
* Implemented dbaccess interface
* Formatting updates
* Create state table if not exists
* Initial set implementation
* Get and set with integration test
* Added delete
* Create date and update date in database
* Integration tests for bulk set and bulk delete
* Etag processing, test refactor
* Added tests for etag processing
* Updated const, removed public metadata fields
* Cleanup edits
* Fixed linting issues
* Ran go mod tidy
* Initial implementation of TransactionalStore
* Fixed linting issues
* Added tests and validation
* Test for creating the state table
* Fixed issue with parsing param value as string
* Update to integration tests
* Fixed linting issues
* Added retry logic from state
* Changed primary key to text
* Updated debug logging
* Review updates
Co-authored-by: Young Bu Park <youngp@microsoft.com>
Co-authored-by: Yaron Schneider <yaronsc@microsoft.com>
* Implement the Multi() interface method adding transaction support in the cosmos db component. The implementation
uses a stored procedure to do this. The stored procedure is registered in Init().
* linter
* code review comments
* code review
* remove comment
* casing and de-dup
* use 'in' in query
Co-authored-by: Yaron Schneider <yaronsc@microsoft.com>
* [DynamoDB] Update BulkSet and BulkDelete with BatchWriteItem
* Validate type of value
* Marshal value before setting for Set and BulkSet
Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
Co-authored-by: Yaron Schneider <yaronsc@microsoft.com>
* Use shared sqs auth package
* Goimports-ed files
* Added support for endpoint param in aws credentials abstraction, since the sqs-sns pubsub module required it
* Added endpoint support for all aws things
* Updated tests
* initial scaffolding
* tidy go.mod
* adding delete functionality and tests
* updating Readme.md
* ETag concurrency support and github review fixes
* removing unused isConflictError
* ping build, some issue with CI timing out
Co-authored-by: Young Bu Park <youngp@microsoft.com>
* Add sql server state
* Fix lint
* Disable docker based tests
* Rename store to SQLServer
Use env variable DAPR_TEST_SQL_CONNSTRING to enable integratoion test
* Add support to Azure SQL on integration test
Co-authored-by: Yaron Schneider <yaronsc@microsoft.com>
* [Commit - PubSub changes ]
Changing the names of the packages directory so to be consistent with name of the company .
It is to be noted that package name can not have hyphen "-" in the name.
So changing the directory names and putting under company names.
For example - servicebus package would be
dapr/components-contrib/pubsub/azure/servicebus
* Commit 2 - Changes for Secret Stores
* Commit 3 - for the state store changes
Lucas Jellema