Commit Graph

47 Commits

Author SHA1 Message Date
Bernd Verst c7a913c552
Minor Az Storage Binding improvements (#3126)
Signed-off-by: Bernd Verst <github@bernd.dev>
2023-09-14 13:16:12 -07:00
Alessandro (Ale) Segala ec05809ee6
[Metadata] Update validator and some other fixes (#2984)
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Co-authored-by: Bernd Verst <github@bernd.dev>
2023-07-17 22:17:22 +00:00
Bernd Verst cfbac4d794
Refactor of Metadata parsing in Bindings (#2720)
Signed-off-by: Bernd Verst <github@bernd.dev>
Signed-off-by: Shivam Kumar <shivamkm07@gmail.com>
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Signed-off-by: Roberto J Rojas <robertojrojas@gmail.com>
Signed-off-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com>
Signed-off-by: Roberto Rojas <robertojrojas@gmail.com>
Signed-off-by: spike <hello@spike.wiki>
Signed-off-by: zhangchao <zchao9100@gmail.com>
Signed-off-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com>
Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>
Signed-off-by: handlerww <handlerww@gmail.com>
Co-authored-by: Shivam Kumar <shivamkm07@gmail.com>
Co-authored-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Co-authored-by: Roberto Rojas <robertojrojas@gmail.com>
Co-authored-by: spike <39330606+SpikeWong@users.noreply.github.com>
Co-authored-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com>
Co-authored-by: Taction <zchao9100@gmail.com>
Co-authored-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>
Co-authored-by: Yiwen Chen <handlerww@gmail.com>
2023-04-05 22:19:08 +00:00
Josh van Leeuwen 4be795d812
Remove `context.TODO()`s with passed context. (#2650)
Signed-off-by: joshvanl <me@joshvanl.dev>
2023-03-08 17:53:02 +00:00
ItalyPaleAle a484d7ebc7 Merge branch 'master' of https://github.com/dapr/components-contrib into newdeps
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2023-02-16 23:35:28 +00:00
Josh van Leeuwen d098e38d6a
Propagate context from caller to appropriate places in the code (#2474)
* Propagates contexts to callers where appropriate.

Signed-off-by: joshvanl <me@joshvanl.dev>

* Updates units tests with new func signature

Signed-off-by: joshvanl <me@joshvanl.dev>

* Fix linting errors

Signed-off-by: joshvanl <me@joshvanl.dev>

* Add atomic gate to alicloud rocketmq close channel.

Signed-off-by: joshvanl <me@joshvanl.dev>

* bindings/aws/kinesis use a separate ctx variable name

Signed-off-by: joshvanl <me@joshvanl.dev>

* binding/kafka: use atomic to prevent closing the channel twice

Signed-off-by: joshvanl <me@joshvanl.dev>

* bindings/mqtt3: use atomic bool to prevent close channel being closed multiple times

Signed-off-by: joshvanl <me@joshvanl.dev>

* bindings/mqtt3: use Background context for handle operations:w

Signed-off-by: joshvanl <me@joshvanl.dev>

* state/cocroachdb: add context to Ping()

Signed-off-by: joshvanl <me@joshvanl.dev>

* bindings/postgres: add comment explaining use of context.

Signed-off-by: joshvanl <me@joshvanl.dev>

* Adds comment header to health/pinger.go

Signed-off-by: joshvanl <me@joshvanl.dev>

* pubsub/aws/snssqs: add waitgroup to wait for all go routines to finish
and block on Close(). Shuts down the subscription if there are no topic
handlers.

Signed-off-by: joshvanl <me@joshvanl.dev>

* pubsub/mqtt3: add atomic bool to prevent multiple channel closes. Add
wait group to block close on all goroutines to finish.

Signed-off-by: joshvanl <me@joshvanl.dev>

* pubsub/rabbitmq: fixes race conditions, uses atomic to prevent multiple
closes, add wait group to block close on all goroutines

Signed-off-by: joshvanl <me@joshvanl.dev>

* pubsub/redis: revert ctx passed when it could be cancelled. Add wait
group wait when closing.

Signed-off-by: joshvanl <me@joshvanl.dev>

* state/postges: pass context in init, and wait group on close.

Signed-off-by: joshvanl <me@joshvanl.dev>

* Update all `Ping()` to `PingContext()` where possible.

Signed-off-by: joshvanl <me@joshvanl.dev>

* state/in-memory: add atomic bool to prevent closing channel multiple
times. Add wait group to block on close()

Signed-off-by: joshvanl <me@joshvanl.dev>

* state/mysql: don't use same ctx variable name

Signed-off-by: joshvanl <me@joshvanl.dev>

* Pass correct loop context to redis go routines

Signed-off-by: joshvanl <me@joshvanl.dev>

* Rename context when creating timeouts in state

Signed-off-by: joshvanl <me@joshvanl.dev>

* Remove state.Features() from requiring a context

Signed-off-by: joshvanl <me@joshvanl.dev>

* Revert wasm request handle Close func to be without context to
implement io.Closer interface. Add 5 second timeout. Add io.Closer
assertion in test.

Signed-off-by: joshvanl <me@joshvanl.dev>

* Remove superfluous go lint vet directive

Signed-off-by: joshvanl <me@joshvanl.dev>

* Change Configuration Init function to take context

Signed-off-by: joshvanl <me@joshvanl.dev>

* Updates input binding interface to include a `Close() error` function. `Close`
blocks until all resources have been released and go routines have returned.

Signed-off-by: joshvanl <me@joshvanl.dev>

* Change `Close() error` in input binding struct to `io.Closer` interface.

Signed-off-by: joshvanl <me@joshvanl.dev>

* Update go.mod files to point to dapr/dapr PR https://github.com/dapr/dapr/pull/5831

Signed-off-by: joshvanl <me@joshvanl.dev>

* pubsub/redis: watch closeCh to shutdown worker instead of init context.

Signed-off-by: joshvanl <me@joshvanl.dev>

* pubsub/aws/snssqs + bindings/kubemq: ensure closeCh is caught so Close
correctly returns

Signed-off-by: joshvanl <me@joshvanl.dev>

* Close kubemq binding client on close. Ensure kafka consumer channel
cannot be closed more than once.

Signed-off-by: joshvanl <me@joshvanl.dev>

* Tweaks

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Fixed cert tests

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* binding/mqtt3: add inline Background context instead of passing to
handleMessage

Signed-off-by: joshvanl <me@joshvanl.dev>

* pubsub/mqtt3: remove context from createSubscriberClientOptions

Signed-off-by: joshvanl <me@joshvanl.dev>

* pubsub/mqtt3: Remove `ResetConnection` func

Signed-off-by: joshvanl <me@joshvanl.dev>

* pubsub/kafka: Don't resubscribe if Subscribe is cancelled.

Signed-off-by: joshvanl <me@joshvanl.dev>

* binding/mqtt3: don't use context to control establishing connection

Signed-off-by: joshvanl <me@joshvanl.dev>

* bindings/mqtt3: Fix linting errors

Signed-off-by: joshvanl <me@joshvanl.dev>

---------

Signed-off-by: joshvanl <me@joshvanl.dev>
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Co-authored-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
2023-02-16 14:18:35 -08:00
ItalyPaleAle a8957062f0 Fixed tests
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2023-02-16 01:42:05 +00:00
ItalyPaleAle 6f30dbf6c6 Updated Azure Event Hubs and Azure Blob Storage SDKs
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2023-02-16 00:21:56 +00:00
Bernd Verst fce17f592e Update SDKs again
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-12-13 12:27:40 -08:00
Bernd Verst 385c6b79b6 remove unnecessary reader close()
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-11-28 15:51:03 -08:00
Bernd Verst 23d36a08d7 more refactoring and address code review
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-11-18 17:56:02 -08:00
Bernd Verst 8811d5e64f AzBlob components: Extract shared code
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-11-17 21:37:17 -08:00
Bernd Verst 70eb9f3a9c Address code review comments
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-11-17 20:35:05 -08:00
Bernd Verst c3027b8fb1 some small improvements
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-11-17 19:54:32 -08:00
Bernd Verst 51e0aa992c Azure Blobstorage Binding: Migrate to Track2 Azure SDK
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-11-17 19:54:32 -08:00
ItalyPaleAle 1a6e11fc63 Do not allow Unicode letters in sanitized metadata
Follow-up from #2112

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-09-22 16:46:14 +00:00
Bernd Verst 0b8b663e3e add warning message for changing metadata key
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-09-21 23:05:22 -07:00
Bernd Verst ddf16a11b2 add test case
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-09-21 22:34:28 -07:00
Bernd Verst 6e4f8e9ca9 sanitize creatBlob request metadata in AzStorageBlob
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-09-21 18:39:14 -07:00
Alessandro (Ale) Segala bcea284c7b
Components' init functions should return interfaces (#1997)
* Components' init functions should return interfaces

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* 💄

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* pin latest berndverst contrib fork

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Fixed component registration in tests

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Updated all cert tests

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-08-25 18:27:45 -07:00
Alessandro (Ale) Segala d38c786771
Add contexts to input bindings (#1831)
This is the last part of the "shutdown sequence fix": it allows shutting down input bindings before output ones.

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-07-12 12:19:27 -07:00
Alessandro (Ale) Segala a2f3a84b96
Moved authentication to be an internal pkg (#1855)
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-07-06 11:05:34 -07:00
Alessandro (Ale) Segala a193cc1ede
Add support for AAD auth in Azure Storage Queues binding (#1842)
* Add support for AAD auth in Azure Storage Queues binding

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* 🧹

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-07-03 14:17:38 -07:00
Alessandro (Ale) Segala cedaacbb7c
Make metadata keys uniform for Azure Storage components (#1837)
* Make metadata keys uniform for Azure Storage components

By using aliases we are preserving backwards-compatibility

Fixes #1832

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Updated bindings.azure.storagequeues too

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Updated state.azure.tablestorage too

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* 💄

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* 🧹

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Co-authored-by: Loong Dai <long.dai@intel.com>
Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com>
2022-07-01 12:05:11 -07:00
ItalyPaleAle 14a9103bec Improvements to blobstorage binding
Spin-off from PR adding contexts to input bindings

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-06-29 19:55:02 +00:00
pigletfly 5673b28b13 Add context in bindings interface
Signed-off-by: pigletfly <wangbing.adam@gmail.com>
2022-04-25 14:16:15 +08:00
Bernd Verst 0f2398670a
Certification tests for Azure Blob Storage Binding (#1435)
* initial blobstorage output binding certification

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* More blobstorage tests

Signed-off-by: GitHub <noreply@github.com>
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Rename some remnants from copy paste

Signed-off-by: GitHub <noreply@github.com>
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Blobstorage Binding - test file upload

Signed-off-by: GitHub <noreply@github.com>
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Blobstorage Output Binding Remove Pascal case

Signed-off-by: GitHub <noreply@github.com>
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Blobstorage Output Binding: Remove list from conf test

Signed-off-by: GitHub <noreply@github.com>
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Azure Blobstorage Binding - include name upon creation

Signed-off-by: GitHub <noreply@github.com>
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* blobstore binding - verify encoding option

Signed-off-by: GitHub <noreply@github.com>
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* go mod tidy

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Additional test cases

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* list blob tests WIP

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* More stable certification tests for AzBlob Binding

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Remaining Blob Storage Binding certification tests

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Lint blob storage certification tests

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* update go.mods

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Fix license header

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* use byte string instead of file

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Run make modtidy-all again

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
2022-01-13 18:06:39 -08:00
Dmitry Shmulevich c7adb917f3
update license to Apache v2.0 (#1406) 2022-01-04 19:53:31 -08:00
Bernd Verst ac85ba2a8e
Add dapr user agent for Azure Components (#1152)
* Add dapr useragent for Service Bus and EventHubs

* Add useragent for tablestorage

* send dapr useragent

* more user agents

* formatting
2021-09-27 17:04:27 -07:00
Long Dai 988fed05ab
ci: standard linter config (#1102)
* ci: standard linter config

Signed-off-by: Long <long.dai@intel.com>

* Update utils.go

* fix pulsar issue

Signed-off-by: Long <long.dai@intel.com>

Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
2021-09-21 15:21:59 -07:00
Christian Kaps 80f0110fa8
Return blob metadata and add list operation (#999)
* Return blob metadata if configured

* Enhance error msg

* Add list functionality

* Fix linting issues

* Finalize list implementation

* Fix linting errors

* Add backward compatibility and align code to be more consistent

* Add tests

* Fix linting errors

* Rename properties

* Fix max result logic

* Fix linting error

* Fix PR suggestions

* Add comments for constants

* Run go mod tidy

Co-authored-by: Yaron Schneider <yaronsc@microsoft.com>
2021-07-14 09:17:48 -07:00
Fabian Steinbach 4412ba019f
unmarshal getBlobRetryCount as int (#919)
Co-authored-by: Yaron Schneider <yaronsc@microsoft.com>
2021-07-08 10:45:52 -07:00
Phil Kedy 688df9bda4
Removed dependency on dapr/dapr in favor of dapr/kit. Fixed go mod dependencies afterwards. Go 1.16 in go.mod. Removed accidental log dependency. (#807) 2021-04-19 12:06:10 -07:00
Fabian Steinbach 3a3998bb60
add delete to blob storage binding (#705)
* add delete to blob storage binding

* adjust linting

* make deletesnapshotsoption configurable

* update linting

Co-authored-by: Yaron Schneider <yaronsc@microsoft.com>
Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
2021-03-09 00:20:43 -08:00
Yaron Schneider 3ef025c604
change headers (#679) 2021-02-09 18:57:55 -08:00
Yaron Schneider d260ec85e6
add access level to azure blob storage (#549) 2020-12-05 07:38:13 -08:00
Yaron Schneider 01f90bdbc8
Fix azure blob storage json payload (#516)
* fix azure blob storage json payload

* handle non-json strings
2020-11-03 10:05:30 -08:00
halspang f647d9eedb
Upgrade golang lint to 1.31 (#490)
Upgrade golang lint to 1.31.0

https://github.com/dapr/components-contrib/issues/439
2020-10-12 12:54:15 -07:00
Yaron Schneider 1e92e0e8e1
Add blob url to azure blob create (#474)
* add blob url to azure blob create

* add missing err statement
2020-09-23 17:42:46 -07:00
Yaron Schneider b91469732c
Add get operation for azure blob storage binding (#473)
* add get operation for azure blob storage binding

* fix retry count

* make retry configurable, add default
2020-09-23 15:00:34 -07:00
Mohammad Taghavi e5a50cb2da
add decodeBase64 option to azure blob storage component (#393)
* add decodeBase64 option to azure blob storage component

* fix lint error

* Apply suggestions from code review

some code improvements (apply review suggestion)

Co-authored-by: Nghia Tran <tcnghia@gmail.com>

* add comment to help assure the reader (resolve review comment)

Co-authored-by: Mohammad Taghavi <Mohammad.Taghavi@plan-b-gmbh.com>
Co-authored-by: Nghia Tran <tcnghia@gmail.com>
Co-authored-by: Young Bu Park <youngp@microsoft.com>
2020-09-22 13:44:36 -07:00
Yaron Schneider e82164f4e7
Add bi-directional bindings support (#350)
* add bi-directional bindings

* added invoke response

* added explicit types

* change operationtype to operationkind
2020-05-26 23:32:20 -07:00
Ben Coleman e97529a4a8
Unescape azure blob data (#313)
Co-authored-by: Yaron Schneider <yaronsc@microsoft.com>
2020-04-30 08:15:18 -07:00
Ricardo Niepel 934ef7a7ba
Add metadata support to Azure Blob binding (#281)
* add metadata support to AzBlobBinding

* change to tabs

* fix lint error

* improve error message

* uncap error message
2020-03-21 16:33:35 -07:00
Young Bu Park ee97c3c56a
Use dapr logger (#238)
* Use dapr logger

* skip linter false alarm
2020-02-28 15:49:49 -08:00
thinkerou a2f4b4c510 upgrade logrus to v1.4.2 (#150)
* fix conflict

* upgrade logrus to v1.4.2

* fix lint error

error: `depguard  `github.com/sirupsen/logrus` is in the blacklist`
2019-12-12 09:34:39 -08:00
Shalabh Mohan Shrivastava 8c9a3bd806 [Commit - Bindings Package names]
Changing the names of the packages directory so to be consistent with name of the company .

It is to be noted that package name can not have hyphen "-" in the name.
So changing the directory names and putting under company names.

For example - dynamodb  package would be

dapr/components-contrib/bindings/aws/dynamodb
2019-11-18 15:33:13 -08:00