Commit Graph

37 Commits

Author SHA1 Message Date
Alessandro (Ale) Segala e7db4cf3ad
Move SQLite "auth" metadata to a separate package `internal/authentication/sqlite` (#3135)
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Co-authored-by: Bernd Verst <github@bernd.dev>
2023-09-20 15:26:12 -07:00
Alessandro (Ale) Segala da50003dd4
HTTP Binding: add "maxResponseBodySize" metadata option (#3040)
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Signed-off-by: Bernd Verst <github@bernd.dev>
Co-authored-by: Bernd Verst <github@bernd.dev>
2023-08-14 21:06:30 -07:00
Alessandro (Ale) Segala 46b7535f4a
DecodeMetadata: add support for aliases via "mapstructurealiases" struct tag (#3034)
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2023-08-03 12:40:23 -07:00
Alessandro (Ale) Segala ec05809ee6
[Metadata] Update validator and some other fixes (#2984)
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Co-authored-by: Bernd Verst <github@bernd.dev>
2023-07-17 22:17:22 +00:00
Alessandro (Ale) Segala 5dad8a867c
Metadata parser: add support for built-in authentication profiles and metadata properties (#2760)
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Co-authored-by: Bernd Verst <github@bernd.dev>
2023-04-12 00:04:34 +00:00
Bernd Verst 2b89d78a2d
Refactor Metadata Parsing of all PubSub Components (#2759)
Signed-off-by: Bernd Verst <github@bernd.dev>
2023-04-09 18:21:56 +00:00
Bernd Verst cfbac4d794
Refactor of Metadata parsing in Bindings (#2720)
Signed-off-by: Bernd Verst <github@bernd.dev>
Signed-off-by: Shivam Kumar <shivamkm07@gmail.com>
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Signed-off-by: Roberto J Rojas <robertojrojas@gmail.com>
Signed-off-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com>
Signed-off-by: Roberto Rojas <robertojrojas@gmail.com>
Signed-off-by: spike <hello@spike.wiki>
Signed-off-by: zhangchao <zchao9100@gmail.com>
Signed-off-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com>
Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>
Signed-off-by: handlerww <handlerww@gmail.com>
Co-authored-by: Shivam Kumar <shivamkm07@gmail.com>
Co-authored-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Co-authored-by: Roberto Rojas <robertojrojas@gmail.com>
Co-authored-by: spike <39330606+SpikeWong@users.noreply.github.com>
Co-authored-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com>
Co-authored-by: Taction <zchao9100@gmail.com>
Co-authored-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>
Co-authored-by: Yiwen Chen <handlerww@gmail.com>
2023-04-05 22:19:08 +00:00
Bernd Verst a069e682be
Refactor metadata parsing in middlewares and workflows (#2715)
Signed-off-by: Bernd Verst <github@bernd.dev>
2023-03-28 09:34:20 -07:00
Bernd Verst 2266e7ad44
Standardize metadata parsing in configstores and lockstore (#2710)
Signed-off-by: Bernd Verst <github@bernd.dev>
2023-03-25 22:34:48 +00:00
Bernd Verst e5cbb34990
Add metadata analyzer tools (#2696)
Signed-off-by: Bernd Verst <github@bernd.dev>
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Co-authored-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com>
2023-03-24 18:34:36 +00:00
ItalyPaleAle 643a2abff2 Removed go.uber.org/atomic and github.com/pkg/errors
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2023-02-15 19:56:20 +00:00
ItalyPaleAle 9920d53f93 Changed SQLite metadata parsing per review feedback
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2023-02-08 23:05:01 +00:00
ItalyPaleAle 254affd133 Make keys in GetMetadataProperties case-insensitive
This method is used by paths including the one that checks for Azure AD credentials. This makes the keys case-insensitive so the matching is more lenient (e.g. `azureClientId` vs `azureClientID`)

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2023-01-24 09:07:15 -08:00
Deepanshu Agarwal b2d8e6013b
Add bulkSubscribe request params to SubscribeRequest (#2405)
* Add bulkSubscribe configuration params

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Replace dapr/dapr package to run cert tests

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Change bulkSubscribe param types

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Update latest dapr/dapr fork

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Update correct dapr/dapr version

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* change BulkSubscribeRequest to Config

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Update dapr/dapr version for cert

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Update with latest dapr fork

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>
2023-01-10 09:30:25 -08:00
Bernd Verst ede060959a
Refactors all state store to standardize metadata parsing (#2257) 2022-11-05 15:36:44 -07:00
deepanshuagarwal bcd101abad
Keys rename (#2135)
Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>
2022-09-28 09:52:59 -07:00
Shubham Sharma cd206609ce
Use request metadata instead of component metadata (#2119)
Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>
2022-09-23 10:49:39 -07:00
Shubham Sharma 4f406a597f
Add bulk subscribe support to Azure Service Bus (#2100)
* Initial implementation

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* 

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Add tests

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Add comment

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Refactor messages.go and add tests

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Update message tests

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Refactor servicebus.go logic

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Refactor handleAsync

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Refactor ReceiveAndBlock

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Update binding

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Review comments addressed

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Lint

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Update metadata name

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Update activeMessagesChan logic for concurrency control

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Fix tests

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Update metadata name

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Use request metadata instead of component metadata

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Review comments addressed

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Fix merge issue with metadata

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Remove unused metadata key

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Fix merge issue with metadata

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Lint!!!

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>
2022-09-23 10:49:06 -07:00
Shubham Sharma b298b65cfe
Add bulk publish support to Azure Service Bus (#2106)
* Initial implementation

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Some improvements

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Refactor and add tests

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Lint

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Refactor

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Use request metadata instead of component metadata

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Remove unused method

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Review comments addressed

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Doc comments

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>
2022-09-23 09:57:00 -07:00
Mukundan Sundararajan e1697ba115 Merge branch 'master' into feature/pubsub-batching 2022-09-23 06:47:47 +05:30
Deepanshu Agarwal 217505fd16 Move common costants to common utils
Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>
2022-09-16 22:22:54 +05:30
Bernd Verst e701aadae2 Add test case for metadat.Duration vs regular time.duration
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-09-12 18:54:20 -07:00
Bernd Verst 1d0ad9c6d8 Use standardized metadata parser
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-09-12 18:50:05 -07:00
Bernd Verst bdcf747143 add metadata decode unit test
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-09-12 17:38:59 -07:00
Bernd Verst 393f24280b Fix Zeebe metadata duration issue
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-09-12 16:42:36 -07:00
Marcos Candeia 8964d8bf41
Define common metadata across components (#1994)
* Add base metadata for all components

Signed-off-by: Marcos Candeia <marrcooos@gmail.com>

* Fix lint issues on consul and mysql tests

Signed-off-by: Marcos Candeia <marrcooos@gmail.com>

* Pin candeia/dapr runtime version

Signed-off-by: Marcos Candeia <marrcooos@gmail.com>

* Add replace directive to use mcandeia/dapr on all go submodules

Signed-off-by: Marcos Candeia <marrcooos@gmail.com>

* Fixed runtime config instantiation

Signed-off-by: Marcos Candeia <marrcooos@gmail.com>

Signed-off-by: Marcos Candeia <marrcooos@gmail.com>
2022-08-25 10:10:53 -07:00
Alessandro (Ale) Segala cedaacbb7c
Make metadata keys uniform for Azure Storage components (#1837)
* Make metadata keys uniform for Azure Storage components

By using aliases we are preserving backwards-compatibility

Fixes #1832

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Updated bindings.azure.storagequeues too

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Updated state.azure.tablestorage too

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* 💄

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* 🧹

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Co-authored-by: Loong Dai <long.dai@intel.com>
Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com>
2022-07-01 12:05:11 -07:00
Alessandro (Ale) Segala e5e9011655
Update Azure Service Bus components to track2 SDK (#1702)
* Updated to Go 1.18

Signed-off-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com>
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>

* Updated Azure SDKs that are on track2
Includes some minor refactoring of auth code

Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>

* Updated Service Bus components to track2 SDK

Co-authored-by: halspang <halspang@microsoft.com>
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>

* Initial Certification test for eventhubs binding [incomplete] (#1670)

* certification test for eventhubs binding

Signed-off-by: tanvigour <tanvi.gour@gmail.com>

* modified go.mod and go.sum

Signed-off-by: tanvigour <tanvi.gour@gmail.com>

* Add connection string testing

Signed-off-by: tanvigour <tanvi.gour@gmail.com>

* iothub testing

Signed-off-by: tanvigour <tanvi.gour@gmail.com>

* address feedback and run test

Signed-off-by: tanvigour <tanvi.gour@gmail.com>

* Install Azure CLI IOT hub extension

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* make modtidy-all

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* covering all eventhubs test cases

Signed-off-by: tanvigour <tanvi.gour@gmail.com>

* dependency changes after go modtidy-all

Signed-off-by: tanvigour <tanvi.gour@gmail.com>

Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
Co-authored-by: Looong Dai <long.dai@intel.com>
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>

* Use revive instead of golint (#1685)

Signed-off-by: pigletfly <wangbing.adam@gmail.com>

Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>

* Updated to Go 1.18 (#1697)

* Updated to Go 1.18

Signed-off-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com>

* Added go.work file
With Go 1.18, this allows gopls (the Go language server used for example in VS Code) to work inside test apps too.
See: https://go.dev/doc/tutorial/workspaces

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Removed go.work

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* 💄

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>

* 💄 & 🧹

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>

* Add metadata property to configure Batching in Pulsar (#1707)

* Add metadata property to configure BatchingMaxSize&batchingMaxMessages in Pulsar
Signed-off-by: saberwang <saberwang@hotmail.com>

* sort field
Signed-off-by: saberwang <saberwang@hotmail.com>

* [pubsub]fix unit test bug
Signed-off-by: saberwang <saberwang@hotmail.com>

* remove unrelated changes
Signed-off-by: saberwang <saberwang@hotmail.com>

* Delete hard coded Metadata
Signed-off-by: saberwang <saberwang@hotmail.com>

* remove  .history

Signed-off-by: saberwang <saberwang@hotmail.com>

* restore .gitignore

Signed-off-by: saberwang <saberwang@hotmail.com>

* Hard coding default values and adding 'BatchingMaxPublishDelay' metadata

Signed-off-by: saberwang <saberwang@hotmail.com>

* fix code format

Signed-off-by: saberwang <saberwang@hotmail.com>

* formatting code

Signed-off-by: saberwang <saberwang@hotmail.com>

Co-authored-by: Looong Dai <long.dai@intel.com>
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>

* This test can't work with track2 SDKs
The methods to create a message with a body are not exported

Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>

* 🧹

Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>

* There's such thing as too much logging

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Refactored subscription.go
This greatly simplifies certain parts of the code, reducing the number of goroutines and likely improving performance.
Performance for end-users improves too as there's no need anymore to pause for 2 seconds every time that we reach `maxActiveMessages`.
Additionally, with this change the config options `prefetchCount` and `maxActiveMessagesRecoveryInSec` are removed as unnecessary anymore.

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* 💄

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Fixed pubsub tests

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* These packages should have never been upgraded

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Ensuring we don't fetch 1 message more than max active

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Adding configurable timeout for servicebusqueues operations

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Persistent connection for invoking SB queues

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Reverted Event Hub SDK update

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Revert "Reverted Event Hub SDK update"

This reverts commit 212220aba0.

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Fix Azure deploy for users with a first.last email

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Added some sleep that should help reduce flakiness in eventhubs binding cert test

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Changed servicebusqueue cert test
In case of a failure in the handler (ie. users' code), the message should be correctly re-enqueued, which means that messages will be re-delivered later and won't be in order.

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Co-authored-by: halspang <halspang@microsoft.com>
Co-authored-by: tanvigour <60332928+tanvigour@users.noreply.github.com>
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
Co-authored-by: Looong Dai <long.dai@intel.com>
Co-authored-by: Wang Bing <wangbing.adam@gmail.com>
Co-authored-by: saber-wang <45062099+saber-wang@users.noreply.github.com>
2022-05-13 16:52:22 -07:00
Dmitry Shmulevich f819b290f7
redis state store: implement state query API (#1488)
Signed-off-by: Dmitry Shmulevich <dmitry.shmulevich@gmail.com>
2022-03-02 10:51:09 -08:00
Dmitry Shmulevich c7adb917f3
update license to Apache v2.0 (#1406) 2022-01-04 19:53:31 -08:00
Long Dai 988fed05ab
ci: standard linter config (#1102)
* ci: standard linter config

Signed-off-by: Long <long.dai@intel.com>

* Update utils.go

* fix pulsar issue

Signed-off-by: Long <long.dai@intel.com>

Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
2021-09-21 15:21:59 -07:00
Long Dai 189d2d6717
bindings: support rabbitmq send any content types (#1147)
Signed-off-by: Long <long0dai@foxmail.com>
2021-09-17 10:21:41 -07:00
Artur Souza 3cc0472a41
Add function to generate CloudEvent for raw subscribed events. (#864)
* Add function to generate CloudEvent for raw subscribed events.

* address code comments.
2021-05-10 10:39:26 -07:00
Christian Kaps 83803f575d
Implement bindings for the Zeebe workflow engine (#806)
* Implement bindings for the Zeebe workflow engine

* Fix PR suggestion regarding error handling

* Refactor cancel_instance test to not use a mocking library

* Refactor all tests to not use a mocking library

* Use error vars

* Fix linting error regarding ineffective error assignment

* Check for mandatory jobType

* Remove double method call

* Fix suggestion regarding mock client instantiation

* Check if either bpmnProcessId or workflowKey is given for workflow instantiation

* Changed logger module to dapr/kit. Misc tweaks.

* Fix go.mod

* Fixing some lint errors

* Ignoring linter false positive

Co-authored-by: Phil Kedy <phil.kedy@gmail.com>
2021-04-28 17:23:11 -07:00
Yaron Schneider 3ef025c604
change headers (#679) 2021-02-09 18:57:55 -08:00
kzmake 66b9734bcc
Add priority parameters to RabbitMQ queue and message (#606)
* Add message priority in RabbitMQ

* Fix integration test for ExclusiveQueue

* Revert priority parameters from pubsub

Co-authored-by: Yaron Schneider <yaronsc@microsoft.com>
Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
2021-01-22 14:03:53 -08:00
Artur Souza e6dadfab6c
TTL in PubSub. (#565)
* TTL in PubSub.

* Handle message TTL only if component cannot handle it.

* Removig dead code for overflow check in pubsub expiration.
2020-12-29 14:28:52 -08:00