Commit Graph

3297 Commits

Author SHA1 Message Date
saber-wang d7eb3b7b27
[pubsub][rabbitmq]add support for binding multiple routingkeys at the same time (#1750)
* Add metadata property to configure BatchingMaxSize&batchingMaxMessages in Pulsar
Signed-off-by: saberwang <saberwang@hotmail.com>

* sort field
Signed-off-by: saberwang <saberwang@hotmail.com>

* [pubsub]fix unit test bug
Signed-off-by: saberwang <saberwang@hotmail.com>

* remove unrelated changes
Signed-off-by: saberwang <saberwang@hotmail.com>

* Delete hard coded Metadata
Signed-off-by: saberwang <saberwang@hotmail.com>

* remove  .history

Signed-off-by: saberwang <saberwang@hotmail.com>

* restore .gitignore

Signed-off-by: saberwang <saberwang@hotmail.com>

* Hard coding default values and adding 'BatchingMaxPublishDelay' metadata

Signed-off-by: saberwang <saberwang@hotmail.com>

* fix code format

Signed-off-by: saberwang <saberwang@hotmail.com>

* formatting code

Signed-off-by: saberwang <saberwang@hotmail.com>

* [pubsub][rabbitmq]Add support for binding multiple routingkeys at the same time

Signed-off-by: saberwang <saberwang@hotmail.com>

* add TestSubscribeBindRoutingKeys

Signed-off-by: saberwang <saberwang@hotmail.com>

* fix: Infof to Debugf

Signed-off-by: saberwang <saberwang@hotmail.com>

* Update pubsub/rabbitmq/rabbitmq_test.go

Co-authored-by: Looong Dai <long.dai@intel.com>
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
2022-06-10 16:34:15 -07:00
Alessandro (Ale) Segala 98aed5c571
Make Service Bus attempt to reconnect forever in case of issues (#1783)
* Make Service Bus attempt to reconnect forever in case of issues

With exponential back-off configurable between min and max time. Fixes #1612

Also includes fixes:

- Binding: make sure it actually retries to connect forever
- Binding: add delay (exponential backoff) before reconnecting
- PubSub: better handling of failures such as topics disabled or other non-connection issues

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* 💄

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Added warning for deprecated metadata options

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* 💄

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-06-10 15:54:46 -07:00
dependabot[bot] 1db4173a9d
Bump github.com/open-policy-agent/opa from 0.23.2 to 0.40.0 (#1757)
* Bump github.com/open-policy-agent/opa from 0.23.2 to 0.40.0

Bumps [github.com/open-policy-agent/opa](https://github.com/open-policy-agent/opa) from 0.23.2 to 0.40.0.
- [Release notes](https://github.com/open-policy-agent/opa/releases)
- [Changelog](https://github.com/open-policy-agent/opa/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-policy-agent/opa/compare/v0.23.2...v0.40.0)

---
updated-dependencies:
- dependency-name: github.com/open-policy-agent/opa
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update go mods

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-06-10 13:00:34 -07:00
Alessandro (Ale) Segala 6f3234d5ac
Rabbitmq publisher confirm (#1762)
* Added `publishConfirm` option to RabbitMQ

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Changed to publisherConfirm per feedback

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-06-10 12:47:02 -07:00
Laurence 7145287978
fix: dubbo binding enhancement (#1782)
* fix: dubbo binding enhancement

Signed-off-by: LaurenceLiZhixin <382673304@qq.com>

* fix: tidy all

Signed-off-by: LaurenceLiZhixin <382673304@qq.com>

* mod tidy

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Pin couchbaselabs/gojcbmock to commit with LICENSE

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* modtidy again

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* commit to retrigger tests

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-06-10 12:16:37 -07:00
Yaron Schneider 37693b8d63
Add postgresql certification test + connection idle time (#1778)
* Add postgresql certification test + connection idle time

Signed-off-by: yaron2 <schneider.yaron@live.com>

* change flow name

Signed-off-by: yaron2 <schneider.yaron@live.com>

* go mod tidy

Signed-off-by: yaron2 <schneider.yaron@live.com>

* add to workflow

Signed-off-by: yaron2 <schneider.yaron@live.com>
2022-06-10 11:07:26 -07:00
Bernd Verst 76bc9acd57
Add additional Event Hub topics for certification tests (#1767)
* Add additional Event Hub topics for certification tests

Signed-off-by: GitHub <noreply@github.com>

* Apply EventHubs changes for pubsub and binding cert tests

Signed-off-by: GitHub <noreply@github.com>

* Remove partitions from EHs because we don't want to pay for premium

Signed-off-by: GitHub <noreply@github.com>
2022-06-08 10:26:23 -07:00
deepanshuagarwal 1af102f6f3
Add Redis StateStore Certification Tests (#1763)
* Add redis certification test

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Adding Test Plan in ReadMe

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Adding redis state cert test to workflow

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Overwrite open telemetry dependency

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-06-07 18:30:54 -07:00
arahja 4322a2280e
Updated sendgrid binding to add the option to add names for the to an… (#1751)
* Updated sendgrid binding to add the option to add names for the to and from addresses. This commit relates to #1749

Signed-off-by: Adam Rahja <adam@resolve.com>

* Updated sendgrid binding from code suggestion.

Signed-off-by: Adam Rahja <adam@resolve.com>
2022-06-07 09:20:45 -07:00
Pravin Pushkar 3f85dd6702
adding query api to the feature list (#1775)
Signed-off-by: Pravin Pushkar <ppushkar@microsoft.com>

Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-06-07 08:38:02 -07:00
Alessandro (Ale) Segala 157ddc6d6b
Bumped dapr/dapr in cert tests (#1773)
* Bumped dapr/dapr in cert tests

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Updated interface

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Added missing go.mod

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* More fixes

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Fix eventhubs not closing on context cancelation

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-06-07 08:24:15 -07:00
Alessandro (Ale) Segala 704f4dd730
Add contexts to pubsub.Subscribe to allow early cancelation (#1756)
This commit is related to dapr/dapr#4624. As noted there, we have an issue in the runtime where all components are shut down after the grace period, when the app is likely already stopped. Because of that, certain input components (the subscribe part of pubsub and the input part of bindings - the latter not in scope of this PR), can continue bringing new work when it's known to fail.

In order to fix the issue linked above properly, we need to implement a way for PubSub components to have the "publish" part closed before the "subscribe" one (and in the future that will need to be done for input bindings too).

This commit achieves precisely that by adding a context in the Subscribe method. When that context is canceled (which can be at any time), the subscription is removed.

PS: This API change was implemented so it can one day be used for dapr/dapr#814 too, as it allows canceling individual subscriptions by using a different context. Although that's not possible today because it requires more work on the runtime, it does implement everything that's needed in the pubsub components already.
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-06-02 15:06:36 -07:00
Alessandro (Ale) Segala 7c35a4e04c
Added test for multiple pubsub handlers (#1743)
* Added test for multiple pubsub handlers

This patch adds conformance tests for PubSub with multiple handlers, each listening on a different topic.

It's meant to catch issues such as #1709

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Added missing go.mod for storagequeues binding test

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-06-02 10:22:15 -07:00
Alessandro (Ale) Segala 6c7ff60c9e
Fixes for SNS/SQS PubSub (#1754)
* Fixes for SNS/SQS PubSub

Including:

- Fixed adding multiple handlers for different topics (tests in #1743 are now passing)
- Fixed delivering wrong message when concurrency mode is parallel
- Avoid having multiple pollers at the same time competing for messages

& more smaller fixes

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Fixed tests

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-06-01 15:49:51 -07:00
Alessandro (Ale) Segala ca9fbf690e
Fixed Kafka PubSub to allow multiple handlers for different topics (#1755)
* Fixed Kafka PubSub to allow multiple handlers for different topics

With this, tests from #1743 are passing

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Removed read lock (for now)

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Updated as requested by @skyao

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Use interface type everywhere

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Fixed panic

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
2022-06-01 15:31:11 -07:00
Laurence a323748d75
Feat: Add apache dubbo output binding (#1768)
* feat: support apache/dubbo

Signed-off-by: LaurenceLiZhixin <382673304@qq.com>

* fix: comments

Signed-off-by: LaurenceLiZhixin <382673304@qq.com>

* fix: gofumpt test file

Signed-off-by: LaurenceLiZhixin <382673304@qq.com>

* feat: dubbo cert test

Signed-off-by: LaurenceLiZhixin <382673304@qq.com>

* fix: add missiing header

Signed-off-by: LaurenceLiZhixin <382673304@qq.com>

Co-authored-by: Loong Dai <long.dai@intel.com>
2022-06-01 10:38:29 -07:00
Sky Ao e2aced1169
Add certification test for kafka bindings component (#1758)
* add certification test for kafka binding, the first version is copied from kafka pubsub to verify github flow

Signed-off-by: Sky Ao <aoxiaojian@gmail.com>

* only run kafka pubsub and binding cert test in development

* fix typo

Signed-off-by: Sky Ao <aoxiaojian@gmail.com>

* run only kafka bindings

Signed-off-by: Sky Ao <aoxiaojian@gmail.com>

* add certification test for kafka binding: with and without retry

Signed-off-by: Sky Ao <aoxiaojian@gmail.com>

* add config.yaml file back for kafka binding certification test

Signed-off-by: Sky Ao <aoxiaojian@gmail.com>

* rollback to run all cert test before PR

Signed-off-by: Sky Ao <aoxiaojian@gmail.com>

* update dependencies for kafka pubsub certification test

Signed-off-by: Sky Ao <aoxiaojian@gmail.com>

* update dependencies for kafka pubsub certification test(again)

Signed-off-by: Sky Ao <aoxiaojian@gmail.com>

* add network interruption test back

Signed-off-by: Sky Ao <aoxiaojian@gmail.com>
2022-05-31 17:15:37 -07:00
Loong Dai d38957ae5e
Merge pull request #1760 from yaron2/natsparallel-1
NATS Streaming: Add parallel processing, fix retry bug
2022-05-31 11:33:40 +08:00
yaron2 05922117c6 add parallel test
Signed-off-by: yaron2 <schneider.yaron@live.com>
2022-05-27 09:52:39 -07:00
yaron2 8066c64284 Merge branch 'natsparallel-1' of github.com:yaron2/components-contrib-1 into natsparallel-1 2022-05-26 11:58:05 -07:00
yaron2 64c4dc206c linter
Signed-off-by: yaron2 <schneider.yaron@live.com>
2022-05-26 11:57:57 -07:00
Yaron Schneider 0d26824967
Merge branch 'master' into natsparallel-1 2022-05-26 11:49:58 -07:00
yaron2 ca36ecc556 add parllel processing, fix retry bug
Signed-off-by: yaron2 <schneider.yaron@live.com>
2022-05-26 11:44:56 -07:00
Shubham Sharma d03aeba3e0
Upgrade rocketmq version (#1740)
* Upgrade rocketmq

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Trigger Build

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>
2022-05-26 09:24:29 -07:00
amulyavarote 5c410fb726
Added certification tests for Azure Storage Queue Bindings (#1736)
* Added certification tests for Azure Storage Queue Bindings

Signed-off-by: Amulya Varote <amulyavarote@microsoft.com>

* Changes based on the review comments

Signed-off-by: Amulya Varote <amulyavarote@microsoft.com>

* Added ttl test case to fetch the unexpired messages

Signed-off-by: Amulya Varote <amulyavarote@microsoft.com>
2022-05-25 17:26:17 -07:00
Alessandro (Ale) Segala 3a2a27a253
Better handling of concurrency in MQTT (#1748)
This helps avoiding issues when multiple subscribers are added in a short period of time

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-05-24 10:25:35 -07:00
Alessandro (Ale) Segala d55ce1dfdb
Fix: route messages to the correct publisher (#1737)
Fixes #1735
Includes:
- Adding a lock in `Subscribe` to prevent concurrency issues with multiple reconnections at the same time
- Increase default timeout from 3 to 30 seconds
- Avoid logging full message body in debug logs
- `connect` supports a context

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-05-24 07:49:08 -07:00
Yaron Schneider 47e95c9a47
Update redis go to v8.11.5 (#1734)
* update redis go to v8.11.5

Signed-off-by: yaron2 <schneider.yaron@live.com>

* go mod update

Signed-off-by: yaron2 <schneider.yaron@live.com>
2022-05-20 11:55:17 -07:00
Alessandro (Ale) Segala f7834bd232
Fixed typo in metadata JSON key (#1733)
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-05-19 12:35:30 -07:00
Bernd Verst c2cb1dad5b
Update Blobstorage Binding certification test (#1730)
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
2022-05-18 23:12:49 -07:00
Laurence 07e2657965
fix: Add nacos cert tests (#1723)
* ftr: nacos cert

Signed-off-by: LaurenceLiZhixin <382673304@qq.com>

* Fix: linter

Signed-off-by: LaurenceLiZhixin <382673304@qq.com>

* Fix: add readme

Signed-off-by: LaurenceLiZhixin <382673304@qq.com>

Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
2022-05-18 15:24:00 -07:00
Jasonli 499f6b2d49
Implement new bindings component for Huawei OBS (#1724)
* Add new bindings component for Huawei OBS

Signed-off-by: jasonlica <jasonli@huawei.com>

* resolve conflicts in go.sum respective to the changes #1710

Signed-off-by: jasonlica <jasonli@huawei.com>

* update go.mod

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Update go.mod

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-05-17 23:12:39 -07:00
Alessandro (Ale) Segala e1b6b01223
Tweaks for ASB pubsub (#1729)
* Tweaks for ASB pubsub:
1. Do not attempt to reconnect when context was canceled
2. Message finalization uses a background context and not the running ctx, in case the app is shutting down

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* ASB: abandoned messages shouldn't be retried right away

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Ported fixes to servicebusqueues binding too

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Use a different approach to limit the number of messages processed in case of error

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Added maxRetriableErrorsPerSec to servicebusqueues binding too
Incl other improvements

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Do not reconnect to ASB every time we retrieve a message.

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* ⚙️

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-05-17 22:33:34 -07:00
seeflood 948540e5a7
Add an in-memory state component (#1326)
* goimports -local github.com/dapr/ -w .

Signed-off-by: seeflood <349895584@qq.com>

* add

Signed-off-by: seeflood <349895584@qq.com>

* rename test cases

Signed-off-by: seeflood <349895584@qq.com>

* fix pointer check

Signed-off-by: seeflood <349895584@qq.com>

* add Close()

Signed-off-by: seeflood <349895584@qq.com>

* doGetWithWriteLock3

Signed-off-by: seeflood <349895584@qq.com>

* fix typo

Signed-off-by: seeflood <349895584@qq.com>

* fix ut

Signed-off-by: seeflood <349895584@qq.com>

* release memory reference

Signed-off-by: seeflood <349895584@qq.com>

* delete test

Signed-off-by: seeflood <349895584@qq.com>

* fix npe

Signed-off-by: seeflood <349895584@qq.com>

* fix

Signed-off-by: seeflood <349895584@qq.com>

Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
2022-05-15 19:36:02 -07:00
Alessandro (Ale) Segala e5e9011655
Update Azure Service Bus components to track2 SDK (#1702)
* Updated to Go 1.18

Signed-off-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com>
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>

* Updated Azure SDKs that are on track2
Includes some minor refactoring of auth code

Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>

* Updated Service Bus components to track2 SDK

Co-authored-by: halspang <halspang@microsoft.com>
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>

* Initial Certification test for eventhubs binding [incomplete] (#1670)

* certification test for eventhubs binding

Signed-off-by: tanvigour <tanvi.gour@gmail.com>

* modified go.mod and go.sum

Signed-off-by: tanvigour <tanvi.gour@gmail.com>

* Add connection string testing

Signed-off-by: tanvigour <tanvi.gour@gmail.com>

* iothub testing

Signed-off-by: tanvigour <tanvi.gour@gmail.com>

* address feedback and run test

Signed-off-by: tanvigour <tanvi.gour@gmail.com>

* Install Azure CLI IOT hub extension

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* make modtidy-all

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* covering all eventhubs test cases

Signed-off-by: tanvigour <tanvi.gour@gmail.com>

* dependency changes after go modtidy-all

Signed-off-by: tanvigour <tanvi.gour@gmail.com>

Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
Co-authored-by: Looong Dai <long.dai@intel.com>
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>

* Use revive instead of golint (#1685)

Signed-off-by: pigletfly <wangbing.adam@gmail.com>

Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>

* Updated to Go 1.18 (#1697)

* Updated to Go 1.18

Signed-off-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com>

* Added go.work file
With Go 1.18, this allows gopls (the Go language server used for example in VS Code) to work inside test apps too.
See: https://go.dev/doc/tutorial/workspaces

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Removed go.work

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* 💄

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>

* 💄 & 🧹

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>

* Add metadata property to configure Batching in Pulsar (#1707)

* Add metadata property to configure BatchingMaxSize&batchingMaxMessages in Pulsar
Signed-off-by: saberwang <saberwang@hotmail.com>

* sort field
Signed-off-by: saberwang <saberwang@hotmail.com>

* [pubsub]fix unit test bug
Signed-off-by: saberwang <saberwang@hotmail.com>

* remove unrelated changes
Signed-off-by: saberwang <saberwang@hotmail.com>

* Delete hard coded Metadata
Signed-off-by: saberwang <saberwang@hotmail.com>

* remove  .history

Signed-off-by: saberwang <saberwang@hotmail.com>

* restore .gitignore

Signed-off-by: saberwang <saberwang@hotmail.com>

* Hard coding default values and adding 'BatchingMaxPublishDelay' metadata

Signed-off-by: saberwang <saberwang@hotmail.com>

* fix code format

Signed-off-by: saberwang <saberwang@hotmail.com>

* formatting code

Signed-off-by: saberwang <saberwang@hotmail.com>

Co-authored-by: Looong Dai <long.dai@intel.com>
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>

* This test can't work with track2 SDKs
The methods to create a message with a body are not exported

Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>

* 🧹

Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>

* There's such thing as too much logging

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Refactored subscription.go
This greatly simplifies certain parts of the code, reducing the number of goroutines and likely improving performance.
Performance for end-users improves too as there's no need anymore to pause for 2 seconds every time that we reach `maxActiveMessages`.
Additionally, with this change the config options `prefetchCount` and `maxActiveMessagesRecoveryInSec` are removed as unnecessary anymore.

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* 💄

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Fixed pubsub tests

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* These packages should have never been upgraded

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Ensuring we don't fetch 1 message more than max active

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Adding configurable timeout for servicebusqueues operations

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Persistent connection for invoking SB queues

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Reverted Event Hub SDK update

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Revert "Reverted Event Hub SDK update"

This reverts commit 212220aba0.

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Fix Azure deploy for users with a first.last email

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Added some sleep that should help reduce flakiness in eventhubs binding cert test

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Changed servicebusqueue cert test
In case of a failure in the handler (ie. users' code), the message should be correctly re-enqueued, which means that messages will be re-delivered later and won't be in order.

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Co-authored-by: halspang <halspang@microsoft.com>
Co-authored-by: tanvigour <60332928+tanvigour@users.noreply.github.com>
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
Co-authored-by: Looong Dai <long.dai@intel.com>
Co-authored-by: Wang Bing <wangbing.adam@gmail.com>
Co-authored-by: saber-wang <45062099+saber-wang@users.noreply.github.com>
2022-05-13 16:52:22 -07:00
Eddie 785ed60ef4
GH-1609 : Fix for MongoDB Atlas conn strings (#1698)
* Update readme of bindings (#1690)

Signed-off-by: pigletfly <wangbing.adam@gmail.com>

Co-authored-by: Looong Dai <long.dai@intel.com>
Signed-off-by: Eddie <eddie@wassef.com>

* Fixing includedHeaders problem with spaces (#1610)

Signed-off-by: Ben Kotvis <kotvisbj@gmail.com>

Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Co-authored-by: Looong Dai <long.dai@intel.com>
Signed-off-by: Eddie <eddie@wassef.com>

* Simplify vault token read (#1560)

* Simplify vault token get

Signed-off-by: zhangchao <zchao9100@gmail.com>

* fix lint

Signed-off-by: zhangchao <zchao9100@gmail.com>

* update tests

Signed-off-by: zhangchao <zchao9100@gmail.com>

Co-authored-by: Looong Dai <long.dai@intel.com>
Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
Signed-off-by: Eddie <eddie@wassef.com>

* GH-1609 : Fix for MongoDB Atlas conn strings

Added recommended fix in the issue

Signed-off-by: Eddie <eddie@wassef.com>

* updating the comment based on PR feedback

Signed-off-by: Eddie <eddie@wassef.com>

* Initial Certification test for eventhubs binding [incomplete] (#1670)

* certification test for eventhubs binding

Signed-off-by: tanvigour <tanvi.gour@gmail.com>

* modified go.mod and go.sum

Signed-off-by: tanvigour <tanvi.gour@gmail.com>

* Add connection string testing

Signed-off-by: tanvigour <tanvi.gour@gmail.com>

* iothub testing

Signed-off-by: tanvigour <tanvi.gour@gmail.com>

* address feedback and run test

Signed-off-by: tanvigour <tanvi.gour@gmail.com>

* Install Azure CLI IOT hub extension

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* make modtidy-all

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* covering all eventhubs test cases

Signed-off-by: tanvigour <tanvi.gour@gmail.com>

* dependency changes after go modtidy-all

Signed-off-by: tanvigour <tanvi.gour@gmail.com>

Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
Co-authored-by: Looong Dai <long.dai@intel.com>
Signed-off-by: Eddie <eddie@wassef.com>

* Use revive instead of golint (#1685)

Signed-off-by: pigletfly <wangbing.adam@gmail.com>

Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
Signed-off-by: Eddie <eddie@wassef.com>

* Updated to Go 1.18 (#1697)

* Updated to Go 1.18

Signed-off-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com>

* Added go.work file
With Go 1.18, this allows gopls (the Go language server used for example in VS Code) to work inside test apps too.
See: https://go.dev/doc/tutorial/workspaces

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Removed go.work

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* 💄

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Signed-off-by: Eddie <eddie@wassef.com>

* Add metadata property to configure Batching in Pulsar (#1707)

* Add metadata property to configure BatchingMaxSize&batchingMaxMessages in Pulsar
Signed-off-by: saberwang <saberwang@hotmail.com>

* sort field
Signed-off-by: saberwang <saberwang@hotmail.com>

* [pubsub]fix unit test bug
Signed-off-by: saberwang <saberwang@hotmail.com>

* remove unrelated changes
Signed-off-by: saberwang <saberwang@hotmail.com>

* Delete hard coded Metadata
Signed-off-by: saberwang <saberwang@hotmail.com>

* remove  .history

Signed-off-by: saberwang <saberwang@hotmail.com>

* restore .gitignore

Signed-off-by: saberwang <saberwang@hotmail.com>

* Hard coding default values and adding 'BatchingMaxPublishDelay' metadata

Signed-off-by: saberwang <saberwang@hotmail.com>

* fix code format

Signed-off-by: saberwang <saberwang@hotmail.com>

* formatting code

Signed-off-by: saberwang <saberwang@hotmail.com>

Co-authored-by: Looong Dai <long.dai@intel.com>
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Signed-off-by: Eddie <eddie@wassef.com>

* Fix 4529: Ignore Subscribe/Get wrong redis configuration type keys. (#1693)

* fix: 4529

Signed-off-by: LaurenceLiZhixin <382673304@qq.com>

* Fix: add test does not throw error for wrong type during get all test case of redis configuration

Signed-off-by: LaurenceLiZhixin <382673304@qq.com>

Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
Co-authored-by: Ian Luo <ian.luo@gmail.com>
Signed-off-by: Eddie <eddie@wassef.com>

* Signed-off-by: Eddie Wassef <eddie@wassef.com>

Fixing leading newline in linter error

Signed-off-by: Eddie <eddie@wassef.com>

* Implment a secret store based on Huawei CSMS (#1710)

Signed-off-by: Chen Cong <chock-cong.chen@outlook.com>

Co-authored-by: Chen Cong <chock-cong.chen@outlook.com>
Signed-off-by: Eddie <eddie@wassef.com>

* Add yet another missing secret to eventhub binding cert test (#1713)

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Signed-off-by: Eddie <eddie@wassef.com>

* Support custom queueEndpoint in Azure Storage Queues (#1692)

* Support custom queueEndpoint in Azure Storage Queues

Signed-off-by: Janusz Dziurzynski <janusz@corechain.tech>

* run gofmt

Signed-off-by: Janusz Dziurzynski <janusz@corechain.tech>

* Add "Url" to JSON field name for clarity

Suggested by @msfussell in dapr/docs#2424

Signed-off-by: Janusz Dziurzynski <janusz@corechain.tech>
Signed-off-by: Eddie <eddie@wassef.com>

* Refactory kafka binding to reuse the kafka common code extracting from kafka pubsub component (#1696)

* refactory kafka pubsub code to extract common kafka code for reuse

Signed-off-by: Sky Ao <aoxiaojian@gmail.com>

* fix lint;add unit test for subscribeAdapter

Signed-off-by: Sky Ao <aoxiaojian@gmail.com>

* move topics filed from internal kafak struct to pubsub kafka struct, since in input binding the topics will confiured in metadata

Signed-off-by: Sky Ao <aoxiaojian@gmail.com>

* reuse internal  kafka code for bindings

Signed-off-by: Sky Ao <aoxiaojian@gmail.com>

* add redis standalone_test back which is delete by mistaken

Signed-off-by: Sky Ao <aoxiaojian@gmail.com>

* small code improvement to trigger test

Signed-off-by: Sky Ao <aoxiaojian@gmail.com>

* add license headers

Signed-off-by: Sky Ao <aoxiaojian@gmail.com>

* try to set disbaleTls to true to verify the kafka connection fail

Signed-off-by: Sky Ao <aoxiaojian@gmail.com>

* don't enable consum retry in kafka binding component;if authenticaion is disabled, need not set TLSDisable at the same time;

Signed-off-by: Sky Ao <aoxiaojian@gmail.com>

* fix lint

Signed-off-by: Sky Ao <aoxiaojian@gmail.com>

Co-authored-by: Loong Dai <long.dai@intel.com>
Signed-off-by: Eddie <eddie@wassef.com>

* Add topic metadata for mqtt input binding and support user defined topic for mqtt output binding (#1674)

* feat(bindings/mqtt): add data incoming topic to metadata

Signed-off-by: lotuc <lotu.c@outlook.com>

* feat(bindings/mqtt): support user defined topic on create action

Signed-off-by: lotuc <lotu.c@outlook.com>

* chore(bindings/mqtt): add integration test and topic response check test

Signed-off-by: lotuc <lotu.c@outlook.com>

* fix(bindings/mqtt): ignore misspell linting error for word mosquitto

Signed-off-by: lotuc <lotu.c@outlook.com>

Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
Signed-off-by: Eddie <eddie@wassef.com>

* Expire -> ExpiryInSeconds (#1721)

Signed-off-by: seeflood <349895584@qq.com>
Signed-off-by: Eddie <eddie@wassef.com>

* running gofmt -s -w state/mongodb/mongodb.go

Signed-off-by: Eddie <eddie@wassef.com>

* Update mongodb.go

Co-authored-by: Wang Bing <wangbing.adam@gmail.com>
Co-authored-by: Looong Dai <long.dai@intel.com>
Co-authored-by: Ben Kotvis <kotvisbj@gmail.com>
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Co-authored-by: Taction <zchao9100@gmail.com>
Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
Co-authored-by: tanvigour <60332928+tanvigour@users.noreply.github.com>
Co-authored-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com>
Co-authored-by: saber-wang <45062099+saber-wang@users.noreply.github.com>
Co-authored-by: Laurence <45508533+LaurenceLiZhixin@users.noreply.github.com>
Co-authored-by: Ian Luo <ian.luo@gmail.com>
Co-authored-by: Chock Chen <46557936+ChenCong0207@users.noreply.github.com>
Co-authored-by: Chen Cong <chock-cong.chen@outlook.com>
Co-authored-by: Janusz Dziurzynski <janusz@forserial.org>
Co-authored-by: Sky Ao <aoxiaojian@gmail.com>
Co-authored-by: lotuc <lotu.c@outlook.com>
Co-authored-by: seeflood <349895584@qq.com>
2022-05-13 16:13:13 -07:00
seeflood 9bb86a08a6
Expire -> ExpiryInSeconds (#1721)
Signed-off-by: seeflood <349895584@qq.com>
2022-05-12 04:13:21 -07:00
lotuc 5e4bd89afd
Add topic metadata for mqtt input binding and support user defined topic for mqtt output binding (#1674)
* feat(bindings/mqtt): add data incoming topic to metadata

Signed-off-by: lotuc <lotu.c@outlook.com>

* feat(bindings/mqtt): support user defined topic on create action

Signed-off-by: lotuc <lotu.c@outlook.com>

* chore(bindings/mqtt): add integration test and topic response check test

Signed-off-by: lotuc <lotu.c@outlook.com>

* fix(bindings/mqtt): ignore misspell linting error for word mosquitto

Signed-off-by: lotuc <lotu.c@outlook.com>

Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
2022-05-11 12:11:51 -07:00
Sky Ao 8b5554dc71
Refactory kafka binding to reuse the kafka common code extracting from kafka pubsub component (#1696)
* refactory kafka pubsub code to extract common kafka code for reuse

Signed-off-by: Sky Ao <aoxiaojian@gmail.com>

* fix lint;add unit test for subscribeAdapter

Signed-off-by: Sky Ao <aoxiaojian@gmail.com>

* move topics filed from internal kafak struct to pubsub kafka struct, since in input binding the topics will confiured in metadata

Signed-off-by: Sky Ao <aoxiaojian@gmail.com>

* reuse internal  kafka code for bindings

Signed-off-by: Sky Ao <aoxiaojian@gmail.com>

* add redis standalone_test back which is delete by mistaken

Signed-off-by: Sky Ao <aoxiaojian@gmail.com>

* small code improvement to trigger test

Signed-off-by: Sky Ao <aoxiaojian@gmail.com>

* add license headers

Signed-off-by: Sky Ao <aoxiaojian@gmail.com>

* try to set disbaleTls to true to verify the kafka connection fail

Signed-off-by: Sky Ao <aoxiaojian@gmail.com>

* don't enable consum retry in kafka binding component;if authenticaion is disabled, need not set TLSDisable at the same time;

Signed-off-by: Sky Ao <aoxiaojian@gmail.com>

* fix lint

Signed-off-by: Sky Ao <aoxiaojian@gmail.com>

Co-authored-by: Loong Dai <long.dai@intel.com>
2022-05-10 20:07:23 -07:00
Janusz Dziurzynski 62c5a2b3d1
Support custom queueEndpoint in Azure Storage Queues (#1692)
* Support custom queueEndpoint in Azure Storage Queues

Signed-off-by: Janusz Dziurzynski <janusz@corechain.tech>

* run gofmt

Signed-off-by: Janusz Dziurzynski <janusz@corechain.tech>

* Add "Url" to JSON field name for clarity

Suggested by @msfussell in dapr/docs#2424

Signed-off-by: Janusz Dziurzynski <janusz@corechain.tech>
2022-05-10 12:27:10 -07:00
Bernd Verst 909d3960db
Add yet another missing secret to eventhub binding cert test (#1713)
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-05-09 19:07:42 -07:00
Chock Chen b818c2f83e
Implment a secret store based on Huawei CSMS (#1710)
Signed-off-by: Chen Cong <chock-cong.chen@outlook.com>

Co-authored-by: Chen Cong <chock-cong.chen@outlook.com>
2022-05-09 15:52:46 -07:00
Laurence e0ea27f2c1
Fix 4529: Ignore Subscribe/Get wrong redis configuration type keys. (#1693)
* fix: 4529

Signed-off-by: LaurenceLiZhixin <382673304@qq.com>

* Fix: add test does not throw error for wrong type during get all test case of redis configuration

Signed-off-by: LaurenceLiZhixin <382673304@qq.com>

Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
Co-authored-by: Ian Luo <ian.luo@gmail.com>
2022-05-07 14:15:07 -07:00
saber-wang a51de3bb09
Add metadata property to configure Batching in Pulsar (#1707)
* Add metadata property to configure BatchingMaxSize&batchingMaxMessages in Pulsar
Signed-off-by: saberwang <saberwang@hotmail.com>

* sort field
Signed-off-by: saberwang <saberwang@hotmail.com>

* [pubsub]fix unit test bug
Signed-off-by: saberwang <saberwang@hotmail.com>

* remove unrelated changes
Signed-off-by: saberwang <saberwang@hotmail.com>

* Delete hard coded Metadata
Signed-off-by: saberwang <saberwang@hotmail.com>

* remove  .history

Signed-off-by: saberwang <saberwang@hotmail.com>

* restore .gitignore

Signed-off-by: saberwang <saberwang@hotmail.com>

* Hard coding default values and adding 'BatchingMaxPublishDelay' metadata

Signed-off-by: saberwang <saberwang@hotmail.com>

* fix code format

Signed-off-by: saberwang <saberwang@hotmail.com>

* formatting code

Signed-off-by: saberwang <saberwang@hotmail.com>

Co-authored-by: Looong Dai <long.dai@intel.com>
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-05-06 17:59:27 -07:00
Alessandro (Ale) Segala e6c67356cd
Updated to Go 1.18 (#1697)
* Updated to Go 1.18

Signed-off-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com>

* Added go.work file
With Go 1.18, this allows gopls (the Go language server used for example in VS Code) to work inside test apps too.
See: https://go.dev/doc/tutorial/workspaces

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Removed go.work

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* 💄

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-05-06 13:44:45 -07:00
Wang Bing db99b43068
Use revive instead of golint (#1685)
Signed-off-by: pigletfly <wangbing.adam@gmail.com>

Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
2022-05-06 12:55:17 -07:00
tanvigour 47db6cc2b1
Initial Certification test for eventhubs binding [incomplete] (#1670)
* certification test for eventhubs binding

Signed-off-by: tanvigour <tanvi.gour@gmail.com>

* modified go.mod and go.sum

Signed-off-by: tanvigour <tanvi.gour@gmail.com>

* Add connection string testing

Signed-off-by: tanvigour <tanvi.gour@gmail.com>

* iothub testing

Signed-off-by: tanvigour <tanvi.gour@gmail.com>

* address feedback and run test

Signed-off-by: tanvigour <tanvi.gour@gmail.com>

* Install Azure CLI IOT hub extension

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* make modtidy-all

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* covering all eventhubs test cases

Signed-off-by: tanvigour <tanvi.gour@gmail.com>

* dependency changes after go modtidy-all

Signed-off-by: tanvigour <tanvi.gour@gmail.com>

Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
Co-authored-by: Looong Dai <long.dai@intel.com>
2022-05-05 19:33:23 -07:00
Taction bfd87eb7a5
Simplify vault token read (#1560)
* Simplify vault token get

Signed-off-by: zhangchao <zchao9100@gmail.com>

* fix lint

Signed-off-by: zhangchao <zchao9100@gmail.com>

* update tests

Signed-off-by: zhangchao <zchao9100@gmail.com>

Co-authored-by: Looong Dai <long.dai@intel.com>
Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
2022-04-28 13:56:50 -07:00
Ben Kotvis 1ef6cc5cdc
Fixing includedHeaders problem with spaces (#1610)
Signed-off-by: Ben Kotvis <kotvisbj@gmail.com>

Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Co-authored-by: Looong Dai <long.dai@intel.com>
2022-04-28 13:53:15 -07:00
Wang Bing e6b8577997
Update readme of bindings (#1690)
Signed-off-by: pigletfly <wangbing.adam@gmail.com>

Co-authored-by: Looong Dai <long.dai@intel.com>
2022-04-28 13:51:58 -07:00