Commit Graph

569 Commits

Author SHA1 Message Date
ItalyPaleAle fc00023206 Small naming fixes
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-12-19 22:03:53 +00:00
mecoding1 0f6aedfd1c
Fix linting issue
Fix linting issue

Signed-off-by: mecoding1 <118708378+mecoding1@users.noreply.github.com>
2022-12-19 14:44:04 +05:30
seachen 5dda098868 feature: add context to lock&pubsub API
Signed-off-by: seachen <seachen@tencent.com>
2022-12-19 09:11:53 +08:00
Mukundan Sundararajan b7817c0ee7 Merge branch 'master' into optimize-bulkpubres-struct 2022-12-17 11:05:39 +05:30
Bernd Verst 3e08736911
Add support for Redis 7 in all Dapr Components (#2228)
* Add support for Redis 7

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Add Redis 7 to conformance tests

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* automatically detect redis version, make redis7 state conformant

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Additions for REDIS without JSON support

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Return helpful query API error when missing redis-json

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Change Redis 7 port for conformance test

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Fix nil reference

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Update Redis7 port

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Enable ContextTimeoutEnabled for Redis v9 client

Co-Authored-By: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Add timeouts, address code review

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Add another timeout

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Redis: so much more context

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* modtidy

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* fix issue from latest merge

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* modtidy

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Add generic redis nil error

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* fix redis version check

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* upgrade redis v9 client

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Remove flaky redis v7 pubsub conformance test

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* modtidy

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Co-authored-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-12-16 14:12:11 -08:00
Bernd Verst 16e1be02d1
Merge branch 'master' into master 2022-12-16 11:54:51 -08:00
Mukundan Sundararajan 0841b3d846 Merge branch 'master' into optimize-bulkpubres-struct 2022-12-15 23:00:34 +05:30
seachen 1ea2cff9fd feature: add context to lock&pubsub API
Signed-off-by: seachen <seachen@tencent.com>
2022-12-15 09:53:32 +08:00
Bernd Verst 64a2c3aad1 Revert "revert amqp changes, eventhubs library update"
This reverts commit b236279d81.
2022-12-14 16:19:06 -08:00
Tomasz Pietrek 72bd117e7e
Rework JetStream delivery policy (#2315)
* Rework JetStream delivery policy

Signed-off-by: Tomasz Pietrek <tomasz@nats.io>

* Add missing deliver policies to error

Co-authored-by: Piotr Piotrowski <piotrpiotrowski94@gmail.com>
Signed-off-by: Tomasz Pietrek <melgaer@gmail.com>
Signed-off-by: Tomasz Pietrek <tomasz@nats.io>

* Rearrange imports

Signed-off-by: Tomasz Pietrek <tomasz@nats.io>

Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
Signed-off-by: Tomasz Pietrek <melgaer@gmail.com>
Co-authored-by: Piotr Piotrowski <piotrpiotrowski94@gmail.com>
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
2022-12-14 15:56:07 -08:00
Yaron Schneider 0b0deea39e
Merge branch 'master' into feat/rabbitmq-tls 2022-12-14 14:34:51 -08:00
seachen e59dfd63c0 feature: add context to lock&pubsub API
Signed-off-by: seachen <seachen@tencent.com>
2022-12-14 19:37:24 +08:00
Mukundan Sundararajan 4b1bb38820 Merge branch 'master' into optimize-bulkpubres-struct 2022-12-14 10:52:40 +05:30
Mukundan Sundararajan ace0d1a03c changed to failed entries for bulk publish response
Signed-off-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com>
2022-12-13 16:22:45 +05:30
Mukundan Sundararajan 72695529f6 optimize bulk pub response to contain only failed entries
Signed-off-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com>
2022-12-13 11:49:04 +05:30
spike 6254982cd7 Raise error if protocol does not match connection string
Signed-off-by: spike <hello@spike.wiki>
2022-12-06 22:27:54 +08:00
Tomasz Pietrek 9df618959e Fix formatting
Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2022-12-05 17:56:02 +01:00
Tomasz Pietrek d2e3a7e533 Test ack policies metadata
Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2022-12-05 17:47:35 +01:00
Tomasz Pietrek 0e80570949 Fix typo
Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2022-12-05 15:57:14 +01:00
Tomasz Pietrek 3a8a9a0ef6 Fix error handling in Ack
Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2022-12-05 14:33:12 +01:00
Tomasz Pietrek 0463c2cf8b Adopt handler to work properly with each ack policy in NATS
Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2022-12-05 14:19:13 +01:00
Tomasz Pietrek 7ec2b8608f Add ack policy to NATS pub/sub
Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2022-12-05 14:15:00 +01:00
Bernd Verst f656319740
Merge branch 'master' into feat/rabbitmq-tls 2022-12-02 13:54:50 -08:00
Tomasz Pietrek 86cf34ef8f
Add missing filter subject in Jetstream pubsub (#2312)
Signed-off-by: Tomasz Pietrek <tomasz@nats.io>

Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com>
2022-11-30 10:46:15 -08:00
thielepaul 47b0d73676
Add mqtt retained information to pubsub message metadata (#2299)
Signed-off-by: Paul Thiele <thielepaul@gmail.com>

Signed-off-by: Paul Thiele <thielepaul@gmail.com>
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Co-authored-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com>
Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com>
2022-11-29 10:06:20 -08:00
Deepanshu Agarwal a6289ccff2 Ack Retained Message too
Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>
2022-11-24 18:36:44 +05:30
mecoding1 71d7acb506
Fix lint issue for eventshub.go
Signed-off-by: mecoding1 <118708378+mecoding1@users.noreply.github.com>
2022-11-23 10:48:59 +05:30
mecoding1 a410556364
Fix lint issue for eventshub.go
Signed-off-by: mecoding1 <118708378+mecoding1@users.noreply.github.com>
2022-11-23 09:59:49 +05:30
mecoding1 94ae646a03
Add requireAllProperties metadata to the subscribe request.
Signed-off-by: mecoding1 <118708378+mecoding1@users.noreply.github.com>
2022-11-23 09:57:03 +05:30
mecoding1 093b6aa525 Add custom application properties to tests/certification/pubsub/azure/eventhubs/send-iot-device-events.sh
Signed-off-by: mecoding1 <118708378+mecoding1@users.noreply.github.com>
2022-11-22 12:48:19 +00:00
mecoding1 20a004b6a7 Asserting that added custom application property is received
Asserting that added custom application property is received

Signed-off-by: mecoding1 <118708378+mecoding1@users.noreply.github.com>
2022-11-22 12:48:19 +00:00
mecoding1 3b6712c68f Fix lint issues.
Signed-off-by: mecoding1 <118708378+mecoding1@users.noreply.github.com>
2022-11-22 12:48:19 +00:00
mecoding1 9e51ac90e4 Update eventHubs.go for lint fix ST1023:
Signed-off-by: mecoding1 <118708378+mecoding1@users.noreply.github.com>
2022-11-22 12:48:19 +00:00
mecoding1 a669c4733b Updating eventshub.go, to ensure receiving custom properties from IoT Hub
Currently, we can receive only the system properties from iot hub. Done changes should allow the retrieval of all custom properties.
It is based on metadata key : requireAllProperties.

By default it is false, if true , looks into the , Properties of Event struct to retrieve the custom properties.

Signed-off-by: mecoding1 <118708378+mecoding1@users.noreply.github.com>
2022-11-22 12:48:19 +00:00
Armin Schlegel 0bbc8c42d6 jetstream pubsub now supports token based auth (#2295)
Signed-off-by: Armin Schlegel <armin.schlegel@gmx.de>

Signed-off-by: Armin Schlegel <armin.schlegel@gmx.de>
Signed-off-by: mecoding1 <118708378+mecoding1@users.noreply.github.com>
2022-11-22 12:48:19 +00:00
spike 17bc6e14b5 Merge branch 'feat/rabbitmq-tls' of git-spike:anoymouscoder/components-contrib into feat/rabbitmq-tls 2022-11-20 15:34:07 +08:00
spike 55a06c40c4 Optimize code according to lint
Signed-off-by: spike <hello@spike.wiki>
2022-11-20 15:33:53 +08:00
Bernd Verst 237d963e45
Merge branch 'master' into feat/rabbitmq-tls 2022-11-18 13:51:42 -08:00
Armin Schlegel 103624468f
jetstream pubsub now supports token based auth (#2295)
Signed-off-by: Armin Schlegel <armin.schlegel@gmx.de>

Signed-off-by: Armin Schlegel <armin.schlegel@gmx.de>
2022-11-17 23:28:18 -08:00
Bernd Verst f5ab3d93eb
Merge branch 'master' into feat/rabbitmq-tls 2022-11-17 12:47:56 -08:00
ItalyPaleAle 1e728597b4 Misc refactorings
- Removed `state.DeleteWithOptions` and `state.SetWithOptions` which were useless at this point (likely a leftover from when there were retries)
- Some improvements in etag handling in postgres
- Other minor refactorings

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-11-17 16:29:29 +00:00
spike 6937857bf8 Change error message and comment
Signed-off-by: spike <hello@spike.wiki>
2022-11-13 15:34:01 +08:00
spike 1d9f3358e4 Add rabbitmq tls metadata
Signed-off-by: spike <hello@spike.wiki>
2022-11-13 15:17:50 +08:00
Bernd Verst 462e2faadc Upgrade linter and fix linter issues
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-11-11 13:19:57 -08:00
Taction 9870c5e33b
Add pubsub rabbitmq TTL support (#2093)
* Add pubsub rabbitmq TTL support

Signed-off-by: zhangchao <zchao9100@gmail.com>

* fix review: remove queue level ttl, both ttl is handled when publishing, to make message ttl has priority over queue level ttl

Signed-off-by: zhangchao <zchao9100@gmail.com>

* add rabbitmq ttl certificate test

Signed-off-by: zhangchao <zchao9100@gmail.com>

* fix lint

Signed-off-by: zhangchao <zchao9100@gmail.com>

* fix ttl certificate test precreate topic

Signed-off-by: zhangchao <zchao9100@gmail.com>

* tiny fix tests

Signed-off-by: zhangchao <zchao9100@gmail.com>

Signed-off-by: zhangchao <zchao9100@gmail.com>
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-11-11 11:34:33 -08:00
Bernd Verst e5a2266fa8 Address lint issues
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-11-10 12:23:04 -08:00
Bernd Verst 9a8bcc5fa5
Merge branch 'master' into explicitly-add-consumer
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-11-10 12:05:51 -08:00
jinjianfeng-chn 875abcda56
Add rocketmq configuration items. Add orderly message. (#2198)
* Add rocketmq configuration items. Add orderly message. Change the version of rocketmq-client-go to v2.1.0.

Signed-off-by: jinjianfeng-chn <feng081212@163.com>

* modtidy

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

Signed-off-by: jinjianfeng-chn <feng081212@163.com>
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com>
2022-11-10 11:54:40 -08:00
yaron2 1f2c987c86 linter
Signed-off-by: yaron2 <schneider.yaron@live.com>
2022-11-10 10:32:24 -08:00
yaron2 beaa223bc2 fix deletion of sqs queue regression
Signed-off-by: yaron2 <schneider.yaron@live.com>
2022-11-10 10:08:35 -08:00
Nick ce87730a73
pubsub jetstream: set Nats-Msg-Id for message deduplication (#2200) (#2202)
Signed-off-by: NickLarsenNZ <nick@aptiv.co.nz>

Signed-off-by: NickLarsenNZ <nick@aptiv.co.nz>
Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-11-08 19:49:34 -08:00
Bernd Verst 6df7f09927 Fix linter issues, update test config, rebase upstream
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-11-02 15:15:45 -07:00
Bernd Verst bfc482d3e4 Merge remote-tracking branch 'upstream/master' into kubemq-io/master 2022-11-02 15:14:33 -07:00
Bernd Verst d562e37a67
Merge branch 'master' into pubsub 2022-10-31 16:16:27 -07:00
Alessandro (Ale) Segala 6b0183244f
Add note that pubsub.hazelcast is deprecated (#2240)
See #1808

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-10-31 13:32:54 -07:00
Alessandro (Ale) Segala e715c169b5
Merge branch 'master' into explicitly-add-consumer 2022-10-31 13:27:27 -07:00
Tomasz Pietrek 54cb93c786 Cleanup the opts
Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2022-10-31 21:07:36 +01:00
Tomasz Pietrek 83d7d09ab8 Explicitly use AddConsumer
Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2022-10-31 20:11:17 +01:00
Tomasz Pietrek e04a44895f Cleanup debugs 2022-10-31 16:43:38 +01:00
Tomasz Pietrek efe706a41d wip on binding by name 2022-10-31 12:52:38 +01:00
Bernd Verst 0819efca5a fix typo in comment
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-10-27 20:37:35 -07:00
Bernd Verst 47a46673cd PubSub: Fall back to 'consumerID' when group not present
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-10-27 14:08:37 -07:00
Alessandro (Ale) Segala 04242ce38a
New PubSub component: Azure Service Bus Queues (#2211)
* Implementation for Azure Service Bus Queue pubsub component

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Fixed syntax errors in unused files

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Moved components to subfolders and enabled conf tests

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-10-24 14:07:23 -07:00
Lior Nabat 2e3a46e04d Update - Add new PubSub component - KubeMQ Kubernetes message broker
Signed-off-by: Lior Nabat <lior.nabat@gmail.clom>
2022-10-23 08:21:08 +03:00
lior.nabat@kubemq.io 04f4f697fd Add new PubSub component - KubeMQ Kubernetes message broker
Signed-off-by: Lior Nabat <lior.nabat@gmail.clom>
2022-10-22 22:30:16 +03:00
Lior Nabat bed3d69a6f Add new PubSub component - KubeMQ Kubernetes message broker 2022-10-22 18:51:11 +03:00
shivam bbae625746 Fix linter error
Signed-off-by: shivam <shivamkm07@gmail.com>
2022-10-21 15:06:26 +05:30
shivam 8533feb87f Upgrade mqtt component to upstream
Signed-off-by: shivam <shivamkm07@gmail.com>
2022-10-21 14:39:46 +05:30
ItalyPaleAle 254a2141ad Moved sender handling to common implementation too
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-10-19 21:01:15 +00:00
ItalyPaleAle 8013d96ad6 More tweaks
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-10-19 20:39:56 +00:00
ItalyPaleAle 065ef86ee5 Moved client creation to common implementation
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-10-19 20:34:10 +00:00
ItalyPaleAle 7ecfb6edd0 Moved code to add metadata to ASB messages to shared impl too
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-10-19 20:08:07 +00:00
ItalyPaleAle 8bce0d5200 Moved metadata to impl package
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-10-19 19:47:26 +00:00
ItalyPaleAle 0057a31b88 Moved errors
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-10-18 21:51:17 +00:00
Dapr Bot 961aa9da72
Merge branch 'master' into merge19intomaster 2022-10-14 13:56:59 -07:00
Bernd Verst 0f0aaf4e9b Merge remote-tracking branch 'upstream/master' into merge19intomaster 2022-10-14 12:16:57 -07:00
ItalyPaleAle 4de7a2fc9d The people demand defers
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-10-12 18:46:38 +00:00
ItalyPaleAle 047f3948c5 Fix for handling "context deadline exceeded" on ASB publishing
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-10-12 18:18:33 +00:00
Tomasz Pietrek 954164216e Fix another lint error
Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2022-10-12 11:50:49 +02:00
Tomasz Pietrek 7a3746b33a Fix lint errors
Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2022-10-12 11:44:00 +02:00
Tomasz Pietrek 84742aa1a8 Add more JetStream options
Signed-off-by: Tomasz Pietrek <tomasz@nats.io>
2022-10-12 11:23:03 +02:00
ItalyPaleAle 3bc20a1de5 Merge branch 'master' into merge-1.9
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-10-11 20:26:35 +00:00
Alessandro (Ale) Segala ef74e7f8b6
Azure Service Bus: consider receiving 0 messages as an error that requires reconnecting (#2169)
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-10-11 12:57:06 -07:00
Bernd Verst f137bc7373 rabbitmq: use ctx for publishing
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-10-10 14:37:21 -07:00
Deepanshu Agarwal f74b4d648d Update EntryID to EntryId
Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>
2022-09-29 15:49:45 +05:30
Shubham Sharma d29c0d6200
Add ctx to BulkPublish API (#2134)
Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>
2022-09-28 09:55:47 -07:00
deepanshuagarwal bcd101abad
Keys rename (#2135)
Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>
2022-09-28 09:52:59 -07:00
Mukundan Sundararajan fbde4c706b
Merge branch 'master' into feature/pubsub-batching 2022-09-27 08:15:37 +05:30
Yash Nisar ad5397ea25
Add time to CloudEvent automatically if not present (#2123)
Closes https://github.com/dapr/dapr/issues/5137

Signed-off-by: Yash Nisar <yashnisar@microsoft.com>

Signed-off-by: Yash Nisar <yashnisar@microsoft.com>
2022-09-26 14:44:57 -07:00
Mukundan Sundararajan 803c0591c2
Merge branch 'master' into feature/pubsub-batching 2022-09-25 12:35:33 +05:30
Mario Reggiori b0d267317d
Add individual connection arguments to pubsub.rabbitmq metadata (#2129)
Signed-off-by: Mario Reggiori <git@marioreggiori.com>

Signed-off-by: Mario Reggiori <git@marioreggiori.com>
Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com>
2022-09-23 23:37:09 -07:00
ItalyPaleAle 4724a38a01 MQTT: Add metadata options "producerID"
Fixes dapr/dapr#5235

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-09-23 18:46:27 +00:00
Shubham Sharma cd206609ce
Use request metadata instead of component metadata (#2119)
Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>
2022-09-23 10:49:39 -07:00
Shubham Sharma 4f406a597f
Add bulk subscribe support to Azure Service Bus (#2100)
* Initial implementation

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* 

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Add tests

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Add comment

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Refactor messages.go and add tests

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Update message tests

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Refactor servicebus.go logic

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Refactor handleAsync

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Refactor ReceiveAndBlock

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Update binding

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Review comments addressed

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Lint

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Update metadata name

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Update activeMessagesChan logic for concurrency control

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Fix tests

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Update metadata name

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Use request metadata instead of component metadata

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Review comments addressed

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Fix merge issue with metadata

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Remove unused metadata key

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Fix merge issue with metadata

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Lint!!!

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>
2022-09-23 10:49:06 -07:00
Shubham Sharma b298b65cfe
Add bulk publish support to Azure Service Bus (#2106)
* Initial implementation

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Some improvements

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Refactor and add tests

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Lint

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Refactor

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Use request metadata instead of component metadata

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Remove unused method

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Review comments addressed

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Doc comments

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>
2022-09-23 09:57:00 -07:00
deepanshuagarwal 8500da577c
Conf test for Bulk Subscribe and Kafka fixes (#2113)
* Conf test for Bulk Sub and Kafka Fixes

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Fix linting

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Incorporating comments

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>
2022-09-23 09:54:21 -07:00
Mukundan Sundararajan e1697ba115 Merge branch 'master' into feature/pubsub-batching 2022-09-23 06:47:47 +05:30
mandarj123 e7be027c28
Update mqtt.go
Signed-off-by: mandarj123 <mandar.jogalekar@capgemini.com>
2022-09-22 10:02:08 +05:30
mandarj123 7ea805a83e
Update mqtt.go
remove mandatory "-producer" to clientId

Signed-off-by: mandarj123 <mandar.jogalekar@capgemini.com>
2022-09-21 16:53:53 +05:30
Mukundan Sundararajan a45f7db8b2 add bulkpublish-conf-test
Signed-off-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com>
2022-09-21 11:46:46 +05:30
Bernd Verst e9e750bce6 fix linter issues
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-09-20 13:02:52 -07:00
Bernd Verst ea0e9770d4
Merge pull request #2089 from shubham1172/shubham1172/add-azure-eh-bulk-publish
Add bulk publish support to Azure Event Hubs
2022-09-20 11:34:51 -07:00
Deepanshu Agarwal 217505fd16 Move common costants to common utils
Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>
2022-09-16 22:22:54 +05:30
Deepanshu Agarwal d602e0ab0c Move constants to utils
Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>
2022-09-16 22:07:38 +05:30
Shubham Sharma 313aa9ece1 Add comment
Signed-off-by: Shubham Sharma <shubhash@microsoft.com>
2022-09-16 20:11:55 +05:30
Deepanshu Agarwal 81898d5ee5 Config Per Topic and using maxAwaitDuration
Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>
2022-09-16 20:04:32 +05:30
Shubham Sharma f7e9d25a5c Return correctly
Signed-off-by: Shubham Sharma <shubhash@microsoft.com>
2022-09-16 18:26:09 +05:30
Shubham Sharma c81eefcc46 Add bulk publish for event hub
Signed-off-by: Shubham Sharma <shubhash@microsoft.com>
2022-09-16 17:51:23 +05:30
Deepanshu Agarwal d31f4d99f5 Merge branch 'feature/pubsub-batching' of github.com:dapr/components-contrib into ps_batch_c_kafka 2022-09-16 11:36:23 +05:30
Mukundan Sundararajan 3f9c42c4b8
Kafka bulk publish (#2059)
* Contrib updates for Batch Subscribe

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* commit

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Commit pubsub

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Update structs

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Update Default Batch struct name

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Update Default Batch

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Update return type for BatchPublish

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Add docs

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Rename batch to bulk

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* snssqs renam batch to bulk

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Renaming Bulker to BulkMessager

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Renaming names as discussed

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Minor updates

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Remove ContentType from BulkPublishRequest

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Update error type

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Changes to bulk subscribe and error type

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Segregating Bulk Publisher and Bulk Subscriber and removing them as mandatory

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Event Type name correction

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Docs update as per comments

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* bulk publish implementation for kafka

Signed-off-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com>

* Add ctx to BulkPublish.

Signed-off-by: Artur Souza <artursouza.ms@outlook.com>

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>
Signed-off-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com>
Signed-off-by: Artur Souza <artursouza.ms@outlook.com>
Co-authored-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>
Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
2022-09-15 13:17:17 -07:00
Deepanshu Agarwal f5542c70b9 Incorporate review comments and remove index based entryID
Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>
2022-09-15 16:02:30 +05:30
Deepanshu Agarwal 70a418bfbb Merge branch 'ps_batch_c' of github.com:DeepanshuA/components-contrib into ps_batch_c_kafka 2022-09-15 15:08:36 +05:30
Deepanshu Agarwal c913c5593c Docs update as per comments
Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>
2022-09-15 14:27:54 +05:30
Bruce Huang a6ad8cb4bf
RocketMQ fix metadata properties (#1817)
* Removes the useless metadata attributes, AccessProto and ConsumerBatchSize, rename GroupName to ProducerGroup

Signed-off-by: helbing <helbingxxx@gmail.com>

* RocketMQ SendTimeout unit is seconds

Signed-off-by: helbing <helbingxxx@gmail.com>

* avoid breaking change

Signed-off-by: helbing <helbingxxx@gmail.com>

* implement ConsumerBatchSize option

Signed-off-by: helbing <helbingxxx@gmail.com>

* avoid breaking change

Signed-off-by: helbing <helbingxxx@gmail.com>

* add deprecated log

Signed-off-by: helbing <helbingxxx@gmail.com>

* backwards-compatibility sendTimeOut

Signed-off-by: helbing <helbingxxx@gmail.com>

* fix lint

Signed-off-by: helbing <helbingxxx@gmail.com>

* add logger import

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

Signed-off-by: helbing <helbingxxx@gmail.com>
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Co-authored-by: addjuarez <addiajuarez@gmail.com>
Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com>
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-09-14 14:45:47 -07:00
Deepanshu Agarwal e483dfe334 Merge branch 'ps_batch_c' of github.com:DeepanshuA/components-contrib into ps_batch_c_kafka 2022-09-14 12:20:51 +05:30
Deepanshu Agarwal 5f50fd96ae Event Type name correction
Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>
2022-09-14 11:54:30 +05:30
Deepanshu Agarwal a1b1e3b871 Segregating Bulk Publisher and Bulk Subscriber and removing them as mandatory
Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>
2022-09-14 10:55:48 +05:30
Crypt Keeper b379f80b75
middleware: changes wasm basic to use waPC (#1833)
Updates wasm basic middleware and simplifies some code inside of it,
notably by using [waPC](https://github.com/wapc/wapc-go).

This changes the entrypoint function to be named "rewrite" and
uses a pool because known memory allocators are not goroutine
safe.

Signed-off-by: Adrian Cole <adrian@tetrate.io>

Signed-off-by: Adrian Cole <adrian@tetrate.io>
Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com>
Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
2022-09-13 17:12:59 -07:00
Xingru ed483dc88b
refactored aws sqs policy inserting (#1807)
* refactored aws sqs policy inserting

Signed-off-by: Xingru <x.xingruxu@student.maastrichtuniversity.nl>

* Add tests and fixed tryInsertCondition not inserting bug

Signed-off-by: Xingru <x.xingruxu@student.maastrichtuniversity.nl>

* fixed lint in snssqs_test.go

Signed-off-by: Xingru <x.xingruxu@student.maastrichtuniversity.nl>

* fixed lint

Signed-off-by: Xingru <x.xingruxu@student.maastrichtuniversity.nl>

* fixed lint error in policy.go

Signed-off-by: Xingru <x.xingruxu@student.maastrichtuniversity.nl>

Signed-off-by: Xingru <x.xingruxu@student.maastrichtuniversity.nl>
Co-authored-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com>
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Co-authored-by: Loong Dai <long.dai@intel.com>
Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com>
Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
2022-09-13 17:11:58 -07:00
Shivam Kumar 9c9df2ff76
Update log level for successful retry (#2066)
* Update log level for successful retry

Signed-off-by: shivam <shivamkm07@gmail.com>

* Logging error after retries exhausted

Signed-off-by: shivam <shivamkm07@gmail.com>

* Fix linter error

Signed-off-by: shivam <shivamkm07@gmail.com>

Signed-off-by: shivam <shivamkm07@gmail.com>
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
2022-09-13 15:57:47 -07:00
Bernd Verst 1d0ad9c6d8 Use standardized metadata parser
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-09-12 18:50:05 -07:00
Deepanshu Agarwal 059461c6f4 Add kafka changes for bulk subscribe
Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>
2022-09-13 01:01:05 +05:30
Deepanshu Agarwal d1d3384d65 Update envelope
Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>
2022-09-12 23:14:48 +05:30
Deepanshu Agarwal ca40728f3d Changes to bulk subscribe and error type
Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>
2022-09-12 10:21:00 +05:30
Deepanshu Agarwal 83055bce1a Update error type
Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>
2022-09-08 16:53:31 +05:30
Deepanshu Agarwal 287bbb12fb Remove ContentType from BulkPublishRequest
Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>
2022-09-08 14:10:28 +05:30
Deepanshu Agarwal 335e8d9935 Minor updates
Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>
2022-09-08 14:08:59 +05:30
Deepanshu Agarwal 9512ed284d Renaming names as discussed
Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>
2022-09-07 18:01:13 +05:30
Deepanshu Agarwal 9988a62f4b Renaming Bulker to BulkMessager
Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>
2022-09-07 12:47:24 +05:30
Deepanshu Agarwal d52d5f5af1 snssqs renam batch to bulk
Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>
2022-09-07 12:05:57 +05:30
Deepanshu Agarwal 9a844a1bb0 Rename batch to bulk
Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>
2022-09-07 12:00:08 +05:30
Deepanshu Agarwal 1ec4efd309 Add docs
Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>
2022-09-05 15:53:02 +05:30
Deepanshu Agarwal 26bc37d867 Update return type for BatchPublish
Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>
2022-09-05 14:00:29 +05:30
Deepanshu Agarwal 4a2da64e94 Update Default Batch
Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>
2022-09-05 13:50:23 +05:30
Deepanshu Agarwal 2e45f528be Update Default Batch struct name
Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>
2022-09-05 13:48:18 +05:30
Deepanshu Agarwal f59f5e509b Update structs
Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>
2022-09-05 13:45:41 +05:30
deepanshuagarwal 36c0cd73c6
Merge branch 'master' into ps_batch_c 2022-08-31 21:57:06 +05:30
Yaron Schneider edd2c9d5a9
Merge branch 'master' into redis-pubsub-fix 2022-08-29 07:00:16 -07:00
Fabian Martinez ce2570cfa2 bugfix: stuck pending messages
Signed-off-by: Fabian Martinez <46371672+famarting@users.noreply.github.com>
2022-08-29 15:25:49 +02:00
Deepanshu Agarwal ae78b77989 Commit pubsub
Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>
2022-08-27 17:20:18 +05:30
Deepanshu Agarwal 3c7eed33ae commit
Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>
2022-08-27 16:56:05 +05:30
Deepanshu Agarwal 23fb488a9c Contrib updates for Batch Subscribe
Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>
2022-08-27 16:49:54 +05:30
Alessandro (Ale) Segala bcea284c7b
Components' init functions should return interfaces (#1997)
* Components' init functions should return interfaces

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* 💄

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* pin latest berndverst contrib fork

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Fixed component registration in tests

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Updated all cert tests

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-08-25 18:27:45 -07:00
Marcos Candeia 8964d8bf41
Define common metadata across components (#1994)
* Add base metadata for all components

Signed-off-by: Marcos Candeia <marrcooos@gmail.com>

* Fix lint issues on consul and mysql tests

Signed-off-by: Marcos Candeia <marrcooos@gmail.com>

* Pin candeia/dapr runtime version

Signed-off-by: Marcos Candeia <marrcooos@gmail.com>

* Add replace directive to use mcandeia/dapr on all go submodules

Signed-off-by: Marcos Candeia <marrcooos@gmail.com>

* Fixed runtime config instantiation

Signed-off-by: Marcos Candeia <marrcooos@gmail.com>

Signed-off-by: Marcos Candeia <marrcooos@gmail.com>
2022-08-25 10:10:53 -07:00
Bernd Verst e87cd5e4cb
Go 1.19 support and linter fixes (#1975)
* Go 1.19 support and linter fixes

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Update workflows for Go1.19 and new linter version

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Remove unnecessary space in nolint directive

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* disable additional linters which aren't used because of Go generics

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* enable gosec linter again

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Update bindings/zeebe/command/publish_message_test.go

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Update bindings/output_binding.go

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Use prepared statement for mysql table creation

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Ping is not ping

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* c'mon linter

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Fix MySQL gosec issue

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* revert mysql to be fixed later

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-08-18 00:45:23 -07:00
Alessandro (Ale) Segala b3e18bb11d
pubsub.in-memory: add support for wildcard topics (#1966)
* pubsub.in-memory: add support for wildcard topics

Fixes #1964

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* make modtidy-all

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-08-12 17:13:47 -07:00
addjuarez cd740b4a7b
Handle service bus exported errors (#1942)
* Add service bus exported error handling

Signed-off-by: Addison Juarez <adjuarez@microsoft.com>

* Add service bus exported error handling

Signed-off-by: Addison Juarez <adjuarez@microsoft.com>

Signed-off-by: Addison Juarez <adjuarez@microsoft.com>
Co-authored-by: Loong Dai <long.dai@intel.com>
2022-08-12 14:47:14 -05:00
Nick 248f7f458d
feat(pubsub/jetstream): allow tls client authentication (#1924)
Signed-off-by: NickLarsenNZ <nick@aptiv.co.nz>

Co-authored-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com>
Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com>
2022-08-04 16:29:53 -07:00
Clare Liguori 6777a4d6c7
pubsub.snssqs: Support regions in other AWS partitions (#1925)
Fixes #1907

Signed-off-by: Clare Liguori <liguori@amazon.com>
2022-08-03 08:59:54 -07:00
Alessandro (Ale) Segala c399efa594
pubsub.mqtt: support multiple wildcards in subscriptions (#1901)
Fixes #1881

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com>
2022-07-21 15:30:26 -07:00
Alessandro (Ale) Segala 801297c724
Added pubsub.FeatureSubscribeWildcards capability (#1887)
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com>
2022-07-18 18:04:59 -07:00
Alessandro (Ale) Segala 66eee69188
pubsub.mqtt: support wildcard and shared subscriptions (#1882)
Fixes #1881

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-07-18 17:26:07 -07:00
Alessandro (Ale) Segala 1a4e2725ec
pubsub.redis: better ack for messages (#1886)
Also do not log errors for "context canceled" during shutdown

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-07-18 11:23:35 -07:00
Alessandro (Ale) Segala d38c786771
Add contexts to input bindings (#1831)
This is the last part of the "shutdown sequence fix": it allows shutting down input bindings before output ones.

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-07-12 12:19:27 -07:00
Alessandro (Ale) Segala a2f3a84b96
Moved authentication to be an internal pkg (#1855)
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-07-06 11:05:34 -07:00
Alessandro (Ale) Segala a193cc1ede
Add support for AAD auth in Azure Storage Queues binding (#1842)
* Add support for AAD auth in Azure Storage Queues binding

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* 🧹

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-07-03 14:17:38 -07:00
ItalyPaleAle 13b822b95f Reverting #1798 for Event Hubs
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-07-01 17:45:38 +00:00
Alessandro (Ale) Segala 4bf6cc4ca6
Updated development docs (#1835)
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com>
2022-06-30 22:23:41 -07:00
ItalyPaleAle b58464ad1c Make RabbitMQ retry more than once
Also updated tests to remove unused options and add more logging

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-06-27 22:31:31 +00:00
ItalyPaleAle 476b95990f Pulsar: do not disconnect in case of error processing message
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-06-27 21:29:54 +00:00
ItalyPaleAle a942b05b4a Added redeliveryDelay to pubsub.pulsar
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-06-27 21:12:34 +00:00
ItalyPaleAle 0c40dccd07 pubsub.mqtt: re-enqueue messages when they fail
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-06-27 20:47:16 +00:00
ItalyPaleAle be2fed6b3a MQTT: re-set "retain" to false as default
Support for "retain" is currently broken

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-06-27 20:36:15 +00:00
Bernd Verst 8974eca74c Fix MQTT metadata test
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-06-27 20:36:15 +00:00
Alessandro (Ale) Segala f55dedbbe9 More sane defaults for default values in MQTT metadata (#1810)
Fixes #1809

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-06-27 20:36:15 +00:00
ItalyPaleAle 6cf1afc264 Re-add built-in retries to pubsub.hazelcast
See: https://github.com/dapr/components-contrib/issues/1808
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-06-27 20:36:15 +00:00
ItalyPaleAle 35034aa877 Disable built-in retries in pubsub components
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-06-27 20:36:15 +00:00
Alessandro (Ale) Segala a08988c25e
Shared implementation for Azure Service Bus subscriptions for binding and pubsub (#1791)
* Moved ASB subscription code to a shared package

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Moved ASBQ binding to use the shared ASB implementation

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Updating Azure Service Bus SDK version

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-06-17 15:12:15 -07:00
Bernd Verst f25b26fc0c
Upgrade to Azure Table Storage Track 2 SDK in State Store and add CosmosDB Table API support (#1792)
* Upgrade to Azure Table Storage Track 2 SDK in State Store

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* make modtidy-all

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* don't panic

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* fix error parsing

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Adds support for empty rowKeys again

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Pin aztables v1.0.1

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Add support for Cosmos DB Table API

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Add Azure Table Storage CosmosDB conformance test

Signed-off-by: GitHub <noreply@github.com>

* Address code review

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Create Table Storage State store subtests

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Address linter issue

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* context instead of ctx

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* load state store in conf tests

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* use cosmosDB mode in cosmosDB test

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Add Azure AD support for Table Storage

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* remove shadow declaration

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* make accountKey optional

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Add options to skip table creation

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-06-17 14:04:53 -07:00
deepanshuagarwal a0ef46265b
Adding Pinger interface and Removing mandatory nature of Ping fuction (#1784)
* add `Pinger` interface.

Signed-off-by: chanyong.moon <dev.chanyongmoon@gmail.com>

* Convert Ping fuction to optional

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Removing unrequired Ping implementations

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Addressing comments

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

Co-authored-by: chanyong.moon <dev.chanyongmoon@gmail.com>
2022-06-14 08:27:54 -07:00
saber-wang d7eb3b7b27
[pubsub][rabbitmq]add support for binding multiple routingkeys at the same time (#1750)
* Add metadata property to configure BatchingMaxSize&batchingMaxMessages in Pulsar
Signed-off-by: saberwang <saberwang@hotmail.com>

* sort field
Signed-off-by: saberwang <saberwang@hotmail.com>

* [pubsub]fix unit test bug
Signed-off-by: saberwang <saberwang@hotmail.com>

* remove unrelated changes
Signed-off-by: saberwang <saberwang@hotmail.com>

* Delete hard coded Metadata
Signed-off-by: saberwang <saberwang@hotmail.com>

* remove  .history

Signed-off-by: saberwang <saberwang@hotmail.com>

* restore .gitignore

Signed-off-by: saberwang <saberwang@hotmail.com>

* Hard coding default values and adding 'BatchingMaxPublishDelay' metadata

Signed-off-by: saberwang <saberwang@hotmail.com>

* fix code format

Signed-off-by: saberwang <saberwang@hotmail.com>

* formatting code

Signed-off-by: saberwang <saberwang@hotmail.com>

* [pubsub][rabbitmq]Add support for binding multiple routingkeys at the same time

Signed-off-by: saberwang <saberwang@hotmail.com>

* add TestSubscribeBindRoutingKeys

Signed-off-by: saberwang <saberwang@hotmail.com>

* fix: Infof to Debugf

Signed-off-by: saberwang <saberwang@hotmail.com>

* Update pubsub/rabbitmq/rabbitmq_test.go

Co-authored-by: Looong Dai <long.dai@intel.com>
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
2022-06-10 16:34:15 -07:00
Alessandro (Ale) Segala 98aed5c571
Make Service Bus attempt to reconnect forever in case of issues (#1783)
* Make Service Bus attempt to reconnect forever in case of issues

With exponential back-off configurable between min and max time. Fixes #1612

Also includes fixes:

- Binding: make sure it actually retries to connect forever
- Binding: add delay (exponential backoff) before reconnecting
- PubSub: better handling of failures such as topics disabled or other non-connection issues

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* 💄

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Added warning for deprecated metadata options

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* 💄

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-06-10 15:54:46 -07:00
Alessandro (Ale) Segala 6f3234d5ac
Rabbitmq publisher confirm (#1762)
* Added `publishConfirm` option to RabbitMQ

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Changed to publisherConfirm per feedback

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-06-10 12:47:02 -07:00
Alessandro (Ale) Segala 157ddc6d6b
Bumped dapr/dapr in cert tests (#1773)
* Bumped dapr/dapr in cert tests

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Updated interface

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Added missing go.mod

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* More fixes

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Fix eventhubs not closing on context cancelation

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-06-07 08:24:15 -07:00
Alessandro (Ale) Segala 704f4dd730
Add contexts to pubsub.Subscribe to allow early cancelation (#1756)
This commit is related to dapr/dapr#4624. As noted there, we have an issue in the runtime where all components are shut down after the grace period, when the app is likely already stopped. Because of that, certain input components (the subscribe part of pubsub and the input part of bindings - the latter not in scope of this PR), can continue bringing new work when it's known to fail.

In order to fix the issue linked above properly, we need to implement a way for PubSub components to have the "publish" part closed before the "subscribe" one (and in the future that will need to be done for input bindings too).

This commit achieves precisely that by adding a context in the Subscribe method. When that context is canceled (which can be at any time), the subscription is removed.

PS: This API change was implemented so it can one day be used for dapr/dapr#814 too, as it allows canceling individual subscriptions by using a different context. Although that's not possible today because it requires more work on the runtime, it does implement everything that's needed in the pubsub components already.
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-06-02 15:06:36 -07:00
Alessandro (Ale) Segala 7c35a4e04c
Added test for multiple pubsub handlers (#1743)
* Added test for multiple pubsub handlers

This patch adds conformance tests for PubSub with multiple handlers, each listening on a different topic.

It's meant to catch issues such as #1709

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Added missing go.mod for storagequeues binding test

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-06-02 10:22:15 -07:00
Alessandro (Ale) Segala 6c7ff60c9e
Fixes for SNS/SQS PubSub (#1754)
* Fixes for SNS/SQS PubSub

Including:

- Fixed adding multiple handlers for different topics (tests in #1743 are now passing)
- Fixed delivering wrong message when concurrency mode is parallel
- Avoid having multiple pollers at the same time competing for messages

& more smaller fixes

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Fixed tests

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-06-01 15:49:51 -07:00
Alessandro (Ale) Segala ca9fbf690e
Fixed Kafka PubSub to allow multiple handlers for different topics (#1755)
* Fixed Kafka PubSub to allow multiple handlers for different topics

With this, tests from #1743 are passing

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Removed read lock (for now)

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Updated as requested by @skyao

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Use interface type everywhere

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Fixed panic

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
2022-06-01 15:31:11 -07:00
yaron2 05922117c6 add parallel test
Signed-off-by: yaron2 <schneider.yaron@live.com>
2022-05-27 09:52:39 -07:00
yaron2 64c4dc206c linter
Signed-off-by: yaron2 <schneider.yaron@live.com>
2022-05-26 11:57:57 -07:00
yaron2 ca36ecc556 add parllel processing, fix retry bug
Signed-off-by: yaron2 <schneider.yaron@live.com>
2022-05-26 11:44:56 -07:00
Alessandro (Ale) Segala 3a2a27a253
Better handling of concurrency in MQTT (#1748)
This helps avoiding issues when multiple subscribers are added in a short period of time

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-05-24 10:25:35 -07:00
Alessandro (Ale) Segala d55ce1dfdb
Fix: route messages to the correct publisher (#1737)
Fixes #1735
Includes:
- Adding a lock in `Subscribe` to prevent concurrency issues with multiple reconnections at the same time
- Increase default timeout from 3 to 30 seconds
- Avoid logging full message body in debug logs
- `connect` supports a context

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-05-24 07:49:08 -07:00
Alessandro (Ale) Segala f7834bd232
Fixed typo in metadata JSON key (#1733)
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-05-19 12:35:30 -07:00
Alessandro (Ale) Segala e1b6b01223
Tweaks for ASB pubsub (#1729)
* Tweaks for ASB pubsub:
1. Do not attempt to reconnect when context was canceled
2. Message finalization uses a background context and not the running ctx, in case the app is shutting down

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* ASB: abandoned messages shouldn't be retried right away

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Ported fixes to servicebusqueues binding too

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Use a different approach to limit the number of messages processed in case of error

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Added maxRetriableErrorsPerSec to servicebusqueues binding too
Incl other improvements

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Do not reconnect to ASB every time we retrieve a message.

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* ⚙️

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-05-17 22:33:34 -07:00
Alessandro (Ale) Segala e5e9011655
Update Azure Service Bus components to track2 SDK (#1702)
* Updated to Go 1.18

Signed-off-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com>
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>

* Updated Azure SDKs that are on track2
Includes some minor refactoring of auth code

Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>

* Updated Service Bus components to track2 SDK

Co-authored-by: halspang <halspang@microsoft.com>
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>

* Initial Certification test for eventhubs binding [incomplete] (#1670)

* certification test for eventhubs binding

Signed-off-by: tanvigour <tanvi.gour@gmail.com>

* modified go.mod and go.sum

Signed-off-by: tanvigour <tanvi.gour@gmail.com>

* Add connection string testing

Signed-off-by: tanvigour <tanvi.gour@gmail.com>

* iothub testing

Signed-off-by: tanvigour <tanvi.gour@gmail.com>

* address feedback and run test

Signed-off-by: tanvigour <tanvi.gour@gmail.com>

* Install Azure CLI IOT hub extension

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* make modtidy-all

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* covering all eventhubs test cases

Signed-off-by: tanvigour <tanvi.gour@gmail.com>

* dependency changes after go modtidy-all

Signed-off-by: tanvigour <tanvi.gour@gmail.com>

Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
Co-authored-by: Looong Dai <long.dai@intel.com>
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>

* Use revive instead of golint (#1685)

Signed-off-by: pigletfly <wangbing.adam@gmail.com>

Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>

* Updated to Go 1.18 (#1697)

* Updated to Go 1.18

Signed-off-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com>

* Added go.work file
With Go 1.18, this allows gopls (the Go language server used for example in VS Code) to work inside test apps too.
See: https://go.dev/doc/tutorial/workspaces

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Removed go.work

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* 💄

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>

* 💄 & 🧹

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>

* Add metadata property to configure Batching in Pulsar (#1707)

* Add metadata property to configure BatchingMaxSize&batchingMaxMessages in Pulsar
Signed-off-by: saberwang <saberwang@hotmail.com>

* sort field
Signed-off-by: saberwang <saberwang@hotmail.com>

* [pubsub]fix unit test bug
Signed-off-by: saberwang <saberwang@hotmail.com>

* remove unrelated changes
Signed-off-by: saberwang <saberwang@hotmail.com>

* Delete hard coded Metadata
Signed-off-by: saberwang <saberwang@hotmail.com>

* remove  .history

Signed-off-by: saberwang <saberwang@hotmail.com>

* restore .gitignore

Signed-off-by: saberwang <saberwang@hotmail.com>

* Hard coding default values and adding 'BatchingMaxPublishDelay' metadata

Signed-off-by: saberwang <saberwang@hotmail.com>

* fix code format

Signed-off-by: saberwang <saberwang@hotmail.com>

* formatting code

Signed-off-by: saberwang <saberwang@hotmail.com>

Co-authored-by: Looong Dai <long.dai@intel.com>
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>

* This test can't work with track2 SDKs
The methods to create a message with a body are not exported

Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>

* 🧹

Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>

* There's such thing as too much logging

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Refactored subscription.go
This greatly simplifies certain parts of the code, reducing the number of goroutines and likely improving performance.
Performance for end-users improves too as there's no need anymore to pause for 2 seconds every time that we reach `maxActiveMessages`.
Additionally, with this change the config options `prefetchCount` and `maxActiveMessagesRecoveryInSec` are removed as unnecessary anymore.

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* 💄

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Fixed pubsub tests

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* These packages should have never been upgraded

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Ensuring we don't fetch 1 message more than max active

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Adding configurable timeout for servicebusqueues operations

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Persistent connection for invoking SB queues

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Reverted Event Hub SDK update

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Revert "Reverted Event Hub SDK update"

This reverts commit 212220aba0.

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Fix Azure deploy for users with a first.last email

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Added some sleep that should help reduce flakiness in eventhubs binding cert test

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Changed servicebusqueue cert test
In case of a failure in the handler (ie. users' code), the message should be correctly re-enqueued, which means that messages will be re-delivered later and won't be in order.

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Co-authored-by: halspang <halspang@microsoft.com>
Co-authored-by: tanvigour <60332928+tanvigour@users.noreply.github.com>
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
Co-authored-by: Looong Dai <long.dai@intel.com>
Co-authored-by: Wang Bing <wangbing.adam@gmail.com>
Co-authored-by: saber-wang <45062099+saber-wang@users.noreply.github.com>
2022-05-13 16:52:22 -07:00
Sky Ao 8b5554dc71
Refactory kafka binding to reuse the kafka common code extracting from kafka pubsub component (#1696)
* refactory kafka pubsub code to extract common kafka code for reuse

Signed-off-by: Sky Ao <aoxiaojian@gmail.com>

* fix lint;add unit test for subscribeAdapter

Signed-off-by: Sky Ao <aoxiaojian@gmail.com>

* move topics filed from internal kafak struct to pubsub kafka struct, since in input binding the topics will confiured in metadata

Signed-off-by: Sky Ao <aoxiaojian@gmail.com>

* reuse internal  kafka code for bindings

Signed-off-by: Sky Ao <aoxiaojian@gmail.com>

* add redis standalone_test back which is delete by mistaken

Signed-off-by: Sky Ao <aoxiaojian@gmail.com>

* small code improvement to trigger test

Signed-off-by: Sky Ao <aoxiaojian@gmail.com>

* add license headers

Signed-off-by: Sky Ao <aoxiaojian@gmail.com>

* try to set disbaleTls to true to verify the kafka connection fail

Signed-off-by: Sky Ao <aoxiaojian@gmail.com>

* don't enable consum retry in kafka binding component;if authenticaion is disabled, need not set TLSDisable at the same time;

Signed-off-by: Sky Ao <aoxiaojian@gmail.com>

* fix lint

Signed-off-by: Sky Ao <aoxiaojian@gmail.com>

Co-authored-by: Loong Dai <long.dai@intel.com>
2022-05-10 20:07:23 -07:00
saber-wang a51de3bb09
Add metadata property to configure Batching in Pulsar (#1707)
* Add metadata property to configure BatchingMaxSize&batchingMaxMessages in Pulsar
Signed-off-by: saberwang <saberwang@hotmail.com>

* sort field
Signed-off-by: saberwang <saberwang@hotmail.com>

* [pubsub]fix unit test bug
Signed-off-by: saberwang <saberwang@hotmail.com>

* remove unrelated changes
Signed-off-by: saberwang <saberwang@hotmail.com>

* Delete hard coded Metadata
Signed-off-by: saberwang <saberwang@hotmail.com>

* remove  .history

Signed-off-by: saberwang <saberwang@hotmail.com>

* restore .gitignore

Signed-off-by: saberwang <saberwang@hotmail.com>

* Hard coding default values and adding 'BatchingMaxPublishDelay' metadata

Signed-off-by: saberwang <saberwang@hotmail.com>

* fix code format

Signed-off-by: saberwang <saberwang@hotmail.com>

* formatting code

Signed-off-by: saberwang <saberwang@hotmail.com>

Co-authored-by: Looong Dai <long.dai@intel.com>
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-05-06 17:59:27 -07:00
Wang Bing db99b43068
Use revive instead of golint (#1685)
Signed-off-by: pigletfly <wangbing.adam@gmail.com>

Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
2022-05-06 12:55:17 -07:00
pigletfly 5673b28b13 Add context in bindings interface
Signed-off-by: pigletfly <wangbing.adam@gmail.com>
2022-04-25 14:16:15 +08:00
Bernd Verst 1cf663c7e6
Merge branch 'master' into master 2022-04-12 14:45:55 -07:00
Amit Mor 136bccd56e
conformance test runs in single mode
Signed-off-by: Amit Mor <amit.mor@hotmail.com>
2022-04-11 11:41:52 +03:00
Amit Mor c31866099c
wg to flush bulks
Signed-off-by: Amit Mor <amit.mor@hotmail.com>
2022-04-11 11:12:17 +03:00
Amit Mor 7a83b28e7e concurrent call handler
Signed-off-by: Amit Mor <amit.mor@hotmail.com>
2022-04-11 11:07:05 +03:00
Amit Mor 13c97c9542 wip parallel calls
Signed-off-by: Amit Mor <amit.mor@hotmail.com>
2022-04-11 11:07:05 +03:00
saberwang 753be58979 Fixing pubsub-pulsar not use format topic problem
Signed-off-by: saberwang <saberwang@hotmail.com>
2022-04-11 14:16:21 +08:00
Bernd Verst 34c7eb591e
Support using AAD auth for Event Hubs Storage account (#1578)
* Support using AAD auth for Event Hubs Storage account

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Address linter issues

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Use correct Azure Environment

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

Co-authored-by: Looong Dai <long.dai@intel.com>
Co-authored-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com>
2022-03-21 15:06:30 -07:00
yellow chicks bc0615652f
fix/typo: Change traceid to traceparent (#1604)
Signed-off-by: 1046102779 <seachen@tencent.com>

Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
2022-03-21 10:21:39 -07:00
Shubham Sharma e0b63062d5
Fix serialization issue with 64 bit data type (#1597)
* Unmarshal precise

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* nit: remove empty line

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* make modtidy-all

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Fix test

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

* Add new test for large numbers

Signed-off-by: Shubham Sharma <shubhash@microsoft.com>

Co-authored-by: Looong Dai <long.dai@intel.com>
2022-03-21 10:15:08 -07:00
Yordan Pavlov 575b530bc7 Merge latest changes from upstream and resolve conflict
Signed-off-by: Yordan Pavlov <yordan.pavlov@dunnhumby.com>
2022-03-10 15:47:11 +00:00