Commit Graph

157 Commits

Author SHA1 Message Date
Bernd Verst f25b26fc0c
Upgrade to Azure Table Storage Track 2 SDK in State Store and add CosmosDB Table API support (#1792)
* Upgrade to Azure Table Storage Track 2 SDK in State Store

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* make modtidy-all

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* don't panic

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* fix error parsing

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Adds support for empty rowKeys again

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Pin aztables v1.0.1

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Add support for Cosmos DB Table API

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Add Azure Table Storage CosmosDB conformance test

Signed-off-by: GitHub <noreply@github.com>

* Address code review

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Create Table Storage State store subtests

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Address linter issue

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* context instead of ctx

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* load state store in conf tests

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* use cosmosDB mode in cosmosDB test

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Add Azure AD support for Table Storage

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* remove shadow declaration

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* make accountKey optional

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Add options to skip table creation

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-06-17 14:04:53 -07:00
Alessandro (Ale) Segala 98aed5c571
Make Service Bus attempt to reconnect forever in case of issues (#1783)
* Make Service Bus attempt to reconnect forever in case of issues

With exponential back-off configurable between min and max time. Fixes #1612

Also includes fixes:

- Binding: make sure it actually retries to connect forever
- Binding: add delay (exponential backoff) before reconnecting
- PubSub: better handling of failures such as topics disabled or other non-connection issues

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* 💄

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Added warning for deprecated metadata options

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* 💄

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-06-10 15:54:46 -07:00
Alessandro (Ale) Segala 157ddc6d6b
Bumped dapr/dapr in cert tests (#1773)
* Bumped dapr/dapr in cert tests

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Updated interface

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Added missing go.mod

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* More fixes

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Fix eventhubs not closing on context cancelation

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-06-07 08:24:15 -07:00
Alessandro (Ale) Segala 704f4dd730
Add contexts to pubsub.Subscribe to allow early cancelation (#1756)
This commit is related to dapr/dapr#4624. As noted there, we have an issue in the runtime where all components are shut down after the grace period, when the app is likely already stopped. Because of that, certain input components (the subscribe part of pubsub and the input part of bindings - the latter not in scope of this PR), can continue bringing new work when it's known to fail.

In order to fix the issue linked above properly, we need to implement a way for PubSub components to have the "publish" part closed before the "subscribe" one (and in the future that will need to be done for input bindings too).

This commit achieves precisely that by adding a context in the Subscribe method. When that context is canceled (which can be at any time), the subscription is removed.

PS: This API change was implemented so it can one day be used for dapr/dapr#814 too, as it allows canceling individual subscriptions by using a different context. Although that's not possible today because it requires more work on the runtime, it does implement everything that's needed in the pubsub components already.
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-06-02 15:06:36 -07:00
Alessandro (Ale) Segala f7834bd232
Fixed typo in metadata JSON key (#1733)
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-05-19 12:35:30 -07:00
Alessandro (Ale) Segala e1b6b01223
Tweaks for ASB pubsub (#1729)
* Tweaks for ASB pubsub:
1. Do not attempt to reconnect when context was canceled
2. Message finalization uses a background context and not the running ctx, in case the app is shutting down

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* ASB: abandoned messages shouldn't be retried right away

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Ported fixes to servicebusqueues binding too

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Use a different approach to limit the number of messages processed in case of error

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Added maxRetriableErrorsPerSec to servicebusqueues binding too
Incl other improvements

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Do not reconnect to ASB every time we retrieve a message.

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* ⚙️

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-05-17 22:33:34 -07:00
Alessandro (Ale) Segala e5e9011655
Update Azure Service Bus components to track2 SDK (#1702)
* Updated to Go 1.18

Signed-off-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com>
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>

* Updated Azure SDKs that are on track2
Includes some minor refactoring of auth code

Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>

* Updated Service Bus components to track2 SDK

Co-authored-by: halspang <halspang@microsoft.com>
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>

* Initial Certification test for eventhubs binding [incomplete] (#1670)

* certification test for eventhubs binding

Signed-off-by: tanvigour <tanvi.gour@gmail.com>

* modified go.mod and go.sum

Signed-off-by: tanvigour <tanvi.gour@gmail.com>

* Add connection string testing

Signed-off-by: tanvigour <tanvi.gour@gmail.com>

* iothub testing

Signed-off-by: tanvigour <tanvi.gour@gmail.com>

* address feedback and run test

Signed-off-by: tanvigour <tanvi.gour@gmail.com>

* Install Azure CLI IOT hub extension

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* make modtidy-all

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* covering all eventhubs test cases

Signed-off-by: tanvigour <tanvi.gour@gmail.com>

* dependency changes after go modtidy-all

Signed-off-by: tanvigour <tanvi.gour@gmail.com>

Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
Co-authored-by: Looong Dai <long.dai@intel.com>
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>

* Use revive instead of golint (#1685)

Signed-off-by: pigletfly <wangbing.adam@gmail.com>

Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>

* Updated to Go 1.18 (#1697)

* Updated to Go 1.18

Signed-off-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com>

* Added go.work file
With Go 1.18, this allows gopls (the Go language server used for example in VS Code) to work inside test apps too.
See: https://go.dev/doc/tutorial/workspaces

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Removed go.work

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* 💄

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>

* 💄 & 🧹

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>

* Add metadata property to configure Batching in Pulsar (#1707)

* Add metadata property to configure BatchingMaxSize&batchingMaxMessages in Pulsar
Signed-off-by: saberwang <saberwang@hotmail.com>

* sort field
Signed-off-by: saberwang <saberwang@hotmail.com>

* [pubsub]fix unit test bug
Signed-off-by: saberwang <saberwang@hotmail.com>

* remove unrelated changes
Signed-off-by: saberwang <saberwang@hotmail.com>

* Delete hard coded Metadata
Signed-off-by: saberwang <saberwang@hotmail.com>

* remove  .history

Signed-off-by: saberwang <saberwang@hotmail.com>

* restore .gitignore

Signed-off-by: saberwang <saberwang@hotmail.com>

* Hard coding default values and adding 'BatchingMaxPublishDelay' metadata

Signed-off-by: saberwang <saberwang@hotmail.com>

* fix code format

Signed-off-by: saberwang <saberwang@hotmail.com>

* formatting code

Signed-off-by: saberwang <saberwang@hotmail.com>

Co-authored-by: Looong Dai <long.dai@intel.com>
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>

* This test can't work with track2 SDKs
The methods to create a message with a body are not exported

Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>

* 🧹

Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>

* There's such thing as too much logging

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Refactored subscription.go
This greatly simplifies certain parts of the code, reducing the number of goroutines and likely improving performance.
Performance for end-users improves too as there's no need anymore to pause for 2 seconds every time that we reach `maxActiveMessages`.
Additionally, with this change the config options `prefetchCount` and `maxActiveMessagesRecoveryInSec` are removed as unnecessary anymore.

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* 💄

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Fixed pubsub tests

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* These packages should have never been upgraded

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Ensuring we don't fetch 1 message more than max active

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Adding configurable timeout for servicebusqueues operations

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Persistent connection for invoking SB queues

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Reverted Event Hub SDK update

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Revert "Reverted Event Hub SDK update"

This reverts commit 212220aba0.

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Fix Azure deploy for users with a first.last email

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Added some sleep that should help reduce flakiness in eventhubs binding cert test

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Changed servicebusqueue cert test
In case of a failure in the handler (ie. users' code), the message should be correctly re-enqueued, which means that messages will be re-delivered later and won't be in order.

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Co-authored-by: halspang <halspang@microsoft.com>
Co-authored-by: tanvigour <60332928+tanvigour@users.noreply.github.com>
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
Co-authored-by: Looong Dai <long.dai@intel.com>
Co-authored-by: Wang Bing <wangbing.adam@gmail.com>
Co-authored-by: saber-wang <45062099+saber-wang@users.noreply.github.com>
2022-05-13 16:52:22 -07:00
Wang Bing db99b43068
Use revive instead of golint (#1685)
Signed-off-by: pigletfly <wangbing.adam@gmail.com>

Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
2022-05-06 12:55:17 -07:00
Bernd Verst 34c7eb591e
Support using AAD auth for Event Hubs Storage account (#1578)
* Support using AAD auth for Event Hubs Storage account

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Address linter issues

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Use correct Azure Environment

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

Co-authored-by: Looong Dai <long.dai@intel.com>
Co-authored-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com>
2022-03-21 15:06:30 -07:00
Bernd Verst 800d22f4b0 Move errors to pseudo constant
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-02-24 15:44:09 -08:00
Bernd Verst e59a67a1c4 Add retriable error conditions for ASB sending
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-02-24 15:33:49 -08:00
Dmitry Shmulevich c7adb917f3
update license to Apache v2.0 (#1406) 2022-01-04 19:53:31 -08:00
Mukundan Sundararajan fe6c9e37d9
rewrite eventhubs component (#1292)
Signed-off-by: Mukundan Sundararajan <msundar.ms@outlook.com>
2022-01-03 19:36:35 -08:00
halspang d5a68041c9
Move Service Bus Pubsub/Binding to common auth (#1201)
* Move Service Bus Pubsub/Binding to common auth

Both the pubsub and input/output binding for Azure Service Bus were
connecting via a connection string. This is still supported but will
now fallback to using AAD from the common auth library. This is also
the recommended auth pattern going forward.

* Move AMPQ specific auth and fix linter issues

* Make conn string and namespace mutually exclusive

* Move resourceName to a constant

* Update auth_amqp.go

* Update auth.go

Co-authored-by: Long Dai <long.dai@intel.com>
Co-authored-by: Simon Leet <31784195+CodeMonkeyLeet@users.noreply.github.com>
Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com>
2021-11-22 18:20:11 -08:00
Simon Leet 2006bcccb0 Support message-id in Event Hubs components 2021-10-25 18:12:16 +00:00
Joni Collinge 893da2a3ef
Enable ASB Message Metadata (#1071)
* add asb message metadata

* add asb message metadata

* fix lint

* add server metadata to receiver

* add metadata prefix

* fix tests

* alway set delivery count

* lint

* nolint on nested if

* lint fix

* remove trailing period on license header

Co-authored-by: Long Dai <long0dai@foxmail.com>
Co-authored-by: Simon Leet <31784195+CodeMonkeyLeet@users.noreply.github.com>
Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
2021-10-07 14:05:05 -07:00
Simon Leet a9cf218c04
Support setting up Azure IoT Hub for integration testing with EventHubs components (#1174)
* Add IoT Hub config to setup-azure-conf-test.sh

* Add EventHubs bindings integration test for IoT device event properties

* Add consumer group for IoT Hub events for pubsub EventHubs testing

* Add EventHubs pubsub integration test for IoT device event properties
2021-10-01 16:17:53 -07:00
Bernd Verst ac85ba2a8e
Add dapr user agent for Azure Components (#1152)
* Add dapr useragent for Service Bus and EventHubs

* Add useragent for tablestorage

* send dapr useragent

* more user agents

* formatting
2021-09-27 17:04:27 -07:00
Long Dai 988fed05ab
ci: standard linter config (#1102)
* ci: standard linter config

Signed-off-by: Long <long.dai@intel.com>

* Update utils.go

* fix pulsar issue

Signed-off-by: Long <long.dai@intel.com>

Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
2021-09-21 15:21:59 -07:00
Phil Kedy b73d817582
Merge changes from Release 1.3 to master (#1038)
* Fixing the handling of detach errors (#1030)

* Improve error message in case of missing property (#1012)

Co-authored-by: Artur Souza <artursouza.ms@outlook.com>

* Remove vestigial pubsub/nats code (#1024)

The pubsub/nats component was replaced by pubsub/natsstreaming as part
of https://github.com/dapr/dapr/pull/2003, but the corresponding code
in dapr/components-contrib was not removed, so this change removes it.

* Fixing the handling of detach errors

Co-authored-by: Maarten Mulders <mthmulders@users.noreply.github.com>
Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
Co-authored-by: Simon Leet <31784195+CodeMonkeyLeet@users.noreply.github.com>

* Revert "Fixing the handling of detach errors (#1030)" (#1031)

This reverts commit 54840c2b83.

* Fixing the handling of detach errors (#1032)

Co-authored-by: Maarten Mulders <mthmulders@users.noreply.github.com>
Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
Co-authored-by: Simon Leet <31784195+CodeMonkeyLeet@users.noreply.github.com>
2021-08-06 10:46:31 -07:00
Simon Leet dc7ee55e2a
Add Event Hubs pubsub and bindings conformance tests (#1040)
* Add conformance test for EventHubs bindings

* Update EventHubs pubsub to pass conformance tests

- Add conformance tests for Azure EventHubs PubSub component
- Add retry & backoff handling on subscriber handling error to EventHubs
  PubSub component for conformance tests.
- Add cancellation context to Azure EventHubs and update Close() to
  invoke cancel prior to closing the hub, which cleans up both senders
  and receivers.
2021-08-02 18:03:24 -07:00
Simon Leet 3877f5c793
Return SystemProperties on events from EventHubs bindings and pubsub (#1009)
- Update azure-event-hubs-go dependency to currently supported v3 with
  IoT Hub SystemProperties.
- Write Event.SystemProperties to ReadResponse.Metadata for EventHubs
  input binding.
- Write Event.SystemProperties to NewMessage.Metadata for EventHubs
  pubsub.
2021-07-13 16:04:22 -07:00
Long Dai fe5e14b78b
Azure Service Bus: ignore amqp🔗detach-forced error (#985)
Signed-off-by: Long Dai <long0dai@foxmail.com>
2021-06-30 17:12:51 -07:00
Phil Kedy 8e6b36ecdf
Importing the retry and config packages from kit instead of internal (#940)
Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
2021-06-22 09:38:55 -07:00
Phil Kedy 1b30e75e60
Configurable retry (#854)
* no message

* Switching over usages of retry to the new package and making use of the DecodeConfig function

* Added decoding configuration using settings with a specific prefix

* Linter fixes

* Fixing linter error

* time.Sleep is the enemy

* Fix typo in comment

* Moving config to a pointer parameter so that the component can pass in the config with default values that make sense for that component.

* Renamed config struct

* Fix comment

Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
2021-05-06 09:43:57 -07:00
Joni Collinge 744e5be6fa
downgrade the verbosity of lock renewal errors from warn to debug. This error is expected when messages are processed before beingrenewed (#803)
Co-authored-by: Phil Kedy <phil.kedy@gmail.com>
Co-authored-by: Yaron Schneider <yaronsc@microsoft.com>
2021-04-21 11:01:19 -07:00
Phil Kedy edfa3e5473
Azure Service Bus: Changed log level of error on close to info (#830) 2021-04-19 17:08:27 -07:00
Phil Kedy 688df9bda4
Removed dependency on dapr/dapr in favor of dapr/kit. Fixed go mod dependencies afterwards. Go 1.16 in go.mod. Removed accidental log dependency. (#807) 2021-04-19 12:06:10 -07:00
Phil Kedy 58b43fbfdb
Azure Service Bus publishing retries when server is "too busy" (#821)
* Adding retry/backoff to publishing to Azure Service Bus when the server is too busy and throttles the connection

* Fixing linter issues
2021-04-14 11:56:55 -07:00
Joni Collinge 9fd461b255 Merge branch 'master' into jjcollinge/pubsub-context 2021-03-23 17:45:05 +00:00
Joni Collinge ca62139f47
Make connection recovery configurable using metadata (#713)
* add metadata properties to control max reconn attempts and recovery delay

* add metadata tests

Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
Co-authored-by: Yaron Schneider <yaronsc@microsoft.com>
2021-03-12 12:41:38 -08:00
Joni Collinge c054ae6ada Merge branch 'master' into jjcollinge/pubsub-context 2021-02-25 16:15:06 +00:00
Yaron Schneider 3ef025c604
change headers (#679) 2021-02-09 18:57:55 -08:00
Yaron Schneider 4ca679172f
Fix service bus new topic creation (#636)
* fix service bus new topic creation

* place topic in list
2021-01-26 13:49:51 -08:00
Artur Souza e6dadfab6c
TTL in PubSub. (#565)
* TTL in PubSub.

* Handle message TTL only if component cannot handle it.

* Removig dead code for overflow check in pubsub expiration.
2020-12-29 14:28:52 -08:00
Joni Collinge 31828b3779
Update ASB message finalization (#534)
* remove servicebus context hack

* add more verbose error message

* make error message lowercase

* update go-autorest

* update azure-sdk-for-go

* remove go mod replace go-autorest

* fix lint issues

* go mod tidy

Co-authored-by: Yaron Schneider <yaronsc@microsoft.com>
2020-12-04 22:13:00 -08:00
Joni Collinge d62e2a9e84 use context.Background() in eventhubs subscriber 2020-11-03 08:29:00 +00:00
Joni Collinge 2e6b84ae40 Merge remote-tracking branch 'origin/master' into jjcollinge/pubsub-context 2020-11-02 09:05:16 +00:00
Yaron Schneider df5be19205
add close method to event hubs pub/sub (#500) 2020-10-15 08:42:14 -07:00
Yaron Schneider 182d204111
Add close method to pub/sub (#494)
* add close method to pub/sub

* linter
2020-10-12 19:06:33 -07:00
halspang f647d9eedb
Upgrade golang lint to 1.31 (#490)
Upgrade golang lint to 1.31.0

https://github.com/dapr/components-contrib/issues/439
2020-10-12 12:54:15 -07:00
jjcollinge 19442e67c8 update pubsub components to use new context 2020-07-28 10:03:23 +01:00
jjcollinge 07b046f140 rename asb handler type to handle 2020-07-07 13:01:24 +01:00
jjcollinge 7734d05288 Merge branch 'master' into jjcollinge/pubsub-context 2020-07-07 12:41:58 +01:00
jjcollinge 410628af3d add context to pubsub handler signature 2020-07-07 12:37:15 +01:00
Joni Collinge 4b2252b017
ASB lock renewal per topic (#365)
* bump asb sdk version to 0.10.2

* add isolation per topic for lock renewal

* added cancellations on async handlers

* refactored out subscription logic from namespace

* updated error handling

* defer async context cancel

* fixed lint issues

* add connection recovery

* fix lint

* update comments

* update comments

Co-authored-by: Yaron Schneider <yaronsc@microsoft.com>
2020-07-02 08:02:42 -07:00
Joni Collinge 1461b0e4a5
Fix issue with ASB pubsub throttling (#329)
* Applied throttling to receiver loop

* added init lock renewal

* added prefect count config option

* renew message locks before processing

* moved lock acquisition per go routine

* fix test

* added max active message throttles

* rename numConcurrentHandlers to maxConcurrentHandlers

* fix typo in abandonMessage func name

* set maxActiveMessages to minimum of prefetchCount

* remove prefetch count constraint

* removed conditional activeMessage tracking

Co-authored-by: Shalabh Mohan Shrivastava <shalabhs@microsoft.com>
Co-authored-by: Yaron Schneider <yaronsc@microsoft.com>
2020-05-22 20:38:28 -07:00
Young Bu Park 0489fb17f3
Fix linter errors (#293) 2020-04-01 17:22:15 -07:00
Abhishek Gupta 807e8893cb
Azure Event Hubs pubsub component (#242)
* event hubs pubsub

* go mod tidy

* go mod

Co-authored-by: yaron2 <yaronsc@microsoft.com>
Co-authored-by: Young Bu Park <youngp@microsoft.com>
2020-03-09 22:02:04 -07:00
Young Bu Park ee97c3c56a
Use dapr logger (#238)
* Use dapr logger

* skip linter false alarm
2020-02-28 15:49:49 -08:00
Yaron Schneider d2a6fa5264
close azure service bus sender connection (#209) 2020-02-06 22:19:55 -08:00
REDDY PRASAD 5f3f850852
Add GCP Secret Manager (#199)
*  Add GCP Secret Manager

* Update tests

* Fix lint errors

* Update as per review

* Run go mod tidy

Co-authored-by: Yaron Schneider <yaronsc@microsoft.com>
2020-01-30 21:51:52 -08:00
Joni Collinge ca2c3821a2 Add concurrency to Azure ServiceBus pubsub (#179)
* added worker pool for concurrent receive

* fixed missing end quote

* added type alias

* removed shadow decl

* added cleanup logic for subscription receiver

* renamed metadata property to numConcurrentHandlers and added handler specific timeout

Co-authored-by: Yaron Schneider <yaronsc@microsoft.com>
Co-authored-by: Young Bu Park <youngp@microsoft.com>
2020-01-24 10:26:11 -08:00
Joni Collinge bcfc66c79b fixed retry logic for servicebus listener (#186) 2020-01-14 12:51:13 -08:00
Abhishek Gupta 9323c44ecf check etag for etcd update and delete ops (#175) 2020-01-09 12:33:23 -08:00
thinkerou a2f4b4c510 upgrade logrus to v1.4.2 (#150)
* fix conflict

* upgrade logrus to v1.4.2

* fix lint error

error: `depguard  `github.com/sirupsen/logrus` is in the blacklist`
2019-12-12 09:34:39 -08:00
Shalabh Mohan Shrivastava bcd23db009
Changes to include Package names pubsub/secret stores/state to be consistent with company name (#137)
* [Commit - PubSub changes ]

Changing the names of the packages directory so to be consistent with name of the company .

It is to be noted that package name can not have hyphen "-" in the name.
So changing the directory names and putting under company names.

For example - servicebus package would be

dapr/components-contrib/pubsub/azure/servicebus

* Commit 2 - Changes for Secret Stores

* Commit 3 - for the state store changes
2019-11-22 12:46:42 -08:00