* Moved ASB subscription code to a shared package
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
* Moved ASBQ binding to use the shared ASB implementation
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
* Updating Azure Service Bus SDK version
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
* Make Service Bus attempt to reconnect forever in case of issues
With exponential back-off configurable between min and max time. Fixes#1612
Also includes fixes:
- Binding: make sure it actually retries to connect forever
- Binding: add delay (exponential backoff) before reconnecting
- PubSub: better handling of failures such as topics disabled or other non-connection issues
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
* 💄
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
* Added warning for deprecated metadata options
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
* 💄
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
* Tweaks for ASB pubsub:
1. Do not attempt to reconnect when context was canceled
2. Message finalization uses a background context and not the running ctx, in case the app is shutting down
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
* ASB: abandoned messages shouldn't be retried right away
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
* Ported fixes to servicebusqueues binding too
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
* Use a different approach to limit the number of messages processed in case of error
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
* Added maxRetriableErrorsPerSec to servicebusqueues binding too
Incl other improvements
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
* Do not reconnect to ASB every time we retrieve a message.
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
* ⚙️
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
* Updated to Go 1.18
Signed-off-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com>
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>
* Updated Azure SDKs that are on track2
Includes some minor refactoring of auth code
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>
* Updated Service Bus components to track2 SDK
Co-authored-by: halspang <halspang@microsoft.com>
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>
* Initial Certification test for eventhubs binding [incomplete] (#1670)
* certification test for eventhubs binding
Signed-off-by: tanvigour <tanvi.gour@gmail.com>
* modified go.mod and go.sum
Signed-off-by: tanvigour <tanvi.gour@gmail.com>
* Add connection string testing
Signed-off-by: tanvigour <tanvi.gour@gmail.com>
* iothub testing
Signed-off-by: tanvigour <tanvi.gour@gmail.com>
* address feedback and run test
Signed-off-by: tanvigour <tanvi.gour@gmail.com>
* Install Azure CLI IOT hub extension
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
* make modtidy-all
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
* covering all eventhubs test cases
Signed-off-by: tanvigour <tanvi.gour@gmail.com>
* dependency changes after go modtidy-all
Signed-off-by: tanvigour <tanvi.gour@gmail.com>
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
Co-authored-by: Looong Dai <long.dai@intel.com>
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>
* Use revive instead of golint (#1685)
Signed-off-by: pigletfly <wangbing.adam@gmail.com>
Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>
* Updated to Go 1.18 (#1697)
* Updated to Go 1.18
Signed-off-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com>
* Added go.work file
With Go 1.18, this allows gopls (the Go language server used for example in VS Code) to work inside test apps too.
See: https://go.dev/doc/tutorial/workspaces
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
* Removed go.work
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
* 💄
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>
* 💄 & 🧹
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>
* Add metadata property to configure Batching in Pulsar (#1707)
* Add metadata property to configure BatchingMaxSize&batchingMaxMessages in Pulsar
Signed-off-by: saberwang <saberwang@hotmail.com>
* sort field
Signed-off-by: saberwang <saberwang@hotmail.com>
* [pubsub]fix unit test bug
Signed-off-by: saberwang <saberwang@hotmail.com>
* remove unrelated changes
Signed-off-by: saberwang <saberwang@hotmail.com>
* Delete hard coded Metadata
Signed-off-by: saberwang <saberwang@hotmail.com>
* remove .history
Signed-off-by: saberwang <saberwang@hotmail.com>
* restore .gitignore
Signed-off-by: saberwang <saberwang@hotmail.com>
* Hard coding default values and adding 'BatchingMaxPublishDelay' metadata
Signed-off-by: saberwang <saberwang@hotmail.com>
* fix code format
Signed-off-by: saberwang <saberwang@hotmail.com>
* formatting code
Signed-off-by: saberwang <saberwang@hotmail.com>
Co-authored-by: Looong Dai <long.dai@intel.com>
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>
* This test can't work with track2 SDKs
The methods to create a message with a body are not exported
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>
* 🧹
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>
* There's such thing as too much logging
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
* Refactored subscription.go
This greatly simplifies certain parts of the code, reducing the number of goroutines and likely improving performance.
Performance for end-users improves too as there's no need anymore to pause for 2 seconds every time that we reach `maxActiveMessages`.
Additionally, with this change the config options `prefetchCount` and `maxActiveMessagesRecoveryInSec` are removed as unnecessary anymore.
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
* 💄
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
* Fixed pubsub tests
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
* These packages should have never been upgraded
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
* Ensuring we don't fetch 1 message more than max active
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
* Adding configurable timeout for servicebusqueues operations
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
* Persistent connection for invoking SB queues
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
* Reverted Event Hub SDK update
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
* Revert "Reverted Event Hub SDK update"
This reverts commit 212220aba0.
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
* Fix Azure deploy for users with a first.last email
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
* Added some sleep that should help reduce flakiness in eventhubs binding cert test
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
* Changed servicebusqueue cert test
In case of a failure in the handler (ie. users' code), the message should be correctly re-enqueued, which means that messages will be re-delivered later and won't be in order.
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Co-authored-by: halspang <halspang@microsoft.com>
Co-authored-by: tanvigour <60332928+tanvigour@users.noreply.github.com>
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
Co-authored-by: Looong Dai <long.dai@intel.com>
Co-authored-by: Wang Bing <wangbing.adam@gmail.com>
Co-authored-by: saber-wang <45062099+saber-wang@users.noreply.github.com>
* Support custom queueEndpoint in Azure Storage Queues
Signed-off-by: Janusz Dziurzynski <janusz@corechain.tech>
* run gofmt
Signed-off-by: Janusz Dziurzynski <janusz@corechain.tech>
* Add "Url" to JSON field name for clarity
Suggested by @msfussell in dapr/docs#2424
Signed-off-by: Janusz Dziurzynski <janusz@corechain.tech>
* Adds retry on CosmosDB Init in case of TooManyRequests error
* Use backoff v4
* missed some permanent errors
* clean up go.mod
* fix error type casting
* Add constant for HTTP 429
* Move Service Bus Pubsub/Binding to common auth
Both the pubsub and input/output binding for Azure Service Bus were
connecting via a connection string. This is still supported but will
now fallback to using AAD from the common auth library. This is also
the recommended auth pattern going forward.
* Move AMPQ specific auth and fix linter issues
* Make conn string and namespace mutually exclusive
* Move resourceName to a constant
* Update auth_amqp.go
* Update auth.go
Co-authored-by: Long Dai <long.dai@intel.com>
Co-authored-by: Simon Leet <31784195+CodeMonkeyLeet@users.noreply.github.com>
Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com>
* Authentication for new Azure SDK
* Updated keyvault to use new Azure SDK
* 🙈
* Update authentication/azure/auth.go
* Update authentication/azure/auth.go
* Update authentication/azure/auth.go
* Reintroduce changes from PR 1132 without updating ASB
* Lint 💄
* Marking contexts as TODO as they'll need a timeout
As per conversation with @berndverst
* Update certification tests with no auth libraries
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
We reconnect/retry forever now in Azure Service Bus Queue binding
while reading. We have to reconnect for every error because the
errors that surface from the client involve connection issues.
* cosmosdb binding - not as flexible as I'd like
conformance tests now pass
* add tests for new UUID injection logic
* Use nested property as partition key value
* avoid repeated string constants
* Disable Identification Hydrator to avoid struct requirement
* Remove struct
* Simplify PR
* Add comments ofor code maintenance
Azure Service Bus Queue names are case-insensitive. When one is
created it will always be forced to lowercase. If a queue with a
non-lowercase is provided, it will be created but can never be
matched causing the next Init to fail.
https://github.com/dapr/components-contrib/issues/1185
* Support Azure AD auth for Cosmos DB
* Fixed linting errors
* Tidying go.sum
* Removed the need for nolint:shadow
Co-authored-by: Simon Leet <31784195+CodeMonkeyLeet@users.noreply.github.com>
Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com>
- Update azure-event-hubs-go dependency to currently supported v3 with
IoT Hub SystemProperties.
- Write Event.SystemProperties to ReadResponse.Metadata for EventHubs
input binding.
- Write Event.SystemProperties to NewMessage.Metadata for EventHubs
pubsub.
* event grid udpates
* Initial event grid input binding
* Added Write() support to EventGrid component
* Updated eventgrid binding to use fasthttp instead of net/http
* Fixed linting issue in EventGrid binding
* Tidy up go.mod
* Added handshakePort to allow custom port for EG callback
* Now properly throwing an http 500 on handler err
* Updated event grid sdk for cloud event 1.0 support
Added support for all event grid sources via scope
Now validating required metadata on read/write
* Updated tests
* Fixed linting issue
Co-authored-by: Yaron Schneider <yaronsc@microsoft.com>
Co-authored-by: Young Bu Park <youngp@microsoft.com>
Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
* event grid udpates
* Initial event grid input binding
* Added Write() support to EventGrid component
* Updated eventgrid binding to use fasthttp instead of net/http
* Fixed linting issue in EventGrid binding
* Tidy up go.mod
* Added handshakePort to allow custom port for EG callback
* Now properly throwing an http 500 on handler err
Co-authored-by: Yaron Schneider <yaronsc@microsoft.com>
Co-authored-by: Young Bu Park <youngp@microsoft.com>
Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
* Add message ttl for Azure Service Bus/Storage Queue
and RabbitMQ
* Rename metadata key to ttlInSeconds
Move defaultQueueTTL to RabbitMQ metadata
* Move integration tests to own files
* Add +build integration_test tag
* Remove integration test skip
Co-authored-by: Aman Bhardwaj <amanbha@users.noreply.github.com>
* folders and files
* boilerplate added
* write queue done
parse metadata done
* refactor of init and write
* All done
* lint updates
* Additional lint fixes
* lint simplifications
* UPdated to make testable
Make read blocking
* working tests
* linting fix
* Added backoff so we dont eat CPU
* go mod tidy to fix error
* fixed test failures
* Fixed lint error
* Add Azure SignalR binding
* Fixes from go lint
* Go lint
* Use "github.com/dgrijalva/jwt-go" for jwt
Fix bug parsing connection string
Normalize usage of len vs == in strings
* Consistent usage of http header Set
Changing the names of the packages directory so to be consistent with name of the company .
It is to be noted that package name can not have hyphen "-" in the name.
So changing the directory names and putting under company names.
For example - dynamodb package would be
dapr/components-contrib/bindings/aws/dynamodb
Alessandro (Ale) Segala
Bernd Verst
Janusz Dziurzynski
Wang Bing
Dmitry Shmulevich
Simon Leet
halspang
Hossam Kandil
Long Dai
Hal Spang
Simon Leet
Javier Vela
Phil Kedy
Christian Kaps
Long Dai
Fabian Steinbach
Yaron Schneider
Artur Souza
Mohammad Taghavi
Nils Schmidt
Ben Coleman
Lynn Orrell
Sivamuthu Kumar
Francisco Beltrao
Young Bu Park
Ricardo Niepel
Carlos Landeras
Abhishek Gupta
Marcin Kosieradzki
thinkerou
stather
Shalabh Mohan Shrivastava