Commit Graph

20 Commits

Author SHA1 Message Date
Alessandro (Ale) Segala 934e86c4f7
Rename folder "internal" to "common" (#3225)
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2023-11-14 11:36:38 -08:00
Alessandro (Ale) Segala f4e73b0e65
Update dapr/kit (#3205)
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Signed-off-by: Bernd Verst <github@bernd.dev>
Co-authored-by: Bernd Verst <github@bernd.dev>
2023-11-02 16:20:56 -07:00
Alessandro (Ale) Segala ec05809ee6
[Metadata] Update validator and some other fixes (#2984)
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Co-authored-by: Bernd Verst <github@bernd.dev>
2023-07-17 22:17:22 +00:00
Alessandro (Ale) Segala cef854fab6
Some linting in metadata.yaml files (#2977)
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2023-07-11 01:53:58 +00:00
Arturo Trenard ef4780beaa
Pubsub pluggable component bulkpublish (#2931)
Signed-off-by: arturo <jarturotrenard@gmail.com>
Signed-off-by: arturo <ajt@ptytechnologies.com>
Signed-off-by: Arturo Trenard <jarturotrenard@gmail.com>
Co-authored-by: arturo <ajt@ptytechnologies.com>
2023-06-22 15:14:18 +00:00
Bernd Verst 1a751931f7
Add Service Bus Topics Pubsub Metadata (#2776)
Signed-off-by: Bernd Verst <github@bernd.dev>
2023-04-12 21:49:49 -07:00
Bernd Verst 2b89d78a2d
Refactor Metadata Parsing of all PubSub Components (#2759)
Signed-off-by: Bernd Verst <github@bernd.dev>
2023-04-09 18:21:56 +00:00
Josh van Leeuwen 569c6be070
PubSub: Wait Group and context catching (#2500)
Signed-off-by: joshvanl <me@joshvanl.dev>
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Signed-off-by: Bernd Verst <github@bernd.dev>
Co-authored-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Co-authored-by: Bernd Verst <github@bernd.dev>
2023-03-03 17:44:26 +00:00
ItalyPaleAle f9cf54ca6f Merge branch 'master' of https://github.com/dapr/components-contrib into release-1.10
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2023-02-16 23:23:23 +00:00
Josh van Leeuwen d098e38d6a
Propagate context from caller to appropriate places in the code (#2474)
* Propagates contexts to callers where appropriate.

Signed-off-by: joshvanl <me@joshvanl.dev>

* Updates units tests with new func signature

Signed-off-by: joshvanl <me@joshvanl.dev>

* Fix linting errors

Signed-off-by: joshvanl <me@joshvanl.dev>

* Add atomic gate to alicloud rocketmq close channel.

Signed-off-by: joshvanl <me@joshvanl.dev>

* bindings/aws/kinesis use a separate ctx variable name

Signed-off-by: joshvanl <me@joshvanl.dev>

* binding/kafka: use atomic to prevent closing the channel twice

Signed-off-by: joshvanl <me@joshvanl.dev>

* bindings/mqtt3: use atomic bool to prevent close channel being closed multiple times

Signed-off-by: joshvanl <me@joshvanl.dev>

* bindings/mqtt3: use Background context for handle operations:w

Signed-off-by: joshvanl <me@joshvanl.dev>

* state/cocroachdb: add context to Ping()

Signed-off-by: joshvanl <me@joshvanl.dev>

* bindings/postgres: add comment explaining use of context.

Signed-off-by: joshvanl <me@joshvanl.dev>

* Adds comment header to health/pinger.go

Signed-off-by: joshvanl <me@joshvanl.dev>

* pubsub/aws/snssqs: add waitgroup to wait for all go routines to finish
and block on Close(). Shuts down the subscription if there are no topic
handlers.

Signed-off-by: joshvanl <me@joshvanl.dev>

* pubsub/mqtt3: add atomic bool to prevent multiple channel closes. Add
wait group to block close on all goroutines to finish.

Signed-off-by: joshvanl <me@joshvanl.dev>

* pubsub/rabbitmq: fixes race conditions, uses atomic to prevent multiple
closes, add wait group to block close on all goroutines

Signed-off-by: joshvanl <me@joshvanl.dev>

* pubsub/redis: revert ctx passed when it could be cancelled. Add wait
group wait when closing.

Signed-off-by: joshvanl <me@joshvanl.dev>

* state/postges: pass context in init, and wait group on close.

Signed-off-by: joshvanl <me@joshvanl.dev>

* Update all `Ping()` to `PingContext()` where possible.

Signed-off-by: joshvanl <me@joshvanl.dev>

* state/in-memory: add atomic bool to prevent closing channel multiple
times. Add wait group to block on close()

Signed-off-by: joshvanl <me@joshvanl.dev>

* state/mysql: don't use same ctx variable name

Signed-off-by: joshvanl <me@joshvanl.dev>

* Pass correct loop context to redis go routines

Signed-off-by: joshvanl <me@joshvanl.dev>

* Rename context when creating timeouts in state

Signed-off-by: joshvanl <me@joshvanl.dev>

* Remove state.Features() from requiring a context

Signed-off-by: joshvanl <me@joshvanl.dev>

* Revert wasm request handle Close func to be without context to
implement io.Closer interface. Add 5 second timeout. Add io.Closer
assertion in test.

Signed-off-by: joshvanl <me@joshvanl.dev>

* Remove superfluous go lint vet directive

Signed-off-by: joshvanl <me@joshvanl.dev>

* Change Configuration Init function to take context

Signed-off-by: joshvanl <me@joshvanl.dev>

* Updates input binding interface to include a `Close() error` function. `Close`
blocks until all resources have been released and go routines have returned.

Signed-off-by: joshvanl <me@joshvanl.dev>

* Change `Close() error` in input binding struct to `io.Closer` interface.

Signed-off-by: joshvanl <me@joshvanl.dev>

* Update go.mod files to point to dapr/dapr PR https://github.com/dapr/dapr/pull/5831

Signed-off-by: joshvanl <me@joshvanl.dev>

* pubsub/redis: watch closeCh to shutdown worker instead of init context.

Signed-off-by: joshvanl <me@joshvanl.dev>

* pubsub/aws/snssqs + bindings/kubemq: ensure closeCh is caught so Close
correctly returns

Signed-off-by: joshvanl <me@joshvanl.dev>

* Close kubemq binding client on close. Ensure kafka consumer channel
cannot be closed more than once.

Signed-off-by: joshvanl <me@joshvanl.dev>

* Tweaks

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Fixed cert tests

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* binding/mqtt3: add inline Background context instead of passing to
handleMessage

Signed-off-by: joshvanl <me@joshvanl.dev>

* pubsub/mqtt3: remove context from createSubscriberClientOptions

Signed-off-by: joshvanl <me@joshvanl.dev>

* pubsub/mqtt3: Remove `ResetConnection` func

Signed-off-by: joshvanl <me@joshvanl.dev>

* pubsub/kafka: Don't resubscribe if Subscribe is cancelled.

Signed-off-by: joshvanl <me@joshvanl.dev>

* binding/mqtt3: don't use context to control establishing connection

Signed-off-by: joshvanl <me@joshvanl.dev>

* bindings/mqtt3: Fix linting errors

Signed-off-by: joshvanl <me@joshvanl.dev>

---------

Signed-off-by: joshvanl <me@joshvanl.dev>
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Co-authored-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
2023-02-16 14:18:35 -08:00
Alessandro (Ale) Segala 12acf8effe
[release-1.10] Azure Service Bus: fix goroutine "explosion" after recovering from failure and many other bugs (#2523)
* Fixed context leak

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Finalize messages in background

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* More tweaks

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Moved a lot more code to the shared implementation package

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Fixed:
- Fixed: if number of messages received in bulk is greater than maxConcurrentHandlers, a deadlock happens
- Fixed: maxRetriableErrorsPerSec was not working

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Some refactorings

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Simplifications

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Avoid checking if topic/queue exists every time we send a message

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Ensuring locks are renewed for all 3 components

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Updated ASB SDK

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Improvements to closing

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Improvements to closing

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Changed context propagation

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Debug logs in cert tests

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Initial delay before renewing locks

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* That last change was actually unnecessary

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Tweaks

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* More tweaks

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* 💄

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

---------

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2023-02-13 09:23:03 -08:00
ItalyPaleAle 7047ad533d More logging when Service Bus reconnects after publishing error
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2023-01-31 17:58:30 +00:00
Deepanshu Agarwal b2d8e6013b
Add bulkSubscribe request params to SubscribeRequest (#2405)
* Add bulkSubscribe configuration params

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Replace dapr/dapr package to run cert tests

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Change bulkSubscribe param types

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Update latest dapr/dapr fork

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Update correct dapr/dapr version

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* change BulkSubscribeRequest to Config

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Update dapr/dapr version for cert

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Update with latest dapr fork

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>
2023-01-10 09:30:25 -08:00
Joni Collinge 382f84f30c
Add support for Azure Service Bus sessions for topics (#2346)
* WIP: ASB sessions

Signed-off-by: Joni Collinge <jonathancollinge@live.com>

* WIP: Support ASB sessions

Signed-off-by: Joni Collinge <jonathancollinge@live.com>

* Add metadata

Signed-off-by: Joni Collinge <jonathancollinge@live.com>

* remove dead code

Signed-off-by: Joni Collinge <jonathancollinge@live.com>

* lint

Signed-off-by: Joni Collinge <jonathancollinge@live.com>

* Add sessionIdleTimeoutInSec md

Signed-off-by: Joni Collinge <jonathancollinge@live.com>

* Support blank session IDs

Signed-off-by: Joni Collinge <jonathancollinge@live.com>

* If require sessions and no session id, set blank

Signed-off-by: Joni Collinge <jonathancollinge@live.com>

* move session config to per subscription metadata

Signed-off-by: Joni Collinge <jonathancollinge@live.com>

* add some unit tests

Signed-off-by: Joni Collinge <jonathancollinge@live.com>

* lint

Signed-off-by: Joni Collinge <jonathancollinge@live.com>

* remove auto blank session id

Signed-off-by: Joni Collinge <jonathancollinge@live.com>

* go mod tidy

Signed-off-by: Joni Collinge <jonathancollinge@live.com>

* add basic cert test

Signed-off-by: Joni Collinge <jonathancollinge@live.com>

* update comment

Signed-off-by: Joni Collinge <jonathancollinge@live.com>

* remove dead code

Signed-off-by: Joni Collinge <jonathancollinge@live.com>

* remove dead code

Signed-off-by: Joni Collinge <jonathancollinge@live.com>

* Update certification test

Signed-off-by: Joni Collinge <jonathancollinge@live.com>

* Add comment

Signed-off-by: Joni Collinge <jonathancollinge@live.com>

* Add cert test for roundrobin

Signed-off-by: Joni Collinge <jonathancollinge@live.com>

* go mod tidy

Signed-off-by: Joni Collinge <jonathancollinge@live.com>

* Update readme

Signed-off-by: Joni Collinge <jonathancollinge@live.com>

* Use struct for NewSubscription to simplify parameters

Signed-off-by: Joni Collinge <jonathancollinge@live.com>

* Address feedback

Signed-off-by: Joni Collinge <jonathancollinge@live.com>

* Address feedback

Signed-off-by: Joni Collinge <jonathancollinge@live.com>

* Only async receive

Signed-off-by: Joni Collinge <jonathancollinge@live.com>

* isTruthy on map without check

Signed-off-by: Joni Collinge <jonathancollinge@live.com>

* Use if not select for context check

Signed-off-by: Joni Collinge <jonathancollinge@live.com>

* Feedback

Signed-off-by: Joni Collinge <jonathancollinge@live.com>

* lint

Signed-off-by: Joni Collinge <jonathancollinge@live.com>

Signed-off-by: Joni Collinge <jonathancollinge@live.com>
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Co-authored-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com>
2023-01-06 09:59:26 -08:00
Mukundan Sundararajan cff6d7b6b1 Merge remote-tracking branch 'upstream/master' into optimize-bulkpubres-struct 2022-12-20 23:04:19 +05:30
ItalyPaleAle fc00023206 Small naming fixes
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-12-19 22:03:53 +00:00
seachen 1ea2cff9fd feature: add context to lock&pubsub API
Signed-off-by: seachen <seachen@tencent.com>
2022-12-15 09:53:32 +08:00
seachen e59dfd63c0 feature: add context to lock&pubsub API
Signed-off-by: seachen <seachen@tencent.com>
2022-12-14 19:37:24 +08:00
Mukundan Sundararajan 72695529f6 optimize bulk pub response to contain only failed entries
Signed-off-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com>
2022-12-13 11:49:04 +05:30
Alessandro (Ale) Segala 04242ce38a
New PubSub component: Azure Service Bus Queues (#2211)
* Implementation for Azure Service Bus Queue pubsub component

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Fixed syntax errors in unused files

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Moved components to subfolders and enabled conf tests

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-10-24 14:07:23 -07:00