Commit Graph

23 Commits

Author SHA1 Message Date
Elena Kolevska dae321130c
Merge 1.14 into master (#3579)
Signed-off-by: Samantha Coyle <sam@diagrid.io>
Signed-off-by: Elena Kolevska <elena@kolevska.com>
Signed-off-by: yaron2 <schneider.yaron@live.com>
Signed-off-by: Artur Souza <asouza.pro@gmail.com>
Signed-off-by: Anton Troshin <anton@diagrid.io>
Co-authored-by: Sam <sam@diagrid.io>
Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
Co-authored-by: yaron2 <schneider.yaron@live.com>
Co-authored-by: Anton Troshin <troll.sic@gmail.com>
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2024-10-24 16:52:48 -07:00
Josh van Leeuwen be7c19b742
Interfaces: Update all component interfaces to implement io.Closer (#3542)
Signed-off-by: joshvanl <me@joshvanl.dev>
2024-09-19 08:53:22 -07:00
Alessandro (Ale) Segala ec05809ee6
[Metadata] Update validator and some other fixes (#2984)
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Co-authored-by: Bernd Verst <github@bernd.dev>
2023-07-17 22:17:22 +00:00
Bernd Verst 2b89d78a2d
Refactor Metadata Parsing of all PubSub Components (#2759)
Signed-off-by: Bernd Verst <github@bernd.dev>
2023-04-09 18:21:56 +00:00
Josh van Leeuwen d098e38d6a
Propagate context from caller to appropriate places in the code (#2474)
* Propagates contexts to callers where appropriate.

Signed-off-by: joshvanl <me@joshvanl.dev>

* Updates units tests with new func signature

Signed-off-by: joshvanl <me@joshvanl.dev>

* Fix linting errors

Signed-off-by: joshvanl <me@joshvanl.dev>

* Add atomic gate to alicloud rocketmq close channel.

Signed-off-by: joshvanl <me@joshvanl.dev>

* bindings/aws/kinesis use a separate ctx variable name

Signed-off-by: joshvanl <me@joshvanl.dev>

* binding/kafka: use atomic to prevent closing the channel twice

Signed-off-by: joshvanl <me@joshvanl.dev>

* bindings/mqtt3: use atomic bool to prevent close channel being closed multiple times

Signed-off-by: joshvanl <me@joshvanl.dev>

* bindings/mqtt3: use Background context for handle operations:w

Signed-off-by: joshvanl <me@joshvanl.dev>

* state/cocroachdb: add context to Ping()

Signed-off-by: joshvanl <me@joshvanl.dev>

* bindings/postgres: add comment explaining use of context.

Signed-off-by: joshvanl <me@joshvanl.dev>

* Adds comment header to health/pinger.go

Signed-off-by: joshvanl <me@joshvanl.dev>

* pubsub/aws/snssqs: add waitgroup to wait for all go routines to finish
and block on Close(). Shuts down the subscription if there are no topic
handlers.

Signed-off-by: joshvanl <me@joshvanl.dev>

* pubsub/mqtt3: add atomic bool to prevent multiple channel closes. Add
wait group to block close on all goroutines to finish.

Signed-off-by: joshvanl <me@joshvanl.dev>

* pubsub/rabbitmq: fixes race conditions, uses atomic to prevent multiple
closes, add wait group to block close on all goroutines

Signed-off-by: joshvanl <me@joshvanl.dev>

* pubsub/redis: revert ctx passed when it could be cancelled. Add wait
group wait when closing.

Signed-off-by: joshvanl <me@joshvanl.dev>

* state/postges: pass context in init, and wait group on close.

Signed-off-by: joshvanl <me@joshvanl.dev>

* Update all `Ping()` to `PingContext()` where possible.

Signed-off-by: joshvanl <me@joshvanl.dev>

* state/in-memory: add atomic bool to prevent closing channel multiple
times. Add wait group to block on close()

Signed-off-by: joshvanl <me@joshvanl.dev>

* state/mysql: don't use same ctx variable name

Signed-off-by: joshvanl <me@joshvanl.dev>

* Pass correct loop context to redis go routines

Signed-off-by: joshvanl <me@joshvanl.dev>

* Rename context when creating timeouts in state

Signed-off-by: joshvanl <me@joshvanl.dev>

* Remove state.Features() from requiring a context

Signed-off-by: joshvanl <me@joshvanl.dev>

* Revert wasm request handle Close func to be without context to
implement io.Closer interface. Add 5 second timeout. Add io.Closer
assertion in test.

Signed-off-by: joshvanl <me@joshvanl.dev>

* Remove superfluous go lint vet directive

Signed-off-by: joshvanl <me@joshvanl.dev>

* Change Configuration Init function to take context

Signed-off-by: joshvanl <me@joshvanl.dev>

* Updates input binding interface to include a `Close() error` function. `Close`
blocks until all resources have been released and go routines have returned.

Signed-off-by: joshvanl <me@joshvanl.dev>

* Change `Close() error` in input binding struct to `io.Closer` interface.

Signed-off-by: joshvanl <me@joshvanl.dev>

* Update go.mod files to point to dapr/dapr PR https://github.com/dapr/dapr/pull/5831

Signed-off-by: joshvanl <me@joshvanl.dev>

* pubsub/redis: watch closeCh to shutdown worker instead of init context.

Signed-off-by: joshvanl <me@joshvanl.dev>

* pubsub/aws/snssqs + bindings/kubemq: ensure closeCh is caught so Close
correctly returns

Signed-off-by: joshvanl <me@joshvanl.dev>

* Close kubemq binding client on close. Ensure kafka consumer channel
cannot be closed more than once.

Signed-off-by: joshvanl <me@joshvanl.dev>

* Tweaks

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* Fixed cert tests

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* binding/mqtt3: add inline Background context instead of passing to
handleMessage

Signed-off-by: joshvanl <me@joshvanl.dev>

* pubsub/mqtt3: remove context from createSubscriberClientOptions

Signed-off-by: joshvanl <me@joshvanl.dev>

* pubsub/mqtt3: Remove `ResetConnection` func

Signed-off-by: joshvanl <me@joshvanl.dev>

* pubsub/kafka: Don't resubscribe if Subscribe is cancelled.

Signed-off-by: joshvanl <me@joshvanl.dev>

* binding/mqtt3: don't use context to control establishing connection

Signed-off-by: joshvanl <me@joshvanl.dev>

* bindings/mqtt3: Fix linting errors

Signed-off-by: joshvanl <me@joshvanl.dev>

---------

Signed-off-by: joshvanl <me@joshvanl.dev>
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Co-authored-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
2023-02-16 14:18:35 -08:00
Mukundan Sundararajan cff6d7b6b1 Merge remote-tracking branch 'upstream/master' into optimize-bulkpubres-struct 2022-12-20 23:04:19 +05:30
seachen e59dfd63c0 feature: add context to lock&pubsub API
Signed-off-by: seachen <seachen@tencent.com>
2022-12-14 19:37:24 +08:00
Mukundan Sundararajan ace0d1a03c changed to failed entries for bulk publish response
Signed-off-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com>
2022-12-13 16:22:45 +05:30
Mukundan Sundararajan 3f9c42c4b8
Kafka bulk publish (#2059)
* Contrib updates for Batch Subscribe

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* commit

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Commit pubsub

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Update structs

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Update Default Batch struct name

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Update Default Batch

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Update return type for BatchPublish

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Add docs

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Rename batch to bulk

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* snssqs renam batch to bulk

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Renaming Bulker to BulkMessager

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Renaming names as discussed

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Minor updates

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Remove ContentType from BulkPublishRequest

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Update error type

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Changes to bulk subscribe and error type

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Segregating Bulk Publisher and Bulk Subscriber and removing them as mandatory

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Event Type name correction

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Docs update as per comments

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* bulk publish implementation for kafka

Signed-off-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com>

* Add ctx to BulkPublish.

Signed-off-by: Artur Souza <artursouza.ms@outlook.com>

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>
Signed-off-by: Mukundan Sundararajan <65565396+mukundansundar@users.noreply.github.com>
Signed-off-by: Artur Souza <artursouza.ms@outlook.com>
Co-authored-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>
Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
2022-09-15 13:17:17 -07:00
Bernd Verst e87cd5e4cb
Go 1.19 support and linter fixes (#1975)
* Go 1.19 support and linter fixes

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Update workflows for Go1.19 and new linter version

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Remove unnecessary space in nolint directive

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* disable additional linters which aren't used because of Go generics

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* enable gosec linter again

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Update bindings/zeebe/command/publish_message_test.go

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Update bindings/output_binding.go

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Use prepared statement for mysql table creation

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Ping is not ping

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* c'mon linter

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* Fix MySQL gosec issue

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

* revert mysql to be fixed later

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
2022-08-18 00:45:23 -07:00
deepanshuagarwal a0ef46265b
Adding Pinger interface and Removing mandatory nature of Ping fuction (#1784)
* add `Pinger` interface.

Signed-off-by: chanyong.moon <dev.chanyongmoon@gmail.com>

* Convert Ping fuction to optional

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Removing unrequired Ping implementations

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

* Addressing comments

Signed-off-by: Deepanshu Agarwal <deepanshu.agarwal1984@gmail.com>

Co-authored-by: chanyong.moon <dev.chanyongmoon@gmail.com>
2022-06-14 08:27:54 -07:00
Alessandro (Ale) Segala 704f4dd730
Add contexts to pubsub.Subscribe to allow early cancelation (#1756)
This commit is related to dapr/dapr#4624. As noted there, we have an issue in the runtime where all components are shut down after the grace period, when the app is likely already stopped. Because of that, certain input components (the subscribe part of pubsub and the input part of bindings - the latter not in scope of this PR), can continue bringing new work when it's known to fail.

In order to fix the issue linked above properly, we need to implement a way for PubSub components to have the "publish" part closed before the "subscribe" one (and in the future that will need to be done for input bindings too).

This commit achieves precisely that by adding a context in the Subscribe method. When that context is canceled (which can be at any time), the subscription is removed.

PS: This API change was implemented so it can one day be used for dapr/dapr#814 too, as it allows canceling individual subscriptions by using a different context. Although that's not possible today because it requires more work on the runtime, it does implement everything that's needed in the pubsub components already.
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
2022-06-02 15:06:36 -07:00
Dmitry Shmulevich c7adb917f3
update license to Apache v2.0 (#1406) 2022-01-04 19:53:31 -08:00
Long Dai 988fed05ab
ci: standard linter config (#1102)
* ci: standard linter config

Signed-off-by: Long <long.dai@intel.com>

* Update utils.go

* fix pulsar issue

Signed-off-by: Long <long.dai@intel.com>

Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
2021-09-21 15:21:59 -07:00
Joni Collinge c054ae6ada Merge branch 'master' into jjcollinge/pubsub-context 2021-02-25 16:15:06 +00:00
Yaron Schneider 3ef025c604
change headers (#679) 2021-02-09 18:57:55 -08:00
Artur Souza e6dadfab6c
TTL in PubSub. (#565)
* TTL in PubSub.

* Handle message TTL only if component cannot handle it.

* Removig dead code for overflow check in pubsub expiration.
2020-12-29 14:28:52 -08:00
Joni Collinge a1ed1c1662 update pubsub and mqtt 2020-11-02 09:54:54 +00:00
Joni Collinge 2e6b84ae40 Merge remote-tracking branch 'origin/master' into jjcollinge/pubsub-context 2020-11-02 09:05:16 +00:00
Yaron Schneider 182d204111
Add close method to pub/sub (#494)
* add close method to pub/sub

* linter
2020-10-12 19:06:33 -07:00
jjcollinge 410628af3d add context to pubsub handler signature 2020-07-07 12:37:15 +01:00
Aman Bhardwaj 04bef3ac3f Adding license header and updating to MIT license. (#26) 2019-10-09 10:58:08 -07:00
yaron2 22fea6f43d initial commit 2019-09-26 15:47:41 -07:00