This commit is related to dapr/dapr#4624. As noted there, we have an issue in the runtime where all components are shut down after the grace period, when the app is likely already stopped. Because of that, certain input components (the subscribe part of pubsub and the input part of bindings - the latter not in scope of this PR), can continue bringing new work when it's known to fail.
In order to fix the issue linked above properly, we need to implement a way for PubSub components to have the "publish" part closed before the "subscribe" one (and in the future that will need to be done for input bindings too).
This commit achieves precisely that by adding a context in the Subscribe method. When that context is canceled (which can be at any time), the subscription is removed.
PS: This API change was implemented so it can one day be used for dapr/dapr#814 too, as it allows canceling individual subscriptions by using a different context. Although that's not possible today because it requires more work on the runtime, it does implement everything that's needed in the pubsub components already.
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
* Move pubsub redis client to internal
* Add failover for common redis. Refactor state redis.
* refactor binding redis
* fix lint
* Refactor to setting partten
* fix db and lint
* fix lint
* for backward compatibility
* Using a type alias to handle decoding duration values of -1 for Redis.
* Linter fixes
Co-authored-by: Phil Kedy <phil.kedy@gmail.com>
* master:
add handle timeout to cancel req for it would be redelivered (#832)
Add optional metadata param maxLen for redis stream PubSub component (#835)
Implement bindings for the Zeebe workflow engine (#806)
Automerge in contrib. (#849)
# Conflicts:
# go.mod
# go.sum
# pubsub/redis/redis.go
Co-authored-by: Phil Kedy <phil.kedy@gmail.com>
Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com>
resolve conflict
* master:
Increasing overall test timeout to allow for conformance tests to run (#834)
Improve SMTP binding performance (#782)
add redis cluter mode for pubsub (#823)
downgrade the verbosity of lock renewal errors from warn to debug. This error is expected when messages are processed before beingrenewed (#803)
Improve pulsar publisher performance. (#827)
optimze(state/redis): output detail redis error (#825)
Consul nameresolution component (#750)
Add DNS name resolver. Add unit tests. (#743)
Azure Service Bus: Changed log level of error on close to info (#830)
Removed dependency on dapr/dapr in favor of dapr/kit. Fixed go mod dependencies afterwards. Go 1.16 in go.mod. Removed accidental log dependency. (#807)
Update state/blobstorage.go to add metadata. (#786)
Azure Service Bus publishing retries when server is "too busy" (#821)
# Conflicts:
# bindings/redis/redis.go
# go.mod
# go.sum
# pubsub/redis/redis.go
When component close function called `r.cancel()` and `r.client.Close()` will be excuted.
The pollNewMessagesLoop loop use client to read from redis which will lead to err. And when err occured this loop will continue.
We need to check context on the top of this loop.
* Improve conformance tests and add pulsar
* Fix linter error
* Fix linter errors
* Tweaks to pulsar pubsub per PR
* Switching uuid import
* Using Stringer interface to print message keys as base64 in logger
* Disabling some checks w/ Redis pubsub until the component can be enhanced
* Adding comment to eventually remove the simulateErrors option
* Add comment asBase64String to explain what it does
* Adding redelivery (reclaims) to the Redis pubsub component
Removed the temporary simulateErrors flag now that Redis is passing
* Fixing linter issues
* using a wait group instead of a sleep to wait for processing to complete
* More comments and refactoring
* comment tweaks
* Tweaks per PR
Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
Fabian Martinez
deepanshuagarwal
Alessandro (Ale) Segala
Dmitry Shmulevich
Long Dai
Taction
Charlie Stanley
Phil Kedy
Joni Collinge
Yaron Schneider
Artur Souza
halspang
Young Bu Park
thinkerou
Aman Bhardwaj