This PR adds conformance tests for Hashicorp Vault secret store component.
This will help brining this component to stable status.
This addresses (but does not fix) #1926.
Signed-off-by: Tiago Alves Macambira <tmacam@burocrata.org>
This commit is related to dapr/dapr#4624. As noted there, we have an issue in the runtime where all components are shut down after the grace period, when the app is likely already stopped. Because of that, certain input components (the subscribe part of pubsub and the input part of bindings - the latter not in scope of this PR), can continue bringing new work when it's known to fail.
In order to fix the issue linked above properly, we need to implement a way for PubSub components to have the "publish" part closed before the "subscribe" one (and in the future that will need to be done for input bindings too).
This commit achieves precisely that by adding a context in the Subscribe method. When that context is canceled (which can be at any time), the subscription is removed.
PS: This API change was implemented so it can one day be used for dapr/dapr#814 too, as it allows canceling individual subscriptions by using a different context. Although that's not possible today because it requires more work on the runtime, it does implement everything that's needed in the pubsub components already.
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
* Updated to Go 1.18
Signed-off-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com>
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>
* Updated Azure SDKs that are on track2
Includes some minor refactoring of auth code
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>
* Updated Service Bus components to track2 SDK
Co-authored-by: halspang <halspang@microsoft.com>
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>
* Initial Certification test for eventhubs binding [incomplete] (#1670)
* certification test for eventhubs binding
Signed-off-by: tanvigour <tanvi.gour@gmail.com>
* modified go.mod and go.sum
Signed-off-by: tanvigour <tanvi.gour@gmail.com>
* Add connection string testing
Signed-off-by: tanvigour <tanvi.gour@gmail.com>
* iothub testing
Signed-off-by: tanvigour <tanvi.gour@gmail.com>
* address feedback and run test
Signed-off-by: tanvigour <tanvi.gour@gmail.com>
* Install Azure CLI IOT hub extension
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
* make modtidy-all
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
* covering all eventhubs test cases
Signed-off-by: tanvigour <tanvi.gour@gmail.com>
* dependency changes after go modtidy-all
Signed-off-by: tanvigour <tanvi.gour@gmail.com>
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
Co-authored-by: Looong Dai <long.dai@intel.com>
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>
* Use revive instead of golint (#1685)
Signed-off-by: pigletfly <wangbing.adam@gmail.com>
Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>
* Updated to Go 1.18 (#1697)
* Updated to Go 1.18
Signed-off-by: Alessandro (Ale) Segala <43508+ItalyPaleAle@users.noreply.github.com>
* Added go.work file
With Go 1.18, this allows gopls (the Go language server used for example in VS Code) to work inside test apps too.
See: https://go.dev/doc/tutorial/workspaces
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
* Removed go.work
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
* 💄
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>
* 💄 & 🧹
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>
* Add metadata property to configure Batching in Pulsar (#1707)
* Add metadata property to configure BatchingMaxSize&batchingMaxMessages in Pulsar
Signed-off-by: saberwang <saberwang@hotmail.com>
* sort field
Signed-off-by: saberwang <saberwang@hotmail.com>
* [pubsub]fix unit test bug
Signed-off-by: saberwang <saberwang@hotmail.com>
* remove unrelated changes
Signed-off-by: saberwang <saberwang@hotmail.com>
* Delete hard coded Metadata
Signed-off-by: saberwang <saberwang@hotmail.com>
* remove .history
Signed-off-by: saberwang <saberwang@hotmail.com>
* restore .gitignore
Signed-off-by: saberwang <saberwang@hotmail.com>
* Hard coding default values and adding 'BatchingMaxPublishDelay' metadata
Signed-off-by: saberwang <saberwang@hotmail.com>
* fix code format
Signed-off-by: saberwang <saberwang@hotmail.com>
* formatting code
Signed-off-by: saberwang <saberwang@hotmail.com>
Co-authored-by: Looong Dai <long.dai@intel.com>
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>
* This test can't work with track2 SDKs
The methods to create a message with a body are not exported
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>
* 🧹
Signed-off-by: Alessandro Segala (ItalyPaleAle) <43508+ItalyPaleAle@users.noreply.github.com>
* There's such thing as too much logging
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
* Refactored subscription.go
This greatly simplifies certain parts of the code, reducing the number of goroutines and likely improving performance.
Performance for end-users improves too as there's no need anymore to pause for 2 seconds every time that we reach `maxActiveMessages`.
Additionally, with this change the config options `prefetchCount` and `maxActiveMessagesRecoveryInSec` are removed as unnecessary anymore.
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
* 💄
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
* Fixed pubsub tests
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
* These packages should have never been upgraded
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
* Ensuring we don't fetch 1 message more than max active
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
* Adding configurable timeout for servicebusqueues operations
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
* Persistent connection for invoking SB queues
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
* Reverted Event Hub SDK update
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
* Revert "Reverted Event Hub SDK update"
This reverts commit 212220aba0.
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
* Fix Azure deploy for users with a first.last email
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
* Added some sleep that should help reduce flakiness in eventhubs binding cert test
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
* Changed servicebusqueue cert test
In case of a failure in the handler (ie. users' code), the message should be correctly re-enqueued, which means that messages will be re-delivered later and won't be in order.
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Co-authored-by: halspang <halspang@microsoft.com>
Co-authored-by: tanvigour <60332928+tanvigour@users.noreply.github.com>
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Co-authored-by: Yaron Schneider <schneider.yaron@live.com>
Co-authored-by: Looong Dai <long.dai@intel.com>
Co-authored-by: Wang Bing <wangbing.adam@gmail.com>
Co-authored-by: saber-wang <45062099+saber-wang@users.noreply.github.com>
* First iteration for the eventhubs certifications tests
Signed-off-by: Surender Singh Malik <surenderssm@users.noreply.github.com>
* Fixing typos, adding more description and publishing with multiple partitions
Signed-off-by: Surender Singh Malik <surenderssm@users.noreply.github.com>
* Moved the secret reference in components definition to secretstore.local.env
Signed-off-by: Surender Singh Malik <surenderssm@users.noreply.github.com>
* moving to latest dapr/dapr github.com/dapr/dapr@v1.7.0-rc.2
Signed-off-by: Surender Singh Malik <surenderssm@users.noreply.github.com>
* updating go.mod
Signed-off-by: Surender Singh Malik <surenderssm@users.noreply.github.com>
* changes in infra and workflow for cert eventhubs
Signed-off-by: Surender Singh Malik <surenderssm@users.noreply.github.com>
* typo fix in local secret store type
Signed-off-by: Surender Singh Malik <surenderssm@users.noreply.github.com>
* correcting the typo in policyName
Signed-off-by: Surender Singh Malik <surenderssm@users.noreply.github.com>
* bug fixes
Signed-off-by: Surender Singh Malik <surenderssm@users.noreply.github.com>
* publish to a topic with no subscriber fix
Signed-off-by: Surender Singh Malik <surenderssm@users.noreply.github.com>
* delete azure eventhub that got created as part of test run
Signed-off-by: Surender Singh Malik <surenderssm@users.noreply.github.com>
* updated readme
Signed-off-by: Surender Singh Malik <surenderssm@users.noreply.github.com>
* removed dummy sidecar and leveraged connectionstring at namespace level for multi topic operations
Signed-off-by: Surender Singh Malik <surenderssm@users.noreply.github.com>
* go mod changes and nit fixes
Signed-off-by: Surender Singh Malik <surenderssm@users.noreply.github.com>
* fixing iot device issue
Signed-off-by: Surender Singh Malik <surenderssm@users.noreply.github.com>
* NIT : CR fixes
* Enabled iothub test and a bit of code refactoring
Signed-off-by: Surender Singh Malik <surenderssm@users.noreply.github.com>
* delete the checkpoint container for the temp eventhub
Signed-off-by: Surender Singh Malik <surenderssm@users.noreply.github.com>
* Revert "NIT : CR fixes"
This reverts commit 260f5ed686.
Signed-off-by: Surender Singh Malik <surenderssm@users.noreply.github.com>
* First iteration for the eventhubs certifications tests
Signed-off-by: Surender Singh Malik <surenderssm@users.noreply.github.com>
* Fixing typos, adding more description and publishing with multiple partitions
Signed-off-by: Surender Singh Malik <surenderssm@users.noreply.github.com>
* Moved the secret reference in components definition to secretstore.local.env
Signed-off-by: Surender Singh Malik <surenderssm@users.noreply.github.com>
* moving to latest dapr/dapr github.com/dapr/dapr@v1.7.0-rc.2
Signed-off-by: Surender Singh Malik <surenderssm@users.noreply.github.com>
* updating go.mod
Signed-off-by: Surender Singh Malik <surenderssm@users.noreply.github.com>
* changes in infra and workflow for cert eventhubs
Signed-off-by: Surender Singh Malik <surenderssm@users.noreply.github.com>
* typo fix in local secret store type
Signed-off-by: Surender Singh Malik <surenderssm@users.noreply.github.com>
* correcting the typo in policyName
Signed-off-by: Surender Singh Malik <surenderssm@users.noreply.github.com>
* bug fixes
Signed-off-by: Surender Singh Malik <surenderssm@users.noreply.github.com>
* publish to a topic with no subscriber fix
Signed-off-by: Surender Singh Malik <surenderssm@users.noreply.github.com>
* delete azure eventhub that got created as part of test run
Signed-off-by: Surender Singh Malik <surenderssm@users.noreply.github.com>
* updated readme
Signed-off-by: Surender Singh Malik <surenderssm@users.noreply.github.com>
* removed dummy sidecar and leveraged connectionstring at namespace level for multi topic operations
Signed-off-by: Surender Singh Malik <surenderssm@users.noreply.github.com>
* go mod changes and nit fixes
Signed-off-by: Surender Singh Malik <surenderssm@users.noreply.github.com>
* fixing iot device issue
Signed-off-by: Surender Singh Malik <surenderssm@users.noreply.github.com>
* NIT : CR fixes
* Enabled iothub test and a bit of code refactoring
Signed-off-by: Surender Singh Malik <surenderssm@users.noreply.github.com>
* delete the checkpoint container for the temp eventhub
Signed-off-by: Surender Singh Malik <surenderssm@users.noreply.github.com>
* Revert "NIT : CR fixes"
This reverts commit 260f5ed686.
Signed-off-by: Surender Singh Malik <surenderssm@users.noreply.github.com>
* Fix eventhub pubsub certification test infra provisioning
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
* Fixing the scoping issue for role assignment on eh
Signed-off-by: Surender Singh Malik <surenderssm@users.noreply.github.com>
Co-authored-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
* conformance tests
Signed-off-by: Amit Mor <amit.mor@hotmail.com>
* latest localstack. snssqs to conformance list
Signed-off-by: Amit Mor <amit.mor@hotmail.com>
Co-authored-by: Mukundan Sundararajan <musundar@microsoft.com>
Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com>
* - Introduce `fmt.Sprintf` where applicable
- Add conformance test for cassandra component.
Signed-off-by: mbimbij <joseph.mbimbi@gmail.com>
* Start cassandra via docker-compose for its conformance test
Signed-off-by: mbimbij <joseph.mbimbi@gmail.com>
* Apply `go fmt` on `tests/conformance/common.go` as part of PR correction
Signed-off-by: mbimbij <joseph.mbimbi@gmail.com>
* Fix test infrastructure setup script
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
* use more unique ACR resource name
Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com>
Co-authored-by: Looong Dai <long.dai@intel.com>
* Add script to update Azure SQL firewall rules from GitHub meta API
* Update state.azure.sql test to use unique DB names
- Add use of `databaseName` metadata to state.azure.sql test.
- Add dynamic generation of test `databaseName` to conformance.yml to
avoid multiple test instances from racing.
- Add wait before clean-up of Azure SQL DB instance in conformance.yml
to prevent test flakiness and accumulation of undeleted DBs.
- Remove dynamic Azure SQL firewall rule injection from conformance.yml.
- The workflow relies on IPs used by GitHub Actions to be provisioned
in the firewall rules already.
- Update documentation for managing Azure SQL and testing instructions.
Co-authored-by: Long Dai <long.dai@intel.com>
Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
* add state conformance test for postgresql #1117
* make postgresql pass comformance test
* include postgresql to github's workflow
* add newline at end of file for Linux text file
* enhance return
* add newline at end of file for valid Linux text file
* fix lint issue
* handle byte[] in special way so that to stick to json type in DB
* use jsonb instead of json
* fix lint issue
* doesn't allow "", a special case
* enhance assertEquals
* remove unused 'expectedReadResponse' to fix lint issue
* go fmt to fix lint issue
* fix lint issue
Co-authored-by: Simon Leet <31784195+CodeMonkeyLeet@users.noreply.github.com>
Co-authored-by: Yaron Schneider <yaronsc@microsoft.com>
Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
Co-authored-by: Long Dai <long.dai@intel.com>
Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com>
- Add removal of AKV_SPAUTH_SP to generated teardown script.
- Fix extraction of `clientId` and `clientSecret` from provided
credentials without line breaks.
* Add mysql conformance test and fix mysql state to make it pass the test #1122
* remove unused code
* add newline at end of Linux text file
* enhance setValue and relevant unit tests
* fix lint issue
* fix lint issue
- Add `OutputData` property to bindings conformance test `TestConfig`.
This is necessary to accommodate bindings like influx, cosmosdbgraph,
alibaba.table and other bindings that require a specific data format.
- Add docker-compose definition and conformance test configuration for
influx binding.
- Add influx bindings to conformance test workflow matrix.
* Fix parameters flag typo in setup-azure-conf-test.sh
* Update Event Hubs conformance tests
- Separate containers and consumer groups used for EventHub pubsub and
bindings conformance tests.
- Update setup-azure-conf-test.sh to reflect new naming targets.
* Add conformance test for EventHubs bindings
* Update EventHubs pubsub to pass conformance tests
- Add conformance tests for Azure EventHubs PubSub component
- Add retry & backoff handling on subscriber handling error to EventHubs
PubSub component for conformance tests.
- Add cancellation context to Azure EventHubs and update Close() to
invoke cancel prior to closing the hub, which cleans up both senders
and receivers.
* Add automation for deploying Azure conformance test resources
- Add .bicep definitions for Azure deployment of conformance test resources.
- Add setup-azure-conf-test.sh script for deploying .bicep definition and
configuring other assets needed for the Azure conformance tests:
- Creates the appropriate service principals and their credentials
for the test.
- Populates the key vault with the test parameters and secrets used by
the GitHub conformance.yml test workflow.
- Generates a .rc file that can be invoked with `source` to configure
the local environment variables and secrets for running conformance
tests described in tests/conformance/README.md.
- Generates a teardown script that can be used to dispose of the
deployed Azure resources when done.
* Move setup-azure-conf-test under azure/ subfolder
* Add documentation for Azure conformance test automation
- Add README.md to .github/infrastructure/conformance/azure describing
use of automation script.
- Update tests/conformance/README.md with reference to Azure
conformance test infrastructure automation.
- Fix spelling and issues in tests/conformance/README.md reported by
markdownlint.
* Improve conformance tests and add pulsar
* Fix linter error
* Fix linter errors
* Tweaks to pulsar pubsub per PR
* Switching uuid import
* Using Stringer interface to print message keys as base64 in logger
* Disabling some checks w/ Redis pubsub until the component can be enhanced
* Adding comment to eventually remove the simulateErrors option
* Add comment asBase64String to explain what it does
* Adding redelivery (reclaims) to the Redis pubsub component
Removed the temporary simulateErrors flag now that Redis is passing
* Fixing linter issues
* using a wait group instead of a sleep to wait for processing to complete
* More comments and refactoring
* comment tweaks
* Tweaks per PR
Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
* Refactor bindings and introduce input bindings
This commit adds Kafka output bindings tests and refactors the
general bindings tests to include input and output bindings in
the same set of tests. Azure Storage Queues are used as the
first input binding.
* Add http input/output bindings test
Co-authored-by: Artur Souza <artursouza.ms@outlook.com>
Ryan Lettieri
Ryan Lettieri
Tiago Alves Macambira
Bernd Verst
amulyavarote
ItalyPaleAle
Surender Singh Malik
Ricardo Corrêa
Dmitry Shmulevich
Amit Mor
mbimbij
Simon Leet
Ian Luo
Simon Leet
Bernd Verst
Phil Kedy
Artur Souza
halspang