Sorry for how big this PR seems (a lot of the change lines are not
significant (DB schema or tests), so I've tried to highlight the
important parts. This is a substantial PR that builds on our recent
changes to use go-memdb for storage. It primarily supports 2 new crucial
features:
- support for duplicate flag keys from multiple sources (namespaced by
`flagSetId`) as described in [this
ADR](https://github.com/open-feature/flagd/blob/main/docs/architecture-decisions/duplicate-flag-keys.md)
- support for a robust query syntax in the "selector" fields and
headers, as proposed by @tangenti in [this
ADR](https://github.com/open-feature/flagd/blob/main/docs/architecture-decisions/decouple-flag-source-and-set.md)
Both of these were accomplished using the new go-memdb module. **The
built-in "watcher" functionality also allows us to _completely delete_
our "mux" layer, which was responsible for fanning out changes from
sync-sources to listeners**; this is something the go-memdb module
providers for free (the ability to watch a query for changes). Now, we
completely rely on this feature for all change notifications.
Additionally, unlike before, change notifications are now scoped to
particular _selectors_ (ie: if a client is only interested in changes
for flags from `flagSetId: x` or `source: y`, they will only get change
notifications pertaining to that selection. Currently, the only
supported query fields for the `selector` are `"source"` and
`"flagSetId"`, but this functionality can easily be extended. By
default, if no `selector` is specified, the previous key-overwrite by
source priority apples (this logic has also been simplified using the
new database). Most of the new functionality is tested
[here](https://github.com/open-feature/flagd/pull/1702/files#diff-0cdd4fe716f4b8a94466279fd1b11187fcf4d74e6434727c33d57ed78c89fe27R163-R478).
Selector in action for `Sync` API:
Selector in action for `Evaluation` API:
To test, run the new make target `make run-flagd-selector-demo`, then
use the OFREP or gRPC endpoints to experiment. This new functionality is
available on all APIs and endpoints. The command I ran in the gifs above
are:
streaming:
```shell
grpcurl -d '{"selector":"flagSetId=example"}' -import-path schemas/protobuf/flagd/sync/v1/ -proto sync.proto -plaintext localhost:8015 flagd.sync.v1.FlagSyncService/SyncFlags | jq
grpcurl -d '{"selector":"flagSetId=example,source=../config/samples/example_flags.flagd.json"}' -import-path schemas/protobuf/flagd/sync/v1/ -proto sync.proto -plaintext localhost:8015 flagd.sync.v1.FlagSyncService/SyncFlags | jq
grpcurl -d '{"selector":"flagSetId=other"}' -import-path schemas/protobuf/flagd/sync/v1/ -proto sync.proto -plaintext localhost:8015 flagd.sync.v1.FlagSyncService/SyncFlags | jq
```
single-evaluations:
```shell
curl -X POST -d '{"context":{}}' 'http://localhost:8016/ofrep/v1/evaluate/flags' | jq
curl -X POST -H 'flagd-selector:flagSetId=other' -d '{"context":{}}' 'http://localhost:8016/ofrep/v1/evaluate/flags' | jq
```
⚠️ There's no breaking changes here. Besides the new features,
there is one behavioral change - the top level "metadata" object
returned for bulk evaluations (and failed evaluations) was previously
very nonsensical in it's behavior (we basically just aggregated the
metadata from all sources, discarding duplicates, and sent it back. This
field was used by providers for telemetry purposes. Now, since flag
evaluations and subscripts are "query based" and can aggregate data from
multiple sources, we've opted to simply reflect the selector queries
contents here.
So if you used a selector like `"flagSetId=1234,source=../my/source"`,
the top-level metadata object in the response would be:
```
"metadata": {
"flagSetId": 1234,
"source": "../my/source"
}
```
This is useful for the provider's ability to report errors, etc in
telemetry.
Fixes: https://github.com/open-feature/flagd/issues/1675
Fixes: https://github.com/open-feature/flagd/issues/1695
Fixes: https://github.com/open-feature/flagd/issues/1611
Fixes: https://github.com/open-feature/flagd/issues/1700
Fixes: https://github.com/open-feature/flagd/issues/1610
---------
Signed-off-by: Todd Baert <todd.baert@dynatrace.com>
Co-authored-by: chrfwow <christian.lutnik@dynatrace.com>
Co-authored-by: Giovanni Liva <giovanni.liva@dynatrace.com>
⚠️ This PR brings a breaking change to the
[flagd-core](https://pkg.go.dev/github.com/open-feature/flagd/core)
library: the `IStore` interface now returns an additional value
representing the flag set metadata. There are no breaking changes in
flagd's behavior.
Changes in flagd:
- returns flag set metadata as metadata for error flags (best effort)
- returns flag set metadata in OFREP and RPC calls
- moves metadata merging logic to evaluator (all flags inherent flag set
metadata, but can override, as as before but now reusable in flagd core)
- removes duplicated flag set metadata keys when the same flag set
metadata key exists in multiple sources
### To Test
- requires `curl`, `grpcurl`, and `jq`
#### RPC
```shell
grpcurl -import-path /...../schemas/protobuf/flagd/evaluation/v1 -proto evaluation.proto -plaintext localhost:8013 flagd.evaluation.v1.Service/ResolveAll | jq
```
### OFREP
```shell
curl --location 'http://localhost:8016/ofrep/v1/evaluate/flags' --header 'Content-Type: application/json' --data '{"context": {"color": "yellow"}}' | jq
```
### Sync
```shell
grpcurl -import-path /...../schemas/protobuf/flagd/sync/v1/ -proto sync.proto -plaintext localhost:8015 flagd.sync.v1.FlagSyncService/FetchAllFlags | jq -r .flagConfiguration | jq
```
---------
Signed-off-by: Todd Baert <todd.baert@dynatrace.com>
## This PR
Fixes#1245 and introduce OFREP support for flagd
OFREP service runs on 8016 by default and can simply use curl for flag
evaluations,
```shell
curl -X POST 'http://localhost:8016/ofrep/v1/evaluate/flags/myBoolFlag'
```
NOTE - This PR touch several files as I had to migrate from
`github.com/golang/mock/gomock` to `go.uber.org/mock/gomock`. This is
because https://github.com/golang/mock is no longer maintained.
---------
Signed-off-by: Kavindu Dodanduwa <kavindudodanduwa@gmail.com>
Signed-off-by: Kavindu Dodanduwa <Kavindu-Dodan@users.noreply.github.com>
Co-authored-by: Giovanni Liva <giovanni.liva@dynatrace.com>
Co-authored-by: Florian Bacher <florian.bacher@dynatrace.com>
Signed-off-by: James Milligan <james@omnant.co.uk>
<!-- Please use this template for your pull request. -->
<!-- Please use the sections that you need and delete other sections -->
## This PR
<!-- add the description of the PR here -->
- updates file paths to conform to schema store regex
### Related Issues
<!-- add here the GitHub issue that this PR resolves if applicable -->
### Notes
<!-- any additional notes for this PR -->
### Follow-up Tasks
<!-- anything that is related to this PR but not done here should be
noted under this section -->
<!-- if there is a need for a new issue, please link it here -->
### How to test
<!-- if applicable, add testing instructions under this section -->
---------
Signed-off-by: James Milligan <james@omnant.co.uk>
Co-authored-by: Michael Beemer <beeme1mr@users.noreply.github.com>
* wip
Signed-off-by: Alex Jones <alexsimonjones@gmail.com>
* wip
Signed-off-by: Alex Jones <alexsimonjones@gmail.com>
* wip
Signed-off-by: Alex Jones <alexsimonjones@gmail.com>
* working https
Signed-off-by: Alex Jones <alexsimonjones@gmail.com>
* working https
Signed-off-by: Alex Jones <alexsimonjones@gmail.com>
* linting
Signed-off-by: Alex Jones <alexsimonjones@gmail.com>
* linting
Signed-off-by: Alex Jones <alexsimonjones@gmail.com>
* updated linter issues
Signed-off-by: Alex Jones <alexsimonjones@gmail.com>
* linting
Signed-off-by: Alex Jones <alexsimonjones@gmail.com>
* passing linter
Signed-off-by: Alex Jones <alexsimonjones@gmail.com>
* passing linter
Signed-off-by: Alex Jones <alexsimonjones@gmail.com>
* updated
Signed-off-by: Alex Jones <alexsimonjones@gmail.com>
* resolvin comments
Signed-off-by: Alex Jones <alexsimonjones@gmail.com>
* updated json eval tests to reflect int64 anf float64 changes
Signed-off-by: James-Milligan <james@omnant.co.uk>
Signed-off-by: Alex Jones <alexsimonjones@gmail.com>
* resolvin comments
Signed-off-by: Alex Jones <alexsimonjones@gmail.com>
* updated readme
Signed-off-by: James-Milligan <james@omnant.co.uk>
Signed-off-by: Alex Jones <alexsimonjones@gmail.com>
* updated example flags and readme
Signed-off-by: James-Milligan <james@omnant.co.uk>
Signed-off-by: Alex Jones <alexsimonjones@gmail.com>
* resolvin comments
Signed-off-by: Alex Jones <alexsimonjones@gmail.com>
* Update cmd/start.go
Co-authored-by: Michael Beemer <beeme1mr@users.noreply.github.com>
Signed-off-by: Alex Jones <alex.jones@canonical.com>
Signed-off-by: Alex Jones <alexsimonjones@gmail.com>
* Update cmd/start.go
Co-authored-by: Michael Beemer <beeme1mr@users.noreply.github.com>
Signed-off-by: Alex Jones <alex.jones@canonical.com>
Signed-off-by: Alex Jones <alexsimonjones@gmail.com>
* resolvin comments
Signed-off-by: Alex Jones <alexsimonjones@gmail.com>
* updated readme
Signed-off-by: Alex Jones <alexsimonjones@gmail.com>
* Update README.md
Signed-off-by: Alex Jones <alexsimonjones@gmail.com>
* merge
Signed-off-by: Alex Jones <alexsimonjones@gmail.com>
* simplifying tls config
Signed-off-by: Alex Jones <alexsimonjones@gmail.com>
Co-authored-by: James-Milligan <james@omnant.co.uk>
Co-authored-by: Michael Beemer <beeme1mr@users.noreply.github.com>
Todd Baert