components-contrib/state
Alessandro (Ale) Segala 86ed8eb2a4
Merge branch 'main' into postgres-delete-with-prefix
2024-02-07 13:24:22 -08:00
..
aerospike Switch to Go 1.21, Updates linter, updates workflows, adds sarama 1.42.1 (#3251) 2023-11-28 18:13:09 -08:00
alicloud/tablestore Switch to Go 1.21, Updates linter, updates workflows, adds sarama 1.42.1 (#3251) 2023-11-28 18:13:09 -08:00
aws/dynamodb secret store: AWS connection validation for parameter store and secrets manager (#3301) 2024-01-16 08:28:29 -08:00
azure CosmosDBState - Add, as option, the partitionKey in QueryMethod (#3227) 2024-01-24 16:49:33 -08:00
cassandra Switch to Go 1.21, Updates linter, updates workflows, adds sarama 1.42.1 (#3251) 2023-11-28 18:13:09 -08:00
cloudflare/workerskv Chore: use a single package for all Feature structs (#3278) 2024-01-08 09:07:10 -08:00
cockroachdb WIP: We need to find a solution because CockroachDB doesn't support UDFs 2024-01-06 00:17:44 +00:00
couchbase Switch to Go 1.21, Updates linter, updates workflows, adds sarama 1.42.1 (#3251) 2023-11-28 18:13:09 -08:00
etcd Switch to Go 1.21, Updates linter, updates workflows, adds sarama 1.42.1 (#3251) 2023-11-28 18:13:09 -08:00
gcp/firestore Switch to Go 1.21, Updates linter, updates workflows, adds sarama 1.42.1 (#3251) 2023-11-28 18:13:09 -08:00
hashicorp/consul Switch to Go 1.21, Updates linter, updates workflows, adds sarama 1.42.1 (#3251) 2023-11-28 18:13:09 -08:00
hazelcast Switch to Go 1.21, Updates linter, updates workflows, adds sarama 1.42.1 (#3251) 2023-11-28 18:13:09 -08:00
in-memory Adding deleteWithPrefix for in-memory statestore (#3314) 2024-01-12 19:40:07 +00:00
jetstream Update dapr/kit (#3205) 2023-11-02 16:20:56 -07:00
memcached Switch to Go 1.21, Updates linter, updates workflows, adds sarama 1.42.1 (#3251) 2023-11-28 18:13:09 -08:00
mongodb Add new filters type in the query of the state component (#3218) 2023-12-06 20:51:27 +00:00
mysql Add ExecuteInTransaction method for db.SQL (#3309) 2024-01-16 08:27:46 -08:00
oci/objectstorage Switch to Go 1.21, Updates linter, updates workflows, adds sarama 1.42.1 (#3251) 2023-11-28 18:13:09 -08:00
oracledatabase Switch to Go 1.21, Updates linter, updates workflows, adds sarama 1.42.1 (#3251) 2023-11-28 18:13:09 -08:00
postgresql WIP: YugabyteDB support 2024-01-06 00:59:59 +00:00
query Add new filters type in the query of the state component (#3218) 2023-12-06 20:51:27 +00:00
redis Redis State Store query: numeric operators do not work correctly on large numbers (#3334) 2024-02-05 13:52:21 -08:00
rethinkdb Switch to Go 1.21, Updates linter, updates workflows, adds sarama 1.42.1 (#3251) 2023-11-28 18:13:09 -08:00
sqlite Remove DeleteWithPrefix from SQLite for now (#3316) 2024-01-17 19:06:29 +00:00
sqlserver Add ExecuteInTransaction method for db.SQL (#3309) 2024-01-16 08:27:46 -08:00
utils ETCD State store v2 (#2932) 2023-06-26 16:08:48 +00:00
zookeeper Switch to Go 1.21, Updates linter, updates workflows, adds sarama 1.42.1 (#3251) 2023-11-28 18:13:09 -08:00
README.md Add new filters type in the query of the state component (#3218) 2023-12-06 20:51:27 +00:00
bulk.go Remove native/transactional BulkSet/BulkDelete from state stores (#2834) 2023-05-17 14:56:34 -07:00
bulk_test.go Switch to Go 1.21, Updates linter, updates workflows, adds sarama 1.42.1 (#3251) 2023-11-28 18:13:09 -08:00
errors.go Remove native/transactional BulkSet/BulkDelete from state stores (#2834) 2023-05-17 14:56:34 -07:00
errors_test.go Refactor state store bulk operations (#2747) 2023-04-07 19:19:47 +00:00
feature.go CosmosDBState - Add, as option, the partitionKey in QueryMethod (#3227) 2024-01-24 16:49:33 -08:00
metadata.go Refactor state store bulk operations (#2747) 2023-04-07 19:19:47 +00:00
request_options.go Refactor state store bulk operations (#2747) 2023-04-07 19:19:47 +00:00
request_options_test.go Switch to Go 1.21, Updates linter, updates workflows, adds sarama 1.42.1 (#3251) 2023-11-28 18:13:09 -08:00
requests.go Added conformance tests for DeleteWithPrefix (#3288) 2023-12-28 04:20:38 +00:00
responses.go DeleteWithPrefix support for SQLite (#3265) 2023-12-18 18:39:58 +00:00
store.go Added conformance tests for DeleteWithPrefix (#3288) 2023-12-28 04:20:38 +00:00

README.md

State Stores

State Stores provide a common way to interact with different data store implementations, and allow users to opt-in to advanced capabilities using defined metadata.

Implementing a new State Store

A compliant state store needs to implement one or more interfaces: Store and TransactionalStore, defined in the store.go file.

See the documentation site for examples.

Implementing State Query API

State Store has an optional API for querying the state.

Please refer to the documentation site for API description and definition.

// Querier is an interface to execute queries.
type Querier interface {
        Query(req *QueryRequest) (*QueryResponse, error)
}

Below are the definitions of structures (including nested) for QueryRequest and QueryResponse.

// QueryResponse is the request object for querying the state.
type QueryRequest struct {
        Query    query.Query       `json:"query"`
        Metadata map[string]string `json:"metadata,omitempty"`
}

type Query struct {
        Filters map[string]interface{} `json:"filter"`
        Sort    []Sorting              `json:"sort"`
        Page    Pagination             `json:"page"`

        // derived from Filters
        Filter Filter
}

type Sorting struct {
        Key   string `json:"key"`
        Order string `json:"order,omitempty"`
}

type Pagination struct {
        Limit int    `json:"limit"`
        Token string `json:"token,omitempty"`
}

// QueryResponse is the response object on querying state.
type QueryResponse struct {
        Results  []QueryItem       `json:"results"`
        Token    string            `json:"token,omitempty"`
        Metadata map[string]string `json:"metadata,omitempty"`
}

// QueryItem is an object representing a single entry in query results.
type QueryItem struct {
        Key   string  `json:"key"`
        Data  []byte  `json:"data"`
        ETag  *string `json:"etag,omitempty"`
        Error string  `json:"error,omitempty"`
}

Upon receiving the query request, Dapr validates it and transforms into object Query, which, in turn, is passed on to the state store component.

The Query object has a member Filter that implements parsing interface per component as described below.

type Filter interface {
	Parse(interface{}) error
}

type FilterEQ struct {
	Key string
	Val interface{}
}

type FilterNEQ struct {
    Key string
    Val interface{}
}

type FilterGT struct {
    Key string
    Val interface{}
}

type FilterGTE struct {
    Key string
    Val interface{}
}

type FilterLT struct {
    Key string
    Val interface{}
}

type FilterLTE struct {
    Key string
    Val interface{}
}

type FilterIN struct {
	Key  string
	Vals []interface{}
}

type FilterAND struct {
	Filters []Filter
}

type FilterOR struct {
	Filters []Filter
}

To simplify the process of query translation, we leveraged visitor design pattern. A state store component developer would need to implement the visit method, and the runtime will use it to construct the native query statement.

type Visitor interface {
	// returns "equal" expression
	VisitEQ(*FilterEQ) (string, error)
	// returns "not equal" expression 
	VisitNEQ(*FilterNEQ) (string, error) 
	// returns "greater than" expression
	VisitGT(*FilterGT) (string, error)
	// returns "greater than equal" expression
	VisitGTE(*FilterGTE) (string, error)
	// returns "less than" expression
	VisitLT(*FilterLT) (string, error)
	// returns "less than equal" expression
	VisitLTE(*FilterLTE) (string, error)
	// returns "in" expression
	VisitIN(*FilterIN) (string, error)
	// returns "and" expression
	VisitAND(*FilterAND) (string, error)
	// returns "or" expression
	VisitOR(*FilterOR) (string, error)
	// receives concatenated filters and finalizes the native query
	Finalize(string, *MidQuery) error
}

The Dapr runtime implements QueryBuilder object that takes in Visitor interface and constructs the native query.

type QueryBuilder struct {
	visitor Visitor
}

func (h *QueryBuilder) BuildQuery(mq *MidQuery) error {...}

The last part is to implement Querier interface in the component:

type Querier interface {
	Query(req *QueryRequest) (*QueryResponse, error)
}

A sample implementation might look like that:

func (m *MyComponent) Query(req *state.QueryRequest) (*state.QueryResponse, error) {
	ctx, cancel := context.WithTimeout(context.Background(), timeout)
	defer cancel()

	query := &Query{} // Query implements Visitor interface
	qbuilder := state.NewQueryBuilder(query)
	if err := qbuilder.BuildQuery(&req.Query); err != nil {
		return &state.QueryResponse{}, err
	}
	data, token, err := query.execute(ctx)
	if err != nil {
		return &state.QueryResponse{}, err
	}
	return &state.QueryResponse{
		Results:  data,
		Token:    token,
	}, nil
}

Some of the examples of State Query API implementation are Redis, MongoDB and CosmosDB state store components.