mirror of https://github.com/dapr/docs.git
Merge branch 'v1.10' into issue_3047
Signed-off-by: Hannah Hunter <94493363+hhunter-ms@users.noreply.github.com>
This commit is contained in:
Hannah Hunter
GitHub
commit
2cdc3a7d56
|
|
@ -1,7 +1,7 @@
|
|||
---
|
||||
type: docs
|
||||
title: "How to: Implement pluggable components"
|
||||
linkTitle: "Pluggable components"
|
||||
linkTitle: "Implement pluggable components"
|
||||
weight: 1100
|
||||
description: "Learn how to author and implement pluggable components"
|
||||
---
|
||||
|
|
@ -105,6 +105,201 @@ After generating the above state store example's service scaffolding code using
|
|||
|
||||
This concrete implementation and auxiliary code are the **core** of your pluggable component. They define how your component behaves when handling gRPC requests from Dapr.
|
||||
|
||||
## Returning semantic errors
|
||||
|
||||
Returning semantic errors are also part of the pluggable component protocol. The component must return specific gRPC codes that have semantic meaning for the user application, those errors are used to a variety of situations from concurrency requirements to informational only.
|
||||
|
||||
| Error | gRPC error code | Source component | Description |
|
||||
| ------------------------ | ------------------------------- | ---------------- | ----------- |
|
||||
| ETag Mismatch | `codes.FailedPrecondition` | State store | Error mapping to meet concurrency requirements |
|
||||
| ETag Invalid | `codes.InvalidArgument` | State store | |
|
||||
| Bulk Delete Row Mismatch | `codes.Internal` | State store | |
|
||||
|
||||
Learn more about concurrency requirements in the [State Management overview]({{< ref "state-management-overview.md#concurrency" >}}).
|
||||
|
||||
The following examples demonstrate how to return an error in your own pluggable component, changing the messages to suit your needs.
|
||||
|
||||
{{< tabs ".NET" "Java" "Go" >}}
|
||||
<!-- .NET -->
|
||||
{{% codetab %}}
|
||||
|
||||
> **Important:** In order to use .NET for error mapping, first install the [`Google.Api.CommonProtos` NuGet package](https://www.nuget.org/packages/Google.Api.CommonProtos/).
|
||||
|
||||
**Etag Mismatch**
|
||||
|
||||
```csharp
|
||||
var badRequest = new BadRequest();
|
||||
var des = "The ETag field provided does not match the one in the store";
|
||||
badRequest.FieldViolations.Add(
|
||||
new Google.Rpc.BadRequest.Types.FieldViolation
|
||||
{
|
||||
Field = "etag",
|
||||
Description = des
|
||||
});
|
||||
|
||||
var baseStatusCode = Grpc.Core.StatusCode.FailedPrecondition;
|
||||
var status = new Google.Rpc.Status{
|
||||
Code = (int)baseStatusCode
|
||||
};
|
||||
|
||||
status.Details.Add(Google.Protobuf.WellKnownTypes.Any.Pack(badRequest));
|
||||
|
||||
var metadata = new Metadata();
|
||||
metadata.Add("grpc-status-details-bin", status.ToByteArray());
|
||||
throw new RpcException(new Grpc.Core.Status(baseStatusCode, "fake-err-msg"), metadata);
|
||||
```
|
||||
|
||||
**Etag Invalid**
|
||||
|
||||
```csharp
|
||||
var badRequest = new BadRequest();
|
||||
var des = "The ETag field must only contain alphanumeric characters";
|
||||
badRequest.FieldViolations.Add(
|
||||
new Google.Rpc.BadRequest.Types.FieldViolation
|
||||
{
|
||||
Field = "etag",
|
||||
Description = des
|
||||
});
|
||||
|
||||
var baseStatusCode = Grpc.Core.StatusCode.InvalidArgument;
|
||||
var status = new Google.Rpc.Status
|
||||
{
|
||||
Code = (int)baseStatusCode
|
||||
};
|
||||
|
||||
status.Details.Add(Google.Protobuf.WellKnownTypes.Any.Pack(badRequest));
|
||||
|
||||
var metadata = new Metadata();
|
||||
metadata.Add("grpc-status-details-bin", status.ToByteArray());
|
||||
throw new RpcException(new Grpc.Core.Status(baseStatusCode, "fake-err-msg"), metadata);
|
||||
```
|
||||
|
||||
**Bulk Delete Row Mismatch**
|
||||
|
||||
```csharp
|
||||
var errorInfo = new Google.Rpc.ErrorInfo();
|
||||
|
||||
errorInfo.Metadata.Add("expected", "100");
|
||||
errorInfo.Metadata.Add("affected", "99");
|
||||
|
||||
var baseStatusCode = Grpc.Core.StatusCode.Internal;
|
||||
var status = new Google.Rpc.Status{
|
||||
Code = (int)baseStatusCode
|
||||
};
|
||||
|
||||
status.Details.Add(Google.Protobuf.WellKnownTypes.Any.Pack(errorInfo));
|
||||
|
||||
var metadata = new Metadata();
|
||||
metadata.Add("grpc-status-details-bin", status.ToByteArray());
|
||||
throw new RpcException(new Grpc.Core.Status(baseStatusCode, "fake-err-msg"), metadata);
|
||||
```
|
||||
|
||||
{{% /codetab %}}
|
||||
|
||||
<!-- Java -->
|
||||
{{% codetab %}}
|
||||
|
||||
Just like the [Dapr Java SDK](https://github.com/tmacam/dapr-java-sdk/), the Java Pluggable Components SDK uses [Project Reactor](https://projectreactor.io/), which provides an asynchronous API for Java.
|
||||
|
||||
Errors can be returned directly by:
|
||||
1. Calling the `.error()` method in the `Mono` or `Flux` that your method returns
|
||||
1. Providing the appropriate exception as parameter.
|
||||
|
||||
You can also raise an exception, as long as it is captured and fed back to your resulting `Mono` or `Flux`.
|
||||
|
||||
**ETag Mismatch**
|
||||
|
||||
```java
|
||||
final Status status = Status.newBuilder()
|
||||
.setCode(io.grpc.Status.Code.FAILED_PRECONDITION.value())
|
||||
.setMessage("fake-err-msg-for-etag-mismatch")
|
||||
.addDetails(Any.pack(BadRequest.FieldViolation.newBuilder()
|
||||
.setField("etag")
|
||||
.setDescription("The ETag field provided does not match the one in the store")
|
||||
.build()))
|
||||
.build();
|
||||
return Mono.error(StatusProto.toStatusException(status));
|
||||
```
|
||||
|
||||
**ETag Invalid**
|
||||
|
||||
```java
|
||||
final Status status = Status.newBuilder()
|
||||
.setCode(io.grpc.Status.Code.INVALID_ARGUMENT.value())
|
||||
.setMessage("fake-err-msg-for-invalid-etag")
|
||||
.addDetails(Any.pack(BadRequest.FieldViolation.newBuilder()
|
||||
.setField("etag")
|
||||
.setDescription("The ETag field must only contain alphanumeric characters")
|
||||
.build()))
|
||||
.build();
|
||||
return Mono.error(StatusProto.toStatusException(status));
|
||||
```
|
||||
|
||||
**Bulk Delete Row Mismatch**
|
||||
|
||||
```java
|
||||
final Status status = Status.newBuilder()
|
||||
.setCode(io.grpc.Status.Code.INTERNAL.value())
|
||||
.setMessage("fake-err-msg-for-bulk-delete-row-mismatch")
|
||||
.addDetails(Any.pack(ErrorInfo.newBuilder()
|
||||
.putAllMetadata(Map.ofEntries(
|
||||
Map.entry("affected", "99"),
|
||||
Map.entry("expected", "100")
|
||||
))
|
||||
.build()))
|
||||
.build();
|
||||
return Mono.error(StatusProto.toStatusException(status));
|
||||
```
|
||||
|
||||
{{% /codetab %}}
|
||||
|
||||
<!-- Go -->
|
||||
{{% codetab %}}
|
||||
|
||||
**ETag Mismatch**
|
||||
|
||||
```go
|
||||
st := status.New(codes.FailedPrecondition, "fake-err-msg")
|
||||
desc := "The ETag field provided does not match the one in the store"
|
||||
v := &errdetails.BadRequest_FieldViolation{
|
||||
Field: etagField,
|
||||
Description: desc,
|
||||
}
|
||||
br := &errdetails.BadRequest{}
|
||||
br.FieldViolations = append(br.FieldViolations, v)
|
||||
st, err := st.WithDetails(br)
|
||||
```
|
||||
|
||||
**ETag Invalid**
|
||||
|
||||
```go
|
||||
st := status.New(codes.InvalidArgument, "fake-err-msg")
|
||||
desc := "The ETag field must only contain alphanumeric characters"
|
||||
v := &errdetails.BadRequest_FieldViolation{
|
||||
Field: etagField,
|
||||
Description: desc,
|
||||
}
|
||||
br := &errdetails.BadRequest{}
|
||||
br.FieldViolations = append(br.FieldViolations, v)
|
||||
st, err := st.WithDetails(br)
|
||||
```
|
||||
|
||||
**Bulk Delete Row Mismatch**
|
||||
|
||||
```go
|
||||
st := status.New(codes.Internal, "fake-err-msg")
|
||||
br := &errdetails.ErrorInfo{}
|
||||
br.Metadata = map[string]string{
|
||||
affected: "99",
|
||||
expected: "100",
|
||||
}
|
||||
st, err := st.WithDetails(br)
|
||||
```
|
||||
|
||||
{{% /codetab %}}
|
||||
|
||||
{{< /tabs >}}
|
||||
|
||||
## Next steps
|
||||
|
||||
- Get started with developing .NET pluggable component using this [sample code](https://github.com/dapr/samples/tree/master/pluggable-components-dotnet-template)
|
||||
|
|
|
|||
|
|
@ -42,9 +42,18 @@ spec:
|
|||
| spec.ignoreErrors | N | Tells the Dapr sidecar to continue initialization if the component fails to load. Default is false | `false`
|
||||
| **spec.metadata** | - | **A key/value pair of component specific configuration. See your component definition for fields**|
|
||||
|
||||
### Special metadata values
|
||||
### Templated metadata values
|
||||
|
||||
Metadata values can contain a `{uuid}` tag that is replaced with a randomly generate UUID when the Dapr sidecar starts up. A new UUID is generated on every start up. It can be used, for example, to have a pod on Kubernetes with multiple application instances consuming a [shared MQTT subscription]({{< ref "setup-mqtt3.md" >}}). Below is an example of using the `{uuid}` tag.
|
||||
Metadata values can contain template tags that are resolved on Dapr sidecar startup. The table below shows the current templating tags that can be used in components.
|
||||
|
||||
| Tag | Details | Example use case |
|
||||
|-------------|--------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
| {uuid} | Randomly generated UUIDv4 | When you need a unique identifier in self-hosted mode; for example, multiple application instances consuming a [shared MQTT subscription]({{< ref "setup-mqtt3.md" >}}) |
|
||||
| {podName} | Name of the pod containing the Dapr sidecar | Use to have a persisted behavior, where the ConsumerID does not change on restart when using StatefulSets in Kubernetes |
|
||||
| {namespace} | Namespace where the Dapr sidecar resides combined with its appId | Using a shared `clientId` when multiple application instances consume a Kafka topic in Kubernetes |
|
||||
| {appID} | The configured `appID` of the resource containing the Dapr sidecar | Having a shared `clientId` when multiple application instances consumer a Kafka topic in self-hosted mode |
|
||||
|
||||
Below is an example of using the `{uuid}` tag in an MQTT pubsub component. Note that multiple template tags can be used in a single metadata value.
|
||||
|
||||
```yaml
|
||||
apiVersion: dapr.io/v1alpha1
|
||||
|
|
@ -67,9 +76,6 @@ spec:
|
|||
value: "false"
|
||||
```
|
||||
|
||||
The consumerID metadata values can also contain a `{podName}` tag that is replaced with the Kubernetes POD's name when the Dapr sidecar starts up. This can be used to have a persisted behavior where the ConsumerID does not change on restart when using StatefulSets in Kubernetes.
|
||||
|
||||
|
||||
## Further reading
|
||||
- [Components concept]({{< ref components-concept.md >}})
|
||||
- [Reference secrets in component definitions]({{< ref component-secrets.md >}})
|
||||
|
|
|
|||
|
|
@ -132,7 +132,7 @@ Follow the steps provided in the [Deploy Dapr on a Kubernetes cluster]({{< ref k
|
|||
|
||||
## Add the pluggable component container in your deployments
|
||||
|
||||
When running in Kubernetes mode, pluggable components are deployed as containers in the same pod as your application.
|
||||
Pluggable components are deployed as containers **in the same pod** as your application.
|
||||
|
||||
Since pluggable components are backed by [Unix Domain Sockets][uds], make the socket created by your pluggable component accessible by Dapr runtime. Configure the deployment spec to:
|
||||
|
||||
|
|
@ -140,7 +140,7 @@ Since pluggable components are backed by [Unix Domain Sockets][uds], make the so
|
|||
2. Hint to Dapr the mounted Unix socket volume location
|
||||
3. Attach volume to your pluggable component container
|
||||
|
||||
Below is an example of a deployment that configures a pluggable component:
|
||||
In the following example, your configured pluggable component is deployed as a container within the same pod as your application container.
|
||||
|
||||
```yaml
|
||||
apiVersion: apps/v1
|
||||
|
|
@ -167,17 +167,51 @@ spec:
|
|||
- name: dapr-unix-domain-socket
|
||||
emptyDir: {}
|
||||
containers:
|
||||
### --------------------- YOUR APPLICATION CONTAINER GOES HERE -----------
|
||||
##
|
||||
### --------------------- YOUR APPLICATION CONTAINER GOES HERE -----------
|
||||
### This is the pluggable component container.
|
||||
containers:
|
||||
### --------------------- YOUR APPLICATION CONTAINER GOES HERE -----------
|
||||
- name: app
|
||||
image: YOUR_APP_IMAGE:YOUR_APP_IMAGE_VERSION
|
||||
### --------------------- YOUR PLUGGABLE COMPONENT CONTAINER GOES HERE -----------
|
||||
- name: component
|
||||
image: YOUR_IMAGE_GOES_HERE:YOUR_IMAGE_VERSION
|
||||
volumeMounts: # required, the sockets volume mount
|
||||
- name: dapr-unix-domain-socket
|
||||
mountPath: /tmp/dapr-components-sockets
|
||||
image: YOUR_IMAGE_GOES_HERE:YOUR_IMAGE_VERSION
|
||||
```
|
||||
|
||||
Alternatively, you can annotate your pods, telling Dapr which containers within that pod are pluggable components, like in the example below:
|
||||
|
||||
```yaml
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: app
|
||||
labels:
|
||||
app: app
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app: app
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: app
|
||||
annotations:
|
||||
dapr.io/pluggable-components: "component" ## the name of the pluggable component container separated by `,`, e.g "componentA,componentB".
|
||||
dapr.io/app-id: "my-app"
|
||||
dapr.io/enabled: "true"
|
||||
spec:
|
||||
containers:
|
||||
### --------------------- YOUR APPLICATION CONTAINER GOES HERE -----------
|
||||
- name: app
|
||||
image: YOUR_APP_IMAGE:YOUR_APP_IMAGE_VERSION
|
||||
### --------------------- YOUR PLUGGABLE COMPONENT CONTAINER GOES HERE -----------
|
||||
- name: component
|
||||
image: YOUR_IMAGE_GOES_HERE:YOUR_IMAGE_VERSION
|
||||
```
|
||||
|
||||
Before applying the deployment, let's add one more configuration: the component spec.
|
||||
|
||||
## Define a component
|
||||
|
|
|
|||
|
|
@ -72,6 +72,30 @@ spec:
|
|||
enabled: false
|
||||
```
|
||||
|
||||
## High cardinality metrics
|
||||
|
||||
Depending on your use case, some metrics emitted by Dapr might contain values that have a high cardinality. This might cause increased memory usage for the Dapr process/container and incur expensive egress costs in certain cloud environments. To mitigate this issue, you can set regular expressions for every metric exposed by the Dapr sidecar. [See a list of all Dapr metrics](https://github.com/dapr/dapr/blob/master/docs/development/dapr-metrics.md).
|
||||
|
||||
The following example shows how to apply a regular expression for the label `method` in the metric `dapr_runtime_service_invocation_req_sent_total`:
|
||||
|
||||
```yaml
|
||||
apiVersion: dapr.io/v1alpha1
|
||||
kind: Configuration
|
||||
metadata:
|
||||
name: daprConfig
|
||||
spec:
|
||||
metric:
|
||||
enabled: true
|
||||
rules:
|
||||
- name: dapr_runtime_service_invocation_req_sent_total
|
||||
labels:
|
||||
- name: method
|
||||
regex:
|
||||
"orders/": "orders/.+"
|
||||
```
|
||||
|
||||
When this configuration is applied, a recorded metric with the `method` label of `orders/a746dhsk293972nz` will be replaced with `orders/`.
|
||||
|
||||
## References
|
||||
|
||||
* [Howto: Run Prometheus locally]({{< ref prometheus.md >}})
|
||||
|
|
|
|||
|
|
@ -15,7 +15,6 @@ For CLI there is no explicit opt-in, just the version that this was first made a
|
|||
|
||||
| Feature | Description | Setting | Documentation | Version introduced |
|
||||
| ---------- |-------------|---------|---------------|-----------------|
|
||||
| **`--image-registry`** flag in Dapr CLI| In self hosted mode you can set this flag to specify any private registry to pull the container images required to install Dapr| N/A | [CLI init command reference]({{<ref "dapr-init.md#self-hosted-environment" >}}) | v1.7 |
|
||||
| **App Middleware** | Allow middleware components to be executed when making service-to-service calls | N/A | [App Middleware]({{<ref "middleware.md#app-middleware" >}}) | v1.9 |
|
||||
| **App health checks** | Allows configuring app health checks | `AppHealthCheck` | [App health checks]({{<ref "app-health.md" >}}) | v1.9 |
|
||||
| **Pluggable components** | Allows creating self-hosted gRPC-based components written in any language that supports gRPC. The following component APIs are supported: State stores, Pub/sub, Bindings | N/A | [Pluggable components concept]({{<ref "components-concept#pluggable-components" >}})| v1.9 |
|
||||
|
|
|
|||
|
|
@ -11,6 +11,9 @@ aliases:
|
|||
|
||||
To setup Kafka binding create a component of type `bindings.kafka`. See [this guide]({{< ref "howto-bindings.md#1-create-a-binding" >}}) on how to create and apply a binding configuration. For details on using `secretKeyRef`, see the guide on [how to reference secrets in components]({{< ref component-secrets.md >}}).
|
||||
|
||||
All component metadata field values can carry [templated metadata values]({{< ref "component-schema.md#templated-metadata-values" >}}), which are resolved on Dapr sidecar startup.
|
||||
For example, you can choose to use `{namespace}` as the `consumerGroup`, to enable using the same `appId` in different namespaces using the same topics as described in [this article]({{< ref "howto-namespace.md#with-namespace-consumer-groups">}}).
|
||||
|
||||
```yaml
|
||||
apiVersion: dapr.io/v1alpha1
|
||||
kind: Component
|
||||
|
|
|
|||
|
|
@ -11,6 +11,9 @@ aliases:
|
|||
|
||||
To setup Apache Kafka pubsub create a component of type `pubsub.kafka`. See [this guide]({{< ref "howto-publish-subscribe.md#step-1-setup-the-pubsub-component" >}}) on how to create and apply a pubsub configuration. For details on using `secretKeyRef`, see the guide on [how to reference secrets in components]({{< ref component-secrets.md >}}).
|
||||
|
||||
All component metadata field values can carry [templated metadata values]({{< ref "component-schema.md#templated-metadata-values" >}}), which are resolved on Dapr sidecar startup.
|
||||
For example, you can choose to use `{namespace}` as the `consumerGroup` to enable using the same `appId` in different namespaces using the same topics as described in [this article]({{< ref "howto-namespace.md#with-namespace-consumer-groups">}}).
|
||||
|
||||
```yaml
|
||||
apiVersion: dapr.io/v1alpha1
|
||||
kind: Component
|
||||
|
|
@ -23,7 +26,7 @@ spec:
|
|||
- name: brokers # Required. Kafka broker connection setting
|
||||
value: "dapr-kafka.myapp.svc.cluster.local:9092"
|
||||
- name: consumerGroup # Optional. Used for input bindings.
|
||||
value: "group1"
|
||||
value: "{namespace}"
|
||||
- name: clientID # Optional. Used as client tracing ID by Kafka brokers.
|
||||
value: "my-dapr-app-id"
|
||||
- name: authType # Required.
|
||||
|
|
|
|||
|
|
@ -38,6 +38,28 @@ spec:
|
|||
value: "-1"
|
||||
- name: disableBatching
|
||||
value: "false"
|
||||
- name: <topic-name>.jsonschema # sets a json schema validation for the configured topic
|
||||
value: |
|
||||
{
|
||||
"type": "record",
|
||||
"name": "Example",
|
||||
"namespace": "test",
|
||||
"fields": [
|
||||
{"name": "ID","type": "int"},
|
||||
{"name": "Name","type": "string"}
|
||||
]
|
||||
}
|
||||
- name: <topic-name>.avroschema # sets an avro schema validation for the configured topic
|
||||
value: |
|
||||
{
|
||||
"type": "record",
|
||||
"name": "Example",
|
||||
"namespace": "test",
|
||||
"fields": [
|
||||
{"name": "ID","type": "int"},
|
||||
{"name": "Name","type": "string"}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## Spec metadata fields
|
||||
|
|
@ -62,6 +84,8 @@ spec:
|
|||
| batchingMaxPublishDelay | N | batchingMaxPublishDelay set the time period within which the messages sent will be batched,if batch messages are enabled. If set to a non zero value, messages will be queued until this time interval or batchingMaxMessages (see below) or batchingMaxSize (see below). There are two valid formats, one is the fraction with a unit suffix format, and the other is the pure digital format that is processed as milliseconds. Valid time units are "ns", "us" (or "µs"), "ms", "s", "m", "h". Default: `"10ms"` | `"10ms"`, `"10"`|
|
||||
| batchingMaxMessages | N | batchingMaxMessages set the maximum number of messages permitted in a batch.If set to a value greater than 1, messages will be queued until this threshold is reached or batchingMaxSize (see below) has been reached or the batch interval has elapsed. Default: `"1000"` | `"1000"`|
|
||||
| batchingMaxSize | N | batchingMaxSize sets the maximum number of bytes permitted in a batch. If set to a value greater than 1, messages will be queued until this threshold is reached or batchingMaxMessages (see above) has been reached or the batch interval has elapsed. Default: `"128KB"` | `"131072"`|
|
||||
| <topic-name>.jsonschema | N | Enforces JSON schema validation for the configured topic. |
|
||||
| <topic-name>.avroschema | N | Enforces Avro schema validation for the configured topic. |
|
||||
|
||||
### Delay queue
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue