dapr
/
docs
mirror of https://github.com/dapr/docs.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Updated conversation api docs to latest alpha v2, refresh sample request, add metadata, remove redundant fields like name, fixes #4687 

Signed-off-by: Bilgin Ibryam <bibryam@gmail.com>
This commit is contained in:
Bilgin Ibryam 2025-09-12 16:15:40 +01:00
parent 95f4442435
commit 23de64e9d7
No known key found for this signature in database
GPG Key ID: F4E44C0A8C57006F
2 changed files with 168 additions and 112 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
daprdocs/content/en/developing-applications/building-blocks/conversation/conversation-overview.md
View File

@ -20,10 +20,10 @@ In addition to enabling critical performance and security functionality (like [p
- **OpenAI-compatible interface** for seamless integration with existing AI workflows and tools
You can also pair the conversation API with Dapr functionalities, like:
- Resiliency circuit breakers and retries to circumvent limit and token errors, or
- Middleware to authenticate requests coming to and from the LLM
Dapr provides observability by issuing metrics for your LLM interactions.
- Resiliency policies including circuit breakers to handle repeated errors, timeouts to safeguards from slow responses, and retries for temporary network failures
- Observability with metrics and distributed tracing using OpenTelemetry and Zipkin
- Middleware to authenticate requests to and from the LLM
## Features
@ -31,7 +31,7 @@ The following features are out-of-the-box for [all the supported conversation co
### Prompt caching
Prompt caching optimizes performance by storing and reusing prompts that are often repeated across multiple API calls. To significantly reduce latency and cost, Dapr stores frequent prompts in a local cache to be reused by your cluster, pod, or other, instead of reprocessing the information for every new request.
Prompt caching improves performance by storing and reusing prompts repeated across API calls. Dapr keeps frequent prompts in a local cache for reuse with your app, reducing latency and cost by avoiding reprocessing for every request.
### Personally identifiable information (PII) obfuscation
@ -67,7 +67,7 @@ Watch the demo presented during [Diagrid's Dapr v1.15 celebration](https://www.d
{{< youtube id=NTnwoDhHIcQ start=5444 >}}
## Try out conversation
## Try out conversation API
### Quickstarts and tutorials

270
daprdocs/content/en/reference/api/conversation_api.md
View File

@ -12,7 +12,7 @@ The conversation API is currently in [alpha]({{% ref "certification-lifecycle.md
Dapr provides an API to interact with Large Language Models (LLMs) and enables critical performance and security functionality with features like prompt caching, PII data obfuscation, and tool calling capabilities.
Tool calling follow's OpenAI's function calling format, making it easy to integrate with existing AI development workflows and tools.
Tool calling follows OpenAI's function calling format, making it easy to integrate with existing AI development workflows and tools.
## Converse
@ -32,32 +32,34 @@ POST http://localhost:<daprPort>/v1.0-alpha2/conversation/<llm-name>/converse
| Field | Description |
| --------- | ----------- |
| `name` | The name of the conversation component. Required |
| `contextId` | The ID of an existing chat (like in ChatGPT). Optional |
| `context_id` | The ID of an existing chat (like in ChatGPT). Optional |
| `inputs` | Inputs for the conversation. Multiple inputs at one time are supported. Required |
| `parameters` | Parameters for all custom fields. Optional |
| `metadata` | [Metadata](#metadata) passed to conversation components. Optional |
| `scrubPii` | A boolean value to enable obfuscation of sensitive information returning from the LLM. Optional |
| `metadata` | Metadata passed to conversation components. Optional |
| `scrub_pii` | A boolean value to enable obfuscation of sensitive information returning from the LLM. Optional |
| `temperature` | A float value to control the temperature of the model. Used to optimize for consistency (0) or creativity (1). Optional |
| `tools` | Tools register the tools available to be used by the LLM during the conversation. Optional |
| `toolChoice` | Controls which (if any) tool is called by the model. Values: `auto`, `required`, or specific tool name. Defaults to `auto` if tools are present. Optional |
| `tool_choice` | Controls which (if any) tool is called by the model. Values: `auto`, `required`, or specific tool name. Defaults to `auto` if tools are present. Optional |
#### Input body
| Field | Description |
| --------- | ----------- |
| `messages` | Array of conversation messages. Required |
| `scrubPii` | A boolean value to enable obfuscation of sensitive information present in the content field. Optional |
| `scrub_pii` | A boolean value to enable obfuscation of sensitive information present in the content field. Optional |
#### Message types
The API supports different message types:
- **`ofDeveloper`**: Developer role messages with optional name and content
- **`ofSystem`**: System role messages with optional name and content
- **`ofUser`**: User role messages with optional name and content
- **`ofAssistant`**: Assistant role messages with optional name, content, and tool calls
- **`ofTool`**: Tool role messages with tool ID, name, and content
| Type | Description |
| ---- | ----------- |
| `of_developer` | Developer role messages with optional name and content |
| `of_system` | System role messages with optional name and content |
| `of_user` | User role messages with optional name and content |
| `of_assistant` | Assistant role messages with optional name, content, and tool calls |
| `of_tool` | Tool role messages with tool ID, name, and content |
#### Tool calling
@ -69,81 +71,136 @@ Tools can be defined using the `tools` field with function definitions:
| `function.description` | A description of what the function does. Optional |
| `function.parameters` | JSON Schema object describing the function parameters. Optional |
#### Tool choice options
The `tool_choice` is an optional parameter that controls how the model can use available tools:
- **`auto`**: The model can pick between generating a message or calling one or more tools (default when tools are present)
- **`required`**: Requires one or more functions to be called
- **`{tool_name}`**: Forces the model to call a specific tool by name
#### Metadata
The `metadata` field serves as a dynamic configuration mechanism that allows you to pass additional configuration and authentication information to conversation components on a per-request basis. This metadata overrides any corresponding fields configured in the component's YAML configuration file, enabling dynamic configuration without modifying static component definitions.
**Common metadata fields:**
| Field | Description | Example |
| ----- | ----------- | ------- |
| `api_key` | API key for authenticating with the LLM service | `"sk-1234567890abcdef"` |
| `model` | Specific model identifier | `"gpt-4-turbo"`, `"claude-3-sonnet"` |
| `version` | API version or service version | `"1.0"`, `"2023-12-01"` |
| `endpoint` | Custom endpoint URL for the service | `"https://api.custom-llm.com/v1"` |
{{% alert title="Note" color="primary" %}}
The exact metadata fields supported depend on the specific conversation component implementation. Refer to the component's documentation for the complete list of supported metadata fields.
{{% /alert %}}
In addition to passing metadata in the request body, you can also pass metadata as URL query parameters without modifying the request payload. Here is the format:
- **Prefix**: All metadata parameters must be prefixed with `metadata.`
- **Format**: `?metadata.<field_name>=<value>`
- **Multiple parameters**: Separate with `&` (e.g., `?metadata.api_key=sk-123&metadata.model=gpt-4`)
Example of model override:
```bash
POST http://localhost:3500/v1.0-alpha2/conversation/openai/converse?metadata.model=sk-gpt-4-turbo
```
URL metadata parameters are merged with request body metadata, URL parameters take precedence if conflicts exist, and both override component configuration in the YAML file.
### Request content examples
#### Basic conversation
```json
REQUEST = {
"name": "openai",
"inputs": [{
"messages": [{
"of_user": {
"content": [{
"text": "What is Dapr?"
}]
}
}]
}],
"parameters": {},
"metadata": {}
}
curl -X POST http://localhost:3500/v1.0-alpha2/conversation/openai/converse \
-H "Content-Type: application/json" \
-d '{
"inputs": [
{
"messages": [
{
"of_user": {
"content": [
{
"text": "What is Dapr?"
}
]
}
}
]
}
],
"parameters": {},
"metadata": {}
}'
```
#### Conversation with tool calling
```json
{
"name": "openai",
"inputs": [{
"messages": [{
"of_user": {
"content": [{
"text": "What is the weather like in San Francisco in celsius?"
}]
}
}],
"scrub_pii": false
}],
"parameters": {
"max_tokens": {
"@type": "type.googleapis.com/google.protobuf.Int64Value",
"value": "100"
},
"model": {
"@type": "type.googleapis.com/google.protobuf.StringValue",
"value": "claude-3-5-sonnet-20240620"
}
},
"metadata": {
"api_key": "test-key",
"version": "1.0"
},
"scrub_pii": false,
"temperature": 0.7,
"tools": [{
"function": {
"name": "get_weather",
"description": "Get the current weather for a location",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA"
curl -X POST http://localhost:3500/v1.0-alpha2/conversation/openai/converse \
-H "Content-Type: application/json" \
-d '{
"inputs": [
{
"messages": [
{
"of_user": {
"content": [
{
"text": "What is the weather like in San Francisco in celsius?"
}
]
}
}
],
"scrub_pii": false
}
],
"parameters": {
"max_tokens": {
"@type": "type.googleapis.com/google.protobuf.Int64Value",
"value": "100"
},
"unit": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"description": "The temperature unit to use"
"model": {
"@type": "type.googleapis.com/google.protobuf.StringValue",
"value": "claude-3-5-sonnet-20240620"
}
},
"required": ["location"]
}
}
}],
"tool_choice": "auto"
}
"metadata": {
"api_key": "test-key",
"version": "1.0"
},
"scrub_pii": false,
"temperature": 0.7,
"tools": [
{
"function": {
"name": "get_weather",
"description": "Get the current weather for a location",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA"
},
"unit": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"description": "The temperature unit to use"
}
},
"required": ["location"]
}
}
}
],
"tool_choice": "auto"
}'
```
### HTTP response codes
@ -159,17 +216,19 @@ Code | Description
#### Basic conversation response
```json
RESPONSE = {
"outputs": [{
"choices": [{
"finish_reason": "stop",
"index": 0,
"message": {
"content": "Dapr is a distributed application runtime that makes it easy for developers to build resilient, stateless and stateful applications that run on the cloud and edge.",
"tool_calls": []
}
}]
}]
{
"outputs": [
{
"choices": [
{
"finishReason": "stop",
"message": {
"content": "Distributed application runtime, open-source."
}
}
]
}
]
}
```
@ -177,32 +236,29 @@ RESPONSE = {
```json
{
"outputs": [{
"choices": [{
"finish_reason": "tool_calls",
"index": 0,
"message": {
"content": null,
"tool_calls": [{
"id": "call_123",
"function": {
"name": "get_weather",
"arguments": "{\"location\": \"San Francisco, CA\", \"unit\": \"celsius\"}"
"outputs": [
{
"choices": [
{
"finishReason": "tool_calls",
"message": {
"toolCalls": [
{
"id": "call_Uwa41pG0UqGA2zp0Fec0KwOq",
"function": {
"name": "get_weather",
"arguments": "{\"location\":\"San Francisco, CA\",\"unit\":\"celsius\"}"
}
}
]
}
}]
}
}]
}]
}
]
}
]
}
```
### Tool choice options
The `tool_choice` is an optional parameter that controls how the model can use available tools:
- **`auto`**: The model can pick between generating a message or calling one or more tools (default when tools are present)
- **`required`**: Requires one or more functions to be called
- **`{tool_name}`**: Forces the model to call a specific tool by name
## Legacy Alpha1 API