47 KiB
Semantic conventions for generative AI events
Status: Development
- Event:
gen_ai.system.message - Event:
gen_ai.user.message - Event:
gen_ai.assistant.message - Event:
gen_ai.tool.message - Event:
gen_ai.choice - Custom events
- Examples
[!Warning]
Existing GenAI instrumentations that are using v1.36.0 of this document (or prior):
- SHOULD NOT change the version of the GenAI conventions that they emit by default. Conventions include, but are not limited to, attributes, metric, span and event names, span kind and unit of measure.
- SHOULD introduce an environment variable
OTEL_SEMCONV_STABILITY_OPT_INas a comma-separated list of category-specific values. The list of values includes:
gen_ai_latest_experimental- emit the latest experimental version of GenAI conventions (supported by the instrumentation) and do not emit the old one (v1.36.0 or prior).- The default behavior is to continue emitting whatever version of the GenAI conventions the instrumentation was emitting (1.36.0 or prior).
This transition plan will be updated to include stable version before the GenAI conventions are marked as stable.
GenAI instrumentations MAY capture user inputs sent to the model and responses received from it as events.
Note: Events are in-development and not yet available in some languages. Check spec-compliance matrix to see the implementation status in corresponding language.
Instrumentations MAY capture inputs and outputs if and only if application has enabled the collection of this data. This is for three primary reasons:
- Data privacy concerns. End users of GenAI applications may input sensitive information or personally identifiable information (PII) that they do not wish to be sent to a telemetry backend.
- Data size concerns. Although there is no specified limit to sizes, there are practical limitations in programming languages and telemetry systems. Some GenAI systems allow for extremely large context windows that end users may take full advantage of.
- Performance concerns. Sending large amounts of data to a telemetry backend may cause performance issues for the application.
Body fields that contain user input, model output, or other potentially sensitive and verbose data SHOULD NOT be captured by default.
Semantic conventions for individual systems which extend content events SHOULD document all additional body fields and specify whether they should be captured by default or need application to opt into capturing them.
Telemetry consumers SHOULD expect to receive unknown body fields.
Instrumentations SHOULD NOT capture undocumented body fields and MUST follow the documented defaults for known fields. Instrumentations MAY offer configuration options allowing to disable events or allowing to capture all fields.
Event: gen_ai.system.message
The event name MUST be gen_ai.system.message.
This event describes the system instructions passed to the GenAI model.
| Attribute | Type | Description | Examples | Requirement Level | Stability |
|---|---|---|---|---|---|
gen_ai.provider.name |
string | The Generative AI provider as identified by the client or server instrumentation. [1] | openai; gcp.gen_ai; gcp.vertex_ai |
Recommended |
[1] gen_ai.provider.name: The attribute SHOULD be set based on the instrumentation's best
knowledge and may differ from the actual model provider.
Multiple providers, including Azure OpenAI, Gemini, and AI hosting platforms are accessible using the OpenAI REST API and corresponding client libraries, but may proxy or host models from different providers.
The gen_ai.request.model, gen_ai.response.model, and server.address
attributes may help identify the actual system in use.
The gen_ai.provider.name attribute acts as a discriminator that
identifies the GenAI telemetry format flavor specific to that provider
within GenAI semantic conventions.
It SHOULD be set consistently with provider-specific attributes and signals.
For example, GenAI spans, metrics, and events related to AWS Bedrock
should have the gen_ai.provider.name set to aws.bedrock and include
applicable aws.bedrock.* attributes and are not expected to include
openai.* attributes.
gen_ai.provider.name has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.
| Value | Description | Stability |
|---|---|---|
anthropic |
Anthropic | |
aws.bedrock |
AWS Bedrock | |
azure.ai.inference |
Azure AI Inference | |
azure.ai.openai |
Azure OpenAI | |
cohere |
Cohere | |
deepseek |
DeepSeek | |
gcp.gemini |
Gemini [2] | |
gcp.gen_ai |
Any Google generative AI endpoint [3] | |
gcp.vertex_ai |
Vertex AI [4] | |
groq |
Groq | |
ibm.watsonx.ai |
IBM Watsonx AI | |
mistral_ai |
Mistral AI | |
openai |
OpenAI | |
perplexity |
Perplexity | |
x_ai |
xAI |
[2]: Used when accessing the 'generativelanguage.googleapis.com' endpoint. Also known as the AI Studio API.
[3]: May be used when specific backend is unknown.
[4]: Used when accessing the 'aiplatform.googleapis.com' endpoint.
Body fields:
⚠️ Body fields will be moved to complex attributes once the semantic convention tooling supports complex attributes (see #1870).
| Body Field | Type | Description | Examples | Requirement Level | Stability |
|---|---|---|---|---|---|
content |
undefined | The contents of the system message. | You're a helpful bot |
Opt-In |
|
role |
string | The actual role of the message author as passed in the message. | system; instruction |
Conditionally Required if available and not equal to system. |
Event: gen_ai.user.message
The event name MUST be gen_ai.user.message.
This event describes the user message passed to the GenAI model.
| Attribute | Type | Description | Examples | Requirement Level | Stability |
|---|---|---|---|---|---|
gen_ai.provider.name |
string | The Generative AI provider as identified by the client or server instrumentation. [1] | openai; gcp.gen_ai; gcp.vertex_ai |
Recommended |
[1] gen_ai.provider.name: The attribute SHOULD be set based on the instrumentation's best
knowledge and may differ from the actual model provider.
Multiple providers, including Azure OpenAI, Gemini, and AI hosting platforms are accessible using the OpenAI REST API and corresponding client libraries, but may proxy or host models from different providers.
The gen_ai.request.model, gen_ai.response.model, and server.address
attributes may help identify the actual system in use.
The gen_ai.provider.name attribute acts as a discriminator that
identifies the GenAI telemetry format flavor specific to that provider
within GenAI semantic conventions.
It SHOULD be set consistently with provider-specific attributes and signals.
For example, GenAI spans, metrics, and events related to AWS Bedrock
should have the gen_ai.provider.name set to aws.bedrock and include
applicable aws.bedrock.* attributes and are not expected to include
openai.* attributes.
gen_ai.provider.name has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.
| Value | Description | Stability |
|---|---|---|
anthropic |
Anthropic | |
aws.bedrock |
AWS Bedrock | |
azure.ai.inference |
Azure AI Inference | |
azure.ai.openai |
Azure OpenAI | |
cohere |
Cohere | |
deepseek |
DeepSeek | |
gcp.gemini |
Gemini [2] | |
gcp.gen_ai |
Any Google generative AI endpoint [3] | |
gcp.vertex_ai |
Vertex AI [4] | |
groq |
Groq | |
ibm.watsonx.ai |
IBM Watsonx AI | |
mistral_ai |
Mistral AI | |
openai |
OpenAI | |
perplexity |
Perplexity | |
x_ai |
xAI |
[2]: Used when accessing the 'generativelanguage.googleapis.com' endpoint. Also known as the AI Studio API.
[3]: May be used when specific backend is unknown.
[4]: Used when accessing the 'aiplatform.googleapis.com' endpoint.
Body fields:
⚠️ Body fields will be moved to complex attributes once the semantic convention tooling supports complex attributes (see #1870).
| Body Field | Type | Description | Examples | Requirement Level | Stability |
|---|---|---|---|---|---|
content |
undefined | The contents of the user message. | What's the weather in Paris? |
Opt-In |
|
role |
string | The actual role of the message author as passed in the message. | user; customer |
Conditionally Required if available and not equal to user. |
Event: gen_ai.assistant.message
The event name MUST be gen_ai.assistant.message.
This event describes the assistant message passed to GenAI system.
| Attribute | Type | Description | Examples | Requirement Level | Stability |
|---|---|---|---|---|---|
gen_ai.provider.name |
string | The Generative AI provider as identified by the client or server instrumentation. [1] | openai; gcp.gen_ai; gcp.vertex_ai |
Recommended |
[1] gen_ai.provider.name: The attribute SHOULD be set based on the instrumentation's best
knowledge and may differ from the actual model provider.
Multiple providers, including Azure OpenAI, Gemini, and AI hosting platforms are accessible using the OpenAI REST API and corresponding client libraries, but may proxy or host models from different providers.
The gen_ai.request.model, gen_ai.response.model, and server.address
attributes may help identify the actual system in use.
The gen_ai.provider.name attribute acts as a discriminator that
identifies the GenAI telemetry format flavor specific to that provider
within GenAI semantic conventions.
It SHOULD be set consistently with provider-specific attributes and signals.
For example, GenAI spans, metrics, and events related to AWS Bedrock
should have the gen_ai.provider.name set to aws.bedrock and include
applicable aws.bedrock.* attributes and are not expected to include
openai.* attributes.
gen_ai.provider.name has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.
| Value | Description | Stability |
|---|---|---|
anthropic |
Anthropic | |
aws.bedrock |
AWS Bedrock | |
azure.ai.inference |
Azure AI Inference | |
azure.ai.openai |
Azure OpenAI | |
cohere |
Cohere | |
deepseek |
DeepSeek | |
gcp.gemini |
Gemini [2] | |
gcp.gen_ai |
Any Google generative AI endpoint [3] | |
gcp.vertex_ai |
Vertex AI [4] | |
groq |
Groq | |
ibm.watsonx.ai |
IBM Watsonx AI | |
mistral_ai |
Mistral AI | |
openai |
OpenAI | |
perplexity |
Perplexity | |
x_ai |
xAI |
[2]: Used when accessing the 'generativelanguage.googleapis.com' endpoint. Also known as the AI Studio API.
[3]: May be used when specific backend is unknown.
[4]: Used when accessing the 'aiplatform.googleapis.com' endpoint.
Body fields:
⚠️ Body fields will be moved to complex attributes once the semantic convention tooling supports complex attributes (see #1870).
| Body Field | Type | Description | Examples | Requirement Level | Stability |
|---|---|---|---|---|---|
content |
undefined | The contents of the tool message. | The weather in Paris is rainy and overcast, with temperatures around 57°F |
Opt-In |
|
role |
string | The actual role of the message author as passed in the message. | assistant; bot |
Conditionally Required if available and not equal to assistant. |
|
tool_calls: |
map[] | The tool calls generated by the model, such as function calls. | Conditionally Required if available |
||
function: |
map | The function call. | Required |
||
arguments |
undefined | The arguments of the function as provided in the LLM response. [1] | {\"location\": \"Paris\"} |
Opt-In |
|
name |
string | The name of the function. | get_weather |
Required |
|
id |
string | The id of the tool call. | call_mszuSIzqtI65i1wAUOE8w5H4 |
Required |
|
type |
enum | The type of the tool. | function |
Required |
[1]: Models usually return arguments as a JSON string. In this case, it's RECOMMENDED to provide arguments as is without attempting to deserialize them. Semantic conventions for individual systems MAY specify a different type for arguments field.
type has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.
| Value | Description | Stability |
|---|---|---|
function |
Function |
Event: gen_ai.tool.message
The event name MUST be gen_ai.tool.message.
This event describes the response from a tool or function call passed to the GenAI model.
| Attribute | Type | Description | Examples | Requirement Level | Stability |
|---|---|---|---|---|---|
gen_ai.provider.name |
string | The Generative AI provider as identified by the client or server instrumentation. [1] | openai; gcp.gen_ai; gcp.vertex_ai |
Recommended |
[1] gen_ai.provider.name: The attribute SHOULD be set based on the instrumentation's best
knowledge and may differ from the actual model provider.
Multiple providers, including Azure OpenAI, Gemini, and AI hosting platforms are accessible using the OpenAI REST API and corresponding client libraries, but may proxy or host models from different providers.
The gen_ai.request.model, gen_ai.response.model, and server.address
attributes may help identify the actual system in use.
The gen_ai.provider.name attribute acts as a discriminator that
identifies the GenAI telemetry format flavor specific to that provider
within GenAI semantic conventions.
It SHOULD be set consistently with provider-specific attributes and signals.
For example, GenAI spans, metrics, and events related to AWS Bedrock
should have the gen_ai.provider.name set to aws.bedrock and include
applicable aws.bedrock.* attributes and are not expected to include
openai.* attributes.
gen_ai.provider.name has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.
| Value | Description | Stability |
|---|---|---|
anthropic |
Anthropic | |
aws.bedrock |
AWS Bedrock | |
azure.ai.inference |
Azure AI Inference | |
azure.ai.openai |
Azure OpenAI | |
cohere |
Cohere | |
deepseek |
DeepSeek | |
gcp.gemini |
Gemini [2] | |
gcp.gen_ai |
Any Google generative AI endpoint [3] | |
gcp.vertex_ai |
Vertex AI [4] | |
groq |
Groq | |
ibm.watsonx.ai |
IBM Watsonx AI | |
mistral_ai |
Mistral AI | |
openai |
OpenAI | |
perplexity |
Perplexity | |
x_ai |
xAI |
[2]: Used when accessing the 'generativelanguage.googleapis.com' endpoint. Also known as the AI Studio API.
[3]: May be used when specific backend is unknown.
[4]: Used when accessing the 'aiplatform.googleapis.com' endpoint.
Body fields:
⚠️ Body fields will be moved to complex attributes once the semantic convention tooling supports complex attributes (see #1870).
| Body Field | Type | Description | Examples | Requirement Level | Stability |
|---|---|---|---|---|---|
content |
undefined | The contents of the tool message. | rainy, 57°F |
Opt-In |
|
id |
string | Tool call id that this message is responding to. | call_mszuSIzqtI65i1wAUOE8w5H4 |
Required |
|
role |
string | The actual role of the message author as passed in the message. | tool; function |
Conditionally Required if available and not equal to tool. |
Event: gen_ai.choice
The event name MUST be gen_ai.choice.
This event describes the Gen AI response message.
| Attribute | Type | Description | Examples | Requirement Level | Stability |
|---|---|---|---|---|---|
gen_ai.provider.name |
string | The Generative AI provider as identified by the client or server instrumentation. [1] | openai; gcp.gen_ai; gcp.vertex_ai |
Recommended |
[1] gen_ai.provider.name: The attribute SHOULD be set based on the instrumentation's best
knowledge and may differ from the actual model provider.
Multiple providers, including Azure OpenAI, Gemini, and AI hosting platforms are accessible using the OpenAI REST API and corresponding client libraries, but may proxy or host models from different providers.
The gen_ai.request.model, gen_ai.response.model, and server.address
attributes may help identify the actual system in use.
The gen_ai.provider.name attribute acts as a discriminator that
identifies the GenAI telemetry format flavor specific to that provider
within GenAI semantic conventions.
It SHOULD be set consistently with provider-specific attributes and signals.
For example, GenAI spans, metrics, and events related to AWS Bedrock
should have the gen_ai.provider.name set to aws.bedrock and include
applicable aws.bedrock.* attributes and are not expected to include
openai.* attributes.
gen_ai.provider.name has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.
| Value | Description | Stability |
|---|---|---|
anthropic |
Anthropic | |
aws.bedrock |
AWS Bedrock | |
azure.ai.inference |
Azure AI Inference | |
azure.ai.openai |
Azure OpenAI | |
cohere |
Cohere | |
deepseek |
DeepSeek | |
gcp.gemini |
Gemini [2] | |
gcp.gen_ai |
Any Google generative AI endpoint [3] | |
gcp.vertex_ai |
Vertex AI [4] | |
groq |
Groq | |
ibm.watsonx.ai |
IBM Watsonx AI | |
mistral_ai |
Mistral AI | |
openai |
OpenAI | |
perplexity |
Perplexity | |
x_ai |
xAI |
[2]: Used when accessing the 'generativelanguage.googleapis.com' endpoint. Also known as the AI Studio API.
[3]: May be used when specific backend is unknown.
[4]: Used when accessing the 'aiplatform.googleapis.com' endpoint.
Body fields:
⚠️ Body fields will be moved to complex attributes once the semantic convention tooling supports complex attributes (see #1870).
| Body Field | Type | Description | Examples | Requirement Level | Stability |
|---|---|---|---|---|---|
finish_reason |
enum | The reason the model stopped generating tokens. | stop; tool_calls; content_filter |
Required |
|
index |
int | The index of the choice in the list of choices. | 0; 1 |
Required |
|
message: |
map | GenAI response message. | Recommended |
||
content |
undefined | The contents of the assistant message. | The weather in Paris is rainy and overcast, with temperatures around 57°F |
Opt-In |
|
role |
string | The actual role of the message author as passed in the message. | assistant; bot |
Conditionally Required if available and not equal to assistant. |
|
tool_calls: |
map[] | The tool calls generated by the model, such as function calls. | Conditionally Required if available |
||
function: |
map | The function that the model called. | Required |
||
arguments |
undefined | The arguments of the function as provided in the LLM response. [1] | {\"location\": \"Paris\"} |
Opt-In |
|
name |
string | The name of the function. | get_weather |
Required |
|
id |
string | The id of the tool call. | call_mszuSIzqtI65i1wAUOE8w5H4 |
Required |
|
type |
enum | The type of the tool. | function |
Required |
[1]: Models usually return arguments as a JSON string. In this case, it's RECOMMENDED to provide arguments as is without attempting to deserialize them. Semantic conventions for individual systems MAY specify a different type for arguments field.
finish_reason has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.
| Value | Description | Stability |
|---|---|---|
content_filter |
Content Filter | |
error |
Error | |
length |
Length | |
stop |
Stop | |
tool_calls |
Tool Calls |
type has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.
| Value | Description | Stability |
|---|---|---|
function |
Function |
Custom events
System-specific events that are not covered in this document SHOULD be documented in corresponding Semantic Conventions extensions and
SHOULD follow {gen_ai.provider.name}.* naming pattern.
Examples
Chat completion
This is an example of telemetry generated for a chat completion call with system and user messages.
%%{init:
{
"sequence": { "messageAlign": "left", "htmlLabels":true },
"themeVariables": { "noteBkgColor" : "green", "noteTextColor": "black", "activationBkgColor": "green", "htmlLabels":true }
}
}%%
sequenceDiagram
participant A as Application
participant I as Instrumented Client
participant M as Model
A->>+I: #U+200D
I->>M: gen_ai.system.message: You are a helpful bot<br/>gen_ai.user.message: Tell me a joke about OpenTelemetry
Note left of I: GenAI Client span
I-->M: gen_ai.choice: Why did the developer bring OpenTelemetry to the party? Because it always knows how to trace the fun!
I-->>-A: #U+200D
GenAI Client span:
| Attribute name | Value |
|---|---|
| Span name | "chat gpt-4" |
gen_ai.provider.name |
"openai" |
gen_ai.request.model |
"gpt-4" |
gen_ai.request.max_tokens |
200 |
gen_ai.request.top_p |
1.0 |
gen_ai.response.id |
"chatcmpl-9J3uIL87gldCFtiIbyaOvTeYBRA3l" |
gen_ai.response.model |
"gpt-4-0613" |
gen_ai.usage.output_tokens |
47 |
gen_ai.usage.input_tokens |
52 |
gen_ai.response.finish_reasons |
["stop"] |
Events:
-
gen_ai.system.messageProperty Value gen_ai.provider.name"openai"Event body (with content enabled) {"content": "You're a helpful bot"} -
gen_ai.user.messageProperty Value gen_ai.provider.name"openai"Event body (with content enabled) {"content":"Tell me a joke about OpenTelemetry"} -
gen_ai.choiceProperty Value gen_ai.provider.name"openai"Event body (with content enabled) {"index":0,"finish_reason":"stop","message":{"content":"Why did the developer bring OpenTelemetry to the party? Because it always knows how to trace the fun!"}}Event body (without content) {"index":0,"finish_reason":"stop","message":{}}
Tools
This is an example of telemetry generated for a chat completion call with user message and function definition that results in a model requesting application to call provided function. Application executes a function and requests another completion now with the tool response.
%%{init:
{
"sequence": { "messageAlign": "left", "htmlLabels":true },
"themeVariables": { "noteBkgColor" : "green", "noteTextColor": "black", "activationBkgColor": "green", "htmlLabels":true }
}
}%%
sequenceDiagram
participant A as Application
participant I as Instrumented Client
participant M as Model
A->>+I: #U+200D
I->>M: gen_ai.user.message: What's the weather in Paris?
Note left of I: GenAI Client span 1
I-->M: gen_ai.choice: Call to the get_weather tool with Paris as the location argument.
I-->>-A: #U+200D
A -->> A: parse tool parameters<br/>execute tool<br/>update chat history
A->>+I: #U+200D
I->>M: gen_ai.user.message: What's the weather in Paris?<br/>gen_ai.assistant.message: get_weather tool call<br/>gen_ai.tool.message: rainy, 57°F
Note left of I: GenAI Client span 2
I-->M: gen_ai.choice: The weather in Paris is rainy and overcast, with temperatures around 57°F
I-->>-A: #U+200D
Here's the telemetry generated for each step in this scenario:
GenAI Client span 1:
| Attribute name | Value |
|---|---|
| Span name | "chat gpt-4" |
gen_ai.provider.name |
"openai" |
gen_ai.request.model |
"gpt-4" |
gen_ai.request.max_tokens |
200 |
gen_ai.request.top_p |
1.0 |
gen_ai.response.id |
"chatcmpl-9J3uIL87gldCFtiIbyaOvTeYBRA3l" |
gen_ai.response.model |
"gpt-4-0613" |
gen_ai.usage.output_tokens |
17 |
gen_ai.usage.input_tokens |
47 |
gen_ai.response.finish_reasons |
["tool_calls"] |
Events:
All the following events are parented to the GenAI chat span 1.
-
gen_ai.user.message(not reported when capturing content is disabled)Property Value gen_ai.provider.name"openai"Event body {"content":"What's the weather in Paris?"} -
gen_ai.choiceProperty Value gen_ai.provider.name"openai"Event body (with content) {"index":0,"finish_reason":"tool_calls","message":{"tool_calls":[{"id":"call_VSPygqKTWdrhaFErNvMV18Yl","function":{"name":"get_weather","arguments":"{\"location\":\"Paris\"}"},"type":"function"}]}Event body (without content) {"index":0,"finish_reason":"tool_calls","message":{"tool_calls":[{"id":"call_VSPygqKTWdrhaFErNvMV18Yl","function":{"name":"get_weather"},"type":"function"}]}
GenAI Client span 2:
| Attribute name | Value |
|---|---|
| Span name | "chat gpt-4" |
gen_ai.provider.name |
"openai" |
gen_ai.request.model |
"gpt-4" |
gen_ai.request.max_tokens |
200 |
gen_ai.request.top_p |
1.0 |
gen_ai.response.id |
"chatcmpl-call_VSPygqKTWdrhaFErNvMV18Yl" |
gen_ai.response.model |
"gpt-4-0613" |
gen_ai.usage.output_tokens |
52 |
gen_ai.usage.input_tokens |
47 |
gen_ai.response.finish_reasons |
["stop"] |
Events:
All the following events are parented to the GenAI chat span 2.
In this example, the event content matches the original messages, but applications may also drop messages or change their content.
-
gen_ai.user.messageProperty Value gen_ai.provider.name"openai"Event body {"content":"What's the weather in Paris?"} -
gen_ai.assistant.messageProperty Value gen_ai.provider.name"openai"Event body (content enabled) {"tool_calls":[{"id":"call_VSPygqKTWdrhaFErNvMV18Yl","function":{"name":"get_weather","arguments":"{\"location\":\"Paris\"}"},"type":"function"}]}Event body (content not enabled) {"tool_calls":[{"id":"call_VSPygqKTWdrhaFErNvMV18Yl","function":{"name":"get_weather"},"type":"function"}]} -
gen_ai.tool.messageProperty Value gen_ai.provider.name"openai"Event body (content enabled) {"content":"rainy, 57°F","id":"call_VSPygqKTWdrhaFErNvMV18Yl"}Event body (content not enabled) {"id":"call_VSPygqKTWdrhaFErNvMV18Yl"} -
gen_ai.choiceProperty Value gen_ai.provider.name"openai"Event body (content enabled) {"index":0,"finish_reason":"stop","message":{"content":"The weather in Paris is rainy and overcast, with temperatures around 57°F"}}Event body (content not enabled) {"index":0,"finish_reason":"stop","message":{}}
Chat completion with multiple choices
This example covers the scenario when user requests model to generate two completions for the same prompt :
%%{init:
{
"sequence": { "messageAlign": "left", "htmlLabels":true },
"themeVariables": { "noteBkgColor" : "green", "noteTextColor": "black", "activationBkgColor": "green", "htmlLabels":true }
}
}%%
sequenceDiagram
participant A as Application
participant I as Instrumented Client
participant M as Model
A->>+I: #U+200D
I->>M: gen_ai.system.message - "You are a helpful bot"<br/>gen_ai.user.message - "Tell me a joke about OpenTelemetry"
Note left of I: GenAI Client span
I-->M: gen_ai.choice - Why did the developer bring OpenTelemetry to the party? Because it always knows how to trace the fun!<br/>gen_ai.choice - Why did OpenTelemetry get promoted? It had great span of control!
I-->>-A: #U+200D
GenAI Client Span:
| Attribute name | Value |
|---|---|
| Span name | "chat gpt-4" |
gen_ai.provider.name |
"openai" |
gen_ai.request.model |
"gpt-4" |
gen_ai.request.max_tokens |
200 |
gen_ai.request.top_p |
1.0 |
gen_ai.response.id |
"chatcmpl-9J3uIL87gldCFtiIbyaOvTeYBRA3l" |
gen_ai.response.model |
"gpt-4-0613" |
gen_ai.usage.output_tokens |
77 |
gen_ai.usage.input_tokens |
52 |
gen_ai.response.finish_reasons |
["stop", "stop"] |
Events:
All events are parented to the GenAI chat span above.
-
gen_ai.system.message: the same as in the Chat Completion example -
gen_ai.user.message: the same as in the Chat Completion example -
gen_ai.choiceProperty Value gen_ai.provider.name"openai"Event body (content enabled) {"index":0,"finish_reason":"stop","message":{"content":"Why did the developer bring OpenTelemetry to the party? Because it always knows how to trace the fun!"}} -
gen_ai.choiceProperty Value gen_ai.provider.name"openai"Event body (content enabled) {"index":1,"finish_reason":"stop","message":{"content":"Why did OpenTelemetry get promoted? It had great span of control!"}}