discourse
/
discourse-ai
mirror of https://github.com/discourse/discourse-ai.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

DEV: Switch translations out of structured output as it returns only a single value (#1503) 

Since translations only require a single key back, there is little point in using structured output. This PR also includes some prompt updates dealing with quotes, details, and code.

Related: #1502

This does mean reverting discourse/discourse-translator#257, but we can see how it goes.
This commit is contained in:
Natalie Tay 2025-07-16 13:38:00 +08:00 committed by GitHub
parent 06743d1939
commit ad6a8cb812
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
6 changed files with 123 additions and 135 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

35
lib/personas/locale_detector.rb
View File

@ -34,43 +34,22 @@ module DiscourseAi
5. Avoid using `und` and prefer `en` over `en-US` or `en-GB` unless the text specifically indicates a regional variant.
6. Format your response as a JSON object with a single key "locale" and the value as the language code.
Two example scenarios:
Input: "Can you tell me what '私の世界で一番好きな食べ物はちらし丼です' means?"
Output: "en"
Your output should be in the following format:
<output>
{"locale": "xx"}
</output>
Where "xx" is replaced by the appropriate language code.
Input: [quote]\nNon smettere mai di credere nella bellezza dei tuoi sogni. Anche quando tutto sembra perduto, c'è sempre una luce che aspetta di essere trovata.\nOgni passo, anche il più piccolo, ti avvicina a ciò che desideri. La forza che cerchi è già dentro di te.\n[/quote]\n¿Cuál es el mensaje principal de esta cita?
Output: "es"
Important: Base your analysis solely on the provided text. Do not use any external information or make assumptions about the text's origin or context beyond what is explicitly provided.
PROMPT
end
def response_format
[{ "key" => "locale", "type" => "string" }]
Your response must be a language code, and nothing else.
PROMPT
end
def temperature
0
end
def examples
spanish = <<~MARKDOWN
[quote]
Non smettere mai di credere nella bellezza dei tuoi sogni. Anche quando tutto sembra perduto, c'è sempre una luce che aspetta di essere trovata.
Ogni passo, anche il più piccolo, ti avvicina a ciò che desideri. La forza che cerchi è già dentro di te.
[/quote]
¿Cuál es el mensaje principal de esta cita?
MARKDOWN
[
["Can you tell me what '私の世界で一番好きな食べ物はちらし丼です' means?", { locale: "en" }.to_json],
[spanish, { locale: "es" }.to_json],
]
end
end
end
end

78
lib/personas/post_raw_translator.rb
View File

@ -8,62 +8,74 @@ module DiscourseAi
end
def system_prompt
examples = [
{
input: {
content:
"**Heathrow fechado**: Suspensão de voos deve continuar nos próximos dias, afirma gerente do aeroporto de Londres\n\n[details=Do site da BBC]\n\nA British Airways estimou que 85% de seus voos planejados seriam realizados no sábado, mas com atrasos em todos os voos. Às 7h GMT, a maioria das partidas havia ocorrido conforme o esperado, mas, das chegadas, nove dos primeiros 20 voos programados para aterrissar foram cancelados.\n\n[/details]",
target_locale: "en",
}.to_json,
output:
"**Heathrow Closed**: Flight Suspension Expected to Continue for the Coming Days, Says London Airport Manager\n\n[details=From the BBC website]\n\nBritish Airways estimated that 85% of its scheduled flights would operate on Saturday, but all flights were delayed. By 7:00 a.m. GMT, most departures had proceeded as expected, but of the arrivals, nine of the first 20 flights scheduled to land were canceled.\n\n[/details]",
},
{
input: {
content:
"[quote] What does the new update include? [/quote]\n\nNew Update for Minecraft Adds Underwater Temples",
target_locale: "es",
}.to_json,
output:
"[quote]¿Qué incluye la nueva actualización?[/quote]\n\nNueva actualización para Minecraft añade templos submarinos",
},
{
input: {
content:
"There has been an error in my update\n\n```ruby\napi_key = \"a quick brown fox\"\nfetch(\"https://api.example.com/data\", headers: { 'Authorization' => api_key })\n```\n\nPlease help me fix it.",
target_locale: "ja",
}.to_json,
output:
"アップデートでエラーが発生しました\n\n```ruby\napi_key = \"a quick brown fox\"\nfetch(\"https://api.example.com/data\", headers: { 'Authorization' => api_key })\n```\n\n修正にご協力ください。\"",
},
]
<<~PROMPT.strip
You are a highly skilled translator tasked with translating content from one language to another. Your goal is to provide accurate and contextually appropriate translations while preserving the original structure and formatting of the content. Follow these instructions strictly:
1. Preserve Markdown elements, HTML elements, or newlines. Text must be translated without altering the original formatting.
2. Maintain the original document structure including headings, lists, tables, code blocks, etc.
3. Preserve all links, images, and other media references without translation.
4. For technical terminology:
4. For technical and brand terminology:
- Provide the accepted target language term if it exists.
- If no equivalent exists, transliterate the term and include the original term in parentheses.
5. For ambiguous terms or phrases, choose the most contextually appropriate translation.
6. Ensure the translation only contains the original language and the target language.
Follow these instructions on what NOT to do:
7. Do not translate code snippets or programming language names, but ensure that any comments within the code are translated.
7. Do not translate code snippets or programming language names, but ensure that any comments within the code are translated. Code can be represented in ``` or in single ` backticks or in <code> HTML tags.
8. Do not add any content besides the translation.
9. Do not add unnecessary newlines.
Here are three examples of correct translations:
Input: #{examples[0][:input]}
Output: #{examples[0][:output]}
Input: #{examples[1][:input]}
Output: #{examples[1][:output]}
Input: #{examples[2][:input]}
Output: #{examples[2][:output]}
The text to translate will be provided in JSON format with the following structure:
{"content": "Text to translate", "target_locale": "Target language code"}
Output your translation in the following JSON format:
{"translation": "Your translated text here"}
You are being consumed via an API. Only return the translated text in the specified JSON format. Do not include any additional information or explanations.
You are being consumed via an API that expects only the translated text. Only return the translated text in the correct language. Do not add questions or explanations.
PROMPT
end
def response_format
[{ "key" => "translation", "type" => "string" }]
end
def temperature
0.3
end
def examples
[
[
{
content:
"**Heathrow fechado**: paralisação de voos deve continuar nos próximos dias, diz gestora do aeroporto de *Londres*",
target_locale: "en",
}.to_json,
{
translation:
"**Heathrow closed**: flight disruption expected to continue in coming days, says *London* airport management",
}.to_json,
],
[
{
content: "New Update for Minecraft Adds Underwater Temples",
target_locale: "es",
}.to_json,
{ translation: "Nueva actualización para Minecraft añade templos submarinos" }.to_json,
],
]
end
end
end
end

51
lib/personas/short_text_translator.rb
View File

@ -8,6 +8,16 @@ module DiscourseAi
end
def system_prompt
examples = [
{ input: { content: "Japan", target_locale: "es" }.to_json, output: "Japón" },
{ input: { content: "Cats and Dogs", target_locale: "zh_CN" }.to_json, output: "猫和狗" },
{
input: { content: "Q&A", target_locale: "pt" }.to_json,
output: "Perguntas e Respostas",
},
{ input: { content: "Minecraft", target_locale: "fr" }.to_json, output: "Minecraft" },
]
<<~PROMPT.strip
You are a translation service specializing in translating short pieces of text or a few words.
These words may be things like a name, description, or title. Adhere to the following guidelines:
@ -17,41 +27,30 @@ module DiscourseAi
3. Translation maintains the original meaning
4. Preserve any Markdown, HTML elements, links, parenthesis, or newlines
Here are four examples of correct translations:
Input: #{examples[0][:input]}
Output: #{examples[0][:output]}
Input: #{examples[1][:input]}
Output: #{examples[1][:output]}
Input: #{examples[2][:input]}
Output: #{examples[2][:output]}
Input: #{examples[3][:input]}
Output: #{examples[3][:output]}
The text to translate will be provided in JSON format with the following structure:
{"content": "Text to translate", "target_locale": "Target language code"}
Provide your translation in the following JSON format:
{"translation": "target_locale translation here"}
Translate the text now and provide your answer in the specified JSON format.
You are being consumed via an API that expects only the translated text. Only return the translated text in the correct language. Do not add questions or explanations.
PROMPT
end
def response_format
[{ "key" => "translation", "type" => "string" }]
end
def temperature
0.3
end
def examples
[
[{ content: "Japan", target_locale: "es" }.to_json, { translation: "Japón" }.to_json],
[
{ content: "Cats and Dogs", target_locale: "zh_CN" }.to_json,
{ translation: "猫和狗" }.to_json,
],
[
{ content: "Q&A", target_locale: "pt" }.to_json,
{ translation: "Perguntas e Respostas" }.to_json,
],
[
{ content: "Minecraft", target_locale: "fr" }.to_json,
{ translation: "Minecraft" }.to_json,
],
]
end
end
end
end

76
lib/personas/topic_title_translator.rb
View File

@ -8,6 +8,32 @@ module DiscourseAi
end
def system_prompt
examples = [
{
input: {
content: "New Update for Minecraft Adds Underwater Temples",
target_locale: "es",
}.to_json,
output: "Nueva actualización para Minecraft añade templos submarinos",
},
{
input: {
content: "Toyota announces revolutionary battery technology",
target_locale: "fr",
}.to_json,
output: "Toyota annonce une technologie de batteries révolutionnaire",
},
{
input: {
content:
"Heathrow fechado: paralisação de voos deve continuar nos próximos dias, diz gestora do aeroporto de Londres",
target_locale: "en",
}.to_json,
output:
"Heathrow closed: flight disruption expected to continue in coming days, says London airport management",
},
]
<<~PROMPT.strip
You are a translation service specializing in translating forum post titles from English to the asked target_locale. Your task is to provide accurate and contextually appropriate translations while adhering to the following guidelines:
@ -24,53 +50,27 @@ module DiscourseAi
4. Adjust the translation if necessary to keep the length similar to the original title.
5. Review your translation for accuracy and naturalness in the target_locale.
Here are three examples of correct translations:
Input: #{examples[0][:input]}
Output: #{examples[0][:output]}
Input: #{examples[1][:input]}
Output: #{examples[1][:output]}
Input: #{examples[2][:input]}
Output: #{examples[2][:output]}
The text to translate will be provided in JSON format with the following structure:
{"content": "Title to translate", "target_locale": "Target language code"}
Provide your translation in the following JSON format:
{"translation": "Your target_locale translation here"}
Remember to keep proper nouns like "Minecraft" and "Toyota" in their original form. Translate the title now and provide your answer in the specified JSON format.
Remember to keep proper nouns like "Minecraft" and "Toyota" in their original form. You are being consumed via an API that expects only the translated title. Only return the translated title in the correct language. Do not add questions or explanations.
PROMPT
end
def response_format
[{ "key" => "translation", "type" => "string" }]
end
def temperature
0.3
end
def examples
[
[
{
content: "New Update for Minecraft Adds Underwater Temples",
target_locale: "es",
}.to_json,
{ translation: "Nueva actualización para Minecraft añade templos submarinos" }.to_json,
],
[
{
content: "Toyota announces revolutionary battery technology",
target_locale: "fr",
}.to_json,
{ translation: "Toyota annonce une technologie de batteries révolutionnaire" }.to_json,
],
[
{
content:
"Heathrow fechado: paralisação de voos deve continuar nos próximos dias, diz gestora do aeroporto de Londres",
target_locale: "en",
}.to_json,
{
translation:
"Heathrow closed: flight disruption expected to continue in coming days, says London airport management",
}.to_json,
],
]
end
end
end
end

9
lib/translation/base_translator.rb
View File

@ -47,12 +47,9 @@ module DiscourseAi
post: @post,
)
structured_output = nil
bot.reply(context) do |partial, _, type|
structured_output = partial if type == :structured_output
end
structured_output&.read_buffered_property(:translation)
result = +""
bot.reply(context) { |partial| result << partial }
result
end
def persona_setting

9
lib/translation/language_detector.rb
View File

@ -42,11 +42,12 @@ module DiscourseAi
post: @post,
)
structured_output = nil
bot.reply(context) do |partial, _, type|
structured_output = partial if type == :structured_output
result = +""
bot.reply(context) do |partial|
next if partial.strip.blank?
result << partial
end
structured_output&.read_buffered_property(:locale) || []
result
end
end
end