discourse-ai/spec/fabricators
Sam 0c9466059c
DEV: improve artifact editing and eval system (#1130)
- Add non-contiguous search/replace support using ... syntax
- Add judge support for evaluating LLM outputs with ratings
- Improve error handling and reporting in eval runner
- Add full section replacement support without search blocks
- Add fabricators and specs for artifact diffing
- Track failed searches to improve debugging
- Add JS syntax validation for artifact versions in eval system
- Update prompt documentation with clear guidelines

* improve eval output

* move error handling

* llm as a judge

* fix spec

* small note on evals
2025-02-19 15:44:33 +11:00
..
ai_artifact_fabricator.rb DEV: improve artifact editing and eval system (#1130) 2025-02-19 15:44:33 +11:00
ai_persona_fabricator.rb FEATURE: UI to update ai personas on admin page (#290) 2023-11-21 16:56:43 +11:00
ai_summary_fabricator.rb FIX: Make summaries backfill job more resilient. (#1071) 2025-01-16 09:42:53 -03:00
ai_tool_fabricator.rb FEATURE: Tool name validation (#842) 2025-02-07 14:34:47 +11:00
classification_result_fabricator.rb Migrate sentiment to a TEI backend (#886) 2024-11-04 09:14:34 -03:00
embedding_definition_fabricator.rb FEATURE: configurable embeddings (#1049) 2025-01-21 12:23:19 -03:00
llm_model_fabricator.rb FEATURE: first class support for OpenRouter (#1011) 2024-12-10 05:59:19 +11:00
llm_quota_fabricator.rb FEATURE: llm quotas (#1047) 2025-01-14 15:54:09 +11:00
llm_quota_usage_fabricator.rb FEATURE: llm quotas (#1047) 2025-01-14 15:54:09 +11:00
rag_document_fragment_fabricator.rb FEATURE: AI Bot RAG support. (#537) 2024-04-01 13:43:34 -03:00