discourse-ai/spec/lib
Sam 0c9466059c
DEV: improve artifact editing and eval system (#1130)
- Add non-contiguous search/replace support using ... syntax
- Add judge support for evaluating LLM outputs with ratings
- Improve error handling and reporting in eval runner
- Add full section replacement support without search blocks
- Add fabricators and specs for artifact diffing
- Track failed searches to improve debugging
- Add JS syntax validation for artifact versions in eval system
- Update prompt documentation with clear guidelines

* improve eval output

* move error handling

* llm as a judge

* fix spec

* small note on evals
2025-02-19 15:44:33 +11:00
..
completions FEATURE: track duration of AI calls (#1082) 2025-01-23 11:32:12 +11:00
discord/bot FEATURE: PDF support for rag pipeline (#1118) 2025-02-14 12:15:07 +11:00
discourse_automation FIX: AI Automation scripts were broken when using seeded models (#991) 2024-12-02 19:07:05 -03:00
inference FEATURE: configurable embeddings (#1049) 2025-01-21 12:23:19 -03:00
modules DEV: improve artifact editing and eval system (#1130) 2025-02-19 15:44:33 +11:00
utils DEV: improve artifact editing and eval system (#1130) 2025-02-19 15:44:33 +11:00
guardian_extensions_spec.rb FIX: Make summaries backfill job more resilient. (#1071) 2025-01-16 09:42:53 -03:00