f6ac5cd0a8
* FEATURE: allow tuning of RAG generation - change chunking to be token based vs char based (which is more accurate) - allow control over overlap / tokens per chunk and conversation snippets inserted - UI to control new settings * improve ui a bit * fix various reindex issues * reduce concurrency * try ultra low queue ... concurrency 1 is too slow. |
||
|---|---|---|
| .. | ||
| all_mpnet_base_v2_tokenizer.rb | ||
| anthropic_tokenizer.rb | ||
| basic_tokenizer.rb | ||
| bert_tokenizer.rb | ||
| bge_large_en_tokenizer.rb | ||
| bge_m3_tokenizer.rb | ||
| llama2_tokenizer.rb | ||
| mixtral_tokenizer.rb | ||
| multilingual_e5_large_tokenizer.rb | ||
| open_ai_tokenizer.rb | ||