vLLM
/
vllm
mirror of https://github.com/vllm-project/vllm.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
7,204 Commits 73 Branches 67 Tags 518 MiB
Commit Graph

65 Commits

Author SHA1 Message Date
Cody Yu d11bf435a0
[MISC] Consolidate cleanup() and refactor offline_inference_with_prefix.py (#9510) 2024-10-18 14:30:55 -07:00
Cyrus Leung 051eaf6db3
[Model] Add user-configurable task for models that support both generation and embedding (#9424) 2024-10-18 11:31:58 -07:00
Joe Runde de4008e2ab
[Bugfix][Core] Use torch.cuda.memory_stats() to profile peak memory usage (#9352) 
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
 2024-10-17 22:47:27 -04:00
Joe Runde 062c89e7c9
[Frontend][Core] Move guided decoding params into sampling params (#8252) 
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
 2024-10-01 09:34:25 +08:00
Cyrus Leung 3b00b9c26c
[Core] rename`PromptInputs` and `inputs` (#8876) 2024-09-26 20:35:15 -07:00
Simon Mo 4f1ba0844b
Revert "rename PromptInputs and inputs with backward compatibility (#8760) (#8810) 2024-09-25 10:36:26 -07:00
Cyrus Leung 28e1299e60
rename PromptInputs and inputs with backward compatibility (#8760) 2024-09-25 09:36:47 -07:00
Andy 2529d09b5a
[Frontend] Batch inference for llm.chat() API (#8648) 
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
 2024-09-24 09:44:11 -07:00
Jiaxin Shan db3bf7c991
[Core] Support load and unload LoRA in api server (#6566) 
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-09-05 18:10:33 -07:00
Cyrus Leung 855c262a6b
[Frontend] Multimodal support in offline chat (#8098) 2024-09-04 05:22:17 +00:00
youkaichao 7d9ffa2ae1
[misc][core] lazy import outlines (#7831) 2024-08-24 00:51:38 -07:00
Maximilien de Bayser e25fee57c2
[BugFix] Fix server crash on empty prompt (#7746) 
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
 2024-08-23 13:12:44 +00:00
nunjunj 3b19e39dc5
Chat method for offline llm (#5049) 
Co-authored-by: nunjunj <ray@g-3ff9f30f2ed650001.c.vllm-405802.internal>
Co-authored-by: nunjunj <ray@g-1df6075697c3f0001.c.vllm-405802.internal>
Co-authored-by: nunjunj <ray@g-c5a2c23abc49e0001.c.vllm-405802.internal>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-08-15 19:41:34 -07:00
Yihuan Bu 654bc5ca49
Support for guided decoding for offline LLM (#6878) 
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2024-08-04 03:12:09 +00:00
Cyrus Leung 9d47f64eb6
[CI/Build] [3/3] Reorganize entrypoints tests (#5966) 2024-06-30 12:58:49 +08:00