vLLM
/
vllm-project.github.io
mirror of https://github.com/vllm-project/vllm-project.github.io.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Update _posts/2023-11-14-notes-vllm-vs-deepspeed.md 

Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
This commit is contained in:
Woosuk Kwon 2023-11-14 12:45:03 -08:00 committed by GitHub
parent 96d1e57523
commit e51ece8b31
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
_posts/2023-11-14-notes-vllm-vs-deepspeed.md
View File

@ -9,7 +9,7 @@ author: "vLLM Team"
- vLLM matches DeepSpeed's speed in common scenarios and surpasses it when handling longer outputs. - vLLM matches DeepSpeed's speed in common scenarios and surpasses it when handling longer outputs.
- DeepSpeed only outperforms vLLM in scenarios with long prompts and short outputs, due to its Dynamic SplitFuse optimization. This optimization is on vLLM’s roadmap. - DeepSpeed only outperforms vLLM in scenarios with long prompts and short outputs, due to its Dynamic SplitFuse optimization. This optimization is on vLLM’s roadmap.
- vLLM’s mission is to build the fastest and easiest-to-use open-source LLM inference and serving engine. It is Apache 2.0 licensed and driven by a community focus, offering extensive model and optimization support. - vLLM’s mission is to build the fastest and easiest-to-use open-source LLM inference and serving engine. It is Apache 2.0 and community-owned, offering extensive model and optimization support.
--- ---