vLLM
/
vllm-project.github.io
mirror of https://github.com/vllm-project/vllm-project.github.io.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Update _posts/2023-11-14-notes-vllm-vs-deepspeed.md 

Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
This commit is contained in:
Woosuk Kwon 2023-11-14 12:44:56 -08:00 committed by GitHub
parent 1e4fef6a7f
commit 96d1e57523
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
_posts/2023-11-14-notes-vllm-vs-deepspeed.md
View File

@ -14,7 +14,7 @@ author: "vLLM Team"
--- ---
The DeepSpeed team recently published [a blog post](https://github.com/microsoft/DeepSpeed/tree/master/blogs/deepspeed-fastgen) claiming 2x throughput improvement over vLLM, achieved by leveraging the Dynamic SplitFuse technique. The DeepSpeed team recently published [a blog post](https://github.com/microsoft/DeepSpeed/tree/master/blogs/deepspeed-fastgen) claiming 2x throughput improvement over vLLM, achieved by leveraging the Dynamic SplitFuse technique.
We are happy to see the technology advancements within the open-source community. We are happy to see the technology advancements from the open-source community.
In our blog today, we'll elucidate the specific scenarios where the Dynamic SplitFuse technique is advantageous, noting that these cases are relatively limited. In our blog today, we'll elucidate the specific scenarios where the Dynamic SplitFuse technique is advantageous, noting that these cases are relatively limited.
For the majority of workloads, vLLM is faster than (or performs comparably to) DeepSpeed MII. For the majority of workloads, vLLM is faster than (or performs comparably to) DeepSpeed MII.