vLLM
/
vllm-project.github.io
mirror of https://github.com/vllm-project/vllm-project.github.io.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

remove the in the author

This commit is contained in:
Zhuohan Li 2024-09-05 00:07:06 -07:00
parent 31cb4a5733
commit 73405ded17
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
_posts/2024-09-05-perf-update.md
View File

@ -1,7 +1,7 @@
--- ---
layout: post layout: post
title: "vLLM v0.6.0: 2.7x Throughput Improvement and 5x Latency Reduction" title: "vLLM v0.6.0: 2.7x Throughput Improvement and 5x Latency Reduction"
author: "The vLLM Team" author: "vLLM Team"
--- ---
**TL;DR:** vLLM achieves 2.7x higher throughput and 5x less TPOT (time per output token) on Llama 8B model, and 1.8x higher throughput and 2x less TPOT on Llama 70B model. **TL;DR:** vLLM achieves 2.7x higher throughput and 5x less TPOT (time per output token) on Llama 8B model, and 1.8x higher throughput and 2x less TPOT on Llama 70B model.