vLLM
/
vllm-project.github.io
mirror of https://github.com/vllm-project/vllm-project.github.io.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Add usage data section

This commit is contained in:
Michael Goin 2025-01-13 12:24:28 -07:00 committed by GitHub
parent 8ec5cdfb3e
commit 26b31f1550
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 13 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

13
_posts/2025-01-10-vllm-2024-wrapped-2025-vision.md
View File

@ -158,6 +158,19 @@ Through all these changes and challenges, our fundamental mission remains clear:
--- ---
## Usage Data Collection
The metrics and insights throughout this post are powered by vLLM's [usage system](https://github.com/vllm-project/vllm/blob/main/vllm/usage/usage_lib.py), which collects anonymized deployment data. Each vLLM instance generates a UUID and reports technical metrics including:
* Hardware specs (GPU count/type, CPU architecture, available memory)
* Model configuration (architecture, dtype, tensor parallelism degree)
* Runtime settings (quantization type, prefix caching enabled)
* Deployment context (cloud provider, platform, vLLM version)
This telemetry helps prioritize optimizations for common hardware configurations and identify which features need performance improvements. The data is collected locally in `~/.config/vllm/usage_stats.json`. Users can opt out by setting `VLLM_NO_USAGE_STATS=1`, `DO_NOT_TRACK=1`, or creating `~/.config/vllm/do_not_track`. The implementation details and full schema are available in our [usage stats documentation](https://docs.vllm.ai/en/latest/serving/usage_stats.html).
---
## Join the Journey ## Join the Journey
vLLM's 2024 journey demonstrates the transformative potential of open-source collaboration. With a clear vision for 2025, the project is poised to redefine AI inference, making it more accessible, scalable, and efficient. Whether through code contributions, attending [vLLM Office Hours](https://hubs.li/Q02TFDTT0), or adopting vLLM in production, every participant helps shape the future of this fast-moving project. vLLM's 2024 journey demonstrates the transformative potential of open-source collaboration. With a clear vision for 2025, the project is poised to redefine AI inference, making it more accessible, scalable, and efficient. Whether through code contributions, attending [vLLM Office Hours](https://hubs.li/Q02TFDTT0), or adopting vLLM in production, every participant helps shape the future of this fast-moving project.