vLLM
/
vllm
mirror of https://github.com/vllm-project/vllm.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
7,421 Commits 73 Branches 67 Tags 518 MiB
Commit Graph

7421 Commits

Author SHA1 Message Date
Woosuk Kwon 515a1915d7 update 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-07-01 19:55:37 +00:00
Woosuk Kwon 7020f662df Merge branch 'main' into woosuk/async-sched 2025-07-01 19:54:26 +00:00
Woosuk Kwon 7f280d69c9
[Optimization] Cache sampled token ids in model runner (#20291) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-07-01 11:01:31 -07:00
TJian 02cabff207
[V1] [ROCm] Enable EP with AITER Fused MoE (#20270) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2025-07-01 16:48:30 +00:00
Shintarou Okada 3d19d47d91
[Frontend] Expand tools even if tool_choice="none" (#17177) 
Signed-off-by: okada shintarou <okada@preferred.jp>
 2025-07-01 12:47:38 -04:00
Woosuk Kwon 8acb4badee
[CUDA graphs] Enable full cuda graphs with FA3 AoT scheduling (#20301) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-07-01 09:07:36 -07:00
Nicolò Lucchesi 314af8617c
[Docs] Update transcriptions API to use openai client with `stream=True` (#20271) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-07-01 15:47:13 +00:00
Woosuk Kwon 0e96cc9b7e
[Misc] Minor refactoring for scheduler (#20299) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-07-01 07:55:32 -07:00
aiyiwang2025 ecad851cbd
[Model]Add Tencent HunYuanMoEV1 Model Support (#20114) 
Signed-off-by: aiyiwang <aiyiwang@tencent.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: quinnrong <quinnrong@tencent.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-07-01 07:28:13 -07:00
Yuxuan Zhang ed70f3c64f
Add GLM4.1V model (Draft) (#19331) 
Signed-off-by: zRzRzRzRzRzRzR <2448370773@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-07-01 12:48:26 +00:00
Nicolò Lucchesi 650d5dbd04
[Misc] Minor refactor of NIXL background handshake (#20068) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-07-01 12:40:14 +01:00
Kyle Sayers 9025a9a705
[Quant] [Bugfix] Fix quantization config matching with `hf_to_vllm_mapper` (#20046) 2025-07-01 19:20:34 +09:00
Lionel Villard c05596f1a3
[Perf] Validate @config in pre-commit instead of dynamically (#20200) 
Signed-off-by: Lionel Villard <villard@us.ibm.com>
 2025-07-01 05:10:28 -04:00
Reid 787b13389e
[doc] fix the incorrect logo in dark mode (#20289) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
 2025-07-01 08:18:09 +00:00
TY-AMD 96453cfa83
[BugFix][V1][ROCm] Triton MLA uses V0 backend on V1 engine (#19067) 
Signed-off-by: Tianyuan Wu <Tianyuan.Wu@amd.com>
 2025-07-01 16:12:19 +08:00
Kebe b1c1fe35a5
[Misc] remove redundant char (#20287) 
Signed-off-by: Kebe <mail@kebe7jun.com>
 2025-07-01 15:33:22 +08:00
Varun Sundar Rabindranath 08d81f1014
[Bugfix] Fix deepep tests (#20288) 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
 2025-07-01 15:29:08 +08:00
Li, Jiang 6cc1e7d96d
[CPU] Update custom ops for the CPU backend (#20255) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-07-01 07:25:03 +00:00
czhu-cohere 9909726d2a
Enable ZP Support for Machete (#20268) 
Signed-off-by: czhu-cohere <conway.zhu@cohere.com>
 2025-07-01 07:12:20 +00:00
Prashant Gupta 22e9d42040
[Misc] add xgrammar for arm64 (#18359) 
Signed-off-by: Prashant Gupta <prashantgupta@us.ibm.com>
 2025-07-01 07:02:20 +00:00
Richard Barnes 86debab54c
Fix `numel()` downcast in vllm/csrc/moe/moe_align_sum_kernels.cu +2 (#17082) 
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-07-01 06:48:10 +00:00
Michael Goin be250bbc67
[V1] Only print cudagraph tqdm on rank 0 with `is_global_first_rank` (#19516) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-07-01 06:02:09 +00:00
Alex Kogan 27949354fa
[Feature] A calibration-free RTN-based quantization for accurate and accelerated INT4/INT8 inference (#18768) 
Signed-off-by: Alex Kogan <alex.kogan@oracle.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-07-01 05:44:38 +00:00
Ernest Wong bd5038af07
[Doc] add config and troubleshooting guide for NCCL & GPUDirect RDMA (#15897) 
Signed-off-by: Ernest Wong <chwong719@gmail.com>
 2025-06-30 21:44:39 -07:00
Chendi.Xue a2f14dc8f9
[CI][Intel Gaudi][vllm-Plugin]Add CI for hpu-plugin-v1-test (#20196) 
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
 2025-07-01 04:17:07 +00:00
Kuntai Du 92ee7baaf9
[Example] add one-click runnable example for P2P NCCL XpYd (#20246) 
Signed-off-by: KuntaiDu <kuntai@uchicago.edu>
 2025-06-30 21:03:55 -07:00
Woosuk Kwon 7151f92241
[Misc] Fix spec decode example (#20296) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-06-30 21:01:48 -07:00
fyuan1316 e28533a16f
[Bugfix] Fix include prompt in stream response when echo=true (#15233) 
Signed-off-by: Yuan Fang <yuanfang@alauda.io>
 2025-07-01 01:30:14 +00:00
Luka Govedič 6d42ce8315
[CLI] Improve CLI arg parsing for `-O`/`--compilation-config` (#20156) 
Signed-off-by: luka <luka@neuralmagic.com>
 2025-07-01 01:03:13 +00:00
Zhonghua Deng ded1fb635b
[Bugfix][V1][P/D]Fix the issue of occasional garbled output for P2pNcclConnector (#20263) 
Signed-off-by: Abatom <abzhonghua@gmail.com>
 2025-06-30 16:45:14 -07:00
Wentao Ye 97d9524fe9
[Refactor] Remove useless pdb comment (#20266) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-06-30 18:15:24 +00:00
Kyle Sayers d8cf819a9a
[Core] [Bugfix] [Multimodal] Fix multimodal profiling and generation for SFT/PTQed models (#20058) 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
 2025-06-30 17:26:49 +00:00
Wentao Ye 551ef1631a
[Unit Test] Add unit test for deep gemm (#20090) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-06-30 10:26:42 -06:00
Woosuk Kwon 3c783e6687 Merge branch 'main' into woosuk/async-sched 2025-06-30 09:12:45 -07:00
Woosuk Kwon 2863befce3
[Optimization] Use Shared `CachedRequestData` Instance Across All Requests (#20232) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-06-30 09:07:50 -07:00
Woosuk Kwon 2965c99c86
[Spec Decode] Clean up spec decode example (#20240) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-06-30 08:28:13 -07:00
Woosuk Kwon 2062c0723d
[Spec Decode] Refactor spec decoding into a separate function (#20238) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-06-30 08:13:50 -07:00
li haoyang 1c50e100a9
[Bugfix] fix quark ptpc (#20251) 
Signed-off-by: Haoyang Li <Haoyang.Li@amd.com>
Co-authored-by: Haoyang Li <307790822@qq.com>
 2025-06-30 22:24:50 +09:00
Michael Yao 3ee56e26be
[Docs] Fix 1-2-3 list in v1/prefix_caching.md (#20243) 
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
 2025-06-30 11:20:51 +00:00
Jee Jee Li 8fe7fc8634
[Quantization] Improve BitsAndBytesModelLoader (#20242) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-06-30 18:22:09 +08:00
Isotr0py e936e401de
[Bugfix] Fix processor initialization in transformers 4.53.0 (#20244) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-06-30 10:16:16 +00:00
noiji f5dfa07531
[Bugfix] Skip loading extra parameters for modelopt Qwen3 MoE model (#19598) 
Signed-off-by: noiji <>
 2025-06-30 18:21:56 +09:00
Reid 022c58b80f
[doc] Add Slack and Forum to the top navigation (#20208) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
 2025-06-30 07:53:45 +00:00
Woosuk Kwon 19108ef311
[Misc] Fix import (#20233) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-06-29 20:34:54 -07:00
Chendi.Xue 5a52f389dd
[BUGFIX][DEEPSEEK][MODEL_LOAD] fix w13, w2 weight not initialized assert (#20202) 
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
 2025-06-29 19:46:19 -07:00
redmoe-moutain 65b1cbb138
[Model] support dots1 (#18254) 
Signed-off-by: redmoe-moutain <agiredmoe@gmail.com>
 2025-06-29 19:34:36 -07:00
Woosuk Kwon 85384796ca fix 2025-06-29 17:37:36 -07:00
Woosuk Kwon 388774b74e Merge branch 'woosuk/async-sched' of https://github.com/vllm-project/vllm into woosuk/async-sched 2025-06-29 17:30:04 -07:00
Woosuk Kwon 8525677926 Merge branch 'woosuk/serial' into woosuk/async-sched 2025-06-29 17:29:20 -07:00
Woosuk Kwon 662a60db9c Merge branch 'main' into woosuk/async-sched 2025-06-29 17:20:44 -07:00