vLLM
/
vllm
mirror of https://github.com/vllm-project/vllm.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
7,204 Commits 73 Branches 67 Tags 518 MiB
Commit Graph

7204 Commits

Author SHA1 Message Date
kourosh hakhamaneshi e6aab5de29
Revert "[Build/CI] Add tracing deps to vllm container image (#15224)" (#19378) 2025-06-12 17:26:40 -07:00
Russell Bryant c57bb199b3
[V1] Resolve failed concurrent structured output requests (#19565) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-06-12 23:30:09 +00:00
Aaron Pham dba68f9159
[Doc] Unify structured outputs examples (#18196) 
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
 2025-06-12 22:50:31 +00:00
Michael Goin a3319f4f04
[Bugfix] Enforce contiguous input for dynamic_per_token FP8/INT8 quant (#19452) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-06-12 15:39:15 -04:00
Varun Sundar Rabindranath 9d880f594d
[Misc] Turn MOE_DP_CHUNK_SIZE into an env var (#19506) 2025-06-12 18:01:16 +00:00
Ekagra Ranjan 017ef648e9
[Spec Decode][Benchmark] Generalize spec decode offline benchmark to more methods and datasets (#18847) 2025-06-12 10:30:56 -07:00
Reid 4b25ab14e2
[doc] Make top navigation sticky (#19540) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
 2025-06-12 15:48:11 +00:00
Luka Govedič f98548b9da
[torch.compile][ROCm] Fuse quantization onto attention using a torch.compile pass (#16756) 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Sage Moore <sage@neuralmagic.com>
 2025-06-12 08:31:04 -07:00
mobicham 96846bb360
Fix TorchAOConfig skip layers (#19265) 
Signed-off-by: mobicham <hicham@mobiuslabs.com>
 2025-06-12 22:22:53 +08:00
Wentao Ye b6efafd9e4
[Perf] Vectorize static / dynamic INT8 quant kernels (#19233) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-06-12 06:51:41 -07:00
Nicolò Lucchesi 1129e2b1ab
[V1][NixlConnector] Drop `num_blocks` check (#19532) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-06-12 12:36:14 +00:00
Cyrus Leung c742438f8b
[Doc] Add V1 column to supported models list (#19523) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-06-12 19:16:44 +08:00
Jee Jee Li 73e2e0118f
[Quantization] Improve AWQ logic (#19431) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-06-12 11:02:11 +00:00
jmswen c9280e6346
[Bugfix] Respect num-gpu-blocks-override in v1 (#19503) 
Signed-off-by: Jon Swenson <jmswen@gmail.com>
 2025-06-12 11:00:23 +00:00
Michael Goin af09b3f0a0
[Bugfix][V1] Allow manual FlashAttention for Blackwell (#19492) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-06-12 10:40:24 +00:00
Russell Bryant 4f6c42fa0a
[Security] Prevent new imports of (cloud)pickle (#18018) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Aaron Pham <Aaronpham0103@gmail.com>
 2025-06-12 10:30:17 +00:00
niu_he dff680001d
Fix typo (#19525) 
Signed-off-by: 2niuhe <carlton2tang@gmail.com>
 2025-06-12 09:24:45 +00:00
rasmith 2e090bd5df
[AMD][Kernel][BugFix] fix test_rocm_compressed_tensors_w8a8 for rocm (#19509) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
 2025-06-12 07:14:24 +00:00
wonjun Jang 1b0b065eb5
[BugFix] Handle missing sep_token for Qwen3-Reranker in Score API (#19522) 
Signed-off-by: strutive07 <strutive07@gmail.com>
 2025-06-12 07:00:47 +00:00
Nick Hill d5bdf899e4
[BugFix] Work-around incremental detokenization edge case error (#19449) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-06-12 06:43:20 +00:00
22quinn 7e3e74c97c
[Frontend] Improve error message in tool_choice validation (#19239) 
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
 2025-06-12 01:13:00 -04:00
Brayden Zhong 3f6341bf7f
Add Triton Fused MoE kernel config for E=16 on B200 (#19518) 
Signed-off-by: Brayden Zhong <b8zhong@uwaterloo.ca>
 2025-06-12 04:31:51 +00:00
Varun Sundar Rabindranath e5d35d62f5
[BugFix] Force registration of w8a8_block_fp8_matmul_deepgemm via lazy import (#19514) 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
 2025-06-12 04:28:12 +00:00
Ning Xie 2f1c19b245
[CI] change spell checker from codespell to typos (#18711) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2025-06-11 19:57:10 -07:00
Richard Zou 42f52cc95b
[CI/Build] Fix torch nightly CI dependencies (#19505) 
Signed-off-by: Richard Zou <zou3519@gmail.com>
 2025-06-11 14:40:42 -07:00
Robert Shaw 97a9465bbc
[UX] Add Feedback During CUDAGraph Capture (#19501) 
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
 2025-06-11 21:09:05 +00:00
rasmith c7ea0b56cd
[AMD] [Quantization] Add override flag for attention dtype instead of using kv_cache_dtype trigger (#17331) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
 2025-06-11 15:53:28 -04:00
bnellnm 29fa5cac1c
[Kernels] Add activation chunking logic to FusedMoEModularKernel (#19168) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2025-06-11 12:53:10 -04:00
Woosuk Kwon b2d9be6f7d
[Docs] Remove WIP features in V1 guide (#19498) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-06-11 09:15:03 -07:00
Jee Jee Li 04a55612dd
[Misc] Fix misleading ROCm warning (#19486) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-06-12 00:12:10 +08:00
David Xia 89b0f84e17
[doc] fix "Other AI accelerators" getting started page (#19457) 
Signed-off-by: David Xia <david@davidxia.com>
 2025-06-11 16:11:17 +00:00
Michael Goin 497a91e9f7
[CI] Update FlashInfer to 0.2.6.post1 (#19297) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-06-11 22:57:28 +08:00
runzhen 943ffa5703
[Bugfix] Update the example code, make it work with the latest lmcache (#19453) 
Signed-off-by: Runzhen Wang <wangrunzhen@gmail.com>
 2025-06-11 12:42:20 +00:00
Louie Tsai 5c8d34a42c
Support no privileged mode on CPU for docker and kubernetes deployments (#19241) 
Signed-off-by: Tsai, Louie <louie.tsai@intel.com>
 2025-06-11 04:11:47 -07:00
Ximingwang-09 3c8694eabe
Fix some typo (#19475) 
Signed-off-by: ximing.wxm <ximing.wxm@antgroup.com>
Co-authored-by: ximing.wxm <ximing.wxm@antgroup.com>
 2025-06-11 10:36:04 +00:00
Michael Goin 7484e1fce2
Add cache to cuda get_device_capability (#19436) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-06-11 17:37:05 +08:00
Cyrus Leung a2142f0196
Support non-string values in JSON keys from CLI (#19471) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-06-11 09:34:04 +00:00
Lu Fang 871d6b7c74
[Misc] Reduce warning message introduced in env_override (#19476) 
Signed-off-by: Lu Fang <lufang@fb.com>
 2025-06-11 17:29:54 +08:00
Cyrus Leung 29a38f0352
[Doc] Support "important" and "announcement" admonitions (#19479) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-06-11 01:39:58 -07:00
Cyrus Leung a5115f4ff5
[Doc] Fix quantization link titles (#19478) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-06-11 01:27:22 -07:00
Cyrus Leung 68b4a26149
[Doc] Update V1 User Guide for Hardware and Models (#19474) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-06-11 00:49:06 -07:00
artetaout b8e809a057
[Kernel] Support deep_gemm for linear methods (#19085) 
Signed-off-by: artetaout <lulala341@gmail.com>
 2025-06-11 15:14:45 +08:00
Lu Fang 5039ec2336
[ROCm] Add rules to automatically label ROCm related PRs (#19405) 
Signed-off-by: Lu Fang <lufang@fb.com>
 2025-06-11 15:09:18 +08:00
leopardracer 7c644ab6d5
Fix Typo in Documentation and Function Name (#19442) 2025-06-10 22:44:11 -07:00
Junhao Li 2d40665fe8
Add fused MOE config for Qwen3 30B A3B on B200 (#19455) 
Signed-off-by: Junhao Li <junhao@ubicloud.com>
 2025-06-11 13:43:46 +08:00
Lukas Geiger 96ada386b7
[Misc] Remove unused `MultiModalHasher.hash_prompt_mm_data` (#19422) 
Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>
 2025-06-11 05:18:57 +00:00
Michael Goin 1e473b3010
[CI] Disable failing GGUF model test (#19454) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-06-11 05:12:38 +00:00
Lu Fang 2b1e2111b0
Fix test_max_model_len in tests/entrypoints/llm/test_generate.py (#19451) 
Signed-off-by: Lu Fang <lufang@fb.com>
 2025-06-11 12:54:59 +08:00
niu_he a45b979d9f
[BugFix] Fix docker build cpu-dev image error (#19394) 
Signed-off-by: niu_he <carlton2tang@gmail.com>
 2025-06-10 20:56:40 -07:00
wang.yuqi 3952731e8f
[New Model]: Support Qwen3 Embedding & Reranker (#19260) 2025-06-10 20:07:30 -07:00