vLLM
/
vllm
mirror of https://github.com/vllm-project/vllm.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
7,462 Commits 73 Branches 67 Tags 518 MiB
Commit Graph

352 Commits

Author SHA1 Message Date
bnellnm c1909e7e8c
[Kernels] MoE refactor (#19636) 
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Co-authored-by: ElizaWszola <ewszola@redhat.com>
 2025-07-02 06:08:27 -07:00
Wentao Ye 7058d7dd5d
[Refactor] Remove duplicate `find_free_port` (#20333) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-07-01 19:03:07 -07:00
czhu-cohere 3abfe22154
Enable group size 64 for Machete (#20290) 
Signed-off-by: czhu-cohere <conway.zhu@cohere.com>
 2025-07-01 18:05:44 -07:00
TY-AMD 96453cfa83
[BugFix][V1][ROCm] Triton MLA uses V0 backend on V1 engine (#19067) 
Signed-off-by: Tianyuan Wu <Tianyuan.Wu@amd.com>
 2025-07-01 16:12:19 +08:00
Varun Sundar Rabindranath 08d81f1014
[Bugfix] Fix deepep tests (#20288) 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
 2025-07-01 15:29:08 +08:00
czhu-cohere 9909726d2a
Enable ZP Support for Machete (#20268) 
Signed-off-by: czhu-cohere <conway.zhu@cohere.com>
 2025-07-01 07:12:20 +00:00
Wentao Ye 551ef1631a
[Unit Test] Add unit test for deep gemm (#20090) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-06-30 10:26:42 -06:00
Wentao Ye 4d36693687
[Refactor] Create a function util and cache the results for `has_deepgemm`, `has_deepep`, `has_pplx` (#20187) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-06-28 22:06:38 +00:00
Wentao Ye 562308816c
[Refactor] Rename commnication utils (#20091) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-06-26 22:19:32 +00:00
Wentao Ye c894c5dc1f
[Bug Fix] Fix address/port already in use error for deep_ep test (#20094) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-06-26 22:33:13 +08:00
bnellnm 015fab8c2f
[Kernels][Bugfix] Use torch op for all kernels in FusedMoE forward. Add additional testing for cudagraphs. (#19717) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2025-06-24 23:22:58 -07:00
Wentao Ye 879f69bed3
[Refactor] Remove duplicate `ceil_div` (#20023) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-06-25 05:19:09 +00:00
Wentao Ye a6c4b87fbc
Revert "[Feature] Integrate new deepgemm (#19820)" (#20049) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-06-24 19:45:22 -07:00
Wentao Ye c6e3bba8e6
[Feature] Integrate new deepgemm (#19820) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-06-24 12:51:56 -07:00
Tyler Michael Smith 68aaeb3749
[EP+DP] Optimize the little operations in the DeepGEMM + DeepEP low latency case (#19885) 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Signed-off-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
 2025-06-23 11:07:47 -07:00
Wentao Ye ffb2cd6b54
[Perf] Optimize `moe_align_block_size` CUDA kernel (#19572) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-06-17 11:49:26 -07:00
Driss Guessous ddfed314f9
Fixes IMA for TP w/ flex-attention (#19712) 
Signed-off-by: drisspg <drisspguessous@gmail.com>
 2025-06-17 04:01:50 +00:00
22quinn 0b73736a0d
[Kernel] Raise verbose error and consolidate `num_heads/num_kv_heads` divisibility check (#19339) 
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
 2025-06-15 13:43:48 +08:00
Wentao Ye b6efafd9e4
[Perf] Vectorize static / dynamic INT8 quant kernels (#19233) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-06-12 06:51:41 -07:00
Ning Xie 2f1c19b245
[CI] change spell checker from codespell to typos (#18711) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2025-06-11 19:57:10 -07:00
bnellnm 29fa5cac1c
[Kernels] Add activation chunking logic to FusedMoEModularKernel (#19168) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2025-06-11 12:53:10 -04:00
leopardracer 7c644ab6d5
Fix Typo in Documentation and Function Name (#19442) 2025-06-10 22:44:11 -07:00
Isotr0py 5f1ac1e1d1
Revert "[v1] Add fp32 support to v1 engine through flex attn" (#19404) 2025-06-10 01:30:20 -07:00
Varun Sundar Rabindranath 5cf2daea9a
[Misc] Fixes and Optimizations for DeepEP + DeepGEMM combination. (#19298) 
Signed-off-by: Varun <vsundarr@redhat.com>
Co-authored-by: Varun <vsundarr@redhat.com>
 2025-06-09 10:50:39 -04:00
Isotr0py b8089195b4
[v1] Add fp32 support to v1 engine through flex attn (#19319) 
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-06-09 22:10:44 +08:00
Driss Guessous cf02f9b283
Add FlexAttention to V1 (#16078) 
Signed-off-by: drisspg <drisspguessous@gmail.com>
 2025-06-06 21:58:55 -07:00
ElizaWszola 84166fee97
[Kernel] Integrate CUTLASS MoE kernel with PPLX (#18762) 
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-06-06 18:26:11 -07:00
Chiyue Wei 61059bee40
[Hardware][NVIDIA] FP4 MoE kernel optimization (#19110) 
Signed-off-by: Chiyue Wei <chiyuew@nvidia.com>
Co-authored-by: Chiyue Wei <chiyuew@nvidia.com>
 2025-06-05 09:48:26 -07:00
Varun Sundar Rabindranath c3fd4d669a
[Kernel] Integrate batched/masked deepgemm kernel (#19111) 
Signed-off-by: Varun <vsundarr@redhat.com>
Co-authored-by: Varun <vsundarr@redhat.com>
 2025-06-04 21:59:18 +00:00
Kaixi Hou 41aa578428
[NVIDIA] Add Cutlass MLA backend (#17625) 2025-06-03 21:40:26 -07:00
Vadim Gimpelson 5d6d1adf15
[KERNEL] Sampler. CUDA kernel for applying repetition penalty (#18437) 2025-06-03 21:13:01 -07:00
Li, Jiang 4555143ea7
[CPU] V1 support for the CPU backend (#16441) 2025-06-03 18:43:01 -07:00
Varun Sundar Rabindranath fa98d77773
[Kernel] DeepEP dispatch-combine kernel integration (#18434) 
Signed-off-by: Varun <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
 2025-06-03 12:30:02 -07:00
Simon Mo 02f0c7b220
[Misc] Add SPDX-FileCopyrightText (#19100) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2025-06-03 11:20:17 -07:00
Charlie Fu 306d60401d
[ROCm][Kernel] Add gfx950 support for skinny gemms (#18010) 
Signed-off-by: charlifu <charlifu@amd.com>
 2025-05-31 07:40:05 -07:00
vllmellm 0f5e0d567e
[FEAT][ROCm] Add AITER grouped topk for DeepSeekV2 (#18825) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
 2025-05-31 03:39:31 -07:00
Cyrus Leung 1aa2f81b43
[Misc] Update type annotation for rotary embedding `base` (#18914) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-05-30 10:17:01 +08:00
Hongxia Yang 269d901734
[Bugfix][ROCm] fix the power of 2 exception from triton_unified_attention.py when running llama4 models and unit test fix (#18100) 
Signed-off-by: Hongxia Yang <hongxia.yang@amd.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2025-05-29 07:21:46 +08:00
Isotr0py 1f1b1bc03b
[V1][Quantization] Add CUDA graph compatible v1 GGUF support (#18646) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-05-27 04:40:28 +00:00
Michael Goin 63934543a0
Speed up the `kernels/quantization/` tests (#18669) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-05-25 05:02:59 +00:00
Tyler Michael Smith 6e588da0f4
[Build/CI] Fix CUDA 11.8 build (#17679) 
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
 2025-05-22 12:13:54 -07:00
Hosang dd5fa7e04f
[ROCm][Kernel][V1] Enable AMD Radeon GPU Custom Paged Attention on v1 (#17004) 
Signed-off-by: Hosang Yoon <hosang.yoon@amd.com>
 2025-05-21 08:35:00 -07:00
bnellnm c6c10ca920
[Bugfix] Reduce moe_sum test size to avoid OOM (#18484) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2025-05-21 06:46:39 -07:00
bnellnm 92247c522e
[Bug] Fix moe_sum signature (#18440) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2025-05-20 22:37:08 -07:00
Lucas Wilkinson 4e1c6a0264
[Bugfix] fix rotary embedding test for _get_padded_tensor_shape (#18229) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
 2025-05-16 01:32:45 +00:00
Thomas Parnell e6b8e65d2d
[Bugfix] Fix fp8 tests for triton_unified_attention for Triton 3.3 (#18013) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
 2025-05-15 13:26:34 +08:00
Lucas Wilkinson d93c976a0d
[Kernel] Have rotary embeddings support tensors (#18046) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
 2025-05-14 15:43:55 -07:00
bnellnm f9c069c85e
Modularize fused experts and integrate PPLX kernels (#15956) 2025-05-14 13:11:54 -07:00
TJian 612c2edb4f
[FEAT] [ROCm]: Add AITER CK 2 Stages MoE support (#17110) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2025-05-14 03:03:11 -07:00
Charlie Fu 7b2f28deba
[AMD][torch.compile] Enable silu+fp8_quant fusion for rocm (#18082) 
Signed-off-by: charlifu <charlifu@amd.com>
 2025-05-13 22:13:56 -07:00