vLLM
/
vllm
mirror of https://github.com/vllm-project/vllm.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
6,557 Commits 73 Branches 67 Tags 518 MiB
Commit Graph

34 Commits

Author SHA1 Message Date
Reid 7377dd0307
[doc] update the issue link (#17782) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
 2025-05-07 20:29:05 +08:00
Michael Goin 98834fefaa
Update nm to rht in doc links + refine fp8 doc (#17678) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-05-06 00:41:14 +00:00
Zhiyu 182f40ea8b
Add NVIDIA TensorRT Model Optimizer in vLLM documentation (#17561) 2025-05-02 11:36:46 -07:00
Reid 3a500cd0b6
[doc] miss result (#17589) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
 2025-05-02 07:04:49 -07:00
Reid 6d1479ca4b
[doc] add the print result (#17584) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
 2025-05-02 05:24:45 -07:00
NaLan ZeYu 1144a8efe7
[Bugfix] Temporarily disable gptq_bitblas on ROCm (#17411) 
Signed-off-by: Yan Cangang <nalanzeyu@gmail.com>
 2025-04-30 19:51:45 -07:00
Reid 2ac74d098e
[doc] add install tips (#17373) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
 2025-04-30 17:02:41 +00:00
Reid 3ad986c28b
[doc] update wrong model id (#17287) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
 2025-04-28 04:20:51 -07:00
Reid df5c879527
[doc] update wrong hf model links (#17184) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
 2025-04-25 16:40:54 +00:00
Michael Yao ef19e67d2c
[Doc] Add headings to improve gptqmodel.md (#17164) 
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
 2025-04-25 01:13:13 -07:00
Reid 9c1244de57
[doc] update to hyperlink (#17096) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
 2025-04-24 00:58:08 -07:00
Michael Yao f7912cba3d
[Doc] Add top anchor and a note to quantization/bitblas.md (#17042) 
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
 2025-04-23 07:32:16 -07:00
Lei Wang 8d32dc603d
[Kernel] Support Microsoft Runtime Kernel Lib for our Low Precision Computation - BitBLAS (#6036) 
Signed-off-by: xinyuxiao <xinyuxiao2024@gmail.com>
Co-authored-by: xinyuxiao <xinyuxiao2024@gmail.com>
 2025-04-22 09:01:36 +01:00
Michael Goin ed37599544
Update supported_hardware.md for TPU INT8 (#16437) 2025-04-11 12:28:07 +08:00
Driss Guessous 652907b354
Torchao (#14231) 
Signed-off-by: drisspg <drisspguessous@gmail.com>
 2025-04-07 19:39:28 -04:00
yihong 95d63f38c0
doc: fix some typos in doc (#16154) 
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
 2025-04-07 05:32:06 +00:00
Tristan Leclercq 4285e423a6
[Misc] Auto detect bitsandbytes pre-quantized models (#16027) 
Signed-off-by: Tristan Leclercq <tristanleclercq@gmail.com>
 2025-04-04 23:30:45 -07:00
chaow-amd 2041c0e360
[Doc] Quark quantization documentation (#15861) 
Signed-off-by: chaow <chaow@amd.com>
 2025-04-01 08:32:45 -07:00
Jee Jee Li 3892e58ad7
[Misc] Upgrade BNB version (#15183) 2025-03-24 05:51:42 +00:00
Jee Jee Li 10f55fe6c5
[Misc] Clean up the BitsAndBytes arguments (#15140) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-03-20 19:17:12 -07:00
Qubitium-ModelCloud cd1d3c3df8
[Docs] Add GPTQModel (#14056) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-03-03 21:59:09 +00:00
Harry Mellor f58f8b5c96
Update AutoAWQ docs (#14042) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-02-28 15:20:29 +00:00
Szymon Ożóg 7f0be2aa24
[Model] Deepseek GGUF support (#13167) 2025-02-27 02:08:35 -08:00
Harry Mellor 2358ca527b
[Doc]: Improve feature tables (#13224) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-02-18 18:52:39 +08:00
Michael Goin c53dc466b1
[Doc] Remove performance warning for auto_awq.md (#12743) 2025-02-04 22:43:11 -08:00
Brian Dellabetta 44bbca78d7
[Doc] int4 w4a16 example (#12585) 
Based on a request by @mgoin , with @kylesayrs we have added an example
doc for int4 w4a16 quantization, following the pre-existing int8 w8a8
quantization example and the example available in
[`llm-compressor`](https://github.com/vllm-project/llm-compressor/blob/main/examples/quantization_w4a16/llama3_example.py)

FIX #n/a (no issue created)

@kylesayrs and I have discussed a couple additional improvements for the
quantization docs. We will revisit at a later date, possibly including:
- A section for "choosing the correct quantization scheme/ compression
technique"
- Additional vision or audio calibration datasets

---------

Signed-off-by: Brian Dellabetta <bdellabe@redhat.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
 2025-01-31 15:38:48 -08:00
Harry Mellor dd6a3a02cb
[Doc] Convert docs to use colon fences (#12471) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-01-29 11:38:29 +08:00
Gregory Shtrasberg e97f802b2d
[FP8][Kernel] Dynamic kv cache scaling factors computation (#11906) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: Micah Williamson <micah.williamson@amd.com>
 2025-01-23 18:04:03 +00:00
Michael Goin 01a55941f5
[Docs] Update FP8 KV Cache documentation (#12238) 
Signed-off-by: mgoin <michael@neuralmagic.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-01-23 11:18:09 +08:00
Kyle Sayers 3f9b7ab9f5
[Doc] Update examples to remove SparseAutoModelForCausalLM (#12062) 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
 2025-01-15 06:36:01 +00:00
TJian 8a1f938e6f
[Doc] Update Quantization Hardware Support Documentation (#12025) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2025-01-14 04:37:52 +00:00
Rafael Vasquez 43f3d9e699
[CI/Build] Add markdown linter (#11857) 
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
 2025-01-12 00:17:13 -08:00
Harry Mellor aba8d6ee00
[Doc] Move examples into categories (#11840) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-01-08 13:09:53 +00:00
Cyrus Leung ee77fdb5de
[Doc][2/N] Reorganize Models and Usage sections (#11755) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-06 21:40:31 +08:00