biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
15,652 Commits 2 Branches 140 Tags
main
Commit Graph

2136 Commits

Author SHA1 Message Date
Harry Mellor
d87fb264df [Docs] Bring README updates into docs README (#39397) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-04-09 10:35:00 +00:00
Michael Goin
eb4205fee5 [UX] Integrate DeepGEMM into vLLM wheel via CMake (#37980) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-04-08 18:56:32 -07:00
Jackmin801
a776a48b1c [MoE] Move DEEP_GEMM into experts/ subdirectory (#39005) 
Signed-off-by: Jackmin801 <ongjackm@gmail.com>
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-04-08 19:23:08 +00:00
Shengqi Chen
75e01a39a1 [Feature] NUMA binding support for GPU workers (#38635) 
Signed-off-by: Shengqi Chen <harry-chen@outlook.com>
Co-authored-by: Jason Li <jasonlizhengjian@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-04-08 09:55:24 -07:00
Kevin H. Luu
6155bbd1dd [Bugfix][Docs] Fix ReadTheDocs build crash from mocked torch decorator (#39284) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-08 09:43:01 +00:00
Varun Sundar Rabindranath
7b80cd8ac3 [Docs] Add Phi-4-reasoning-vision to supported models + examples (#39232) 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
 2026-04-08 02:02:26 +00:00
Andrey Talman
2111997f96 [release 2.11] Update to torch 2.11 (#34644) 2026-04-07 18:55:48 -07:00
Harry Mellor
79df4a794d Automatically add links to API docs for matching strings in docs (#37434) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-04-07 21:21:18 +08:00
bnellnm
b2b2c5239e [MoE Refactor] Split up compressed_tensors_moe.py (#38960) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2026-04-06 20:07:54 -04:00
bhargav-patel-29
c5e3454e5a [Model] Add support for BharatGen's Param2MoE model (#38000) 
Signed-off-by: bhargav-patel-29 <bhargav.patel@tihiitb.org>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-04-06 16:19:56 +08:00
Carl Y
1f5ec2889c [mla] Support fused FP8/NVFP4 output quantization in MLA attention (#35792) (#36205) 
Signed-off-by: Carl You <4531192+carlyou@users.noreply.github.com>
Signed-off-by: Carl Y <4531192+carlyou@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-04-02 21:16:11 -04:00
1096125073
71a9125c67 [New Model]: add support for telechat3 (#38510) 
Signed-off-by: xiayongqiang <xiayq1@chinatelecom.cn>
Co-authored-by: xiayongqiang <xiayq1@chinatelecom.cn>
 2026-04-03 08:26:22 +08:00
bsliu
c0817e4d39 [Model] Add support for Cheers multimodal model (#38788) 
Signed-off-by: bsliu <1187291748@qq.com>
Signed-off-by: 吴炳贤 <wubingxian24@mails.ucas.ac.cn>
 2026-04-02 21:01:40 +08:00
JartX
2ce3d0ce36 [Feature] KV cache per-token-head INT8/FP8 quantization (#38378) 
Signed-off-by: JartX <sagformas@epdcenter.es>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: yangyang4991 <yangyang4991@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
 2026-04-02 08:13:26 -04:00
Stefano Castagnetta
6183cae1bd [Bugfix] Restrict TRTLLM attention to SM100, fixing GB300 (SM103) hang (#38730) 
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
 2026-04-01 12:08:40 -07:00
Monishver
c09ad767cd Feature/silu block quant fusion v1 (#32996) 
Signed-off-by: Monishver Chandrasekaran <monishverchandrasekaran@gmail.com>
 2026-04-01 18:50:43 +00:00
Chauncey
cbe7d18096 [Misc] Rename think_start_str/think_end_str to reasoning_start_str/reasoning_end_str (#38242) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-04-01 09:56:45 -07:00
Jesus Talavera
148c2072ec Add ibm-granite/granite-vision-3.3-2b to supported models documentation (#38714) 
Signed-off-by: Jesus Talavera <jesus.talavera@ibm.com>
 2026-04-01 08:22:25 -07:00
Li, Jiang
36d7f19897 [CPU] Support head_size 512 in cpu_attn (#38676) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2026-04-01 05:42:27 +00:00
Vedant V Jhaveri
2e56975657 Generative Scoring (#34539) 
Signed-off-by: Vedant Jhaveri <vjhaveri@linkedin.com>
Co-authored-by: Vedant Jhaveri <vjhaveri@linkedin.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-03-31 16:02:11 -07:00
BadrBasowid
077a9a8e37 [torch.compile] Refactor Attention Quant Fusion Pass and Remove Boilerplate (#37373) 
Signed-off-by: BadrBasowid <badr.basowid@gmail.com>
Co-authored-by: vllmellm <vllm.ellm@embeddedllm.com>
 2026-03-31 14:15:50 -04:00
Nicolò Lucchesi
7337ff7f03 [Docs] PD with Nixl compat matrix (#38628) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-03-31 15:01:21 +00:00
Flora Feng
3e802e8786 [Mypy] Fix adjust_request typing (#38264) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-03-31 04:21:18 +00:00
TJian
03ac6ca895 [ROCm] [DOC] Update the Documentation to include ROCm Nightly Wheel support (#38457) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2026-03-30 02:25:46 -07:00
Juan Pérez de Algaba
57861ae48d (security) Fix SSRF in batch runner download_bytes_from_url (#38482) 
Signed-off-by: jperezde <jperezde@redhat.com>
 2026-03-30 07:10:01 +00:00
Andreas Karatzas
43cc5138e5 [ROCm][CI] Fix cross-attention dispatch for encoder-decoder models (#38450) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-03-28 22:08:03 -07:00
haosdent
d39b8daf5f [Feature] Add Qwen3-ForcedAligner support via token classification pooling (#35367) 
Signed-off-by: haosdent <haosdent@gmail.com>
 2026-03-29 00:27:52 +00:00
whyiug
58c959a767 [Misc]: clean up non-core lint issues (#37049) 
Signed-off-by: whyiug <whyiug@hotmail.com>
 2026-03-28 10:28:16 -04:00
Hongxia Yang
83a4df049d [ROCm][Documentation] update quickstart and installation to include rocm nightly docker tips (#38367) 
Signed-off-by: Hongxia Yang <hongxiay.yang@amd.com>
Co-authored-by: Hongxia Yang <hongxiay.yang@amd.com>
 2026-03-27 23:20:19 +00:00
Juan Pérez de Algaba
b111f8a61f fix(security): Add VLLM_MAX_N_SEQUENCES environment variable and enforce limit (#37952) 
Signed-off-by: jperezde <jperezde@redhat.com>
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
 2026-03-27 09:02:10 -04:00
Yuichiro Utsumi
a1746ff9ec [Doc] Clarify Helm chart location in deployment guide (#38328) 
Signed-off-by: Yuichiro Utsumi <utsumi.yuichiro@fujitsu.com>
Signed-off-by: Yuichiro Utsumi <81412151+utsumi-fj@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-03-27 15:43:02 +08:00
Cyrus Leung
a9213c0ffe [Doc] Fix outdated reference to CUDAGraphManager (#38209) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-03-26 01:52:38 -07:00
Ekagra Ranjan
7b54f60db0 [Cohere] Enable Cohere-Transcribe (#38120) 
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
 2026-03-25 16:13:51 -07:00
Rohan Potdar
a0e8c74005 [ROCm]: Update rope+kvcache fusion conditions and disable custom op by default (#36716) 
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
 2026-03-25 20:58:44 +00:00
Cyrus Leung
ba2f0acc2d [Misc] Reorganize inputs (#35182) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-03-25 10:22:54 -07:00
Ben Browning
a1a2566447 [Docs] Add guide for editing agent instruction files (#37819) 
Signed-off-by: Ben Browning <bbrownin@redhat.com>
 2026-03-25 13:54:09 +00:00
R0CKSTAR
242c93f744 [Docs] Adds vllm-musa to custom_op.md (#37840) 
Signed-off-by: Xiaodong Ye <yeahdongcn@gmail.com>
 2026-03-25 11:54:36 +00:00
Gregory Shtrasberg
189ddefbfd [ROCm] Attention selector reordering (#36702) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
Co-authored-by: Micah Williamson <micah.williamson@amd.com>
 2026-03-25 17:42:56 +08:00
Baorun (Lauren) Mu
9d0351c91d [Docs] Add Encoder (ViT) CUDA Graphs section to CUDA Graphs design doc (#37914) 
Signed-off-by: Baorun Mu <bmu@nvidia.com>
 2026-03-24 19:53:24 -07:00
Terry Gao
82580b10ac [Perf] Disable inductor runtime asserts by default for serving perfor… (#37485) 
Signed-off-by: tianrengao <terrygao87@gmail.com>
Co-authored-by: Tianren Gao <tianren@fb.com>
 2026-03-24 19:37:51 -04:00
Nick Cao
935c46dd9b [Model] Add Granite 4.0 1B speech to supported models (#38019) 
Signed-off-by: Nick Cao <ncao@redhat.com>
 2026-03-24 18:23:41 +00:00
Vineeta Tiwari
b58c5f28aa docs: fix broken offline inference paths in documentation (#37998) 
Signed-off-by: Vineeta Tiwari <vineeta.tiwari2@ibm.com>
Signed-off-by: Vineeta Tiwari <vineetatiwari2000@gmail.com>
Co-authored-by: Vineeta Tiwari <vineeta.tiwari2@ibm.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-03-24 17:35:14 +00:00
Sungjae Lee
4731884796 [Feature] limit thinking tokens (hard limit) (#20859) 
Signed-off-by: Sungjae Lee <33976427+llsj14@users.noreply.github.com>
Signed-off-by: Sungjae Lee <sung-jae.lee@navercorp.com>
Signed-off-by: Chauncey <chaunceyjiang@gmail.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-24 09:53:07 -07:00
wang.yuqi
1b6cb920e6 [Deprecate] Deprecate pooling multi task support. (#37956) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-03-24 14:07:47 +00:00
Andrew Xia
9ace378a63 [Frontend][Responses API] Fix arrival_time recording for TTFT on initial request (#37498) 
Signed-off-by: Andrew Xia <axia@meta.com>
 2026-03-23 09:58:08 +00:00
Yan Ma
d3fe857135 update doc for online fp8 quantization (#37851) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
 2026-03-23 05:19:03 +00:00
Lasha Koroshinadze
e7767eccae Fix AudioFlamingo3/MusicFlamingo HF parity and RoTE handling (#37643) 
Signed-off-by: Lasha <26011196+lashahub@users.noreply.github.com>
 2026-03-23 10:29:07 +08:00
Robert Shaw
4383f1532e [MoE] Move PF Methods to Folder (#35927) 2026-03-22 02:42:59 -06:00
Yongye Zhu
87bd91892f [MoE Refactor] Mxfp4 oracle rebased (#37128) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-21 03:37:04 +00:00
Ilya Boytsov
8b6c6b9505 [Model] Add LFM2-ColBERT-350M support (#37528) 
Signed-off-by: Ilya Boytsov <ilyaboytsov1805@gmail.com>
 2026-03-20 14:57:57 +00:00