biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
15,440 Commits 2 Branches 140 Tags
7cf56a59a267d2f32860c96a601e56bb29f6627a
Commit Graph

4539 Commits

Author SHA1 Message Date
bnellnm
7cf56a59a2 [MoE Refactor] Make SharedExperts class for use with DefaultMoERunner (#35153) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2026-04-01 09:44:08 -04:00
Zhanda Zhu
c75a313824 [Perf] triton bilinear_pos_embed kernel for ViT (#37948) 
Signed-off-by: Zhanda Zhu <zhandazhu@gmail.com>
 2026-04-01 01:52:02 -07:00
Lukas Geiger
4f6eed3bd4 [Core] Simplify multimodal masking (#34246) 
Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>
 2026-04-01 01:18:22 -07:00
Lucas Wilkinson
eb47454987 [Bugfix][MLA] Add logits size budget to sparse indexer prefill chunking (#36178) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-04-01 00:15:53 -04:00
Matthew Bonanni
116f4be405 [1/N][Cleanup] Standardize on use of is_quantized_kv_cache (#38659) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-04-01 04:08:01 +00:00
Wentao Ye
7b01d97a22 [Perf] Optimize mean pooling using chunks and index_add, 5.9% E2E throughput improvement (#38559) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-04-01 03:54:58 +00:00
Luka Govedič
40bb175027 [vLLM IR] 1/N Implement IR skeleton and rms_norm op (#33825) 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: Xinyu Chen <xinyu1.chen@intel.com>
Signed-off-by: chzhang <chaojun.zhang@intel.com>
Signed-off-by: Luka Govedic <luka.govedic@gmail.com>
Co-authored-by: Xinyu Chen <xinyu1.chen@intel.com>
Co-authored-by: Chaojun Zhang <chaojun.zhang@intel.com>
Co-authored-by: Luka Govedič <ProExpertProg@h100-01.nemg-001.lab.rdu2.dc.redhat.com>
 2026-03-31 22:15:05 -04:00
Asaf Gardin
3dc01ef352 [Quantization] Consolidate dummy format logic into DummyModelLoader (#38637) 
Signed-off-by: Josephasafg <ajgard7@gmail.com>
 2026-03-31 22:20:45 +00:00
czhu-cohere
517b769b58 [Perf] Fix DBO overlap: capture DeepEP event before yield (#38451) 
Signed-off-by: root <conway.zhu@cohere.com>
 2026-03-31 20:38:59 +00:00
yzong-rh
d9b90a07ac [MoE Refactor] Migrate Unquantized to Full Oracle Flow (#36286) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Signed-off-by: yzong-rh <yzong@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2026-03-31 15:43:33 -04:00
Olya Kozlova
598190aac3 [fix] Remove trtllm ragged mla prefills (#36540) 
Signed-off-by: Olya Kozlova <okozlova@nvidia.com>
 2026-03-31 12:30:27 -07:00
Xu Jinyang
b779eb3363 [Model] Sync upstream BT=chunk_size fix for GDN chunk_fwd_kernel_o, simplify warmup to single pass (#38343) 
Signed-off-by: AuYang <459461160@qq.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
 2026-03-31 23:03:24 +04:00
Yi Liu
0dd25a44ea [Quantization][Autoround][XPU] Add W4A16 Support (#37986) 
Signed-off-by: yiliu30 <yi4.liu@intel.com>
 2026-03-31 16:48:24 +00:00
SandishKumarHN
3896e021a0 [Bugfix] Fix FusedMoE weight loading with padded hidden dimensions (#37010) 
Signed-off-by: SandishKumarHN <sandish@fb.com>
 2026-03-31 12:22:26 -04:00
zhang-prog
b6e636c12c [Fix] handle PaddleOCR-VL image processor max_pixels across Transformers v4/v5 (#38629) 
Signed-off-by: zhangyue66 <zhangyue66@baidu.com>
 2026-03-31 15:50:41 +00:00
Jingu Kang
f1ff50c86c [Bugfix] clamp dA_cumsum differences to prevent Inf in Mamba2 SSD kernels (#37501) 
Signed-off-by: Jingu Kang <jg.k@navercorp.com>
 2026-03-31 17:35:51 +02:00
Kyle Sayers
5869f69c5f [Online Quant] [QeRL] Minor code cleanup (#38574) 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
 2026-03-31 14:56:43 +00:00
ElizaWszola
202f147cf2 Fix MLA runs when use_inductor_graph_partition=True (#38631) 
Signed-off-by: ElizaWszola <ewszola@redhat.com>
 2026-03-31 13:37:43 +00:00
Yintong Lu
f09daea261 [CPU] Support int8 compute mode in CPU AWQ (#35697) 
Signed-off-by: Yintong Lu <yintong.lu@intel.com>
 2026-03-31 15:27:37 +08:00
zhangyiming
1ac6694297 [OOT] Add OOT support for linear kernel. (#37989) 
Signed-off-by: menogrey <1299267905@qq.com>
 2026-03-31 14:33:21 +08:00
Netanel Haber
e812bf70bd Restore non-hf processor path for Nano-Nemotron-VL (bypass call_hf_processor_mm_only) - fixes #38018 (#38567) 
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
Co-authored-by: tomeras91 <57313761+tomeras91@users.noreply.github.com>
 2026-03-30 21:56:52 +00:00
SandishKumarHN
bcc6f67447 [Bugfix] Use null block (0) for padded block table entries (#35431) 
Signed-off-by: SandishKumarHN <sandish@fb.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-03-30 14:02:51 -07:00
Asaf Gardin
1fc69f59bb [Bug fix][Quantization] Fix dummy weight loading (#38478) 
Signed-off-by: Josephasafg <ajgard7@gmail.com>
 2026-03-30 16:38:02 -04:00
Ilya Markov
12701e8af2 [EPLB] Optmize eplb mapping and record in router for prefill (#36261) 
Signed-off-by: ilmarkov <markovilya197@gmail.com>
 2026-03-30 19:48:33 +00:00
Benjamin Chislett
494636b29d [Feat][Spec Decode] DFlash (#36847) 
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
 2026-03-30 15:03:15 -04:00
Chendi.Xue
3b1dbaad4e [HMA]Fix corner case when hybrid page_size can not be evenly divided issue (blk_size=64,tp=4) (#37467) 
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2026-03-30 16:47:30 +00:00
Johnny
b4a2f3ac36 [NVIDIA] Bugfix NVFP4 DGX Spark and RTX50 (#38423) 
Signed-off-by: johnnynunez <johnnynuca14@gmail.com>
Signed-off-by: Johnny <johnnynuca14@gmail.com>
 2026-03-30 09:36:18 -07:00
roikoren755
8e6293e838 [Mamba] Add stochastic rounding support (#35753) 
Signed-off-by: Roi Koren <roik@nvidia.com>
 2026-03-30 12:33:49 -04:00
Hongxia Yang
dbdd9ae067 [ROCm][Bugfix] fix exception related to trust_remote_code for MiniMax-M2.1-MXFP4 (#37698) 
Signed-off-by: Hongxia Yang <hongxiay.yang@amd.com>
Co-authored-by: Hongxia Yang <hongxiay.yang@amd.com>
 2026-03-30 15:49:23 +00:00
Matthias Gehre
e8b055a5ac [Bugfix] Handle ParallelLMHead in compressed-tensors get_quant_method (#37291) 
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-03-30 07:30:52 -07:00
Thomas Parnell
7c3f88b2a8 [Bugfix] Remove false-positive format mismatch warnings in FLA ops (#38255) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
 2026-03-30 12:32:26 +00:00
Tan Pin Siang
85c0950b1f [ROCm] Enable MORI EP for unquantized MoE with AITER backend (#37529) 
Signed-off-by: Tan Pin Siang <pinsiang.tan@amd.com>
 2026-03-30 15:19:33 +08:00
Jee Jee Li
ac30a8311e [Bugfix][Model] Fix PixtralForConditionalGeneration LoRA (#36963) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-03-29 23:59:42 -07:00
PikaPikachu
63babd17f1 [Model][Quantization] Add GGUF support for MiniMax-M2.1 (#36965) 
Signed-off-by: kangletian <Letian.Kang@amd.com>
 2026-03-30 14:24:06 +08:00
Jaewon
d816834c1a [MoE] Add RoutingMethodType.Simulated to TRT-LLM FP8/NVFP4 kernel allowlists (#38329) 
Signed-off-by: Jaewon Lee <jaewon@meta.com>
 2026-03-29 22:53:43 -07:00
Roger Wang
92f0db57a8 [Misc] Always use forward_mulmat for Conv3d on newer versions of torch. (#38487) 2026-03-30 05:39:41 +00:00
Kyle Sayers
d28d86e8a3 [QeRL] Fix online quantized reloading (#38442) 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
 2026-03-29 14:56:41 -06:00
Wentao Ye
995dea1354 [Perf] Remove redundant device copies for CPU-only pooling token IDs, 48.9% E2E throughput improvement (#38139) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-03-29 18:12:50 +00:00
allgather
8c0b6267d7 [Transformers v5] fix missing pixtral/voxtral multimodal dispatch (#38410) 
Signed-off-by: allgather <all2allops@gmail.com>
 2026-03-29 09:59:06 +00:00
haosdent
d39b8daf5f [Feature] Add Qwen3-ForcedAligner support via token classification pooling (#35367) 
Signed-off-by: haosdent <haosdent@gmail.com>
 2026-03-29 00:27:52 +00:00
Kunshang Ji
aa4eb0db78 [CI]revert initialize_model context manager (#38426) 
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-03-28 16:56:50 +00:00
IriKa
148a5c1226 [Bugfix]fix output Nan/Inf in marlin if dtype=float16 (#33972) 
Signed-off-by: IriKa Qiu <qiujie.jq@gmail.com>
 2026-03-27 16:36:08 -07:00
Wei Zhao
b69bf2f0b1 [Perf] Use torch compile to fuse pack topk in trtllm moe (#37695) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
 2026-03-27 17:30:46 -06:00
rongfu.leng
88149b635e Add nvidia h800 moe config (#31201) 
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
 2026-03-27 16:28:48 -07:00
Kyle Sayers
648edcf729 [QeRL] Compose online quantization with quantized reloading (#38032) 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
 2026-03-27 13:22:33 -07:00
Gregory Shtrasberg
b8665383df [ROCm] Fix GPT-OSS import for triton 3.6 (#37453) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2026-03-27 18:00:57 +00:00
Rohan Potdar
0e9358c11d {ROCm]: gpt-oss fusion/padding fixes (#38043) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
Signed-off-by: Rohan Potdar <66227218+Rohan138@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-03-27 12:19:15 -04:00
Jonas M. Kübler
98e7f223b9 enable skipping of SW attention layers when using FP8 KV cache (#33695) 
Signed-off-by: Jonas Kuebler <kuebj@amazon.com>
 2026-03-27 07:25:02 -06:00
Bowen Bao
0ae89f18fd [Refactor] Move FusedMoE hidden_size roundup to quant_method (#34285) 
Signed-off-by: Bowen Bao <bowenbao@amd.com>
 2026-03-26 23:38:26 -07:00
Li, Jiang
becaed6ec8 [CPU] Support CT W4A16 on CPU MP kernel (#38219) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2026-03-27 14:15:28 +08:00