biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
13,638 Commits 2 Branches 140 Tags
c1858b7ec8aa571dc0c0e00aded01019cca6a7e6
Commit Graph

198 Commits

Author SHA1 Message Date
Aaron Hao
c1858b7ec8 [Feat][RL][1/2] Native Weight Syncing API: NCCL (#31943) 
Signed-off-by: ahao-anyscale <ahao@anyscale.com>
Signed-off-by: Aaron Hao <ahao@anyscale.com>
Co-authored-by: SumanthRH <sumanthrh99@gmail.com>
 2026-02-05 12:13:23 -05:00
Michael Goin
2a99c5a6c8 [Bugfix] Disable TRTLLM FP8 MoE if router_logits_dtype==float32 and routing_method!=DeepSeekV3 (#33613) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-02-03 13:26:51 -08:00
Kunshang Ji
e10604480b [XPU][1/N] Deprecate ipex and switch to vllm-xpu-kernels for xpu platform (#33379) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-02-02 22:46:10 -08:00
Vasiliy Kuznetsov
0130223bd9 fix memory for online fp8 quantization with streaming weight load (#31914) 
Signed-off-by: vasiliy <vasiliy@fb.com>
 2026-02-02 14:17:42 -05:00
Michael Goin
fd0e377244 Support FP8 block quant for CompressedTensorsW8A16Fp8 (#33280) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-01-30 11:15:20 -05:00
杨朱 · Kiki
1a7894dbdf [Misc] Replace Optional[X] with X | None syntax (#33332) 
Signed-off-by: carlory <baofa.fan@daocloud.io>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
 2026-01-30 01:56:59 -08:00
Matthew Bonanni
a608b4c6c2 [5/N][Attention] Finish eliminating vllm/attention folder (#32064) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-01-27 10:02:51 -05:00
Robert Shaw
5a93b9162b [MoE Refactor] Integrate Naive Prepare Finalize into MK (#32567) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Signed-off-by: Amir Klein <203507526+amirkl94@users.noreply.github.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: amirkl94 <203507526+amirkl94@users.noreply.github.com>
 2026-01-27 01:28:02 +00:00
bnellnm
dc917cceb8 [MoE Refactor] Move select_experts from FusedMoEQuantMethod -> FusedMoE (#31996) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2026-01-22 18:21:35 -05:00
Robert Shaw
42135d6898 [MoE Refactor] Oracle Select FP8+NVFP4 Kernels In Priority (#32414) 2026-01-21 08:22:33 -05:00
Vasiliy Kuznetsov
d2389c1262 fp8 online quant: split out Fp8OnlineLinearMethod (#32189) 2026-01-20 18:13:22 -05:00
vllmellm
148117ea2e [Refactor] Make FP8 Linear Ops use kernel abstraction (#27814) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
 2026-01-20 14:48:20 +08:00
bnellnm
327a02d8db [MoE Refactor] Separate Router into OO Classes (#30623) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2026-01-18 11:40:49 -05:00
TomerBN-Nvidia
c277fbdf31 [Feat] Support non-gated MoE with Marlin, NVFP4 CUTLASS, FP8, INT8, compressed-tensors (#32257) 
Signed-off-by: Tomer Natan <tbarnatan@computelab-frontend-8.nvidia.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Tomer Natan <tbarnatan@computelab-frontend-8.nvidia.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Tomer Natan <tbarnatan@ipp1-1429.ipp1a1.colossus.nvidia.com>
 2026-01-15 16:15:05 -08:00
bnellnm
e74698c27a [Misc][Refactor] Add FusedMoERouter object (#30519) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2026-01-08 20:52:55 +00:00
Robert Shaw
5dcd7ef1f2 [MoE Refactor][15/N] Apply Refactor to Fp8 (#31415) 2026-01-07 19:42:33 -05:00
Robert Shaw
af8fd73051 [MoE Refactor][14/N] Clean Up FI Quant Config Smuggling (#31593) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2026-01-06 15:47:04 +00:00
Robert Shaw
f6c0009afa [Bugfix] Fix Broken ModelOpt NVFP4 MoE (#31742) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2026-01-05 23:18:38 +00:00
Robert Shaw
268b1c55ad [MoE Refactor][13/N] Convert FI to Use PFNoEP (#31533) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-01-03 12:26:36 -08:00
Andreas Karatzas
6ef770df7c [MoE] Fix output_shape calculation in Attention layer to handle 3D query inputs (#31596) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-01-02 15:46:23 +00:00
Robert Shaw
56f516254c [Bugfix][ROCm] Fix Static Quant Issue (#31502) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
 2025-12-29 13:27:55 -08:00
Robert Shaw
9152a30d8f [MoE Refactor][12/N] Marlin Fp8 MoE Pure Function (#31499) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2025-12-29 13:27:00 -08:00
Robert Shaw
727c41f3fd [MoE Refactor][10/N] Cleanup Fp8 Process Weights After Loading (#31169) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2025-12-27 20:22:48 +00:00
Robert Shaw
b57b967386 [MoE Refactor][7/N] AITER MK (#31102) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2025-12-22 16:42:58 -07:00
Kevin McKay
cf8eed7bef [Bugfix][ROCm] Fix typo: is_linear_fp8_enaled -> is_linear_fp8_enabled (#31109) 
Signed-off-by: c0de128 <kevin.mckay@outlook.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
 2025-12-21 21:14:58 -08:00
Robert Shaw
b471092d3a [MoE Refactor][4/N] Marlin Fp8 Mk (#31036) 2025-12-21 12:37:42 -05:00
baonudesifeizhai
54c8924384 [MoE Refactor][5/N] Isolate zero expert to LongCatFlash (#28891) 
Signed-off-by: baonudesifeizhai <85092850+baonudesifeizhai@users.noreply.github.com>
Signed-off-by: Dongjie Zou <85092850+baonudesifeizhai@users.noreply.github.com>
Signed-off-by: baonudesifeizhai <baonudesifeizhai@gmail.com>
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com>
 2025-12-20 18:22:04 +00:00
Yan Ma
560ae9638c [XPU] enable fp8 online streaming quantization (#30944) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
 2025-12-20 13:45:27 +00:00
Robert Shaw
95befecc18 [MoE Refactor][2/N] Use Modular Kernels for Fp8 (#30825) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2025-12-19 23:36:38 +00:00
Robert Shaw
83a317f650 [MoE Refactor][3/N] Deprecate cutlass block quant fp8 (b200) (#30990) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2025-12-19 13:09:54 -08:00
Vasiliy Kuznetsov
f4ee2c3d90 fix fp8 online quantization streaming with tp > 1 (#30900) 
Signed-off-by: vasiliy <vasiliy@fb.com>
 2025-12-18 11:45:15 -05:00
Jinzhen Lin
ce96857fdd [Kernel][Quantization][MoE] add marlin kernel support for turing (sm75) (#29901) 
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-12-16 14:35:28 -08:00
Robert Shaw
d0502b4928 [MoE][Refactor 1/N] Separate Online Quantization (#30627) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2025-12-15 06:54:53 -08:00
Roberto L. Castro
4fa7ce46f3 [Feature] Add SM103 (Blackwell Ultra) Support to vLLM (#30484) 
Signed-off-by: LopezCastroRoberto <robertol.c510@gmail.com>
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2025-12-12 19:34:23 -08:00
Kyle Sayers
fccd532587 [Quantization] FP8 Weight Reloading for Quantized RL Rollout (#28480) 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
 2025-12-09 13:54:32 -08:00
bnellnm
00e5cbb967 [MoE][Refactor] Remove most arguments to FusedMoEMethodBase.apply (#29066) 
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2025-12-09 13:48:25 -08:00
Vasiliy Kuznetsov
0d402d2600 online fp8 quant with streaming weight post-processing (#29196) 
Signed-off-by: vasiliy <vasiliy@fb.com>
 2025-12-08 20:15:10 +00:00
Jee Jee Li
652ba93da3 [Bugfix] Fix FP8 MoE LoRA (#29890) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-12-04 18:17:49 +00:00
Jinzhen Lin
1656ad3704 [Kernel][Quantization] add w4a8 support for marlin kernel (#24722) 
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin@redhat.com>
 2025-11-29 07:19:33 -08:00
Matthew Bonanni
fc1d8be3dc [Attention] Update attention imports (#29540) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-11-27 11:19:09 -05:00
Michael Goin
6f1355a1b7 [Perf] Disable DeepGEMM MoE by default when TP=8 is used (#29346) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-11-24 19:01:40 -07:00
bnellnm
8f066146c3 [MoE][Refactor] Make select_experts a non-static method (#29067) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2025-11-24 13:38:04 -05:00
Wentao Ye
a42ab317ac [Log] Optimize startup log (#28948) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-11-21 08:46:20 -08:00
Chen Bruce
da2f6800e0 [Feat][Perf] Enable deepep-low-latency with round-robin expert placement. (#28449) 
Signed-off-by: bruceszchen <bruceszchen@tencent.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-11-19 13:46:24 +01:00
Duncan Moss
3f8a874065 [Kernels] Enable FlashInfer FP8 Blockscale on SM90 (for TEP DSR1) (#27134) 
Signed-off-by: Duncan Moss <djm.moss@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-11-14 08:02:44 -08:00
Varun Sundar Rabindranath
74a9a9faad [Performance][B200] Fix deepgemm prologue (#27897) 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
 2025-11-12 13:13:03 -08:00
Michael Goin
f9a4087182 Remove weight_scale.T special case for SM90 Block FP8 CUTLASS kernel (#28431) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-11-11 11:46:04 -05:00
Wentao Ye
de540c0354 [Feature] Add env var VLLM_MOE_USE_DEEP_GEMM (#28422) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-11-11 02:29:48 +00:00
Wentao Ye
35d801f13f [Feature] Refactor batch invariant fp8 DeepGEMM (#27606) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-11-11 00:08:40 +00:00
jiahanc
34553b9d27 [Performance] Support FP8 flashinfer TRTLLM MOE on Qwen3 and Qwen-3next (#27492) 
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
 2025-11-10 12:34:57 -05:00