biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
14,087 Commits 2 Branches 140 Tags
5719a4e4e601fb91274294d25370b7aad656d629
Commit Graph

186 Commits

Author SHA1 Message Date
Xin Yang
b1c4f0b265 [Kernel] Optimize grouped topk kernel (#34206) 
Signed-off-by: Xin Yang <xyangx@amazon.com>
 2026-02-20 01:34:45 -08:00
Wenlong Wang
847a57cd12 [Bugfix][MoE Kernel] Fix incorrect routing selection for models without expert groups (e.g., MiniMax-M2.1) (#34673) 
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-02-18 13:03:24 -08:00
rasmith
fcd6ac97ed [CI][AMD][BugFix] Skip tests in test_unquantized_backend_selection that should not run on ROCm (#34655) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
 2026-02-18 15:00:40 -05:00
haosdent
b68fd899d1 [Bugfix] Fix fused MoE int32 overflow in stride*offset without perf regression (#34507) 
Signed-off-by: haosdent <haosdent@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-02-16 17:58:49 -08:00
haosdent
79f3fab05a [Bugfix] Handle num_expert_group=None in flashinfer block-scale FP8 MoE (#34494) 
Signed-off-by: haosdent <haosdent@gmail.com>
 2026-02-14 23:25:46 -08:00
Marek Michalowski
742d214d6e [Bugfix] fix the import path in moe test utils.py (#34245) 
Signed-off-by: Marek Michalowski <marek.michalowski@arm.com>
 2026-02-13 00:13:45 -08:00
amitz-nv
f120bd42d3 [Kernel] Support Flashinfer trtllm fused MoE non gated FP8 & NVFP4 (#33506) 
Signed-off-by: amitz-nv <203509407+amitz-nv@users.noreply.github.com>
 2026-02-12 13:06:58 -08:00
Michael Goin
ff1f83b056 [Refactor] Replace activation: str with MoEActivation enum (#33843) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
 2026-02-11 17:29:32 -08:00
Wei Zhao
5aff2699bd Fix CI failure - Flashinfer Kernel tests (#34316) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
 2026-02-11 14:17:16 -08:00
Linda
275e0d2a99 [NVIDIA][test] Tests for flashinfer TRTLLM BF16 MoE (#33715) 
Signed-off-by: Linda-Stadter <57756729+Linda-Stadter@users.noreply.github.com>
Co-authored-by: Pavani Majety <pmajety@nvidia.com>
 2026-02-11 12:38:11 +00:00
bnellnm
d1481ba783 [MoE Refactor] Introduce MoERunner abstraction and move execution logic from FusedMoE to DefaultMoERunner (#32344) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2026-02-10 19:51:07 -05:00
xuebwang-amd
b129136c7a [ROCm][Quantization] GPT_OSS in amd-quark format model loading and emulations (#29008) 
Signed-off-by: xuebwang-amd <xuebwang@amd.com>
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-02-10 10:08:05 -05:00
Andrey Talman
f97ca67176 [Release 2.10] Update to Torch 2.10 - final release (#30525) 2026-02-08 13:51:09 -08:00
lukec
15a0b9e570 Fix spelling errors (#33978) 2026-02-06 23:58:50 -08:00
Wentao Ye
77c09e1130 [Refactor] Remove align block size logic in moe_permute (#33449) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-02-06 10:57:06 -08:00
bnellnm
a57c8228ff [Moe Refactor] Make Inplace Flag for FusedMoEModularKernel part of the constructor (#33375) 
Signed-off-by: Bill Nell <bnell@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-02-05 18:07:18 +00:00
rasmith
c1395f72cd [CI][AMD][BugFix] Ensure VLLM_ROCM_USE_AITER is set so test_rocm_aiter_topk.py can run correctly (#33840) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
 2026-02-05 05:05:48 +00:00
杨朱 · Kiki
b95cc5014d [Misc] Remove deprecated VLLM_ALL2ALL_BACKEND environment variable (#33535) 
Signed-off-by: carlory <baofa.fan@daocloud.io>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
 2026-02-03 15:01:59 +08:00
danielafrimi
0aca8b8c62 [MoE] Enable Shared/Routed Overlap For Latent MoE (Nemotron-H) (#32790) 
Signed-off-by: dafrimi <dafrimi@nvidia.com>
 2026-02-02 09:18:50 -05:00
Roy Wang
63c0889416 [Misc] Fix flashinfer related tests (#33462) 
Signed-off-by: esmeetu <jasonailu87@gmail.com>
 2026-01-31 16:10:24 -05:00
Dimitrios Bariamis
f0bca83ee4 Add support for Mistral Large 3 inference with Flashinfer MoE (#33174) 
Signed-off-by: Dimitrios Bariamis <12195802+dbari@users.noreply.github.com>
Co-authored-by: Dimitrios Bariamis <12195802+dbari@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-01-30 22:48:27 -08:00
Michael Goin
67ebaff528 Refactor NVFP4 Linear utils for ModelOpt and CT (#33201) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-01-30 16:37:42 -08:00
Pavani Majety
c3a9752b0c [Hardware][SM100] Add TRTLLM Kernel for INT4 W4A16 Kernel. (#32437) 
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
 2026-01-30 10:30:46 -08:00
Harry Mellor
a11bc12d53 Fix test_moe.py for Transformers v5 (#33413) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-01-30 14:03:25 +00:00
Richard Zou
d9aa39a3bb [torch.compile] Speed up MOE handling in forward_context (#33184) 
Signed-off-by: Richard Zou <zou3519@gmail.com>
 2026-01-27 15:17:54 -08:00
Robert Shaw
5a93b9162b [MoE Refactor] Integrate Naive Prepare Finalize into MK (#32567) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Signed-off-by: Amir Klein <203507526+amirkl94@users.noreply.github.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: amirkl94 <203507526+amirkl94@users.noreply.github.com>
 2026-01-27 01:28:02 +00:00
Fadi Arafeh
aac0b817fa [CPU Backend][BugFix] Fix failing CPU MoE test (#32876) 
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
 2026-01-23 12:06:51 +00:00
Karan Bansal
fa6e599a61 [Bugfix] Fix _CPU_MOE_ACT AssertionError when vLLM config not set (#32777) 
Signed-off-by: Karan Bansal <karanb192@gmail.com>
 2026-01-23 08:22:37 +00:00
bnellnm
dc917cceb8 [MoE Refactor] Move select_experts from FusedMoEQuantMethod -> FusedMoE (#31996) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2026-01-22 18:21:35 -05:00
Richard Zou
654a71fc3c [torch.compile] Improve Cold Start for MoEs (#32805) 
Signed-off-by: Richard Zou <zou3519@gmail.com>
 2026-01-22 10:44:40 -05:00
Alex Sun
49a1262267 [AMD][ROCm] MoRI EP: a high-performance all2all backend (#28664) 
Signed-off-by: Alex Sun <alex.s@amd.com>
 2026-01-22 16:33:18 +08:00
Xin Yang
63227accf5 [Kernel] Add topk_sigmoid kernel (#31246) 
Signed-off-by: Xin Yang <xyangx@amazon.com>
 2026-01-21 22:49:51 +00:00
elvischenv
808d6fd7b9 Bump Flashinfer to v0.6.1 (#30993) 
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
 2026-01-21 08:49:50 -08:00
Robert Shaw
42135d6898 [MoE Refactor] Oracle Select FP8+NVFP4 Kernels In Priority (#32414) 2026-01-21 08:22:33 -05:00
bnellnm
327a02d8db [MoE Refactor] Separate Router into OO Classes (#30623) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2026-01-18 11:40:49 -05:00
Robert Shaw
4a6af8813f [MoE Refactor] Move Test Impl into Test Dirs (#32129) 
Signed-off-by: Robert Shaw <rshaw@neuralmagic.com>
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
 2026-01-18 12:16:59 +08:00
TomerBN-Nvidia
c277fbdf31 [Feat] Support non-gated MoE with Marlin, NVFP4 CUTLASS, FP8, INT8, compressed-tensors (#32257) 
Signed-off-by: Tomer Natan <tbarnatan@computelab-frontend-8.nvidia.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Tomer Natan <tbarnatan@computelab-frontend-8.nvidia.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Tomer Natan <tbarnatan@ipp1-1429.ipp1a1.colossus.nvidia.com>
 2026-01-15 16:15:05 -08:00
Hongxia Yang
048bb59728 AMD CI Test - unskip moe_sum test and moe_align_block_size tests (#32039) 
Signed-off-by: Hongxia Yang <hongxia.yang@amd.com>
 2026-01-13 23:25:10 -08:00
Rabi Mishra
69f8a0ea37 fix(rocm): Use refresh_env_variables() for rocm_aiter_ops in test_moe (#31711) 
Signed-off-by: rabi <ramishra@redhat.com>
 2026-01-13 19:11:54 +00:00
danielafrimi
3f72639d36 [FIX] Add NO_MUL activation support for modular kernel path (#31528) 
Signed-off-by: dafrimi <dafrimi@nvidia.com>
Signed-off-by: <>
Co-authored-by: root <root@gpu-267.slurm-workers-slurm.slurm.svc.cluster.local>
Co-authored-by: root <root@gpu-537.slurm-workers-slurm.slurm.svc.cluster.local>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: root <root@pool0-01777.cm.cluster>
 2026-01-12 11:55:49 -05:00
Runkai Tao
a4d5d663e2 Add unpermute-aware fused MoE path and small-batch fallback (#29354) 
Signed-off-by: Runkai Tao <rt572@physics.rutgers.edu>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-01-09 12:58:39 -07:00
Lucas Wilkinson
6cdf015c3c [Misc] Fix Current vLLM config is not set. warnings, assert to avoid issues in the future (#31747) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2026-01-08 15:20:49 -08:00
Michael Goin
87e07a6b46 Revert "feat(moe): Add is_act_and_mul=False support for Triton MoE kernels" (#31978) 2026-01-08 11:31:53 -08:00
Robert Shaw
9f6dcb71ae [MoE Refactor][16/N] Apply Refactor to NVFP4 (#31692) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Pavani Majety <pmajety@nvidia.com>
 2026-01-08 03:46:27 +00:00
Rabi Mishra
25eef3dc2e feat(moe): Add is_act_and_mul=False support for Triton MoE kernels (#31645) 
Signed-off-by: rabi <ramishra@redhat.com>
 2026-01-08 10:27:09 +08:00
Robert Shaw
5dcd7ef1f2 [MoE Refactor][15/N] Apply Refactor to Fp8 (#31415) 2026-01-07 19:42:33 -05:00
Xin Yang
0ada960a20 [Kernel] Support bias type in grouped_topk kernel (#31781) 
Signed-off-by: Xin Yang <xyangx@amazon.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-01-07 12:16:32 -08:00
Robert Shaw
af8fd73051 [MoE Refactor][14/N] Clean Up FI Quant Config Smuggling (#31593) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2026-01-06 15:47:04 +00:00
wangxiyuan
bb4337b34c [Platform] Deprecate seed_everything (#31659) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2026-01-04 18:34:04 -08:00
Xinyu Chen
08f425bad1 CustomOp: test forward dispatch for grouped_topk (#31530) 
Signed-off-by: Xinyu Chen <xinyu1.chen@intel.com>
 2026-01-02 10:04:01 -05:00