biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
15,552 Commits 2 Branches 140 Tags
e8ebbdde8304a8cf89bbd4e101ebdfc25118b125
Commit Graph

223 Commits

Author SHA1 Message Date
Yongye Zhu
e8ebbdde83 [Quantization] Add FlashInfer CuteDSL batched experts backend for NVFP4 MoE (#38251) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-04-06 11:57:53 -07:00
bnellnm
f01482408c [MoE Refactor][Test] FusedMoE layer test (#24675) 
Signed-off-by: Bill Nell <bnell@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-04-06 17:17:23 +00:00
Li, Jiang
c6f722b93e [CPU] Support gelu act in cpu_fused_moe (#38770) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2026-04-02 14:14:32 +08:00
Xin Yang
9bd7231106 Revert "[Kernel] Add gpt-oss Router GEMM kernel (#37205)" (#38778) 
Signed-off-by: Xin Yang <xyangx@amazon.com>
 2026-04-01 22:02:32 -07:00
Michael Goin
db5d0719e1 [Kernel] Add MXFP8 to Marlin GEMM/MoE and refactor Mxfp8LinearOp (#34664) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-04-01 09:41:42 -07:00
bnellnm
7cf56a59a2 [MoE Refactor] Make SharedExperts class for use with DefaultMoERunner (#35153) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2026-04-01 09:44:08 -04:00
yzong-rh
d9b90a07ac [MoE Refactor] Migrate Unquantized to Full Oracle Flow (#36286) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Signed-off-by: yzong-rh <yzong@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2026-03-31 15:43:33 -04:00
SandishKumarHN
3896e021a0 [Bugfix] Fix FusedMoE weight loading with padded hidden dimensions (#37010) 
Signed-off-by: SandishKumarHN <sandish@fb.com>
 2026-03-31 12:22:26 -04:00
wliao2
4dfad17ed1 replace cuda_device_count_stateless() to current_platform.device_count() (#37841) 
Signed-off-by: Liao, Wei <wei.liao@intel.com>
Signed-off-by: wliao2 <wei.liao@intel.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-03-31 22:32:54 +08:00
Ilya Markov
12701e8af2 [EPLB] Optmize eplb mapping and record in router for prefill (#36261) 
Signed-off-by: ilmarkov <markovilya197@gmail.com>
 2026-03-30 19:48:33 +00:00
Johnny
b4a2f3ac36 [NVIDIA] Bugfix NVFP4 DGX Spark and RTX50 (#38423) 
Signed-off-by: johnnynunez <johnnynuca14@gmail.com>
Signed-off-by: Johnny <johnnynuca14@gmail.com>
 2026-03-30 09:36:18 -07:00
Andreas Karatzas
bdc1719eb9 [ROCm][CI] Fix AITER state leak in shared_fused_moe_routed_transform test (#38137) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-03-26 09:26:46 -07:00
Zhewen Li
be1a85b7a2 Revert "[MoE Kernel] Flashinfer nvfp4 cutedsl moe kernel integration" (#38050) (#38169) 
Co-authored-by: Zhewen Li <zhewenli@inferact.ai>
 2026-03-26 07:59:09 -07:00
Andreas Karatzas
7d6917bef5 [ROCm] Fix MoE kernel test failures on gfx950 (#37833) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Matthew Wong <Matthew.Wong2@amd.com>
 2026-03-25 13:46:40 -05:00
Yongye Zhu
678b3c99e8 [MoE Kernel] Flashinfer nvfp4 cutedsl moe kernel integration (#38050) 2026-03-25 10:16:40 -07:00
Ranran
dc6908ac6a [Bugfix] Register VLLM_BATCH_INVARIANT in envs.py to fix spurious unknown env var warning (#35007) 
Signed-off-by: Ranran <1012869439@qq.com>
Signed-off-by: Ranran <hzz5361@psu.edu>
Signed-off-by: ran <hzz5361@psu.edu>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-03-23 18:31:14 -04:00
Robert Shaw
4383f1532e [MoE] Move PF Methods to Folder (#35927) 2026-03-22 02:42:59 -06:00
Robert Shaw
6b2fa3a762 [MoE] Move FlashInfer CuteDSL experts into fused_moe/experts/ (#37759) 
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
 2026-03-21 19:15:16 -04:00
Yongye Zhu
87bd91892f [MoE Refactor] Mxfp4 oracle rebased (#37128) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-21 03:37:04 +00:00
Xin Yang
d0532bf38d [Perf] Eliminate redundant SparseMatrix creation in gpt_oss_triton_kernels (#37683) 
Signed-off-by: Xin Yang <xyangx@amazon.com>
 2026-03-20 11:28:41 -06:00
rasmith
98ff042917 [CI][BugFix][AMD] Don't set VLLM_ROCM_USE_AITER anymore in test_rocm_aiter_topk since its not necessary (#36996) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
 2026-03-20 07:12:45 +08:00
Xin Yang
b1169d7be8 [Kernel] Add gpt-oss Router GEMM kernel (#37205) 
Signed-off-by: Xin Yang <xyangx@amazon.com>
 2026-03-18 08:15:56 -07:00
Andrey Talman
68f783a727 [Torch 2.11] Guard torch._C._cpu attribute checks for forward compatibility (#35673) 
Signed-off-by: atalman <atalman@fb.com>
 2026-03-17 18:47:59 +00:00
leo-cf-tian
2754231ba3 [Kernel] Add FlashInfer MoE A2A Kernel (#36022) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Signed-off-by: Leo Tian <lctian@nvidia.com>
Co-authored-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Stefano Castagnetta <scastagnetta@nvidia.com>
Co-authored-by: root <root@lyris0267.lyris.clusters.nvidia.com>
 2026-03-15 23:45:32 -07:00
Xinan Miao
2cdf92228c [Feature]: Remove Chunking From FusedMoE (#34086) 
Signed-off-by: SouthWest7 <am1ao@qq.com>
Signed-off-by: Southwest <1403572259@qq.com>
Signed-off-by: southwest <am1ao@qq.com>
Signed-off-by: Xinan Miao <1403572259@qq.com>
Co-authored-by: SouthWest7 <am1ao@qq.com>
 2026-03-12 14:24:38 -04:00
Kunshang Ji
53ec16a705 [Hardware] Replace torch.cuda.device_count/current_device/set_device API (#36145) 
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-03-12 07:57:47 -07:00
Wei Zhao
2e693f48e7 [Perf] Add TRTLLM FP8 MoE Modular Kernel (#36307) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-03-12 07:32:31 -07:00
danisereb
0a6a3a1290 Add support for ModelOpt MXFP8 MoE models (#35986) 
Signed-off-by: Daniel Serebrenik <daserebrenik@nvidia.com>
 2026-03-08 13:00:05 -07:00
Jiayi Yan
6a895197fa [Bugfix][CI] fix typos (#34934) 
Signed-off-by: 1195343015 <1195343015@qq.com>
Signed-off-by: Jiayi Yan <66017932+1195343015@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-05 17:05:46 +00:00
Kunshang Ji
66a2209645 [Hardware] Replace torch.cuda.synchronize() api with torch.accelerator.synchronize (#36085) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-03-05 10:36:39 +00:00
Kunshang Ji
16d2ad1d38 [Hardware] Replace torch.cuda.empty_cache with torch.accelerator.empty_cache (#30681) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-04 09:49:47 +00:00
Robert Shaw
97995f6376 [MoE Refactor] Create MK for TRTLLM Kernels (#32564) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Signed-off-by: Robert Shaw <rshaw@neuralmagic.com>
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
 2026-03-03 10:39:50 -08:00
Turner Jabbour
4034c3d32e [Core] Move test utility to test file (#35672) 
Signed-off-by: Turner Jabbour <doubleujabbour@gmail.com>
 2026-03-02 10:56:03 -05:00
EdalatiAli
cb21972a97 [Kernel] Integrate SM100 MXFP8 blockscaled grouped MM and quant kernels (#34448) 
Signed-off-by: EdalatiAli <aliedalati@cohere.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-03-01 23:31:19 -08:00
Tyler Michael Smith
eb19955c37 [WideEP] Remove pplx all2all backend (#33724) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-26 14:30:10 -08:00
Xin Yang
3bbb2046ff [Bugfix] Fix expert_ids padding values in moe_align_block_size kernel (#35161) 
Signed-off-by: Xin Yang <xyangx@amazon.com>
 2026-02-24 17:14:24 -08:00
tacos8me
b7892a3bef [Model] Add NVFP4 quantization support for Step3.5-Flash (#34478) 
Signed-off-by: tacos8me <ian@cloudhabit.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-02-22 12:30:46 -07:00
Xin Yang
b1c4f0b265 [Kernel] Optimize grouped topk kernel (#34206) 
Signed-off-by: Xin Yang <xyangx@amazon.com>
 2026-02-20 01:34:45 -08:00
Wenlong Wang
847a57cd12 [Bugfix][MoE Kernel] Fix incorrect routing selection for models without expert groups (e.g., MiniMax-M2.1) (#34673) 
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-02-18 13:03:24 -08:00
rasmith
fcd6ac97ed [CI][AMD][BugFix] Skip tests in test_unquantized_backend_selection that should not run on ROCm (#34655) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
 2026-02-18 15:00:40 -05:00
haosdent
b68fd899d1 [Bugfix] Fix fused MoE int32 overflow in stride*offset without perf regression (#34507) 
Signed-off-by: haosdent <haosdent@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-02-16 17:58:49 -08:00
haosdent
79f3fab05a [Bugfix] Handle num_expert_group=None in flashinfer block-scale FP8 MoE (#34494) 
Signed-off-by: haosdent <haosdent@gmail.com>
 2026-02-14 23:25:46 -08:00
Marek Michalowski
742d214d6e [Bugfix] fix the import path in moe test utils.py (#34245) 
Signed-off-by: Marek Michalowski <marek.michalowski@arm.com>
 2026-02-13 00:13:45 -08:00
amitz-nv
f120bd42d3 [Kernel] Support Flashinfer trtllm fused MoE non gated FP8 & NVFP4 (#33506) 
Signed-off-by: amitz-nv <203509407+amitz-nv@users.noreply.github.com>
 2026-02-12 13:06:58 -08:00
Michael Goin
ff1f83b056 [Refactor] Replace activation: str with MoEActivation enum (#33843) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
 2026-02-11 17:29:32 -08:00
Wei Zhao
5aff2699bd Fix CI failure - Flashinfer Kernel tests (#34316) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
 2026-02-11 14:17:16 -08:00
Linda
275e0d2a99 [NVIDIA][test] Tests for flashinfer TRTLLM BF16 MoE (#33715) 
Signed-off-by: Linda-Stadter <57756729+Linda-Stadter@users.noreply.github.com>
Co-authored-by: Pavani Majety <pmajety@nvidia.com>
 2026-02-11 12:38:11 +00:00
bnellnm
d1481ba783 [MoE Refactor] Introduce MoERunner abstraction and move execution logic from FusedMoE to DefaultMoERunner (#32344) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2026-02-10 19:51:07 -05:00
xuebwang-amd
b129136c7a [ROCm][Quantization] GPT_OSS in amd-quark format model loading and emulations (#29008) 
Signed-off-by: xuebwang-amd <xuebwang@amd.com>
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-02-10 10:08:05 -05:00
Andrey Talman
f97ca67176 [Release 2.10] Update to Torch 2.10 - final release (#30525) 2026-02-08 13:51:09 -08:00