biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
15,614 Commits 2 Branches 140 Tags
8332078cfdbd5e44e527893b695e79052d008172
Commit Graph

112 Commits

Author SHA1 Message Date
Xin Yang
9bd7231106 Revert "[Kernel] Add gpt-oss Router GEMM kernel (#37205)" (#38778) 
Signed-off-by: Xin Yang <xyangx@amazon.com>
 2026-04-01 22:02:32 -07:00
Michael Goin
db5d0719e1 [Kernel] Add MXFP8 to Marlin GEMM/MoE and refactor Mxfp8LinearOp (#34664) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-04-01 09:41:42 -07:00
IriKa
148a5c1226 [Bugfix]fix output Nan/Inf in marlin if dtype=float16 (#33972) 
Signed-off-by: IriKa Qiu <qiujie.jq@gmail.com>
 2026-03-27 16:36:08 -07:00
Xin Yang
b1169d7be8 [Kernel] Add gpt-oss Router GEMM kernel (#37205) 
Signed-off-by: Xin Yang <xyangx@amazon.com>
 2026-03-18 08:15:56 -07:00
Wentao Ye
e855d380fa [Compile] Fix compile warning in moe_permute (#36529) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-03-16 10:16:14 -04:00
Jiayi Yan
6a895197fa [Bugfix][CI] fix typos (#34934) 
Signed-off-by: 1195343015 <1195343015@qq.com>
Signed-off-by: Jiayi Yan <66017932+1195343015@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-05 17:05:46 +00:00
EdalatiAli
cb21972a97 [Kernel] Integrate SM100 MXFP8 blockscaled grouped MM and quant kernels (#34448) 
Signed-off-by: EdalatiAli <aliedalati@cohere.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-03-01 23:31:19 -08:00
roikoren755
38c498b8e3 [Performance] Cublas Bf16 Gate with Fp32 Output (#35121) 
Signed-off-by: Roi Koren <roik@nvidia.com>
 2026-02-26 16:51:28 -08:00
Xin Yang
3bbb2046ff [Bugfix] Fix expert_ids padding values in moe_align_block_size kernel (#35161) 
Signed-off-by: Xin Yang <xyangx@amazon.com>
 2026-02-24 17:14:24 -08:00
Michael Goin
3ef9fd0f98 [Bugfix] Fix DSV3 kernels breaking _C and _moe_C on unsupported arches (#35123) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-02-23 17:11:27 -08:00
Robert Shaw
8435b2e049 [ModelBash][DSV3] Add TRTLLM DSV3 Router GEMM kernel (6% B1 Speedup) (#34302) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2026-02-23 14:02:26 +00:00
Xin Yang
b1c4f0b265 [Kernel] Optimize grouped topk kernel (#34206) 
Signed-off-by: Xin Yang <xyangx@amazon.com>
 2026-02-20 01:34:45 -08:00
Wentao Ye
77c09e1130 [Refactor] Remove align block size logic in moe_permute (#33449) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-02-06 10:57:06 -08:00
linhaifeng
fedf64332e [Bugfix]: Fix display errors in TORCH_CHECK messages (#32942) 
Signed-off-by: linhaifeng <1371675203@qq.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-01-31 09:48:48 -08:00
Wentao Ye
c4e744dbd4 [Perf] Optimize moe_permute for CUTLASS FP8 (#32892) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-01-28 10:15:24 -08:00
Michael Goin
4561f13985 [Refactor] Rename gptq_marlin to marlin to match MoE (#32952) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-01-23 16:48:12 -05:00
Xin Yang
63227accf5 [Kernel] Add topk_sigmoid kernel (#31246) 
Signed-off-by: Xin Yang <xyangx@amazon.com>
 2026-01-21 22:49:51 +00:00
Wentao Ye
6c97b9b9b6 [Perf] Only clone when needed for moe_permute (#32273) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-01-20 11:34:39 -05:00
Michael Goin
83239ff19a Add thread_n=64 support to Marlin MoE (#32360) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-01-15 16:45:44 -08:00
Wentao Ye
f28125d87b [Perf] Optimize grouped topk kernel, 1.2%~2% E2E Throughput improvement (#32058) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-01-13 10:58:18 -08:00
Xin Yang
0ada960a20 [Kernel] Support bias type in grouped_topk kernel (#31781) 
Signed-off-by: Xin Yang <xyangx@amazon.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-01-07 12:16:32 -08:00
Jinzhen Lin
2f4bdee61e [Quantization][MoE] remove unused ep logic from moe marlin (#31571) 
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-01-06 09:07:19 -08:00
Jinzhen Lin
ce96857fdd [Kernel][Quantization][MoE] add marlin kernel support for turing (sm75) (#29901) 
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-12-16 14:35:28 -08:00
Wentao Ye
f21f5ea38c [Refactor] Small refactor for group topk (#30562) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2025-12-16 14:50:59 -05:00
Bhanu Prakash Voutharoja
6a6fc41c79 gptq marlin quantization support for fused moe with lora (#30254) 
Signed-off-by: Bhanu068 <voutharoja.bhanu06@gmail.com>
 2025-12-12 02:27:22 +00:00
Wentao Ye
61249b177d [Refactor] Remove useless syncwarp (#30510) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-12-11 17:43:41 -05:00
gnovack
ea657f2078 Lora MoE Align Improvements (#29257) 
Signed-off-by: gnovack <gnovack@amazon.com>
 2025-12-09 10:35:16 +08:00
Wentao Ye
0ee6416f67 [Perf] Optimize group_topk kernel, 1.9% Throughput improvement, 2.1% TPOT improvemnt (#30159) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-12-08 19:44:01 -05:00
Jinzhen Lin
879ddb09c3 [Kernel][MoE] optimize moe_align_block_size (#29642) 
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-12-07 01:58:47 -08:00
Zhang Xiangze
13ea39bc09 [CPU]Parallelize over tokens in int4 moe (#29600) 
Signed-off-by: Zhang Xiangze <Xiangze.Zhang@arm.com>
 2025-12-02 06:21:39 +00:00
Jinzhen Lin
1656ad3704 [Kernel][Quantization] add w4a8 support for marlin kernel (#24722) 
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin@redhat.com>
 2025-11-29 07:19:33 -08:00
Jinzhen Lin
a67dec7cba [Bugfix] fix IMA issue in certain cases of the moe marlin kernel (#28619) 
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-11-26 19:02:21 -08:00
Michael Goin
0852527647 [Perf][DeepSeek] Add sigmoid+bias fusion to fused_grouped_topk from TRTLLM (#28124) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-11-07 18:20:55 -08:00
xiangze-arm
f32cbc9a0c [CPU]Improve dynamic 4bit moe performance (#27240) 
Signed-off-by: Zhang Xiangze <Xiangze.Zhang@arm.com>
 2025-11-04 06:33:23 +00:00
gnovack
294c805f1d Early exit for MoE LoRA kernels (#27131) 
Signed-off-by: gnovack <gnovack@amazon.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-11-03 20:22:17 +08:00
gnovack
8e4ca4d14e Bugfix - pass 'max_num_tokens_padded' into 'moe_lora_align_block_size' (#27311) 
Signed-off-by: gnovack <gnovack@amazon.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-10-22 12:23:57 +00:00
Chen Wu
5f6cbf60d6 [Feature][Kernel]FusedMoE LoRA (#21229) 
Signed-off-by: wuchen <cntryroa@gmail.com>
Signed-off-by: banjuede <lmklhc@163.com>
Signed-off-by: Chen Wu <cntryroa@gmail.com>
Signed-off-by: Danielle Robinson <dmmaddix@amazon.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: bk-201 <joy25810@foxmail.com>
Co-authored-by: wuchen <wuchen@zetyun.com>
Co-authored-by: Nathan Van Gheem <vangheem@gmail.com>
Co-authored-by: banjuede <lmklhc@163.com>
Co-authored-by: Danielle Robinson <dmmaddix@amazon.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: bk-201 <joy25810@foxmail.com>
 2025-10-21 03:01:37 +00:00
zhrrr
75c7ad9918 [Kernel][Performance] Fuse float cast and renormalize to topk softmax kernel (#26717) 
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
Signed-off-by: izhuhaoran <izhuhaoran@qq.com>
 2025-10-17 07:30:35 +00:00
Varun Sundar Rabindranath
fb0571b077 [GPTOSS][DP/EP][Marlin] Enable GPTOSS Batched DP/EP using Marlin kernels (#25997) 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
 2025-10-16 12:53:11 -07:00
Bram Wasti
3263799056 [unrevert] Add batch invariant kernel override for FlashInfer backend [2/n] (#26373) 
Signed-off-by: Bram Wasti <bwasti@meta.com>
Signed-off-by: Bram Wasti <bwasti@fb.com>
 2025-10-13 10:24:53 -04:00
Harry Mellor
d6953beb91 Convert formatting to use ruff instead of yapf + isort (#26247) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 07:06:22 -07:00
Bram Wasti
dc48ba0c75 Kernel-override Determinism [1/n] (#25603) 
Signed-off-by: Bram Wasti <bwasti@meta.com>
 2025-09-26 16:59:09 -07:00
XuruiYang
845adb3ec6 [Model] Add LongCat-Flash (#23991) 
Signed-off-by: yangxurui <yangxurui@meituan.com>
Co-authored-by: yangxurui <yangxurui@meituan.com>
 2025-09-24 21:53:40 -07:00
Nikhil Gupta
359d293006 [fix]: add Arm 4bit fused moe support (#23809) 
Signed-off-by: Nikhil Gupta <nikhil.gupta2@arm.com>
 2025-09-24 01:32:22 +00:00
Ming Yang
527821d191 Use macro guard CUDA functions for back compatibility in grouped_topk_kernel.cu (#25346) 
Signed-off-by: Ming Yang <minos.future@gmail.com>
Signed-off-by: Rahul Tuli <rtuli@redhat.com>
Co-authored-by: Rahul Tuli <rtuli@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com>
Co-authored-by: Ye (Charlotte) Qi <yeq@meta.com>
 2025-09-23 09:45:39 -07:00
Lumina
81b16a2bc9 [Kernel] Better inf handling for grouped topk cu (#24886) 
Signed-off-by: lumina37 <starry.qvq@gmail.com>
 2025-09-18 05:53:55 +00:00
Aidyn-A
bfe9380161 Apply fixes for CUDA 13 (#24599) 
Signed-off-by: Aidyn-A <aidyn.b.aitzhan@gmail.com>
 2025-09-17 09:15:42 -04:00
Qiming Zhang
e919d6f549 [Kernel][Bugfix] Fix grouped topk cu (#24146) 
Signed-off-by: mayuyuace <qiming1.zhang@intel.com>
 2025-09-04 12:37:37 +08:00
Luka Govedič
4f35be10a9 [BugFix] Fix topk_softmax assert (#19764) 
Signed-off-by: Luka Govedic <lgovedic@redhat.com>
 2025-08-27 09:47:28 -07:00
Xin Yang
8a3cd90af5 [Kernel] Add fused grouped_topk kernel for MoE (#23274) 
Signed-off-by: Xin Yang <xyangx@amazon.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-08-25 11:47:52 -07:00