biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
15,614 Commits 2 Branches 140 Tags
8332078cfdbd5e44e527893b695e79052d008172
Commit Graph

152 Commits

Author SHA1 Message Date
Roberto L. Castro
b55d830ec7 [Perf][Kernel] Persistent TopK scheduler: unified CUDAGraph-safe kernel with dynamic per-row dispatch - DeepSeek-V3.2 DSA decode (#37421) 
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2026-04-08 13:35:57 -04:00
Gregory Shtrasberg
56c976c1b5 [ROCm] Enable fused_silu_mul_block_quant on ROCm (#38817) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2026-04-08 11:23:32 -05:00
Carl Y
3bc2734dd0 [Kernel] Fuse FP8 output quantization into merge_attn_states (#36518) 
Signed-off-by: Carl You <4531192+carlyou@users.noreply.github.com>
 2026-04-03 01:47:04 +00:00
Gregory Shtrasberg
3aab680e3e [ROCm][Bugfix] Fix ROCm runtime failure due to missing symbol (#38750) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Signed-off-by: Gregory Shtrasberg <156009573+gshtras@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: tjtanaavllm <tunjian.tan@amd.com>
 2026-04-01 21:30:11 -07:00
Monishver
c09ad767cd Feature/silu block quant fusion v1 (#32996) 
Signed-off-by: Monishver Chandrasekaran <monishverchandrasekaran@gmail.com>
 2026-04-01 18:50:43 +00:00
Olya Kozlova
598190aac3 [fix] Remove trtllm ragged mla prefills (#36540) 
Signed-off-by: Olya Kozlova <okozlova@nvidia.com>
 2026-03-31 12:30:27 -07:00
mikaylagawarecki
7c080dd3c5 [4/n] Migrate FP4/W4A8 CUTLASS kernels to torch stable ABI (#37503) 
Signed-off-by: Mikayla Gawarecki <mikaylagawarecki@gmail.com>
 2026-03-31 10:21:13 -07:00
SandishKumarHN
bcc6f67447 [Bugfix] Use null block (0) for padded block table entries (#35431) 
Signed-off-by: SandishKumarHN <sandish@fb.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-03-30 14:02:51 -07:00
mikaylagawarecki
ab1a6a43fa [3/n] Migrate cutlass/scaled_mm_entry.cu torch stable ABI (#37221) 
Signed-off-by: Mikayla Gawarecki <mikaylagawarecki@gmail.com>
 2026-03-30 11:20:13 -07:00
mikaylagawarecki
bf4cc9ed2d [2/n] Migrate per_token_group_quant to torch stable ABI (#36058) 
Signed-off-by: Mikayla Gawarecki <mikaylagawarecki@gmail.com>
 2026-03-25 10:15:13 -07:00
Kyle Sayers
38364a7e32 [Sparse24] [Deprecation] Remove Sparse24 CT integration and kernels (#36799) 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
 2026-03-23 16:03:29 -04:00
mikaylagawarecki
8b10e4fb31 [1/n] Migrate permute_cols to libtorch stable ABI (#31509) 
Signed-off-by: Mikayla Gawarecki <mikaylagawarecki@gmail.com>
 2026-03-19 11:27:26 -04:00
Michael Goin
09e4576f65 [Kernel] Add non-gated support for NVFP4 CUTLASS MoE (#37320) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-03-17 18:12:04 -04:00
Terry Gao
3e6a1e1686 [Custom Ops] Add functional + out variant for scaled_fp4_quant (#34389) 
Signed-off-by: tianrengao <terrygao87@gmail.com>
 2026-03-16 18:51:46 -04:00
Asaf Gardin
bbf81f9a92 [Mamba1] - Kernel Level Chunk Alignment for Prefix Caching (#34798) 
Signed-off-by: Josephasafg <ajgard7@gmail.com>
 2026-03-01 20:40:23 +08:00
Tyler Michael Smith
eb19955c37 [WideEP] Remove pplx all2all backend (#33724) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-26 14:30:10 -08:00
Robert Shaw
6874638bc4 [Model Bash] DeepSeek R1 BF16 Min Latency QKV A GEMM (0.5% E2E Speedup) (#34758) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2026-02-18 07:42:36 -08:00
ElizaWszola
a88b3be7c4 [Bugfix] Fix quant RMS norm fusion for quantization with TMA-aligned scales (#33255) 
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2026-02-17 23:35:04 -08:00
Roberto L. Castro
afdce12c89 [Perf][Kernel] Add faster topKperRow decode kernel for DeepSeek-V3.2 sparse attention (#33680) 
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
 2026-02-10 10:29:52 -05:00
Roberto L. Castro
fcb9df99bd [Perf][Kernel] Optimize FP4 quantization kernels (SM100F) (#32520) 
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
 2026-01-24 18:45:27 -07:00
Wentao Ye
eebc58df0c [Refactor] Remove unused cutlass moe problem size function (#32047) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-01-18 12:46:59 -08:00
Lucas Wilkinson
0a0aa07747 [Quant] Make static quant support all group shapes (#30833) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-01-09 12:49:27 -08:00
Wentao Ye
308feab33f [Perf] Optimize cutlass moe problem size calculation, 5.3% E2E Throughput improvement, 2.2% TTFT improvement (#31830) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2026-01-09 11:13:43 -08:00
Michael Goin
34cd32fe30 [Perf][Kernel] Fused SiLU+Mul+Quant kernel for NVFP4 cutlass_moe (#31832) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
 2026-01-09 07:40:33 -07:00
czhu-cohere
f6227c22ab [Kernel]Support W4A8 Grouped GEMM on Hopper (#29691) 
Signed-off-by: czhu-cohere <conway.zhu@cohere.com>
 2025-12-08 19:29:06 -08:00
Daniel Cámpora
184076c3fe [DeepSeek v3.2] Make top-k work for any logit values. (#27568) 
Signed-off-by: Daniel Campora <961215+dcampora@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-12-08 06:55:58 -08:00
ElizaWszola
af0444bf40 [Performance] Fused blockwise quant RMS norm (#27883) 
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
 2025-12-07 16:38:04 +00:00
Wentao Ye
541a2ef892 [Perf] Deepgemm fused layout kernel for activations, 4.3% throughput improvement, 10.7% TTFT improvement. (#29546) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-12-07 20:31:14 +08:00
Pleaplusone
d9d342d214 [Performance][MLA][ROCm] Remove redundant D2D copy in deepseek (#27457) 
Signed-off-by: ganyi <ygan@amd.com>
 2025-11-26 12:45:28 +08:00
zhrrr
68c09efc37 [Kernel][Perf] fuse QK Norm and RoPE into one cuda kernel for Qwen Model (#27165) 
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
 2025-11-11 12:00:31 -05:00
Asaf Joseph Gardin
00b31a36a2 [V1] [Hybrid] Mamba1 Automatic Prefix Caching (#26377) 
Signed-off-by: asafg <39553475+Josephasafg@users.noreply.github.com>
 2025-11-02 04:16:23 -08:00
Xiangyu Li
5cc6bddb6e [Kernel] Add GPTQv2 format support for low-bit or asymmetric quantization, by adapting gptq_gemm (#26092) 2025-10-23 23:26:13 -04:00
Lain
09a7e6f617 [Deepseek v3.2] Remove extra logics in indexer (#26465) 
Signed-off-by: Siyuan Fu <siyuanf@nvidia.com>
Signed-off-by: Daniel Campora <961215+dcampora@users.noreply.github.com>
Signed-off-by: Lain <siyuanf@nvidia.com>
Co-authored-by: Daniel Campora <961215+dcampora@users.noreply.github.com>
 2025-10-21 23:34:03 +00:00
Daniel Cámpora
80e9452984 [Deepseek v3.2] Optimize top_k_per_row (#26763) 
Signed-off-by: Daniel Campora <961215+dcampora@users.noreply.github.com>
 2025-10-21 08:30:07 +00:00
Isotr0py
3125d79950 [Chore] Remove unused PolyNorm layer (#27110) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-10-17 19:03:43 +00:00
Elvir Crnčević
7b03584de8 Silu v2 (#25074) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: elvircrn <elvircrn@gmail.com>
Signed-off-by: Elvir Crnčević <elvircrn@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Varun Sundar Rabindranath <varunsundar08@gmail.com>
 2025-10-10 15:19:53 +00:00
Daniel Cámpora
e1098ced95 Add topk logits torch op for DS3.2. (#25945) 
Signed-off-by: Daniel Campora <961215+dcampora@users.noreply.github.com>
Signed-off-by: Daniel Cámpora <961215+dcampora@users.noreply.github.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2025-10-07 10:07:32 +00:00
Nikhil Gupta
359d293006 [fix]: add Arm 4bit fused moe support (#23809) 
Signed-off-by: Nikhil Gupta <nikhil.gupta2@arm.com>
 2025-09-24 01:32:22 +00:00
Kyle Sayers
a0b26701c9 [Transform] Deterministic Hadacore Transforms (#24106) 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
 2025-09-15 12:59:31 -06:00
Elvir Crnčević
98229db244 [Kernels][DP/EP] Optimize Silu Kernel for R1 (#24054) 
Signed-off-by: elvircrn <elvircrn@gmail.com>
 2025-09-13 00:17:27 -07:00
Woosuk Kwon
5febdc8750 [Chore] Remove unused batched RoPE op & kernel (#24789) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-09-13 00:08:20 -07:00
TaehyunKim
9bd831f501 [Model] New model support for Motif-1-Tiny (#23414) 
Signed-off-by: ca1207 <ca1207zzz@gmail.com>
Signed-off-by: TaehyunKim <73943231+ca1207@users.noreply.github.com>
Co-authored-by: WyldeCat <skan1543@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-10 23:29:40 -07:00
elvischenv
adc3ddb430 [Bugfix][Misc] Fix silu_and_mul_nvfp4_quant issue and extract common utils for nvfp4 kernel source files (#23727) 
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2025-09-04 14:25:45 -07:00
yzds
0dc9532065 [BUGFIX ] fix undefined silu_and_mul_nvfp4_quant (#23929) 
Signed-off-by: hongchao <hongchao@msh.team>
Signed-off-by: Richard Zou <zou3519@gmail.com>
Co-authored-by: hongchao <hongchao@msh.team>
Co-authored-by: Richard Zou <zou3519@gmail.com>
Co-authored-by: Richard Zou <zou3519@users.noreply.github.com>
 2025-08-29 09:36:39 -07:00
elvischenv
16a45b3a28 [NVIDIA] Support SiluMul + NVFP4 quant fusion (#23671) 
Signed-off-by: jindih <jindih@nvidia.com>
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
Co-authored-by: jindih <jindih@nvidia.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Luka Govedic <lgovedic@redhat.com>
 2025-08-28 19:36:50 +00:00
shixianc
b17109beea [Kernel] CUTLASS MoE FP8: Integrate cuda moe permute/unpermute (#23045) 
Signed-off-by: Shixian Cui <shixian@amazon.com>
 2025-08-20 10:35:26 -04:00
Jee Jee Li
4d4061b6e7 [Kernel] Add cuda kernel for gpt_oss activation (#22951) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-08-17 05:03:24 +00:00
Michael Goin
4fc722eca4 [Kernel/Quant] Remove AQLM (#22943) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-08-16 19:38:21 +00:00
Woosuk Kwon
1c859a1387 [V0 Deprecation] Remove advance_step (#22969) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-08-15 08:22:31 -07:00
Simon Mo
f1f0d2fab8 Revert "[Kernel] Add cuda kernel for gpt_oss activation" (#22948) 2025-08-14 17:38:10 -07:00