biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
15,652 Commits 2 Branches 140 Tags
main
Commit Graph

770 Commits

Author SHA1 Message Date
Li, Jiang
5da4c7d789 [CI/Build][CPU] Fix failed pooling tests and macos smoke test (#32907) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
Signed-off-by: Li, Jiang <bigpyj64@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-01-23 10:48:20 +00:00
Eldar Kurtić
44f08af3a7 Add llmcompressor fp8 kv-cache quant (per-tensor and per-attn_head) (#30141) 
Signed-off-by: Eldar Kurtic <8884008+eldarkurtic@users.noreply.github.com>
Signed-off-by: eldarkurtic <8884008+eldarkurtic@users.noreply.github.com>
 2026-01-22 13:29:57 -07:00
Fadi Arafeh
744ef30484 [CPU Backend] [Perf] Accelerate tensor-parallel/data-parallel inference across NUMA domains on Arm (#32792) 
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
 2026-01-22 18:55:23 +00:00
Or Ozeri
421012b63a OffloadingConnector: Support kernel_block_size != block_size (#30692) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2026-01-22 12:30:04 +00:00
Xin Yang
63227accf5 [Kernel] Add topk_sigmoid kernel (#31246) 
Signed-off-by: Xin Yang <xyangx@amazon.com>
 2026-01-21 22:49:51 +00:00
Robert Shaw
85f55c943c [Quantization][Deprecation] Deprecate HQQ (#32681) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2026-01-21 09:32:40 -05:00
Wentao Ye
6c97b9b9b6 [Perf] Only clone when needed for moe_permute (#32273) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-01-20 11:34:39 -05:00
Wentao Ye
eebc58df0c [Refactor] Remove unused cutlass moe problem size function (#32047) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-01-18 12:46:59 -08:00
Hashem Hashemi
7a1030431a Atomics Reduce Counting Optimization for SplitK Skinny GEMMs. (#29843) 
Signed-off-by: Hashem Hashemi <hashem.hashemi@amd.com>
 2026-01-16 11:45:04 -06:00
Michael Goin
83239ff19a Add thread_n=64 support to Marlin MoE (#32360) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-01-15 16:45:44 -08:00
Wentao Ye
f28125d87b [Perf] Optimize grouped topk kernel, 1.2%~2% E2E Throughput improvement (#32058) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-01-13 10:58:18 -08:00
Kevin McKay
c60578de0a [Bugfix][Hardware][AMD] Use dynamic WARP_SIZE in sampler vectorized_process (#31295) 
Signed-off-by: c0de128 <kevin.mckay@outlook.com>
 2026-01-10 03:57:38 +00:00
PatrykSaffer
80fead8bf6 Fuse RoPE and MLA KV-cache write (#25774) 
Signed-off-by: Patryk Saffer <patryk.saffer99@gmail.com>
Signed-off-by: PatrykSaffer <patryk.saffer@mistral.ai>
Co-authored-by: Patryk Saffer <patryk.saffer99@gmail.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-01-09 19:18:37 -08:00
Lucas Wilkinson
0a0aa07747 [Quant] Make static quant support all group shapes (#30833) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-01-09 12:49:27 -08:00
Wentao Ye
308feab33f [Perf] Optimize cutlass moe problem size calculation, 5.3% E2E Throughput improvement, 2.2% TTFT improvement (#31830) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2026-01-09 11:13:43 -08:00
Michael Goin
34cd32fe30 [Perf][Kernel] Fused SiLU+Mul+Quant kernel for NVFP4 cutlass_moe (#31832) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
 2026-01-09 07:40:33 -07:00
R3hankhan
8e27663b6a [CPU] Add head sizes 80 and 112 with vec16 fallback (#31968) 
Signed-off-by: Rehan Khan <Rehan.Khan7@ibm.com>
 2026-01-09 22:14:46 +08:00
Xin Yang
0ada960a20 [Kernel] Support bias type in grouped_topk kernel (#31781) 
Signed-off-by: Xin Yang <xyangx@amazon.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-01-07 12:16:32 -08:00
Michael Goin
f347ac6c34 [Perf] Fuse stride preparation for NVFP4 cutlass_moe (#31837) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-01-07 13:31:26 -05:00
Jinzhen Lin
2f4bdee61e [Quantization][MoE] remove unused ep logic from moe marlin (#31571) 
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-01-06 09:07:19 -08:00
Andreas Karatzas
3ecfdc3776 [ROCm][GPTQ][Bugfix] Fix GPTQ GEMM kernel output zeroing race condition (#30719) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2025-12-29 01:13:14 -08:00
skaraban3807
7cd288a4b3 [PERF] Add interleaved memory allocation to NUMA module (#30800) 2025-12-24 13:47:49 +00:00
Matthew Bonanni
369f47aa0f [DeepSeek v3.2] Remove unnecessary syncwarps (#31047) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-12-23 21:33:30 -08:00
rongfu.leng
4ed11105d7 [Misc] Remove unused custom ops copy_blocks and copy_blocks_mla (#30967) 
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
 2025-12-23 18:22:35 -08:00
danielafrimi
b94f80ffb8 [FIX] FP4 quantization kernel padding initialization bug (#31097) 
Signed-off-by: <>
Co-authored-by: root <root@gpu-193.slurm-workers-slurm.slurm.svc.cluster.local>
Co-authored-by: root <root@gpu-951.slurm-workers-slurm.slurm.svc.cluster.local>
 2025-12-23 08:45:18 -08:00
TJian
022f3cea53 [ROCm] [Critical]: Remove unused variable (#31156) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2025-12-22 08:28:22 -08:00
Jee Jee Li
097978a15d [Kernel] Enable fused_qknorm_rope_kernel supports partial rope (#30821) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-12-21 18:39:22 -08:00
Michael Goin
06d490282f [NVFP4][Perf] Tune NVFP4 input quant kernel for small batch size (#30897) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-12-21 09:41:57 -08:00
Robert Shaw
83a317f650 [MoE Refactor][3/N] Deprecate cutlass block quant fp8 (b200) (#30990) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2025-12-19 13:09:54 -08:00
Nishidha Panpaliya
bd2b52fc2d [CPU][Bugfix] Fix ppc64le CPU build (#30871) 
Signed-off-by: Nishidha Panpaliya <nishidha.panpaliya@partner.ibm.com>
 2025-12-19 12:26:35 +00:00
Li, Jiang
f90d3636e2 [Bugfix][CPU] Fix Mac CPU build (#30955) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-12-18 01:38:22 -08:00
Li, Jiang
e3ab93c896 [CPU] Refactor CPU fused MOE (#30531) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-12-18 14:36:49 +08:00
Sheng Lin
f4e884f222 [NIXL][Bugfix] Fix NIXL/RDMA registration failure over CuMemAllocator (#29569) 
Signed-off-by: Somoku <linsh0@protonmail.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2025-12-17 01:52:58 -08:00
Michael Goin
0a1ab1e565 [Perf][Kernels] Vectorize csrc/activations_kernels.cu (#29512) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-12-16 14:56:02 -08:00
Jinzhen Lin
ce96857fdd [Kernel][Quantization][MoE] add marlin kernel support for turing (sm75) (#29901) 
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-12-16 14:35:28 -08:00
Daniel Cámpora
eaa82a709a [Bugfix][DSV32] Fix overflow in topk. (#30754) 
Signed-off-by: Daniel Campora <961215+dcampora@users.noreply.github.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-12-16 14:21:17 -08:00
Wentao Ye
f21f5ea38c [Refactor] Small refactor for group topk (#30562) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2025-12-16 14:50:59 -05:00
Wentao Ye
1e6b115300 [Refactor] Reduce duplicate code in per_token_group_quant cuda kernels (#30496) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-12-12 16:45:23 -05:00
Lucas Wilkinson
3e41992fec [Attention] Use sparse prefill kernel for fp8 kv-cache in DeepSeek-v3.2 (#27532) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-12-12 05:57:47 -08:00
Bhanu Prakash Voutharoja
6a6fc41c79 gptq marlin quantization support for fused moe with lora (#30254) 
Signed-off-by: Bhanu068 <voutharoja.bhanu06@gmail.com>
 2025-12-12 02:27:22 +00:00
Wentao Ye
61249b177d [Refactor] Remove useless syncwarp (#30510) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-12-11 17:43:41 -05:00
Aditya Tewari
cebda2a4af [CPU] Support for Whisper (#30062) 
Signed-off-by: Aditya Tewari <aditya.tewari@arm.com>
 2025-12-10 04:58:42 -08:00
Wilson Wu
3bdd426636 Fix typos in comments across multiple files (#30345) 
Signed-off-by: Wilson Wu <iwilsonwu@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-12-09 20:05:28 -08:00
Hashem Hashemi
2e7054da06 Improve wvsplitK tile and balance heristics. (#29937) 
Signed-off-by: Hashem Hashemi <hashem.hashemi@amd.com>
 2025-12-09 23:51:32 +00:00
czhu-cohere
f6227c22ab [Kernel]Support W4A8 Grouped GEMM on Hopper (#29691) 
Signed-off-by: czhu-cohere <conway.zhu@cohere.com>
 2025-12-08 19:29:06 -08:00
gnovack
ea657f2078 Lora MoE Align Improvements (#29257) 
Signed-off-by: gnovack <gnovack@amazon.com>
 2025-12-09 10:35:16 +08:00
Wentao Ye
0ee6416f67 [Perf] Optimize group_topk kernel, 1.9% Throughput improvement, 2.1% TPOT improvemnt (#30159) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-12-08 19:44:01 -05:00
Daniel Cámpora
184076c3fe [DeepSeek v3.2] Make top-k work for any logit values. (#27568) 
Signed-off-by: Daniel Campora <961215+dcampora@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-12-08 06:55:58 -08:00
ElizaWszola
af0444bf40 [Performance] Fused blockwise quant RMS norm (#27883) 
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
 2025-12-07 16:38:04 +00:00
Wentao Ye
541a2ef892 [Perf] Deepgemm fused layout kernel for activations, 4.3% throughput improvement, 10.7% TTFT improvement. (#29546) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-12-07 20:31:14 +08:00