biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
15,552 Commits 2 Branches 140 Tags
e8ebbdde8304a8cf89bbd4e101ebdfc25118b125
Commit Graph

139 Commits

Author SHA1 Message Date
Carl Y
3bc2734dd0 [Kernel] Fuse FP8 output quantization into merge_attn_states (#36518) 
Signed-off-by: Carl You <4531192+carlyou@users.noreply.github.com>
 2026-04-03 01:47:04 +00:00
Stefano Castagnetta
58262dec6e [Bugfix] Fix test mocks after SM100 restriction in #38730 (#38791) 
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-04-02 13:12:58 -04:00
Li, Jiang
36d7f19897 [CPU] Support head_size 512 in cpu_attn (#38676) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2026-04-01 05:42:27 +00:00
Olya Kozlova
598190aac3 [fix] Remove trtllm ragged mla prefills (#36540) 
Signed-off-by: Olya Kozlova <okozlova@nvidia.com>
 2026-03-31 12:30:27 -07:00
Ranran
dc6908ac6a [Bugfix] Register VLLM_BATCH_INVARIANT in envs.py to fix spurious unknown env var warning (#35007) 
Signed-off-by: Ranran <1012869439@qq.com>
Signed-off-by: Ranran <hzz5361@psu.edu>
Signed-off-by: ran <hzz5361@psu.edu>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-03-23 18:31:14 -04:00
Andreas Karatzas
3ffa52009f [ROCm][CI] Guard CudaPlatform/RocmPlatform imports to fix test collection on cross-platform builds (#37617) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-03-21 11:58:58 +08:00
Andreas Karatzas
58cde5c026 [ROCm][CI] Skip trtllm kvfp8 dequant tests on ROCm (#37330) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-03-18 11:12:26 +08:00
Andrey Talman
68f783a727 [Torch 2.11] Guard torch._C._cpu attribute checks for forward compatibility (#35673) 
Signed-off-by: atalman <atalman@fb.com>
 2026-03-17 18:47:59 +00:00
Vadim Gimpelson
6c1cfbad32 Support non-contiguous KV cache in TRTLLM fp8 dequant kernel (#36867) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Signed-off-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
Co-authored-by: Pavani Majety <pavanimajety@gmail.com>
 2026-03-16 17:48:42 -07:00
grimulkan
a1257fd1ea [Kernel] Add FP8 KV cache support to Triton MLA decode attention (#34597) 
Signed-off-by: grimulkan <grimulkan@gmail.com>
 2026-03-12 08:32:34 -07:00
Kunshang Ji
53ec16a705 [Hardware] Replace torch.cuda.device_count/current_device/set_device API (#36145) 
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-03-12 07:57:47 -07:00
Shanshan Shen
f0d3658c0f [MM][OOT] Support CPU seq_lens for OOT MMEncoderAttention kernels (#36605) 
Signed-off-by: shen-shanshan <467638484@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-03-12 03:28:23 -07:00
Julien Denize
a5d06dc557 Add 320 dimension size support to MLA (#36161) 
Signed-off-by: Julien Denize <julien.denize@mistral.ai>
 2026-03-11 10:21:22 -07:00
Wuxun Zhang
e584dce52b Add XPU MLA Sparse backend for DeepSeek v3.2 (#33230) 
Signed-off-by: Zhang, Wuxun <wuxun.zhang@intel.com>
 2026-03-11 19:19:15 +08:00
Matthew Bonanni
77a73458e3 Reapply [Attention] Refactor check_and_update_config (#35122) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-03-09 07:17:14 -07:00
Isotr0py
b0906d8b02 [MM Encoder] Default to use TORCH_SDPA backend for ViT on Volta/Turing GPU (#36472) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-03-09 03:43:44 -07:00
Alexei-V-Ivanov-AMD
225d1090a0 Enabling some B200-specific tests on MI355 (#35253) 
Signed-off-by: Alexei V. Ivanov <alexei.ivanov@amd.com>
Signed-off-by: Alexei-V-Ivanov-AMD <156011006+Alexei-V-Ivanov-AMD@users.noreply.github.com>
 2026-03-06 19:27:20 +00:00
Jiayi Yan
6a895197fa [Bugfix][CI] fix typos (#34934) 
Signed-off-by: 1195343015 <1195343015@qq.com>
Signed-off-by: Jiayi Yan <66017932+1195343015@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-05 17:05:46 +00:00
Sage Moore
8c760b6ab6 [ROCm] Refactor ROCm attention backend selection logic (#35246) 
Signed-off-by: Sage Moore <sage@neuralmagic.com>
 2026-03-05 10:51:26 -06:00
Kunshang Ji
66a2209645 [Hardware] Replace torch.cuda.synchronize() api with torch.accelerator.synchronize (#36085) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-03-05 10:36:39 +00:00
Nicolò Lucchesi
18e01a0a10 [Misc] Add --attention-backend auto option (#35738) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-03-04 15:12:27 +00:00
ojhaanshika
e05cb3b93e TRTLLM gen-full attn Test Coverage (#34986) 
Signed-off-by: Anshika Ojha <anshikao@nvidia.com>
Co-authored-by: Anshika Ojha <anshikao@gb-nvl-059-compute09.nvidia.com>
 2026-03-03 11:35:34 -05:00
Max Hu
9c3fe9936b Flashinfer cuDNN backend for Qwen3 VL ViT attention (#34580) 
Signed-off-by: Max Hu <maxhu@nvidia.com>
Signed-off-by: Max Hu <hyoung2991@gmail.com>
Co-authored-by: Max Hu <maxhu@nvidia.com>
Co-authored-by: Shang Wang <shangw@nvidia.com>
 2026-02-27 20:20:23 +08:00
Andrii Skliar
56a6371706 [Update] Use FlashInfer fast_decode_plan directly instead of replication (#34687) 
Signed-off-by: Andrii <askliar@nvidia.com>
Co-authored-by: Andrii <askliar@nvidia.com>
 2026-02-26 16:31:43 -08:00
Kunshang Ji
8ad54a991b [Platform] Add current_platform.num_compute_units interface (#35042) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
 2026-02-24 22:22:49 -08:00
Eldar Kurtić
a87cc50859 [Attn,KV-cache] Use per-head scales in the attention selector (#34281) 
Signed-off-by: Your Name <you@example.com>
Signed-off-by: Eldar Kurtic <research@neuralmagic.com>
Co-authored-by: Eldar Kurtic <research@neuralmagic.com>
Co-authored-by: Your Name <you@example.com>
 2026-02-24 09:02:43 -05:00
Burkhard Ringlein
e24663c5a9 Add unit tests for fp8 output fusion of triton_attn (#34228) 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2026-02-18 06:22:49 -05:00
Isotr0py
71cd89264f [MM Encoder] Add Triton ViT attention backend (#32183) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-02-15 06:32:47 -08:00
Andreas Karatzas
350ca72c04 [ROCm][AITER] Fix AITER import regression for explicit backend selection (#33749) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-02-06 15:08:16 +00:00
Xin Yang
79028d4388 [Perf] Disable clean_logits in deepgemm fp8_mqa_logits kernel (#33568) 2026-02-05 20:34:00 -05:00
R3hankhan
4dffc5e044 [CPU] Split attention dispatch by head_dim alignment (#32161) 
Signed-off-by: Rehan Khan <Rehan.Khan7@ibm.com>
 2026-02-03 19:37:15 -08:00
Matthew Bonanni
a608b4c6c2 [5/N][Attention] Finish eliminating vllm/attention folder (#32064) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-01-27 10:02:51 -05:00
Matt
305e53ade8 [Hardware][AMD][CI][Bugfix] Fix Kernels Attention Cache test (#32904) 
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
 2026-01-23 16:24:26 +00:00
Eldar Kurtić
44f08af3a7 Add llmcompressor fp8 kv-cache quant (per-tensor and per-attn_head) (#30141) 
Signed-off-by: Eldar Kurtic <8884008+eldarkurtic@users.noreply.github.com>
Signed-off-by: eldarkurtic <8884008+eldarkurtic@users.noreply.github.com>
 2026-01-22 13:29:57 -07:00
Or Ozeri
421012b63a OffloadingConnector: Support kernel_block_size != block_size (#30692) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2026-01-22 12:30:04 +00:00
Lucas Wilkinson
b4f64e5b02 Update FlashMLA (#32491) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-01-21 13:03:37 +08:00
shyeh25
1c46dea001 Revert "[Kernels][FI] Skip trtllm attention when num_kv_heads=1 (#308… (#31617) 
Signed-off-by: shyeh25 <206795756+shyeh25@users.noreply.github.com>
 2026-01-10 12:39:59 -08:00
Matthew Bonanni
2612ba9285 [1/N][Attention] Restructure attention: move files (#31916) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-01-09 13:10:24 -08:00
vllmellm
1a19e9cd87 [Bugfix][ROCm]Fix Qwen3-Next-80B-A3B-Thinking inference and optimize non-standard block size (544) support under rocm_atten (#31380) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
 2026-01-09 19:28:02 +08:00
Lucas Wilkinson
6cdf015c3c [Misc] Fix Current vLLM config is not set. warnings, assert to avoid issues in the future (#31747) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2026-01-08 15:20:49 -08:00
Isotr0py
6aa5b18e1d [v1] Add encoder-only/cross attention support to Triton Attention backend (#31406) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-01-06 00:00:23 +08:00
wangxiyuan
bb4337b34c [Platform] Deprecate seed_everything (#31659) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2026-01-04 18:34:04 -08:00
rongfu.leng
4ed11105d7 [Misc] Remove unused custom ops copy_blocks and copy_blocks_mla (#30967) 
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
 2025-12-23 18:22:35 -08:00
Isotr0py
700a5ad6c6 [MM Encoder]: Migrate legacy ViT MultiHeadAttention to new MMEncoderAttention interface (#30684) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-12-19 02:04:19 +08:00
Matthew Bonanni
7eb6cb6c18 [Attention] Update tests to remove deprecated env vars (#30563) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-12-17 09:49:59 -08:00
Ye (Charlotte) Qi
a100152288 [Kernels][FI] Skip trtllm attention when num_kv_heads=1 (#30842) 
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
 2025-12-17 01:54:21 -08:00
Roberto L. Castro
4fa7ce46f3 [Feature] Add SM103 (Blackwell Ultra) Support to vLLM (#30484) 
Signed-off-by: LopezCastroRoberto <robertol.c510@gmail.com>
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2025-12-12 19:34:23 -08:00
jvlunteren
9c0ee995a8 [Kernel] Support CUDA Graphs in 3D Triton Attention Kernel (#28306) 
Signed-off-by: Jan van Lunteren <jvl@zurich.ibm.com>
Signed-off-by: jvlunteren <161835099+jvlunteren@users.noreply.github.com>
Co-authored-by: Thomas Parnell <tom.parnell@gmail.com>
Co-authored-by: Thomas Parnell <tpa@zurich.ibm.com>
 2025-12-12 16:55:40 +01:00
Fadi Arafeh
434ac76a7c [cpu][ci] Add CPU Attention Tests for Neon Backend (#30347) 
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
 2025-12-10 05:37:35 +00:00
rasmith
7618dc973d [CI/Build] Make test_mha_attn.py run on correct platform only and check for flash_attn_varlen_func in layer.py (#29145) 2025-12-09 20:18:17 +00:00