biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
7,580 Commits 2 Branches 140 Tags
47043eb6787bc8f62e4e746d7a9ad2451aaf7983
Commit Graph

140 Commits

Author SHA1 Message Date
Tuan, Hoang-Trong
47043eb678 [Kernel] Triton implementation of causal-conv1d for Mamba-based models (#18218) 
Signed-off-by: Tuan M. Hoang-Trong <tmhoangt@us.ibm.com>
Co-authored-by: Tuan M. Hoang-Trong <tmhoangt@us.ibm.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-07-09 12:53:55 -07:00
Akash kaothalkar
6db31e7a27 [Hardware][PPC64LE] Enable V1 for ppc64le and ARM (#20554) 
Signed-off-by: Akash Kaothalkar <akash.kaothalkar@ibm.com>
Co-authored-by: Akash Kaothalkar <akash.kaothalkar@ibm.com>
Co-authored-by: Nikhil Gupta <nikhil.gupta2@arm.com>
 2025-07-08 20:00:41 -07:00
Chenyaaang
e34d130c16 [TPU] Temporary fix vmem oom for long model len by reducing page size (#20278) 
Signed-off-by: Chenyaaang <chenyangli@google.com>
 2025-07-08 05:16:16 +00:00
Li, Jiang
7721ef1786 [CI/Build][CPU] Fix CPU CI and remove all CPU V0 files (#20560) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-07-07 22:13:44 -07:00
Cyrus Leung
9fb52e523a [V1] Support any head size for FlexAttention backend (#20467) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-07-06 09:54:36 -07:00
Isotr0py
32c9be2200 [v1] Re-add fp32 support to v1 engine through FlexAttention (#19754) 
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-07-05 09:41:10 +00:00
Jee Jee Li
1caca5a589 [Misc] Add SPDX-FileCopyrightText (#20428) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-07-04 07:40:42 +00:00
Nicolò Lucchesi
8d775dd30a [Misc] Fix Unable to detect current VLLM config. Defaulting to NHD kv cache layout warning (#20400) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-07-03 14:56:09 -07:00
vllmellm
a1aafc827a [ROCm][FEAT] Enable Full Graph Mode in AITER MLA V1 Attn Backend (Decode Phase only) (#20254) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
 2025-07-02 16:25:46 +00:00
Chengji Yao
7da296be04 [TPU] kv cache update kernel supports dynamic grid (#20235) 
Signed-off-by: Chengji Yao <chengjiyao@google.com>
 2025-07-02 06:33:37 +00:00
Liangliang Ma
a0389e0554 [UT][intel GPU] use current_platform instead of device hardcode in v1 tests (#20169) 
Signed-off-by: Ma, Liangliang <liangliang.ma@intel.com>
 2025-07-02 09:06:04 +08:00
Woosuk Kwon
8acb4badee [CUDA graphs] Enable full cuda graphs with FA3 AoT scheduling (#20301) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-07-01 09:07:36 -07:00
TY-AMD
96453cfa83 [BugFix][V1][ROCm] Triton MLA uses V0 backend on V1 engine (#19067) 
Signed-off-by: Tianyuan Wu <Tianyuan.Wu@amd.com>
 2025-07-01 16:12:19 +08:00
Chendi.Xue
dec197e3e5 Quick Fix by adding conditional import for flash_attn_varlen_func in flash_attn (#20143) 
Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
 2025-06-27 05:48:13 +00:00
Chengji Yao
04e1642e32 [TPU] add kv cache update kernel (#19928) 
Signed-off-by: Chengji Yao <chengjiyao@google.com>
 2025-06-26 10:01:37 -07:00
Kunshang Ji
b69781f107 [Hardware][Intel GPU] Add v1 Intel GPU support with Flash attention backend. (#19560) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2025-06-26 09:27:18 -07:00
TJian
27c065df50 [Bugfix][V1][ROCm] Fix AITER Flash Attention Backend (Fix API Break and Local Attention Logic: affecting Llama4) (#19904) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2025-06-26 12:42:31 +00:00
Chenyaaang
2d7620c3eb [TPU] Add TPU specific var VLLM_TPU_MOST_MODEL_LEN (#19919) 
Signed-off-by: Chenyaaang <chenyangli@google.com>
 2025-06-25 15:51:02 -07:00
Chengji Yao
2cc2069970 [TPU][Bugfix] fix kv cache padding (#20048) 
Signed-off-by: Chengji Yao <chengjiyao@google.com>
 2025-06-25 21:24:10 +00:00
Lucas Wilkinson
0f9e7354f5 [BugFix] Fix full-cuda-graph illegal memory access in FA3 (#20057) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-06-25 08:39:04 +00:00
qli88
e3a3e4db46 [Bugfix] Enable PP with AITER+V1 (#19822) 
Signed-off-by: Qiang Li <qiang.li2@amd.com>
 2025-06-20 12:43:20 +08:00
zsolt-borbely-htec
aa20d10a91 [Misc] [ROCm] Prevent surplus tensor reshape (#19803) 
Signed-off-by: Zsolt Borbely <zsolt.borbely@htecgroup.com>
 2025-06-19 13:57:16 +08:00
Chen Zhang
a89209b78d [v1] Support mamba2 (#19327) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-06-18 20:34:15 +00:00
Zzz9990
8b6e1d639c [Hardware][AMD] integrate aiter chunked prefill into vllm (#18596) 
Signed-off-by: fsx950223 <fsx950223@outlook.com>
Signed-off-by: charlifu <charlifu@amd.com>
Co-authored-by: fsx950223 <fsx950223@outlook.com>
Co-authored-by: charlifu <charlifu@amd.com>
 2025-06-18 08:46:51 -07:00
Charlie Fu
a44b1c951d [Feature][ROCm] Add full graph capture support for TritonAttentionBackend (#19158) 
Signed-off-by: charlifu <charlifu@amd.com>
 2025-06-17 17:03:06 -04:00
Nicolò Lucchesi
4c8f64faa7 [V1][Kernel] Flashinfer HND KV cache layout (#19280) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-06-17 09:09:22 -04:00
Driss Guessous
ddfed314f9 Fixes IMA for TP w/ flex-attention (#19712) 
Signed-off-by: drisspg <drisspguessous@gmail.com>
 2025-06-17 04:01:50 +00:00
Isotr0py
1173804dca [Bugfix] Fix TP inference for Flex attention backend (#19657) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-06-16 11:21:37 +00:00
Chengji Yao
a77aea59fd [TPU] support attention head dim smaller than 128 (#19620) 
Signed-off-by: Chengji Yao <chengjiyao@google.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-06-16 06:40:53 +00:00
22quinn
0b73736a0d [Kernel] Raise verbose error and consolidate num_heads/num_kv_heads divisibility check (#19339) 
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
 2025-06-15 13:43:48 +08:00
Luka Govedič
3597b06a4f [CUDA] Enable full cudagraph for FlashMLA (#18581) 
Signed-off-by: luka <luka@neuralmagic.com>
 2025-06-13 18:12:26 +00:00
Luka Govedič
f98548b9da [torch.compile][ROCm] Fuse quantization onto attention using a torch.compile pass (#16756) 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Sage Moore <sage@neuralmagic.com>
 2025-06-12 08:31:04 -07:00
Rachel Guo
467bef18a3 [BugFix][FlashInfer] Fix attention backend interface mismatch with unexpected keyword use_irope (#19134) 
Signed-off-by: Yunqiu Guo <guorachel@meta.com>
 2025-06-10 16:48:51 +08:00
Pavani Majety
8058c91108 [HOT-FIX] Add kv_sharing_target_layer_name argument to cutlass_mla backend (#19374) 
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
 2025-06-09 19:00:07 -04:00
Driss Guessous
cf02f9b283 Add FlexAttention to V1 (#16078) 
Signed-off-by: drisspg <drisspguessous@gmail.com>
 2025-06-06 21:58:55 -07:00
vllmellm
18093084be [Misc] Remove unnecessary fallback to prefill-decode attention (#19138) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
 2025-06-05 16:08:26 +08:00
Nicolò Lucchesi
b2fac67130 [P/D] Heterogeneous TP (#18833) 
Signed-off-by: nicklucche <nlucches@redhat.com>
 2025-06-04 23:25:34 +00:00
Woosuk Kwon
b124e1085b [Bugfix] Fix FA3 full cuda graph correctness (#19106) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-06-03 23:10:15 -07:00
Kaixi Hou
41aa578428 [NVIDIA] Add Cutlass MLA backend (#17625) 2025-06-03 21:40:26 -07:00
Li, Jiang
4555143ea7 [CPU] V1 support for the CPU backend (#16441) 2025-06-03 18:43:01 -07:00
Yong Hoon Shin
bdf13965ab [V1] Support cross-layer KV sharing (#18212) 
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
 2025-06-03 20:33:07 +00:00
Simon Mo
02f0c7b220 [Misc] Add SPDX-FileCopyrightText (#19100) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2025-06-03 11:20:17 -07:00
vllmellm
77b6e74fe2 [ROCm] Remove unnecessary assertion of max_model_len in ROCM_AITER_MLA attention backend. (#18938) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
 2025-05-29 22:33:17 -07:00
Gregory Shtrasberg
da4b69d0b4 [Attention][V1] Toggle for v1 attention backend (#18275) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2025-05-29 10:48:24 -04:00
Lucas Wilkinson
ce75efeecb [BugFix] FA2 MLA Accuracy Issue (#18807) 
Signed-off-by: LucasWilkinson <lwilkinson@neuralmagic.com>
 2025-05-28 08:59:39 +00:00
Chen Zhang
6550114c9c [v1] Redo "Support multiple KV cache groups in GPU model runner (#17945)" (#18593) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-05-23 09:39:47 -07:00
vllmellm
94d8ec8d2b [FEAT][ROCm] Upgrade AITER MLA v1 backend (#18338) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2025-05-21 10:34:28 -07:00
Mark McLoughlin
bb0a311213 Revert "[v1] Support multiple KV cache groups in GPU model runner (#17945) (#18459) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2025-05-21 10:25:23 -07:00
kliuae
ee659e3b60 [Bugfix][ROCm] Use chunked_prefill_paged_decode as fallback for V1 attention on ROCm (#18093) 
Signed-off-by: kf <kuanfu.liu@embeddedllm.com>
 2025-05-15 19:30:17 -07:00
Thomas Parnell
01c22335ba [Kernel] [V1] Fix performance regression for triton unified attention (#18161) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
 2025-05-15 06:39:00 -07:00