biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
8,536 Commits 2 Branches 140 Tags
fceafaf582cd72e6636f47127a665afb9e0ea0aa
Commit Graph

25 Commits

Author SHA1 Message Date
Michael Goin
c6b928798e Force TRTLLM attention for gpt-oss on SM100 (#22678) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-08-12 21:22:16 -07:00
Lucas Wilkinson
1dc8a70b6d [Attention] Support multiple attention metadata builders per kv_cache_spec + proper local attention no hybrid kv cache fix (#21588) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-08-06 18:40:52 -07:00
Lain
9a3835aaa9 Fix trtllm-gen attention env and add attention sink (#22378) 
Signed-off-by: Siyuan Fu <siyuanf@nvidia.com>
Signed-off-by: Lain <fusiyuan2000@hotmail.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
 2025-08-06 18:07:41 -07:00
Woosuk Kwon
6e20924350 Add attention sink in attention backends (#22320) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

Co-authored-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com>
Co-authored-by: Minseok Lee <47620120+minseokl@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
 2025-08-05 22:37:21 -07:00
Giancarlo Delfin
aa7012eb6d Add tree attention backend for v1 (part 1) (#20401) 
Signed-off-by: Giancarlo Delfin <gdelfin@meta.com>
 2025-08-03 22:13:26 -07:00
ZiTian.Zhao
e27d25a0dc [fix] fix correct assertion syntax error in attention utils. (#22154) 
Signed-off-by: zitian.zhao <zitian.zhao@tencentmusic.com>
 2025-08-03 19:24:02 -07:00
Thomas Parnell
4abfd8796f [V1] [Hybrid] Validate compatibility of attention backend batch reordering at init time (#21557) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
 2025-08-02 05:29:40 -07:00
Sage Moore
0edaf752d7 [Attention][DBO] Add support for "splitting" the CommonAttentionMetadata (#21153) 
Signed-off-by: Sage Moore <sage@neuralmagic.com>
 2025-08-01 19:47:53 -07:00
fhl2000
23322431c8 [V1][CUDA] Full cudagraph support for FlashInfer (#21367) 2025-08-01 21:49:34 -04:00
Yong Hoon Shin
ad510309ee Override attention metadata for fast prefill in some KV sharing setups (#21590) 
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
 2025-07-30 08:54:15 -07:00
Chen Zhang
555e7225bc [v1][attention] Support Hybrid Allocator + FlashInfer (#21412) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-07-30 01:45:29 +00:00
Maximilien de Bayser
1cd6eaba54 Support encoder-only models without KV-Cache (#21270) 
Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
 2025-07-26 21:09:52 +08:00
Lucas Wilkinson
59f935300c [BugFix] Fix potential cuda-graph IMA (#21196) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-07-19 02:18:47 -07:00
Lucia Fang
9a9fda1423 [Core] Support Local Chunked Attention for Hybrid KV Cache (#19351) 
Signed-off-by: Lucia Fang <fanglu@fb.com>
Signed-off-by: Lu Fang <fanglu@meta.com>
Signed-off-by: Lu Fang <fanglu@fb.com>
Co-authored-by: Lu Fang <fanglu@meta.com>
 2025-07-18 20:48:38 -07:00
Lucas Wilkinson
89cab4d01f [Attention] Make local attention backend agnostic (#21093) 2025-07-18 00:10:42 -04:00
Lucas Wilkinson
76b494444f [Attention] Refactor attention metadata builder interface (#20466) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-07-17 04:44:25 +00:00
Pavani Majety
7bd4c37ae7 [Core] Add Flashinfer TRTLLM Backend for Flashinfer decode path (SM100). (#19825) 
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: shuw <shuw@nvidia.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-07-11 09:23:23 +00:00
Alexander Matveev
5b032352cc [Attention] MLA - Flashinfer Ragged Prefill (#20034) 2025-07-10 20:17:47 -07:00
Nicolò Lucchesi
8d775dd30a [Misc] Fix Unable to detect current VLLM config. Defaulting to NHD kv cache layout warning (#20400) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-07-03 14:56:09 -07:00
Charlie Fu
a44b1c951d [Feature][ROCm] Add full graph capture support for TritonAttentionBackend (#19158) 
Signed-off-by: charlifu <charlifu@amd.com>
 2025-06-17 17:03:06 -04:00
Nicolò Lucchesi
4c8f64faa7 [V1][Kernel] Flashinfer HND KV cache layout (#19280) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-06-17 09:09:22 -04:00
Luka Govedič
3597b06a4f [CUDA] Enable full cudagraph for FlashMLA (#18581) 
Signed-off-by: luka <luka@neuralmagic.com>
 2025-06-13 18:12:26 +00:00
Yong Hoon Shin
bdf13965ab [V1] Support cross-layer KV sharing (#18212) 
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
 2025-06-03 20:33:07 +00:00
Simon Mo
02f0c7b220 [Misc] Add SPDX-FileCopyrightText (#19100) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2025-06-03 11:20:17 -07:00
Chen Zhang
cba31c47c4 [v1] AttentionMetadata for each layer (#17394) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-05-06 07:58:37 -07:00