biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
14,985 Commits 2 Branches 140 Tags
738d0a281fab2e151a67b370c26b4e4360362f8f
Commit Graph

185 Commits

Author SHA1 Message Date
Wei Zhao
b36adfa349 [Perf] Set Flashinfer sparse MLA as default backend for FP8 kv cache (#37252) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
 2026-03-17 20:09:20 +00:00
Matthew Bonanni
93f3c8e531 [Misc] Add float16 to CacheDType (#37199) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-03-16 13:24:48 -07:00
Lucas Kabela
714c6e0eab [torch.compile][BE] Modify cudagraph callable to check for is_forward_context_set (#36288) 
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
 2026-03-16 19:42:34 +00:00
leo-cf-tian
2754231ba3 [Kernel] Add FlashInfer MoE A2A Kernel (#36022) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Signed-off-by: Leo Tian <lctian@nvidia.com>
Co-authored-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Stefano Castagnetta <scastagnetta@nvidia.com>
Co-authored-by: root <root@lyris0267.lyris.clusters.nvidia.com>
 2026-03-15 23:45:32 -07:00
Xinan Miao
2cdf92228c [Feature]: Remove Chunking From FusedMoE (#34086) 
Signed-off-by: SouthWest7 <am1ao@qq.com>
Signed-off-by: Southwest <1403572259@qq.com>
Signed-off-by: southwest <am1ao@qq.com>
Signed-off-by: Xinan Miao <1403572259@qq.com>
Co-authored-by: SouthWest7 <am1ao@qq.com>
 2026-03-12 14:24:38 -04:00
grimulkan
a1257fd1ea [Kernel] Add FP8 KV cache support to Triton MLA decode attention (#34597) 
Signed-off-by: grimulkan <grimulkan@gmail.com>
 2026-03-12 08:32:34 -07:00
Wuxun Zhang
e584dce52b Add XPU MLA Sparse backend for DeepSeek v3.2 (#33230) 
Signed-off-by: Zhang, Wuxun <wuxun.zhang@intel.com>
 2026-03-11 19:19:15 +08:00
JartX
a40ee486f2 [Bugfix] Add Multiple of 16 block_size to triton fallback on rocm Attention to support qwen3_5 (#35923) 
Signed-off-by: JartX <sagformas@epdcenter.es>
Co-authored-by: akaratza <akaratza@amd.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
 2026-03-11 07:45:57 +00:00
Lucas Kabela
3fd03f1ec2 [BE] Rename should_torch_compile_mm_vit to should_torch_compile_mm_encoder (#36281) 
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
 2026-03-09 18:22:05 +00:00
Andreas Karatzas
c174d54f86 [ROCm][CI] Fix ROCm attention backend validation for head sizes, block sizes, and compute capability checks (#36292) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-03-09 12:02:41 -05:00
Cyrus Leung
f96c3ab08c [Deprecation][1/2] Remove items deprecated in v0.18 (#36470) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-03-09 03:43:23 -07:00
Harry Mellor
a0f44bb616 Allow markdownlint to run locally (#36398) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-08 20:05:24 -07:00
Wei Zhao
379689d533 [Perf] Support FP8 KV cache for Flashinfer MLA Sparse (#35891) 2026-03-07 13:51:54 -08:00
lif
00b814ba5a [V0 Deprecation] Remove unused swap_space parameter (#36216) 
Signed-off-by: majiayu000 <1835304752@qq.com>
Co-authored-by: mcelrath
 2026-03-07 22:09:55 +08:00
Copilot
ce8546a12b [docs][torch.compile] Add fusions.md — kernel/operator fusion reference page (#35538) 
Signed-off-by: ProExpertProg <luka.govedic@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: ProExpertProg <11367180+ProExpertProg@users.noreply.github.com>
Co-authored-by: ProExpertProg <luka.govedic@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2026-03-06 23:55:06 +00:00
Andreas Karatzas
807d680337 [ROCm][CI] Fix tool use test stability - disable skinny GEMM, prefix caching, eliminate batch variance (#35553) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-03-06 15:15:12 +08:00
Rohan Potdar
c5362c739f Reenable features for ROCm attention backends (#36185) 
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
 2026-03-05 20:21:06 -08:00
Jiayi Yan
6a895197fa [Bugfix][CI] fix typos (#34934) 
Signed-off-by: 1195343015 <1195343015@qq.com>
Signed-off-by: Jiayi Yan <66017932+1195343015@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-05 17:05:46 +00:00
Sage Moore
8c760b6ab6 [ROCm] Refactor ROCm attention backend selection logic (#35246) 
Signed-off-by: Sage Moore <sage@neuralmagic.com>
 2026-03-05 10:51:26 -06:00
Kunshang Ji
66a2209645 [Hardware] Replace torch.cuda.synchronize() api with torch.accelerator.synchronize (#36085) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-03-05 10:36:39 +00:00
Michael Yao
fd3bfe74c9 [Docs] Update design/multiprocessing.md (#30677) 
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
 2026-03-04 17:58:59 +00:00
simone-dotolo
e86221deb6 [Doc] Fix GPU Worker count in Process Count Summary (#36000) 
Signed-off-by: simone-dotolo <simonedotolo@libero.it>
Signed-off-by: simone-dotolo <84937474+simone-dotolo@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-03-04 17:03:14 +00:00
Shanshan Shen
77e6dcbbfa [PluggableLayer][MM] Add PluggableLayer for RelPosAttention (#33753) 
Signed-off-by: shen-shanshan <467638484@qq.com>
 2026-03-03 19:41:27 -08:00
Robert Shaw
97995f6376 [MoE Refactor] Create MK for TRTLLM Kernels (#32564) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Signed-off-by: Robert Shaw <rshaw@neuralmagic.com>
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
 2026-03-03 10:39:50 -08:00
Woosuk Kwon
4f85bae9d6 [Docs][Model Runner V2] Add Design Docs (#35819) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-03-02 19:58:14 -08:00
Lucas Wilkinson
8b5014d3dd [Attention] FA4 integration (#32974) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2026-03-01 23:44:57 +00:00
Augusto Yao
8e75d88554 add io_process_plugin for sparse embedding (#34214) 
Signed-off-by: augusto.yjh <augusto.yjh@antgroup.com>
Signed-off-by: Augusto Yao <augusto.yjh@antgroup.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-02-28 09:16:37 +00:00
Micah Williamson
0edf101d2b [ROCm] Add stablelm Head Size 80 To Supported Head Sizes For ROCM_ATTN (#35527) 
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
 2026-02-28 12:16:34 +08:00
Gregory Shtrasberg
9fa6c68fa6 [ROCm] Enabling encoder and encoder-decoder on ROCm and AITER unified backends (#35334) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2026-02-27 21:32:55 +00:00
Wentao Ye
062b789632 [Bug] Fix outdated links in source code (#35314) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-02-27 03:50:46 +00:00
Tyler Michael Smith
eb19955c37 [WideEP] Remove pplx all2all backend (#33724) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-26 14:30:10 -08:00
Joao Gante
709eadbb0b Doc link typo (#35281) 
Signed-off-by: Joao Gante <joaofranciscocardosogante@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-02-25 03:00:31 -08:00
lichuang
2c619e5e3f [Docs]Fix documentation formatting in architecture overview (#34679) 
Signed-off-by: codedump <lichuang1982@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-02-25 08:00:15 +00:00
Cyrus Leung
a766b30349 [Renderer] Deprecate code paths for old input processing (#34775) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-18 00:35:04 -08:00
Matthew Bonanni
dc5fa77a4e [Bugfix][MTP][Sparse MLA] Allow sparse MLA with MTP to run with FULL cudagraphs (#34457) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-02-17 14:01:27 -05:00
Matthew Bonanni
f2c47886fd [Attention] Add FlashInfer Sparse MLA backend (#33451) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2026-02-12 17:21:54 +00:00
Cyrus Leung
c9a1923bb4 [Plugin] Simplify IO Processor Plugin interface (#34236) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-10 19:47:39 -08:00
bnellnm
d1481ba783 [MoE Refactor] Introduce MoERunner abstraction and move execution logic from FusedMoE to DefaultMoERunner (#32344) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2026-02-10 19:51:07 -05:00
Michael Goin
5e75a14a66 [Doc] Add DCP support to attention backend doc (#33936) 2026-02-09 18:33:43 -05:00
果冻虾仁
6f7adc533a fix description in plugin_system.md (#33999) 2026-02-06 19:37:02 -08:00
Michael Goin
c39ee9ee2b [Docs] Add sections on process architecture and minimum CPU resources (#33940) 
It seems users can be confused about vLLM's performance when running
with very small amounts of CPU cores available. We are missing a clear
overview of what vLLM's process architecture is, so I added this along with
some diagrams in arch_overview.md, and included a section on CPU resource
recommendations in optimization.md

Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-02-06 15:26:43 +00:00
Nicolò Lucchesi
81a90e5277 [Docs] Add bart-plugin to docs (#33905) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-02-05 12:20:25 +00:00
Richard Zou
fd9c83d0e0 [torch.compile] Document the workaround to standalone_compile failing (#33571) 
Signed-off-by: Richard Zou <zou3519@gmail.com>
 2026-02-03 07:16:55 +00:00
Cyrus Leung
793af538a3 [Doc] Update plugin deprecation notices (#33476) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-31 22:48:28 +08:00
jennyyyyzhen
527bcd14d4 [ROCM] Enable aiter attn backend for qwen3-next model (#32492) 
Signed-off-by: jennyyyyzhen <yzhen@hmc.edu>
 2026-01-31 17:03:57 +08:00
Didier Durand
31b25f6516 [Doc]: fixing multiple typos in diverse files (#33256) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
Signed-off-by: Didier Durand <2927957+didier-durand@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-01-29 16:52:03 +08:00
Matthew Bonanni
77c4f45c6c [7/N][Attention][Docs] Add documentation for attention backends (#32477) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-01-28 17:20:22 -05:00
Angela Yi
fb7abfc1d0 [docs] Improve tlparse section (#33211) 
Signed-off-by: angelayi <yiangela7@gmail.com>
 2026-01-28 02:07:37 +00:00
Matthew Bonanni
a608b4c6c2 [5/N][Attention] Finish eliminating vllm/attention folder (#32064) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-01-27 10:02:51 -05:00
Robert Shaw
5a93b9162b [MoE Refactor] Integrate Naive Prepare Finalize into MK (#32567) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Signed-off-by: Amir Klein <203507526+amirkl94@users.noreply.github.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: amirkl94 <203507526+amirkl94@users.noreply.github.com>
 2026-01-27 01:28:02 +00:00