biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
10,251 Commits 2 Branches 140 Tags
76879cc160425960b9101eaade27a0565a461afc
Commit Graph

373 Commits

Author SHA1 Message Date
Matthew Bonanni
76879cc160 [Attention] Implement universal BACKEND_MAP (#25900) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-10-08 12:00:25 -07:00
Utkarsh Sharma
335b28f7d1 [TPU] Rename tpu_commons to tpu_inference (#26279) 
Signed-off-by: Utkarsh Sharma <utksharma@google.com>
Co-authored-by: Utkarsh Sharma <utksharma@google.com>
Co-authored-by: Chengji Yao <chengjiyao@google.com>
 2025-10-07 23:30:52 -07:00
Harry Mellor
6c04638214 Fix per file ruff ignores related to line length (#26262) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-06 05:12:40 +00:00
Harry Mellor
b893d661b1 Fix per file ruff ignores related to simplification (#26259) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 20:31:53 +00:00
Harry Mellor
4e256cadc2 Remove all references to yapf as it's no longer used (#26251) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 09:18:11 -07:00
Harry Mellor
d6953beb91 Convert formatting to use ruff instead of yapf + isort (#26247) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 07:06:22 -07:00
Nicolò Lucchesi
2a6dc67eb5 [Bugfix] Fix _reqs_to_process leak on abort (#26012) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-10-04 11:39:31 +00:00
Nicolò Lucchesi
48f309029a [NIXL][Misc] Expose metrics from NIXL for logging to CLI (#25388) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-10-03 10:47:59 +00:00
Matthew Bonanni
2aaa423842 [Attention] Move Backend enum into registry (#25893) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-10-02 20:32:24 -07:00
Lucia Fang
f48b6a03ba [Misc]allow disable pynccl (#25421) 
Signed-off-by: Lu Fang <fanglu@fb.com>
Co-authored-by: Lucia (Lu) Fang <fanglu@meta.com>
 2025-10-01 06:04:13 +00:00
David Ben-David
9a9f48dff7 [V1] [P/D] Add Support for KV Load Failure Recovery (#19330) 
Signed-off-by: David Ben-David <davidb@pliops.com>
Co-authored-by: David Ben-David <davidb@pliops.com>
 2025-09-30 14:57:08 -07:00
Or Ozeri
cfd302db9b OffloadingConnector: Fix GPU block tracking bug (#25856) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2025-09-30 19:53:04 +00:00
Nicolò Lucchesi
80608ba5af [NIXL] Add support for MLA caches with different latent dim (#25902) 
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
 2025-09-30 12:18:29 +00:00
Gregory Shtrasberg
61a3431613 [Bugfix][ROCm] Fixing trying to import non-existent symbols from libnccl.so (#25605) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2025-09-29 17:01:50 -04:00
Chenxi Yang
d0d138bc55 [Nixl][P/D] Add cuda2cpu support (HD->DH transfer) (#24690) 
Signed-off-by: Chenxi Yang <cxyang@fb.com>
Co-authored-by: Chenxi Yang <cxyang@fb.com>
 2025-09-29 14:31:51 +00:00
Robert Shaw
9b44a7d926 [P/D] NIXL Updates (#25844) 
Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: simon-mo <simon.mo@hey.com>
Signed-off-by: rentianyue-jk <rentianyue-jk@360shuke.com>
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Chenheli Hua <huachenheli@outlook.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Sage Moore <sage@neuralmagic.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: rentianyue-jk <rentianyue-jk@360shuke.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Chenheli Hua <huachenheli@outlook.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2025-09-29 04:46:30 +00:00
Nicolò Lucchesi
da63274d9f [Bugfix][NIXL] Fix Async Scheduler timeout issue (#25808) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-09-27 15:17:35 -04:00
Tyler Michael Smith
a5354b3ed2 [Bugfix][WideEP] Apply TP Attn + EP MoE fix to other models (#24982) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2025-09-27 14:22:28 +00:00
Nick Hill
983056e456 [Misc] Remove unnecessary memoryviews in shm_broadcast.py (#25721) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-09-26 03:11:44 +00:00
Nick Hill
8b77328ffe [Misc] Don't log shm dequeue delay warning on worker side (#25720) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-09-26 01:08:30 +00:00
Matthew Bonanni
3468f17ebe [V0 deprecation] Remove _VLLM_V1 suffixes from attention backend names (#25489) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
 2025-09-25 17:37:50 +00:00
youkaichao
6c340da4df [misc] log info messages by default for hanging / busy / idle (#25627) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-09-25 21:14:57 +08:00
Cyrus Leung
2f17117606 [mypy] Fix wrong type annotations related to tuple (#25660) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-09-25 13:00:45 +00:00
Shu Wang
54e42b72db Support mnnvl all2allv from Flashinfer (#21003) 
Signed-off-by: Shu Wang <shuw@nvidia.com>
Signed-off-by: Shu Wang. <shuw@nvidia.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2025-09-24 14:38:16 -04:00
youkaichao
b67dece2d8 [misc] update the warning message (#25566) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-09-24 17:24:35 +08:00
Michael Goin
7361ab379f Remove redundant mutates_args and dispatch_key for direct_register_custom_op (#25512) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-09-23 22:48:40 +00:00
Thomas Parnell
969b4da3a6 [V0 Deprecation] Remove placeholder attn (#25510) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
 2025-09-23 22:12:14 +00:00
Ilya Markov
8bdd8b5c51 Enable symmetric memory all reduce by default only enabling for TP (#25070) 
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-09-23 15:53:00 -04:00
Amir Samani
8c1c81a3de [core] add nccl symmetric memory for all reduce (#24532) 
Signed-off-by: Amir Samani <asamani@nvidia.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-09-23 14:33:06 -04:00
Lucas Wilkinson
cc1dc7ed6d [Core/DBO][2/N] Dual-Batch Overlap add DeepEP High Throughput support and Prefill support (#24845) 
Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Sage Moore <sage@neuralmagic.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-09-23 16:02:10 +00:00
Fanli Lin
4c966e440e [XPU] Fix MOE DP accuracy issue on XPU (#25465) 2025-09-23 14:32:57 +00:00
Chauncey
f05a4f0e34 [P/D] Support NIXL connector to disconnect during a clean shutdown (#24423) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: Mark McLoughlin <markmc@redhat.com>
 2025-09-23 16:08:02 +02:00
Chendi.Xue
5774b0a1da [NIXL][OOT platform] support nixl_connector with oot platform and other nixl_backend (#25121) 
Signed-off-by: Chendi Xue <Chendi.Xue@intel.com>
 2025-09-23 04:17:42 +00:00
Lucia Fang
922979bfcc [DP] support torchrun external launcher with Data Parallelism (#24899) 
Signed-off-by: Lu Fang <fanglu@fb.com>
Signed-off-by: Zhuohan Li <zhuohan123@gmail.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
 2025-09-22 12:06:05 -07:00
Csrayz
c10101a3eb [Bugfix] Fix several issues with p2p xPyD in GET type (#23993) 
Signed-off-by: Csrayz <jover@cmbchina.com>
Signed-off-by: ivyilike <pww123@cmbchina.com>
Co-authored-by: ivyilike <pww123@cmbchina.com>
 2025-09-22 14:53:13 +00:00
Woosuk Kwon
bc6e542d9f Remove V0 attention backends (#25351) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-09-21 16:03:28 -07:00
Nick Hill
a25ade5d47 [BugFix] Ensure appropriate guards in destructors (#25284) 
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-20 09:06:34 +08:00
Or Ozeri
c59a0eca42 [KV offload][4/N] Offloading KV connector (#22595) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2025-09-19 19:07:17 +00:00
Nicolò Lucchesi
a3d087adec [P/D][Nixl] Introduce KVTransferMetrics and aggregation strategy (#22188) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-09-19 11:09:14 +00:00
Shu Wang
2ea50e977a Enable Allgather/ReduceScatter backend for NaiveAllToAll (#23964) 
Signed-off-by: Shu Wang. <shuw@nvidia.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Signed-off-by: Shu Wang <shuw@nvidia.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-09-18 15:52:58 +00:00
Hyogeun Oh (오효근)
b419937c78 [Docs] Fix warnings in mkdocs build (continued) (#25163) 
Signed-off-by: Zerohertz <ohg3417@gmail.com>
 2025-09-18 08:23:26 -07:00
Punitvara
05b044e698 [Doc] Fix cross-reference warnings (#25058) 
Signed-off-by: Punit Vara <punitvara@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-18 02:05:16 -07:00
samzong
47f670b03b [Docs] improve code formatting and comments for eliminate griffe build warning. (#25010) 
Signed-off-by: samzong <samzong.lu@gmail.com>
 2025-09-17 07:31:20 -07:00
Sage Moore
567939953b [Core/DBO][1/N] Add Dual-Batch Overlap mechanism to VLLM (#23693) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2025-09-16 12:21:48 -04:00
cascade
17871983a2 [Bugfix] Fix sequence parallelism bug when enable pipeline parallelism (#24021) 
Signed-off-by: cascade812 <cascade812@outlook.com>
 2025-09-16 04:32:32 +00:00
Nicolò Lucchesi
2e41f5abca [XPU] Set consistent default KV cache layout (#24745) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-09-15 18:09:34 +08:00
Chao Lei
8de261b04a [P/D]kv_output_aggregator support P TP > D TP (#23917) 
Signed-off-by: LCAIZJ <leichao139636@163.com>
Co-authored-by: leichao.lc <leichao.lc@antgroup.com>
 2025-09-15 11:36:06 +02:00
TaoYu Chen
15b8fef453 Remove redundant assignment in xfer_buffers, This is a little fix (#24732) 
Signed-off-by: ChenTaoyu-SJTU <ctynb@qq.com>
 2025-09-13 08:11:59 +00:00
Hyogeun Oh (오효근)
9a8966bcc2 [Docs] Fix warnings in mkdocs build (continued) (#24791) 
Signed-off-by: Zerohertz <ohg3417@gmail.com>
 2025-09-13 00:13:44 -07:00
dongluw
a5b84f1cbf [Core] Shared memory based object store for Multimodal data caching and IPC (#20452) 
Signed-off-by: donglu <donglu@cohere.com>
 2025-09-12 07:54:17 -07:00