biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
15,025 Commits 2 Branches 140 Tags
96266f119bb93516703328f9e37ec99cce45f792
Commit Graph

646 Commits

Author SHA1 Message Date
zhanqiuhu
d49f273144 [SSM/Mamba] Follow-up: N-1 prefill for P/D disaggregation (#37310) 2026-03-19 08:22:00 +01:00
Sage Moore
c32a58cc2a [EPLB] Simplify EPLB rearrange by only returning one map (#36267) 
Signed-off-by: Sage Moore <sage@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2026-03-18 20:34:00 -04:00
Wentao Ye
0d81a1fe61 [V0 Deprecation] Deprecate virtual engine (#37195) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-03-18 14:30:14 -07:00
Or Ozeri
5dd8df0701 [kv_offload+HMA][2/N]: Support multiple KV groups in GPULoadStoreSpec (#36642) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2026-03-18 19:26:40 +02:00
Itay Alroy
de1a86b7de elastic_ep: Fix stateless group port races (#36330) 
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
 2026-03-18 14:36:18 +00:00
Or Ozeri
525f2eeb0b [kv_offload+HMA][6/N]: Split offloading_connector.py (#37405) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2026-03-18 14:42:46 +01:00
Andy Lo
98b09ddc27 [NIXL][Bugfix] metrics & testing minor bug (#36051) 
Signed-off-by: Andy Lo <andy@mistral.ai>
 2026-03-18 14:39:14 +01:00
Or Ozeri
fcf0687b27 [kv_offload+HMA][0/N]: Support block-level preemption handling (#34805) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2026-03-18 08:49:53 +02:00
Wentao Ye
bdb903bb5f [Bug] Fix FlashInfer MNNVL socket collisions under concurrent vLLM jobs (#36674) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-03-17 15:19:52 -04:00
Isotr0py
a836524d20 [Chore] Replace all base64 usages with faster pybase64 package (#37290) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-03-17 14:44:19 +00:00
Siew's Capital Jarvis
c25dbc2d27 [Bugfix] Fix unclean shutdown crash with AllReduce Fusion workspace (#36955) 
Signed-off-by: Jarvis <brayden.stanley.0127@gmail.com>
 2026-03-17 14:22:09 +00:00
youkaichao
0fb142a454 [perf][connector] optimize build_connector_meta when host buffer transfer is not used (#37165) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2026-03-17 11:59:35 +00:00
zhanqiuhu
2dccb38f73 [Bugfix][MultiConnector] Fix MultiConnector for SupportsHMA sub-connectors (#36549) 2026-03-16 20:51:04 +00:00
Nicolò Lucchesi
f5c081d432 [PD][Nixl] Add support for hybrid SSM-FA models (#36687) 2026-03-16 19:58:06 +01:00
haosdent
ca1954d58c [Bugfix] Disable cross-layer KV cache for MLA attention backends (#37090) 
Signed-off-by: haosdent <haosdent@gmail.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
 2026-03-16 19:03:10 +02:00
Fynn Schmitt-Ulms
04bf5a35fa [Spec Decode] Update extract_hidden_states to use deferred kv_connector clear (#37013) 2026-03-16 14:53:45 +01:00
leo-cf-tian
2754231ba3 [Kernel] Add FlashInfer MoE A2A Kernel (#36022) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Signed-off-by: Leo Tian <lctian@nvidia.com>
Co-authored-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Stefano Castagnetta <scastagnetta@nvidia.com>
Co-authored-by: root <root@lyris0267.lyris.clusters.nvidia.com>
 2026-03-15 23:45:32 -07:00
rasmith
0024f39a32 [ROCm][P/D][MORI][BugFix] Add transfer_id for moriio_connector so moriio_connector to restore P/D functionality (#34907) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
 2026-03-16 10:36:51 +08:00
Harry Mellor
5a3f1eb62f [Misc] Set default kv_buffer_device in a better way (#36862) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-13 19:07:33 +00:00
Itay Alroy
d5af196c18 [2/N] Elastic EP Milestone 2: Integrating NIXL-EP (#35627) 
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
Co-authored-by: Yongji Wu <wuyongji317@gmail.com>
Co-authored-by: Ron Tourgeman <rtourgeman@nvidia.com>
 2026-03-13 09:25:33 -04:00
Or Ozeri
cfaf4668f7 [kv_offload+HMA][1/N]: Support multiple KV groups in OffloadingSpec (#36610) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2026-03-13 08:04:21 +00:00
Aaron Hao
5e1a373d2e [BUG] Fix rank calculation in NCCLWeightTransferEngine (#36940) 
Signed-off-by: hao-aaron <ahao@anyscale.com>
 2026-03-13 01:56:51 +00:00
Kunshang Ji
53ec16a705 [Hardware] Replace torch.cuda.device_count/current_device/set_device API (#36145) 
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-03-12 07:57:47 -07:00
sfeiqiang
8cb24d3aed [KV Connector] Support using FlexKV as KV Cache Offloading option. (#34328) 
Signed-off-by: phaedonsun <phaedonsun@tencent.com>
Co-authored-by: phaedonsun <phaedonsun@tencent.com>
 2026-03-12 00:46:20 -07:00
Yan Ma
894843eb25 replace with torch.cuda.device with with torch.accelerator.device_index (#36144) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
 2026-03-11 23:12:57 -07:00
Yuwei An
2ef69456f5 [LMCache] Fault Tolerance Mechanism (#36586) 
Signed-off-by: Oasis-Git <ayw.sirius19@gmail.com>
 2026-03-12 03:54:39 +00:00
maobaolong
12001f2ebc [LMCache] Pass TP size in lookup for MLA multi-reader locking (#36129) 
Signed-off-by: baoloongmao <baoloongmao@tencent.com>
Co-authored-by: Yihua Cheng <yihua98@uchicago.edu>
 2026-03-11 20:45:20 +00:00
Or Ozeri
7ee5d5093b [BugFix][kv_offload] Fix offloading decodes with async scheduling (#33881) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2026-03-11 20:43:40 +00:00
Wentao Ye
35bdca5431 [Refactor] Remove dead code in KV connector (#36424) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-03-11 19:40:17 +00:00
Or Ozeri
a1a3523a56 [KVConnector] Support worker -> scheduler metadata (#31964) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2026-03-11 17:36:37 +00:00
YiSheng5
c910eeb125 [XPU]Bug fix for some unexpected error when use AgRs backend on XPU device. (#36593) 
Signed-off-by: yisheng <yi.sheng@intel.com>
 2026-03-11 09:17:46 +00:00
Nicolò Lucchesi
098d844731 [NIXL][1/N] Refactor kernel_block_size detection (#35752) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-03-11 01:11:23 -07:00
liuzhenwei
f22d6e0267 [Hardware][NIXL] set default kv buffer type for different platform (#36438) 
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-03-11 05:19:28 +00:00
Harry Mellor
f83b933b84 [CI] Bump mypy version to 1.19.1 (#36104) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-10 09:18:28 -07:00
youkaichao
f85b4eda3a [bugfix] fix nvlink for nixl/ucx (#36475) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2026-03-10 07:49:47 +08:00
Matthew Bonanni
77a73458e3 Reapply [Attention] Refactor check_and_update_config (#35122) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-03-09 07:17:14 -07:00
Samuel Shen
5d6aae4577 [LMCache MP Patch]: Race Condition + Duplicated Block Ids (#35831) 2026-03-07 13:52:48 -08:00
Roy Huang
63298ee173 [Bugfix][LMCache][KVConnector] fix potential memory leak in LMCache multiprocess mode (#35931) 2026-03-07 13:52:35 -08:00
Nicolò Lucchesi
5b3ba94ab4 [Core][KVConnector] Support HMA+NixlConnector (#35758) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-03-06 08:51:21 +01:00
Jiayi Yan
6a895197fa [Bugfix][CI] fix typos (#34934) 
Signed-off-by: 1195343015 <1195343015@qq.com>
Signed-off-by: Jiayi Yan <66017932+1195343015@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-05 17:05:46 +00:00
Kunshang Ji
66a2209645 [Hardware] Replace torch.cuda.synchronize() api with torch.accelerator.synchronize (#36085) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-03-05 10:36:39 +00:00
nkm-meta
792cbd64ca Add platform method to enable custom collective ops registration (#34760) 
Signed-off-by: Naina Kuruballi Mahesh <nainakm@meta.com>
 2026-03-05 00:50:32 +00:00
Simon Mo
f678c3f61a [RL] [Weight Sync] Guard IPC update-info pickle deserialization behind insecure serialization flag (#35928) 
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
 2026-03-04 17:05:32 -05:00
Harry Mellor
17dc9c7fc9 [CI] Bump mypy version (#34950) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-04 20:55:11 +00:00
Qi Wang
6aa6ad8992 [BugFix] Fix implicit and incorrect assumption on ECConnector is_producer (#34783) 
Signed-off-by: Qi Wang <qiwa@nvidia.com>
 2026-03-04 15:01:30 +01:00
Kunshang Ji
16d2ad1d38 [Hardware] Replace torch.cuda.empty_cache with torch.accelerator.empty_cache (#30681) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-04 09:49:47 +00:00
Joe Runde
6f0dd93801 [Core] Remove busy loop from idle buffer readers (#28053) 
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Travis Johnson <tsjohnso@us.ibm.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-03-04 07:44:20 +00:00
Li, Jiang
440f0e7dc6 [Bugfix] Avoid src/dst as None in irecv/isend_tensor_dict (#35754) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2026-03-03 05:56:08 -08:00
liuzhenwei
9dd656f0ea [XPU][NIXL] Add GPUDirect RDMA support for XPU (#35270) 
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-03-03 08:42:49 +08:00
Yashwant Bezawada
a13d8c03c9 [KVConnector] Auto-downgrade to PIECEWISE cudagraph mode for layerwise async ops (#31057) 
Signed-off-by: Yashwant Bezawada <yashwant_b@me.com>
 2026-03-02 15:04:47 -05:00