biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
14,640 Commits 2 Branches 140 Tags
f85b4eda3a22fedd885ef31650c825d56867587e
Commit Graph

612 Commits

Author SHA1 Message Date
youkaichao
f85b4eda3a [bugfix] fix nvlink for nixl/ucx (#36475) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2026-03-10 07:49:47 +08:00
Matthew Bonanni
77a73458e3 Reapply [Attention] Refactor check_and_update_config (#35122) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-03-09 07:17:14 -07:00
Samuel Shen
5d6aae4577 [LMCache MP Patch]: Race Condition + Duplicated Block Ids (#35831) 2026-03-07 13:52:48 -08:00
Roy Huang
63298ee173 [Bugfix][LMCache][KVConnector] fix potential memory leak in LMCache multiprocess mode (#35931) 2026-03-07 13:52:35 -08:00
Nicolò Lucchesi
5b3ba94ab4 [Core][KVConnector] Support HMA+NixlConnector (#35758) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-03-06 08:51:21 +01:00
Jiayi Yan
6a895197fa [Bugfix][CI] fix typos (#34934) 
Signed-off-by: 1195343015 <1195343015@qq.com>
Signed-off-by: Jiayi Yan <66017932+1195343015@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-05 17:05:46 +00:00
Kunshang Ji
66a2209645 [Hardware] Replace torch.cuda.synchronize() api with torch.accelerator.synchronize (#36085) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-03-05 10:36:39 +00:00
nkm-meta
792cbd64ca Add platform method to enable custom collective ops registration (#34760) 
Signed-off-by: Naina Kuruballi Mahesh <nainakm@meta.com>
 2026-03-05 00:50:32 +00:00
Simon Mo
f678c3f61a [RL] [Weight Sync] Guard IPC update-info pickle deserialization behind insecure serialization flag (#35928) 
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
 2026-03-04 17:05:32 -05:00
Harry Mellor
17dc9c7fc9 [CI] Bump mypy version (#34950) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-04 20:55:11 +00:00
Qi Wang
6aa6ad8992 [BugFix] Fix implicit and incorrect assumption on ECConnector is_producer (#34783) 
Signed-off-by: Qi Wang <qiwa@nvidia.com>
 2026-03-04 15:01:30 +01:00
Kunshang Ji
16d2ad1d38 [Hardware] Replace torch.cuda.empty_cache with torch.accelerator.empty_cache (#30681) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-04 09:49:47 +00:00
Joe Runde
6f0dd93801 [Core] Remove busy loop from idle buffer readers (#28053) 
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Travis Johnson <tsjohnso@us.ibm.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-03-04 07:44:20 +00:00
Li, Jiang
440f0e7dc6 [Bugfix] Avoid src/dst as None in irecv/isend_tensor_dict (#35754) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2026-03-03 05:56:08 -08:00
liuzhenwei
9dd656f0ea [XPU][NIXL] Add GPUDirect RDMA support for XPU (#35270) 
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-03-03 08:42:49 +08:00
Yashwant Bezawada
a13d8c03c9 [KVConnector] Auto-downgrade to PIECEWISE cudagraph mode for layerwise async ops (#31057) 
Signed-off-by: Yashwant Bezawada <yashwant_b@me.com>
 2026-03-02 15:04:47 -05:00
Fynn Schmitt-Ulms
9433acb8df [Spec Decode] Add hidden states extraction system (#33736) 
Signed-off-by: Fynn Schmitt-Ulms <fschmitt@redhat.com>
 2026-03-02 14:29:09 -05:00
wangxiyuan
510bc9e1df [Misc] Cleanup useless current_platform import (#35715) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2026-03-02 09:36:54 +00:00
Charles Ashby
cbd361fd46 [CPU][Distributed] Fix Enable _CPUSHMDistributed only when TP/PP ranks share the same SHM group name (#34169) 
Signed-off-by: Charles Ashby <charlesa.l@hotmail.com>
 2026-03-02 09:34:35 +00:00
Ryan Rock
87d319c52f [AMD][CI] Support Triton attention with ExampleConnector (#34931) 
Signed-off-by: Ryan Rock <ryan.rock@amd.com>
 2026-03-01 09:58:07 +02:00
Itay Alroy
dea268336f [1/N] Elastic EP Milestone 2 (#34861) 
Signed-off-by: Yongji Wu <wuyongji317@gmail.com>
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Signed-off-by: Ron Tourgeman <rtourgeman@nvidia.com>
Co-authored-by: Yongji Wu <wuyongji317@gmail.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Ron Tourgeman <rtourgeman@nvidia.com>
 2026-02-28 04:46:42 +00:00
youkaichao
5323672bc2 [misc] cleanup one level of error stack when nixl fails to initialize (#35517) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2026-02-28 08:42:37 +08:00
Aaron Hao
2ce6f3cf67 [Feat][RL][2/2] Native Weight Syncing API: IPC (#34171) 
Signed-off-by: hao-aaron <ahao@anyscale.com>
Signed-off-by: Aaron Hao <ahao@anyscale.com>
Signed-off-by: ahao-anyscale <ahao@anyscale.com>
 2026-02-27 13:45:21 -07:00
pkousha
31fb6f43da [Kernel][perf] optimize NCCL symm_mem vs custom_AR selection thresholds (#33839) 
Signed-off-by: <>
Signed-off-by: pkousha <43781676+pkousha@users.noreply.github.com>
Co-authored-by: Pouya Kousha <pkousha@login-eos01.eos.clusters.nvidia.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-02-26 14:35:58 -08:00
Tyler Michael Smith
eb19955c37 [WideEP] Remove pplx all2all backend (#33724) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-26 14:30:10 -08:00
Hanjie Qiu
71dfce6aa6 [Kernel] Refactor FlashInfer allreduce for mnnvl backend (#34109) 
Signed-off-by: hjjq <50634613+hjjq@users.noreply.github.com>
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
 2026-02-26 03:17:20 +00:00
Martin Hickey
e97c46a92d [BugFix]: Fix local mypy issues (#34739) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-02-23 00:40:29 -08:00
zhongdaor-nv
a0fe7ea2f0 [feat] Add per-block extra_keys to KV events (#33304) 
Signed-off-by: zhongdaor-nv <zhongdaor@nvidia.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-02-20 20:11:40 -08:00
Nick Hill
40b2f1c3d9 [Model Runner V2] Minor CPU optimizations (#34856) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-02-19 16:05:37 -08:00
Wentao Ye
3d2a026fd0 [Feature] Pipeline Parallel Async send/recv, 2.9% E2E throughput improvement (#33368) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2026-02-13 16:38:16 +08:00
Martin Hickey
47e9b63e1a [KVConnector] Clean up redundant code in KV connectors (#34147) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-02-13 00:14:30 -08:00
Jaewon
4453ba8d9e [Core] Profiler improvements and lazy initialization (#33198) 
Signed-off-by: Jaewon Lee <jaewon@meta.com>
Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com>
 2026-02-12 16:16:38 -08:00
Ilya Markov
bb2fc8b5e7 [BugFix] Fix async EPLB hang with DeepEP LL all2all backend (#32860) 
Signed-off-by: ilmarkov <markovilya197@gmail.com>
 2026-02-10 22:34:47 +00:00
Ilya Markov
67132945bb [Perf] Move eplb rebalance algo to async thread (#30888) 
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2026-02-10 22:19:10 +00:00
Qi Wang
33bcd3dc3b [Misc] Introduce ec_both role EC (encoder cache) connector (#34182) 
Signed-off-by: Qi Wang <qiwa@nvidia.com>
 2026-02-10 18:55:35 +00:00
Zetong Li
5f970120f0 [Bugfix] Fix memory inconsistency in cross-process shared memory (#32022) 
Signed-off-by: Zetong Li <slippersss@126.com>
 2026-02-10 08:22:03 +00:00
Yuwei An
e94ec59733 [LMCache] Token Base IPC API (#34175) 
Signed-off-by: Oasis-Git <ayw.sirius19@gmail.com>
 2026-02-10 01:18:42 +00:00
ZhengHongming888
cb62e86f83 Add NUMA Core binding in nixl_connector for CPU xPyD (#32365) 
Signed-off-by: Hongming Zheng <hongming.zheng@intel.com>
Signed-off-by: ZhengHongming888 <hongming.zheng@intel.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-02-09 15:39:12 +00:00
Seiji Eicher
aca5967416 [KV Connector] Add missing method overrides to MultiConnector (#33292) 
Signed-off-by: Seiji Eicher <seiji@anyscale.com>
 2026-02-06 12:58:21 -05:00
zackyoray
1ee95841bd [Bugfix] Fix swapped engine_ids in NIXL Llama 4 local attention path (#33795) 
Signed-off-by: Yoray Zack <yorayz@nvidia.com>
 2026-02-05 17:51:58 +00:00
Nicolò Lucchesi
7d8c6804e2 [Misc] Add debug logs (#33931) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-02-05 09:42:40 -08:00
Aaron Hao
c1858b7ec8 [Feat][RL][1/2] Native Weight Syncing API: NCCL (#31943) 
Signed-off-by: ahao-anyscale <ahao@anyscale.com>
Signed-off-by: Aaron Hao <ahao@anyscale.com>
Co-authored-by: SumanthRH <sumanthrh99@gmail.com>
 2026-02-05 12:13:23 -05:00
liranschour
8322d4e47f Enable Cross layers KV cache layout at NIXL Connector V2 (#33339) 
Signed-off-by: Liran Schour <lirans@il.ibm.com>
Signed-off-by: liranschour <liranschour@users.noreply.github.com>
Co-authored-by: Or Ozeri <or@ozery.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@gmail.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2026-02-05 02:17:02 -08:00
Sage Moore
ce498a6d61 Change the type signature of MixtureOfExperts.expert_weights to MutableSequence[Sequence[Tensor]] (#33573) 
Signed-off-by: Sage Moore <sagmoore@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-02-04 17:02:46 -05:00
Micah Williamson
1d367a738e [Bugfix][ROCm] Include float8_e4m3fnuz in NCCL Dtype Dispatching (#33713) 
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
 2026-02-04 05:36:29 -08:00
dtc
0d6ccf68fa [P/D] rework mooncake connector and introduce its bootstrap server (#31034) 
Signed-off-by: Tianchen Ding <dtcccc@linux.alibaba.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@gmail.com>
 2026-02-03 08:08:25 -08:00
杨朱 · Kiki
1a7894dbdf [Misc] Replace Optional[X] with X | None syntax (#33332) 
Signed-off-by: carlory <baofa.fan@daocloud.io>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
 2026-01-30 01:56:59 -08:00
Li, Jiang
8311f083bd [Bugfix][CPU] Fix thread num for shared memory communication (#33317) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
Signed-off-by: Li, Jiang <bigpyj64@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-01-29 03:26:58 -08:00
Ilya Markov
d09135fbd0 [BugFix] Async Eplb fix potential race condition (#32881) 
Signed-off-by: ilmarkov <markovilya197@gmail.com>
 2026-01-29 10:31:40 +00:00
Angela Yi
4197168ea5 [ez] Remove checks for torch version <= 2.8 (#33209) 
Signed-off-by: angelayi <yiangela7@gmail.com>
 2026-01-28 16:03:56 -05:00