biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
11,063 Commits 2 Branches 140 Tags
72b1c2ae2c2d2782ef1850ca1b96a21f797f7af6
Commit Graph

435 Commits

Author SHA1 Message Date
Samuel Shen
40db194446 [CI]: Add LMCacheConnector Unit Tests (#27852) 
Signed-off-by: Samuel Shen <slshen@uchciago.edu>
Co-authored-by: Samuel Shen <slshen@uchciago.edu>
Co-authored-by: Yihua Cheng <yihua98@uchicago.edu>
 2025-11-05 09:45:57 -08:00
Ilya Markov
e50c454672 [BugFix] Support EP/DP + EPLB with MTP (#25311) 
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Signed-off-by: Sage Moore <sage@neuralmagic.com>
Co-authored-by: Sage Moore <sage@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2025-11-05 15:22:17 +00:00
wangxiyuan
428bc7bf1c [V0 deprecation] Remove VLLM_USE_V1 usage in most modules (#27955) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2025-11-04 20:51:16 -08:00
Nick Hill
c9f66da8fd [PerfFix] Avoid separate thread for MP executor shm spin (#28012) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-11-04 08:33:55 -08:00
bnellnm
938772af03 [Kernels] Isolate modular kernel code from FusedMoEMethodBase subclasses. (#27123) 2025-11-04 21:59:45 +08:00
Mark McLoughlin
58279c60b5 [KV Connector] Make KVCacheConfig an explicit constructor argument (#27887) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2025-11-03 23:00:49 -08:00
Yue Zhang
685c99ee77 [KV offload] Offloading connector async scheduling support (#27648) 
Signed-off-by: KevinCheung2259 <2651309292@qq.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-11-01 21:08:56 +00:00
Nick Hill
0cdbe7b744 [Core] Async scheduling + structured outputs compatibility (#26866) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-11-01 00:35:04 +00:00
GuanLuo
d6517be3cd [Bugfix] Missing NIXL metadata for handshake initialization if instance spans multi-node (#26338) 
Signed-off-by: Guan Luo <gluo@nvidia.com>
Signed-off-by: GuanLuo <41310872+GuanLuo@users.noreply.github.com>
Signed-off-by: Guan Luo <41310872+GuanLuo@users.noreply.github.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2025-10-31 10:16:00 -07:00
Wentao Ye
a8141fa649 [Refactor] Remove VLLM_DEEPEP_LOW_LATENCY_ALLOW_NVLINK (#27750) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-30 15:32:39 -04:00
Ilya Markov
60f76baa66 [Misc] Replace CUDA_VISIBLE_DEVICES in DP with torch.cuda.set_device for device selection on cuda-like devices (#27564) 
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2025-10-30 11:41:44 -04:00
Nick Hill
2ce5c5d3d6 [BugFix] Handle unscheduled requests properly when async scheduling (#27756) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-10-29 21:04:25 -07:00
Nick Hill
d4aa144343 [BugFix] Fix handling of resumed reqs in SharedStorageConnector (#27719) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-10-29 20:16:52 +00:00
Nicolò Lucchesi
accb8fab07 [KVConnector] Add metrics to Prometheus-Grafana dashboard (#26811) 
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: Mark McLoughlin <markmc@redhat.com>
 2025-10-29 18:44:49 +00:00
Wentao Ye
5522fb274b [Chore] Optimize P2PNCCLEngine http_address (#27488) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-30 00:05:09 +08:00
Shaoting
a4a4f0f617 [KV Connector] Update lmcache connector with latest compatibility (#27681) 
Signed-off-by: Samuel Shen <slshen@uchicago.edu>
Co-authored-by: Samuel Shen <slshen@uchicago.edu>
 2025-10-29 05:38:37 +00:00
Kero Liang
02af36df36 [Bugfix] Fix allocation & free logic of SingleWriterShmRingBuffer (#27117) 
Signed-off-by: Kero Liang <kerorek@outlook.com>
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: donglu <donglu@cohere.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2025-10-28 15:01:24 +00:00
Samuel Shen
05e034f085 [nit]: Fix import for the lmcache integration (#27600) 
Signed-off-by: Samuel Shen <slshen@uchicago.edu>
Co-authored-by: Samuel Shen <slshen@uchicago.edu>
 2025-10-28 14:40:55 +00:00
Cyrus Leung
7c2bdb83dc [Misc] Clean up utils (#27552) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-27 09:05:40 +00:00
Kuntai Du
b853540388 [Core][Hybrid allocator + kv connector 1/n] Enable hybrid allocator + KV cache connector (#25712) 
Signed-off-by: KuntaiDu <kuntai@uchicago.edu>
Signed-off-by: Kuntai Du <kuntai@uchicago.edu>
 2025-10-24 23:34:18 -07:00
Zhuohan Li
56ed7609a9 Revert "[Misc] Remove use of CUDA_VISIBLE_DEVICES for device selectio… (#27502) 2025-10-25 05:31:43 +00:00
Yihua Cheng
83f478bb19 [KVConnector] Migrate the LMCache integration code to be vLLM native (#25542) 
Signed-off-by: ApostaC <yihua98@uchicago.edu>
 2025-10-25 00:23:53 +00:00
Wentao Ye
52efc34ebf [Log] Optimize Startup Log (#26740) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-24 19:27:04 -04:00
Pengchao Wang
d95d0f4b98 [Distributed] Basic set of configuration for large EP deployment on GB200 (#27328) 
Signed-off-by: Pengchao Wang <wpc@fb.com>
Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com>
 2025-10-24 14:16:44 -07:00
kourosh hakhamaneshi
7e1d697b56 [Bugfix] Fix MultiConnector stats reconstruction across process boundaries (#27366) 
Signed-off-by: Kourosh Hakhamaneshi <Kourosh@anyscale.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2025-10-24 17:08:05 +00:00
Chendi.Xue
699d62e6cf [NIXL][BUGFIX] delay done_recving queue cleanup to bottom of get_finished (#27297) 
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
 2025-10-24 17:01:41 +00:00
Rui Qiao
09a6a49eaf [Misc] Avoid "PyTorch non-writable tensors" warning in RayPPCommunicator (#27443) 
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
 2025-10-24 14:53:09 +08:00
usberkeley
c528b9006a Fix EventPublisherFactory logic for disabled KV cache events (#27419) 
Signed-off-by: Bradley <bradley.b.pitt@gmail.com>
 2025-10-24 05:00:01 +00:00
Ilya Markov
237cf6d32a [Misc] Remove use of CUDA_VISIBLE_DEVICES for device selection (fix DP slow startup time &c) (#26709) 
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2025-10-23 20:58:39 +08:00
dongbo910220
a0003b56b0 [Chore] Separate out system utilities from vllm.utils (#27201) 
Signed-off-by: dongbo910220 <1275604947@qq.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-10-22 20:25:25 +00:00
rongfu.leng
8669c69afa [Feature] publisher default set zmq in kv_event config (#26915) 
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-10-22 19:19:33 +00:00
Nicolò Lucchesi
4dfdb821c8 [P/D] Dynamic kv_output_aggregator collect size (#26734) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-10-22 18:07:58 +02:00
Chendi.Xue
7c4767f1eb [NIXL] use Host buffer to support TP_ratio > 1 for XPU (#27140) 
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@gmail.com>
 2025-10-22 15:28:13 +00:00
Mark McLoughlin
4ca13a8667 [NIXL] Terminate handshake listener thread in shutdown (#26404) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2025-10-22 16:59:53 +02:00
dongbo910220
3ae082c373 [Chore] Separate out optional dependency checks from vllm.utils (#27207) 
Signed-off-by: dongbo910220 <1275604947@qq.com>
Signed-off-by: dongbo910220 <32610838+dongbo910220@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-10-22 10:44:21 -04:00
Tyler Michael Smith
6c2eef5a5d [P/D] KVConnector for decode benchmarking (#25986) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-10-21 16:30:47 -07:00
Nick Hill
647214f3d5 [V0 Deprecation] Remove V0 executors (#27142) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-10-21 11:09:37 -07:00
David Whyte-Gray
ddeec11ba9 [Bugfix][P/D] Reduce num_threads used by nixl ucx backend (#27196) 
Signed-off-by: David Whyte-Gray <40244437+dagrayvid@users.noreply.github.com>
 2025-10-21 13:41:52 -04:00
dongbo910220
6c728f7771 [Chore] Separate out NCCL utilities from vllm.utils (#27197) 
Signed-off-by: dongbo910220 <1275604947@qq.com>
 2025-10-21 06:18:23 -07:00
Nicolò Lucchesi
72f431e709 [Nixl] Minor refactor to handshake related metadata (#26410) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-10-21 09:07:47 +02:00
Lunwen He
0eb8f2b880 create is_in_the_same_node on cpu (#26832) 
Co-authored-by: Lunwen He <lunwenh@meta.com>
 2025-10-21 02:04:14 +00:00
iAmir97
7a6c8c3fa1 [Chore] Separate out vllm.utils.network_utils (#27164) 
Signed-off-by: iAmir97 <Amir.balwel@embeddedllm.com>
Co-authored-by: iAmir97 <Amir.balwel@embeddedllm.com>
 2025-10-19 03:06:32 -07:00
Nick Hill
3b45075206 [Minor] Add some clarifying comments to recent changes (#27130) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-10-18 09:52:45 -07:00
Yongtao Huang
168e578efc Fix incorrect string formatting in barrier timeout exceptions (#27149) 
Signed-off-by: Yongtao Huang <yongtaoh2022@gmail.com>
 2025-10-18 09:51:57 -07:00
Isotr0py
6ac5e06f7c [Chore] Clean up pytorch helper functions in vllm.utils (#26908) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: isotr0py <2037008807@qq.com>
 2025-10-18 09:48:22 -07:00
Harry Mellor
6c9fdbf725 [Docs] Replace rst style double-backtick with md single-backtick (#27091) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-17 02:47:34 -07:00
Nick Hill
ab81379ea6 [Perf] Exploit out-of-band buffers in shm_broadcast (#26961) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-10-16 20:08:03 -07:00
Cyrus Leung
4d4d6bad19 [Chore] Separate out vllm.utils.importlib (#27022) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-17 00:48:59 +00:00
Bram Wasti
b2f78cbad4 [small][batch invariance] Rename the env and internal flags to simplify usage (#26855) 
Signed-off-by: Bram Wasti <bwasti@meta.com>
 2025-10-16 21:40:25 +00:00
Mark McLoughlin
4a510ab487 [NIXL] Improve request_finished() debug logs (#25665) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2025-10-16 15:55:17 +02:00