biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
14,314 Commits 2 Branches 140 Tags
405f28d38df2ea4f320635c6e87e206b3ccbea2f
Commit Graph

33 Commits

Author SHA1 Message Date
Cyrus Leung
cd8b405bd0 [Refactor] Consolidate sequence normalization and enc-dec parsing (#33928) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-06 15:43:47 +00:00
Cyrus Leung
a502831d36 [Chore] Remove redundant input parsing methods (#33542) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-02 10:50:47 +00:00
Nick Hill
0cd259b2d8 [BugFix] Fix P/D with non-MoE DP (#33037) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-01-27 08:03:47 -08:00
Joshua Deng
91601ff478 [Feature] add session based streaming input support to v1 (#28973) 
Signed-off-by: Joshua Deng <joshuakdeng@gmail.com>
Signed-off-by: Patrick von Platen <patrick.v.platen@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-01-24 12:06:28 -08:00
dtc
1e584823f8 [Bugfix] Strengthen the check of X-data-parallel-rank in Hybrid LB mode (#32314) 
Signed-off-by: Tianchen Ding <dtcccc@linux.alibaba.com>
 2026-01-15 16:31:16 +08:00
Nick Hill
bd877162eb [BugFix] Support online dense model DP without overhead (#30739) 
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: njhill <nickhill123@gmail.com>
 2026-01-02 23:36:38 +08:00
maang-h
cc0f2a0e19 [Doc] Improve abnormal information string (#29655) 
Signed-off-by: maang <maang_h@163.com>
 2025-11-28 00:12:20 -08:00
Lucia Fang
b316ac6589 [V1] Support MP Executor for multi node distributed inference (#23691) 
Signed-off-by: Lu Fang <fanglu@fb.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Signed-off-by: Lucia Fang <fanglu@fb.com>
Signed-off-by: Lucia Fang <116399278+luccafong@users.noreply.github.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-11-16 09:01:21 +00:00
Ilya Markov
60f76baa66 [Misc] Replace CUDA_VISIBLE_DEVICES in DP with torch.cuda.set_device for device selection on cuda-like devices (#27564) 
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2025-10-30 11:41:44 -04:00
Cyrus Leung
6ebffafbb6 [Misc] Clean up more utils (#27567) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-27 15:30:38 +00:00
Zhuohan Li
56ed7609a9 Revert "[Misc] Remove use of CUDA_VISIBLE_DEVICES for device selectio… (#27502) 2025-10-25 05:31:43 +00:00
Kebe
51dd14ac2b [Bugfix][DP] Fix creating too many DP Placement Groups (#26880) 
Signed-off-by: Kebe <mail@kebe7jun.com>
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
Co-authored-by: Rui Qiao <ruisearch42@gmail.com>
 2025-10-23 20:16:51 +00:00
Ilya Markov
237cf6d32a [Misc] Remove use of CUDA_VISIBLE_DEVICES for device selection (fix DP slow startup time &c) (#26709) 
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2025-10-23 20:58:39 +08:00
Nick Hill
647214f3d5 [V0 Deprecation] Remove V0 executors (#27142) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-10-21 11:09:37 -07:00
iAmir97
7a6c8c3fa1 [Chore] Separate out vllm.utils.network_utils (#27164) 
Signed-off-by: iAmir97 <Amir.balwel@embeddedllm.com>
Co-authored-by: iAmir97 <Amir.balwel@embeddedllm.com>
 2025-10-19 03:06:32 -07:00
Patrick von Platen
b038d9c40c [Data-parallel] Allow DP>1 for world_size > num_gpus on node (8) (#26367) 
Signed-off-by: Patrick von Platen <patrick.v.platen@gmail.com>
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Rui Qiao <ruisearch42@gmail.com>
 2025-10-17 08:24:42 -07:00
Michael Goin
3e051bda82 [UX] Replace VLLM_ALL2ALL_BACKEND with --all2all-backend (#26732) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-10-13 18:12:52 -07:00
Harry Mellor
8fcaaf6a16 Update Optional[x] -> x | None and Union[x, y] to x | y (#26633) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-12 09:51:31 -07:00
Rui Qiao
757fa4a4da [DP][ray] Support different VLLM_RAY_DP_PACK_STRATEGY (#23849) 
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
 2025-10-09 19:53:43 -07:00
Ayush Satyam
5e65d6b2ad fix[DP][v1]: Prevent hangs from mismatched worker configurations (#26218) 
Signed-off-by: Ayush Satyam <ayushsatyam146@gmail.com>
 2025-10-07 22:55:08 -07:00
Harry Mellor
6c04638214 Fix per file ruff ignores related to line length (#26262) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-06 05:12:40 +00:00
Harry Mellor
d6953beb91 Convert formatting to use ruff instead of yapf + isort (#26247) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 07:06:22 -07:00
Xinyu Chen
bb58dc8c20 [DP] Create placement groups by ray_device_key (#25026) 
Signed-off-by: Xinyu Chen <xinyu1.chen@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2025-09-17 08:57:25 +00:00
22quinn
0cdd213641 [Misc] Improve Worker process title and logging prefix (#22205) 
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
 2025-09-08 21:43:48 -07:00
Kebe
a43a3f1770 [Bugfix][DP] DP distribution does not require ray[default] (#23822) 
Signed-off-by: Kebe <mail@kebe7jun.com>
 2025-09-03 13:21:36 -07:00
Fanli Lin
8f0d7eaea8 [XPU] Fix OOM issue for data parallel with Ray backend (#22500) 
Signed-off-by: Fanli Lin <fanli.lin@intel.com>
Signed-off-by: Fanli Lin <fanli0116@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-08-27 19:57:38 +08:00
Ming Yang
10f535c086 [Bugfix] Fix port conflict by obtaining a list of open ports upfront (#21894) 
Signed-off-by: Ming Yang <minos.future@gmail.com>
 2025-08-21 10:22:18 -07:00
Rui Qiao
302962e806 [Bugfix] Skip dead and non-GPU nodes for Ray DP engine allocation (#22275) 
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
 2025-08-05 20:35:32 -07:00
Yinghai Lu
11ef7a611e [BugFix] Set CUDA_VISIBLE_DEVICES before spawning the subprocesses (#21211) 
Signed-off-by: Yinghai Lu <yinghai@thinkingmachines.ai>
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Rui Qiao <ruisearch42@gmail.com>
 2025-07-23 21:44:04 -07:00
Robert Shaw
d5b981f8b1 [DP] Internal Load Balancing Per Node [one-pod-per-node] (#21238) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-07-23 20:57:32 -07:00
Rui Qiao
217937221b Elastic Expert Parallel Initial Support (#20775) 
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
 2025-07-18 17:46:09 -07:00
Rui Qiao
a6d795d593 [DP] Copy environment variables to Ray DPEngineCoreActors (#20344) 
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
 2025-07-07 10:14:22 -07:00
Nick Hill
657f2f301a [DP] Support external DP Load Balancer mode (#19790) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-07-02 10:21:52 -07:00