biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
11,878 Commits 2 Branches 140 Tags
b08025a83bf416d97d0547ac52c3909356e118c4
Commit Graph

57 Commits

Author SHA1 Message Date
Russell Bryant
b08025a83b [Docs] Discuss api key limitations in security guide (#29922) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-12-02 20:57:28 -08:00
Lucia Fang
b316ac6589 [V1] Support MP Executor for multi node distributed inference (#23691) 
Signed-off-by: Lu Fang <fanglu@fb.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Signed-off-by: Lucia Fang <fanglu@fb.com>
Signed-off-by: Lucia Fang <116399278+luccafong@users.noreply.github.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-11-16 09:01:21 +00:00
Iceber Gu
e0d6b4a867 [CLI] add --max-tokens to vllm complete (#28109) 
Signed-off-by: Iceber Gu <caiwei95@hotmail.com>
 2025-11-07 12:21:40 +00:00
wangxiyuan
428bc7bf1c [V0 deprecation] Remove VLLM_USE_V1 usage in most modules (#27955) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2025-11-04 20:51:16 -08:00
Cyrus Leung
ecca3fee76 [Frontend] Add vllm bench sweep to CLI (#27639) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-29 05:59:48 -07:00
Cyrus Leung
7c2bdb83dc [Misc] Clean up utils (#27552) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-27 09:05:40 +00:00
Yeshwanth N
71b1c8b667 [Chore]:Extract math and argparse utilities to separate modules (#27188) 
Signed-off-by: Yeshwanth Surya <yeshsurya@gmail.com>
Signed-off-by: Yeshwanth N <yeshsurya@gmail.com>
Signed-off-by: yeshsurya <yeshsurya@gmail.com>
 2025-10-26 04:03:32 -07:00
dongbo910220
a0003b56b0 [Chore] Separate out system utilities from vllm.utils (#27201) 
Signed-off-by: dongbo910220 <1275604947@qq.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-10-22 20:25:25 +00:00
Nick Hill
647214f3d5 [V0 Deprecation] Remove V0 executors (#27142) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-10-21 11:09:37 -07:00
iAmir97
7a6c8c3fa1 [Chore] Separate out vllm.utils.network_utils (#27164) 
Signed-off-by: iAmir97 <Amir.balwel@embeddedllm.com>
Co-authored-by: iAmir97 <Amir.balwel@embeddedllm.com>
 2025-10-19 03:06:32 -07:00
Harry Mellor
8fcaaf6a16 Update Optional[x] -> x | None and Union[x, y] to x | y (#26633) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-12 09:51:31 -07:00
Aydin Abiar
76afe4edf8 [Bugfix] Fix vllm bench ... on CPU-only head nodes (#25283) 
Signed-off-by: Aydin Abiar <aydin@anyscale.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Aydin Abiar <aydin@anyscale.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-10-08 16:06:42 +00:00
Harry Mellor
d6953beb91 Convert formatting to use ruff instead of yapf + isort (#26247) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 07:06:22 -07:00
Harry Mellor
e7f27ea648 Improve --help for enhanced user experience (#24903) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-24 23:08:18 +00:00
Cyrus Leung
6c117cff7d [Frontend] Pass API server count to each process (#23717) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-09-20 01:15:19 +08:00
Simon Mo
e111d5b0ae [CLI] Use streaming in CLI chat and completion commands (#23769) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2025-09-17 22:30:26 -07:00
Harry Mellor
bc1d02ac85 [Docs] Add comprehensive CLI reference for all large vllm subcommands (#22601) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-08-11 00:13:33 -07:00
Cyrus Leung
139d155781 [Frontend] Use engine argument to control MM cache size (#22441) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-08-07 09:47:10 -07:00
Cyrus Leung
766bc8162c [Core] Store only the keys for multi-modal data in P0 (#22198) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-08-07 01:45:04 -07:00
Nick Hill
3146519add [BugFix] Don't change title of top-level process (#22032) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-08-01 07:37:55 -07:00
wuhang
e6680f9e25 [Bugfix] Add log prefix in non-dp mode engine core (#21889) 
Signed-off-by: wuhang <wuhang6@huawei.com>
 2025-08-01 09:04:16 +00:00
Cyrus Leung
b4e081cb15 [Bugfix] Disable multi-modal preprocessor cache for DP (#21896) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-08-01 08:03:56 +01:00
Cyrus Leung
44bc46da60 [Bugfix] Actually disable processing cache when API server is scaled out (#21839) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-07-29 20:36:04 -07:00
Nick Hill
7234fe2685 [Misc] Rework process titles (#21780) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-07-29 05:14:47 +00:00
Nick Hill
9c8b2c2a8a [DP] Support api-server-count > 0 in hybrid DP LB mode (#21510) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-07-24 20:18:16 -07:00
Chauncey
6da0078523 [Feat] Allow custom naming of vLLM processes (#21445) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-07-24 03:15:23 -07:00
Robert Shaw
d5b981f8b1 [DP] Internal Load Balancing Per Node [one-pod-per-node] (#21238) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-07-23 20:57:32 -07:00
Harry Mellor
b637e9dcb8 Add full serve CLI reference back to docs (#20978) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-07-15 17:42:30 +00:00
Reid
9887e8ec50 [Misc] Remove unused function (#20909) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
 2025-07-14 10:48:55 +00:00
Reid
2c7fa47161 Fix: Add missing EOFError handling in CLI complete command (#20896) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
 2025-07-14 07:09:57 +00:00
Reid
8369b7c2a9 [Misc] improve error msg (#20604) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
 2025-07-07 21:45:18 -07:00
Reid
619b9f5c7e [Frontend] fix duplicate output for bench subcmd (#20446) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
 2025-07-03 08:02:06 -07:00
Reid
9854dc9040 [Frontend] improve vllm bench <bench_type> --help display (#20430) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
 2025-07-03 14:22:16 +00:00
Nick Hill
059d4cdb49 [BugFix] Fix DP headless mode arg validation (#20398) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-07-02 17:15:32 -07:00
Nick Hill
657f2f301a [DP] Support external DP Load Balancer mode (#19790) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-07-02 10:21:52 -07:00
Aaron Pham
c4cf260677 [Perf][CLI] Improve overall startup time (#19941) 2025-06-22 23:11:22 +00:00
Ye (Charlotte) Qi
2c11a29f0b [Misc] Simplify vllm bench cli subcommand implementation (#19948) 2025-06-22 12:34:48 -04:00
Hyogeun Oh (오효근)
bb4a0decef [Misc] Correct broken docs link (#19553) 
Signed-off-by: Zerohertz <ohg3417@gmail.com>
 2025-06-12 22:27:13 -07:00
Reid
1aeb925f34 [Frontend] improve vllm run-batch --help display (#19187) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
 2025-06-05 11:16:25 +00:00
Simon Mo
02f0c7b220 [Misc] Add SPDX-FileCopyrightText (#19100) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2025-06-03 11:20:17 -07:00
Rui Qiao
bdce64f236 [V1] Support DP with Ray (#18779) 2025-06-02 21:15:13 -07:00
Nick Hill
2dbe8c0774 [Perf] API-server scaleout with many-to-many server-engine comms (#17546) 2025-05-30 08:17:00 -07:00
Reid
435fa95444 [Frontend] add run batch to CLI (#18804) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
 2025-05-28 07:08:57 -07:00
Reid
441dc63ac7 [Frontend] improve vllm serve --help display (#18643) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
 2025-05-24 07:53:22 +00:00
Reid
20d8ce81eb [Frontend] add --quick option for vllm chat/complete (#18297) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
 2025-05-19 09:36:13 -07:00
Nick Hill
55aa7af994 [V1] DP scale-out (2/N): Decouple engine process management and comms (#15977) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-05-13 10:48:21 -07:00
Reid
ada50aa295 [bugfix] fix the wrong parser (#17958) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
 2025-05-12 04:58:02 +00:00
rongfu.leng
7bdfd29a35 [Misc] add collect_env to cli and docker image (#16759) 
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
 2025-04-17 22:13:35 -07:00
Michael Goin
b4fe16c75b Add vllm bench [latency, throughput] CLI commands (#16508) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-04-14 23:10:35 -07:00
Reid
fad6e2538e [Misc] add description attribute in CLI (#15921) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
 2025-04-07 22:30:35 +00:00