biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
10,671 Commits 2 Branches 140 Tags
5beacce2eae2cb36e44498e2ea80b2d3ee34fe61
Commit Graph

1006 Commits

Author SHA1 Message Date
RED
c9461e05a4 Support Anthropic API /v1/messages Endpoint (#22627) 
Signed-off-by: liuli <ll407707@alibaba-inc.com>
Co-authored-by: liuli <ll407707@alibaba-inc.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-10-22 09:13:18 -07:00
Russell Bryant
58fab50d82 [Frontend] Require flag for loading text and image embeds (#27204) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-22 15:52:02 +00:00
Wentao Ye
1a0f4defb7 [Log] Add Warning for LLM(data_parallel_size=k) single-process DP Usage (#27282) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-22 12:12:21 +00:00
wang.yuqi
1f633b8632 [Frontend][3/N] Improve all pooling task | Support binary embedding response (#27066) 
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-10-22 18:38:57 +08:00
ExtReMLapin
a4c29e6e82 fixed reasoning streaming with tool_choice="required" (#24108) 
Signed-off-by: CNE Pierre FICHEPOIL <pierre-1.fichepoil@gendarmerie.interieur.gouv.fr>
Signed-off-by: ExtReMLapin <3909752+ExtReMLapin@users.noreply.github.com>
Co-authored-by: CNE Pierre FICHEPOIL <pierre-1.fichepoil@gendarmerie.interieur.gouv.fr>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2025-10-22 09:42:55 +00:00
Nick Hill
647214f3d5 [V0 Deprecation] Remove V0 executors (#27142) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-10-21 11:09:37 -07:00
Russell Bryant
3ada34f9cb [Frontend] Enforce tokenize=False when applying chat template (#27205) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-10-21 02:57:34 +00:00
Cyrus Leung
d31f7844f8 [Misc] Move utils to avoid conflicts with stdlib, and move tests (#27169) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-19 05:20:55 -07:00
iAmir97
7a6c8c3fa1 [Chore] Separate out vllm.utils.network_utils (#27164) 
Signed-off-by: iAmir97 <Amir.balwel@embeddedllm.com>
Co-authored-by: iAmir97 <Amir.balwel@embeddedllm.com>
 2025-10-19 03:06:32 -07:00
Cyrus Leung
b3aba04e5a [Benchmark] Convenience script for multiple parameter combinations (#27085) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-18 23:57:01 -07:00
Hanchenli
7c572544e4 [GPT-OSS] Structure_Tag support for gpt-oss tool-call in cot (#25515) 
Signed-off-by: Hanchenli <lihanc2002@gmail.com>
Signed-off-by: Hanchenli <61769611+Hanchenli@users.noreply.github.com>
Signed-off-by: Wei Wei <wwei6@meta.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Wei Wei <wwei6@meta.com>
Co-authored-by: Wei Wei <weiweinpu@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-10-17 21:55:54 -07:00
Harry Mellor
6c9fdbf725 [Docs] Replace rst style double-backtick with md single-backtick (#27091) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-17 02:47:34 -07:00
Cyrus Leung
4d4d6bad19 [Chore] Separate out vllm.utils.importlib (#27022) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-17 00:48:59 +00:00
Andrew Xia
e6ba2000ae [gpt-oss][1/N] EZ: refactor serving_responses for modularity (#26948) 
Signed-off-by: Andrew Xia <axia@meta.com>
 2025-10-16 18:44:06 +00:00
Cyrus Leung
d2740fafbf [Chore] Separate out vllm.utils.collections (#26990) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-16 08:35:35 +00:00
Cyrus Leung
f6cdc9a02f [Chore] Rename utils submodules (#26920) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-16 03:58:13 +00:00
InChang Jeong
0ecc553ee6 [Bugfix] reasoning_parser parameter handling in run_batch.py (#26225) 
Signed-off-by: inc-jeong <inc.jeong@navercorp.com>
Signed-off-by: InChang Jeong <inc.jeong@navercorp.com>
Co-authored-by: USER <user@AL02367916.local>
 2025-10-16 10:24:05 +08:00
Pradeep Dasigi
4794c2bd92 Olmo 3 tool parser and tests (#26143) 
Signed-off-by: Pradeep Dasigi <pradeepd@allenai.org>
 2025-10-15 16:36:12 +00:00
Cyrus Leung
828523ad8e [Chore] Separate out vllm.utils.async_utils (#26913) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-15 15:33:00 +00:00
Cyrus Leung
136a17fe6e [Chore] Separate out vllm.utils.func (#26904) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-15 13:03:58 +00:00
wangxiyuan
8f4b313c37 [Misc] rename torch_dtype to dtype (#26695) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2025-10-15 12:11:48 +00:00
wang.yuqi
f54f85129e [Model][2/N] Improve all pooling task | Support multi-vector retrieval (#25370) 
Signed-off-by: wang.yuqi <noooop@126.com>
 2025-10-15 11:14:41 +00:00
Tao Hui
85a65e7f51 [Model] Add DeepSeek-V3.1 reasoning parser (split from PR #24972) (#25589) 
Signed-off-by: taohui <taohui3@gmail.com>
Signed-off-by: Tao Hui <taohui3@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2025-10-15 11:09:52 +08:00
Morrison Turnansky
96b9aa5aa0 [Frontend][torch.compile] CompilationConfig Overhaul (#20283): name change compilation level to compilation mode, deprecation compilation level (#26355) 
Signed-off-by: morrison-turnansky <mturnans@redhat.com>
Signed-off-by: Morrison Turnansky <mturnans@redhat.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2025-10-15 02:51:16 +00:00
Zhikaiiii
9354660036 [Bugfix]fix Qwen3 xml tool parser (#26345) 
Signed-off-by: Zhikaiiii <1658973216@qq.com>
 2025-10-15 09:50:30 +08:00
Ye Hu
0512c04aee [frontend][gptoss] Add per turn stats into Harmony Context (#25061) 
Signed-off-by: lacora <hyelacora@gmail.com>
Co-authored-by: Ye Hu <yehu@fb.com>
 2025-10-14 16:48:13 -07:00
Nan Qin
9d6964926e fix: response_format for completion (#23212) 
Signed-off-by: Nan2018 <qinnanjoshua@gmail.com>
 2025-10-14 21:23:22 +00:00
Chauncey
df850c4912 [Feature][Responses API] Stream Function Call - harmony (#24317) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-10-14 08:31:43 -07:00
Max Wittig
fd85c9f426 [Bugfix][FE]: Always include usage with --enable-force-include-usage (#20983) 
Signed-off-by: Max Wittig <max.wittig@siemens.com>
Signed-off-by: Antoine Auger <antoineauger@users.noreply.github.com>
Co-authored-by: Antoine Auger <antoineauger@users.noreply.github.com>
 2025-10-14 09:17:39 +02:00
Lucia Fang
8317f72354 [Misc][DP] support customized aggregated logger for dp (#24354) 
Signed-off-by: Lu Fang <fanglu@fb.com>
 2025-10-13 17:45:59 -07:00
Jialin Ouyang
35bc22f23c [ResponseAPI] Further polish message serialization and unit tests (#26728) 
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
 2025-10-13 23:31:35 +00:00
wang.yuqi
d2a7938582 [Frontend][1/N] Improve all pooling task | Support FP16 Embedding Base64 (Still uses fp32 by default). (#26414) 
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: Maximilien de Bayser <maxdebayser@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-10-13 19:06:43 +00:00
Aleksei Tsvetkov
5c7fe25491 [Misc] Separate prompt logging to debug (#26713) 
Signed-off-by: Aleksei Tsvetkov <aitsvet@ya.ru>
 2025-10-13 09:04:18 -07:00
Jialin Ouyang
4073c82c4e [ResponseAPI] Simplify input/output message serialization (#26620) 
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
 2025-10-13 09:59:15 +00:00
CSWYF3634076
782505ed8e [Model] Add reasoning_parser and tool_parser for Ernie45 thinking (#25027) 
Signed-off-by: wangyafeng <wangyafeng@baidu.com>
 2025-10-13 15:55:20 +08:00
Harry Mellor
8fcaaf6a16 Update Optional[x] -> x | None and Union[x, y] to x | y (#26633) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-12 09:51:31 -07:00
Chauncey
d0bed837ac [Refactor]Reduce duplicate code in serving_chat (#26627) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-10-11 12:04:49 +00:00
Chauncey
910abdbd08 [Bugfix] fixed top_logprobs: -1 does not appear to work as intended (#26470) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-10-11 00:41:17 +08:00
Chauncey
720d3cd0f0 [CI] fix ruff format (#26579) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-10-10 03:02:12 -07:00
Luis Tomas Bolivar
3ee202ea1e [GPT-OSS] Add support for arrays at tool message content (#25593) 
Signed-off-by: Luis Tomas Bolivar <ltomasbo@redhat.com>
 2025-10-10 09:00:45 +00:00
Cyrus Leung
ad430a67ca [Metrics] Log multi-modal cache stats and fix reset (#26285) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-10 01:45:55 -07:00
Julien Denize
c6187f55f7 Refactor MistralTokenizer (#26358) 
Signed-off-by: Julien Denize <julien.denize@mistral.ai>
 2025-10-09 22:48:58 +00:00
Cyrus Leung
4bdf7ac593 [Bugfix] Fix SHM cache initialization (#26427) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-09 02:48:04 -07:00
Aydin Abiar
76afe4edf8 [Bugfix] Fix vllm bench ... on CPU-only head nodes (#25283) 
Signed-off-by: Aydin Abiar <aydin@anyscale.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Aydin Abiar <aydin@anyscale.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-10-08 16:06:42 +00:00
Harry Mellor
2f99f2f506 Tidy vllm/config/__init__.py to only add classes and functions (#26405) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-08 07:10:00 -07:00
Grant Holmes (Ren)
d100d78eb3 Optimize KV cache distribution for asymmetric pipeline parallelism (#25164) 
Signed-off-by: gholmes829 <g.holmes429@gmail.com>
 2025-10-07 09:20:30 +00:00
Andrew Xia
185d8ed44f [responsesAPI][bugfix] serialize harmony messages (#26185) 
Signed-off-by: Andrew Xia <axia@meta.com>
Co-authored-by: Ye (Charlotte) Qi <yeq@meta.com>
 2025-10-07 07:07:53 +00:00
Cyrus Leung
d9836d4517 [Deprecation] Deprecate LLM.set_tokenizer (#26333) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-07 06:50:57 +00:00
Cyrus Leung
44b9af5bb2 [Benchmark] Enable MM Embedding benchmarks (#26310) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-06 19:51:58 +00:00
Cyrus Leung
391612e78b [Frontend] Consolidate tokenizer init code (#26276) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-06 09:34:52 +00:00