biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
15,412 Commits 2 Branches 140 Tags
598190aac38a42d8c51ea46a3061e46d9078b3a5
Commit Graph

1025 Commits

Author SHA1 Message Date
Chang Su
acf7292bf2 [Misc] Move --grpc CLI argument into make_arg_parser (#38570) 
Signed-off-by: Chang Su <chang.s.su@oracle.com>
 2026-03-31 03:24:05 -07:00
Chauncey
ce884756f0 [Feature]: add presence_penalty and frequency_penalty fields to Responses API (#38613) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-03-31 08:45:57 +00:00
wang.yuqi
d9d21eb8e3 [Frontend][3/n] Improve pooling entrypoints | scoring. (#28631) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-03-31 07:52:00 +00:00
yzong-rh
3683fe6c06 [Bugfix] Fix shared-object aliasing in n>1 streaming with tool calls (#38158) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
Signed-off-by: Yifan <yzong@redhat.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-03-30 10:12:13 +00:00
Juan Pérez de Algaba
57861ae48d (security) Fix SSRF in batch runner download_bytes_from_url (#38482) 
Signed-off-by: jperezde <jperezde@redhat.com>
 2026-03-30 07:10:01 +00:00
cjackal
2babac0bed [frontend] dump openai responses type by alias (#38262) 
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
 2026-03-27 05:58:20 +00:00
wang.yuqi
dcdc145893 [CI] Reorganize scoring tests (#38207) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-03-26 12:07:01 +00:00
Matej Rojec
2908094567 Add /v1/chat/completions/batch endpoint for batched chat completions (#38011) 
Signed-off-by: Matej Rojec <64556640+MatejRojec@users.noreply.github.com>
 2026-03-26 12:13:33 +08:00
Flora Feng
e2db2b4234 [Tool Parser][1/3] Pass tools to ToolParser constructor (#38029) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-03-26 10:29:06 +08:00
Cyrus Leung
ba2f0acc2d [Misc] Reorganize inputs (#35182) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-03-25 10:22:54 -07:00
Andreas Karatzas
9ac2fcafbb [CI] Fix realtime WebSocket timeout deadlock and unhandled model validation errors (#37483) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-03-25 11:24:33 +01:00
Chauncey
a32783bb35 [Bugfix] Fix IndexError when accessing prev_tool_call_arr in OpenAIToolParser (#37958) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-03-25 12:06:21 +08:00
Sungjae Lee
4731884796 [Feature] limit thinking tokens (hard limit) (#20859) 
Signed-off-by: Sungjae Lee <33976427+llsj14@users.noreply.github.com>
Signed-off-by: Sungjae Lee <sung-jae.lee@navercorp.com>
Signed-off-by: Chauncey <chaunceyjiang@gmail.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-24 09:53:07 -07:00
Flora Feng
2e67fa756d Fix tool_parser_cls type annotation from Callable to type[ToolParser] (#37957) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-03-23 22:58:27 -07:00
jetxa
16a664df24 [Frontend][Bugfix] Pass default_chat_template_kwargs to AnthropicServingMessages (#37899) 
Signed-off-by: jetxa <jetxzhang@outlook.com>
 2026-03-24 05:00:12 +00:00
Andrew Xia
9ace378a63 [Frontend][Responses API] Fix arrival_time recording for TTFT on initial request (#37498) 
Signed-off-by: Andrew Xia <axia@meta.com>
 2026-03-23 09:58:08 +00:00
Bongwoo Bak
17ee641c45 [Responses API] Add kv_transfer_params for PD disaggregation (#37424) 
Signed-off-by: bongwoobak <bongwoobak@gmail.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-03-21 13:48:54 +08:00
Isotr0py
c7f98b4d0a [Frontend] Remove librosa from audio dependency (#37058) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-03-21 11:36:15 +08:00
wang.yuqi
ed359c497a [Model] Deprecate the score task (this will not affect users). (#37537) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-03-20 08:07:56 +00:00
Flora Feng
9040151fe1 [V0 Deprecation] Deprecate --disable-frontend-multiprocessing (#37612) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-03-20 11:31:43 +08:00
Ifta khairul Alam Adil
104605cbf2 Remove deprecated reasoning_content message field(part-2) (#37480) 
Signed-off-by: JartX <sagformas@epdcenter.es>
Signed-off-by: Ifta Khairul Alam Adil <ikaadil007@gmail.com>
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Philip Ottesen <phiott256@gmail.com>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
Signed-off-by: Andy Lo <andy@mistral.ai>
Signed-off-by: Thillai Chithambaram <thillaichithambaram.a@gmail.com>
Signed-off-by: sihao.li <sihao.li@intel.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: JartX <sagformas@epdcenter.es>
Co-authored-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Philip Ottesen <phiott256@gmail.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Giancarlo Delfin <32987265+TheEpicDolphin@users.noreply.github.com>
Co-authored-by: Andy Lo <andy@mistral.ai>
Co-authored-by: Thillai Chithambaram <79466435+thillai-c@users.noreply.github.com>
Co-authored-by: sihao_li <165983188+1643661061leo@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-19 15:20:08 +00:00
DorBernsohn
c63ca2b2e6 [Bugfix] Add Kimi-K2.5 reasoning/tool parser aliases and tool_call_id support (#37438) 
Signed-off-by: DorBernsohn <dor.bernsohn@gmail.com>
 2026-03-19 21:08:00 +08:00
Chauncey
b322b197f1 [Build] Bump python openai version (#32316) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-03-18 18:20:10 +08:00
Andrew Xia
0e95916155 [responsesAPI] parser.extract_response_outputs can take in token IDs (#37130) 
Signed-off-by: Andrew Xia <axia@meta.com>
 2026-03-18 05:31:31 +00:00
Ekagra Ranjan
b5ca9c3557 [Models] Cohere ASR (#35809) 
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
 2026-03-17 21:04:17 +00:00
Ning Xie
c9e5096256 [openapi] remove redundant exception stack trace[4/N] (#37157) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2026-03-17 15:06:25 +00:00
Isotr0py
a836524d20 [Chore] Replace all base64 usages with faster pybase64 package (#37290) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-03-17 14:44:19 +00:00
Sage
59192dfd39 [Frontend] Complete OpenAI render delegation (#37287) 
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com>
 2026-03-17 13:53:55 +00:00
Umut Polat
56cb1baa66 [Misc] Use VLLMValidationError in batch, pooling, and tokenize protocol validators (#36256) 
Signed-off-by: umut-polat <52835619+umut-polat@users.noreply.github.com>
 2026-03-17 13:52:30 +00:00
Sage
00f8e0d211 [Frontend] Delegate tokenization serving preprocessing to OpenAIServingRender (#37266) 
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com>
 2026-03-17 11:22:54 +00:00
Julien Denize
5db91f0aaf Fix some Mistral parser issues (#37209) 
Signed-off-by: juliendenize <julien.denize@mistral.ai>
 2026-03-17 00:08:56 +00:00
Chauncey
6682c231fa [Bugfix] Add error handling for FINISHED_ERROR in OpenAIServing (#37148) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-03-16 16:27:47 +00:00
Andreas Karatzas
57a314d155 [CI][Bugfix] Fix 500 errors from priority overflow and TemplateError subclasses in schema fuzz tests (#37127) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-03-16 05:27:21 +00:00
Isotr0py
6590a3ecda [Frontend] Remove torchcodec from audio dependency (#37061) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-03-15 05:15:59 +00:00
seanmamasde
84868e4793 [Bugfix][Frontend] Fix audio transcription for MP4, M4A, and WebM formats (#35109) 
Signed-off-by: seanmamasde <seanmamasde@gmail.com>
 2026-03-14 08:44:03 -07:00
Sergey Zinchenko
4a718e770d [Bug] Fix Failure in /v1/chat/completions/render for Multimodal Requests (https://github.com/vllm-project/vllm/issues/35665) (#35684) 2026-03-14 14:10:11 +00:00
Andrew Xia
f680dc1b39 [responsesAPI] prioritize content over summary in reasoning item input (#36516) 
Signed-off-by: Andrew Xia <axia@meta.com>
Signed-off-by: Andrew Xia <mitandrewxia@gmail.com>
Signed-off-by: Andrew Xia <axia@fb.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Andrew Xia <axia@fb.com>
 2026-03-14 09:20:30 +08:00
Sage
a2268617cf [Frontend] Delegate preprocessing to OpenAIServingRender (#36483) 
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com>
 2026-03-13 00:39:43 -07:00
Csrayz
bc2c0c86ef [Frontend] Fix usage incorrectly returned with empty stream_options` (#36379) 
Signed-off-by: Csrayz <33659823+Csrayz@users.noreply.github.com>
 2026-03-13 03:33:04 +00:00
Eunkwang Jeon
bdc2343454 [Bugfix] Fix KeyError in parse_response_input for reasoning items with optional content (#34499) 
Signed-off-by: jeonsworld <jeonsworld@gmail.com>
 2026-03-13 00:13:36 +08:00
Sage
06e0bc21d2 [Frontend] Split OpenAIServingModels into OpenAIModelRegistry + OpenAIServingModels (#36536) 
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com>
 2026-03-12 03:29:37 -07:00
Chauncey
5a71cdd76e [Bugfix] Fix crash when tool_choice=required exceeds max_tokens (#36841) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-03-12 03:28:45 -07:00
Chauncey
9fe404ed04 [Frontend] OpenAI Responses API supports Tool/Function calling with streaming (#29947) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-03-12 15:03:50 +08:00
Julien Denize
a3ea760ea5 Add 'none' reasoning effort to ChatCompletionRequest (#36238) 
Signed-off-by: Julien Denize <julien.denize@mistral.ai>
 2026-03-11 15:45:34 +00:00
Flora Feng
d5080aeaa4 [Refactor] Remove deadcode in Responses API serving (#36726) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-03-11 07:11:41 +00:00
Ning Xie
fe714dd507 [openapi server] log exception in exception handler(2/N) (#36201) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2026-03-10 20:16:30 -07:00
Alvin Tang
cf88b23749 fix: check HTTP status in batch read_file to prevent silent failures (#36397) 
Signed-off-by: gambletan <ethanchang32@gmail.com>
Co-authored-by: gambletan <ethanchang32@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-10 07:22:40 -07:00
Wentao Ye
7279374f91 [Perf] Compute maxsim in worker side, reducing redundant copies, 2.7% E2E throughput improvement (#36159) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-03-09 20:55:58 -07:00
Wentao Ye
941e52c298 [Refactor] Simplify chat_completion_full_generator for tool parsers (#35634) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-03-09 23:33:46 +08:00
Tianyu Guo
5578f2a4d3 Support online use_audio_in_video (#36319) 
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-03-09 07:16:44 -07:00