biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
14,921 Commits 2 Branches 140 Tags
56cb1baa667e413c9bfa38c7c44da38bd41fc612
Commit Graph

115 Commits

Author SHA1 Message Date
Harry Mellor
f83b933b84 [CI] Bump mypy version to 1.19.1 (#36104) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-10 09:18:28 -07:00
Zhuohan Li
04b67d8f62 Remove unused disable_fallback field (#36546) 2026-03-09 20:56:54 -07:00
aykoppol
25e02647c2 [Core] Add optional flags to check for repetitive token patterns in engine output (#35451) 
Signed-off-by: aykoppol <aykoppol+git@gmail.com>
 2026-03-03 12:23:25 +08:00
Wentao Ye
e113a30113 [Deprecation] Deprecate code in 0.17 as scheduled (#35441) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-02-28 17:32:37 +00:00
Nick Hill
b1d9f5372d [Model Runner V2] Warmup kernels (#35172) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-02-27 10:43:30 -08:00
Yiliu Dong
d940607629 [Core] Support min_tokens with speculative decoding (#32642) 
Signed-off-by: qianlihuang <yiliu.dong@qq.com>
Co-authored-by: qianlihuang <yiliu.dong@qq.com>
 2026-02-26 12:31:28 -05:00
Neil Schemenauer
54e2f83d0a [Feature] Lazy import for the "mistral" tokenizer module. (#34651) 
Signed-off-by: Neil Schemenauer <nas@arctrix.com>
 2026-02-23 00:43:01 -08:00
Martin Hickey
e97c46a92d [BugFix]: Fix local mypy issues (#34739) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-02-23 00:40:29 -08:00
Cyrus Leung
ea5ff3a1f6 [Refactor] Simplify BOS/EOS token handling (#34435) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-12 18:18:24 -08:00
Cyrus Leung
fb455ed547 [V0 Deprecation] Remove code related to per-request logits processors (#34400) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-12 20:44:28 +08:00
Cyrus Leung
ced2a92f40 [Refactor] Move validation to params definitions (#34362) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-11 19:33:15 -08:00
cjackal
35bf5d08e8 [bugfix] Fix online serving crash when text type response_format is received (#26822) 
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
Signed-off-by: j0shuajun <59368606+j0shuajun@users.noreply.github.com>
Co-authored-by: j0shuajun <59368606+j0shuajun@users.noreply.github.com>
 2026-01-16 12:23:54 +08:00
R3hankhan
1ab055efe6 [OpenAI] Extend VLLMValidationError to additional validation parameters (#31870) 
Signed-off-by: Rehan Khan <Rehan.Khan7@ibm.com>
 2026-01-07 14:45:49 +00:00
Michael Goin
bc5ef333e0 [Perf] Add skip_clone to SamplingParams for internal request handling (#31041) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-12-24 14:35:57 -08:00
Cyrus Leung
34a984274e [Misc] Refactor tokenizer interface (#29693) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-11-29 04:02:21 -08:00
kflu
ce58fdc1c3 Fix PoolingParams.skip_reading_prefix_cache type (#29364) 
Signed-off-by: KFL <kludev@gmail.com>
 2025-11-25 06:39:29 +00:00
Harry Mellor
316c8492bf Scheduled removal of guided_* config fields (#29326) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-11-25 05:24:05 +00:00
Cyrus Leung
56e96b37e4 [V0 Deprecation] Remove best_of (#29090) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-11-21 11:40:40 +08:00
Jialin Ouyang
40b6b38f2c [Core] Switch Flat logprob control from environment variable to SamplingParams (#28914) 
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
Co-authored-by: 22quinn <33176974+22quinn@users.noreply.github.com>
 2025-11-19 02:10:02 +00:00
wang.yuqi
a55b64635c [Model] Allow users to control skip reading cache per request. (#28194) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
 2025-11-16 00:04:50 -08:00
Nicolò Lucchesi
6f1e7f7226 [DisaggEverything] Tokens in<>out /generate endpoint (#24261) 
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-11-14 09:58:01 -07:00
Hanchenli
7c572544e4 [GPT-OSS] Structure_Tag support for gpt-oss tool-call in cot (#25515) 
Signed-off-by: Hanchenli <lihanc2002@gmail.com>
Signed-off-by: Hanchenli <61769611+Hanchenli@users.noreply.github.com>
Signed-off-by: Wei Wei <wwei6@meta.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Wei Wei <wwei6@meta.com>
Co-authored-by: Wei Wei <weiweinpu@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-10-17 21:55:54 -07:00
Harry Mellor
6c9fdbf725 [Docs] Replace rst style double-backtick with md single-backtick (#27091) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-17 02:47:34 -07:00
Harry Mellor
8fcaaf6a16 Update Optional[x] -> x | None and Union[x, y] to x | y (#26633) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-12 09:51:31 -07:00
Harry Mellor
d6953beb91 Convert formatting to use ruff instead of yapf + isort (#26247) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 07:06:22 -07:00
Harry Mellor
875d6def90 Add backward compatibility for GuidedDecodingParams (#25422) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-23 17:07:30 +01:00
William Song
c9ff9e6f0c [Docs] add the parallel sampling usage in LLMEngine and AsyncLLM (#24222) 2025-09-18 04:37:08 -07:00
Aaron Pham
29283e8976 [Chore] Cleanup guided namespace, move to structured outputs config (#22772) 
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-18 09:20:27 +00:00
Xingyu Liu
b3d7e3c845 [Sampler] Support returning all prompt logprobs (#23868) 
Signed-off-by: Xingyu Liu <charlotteliu12x@gmail.com>
Co-authored-by: 22quinn <33176974+22quinn@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-09-07 19:34:31 -07:00
Gabriel Marinho
5b8077b8ac Fix wrong truncate_prompt_tokens type hint (#22761) 
Signed-off-by: Gabriel Marinho <gmarinho@ibm.com>
Signed-off-by: Gabriel Marinho <104592062+gmarinho2@users.noreply.github.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Max de Bayser <mbayser@br.ibm.com>
 2025-08-30 20:39:38 +00:00
Harry Mellor
00976db0c3 [Docs] Fix warnings in docs build (#22588) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-08-10 05:49:51 -07:00
22quinn
54de71d0df [Sampler] Support returning all logprobs or logits (#21792) 
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
 2025-08-04 03:04:12 -07:00
Harry Mellor
313ae8c16a [Deprecation] Remove everything scheduled for removal in v0.10.0 (#20979) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-07-15 15:57:53 +00:00
afeldman-nm
dfada85eee [Frontend] Expose custom args in OpenAI APIs (#16862) 
Signed-off-by: Andrew Feldman <afeldman@neuralmagic.com>
Signed-off-by: Andrew Feldman <afeldman@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-06-18 17:41:11 -07:00
Simon Mo
02f0c7b220 [Misc] Add SPDX-FileCopyrightText (#19100) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2025-06-03 11:20:17 -07:00
CYJiang
d054da1992 [Misc] fix: add miss best_of param validation (#18555) 
Signed-off-by: googs1025 <googs1025@gmail.com>
 2025-06-03 11:02:07 -07:00
Harry Mellor
c6798baa9c Change top_k to be disabled with 0 (still accept -1 for now) (#17773) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-05-09 10:01:49 +00:00
Qiong Zhou Huang
39956efb3f [Bugfix] Fix bad words for Mistral models (#17753) 
Signed-off-by: Qiong Zhou Huang <qiong@phonic.co>
 2025-05-07 23:32:10 -07:00
Yang Wang
b8b0859b5c add more pytorch related tests for torch nightly (#17422) 
Signed-off-by: Yang Wang <elainewy@meta.com>
 2025-05-02 03:29:59 -07:00
Gabriel Marinho
1c2bc7ead0 Truncation control for embedding models (#14776) 
Signed-off-by: Gabriel Marinho <gmarinho@ibm.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Max de Bayser <mbayser@br.ibm.com>
 2025-04-30 09:24:57 +08:00
Harry Mellor
a6977dbd15 Simplify (and fix) passing of guided decoding backend options (#17008) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-04-29 19:02:23 +00:00
Nick Hill
8262a3e23b [Misc] Validate stop_token_ids contents (#17268) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-04-28 03:54:05 +00:00
Russell Bryant
f8acd01ff7 [V1] Add structural_tag support using xgrammar (#17085) 2025-04-26 14:06:37 +00:00
Chauncey
acba33a0f1 [Bugfix] Fix the issue where llm.generate cannot be called repeatedly after setting GuidedDecodingParams (#16767) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
 2025-04-22 06:02:20 +00:00
Alex Brooks
f8f9c0ba62 [Bugfix] Don't set an upper bound on repetition penalty (#16403) 
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-04-11 14:19:40 +08:00
yihong
04149cce27 [BugFix] fix some typos found by typos. (#16314) 
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
 2025-04-09 03:43:59 -07:00
Nick Hill
623e2ed29f [BugFix][V1] Quick fix for min_tokens with multiple EOS (#15407) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-03-24 15:58:59 -07:00
Nick Hill
fc1f67715d [BugFix][V1] Fix overhead related to bad_words sampling when not in use (#14894) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-03-16 14:53:34 -07:00
Nick Hill
4059adc31b [Misc][Minor] Simplify SamplingParams.__post_init__() (#14772) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-03-14 11:44:20 +08:00
22quinn
eb8b5eb183 [V1] Support bad_words in sampler (#13376) 
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-03-08 14:50:26 -08:00