biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
15,337 Commits 2 Branches 140 Tags
4f2ed5fddb52b45ea8a8b0dad4a354802a6b3bea
Commit Graph

300 Commits

Author SHA1 Message Date
Cyrus Leung
ba2f0acc2d [Misc] Reorganize inputs (#35182) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-03-25 10:22:54 -07:00
Harry Mellor
d215d1efca [Mypy] Better fixes for the mypy issues in vllm/config (#37902) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-25 06:14:43 -07:00
wang.yuqi
1b6cb920e6 [Deprecate] Deprecate pooling multi task support. (#37956) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-03-24 14:07:47 +00:00
wang.yuqi
ed359c497a [Model] Deprecate the score task (this will not affect users). (#37537) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-03-20 08:07:56 +00:00
wang.yuqi
a3189a08b0 [Model] Consolidate score logic by introduce score_type (#36479) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-03-10 13:32:25 +00:00
wang.yuqi
fff3711a24 [Frontend][2/n] Improve pooling entrypoints | embed. (#36110) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
 2026-03-09 11:42:19 +08:00
lif
00b814ba5a [V0 Deprecation] Remove unused swap_space parameter (#36216) 
Signed-off-by: majiayu000 <1835304752@qq.com>
Co-authored-by: mcelrath
 2026-03-07 22:09:55 +08:00
Alex Brooks
10f4db4dbe [Frontend] Add Support for MM Encoder/Decoder Beam Search (Offline) (#36153) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-03-06 01:16:56 -08:00
wang.yuqi
ea463978bb [Frontend][1/n] Improve pooling entrypoints | classify. (#35604) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-03-03 06:05:36 -08:00
Wentao Ye
e113a30113 [Deprecation] Deprecate code in 0.17 as scheduled (#35441) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-02-28 17:32:37 +00:00
Ming Yang
6831650c40 [offloader] v2: Hide weight onloading latency via prefetching (#29941) 
Signed-off-by: Ming Yang <minos.future@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-02-25 17:20:59 -08:00
Nick Hill
dbf0da817a [Core] Cleanup engine pause/sleep logic (#34528) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-02-24 19:33:34 -08:00
Neil Schemenauer
54e2f83d0a [Feature] Lazy import for the "mistral" tokenizer module. (#34651) 
Signed-off-by: Neil Schemenauer <nas@arctrix.com>
 2026-02-23 00:43:01 -08:00
Kata Coder
5719a4e4e6 [Frontend] Support multimodal inputs for late-interaction scoring (ColQwen3) + NewModel: nvidia/nemotron-colembed (#34574) 
Signed-off-by: craftsangjae <craftsangjae@gmail.com>
 2026-02-20 20:01:40 -08:00
Cyrus Leung
ac900c89bb [Refactor] Implement output type check in LLM (#34794) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-19 19:57:55 -08:00
Cyrus Leung
61cf087680 [Bugfix] Fix lora tests (#34834) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-02-18 13:22:31 -08:00
Cyrus Leung
a766b30349 [Renderer] Deprecate code paths for old input processing (#34775) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-18 00:35:04 -08:00
Cyrus Leung
574fe75245 [Renderer] Move InputPreprocessor into Renderer (2/2) (#34560) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-17 05:29:01 -08:00
Christian Pinto
6930becd45 (bugfix): Fixed encode in LLM entrypoint for IOProcessr plugin prompts (#34618) 
Signed-off-by: Christian Pinto <christian.pinto@ibm.com>
 2026-02-16 07:33:55 -08:00
Cyrus Leung
73391a1baa [Renderer] Move InputPreprocessor into Renderer (1/2) (#34510) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-02-14 10:14:21 -08:00
Cyrus Leung
ec090c2429 [Refactor] Call renderer for online IO processor request (#34490) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-02-12 22:48:45 -08:00
Jaewon
4453ba8d9e [Core] Profiler improvements and lazy initialization (#33198) 
Signed-off-by: Jaewon Lee <jaewon@meta.com>
Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com>
 2026-02-12 16:16:38 -08:00
Jaewon
aa181c923b [Core] Add sleep level 0 mode with enqueue/wait pattern (#33195) 
Signed-off-by: Jaewon Lee <jaewon@meta.com>
Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com>
 2026-02-12 16:16:25 -08:00
Cyrus Leung
b96f7314b4 [Refactor] Pass Renderer to Input Processor (#34329) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-11 19:38:11 -08:00
Cyrus Leung
c9a1923bb4 [Plugin] Simplify IO Processor Plugin interface (#34236) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-10 19:47:39 -08:00
Cyrus Leung
edb359cce4 [Renderer] Define render_cmpl and render_chat (#34039) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-07 05:24:40 -08:00
Cyrus Leung
cd8b405bd0 [Refactor] Consolidate sequence normalization and enc-dec parsing (#33928) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-06 15:43:47 +00:00
SorenDreano
6e7b1c4b59 [Docs] Improve documentation (#33799) 
Co-authored-by: Soren Dreano <soren@numind.ai>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-02-06 12:57:09 +00:00
Aaron Hao
c1858b7ec8 [Feat][RL][1/2] Native Weight Syncing API: NCCL (#31943) 
Signed-off-by: ahao-anyscale <ahao@anyscale.com>
Signed-off-by: Aaron Hao <ahao@anyscale.com>
Co-authored-by: SumanthRH <sumanthrh99@gmail.com>
 2026-02-05 12:13:23 -05:00
Cyrus Leung
038914b7c8 [Refactor] Move task outside of PoolingParams.verify (#33796) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-02-05 09:33:11 +00:00
Ilya Boytsov
439afa4eea feat: Add ColBERT late interaction model support (#33686) 
Signed-off-by: Ilya Boytsov <ilyaboytsov1805@gmail.com>
Signed-off-by: Ilya Boytsov <boytsovpanamera@mail.ru>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-02-05 08:05:13 +08:00
Cyrus Leung
80f921ba4b [Bugfix] Fix normalize still being passed to PoolerConfig (#33794) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-04 23:56:02 +08:00
wang.yuqi
1b8fe6f7c4 [Frontend][4/n] Make pooling entrypoints request schema consensus | ScoreRequest (#33060) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-02-04 01:48:40 +00:00
Cyrus Leung
f0a1c8453a [Frontend] Use new Renderer for Completions and Tokenize API (#32863) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-31 04:51:15 -08:00
Cyrus Leung
11b556878b [Refactor] Use data parser for matching data items to multi-modal UUIDs (#32955) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-26 15:00:28 +08:00
Cyrus Leung
d117a4d1a9 [Frontend] Introduce Renderer for processing chat messages (using ModelConfig) (#30200) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-22 12:44:22 +00:00
wang.yuqi
4ae77dfd42 [Frontend][1/n] Make pooling entrypoints request schema consensus | CompletionRequest (#32395) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-01-16 06:17:04 +00:00
Hongxin Xu
49e6b86c91 [Feature] Support recording expert indices for rollout router replay (#28284) 
Signed-off-by: xhx1022 <1737006628@qq.com>
Signed-off-by: Hongxin Xu <70438206+xhx1022@users.noreply.github.com>
Signed-off-by: arlenxu <arlenxu@tencent.com>
Co-authored-by: 22quinn <33176974+22quinn@users.noreply.github.com>
Co-authored-by: arlenxu <arlenxu@tencent.com>
 2026-01-12 06:23:04 -08:00
Cyrus Leung
583a90e005 [Refactor] Separate sequence and token pooling types (#32026) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-10 04:53:24 +00:00
Michael Goin
d5ec6c056f [UX] Add vLLM model inspection view (#29450) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-01-09 10:12:35 -07:00
Michael Goin
bc5ef333e0 [Perf] Add skip_clone to SamplingParams for internal request handling (#31041) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-12-24 14:35:57 -08:00
Mark McLoughlin
f790068600 [Core] Add a random suffix to frontend-provided request IDs (#27987) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-12-23 13:05:39 -08:00
Jakub Zakrzewski
23daef548d [Frontend] Support using chat template as custom score template for reranking models (#30550) 
Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com>
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
 2025-12-23 11:19:16 +00:00
Matthew Bonanni
a182be4308 [UX][Attention] Add attention_config argument to LLM() (#30710) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-12-15 17:29:09 -05:00
Cyrus Leung
64251f48df [Chore] Adjust tokenizer import to avoid circular imports (#30601) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-12-13 04:42:39 -08:00
Cyrus Leung
7e24e5d4d6 [Deprecation] Remove deprecated task, seed and MM settings (#30397) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-12-10 19:59:39 -08:00
Cyrus Leung
e72d65b959 {Deprecation] Remove tokenizer setter (#30400) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-12-10 19:10:58 +00:00
Benjamin Chislett
e858bfe051 [Cleanup] Refactor profiling env vars into a CLI config (#29912) 
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-12-09 13:29:33 -05:00
Cyrus Leung
e83b7e379c Revert "[Renderer] Separate out RendererConfig from ModelConfig (#30145)" (#30199) 2025-12-07 00:00:22 -08:00
Cyrus Leung
27f4c2fd46 [Renderer] Separate out RendererConfig from ModelConfig (#30145) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-12-06 23:15:42 -08:00