biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
12,117 Commits 2 Branches 140 Tags
d017bceb08eaac7bae2c499124ece737fb4fb22b
Commit Graph

435 Commits

Author SHA1 Message Date
Christina Norman
e41312a2f5 [Bugfix] Skip generation config fallback for GGUF to prevent multi-process hang (#30209) 
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
 2025-12-09 01:52:43 +00:00
shaharmor98
fcd5306f65 Add latent MoE support (#30203) 
Signed-off-by: Shahar Mor <smor@nvidia.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-12-08 17:35:01 +00:00
Cyrus Leung
e83b7e379c Revert "[Renderer] Separate out RendererConfig from ModelConfig (#30145)" (#30199) 2025-12-07 00:00:22 -08:00
Cyrus Leung
27f4c2fd46 [Renderer] Separate out RendererConfig from ModelConfig (#30145) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-12-06 23:15:42 -08:00
Luke
a49d813fa8 Lazy loading to avoid importing all files (#29716) 
Signed-off-by: Luke <yq0536@gmail.com>
 2025-12-07 07:13:14 +00:00
Ye (Charlotte) Qi
92c35abb24 [Misc] Fix circular import in vllm.transformers_utils.config (#30179) 
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
 2025-12-06 09:24:03 +00:00
Peter Salas
e858bc4d14 [Model] Add support for transformer-based Ultravox v0.7 projector (#30089) 
Signed-off-by: Peter Salas <peter@fixie.ai>
 2025-12-05 20:55:43 -08:00
Harry Mellor
e10c84e06a Access partial_rotary_factor from rope_parameters (#29966) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-12-04 18:42:49 +00:00
Harry Mellor
5c32a06a04 Use Transformers v5 RoPE standardisation and validation (#30046) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-12-04 14:54:28 +00:00
Noa Neria
6366c098d7 Validating Runai Model Streamer Integration with S3 Object Storage (#29320) 
Signed-off-by: Noa Neria <noa@run.ai>
 2025-12-04 18:04:43 +08:00
JackieWu
0bec63fa31 [BugFix] fix imgs_pos in hunyuan_vl (#29879) 
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-12-03 06:20:37 +00:00
Harry Mellor
6fc5841db1 Fix some more Transformers nightly tests (#29872) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-12-02 21:49:44 +00:00
Isotr0py
63b1da76ba [Chore]: Reorganize gguf utils funtions under transformers_utils (#29891) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-12-02 17:33:23 +00:00
Cyrus Leung
68ffbca7e4 [Chore] Use tokenizer.encode and tokenizer.decode directly (#29851) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-12-02 12:30:40 +00:00
Julien Denize
d8c6210eea Add Mistral Large 3 and Ministral 3 (#29757) 
Signed-off-by: Julien Denize <julien.denize@mistral.ai>
Signed-off-by: Julien Denize <40604584+juliendenize@users.noreply.github.com>
Signed-off-by: Mickael Seznec <mickael@mistral.ai>
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Mickael Seznec <mickael@mistral.ai>
 2025-12-02 10:29:00 +00:00
Cyrus Leung
653591d5e7 [Chore] Move tokenizer initialization methods (#29793) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-12-02 13:33:37 +08:00
Cyrus Leung
f0a28bf661 [Misc] Unify tokenizer registration (#29767) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-12-01 11:34:58 +00:00
Harry Mellor
cd719de5cb Fix RoPE failures in Transformers nightly (#29700) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-11-30 14:29:32 +00:00
Cyrus Leung
2afcec4dec [Misc] Update TokenizerLike interface and move get_cached_tokenizer (#29730) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-11-30 14:59:47 +08:00
Cyrus Leung
fa59fe417f [Chore] Move detokenizer_utils to vllm/tokenizers (#29727) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-11-29 06:25:17 -08:00
Cyrus Leung
34a984274e [Misc] Refactor tokenizer interface (#29693) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-11-29 04:02:21 -08:00
Didier Durand
04a797cd0e [Doc]: fixing typos in various files. (#29717) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
 2025-11-29 01:15:39 -08:00
Harry Mellor
fecae12cd7 Remove all_special_tokens_extended from tokenizer code (#29686) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-11-28 20:26:51 +00:00
Isotr0py
f946a8d743 [Chore]: Reorganize model repo operating functions in transformers_utils (#29680) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-11-28 08:46:51 -08:00
Cyrus Leung
9e6bcda3ac [mypy] Enable type checking for more directories (#29674) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-11-28 08:39:27 -08:00
Julien Denize
b2c1d294fa [BUGFIX] MistralTokenizer._call__ adds an invalid EOS token (#29607) 
Signed-off-by: Julien Denize <julien.denize@mistral.ai>
Signed-off-by: Julien Denize <40604584+juliendenize@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-11-28 16:44:47 +08:00
Injae Ryou
0840abdd24 [BugFix] Optional tokenizer argument when loading GGUF models (#29582) 
Signed-off-by: Injae Ryou <injaeryou@gmail.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-11-27 16:53:10 +00:00
Roger Wang
cf348c8d27 [Bugfix] Fix HunyuanVL XD-RoPE (#29593) 
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored by: grider-transwithai <grider@transwith.ai>
 2025-11-27 12:36:24 +00:00
Harry Mellor
0353d2e162 Fix RoPE related failures in Transformers nightly tests (#29333) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-11-25 16:23:45 +00:00
Injae Ryou
794029f012 [Feature]: Improve GGUF loading from HuggingFace user experience like repo_id:quant_type (#29137) 
Signed-off-by: Injae Ryou <injaeryou@gmail.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-11-25 14:28:53 +00:00
Isotr0py
92effb07a4 [Model] Add HunyuanOCR support (#29327) 
Signed-off-by: manayang <jackmanayang@gmail.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: sergeywang <sergeywang@tencent.com>
Co-authored-by: manayang <jackmanayang@gmail.com>
Co-authored-by: manayang <manayang@tencent.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2025-11-25 03:28:51 +00:00
Luke
6fb0215eee [Bugfix] Use lazy string reference for DeepseekV3Config in config registry (#28958) 
Signed-off-by: Luke <yq0536@gmail.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-11-23 11:43:21 +00:00
Cyrus Leung
d1cf8214e5 [Bugfix] Use HF config fields as fallback when loading Mistral config (#29239) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-11-22 11:22:48 -07:00
Julien Denize
57430fc95c Default model load/config/tokenizer to mistral format if relevant files exist (#28659) 
Signed-off-by: Julien Denize <julien.denize@mistral.ai>
Signed-off-by: Julien Denize <40604584+juliendenize@users.noreply.github.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-11-21 13:58:59 -08:00
Julien Denize
434f3d3eb8 Fix mistral config (#29172) 
Signed-off-by: Julien Denize <julien.denize@mistral.ai>
Signed-off-by: Julien Denize <40604584+juliendenize@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2025-11-21 14:01:20 +00:00
Isotr0py
64192d5624 [Bugfix] Revert custom attention mask for gemma3-mm (#28995) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-11-20 13:23:22 +08:00
Harry Mellor
a8b70304d6 Update rope_scaling to rope_parameters in preparation for Transformers v5 (#28542) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-11-19 09:06:36 -08:00
Luciano Martins
c2612371ad [Model] Add Gemma3 GGUF multimodal support (#27772) 
Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-11-18 08:56:29 -08:00
Pranav
f77bce001a [Model] Add Afmoe architecture implementation (#28332) 
Signed-off-by: Maziyar Panahi <maziyar.panahi@iscpif.fr>
Signed-off-by: Pranav <veldurthipranav@gmail.com>
Co-authored-by: Maziyar Panahi <maziyar.panahi@iscpif.fr>
 2025-11-17 15:11:20 -08:00
Yong Hoon Shin
11ac9ddd03 Support all interleaved layer types (#28485) 
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
 2025-11-13 08:57:20 +00:00
Harry Mellor
d9ab1ad9d1 reasoning_content -> reasoning (#27752) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-11-08 12:15:08 +00:00
Junhong Liu
59b453eaa2 Speed up mm processor kwargs per request by spliting dynamic and static kwargs (#26483) 
Signed-off-by: Junhong <liujunhong11@huawei.com>
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com>
Co-authored-by: Junhong <liujunhong11@huawei.com>
 2025-11-07 07:51:28 +08:00
Julien Denize
7a8375f8a0 Add llama 4 scaling support (#28145) 
Signed-off-by: Julien Denize <julien.denize@mistral.ai>
 2025-11-06 18:55:17 +00:00
Julien Denize
a404e2c0f1 Patch Mistral Tokenizer (#28146) 
Signed-off-by: Julien Denize <julien.denize@mistral.ai>
 2025-11-06 06:43:16 +00:00
Isotr0py
43ecd0a900 [Chore] Clean up deepseek v2/v3 config copy (#28055) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-11-06 03:46:30 +00:00
Isotr0py
ffb08379d8 [Chore] Remove Nemotron-Nano-VL config copy (#28126) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-11-05 20:06:45 +00:00
Isotr0py
3f5a4b6473 [Bugfix] Validate custom logits processor xargs for online serving (#27560) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-11-05 16:53:33 +00:00
pwschuurman
f7d2946e99 [Bugfix] Skip gs:// model paths for speculator detection (#27846) 
Signed-off-by: Peter Schuurman <psch@google.com>
 2025-11-03 14:31:03 +00:00
Cyrus Leung
6c317a656e [Misc] Provide Siglip2 chat template (#27939) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-11-02 13:42:38 +00:00
Julien Denize
73444b7b56 Performance fix MistralTokenizer: cache special ids and tokens (#27925) 
Signed-off-by: Julien Denize <julien.denize@mistral.ai>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2025-11-02 08:48:33 +00:00