biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
9,354 Commits 2 Branches 140 Tags
25bb9e8c65424e3bf24d2eab259743f9a97b7a3c
Commit Graph

76 Commits

Author SHA1 Message Date
wang.yuqi
25bb9e8c65 [CI Failure] fix models/language/pooling/test_auto_prefix_cache_support.py (#24636) 
Signed-off-by: wang.yuqi <noooop@126.com>
 2025-09-11 03:31:23 -07:00
Tao He
e93f4cc9e3 Add the support for the qwen3 next model (a hybrid attention model). (#24526) 
Signed-off-by: Tao He <linzhu.ht@alibaba-inc.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-11 15:32:09 +08:00
Didier Durand
e2b1f863aa [Doc]: fixing doc typos (#24635) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
 2025-09-10 23:19:28 -07:00
shengshiqi-google
41329a0ff9 [Core] feat: Add --safetensors-load-strategy flag for faster safetensors loading from Lustre (#24469) 
Signed-off-by: Shiqi Sheng <shengshiqi@google.com>
Signed-off-by: shengshiqi-google <160179165+shengshiqi-google@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-10 23:10:01 -07:00
Peter Salas
f17a6aa4ec [Ultravox] Fix Gemma instantiation, support quantization via --hf-overrides (#24131) 
Signed-off-by: Peter Salas <peter@fixie.ai>
 2025-09-10 22:25:34 -07:00
Russell Bryant
37e8182bfe [v1] Add Whisper model support (encoder-decoder) (#21088) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: NickLucche <nlucches@redhat.com>
 2025-09-10 13:53:35 -07:00
Xingyu Liu
9fb74c27a7 [Core] Support configuration parsing plugin (#24277) 
Signed-off-by: Xingyu Liu <charlotteliu12x@gmail.com>
Signed-off-by: Xingyu Liu <38244988+charlotte12l@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-10 11:32:43 -07:00
Harry Mellor
f36355abfd Move LoadConfig from config/__init__.py to config/load.py (#24566) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-10 06:14:18 -07:00
danielafrimi
72d30108a0 Support for NemotronH Nano VLM (#23644) 
Signed-off-by: Daniel Afrimi <danielafrimi8@gmail.com>
 2025-09-10 06:10:06 -07:00
Remy
feaf202e93 [Bugfix] Guard _may_reorder_batch for encoder-only models on CPU (#24319) (#24348) 
Signed-off-by: Remy <eunhwan.shin@dtonic.io>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2025-09-10 14:24:42 +08:00
pwschuurman
4377b1ae3b [Bugfix] Update Run:AI Model Streamer Loading Integration (#23845) 
Signed-off-by: Omer Dayan (SW-GPU) <omer@run.ai>
Signed-off-by: Peter Schuurman <psch@google.com>
Co-authored-by: Omer Dayan (SW-GPU) <omer@run.ai>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-09-09 21:37:17 -07:00
Wenlong Wang
53b42f4102 [BugFix][Spec Decode] Fix out-of-range index triggered by eagle3; re-enable test for LlamaForCausalLMEagle3 (#24392) 
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
 2025-09-09 21:24:23 -07:00
Tyler Michael Smith
561f38dc3c [Bugfix] Improve EPLB config validation error message (#24524) 
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-09-10 00:32:36 +00:00
wang.yuqi
19332c0479 [Model] Systematic support for fp32 head, pooling models part (#23810) 
Signed-off-by: wang.yuqi <noooop@126.com>
 2025-09-09 07:29:50 -07:00
Wentao Ye
a55cf41a09 [Compilation][WideEP] Enable Piecewise CUDAGraph for DeepEPHT (#24123) 2025-09-09 10:21:10 -04:00
cong-meta
b2f7745774 Add data_parallel_size to VllmConfig string representation (#24298) 
Co-authored-by: Cong Chen <congc@meta.com>
 2025-09-08 21:35:18 -07:00
Zebing Lin
82dfb12e52 [Core] Use sha256 bytes instead of BlockHash to reduce GC overhead (#23673) 
Signed-off-by: linzebing <linzebing1995@gmail.com>
 2025-09-08 21:34:37 -07:00
Harry Mellor
3e0d4a3475 Move KVTransferConfig from config/__init__.py to config/kv_transfer.py (#24434) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-08 20:30:32 -07:00
Harry Mellor
03dd652c16 Move KVEventsConfig from config/__init__.py to config/kv_events.py (#24433) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-08 06:41:27 -07:00
Didier Durand
f4962a6d55 [Doc]: fix typos in Python comments (#24417) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
 2025-09-08 00:22:16 -07:00
Woosuk Kwon
4172235ab7 [V0 deprecation] Deprecate V0 Neuron backend (#21159) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-09-06 16:15:18 -07:00
Ashwin Phadke
6024d115cd Lora bias(enable_lora_bias) deprecate warning (#24339) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-07 00:42:19 +08:00
wang.yuqi
6d6c6b05d3 [New Model]: google/embeddinggemma-300m (#24318) 
Signed-off-by: wang.yuqi <noooop@126.com>
 2025-09-05 22:58:36 -07:00
yzds
ac201a0eaf [Feature] Support Decode Context Parallel (DCP) for MLA (#23734) 
Signed-off-by: hongchao <hongchao@msh.team>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: hongchao <hongchao@msh.team>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2025-09-06 13:24:05 +08:00
Yong Hoon Shin
3c529fc994 [KV Sharing] Raise error if using eagle with fast prefill (#24350) 
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
 2025-09-05 20:22:40 -07:00
nopperl
2b30afa442 Use hidden_size_per_head as head_size fallback (#24221) 
Signed-off-by: nopperl <54780682+nopperl@users.noreply.github.com>
 2025-09-04 12:59:16 +01:00
mgazz
51d5e9be7d [Core][Model] Terratorch backend integration (#23513) 
Signed-off-by: Michele Gazzetti <michele.gazzetti1@ibm.com>
Signed-off-by: Christian Pinto <christian.pinto@ibm.com>
Co-authored-by: Christian Pinto <christian.pinto@ibm.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-09-04 00:22:41 -07:00
Peter Pan
b5ee1e3261 Remove deprecated PyNcclConnector (#24151) 
Signed-off-by: Peter Pan <Peter.Pan@daocloud.io>
 2025-09-03 22:49:16 +00:00
nopperl
fa4311d85f [V1] v1 engine + full CUDA graph support for PLaMo2 (#23998) 
Signed-off-by: Hemmi Shinichi <shemmi@preferred.jp>
Signed-off-by: nopperl <54780682+nopperl@users.noreply.github.com>
Co-authored-by: Hemmi Shinichi <shemmi@preferred.jp>
Co-authored-by: Thomas Parnell <tom.parnell@gmail.com>
 2025-09-03 08:24:02 -07:00
Jiangyun Zhu
e81d4e69c1 [Misc] Add check for dual_chunk_attention (#24070) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2025-09-03 04:19:14 +00:00
co63oc
1bd007f234 fix some typos (#24071) 
Signed-off-by: co63oc <co63oc@users.noreply.github.com>
 2025-09-02 20:44:50 -07:00
wang.yuqi
e0653f6c0b [Model] Classification models support logit_bias / sigmoid_normalize (#24031) 
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-02 16:48:57 +00:00
wang.yuqi
55602bb2e6 [Frontend] Update the warning log when using VLLM_ALLOW_LONG_MAX_MODEL_LEN (#20904) 
Signed-off-by: wang.yuqi <noooop@126.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-01 08:50:25 +00:00
Christian Pinto
1cb39dbcdd [Misc] IO Processor plugins for pooling models (#22820) 
Signed-off-by: Christian Pinto <christian.pinto@ibm.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Max de Bayser <mbayser@br.ibm.com>
 2025-08-31 23:07:12 -07:00
Yong Hoon Shin
cb293f6a79 [V1] Enable prefill optimization for Gemma3n (#22628) 
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
 2025-08-28 14:54:30 -07:00
Didier Durand
d3da2eea54 [Doc]: fix typos in Python scripts (#23828) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
 2025-08-28 05:37:38 -07:00
Jinheng
c8851a4723 Add deprecation warning for lora_extra_vocab_size (#23635) 
Signed-off-by: Jinheng Li <ahengljh@gmail.com>
 2025-08-27 22:34:29 -07:00
Thomas Parnell
dd58932280 [V1] [Hybrid] Enable compile and piecewise CUDA graph for MiniMax-Text models (#22589) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
 2025-08-27 10:05:16 -07:00
Cyrus Leung
52883ed084 [Model] Merge SupportsMultiModalWithRawInput with SupportsMultiModal (#23749) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-08-27 10:01:50 -07:00
Cyrus Leung
fe8d7b6f03 [Model] Interface to enable batch-level DP support (#23733) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-08-27 06:41:22 -07:00
Cyrus Leung
69244e67e6 [Core] Use key-only cache for BaseMultiModalProcessor (#23018) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-08-27 14:19:13 +08:00
Harry Mellor
b00e69f8ca Fix nits from #20059 (#23548) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-08-26 03:27:20 -07:00
Noam Gat
39971db3aa Frontend: Adding LM Format Enforcer support to V1 engine (#22564) 
Signed-off-by: Noam Gat <noamgat@gmail.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-08-24 19:31:22 -07:00
Didier Durand
47455c424f [Doc: ]fix various typos in multiple files (#23487) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-08-25 00:04:04 +00:00
rongfu.leng
1b9b16649c [Misc] update dict parse to EPLBConfig from json dumps to dict unpacking (#23305) 
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
 2025-08-24 08:06:34 +00:00
Daifeng Li
fa78de9dc3 Quantization: support FP4 quantized models on AMD CDNA2/CDNA3 GPUs (#22527) 
Signed-off-by: feng <fengli1702@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-08-22 20:53:21 -06:00
22quinn
480bdf5a7b [Core] Support custom executor qualname (#23314) 
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
 2025-08-22 09:40:54 +08:00
Ming Yang
10f535c086 [Bugfix] Fix port conflict by obtaining a list of open ports upfront (#21894) 
Signed-off-by: Ming Yang <minos.future@gmail.com>
 2025-08-21 10:22:18 -07:00
Roger Wang
79f05e4436 [Multimodal] Always enable hashing mm data (#23308) 
Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-08-21 07:23:28 -07:00
Paul Pak
2e2000f352 [Model] Add LFM2 architecture (#22845) 
Signed-off-by: Paul Pak <paulpak58@gmail.com>
 2025-08-21 09:35:07 +02:00