biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
9,748 Commits 2 Branches 140 Tags
45d7d852d362b055b7c02eb967c4ea00d6f9e130
Commit Graph

2752 Commits

Author SHA1 Message Date
Bowen Wang
06a41334c7 [EPLB] Reduce EPLB Inference Overhead (#24573) 
Signed-off-by: Bowen Wang <abmfy@icloud.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-09-22 16:31:05 +00:00
Sara-KS
ac243886b0 [Kernel] MI-300X triton moe configs (#23445) 
Signed-off-by: Sara Kokkila Schumacher <saraks@ibm.com>
 2025-09-22 14:29:54 +00:00
Nicolò Lucchesi
4cf71cc88a [TPU] Deprecate xm.mark_step in favor of `torch_xla.sync (#25254) 
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Ye (Charlotte) Qi <yeq@meta.com>
 2025-09-22 10:12:57 +00:00
Eldar Kurtić
21467f9a1c Enable Eagle3 speculative decoding for GPT-OSS model (#25246) 
Signed-off-by: Eldar Kurtic <8884008+eldarkurtic@users.noreply.github.com>
 2025-09-22 08:50:39 +00:00
Roger Wang
7b57a433da [Model] Support Dots OCR (#24645) 
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: yinz-aizip <yinz@aizip.ai>
 2025-09-22 02:24:40 +00:00
Woosuk Kwon
bc6e542d9f Remove V0 attention backends (#25351) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-09-21 16:03:28 -07:00
Isotr0py
af7dfb0d1a [Perf] Further optimization for Qwen3-VL fast_pos_embed_interpolate (#25347) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-09-21 20:12:45 +00:00
Woosuk Kwon
1c3ffdbecc [V0 Deprecation] Remove V0 sampling metadata (#25345) 
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
 2025-09-21 10:37:11 -07:00
Woosuk Kwon
26e673fe93 [V0 Deprecation] Remove V0 Sequence class & Sampler (#25332) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
 2025-09-21 08:52:15 -07:00
Roger Wang
30d08911f7 [MM][Perf] Minor Optimization on Qwen3-VL fast_pos_embed_interpolate (#25337) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2025-09-21 11:05:20 +00:00
Wenlong Wang
035fd2bd2c [Multi Modal][Performance] Fused Q,K's apply_rope in more models (#25005) 
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2025-09-21 03:55:10 +00:00
Woosuk Kwon
572ddf83ce [Chore] Remove unused sampler in models (#25324) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-09-20 19:53:20 -07:00
Woosuk Kwon
52c2a8d4ad [V0 Deprecation] Remove LLMEngine (#25033) 
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-09-20 17:56:30 -07:00
Isotr0py
3c713a9711 [Model] Cleanup InternViT's data parallel implementation (#25306) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-09-20 05:46:24 -07:00
Manoel Marques
bf8b26cad1 Generate _ModelInfo properties file when loading to improve loading speed (#23558) 
Signed-off-by: Manoel Marques <manoel.marques@ibm.com>
Signed-off-by: Manoel Marques <manoelmrqs@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2025-09-20 11:51:13 +00:00
Michael Goin
e08a3a3fdb [CI Failure] Disable FlashInfer RoPE to unblock CI (#25299) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-09-20 08:16:56 +00:00
Roger Wang
be874c0201 [Bugfix] Fix Qwen3-VL-MoE weight loading for EP (#25300) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2025-09-20 00:04:05 -07:00
Cyrus Leung
c60e6137f0 [Optimization] Avoid repeated model architecture conversion for pooling models (#25261) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-09-20 13:30:22 +08:00
JartX
3642909617 [BUGFIX] GPTQ quantization compatibility for Qwen3 Next MOE models (AutoGPTQ and AutoRound-GPTQ) (#25268) 
Signed-off-by: JartX <sagformas@epdcenter.es>
 2025-09-20 11:18:13 +08:00
Harry Mellor
c308501cb6 Improve weight loading for encoder models in Transformers backend (#25289) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-20 03:11:03 +00:00
Zhiyu
431535b522 Enable modelopt gemma3 nvfp4/fp8, make workflow more robust (#22771) 
Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-09-19 22:40:33 +00:00
David-Wen
ddc9048394 Fix: Correct FusedMoE layer reference in auto_round quantization (#24818) 
Signed-off-by: David-Wen <18927700430@163.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-19 20:44:24 +00:00
Michael Goin
48ecb4438b [Perf] Use FlashInfer RoPE for RotaryEmbedding.forward_cuda when available (#21126) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2025-09-19 14:06:49 -06:00
bnellnm
4bdf400218 [Bugfix] Fix chunked a2_scales in modular kernels (#25264) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2025-09-19 19:42:01 +00:00
Varun Sundar Rabindranath
7852b82b93 [Bugfix] GPT OSS Attritbute error on H100 (#25228) 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
 2025-09-19 13:14:09 -06:00
samzong
138f0d1e75 [Docs] add __init__.py to vllm/model_executor/layers/quantization/compressed_tensors/transform (#24974) 
Signed-off-by: samzong <samzong.lu@gmail.com>
 2025-09-19 18:32:27 +00:00
Harry Mellor
12aed7e453 Encoder model support for the Transformers backend (#25174) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-19 19:15:22 +01:00
LJH-LBJ
d90e212a3a Remove Redundant Assignment in Qwen3_VisionPatchMerger (#25224) 
Signed-off-by: Junhong <liujunhong11@huawei.com>
Co-authored-by: Junhong <liujunhong11@huawei.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2025-09-19 12:15:13 -06:00
Harry Mellor
aed16879a9 Move ModelConfig from config/__init__.py to config/model.py (#25252) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-19 16:22:33 +00:00
Icey
838d7116ba [Qwen] Remove cuda hard-code in qwen3 next (#25243) 
Signed-off-by: Icey <1790571317@qq.com>
 2025-09-19 12:25:12 +00:00
Cyrus Leung
5089fd749c [V0 Deprecation] Remove V0 logic from get_input_embeddings interface (#25242) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-09-19 11:10:52 +00:00
Roger Wang
1dfea5f4a9 [Bugfix][Perf] Misc fixes for Qwen3 VL (#25238) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2025-09-19 10:46:16 +00:00
Isotr0py
cea91a32f2 [Kernel][Performance] Add Triton kernel for Qwen3-VL interleaved MRoPE (#25055) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-09-19 10:27:49 +00:00
Li, Jiang
825fdb11ad [Bugfix][CPU] Add placeholder to avoid import errors when using fused_moe ops on platforms without triton (#25137) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-09-19 07:41:12 +00:00
Li, Jiang
8c1d4acbfe [CPU] Disable oneDNN linear on non-x86 platforms (#25166) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-09-19 07:27:22 +00:00
Chendi.Xue
a6149aa587 [OOT] Support sync_model_loading for OOT (#25126) 
Signed-off-by: Chendi Xue <Chendi.Xue@intel.com>
 2025-09-19 05:41:53 +00:00
Chen Ding
1a0a04dae9 [Perf] Optimize memory peak during EAGLE model loading. (#24585) 
Signed-off-by: Chen Ding <candy.dc@alibaba-inc.com>
 2025-09-19 03:31:16 +00:00
Lucas Wilkinson
9fac6aa30b [BugFix] Fix DeepGEMM warmup, no m.weight_scale_inv (#25206) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-09-18 14:26:28 -07:00
Aziz
38db529f66 [feat]: Create interface for model-specific M-RoPE (#24194) 
Signed-off-by: AzizCode92 <azizbenothman76@gmail.com>
Signed-off-by: Aziz <azizbenothman76@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-09-18 19:18:56 +00:00
Nikhil Gupta
064cac7bb7 [fix]: remove data type hardcoding from gptoss model implementation (#23807) 
Signed-off-by: Nikhil Gupta <nikhil.gupta2@arm.com>
 2025-09-18 18:15:23 +00:00
qizixi
c4cb0af98a [spec decode] Fix MTP inference path for MiMo-7B model (#25136) 
Signed-off-by: zixi-qi <qizixi@meta.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-09-18 09:12:19 -07:00
wang.yuqi
5f696c33b1 [New Model] Support BertForTokenClassification / Named Entity Recognition (NER) task (#24872) 
Signed-off-by: wang.yuqi <noooop@126.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-09-18 23:22:01 +08:00
Vadim Gimpelson
072d7e53e5 [PERF] Add conv1d metadata to GDN attn (#25105) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
 2025-09-18 14:27:49 +00:00
Michael Goin
fbd6523ac0 Refactor dense FP8 tensor/channel/block utils and add CT FP8 block (#21404) 2025-09-18 08:53:45 -04:00
Asaf Joseph Gardin
66072b36db [Bugfix][Mamba] - Fix Conv State Kernel FP32 Support (#24883) 
Signed-off-by: asafg <39553475+Josephasafg@users.noreply.github.com>
 2025-09-18 12:21:17 +00:00
Jee Jee Li
37970105fe [Model] Improve Pooling Model (#25149) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-18 11:04:21 +00:00
Aaron Pham
29283e8976 [Chore] Cleanup guided namespace, move to structured outputs config (#22772) 
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-18 09:20:27 +00:00
Punitvara
05b044e698 [Doc] Fix cross-reference warnings (#25058) 
Signed-off-by: Punit Vara <punitvara@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-18 02:05:16 -07:00
Tao He
ef7eefe17a [Qwen] Add fp8 checkpoint support for qwen3-next. (#25079) 
Signed-off-by: Tao He <linzhu.ht@alibaba-inc.com>
 2025-09-18 08:16:04 +00:00
Harry Mellor
f4cd80f944 Retrieve sliding_window from text config in Gemma3 MM (#25085) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-18 06:29:05 +00:00