biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
9,670 Commits 2 Branches 140 Tags
48ecb4438b2845f757edf228c5455ca6095938af
Commit Graph

2730 Commits

Author SHA1 Message Date
Michael Goin
48ecb4438b [Perf] Use FlashInfer RoPE for RotaryEmbedding.forward_cuda when available (#21126) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2025-09-19 14:06:49 -06:00
bnellnm
4bdf400218 [Bugfix] Fix chunked a2_scales in modular kernels (#25264) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2025-09-19 19:42:01 +00:00
Varun Sundar Rabindranath
7852b82b93 [Bugfix] GPT OSS Attritbute error on H100 (#25228) 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
 2025-09-19 13:14:09 -06:00
samzong
138f0d1e75 [Docs] add __init__.py to vllm/model_executor/layers/quantization/compressed_tensors/transform (#24974) 
Signed-off-by: samzong <samzong.lu@gmail.com>
 2025-09-19 18:32:27 +00:00
Harry Mellor
12aed7e453 Encoder model support for the Transformers backend (#25174) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-19 19:15:22 +01:00
LJH-LBJ
d90e212a3a Remove Redundant Assignment in Qwen3_VisionPatchMerger (#25224) 
Signed-off-by: Junhong <liujunhong11@huawei.com>
Co-authored-by: Junhong <liujunhong11@huawei.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2025-09-19 12:15:13 -06:00
Harry Mellor
aed16879a9 Move ModelConfig from config/__init__.py to config/model.py (#25252) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-19 16:22:33 +00:00
Icey
838d7116ba [Qwen] Remove cuda hard-code in qwen3 next (#25243) 
Signed-off-by: Icey <1790571317@qq.com>
 2025-09-19 12:25:12 +00:00
Cyrus Leung
5089fd749c [V0 Deprecation] Remove V0 logic from get_input_embeddings interface (#25242) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-09-19 11:10:52 +00:00
Roger Wang
1dfea5f4a9 [Bugfix][Perf] Misc fixes for Qwen3 VL (#25238) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2025-09-19 10:46:16 +00:00
Isotr0py
cea91a32f2 [Kernel][Performance] Add Triton kernel for Qwen3-VL interleaved MRoPE (#25055) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-09-19 10:27:49 +00:00
Li, Jiang
825fdb11ad [Bugfix][CPU] Add placeholder to avoid import errors when using fused_moe ops on platforms without triton (#25137) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-09-19 07:41:12 +00:00
Li, Jiang
8c1d4acbfe [CPU] Disable oneDNN linear on non-x86 platforms (#25166) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-09-19 07:27:22 +00:00
Chendi.Xue
a6149aa587 [OOT] Support sync_model_loading for OOT (#25126) 
Signed-off-by: Chendi Xue <Chendi.Xue@intel.com>
 2025-09-19 05:41:53 +00:00
Chen Ding
1a0a04dae9 [Perf] Optimize memory peak during EAGLE model loading. (#24585) 
Signed-off-by: Chen Ding <candy.dc@alibaba-inc.com>
 2025-09-19 03:31:16 +00:00
Lucas Wilkinson
9fac6aa30b [BugFix] Fix DeepGEMM warmup, no m.weight_scale_inv (#25206) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-09-18 14:26:28 -07:00
Aziz
38db529f66 [feat]: Create interface for model-specific M-RoPE (#24194) 
Signed-off-by: AzizCode92 <azizbenothman76@gmail.com>
Signed-off-by: Aziz <azizbenothman76@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-09-18 19:18:56 +00:00
Nikhil Gupta
064cac7bb7 [fix]: remove data type hardcoding from gptoss model implementation (#23807) 
Signed-off-by: Nikhil Gupta <nikhil.gupta2@arm.com>
 2025-09-18 18:15:23 +00:00
qizixi
c4cb0af98a [spec decode] Fix MTP inference path for MiMo-7B model (#25136) 
Signed-off-by: zixi-qi <qizixi@meta.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-09-18 09:12:19 -07:00
wang.yuqi
5f696c33b1 [New Model] Support BertForTokenClassification / Named Entity Recognition (NER) task (#24872) 
Signed-off-by: wang.yuqi <noooop@126.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-09-18 23:22:01 +08:00
Vadim Gimpelson
072d7e53e5 [PERF] Add conv1d metadata to GDN attn (#25105) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
 2025-09-18 14:27:49 +00:00
Michael Goin
fbd6523ac0 Refactor dense FP8 tensor/channel/block utils and add CT FP8 block (#21404) 2025-09-18 08:53:45 -04:00
Asaf Joseph Gardin
66072b36db [Bugfix][Mamba] - Fix Conv State Kernel FP32 Support (#24883) 
Signed-off-by: asafg <39553475+Josephasafg@users.noreply.github.com>
 2025-09-18 12:21:17 +00:00
Jee Jee Li
37970105fe [Model] Improve Pooling Model (#25149) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-18 11:04:21 +00:00
Aaron Pham
29283e8976 [Chore] Cleanup guided namespace, move to structured outputs config (#22772) 
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-18 09:20:27 +00:00
Punitvara
05b044e698 [Doc] Fix cross-reference warnings (#25058) 
Signed-off-by: Punit Vara <punitvara@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-18 02:05:16 -07:00
Tao He
ef7eefe17a [Qwen] Add fp8 checkpoint support for qwen3-next. (#25079) 
Signed-off-by: Tao He <linzhu.ht@alibaba-inc.com>
 2025-09-18 08:16:04 +00:00
Harry Mellor
f4cd80f944 Retrieve sliding_window from text config in Gemma3 MM (#25085) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-18 06:29:05 +00:00
YiwenC
9d8a2d86d2 [EPLB] Add EPLB support for hunyuan_v1 (#23078) 2025-09-18 04:51:35 +00:00
YiwenC
52bc9d5b3e [Model] enable data parallel for InternVL vision encoder (#23909) 
Signed-off-by: Yiwen Chen <yiwen66@berkeley.edu>
Signed-off-by: YiwenC <54658925+666even666@users.noreply.github.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2025-09-17 21:11:46 -07:00
bnellnm
dc2979c585 [Kernels] Overlap shared experts with combine instead of dispatch (#24254) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2025-09-18 12:10:21 +08:00
toncao
027d37df38 [Bugfix][Qwen3-Next] add prefixes to shared_expert in qwen3-next and mlp in qwen2moe to successfully load ignored params in quantized models (#24960) 
Signed-off-by: toncao <cpatonn@gmail.com>
Co-authored-by: toncao <cpatonn@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-18 12:08:50 +08:00
Roger Wang
3127274d02 [MM Encoder] Apply DP ViT for Qwen3-VL model series (#24955) 
Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Huang Jie <92386084+JJJYmmm@users.noreply.github.com>
Co-authored-by: 松灵 <26085463+wulipc@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-09-17 21:04:21 -07:00
bnellnm
5963b98b46 [Kernel] Delegate construction of FusedMoEQuantConfig to FusedMoEMethodBase subclasses (#22537) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2025-09-17 17:43:31 -06:00
Tao He
dd6a910aac [Bugfix][Qwen3-Next] fixes the varlen issue in qwen3-next's MTP implementation. (#24957) 
Signed-off-by: Tao He <linzhu.ht@alibaba-inc.com>
 2025-09-17 21:59:09 +08:00
danielafrimi
252ada5559 Add RADIO Vision Encoder Support to vLLM (#24595) 
Signed-off-by: Daniel Afrimi <danielafrimi8@gmail.com>
Co-authored-by: root <root@cw-dfw-h100-001-305-026.cm.cluster>
 2025-09-17 05:53:30 -07:00
whx
c15309a730 [Model] Apply SharedFusedMoE to glm4_moe. (#24849) 
Signed-off-by: whx-sjtu <2952154980@qq.com>
 2025-09-17 16:02:31 +08:00
whx
4a9375fe9d [Model] Pass param prefix to LLMHead (#24862) 
Signed-off-by: whx-sjtu <2952154980@qq.com>
 2025-09-17 16:01:27 +08:00
rouchenzi
b77bf34e53 [EPLB] Support EPLB for Mixtral Model (#22842) 
Signed-off-by: rouchenzi <ruochenwen@gmail.com>
Signed-off-by: rouchenzi <40842833+rouchenzi@users.noreply.github.com>
Co-authored-by: Bowen Wang <abmfy@icloud.com>
 2025-09-17 07:27:34 +00:00
haoyangli-amd
ca2d1925ef [Rocm] [quantization] Fix quark ptpc moe and add test case (#24649) 
Signed-off-by: Haoyang Li <lihaoyang0109@gmail.com>
Co-authored-by: Haoyang Li <haoyang.li@amd.com>
 2025-09-16 22:15:13 -07:00
Roger Wang
0f7acdd73c [Model] Support Qwen3-VL Model Series (#24727) 
Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Huang Jie <92386084+JJJYmmm@users.noreply.github.com>
Co-authored-by: 松灵 <26085463+wulipc@users.noreply.github.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-09-17 05:01:04 +00:00
Tahsin Tunan
cef32104b4 [FP8] Extend per-token-group quantization support to QuantFP8 (#24342) 
Signed-off-by: Tahsin Tunan <tahsintunan@gmail.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>
 2025-09-16 18:31:06 -07:00
Matthew Bonanni
d119fc8614 [CI][Bugfix] Fix failing Blackwell test (#24993) 
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2025-09-16 15:55:02 -07:00
Michael Goin
dbebb7f812 [Perf] Reuse workspace for FP8+FP4 Marlin MoE (#20500) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-09-16 15:45:10 -06:00
Aleksandr Malyshev
3053a22b33 fp8 kv cache support fix for torch.compile (#22758) 
Signed-off-by: Aleksandr Malyshev <maleksan@amd.com>
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: Aleksandr Malyshev <maleksan@amd.com>
Co-authored-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: Gregory Shtrasberg <156009573+gshtras@users.noreply.github.com>
 2025-09-16 21:27:11 +00:00
Isotr0py
d593cf28fa [Misc] Add removed encoder-decoder models to previously supported models list (#24961) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-09-16 10:46:46 -07:00
Sage Moore
567939953b [Core/DBO][1/N] Add Dual-Batch Overlap mechanism to VLLM (#23693) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2025-09-16 12:21:48 -04:00
Chih-Chieh Yang
73cfb3c5ee [Model] Clean up and simplify Mamba2 Metadata Usage in both V0 and V1 (#24331) 
Signed-off-by: Chih-Chieh-Yang <7364402+cyang49@users.noreply.github.com>
 2025-09-16 14:53:43 +00:00
Chen Bruce
7ea5c73ad7 [Feat][EPLB] A novel static EPLB placement strategy for MoE models. (#23745) 
Signed-off-by: bruceszchen <bruceszchen@tencent.com>
Signed-off-by: Chen Bruce <bruceszchen@tencent.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Chen Bruce <cszwwdz@vip.qq.com>
Co-authored-by: lemon412 <lemon412@foxmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-16 10:55:16 +00:00
tomeras91
27fcfe7bcf [Mamba] Support TP>1 with quantization for mamba2 mixer in case n_groups % tp_size == 0 (#24593) 
Signed-off-by: Tomer Asida <57313761+tomeras91@users.noreply.github.com>
Signed-off-by: tomeras91 <57313761+tomeras91@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-16 10:51:01 +00:00