biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
9,506 Commits 2 Branches 140 Tags
73cfb3c5eeb8b00a6e222751a28fd89a5f6229dc
Commit Graph

2683 Commits

Author SHA1 Message Date
Chih-Chieh Yang
73cfb3c5ee [Model] Clean up and simplify Mamba2 Metadata Usage in both V0 and V1 (#24331) 
Signed-off-by: Chih-Chieh-Yang <7364402+cyang49@users.noreply.github.com>
 2025-09-16 14:53:43 +00:00
Chen Bruce
7ea5c73ad7 [Feat][EPLB] A novel static EPLB placement strategy for MoE models. (#23745) 
Signed-off-by: bruceszchen <bruceszchen@tencent.com>
Signed-off-by: Chen Bruce <bruceszchen@tencent.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Chen Bruce <cszwwdz@vip.qq.com>
Co-authored-by: lemon412 <lemon412@foxmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-16 10:55:16 +00:00
tomeras91
27fcfe7bcf [Mamba] Support TP>1 with quantization for mamba2 mixer in case n_groups % tp_size == 0 (#24593) 
Signed-off-by: Tomer Asida <57313761+tomeras91@users.noreply.github.com>
Signed-off-by: tomeras91 <57313761+tomeras91@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-16 10:51:01 +00:00
Jee Jee Li
04ad0dc275 [benchmark] Add triton version in the moe tuned config (#24769) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-16 14:10:54 +08:00
Saman A. Pour
238c4c1705 [QWEN NEXT] Fused MoE kernels Optimization configs (#24924) 
Signed-off-by: Saman Keon <samanamp@outlook.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-16 13:06:03 +08:00
vllmellm
8c54610265 [Bug] [Spec Dec]: Fix kv_cache dtype mismatch for Eagle3 drafter on FP8 target (#24505) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
 2025-09-16 04:45:38 +00:00
Woosuk Kwon
759ef49b15 Remove V0 Encoder-Decoder Support (#24907) 
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
 2025-09-15 21:17:14 -07:00
Gregory Shtrasberg
2891603efd [ROCm][Bugfix] Fix the case where there's bias (#24895) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2025-09-15 20:05:12 -06:00
Kyle Sayers
a0b26701c9 [Transform] Deterministic Hadacore Transforms (#24106) 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
 2025-09-15 12:59:31 -06:00
Harry Mellor
c4afdb69cc Move MultiModalConfig from config/__init__.py to config/multimodal.py (#24659) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-15 17:43:16 +00:00
Rafael Marcelino Koike
b834b4cbf1 [USAGE] Improve error handling for weight initialization in Unquantized… (#20321) 
Signed-off-by: Rafael Marcelino Koike <rafael.koike@oracle.com>
Signed-off-by: Rafael Koike <koike.rafael@gmail.com>
 2025-09-15 16:45:49 +00:00
Isotr0py
0e219cd50b [Bugfix] Fix GLM4.1V multimodal processor with compatability for Transformers v4.56 (#24822) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-09-15 20:45:06 +08:00
ant-yy
72c99f2a75 [Model]: support Ling2.0 (#24627) 
Signed-off-by: vito.yy <vito.yy@antgroup.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-09-15 05:09:30 -07:00
Didier Durand
4979eb79da [Doc]: fix typos in various files (#24821) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
 2025-09-15 01:08:52 -07:00
bingchen-mi
a8c0f59973 [Bugfix] MiDashengLM model contact error under concurrent testing (#24738) 
Signed-off-by: chenbing8 <chenbing8@xiaomi.com>
Signed-off-by: bingchen-mi <chenbing8@xiaomi.com>
 2025-09-15 06:38:12 +00:00
Wentao Ye
fc2dbcda8b [Perf] Fix DeepGEMM Contiguous Layout Issue, 5.5% Throughput Improvement (#24783) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2025-09-14 11:20:17 -04:00
Wenlong Wang
cc3173ae98 [Multi Modal][Performance] Fused Q,K's apply_rope into one (#24511) 
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-09-14 08:10:21 +00:00
Didier Durand
41ae4a1eab [Doc]: fix typos in various files (#24798) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
 2025-09-13 00:43:33 -07:00
Elvir Crnčević
98229db244 [Kernels][DP/EP] Optimize Silu Kernel for R1 (#24054) 
Signed-off-by: elvircrn <elvircrn@gmail.com>
 2025-09-13 00:17:27 -07:00
Hyogeun Oh (오효근)
9a8966bcc2 [Docs] Fix warnings in mkdocs build (continued) (#24791) 
Signed-off-by: Zerohertz <ohg3417@gmail.com>
 2025-09-13 00:13:44 -07:00
Woosuk Kwon
5febdc8750 [Chore] Remove unused batched RoPE op & kernel (#24789) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-09-13 00:08:20 -07:00
Shane A
89e08d6d18 [Model] Add Olmo3 model implementation (#24534) 
Signed-off-by: Shane A <shanea@allenai.org>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-09-13 03:26:21 +00:00
Matthew Bonanni
7ba32aa60b [Attention][FlashInfer] Enable FP8 FlashInfer (TRTLLM) MLA decode (#24705) 
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
 2025-09-12 15:45:53 -06:00
Alexandre Marques
c89ed8de43 Invert pattern order to make sure that out_proj layers are identified (#24781) 
Signed-off-by: Alexandre Marques <almarque@redhat.com>
 2025-09-12 14:45:29 -07:00
Wentao Ye
3beadc2f25 [Compilation Bug] Fix Inductor Graph Output with Shape Issue (#24772) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-09-12 21:23:05 +00:00
Samit
f17c075884 [Model] Switch to Fused RMSNorm in GLM-4.1V model (#24733) 
Signed-off-by: SamitHuang <285365963@qq.com>
 2025-09-12 09:12:23 -07:00
Lukas Geiger
b0d1213ac3 [Models] Prevent CUDA sync in Qwen2.5-VL (#24741) 
Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>
 2025-09-12 16:03:55 +00:00
Lukas Geiger
57f94e88ea [Models] Optimise and simplify _validate_and_reshape_mm_tensor (#24742) 
Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>
 2025-09-12 15:37:37 +00:00
Elvir Crnčević
9f04d9d55f [Qwen3-Next] MoE configs for H100 TP=1,2 and TP2/EP (#24739) 
Signed-off-by: elvircrn <elvircrn@gmail.com>
 2025-09-12 07:54:04 -07:00
Yan Ma
4d7c1d531b [Bugfix] Fix MRoPE dispatch on XPU (#24724) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
 2025-09-12 21:43:56 +08:00
Hyogeun Oh (오효근)
41f17bf290 [Docs] Fix warnings in mkdocs build (continued) (#24740) 
Signed-off-by: Zerohertz <ohg3417@gmail.com>
 2025-09-12 06:43:15 -07:00
Didier Durand
bcb06d7baf [Doc]: fix typos in various files (#24726) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
 2025-09-12 06:43:12 -07:00
Wenlong Wang
72fc8aa412 [Multi Modal] Add FA3 in VIT (#24347) 
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
 2025-09-12 21:27:24 +08:00
Jee Jee Li
60a0951924 [Bugfix] Fix BNB name match (#24735) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-12 11:12:01 +00:00
Nick Hill
f592b3174b [BugFix] Fix Qwen3-Next PP (#24709) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-09-11 23:35:04 -07:00
Li, Jiang
7920de0a2a [Bugfix] Fix MRoPE dispatch on CPU (#24712) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-09-12 04:56:31 +00:00
Andrew Sansom
ddcec289c7 Fix implementation divergence for BLOOM models between vLLM and HuggingFace when using prompt embeds (#24686) 
Signed-off-by: Andrew Sansom <andrew@protopia.ai>
 2025-09-12 04:35:48 +00:00
Maximilien de Bayser
e090b7b45b Enable conversion of multimodal models to pooling tasks (#24451) 
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
 2025-09-12 03:30:41 +00:00
Jee Jee Li
12a8414d81 [Qwen3-Next] MoE configs for H20 TP=1,2,4,8 (#24707) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-12 10:06:26 +08:00
Tao He
880c741bb6 [Bugfix] fixes the causal_conv1d_update kernel update non-speculative decoding cases (#24680) 
Signed-off-by: Tao He <linzhu.ht@alibaba-inc.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-09-11 18:16:43 -07:00
Lucas Wilkinson
2e6bc46821 [Startup] Make DeepGEMM warmup scale with max-num-batched-tokens (#24693) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-09-11 20:10:19 -04:00
Wentao Ye
fcba05c435 [Bug] Fix Layer weight_block_size Assertion Issue (#24674) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-09-11 19:47:59 -04:00
Chen Zhang
f82f7a8990 [Qwen3-Next] MOE configs for H100 TP4 (#24699) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-09-11 15:45:52 -07:00
Michael Goin
c3aea10dc8 [Perf] Use upstream CUTLASS for SM90 Block FP8 kernel (#23280) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-09-11 15:43:14 -07:00
Vadim Gimpelson
7a70a71892 [Qwen3-Next] Add B200 MoE configs for Qwen3-next (#24698) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
 2025-09-11 15:34:58 -07:00
Woosuk Kwon
569bf1c9c0 [Qwen3-Next] MoE configs for H200 TP=1,2,4 (#24695) 
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
 2025-09-11 14:38:16 -07:00
Duncan Moss
074854b24f [Kernel][B200] mxfp4 fused cutlass moe (#23696) 
Signed-off-by: Duncan Moss <djm.moss@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-09-11 17:04:56 -04:00
Woosuk Kwon
c733bd5e87 [Qwen3-Next] Add MoE Config for H200 (#24688) 
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
 2025-09-11 12:40:15 -07:00
Wentao Ye
a892b259b4 [Doc] Remove Useless Comments (#24687) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-09-11 12:25:47 -07:00
Peter Salas
127ded0a9e [Ultravox] Use wrapped_model_config to instantiate inner model (#24679) 
Signed-off-by: Peter Salas <peter@fixie.ai>
 2025-09-11 18:52:24 +00:00