biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
11,267 Commits 2 Branches 140 Tags
6e25b1cddfd78eab307acdb5e3ec14475e465d90
Commit Graph

261 Commits

Author SHA1 Message Date
elvischenv
5d6ce2b960 [Perf] Support stream interval for reducing host overhead (#27869) 
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-11-13 13:21:25 -05:00
Roger Wang
d3387750f1 [Misc] Turn off encoder torch compile by default (#28634) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2025-11-13 08:38:08 -08:00
Harry Mellor
b230286fbc Fix get_num_experts when config sets it explicitly to None (#28652) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: bruceszchen <bruceszchen@tencent.com>
 2025-11-13 16:02:42 +00:00
Nick Hill
8832fff972 [BugFix] Fix mm_encoder_attn_backend arg type checking (#28599) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-11-13 03:06:03 +00:00
ℍ𝕠𝕝𝕝𝕠𝕨 𝕄𝕒𝕟
4ca5cd5740 [Core][AMD] Migrate fully transparent sleep mode to ROCm platform (#12695) 
Signed-off-by: Hollow Man <hollowman@opensuse.org>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: kliuae <kuanfu.liu@embeddedllm.com>
 2025-11-12 15:24:12 -08:00
Thomas Parnell
64d57c3be7 [Model] [Config] Correctly identify granite-4.0-micro as non-hybrid model (#28563) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
 2025-11-12 18:17:55 +00:00
PerryZhang01
a1e7fa362a [EPLB][ROCm]: support EPBL for ROCm backend (#27731) 
Signed-off-by: Perry Zhang <perzhang@amd.com>
Co-authored-by: Perry Zhang <perzhang@amd.com>
 2025-11-12 18:16:35 +00:00
Harry Mellor
a742134cc5 Remove deprecated fields from CompilationConfig (#27593) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-11-12 16:10:28 +00:00
TJian
edb59a9470 [ROCm] [Bugfix] Fix fused_qknorm_rope_kernel rocm compatibility (#28500) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2025-11-12 05:01:14 -08:00
Chenguang Zheng
4ccffe561f [Core] Encoder separation for Encode-Prefill-Decode Disaggregation (#25233) 
Signed-off-by: n00909098 <nguyen.kha.long@huawei.com>
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Signed-off-by: herotai214 <herotai214@gmail.com>
Signed-off-by: Khuong Le <khuong.le.manh@huawei.com>
Signed-off-by: Khuong Le <lemanhkhuong2611@gmail.com>
Co-authored-by: n00909098 <nguyen.kha.long@huawei.com>
Co-authored-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Co-authored-by: herotai214 <herotai214@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Khuong Le <khuong.le.manh@huawei.com>
Co-authored-by: Khuong Le <lemanhkhuong2611@gmail.com>
 2025-11-11 18:58:33 -08:00
Yanan Cao
48c879369f [Frontend] Change CompilationMode to a proper Enum (#28165) 
Signed-off-by: Yanan Cao <gmagogsfm@gmail.com>
 2025-11-11 19:46:18 -05:00
zhrrr
68c09efc37 [Kernel][Perf] fuse QK Norm and RoPE into one cuda kernel for Qwen Model (#27165) 
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
 2025-11-11 12:00:31 -05:00
Matthew Bonanni
b30dfa03c5 [Attention] Refactor CUDA attention backend selection logic (#24794) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2025-11-11 07:40:44 -05:00
Ilya Markov
d17ecc6b19 [PERF] Allreduce fusion. Support torch native matching. Tuning of the thresholds (#24248) 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2025-11-10 18:33:11 -05:00
Robert Shaw
30700b1cd7 [CI] Fix Plugin Tests Tests (#28413) 
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
 2025-11-10 22:36:11 +00:00
zhangsicheng5
2108a571d7 [DCP] Support dcp kv_cache interleave size > 1 (#26696) 
Signed-off-by: zhangsicheng5 <zhangsicheng5@huawei.com>
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
Signed-off-by: Qiu <qiuchunshuo@huawei.com>
Co-authored-by: QiuChunshuo <qiuchunshuo@huawei.com>
 2025-11-09 04:45:27 +09:00
Aurick Qiao
781f5ebf52 Bump arctic-inference requirement (#28174) 
Co-authored-by: Aurick Qiao <aurick.qiao@snowflake.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-11-07 18:31:18 -08:00
Nick Hill
da786e339e [Core] Rework handling of async scheduling config (#28250) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-11-07 20:01:23 +00:00
Harry Mellor
c0a4b95d64 Fix issues from #28242 (#28257) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-11-07 04:23:17 +00:00
Lucas Kabela
4bf56c79cc [Multimodal][torch.compile] Add compilation config field for turning off ViT/MM compile (#28242) 
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
 2025-11-07 00:16:03 +00:00
Vadim Gimpelson
b6a248bdd7 [PERF] Decouple projections from GDN custom op. Attempt 2 (#28083) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
 2025-11-05 17:01:12 -08:00
Walter Beller-Morales
752ddeacaa [Core] add support for reasoning parser plugins (#28075) 
Signed-off-by: walter beller-morales <walter.beller.morales@gmail.com>
 2025-11-06 01:15:06 +08:00
Vadim Gimpelson
d4e547bb7e Revert "[PERF] Decouple projections from GDN custom op" (#28080) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
 2025-11-04 15:58:23 -08:00
Harry Mellor
2f1cc8cef1 Remove deprecated --rope-scaling and --rope-theta (#28006) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-11-04 18:01:56 +00:00
yt0428
05cae69f0f [model] Add support for openPangu_Ultra_MoE (#27521) 
Signed-off-by: yuantao <2422264527@qq.com>
Signed-off-by: yt0428 <51468697+yt0428@users.noreply.github.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-11-04 08:17:20 -08:00
Vadim Gimpelson
5fd8f02ea9 [PERF] Decouple projections from GDN custom op (#27512) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
 2025-11-04 08:11:41 -08:00
Ning Xie
786030721e [Docs] add runai_streamer_sharded to LoadConfig (#27937) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2025-11-03 20:35:16 +00:00
Aurick Qiao
2c19d96777 [Spec Decode] Integrate Suffix Decoding from Arctic Inference (#25784) 
Co-authored-by: Aurick Qiao <aurick.qiao@snowflake.com>
 2025-11-03 09:23:31 -08:00
ahao-anyscale
cac4c10ef0 [BUG] Make 'binary' default option for saving torch compile artifacts when using standalone_compile (#27616) 
Signed-off-by: ahao-anyscale <ahao@anyscale.com>
 2025-11-03 11:13:51 -05:00
Asaf Joseph Gardin
00b31a36a2 [V1] [Hybrid] Mamba1 Automatic Prefix Caching (#26377) 
Signed-off-by: asafg <39553475+Josephasafg@users.noreply.github.com>
 2025-11-02 04:16:23 -08:00
Cyrus Leung
99d69af9ec [Bugfix] Python 3.10 compatibility for Self (#27918) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-11-01 15:28:54 +00:00
Yihua Cheng
e675118849 [Add] cmdline argument parsing for KV cache offloading modules (#27621) 
Signed-off-by: ApostaC <yihua98@uchicago.edu>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-11-01 07:17:07 +00:00
Vinay R Damodaran
5e8862e9e0 [Feature] Pydantic validation for scheduler.py and structured_outputs.py (#26519) 
Signed-off-by: Vinay Damodaran <vrdn@hey.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-31 18:05:50 +00:00
Nick Hill
9e5bd3076e [Cleanup] Remove no-longer-used SpeculativeConfig.enable_chunked_prefill (#27826) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-10-31 10:57:45 -07:00
Paul Zhang
e7acb20076 [Feature] Batch invariant torch.compile (#27660) 
Signed-off-by: PaulZhang12 <paulzhan@fb.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-10-30 13:11:29 -07:00
Zhiyuan Li
4e68cc9b6a [Model] Introduce Kimi Linear to vLLM (#27809) 
Signed-off-by: lizhiyuan <lizhiyuan@moonshot.cn>
Signed-off-by: Zhiyuan Li <uniartisan2017@gmail.com>
 2025-10-30 21:02:27 +08:00
wang.yuqi
4464723f22 [Frontend][Doc][5/N] Improve all pooling task | Polish encode (pooling) api & Document. (#25524) 
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-10-30 12:13:05 +00:00
Sairam Pillai
74374386e2 [Bugfix] Improve GPU validation logging in Ray fallback scenarios (#25775) 
Signed-off-by: Sairam Pillai <sairam.pillai61@gmail.com>
 2025-10-30 11:57:59 +00:00
Wentao Ye
c01f6e525f [CI] Fix mypy for vllm/v1/core and vllm/v1/engine (#27108) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-30 11:32:17 +00:00
wangxiyuan
af826e0820 [V0 deprecation] Remove VLLM_USE_V1 usage in config module (#27784) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2025-10-30 09:42:49 +00:00
Chen Zhang
d7fb10c574 [Bugfix] mamba-block-size is set for vision language model (#27773) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-10-29 19:39:57 -07:00
Cyrus Leung
4fb8771cc0 [CI/Build] Move pre-commit only scripts to tools/pre_commit (#27657) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-29 08:04:33 +00:00
Lucas Kabela
94666612a9 [Misc][qwen2_5_vl][torch.compile] Enable supports_torch_compile on generic nn.Module and demonstrate speedup on Qwen Vision model (#23207) 
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
Signed-off-by: Lucas Kabela <lucasakabela@gmail.com>
 2025-10-28 22:36:43 +00:00
Cyrus Leung
f5710ef02a [Misc] Make LayerBlockType a Literal instead of Enum (#27658) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-28 16:23:35 +00:00
Asaf Joseph Gardin
05181cc57f [Hybrid] Add mamba_block_size to Engine Args (#27289) 
Signed-off-by: asafg <39553475+Josephasafg@users.noreply.github.com>
 2025-10-28 12:54:24 +00:00
Zhengxu Chen
259504e147 [compile] Add enable_prompt_embeds to compile hash. (#27285) 
Signed-off-by: zhxchen17 <zhxchen17@fb.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-10-28 20:46:03 +08:00
Matthew Bonanni
44b5ce956d [Bugfix] In LongRoPE, decide short vs long based on max_model_len (#27431) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-10-28 12:00:56 +00:00
Kuntai Du
255e34ca50 [Stability fix] turn off HMA allocator when connector is set (#27592) 
Signed-off-by: KuntaiDu <kuntai@uchicago.edu>
Signed-off-by: Kuntai Du <kuntai@uchicago.edu>
 2025-10-27 18:32:23 -07:00
Cyrus Leung
6ebffafbb6 [Misc] Clean up more utils (#27567) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-27 15:30:38 +00:00
Asaf Joseph Gardin
9273754222 [Hybrid] Added supports_mamba_prefix_caching Protocol (#27339) 
Signed-off-by: asafg <39553475+Josephasafg@users.noreply.github.com>
 2025-10-27 13:05:20 +00:00