biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
14,470 Commits 2 Branches 140 Tags
2f2c1d73a745d8a38d1a21a5865a7d53d8d616b7
Commit Graph

14470 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Russell Bryant
2f2c1d73a7 [Docs] Upgrade dynamic LoRA warning to admonition block (#35218) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2026-03-04 10:01:42 -08:00
Bhuminjay Soni
fb3e78ab09 [Feature][CI]: compare func & no_func outputs in test_functionalization.py (#35481) 
Signed-off-by: Bhuminjay <bhuminjaysoni@gmail.com>
Signed-off-by: Bhuminjay Soni <Soni5Happy@gmail.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2026-03-04 18:01:16 +00:00
Michael Yao
fd3bfe74c9 [Docs] Update design/multiprocessing.md (#30677) 
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
 2026-03-04 17:58:59 +00:00
tc-mb
bfdb512f11 fix minicpmo4.5: fix attn_mask in vit attn && fix resampler pos_emb i… (#34127) 
Signed-off-by: tc-mb <caitianchi@modelbest.cn>
Co-authored-by: hezhihui <hezhihui@modelbest.cn>
 2026-03-04 17:46:17 +00:00
Sage
d25c1ec3c9 docs(cpu): Clarify pre-built wheels requirement for CPU Python-only build (#35090) 
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com>
 2026-03-04 17:45:35 +00:00
Xing Liu
7cc6058ac6 [Doc] Add MTP docs and update speculative decoding guidance (#35197) 
Signed-off-by: liuxing <945764858@qq.com>
 2026-03-04 17:23:34 +00:00
Manrique Vargas
28028dff2f fix(docs): use static rdzv backend in multi-node troubleshooting script (#34784) 
Signed-off-by: machov <mv1742@nyu.edu>
 2026-03-04 17:15:35 +00:00
Dr Alex Mitre
3417ba5648 docs: add README for logits_processor examples (#35933) 2026-03-04 17:09:19 +00:00
Yan Ma
58cfe0dc44 Fix phi4-mm and remove cuda binding (#35964) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
 2026-03-05 01:08:05 +08:00
simone-dotolo
e86221deb6 [Doc] Fix GPU Worker count in Process Count Summary (#36000) 
Signed-off-by: simone-dotolo <simonedotolo@libero.it>
Signed-off-by: simone-dotolo <84937474+simone-dotolo@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-03-04 17:03:14 +00:00
Netanel Haber
289fc48ab7 Use MMEncoderAttention (=use FlashAttention) instead of torch.sdpa in radio.py (#35653) 2026-03-04 08:43:13 -08:00
Christian Pinto
2f2212e6cc Split generic IO Processor plugins tests from Terratorch specific ones (#35756) 
Signed-off-by: Christian Pinto <christian.pinto@ibm.com>
 2026-03-05 00:01:03 +08:00
Nicolò Lucchesi
18e01a0a10 [Misc] Add --attention-backend auto option (#35738) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-03-04 15:12:27 +00:00
sungsoo ha
6cb901093f [Core] Add All-to-All communication backend for DCP (#34883) 
Signed-off-by: Sungsoo Ha <sungsooh@nvidia.com>
Signed-off-by: sungsoo ha <hasungsoo@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-04 10:01:57 -05:00
Cyrus Leung
ead7bde1ab [Bugfix] Make kaldi_native_fbank optional (#35996) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-03-04 06:47:32 -08:00
Qi Wang
6aa6ad8992 [BugFix] Fix implicit and incorrect assumption on ECConnector is_producer (#34783) 
Signed-off-by: Qi Wang <qiwa@nvidia.com>
 2026-03-04 15:01:30 +01:00
Raghavan
c8c3935b70 [Bugfix][Model] Fix FP8 k_scale/v_scale not loaded for Qwen3-MoE (#35656) 
Signed-off-by: raghavan <oneraghavan@gmail.com>
 2026-03-04 13:15:38 +00:00
Ronen Schaffer
bb6888b8b1 [Bugfix][CPUOffloadingManager] Prevent eviction of already-stored blocks in LRU/ARC prepare_store() (#35846) 
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
 2026-03-04 14:25:33 +02:00
Taneem Ibrahim
1aaec59d79 [MISC] fixed tool_parser mypy errors (#35640) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-04 12:23:12 +00:00
pougetat
1659b2e058 [Feature] Add basic metrics for /realtime endpoint (#35500) 
Signed-off-by: Thomas Pouget-Abadie <thomaspou@microsoft.com>
Signed-off-by: pougetat <thomas.pougetabadie@gmail.com>
Co-authored-by: Thomas Pouget-Abadie <thomaspou@microsoft.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-03-04 19:56:32 +08:00
haosdent
d6e04f4c43 [Bugfix] Cap FULL decode cudagraph sizes for Mamba/hybrid models (#34094) (#34571) 
Signed-off-by: haosdent <haosdent@gmail.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
 2026-03-04 11:56:22 +01:00
Kunshang Ji
a8f66cbde8 [XPU] bump vllm-xpu-kernels to v0.1.3 (#35984) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-03-04 18:23:31 +08:00
Kunshang Ji
16d2ad1d38 [Hardware] Replace torch.cuda.empty_cache with torch.accelerator.empty_cache (#30681) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-04 09:49:47 +00:00
Chuan (Richard) Li
5dc3538736 [ROCm][Bugfix] Fall back from CK MXFP4 MoE when GEMM dimensions are unsupported (#35893) 
Signed-off-by: Li <chuali@amd.com>
 2026-03-04 08:30:54 +00:00
Nathan Price
36bf213181 [Bugfix] Add missing dynamic_arg_dims for Qwen3-ASR torch.compile (#35869) 
Signed-off-by: Nathan Price <nathan@abridge.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-03-04 08:29:01 +00:00
Joe Runde
6f0dd93801 [Core] Remove busy loop from idle buffer readers (#28053) 
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Travis Johnson <tsjohnso@us.ibm.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-03-04 07:44:20 +00:00
Andrii Skliar
5d199ac8f2 Support Audio Extraction from MP4 Video for Nemotron Nano VL (#35539) 
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Signed-off-by: Andrii <askliar@nvidia.com>
Co-authored-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
Co-authored-by: Andrii Skliar <askliar@oci-nrt-cs-001-vscode-01.cm.cluster>
Co-authored-by: Andrii <askliar@nvidia.com>
Co-authored-by: root <root@pool0-03748.cm.cluster>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: root <root@pool0-02416.cm.cluster>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: wangxiyuan <wangxiyuan1007@gmail.com>
Co-authored-by: root <root@pool0-04880.cm.cluster>
 2026-03-03 23:20:33 -08:00
Komal Kumar Teru
9e0f44bec4 [cohere][fix][spec-decode]: fix crash when allowed_token_ids is set without penalties (#35654) 
Signed-off-by: kkt-cohere <komal@cohere.com>
 2026-03-03 23:20:15 -08:00
lailoo
097eb544e9 [Bugfix] Improve engine ready timeout error message (#35616) 
Signed-off-by: damaozi <1811866786@qq.com>
v0.17.0rc0 		2026-03-04 05:54:32 +00:00
ShiJie Zhong
7cdba98edf [BugFix] Support tool_choice=none in the Anthropic API (#35835) 
Signed-off-by: ZhongsJie <zhongsjie@gmail.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-03-04 05:24:46 +00:00
Charlie Fu
3c85cd9d74 [Rocm][CI] Fix ROCm LM Eval Large Models (8 Card) (#35913) 
Signed-off-by: charlifu <charlifu@amd.com>
 2026-03-04 04:50:13 +00:00
Andreas Karatzas
edba15045a [Bugfix] Guard mm_token_type_ids kwarg in get_mrope_input_positions (#35711) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-03-04 04:12:51 +00:00
Cyrus Leung
e379396167 [Refactor] Clean up processor kwargs extraction (#35872) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-03-03 19:53:53 -08:00
Isotr0py
6e9f21e8a2 [Chore] Remove debug code in model implementation (#35883) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-03-03 19:50:58 -08:00
AllenDou
c1d963403c [model] support FireRedASR2 (#35727) 
Signed-off-by: zixiao <shunli.dsl@alibaba-inc.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: zixiao <shunli.dsl@alibaba-inc.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-03-03 19:41:30 -08:00
Shanshan Shen
77e6dcbbfa [PluggableLayer][MM] Add PluggableLayer for RelPosAttention (#33753) 
Signed-off-by: shen-shanshan <467638484@qq.com>
 2026-03-03 19:41:27 -08:00
William Zhang
70c73df69e [Bugfix] Fix EVS implementation for Qwen3 VL (#33607) 
Signed-off-by: 2ez4bz <133824995+2ez4bz@users.noreply.github.com>
 2026-03-04 02:18:11 +00:00
xjx
9a9d442464 Enable bnb for multiple indices weight (#35838) 
Signed-off-by: xjx <493337577@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-03-04 01:46:47 +00:00
Andreas Karatzas
f7da9cdffc [ROCm][CI] Support async weight transfer example with platform-aware determinism (#35710) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-03-04 09:44:14 +08:00
Jaewon
f22ff2958c [Bugfix] Fix coord_socket assertion in DPEngineCoreProc for offline DP mode (#35916) 
Signed-off-by: Jaewon Lee <jaewon@meta.com>
 2026-03-04 00:10:11 +00:00
Nick Hill
d15c3b90fc [Core] Move save_tensorized_model logic to Worker (#35825) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-03-03 15:31:59 -08:00
zhrrr
97286a20ed [Model Runner V2] support dp & ep for spec decoding (#35294) 
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai>
 2026-03-03 15:19:45 -08:00
Amr Mahdi
12b38c0f45 [CI/Build] Allow mounting AWS credentials for sccache S3 auth (#35912) 
Signed-off-by: Amr Mahdi <amrmahdi@meta.com>
 2026-03-03 14:30:47 -08:00
Woosuk Kwon
467886a0c4 [Model Runner V2] Fix inputs_embeds=None bug for MM models (#35917) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-03-03 13:47:45 -08:00
bnellnm
a9b8b13e5c [Bugfix] Fix misnamed parameter in compressed_tensors_moe.py (#35813) 
Signed-off-by: Bill Nell <bnell@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-03-03 16:29:57 -05:00
Micah Williamson
e7213003cb [ROCm][CI] Fix TP size issue for test_gpt_oss (#35887) 
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
 2026-03-03 20:57:34 +00:00
Rohan Potdar
3a8eef5869 [ROCm][Bugfix]: Disable AITER Triton ROPE by default (#35601) 
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
 2026-03-03 13:43:56 -06:00
Robert Shaw
97995f6376 [MoE Refactor] Create MK for TRTLLM Kernels (#32564) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Signed-off-by: Robert Shaw <rshaw@neuralmagic.com>
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
 2026-03-03 10:39:50 -08:00
Robert Shaw
881a6b011b [CI] Temporarily Disable Llama4 MoE Refactor Test (#35870) 
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-03-03 10:36:15 -08:00
Matthew Bonanni
8e1fd5baf0 [CI] Bump num_speculative_tokens to 3 in nightly DeepSeek tests (#35882) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-03-03 09:26:44 -08:00