biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
14,099 Commits 2 Branches 140 Tags
272b535ab3315a2ed3cd1a5e9803df2b86da4f07
Commit Graph

9985 Commits

Author SHA1 Message Date
Cyrus Leung
f74f1572ca [Benchmark] Improve benchmarks (#35012) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-21 10:31:58 +00:00
Nick Hill
820d7815eb [Core] Minor structured-output related scheduler optimization (#34765) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-02-21 01:38:28 -08:00
Nicolò Lucchesi
ab6f3487a6 [PD] Change kv_load_failure_policy Default from "recompute" to "fail" (#34896) 
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-02-21 01:34:57 -08:00
BADAOUI Abdennacer
8dc8a99b56 [ROCm] Enable bitsandbytes quantization support on ROCm (#34688) 
Signed-off-by: badaoui <abdennacerbadaoui0@gmail.com>
 2026-02-21 00:34:55 -08:00
jennyyyyzhen
2aab2bb543 [ROCM] Optimize ROCM_AITER_FA spec decode eagle performance (#34541) 
Signed-off-by: jennyyyyzhen <yzhen@hmc.edu>
 2026-02-20 20:32:05 -08:00
Andreas Karatzas
cf93c1a128 [ROCm][AITER] Fix aiter paged_attention_v1 decode for sliding window and head_size < 64 (#34570) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-02-20 20:25:07 -08:00
zhongdaor-nv
a0fe7ea2f0 [feat] Add per-block extra_keys to KV events (#33304) 
Signed-off-by: zhongdaor-nv <zhongdaor@nvidia.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-02-20 20:11:40 -08:00
Andreas Karatzas
991d6bff38 [CI][MCP][Harmony] Heavy refactoring Harmony & MCP response tests and stabilizing with deterministic test infrastructure (#33949) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-02-20 20:03:32 -08:00
Kata Coder
5719a4e4e6 [Frontend] Support multimodal inputs for late-interaction scoring (ColQwen3) + NewModel: nvidia/nemotron-colembed (#34574) 
Signed-off-by: craftsangjae <craftsangjae@gmail.com>
 2026-02-20 20:01:40 -08:00
pougetat
11be2c74dc [Realtime] Add Qwen3-ASR realtime streaming support (#34613) 
Signed-off-by: Thomas Pouget-Abadie <thomaspou@microsoft.com>
Co-authored-by: Thomas Pouget-Abadie <thomaspou@microsoft.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2026-02-20 19:59:42 -08:00
Xin Yang
7a5adad480 [Kernel] Optimize sample_recovered_tokens_kernel (#34974) 
Signed-off-by: Xin Yang <xyangx@amazon.com>
 2026-02-20 19:59:06 -08:00
Li
59c6233297 Support prompt_embeds for pooling requests in output processor (#34904) 
Signed-off-by: Li Zhang <lzhanga@amazon.com>
Co-authored-by: Li Zhang <lzhanga@amazon.com>
 2026-02-20 19:57:38 -08:00
Taneem Ibrahim
d38cd3dde5 [Misc] Fix mypy errors in vllm/profiler and remove from exclude list (#34959) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-02-20 19:56:33 -08:00
Rohan Potdar
ded333fb9b [ROCm][Bugfix]: Only save unpadded sizes for shared_experts in MoERunner to fix rmsnorm pad fusion (#34636) 
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
 2026-02-20 19:56:16 -08:00
Yanan Cao
9d7577b2bd [Kernel] [Helion] [9/N] Canonicalize GPU variant names to base model names (#34928) 
Signed-off-by: Yanan Cao <gmagogsfm@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-20 19:55:51 -08:00
yugong333
a55caf6ae9 [LoRA] Support Quantized Adapters (#30286) 
Signed-off-by: Yu Gong <yu3.gong@gmail.com>
Signed-off-by: wz1qqx <ziqi.wang@novita.ai>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: wz1qqx <55830058+wz1qqx@users.noreply.github.com>
Co-authored-by: wz1qqx <ziqi.wang@novita.ai>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-02-20 19:54:35 -08:00
Lucas Wilkinson
0e22cd618b Revert "[Llama4,Quantization] Simplify and generalize logic for Q/K permutations in quantized self-attn layers " (#34997) 2026-02-20 17:19:19 -08:00
Wei Zhao
ea5f903f80 Bump Flashinfer Version and Re-enable DeepSeek NVFP4 AR+Norm Fusion (#34899) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-02-20 13:37:31 -08:00
Lucas Wilkinson
aaefc58ee0 [CI] Revert PRs 34818 and 33600 (#34979) 2026-02-20 13:25:50 -08:00
Zhengxu Chen
f863994084 [compile] Fix torch.compile time discrepancy in logging. (#34912) 
Signed-off-by: zhxchen17 <zhxchen17@fb.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-02-20 08:47:14 -08:00
Zhengxu Chen
e4a5d8c653 [compile] Move torch_aot_compile directory under torch_compile_cache (#34831) 
Signed-off-by: zhxchen17 <zhxchen17@fb.com>
 2026-02-20 08:46:45 -08:00
Yanan Cao
a6d0299c75 [Kernel] [Helion] [6/N] Add num_tokens dimension to silu_mul autotuning and dispatching (#34185) 
Signed-off-by: Yanan Cao <gmagogsfm@gmail.com>
 2026-02-20 08:36:51 -08:00
Huamin Li
1fe462168c [perf] Avoid dtype promotion sync in mamba_get_block_table_tensor (#34870) 
Signed-off-by: Huamin Li <3ericli@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-02-20 06:21:56 -08:00
Flora Feng
ed31a020ee [Refactor] Extract Harmony streaming SSE event builders into streaming_events.py (#34909) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-02-20 06:20:46 -08:00
Cyrus Leung
f9ac19204f [V0 Deprecation] Remove unused MM placeholders in request output (#34944) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-20 06:19:23 -08:00
Vadim Gimpelson
59965affbd [BUGFIX] Fix _dummy_run missing prepare_inputs_event synchronization (#34866) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
 2026-02-20 05:54:27 -08:00
Kevin McKay
8de7c636cc [Bugfix][Hardware][AMD] Fix ROCM_AITER_FA speculative decoding support (#32877) 
Signed-off-by: c0de128 <kevin.mckay@outlook.com>
Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com>
 2026-02-19 22:25:46 -08:00
Frank Wang
059779231f [Minor] Add logging when using MXFP4 MXFP8 TRTLLM backend (#34916) 
Signed-off-by: frankwang28 <frank.wbb@hotmail.com>
Signed-off-by: Frank Wang <41319051+frankwang28@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-02-19 22:07:57 -08:00
tianshu-Michael-yu
ea37530b47 [Models] LFM2: Support LoRA (#34921) 
Co-authored-by: Piotr Mazurek <piotr635@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-02-19 22:07:23 -08:00
杨朱 · Kiki
07cab212f0 [Misc] Add deprecated environment variable utilities (#33677) 
Signed-off-by: carlory <baofa.fan@daocloud.io>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-02-19 21:33:25 -08:00
Varun Chawla
676f82ae81 Add validation to reject non-text content in system messages (#34072) 
Signed-off-by: Varun Chawla <varun_6april@hotmail.com>
 2026-02-19 21:30:33 -08:00
Elizabeth Thomas
81bfc21a6a [Model Bash]: Improve FP8 Oracle for Config Specific Kernel Selection (#34260) 
Signed-off-by: Elizabeth Thomas <email2eliza@gmail.com>
Signed-off-by: Robert Shaw <robertgshaw2-redhat@h100-02.nemg-001.lab.rdu2.dc.redhat.com>
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <robertgshaw2-redhat@h100-02.nemg-001.lab.rdu2.dc.redhat.com>
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com>
 2026-02-19 21:29:08 -08:00
Bowen Bao
d9e62c03eb [Quark] Fix MoE fp8 activation scale handling on mi300 (#34386) 
Signed-off-by: Bowen Bao <bowenbao@amd.com>
 2026-02-19 21:27:14 -08:00
Cyrus Leung
ac900c89bb [Refactor] Implement output type check in LLM (#34794) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-19 19:57:55 -08:00
Mark McLoughlin
76df6072ff [Core] Fix state names in pause_scheduler() (#34840) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2026-02-19 17:21:46 -08:00
Nick Hill
40b2f1c3d9 [Model Runner V2] Minor CPU optimizations (#34856) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-02-19 16:05:37 -08:00
Michael Goin
f72061a19a [UX] More descriptive reasons in is_supported_config for MoE (#34908) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-02-19 15:20:52 -08:00
Matthew Bonanni
662205d34e [Bugfix] Fix Basic Models Test (#34818) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-02-19 14:49:07 -08:00
Roger Wang
4fb8beefaa [Bugfix] Fix cutlass fp8 kernel on hopper for Qwen3.5 (#34914) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2026-02-19 13:34:55 -08:00
Wentao Ye
c683d11c94 [Refactor] Deprecate head_first for chunk_gated_delta_rule (#34263) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-02-19 13:23:49 -05:00
roikoren755
3eff45d793 Revert "[NemotronH] Do not force router to run in fp32 (#34582)" (#34808) 
Signed-off-by: Roi Koren <roik@nvidia.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-02-19 09:47:05 -08:00
Robert Shaw
4685a630a2 [Model Bash][DeepSeekR1] Remove Shared Expert Clone (#34344) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2026-02-19 07:56:14 -08:00
Eldar Kurtić
ee1d25f199 [Llama4,Quantization] Simplify and generalize logic for Q/K permutations in quantized self-attn layers (#34471) 
Signed-off-by: Your Name <you@example.com>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-02-19 07:55:41 -08:00
Linda
6fff24f30f [Bugfix] Qwen3.5 kv-scale weight remapping (#34719) 
Signed-off-by: Linda-Stadter <57756729+Linda-Stadter@users.noreply.github.com>
 2026-02-19 04:13:37 -08:00
Cyrus Leung
1391378861 [Bugfix] Fix edge case in UUID data parsing (#34884) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-19 02:24:30 -08:00
Tal Nir
f75b61a9e9 [Voxtral Realtime] Fix engine crash on empty multimodal embeddings (#34862) 
Signed-off-by: Tal Nir <tal@nervexneurotech.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-18 23:21:47 -08:00
Wei Zhao
7f51e93864 [Bug] Fix DeepSeek V3 weight loading caused by incorrect prefix (#34876) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
 2026-02-18 23:20:30 -08:00
Alex Brooks
4611af1663 [Bugfix] Add Quant Config to Llava Next Projector (#34847) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
 2026-02-18 23:18:23 -08:00
Manrique Vargas
ad5aa6bd9f fix(docs): fix typos in comments and docstrings (#34836) 
Signed-off-by: machov <mv1742@nyu.edu>
 2026-02-18 23:17:41 -08:00
Jaeyeon Kim(김재연)
9681068cf9 [Frontend] Fix reasoning_tokens for text-based parsers in Responses API (#33513) 
Signed-off-by: Jaeyeon Kim <anencore94@gmail.com>
 2026-02-18 23:16:41 -08:00