biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
13,302 Commits 2 Branches 140 Tags
57eae2f8918b42fc0fd3ed0b440b4eef6eae42e9
Commit Graph

4252 Commits

Author SHA1 Message Date
Kyungmin Lee
63ed2409e8 Add K-EXAONE-236B-A23B (#31621) 
Signed-off-by: lkm2835 <lkm2835@gmail.com>
Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: lgai-exaone <exaonemodels@lgresearch.ai>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-01-12 16:30:50 +00:00
Or Ozeri
9cddbdba6d OffloadingConnector: Add cpu_bytes_to_use configuration (#24498) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2026-01-12 15:00:43 +00:00
daniel-salib
d7b2e57097 [Frontend] Fix Flaky MCP Streaming Test (#32153) 
Signed-off-by: Daniel Salib <danielsalib@meta.com>
 2026-01-12 18:03:32 +08:00
Jiangyun Zhu
3df619ac94 [CI] fix test_concat_and_cache_mla_rope_fused (#32117) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2026-01-11 15:11:11 +00:00
Cyrus Leung
a374532111 [CI/Build] Separate out flaky responses API tests (#32110) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-11 05:01:12 -08:00
Or Ozeri
4c16ba617f [KVConnector] OffloadingConnector: Fix bug in handling of preemptions (#29870) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2026-01-11 08:05:36 +00:00
Matt
bde57ab2ed [Hardware][AMD][CI][Bugfix] Fix AMD Quantization test group (#31713) 
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
 2026-01-10 23:19:46 -08:00
Cyrus Leung
ef96fa3f1f [Benchmark][2/2] Use spline interpolation to tune SLA variables (#32095) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-10 20:27:27 -08:00
Or Ozeri
2a4dbe24ea [BugFix] Wait for compute before offloading KV to CPU (#31341) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2026-01-10 22:25:08 +00:00
shyeh25
1c46dea001 Revert "[Kernels][FI] Skip trtllm attention when num_kv_heads=1 (#308… (#31617) 
Signed-off-by: shyeh25 <206795756+shyeh25@users.noreply.github.com>
 2026-01-10 12:39:59 -08:00
Or Ozeri
028599739d [BugFix] scheduler: Fix resuming of preempted requests after async load (#31583) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2026-01-10 12:39:25 -08:00
Jeremy Teboul
07286ec5a6 [Bugfix] Fix integer overflow in Gemma3n audio processing (#31657) 
Signed-off-by: Jeremy Teboul <jeremyte@meta.com>
 2026-01-10 17:52:53 +08:00
Cyrus Leung
5f2385a4c8 [Benchmark][1/2] Generalize SLA criterion validation from binary flags to margins (#32075) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-10 07:11:03 +00:00
Andreas Karatzas
d83becd503 [ROCm][CI] Fix flaky test_function_calling_with_stream and reduce schema test examples (#32063) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-01-10 05:02:35 +00:00
Cyrus Leung
583a90e005 [Refactor] Separate sequence and token pooling types (#32026) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-10 04:53:24 +00:00
PatrykSaffer
80fead8bf6 Fuse RoPE and MLA KV-cache write (#25774) 
Signed-off-by: Patryk Saffer <patryk.saffer99@gmail.com>
Signed-off-by: PatrykSaffer <patryk.saffer@mistral.ai>
Co-authored-by: Patryk Saffer <patryk.saffer99@gmail.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-01-09 19:18:37 -08:00
Akshat Shrivastava
e45946bd91 feature/issac 0.2 (#31550) 
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-01-10 03:18:05 +00:00
Lucas Kabela
ea6d067a2a [Misc][LLaMa4] Compile LLaMa Vision Encoder (#30709) 
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
 2026-01-09 22:01:38 -05:00
Ning Xie
abd9224280 resolve pydantic error in startup benchmark (#31348) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2026-01-10 02:41:27 +00:00
Matthew Bonanni
2612ba9285 [1/N][Attention] Restructure attention: move files (#31916) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-01-09 13:10:24 -08:00
Andrew Xia
1f8b7c536b [responsesAPI] fix incomplete_messages for simple/parsable context (#31836) 
Signed-off-by: Andrew Xia <axia@fb.com>
Co-authored-by: Andrew Xia <axia@fb.com>
 2026-01-09 21:00:57 +00:00
Lucas Wilkinson
0a0aa07747 [Quant] Make static quant support all group shapes (#30833) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-01-09 12:49:27 -08:00
Runkai Tao
a4d5d663e2 Add unpermute-aware fused MoE path and small-batch fallback (#29354) 
Signed-off-by: Runkai Tao <rt572@physics.rutgers.edu>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-01-09 12:58:39 -07:00
Jeremy Teboul
657e9c0e18 [Fix] Introduce audio channels spec (#31595) 
Signed-off-by: Jeremy Teboul <jeremyte@meta.com>
 2026-01-09 19:34:51 +00:00
Andrew Xia
f32c629eb4 [Frontend][gpt-oss] Allow system message to overwrite model identity (#31737) 
Signed-off-by: lacora <hyelacora@gmail.com>
Signed-off-by: Andrew Xia <axia@fb.com>
Co-authored-by: lacora <hyelacora@gmail.com>
Co-authored-by: Andrew Xia <axia@fb.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-01-09 14:03:57 -05:00
Yifan Qiao
cd4a95e3aa [Feat][Core] Support multiple KV cache groups in Hybrid KV Coordinator (#31707) 
Signed-off-by: Yifan Qiao <yifanqiao@berkeley.edu>
 2026-01-09 10:53:20 -08:00
inkcherry
4505849b30 [ROCm][PD] add moriio kv connector. (#29304) 
Signed-off-by: inkcherry <mingzhi.liu@amd.com>
 2026-01-09 14:01:57 +00:00
Andreas Karatzas
e02706d2d2 [ROCm][CI][V1] Fix nixl_connector test failure and achieve CUDA parity in test_async_scheduling (#32000) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-01-09 20:48:32 +08:00
Xin Yang
e7b68f4d6c [Bugfix] Fix Triton FusedMoE LoRA (#30585) 
Signed-off-by: Xin Yang <xyangx@amazon.com>
 2026-01-09 11:46:59 +00:00
vllmellm
1a19e9cd87 [Bugfix][ROCm]Fix Qwen3-Next-80B-A3B-Thinking inference and optimize non-standard block size (544) support under rocm_atten (#31380) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
 2026-01-09 19:28:02 +08:00
Cyrus Leung
c8ed39b9dd [Model] Reorganize pooling layers (#31973) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-09 11:02:14 +00:00
Andreas Karatzas
020732800c [Bugfix] Fix OpenAPI schema test failures (#31921) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-01-09 10:56:20 +00:00
gnovack
bde38c11df fix lora moe sharding when rank < max_lora_rank (#31994) 
Signed-off-by: gnovack <gnovack@amazon.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-01-09 14:43:25 +08:00
Nick Hill
29ce48221c [Cleanup] Remove obsolete spec decoding compatibility logic (#32003) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-01-09 05:44:18 +00:00
TJian
7a05d2dc65 [CI] [ROCm] Fix tests/entrypoints/test_grpc_server.py on ROCm (#31970) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2026-01-09 12:54:20 +08:00
Divakar Verma
a1648c4045 [ROCm][CI] Fix test_token_classification.py::test_bert_models (#31993) 
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
 2026-01-09 04:04:33 +00:00
zhrrr
8ff4a99566 [Async][Feat] support apply penalty or bad_words for async + spec (#30495) 
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
Signed-off-by: izhuhaoran <izhuhaoran@qq.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-01-09 02:31:50 +00:00
daniel-salib
a4ec0c5595 [Frontend] Add MCP tool streaming support to Responses API (#31761) 
Signed-off-by: Daniel Salib <danielsalib@meta.com>
 2026-01-09 09:19:34 +08:00
Robert Shaw
0fa8dd24d2 [Bugfix] Fix Typo from NVFP4 Refactor (#31977) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2026-01-08 16:18:50 -08:00
Nick Hill
11cec296dd [BugFix] Add spec-decode-incompatible request param validation (#31982) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-01-09 00:08:21 +00:00
Robert Shaw
5825bbc1f7 [Quantization] Deprecate Long Tail of Schemes (#31688) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-01-08 19:07:45 -05:00
Lucas Wilkinson
6cdf015c3c [Misc] Fix Current vLLM config is not set. warnings, assert to avoid issues in the future (#31747) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2026-01-08 15:20:49 -08:00
bnellnm
e74698c27a [Misc][Refactor] Add FusedMoERouter object (#30519) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2026-01-08 20:52:55 +00:00
Cyrus Leung
aa125ecf0e [Frontend] Improve error message (#31987) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-08 20:07:03 +00:00
Michael Goin
87e07a6b46 Revert "feat(moe): Add is_act_and_mul=False support for Triton MoE kernels" (#31978) 2026-01-08 11:31:53 -08:00
yxing-bj
fe86be66c5 [Model] Support IQuestCoder model (#31575) 
Signed-off-by: yxing <yxing@iquestlab.com>
 2026-01-08 14:42:57 +00:00
Mary
7645bc524b [OpenAI] Fix tool_choice=required streaming when output has trailing extra data (#31610) 
Signed-off-by: maylikenoother <ogedengbemary19@gmail.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-01-08 21:01:42 +08:00
tianshu-Michael-yu
03fd76c570 [Model] Add LFM2-VL model support (#31758) 
Signed-off-by: Tianshu Yu <tianshuyu.formal@gmail.com>
Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-01-08 05:00:27 -08:00
Bijaya Dangol
59d260f5e4 [Model] Add Grok-2 (#31847) 
Signed-off-by: dangoldbj <dangoldbj23@gmail.com>
 2026-01-08 04:59:48 -08:00
Cyrus Leung
d1b6fe007f [Chore] Further cleanup pooler (#31951) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-08 02:16:21 -08:00