biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
13,302 Commits 2 Branches 140 Tags
57eae2f8918b42fc0fd3ed0b440b4eef6eae42e9
Commit Graph

4252 Commits

Author SHA1 Message Date
Yifan Qiao
f0d005864a [Fix] prefix cache hit rate == 0 bug with gpt-oss style models (#33524) 
Signed-off-by: Yifan Qiao <yifanqiao@berkeley.edu>
(cherry picked from commit a01ef3fa51)
 2026-02-02 10:31:50 -08:00
Robert Shaw
94cbe0a328 [Nightly CI] Remove CT Model (#33530) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
(cherry picked from commit 318b120766)
 2026-02-02 02:17:42 -08:00
csy0225
8b45c58fe9 [Models] Step-3.5-Flash (#33523) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: i-zhangmingming <i-zhangmingming@stepfun.com>
Co-authored-by: xiewuxun <xiewuxun@stepfun.com>
Co-authored-by: zetaohong <i-hongzetao@stepfun.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
(cherry picked from commit c3b40dc3e7)
 2026-02-02 02:16:23 -08:00
Luka Govedič
2915268369 [fix][torch.compile] Fix cold-start compilation time increase by adding kv cache update to splitting ops (#33441) 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Richard Zou <zou3519@gmail.com>
(cherry picked from commit 15f40b20aa)
 2026-02-02 00:14:07 -08:00
Gregory Shtrasberg
5f45b0b7e0 [Bugfix][ROCm] Fixing the skinny gemm dispatch logic from #32831 (#33366) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
(cherry picked from commit 31aedfe7d6)
 2026-02-02 00:13:45 -08:00
wang.yuqi
1ed963d43a [Bugfix] Fix Qwen3-VL-Reranker load. (#33298) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
(cherry picked from commit abb34ac43a)
 2026-02-02 00:13:12 -08:00
Or Ozeri
fe18ce4d3f Revert "Enable Cross layers KV cache layout at NIXL Connector (#30207)" (#33241) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
(cherry picked from commit 2e8de86777)
 2026-01-28 11:44:59 -08:00
Roger Wang
5042815ab6 [Models] Kimi-K2.5 (#33131) 
Signed-off-by: wanglinian <wanglinian@stu.pku.edu.cn>
Signed-off-by: wangln19 <96399074+wangln19@users.noreply.github.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: wanglinian <wanglinian@stu.pku.edu.cn>
Co-authored-by: wangln19 <96399074+wangln19@users.noreply.github.com>
Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
(cherry picked from commit b539f988e1)
 2026-01-28 02:16:28 -08:00
Chauncey
afb390ab02 [CI] Fix AssertionError: MCP tool call not found in output_messages (#33093) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
(cherry picked from commit a2393ed496)
 2026-01-28 02:16:14 -08:00
Cyrus Leung
11b556878b [Refactor] Use data parser for matching data items to multi-modal UUIDs (#32955) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-26 15:00:28 +08:00
Robert Shaw
254db42ede [Tests] Remove Duplicates (#33032) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2026-01-26 05:23:54 +00:00
JJJYmmm
7e67df5570 [Bugfix] fix encoder cache hang in Qwen3VL (#32684) 
Signed-off-by: JJJYmmm <92386084+JJJYmmm@users.noreply.github.com>
Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-01-25 05:17:31 +00:00
Roberto L. Castro
fcb9df99bd [Perf][Kernel] Optimize FP4 quantization kernels (SM100F) (#32520) 
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
 2026-01-24 18:45:27 -07:00
Joshua Deng
91601ff478 [Feature] add session based streaming input support to v1 (#28973) 
Signed-off-by: Joshua Deng <joshuakdeng@gmail.com>
Signed-off-by: Patrick von Platen <patrick.v.platen@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-01-24 12:06:28 -08:00
7. Sun
cd775bdbe0 [Tests] Replace flaky sleep with polling in test_background_cancel (#32986) 
Signed-off-by: 7. Sun <jhao.sun@gmail.com>
 2026-01-24 16:39:07 +00:00
7. Sun
0ccecf8833 [Tests] Standardize RNG seed utility across test files (#32982) 
Signed-off-by: 7. Sun <jhao.sun@gmail.com>
 2026-01-24 06:47:14 +00:00
7. Sun
0b9a735e11 [Tests] Clarify pytest skip reasons with actionable context (#32981) 
Signed-off-by: 7. Sun <jhao.sun@gmail.com>
 2026-01-24 06:38:50 +00:00
ElizaWszola
a28b94e6ef [Performance] Split FlashAttn attention and cache update (#25954) 
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Luka Govedič <luka.govedic@gmail.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Varun Sundar Rabindranath <varunsundar08@gmail.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Luka Govedič <luka.govedic@gmail.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>
 2026-01-23 17:28:06 -08:00
dolpm
0118cdcc02 [fix] add VLLM_OBJECT_STORAGE_SHM_BUFFER_NAME to compile factors (#32912) 
Signed-off-by: dolpm <34420038+dolpm@users.noreply.github.com>
 2026-01-23 22:53:10 +00:00
Michael Goin
4561f13985 [Refactor] Rename gptq_marlin to marlin to match MoE (#32952) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-01-23 16:48:12 -05:00
Lucas Wilkinson
3a41459501 [cudagraphs] Refactor cudagraph capture loop (#32946) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-01-23 13:22:20 -07:00
Harry Huang
5206e5e28c [V1][Hybrid] Mamba Prefix Caching with align mode (#30877) 
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
 2026-01-23 09:56:48 -08:00
Luka Govedič
bbbd696af9 [torch.compile][CI] Add back attn fusion on hopper/ada (#32940) 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
 2026-01-23 16:49:20 +00:00
sangbumlikeagod
9b77bb790d [Frontend] add logprob, compression_rate to 'verbose_json' features (#31059) 
Signed-off-by: sangbumlikeagod <oironese@naver.com>
Signed-off-by: sangbumlikeagod <98077576+sangbumlikeagod@users.noreply.github.com>
 2026-01-23 16:35:13 +00:00
Matt
305e53ade8 [Hardware][AMD][CI][Bugfix] Fix Kernels Attention Cache test (#32904) 
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
 2026-01-23 16:24:26 +00:00
Xin Yang
90c2007932 [Bugfix] Disable tma_aligned_scales in test_fusions_e2e (#32916) 
Signed-off-by: Xin Yang <xyangx@amazon.com>
 2026-01-23 14:34:30 +00:00
Fadi Arafeh
aac0b817fa [CPU Backend][BugFix] Fix failing CPU MoE test (#32876) 
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
 2026-01-23 12:06:51 +00:00
wang.yuqi
05f3d714db [Frontend][3/n] Make pooling entrypoints request schema consensus | EmbedRequest & ClassifyRequest (#32905) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-01-23 12:03:44 +00:00
Patrick von Platen
3f3f89529d [Voxtral] Add new streaming arch (#32861) 
Signed-off-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-01-23 12:41:52 +01:00
Karan Bansal
fa6e599a61 [Bugfix] Fix _CPU_MOE_ACT AssertionError when vLLM config not set (#32777) 
Signed-off-by: Karan Bansal <karanb192@gmail.com>
 2026-01-23 08:22:37 +00:00
Luka Govedič
5e4e0e51f4 [torch.compile] Compile CustomOp.forward_native for SiluAndMul and QuantFP8 to avoid raw torch ops inside opaque custom ops (#32806) 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-01-22 19:52:26 -08:00
bnellnm
dc917cceb8 [MoE Refactor] Move select_experts from FusedMoEQuantMethod -> FusedMoE (#31996) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2026-01-22 18:21:35 -05:00
Xin Yang
d08b356ee0 [Perf] Create TMA-aligned input scale tensor for DeepGemm on Hopper (#32619) 
Signed-off-by: Xin Yang <xyangx@amazon.com>
 2026-01-22 15:47:04 -05:00
Eldar Kurtić
44f08af3a7 Add llmcompressor fp8 kv-cache quant (per-tensor and per-attn_head) (#30141) 
Signed-off-by: Eldar Kurtic <8884008+eldarkurtic@users.noreply.github.com>
Signed-off-by: eldarkurtic <8884008+eldarkurtic@users.noreply.github.com>
 2026-01-22 13:29:57 -07:00
David Ramon Prados
3a63be0faa Support custom URI schemes and trace handlers for profiler (#32393) 2026-01-22 09:45:40 -08:00
Matt
c517d8c934 [Hardware][AMD][CI][Bugfix] Fix regressions from deprecated env vars (#32837) 
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
 2026-01-23 00:59:15 +08:00
Maximilien de Bayser
ff365eea94 Support bge-m3 sparse embeddings and colbert embeddings (#14526) 
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
 2026-01-22 23:52:57 +08:00
Isotr0py
444e2e7e1f [Misc] Bump opencv-python dependecy version to 4.13 (#32668) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-01-22 15:51:15 +00:00
Richard Zou
654a71fc3c [torch.compile] Improve Cold Start for MoEs (#32805) 
Signed-off-by: Richard Zou <zou3519@gmail.com>
 2026-01-22 10:44:40 -05:00
Lucas Kabela
15e302dfce [Misc][BE] Turn on strict type coverage for vllm/compilation (#31756) 
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
 2026-01-22 15:12:26 +00:00
Cyrus Leung
d117a4d1a9 [Frontend] Introduce Renderer for processing chat messages (using ModelConfig) (#30200) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-22 12:44:22 +00:00
Or Ozeri
421012b63a OffloadingConnector: Support kernel_block_size != block_size (#30692) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2026-01-22 12:30:04 +00:00
Nicolò Lucchesi
ea6102b85d [Bugfix] Fix Whisper/encoder-decoder GPU memory leak (#32789) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-01-22 10:50:37 +00:00
wang.yuqi
328cbb2773 [Frontend][2/n] Make pooling entrypoints request schema consensus | ChatRequest (#32574) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-01-22 10:32:44 +00:00
liranschour
64e3d67ac0 Enable Cross layers KV cache layout at NIXL Connector (#30207) 
Signed-off-by: Liran Schour <lirans@il.ibm.com>
Signed-off-by: liranschour <liranschour@users.noreply.github.com>
Co-authored-by: Or Ozeri <or@ozery.com>
 2026-01-22 10:12:58 +00:00
Alex Sun
49a1262267 [AMD][ROCm] MoRI EP: a high-performance all2all backend (#28664) 
Signed-off-by: Alex Sun <alex.s@amd.com>
 2026-01-22 16:33:18 +08:00
Cyrus Leung
2b8a38b6d6 [Model] Extend collect_children and no_init_weights contexts (#32757) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-22 08:20:27 +00:00
Andreas Karatzas
a810299838 [ROCm][CI][Docs] Add comment explaining TRITON_ATTN fallback for ROCm (#32835) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-01-21 22:11:09 -08:00
Andreas Karatzas
eb1629da24 [ROCm][CI] Fix AITER test flakiness by using explicit attention backend (#32346) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: Matthew Wong <Matthew.Wong2@amd.com>
 2026-01-22 13:55:25 +08:00
Micah Williamson
019e2c3b7c [ROCm][CI] Lower Acceptance Len Threshold For test_draft_model_quantization (#32731) 
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
 2026-01-22 05:47:33 +00:00