biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
12,929 Commits 2 Branches 140 Tags
0a7dd23754ed5e01303e0eb4e64ace5e70251f46
Commit Graph

12929 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Woosuk Kwon
0a7dd23754 [Model Runner V2] Add support for M-RoPE (#32143) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2026-01-12 13:37:43 -08:00
Woosuk Kwon
dec28688c5 [Model Runner V2] Minor refactor for logit_bias (#32209) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2026-01-12 13:08:30 -08:00
Vadim Gimpelson
9f430c94bd [BUGFIX] Add missed remaping of the names of fp8 kv-scale (#32199) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
 2026-01-12 20:42:06 +00:00
Nicolò Lucchesi
f8bd8394e3 [NIXL][Bugfix] Failure logging overhaul + early metadata free on failure (#32031) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-01-12 20:38:49 +00:00
Woosuk Kwon
ca81811bfe [Model Runner V2] Support logit_bias, allowed_token_ids, min_tokens (#32163) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2026-01-12 11:31:10 -08:00
Lucas Kabela
ad8818bb5e [Misc][BE] Type coverage for vllm/compilation [3/3] (#31748) 
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
 2026-01-12 19:24:38 +00:00
Nicolò Lucchesi
08e8e99ce7 [Misc] Change log level for batch queue log (#32192) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-01-12 18:59:31 +00:00
Or Ozeri
2be765b68a [BugFix] scheduler: Fix ordering preserving of skipped requests (#32173) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2026-01-12 18:39:38 +00:00
Roger Wang
16abe6b85a [Misc] Set default torch num threads for input processing (#31879) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2026-01-12 10:28:16 -08:00
Ilya Markov
1eb61ab34b [Refactor] EPLB rebalance algo to NumPy (#30697) 
Signed-off-by: ilmarkov <markovilya197@gmail.com>
 2026-01-12 18:13:23 +00:00
Kyungmin Lee
3d962d72ab [BugFix] fix FusedMoE.make_expert_params_mapping in EXAONE-MoE (#32196) 
Signed-off-by: lkm2835 <lkm2835@gmail.com>
 2026-01-12 10:00:45 -08:00
Matthew Bonanni
20228cb851 [3/N][Attention] Move AttentionMetadata-related code from utils.py to backend.py (#32054) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-01-12 09:13:56 -08:00
Cyrus Leung
7c0d3c5152 [Benchmark] Share data between SLA runs (#32184) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-13 01:12:22 +08:00
Nicolò Lucchesi
5b68107411 [Misc][PD] Fix get_attn_backend usage in transfer connectors (#31988) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-01-12 18:10:05 +01:00
Asaf Joseph Gardin
8fb2c135be [Bugfix] Fix stale SSM state for new Mamba requests scheduled as decode (#32118) 
Signed-off-by: Josephasafg <ajgard7@gmail.com>
 2026-01-12 17:02:38 +00:00
Cyrus Leung
8863c2b25c [Model] Standardize pooling heads (#32148) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-12 17:01:49 +00:00
danielafrimi
3f72639d36 [FIX] Add NO_MUL activation support for modular kernel path (#31528) 
Signed-off-by: dafrimi <dafrimi@nvidia.com>
Signed-off-by: <>
Co-authored-by: root <root@gpu-267.slurm-workers-slurm.slurm.svc.cluster.local>
Co-authored-by: root <root@gpu-537.slurm-workers-slurm.slurm.svc.cluster.local>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: root <root@pool0-01777.cm.cluster>
 2026-01-12 11:55:49 -05:00
Jaehyun An
6bc9c8473e [MODEL] New model support for kakaocorp/kanana-1.5-v-3b-instruct (#29384) 
Signed-off-by: Jaehyun An <steve.ai@kakaocorp.com>
 2026-01-12 16:39:02 +00:00
Kyungmin Lee
63ed2409e8 Add K-EXAONE-236B-A23B (#31621) 
Signed-off-by: lkm2835 <lkm2835@gmail.com>
Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: lgai-exaone <exaonemodels@lgresearch.ai>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-01-12 16:30:50 +00:00
Andy Zhang
95e53d907c doc: Update model references in supported_models.md (#32188) 2026-01-12 08:15:28 -08:00
TJian
0346396e94 [ROCm] [Bugfix] Fix order of mori build in Dockerfile.rocm_base (#32179) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2026-01-12 15:33:21 +00:00
Andy Zhang
e68b0dad8b doc: Update model name for Qwen3-Coder in documentation (#32185) 
Signed-off-by: Andy Zhang <xiazhang@microsoft.com>
 2026-01-12 07:10:50 -08:00
Or Ozeri
9cddbdba6d OffloadingConnector: Add cpu_bytes_to_use configuration (#24498) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2026-01-12 15:00:43 +00:00
Hongxin Xu
49e6b86c91 [Feature] Support recording expert indices for rollout router replay (#28284) 
Signed-off-by: xhx1022 <1737006628@qq.com>
Signed-off-by: Hongxin Xu <70438206+xhx1022@users.noreply.github.com>
Signed-off-by: arlenxu <arlenxu@tencent.com>
Co-authored-by: 22quinn <33176974+22quinn@users.noreply.github.com>
Co-authored-by: arlenxu <arlenxu@tencent.com>
 2026-01-12 06:23:04 -08:00
dtc
0565f1fdec [P/D] Refactor mooncake connector sender thread using async coroutines (#31573) 
Signed-off-by: Tianchen Ding <dtcccc@linux.alibaba.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@gmail.com>
 2026-01-12 12:35:35 +00:00
Isotr0py
9dbe1fe960 [Bugfix] Fix missing scale passing for encoder Triton Attention implementation (#32149) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-01-12 11:13:41 +00:00
RickyChen / 陳昭儒
a5f89ae296 [Doc] Add documentation for offline API docs feature (#32134) 
Signed-off-by: rickychen-infinirc <ricky.chen@infinirc.com>
 2026-01-12 10:33:48 +00:00
Jee Jee Li
05e8981234 [Doc] Improve LoRA docs (#32159) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-01-12 02:19:17 -08:00
XlKsyt
899541bdb1 [doc] fix broken links (#32158) 
Signed-off-by: minimAluminiumalism <caixuesen@outlook.com>
 2026-01-12 10:18:38 +00:00
daniel-salib
d7b2e57097 [Frontend] Fix Flaky MCP Streaming Test (#32153) 
Signed-off-by: Daniel Salib <danielsalib@meta.com>
 2026-01-12 18:03:32 +08:00
Andika Rachman
5e034f2e3d [cpu][bench] Add Fused MoE Micro Benchmark for CPU Backend (#32092) 
Signed-off-by: andikarachman <andika.rachman.y@gmail.com>
 2026-01-12 10:03:28 +00:00
Nicolò Lucchesi
22970c1626 [Misc] Disable default --ready-check-timeout-sec extra call in vllm bench (#30975) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-01-12 01:58:21 -08:00
Cyrus Leung
600aaab8d6 [Model] Remove incorrect SupportsPP from MTP models (#32150) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-12 01:19:30 -08:00
wang.yuqi
60446cd684 [Model] Improve multimodal pooling examples (#32085) 
Signed-off-by: wang.yuqi <noooop@126.com>
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-01-12 07:54:09 +00:00
Cyrus Leung
9101dc756c [Model] Avoid hardcoding pooling type (#32119) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-11 21:28:12 -08:00
Woosuk Kwon
025a32f9ed [Model Runner V2] Remove async barrier (#32083) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2026-01-11 20:24:30 -08:00
Woosuk Kwon
19504ac07f [Model Runner V2] Skip building deprecated fields in attn metadata (#32132) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2026-01-11 14:31:04 -08:00
Jiangyun Zhu
3df619ac94 [CI] fix test_concat_and_cache_mla_rope_fused (#32117) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2026-01-11 15:11:11 +00:00
Ning Xie
d74132ca3b fix offline inference chat response prompt (#32088) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2026-01-11 14:01:18 +00:00
maang
a34abc49b7 [FixBug] Improve exception string in tensorizer.py (#31680) 
Signed-off-by: maang <maang_h@163.com>
Signed-off-by: maang-h <55082429+maang-h@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-01-11 05:01:53 -08:00
rongfu.leng
d70249e2e9 [Misc] fix this log format not space (#32112) 
Signed-off-by: lengrongfu <lenronfu@gmail.com>
 2026-01-11 05:01:16 -08:00
Cyrus Leung
a374532111 [CI/Build] Separate out flaky responses API tests (#32110) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-11 05:01:12 -08:00
Isotr0py
cee7436a26 [Misc] Make scipy as optional audio/benchmark dependency (#32096) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-01-11 00:18:57 -08:00
Or Ozeri
4c16ba617f [KVConnector] OffloadingConnector: Fix bug in handling of preemptions (#29870) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2026-01-11 08:05:36 +00:00
Matt
bde57ab2ed [Hardware][AMD][CI][Bugfix] Fix AMD Quantization test group (#31713) 
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
 2026-01-10 23:19:46 -08:00
Fadi Arafeh
9103ed1696 [CPU][BugFix] Disable AOT Compile for CPU (#32037) 
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
 2026-01-10 23:15:49 -08:00
Laith Sakka
46eb30f519 make assume_32_bit_indexing configurable (#32044) 
Signed-off-by: Laith Sakka <lsakka@meta.com>
 2026-01-10 23:15:46 -08:00
Andy Liu
0dd63639be [MTP][GLM][Bugfix] Fixed .weight_scale loading logic that dropped MTP prediction accuracy with fp8+mtp (#32101) 
Signed-off-by: Andy Liu <andyliu@roblox.com>
 2026-01-10 23:14:54 -08:00
Cyrus Leung
ef96fa3f1f [Benchmark][2/2] Use spline interpolation to tune SLA variables (#32095) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-10 20:27:27 -08:00
Or Ozeri
2a4dbe24ea [BugFix] Wait for compute before offloading KV to CPU (#31341) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2026-01-10 22:25:08 +00:00