biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
15,459 Commits 2 Branches 140 Tags
73f48ce559e230fd0d738c52cb2e99bd0dd08754
Commit Graph

1120 Commits

Author SHA1 Message Date
Chauncey
cbe7d18096 [Misc] Rename think_start_str/think_end_str to reasoning_start_str/reasoning_end_str (#38242) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-04-01 09:56:45 -07:00
yzong-rh
dc0428ebb8 [NIXL][BUG] Fix Triton heterogeneous TP (#37940) 
Signed-off-by: Yifan <yzong@redhat.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2026-04-01 17:23:15 +02:00
Lucas Wilkinson
eb47454987 [Bugfix][MLA] Add logits size budget to sparse indexer prefill chunking (#36178) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-04-01 00:15:53 -04:00
HarshRathva
17b72fd1c8 Fix priority preemption regression test in scheduler (#37051) 
Signed-off-by: HarshRathva <harshrathvaai@gmail.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
 2026-04-01 06:36:12 +03:00
Yifan Qiao
91e4521f9f [Feat][v1] Simple yet General CPU KV Cache Offloading (#37160) 
Signed-off-by: Yifan Qiao <yifanqiao@berkeley.edu>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
 2026-03-31 17:58:37 -07:00
Wentao Ye
856589ed9a [Refactor] Remove dead code in kv connector and model runner (#38383) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-03-31 17:05:23 -04:00
Matthew Bonanni
757068dc65 [Bugfix][Async] Fix async spec decoding with hybrid models (#38556) 
Signed-off-by: SandishKumarHN <sandishkumarhn@gmail.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: SandishKumarHN <sandishkumarhn@gmail.com>
 2026-03-31 11:08:54 -04:00
wliao2
4dfad17ed1 replace cuda_device_count_stateless() to current_platform.device_count() (#37841) 
Signed-off-by: Liao, Wei <wei.liao@intel.com>
Signed-off-by: wliao2 <wei.liao@intel.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-03-31 22:32:54 +08:00
Nicolò Lucchesi
7430389669 [Bugfix][CI] Skip flaky test_eagle test (#38566) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-03-31 09:42:37 -04:00
Matthew Bonanni
7d65463528 [WIP][CI][Bugfix] Fix test_run_eagle_dp (#38584) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-03-31 12:30:25 +02:00
Benjamin Chislett
494636b29d [Feat][Spec Decode] DFlash (#36847) 
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
 2026-03-30 15:03:15 -04:00
Chendi.Xue
3b1dbaad4e [HMA]Fix corner case when hybrid page_size can not be evenly divided issue (blk_size=64,tp=4) (#37467) 
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2026-03-30 16:47:30 +00:00
Collin McCarthy
1031c84c36 Fix ambiguous num_blocks for hybrid attn mamba (#37236) 
Signed-off-by: Collin McCarthy <cmccarthy@nvidia.com>
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
Co-authored-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
 2026-03-30 11:09:45 +00:00
Nicolò Lucchesi
cc06b4e86b [Mamba][Bugfix] Raise on insufficient cache blocks instead of silently capping cudagraph sizes (#38270) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-03-30 09:41:50 +00:00
Andreas Karatzas
4f2ed5fddb [ROCm][CI] Enable hybrid chunked prefill test (#38317) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-03-30 10:30:26 +08:00
Wentao Ye
995dea1354 [Perf] Remove redundant device copies for CPU-only pooling token IDs, 48.9% E2E throughput improvement (#38139) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-03-29 18:12:50 +00:00
yzong-rh
6dad4c5722 [Test] Fix flaky race condition in test_abort_final_step (#38414) 
Signed-off-by: Yifan <yzong@redhat.com>
 2026-03-28 09:06:56 +00:00
dtc
6287e7fa20 [P/D] Mooncake: Add unit tests and minor fixes for mooncake connector (#36946) 
Signed-off-by: Tianchen Ding <dtcccc@linux.alibaba.com>
 2026-03-27 09:26:40 +01:00
Or Ozeri
7cc302dd87 [kv_offload+HMA][7/N]: Support register_kv_caches for hybrid models (#37853) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2026-03-27 08:38:33 +03:00
Giancarlo Delfin
c32e97602d [Model Runner V2] Enable forcing a specific acceptance rate during rejection sampling (#38045) 
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
 2026-03-26 13:38:12 -07:00
Woosuk Kwon
144030c84e Relocate Encoder CUDA graph manager (#38116) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-03-25 20:52:12 -07:00
Cyrus Leung
ba2f0acc2d [Misc] Reorganize inputs (#35182) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-03-25 10:22:54 -07:00
Wentao Ye
1bf2ddd0ee [Refactor] Rename WAITING_FOR_FSM to WAITING_FOR_STRUCTURED_OUTPUT_GRAMMAR (#38048) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-03-25 11:41:44 -04:00
Wentao Ye
d7e93e13fb [Feature] EPLB Support for GPU Model Runner v2 (#37488) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-03-25 08:16:39 -07:00
Andrii Skliar
cd7643015e [Feature] Support per-draft-model MoE backend via --speculative-config (#37880) 
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Signed-off-by: [Andrii Skliar] <askliar@nvidia.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
 2026-03-25 14:31:52 +00:00
Harry Mellor
d215d1efca [Mypy] Better fixes for the mypy issues in vllm/config (#37902) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-25 06:14:43 -07:00
Gregory Shtrasberg
189ddefbfd [ROCm] Attention selector reordering (#36702) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
Co-authored-by: Micah Williamson <micah.williamson@amd.com>
 2026-03-25 17:42:56 +08:00
Sungjae Lee
4731884796 [Feature] limit thinking tokens (hard limit) (#20859) 
Signed-off-by: Sungjae Lee <33976427+llsj14@users.noreply.github.com>
Signed-off-by: Sungjae Lee <sung-jae.lee@navercorp.com>
Signed-off-by: Chauncey <chaunceyjiang@gmail.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-24 09:53:07 -07:00
Ronen Schaffer
e3c6c10cad [KV Offload] Refactor CPU offloading: pluggable CachePolicy, remove Backend abstraction, restructure into cpu/ package (#37874) 
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
 2026-03-24 07:02:51 +02:00
Wentao Ye
c59a132f96 [V0 Deprecation] Refactor kv cache from list to element (#37487) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-03-23 20:10:11 -07:00
Ranran
dc6908ac6a [Bugfix] Register VLLM_BATCH_INVARIANT in envs.py to fix spurious unknown env var warning (#35007) 
Signed-off-by: Ranran <1012869439@qq.com>
Signed-off-by: Ranran <hzz5361@psu.edu>
Signed-off-by: ran <hzz5361@psu.edu>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-03-23 18:31:14 -04:00
Matthew Bonanni
fafe76b4af [Async][Spec Decoding] Zero-bubble async scheduling + spec decoding (#32951) 
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
Co-authored-by: zhrrr <43847754+izhuhaoran@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
 2026-03-23 15:37:22 -04:00
Nicolò Lucchesi
1cbbcfe8a3 [CI][PD] Add Hybrid SSM integration tests to CI (#37657) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-03-23 23:58:19 +08:00
Baorun (Lauren) Mu
f85e479e66 [Feature] ViT Full CUDA Graph (#35963) 
Signed-off-by: Baorun Mu <bmu@nvidia.com>
 2026-03-23 13:01:10 +08:00
Wentao Ye
eaf4978621 [Test] Only Run MLA model when user explicitly set for batch invariance (#37719) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-03-22 09:09:12 -04:00
Andreas Karatzas
66f927f205 [Bugfix] Fix pooling non-determinism from pinned prompt_lens aliasing (#37775) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-03-22 03:22:24 +00:00
Brandon Pelfrey
80b70884eb Add tensor IPC transfer mechanism for multimodal data (#32104) 
Signed-off-by: Brandon Pelfrey <bpelfrey@nvidia.com>
Signed-off-by: Brandon Pelfrey <brandonpelfrey@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-03-21 20:10:20 +00:00
Francesco Fusco
298e510848 [Hybrid] calling get_mamba_groups() once at MambaCopyBuffers.create() (#37318) 
Signed-off-by: Francesco Fusco <ffu@zurich.ibm.com>
 2026-03-21 09:29:43 +00:00
Santino Ramos
85f671b8e1 [Model Runner V2] Support Streaming Inputs (#37028) 
Signed-off-by: Santino Ramos <elsantinoramos@gmail.com>
 2026-03-20 20:42:25 +00:00
Lucas Wilkinson
e1d85e5c24 [Attention] Support distinguishing between short extends and decodes (#37303) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-03-20 10:49:36 -07:00
Flora Feng
b4c1aef21c [Refactor] Relocate tests from tests/v1/entrypoints/ to tests/entrypoints/ (#37500) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-03-20 02:50:34 -07:00
Flora Feng
9040151fe1 [V0 Deprecation] Deprecate --disable-frontend-multiprocessing (#37612) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-03-20 11:31:43 +08:00
tianshu-Michael-yu
269bf46d99 fix: disambiguate multimodal prefix cache keys (#36708) 
Signed-off-by: tianshu.yu <tianshuyu.formal@gmail.com>
 2026-03-20 10:33:20 +08:00
zhanqiuhu
d49f273144 [SSM/Mamba] Follow-up: N-1 prefill for P/D disaggregation (#37310) 2026-03-19 08:22:00 +01:00
Thillai Chithambaram
828f862acb [Bugfix] Expand quantization method support in perf metrics (#37231) 
Signed-off-by: Thillai Chithambaram <thillaichithambaram.a@gmail.com>
 2026-03-18 23:54:19 +00:00
Andy Lo
577df69b26 [Bugfix] Fix KV scales inconsistency in fp8 MLA & FlashInfer kv_cache_dtype "auto" leading to gibberish (#37054) 
Signed-off-by: Andy Lo <andy@mistral.ai>
 2026-03-18 23:07:29 +00:00
Wentao Ye
0d81a1fe61 [V0 Deprecation] Deprecate virtual engine (#37195) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-03-18 14:30:14 -07:00
Or Ozeri
5dd8df0701 [kv_offload+HMA][2/N]: Support multiple KV groups in GPULoadStoreSpec (#36642) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2026-03-18 19:26:40 +02:00
Or Ozeri
525f2eeb0b [kv_offload+HMA][6/N]: Split offloading_connector.py (#37405) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2026-03-18 14:42:46 +01:00
Andy Lo
98b09ddc27 [NIXL][Bugfix] metrics & testing minor bug (#36051) 
Signed-off-by: Andy Lo <andy@mistral.ai>
 2026-03-18 14:39:14 +01:00