biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
9,940 Commits 2 Branches 140 Tags
4e33a7ea85cb702090c07fb7a8ebdbf44c472f5c
Commit Graph

9940 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Naman Lalit
4e33a7ea85 [Bugfix] Optimize CpuGpuBuffer initialization (#25447) 
Signed-off-by: Naman Lalit <nl2688@nyu.edu>
 2025-09-27 08:07:36 +08:00
Bram Wasti
dc48ba0c75 Kernel-override Determinism [1/n] (#25603) 
Signed-off-by: Bram Wasti <bwasti@meta.com>
 2025-09-26 16:59:09 -07:00
Sage Moore
4778b42660 Reduce the Cuda Graph memory footprint when running with DBO (#25779) 
Signed-off-by: Sage Moore <sage@neuralmagic.com>
 2025-09-26 22:29:56 +00:00
qizixi
c70ac4b8ff [spec decode] Consolidate speculative decode method name for MTP (#25232) 
Signed-off-by: zixi-qi <qizixi@meta.com>
 2025-09-26 22:27:05 +00:00
Michael Goin
cf89202855 [CI] Fix FlashInfer AOT in release docker image (#25730) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-09-26 14:11:40 -07:00
fhl2000
f075693da7 [V1] address post issues related to #20059 (part 1) (#23046) 
Signed-off-by: fhl2000 <63384265+fhl2000@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2025-09-26 15:58:19 -04:00
Michael Goin
f708bd4904 [CI] Add E2E Blackwell Quantized MoE Test (#25723) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-09-26 12:23:00 -07:00
Michael Goin
0002b7f0d1 [Docs] Add Toronto Meetup (#25773) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-09-26 12:00:46 -07:00
Frank Wang
11aafd9886 [Bugfix] Improve GLM4 MoE Reasoning Parser's is_reasoning_end Condition (#25355) 
Signed-off-by: frankwang28 <frank.wbb@hotmail.com>
Signed-off-by: Frank Wang <41319051+frankwang28@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2025-09-26 11:54:00 -07:00
Clouddude
b761df963c [Doc]: improve CPU(x86) build-wheel-from-source section (#25617) 
Signed-off-by: Kosseila (CloudThrill) <klouddude@gmail.com>
v0.11.0rc1 v0.11.1rc0 		2025-09-26 10:26:33 -07:00
阿丹(adan)
33f6aaf972 Eagle3 that supports the Minicpm3 model (#24243) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: liudan <adan@minicpm.com>
Co-authored-by: liudan <liudan@qq.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Lucia Fang <116399278+luccafong@users.noreply.github.com>
 2025-09-26 10:04:57 -07:00
Jiangyun Zhu
56aafa8c0b [Misc] fix unique_filepath (#25732) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2025-09-26 16:56:15 +00:00
Seiji Eicher
8d52f2b3a7 [ray][metrics] Replace ':' with '_' for OpenTelemetry compatibility in Ray (#25439) 
Signed-off-by: Seiji Eicher <seiji@anyscale.com>
Signed-off-by: Seiji Eicher <58963096+eicherseiji@users.noreply.github.com>
Co-authored-by: Rui Qiao <161574667+ruisearch42@users.noreply.github.com>
 2025-09-26 09:43:30 -07:00
Lucas Wilkinson
984d18498a [BugFix] Fix using dbo_decode_token_threshold always (and ignoring dbo_prefill_token_threshold) (#25622) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-09-26 16:22:49 +00:00
Isotr0py
d4d9899860 [Quantization] Add field to skip unquantized modules for GPTQ config (#25455) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-09-26 15:47:41 +00:00
Cyrus Leung
db1e42f627 [CI/Build] Fix some V1 tests not being run (#25569) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-09-26 20:52:36 +08:00
Cyrus Leung
bc9d7b5595 [CI/Build] Split up Distributed Tests (#25572) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-09-26 14:49:33 +02:00
wang.yuqi
fe6b19c314 [Bugfix] Properly abort pooling request. (#25734) 
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-09-26 05:47:34 -07:00
Chauncey
2827b3f4a3 [CI] Fix test_shared_storage_connector_hashes (#25748) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-09-26 20:46:17 +08:00
Chih-Chieh Yang
2b6b1d7809 [Model] Mamba2 varlen refactor (#21467) 
Signed-off-by: Chih-Chieh-Yang <7364402+cyang49@users.noreply.github.com>
Co-authored-by: RishiAstra <40644327+RishiAstra@users.noreply.github.com>
 2025-09-26 11:31:14 +00:00
Cyrus Leung
633f943e30 [Doc] Update Batch-level DP docs (#25757) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-09-26 02:37:40 -07:00
Xu Wenqing
b03b1b97f6 Support LongCat-Flash-Chat tool call (#24083) 
Signed-off-by: 许文卿 <xwq391974@alibaba-inc.com>
 2025-09-26 09:25:39 +00:00
Sage Moore
dfb9af2014 [Bugfix] Fix Shared Expert/Zero expert code in FusedMoE.process_chunk (#25698) 
Signed-off-by: Sage Moore <sage@neuralmagic.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2025-09-26 01:25:28 -07:00
yyzxw
19f76ee68e [misc] refactor speculative config (#25657) 
Signed-off-by: zxw <1020938856@qq.com>
 2025-09-26 01:22:06 -07:00
Icey
dd70437a4f Remove cuda hard-code in compute_causal_conv1d_metadata (#25555) 
Signed-off-by: Icey <1790571317@qq.com>
 2025-09-26 01:19:20 -07:00
Tao He
99b3a504c5 [Qwen3-Next][GDN] fixes cuda graph capturing bug in GDN metadata and a stride bug in causal_conv_1d. (#25743) 
Signed-off-by: Tao He <linzhu.ht@alibaba-inc.com>
 2025-09-26 01:18:58 -07:00
Iceber Gu
6e30010d2f fix: print outputt offline_inference/base/chat.py example (#25744) 
Signed-off-by: Iceber Gu <caiwei95@hotmail.com>
 2025-09-26 01:18:24 -07:00
xaguilar-amd
52621c8f5c [Harware][AMD][Model] Triton MoE tuning configs for GLM-4.5 for MI300X (#25703) 
Signed-off-by: xaguilar <Xavier.AguilarFruto@amd.com>
 2025-09-26 01:18:20 -07:00
Andrew Sansom
d48f4d6daf perf: Avoid copying inputs_embeds tensors to GPU unless prompt_embeds is enabled (#25739) 
Signed-off-by: Andrew Sansom <andrew@protopia.ai>
 2025-09-26 01:18:09 -07:00
Andrew Sansom
e84e0735c7 fix: revert cast to cpu in MsgpackEncoder._encode_tensor to avoid hidden performance regressions (#25738) 
Signed-off-by: Andrew Sansom <andrew@protopia.ai>
 2025-09-26 01:18:05 -07:00
yitingdc
3edf87d25f [CI/Build] fix doc build warning: Failed to get 'name: description' pair (#25733) 
Signed-off-by: yiting.jiang <yiting.jiang@daocloud.io>
 2025-09-26 01:18:02 -07:00
Eugene Khvedchenya
392edee34a EVS Support (Video tokens pruning) (#22980) 
Signed-off-by: Eugene Khvedchenia <ekhvedchenia@nvidia.com>
Signed-off-by: Eugene Khvedchenya <ekhvedchenya@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2025-09-26 11:54:54 +08:00
Nick Hill
983056e456 [Misc] Remove unnecessary memoryviews in shm_broadcast.py (#25721) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-09-26 03:11:44 +00:00
Russell Bryant
13dd93c667 [Core] Force PIECEWISE CUDAGraph mode for encoder-decoder (#25701) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-09-25 18:21:56 -07:00
Aleksandr Malyshev
53a30845be Llamas 3.1 405B fp4 changes upstreaming from 355_wip (#25135) 
Signed-off-by: Aleksandr Malyshev <maleksan@amd.com>
Co-authored-by: Aleksandr Malyshev <maleksan@amd.com>
Co-authored-by: Doug Lehr <douglehr@amd.com>
 2025-09-25 19:16:53 -06:00
Nick Hill
8b77328ffe [Misc] Don't log shm dequeue delay warning on worker side (#25720) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-09-26 01:08:30 +00:00
Wentao Ye
9fe4c2bdb9 [Refactor] Remove DeepGEMM OP Register (#25710) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-09-25 20:13:41 -04:00
Shu Wang
081b5594a2 Fix routing_bias dtype (#25711) 
Signed-off-by: Shu Wang. <shuw@nvidia.com>
 2025-09-25 23:35:14 +00:00
tomeras91
57329a8c01 [Model] rename NemotronH_Nano_VL -> NemotronH_Nano_VL_V2 (#25708) 
Signed-off-by: Tomer Asida <57313761+tomeras91@users.noreply.github.com>
 2025-09-25 16:10:29 -07:00
Zhuohan Li
8c435c9bce [Core] Enable command line logging for LLMEngine (#25610) 
Co-authored-by: Ye (Charlotte) Qi <yeq@meta.com>
Signed-off-by: Zhuohan Li <zhuohan123@gmail.com>
 2025-09-25 15:31:17 -07:00
Ekagra Ranjan
e71b8e210d [Spec Decode] Add Batch Parallel Ngram. Upto 8x lower overhead. (#24986) 
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-09-25 15:22:03 -07:00
Cyrus Leung
89fa54e6f7 [Optimization] Use a cheaper cache key in get_model_architecture (#25682) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-09-25 17:54:20 -04:00
Cyrus Leung
3d54bdcb73 [Optimization] Streamline InputPreprocessor (#25702) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-09-25 21:06:49 +00:00
Cyrus Leung
6b0fcbbf43 [Misc] Simplify test_argsort_mm_positions (#25690) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-09-25 18:23:01 +00:00
Jee Jee Li
0fa673af4c [V0 deprecation] Clean up LoRA (#25686) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-25 18:12:33 +00:00
Matthew Bonanni
3468f17ebe [V0 deprecation] Remove _VLLM_V1 suffixes from attention backend names (#25489) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
 2025-09-25 17:37:50 +00:00
Isotr0py
71b25b0d48 [V0 deprecation] Clean up V0 fallback in compilation config (#25675) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-09-25 17:29:51 +00:00
Cyrus Leung
0ea80c87d9 [Model] Define merge_by_field_config MM interface (#25676) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-09-25 17:13:07 +00:00
Tao Hui
b8d9e4a326 [Model] Add optional parameter to reasoning parser constructor (#25554) 
Signed-off-by: taohui <taohui3@gmail.com>
Signed-off-by: Tao Hui <taohui3@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-26 01:12:50 +08:00
Lucas Wilkinson
13cc7f5370 [BugFix] Fix DBO hang (#25625) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-09-25 17:04:48 +00:00