biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
10,620 Commits 2 Branches 140 Tags
be4445072c4e1e5e3a2ebf0552e432fc86f137ca
Commit Graph

10620 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Zebing Lin
be4445072c [Fix][Spec Decode] Fix llama4 draft loading with different quantization (#27136) 
Signed-off-by: linzebing <linzebing1995@gmail.com>
 2025-10-20 23:19:00 -07:00
Benjamin Chislett
f381cf2302 [Bugfix] Fix broken MTP weight loading for FP8 KV Scales (#27227) 
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
 2025-10-20 22:51:44 -07:00
Varun Sundar Rabindranath
5ff5d94e77 [Bugfix] Fix gpt-oss w4a8 DP/EP on B200 (#26729) 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-10-21 01:51:14 -04:00
Shu Wang
f95da13c3d [ModelOpt] Load w13/w2_input_scale for all experts, nvfp4 (#26135) 
Signed-off-by: Shu Wang <shuw@nvidia.com>
Signed-off-by: Shu Wang. <shuw@nvidia.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-10-21 01:50:31 -04:00
Po-Han Huang (NVIDIA)
aef368aa08 [BugFix] GPT-OSS Attention DP + MoE TP weight loading issue (#24032) 
Signed-off-by: Po-Han Huang <pohanh@nvidia.com>
 2025-10-21 04:03:47 +00:00
Chen Wu
5f6cbf60d6 [Feature][Kernel]FusedMoE LoRA (#21229) 
Signed-off-by: wuchen <cntryroa@gmail.com>
Signed-off-by: banjuede <lmklhc@163.com>
Signed-off-by: Chen Wu <cntryroa@gmail.com>
Signed-off-by: Danielle Robinson <dmmaddix@amazon.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: bk-201 <joy25810@foxmail.com>
Co-authored-by: wuchen <wuchen@zetyun.com>
Co-authored-by: Nathan Van Gheem <vangheem@gmail.com>
Co-authored-by: banjuede <lmklhc@163.com>
Co-authored-by: Danielle Robinson <dmmaddix@amazon.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: bk-201 <joy25810@foxmail.com>
 2025-10-21 03:01:37 +00:00
Russell Bryant
3ada34f9cb [Frontend] Enforce tokenize=False when applying chat template (#27205) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-10-21 02:57:34 +00:00
Lunwen He
0eb8f2b880 create is_in_the_same_node on cpu (#26832) 
Co-authored-by: Lunwen He <lunwenh@meta.com>
 2025-10-21 02:04:14 +00:00
Fadi Arafeh
163965d183 [cpu] Dispatch un-quantized linear to oneDNN/ACL by default for AArch64 (#27183) 
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
Co-authored-by: Michael Yang <Michael.Yang@arm.com>
 2025-10-21 02:02:58 +00:00
Nick Hill
a03cf9bc70 [V0 Deprecation] Remove V0 metrics code (#27215) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-10-21 02:02:10 +00:00
Isotr0py
352c0c8a28 [Quantization] Automatically infer AWQ modules_to_not_convert field (#26909) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-10-21 01:49:28 +00:00
Andrew Xia
bfe0b4bd2a [ez] add uv lock to gitignore (#27212) 
Signed-off-by: Andrew Xia <axia@fb.com>
Co-authored-by: Andrew Xia <axia@fb.com>
 2025-10-21 00:37:44 +00:00
Concurrensee
58fbbcb2f5 [ROCm] enable some tests in entrypoints test groups on AMD (#26725) 
Signed-off-by: Yida <yida.wu@amd.com>
 2025-10-21 00:37:16 +00:00
Heng Guo
87778d5f00 [Feature][Quantization] auto_round support for mixed bits quantization (#23812) 
Signed-off-by: n1ck-guo <heng.guo@intel.com>
Signed-off-by: Heng Guo <heng.guo@intel.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-10-20 22:23:30 +00:00
Nicolò Lucchesi
f9e7ad5400 [Bugfix][CI] Fix Distributed Tests (4 GPUs) async_sched+ray test (#27195) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-10-20 16:34:54 +00:00
shivampr
4d0f266113 [Kernel][Model] Tune fused_moe Triton configs for Qwen3-30B A3/A3B on H100 (FP8/BF16) (#26268) 
Signed-off-by: Shivam <shivampr.dev@gmail.com>
 2025-10-20 07:48:01 -07:00
Eugene Khvedchenya
e93ff6c8b9 Nemotron Nano V2 VL + EVS Video Support (#27107) 
Signed-off-by: Eugene Khvedchenia <ekhvedchenia@nvidia.com>
Signed-off-by: Natan Bagrov <nbagrov@nvidia.com>
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Natan Bagrov <nbagrov@nvidia.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2025-10-20 22:19:11 +08:00
ioana ghiban
1c691f4a71 AArch64 CPU Docker pipeline (#26931) 2025-10-20 07:09:40 -04:00
Jiangyun Zhu
9fce7bee74 [Kernel] Accelerate solve_tril with TMA (#26746) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2025-10-20 05:39:02 +00:00
Andy Lo
b63f2143f8 [LoRA] LoRA cuda graph specialization (#25914) 
Signed-off-by: Andy Lo <andy@mistral.ai>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-10-20 04:21:09 +00:00
Yi Zhang
f32bf7582e [Model][VLM] Support Bee-8B Model (#27012) 
Signed-off-by: uyzhang <yi.zhang.4096@gmail.com>
Signed-off-by: Yi Zhang <zhangyi970819@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2025-10-20 02:31:26 +00:00
Yongtao Huang
8a81d776ce Fix typo in ValueError message: use kv_role instead of kv_disagg_role (#27166) 
Signed-off-by: Yongtao Huang <yongtaoh2022@gmail.com>
 2025-10-19 19:47:19 +00:00
Sergei Skvortsov
f6fdacd82c [Bugfix] Fix error with penalties when speculative decoding and structural output are enabled (#26586) 
Signed-off-by: southfreebird <yvorott@gmail.com>
 2025-10-19 19:24:46 +00:00
Cyrus Leung
d31f7844f8 [Misc] Move utils to avoid conflicts with stdlib, and move tests (#27169) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-19 05:20:55 -07:00
iAmir97
7a6c8c3fa1 [Chore] Separate out vllm.utils.network_utils (#27164) 
Signed-off-by: iAmir97 <Amir.balwel@embeddedllm.com>
Co-authored-by: iAmir97 <Amir.balwel@embeddedllm.com>
 2025-10-19 03:06:32 -07:00
Jianyu Huang
221bf72577 output type conversion fix (#27159) 2025-10-19 08:10:07 +00:00
Cyrus Leung
b3aba04e5a [Benchmark] Convenience script for multiple parameter combinations (#27085) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-18 23:57:01 -07:00
dongbo910220
8a297115e2 [Chore] Separate out hashing utilities from vllm.utils (#27151) 
Signed-off-by: dongbo910220 <1275604947@qq.com>
 2025-10-19 11:09:38 +08:00
22quinn
191eed0bb9 [BugFix] Fix lazy imports involving outlines_core (#27158) 
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
 2025-10-19 02:35:32 +00:00
Woosuk Kwon
fb860670da [Minor] Remove unused env variable (#27161) 2025-10-18 18:48:35 -07:00
Tova Movshovitz
83e760c57d [V1][Metrics][Plugin] Add plugin support for custom StatLoggerBase implementations (#22456) 
Signed-off-by: tovam <tovam@pliops.com>
 2025-10-18 15:12:46 -07:00
Lucas Wilkinson
c2bba69065 [BugFix] Disable fp8 kv-cache by default for DeepSeek V3.2 (#27121) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-10-18 22:05:23 +00:00
Boyuan Feng
e133d6d218 [BugFix] fix graph partition signature (#27139) 
Signed-off-by: Boyuan Feng <boyuan@meta.com>
 2025-10-18 17:34:36 -04:00
dongbo910220
a1946c9f61 [Chore] Separate out profiling utilities from vllm.utils (#27150) 
Signed-off-by: dongbo910220 <1275604947@qq.com>
 2025-10-18 19:12:01 +00:00
Lucas Wilkinson
9f020f4f31 [BugFix] Fix failing gemma-3-1b-it test: test_lm_eval_accuracy_v1_engine[google/gemma-3-1b-it] (#27111) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-10-18 12:44:39 -06:00
Nick Hill
3b45075206 [Minor] Add some clarifying comments to recent changes (#27130) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-10-18 09:52:45 -07:00
Yongtao Huang
168e578efc Fix incorrect string formatting in barrier timeout exceptions (#27149) 
Signed-off-by: Yongtao Huang <yongtaoh2022@gmail.com>
 2025-10-18 09:51:57 -07:00
Isotr0py
6ac5e06f7c [Chore] Clean up pytorch helper functions in vllm.utils (#26908) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: isotr0py <2037008807@qq.com>
 2025-10-18 09:48:22 -07:00
Lukas Geiger
5c2acb270a [Models][QwenVL] Remove unnecessary .contiguous() calls (#27106) 
Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>
 2025-10-18 07:05:05 -07:00
Nicolò Lucchesi
b26b70bec4 [Misc] Refactor get_kv_cache_spec into AttentionLayerBase (#26587) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-10-18 13:51:21 +00:00
Fadi Arafeh
ab4be40fc5 [fix][cpu] fix prefill attention in CPU attention backend (#27035) 
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
 2025-10-18 13:30:21 +00:00
Wentao Ye
245e4f2c01 [Feature] Batch Invariant: Support DeepGEMM and Blackwell (#27127) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-18 09:28:05 -04:00
iAmir97
1d165d6d85 [Chore] Separate out vllm.utils.mem_utils (#27143) 
Signed-off-by: iAmir97 <Amir.balwel@embeddedllm.com>
Signed-off-by: iAmir97 <71513472+iAmir97@users.noreply.github.com>
Co-authored-by: iAmir97 <Amir.balwel@embeddedllm.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-10-18 10:06:59 +00:00
dongbo910220
83004020fd [Test] Add test for /health endpoint on engine failure (#26074) 
Signed-off-by: dongbo910220 <1275604947@qq.com>
 2025-10-18 09:59:05 +00:00
Chendi.Xue
12e21701e7 [DOC][FEATURES][CPU]update cpu feature for v1 (#27135) 
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
 2025-10-18 01:10:45 -07:00
Varun Sundar Rabindranath
30a33b92ee [Misc] Rev DeepEP (#27122) 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
 2025-10-18 14:54:29 +08:00
Hanchenli
7c572544e4 [GPT-OSS] Structure_Tag support for gpt-oss tool-call in cot (#25515) 
Signed-off-by: Hanchenli <lihanc2002@gmail.com>
Signed-off-by: Hanchenli <61769611+Hanchenli@users.noreply.github.com>
Signed-off-by: Wei Wei <wwei6@meta.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Wei Wei <wwei6@meta.com>
Co-authored-by: Wei Wei <weiweinpu@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-10-17 21:55:54 -07:00
Huamin Li
c312320764 [CI/Build] tests(v1): feed Triton attention the (num_blocks, 2, …) KV cache layout in backend-correctness tests (#26663) 
Signed-off-by: Huamin Li <3ericli@gmail.com>
Co-authored-by: Ye (Charlotte) Qi <yeq@meta.com>
 2025-10-17 21:11:26 -07:00
ZiTian Zhao
c981f0ea78 [Perf] Add H100 fused MoE config (#25398) 
Signed-off-by: zitian.zhao <zitian.zhao@tencentmusic.com>
 2025-10-18 02:21:27 +00:00
Lehua Ding
6367bde739 [BugFix][Core] Fix error when enable async-scheduling in multi-node env (#25887) 
Signed-off-by: Lehua Ding <lehuading@tencent.com>
Signed-off-by: Lehua Ding <lehuading@qq.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
 2025-10-17 22:16:18 +00:00