biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
13,376 Commits 2 Branches 140 Tags
39037d258e68da3926d99681ea63e46212e519f9
Commit Graph

249 Commits

Author SHA1 Message Date
Michael Goin
ca1969186d [UX] Enable nested configs in config yaml files (#33193) 2026-01-28 16:54:25 -05:00
Angela Yi
4197168ea5 [ez] Remove checks for torch version <= 2.8 (#33209) 
Signed-off-by: angelayi <yiangela7@gmail.com>
 2026-01-28 16:03:56 -05:00
Cyrus Leung
11b556878b [Refactor] Use data parser for matching data items to multi-modal UUIDs (#32955) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-26 15:00:28 +08:00
7. Sun
0ccecf8833 [Tests] Standardize RNG seed utility across test files (#32982) 
Signed-off-by: 7. Sun <jhao.sun@gmail.com>
 2026-01-24 06:47:14 +00:00
ElizaWszola
a28b94e6ef [Performance] Split FlashAttn attention and cache update (#25954) 
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Luka Govedič <luka.govedic@gmail.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Varun Sundar Rabindranath <varunsundar08@gmail.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Luka Govedič <luka.govedic@gmail.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>
 2026-01-23 17:28:06 -08:00
Xin Yang
d08b356ee0 [Perf] Create TMA-aligned input scale tensor for DeepGemm on Hopper (#32619) 
Signed-off-by: Xin Yang <xyangx@amazon.com>
 2026-01-22 15:47:04 -05:00
Isotr0py
8ebf271bb6 [Misc] Replace urllib's urlparse with urllib3's parse_url (#32746) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-01-22 16:37:15 +08:00
Alex Sun
49a1262267 [AMD][ROCm] MoRI EP: a high-performance all2all backend (#28664) 
Signed-off-by: Alex Sun <alex.s@amd.com>
 2026-01-22 16:33:18 +08:00
Cyrus Leung
2b8a38b6d6 [Model] Extend collect_children and no_init_weights contexts (#32757) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-22 08:20:27 +00:00
Yanwen Lin
6bb2bc71e2 [Bugfix] Force using spawn multiprocess method when it's the WSL platform (#32749) 
Signed-off-by: Yanwen Lin <lyw1124278064@gmail.com>
 2026-01-21 09:35:55 +00:00
Chauncey
c4e5bdf61b [Bugfix] Fix the fp8_mqa_logits dim mismatch (#32652) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-01-20 18:48:07 +08:00
Yanan Cao
9d1e611f0e [CI] Add Helion as an optional dependency (#32482) 
Signed-off-by: Yanan Cao <gmagogsfm@gmail.com>
 2026-01-19 19:09:56 +00:00
Isotr0py
8cc26acd8b [Performance] Improve Triton prefill attention kernel's performance (#32403) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-01-17 20:19:59 -08:00
Shanshan Shen
ce0946249d [Misc] Make mem utils can be reused by other platforms (#32322) 
Signed-off-by: shen-shanshan <467638484@qq.com>
 2026-01-14 03:46:01 -08:00
Angela Yi
7933638051 [misc] Remove is_torch_equal_or_newer(2.4) cases (#32296) 
Signed-off-by: angelayi <yiangela7@gmail.com>
 2026-01-13 23:22:07 -08:00
Roberto L. Castro
8ef50d9a6b [Kernel][Performance] Enable smaller Scaling Factor tiling for NVFP4 small-batch decoding (#30885) 
Signed-off-by: LopezCastroRoberto <roberto.lopez.castro@udc.es>
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com>
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
 2026-01-13 15:22:53 -08:00
Vadim Gimpelson
e15a5ff07b [MISC] Add strict contiguity check for FlashInfer attention tensors (#32008) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Signed-off-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
 2026-01-10 12:40:05 -08:00
shyeh25
1c46dea001 Revert "[Kernels][FI] Skip trtllm attention when num_kv_heads=1 (#308… (#31617) 
Signed-off-by: shyeh25 <206795756+shyeh25@users.noreply.github.com>
 2026-01-10 12:39:59 -08:00
roikoren755
0c9614876e Update modelopt KV cache quantization resolution to new scheme (#31895) 
Signed-off-by: Roi Koren <roik@nvidia.com>
 2026-01-10 04:54:13 +00:00
Ning Xie
c907d22158 [refactor] refactor memory constants usage (#31865) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2026-01-07 18:37:31 +00:00
Kate Cheng
cc6dafaef2 [Perf][Kernels] Enable FlashInfer DeepGEMM swapAB on SM90 (for W8A8 Linear Op) (#29213) 
Signed-off-by: Kate Cheng <yunhsuanc@nvidia.com>
Signed-off-by: Jhao-Ting Chen <jhaotingc@nvidia.com>
Co-authored-by: Jhao-Ting Chen <jhaotingc@nvidia.com>
 2026-01-07 10:53:54 -05:00
Kevin McKay
4614c5a539 [Bugfix][Hardware][AMD] Consolidate FP8 min/max values helper function (#31106) 
Signed-off-by: c0de128 <kevin.mckay@outlook.com>
Signed-off-by: Kevin McKay <kevin@example.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
 2026-01-07 06:55:03 +00:00
tjp_zju
55caa6051d refactor: find_loaded_library (#31866) 
Signed-off-by: tjp_zju <tanjianpingzju1990@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-01-07 06:42:20 +00:00
Cyrus Leung
aafd4d2354 [Chore] Try remove init_cached_hf_modules (#31786) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-07 12:34:04 +08:00
Ning Xie
6f5e653383 [Log] add log about gpu worker init snapshot and requested memory (#29493) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2026-01-06 17:32:55 +00:00
Cyrus Leung
14df02b4e1 [Chore] Cleanup mem_utils.py (#31793) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-01-06 19:55:59 +08:00
maang
d386ab1412 [Docs] Improve malformed exception caused by backslash line continuations (#31694) 
Signed-off-by: maang <maang_h@163.com>
Signed-off-by: maang <55082429+maang-h@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-01-05 17:51:54 -08:00
wangxiyuan
bb4337b34c [Platform] Deprecate seed_everything (#31659) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2026-01-04 18:34:04 -08:00
Amir Samani
030fc44914 use the same stream for cuda graph catpure and replay for NCCL (#29207) 
Signed-off-by: Amir Samani <asamani@nvidia.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2025-12-25 19:10:03 +08:00
Cyrus Leung
09dc7c690c [Chore][1/2] Drop v0.14 deprecations (#31285) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-12-24 09:54:01 -08:00
Kevin McKay
ec58c10ce1 [Misc] Fix quantization-related typos (#31116) 
Signed-off-by: c0de128 <kevin.mckay@outlook.com>
 2025-12-21 21:13:48 -08:00
Wentao Ye
3bd8335bd0 [Refactor] Refactor for DeepGemmQuantScaleFMT using cache (#30898) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-12-19 13:50:39 -07:00
Cyrus Leung
2497228ad4 [Chore] Factor out logic for requesting initial memory (#30868) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-12-17 07:32:17 -08:00
danielafrimi
7b966ae2ba [Fix]Load kv-cache dtype from hf_quant_config.json automatically (fix for reverted PR) (#30785) 
Signed-off-by: <>
Co-authored-by: root <root@gpu-937.slurm-workers-slurm.slurm.svc.cluster.local>
 2025-12-17 01:56:38 -08:00
Ye (Charlotte) Qi
a100152288 [Kernels][FI] Skip trtllm attention when num_kv_heads=1 (#30842) 
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
 2025-12-17 01:54:21 -08:00
jiahanc
254a7f8fd6 [Perf] Do FP4 quant before All gather on flashinfer trtllmgen MOE (#30014) 
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
 2025-12-16 13:01:48 -08:00
Robert Shaw
e2ed238885 Revert "[Fix]Load kv-cache dtype from hf_quant_config.json automatically" (#30653) 2025-12-14 19:33:41 -05:00
ElizaWszola
994acec0cc [Bugfix] Fix fusion for VL models (#30244) 
Signed-off-by: ElizaWszola <ewszola@redhat.com>
 2025-12-14 21:22:37 +08:00
Roberto L. Castro
4fa7ce46f3 [Feature] Add SM103 (Blackwell Ultra) Support to vLLM (#30484) 
Signed-off-by: LopezCastroRoberto <robertol.c510@gmail.com>
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2025-12-12 19:34:23 -08:00
danielafrimi
6ec0d8dbe4 [Fix]Load kv-cache dtype from hf_quant_config.json automatically (#29980) 
Signed-off-by: Daniel Afrimi <dafrimi@nvidia.com>
 2025-12-12 11:27:47 -08:00
Po-Han Huang (NVIDIA)
eea41804a4 [bug] Fix "Current vLLM config is not set." warnings when FlashInfer attention is used (#30241) 
Signed-off-by: Po-Han Huang <pohanh@nvidia.com>
 2025-12-10 11:18:51 -08:00
Zhewen Li
ae339b1a67 [Bugfix] Fix DeepGEMM after #29546 (#30267) 
Signed-off-by: zhewenli <zhewenli@meta.com>
Signed-off-by: Zhewen Li <zhewenli@meta.com>
 2025-12-09 01:05:27 +00:00
wang.yuqi
2e660c2434 [Frontend] Binary embedding response does not return metadata by setting encoding_format to bytes_only. (#30249) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-08 12:01:21 +00:00
ElizaWszola
af0444bf40 [Performance] Fused blockwise quant RMS norm (#27883) 
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
 2025-12-07 16:38:04 +00:00
Yanan Cao
cbedb703cc [Frontend] Remove confusing -O.xx flag error (#30169) 
Signed-off-by: Yanan Cao <gmagogsfm@gmail.com>
 2025-12-07 02:53:42 +00:00
Matthew Bonanni
66e674cdd5 [Attention][UX][1/N] Add AttentionConfig and change attention env vars to CLI arguments (#26315) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2025-12-05 09:48:43 -08:00
Max Hu
c2894d3883 [Feature] Add Layer-wise NVTX Support (#29990) 
Signed-off-by: Max Hu <hyoung2991@gmail.com>
Signed-off-by: Max Hu <maxhu@nvidia.com>
Co-authored-by: Max Hu <maxhu@nvidia.com>
 2025-12-05 11:20:07 +00:00
Yanan Cao
62b3333448 [Frontend] Remove deprecated -O.xx flag (#29991) 
Signed-off-by: Yanan Cao <gmagogsfm@gmail.com>
 2025-12-05 00:47:22 -08:00
amitz-nv
6038b1b04b [Frontend][Model] Add 'float16' to possible mamba cache dtype values, override mamba SSM cache dtype value for NemotronH (#29978) 
Signed-off-by: amitz-nv <203509407+amitz-nv@users.noreply.github.com>
 2025-12-05 00:34:33 -08:00
Cyrus Leung
9ae2f60374 [Misc] Various cleanups for MM input processing (#29970) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-12-04 06:22:20 +00:00