biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
10,506 Commits 2 Branches 140 Tags
17838e50efc9e686fffc901dfed811b15b7d614a
Commit Graph

10506 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Cyrus Leung
17838e50ef [Benchmark] Use truncation by default for pooling benchmarks (#26992) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-16 16:02:39 +08:00
Zhewen Li
44c8555621 [CI/Build] Fix AMD import failures in CI (#26841) 
Signed-off-by: zhewenli <zhewenli@meta.com>
 2025-10-16 07:28:20 +00:00
Akash kaothalkar
f7d318de2b [Hardware][CPU][PowerPC]Disable torch.compile() in toptopk sampling (#26987) 
Signed-off-by: Akash Kaothalkar <akash.kaothalkar@ibm.com>
Co-authored-by: Akash Kaothalkar <akash.kaothalkar@ibm.com>
 2025-10-15 22:36:59 -07:00
Cyrus Leung
76f0d05bc6 [CI/Build] Update expected beam search output for Phi3V (#26978) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-16 05:12:44 +00:00
Bram Wasti
7d8975de84 Deepseek-v3 Batch Invariant on 8xH100 (#26609) 
Signed-off-by: Bram Wasti <bwasti@meta.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-10-15 22:06:02 -07:00
Vadim Gimpelson
785d8b6410 [PERF] Qwen3-next MTP speedup (change bool mask indexing to index_select / index_copy to reduce d2h) (#26437) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
 2025-10-16 12:18:31 +08:00
Cyrus Leung
f6cdc9a02f [Chore] Rename utils submodules (#26920) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-16 03:58:13 +00:00
Chendi.Xue
509cdc0370 [DOC][XPU]update feature parity with Intel GPU (#26954) 
Signed-off-by: Chendi Xue <Chendi.Xue@intel.com>
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
 2025-10-15 20:07:10 -07:00
Richard Zou
9b6504c307 [BugFix] Work around graph partition x torch.compile cache issue (#26956) 
Signed-off-by: Richard Zou <zou3519@gmail.com>
 2025-10-15 20:06:11 -07:00
Angela Yi
e19b16dde6 [bugfix] Fix SP + PP without specifying compile size (#26955) 
Signed-off-by: angelayi <yiangela7@gmail.com>
 2025-10-15 20:05:33 -07:00
ahao-anyscale
582f2c6be7 [BUG] Allow runai_streamer_sharded in config check (#26958) 
Signed-off-by: ahao-anyscale <ahao@anyscale.com>
 2025-10-15 20:05:14 -07:00
Michael Goin
f8a0acbdbe [CI] Enable Blackwell Llama4 MoE tests (#26731) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-10-15 21:02:57 -06:00
kliuae
1317034379 [ROCm][FEAT] Fuse DeepSeek shared experts into AITER fused_moe ops (#24097) 
Signed-off-by: chenjun <junchen2@amd.com>
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com>
Co-authored-by: valarLip <103567126+valarLip@users.noreply.github.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
 2025-10-16 10:41:34 +08:00
InChang Jeong
0ecc553ee6 [Bugfix] reasoning_parser parameter handling in run_batch.py (#26225) 
Signed-off-by: inc-jeong <inc.jeong@navercorp.com>
Signed-off-by: InChang Jeong <inc.jeong@navercorp.com>
Co-authored-by: USER <user@AL02367916.local>
 2025-10-16 10:24:05 +08:00
felixzhu555
f96bc3649c [Qwen3-Next] Add tuned MoE config for Qwen3-Next FP8 on H100 tp2 (#26887) 
Signed-off-by: Felix Zhu <felixzhu555@gmail.com>
 2025-10-15 18:55:05 -07:00
Alexei-V-Ivanov-AMD
938c43ea7f [ci] Adjusting AMD test composition 2025-10-14 (#26852) 
Signed-off-by: Alexei V. Ivanov <alexei.ivanov@amd.com>
 2025-10-15 23:52:13 +00:00
Adrian Abeyta
0a9ef0cfce Move query quantization to attention layer for Flashinfer & Triton. (#26534) 
Signed-off-by: adabeyta <aabeyta@redhat.com>
Signed-off-by: Adrian Abeyta <aabeyta@redhat.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2025-10-15 19:01:38 -04:00
Wentao Ye
e5b438a247 [Bug] Temporally Disable VLLM_ALLREDUCE_USE_SYMM_MEM by Default (#26925) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-15 16:18:50 -04:00
XiaobingZhang
0b99f5d302 support flashinfer_fp4 moe for 5090 gpu (#26669) 
Signed-off-by: XiaobingSuper <xiaobingzhangupc@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-10-15 15:06:47 -04:00
Benji Beck
1f491aa0c8 Vectorize RMS norm variance using vectorize_read_with_alignment (#26234) 
Signed-off-by: Benji Beck <benjibeck@meta.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-10-15 11:54:41 -07:00
Kaixi Hou
de92d916fe [NVIDIA] Add support for cudnn fp4 gemm via flashinfer (#26107) 
Signed-off-by: kaixih <kaixih@nvidia.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-10-15 13:53:00 -04:00
Woosuk Kwon
a1063628a4 [Chore] Clean up CODEOWNERS (#26923) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-10-15 10:52:54 -07:00
XiaobingZhang
d796375258 [ModelOpt] Remove NVFP4 MoE K%16==0 constraint (#26891) 
Signed-off-by: XiaobingSuper <xiaobingzhangupc@gmail.com>
 2025-10-15 13:06:17 -04:00
Sam/Samuel
14f8456344 [Feature]: Use pydantic validation in observability.py config (#26637) 
Signed-off-by: Samuel Wu <cernunnos1710@gmail.com>
Signed-off-by: Sam/Samuel <57896620+cern1710@users.noreply.github.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-15 16:44:03 +00:00
Pradeep Dasigi
4794c2bd92 Olmo 3 tool parser and tests (#26143) 
Signed-off-by: Pradeep Dasigi <pradeepd@allenai.org>
 2025-10-15 16:36:12 +00:00
Harry Mellor
d3cbaa08dc Lower sevarity of log when model info cache misses due to exception (#26917) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-15 09:01:09 -07:00
Cyrus Leung
828523ad8e [Chore] Separate out vllm.utils.async_utils (#26913) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-15 15:33:00 +00:00
Cyrus Leung
136a17fe6e [Chore] Separate out vllm.utils.func (#26904) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-15 13:03:58 +00:00
Boyuan Feng
f57438338d [BugFix] Patch inductor memory plan logic (#26878) 
Signed-off-by: Boyuan Feng <boyuan@meta.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-15 12:51:45 +00:00
Max Wittig
5d598680e3 chore: remove unused marker (#26890) 
Signed-off-by: Max Wittig <max.wittig@siemens.com>
 2025-10-15 05:40:33 -07:00
wangxiyuan
8f4b313c37 [Misc] rename torch_dtype to dtype (#26695) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2025-10-15 12:11:48 +00:00
Cyrus Leung
f93e348010 [Misc] Remove isort and yapf ignores (#26888) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-15 12:09:03 +00:00
wang.yuqi
f54f85129e [Model][2/N] Improve all pooling task | Support multi-vector retrieval (#25370) 
Signed-off-by: wang.yuqi <noooop@126.com>
 2025-10-15 11:14:41 +00:00
li2haipeng
d4d1a6024f [Lora]Load tuned multi-lora kernel configs from json files (#26319) 
Signed-off-by: li2haipeng <44383182+li2haipeng@users.noreply.github.com>
Signed-off-by: Haipeng Li <li2haipeng@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-10-15 09:45:14 +00:00
wangxiyuan
db1764e4e0 [Platform] allow platform to init dp group (#22243) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2025-10-15 02:32:17 -07:00
Jialin Ouyang
7f83b4ee8e [Easy] Get rid of unnecessary paraenthesis in kv_cache_manager (#26842) 
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
 2025-10-15 09:17:43 +00:00
ant-yy
5c3bae1a6a [Fix] Remove divisibility requirement between num_kv_heads and tp_size in bailing_moe (#26876) 
Signed-off-by: vito.yy <vito.yy@antgroup.com>
 2025-10-15 16:44:04 +08:00
Xudong Ma
5210dc3940 [Misc] Update TritonLanguagePlaceholder to have attributes that are used by Flash Linear Attention ops. (#26853) 
Co-authored-by: Xudong Ma <mxd@meta.com>
 2025-10-15 08:37:49 +00:00
youkaichao
650b51f9f9 [doc] add Context Parallel Deployment doc (#26877) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-10-15 16:33:52 +08:00
Cyrus Leung
6256697997 [Doc] ruff format remaining Python examples (#26795) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-15 01:25:49 -07:00
Wentao Ye
71557a5f7c [CI] Fix mypy for vllm/executor (#26845) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-15 01:23:33 -07:00
Zhewen Li
f3c378ffa7 [CI/Build] Add Qwen2.5-VL-7B-Instruct ChartQA Accuracy Tests in CI (#21810) 
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
Signed-off-by: zhewenli <zhewenli@meta.com>
Co-authored-by: Ye (Charlotte) Qi <yeq@meta.com>
Co-authored-by: Ye (Charlotte) Qi <ye.charlotte.qi@gmail.com>
 2025-10-15 08:09:56 +00:00
Yongye Zhu
f5ed68ef63 [Deepseek-V3.2][Kernel] Integrate cuda indexer k cache gather (#26456) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
 2025-10-15 16:05:01 +08:00
Angela Yi
efdef57b1f [bugfix] Lazy import cv2 (#26869) 
Signed-off-by: angelayi <yiangela7@gmail.com>
 2025-10-15 07:47:50 +00:00
Cyrus Leung
b8a4572157 [Misc] Use helper function to generate dummy messages in OpenAI MM tests (#26875) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-15 07:17:37 +00:00
Mengqing Cao
302ef403a2 [DSA][MLA] Tiny refactor on DeepSeek to make it reusable for different backends (#26656) 
Signed-off-by: MengqingCao <cmq0113@163.com>
 2025-10-15 00:16:44 -07:00
sangho.lee
8865da157b [Bugfix][Multi Modal] Fix incorrect Molmo token processing (#26873) 
Signed-off-by: sanghol <sanghol@allenai.org>
 2025-10-15 07:13:59 +00:00
Boyuan Feng
f0862eae43 [Graph Partition] pass tests for decorator (#26831) 
Signed-off-by: Boyuan Feng <boyuan@meta.com>
 2025-10-15 06:39:48 +00:00
Isotr0py
8c851f6d04 [Bugfix] Fix qwen3-omni audio truncation issue (#26815) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-10-15 05:38:36 +00:00
Angela Yi
7cfa420f49 [BugFix] Patch inductor partitioning logic (#26735) 
Signed-off-by: angelayi <yiangela7@gmail.com>
 2025-10-15 05:04:32 +00:00