biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
11,559 Commits 2 Branches 140 Tags
e9056056fbacecbac4318bd0323745fdd7fe55b6
Commit Graph

11559 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Woosuk Kwon
e9056056fb [Model Runner V2] Limit cudagraph size to max decode batch size (#29221) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-11-21 20:21:35 -08:00
Jee Jee Li
1489902b53 [LoRA] Cleanup FusedMoEWithLoRA (#29187) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-11-22 04:01:30 +00:00
Yanan Cao
933f67ecd8 [Bugfix]Fix a conditional to not check zero value (#28754) 
Signed-off-by: Yanan Cao <gmagogsfm@gmail.com>
 2025-11-21 19:59:07 -08:00
rasmith
fd65015a14 [CI/Build] Only use supported types and features on ROCm in MoE kernel tests (#29149) 
Signed-off-by: Randall Smith <ransmith@amd.com>
Co-authored-by: Randall Smith <ransmith@amd.com>
 2025-11-21 20:34:33 -07:00
Yihua Cheng
77e1c035d0 [chore][LMCache connector] Remove useless logs from lmcache connector (#29069) 
Signed-off-by: ApostaC <yihua98@uchicago.edu>
 2025-11-22 03:18:00 +00:00
rasmith
6f403501a0 [CI/Build][AMD] Enable Entrypoints Integration Test (Pooling) to run without error on ROCm (#29212) 
Signed-off-by: Randall Smith <ransmith@amd.com>
Co-authored-by: Randall Smith <ransmith@amd.com>
 2025-11-22 02:13:18 +00:00
FlintyLemming
052950e5b3 Add fused MoE config for H200 E160 N192 fp8 (#29182) 
Signed-off-by: FlintyLemming <admin@flinty.moe>
 2025-11-21 17:37:51 -08:00
qli88
1ef9c9e294 [CI/Build] Disable test_gptoss_tp.py in 'LoRA TP Test' group for ROCm platform (#29204) 
Signed-off-by: qli88 <qiang.li2@amd.com>
 2025-11-21 17:36:19 -08:00
Jie Luo
5c8f2adf50 [Bugfix] Fix block size in block_table with PCP (#29094) 
Signed-off-by: Livinfly <luojie3m@gmail.com>
 2025-11-22 01:34:28 +00:00
Ryan Rock
ed8e6843cc [CI/Build] Add terratorch for AMD (#29205) 
Signed-off-by: Ryan Rock <ryan.rock@amd.com>
 2025-11-21 17:31:22 -08:00
Lukas Geiger
d045e22dfe [Model][Qwen3VL] Tune Triton w8a8 block fp8 kernel for L40s (#29217) 
Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>
 2025-11-21 17:30:55 -08:00
Wentao Ye
1d34eb11e0 [CI] Bug: Fix triton import issue (#29202) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-11-21 17:14:49 -08:00
Charlie Fu
9a3101b2ba [Rocm][CI] Fix DeekSeek V2-Lite Accuracy CI (#29135) 
Signed-off-by: charlifu <charlifu@amd.com>
 2025-11-21 17:11:02 -08:00
Angela Yi
d5dbdbfcb2 [docs] Fix cudagraph mode config (#29170) 
Signed-off-by: angelayi <yiangela7@gmail.com>
 2025-11-21 17:10:27 -08:00
Lucas Wilkinson
30d6466238 [BugFix] Fix Eagle IndexError: list index out of range for even num_speculative_tokens (#29102) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-11-22 00:47:05 +00:00
Woosuk Kwon
e9af6ba62a [Model Runner V2] Optimize Gumbel Sampling Kernel (#29210) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-11-21 15:52:28 -08:00
Mark McLoughlin
c6fa3895e9 [KV Connector] Fix async connector prefix cache metrics (#28585) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2025-11-21 17:45:00 -05:00
Varun Sundar Rabindranath
3137991f55 [BugFix] EPLB + B200 + DeepGEMM : Handle column-major scales tensor (#29162) 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
 2025-11-21 14:28:17 -08:00
Julien Denize
57430fc95c Default model load/config/tokenizer to mistral format if relevant files exist (#28659) 
Signed-off-by: Julien Denize <julien.denize@mistral.ai>
Signed-off-by: Julien Denize <40604584+juliendenize@users.noreply.github.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-11-21 13:58:59 -08:00
Lucas Wilkinson
c68c7b403d [BugFix] Fix missing symbol triggering FA2 fallback on Hopper (#29107) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-11-21 13:58:32 -08:00
Ning Xie
53a1ba6ec5 [log] add weights loading time log to sharded_state loader (#28628) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2025-11-21 21:06:09 +00:00
Lucas Wilkinson
1840c5cb18 [BugFix] Make sure to allocate worst case MoE workspace during profile run in the DP + EP case (#27426) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-11-21 11:41:52 -08:00
Woosuk Kwon
1bed891f72 [Chore] Fix pre-commit error after #25266 (#29190) 2025-11-21 10:21:40 -08:00
Cyrus Leung
ceca060501 [Deprecation] Deprecate seed=None (#29185) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-11-21 18:19:25 +00:00
Charlie Fu
75648b16dd [ROCm][CI] Fix config/test_config_generation.py (#29142) 
Signed-off-by: charlifu <charlifu@amd.com>
 2025-11-21 17:12:16 +00:00
Chendi.Xue
460d02a417 [NIXL] Fix after virtual block_size for host_buffer with heter kv_layout (#29122) 
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
 2025-11-21 08:55:27 -08:00
Mingyuan Ma
b4c8fbaae2 Add TRTLLM MoE NVFP4 kernel to CompressedTensorsW4A4MoeMethod (#28892) 
Signed-off-by: mingyuanm <mingyuanm@nvidia.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-11-21 09:54:11 -07:00
rasmith
e99e467384 [CI/Build][Kernel][AMD] Move extra dim to after load in _fwd_kv_parallel in lighting_attn.py (#29132) 
Signed-off-by: Randall Smith <ransmith@amd.com>
Co-authored-by: Randall Smith <ransmith@amd.com>
 2025-11-21 11:53:09 -05:00
Wentao Ye
a42ab317ac [Log] Optimize startup log (#28948) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-11-21 08:46:20 -08:00
Aleksandr Malyshev
b7f1f490a6 Upstream triton fp4 weight preshuffle (#28888) 
Signed-off-by: Aleksandr Malyshev <maleksan@amd.com>
Co-authored-by: Aleksandr Malyshev <maleksan@amd.com>
 2025-11-21 11:34:46 -05:00
Woosuk Kwon
30b44a1598 GPU Model Runner V2 (#25266) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-11-21 08:20:55 -08:00
Wentao Ye
1f400c58b8 [CI] Add batch invariant test to ci (#27842) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-11-21 09:20:33 -07:00
rasmith
711241c13c [CI/Build] Fix illegal memory access and unsupported test in kernels/attention/test_cache.py (#29118) 
Signed-off-by: Randall Smith <ransmith@amd.com>
Co-authored-by: Randall Smith <ransmith@amd.com>
 2025-11-21 10:58:38 -05:00
Cyrus Leung
d7219bcda3 [Misc] Move dynamic seed initialization to EngineArgs (#29165) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-11-21 15:27:44 +00:00
wangxiyuan
4050bae417 [Doc] Update plugin doc (#28532) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2025-11-21 14:57:26 +00:00
skaraban3807
f1805db1a6 [Perf] These changes enhance the NUMA functionality of vllm for systems with more than one NUMA nodes per socket (#25559) 
Signed-off-by: Siddappa Karabannavar <siddappa.karabannavar@amd.com>
 2025-11-21 14:13:52 +00:00
Julien Denize
434f3d3eb8 Fix mistral config (#29172) 
Signed-off-by: Julien Denize <julien.denize@mistral.ai>
Signed-off-by: Julien Denize <40604584+juliendenize@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2025-11-21 14:01:20 +00:00
sfbemerk
2092ce8c39 Tool Call Parser logs should not contain user input / model output except on DEBUG (#29160) 
Signed-off-by: Benjamin Merkel <benjamin.merkel@tngtech.com>
Co-authored-by: Benjamin Merkel <benjamin.merkel@tngtech.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2025-11-21 20:57:19 +08:00
who who who
fc9f821d20 fix cross attention (#28346) 
Signed-off-by: fsx950223 <fsx950223@outlook.com>
 2025-11-21 04:55:43 -08:00
Cyrus Leung
9452863088 Revert "Revert #28875 (#29159)" (#29179) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-11-21 04:27:43 -08:00
Bhagyashri
2b1b3dfa4b Update Dockerfile to use gcc-toolset-14 and fix test case failures on power (ppc64le) (#28957) 
Signed-off-by: Bhagyashri <Bhagyashri.Gaikwad2@ibm.com>
 2025-11-21 12:24:09 +00:00
Russell Bryant
cca2d2cdbe [Core] Align whisper closer to other multimodal models (#27292) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-11-21 12:01:54 +00:00
Cyrus Leung
aab0102a26 [V0 deprecation] Remove more V0 references (#29088) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-11-21 11:56:59 +00:00
WeiQing Chen
b34129bf8e [Misc] remove useless v1 env (#29164) 
Signed-off-by: David Chen <530634352@qq.com>
 2025-11-21 01:41:20 -08:00
Cyrus Leung
4d7231e774 Revert #28875 (#29159) 2025-11-21 01:40:17 -08:00
Huamin Li
8ac3a41487 [CI Failure] Fix Gemma3 RoPE configuration for sliding attention layers (#29111) 
Signed-off-by: Huamin Li <3ericli@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-11-20 23:53:30 -08:00
Canlin Guo
7d6da483b0 [Minor][Clean] Remove the legacy assertion in video (#29150) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2025-11-20 23:52:34 -08:00
Chenheli Hua
e4c3182c68 [Small] Capture AttributeError when checking ray dependency. (#29024) 
Signed-off-by: Chenheli Hua <huachenheli@outlook.com>
 2025-11-20 22:54:10 -08:00
Alex Brooks
b4734b9550 [Bugfix] Fix default MM LoRA alignment for single str prompts (#29140) 
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
 2025-11-21 13:32:30 +08:00
Jialin Ouyang
30b9c67743 Revert "[Redo] #26368 (#28771)" (#29121) 
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
 2025-11-20 21:27:45 -08:00