biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
14,055 Commits 2 Branches 140 Tags
4e2c7caf2d11444ce6c1e4895bc921c93610bd7c
Commit Graph

14055 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Matthias Gehre
4e2c7caf2d [Bugfix] Add regression test for MoE quant_config under torch.compile (#34335) 
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com>
 2026-02-20 13:27:26 +08:00
Bowen Bao
d9e62c03eb [Quark] Fix MoE fp8 activation scale handling on mi300 (#34386) 
Signed-off-by: Bowen Bao <bowenbao@amd.com>
 2026-02-19 21:27:14 -08:00
Kevin H. Luu
a1a2d79442 [ci] Use the right tag for CPU arm64 image (#34915) 
Signed-off-by: Kevin H. Luu <khluu000@gmail.com>
 2026-02-19 19:59:15 -08:00
Cyrus Leung
ac900c89bb [Refactor] Implement output type check in LLM (#34794) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-19 19:57:55 -08:00
Mark McLoughlin
76df6072ff [Core] Fix state names in pause_scheduler() (#34840) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2026-02-19 17:21:46 -08:00
Michael Goin
16f24e8797 [CI] Add GPT-OSS Eval job for H100 (#34359) 
Signed-off-by: Michael Goin <mgoin64@gmail.com>
 2026-02-19 17:14:54 -08:00
Nick Hill
40b2f1c3d9 [Model Runner V2] Minor CPU optimizations (#34856) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-02-19 16:05:37 -08:00
Mayank Ketkar
648951a9c3 [Bugfix] Fix benchmark_fused_collective crash on CustomOp init (#34665) 
Signed-off-by: Mayank Ketkar <mketkar@zoox.com>
Signed-off-by: Mayank Ketkar <mayket04@gmail.com>
Co-authored-by: Mayank Ketkar <mketkar@zoox.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-02-19 19:01:00 -05:00
Michael Goin
f72061a19a [UX] More descriptive reasons in is_supported_config for MoE (#34908) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-02-19 15:20:52 -08:00
Matthew Bonanni
662205d34e [Bugfix] Fix Basic Models Test (#34818) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-02-19 14:49:07 -08:00
Roger Wang
4fb8beefaa [Bugfix] Fix cutlass fp8 kernel on hopper for Qwen3.5 (#34914) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2026-02-19 13:34:55 -08:00
Alexei-V-Ivanov-AMD
304319c4ed Change targets for AMD build in the "CI" pipeline (#34918) 
Signed-off-by: Alexei V. Ivanov <alexei.ivanov@amd.com>
 2026-02-19 21:26:53 +00:00
Wentao Ye
c683d11c94 [Refactor] Deprecate head_first for chunk_gated_delta_rule (#34263) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-02-19 13:23:49 -05:00
roikoren755
3eff45d793 Revert "[NemotronH] Do not force router to run in fp32 (#34582)" (#34808) 
Signed-off-by: Roi Koren <roik@nvidia.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-02-19 09:47:05 -08:00
Robert Shaw
4685a630a2 [Model Bash][DeepSeekR1] Remove Shared Expert Clone (#34344) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2026-02-19 07:56:14 -08:00
Eldar Kurtić
ee1d25f199 [Llama4,Quantization] Simplify and generalize logic for Q/K permutations in quantized self-attn layers (#34471) 
Signed-off-by: Your Name <you@example.com>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-02-19 07:55:41 -08:00
Linda
6fff24f30f [Bugfix] Qwen3.5 kv-scale weight remapping (#34719) 
Signed-off-by: Linda-Stadter <57756729+Linda-Stadter@users.noreply.github.com>
 2026-02-19 04:13:37 -08:00
Cyrus Leung
23210a911e [CI/Build] Try to make beam search test less flaky (#34885) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-19 19:16:58 +08:00
Cyrus Leung
1391378861 [Bugfix] Fix edge case in UUID data parsing (#34884) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-19 02:24:30 -08:00
Andreas Karatzas
f6220f9877 [ROCm][Test] Fix beam search determinism failures from batch-size-dependent FP divergence and removed wrong marker (#34878) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-02-19 08:25:26 +00:00
Andreas Karatzas
2df2bb27b0 [ROCm][CI] Removing all blocking labels from MI355 until stable infra (#34879) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-02-19 07:53:08 +00:00
Tal Nir
f75b61a9e9 [Voxtral Realtime] Fix engine crash on empty multimodal embeddings (#34862) 
Signed-off-by: Tal Nir <tal@nervexneurotech.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-18 23:21:47 -08:00
Wei Zhao
7f51e93864 [Bug] Fix DeepSeek V3 weight loading caused by incorrect prefix (#34876) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
 2026-02-18 23:20:30 -08:00
Alex Brooks
4611af1663 [Bugfix] Add Quant Config to Llava Next Projector (#34847) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
 2026-02-18 23:18:23 -08:00
Manrique Vargas
ad5aa6bd9f fix(docs): fix typos in comments and docstrings (#34836) 
Signed-off-by: machov <mv1742@nyu.edu>
 2026-02-18 23:17:41 -08:00
Jaeyeon Kim(김재연)
9681068cf9 [Frontend] Fix reasoning_tokens for text-based parsers in Responses API (#33513) 
Signed-off-by: Jaeyeon Kim <anencore94@gmail.com>
 2026-02-18 23:16:41 -08:00
Kevin H. Luu
b6101d384d Deprecate test-pipeline.yaml (#34864) 
Signed-off-by: Kevin H. Luu <khluu000@gmail.com>
 2026-02-19 02:15:27 +00:00
Woosuk Kwon
5fcb0cdd68 [Model Runner V2] Use FP32 for Gumbel Noise (#34854) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-02-18 17:07:37 -08:00
Woosuk Kwon
c878b43b64 [Model Runner V2] Remove unnecessary copies in PW CUDA graph capture (#34849) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-02-18 15:52:50 -08:00
rasmith
2b84ac669c [CI][AMD][BugFix] Use torch.testing.assert_close instead of assert torch.allclose in test_rocm_skinny_gemms.py (#34181) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
 2026-02-18 23:10:19 +00:00
zhrrr
11d3976b88 [Model Runner V2] support piecewise & mixed cudagraph (#32771) 
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
 2026-02-18 15:03:17 -08:00
Yongye Zhu
40da9625a1 [MoE Refactor] Convert mxfp4 marlin into modular kernel format (#34588) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-02-18 14:37:14 -08:00
Flora Feng
8d9babd4de Fix empty tool_call_id in Anthropic messages API tool result conversion (#34745) 
Signed-off-by: <>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: Flora Feng <sfeng33@h100-01.nemg-001.lab.rdu2.dc.redhat.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2026-02-18 14:31:59 -08:00
Aaron Hao
e99ba957ec [BUG] Fixing Weight Sync unit test (#34841) 
Signed-off-by: ahao-anyscale <ahao@anyscale.com>
 2026-02-18 17:20:10 -05:00
Kyle Sayers
64ac1395e8 [Docs] Clean up speculators docs (#34065) 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
 2026-02-18 13:48:11 -08:00
Cyrus Leung
61cf087680 [Bugfix] Fix lora tests (#34834) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-02-18 13:22:31 -08:00
Wenlong Wang
847a57cd12 [Bugfix][MoE Kernel] Fix incorrect routing selection for models without expert groups (e.g., MiniMax-M2.1) (#34673) 
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-02-18 13:03:24 -08:00
rasmith
fcd6ac97ed [CI][AMD][BugFix] Skip tests in test_unquantized_backend_selection that should not run on ROCm (#34655) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
 2026-02-18 15:00:40 -05:00
Woosuk Kwon
95be2a7f22 [Model Runner V2] Minor simplification for DCP (#34786) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-02-18 11:04:53 -08:00
Jaden Mathias
0e60c925cf [Bugfix] Remove assert causing hipErrorStreamCaptureUnsupported (#34455) 
Signed-off-by: Jaden Mathias <jaden.mathias@amd.com>
 2026-02-18 18:54:54 +00:00
Teng Ma
d7ff22204a [Misc] Add mooncake-transfer-engine to kv_connectors requirements (#34826) 
Signed-off-by: Teng Ma <teng-ma@linux.alibaba.com>
 2026-02-18 18:26:24 +00:00
Isotr0py
c0bd8b13da [Bugfix] Redo Qwen3.5/Qwen3-Next GDN projector fusion (#34697) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: JJJYmmm <92386084+JJJYmmm@users.noreply.github.com>
 2026-02-18 09:46:53 -08:00
Michael Goin
caeb887bf6 [Bugfix] Fix NVFP4 TRTLLM MoE non-gated support; add gsm8k for Nemotron-3-Nano FP8+NVFP4 (#34725) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-02-18 09:39:22 -08:00
Ilya Markov
6b3166a7c7 [CI][Bugfix] Fix multinode test script (#34820) 
Signed-off-by: ilmarkov <markovilya197@gmail.com>
 2026-02-18 11:45:10 -05:00
Robert Shaw
25e2e136ef [CI] temporarily disable multi-node tests (#34825) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2026-02-18 11:32:44 -05:00
Robert Shaw
6874638bc4 [Model Bash] DeepSeek R1 BF16 Min Latency QKV A GEMM (0.5% E2E Speedup) (#34758) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2026-02-18 07:42:36 -08:00
Burkhard Ringlein
e24663c5a9 Add unit tests for fp8 output fusion of triton_attn (#34228) 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2026-02-18 06:22:49 -05:00
Nick Hill
c50e105a88 [Model Runner V2] Avoid prepare prefill kernel launch overhead (#34780) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-02-18 00:49:21 -08:00
Cyrus Leung
a766b30349 [Renderer] Deprecate code paths for old input processing (#34775) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-18 00:35:04 -08:00
Asaf Joseph Gardin
1faa8cb73c [Quantization] - Added uses_meta_device_weights to quant config (#34645) 
Signed-off-by: Josephasafg <ajgard7@gmail.com>
 2026-02-17 23:43:44 -08:00