biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
15,632 Commits 2 Branches 140 Tags
b6c9be509e869a8e1264bf1d8f4f33ccb756a365
Commit Graph

15632 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Shengqi Chen
b6c9be509e [CI] fix possible user permission issues in nightly index generation (#39390) 
Signed-off-by: Shengqi Chen <harry-chen@outlook.com>
 2026-04-09 08:14:07 +00:00
Qidong Su
ed733802f0 Fix NUMA binding on non-CDMM Grace-Blackwell systems (#39361) 
Signed-off-by: Qidong Su <soodoshll@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-09 07:36:51 +00:00
Andrew Barnes
8a34c5087a [ROCm] Remove unnecessary fp8 roundtrip in gather cache NHD dequant (#39122) 
Signed-off-by: Bortlesboat <bortstheboat@gmail.com>
 2026-04-09 15:12:22 +08:00
Wentao Ye
ed2f282bc8 [Perf] Optimize redundant sync for pooling model, 3.7% Throughput Improvement (#39113) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-04-08 23:12:23 -07:00
Zhewen Li
9e78555743 [Docker] Add fastsafetensors to NVIDIA Dockerfile (#38950) 2026-04-08 22:21:37 -07:00
sihao_li
e80e633927 [XPU] Skip VLLM_BATCH_INVARIANT for XPU in EAGLE DP test (#39164) 
Signed-off-by: sihao.li <sihao.li@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-04-09 12:45:16 +08:00
Khairul Kabir
490f17d0c7 [Multimodal] Fix nested_tensors_equal: add length check for lists and tuple support (#38388) 
Signed-off-by: khairulkabir1661 <khairulkabir1661@users.noreply.github.com>
Co-authored-by: khairulkabir1661 <khairulkabir1661@users.noreply.github.com>
 2026-04-09 04:40:37 +00:00
Yongye Zhu
2e98406048 [Refactor] Improve indexer decode path metadata preparation (#38865) 2026-04-08 20:49:15 -07:00
Chendi.Xue
ef5a226819 [PD][HeteroArch]Fix accuracy issue with CPU_ATTN as Decoder and Flash_ATTN as prefiller (#38935) 
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
 2026-04-09 11:19:07 +08:00
Wentao Ye
aec18492d0 [CI] Fix mypy for vllm/v1/ops (#39219) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-04-09 11:06:34 +08:00
noobHappylife
2a49284c8a Fix Responses JSON schema alias serialization (#38519) 
Signed-off-by: noobhappylife <aratar1991@hotmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-04-09 10:50:16 +08:00
Ilya Boytsov
d37b378762 [Model] Update ColModernVBERT to support latest HF checkpoint (#39307) 
Signed-off-by: Ilya Boytsov <ilyaboytsov1805@gmail.com>
 2026-04-09 10:48:51 +08:00
Wei Zhao
92fbec391b [Bug] Fix routing bias dtype for trtllm per-block fp8 moe (#38989) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-04-08 19:42:43 -07:00
Ajay Anubolu
2f41d6c063 [Bugfix] Fix cpu-offload-gb assertion with non-default block sizes (#36461) 
Signed-off-by: AjAnubolu <anuboluajay@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-04-08 19:42:16 -07:00
Dipika Sikka
3aecdf08b4 [Gemma4] Support quantized MoE (#39045) 
Signed-off-by: Dipika Sikka <dipikasikka1@gmail.com>
 2026-04-08 21:57:53 -04:00
Michael Goin
eb4205fee5 [UX] Integrate DeepGEMM into vLLM wheel via CMake (#37980) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-04-08 18:56:32 -07:00
liuzhenwei
83aea2147f [XPU][UT] update UTs in CI (#39296) 
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: Kunshang Ji <jikunshang95@gmail.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-04-09 09:38:16 +08:00
Maral
2e9034c998 [W8A8 Block Linear Refactor][2/N] Remove W8A8Fp8BlockLinearOp and adopt Fp8 block linear kernel selections. (#33892) 
Signed-off-by: maral <maralbahari.98@gmail.com>
Signed-off-by: Maral <maralbahari.98@gmail.com>
 2026-04-09 08:50:39 +08:00
Benjamin Chislett
8332078cfd [Bugfix] FlashInfer MXINT4 MoE crashes, missing do_finalize (#39315) 
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-04-08 20:36:33 -04:00
Richard Zou
ba4a78eb5d [torch.compile] Allow usage of Opaque Objects in PyTorch 2.11 (#39286) 
Signed-off-by: Richard Zou <zou3519@gmail.com>
 2026-04-08 23:21:10 +00:00
Kai Song
f3c7941ec8 [Bugfix]Fix EP precision for Qwen3.5, Qwen3-Next (#39181) 
Signed-off-by: Song Kai <songkai05@baidu.com>
 2026-04-09 01:47:48 +04:00
Wentao Ye
3352bf8b03 [CI Bug] Fix pre-commit issue in main (#39347) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-04-08 14:10:05 -07:00
triangleXIV
7c94ae16c6 [BugFix] --max-model-len=-1 causes over-limit requests to hang and starve the entire service (#39102) 
Signed-off-by: triangle14 <y1019026570@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-04-08 14:03:17 -07:00
Rishi Puri
ad05edfbca tests/v1/e2e/spec_decode: assert async scheduling is used (#39206) 
Signed-off-by: Rishi Puri <riship@nvidia.com>
Signed-off-by: Rishi Puri <puririshi98@berkeley.edu>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-04-08 20:30:03 +00:00
Wentao Ye
2018137242 [Feature] Batch invariant nvfp4 linear support (#39322) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-04-08 16:29:13 -04:00
Jackmin801
a776a48b1c [MoE] Move DEEP_GEMM into experts/ subdirectory (#39005) 
Signed-off-by: Jackmin801 <ongjackm@gmail.com>
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-04-08 19:23:08 +00:00
Ben Browning
8477fe427d [Tool] adjust_request to reasoning parser, and Gemma4 fixes (#39027) 
Signed-off-by: Ben Browning <bbrownin@redhat.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-04-08 19:04:04 +00:00
Lain
e24e0a43a4 [Attention] relax the head dim 512 and paged kv for sm90+FA4 (#38835) 
Signed-off-by: Siyuan Fu <siyuanf@nvidia.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-04-08 18:23:18 +00:00
Roberto L. Castro
b55d830ec7 [Perf][Kernel] Persistent TopK scheduler: unified CUDAGraph-safe kernel with dynamic per-row dispatch - DeepSeek-V3.2 DSA decode (#37421) 
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2026-04-08 13:35:57 -04:00
Shengqi Chen
75e01a39a1 [Feature] NUMA binding support for GPU workers (#38635) 
Signed-off-by: Shengqi Chen <harry-chen@outlook.com>
Co-authored-by: Jason Li <jasonlizhengjian@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-04-08 09:55:24 -07:00
Or Ozeri
512c5eb455 [kv_offload+HMA][5/N]: Track group block hashes and block IDs (#37109) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2026-04-08 19:50:28 +03:00
Flora Feng
13151a4df4 [Bugfix] Fix Gemma4 streaming tool call corruption for split boolean/number values (#39114) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-04-08 16:46:27 +00:00
Gregory Shtrasberg
56c976c1b5 [ROCm] Enable fused_silu_mul_block_quant on ROCm (#38817) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2026-04-08 11:23:32 -05:00
Frederik Gossen
d74a306c4b [Core] Use tuple_return in split_module for tuple-conformant subgraphs (#38752) 
Signed-off-by: Frederik Gossen <frgossen@meta.com>
Co-authored-by: Boyuan Feng <boyuan@meta.com>
 2026-04-08 09:09:58 -07:00
Gregory Shtrasberg
0e9f0a516c [ROCm][CI-Build] Cherry pick triton BUFFER_OPS fix and update AITER (#38580) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2026-04-08 10:38:03 -05:00
haosdent
8904fc4d19 [Bugfix] Fix V1 logprobs empty strings for multi-byte UTF-8 tokens when logprobs > 0 (#34875) 
Signed-off-by: haosdent <haosdent@gmail.com>
 2026-04-08 15:30:00 +00:00
nemanjaudovic
1a2c17634e [Bugfix] Add missing ASRDataset import and CLI args in benchmarks/throughput.py (#38114) 
Signed-off-by: nemanjaudovic <nudovic@amd.com>
 2026-04-08 13:53:53 +00:00
Matthew Bonanni
308cec5864 [FlashAttention] Symlink FA4 instead of copying when using VLLM_FLASH_ATTN_SRC_DIR (#38814) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-04-08 12:04:34 +00:00
wang.yuqi
4e2ab1861d [CI Failure] pin nomic-embed-text-v1 revision (#39292) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-04-08 11:43:06 +00:00
JartX
140cbb1186 [Bugfix] Cuda Clean up scales Kvcache fp8/int8_per_token_head (#39224) 
Signed-off-by: JartX <sagformas@epdcenter.es>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-04-08 04:08:04 -07:00
Kevin H. Luu
6155bbd1dd [Bugfix][Docs] Fix ReadTheDocs build crash from mocked torch decorator (#39284) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-08 09:43:01 +00:00
rasmith
78434b923c [CI][AMD][BugFix][Kernel] Cast induction variable to int64 on MI350 for chunk_gated_delta_rule_fwd_kernel_h_blockdim64 to avoid illegal memory access (#39087) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
 2026-04-08 16:57:18 +08:00
Michael Goin
2488d1dca2 [Docs] Update README (#39251) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-04-08 11:34:07 +08:00
yoke
d734445fcd [Bugfix][Frontend] Fix Gemma4 streaming HTML duplication after tool calls (#38909) 
Signed-off-by: yoke233 <yoke2012@gmail.com>
 2026-04-08 11:03:54 +08:00
Flora Feng
927975ead8 [Parser] Migrate response api streaming to unified parser (#38755) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Signed-off-by: Andrew Xia <axia@meta.com>
 2026-04-08 10:09:00 +08:00
Flora Feng
9ea7d670d8 [Bugfix] Fix Qwen3 tool parser for Responses API tools (#38848) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-04-08 10:08:51 +08:00
Varun Sundar Rabindranath
7b80cd8ac3 [Docs] Add Phi-4-reasoning-vision to supported models + examples (#39232) 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
 2026-04-08 02:02:26 +00:00
Andrey Talman
2111997f96 [release 2.11] Update to torch 2.11 (#34644) 2026-04-07 18:55:48 -07:00
Flora Feng
5af684c319 [CI] Add reasoning parser tests to CI (#37025) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-04-08 00:57:36 +00:00
Md. Mekayel Anik
d521dcdbcc docs: clarify SMT and OMP acronyms in CpuPlatform (#39085) 2026-04-07 17:42:07 -07:00