biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
14,230 Commits 2 Branches 140 Tags
186ea22efefd2c6f4f9b7fcb657bd00f50cb465a
Commit Graph

10081 Commits

Author SHA1 Message Date
Flora Feng
186ea22efe [Misc][Harmony] Move Responses API only harmony utils to responses/harmony.py (#35339) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-02-26 14:35:16 +08:00
Daniele
4a9c07a0a2 [BugFix] anthropic/serving_messages: fix tool call arguments streaming (#34887) 
Signed-off-by: Daniele Trifirò <dtrifiro@redhat.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2026-02-26 05:39:48 +00:00
Jason Li
9d37941017 [torch.compile] Sequence Parallelism threshold compile ranges (#28672) 
Signed-off-by: jasonlizhengjian <jasonlizhengjian@gmail.com>
Signed-off-by: Jason Li <jasonlizhengjian@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2026-02-26 05:00:12 +00:00
Fadi Arafeh
4171ff6dd9 [CPU][Feat] Enable KleidiAI INT8_W4A8 for all input dtypes (#34890) 
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2026-02-26 05:00:10 +00:00
Woosuk Kwon
13025e71e8 [Model Runner V2] Add coding style guide (#35325) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-02-25 20:42:40 -08:00
Hanjie Qiu
71dfce6aa6 [Kernel] Refactor FlashInfer allreduce for mnnvl backend (#34109) 
Signed-off-by: hjjq <50634613+hjjq@users.noreply.github.com>
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
 2026-02-26 03:17:20 +00:00
hujiaxin0
2aa4140402 openpangu-vl support video input (#34134) 
Signed-off-by: hujiaxin <524446785@qq.com>
Signed-off-by: Emilie1001 <79921183+Emilie1001@users.noreply.github.com>
Co-authored-by: Emilie1001 <79921183+Emilie1001@users.noreply.github.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-02-26 03:08:09 +00:00
Seungmin Kim
160424a937 [Bugfix] Fix CUDA compatibility path setting for both datacenter and consumer NVIDIA GPUs (#33992) 
Signed-off-by: Seungmin Kim <8457324+ehfd@users.noreply.github.com>
Signed-off-by: Andrew Mello <19512127+88plug@users.noreply.github.com>
Co-authored-by: 88plug <19512127+88plug@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-02-25 18:15:51 -08:00
Lucas Wilkinson
9511a3f8ee [Bugfix] Fix AttributeError in SMControlContextManager (#35338) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-02-25 18:01:10 -08:00
Michael Goin
de527e1cec [UX] Add --moe-backend arg for explicit kernel selection (#33807) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-02-25 17:44:44 -08:00
Yongye Zhu
1976356ee6 [MoE Refactor] MXFP4 Cutlass Experts to MK (#34542) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-02-25 17:32:39 -08:00
Michael Goin
cbf8f7028c [UX] Add --performance-mode {balanced,interactivity,throughput} (#34936) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-02-25 17:28:31 -08:00
Ming Yang
6831650c40 [offloader] v2: Hide weight onloading latency via prefetching (#29941) 
Signed-off-by: Ming Yang <minos.future@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-02-25 17:20:59 -08:00
Elizabeth Thomas
c97234c08b fix(mxfp4): Disable monolithic path for TRITON backend with EP (#34270) 
Signed-off-by: Elizabeth Thomas <email2eliza@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-02-25 13:33:42 -08:00
Lucas Wilkinson
15d76f74e2 Revert "[Misc] Enable weights loading tracking for quantized models" (#35309) 2026-02-25 09:20:15 -08:00
pushkar
5d18bf8b32 [Bugfix] Fix Harmony preamble visibility in Responses API (#32114) 
Signed-off-by: Pushkar Patel <git@thepushkarp.com>
Signed-off-by: pupa <pupa@users.noreply.github.com>
 2026-02-25 08:08:16 -08:00
haosdent
0788ff0a15 [Bugfix] Gracefully disable AllReduceFusionPass on GPUs without multicast support (#35085) 
Signed-off-by: haosdent <haosdent@gmail.com>
 2026-02-25 07:31:45 -08:00
Chendi.Xue
d72b0be33c [XPU]Fix for Qwen-OMNI crash (#35249) 
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
 2026-02-25 07:31:07 -08:00
Bhoomit
42489e43c2 [Misc][LoRA] Increase max vocab size limit to 258048 in logits processor (#34773) 
Signed-off-by: Bhoomit Vasani <vbhoomit@amazon.com>
 2026-02-25 23:30:55 +08:00
Mario Hong
af5e6afa0a [Bugfix] Fix step3p5 reasoning with interleaved thinking (#34211) 
Signed-off-by: mariohong <mariohong128@gmail.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-02-25 15:13:01 +00:00
Harry Mellor
90fc7f9109 Fix custom processors that use deleted behaviour for Transformers v5 (#35107) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-02-25 02:36:21 -08:00
Laura Wang
2465071510 [Perf] Add opt-in SM100 Oink RMSNorm custom-op path (#31828) 
Signed-off-by: Laura Wang <3700467+Laurawly@users.noreply.github.com>
Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com>
 2026-02-24 23:01:53 -08:00
Xinyu Chen
35d44b4557 [XPU]Support CUDAGraph on XPU Platform (#34482) 
Signed-off-by: Xinyu Chen <xinyu1.chen@intel.com>
Co-authored-by: chzhang <chaojun.zhang@intel.com>
Co-authored-by: zhenwei-intel <zhenwei.liu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-02-24 22:22:52 -08:00
Kunshang Ji
8ad54a991b [Platform] Add current_platform.num_compute_units interface (#35042) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
 2026-02-24 22:22:49 -08:00
Kunshang Ji
92510edc32 remove cuda check in top_k_top_p_triton kernel (#35011) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-02-24 22:22:31 -08:00
Isotr0py
a6c137521c [Misc] Add shard_id validation for MergedColumnLinear (#35055) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-02-24 22:12:28 -08:00
Isotr0py
4572a06afe [Misc] Enable weights loading tracking for quantized models (#35074) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-02-24 22:11:03 -08:00
Zhengxu Chen
5cc29cfb8b [compile] Improve error message during artifacts load failure. (#35115) 
Signed-off-by: zhxchen17 <zhxchen17@fb.com>
 2026-02-24 22:01:09 -08:00
Chen Zhang
8fae54faff [Linear Attention] fix bug for linear attention + prefix caching + reset_prefix_cache (#35157) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2026-02-24 22:00:19 -08:00
Harry Mellor
f7967577f5 Remove requirement to use --hf-overrides for DeepseekVLV2ForCausalLM (#35203) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-02-24 22:00:06 -08:00
pks
af770b8e7b [Bugfix] Fix AttributeError when passing StructuredOutputsParams to CompletionRequest (#35237) 
Signed-off-by: Patrick Simianer <patrick@lilt.com>
 2026-02-24 22:00:03 -08:00
Andreas Karatzas
2ff3e436ad [Responses][CI] Filter negative token IDs in schema fuzz test to avoid 500 errors (#35231) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-02-25 05:52:44 +00:00
Jhao-Ting Chen
c2c4c4611a [FIX] fused moe with lora shared expert dual stream (1.07x otps) (#34933) 
Signed-off-by: Jhao-Ting Chen <jhaotingc@nvidia.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-02-25 04:40:45 +00:00
Rohan Potdar
f38f8c9742 [ROCm]: Enable customop and rope+kvcache fusion for AITER RoPE (#35180) 
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
 2026-02-25 04:36:40 +00:00
Flora Feng
ec1d30c0f6 [Responses] Decouple SSE event helpers from Harmony context (#35148) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-02-24 20:05:25 -08:00
Pooya Davoodi
e3b2324ec4 [Frontend] Use init_app_state and FrontendArgs in run_batch (#32967) 
Signed-off-by: Pooya Davoodi <pooya.davoodi@parasail.io>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-02-24 19:40:39 -08:00
Nick Hill
dbf0da817a [Core] Cleanup engine pause/sleep logic (#34528) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-02-24 19:33:34 -08:00
yugong333
576fe50333 Adding Nemotron fp8 Triton MoE Config (#34674) 
Signed-off-by: Yu Gong <yu3.gong@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-02-24 15:56:38 -08:00
Benjamin Chislett
9fa5b25a23 [Bug][DSV3.2] Always prepare metadata for DeepGEMM Sparse Attention (#35075) 
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
 2026-02-24 14:55:22 -08:00
Benjamin Chislett
f5972a872f [Model][Spec Decode] Nemotron-H MTP and Mamba Speculative Decoding Support (#33726) 
Signed-off-by: Shahar Mor <smor@nvidia.com>
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Shahar Mor <smor@nvidia.com>
Co-authored-by: Roi Koren <roik@nvidia.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-02-24 09:49:56 -08:00
Wentao Ye
9ce8fad2a9 [Perf] Optimize Python Slice for Structured Output using islice instead of [:] (#33593) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-02-24 09:02:36 -08:00
Harry Mellor
c38b8d5a31 Remove padding_index from models that don't use it for better Transformers v5 compatibility (#35189) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-02-24 08:04:46 -08:00
danisereb
9609b1f18d Integrate flashinfer mm_mxfp8 in ModelOpt MXFP8 (#35053) 
Signed-off-by: Daniel Serebrenik <daserebrenik@nvidia.com>
 2026-02-24 08:45:13 -07:00
danisereb
a0c7081695 Fix fallback to default tactic (flashinfer autotuner) with trtllm_fp4_block_scale_moe (#35088) 
Signed-off-by: Daniel Serebrenik <daserebrenik@nvidia.com>
 2026-02-24 07:25:44 -08:00
R3hankhan
34ce0ffd1f [CPU][Perf] Accelerate Attention head for s390x using vector intrinsics (#34434) 
Signed-off-by: Rehan Khan <Rehan.Khan7@ibm.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2026-02-24 07:25:39 -08:00
Eldar Kurtić
a87cc50859 [Attn,KV-cache] Use per-head scales in the attention selector (#34281) 
Signed-off-by: Your Name <you@example.com>
Signed-off-by: Eldar Kurtic <research@neuralmagic.com>
Co-authored-by: Eldar Kurtic <research@neuralmagic.com>
Co-authored-by: Your Name <you@example.com>
 2026-02-24 09:02:43 -05:00
Cyrus Leung
761e63e541 [Frontend] Always pass supported_tasks to validation (#35186) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-24 04:16:33 -08:00
Isotr0py
d12d201409 [Bugfix] Fix failing FunASR processor test (#35111) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-02-24 04:13:45 -08:00
Wentao Ye
14561fabfd [Perf] Optimize pooling model redundant copy, 1.8% throughput improvement (#35127) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-02-24 04:13:11 -08:00
Zhengxu Chen
c77f3e1207 [compile] Save aot compile artifacts atomically. (#35117) 
Signed-off-by: zhxchen17 <zhxchen17@fb.com>
 2026-02-24 04:11:01 -08:00