biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
15,309 Commits 2 Branches 140 Tags
0e9358c11daf3f5a2d4e8f80a100b6d5e070e1a1
Commit Graph

15309 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Ekagra Ranjan
d0b402974f [Bugfix][Spec Decode] Avoid double call of Ngram CPU (#36952) 
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
 2026-03-13 20:33:19 +00:00
Divakar Verma
6341d43043 [ROCm][Quantization] add quark w4a8 mxfp4_fp8 for LinearLayer (#35316) 
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
 2026-03-13 19:44:24 +00:00
Mark McLoughlin
7afe0faab1 [Frontend][Core] Re-add shutdown timeout - allowing in-flight requests to finish (#36666) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-03-13 12:10:06 -07:00
Harry Mellor
5a3f1eb62f [Misc] Set default kv_buffer_device in a better way (#36862) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-13 19:07:33 +00:00
yugong333
b3ce711b93 Fp8 lora dense kernel (#35242) 
Signed-off-by: Yu Gong <yu3.gong@gmail.com>
 2026-03-13 19:05:08 +00:00
Isotr0py
abf61aaa8e [Bugfix] Fix Qwen2.5-omni/Qwen3-omni mm_processor cache for audio_in_video request (#36800) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-03-13 18:16:05 +00:00
bigmoyan
4508532fbd [Bugfix] fix paddleocr crash on some image shape (#36959) 
Signed-off-by: wangzhengtao <wangzhengtao@msh.team>
Signed-off-by: bigmoyan <moyan_work@foxmail.com>
Co-authored-by: wangzhengtao <wangzhengtao@msh.team>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-03-13 13:46:55 +00:00
Itay Alroy
d5af196c18 [2/N] Elastic EP Milestone 2: Integrating NIXL-EP (#35627) 
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
Co-authored-by: Yongji Wu <wuyongji317@gmail.com>
Co-authored-by: Ron Tourgeman <rtourgeman@nvidia.com>
 2026-03-13 09:25:33 -04:00
Chaojun Zhang
82f836d976 [XPU] Support LoRA via torch.compile on XPU platform (#36962) 
Signed-off-by: chzhang <chaojun.zhang@intel.com>
 2026-03-13 10:34:59 +00:00
Andreas Karatzas
4fccd30f19 [ROCm][CI] Upgrading orchestrator to handle python pipeline markers and options (#36181) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-03-13 02:04:22 -07:00
Or Ozeri
cfaf4668f7 [kv_offload+HMA][1/N]: Support multiple KV groups in OffloadingSpec (#36610) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2026-03-13 08:04:21 +00:00
Andreas Karatzas
99a57bdf74 [ROCm][CI] Corrected the GPT-OSS test root path (#36711) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-03-13 15:53:43 +08:00
Sage
a2268617cf [Frontend] Delegate preprocessing to OpenAIServingRender (#36483) 
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com>
 2026-03-13 00:39:43 -07:00
Rohan Potdar
a4ad9db541 Enable RoPE+KV cache fusion for ROCm AITER FA (non-shuffle layout) (#35786) 
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
 2026-03-13 07:33:22 +00:00
Nick Hill
b373b5102a [Tests] Shutdown test RemoteVLLMServer cleanly (#36950) 
Recent PR #33949 changed the teardown logic of the RemoteVLLMServer test utility class to
send SIGTERM to all vllm (sub)processes at once, which breaks the clean/coordinated
shutdown logic that assumes only the top-level process will receive a signal (for example
when running in a container that's shut down).

This caused a bunch of errors and stacktraces in some test logs, even though those tests
still pass. We should still attempt a normal shutdown and only kill other procs if they are
still running after a few seconds.

Example: tests/v1/distributed/test_external_lb_dp.py::test_external_lb_completion_streaming

Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-03-13 07:32:55 +00:00
Thomas Parnell
f296a1966d [Bugfix] Fix FlashInfer GDN warmup ValueError on SM90 GPUs (#36876) 2026-03-13 07:09:39 +01:00
Csrayz
bc2c0c86ef [Frontend] Fix usage incorrectly returned with empty stream_options` (#36379) 
Signed-off-by: Csrayz <33659823+Csrayz@users.noreply.github.com>
 2026-03-13 03:33:04 +00:00
jaime campos salas
891c60dcd5 fix(kv-cache): increase hybrid attention grouping threshold from 1.25 to 1.5 (#36684) 
Signed-off-by: Jaime Campos Salas <jaime.campos.salas@gmail.com>
 2026-03-12 23:28:27 -04:00
whyiug
1ce13cf992 [Model] Add support for BERT-like Chinese ERNIE pooling models (#36385) 
Signed-off-by: whyiug <whyiug@hotmail.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-03-13 03:23:53 +00:00
Nikita
10f08dedfa [Model] Add ColPali late interaction model for multi-modal retrieval (#36818) 
Signed-off-by: Nikita Sukharev <kaonael@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-03-13 02:18:57 +00:00
Aaron Hao
5e1a373d2e [BUG] Fix rank calculation in NCCLWeightTransferEngine (#36940) 
Signed-off-by: hao-aaron <ahao@anyscale.com>
 2026-03-13 01:56:51 +00:00
Simo Lin
572c776bfb build: update smg-grpc-servicer to use vllm extra (#36938) 
Signed-off-by: Simo Lin <linsimo.mark@gmail.com>
 2026-03-13 01:31:36 +00:00
Yifan Qiao
55d8073d06 [Bugfix] ep_scatter kernel store-load race condition (#34991) 
Signed-off-by: Yifan Qiao <yifanqiao@berkeley.edu>
 2026-03-13 01:07:59 +00:00
Nick Hill
cd32d6f586 [Model Runner V2] Some code simplification (#36929) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-03-13 00:59:23 +00:00
Jaewon
aaa3092f51 [MoE] Add routing simulation override for MXFP4 quantized MoE (#33595) 
Signed-off-by: Jaewon Lee <jaewon@meta.com>
 2026-03-13 00:30:44 +00:00
Shubhra Pandit
87985077a4 [Speculative Decoding] Add norm_before_fc for gpt-oss draft models (#36545) 
Signed-off-by: Shubhra Pandit <shubhra.pandit@gmail.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
 2026-03-12 23:03:32 +00:00
Ryan Rock
a79c1c2c80 [AMD][Build] Add DeepEP to ROCm Dockerfile (#36086) 
Signed-off-by: Ryan Rock <ryan.rock@amd.com>
 2026-03-12 21:33:32 +00:00
Andreas Karatzas
cc8f1f4764 [ROCm][CI] Preparing gfx90a mirroring (#36210) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-03-12 13:42:25 -07:00
Michael Goin
05b9e8ab5b Revise environment setup in AGENTS.md (#36909) 
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-12 19:21:11 +00:00
Xinan Miao
2cdf92228c [Feature]: Remove Chunking From FusedMoE (#34086) 
Signed-off-by: SouthWest7 <am1ao@qq.com>
Signed-off-by: Southwest <1403572259@qq.com>
Signed-off-by: southwest <am1ao@qq.com>
Signed-off-by: Xinan Miao <1403572259@qq.com>
Co-authored-by: SouthWest7 <am1ao@qq.com>
 2026-03-12 14:24:38 -04:00
Marc Sun
c973ecdead [bnb] Skip moe + bnb test (#36896) 
Signed-off-by: Marc Sun <marc@huggingface.co>
 2026-03-12 18:03:25 +00:00
Harry Mellor
e39257a552 Add AGENTS.md (#36877) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-03-12 10:20:50 -07:00
Dimitrios Bariamis
cc16b24b17 Update Flashinfer to 0.6.6 (#36768) 
Signed-off-by: Dimitrios Bariamis <12195802+dbari@users.noreply.github.com>
Co-authored-by: Dimitrios Bariamis <12195802+dbari@users.noreply.github.com>
 2026-03-12 13:19:19 -04:00
Eunkwang Jeon
bdc2343454 [Bugfix] Fix KeyError in parse_response_input for reasoning items with optional content (#34499) 
Signed-off-by: jeonsworld <jeonsworld@gmail.com>
 2026-03-13 00:13:36 +08:00
Matthew Bonanni
f444c05c32 [Attention] Use FA4 for MLA prefill (#34732) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-03-12 12:10:17 -04:00
SoluMilken
85199f9681 [Bugfix] fix main branch pre-commit error (1 line change) (#36897) 
Signed-off-by: SoluMilken <ypiheyn.imm02g@g2.nctu.edu.tw>
 2026-03-12 09:08:37 -07:00
grimulkan
a1257fd1ea [Kernel] Add FP8 KV cache support to Triton MLA decode attention (#34597) 
Signed-off-by: grimulkan <grimulkan@gmail.com>
 2026-03-12 08:32:34 -07:00
Thomas Parnell
abcffbba8c [CI] Fix mypy pre-commit errors on main (#36882) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-12 08:22:29 -07:00
Kunshang Ji
53ec16a705 [Hardware] Replace torch.cuda.device_count/current_device/set_device API (#36145) 
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-03-12 07:57:47 -07:00
Wei Zhao
2e693f48e7 [Perf] Add TRTLLM FP8 MoE Modular Kernel (#36307) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-03-12 07:32:31 -07:00
Martin Hickey
7f1f36bf91 [CI] Fix mypy for vllm/reasoning (#35742) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-12 12:21:33 +00:00
Mark McLoughlin
5282c7d4d0 [docs] Add lightweight AI assisted contribution policy (#30947) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2026-03-12 11:46:13 +00:00
caozuoba
9e19f8338b [Perf] add packed recurrent fast path for decode (#36596) 
Signed-off-by: hdj <1293066020@qq.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-03-12 04:01:57 -07:00
Sage
06e0bc21d2 [Frontend] Split OpenAIServingModels into OpenAIModelRegistry + OpenAIServingModels (#36536) 
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com>
 2026-03-12 03:29:37 -07:00
Chauncey
5a71cdd76e [Bugfix] Fix crash when tool_choice=required exceeds max_tokens (#36841) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-03-12 03:28:45 -07:00
Shanshan Shen
f0d3658c0f [MM][OOT] Support CPU seq_lens for OOT MMEncoderAttention kernels (#36605) 
Signed-off-by: shen-shanshan <467638484@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-03-12 03:28:23 -07:00
Michael Goin
57431d8231 [UX] Only show FP4 Marlin fallback warning for w4a4 models (#36806) 
Co-authored-by: Claude <noreply@anthropic.com>
 2026-03-12 05:19:35 -04:00
Xu Jinyang
3e64fe4a18 [Bugfix] Warm up Triton autotuner for GDN layers during V1 profiling (#36599) 
Signed-off-by: AuYang <459461160@qq.com>
 2026-03-12 00:51:09 -07:00
sfeiqiang
8cb24d3aed [KV Connector] Support using FlexKV as KV Cache Offloading option. (#34328) 
Signed-off-by: phaedonsun <phaedonsun@tencent.com>
Co-authored-by: phaedonsun <phaedonsun@tencent.com>
 2026-03-12 00:46:20 -07:00
István Ketykó
00726c74c9 [Bugfix][Model] Fix DeepSeek-OCR TensorSchema crash on empty images_crop (#36670) 
Signed-off-by: István Ketykó <istvan.ketyko@gmail.com>
 2026-03-12 15:35:54 +08:00