biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
14,386 Commits 2 Branches 140 Tags
fa6a6be51978bd4b49ba0da17039e60f96dc5b13
Commit Graph

14386 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Nicolò Lucchesi
8e962fef5f [CI][Nixl] Add CrossLayer KV layout tests (#34615) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-02-17 21:35:40 +08:00
Cyrus Leung
574fe75245 [Renderer] Move InputPreprocessor into Renderer (2/2) (#34560) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-17 05:29:01 -08:00
junuxyz
c61a98f529 [CI][BugFix] ShellCheck cleanup to remove baseline and preserve runtime behavior (#34514) 
Signed-off-by: junuxyz <216036880+junuxyz@users.noreply.github.com>
 2026-02-17 12:22:56 +00:00
Harry Mellor
28bffe9466 Fix docs build warning (#34686) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-02-17 02:31:40 -08:00
ChenqianCao
ad65177a19 [Bugfix] Fix 'remove_instance_endpoint' method logic in disagg_proxy_demo (#32922) 
Signed-off-by: ChenqianCao <39755070+ChenqianCao@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-02-17 10:06:53 +00:00
Tim Dettmers
d44a5b6c47 Remove dead bitsandbytes CxB code from 8-bit inference path (#34633) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-17 01:49:14 -08:00
Jiangyun Zhu
1d65283e95 Revert "[Models] Fuse Qwen3.5 GDN's qkvz_proj and ba_proj" (#34683) 2026-02-17 01:29:27 -08:00
kourosh hakhamaneshi
c464b57374 [Ray] Propagate third-party env vars to Ray workers via prefix matching (#34383) 
Signed-off-by: Kourosh Hakhamaneshi <kourosh@anyscale.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-02-17 01:08:42 -08:00
Amr Mahdi
c5c38e152a [CI] Fix bake config artifact path for AMI rebuild pipeline (#34656) 
Signed-off-by: Amr Mahdi <amrmahdi@meta.com>
 2026-02-17 06:39:44 +00:00
Woosuk Kwon
d00df624f3 [Model Runner V2] Minor refactoring for penalties (#34662) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-02-16 21:43:00 -08:00
Woosuk Kwon
9752da9d9c [Model Runner V2] Minor simplification for BadWordsState (#34669) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-02-16 21:27:24 -08:00
Woosuk Kwon
04925b2202 [Model Runner V2] Minor cleanup for PP (#34666) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-02-16 19:15:31 -08:00
Woosuk Kwon
d74278fb67 [Model Runner V2] Fix unintended CPU-GPU sync in make_dummy (#34667) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-02-16 19:00:29 -08:00
haosdent
b68fd899d1 [Bugfix] Fix fused MoE int32 overflow in stride*offset without perf regression (#34507) 
Signed-off-by: haosdent <haosdent@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-02-16 17:58:49 -08:00
Aneesh Puttur
0b5f9b7204 [CI] Enable mypy import following for vllm/v1/kv_offload (#34639) 
Signed-off-by: Aneesh Puttur <aneeshputtur@gmail.com>
 2026-02-17 09:58:15 +08:00
zhanqiuhu
9a8853f781 [Core] Pipeline Parallel support for Model Runner V2 (#33960) 
Signed-off-by: Zhanqiu Hu <zh338@cornell.edu>
 2026-02-16 17:48:16 -08:00
zhrrr
387a1898d9 [Model Runner V2] support bad_words sampling param (#33433) 
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-02-16 16:36:06 -08:00
roikoren755
3b30e61507 [NemotronH] Do not force router to run in fp32 (#34582) 
Signed-off-by: Roi Koren <roik@nvidia.com>
 2026-02-16 10:15:32 -08:00
Alexei-V-Ivanov-AMD
824f9e8f3c Targeting the MI355 agent pool with all existing tests (#34629) 
Signed-off-by: Alexei V. Ivanov <alexei.ivanov@amd.com>
 2026-02-16 17:02:27 +00:00
Nicolò Lucchesi
6cc403e67d [Bugfix][CI] Fix flaky entrypoints/openai/test_response_api_with_harmony.py::test_function_calling[openai/gpt-oss-20b] (#34624) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-02-16 16:11:07 +00:00
Almog Tavor
72d5951d02 [Bugfix] Treat generation_config max_tokens as default not ceiling (#34063) 
Signed-off-by: almogtavor <almogtavor@gmail.com>
 2026-02-16 07:58:24 -08:00
Lucas Kabela
a3205beffb [CI] Enable mypy coverage for individual excluded files (#34292) 
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-02-16 07:34:29 -08:00
Christian Pinto
6930becd45 (bugfix): Fixed encode in LLM entrypoint for IOProcessr plugin prompts (#34618) 
Signed-off-by: Christian Pinto <christian.pinto@ibm.com>
 2026-02-16 07:33:55 -08:00
Andreas Karatzas
03a8770a6d [ROCm][CI] Fix plugins test group; updating terratorch and dependencies (#34589) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-02-16 07:33:42 -08:00
Yiqi Xue
bc56a1d56e [Bugfix] Fix ARC touch KeyError for non-ready T1 blocks in kv offload (#34576) 
Signed-off-by: Yiqi Xue <xuey666@gmail.com>
 2026-02-16 07:33:19 -08:00
danisereb
ec7d9e6745 Fix call to moe_mk in modelopt MoE modules (required for LoRA) (#34575) 
Signed-off-by: Daniel Serebrenik <daserebrenik@nvidia.com>
 2026-02-16 07:33:09 -08:00
Isotr0py
3bb4e4311c [Models] Fuse Qwen3.5 GDN's qkvz_proj and ba_proj (#34492) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-02-16 07:32:51 -08:00
Amr Mahdi
08f8c198ae [CI] Disable precompiled wheel path in CI image builds (#34606) 
Signed-off-by: Amr Mahdi <amrmahdi@meta.com>
 2026-02-16 15:14:43 +00:00
Harry Mellor
a21cedf4ff Bump lm-eval version for Transformers v5 compatibility (#33994) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-02-16 05:24:35 -08:00
emricksini-h
3ef74cde5d [CI][Tracing] Fix race condition by adding server readiness check (#34364) 
Attempt to resolve #34284: "Metrics Tracing (2GPU)" fails with a
segmentation fault.

Signed-off-by: emricksini-h <emrick.birivoutin@hcompany.ai>
 2026-02-16 12:57:39 +00:00
Ekagra Ranjan
cd81cdb399 [Scheduler][ASR] Fix CrossAttn blocks per-request for Variable length encoder inputs (#31058) 
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2026-02-16 11:08:44 +00:00
Andreas Karatzas
1e828573b4 [CI][Metrics] Stabilize tests with polling and subprocess guards (#34566) 
test_abort_metrics_reset is flaky due to hardware-dependent
fixed sleeps: replace fixed sleeps with polling.

test_metrics_exist_run_batch passes even when the engine crashes
on startup (false positive): add subprocess lifecycle guards.

Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-02-16 10:52:02 +00:00
Samu Tamminen
a5ccc85c8c [Bugfix] Fix Dynamo unexpected keyword argument (#34320) 
Signed-off-by: Samu Tamminen <stammine@amd.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-02-16 01:32:30 -08:00
Roger Wang
b5475d0534 Revert "[Misc] fix qwen3.5 config" (#34610) 2026-02-16 01:06:05 -08:00
JJJYmmm
9521002f0a [Misc] fix qwen3.5 config (#34604) 2026-02-16 00:25:38 -08:00
Cyrus Leung
ec17bdd894 [Renderer] Move InputPreprocessor into Renderer (1.5/2) (#34598) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-15 23:46:33 -08:00
Amr Mahdi
bb59c90248 [CI] Write bake config to temp directory instead of repo root (#34569) 
Signed-off-by: Amr Mahdi <amrmahdi@meta.com>
 2026-02-15 22:15:47 -08:00
bnellnm
5bff999d12 [Bugfix] Add method to swap quant_method on FusedMoE to fix LoRA issues (#34453) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2026-02-15 20:10:50 -08:00
Lucas Wilkinson
bb85929aa6 [BugFix] Fix Python 3.13 FlashMLA import error (#34548) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-02-15 20:09:18 -08:00
Parth Bansal
5653021094 [Doc] Add Mistral-7b-v0.3 model to the batch invariance validated model (#34584) 
Signed-off-by: Parth Bansal <parthbansal127@gmail.com>
 2026-02-16 12:09:00 +08:00
Andreas Karatzas
974d829b05 [CI][Frontend] Return 422 instead of 500 for invalid Anthropic tool_choice (#34590) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-02-15 20:06:48 -08:00
Isotr0py
91ac5d9bfd [CI/Build] Enable tests for recent day-0 new models (#34585) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-02-15 18:17:04 -08:00
Luka Govedič
23d825aba1 [torch.compile] Disable ar-rms fusion for ds3-fp4 & DP, fix CI test (#34392) 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-02-15 06:33:57 -08:00
Maryam Tahhan
f07a128413 [CPU][ARM] Add ARM BF16 cross-compilation support and improve documen… (#33079) 
Signed-off-by: Maryam Tahhan <mtahhan@redhat.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2026-02-15 06:33:08 -08:00
Isotr0py
71cd89264f [MM Encoder] Add Triton ViT attention backend (#32183) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-02-15 06:32:47 -08:00
Isotr0py
19fab44152 [Doc] Update Encoder-Decoder models support doc with Florence-2 (#34581) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-02-15 04:18:57 -08:00
Seiji Eicher
79c7e09235 [KV Connector] Add temporary, off-by-default VLLM_DISABLE_REQUEST_ID_RANDOMIZATION workaround (#34415) 
Signed-off-by: Seiji Eicher <seiji@anyscale.com>
 2026-02-14 23:26:10 -08:00
haosdent
79f3fab05a [Bugfix] Handle num_expert_group=None in flashinfer block-scale FP8 MoE (#34494) 
Signed-off-by: haosdent <haosdent@gmail.com>
 2026-02-14 23:25:46 -08:00
Vadim Gimpelson
604b9eaec5 [BUGFIX] Fix accuracy regression for NVIDIA-Nemotron-3-Nano-30B-A3B-NVFP4 with TP>1 (#34476) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
 2026-02-14 23:25:17 -08:00
Stanislav Kirillov
50dbd6c9e6 [bugfix] Fix critical bug when reporting for all paths where handler.create_error_response is used (#34516) 
Signed-off-by: Stanislav Kirillov <stas@nebius.com>
Co-authored-by: Stanislav Kirillov <stas@nebius.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-02-14 23:24:25 -08:00