biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
13,740 Commits 2 Branches 140 Tags
9bdb06b4368e304bc5e23c8df2dff8f8b2ccf0f6
Commit Graph

13740 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
zofia
9bdb06b436 [XPU][6/N] add xpu scaled_mm kernel (#34117) 
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
 2026-02-09 20:17:35 +08:00
Nikhil Gupta
caad9f1e01 [Fix] [CPU Backend] : Prepack weights for w8a8 oneDNN matmul (#33901) 
Signed-off-by: nikhil-arm <nikhil.gupta2@arm.com>
 2026-02-09 18:04:41 +08:00
Ekagra Ranjan
1d5922fade [ASR] Fix audio benchmark and add RTFx metric (#32300) 
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@gmail.com>
 2026-02-09 10:02:37 +00:00
Andreas Karatzas
3025b3cebb [CI] Remove empty image_size_factors for fuyu, glm4_1v, glm_ocr (#34107) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-02-09 17:37:04 +08:00
Jee Jee Li
978a37c823 [Model] GLM adaptation (#34124) 2026-02-09 17:32:52 +08:00
ihb2032
5a5c43511a fix(cpu): fix mla_decode compilation on x86 without AVX512 (#34052) 
Signed-off-by: ihb2032 <hebome@foxmail.com>
Co-authored-by: root <root@LAPTOP-FKNHV411.localdomain>
 2026-02-09 08:55:41 +00:00
Nick Hill
d9bede0314 [BugFix] Fix fastsafetensors TP all procs using all GPUs (#34070) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-02-09 15:15:46 +08:00
wang.yuqi
22b64948f6 [Frontend][last/5] Make pooling entrypoints request schema consensus. (#31127) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
v0.16.0rc1 		2026-02-09 06:42:38 +00:00
Reagan Lee
7c233dbb36 [Tiny] Rename encoder budget file to more specific name (#34103) 
Signed-off-by: Reagan Lee <“reaganjlee@gmail.com”>
Co-authored-by: Reagan Lee <“reaganjlee@gmail.com”>
 2026-02-09 03:48:19 +00:00
kourosh hakhamaneshi
a75a5b54c7 [bug-fix] supported_tasks is breaking backward compatibility at init_app_state (#34027) 
Signed-off-by: Kourosh Hakhamaneshi <kourosh@anyscale.com>
Signed-off-by: kourosh hakhamaneshi <31483498+kouroshHakha@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-02-09 09:46:46 +08:00
Andrey Talman
f97ca67176 [Release 2.10] Update to Torch 2.10 - final release (#30525) 2026-02-08 13:51:09 -08:00
danisereb
084aa19f02 Add support for ModelOpt MXFP8 dense models (#33786) 
Signed-off-by: Daniel Serebrenik <daserebrenik@nvidia.com>
 2026-02-08 11:16:48 -08:00
navmarri14
1ecfabe525 glm 4.6 fused tuned inference config for B200 (#32958) 2026-02-08 18:55:47 +00:00
Richard Zou
4df841fe75 [torch.compile] Add an option to force-enable the MOE cold start optimization (#33735) 
Signed-off-by: Richard Zou <zou3519@gmail.com>
 2026-02-08 18:42:56 +00:00
TomerBN-Nvidia
a263aa6140 [BugFix] Change support no act and mul for marlin (#34088) 
Signed-off-by: Tomer Natan <tbarnatan@computelab-frontend-8.nvidia.com>
Co-authored-by: Tomer Natan <tbarnatan@computelab-frontend-8.nvidia.com>
 2026-02-08 17:18:22 +00:00
aabbccddwasd
179ae7da8f [Revert] Fix performance regression for GLM-4.7-GPTQ decode and MTP acceptance rate (#33771) 
Signed-off-by: aabbccddwasd <aabbccddwasd@qq.com>
 2026-02-08 08:13:24 -08:00
Reagan Lee
c4df59ad43 Add embedding input functionality for disabled modalities [remake] (#32493) 
Signed-off-by: Reagan Lee <“reaganjlee@gmail.com”>
Signed-off-by: Reagan Lee <reaganjlee@gmail.com>
Signed-off-by: Reagan Lee <96998476+reaganjlee@users.noreply.github.com>
Co-authored-by: Reagan Lee <“reaganjlee@gmail.com”>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-02-08 04:57:16 -08:00
TJian
785cf28fff [ROCm] [CI] Reduce Resource of two test groups (#34059) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2026-02-08 15:17:26 +08:00
Nick Hill
a96197f564 [Perf] Simplify DeepseekV32 tokenizer, ensure fast detokenization used (#33855) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-02-08 07:16:34 +00:00
Andreas Karatzas
ab10d79855 [ROCm][Bugfix] fix act_quant_fusion module import error (#34069) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-02-07 19:21:12 -08:00
Cyrus Leung
7fcb705b80 [CI/Build] Skip GCS test (#34057) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-07 08:52:38 -08:00
Cyrus Leung
b956cdf818 [Doc] Fix run_batch docs (#34056) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-07 06:18:16 -08:00
Hashem Hashemi
ed17f54c8b Perf tuning and expansion of cases covered for wvSplitKrc (#33493) 
Signed-off-by: Hashem Hashemi <hashem.hashemi@amd.com>
 2026-02-07 05:33:11 -08:00
Jiang Wu
860981d8d8 Make directory exist ok for ray spinning up multiple replicas on a single instance (#33604) 
Signed-off-by: Jiang Wu <jwu@cclgroup.com>
 2026-02-07 05:30:49 -08:00
zifeitong
52181baaea Update DeepGEMM version pin in Dockerfile to match #32479 (#33935) 
Signed-off-by: Zifei Tong <zifeitong@gmail.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-02-07 05:30:22 -08:00
Rohan Potdar
de3869bb4d move checks out of unified_kv_cache_update custom op (#33943) 
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
 2026-02-07 05:30:09 -08:00
whx
ce9b3cd3e9 [PluggableLayer][3/N] Apply PluggableLayer to mamba layers. (#33660) 
Signed-off-by: whx-sjtu <2952154980@qq.com>
 2026-02-07 05:26:05 -08:00
Jee Jee Li
db4ede9743 [Model] Enable Step3p5ForCausalLM testing (#33755) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-02-07 05:25:24 -08:00
Pooya Davoodi
2cb2340f7a [Frontend]Add support for transcriptions and translations to run_batch (#33934) 
Signed-off-by: Pooya Davoodi <pooya.davoodi@parasail.io>
Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-02-07 05:24:57 -08:00
TundeAtSN
4df44c16ba Enable Eagle3 speculative decoding for Mistral3ForConditionalGeneration to support eagle3 (#33939) 
Signed-off-by: Akintunde Oladipo <akintunde.oladipo@servicenow.com>
Signed-off-by: TundeAtSN <akintunde.oladipo@servicenow.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-02-07 05:24:52 -08:00
Richard Zou
81fe69cae5 [torch.compile] Stop compiling identical artifacts (#34003) 
Signed-off-by: Richard Zou <zou3519@gmail.com>
 2026-02-07 05:24:48 -08:00
Mohammad Miadh Angkad
dd6a6e1190 [Kernel] Add KernelConfig flag to enable/disable FlashInfer autotune (#34006) 
Signed-off-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2026-02-07 05:24:44 -08:00
Cyrus Leung
edb359cce4 [Renderer] Define render_cmpl and render_chat (#34039) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-07 05:24:40 -08:00
wang.yuqi
6ed5eda300 [CI][Build] Pin grpcio-tools==1.78.0 (#34048) 
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-02-07 05:24:35 -08:00
Cyrus Leung
11a4c9d30d [Misc] Simplify get_max_tokens (#34036) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-07 00:59:49 -08:00
lukec
15a0b9e570 Fix spelling errors (#33978) 2026-02-06 23:58:50 -08:00
Andreas Karatzas
c490d8cc73 [ROCm][CI] Pinning lm-eval version to resolve multi-modal small eval bug (#34038) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-02-06 22:21:08 -08:00
Cyrus Leung
48312e579a [Misc] Make PlaceholderRange.get_num_embeds a method (#34035) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-07 05:30:17 +00:00
Vel
bc32444b23 [Kernel] Add enable_sm120_or_later for SM121 (DGX Spark) CUTLASS support (#33517) 
Signed-off-by: code4me2 <velvetmoon222999@gmail.com>
 2026-02-06 20:28:01 -08:00
Wentao Ye
18e8545297 [Revert] Add util handle_deprecated back (#33998) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-02-07 04:14:45 +00:00
果冻虾仁
6f7adc533a fix description in plugin_system.md (#33999) 2026-02-06 19:37:02 -08:00
Nick Hill
40218a82ba [ModelRunner V2] Revert token rank comparison difference for now (#34017) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-02-07 11:11:05 +08:00
kourosh hakhamaneshi
1c3b22058f [Misc] Add backward-compatible import aliases for renamed translations module (#34015) 
Signed-off-by: Kourosh Hakhamaneshi <kourosh@anyscale.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-02-07 11:01:41 +08:00
Xin Yang
3920cafdd6 [Bugfix] Fix _fused_moe_lora_expand signature mismatch (#33821) 
Signed-off-by: Xin Yang <xyangx@amazon.com>
 2026-02-07 10:45:59 +08:00
rasmith
ec28784fdc [CI][AMD]Bugfix] Check that model_config is not None in enable_norm_pad_fusion (#34007) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
 2026-02-07 02:43:25 +00:00
Nicolò Lucchesi
55aeec04f5 [Bugfix] Fix Whisper tokenization (#34011) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-02-07 10:42:52 +08:00
Ikenna
906077181b [Bugfix] Fix QK Norm+RoPE fusion pattern matching on B200+FP8 (#33967) 
Signed-off-by: Ikenna <ikennachifo@gmail.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2026-02-07 02:27:33 +00:00
Aaron Hao
89a385d79f [Feat][RL] Pause and Resume with keep requests for single engine (#32351) 
Signed-off-by: ahao-anyscale <ahao@anyscale.com>
Signed-off-by: Aaron Hao <ahao@anyscale.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-02-07 00:08:58 +00:00
kourosh hakhamaneshi
4a2d00eafd [bugfix] [ROCm] Fix premature CUDA initialization in platform detection (#33941) 
Signed-off-by: Kourosh Hakhamaneshi <kourosh@anyscale.com>
 2026-02-06 16:17:55 -06:00
Dimitrios Bariamis
207c3a0c20 Fix RoutingMethodType logic (#33919) 
Signed-off-by: Dimitrios Bariamis <12195802+dbari@users.noreply.github.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Dimitrios Bariamis <12195802+dbari@users.noreply.github.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-02-06 14:03:34 -08:00