biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
13,562 Commits 2 Branches 140 Tags
f0d525171557e3fe74e8e6df52257f9d66831d3f
Commit Graph

4336 Commits

Author SHA1 Message Date
gmagogsfm
bde5039325 [CI] Add compile/test_multimodal_compile.py to CI (#28151) 
Signed-off-by: Yanan Cao <gmagogsfm@gmail.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-11-06 05:41:47 +00:00
Vadim Gimpelson
f948ab6945 [CI Failure] nm-testing/Qwen2-0.5B-Instruct-FP8-SkipQKV was removed from HF. Skip it in tests (#28170) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
 2025-11-06 01:22:13 +00:00
Zhewen Li
0b8e871e5e [CI/Build] Fix test_defaults_with_usage_context in AMD CI (#27926) 
Signed-off-by: zhewenli <zhewenli@meta.com>
 2025-11-05 15:40:24 -08:00
Snehlata
e15601789b [Feature]: Add corrupted request metric to V1 metrics system. (#27306) 
Signed-off-by: atalhens <sneh.lata@nutanix.com>
 2025-11-05 13:45:29 -08:00
Paul Zhang
faedbb4d4f [Feature] Extend batch invariant torch.compile to B200 (#27856) 
Signed-off-by: PaulZhang12 <paulzhan@fb.com>
 2025-11-05 10:04:49 -08:00
Samuel Shen
40db194446 [CI]: Add LMCacheConnector Unit Tests (#27852) 
Signed-off-by: Samuel Shen <slshen@uchciago.edu>
Co-authored-by: Samuel Shen <slshen@uchciago.edu>
Co-authored-by: Yihua Cheng <yihua98@uchicago.edu>
 2025-11-05 09:45:57 -08:00
Isotr0py
3f5a4b6473 [Bugfix] Validate custom logits processor xargs for online serving (#27560) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-11-05 16:53:33 +00:00
Pleaplusone
6cae1e5332 [ROCm][MLA] Support block-size > 1 for AITER MLA backend (#27224) 
Signed-off-by: ganyi <ygan@amd.com>
Co-authored-by: wuhuikx <hattie.wu@amd.com>
 2025-11-05 10:43:02 -05:00
Ilya Markov
e50c454672 [BugFix] Support EP/DP + EPLB with MTP (#25311) 
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Signed-off-by: Sage Moore <sage@neuralmagic.com>
Co-authored-by: Sage Moore <sage@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2025-11-05 15:22:17 +00:00
amirkl94
6b7a81185d Bugfix: Cutlass FP8 FusedMoE bad scaling factors (#27255) 
Signed-off-by: Amir Klein <203507526+amirkl94@users.noreply.github.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-11-05 06:06:06 -05:00
Kuntai Du
86dca07d9b [Hybrid allocator + kv connector] revert connector test changes related to hybrid allocator (#28011) 
Signed-off-by: KuntaiDu <kuntai@uchicago.edu>
 2025-11-05 10:36:31 +00:00
Chauncey
e261d37c9a [Refactor] Lazy-loaded reasoning_parser (#28092) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-11-05 15:37:02 +08:00
Alex Brooks
b7cbc25416 [Model, Core] Support Granite Speech & LoRA for STT (#24455) 2025-11-05 08:33:48 +01:00
Isotr0py
0ff05e3770 [Bugfix] Fix encoder-only model support for transformers backend (#28021) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-11-04 22:24:41 -08:00
wangxiyuan
428bc7bf1c [V0 deprecation] Remove VLLM_USE_V1 usage in most modules (#27955) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2025-11-04 20:51:16 -08:00
Nick Hill
938a81692e [AsyncScheduling] Don't schedule past request max_tokens (#27922) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-11-04 17:06:28 +00:00
Nick Hill
c9f66da8fd [PerfFix] Avoid separate thread for MP executor shm spin (#28012) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-11-04 08:33:55 -08:00
yt0428
05cae69f0f [model] Add support for openPangu_Ultra_MoE (#27521) 
Signed-off-by: yuantao <2422264527@qq.com>
Signed-off-by: yt0428 <51468697+yt0428@users.noreply.github.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-11-04 08:17:20 -08:00
Jerry Zhang
03c4c4aa9d Support using Int4PreshuffledTensor after loading (#26066) 
Signed-off-by: Jerry Zhang <jerryzh168@gmail.com>
 2025-11-04 06:00:57 -05:00
yugong333
2ec401bc39 Load tuned fused_moe_lora shrink and expand kernel configs separately (#27435) 
Signed-off-by: Yu Gong <yu3.gong@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-11-04 18:27:35 +08:00
Varun Sundar Rabindranath
4022a9d279 [BugFix][Performance] Restore flashinfer autotuning for all scenarios (#27904) 2025-11-04 15:56:21 +08:00
Mark McLoughlin
58279c60b5 [KV Connector] Make KVCacheConfig an explicit constructor argument (#27887) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2025-11-03 23:00:49 -08:00
Chauncey
c02fccdbd2 [Refactor] Lazy import tool_parser (#27974) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-11-04 10:10:10 +08:00
Matthew Bonanni
01baefe674 Add TP parameter to attention tests (#27683) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-11-03 13:04:40 -08:00
Aurick Qiao
2c19d96777 [Spec Decode] Integrate Suffix Decoding from Arctic Inference (#25784) 
Co-authored-by: Aurick Qiao <aurick.qiao@snowflake.com>
 2025-11-03 09:23:31 -08:00
Lucas Wilkinson
4bc400f47e [CI/Testing] Add basic single node dual batch overlap test (#27235) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-11-03 17:00:46 +00:00
pwschuurman
f7d2946e99 [Bugfix] Skip gs:// model paths for speculator detection (#27846) 
Signed-off-by: Peter Schuurman <psch@google.com>
 2025-11-03 14:31:03 +00:00
gnovack
294c805f1d Early exit for MoE LoRA kernels (#27131) 
Signed-off-by: gnovack <gnovack@amazon.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-11-03 20:22:17 +08:00
zhang-prog
40b69e33e7 [Model] Add PaddleOCR-VL Model Support (#27758) 
Signed-off-by: zhangyue <zhangyue66@baidu.com>
Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: zhangyue66 <zhangyue66@baidu.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-11-03 19:04:22 +08:00
Jee Jee Li
32257297dd [CI/Build] Remove the flaky gpt-oss lora test (#27966) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-11-03 16:50:06 +08:00
Misha Efimov
ba464e6ae2 Add ORCA endpoint load metrics support (#24905) 
Signed-off-by: Misha Efimov <mef@google.com>
 2025-11-03 08:21:31 +00:00
Rémi Delacourt
cec7c28833 [Bugfix] Padded Eagle Specdec with Chunked Prefill (#26263) 
Signed-off-by: Rémi Delacourt <remi@mistral.ai>
Signed-off-by: Rémi Delacourt <54138269+Flechman@users.noreply.github.com>
Signed-off-by: remi <remi@mistral.ai>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
 2025-11-03 02:22:46 -05:00
Biswa Panda
1bf43ae35d [BugFix][LoRA] use adapter_id instead of id field of lora_request (#27728) 
Signed-off-by: Biswa Panda <biswa.panda@gmail.com>
 2025-11-03 10:08:08 +08:00
Vensen
0ce743f4e1 Fix(llm): Abort orphaned requests when llm.chat() batch fails Fixes #26081 (#27420) 
Signed-off-by: vensenmu <vensenmu@gmail.com>
 2025-11-02 16:24:01 +00:00
Asaf Joseph Gardin
00b31a36a2 [V1] [Hybrid] Mamba1 Automatic Prefix Caching (#26377) 
Signed-off-by: asafg <39553475+Josephasafg@users.noreply.github.com>
 2025-11-02 04:16:23 -08:00
Ben Browning
758ea2e980 [CI/Build] Fix flaky test_transcription_validation.py::test_basic_audio_gemma (#27924) 
Signed-off-by: Ben Browning <bbrownin@redhat.com>
 2025-11-02 03:45:02 +00:00
Benjamin Bartels
1e88fb751b Adds anthropic /v1/messages endpoint to openai api_server (#27882) 
Signed-off-by: bbartels <benjamin@bartels.dev>
Signed-off-by: Benjamin Bartels <benjamin@bartels.dev>
 2025-11-01 12:45:42 -07:00
Yihua Cheng
e675118849 [Add] cmdline argument parsing for KV cache offloading modules (#27621) 
Signed-off-by: ApostaC <yihua98@uchicago.edu>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-11-01 07:17:07 +00:00
TJian
e2347dbf58 [Bugfix] [Model] Missing MRoPE function definition from KeyeForConditionalGeneration (#27895) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2025-11-01 13:45:23 +08:00
Cyrus Leung
879a06579e [CI/Build] Bump transformers version (#27528) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-31 22:11:07 -07:00
Nick Hill
0cdbe7b744 [Core] Async scheduling + structured outputs compatibility (#26866) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-11-01 00:35:04 +00:00
Chen Zhang
df334868ca [Hybrid] A simpler algorithm to find kernel_block_size (#26476) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-10-31 21:30:28 +00:00
Matthew Bonanni
f29aeb5a25 Add FLASHINFER_MLA to test_mla_backends and add B200 CI run (#27663) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-10-31 11:12:19 -07:00
GuanLuo
d6517be3cd [Bugfix] Missing NIXL metadata for handshake initialization if instance spans multi-node (#26338) 
Signed-off-by: Guan Luo <gluo@nvidia.com>
Signed-off-by: GuanLuo <41310872+GuanLuo@users.noreply.github.com>
Signed-off-by: Guan Luo <41310872+GuanLuo@users.noreply.github.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2025-10-31 10:16:00 -07:00
Jee Jee Li
0384aa7150 [CI/Build] Add gpt-oss LoRA test (#27870) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-10-31 22:17:21 +08:00
Sumanth R Hegde
4917002523 [Fix] Skip record_sleep_state logic in PrometheusStatsLogger if not in dev mode (#27789) 
Signed-off-by: SumanthRH <sumanthrh99@gmail.com>
 2025-10-30 19:26:27 +00:00
cong-meta
a2981c4272 [EP/DP][API Server] Enable DP-aware routing in OpenAI API requests (#24945) 
Co-authored-by: Cong Chen <prowindy@gmail.com>
 2025-10-30 12:10:16 -07:00
Fan Yin
9956aae4ea [Model][Ouro] Support Ouro Model (#27794) 
Signed-off-by: yinfan.1024 <yinfan.1024@bytedance.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: yinfan.1024 <yinfan.1024@bytedance.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-10-30 22:34:41 +08:00
Zhewen Li
0fe0140408 [KV offload] Enable CPU KV offload on CUDA alike Platforms (#27770) 
Signed-off-by: zhewenli <zhewenli@meta.com>
 2025-10-30 22:10:29 +08:00
Zhiyuan Li
4e68cc9b6a [Model] Introduce Kimi Linear to vLLM (#27809) 
Signed-off-by: lizhiyuan <lizhiyuan@moonshot.cn>
Signed-off-by: Zhiyuan Li <uniartisan2017@gmail.com>
 2025-10-30 21:02:27 +08:00