biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
9,263 Commits 2 Branches 140 Tags
b9a1c4c8a2430555f1959938b87b10640c751e57
Commit Graph

468 Commits

Author SHA1 Message Date
Zebing Lin
82dfb12e52 [Core] Use sha256 bytes instead of BlockHash to reduce GC overhead (#23673) 
Signed-off-by: linzebing <linzebing1995@gmail.com>
 2025-09-08 21:34:37 -07:00
Matthew Bonanni
620db1fc58 [Attention] FlashAttention MLA cudagraph support (#23958) 
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2025-09-08 22:05:26 +00:00
22quinn
3a3e91bdfe [CI/Build] Disable flaky test_structured_output tests (#24404) 
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
 2025-09-08 02:51:59 +00:00
Xingyu Liu
b3d7e3c845 [Sampler] Support returning all prompt logprobs (#23868) 
Signed-off-by: Xingyu Liu <charlotteliu12x@gmail.com>
Co-authored-by: 22quinn <33176974+22quinn@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-09-07 19:34:31 -07:00
Woosuk Kwon
105d3d62ef [TPU] Remove TopKTopPSampler dependency for TPU sampler (#24391) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-09-07 01:12:36 -07:00
Aaron Pham
e67597545b [CI][Fix] deterministic seed for flaky CI runs on structured outputs (#24380) 
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
 2025-09-07 11:10:40 +08:00
Roger Wang
b121ca22ad [CI] Disable flaky structured output test from CI (#24366) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2025-09-06 13:31:56 +00:00
Seiji Eicher
60b755cbcb [Misc] Have AsyncLLM custom_stat_loggers extend default logger list (#20952) 
Signed-off-by: Seiji Eicher <seiji@anyscale.com>
Signed-off-by: Seiji Eicher <58963096+eicherseiji@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-09-04 14:25:30 -07:00
Kebe
8f423e5f43 [Feature][Response API] Add streaming support for non-harmony (#23741) 
Signed-off-by: Kebe <mail@kebe7jun.com>
 2025-09-04 17:49:06 +08:00
Lucas Wilkinson
402759d472 [Attention] FlashAttn MLA (#14258) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
Co-authored-by: Matthew Bonanni <mbonanni001@gmail.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-09-04 02:47:59 -07:00
dsinghvi
70549c1245 [CI/Build] Serve images used by multimodal tests through local HTTP Server (#23907) 
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-09-03 16:13:11 +08:00
Didier Durand
d7e1e59972 [Doc]: fix typos in Python comments (#24093) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
 2025-09-02 21:05:45 -07:00
co63oc
1bd007f234 fix some typos (#24071) 
Signed-off-by: co63oc <co63oc@users.noreply.github.com>
 2025-09-02 20:44:50 -07:00
afeldman-nm
136d853e65 [V1] Wrapper which plumbs request-level logits processors into vLLM batch-level logits processing (#23656) 
Signed-off-by: Andrew Feldman <afeldman@redhat.com>
 2025-09-03 02:52:51 +00:00
Didier Durand
fad73be1a5 [Doc]: fix typos in Python comments (#24077) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
 2025-09-02 02:38:55 -07:00
Roger Wang
749be00a98 [Core][Multimodal] Allow passing multi_modal_uuids as multimodal identifiers. (#23394) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2025-08-30 18:01:22 -07:00
Ning Xie
5490d633ce [UT] fix unify_kv_cache_configs when kv cache config needs sort (#23843) 2025-08-30 11:22:14 +00:00
Yong Hoon Shin
8c3e199998 Revert gemma3n fast prefill changes (#23897) 
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
 2025-08-29 12:16:57 -07:00
Nick Hill
d90d8eb674 [BugFix] Async scheduling and PP compatibility with DP (#23770) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-08-29 08:17:27 -07:00
Flora Feng
69f46359dd [Multimodal] Consolidate mm inputs into MultiModalFeatureSpec (#23779) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2025-08-29 18:36:57 +08:00
Yong Hoon Shin
cb293f6a79 [V1] Enable prefill optimization for Gemma3n (#22628) 
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
 2025-08-28 14:54:30 -07:00
Hanchenli
5da4f5d857 [Bugfix] Fix for V1 priority scheduling crashes at preemption (#23713) 
Signed-off-by: Hanchenli <lihanc2002@gmail.com>
 2025-08-28 00:44:52 +00:00
Nick Hill
3ce8285d6d [LogitsProcs] Deduplicate built-in LP implementation logic (#23362) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-08-27 23:11:33 +08:00
Roger Wang
b5d34af328 [Bugfix] Fix scheduling when repeated images in one request (#23544) 
Signed-off-by: Roger Wang <hey@rogerw.me>
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Roger Wang <hey@rogerw.me>
Co-authored-by: knlnguyen1802 <knlnguyen1802@gmail.com>
 2025-08-26 09:46:28 +00:00
Driss Guessous
e0329ed4b4 Updates to Flex + VLLm integration (#21416) 
Signed-off-by: drisspg <drisspguessous@gmail.com>
 2025-08-25 09:32:42 -04:00
Ayush Satyam
5c4b6e66fe [Attention] Unify mamba and attention backend selection (#23171) 
Signed-off-by: Ayush Satyam <ayushsatyam146@gmail.com>
 2025-08-25 09:09:36 +00:00
Chenguang Zheng
d765cf01fe [Core][Multimodal] Track encode cache entries by mm_hash and enable embedding sharing between requests (#22711) 
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2025-08-25 00:41:17 -07:00
Noam Gat
39971db3aa Frontend: Adding LM Format Enforcer support to V1 engine (#22564) 
Signed-off-by: Noam Gat <noamgat@gmail.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-08-24 19:31:22 -07:00
Yong Hoon Shin
b6d7d34fc6 Add unit tests for batched guided and non-guided requests (#23389) 
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
 2025-08-22 10:31:24 -07:00
Flora Feng
53415653ff [P/D][Nixl] Make kv cache register compatible with hybrid memory allocator (#23079) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2025-08-21 22:30:48 -07:00
Chen Zhang
17373dcd93 [Attention] Refactor AttentionMetadata Preparation for Encoder-only Models (#23154) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-08-22 05:05:59 +00:00
Arjun Reddy
111692bb8c [CI] Add end-to-end V1 min_tokens test coverage (#22495) 
Signed-off-by: Arjun Reddy <189282188+arjunbreddy22@users.noreply.github.com>
Co-authored-by: Arjun Reddy <189282188+arjunbreddy22@users.noreply.github.com>
 2025-08-21 22:04:07 -06:00
Cyrus Leung
8896eb72eb [Deprecation] Remove prompt_token_ids arg fallback in LLM.generate and LLM.embed (#18800) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-08-22 10:56:57 +08:00
22quinn
480bdf5a7b [Core] Support custom executor qualname (#23314) 
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
 2025-08-22 09:40:54 +08:00
Kebe
5368f76855 [Feature][Responses API] Support logprobs(non-stream) (#23319) 
Signed-off-by: Kebe <mail@kebe7jun.com>
 2025-08-21 23:09:16 +00:00
22quinn
f571ff8eb6 [Sampler] Support returning final logprobs (#22387) 
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-08-20 21:28:32 -07:00
Matthew Bonanni
10cc12ba66 Feature/mla tests (#23195) 
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-08-20 21:46:47 +00:00
Woosuk Kwon
d6d13bd49e [Misc] Add max_seq_len to CommonAttentionMetadata (#23216) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-08-20 09:05:29 -07:00
Xin Yang
83e69a09d6 [Model] Support deepseek with eagle (#21086) 
Signed-off-by: Xin Yang <xyangx@amazon.com>
 2025-08-20 19:01:31 +08:00
Woosuk Kwon
c9b38be8aa [Spec Decode] Make propose_draft_token_ids non-blocking for lower TTFT (#23041) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-08-18 17:20:38 -07:00
Cyrus Leung
27e8d1ea3e [Refactor] Define MultiModalKwargsItems separate from MultiModalKwargs (#23053) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-08-18 09:52:00 +00:00
Cyrus Leung
5c32143b9d [Refactor] Defer tensor data construction in MultiModalKwargs (#23030) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-08-16 21:05:50 -07:00
afeldman-nm
bf7f470b22 [V1] Logits processors extensibility (#19912) 
Signed-off-by: Andrew Feldman <afeldman@redhat.com>
Signed-off-by: Andrew Feldman <afeld2012@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Andrew Feldman <afeld2012@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-08-16 12:59:17 -07:00
Cyrus Leung
4dff91c93d [Refactor] Allow optional MultiModalKwargsItem in IPC (#23022) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-08-16 11:30:49 +00:00
Nick Hill
ad0297d113 [Misc] Support passing multiple request ids at once to AsyncLLM.abort() (#22944) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-08-15 17:00:36 -07:00
Yong Hoon Shin
3e2f7985a2 Support multiple attention groups for KV sharing (#22672) 
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
 2025-08-15 16:54:10 -07:00
Or Ozeri
c280066f9d [v1] Move block_hashes from KVCacheManager to Request.block_hashes (#19728) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2025-08-15 16:52:52 -07:00
Nick Hill
b9dc9d2607 [BugFix] Handle case where async utility call is cancelled (#22996) 
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Yinghai Lu <yinghai@thinkingmachines.ai>
 2025-08-15 17:38:42 -06:00
fhl2000
74f441f4b5 [Core] Allow full cudagraph with separate attention routines and orthogonal to compilation, add support for FA2 and FlashInfer (#20059) 
Signed-off-by: fhl <2410591650@qq.com>
Signed-off-by: fhl2000 <63384265+fhl2000@users.noreply.github.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2025-08-15 10:01:39 -04:00
Thomas Parnell
75531a6c13 [V1] [Hybrid] Support using float32 for state in Hybrid Models (Mamba2, Mamba1, Minimax) (#22928) 
Signed-off-by: Daniel Afrimi <danielafrimi8@gmail.com>
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Daniel Afrimi <danielafrimi8@gmail.com>
Co-authored-by: Burkhard Ringlein <ngl@zurich.ibm.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
 2025-08-15 12:57:06 +00:00