biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
8,158 Commits 2 Branches 140 Tags
ec02e536dfa46c7e8785cb5aaf5dd4eaad88f405
Commit Graph

417 Commits

Author SHA1 Message Date
Harry Mellor
f94886946e Improve conversion from dataclass configs to argparse arguments (#17303) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-04-28 16:22:12 +00:00
Lucas Wilkinson
d8bccde686 [BugFix] Fix vllm_flash_attn install issues (#17267) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Aaron Pham <contact@aarnphm.xyz>
 2025-04-27 17:27:56 -07:00
rasmith
68af5f6c5c [AMD][FP8][BugFix] Remove V1 check in arg_utils.py for FP8 since it is not necessary (#17215) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
 2025-04-25 19:55:05 -07:00
Benjamin Chislett
a0e619e62a [V1][Spec Decode] EAGLE-3 Support (#16937) 
Signed-off-by: Bryan Lu <yuzhelu@amazon.com>
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
Co-authored-by: Bryan Lu <yuzhelu@amazon.com>
 2025-04-25 15:43:07 -07:00
rasmith
a41351f363 [Quantization][FP8] Add support for FP8 models with input_scale for output projection and QK quantization (#15734) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>
 2025-04-25 00:45:02 -07:00
Harry Mellor
6ca0234478 Move missed SchedulerConfig args into scheduler config group in EngineArgs (#17131) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-04-24 22:48:53 -07:00
Harry Mellor
0fa939e2d1 Improve configs - LoRAConfig + PromptAdapterConfig (#16980) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-04-24 10:29:34 -07:00
Harry Mellor
0a05ed57e6 Simplify TokenizerGroup (#16790) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-04-24 04:43:56 -07:00
Woosuk Kwon
46e678bcff [Minor] Use larger batch sizes for A100/B100/B200/MI300x (#17073) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-04-23 19:18:59 -07:00
Travis Johnson
3cde34a4a4 [Frontend] Support guidance:no-additional-properties for compatibility with xgrammar (#15949) 
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
 2025-04-23 18:34:41 +00:00
Harry Mellor
d059110498 Improve configs - SpeculativeConfig (#16971) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-04-22 12:55:36 +00:00
Cyrus Leung
8f7bace7c3 [Doc] Improve documentation for multimodal CLI args (#16960) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-22 08:35:35 +00:00
wangxiyuan
b9b4746950 [V1] Remove additional_config check (#16710) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2025-04-21 20:45:27 -07:00
Michael Goin
986537f1c3 [V1] V1 FlashInfer Attention (#16684) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Aurick Qiao <qiao@aurick.net>
 2025-04-22 00:38:41 +00:00
Harry Mellor
4b07d36891 Improve configs - CacheConfig (#16835) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-04-20 12:25:04 +08:00
Harry Mellor
686623c5e7 Fix nullable_kvs fallback (#16837) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-04-18 05:58:39 -07:00
Harry Mellor
e78587a64c Improve-mm-and-pooler-and-decoding-configs (#16789) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-04-17 22:13:32 -07:00
Cyrus Leung
c16fb5dae8 [Doc] Improve help examples for --compilation-config (#16729) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-17 21:22:34 -07:00
Yihua Cheng
3408e47159 [P/D][V1] KV Connector API V1 (#15960) 
Signed-off-by: ApostaC <yihua98@uchicago.edu>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Signed-off-by: remi <remi@mistral.ai>
Co-authored-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Rémi Delacourt <54138269+Flechman@users.noreply.github.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
 2025-04-17 13:22:40 -07:00
Harry Mellor
d27ea94034 Improve configs - TokenizerPoolConfig + DeviceConfig (#16603) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-04-17 11:19:42 +00:00
Harry Mellor
e51929ebca Improve configs - SchedulerConfig (#16533) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-04-14 17:24:16 +08:00
Cyrus Leung
d9fc8cd9da [V1] Enable multi-input by default (#15799) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-12 08:52:39 +00:00
Harry Mellor
cd77382ac1 Improve configs - LoadConfig (#16422) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-04-11 20:27:27 +00:00
Michael Goin
aa3b3d76e0 Enforce valid max_num_batched_tokens when disable_chunked_mm_input=True (#16447) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-04-11 08:09:52 +00:00
Russell Bryant
9665313c39 [V1] Set structured output backend to auto by default (#15724) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-04-10 17:53:26 +00:00
Harry Mellor
0c54fc7273 Improve configs - ParallelConfig (#16332) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-04-10 17:34:37 +00:00
Michael Goin
8e5314a468 [V1] Add disable_chunked_mm_input arg to disable partial mm input prefill (#15837) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-04-07 23:24:07 -07:00
Nicolò Lucchesi
090c856d76 [Misc] Human-readable max-model-len cli arg (#16181) 
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-04-07 14:40:58 -04:00
Cyrus Leung
66d433b94f [V1] Revert the default max_num_seqs to V0 values for most hardware (#16158) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-07 13:54:36 -04:00
paolovic
da224daaa9 [Bugfix] add hf_token to EngineArgs (#16093) 
Signed-off-by: paolovic <paul-philipp.luley@uzh.ch>
Co-authored-by: paolovic <paul-philipp.luley@uzh.ch>
 2025-04-06 14:47:33 +00:00
Tristan Leclercq
4285e423a6 [Misc] Auto detect bitsandbytes pre-quantized models (#16027) 
Signed-off-by: Tristan Leclercq <tristanleclercq@gmail.com>
 2025-04-04 23:30:45 -07:00
Michael Goin
4708f13a9c [Bugfix] Fix default behavior/fallback for pp in v1 (#16057) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-04-04 17:58:08 +00:00
Woosuk Kwon
e75a6301bd [V1][Spec Decode] Implement Eagle Proposer [1/N] (#15729) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-04-01 12:33:16 -07:00
Varun Sundar Rabindranath
79455cf421 [Misc] Enable V1 LoRA by default (#15320) 
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
 2025-04-01 16:53:56 +08:00
Gregory Shtrasberg
c7e63aa4d8 [ROCm] Use device name in the warning (#15838) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2025-04-01 00:10:48 -07:00
shangmingc
239b7befdd [V1][Spec Decode] Remove deprecated spec decode config params (#15466) 
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
 2025-03-31 09:19:35 -07:00
youkaichao
555aa21905 [V1] Fully Transparent Implementation of CPU Offloading (#15354) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-03-31 20:22:34 +08:00
Reid
26df46ee59 [Misc] cli auto show default value (#15582) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
 2025-03-28 22:23:00 +00:00
Russell Bryant
7329ff5468 [V1] Support disable_any_whtespace for guidance backend (#15584) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-03-28 23:46:45 +08:00
Ce Gao
32b14baf8a [Refactor][Frontend] Keep all logic about reasoning into one class (#14428) 
Signed-off-by: Ce Gao <cegao@tensorchord.ai>
 2025-03-28 00:23:30 -07:00
Jee Jee Li
726efc6a32 [Quantization][V1] BitsAndBytes support V1 (#15611) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-03-28 10:12:47 +08:00
Nick Hill
15dac210f0 [V1] AsyncLLM data parallel (#13923) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-03-27 16:14:41 -07:00
Cyrus Leung
247181536f [Misc] Replace is_encoder_decoder_inputs with split_enc_dec_inputs (#15620) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-27 17:36:32 +00:00
Rui Qiao
df8d3d1287 [Misc] Restrict ray version dependency and update PP feature warning in V1 (#15556) 2025-03-27 06:21:07 +00:00
marko
27df5199d9 Support SHA256 as hash function in prefix caching (#15297) 
Signed-off-by: Marko Rosenmueller <5467316+dr75@users.noreply.github.com>
 2025-03-26 11:11:28 -07:00
Alex Brooks
1711b929b6 [Model] Add Reasoning Parser for Granite Models (#14202) 
Signed-off-by: Alex-Brooks <Alex.brooks@ibm.com>
Co-authored-by: Joe Runde <joe@joerun.de>
 2025-03-26 14:28:07 +00:00
Harry Mellor
e42389f9d7 Transformers backend already supports V1 (#15463) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-03-25 20:26:16 -07:00
Lu Fang
082ab86f5f [V1] Support long_prefill_token_threshold in v1 scheduler (#15419) 
Signed-off-by: Lu Fang <lufang@fb.com>
 2025-03-25 14:22:26 -07:00
Joe Runde
5f063a80bd [bugfix] add supports_v1 platform interface (#15417) 
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
 2025-03-25 15:00:32 -04:00
Russell Bryant
a09ad90a72 [V1] guidance backend for structured output + auto fallback mode (#14779) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Loc Huynh <jc1da.3011@gmail.com>
Co-authored-by: Michal Moskal <michal@moskal.me>
 2025-03-24 21:02:33 -07:00