biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
12,645 Commits 2 Branches 140 Tags
b471aad41f0e20634a3802e843e21f0e969c0553
Commit Graph

55 Commits

Author SHA1 Message Date
wang.yuqi
8be6432bda [CI Failure] Fix NomicBert max_model_len validation (#31662) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-01-05 11:06:52 +08:00
wang.yuqi
bd89ce16d2 [Model] Introduce verify_and_update_model_config for VerifyAndUpdateConfig. (#31131) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
 2025-12-24 09:54:57 +00:00
Jakub Zakrzewski
23daef548d [Frontend] Support using chat template as custom score template for reranking models (#30550) 
Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com>
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
 2025-12-23 11:19:16 +00:00
Kevin McKay
14c3e6ade3 [Misc] Fix spelling typos in model comments (#31117) 
Signed-off-by: c0de128 <kevin.mckay@outlook.com>
 2025-12-21 21:14:14 -08:00
Asaf Joseph Gardin
a9e15c21ef [Mamba] Removed disable cascade attn in MambaModelConfig (#30712) 
Signed-off-by: asafg <39553475+Josephasafg@users.noreply.github.com>
 2025-12-17 08:48:53 +00:00
Roberto L. Castro
4fa7ce46f3 [Feature] Add SM103 (Blackwell Ultra) Support to vLLM (#30484) 
Signed-off-by: LopezCastroRoberto <robertol.c510@gmail.com>
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2025-12-12 19:34:23 -08:00
Harry Mellor
cf3eacfe58 Standardise get_rope to use rope_parameters["partial_rotary_factor"], not rotary_dim (#30389) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-12-11 20:45:23 +00:00
Harry Mellor
93db3256a4 Give pooling examples better names (#30488) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-12-11 16:22:58 +00:00
Matthew Bonanni
66e674cdd5 [Attention][UX][1/N] Add AttentionConfig and change attention env vars to CLI arguments (#26315) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2025-12-05 09:48:43 -08:00
amitz-nv
6038b1b04b [Frontend][Model] Add 'float16' to possible mamba cache dtype values, override mamba SSM cache dtype value for NemotronH (#29978) 
Signed-off-by: amitz-nv <203509407+amitz-nv@users.noreply.github.com>
 2025-12-05 00:34:33 -08:00
Harry Mellor
e10c84e06a Access partial_rotary_factor from rope_parameters (#29966) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-12-04 18:42:49 +00:00
Yifan Qiao
48ddb02b79 [Hybrid Allocator] Support KV cache groups with different block_size (#29143) 
Signed-off-by: Yifan Qiao <yifanqiao@berkeley.edu>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
 2025-11-25 10:30:57 -05:00
Harry Mellor
a8b70304d6 Update rope_scaling to rope_parameters in preparation for Transformers v5 (#28542) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-11-19 09:06:36 -08:00
Mohammad Miadh Angkad
404d7a9d14 [Performance][gpt-oss] Revert gpt-oss max cudagraph size to 1024 (#28345) 
Signed-off-by: Mohammad Miadh Angkad <MAngkad.BSDSBA2027@aim.edu>
 2025-11-08 15:50:10 -07:00
Chen Zhang
c765f0b443 [FlashInfer] Avoid FlashInfer block_size 16 + head_size 256 on blackwell (#27994) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-11-05 09:25:32 -08:00
wangxiyuan
428bc7bf1c [V0 deprecation] Remove VLLM_USE_V1 usage in most modules (#27955) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2025-11-04 20:51:16 -08:00
Asaf Joseph Gardin
00b31a36a2 [V1] [Hybrid] Mamba1 Automatic Prefix Caching (#26377) 
Signed-off-by: asafg <39553475+Josephasafg@users.noreply.github.com>
 2025-11-02 04:16:23 -08:00
Tyler Michael Smith
ab98f6556f [Bugfix] Fix 2 precommit issues - (mamba_block_size, kv_cache_config) (#27811) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Signed-off-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-10-30 11:52:18 -07:00
Zhiyuan Li
4e68cc9b6a [Model] Introduce Kimi Linear to vLLM (#27809) 
Signed-off-by: lizhiyuan <lizhiyuan@moonshot.cn>
Signed-off-by: Zhiyuan Li <uniartisan2017@gmail.com>
 2025-10-30 21:02:27 +08:00
wang.yuqi
4464723f22 [Frontend][Doc][5/N] Improve all pooling task | Polish encode (pooling) api & Document. (#25524) 
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-10-30 12:13:05 +00:00
Asaf Joseph Gardin
05181cc57f [Hybrid] Add mamba_block_size to Engine Args (#27289) 
Signed-off-by: asafg <39553475+Josephasafg@users.noreply.github.com>
 2025-10-28 12:54:24 +00:00
Asaf Joseph Gardin
9273754222 [Hybrid] Added supports_mamba_prefix_caching Protocol (#27339) 
Signed-off-by: asafg <39553475+Josephasafg@users.noreply.github.com>
 2025-10-27 13:05:20 +00:00
Yeshwanth N
71b1c8b667 [Chore]:Extract math and argparse utilities to separate modules (#27188) 
Signed-off-by: Yeshwanth Surya <yeshsurya@gmail.com>
Signed-off-by: Yeshwanth N <yeshsurya@gmail.com>
Signed-off-by: yeshsurya <yeshsurya@gmail.com>
 2025-10-26 04:03:32 -07:00
fhl2000
284cc92275 [MISC] cudagraph_capture_sizes related improvements (#26016) 
Signed-off-by: fhl <2410591650@qq.com>
Signed-off-by: fhl2000 <63384265+fhl2000@users.noreply.github.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-24 05:11:05 -07:00
Lucas Wilkinson
c2bba69065 [BugFix] Disable fp8 kv-cache by default for DeepSeek V3.2 (#27121) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-10-18 22:05:23 +00:00
Isotr0py
6ac5e06f7c [Chore] Clean up pytorch helper functions in vllm.utils (#26908) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: isotr0py <2037008807@qq.com>
 2025-10-18 09:48:22 -07:00
Isotr0py
8e67b2557a [Bugfix] Fix out of bound index issue for Jina-embedding-v3 RoPE with cuda graph (#26687) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-10-13 03:21:48 -07:00
Zhiyuan Li
d24cf322e1 [Hybrid]: Decouple Kernel Block Size from KV Page Size (#24486) 
Signed-off-by: lizhiyuan <uniartisan2017@gmail.com>
Signed-off-by: Zhiyuan Li <uniartisan2017@gmail.com>
 2025-10-08 23:43:39 -07:00
Harry Mellor
d6953beb91 Convert formatting to use ruff instead of yapf + isort (#26247) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 07:06:22 -07:00
Thomas Parnell
ed3aeb25a4 [V1] [Hybrid] Remove code to override default CUDA graph configuration (#26226) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
 2025-10-04 13:47:48 +00:00
Stan Wozniak
ea507c3a93 [V1] [Hybrid] Mamba2 Automatic Prefix Caching (#25752) 
Signed-off-by: Stanislaw Wozniak <stw@zurich.ibm.com>
Signed-off-by: Thomas Ortner <boh@zurich.ibm.com>
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Thomas Ortner <boh@zurich.ibm.com>
Co-authored-by: Thomas Parnell <tpa@zurich.ibm.com>
 2025-10-04 06:34:22 +02:00
Lucas Wilkinson
8d7da92fd7 [BugFix] Fix default kv-cache-dtype default for DeepseekV3.2 (#25988) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-09-30 21:58:31 -07:00
Yongye Zhu
fa7e254a7f [New Model] DeepSeek-V3.2 (Rebased to Main) (#25896) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Barry Kang <43644113+Barry-Delaney@users.noreply.github.com>
Signed-off-by: Lucia Fang <fanglu@meta.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Lucia Fang <116399278+luccafong@users.noreply.github.com>
Co-authored-by: Lucia Fang <fanglu@meta.com>
Co-authored-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Siyuan Fu <siyuanf@nvidia.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Xiaozhu Meng <mxz297@gmail.com>
Co-authored-by: Barry Kang <43644113+Barry-Delaney@users.noreply.github.com>
 2025-09-30 17:14:41 +08:00
Woosuk Kwon
2e19a848d4 [V0 Deprecation] Remove max_seq_len_to_capture (#25543) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-09-24 01:51:39 -07:00
Michael Goin
a8ffc4f0f2 [Bugfix] Lower gpt-oss max cudagraph size to 992 to be compatible with FA3 (#25508) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-09-23 12:49:55 -07:00
Aaron Pham
29283e8976 [Chore] Cleanup guided namespace, move to structured outputs config (#22772) 
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-18 09:20:27 +00:00
Tao He
e93f4cc9e3 Add the support for the qwen3 next model (a hybrid attention model). (#24526) 
Signed-off-by: Tao He <linzhu.ht@alibaba-inc.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-11 15:32:09 +08:00
wang.yuqi
6d6c6b05d3 [New Model]: google/embeddinggemma-300m (#24318) 
Signed-off-by: wang.yuqi <noooop@126.com>
 2025-09-05 22:58:36 -07:00
Aaron Pham
c29fb540ff [gpt-oss] tool parser supports for /chat/completions [1/n] (#22386) 
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: Simon Mo <simon.mo@hey.com>
 2025-09-04 20:39:12 -07:00
wang.yuqi
e0653f6c0b [Model] Classification models support logit_bias / sigmoid_normalize (#24031) 
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-02 16:48:57 +00:00
wang.yuqi
11a7fafaa8 [New Model]: Support GteNewModelForSequenceClassification (#23524) 
Signed-off-by: wang.yuqi <noooop@126.com>
 2025-08-28 15:36:42 +08:00
Asaf Joseph Gardin
853c371fc3 [V1][Mamba] - Enable V1 by default for Mamba Models (#23650) 
Signed-off-by: asafg <39553475+Josephasafg@users.noreply.github.com>
 2025-08-27 20:53:30 +00:00
Thomas Parnell
704432af3c [V1] [Hybrid] Disable prefix caching by default for hybrid or mamba-based models (#23716) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
 2025-08-27 12:51:54 +00:00
Thomas Parnell
5f1af97f86 [V1] [Hybrid] Enable Full CUDA graph by default for hybrid models in V1 (#22594) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
 2025-08-26 23:28:55 +00:00
Thomas Parnell
75531a6c13 [V1] [Hybrid] Support using float32 for state in Hybrid Models (Mamba2, Mamba1, Minimax) (#22928) 
Signed-off-by: Daniel Afrimi <danielafrimi8@gmail.com>
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Daniel Afrimi <danielafrimi8@gmail.com>
Co-authored-by: Burkhard Ringlein <ngl@zurich.ibm.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
 2025-08-15 12:57:06 +00:00
Chen Zhang
a47e6ffe93 [GptOss] Add GptOss reasoning parser to support structure output (#22322) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com>
Co-authored-by: Minseok Lee <47620120+minseokl@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
 2025-08-05 23:39:13 -07:00
Woosuk Kwon
de98252f49 Add GPT-OSS model code and config [1/N] (#22327) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-08-05 23:26:00 -07:00
wang.yuqi
586f286789 [Model] Pooling model activation supports per request control by PoolingParams (#20538) 
Signed-off-by: wang.yuqi <noooop@126.com>
 2025-08-05 00:37:00 -07:00
wang.yuqi
2836dd73f1 [Model][CI] Let more pooling models support v1 (#21747) 
Signed-off-by: wang.yuqi <noooop@126.com>
 2025-07-31 01:51:15 -07:00
Cyrus Leung
86ae693f20 [Deprecation][2/N] Replace --task with --runner and --convert (#21470) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-07-27 19:42:40 -07:00