biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
5,802 Commits 2 Branches 140 Tags
bd6028d6b0bbc0c569ece0535067081c5e8bdc14
Commit Graph

50 Commits

Author SHA1 Message Date
Michael Goin
bd6028d6b0 Optimized topk for topk=1 (Llama-4) (#16512) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-04-12 14:21:08 +08:00
Alex Brooks
c2e7507ad4 [Bugfix] Fix Crashing When Loading Modules With Batchnorm Stats (#15813) 
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
 2025-03-31 13:23:53 +00:00
Naveassaf
3aa2b6a637 [Model] Update support for NemotronNAS models (#15008) 
Signed-off-by: Nave Assaf <nassaf@nvidia.com>
 2025-03-31 20:35:14 +08:00
youkaichao
555aa21905 [V1] Fully Transparent Implementation of CPU Offloading (#15354) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-03-31 20:22:34 +08:00
Harry Mellor
97cfa65df7 Add pipeline parallel support to TransformersModel (#12832) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
 2025-03-25 10:41:45 +08:00
Nicolò Lucchesi
69ff99fdcd [Core] Optimizing cross-attention QKVParallelLinear computation (#12325) 
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: NickLucche <nick@nlucches-4xa100.c.openshift-330514.internal>
Co-authored-by: NickLucche <nick@nlucches-4xa100.c.openshift-330514.internal>
 2025-03-06 09:37:26 +00:00
Isotr0py
7864875879 [Bugfix] Fix qwen2.5-vl overflow issue (#13968) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-02-27 17:30:39 +00:00
Russell Bryant
e489ad7a21 [Misc] Add SPDX-License-Identifier headers to python source files (#12628) 
- **Add SPDX license headers to python source files**
- **Check for SPDX headers using pre-commit**

commit 9d7ef44c3cfb72ca4c32e1c677d99259d10d4745
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:18:24 2025 -0500

    Add SPDX license headers to python source files
    
This commit adds SPDX license headers to python source files as
recommended to
the project by the Linux Foundation. These headers provide a concise way
that is
both human and machine readable for communicating license information
for each
source file. It helps avoid any ambiguity about the license of the code
and can
    also be easily used by tools to help manage license compliance.
    
The Linux Foundation runs license scans against the codebase to help
ensure
    we are in compliance with the licenses of the code we use, including
dependencies. Having these headers in place helps that tool do its job.
    
    More information can be found on the SPDX site:
    
    - https://spdx.dev/learn/handling-license-info/
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

commit 5a1cf1cb3b80759131c73f6a9dddebccac039dea
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:36:32 2025 -0500

    Check for SPDX headers using pre-commit
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

---------

Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-02-02 11:58:18 -08:00
Harry Mellor
823ab79633 Update pre-commit hooks (#12475) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-01-27 17:23:08 -07:00
Cyrus Leung
ca47e176af [Misc] Move some model utils into vision file (#11848) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-08 17:04:46 +00:00
Cyrus Leung
91445c7bc8 [Bugfix] Fix image input for Pixtral-HF (#11741) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-08 10:17:16 +08:00
Cyrus Leung
8c38ee7007 [VLM] Merged multi-modal processor for LLaVA-NeXT (#11682) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-02 16:39:27 +00:00
Roger Wang
a11f326528 [V1] Initial support of multimodal models for V1 re-arch (#10699) 
Signed-off-by: Roger Wang <ywang@roblox.com>
 2024-12-08 12:50:51 +00:00
Cyrus Leung
bf0e382e16 [Model] Composite weight loading for multimodal Qwen2 (#10944) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-07 07:22:52 -07:00
Cyrus Leung
133707123e [Model] Replace embedding models with pooling adapter (#10769) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-01 08:02:54 +08:00
Cyrus Leung
fa6ecb9aa7 [Model] Clean up MiniCPMV (#10751) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-29 04:47:06 +00:00
Roger Wang
2f0a0a17a4 [V1] Refactor model executable interface for multimodal models (#10570) 
Signed-off-by: Roger Wang <ywang@roblox.com>
 2024-11-26 20:46:11 +00:00
youkaichao
c055747867 [model][utils] add extract_layer_index utility function (#10599) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-23 22:22:54 -08:00
Mengqing Cao
8c1fb50705 [Platform][Refactor] Extract func get_default_attn_backend to Platform (#10358) 
Signed-off-by: Mengqing Cao <cmq0113@163.com>
 2024-11-19 11:22:26 +08:00
Isotr0py
e7ebb662d7 [Model] Remove transformers attention porting in VITs (#10414) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2024-11-18 21:45:21 +08:00
Isotr0py
c4e464333e [Misc] Add uninitialized params tracking for AutoWeightsLoader (#10327) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2024-11-18 09:07:46 +08:00
youkaichao
504ac53d18 [misc] error early for old-style class (#10304) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-13 18:55:39 -08:00
Cyrus Leung
0b8bb86bf1 [1/N] Initial prototype for multi-modal processor (#10044) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-13 12:39:03 +00:00
Cyrus Leung
51c2e1fcef [CI/Build] Split up models tests (#10069) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-09 11:39:14 -08:00
youkaichao
1a95f10ee7 [5/N] pass the whole config to model (#9983) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-09 14:17:28 +08:00
youkaichao
3bb4befea7 [bugfix] fix tsts (#9959) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-02 15:54:05 -07:00
Peter Salas
6c0b7f548d [Core][VLM] Add precise multi-modal placeholder tracking (#8346) 
Signed-off-by: Peter Salas <peter@fixie.ai>
 2024-11-01 16:21:10 -07:00
Michael Goin
bc73e9821c [Bugfix] Fix prefix strings for quantized VLMs (#9772) 2024-10-29 16:02:59 -07:00
Cyrus Leung
836e8ef6ee [Bugfix] Fix PP for ChatGLM and Molmo (#9422) 2024-10-24 06:12:05 +00:00
Cyrus Leung
831540cf04 [Model] Support E5-V (#9576) 2024-10-23 11:35:29 +08:00
wangshuai09
3ddbe25502 [Hardware][CPU] using current_platform.is_cpu (#9536) 2024-10-22 00:50:43 -07:00
Isotr0py
cf1d62a644 [Model] Support SDPA attention for Molmo vision backbone (#9410) 2024-10-16 11:52:01 +00:00
Cyrus Leung
7abba39ee6 [Model] VLM2Vec, the first multimodal embedding model in vLLM (#9303) 2024-10-16 14:31:00 +08:00
Cyrus Leung
8bfaa4e31e [Bugfix] fix composite weight loading and EAGLE weight loading (#9160) 2024-10-09 00:36:55 -07:00
Cyrus Leung
b22b798471 [Model] PP support for embedding models and update docs (#9090) 
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
 2024-10-06 16:35:27 +08:00
Murali Andoorveedu
0f6d7a9a34 [Models] Add remaining model PP support (#7168) 
Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
Signed-off-by: Murali Andoorveedu <muralidhar.andoorveedu@centml.ai>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-10-04 10:56:58 +08:00
Jee Jee Li
3d49776bbb [Model][LoRA]LoRA support added for MiniCPMV2.5 (#7199) 2024-09-29 06:59:45 +00:00
Isotr0py
13d88d4137 [Bugfix] Refactor composite weight loading logic (#8656) 2024-09-22 04:33:27 +00:00
manikandan.tm@zucisystems.com
8685ba1a1e Inclusion of InternVLChatModel In PP_SUPPORTED_MODELS(Pipeline Parallelism) (#7860) 2024-09-05 11:33:37 +00:00
Peter Salas
74d5543ec5 [VLM][Core] Fix exceptions on ragged NestedTensors (#7974) 2024-08-29 03:24:31 +00:00
Cyrus Leung
ef9baee3c5 [Bugfix][VLM] Fix incompatibility between #7902 and #7230 (#7948) 2024-08-28 08:11:18 -07:00
Peter Salas
fab5f53e2d [Core][VLM] Stack multimodal tensors to represent multiple images within each prompt (#7902) 2024-08-28 01:53:56 +00:00
Peter Salas
00c3d68e45 [Frontend][Core] Add plumbing to support audio language models (#7446) 2024-08-13 17:39:33 +00:00
Cyrus Leung
1f26efbb3a [Model] Support SigLIP encoder and alternative decoders for LLaVA models (#7153) 
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
 2024-08-06 16:55:31 +08:00
Michael Goin
460c1884e3 [Bugfix] Support cpu offloading with fp8 quantization (#6960) 2024-07-31 12:47:46 -07:00
Robert Shaw
dbe5588554 [ Misc ] non-uniform quantization via compressed-tensors for Llama (#6515) 2024-07-18 22:39:18 -04:00
youkaichao
1c27d25fb5 [core][model] yet another cpu offload implementation (#6496) 
Co-authored-by: Michael Goin <michael@neuralmagic.com>
 2024-07-17 20:54:35 -07:00
youkaichao
4cf256ae7f [misc][distributed] fix pp missing layer condition (#6446)
Some checks failed
Create Release / Create Release (push) Has been cancelled
Details
Create Release / Build Wheel (11.8, ubuntu-20.04, 3.10, 2.3.1) (push) Has been cancelled
Details
Create Release / Build Wheel (11.8, ubuntu-20.04, 3.11, 2.3.1) (push) Has been cancelled
Details
Create Release / Build Wheel (11.8, ubuntu-20.04, 3.8, 2.3.1) (push) Has been cancelled
Details
Create Release / Build Wheel (11.8, ubuntu-20.04, 3.9, 2.3.1) (push) Has been cancelled
Details
Create Release / Build Wheel (12.1, ubuntu-20.04, 3.10, 2.3.1) (push) Has been cancelled
Details
Create Release / Build Wheel (12.1, ubuntu-20.04, 3.11, 2.3.1) (push) Has been cancelled
Details
Create Release / Build Wheel (12.1, ubuntu-20.04, 3.8, 2.3.1) (push) Has been cancelled
Details
Create Release / Build Wheel (12.1, ubuntu-20.04, 3.9, 2.3.1) (push) Has been cancelled
Details
2024-07-15 10:32:35 -07:00
youkaichao
69672f116c [core][distributed] simplify code to support pipeline parallel (#6406) 2024-07-14 21:20:51 -07:00
Cyrus Leung
9831aec49f [Core] Dynamic image size support for VLMs (#5276) 
Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: ywang96 <ywang@roblox.com>
Co-authored-by: xwjiang2010 <87673679+xwjiang2010@users.noreply.github.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
 2024-07-02 20:34:00 -07:00