biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
9,263 Commits 2 Branches 140 Tags
b9a1c4c8a2430555f1959938b87b10640c751e57
Commit Graph

674 Commits

Author SHA1 Message Date
Cyrus Leung
027b204ff1 [Bugfix] Re-enable support for ChatGLMForConditionalGeneration (#16187) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-07 23:15:58 +08:00
Lu Fang
55dcce91df Upstream Llama4 Support to Main (#16113) 
Signed-off-by: Aston Zhang <22279212+astonzhang@users.noreply.github.com>
Signed-off-by: Chris Thi <chris.c.thi@gmail.com>
Signed-off-by: drisspg <drisspguessous@gmail.com>
Signed-off-by: Jon Swenson <jmswen@gmail.com>
Signed-off-by: Keyun Tong <tongkeyun@gmail.com>
Signed-off-by: Lu Fang <fanglu@meta.com>
Signed-off-by: Xiaodong Wang <xdwang@meta.com>
Signed-off-by: Yang Chen <yangche@fb.com>
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
Signed-off-by: Lu Fang <lufang@fb.com>
Signed-off-by: Lu Fang <fanglu@fb.com>
Signed-off-by: Lucia Fang <fanglu@fb.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Lu Fang <fanglu@fb.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-07 08:06:27 -07:00
YamPengLi
7699258ef0 [Model] Add Qwen3 and Qwen3MoE (#15289) 
Signed-off-by: YamPengLi <yampayne.lyp@alibaba-inc.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-04-07 04:06:41 -07:00
Roger Wang
bb8dab821e [CI] Set max transformers version for Ultravox model test (#16149) 
Signed-off-by: Roger Wang <ywang@roblox.com>
 2025-04-07 04:37:58 +00:00
Isotr0py
fc0f87768a [Bugfix] Make dummy encoder prompt padding alternative and add missing warnings (#16129) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-04-07 04:07:15 +00:00
Roger Wang
af51d80fa1 Revert "[V1] Scatter and gather placeholders in the model runner" (#16075) 2025-04-04 14:50:57 -07:00
Cyrus Leung
f5722a5052 [V1] Scatter and gather placeholders in the model runner (#15712) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
 2025-04-04 21:26:44 +00:00
Gerald
9ef98d527e [Model][MiniMaxText01] Support MiniMaxText01 model inference (#13454) 
Signed-off-by: qscqesze <475517977@qq.com>
Co-authored-by: qingjun <qingjun@minimaxi.com>
Co-authored-by: qscqesze <475517977@qq.com>
 2025-04-01 16:23:55 -04:00
Jennifer Zhao
38327cf454 [Model] Aya Vision (#15441) 
Signed-off-by: Jennifer Zhao <ai.jenniferzhao@gmail.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
 2025-04-01 16:30:43 +00:00
wang.yuqi
085cbc4f9f [New Model]: jinaai/jina-reranker-v2-base-multilingual (#15876) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-01 08:32:26 -07:00
Michael Goin
51d7c6a2b2 [Model] Support Mistral3 in the HF Transformers format (#15505) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-04-01 06:10:05 -07:00
Yan Ma
ff6473980d [Bugfix][Model] fix mllama multi-image (#14883) 
Signed-off-by: yan ma <yan.ma@intel.com>
 2025-03-31 22:53:37 -07:00
Harry Mellor
a76f547e11 Rename fallback model and refactor supported models section (#15829) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-03-31 22:49:41 -07:00
shangmingc
239b7befdd [V1][Spec Decode] Remove deprecated spec decode config params (#15466) 
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
 2025-03-31 09:19:35 -07:00
Cyrus Leung
09e974d483 [Bugfix] Check dimensions of multimodal embeddings in V1 (#15816) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-31 09:01:35 -07:00
Harry Mellor
e5ef4fa99a Upgrade transformers to v4.50.3 (#13905) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-03-31 08:59:37 -07:00
Alex Brooks
c2e7507ad4 [Bugfix] Fix Crashing When Loading Modules With Batchnorm Stats (#15813) 
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
 2025-03-31 13:23:53 +00:00
Naveassaf
3aa2b6a637 [Model] Update support for NemotronNAS models (#15008) 
Signed-off-by: Nave Assaf <nassaf@nvidia.com>
 2025-03-31 20:35:14 +08:00
yihong
248e76c4df fix: lint fix a ruff checkout syntax error (#15767) 
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
 2025-03-30 03:36:02 -07:00
Cyrus Leung
803d5c35f3 [V1] Override mm_counts for dummy data creation (#15703) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-30 03:20:42 -07:00
pansicheng
7fd8c0f85c fix test_phi3v (#15321) 
Signed-off-by: pansicheng <sicheng.pan.chn@gmail.com>
 2025-03-30 02:01:34 -07:00
Roger Wang
c67abd614f [V1] Support interleaved modality items (#15605) 
Signed-off-by: Roger Wang <ywang@roblox.com>
 2025-03-29 06:30:09 -07:00
pengyuange
de1cb38769 [Model] Support Skywork-R1V (#15397) 
Signed-off-by: jiacai.liu <932997367@qq.com>
Co-authored-by: jiacai.liu <932997367@qq.com>
 2025-03-28 20:39:21 -07:00
Cyrus Leung
355f66348c [V1] Remove legacy input registry (#15673) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-27 23:34:34 -07:00
Jee Jee Li
726efc6a32 [Quantization][V1] BitsAndBytes support V1 (#15611) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-03-28 10:12:47 +08:00
Cyrus Leung
247181536f [Misc] Replace is_encoder_decoder_inputs with split_enc_dec_inputs (#15620) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-27 17:36:32 +00:00
Harry Mellor
cf5c8f1686 Separate base model from TransformersModel (#15467) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-03-26 18:13:38 +08:00
vllmellm
5ebf66748b [FEAT][ROCm] Integrate Fused MoE Kernels from AITER (#14967) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2025-03-26 16:30:30 +08:00
Cyrus Leung
997c8811d6 [Model] Support multi-image for Molmo (#15438) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-26 11:26:33 +08:00
Harry Mellor
e42389f9d7 Transformers backend already supports V1 (#15463) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-03-25 20:26:16 -07:00
Cyrus Leung
a9e879b316 [Misc] Clean up MiniCPM-V/O code (#15337) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-25 10:22:52 +00:00
Naitong Yu
2f4bd358f1 [Model] Support Tele-FLM Model (#15023) 
Signed-off-by: Naitong Yu <ntyu@baai.ac.cn>
Signed-off-by: jiangxin <horizon94@outlook.com>
Co-authored-by: Jason Fang <jasonfang3900@gmail.com>
Co-authored-by: jiangxin <horizon94@outlook.com>
 2025-03-22 02:04:44 -07:00
TJian
ec870fba9a [FEAT] [ROCm]: Add AITER RMS Norm (Layer Norm) Feature (#14959) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2025-03-21 22:36:14 -07:00
Isotr0py
1e508343e1 [Bugfix] Fix incorrect qwen2.5-vl attention mask pre-computation (#15200) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-03-20 19:18:04 -07:00
Matt Ritter
a8652f4f0f Enable CUDA graph support for llama 3.2 vision (#14917) 
Signed-off-by: Matt Ritter <100659061+mritterfigma@users.noreply.github.com>
 2025-03-19 23:29:16 -07:00
Cyrus Leung
f690372b68 [Core] Update dtype detection and defaults (#14858) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-19 13:49:33 +08:00
Jee Jee Li
46c759c165 [Bugfix] Fix LoRA extra vocab size (#15047) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-03-18 09:40:29 -07:00
yury-tokpanov
452e8fd968 [MODEL] Add support for Zamba2 models (#13185) 
Signed-off-by: Yury Tokpanov <yury@zyphra.com>
Signed-off-by: Quentin Anthony <qganthony@yahoo.com>
Co-authored-by: Quentin Anthony <qganthony@yahoo.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-03-18 08:56:21 -07:00
Patrick von Platen
f863ffc965 [Mistral-Small 3.1] Update docs and tests (#14977) 
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
 2025-03-18 03:29:42 -07:00
Cyrus Leung
b89fb2a4a1 [CI/Build] Use AutoModelForImageTextToText to load VLMs in tests (#14945) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-17 18:35:17 +00:00
vllmellm
2bb0e1a799 [Bugfix][ROCm] running new process using spawn method for rocm in tests. (#14810) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-03-17 11:33:35 +00:00
Sibi
a73e183e36 [Misc] Replace os environ to monkeypatch in test suite (#14516) 
Signed-off-by: sibi <85477603+t-sibiraj@users.noreply.github.com>
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Aaron Pham <contact@aarnphm.xyz>
 2025-03-16 20:35:57 -07:00
Robert Shaw
bb3aeddfaf [CI] Nightly Tests (#14898) 
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
 2025-03-17 02:06:43 +00:00
Isotr0py
def232e122 [VLM] Clean up Phi-4-MM ViT implementation (#14812) 
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-03-15 18:53:52 -07:00
Rémi Delacourt
61c6a5a796 [VLM] Merged multi-modal processor for Pixtral (#12211) 
Signed-off-by: remi <remi@mistral.ai>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-15 06:28:27 -07:00
Cyrus Leung
3556a41434 [VLM] Limit multimodal input cache by memory (#14805) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-15 02:52:05 -07:00
Li, Jiang
a2ae496589 [CPU] Support FP8 KV cache (#14741) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-03-14 22:07:36 -07:00
Robert Shaw
d4d93db2c5 [V1] V1 Enablement Oracle (#13726) 
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
 2025-03-14 22:02:20 -07:00
Cyrus Leung
613c5bb945 [Bugfix] Fix Aria test loading (#14823) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-14 09:11:23 -07:00
Roger Wang
0c2af17c76 [CI] Fix missing example model id in processor test (#14787) 
Signed-off-by: Roger Wang <ywang@roblox.com>
 2025-03-14 13:52:15 +08:00