biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
5,758 Commits 2 Branches 140 Tags
7678fcd5b6d64084aeabeb17251d388e251ba4c9
Commit Graph

490 Commits

Author SHA1 Message Date
Lu Fang
7678fcd5b6 Fix the torch version parsing logic (#15857) 2025-04-10 07:37:47 -07:00
wang.yuqi
1f5d13ab9f [New Model]: jinaai/jina-embeddings-v3 (#16120) 2025-04-08 08:39:12 -07:00
Michael Goin
8e5314a468 [V1] Add disable_chunked_mm_input arg to disable partial mm input prefill (#15837) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-04-07 23:24:07 -07:00
Lu Fang
55dcce91df Upstream Llama4 Support to Main (#16113) 
Signed-off-by: Aston Zhang <22279212+astonzhang@users.noreply.github.com>
Signed-off-by: Chris Thi <chris.c.thi@gmail.com>
Signed-off-by: drisspg <drisspguessous@gmail.com>
Signed-off-by: Jon Swenson <jmswen@gmail.com>
Signed-off-by: Keyun Tong <tongkeyun@gmail.com>
Signed-off-by: Lu Fang <fanglu@meta.com>
Signed-off-by: Xiaodong Wang <xdwang@meta.com>
Signed-off-by: Yang Chen <yangche@fb.com>
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
Signed-off-by: Lu Fang <lufang@fb.com>
Signed-off-by: Lu Fang <fanglu@fb.com>
Signed-off-by: Lucia Fang <fanglu@fb.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Lu Fang <fanglu@fb.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-07 08:06:27 -07:00
paolovic
da224daaa9 [Bugfix] add hf_token to EngineArgs (#16093) 
Signed-off-by: paolovic <paul-philipp.luley@uzh.ch>
Co-authored-by: paolovic <paul-philipp.luley@uzh.ch>
 2025-04-06 14:47:33 +00:00
Gregory Shtrasberg
a6d042df0a [ROCm][Bugfix] Bring back fallback to eager mode removed in #14917, but for ROCm only (#15413) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2025-04-04 09:40:37 -07:00
Ilya Markov
ef608c37a7 [Distributed] [ROCM] Fix custom allreduce enable checks (#16010) 
Signed-off-by: ilmarkov <imarkov@redhat.com>
Co-authored-by: ilmarkov <imarkov@redhat.com>
 2025-04-04 09:39:08 -07:00
youkaichao
5e125e74d1 [misc] improve error message for "Failed to infer device type" (#15994) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-04-03 14:45:03 +08:00
youkaichao
8b664706aa [bugfix] add seed in torchrun_example.py (#15980) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-04-03 12:25:01 +08:00
Jee Jee Li
58e234a754 [Misc] V1 LoRA support CPU offload (#15843) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-04-02 23:04:43 +08:00
Gerald
9ef98d527e [Model][MiniMaxText01] Support MiniMaxText01 model inference (#13454) 
Signed-off-by: qscqesze <475517977@qq.com>
Co-authored-by: qingjun <qingjun@minimaxi.com>
Co-authored-by: qscqesze <475517977@qq.com>
 2025-04-01 16:23:55 -04:00
Woosuk Kwon
e75a6301bd [V1][Spec Decode] Implement Eagle Proposer [1/N] (#15729) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-04-01 12:33:16 -07:00
Jennifer Zhao
38327cf454 [Model] Aya Vision (#15441) 
Signed-off-by: Jennifer Zhao <ai.jenniferzhao@gmail.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
 2025-04-01 16:30:43 +00:00
shangmingc
656fd72976 [Misc] Fix speculative config repr string (#15860) 
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
 2025-04-01 02:26:22 -07:00
Chen Zhang
3a5f0afcd2 [V1] Implement sliding window attention in kv_cache_manager (#14097) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-04-01 00:33:17 -07:00
Ilya Markov
b7b7676d67 [Distributed] Add custom allreduce support for ROCM (#14125) 
Signed-off-by: ilmarkov <imarkov@redhat.com>
Co-authored-by: ilmarkov <imarkov@redhat.com>
 2025-03-31 22:49:12 -07:00
Harry Mellor
e6e3c55ef2 Move dockerfiles into their own directory (#14549) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-03-31 13:47:32 -07:00
shangmingc
239b7befdd [V1][Spec Decode] Remove deprecated spec decode config params (#15466) 
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
 2025-03-31 09:19:35 -07:00
Naveassaf
3aa2b6a637 [Model] Update support for NemotronNAS models (#15008) 
Signed-off-by: Nave Assaf <nassaf@nvidia.com>
 2025-03-31 20:35:14 +08:00
youkaichao
555aa21905 [V1] Fully Transparent Implementation of CPU Offloading (#15354) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-03-31 20:22:34 +08:00
shangmingc
d03308be0c [Misc] Remove stale func in KVTransferConfig (#14746) 
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
 2025-03-28 17:33:32 +00:00
Jee Jee Li
726efc6a32 [Quantization][V1] BitsAndBytes support V1 (#15611) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-03-28 10:12:47 +08:00
Nick Hill
15dac210f0 [V1] AsyncLLM data parallel (#13923) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-03-27 16:14:41 -07:00
wang.yuqi
3f532cb6a6 [Misc] Use model_redirect to redirect the model name to a local folder. (#14116) 2025-03-27 02:21:23 -07:00
Rui Qiao
df8d3d1287 [Misc] Restrict ray version dependency and update PP feature warning in V1 (#15556) 2025-03-27 06:21:07 +00:00
Matthew Vine
7a6d45bc8a Support FIPS enabled machines with MD5 hashing (#15299) 
Signed-off-by: Matthew Vine <32849887+MattTheCuber@users.noreply.github.com>
 2025-03-26 20:19:46 -04:00
marko
27df5199d9 Support SHA256 as hash function in prefix caching (#15297) 
Signed-off-by: Marko Rosenmueller <5467316+dr75@users.noreply.github.com>
 2025-03-26 11:11:28 -07:00
Bryan Lu
781d056280 [Feature] Enhance EAGLE Architecture with Proper RMS Norms (#14990) 
Signed-off-by: Bryan Lu <yuzhelu@amazon.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-03-26 08:24:07 +00:00
youkaichao
d0cfec7ab9 [bugfix] fix inductor cache on max_position_embeddings (#15436) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-03-25 07:05:39 -07:00
Russell Bryant
a09ad90a72 [V1] guidance backend for structured output + auto fallback mode (#14779) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Loc Huynh <jc1da.3011@gmail.com>
Co-authored-by: Michal Moskal <michal@moskal.me>
 2025-03-24 21:02:33 -07:00
Jee Jee Li
6db94571d7 [Misc] Remove LoRA log (#15388) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-03-24 20:43:48 -07:00
Manish Sethi
761702fd19 [Core] Integrate fastsafetensors loader for loading model weights (#10647) 
Signed-off-by: Manish Sethi <Manish.sethi1@ibm.com>
 2025-03-24 08:08:02 -07:00
Luka Govedič
f622dbcf39 [Fix] [torch.compile] Improve UUID system for custom passes (#15249) 
Signed-off-by: luka <luka@neuralmagic.com>
 2025-03-24 01:54:07 +00:00
Lucas Wilkinson
dccf535f8e [V1] Enable V1 Fp8 cache for FA3 in the oracle (#15191) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-03-23 15:07:04 -07:00
Roger Wang
9c5c81b0da [Misc][Doc] Add note regarding loading generation_config by default (#15281) 
Signed-off-by: Roger Wang <ywang@roblox.com>
 2025-03-23 14:00:55 -07:00
Woosuk Kwon
bc8ed3c4ba [V1][Spec Decode] Use better defaults for N-gram (#15358) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-03-23 10:52:30 -07:00
shangmingc
50c9636d87 [V1][Usage] Refactor speculative decoding configuration and tests (#14434) 
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
 2025-03-22 19:28:10 -10:00
Russell Bryant
b877031d80 Remove openvino support in favor of external plugin (#15339) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-03-22 14:06:39 -07:00
wwl2755
1c2bec0f82 [Doc] add load_format items in docs (#14804) 
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
 2025-03-21 22:36:43 -07:00
Woosuk Kwon
2b22290ce0 [V1] Add flag to disable cascade attention (#15243) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-03-20 15:24:16 -07:00
Wang Ran (汪然)
bfe2fe0af4 typo: Update config.py (#15189) 2025-03-19 23:31:21 -07:00
Matt Ritter
a8652f4f0f Enable CUDA graph support for llama 3.2 vision (#14917) 
Signed-off-by: Matt Ritter <100659061+mritterfigma@users.noreply.github.com>
 2025-03-19 23:29:16 -07:00
Russell Bryant
1f16b7fe74 [Core][V0] Add guidance backend for structured output (#14589) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Loc Huynh <lohuynh@microsoft.com>
Co-authored-by: Michal Moskal <michal@moskal.me>
Co-authored-by: Aaron Pham <contact@aarnphm.xyz>
 2025-03-19 21:33:51 -07:00
Alexander Matveev
cfbca8a2f2 [V1] TPU - Tensor parallel MP support (#15059) 2025-03-20 00:55:18 +00:00
Cyrus Leung
f690372b68 [Core] Update dtype detection and defaults (#14858) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-19 13:49:33 +08:00
Jee Jee Li
46c759c165 [Bugfix] Fix LoRA extra vocab size (#15047) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-03-18 09:40:29 -07:00
yury-tokpanov
452e8fd968 [MODEL] Add support for Zamba2 models (#13185) 
Signed-off-by: Yury Tokpanov <yury@zyphra.com>
Signed-off-by: Quentin Anthony <qganthony@yahoo.com>
Co-authored-by: Quentin Anthony <qganthony@yahoo.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-03-18 08:56:21 -07:00
hoshi-hiyouga
414919138b [Bugfix] torchrun compatibility (#14899) 
Signed-off-by: hiyouga <hiyouga@buaa.edu.cn>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2025-03-18 05:49:27 -07:00
Robert Shaw
d4d93db2c5 [V1] V1 Enablement Oracle (#13726) 
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
 2025-03-14 22:02:20 -07:00
Michael Goin
14f301b541 Update to torch==2.6.0 (#12721) 
Signed-off-by: mgoin <michael@neuralmagic.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: luka <luka@neuralmagic.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-03-14 16:58:30 -04:00