biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
3,481 Commits 2 Branches 140 Tags
74f8c2cf5f6a34fd21cfbe6d72bcc1b2a2a6754a
Commit Graph

2291 Commits

Author SHA1 Message Date
Mengqing Cao
8c1fb50705 [Platform][Refactor] Extract func get_default_attn_backend to Platform (#10358) 
Signed-off-by: Mengqing Cao <cmq0113@163.com>
 2024-11-19 11:22:26 +08:00
Jee Jee Li
7eb719df13 [Bugfix]Fix Phi-3 BNB online quantization (#10417) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-11-19 03:21:42 +00:00
youkaichao
2298e69b5f [ci][bugfix] fix kernel tests (#10431) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-18 15:29:37 -08:00
youkaichao
a03ea40792 [3/N][torch.compile] consolidate custom op logging (#10399) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-18 15:14:59 -08:00
Lucas Wilkinson
96d999fbe8 [Kernel] Initial Machete W4A8 support + Refactors (#9855) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
 2024-11-18 12:59:29 -07:00
Angus Wang
c2170a5b39 [Kernel] Explicitly specify other value in tl.load calls (#9014) 
Signed-off-by: Angus Wang <wangjadehao@gmail.com>
 2024-11-18 11:39:40 -08:00
Yan Ma
6b2d25efc7 [Hardware][XPU] AWQ/GPTQ support for xpu backend (#10107) 
Signed-off-by: yan ma <yan.ma@intel.com>
 2024-11-18 11:18:05 -07:00
Michael Goin
281cc4b3cd [Model][Bugfix] Support TP for PixtralHF ViT (#10405) 
Signed-off-by: mgoin <michael@neuralmagic.com>
 2024-11-18 10:04:14 -08:00
youkaichao
7851b45196 [5/N][torch.compile] torch.jit.script --> torch.compile (#10406) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-18 23:20:06 +08:00
Isotr0py
e7ebb662d7 [Model] Remove transformers attention porting in VITs (#10414) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2024-11-18 21:45:21 +08:00
B-201
5be4e52b65 [Model][LoRA]LoRA support added for glm-4v (#10418) 
Signed-off-by: B-201 <Joy25810@foxmail.com>
 2024-11-18 12:57:10 +00:00
Maybewuss
01aae1cc68 [Model] Remove redundant softmax when using PoolingType.STEP (#10415) 2024-11-18 10:05:36 +00:00
lkchen
c7dec926f6 [VLM] Report multi_modal_placeholders in output (#10407) 
Signed-off-by: Linkun Chen <lkchen+anyscale@github.com>
 2024-11-18 16:06:16 +08:00
youkaichao
51bb12d17b [4/N][torch.compile] clean up set_torch_compile_backend (#10401) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-17 23:57:20 -08:00
ℍ𝕠𝕝𝕝𝕠𝕨 𝕄𝕒𝕟
47826cacf0 [Bugfix] Ignore ray reinit error when current platform is ROCm or XPU (#10375) 
Signed-off-by: Hollow Man <hollowman@opensuse.org>
 2024-11-18 11:29:26 +08:00
Isotr0py
c4e464333e [Misc] Add uninitialized params tracking for AutoWeightsLoader (#10327) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2024-11-18 09:07:46 +08:00
电脑星人
80d85c5d7b [Bugfix] Fix mrope_position_delta in non-last prefill chunk (#10403) 
Signed-off-by: imkero <kerorek@outlook.com>
 2024-11-17 08:50:24 +00:00
Kunshang Ji
76aab90ab6 [Hardware] [HPU]add mark_step for hpu (#10239) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2024-11-17 00:44:44 -08:00
youkaichao
8d74b5aee9 [platforms] refactor cpu code (#10402) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-16 23:14:23 -08:00
Isotr0py
cf349c4a97 [Bugfix][CPU] Fix CPU embedding runner with tensor parallel (#10394) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2024-11-16 23:12:04 -08:00
Roger Wang
643ecf7b11 [V1] Refactor model executable interface for all text-only language models (#10374) 
Signed-off-by: Roger Wang <ywang@roblox.com>
 2024-11-17 05:18:46 +00:00
youkaichao
4fd9375028 [2/N][torch.compile] make compilation cfg part of vllm cfg (#10383) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-16 18:02:14 -08:00
电脑星人
361c29e174 [Bugfix] Fix M-RoPE position calculation when chunked prefill is enabled (#10388) 
Signed-off-by: imkero <kerorek@outlook.com>
 2024-11-17 02:10:00 +08:00
Sky Lee
b98d89efd4 [Misc] Medusa supports custom bias (#10361) 2024-11-16 16:33:01 +00:00
rasmith
1d75472626 [BugFix] [Kernel] Fix GPU SEGV occuring in fused_moe kernel (#10385) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
 2024-11-16 09:55:05 +00:00
youkaichao
2f427c2d16 [misc][plugin] improve log messages (#10386) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-16 01:23:20 -08:00
youkaichao
755b85359b [doc] add doc for the plugin system (#10372) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-15 21:46:27 -08:00
Cyrus Leung
32e46e000f [Frontend] Automatic detection of chat content format from AST (#9919) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-16 13:35:40 +08:00
Russell Bryant
a067f85e08 [Frontend] Add --version flag to CLI (#10369) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2024-11-15 13:13:53 -08:00
Guillaume Calmettes
691a3ec047 [Bugfix] Ensure special tokens are properly filtered out for guided structured output with MistralTokenizer (#10363) 
Signed-off-by: Guillaume Calmettes <gcalmettes@scaleway.com>
 2024-11-15 14:50:40 +00:00
youkaichao
3a763ba0c3 [core][misc] keep compatibility for old-style classes (#10356) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-15 13:55:51 +00:00
shangmingc
f2056f726d [Misc] Fix some help info of arg_utils to improve readability (#10362) 2024-11-15 12:40:30 +00:00
Jee Jee Li
1d65ec7eeb [Bugfix] Fix fully sharded LoRA bug (#10352) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-11-15 10:34:58 +00:00
Xin Yang
26908554b2 [Doc] Remove float32 choice from --lora-dtype (#10348) 
Signed-off-by: Xin Yang <xyang19@gmail.com>
 2024-11-15 10:22:57 +00:00
Cyrus Leung
b311efd0bd [Misc] Fix import error in tensorizer tests and cleanup some code (#10349) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-15 09:34:17 +00:00
wchen61
3d158cdc8d Add default value to avoid Falcon crash (#5363) (#10347) 
Signed-off-by: wchen61 <wchen61@foxmail.com>
 2024-11-15 08:52:20 +00:00
Cyrus Leung
2ac6d0e75b [Misc] Consolidate pooler config overrides (#10351) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-15 06:59:00 +00:00
Sky Lee
2ec8827288 [Bugfix] Qwen-vl output is inconsistent in speculative decoding (#10350) 2024-11-15 05:40:10 +00:00
Cyrus Leung
b40cf6402e [Model] Support Qwen2 embeddings and use tags to select model tests (#10184) 2024-11-14 20:23:09 -08:00
Tyler Michael Smith
2885ba0e24 [Misc] Change RedundantReshapesPass and FusionPass logging from info to debug (#10308) 
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2024-11-15 02:44:26 +00:00
Luka Govedič
bf2ddc6610 [bugfix] Fix static asymmetric quantization case (#10334) 
Signed-off-by: Daniël de Kok <me@danieldk.eu>
Signed-off-by: luka <luka@neuralmagic.com>
Co-authored-by: Daniël de Kok <me@danieldk.eu>
 2024-11-15 09:35:11 +08:00
Cyrus Leung
972112d82f [Bugfix] Fix unable to load some models (#10312) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-14 16:55:54 -08:00
Patrick von Platen
11cd1ae6ad [Tool parsing] Improve / correct mistral tool parsing (#10333) 2024-11-15 00:42:49 +00:00
Zijin Xiao
554af9228d [Bugfix] use AF_INET6 for OpenAI Compatible Server with ipv6 (#9583) 
Signed-off-by: xiaozijin <xiaozijin@bytedance.com>
 2024-11-14 16:38:53 -08:00
Murali Andoorveedu
b2e0ad3b59 [Perf] Reduce peak memory usage of llama (#10339) 
Signed-off-by: andoorve <37849411+andoorve@users.noreply.github.com>
 2024-11-15 00:38:20 +00:00
Maximilien de Bayser
4a18fd14ba Support Roberta embedding models (#9387) 
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Signed-off-by: Flavia Beo <flavia.beo@ibm.com>
Co-authored-by: Flavia Beo <flavia.beo@ibm.com>
 2024-11-14 21:23:29 +00:00
B-201
294bf467ba [Model] Add BNB quantization support for Idefics3 (#10310) 
Signed-off-by: B-201 <Joy25810@foxmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-11-14 06:31:44 +00:00
Guillaume Calmettes
52b48c1ead [BugFix]: properly deserialize tool_calls iterator before processing by mistral-common when MistralTokenizer is used (#9951) 
Signed-off-by: Guillaume Calmettes <gcalmettes@scaleway.com>
 2024-11-14 04:48:16 +00:00
Mike Depinet
f67ce05d0b [Frontend] Pythonic tool parser (#9859) 
Signed-off-by: Mike Depinet <mike@fixie.ai>
 2024-11-14 04:14:34 +00:00
youkaichao
504ac53d18 [misc] error early for old-style class (#10304) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-13 18:55:39 -08:00