biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
8,877 Commits 2 Branches 140 Tags
56dcf4e7e965e34043acf20ca4e4aceda21d41ec
Commit Graph

8877 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
youkaichao
eb881ed006 [misc] fix typing (#11540) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-12-27 11:05:08 +08:00
Robert Shaw
46d4359450 [CI] Fix broken CI (#11543) 2024-12-26 18:49:16 -08:00
Woosuk Kwon
81b979f2a8 [V1] Fix yapf (#11538) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2024-12-27 09:47:10 +09:00
Woosuk Kwon
371d04d39b [V1] Use FlashInfer Sampling Kernel for Top-P & Top-K Sampling (#11394) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2024-12-27 09:32:38 +09:00
Robert Shaw
0c0c2015c5 Update openai_compatible_server.md (#11536) 
Co-authored-by: Simon Mo <simon.mo@hey.com>
 2024-12-26 16:26:18 -08:00
Simon Mo
82d24f7aac [Docs] Document Deepseek V3 support (#11535) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2024-12-26 16:21:56 -08:00
Simon Mo
f49777ba62 Deepseek v3 (#11502)
Some checks failed
Create Release / Create Release (push) Has been cancelled
Details 
Signed-off-by: mgoin <michael@neuralmagic.com>
Co-authored-by: mgoin <michael@neuralmagic.com>
Co-authored-by: robertgshaw2-neuralmagic <rshaw@neuralmagic.com>
v0.6.6 		2024-12-26 16:09:44 -08:00
Robert Shaw
55fb97f7bd [2/N] API Server: Avoid ulimit footgun (#11530) 2024-12-26 23:43:05 +00:00
Michael Goin
2072924d14 [Model] [Quantization] Support deepseek_v3 w8a8 fp8 block-wise quantization (#11523) 
Signed-off-by: mgoin <michael@neuralmagic.com>
Signed-off-by: simon-mo <simon.mo@hey.com>
Signed-off-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: simon-mo <simon.mo@hey.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: HandH1998 <1335248067@qq.com>
 2024-12-26 15:33:30 -08:00
Robert Shaw
720b10fdc6 [1/N] API Server (Remove Proxy) (#11529) 2024-12-26 23:03:43 +00:00
Isotr0py
b85a977822 [Doc] Add video example to openai client for multimodal (#11521) 
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2024-12-26 17:31:29 +00:00
Cyrus Leung
eec906d811 [Misc] Add placeholder module (#11501) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-26 13:12:51 +00:00
Jee Jee Li
f57ee5650d [Model] Modify MolmoForCausalLM MLP (#11510) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-12-26 13:12:05 +00:00
sroy745
dcb1a944d4 [V1] Adding min tokens/repetition/presence/frequence penalties to V1 sampler (#10681) 
Signed-off-by: Sourashis Roy <sroy@roblox.com>
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2024-12-26 19:02:58 +09:00
Roger Wang
7492a36207 [Doc] Add QVQ and QwQ to the list of supported models (#11509) 
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2024-12-26 09:44:32 +00:00
Jee Jee Li
aa25985bd1 [Misc][LoRA] Fix LoRA weight mapper (#11495) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-12-26 15:52:48 +08:00
Lucas Tucker
dbeac95dbb Mypy checking for vllm/compilation (#11496) 
Signed-off-by: lucast2021 <lucast2021@headroyce.org>
Co-authored-by: lucast2021 <lucast2021@headroyce.org>
 2024-12-26 05:04:07 +00:00
Cyrus Leung
51a624bf02 [Misc] Move some multimodal utils to modality-specific modules (#11494) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-26 04:23:20 +00:00
Cyrus Leung
6ad909fdda [Doc] Improve GitHub links (#11491) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-25 14:49:26 -08:00
Cyrus Leung
b689ada91e [Frontend] Enable decord to load video from base64 (#11492) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-25 16:33:55 +00:00
Jiaxin Shan
fc601665eb [Misc] Update disaggregation benchmark scripts and test logs (#11456) 
Signed-off-by: Jiaxin Shan <seedjeffwan@gmail.com>
 2024-12-25 06:58:48 +00:00
Rui Qiao
9832e5572a [V1] Unify VLLM_ENABLE_V1_MULTIPROCESSING handling in RayExecutor (#11472) 2024-12-24 19:49:46 -08:00
Cyrus Leung
3f3e92e1f2 [Model] Automatic conversion of classification and reward models (#11469) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-24 18:22:22 +00:00
Yuan Tang
409475a827 [Bugfix] Fix issues in CPU build Dockerfile. Fixes #9182 (#11435) 
Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
 2024-12-24 16:53:28 +00:00
Jee Jee Li
196c34b0ac [Misc] Move weights mapper (#11443) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-12-24 13:05:25 +00:00
Mengqing Cao
5c7963249d [attn][tiny fix] fix attn backend in MultiHeadAttention (#11463) 
Signed-off-by: Mengqing Cao <cmq0113@163.com>
 2024-12-24 12:39:36 +00:00
Ilya Lavrenov
461cde2080 [OpenVINO] Fixed installation conflicts (#11458) 
Signed-off-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
 2024-12-24 11:38:21 +00:00
Isotr0py
7a5286cc04 [Bugfix][Hardware][CPU] Fix CPU input_positions creation for text-only inputs with mrope (#11434) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2024-12-24 17:59:51 +08:00
Jee Jee Li
b1b1038fbd [Bugfix] Fix Qwen2-VL LoRA weight loading (#11430) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-12-24 09:56:10 +00:00
Cyrus Leung
9edca6bf8f [Frontend] Online Pooling API (#11457) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-24 17:54:30 +08:00
dpxa
4f074fbf53 [Misc]Suppress irrelevant exception stack trace information when CUDA… (#11438) 
Co-authored-by: shiquan <shiquan>
 2024-12-24 08:43:39 +00:00
Rui Qiao
a491d6f535 [V1] TP Ray executor (#11107) 
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
 2024-12-23 23:00:12 +00:00
Rafael Vasquez
32aa2059ad [Docs] Convert rST to MyST (Markdown) (#11145) 
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
 2024-12-23 22:35:38 +00:00
yansh97
94d545a1a1 [Doc] Fix typo in the help message of '--guided-decoding-backend' (#11440) 2024-12-23 20:20:44 +00:00
Michael Goin
60fb4f3bcf [Bugfix] Add kv cache scales to gemma2.py (#11269) 2024-12-23 19:30:45 +00:00
Michael Goin
63afbe9215 [CI] Expand OpenAI test_chat.py guided decoding tests (#11048) 
Signed-off-by: mgoin <michael@neuralmagic.com>
 2024-12-23 18:35:38 +00:00
Dipika Sikka
8cef6e02dc [Misc] add w8a8 asym models (#11075) 2024-12-23 13:33:20 -05:00
Dipika Sikka
b866cdbd05 [Misc] Add assertion and helpful message for marlin24 compressed models (#11388) 2024-12-24 02:23:38 +08:00
Yuan Tang
2e726680b3 [Bugfix] torch nightly version in ROCm installation guide (#11423) 
Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
 2024-12-23 17:20:22 +00:00
Michael Goin
5bfb30a529 [Bugfix] Fix CFGGuide and use outlines for grammars that can't convert to GBNF (#11389) 
Signed-off-by: mgoin <michael@neuralmagic.com>
 2024-12-23 23:06:20 +08:00
Lucas Tucker
e51719ae72 mypy type checking for vllm/worker (#11418) 
Signed-off-by: lucast2021 <lucast2021@headroyce.org>
Co-authored-by: lucast2021 <lucast2021@headroyce.org>
 2024-12-23 13:55:49 +00:00
youkaichao
f30581c518 [misc][perf] remove old code (#11425) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-12-23 08:01:08 +00:00
Simon Mo
048fc57a0f [CI] Unboock H100 Benchmark (#11419) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2024-12-22 14:17:43 -08:00
Jason T. Greene
f1d1bf6288 [Bugfix] Fix fully sharded LoRAs with Mixtral (#11390) 
Signed-off-by: Jason Greene <jason.greene@redhat.com>
 2024-12-22 23:25:10 +08:00
youkaichao
72d9c316d3 [cd][release] fix race conditions (#11407) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-12-22 00:39:11 -08:00
youkaichao
4a9139780a [cd][release] add pypi index for every commit and nightly build (#11404) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
 2024-12-21 23:53:44 -08:00
Roger Wang
29c748930e [CI] Fix flaky entrypoint tests (#11403) 
Signed-off-by: Roger Wang <ywang@roblox.com>
 2024-12-21 21:08:44 -08:00
Roger Wang
c2d1b075ba [Bugfix] Fix issues for Pixtral-Large-Instruct-2411 (#11393) 
Signed-off-by: ywang96 <ywang@example.com>
Co-authored-by: ywang96 <ywang@example.com>
 2024-12-21 10:15:03 +00:00
Ricky Xu
584f0ae40d [V1] Make AsyncLLMEngine v1-v0 opaque (#11383) 
Signed-off-by: Ricky Xu <xuchen727@hotmail.com>
 2024-12-21 15:14:08 +08:00
George
51ff216d85 [Bugfix] update should_ignore_layer (#11354) 
Signed-off-by: George Ohashi <george@neuralmagic.com>
 2024-12-21 06:36:23 +00:00