biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
4,258 Commits 2 Branches 140 Tags
68ad4e3a8d8a66fb2a43be57471ee13a8bec4ec0
Commit Graph

377 Commits

Author SHA1 Message Date
youkaichao
68ad4e3a8d [Core] Support fully transparent sleep mode (#11743) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-01-22 14:39:32 +08:00
Liangfu Chen
016e3676e7 [CI] add docker volume prune to neuron CI (#12291) 
Signed-off-by: Liangfu Chen <liangfc@amazon.com>
 2025-01-22 10:47:49 +08:00
youkaichao
2fc6944c5e [ci/build] disable failed and flaky tests (#12240) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-01-21 13:25:03 +08:00
Harry Mellor
3ea7b94523 Move linting to pre-commit (#11975) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-01-20 14:58:01 +08:00
Isotr0py
02798ecabe [Model] Port deepseek-vl2 processor, remove dependency (#12169) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-01-18 13:59:39 +08:00
youkaichao
87a0c076af [core] allow callable in collective_rpc (#12151) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-01-17 20:47:01 +08:00
Li, Jiang
d4e6194570 [CI/Build][CPU][Bugfix] Fix CPU CI (#12150) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-01-17 19:39:52 +08:00
Kunshang Ji
fead53ba78 [CI]add genai-perf benchmark in nightly benchmark (#10704) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2025-01-17 04:15:09 +00:00
youkaichao
92e793d91a [core] LLM.collective_rpc interface and RLHF example (#12084) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-01-16 20:19:52 +08:00
youkaichao
bf53e0c70b Support torchrun and SPMD-style offline inference (#12071) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-01-16 19:58:53 +08:00
youkaichao
ff39141a49 [HPU][misc] add comments for explanation (#12034) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-01-14 19:24:06 +08:00
Konrad Zawora
078da31903 [HPU][Bugfix] set_forward_context and CI test execution (#12014) 
Signed-off-by: Konrad Zawora <kzawora@habana.ai>
 2025-01-14 11:04:18 +08:00
Sungjae Lee
80ea3af1a0 [CI][Spec Decode] fix: broken test for EAGLE model (#11972) 
Signed-off-by: Sungjae Lee <33976427+llsj14@users.noreply.github.com>
 2025-01-13 06:50:35 +00:00
Akshat Tripathi
8bddb73512 [Hardware][CPU] Multi-LoRA implementation for the CPU backend (#11100) 
Signed-off-by: Akshat Tripathi <akshat@krai.ai>
Signed-off-by: Oleg Mosalov <oleg@krai.ai>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Oleg Mosalov <oleg@krai.ai>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
 2025-01-12 13:01:52 +00:00
Isotr0py
f967e51f38 [Model] Initialize support for Deepseek-VL2 models (#11578) 
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-01-12 00:17:24 -08:00
Cyrus Leung
7a3a83e3b8 [CI/Build] Move model-specific multi-modal processing tests (#11934) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-11 13:50:05 +08:00
Harry Mellor
482cdc494e [Doc] Rename offline inference examples (#11927) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-01-10 23:50:29 +08:00
youkaichao
241ad7b301 [ci] Fix sampler tests (#11922) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-01-10 20:45:33 +08:00
Harry Mellor
d85c47d6ad Replace "online inference" with "online serving" (#11923) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-01-10 12:05:56 +00:00
Cyrus Leung
65097ca0af [Doc] Add model development API Reference (#11884) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-09 09:43:40 +00:00
Robert Shaw
56fe4c297c [TPU][Quantization] TPU W8A8 (#11785) 
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-01-08 19:33:29 +00:00
Li, Jiang
2f7024987e [CI/Build][Bugfix] Fix CPU CI image clean up (#11836) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-01-08 15:18:28 +00:00
Cyrus Leung
6cd40a5bfe [Doc][4/N] Reorganize API Reference (#11843) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-08 21:34:44 +08:00
Harry Mellor
aba8d6ee00 [Doc] Move examples into categories (#11840) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-01-08 13:09:53 +00:00
Yuan
1e4ce295ae [CI][CPU] adding build number to docker image name (#11788) 
Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>
 2025-01-07 07:28:01 +00:00
Liangfu Chen
898cdf033e [CI] Fix neuron CI and run offline tests (#11779) 
Signed-off-by: Liangfu Chen <liangfc@amazon.com>
 2025-01-06 21:36:10 -08:00
Jee Jee Li
b278557935 [Kernel][LoRA]Punica prefill kernels fusion (#11234) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Abatom <abzhonghua@gmail.com>
Co-authored-by: Zhonghua Deng <abatom@163.com>
 2025-01-07 04:01:39 +00:00
Aurick Qiao
e1a5c2f0a1 [Model] Whisper model implementation (#11280) 
Co-authored-by: Aurick Qiao <aurick.qiao@snowflake.com>
 2025-01-03 16:39:19 +08:00
Kevin H. Luu
fd3a62a122 [perf-benchmark] Fix dependency for steps in benchmark pipeline (#11710) 2025-01-02 22:38:37 -08:00
Kevin H. Luu
ccb1aabcca [benchmark] Remove dependency for H100 benchmark step (#11572) 2024-12-30 12:27:07 -08:00
Cyrus Leung
8d9b6721e7 [VLM] Abstract out multi-modal data parsing in merged processor (#11620) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-30 15:01:35 +00:00
youkaichao
b12e87f942 [platforms] enable platform plugins (#11602) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-12-30 20:24:45 +08:00
Simon Mo
048fc57a0f [CI] Unboock H100 Benchmark (#11419) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2024-12-22 14:17:43 -08:00
youkaichao
72d9c316d3 [cd][release] fix race conditions (#11407) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-12-22 00:39:11 -08:00
youkaichao
4a9139780a [cd][release] add pypi index for every commit and nightly build (#11404) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
 2024-12-21 23:53:44 -08:00
youkaichao
7801f56ed7 [ci][gh200] dockerfile clean up (#11351) 
Signed-off-by: drikster80 <ed.sealing@gmail.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: drikster80 <ed.sealing@gmail.com>
Co-authored-by: cenzhiyao <2523403608@qq.com>
 2024-12-19 18:13:06 -08:00
Yuan
a985f7af9f [CI] Adding CPU docker pipeline (#11261) 
Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>
Co-authored-by: Kevin H. Luu <kevin@anyscale.com>
 2024-12-19 11:46:55 -08:00
Wallas Henrique
8b79f9e107 [Bugfix] Fix guided decoding with tokenizer mode mistral (#11046) 2024-12-17 22:34:08 -08:00
youkaichao
35bae114a8 fix gh200 tests on main (#11246) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-12-16 17:22:38 -08:00
youkaichao
c301616ed2 [ci][tests] add gh200 tests (#11244) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-12-16 15:53:18 -08:00
Varun Sundar Rabindranath
efbce85f4d [misc] Layerwise profile updates (#10242) 
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
 2024-12-16 18:14:57 +00:00
Cyrus Leung
eeec9e3390 [Frontend] Separate pooling APIs in offline inference (#11129) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-13 10:40:07 +00:00
youkaichao
62de37a38e [core][distributed] initialization from StatelessProcessGroup (#10986) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-12-12 09:04:19 +00:00
Cyrus Leung
d1e21a979b [CI/Build] Split up VLM tests (#11083) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-12 06:18:16 +08:00
hissu-hyvarinen
b2f775456e [CI/Build] Enable prefix caching test for AMD (#11098) 
Signed-off-by: Hissu Hyvarinen <hissu.hyvarinen@amd.com>
 2024-12-11 15:23:37 +00:00
Richard Liu
5ed5d5f128 Build tpu image in release pipeline (#10936) 
Signed-off-by: Richard Liu <ricliu@google.com>
Co-authored-by: Kevin H. Luu <kevin@anyscale.com>
 2024-12-09 23:07:48 +00:00
Cyrus Leung
39e227c7ae [Model] Update multi-modal processor to support Mantis(LLaVA) model (#10711) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-07 17:10:05 +00:00
Jee Jee Li
acf092d348 [Bugfix] Fix test-pipeline.yaml (#10973) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-12-07 12:08:54 +08:00
youkaichao
9743d64e4e [ci][build] add tests for python only compilation (#10915) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-12-05 08:54:47 -08:00
Kevin H. Luu
7883c2bbe7 [benchmark] Make H100 benchmark optional (#10908) 2024-12-04 17:02:17 -08:00