biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
9,263 Commits 2 Branches 140 Tags
b9a1c4c8a2430555f1959938b87b10640c751e57
Commit Graph

707 Commits

Author SHA1 Message Date
Li, Jiang
4555143ea7 [CPU] V1 support for the CPU backend (#16441) 2025-06-03 18:43:01 -07:00
Yan Ru Pei
b712be98c7 feat: add data parallel rank to KVEventBatch (#18925) 2025-06-03 17:14:20 -07:00
Simon Mo
02f0c7b220 [Misc] Add SPDX-FileCopyrightText (#19100) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2025-06-03 11:20:17 -07:00
Li, Jiang
8655f47f37 [CPU][CI] Re-enable the CPU CI tests (#19046) 
Signed-off-by: jiang.li <jiang1.li@intel.com>
 2025-06-02 20:46:47 -07:00
Concurrensee
4ce42f9204 Adding "LoRA Test %N" to AMD production tests (#18929) 
Signed-off-by: Yida Wu <yidawu@alumni.cmu.edu>
 2025-06-02 20:46:44 -07:00
Siyuan Liu
9112b443a0 [Hardware][TPU] Initial support of model parallelism with single worker using SPMD (#18011) 
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
Co-authored-by: Hossein Sarshar <hossein.sarshar@gmail.com>
Co-authored-by: Chengji Yao <chengjiyao@google.com>
 2025-06-03 00:06:20 +00:00
Nick Hill
2dbe8c0774 [Perf] API-server scaleout with many-to-many server-engine comms (#17546) 2025-05-30 08:17:00 -07:00
Rabi Mishra
5f1d0c8118 [Bugfix][Failing Test] Fix test_vllm_port.py (#18618) 
Signed-off-by: rabi <ramishra@redhat.com>
 2025-05-30 17:13:47 +08:00
Carol Zheng
3132290a14 [TPU][CI/CD] Clean up docker for TPU tests. (#18926) 
Signed-off-by: Carol Zheng <cazheng@google.com>
 2025-05-30 10:24:19 +08:00
Brent Salisbury
fd7bb88d72 Fixes a dead link in nightly benchmark readme (#18856) 
Signed-off-by: Brent Salisbury <bsalisbu@redhat.com>
 2025-05-29 04:41:39 +00:00
Akshat Tripathi
643622ba46 [Hardware][TPU][V1] Multi-LoRA Optimisations for the V1 TPU backend (#15655) 
Signed-off-by: Akshat Tripathi <akshat@krai.ai>
Signed-off-by: Chengji Yao <chengjiyao@google.com>
Signed-off-by: xihajun <junfan@krai.ai>
Signed-off-by: Jorge de Freitas <jorge.de-freitas22@imperial.ac.uk>
Signed-off-by: Jorge de Freitas <jorge@krai.ai>
Co-authored-by: Chengji Yao <chengjiyao@google.com>
Co-authored-by: xihajun <junfan@krai.ai>
Co-authored-by: Jorge de Freitas <jorge.de-freitas22@imperial.ac.uk>
Co-authored-by: Jorge de Freitas <jorge@krai.ai>
 2025-05-28 19:59:09 +00:00
Rabi Mishra
b78f844a67 [Bugfix][FailingTest]Fix test_model_load_with_params.py (#18758) 
Signed-off-by: rabi <ramishra@redhat.com>
 2025-05-28 05:42:54 +00:00
Carol Zheng
b48d5cca16 [CI/Build] [TPU] Fix TPU CI exit code (#18282) 
Signed-off-by: Carol Zheng <cazheng@google.com>
 2025-05-27 14:54:59 -07:00
Mark McLoughlin
06a0338015 [V1][Metrics] Add API for accessing in-memory Prometheus metrics (#17010) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2025-05-27 09:37:06 +00:00
Łukasz Durejko
bbd9a84dc5 [Hardware][Intel-Gaudi] [CI/Build] Fix multiple containers using the same name in run-hpu-test.sh (#18752) 
Signed-off-by: Lukasz Durejko <ldurejko@habana.ai>
 2025-05-27 00:10:26 -07:00
Cyrus Leung
82e2339b06 [Doc] Move examples and further reorganize user guide (#18666) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-05-26 07:38:04 -07:00
Łukasz Durejko
e76be06550 [Hardware][Intel-Gaudi] [CI/Build] Add tensor parallel size = 2 test to HPU CI (#18709) 
Signed-off-by: Lukasz Durejko <ldurejko@habana.ai>
 2025-05-26 05:26:07 -07:00
Isotr0py
0877750029 [CI/Build] Split pooling and generation extended language models tests in CI (#18705) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-05-26 04:00:08 -07:00
Michael Goin
0ddf88e16e [CI] Enable test_initialization to run on V1 (#16736) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-05-23 15:09:44 -07:00
Cyrus Leung
6dd51c7ef1 [CI/Build] Fix V1 flag being set in entrypoints tests (#18598) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-05-23 05:51:53 -07:00
Harry Mellor
a1fe24d961 Migrate docs from Sphinx to MkDocs (#18145) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-05-23 02:09:53 -07:00
cascade
71ea614d4a [Feature]Add async tensor parallelism using compilation pass (#17882) 
Signed-off-by: cascade812 <cascade812@outlook.com>
 2025-05-23 01:03:34 -07:00
aws-elaineyz
ed5d408255 [Neuron] Remove bypass on EAGLEConfig and add a test (#18514) 
Signed-off-by: Elaine Zhao <elaineyz@amazon.com>
 2025-05-22 21:26:32 -07:00
Sanger Steel
c32e249a23 [Frontend] [Core] Add Tensorizer support for V1, LoRA adapter serialization and deserialization (#17926) 
Signed-off-by: Sanger Steel <sangersteel@gmail.com>
 2025-05-22 18:44:18 -07:00
David Xia
1f3a1200e4 [Bugfix] make test_openai_schema.py pass (#18224) 
Signed-off-by: David Xia <david@davidxia.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-05-22 18:34:06 +00:00
lkchen
a35a494745 [Bugfix] Add kwargs to RequestOutput __init__ to be forward compatible (#18513) 
Signed-off-by: Linkun <github@lkchen.net>
 2025-05-22 05:24:43 -07:00
Rabi Mishra
61acfc45bc [Bugfix][Failing Test] Fix test_events.py (#18460) 
Signed-off-by: rabi <ramishra@redhat.com>
 2025-05-21 04:57:28 -07:00
Kevin H. Luu
d981396778 [release] Change dockerhub username for TPU release (#18389) 2025-05-19 23:49:23 -07:00
Liangfu Chen
d565e0976f [neuron] fix authorization issue (#18364) 
Signed-off-by: Liangfu Chen <liangfc@amazon.com>
 2025-05-19 23:30:32 +00:00
Simon Mo
47fda6d089 [Build] Supports CUDA 12.6 and 11.8 after Blackwell Update (#18316) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2025-05-18 23:19:33 -07:00
Lucia Fang
3d2779c29a [Feature] Support Pipeline Parallism in torchrun SPMD offline inference for V1 (#17827) 
Signed-off-by: Lucia Fang <fanglu@fb.com>
 2025-05-15 22:28:27 -07:00
Alexei-V-Ivanov-AMD
0b34593017 Adding "AMD: Tensorizer Test" to amdproduction. (#18216) 2025-05-15 11:01:25 -07:00
Alexei-V-Ivanov-AMD
566ec04c3d Adding "Basic Models Test" and "Multi-Modal Models Test (Extended) 3" in AMD Pipeline (#18106) 
Signed-off-by: Alexei V. Ivanov <alexei.ivanov@amd.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-05-15 08:49:23 -07:00
Mark McLoughlin
65334ef3b9 [V1][Metrics] Remove unused code (#18158) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2025-05-14 20:13:17 -07:00
Andrey Talman
09f106a91e Upload vllm index for the rc builds (#18173) 2025-05-14 16:35:56 -07:00
Charlie Fu
7b2f28deba [AMD][torch.compile] Enable silu+fp8_quant fusion for rocm (#18082) 
Signed-off-by: charlifu <charlifu@amd.com>
 2025-05-13 22:13:56 -07:00
Harry Mellor
009d9e7590 Convert benchmarks to ruff format (#18068) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-05-13 13:43:29 +00:00
Harry Mellor
98fcba1575 Convert .buildkite to ruff format (#17656) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-05-13 09:28:31 +00:00
Nick Hill
ee5be834e7 [BugFix] Fix 4-GPU RLHF tests (#18007) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-05-12 23:03:55 -07:00
Yang Wang
2b0db9b0e2 Enable standard language model for torhc nightly (#18004) 
Signed-off-by: Yang Wang <elainewy@meta.com>
 2025-05-12 14:00:04 -07:00
Alexei-V-Ivanov-AMD
e9c730c9bd Enabling "Weight Loading Multiple GPU Test - Large Models" (#18020) 2025-05-12 13:05:33 -07:00
Carol Zheng
b9fd0d7a69 [CI/Build] Fix TPU V1 Test mixed use of & and && across tests (#17968) 2025-05-12 12:06:59 -07:00
Jonathan Berkhahn
98ea35601c [Lora][Frontend]Add default local directory LoRA resolver plugin. (#16855) 
Signed-off-by: jberkhahn <jaberkha@us.ibm.com>
 2025-05-12 10:39:10 -07:00
Robert Shaw
d19110204c [P/D] NIXL Integration (#17751) 
Signed-off-by: ApostaC <yihua98@uchicago.edu>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Signed-off-by: Robert Shaw <rshaw@neuralmagic.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: Brent Salisbury <bsalisbu@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: ApostaC <yihua98@uchicago.edu>
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: Brent Salisbury <bsalisbu@redhat.com>
 2025-05-12 09:46:16 -07:00
Aaruni Aggarwal
9fbf2bfbd5 Correcting testcases in builkite job for IBM Power (#17675) 
Signed-off-by: Aaruni Aggarwal <aaruniagg@gmail.com>
 2025-05-12 08:11:55 +00:00
Alexei-V-Ivanov-AMD
3b602cdea7 AMD conditional all test execution // new test groups (#17556) 
Signed-off-by: Alexei V. Ivanov <alexei.ivanov@amd.com>
Signed-off-by: Yida Wu <yidawu@alumni.cmu.edu>
 2025-05-09 15:35:58 -07:00
Michael Goin
8342e3abd1 [CI] Prune down lm-eval small tests (#17012) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-05-08 19:00:26 +00:00
yarongmu-google
a83a0f92b5 [Test] Attempt all TPU V1 tests, even if some of them fail. (#17334) 
Signed-off-by: Yarong Mu <ymu@google.com>
 2025-05-08 17:20:54 +00:00
Akshat Tripathi
c20ef40fd0 [Hardware][TPU][V1] Multi-LoRA implementation for the V1 TPU backend (#14238) 
Signed-off-by: Akshat Tripathi <akshat@krai.ai>
Signed-off-by: Chengji Yao <chengjiyao@google.com>
Co-authored-by: Chengji Yao <chengjiyao@google.com>
 2025-05-07 16:28:47 -04:00
Michael Goin
950b71186f Replace lm-eval bash script with pytest and use enforce_eager for faster CI (#17717) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-05-06 18:00:10 -07:00