biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
9,263 Commits 2 Branches 140 Tags
b9a1c4c8a2430555f1959938b87b10640c751e57
Commit Graph

359 Commits

Author SHA1 Message Date
Huy Do
2c4f59afc3 Update PyTorch to 2.7.0 (#16859) 2025-04-29 19:08:04 -07:00
Alexei-V-Ivanov-AMD
608968b7c5 Enabling multi-group kernel tests. (#17115) 
Signed-off-by: Alexei V. Ivanov <alexei.ivanov@amd.com>
 2025-04-29 10:27:27 -07:00
cascade
690fe019f0 [Feature] support sequence parallelism using compilation pass (#16155) 
Signed-off-by: cascade812 <cascade812@outlook.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-04-27 06:29:35 -07:00
Michael Goin
6317a5174a Categorize tests/kernels/ based on kernel type (#16799) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-04-23 09:21:07 -04:00
Russell Bryant
ce17db8085 [CI] Run v1/test_serial_utils.py in CI (#16996) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-04-23 01:13:34 -07:00
Yang Wang
f67e9e9f22 add Dockerfile build vllm against torch nightly (#16936) 
Signed-off-by: Yang Wang <elainewy@meta.com>
 2025-04-22 19:08:27 -07:00
Alexei-V-Ivanov-AMD
5536b30a4c Fencing Kernels Tests for enabling on AMD (#16929) 
Signed-off-by: Alexei V. Ivanov <alexei.ivanov@amd.com>
 2025-04-22 09:32:40 -07:00
Woosuk Kwon
4c41278b77 [CI/CD][V1] Add spec decode tests to CI (#16900) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-04-20 22:37:16 -07:00
Tarun Kumar
e37073efd7 Add property-based testing for vLLM endpoints using an API defined by an OpenAPI 3.1 schema (#16721) 
Signed-off-by: Tarun Kumar <takumar@redhat.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-04-17 21:08:27 -07:00
Robert Shaw
2b05b8ce69 [V1][Frontend] Improve Shutdown And Logs (#11737) 
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Signed-off-by: Andrew Feldman <afeldman@neuralmagic.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Andrew Feldman <afeldman@neuralmagic.com>
Co-authored-by: afeldman-nm <156691304+afeldman-nm@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-04-16 19:48:34 -07:00
Shinichi Hemmi
3badb0213b [Model] Add PLaMo2 (#14323) 
Signed-off-by: Shinichi Hemmi <50256998+Alnusjaponica@users.noreply.github.com>
Signed-off-by: shemmi <shemmi@preferred.jp>
Co-authored-by: Kento Nozawa <nzw0301@preferred.jp>
Co-authored-by: Hiroaki Mikami <mhiroaki@preferred.jp>
Co-authored-by: Calvin Metzger <metzger@preferred.jp>
 2025-04-15 19:31:30 -07:00
Michael Goin
b4fe16c75b Add vllm bench [latency, throughput] CLI commands (#16508) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-04-14 23:10:35 -07:00
Michael Goin
57504a4bcf [CI][Bugfix] Add mistral_tool_use to Ci (#16517) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-04-11 17:52:38 -07:00
Cyrus Leung
3d4c87758e [Misc] Update transformers version limits of multi-modal tests (#16381) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-09 23:03:33 -07:00
Russell Bryant
fee5b8d37f [Build/CI] Add tracing deps to vllm container image (#15224) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-04-09 19:14:06 +00:00
Luka Govedič
9cdde47289 [BugFix] Fix fusion test and add them to CI (#16287) 
Signed-off-by: luka <luka@neuralmagic.com>
 2025-04-08 23:46:45 -07:00
Lu Fang
55dcce91df Upstream Llama4 Support to Main (#16113) 
Signed-off-by: Aston Zhang <22279212+astonzhang@users.noreply.github.com>
Signed-off-by: Chris Thi <chris.c.thi@gmail.com>
Signed-off-by: drisspg <drisspguessous@gmail.com>
Signed-off-by: Jon Swenson <jmswen@gmail.com>
Signed-off-by: Keyun Tong <tongkeyun@gmail.com>
Signed-off-by: Lu Fang <fanglu@meta.com>
Signed-off-by: Xiaodong Wang <xdwang@meta.com>
Signed-off-by: Yang Chen <yangche@fb.com>
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
Signed-off-by: Lu Fang <lufang@fb.com>
Signed-off-by: Lu Fang <fanglu@fb.com>
Signed-off-by: Lucia Fang <fanglu@fb.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Lu Fang <fanglu@fb.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-07 08:06:27 -07:00
Kevin H. Luu
4dc52e1c53 [CI] Reorganize .buildkite directory (#16001) 
Signed-off-by: kevin <kevin@anyscale.com>
 2025-04-04 12:16:20 -07:00
Alexei-V-Ivanov-AMD
03a70eacaf Re-enable the AMD Testing for the passing tests. (#15586) 
Signed-off-by: Alexei V. Ivanov <alexei.ivanov@amd.com>
 2025-04-03 11:05:17 -07:00
Kay Yan
44f990515b [CI] Remove duplicate entrypoints-test (#15940) 
Signed-off-by: Kay Yan <kay.yan@daocloud.io>
 2025-04-02 02:44:01 -07:00
Jee Jee Li
4203926f10 [CI/Build] Further clean up LoRA tests (#15920) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-04-02 01:39:09 -07:00
Roger Wang
c67abd614f [V1] Support interleaved modality items (#15605) 
Signed-off-by: Roger Wang <ywang@roblox.com>
 2025-03-29 06:30:09 -07:00
wwl2755
94744ba41a [V1] [Feature] Collective RPC (#15444) 
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
 2025-03-29 03:39:14 -07:00
Nick Hill
15dac210f0 [V1] AsyncLLM data parallel (#13923) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-03-27 16:14:41 -07:00
Varun Sundar Rabindranath
ff38f0a32c [CI/Build] LoRA: Delete long context tests (#15503) 
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
 2025-03-25 17:18:34 -07:00
Robin
d6cd59f122 [Frontend] Support tool calling and reasoning parser (#14511) 
Signed-off-by: WangErXiao <863579016@qq.com>
 2025-03-23 14:00:07 -07:00
youkaichao
f68cce8e64 [ci/build] fix broken tests in LLM.collective_rpc (#15350) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-03-23 14:49:48 +08:00
hijkzzz
0661cfef7a Fix v1 supported oracle for worker-cls and worker-extension-cls (#15324) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2025-03-23 10:23:35 +08:00
Chi Zhang
086b56824c [ci] feat: make the test_torchrun_example run with tp=2, external_dp=2 (#15172) 
Signed-off-by: Chi Zhang <zhangchi.usc1992@bytedance.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2025-03-21 00:30:04 +08:00
Jovan Sardinha
70e500cad9 Fix broken tests (#14713) 
Signed-off-by: JovanSardinha <jovan.sardinha@gmail.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2025-03-20 02:06:49 +00:00
Aaron Pham
c0efdd655b [Fix][Structured Output] using vocab_size to construct matcher (#14868) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2025-03-17 11:42:45 -04:00
Cyrus Leung
6eaf1e5c52 [Misc] Add --seed option to offline multi-modal examples (#14934) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-17 03:00:17 -07:00
Sibi
a73e183e36 [Misc] Replace os environ to monkeypatch in test suite (#14516) 
Signed-off-by: sibi <85477603+t-sibiraj@users.noreply.github.com>
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Aaron Pham <contact@aarnphm.xyz>
 2025-03-16 20:35:57 -07:00
Robert Shaw
aecc780dba [V1] Enable Entrypoints Tests (#14903) 2025-03-16 17:56:16 -07:00
Robert Shaw
d4d93db2c5 [V1] V1 Enablement Oracle (#13726) 
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
 2025-03-14 22:02:20 -07:00
Kevin H. Luu
f1f632d9ec [ci] Reduce number of tests in fastcheck (#14782) 2025-03-13 20:43:45 -07:00
Harry Mellor
206e2577fa Move requirements into their own directory (#12547) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-03-08 16:44:35 +00:00
Aaron Pham
80e9afb5bc [V1][Core] Support for Structured Outputs (#12388) 
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-03-07 07:19:11 -08:00
Thomas Parnell
8ca2b21c98 [CI] Disable spawn when running V1 Test (#14345) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
 2025-03-06 21:52:46 +00:00
youkaichao
151b08e0fe [RLHF] use worker_extension_cls for compatibility with V0 and V1 (#14185) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-03-07 00:32:46 +08:00
Russell Bryant
ffad94397d [CI/Build] Use spawn multiprocessing mode for V1 test pipeline (#14243) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-03-05 17:08:02 -08:00
Jee Jee Li
cc5e8f6db8 [Model] Add LoRA support for TransformersModel (#13770) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-03-02 09:17:34 +08:00
youkaichao
eb24dc4a45 [v1] torchrun compatibility (#13642) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-02-23 22:47:24 +08:00
Kevin H. Luu
2c5e637b57 [ci] Use env var to control whether to use S3 bucket in CI (#13634) 2025-02-22 19:19:45 -08:00
youkaichao
3e472d882a [core] set up data parallel communication (#13591) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-02-22 19:28:59 +08:00
Harry Mellor
992e5c3d34 Merge similar examples in offline_inference into single basic example (#12737) 2025-02-20 04:53:51 -08:00
Yannick Schnider
423330263b [Feature] Pluggable platform-specific scheduler (#13161) 
Signed-off-by: Yannick Schnider <yannick.schnider1@ibm.com>
Signed-off-by: Yannick Schnider <Yannick.Schnider1@ibm.com>
 2025-02-19 17:16:38 +08:00
Lucia Fang
f525c0be8b [Model][Speculative Decoding] DeepSeek MTP spec decode (#12755) 
Signed-off-by: Lu Fang <fanglu@fb.com>
Co-authored-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
 2025-02-19 17:06:23 +08:00
Nicolò Lucchesi
d84cef76eb [Frontend] Add /v1/audio/transcriptions OpenAI API endpoint (#12909) 2025-02-13 07:23:45 -08:00
Kevin H. Luu
842b0fd402 [ci] Add more source file dependencies for some tests (#13123) 
Signed-off-by: <>
Co-authored-by: EC2 Default User <ec2-user@ip-172-31-20-117.us-west-2.compute.internal>
 2025-02-11 20:38:10 -08:00