biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
14,386 Commits 2 Branches 140 Tags
fa6a6be51978bd4b49ba0da17039e60f96dc5b13
Commit Graph

528 Commits

Author SHA1 Message Date
Jee Jee Li
acf092d348 [Bugfix] Fix test-pipeline.yaml (#10973) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-12-07 12:08:54 +08:00
youkaichao
9743d64e4e [ci][build] add tests for python only compilation (#10915) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-12-05 08:54:47 -08:00
Jee Jee Li
a4cf256159 [Bugfix] Fix QKVParallelLinearWithShardedLora bias bug (#10844) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-12-03 12:10:29 +08:00
Kuntai Du
0590ec3fd9 [Core] Implement disagg prefill by StatelessProcessGroup (#10502) 
This PR provides initial support for single-node disaggregated prefill in 1P1D scenario.
Signed-off-by: KuntaiDu <kuntai@uchicago.edu>
Co-authored-by: ApostaC <yihua98@uchicago.edu>
Co-authored-by: YaoJiayi <120040070@link.cuhk.edu.cn>
 2024-12-01 19:01:00 -06:00
Cyrus Leung
133707123e [Model] Replace embedding models with pooling adapter (#10769) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-12-01 08:02:54 +08:00
Ricky Xu
519e8e4182 [v1] EngineArgs for better config handling for v1 (#10382) 
Signed-off-by: rickyx <rickyx@anyscale.com>
 2024-11-25 21:09:43 -08:00
youkaichao
eda2b3589c Revert "Print running script to enhance CI log readability" (#10601) 2024-11-23 21:31:47 -08:00
Jee Jee Li
1c445dca51 [CI/Build] Print running script to enhance CI log readability (#10594) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-11-24 03:57:13 +00:00
Jee Jee Li
1700c543a5 [Bugfix] Fix LoRA weight sharding (#10450) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2024-11-23 17:23:17 -08:00
youkaichao
388ee3de66 [torch.compile] limit inductor threads and lazy import quant (#10482) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-20 18:36:33 -08:00
Kevin H. Luu
ed701ca963 [ci/build] Combine nightly and optional (#10465) 2024-11-19 21:36:03 -08:00
Cyrus Leung
b40cf6402e [Model] Support Qwen2 embeddings and use tags to select model tests (#10184) 2024-11-14 20:23:09 -08:00
Cyrus Leung
972112d82f [Bugfix] Fix unable to load some models (#10312) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-14 16:55:54 -08:00
Robert Shaw
6ace6fba2c [V1] AsyncLLM Implementation (#9826) 
Signed-off-by: Nick Hill <nickhill@us.ibm.com>
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2024-11-11 23:05:38 +00:00
Cyrus Leung
51c2e1fcef [CI/Build] Split up models tests (#10069) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-09 11:39:14 -08:00
Cyrus Leung
b489fc3c91 [CI/Build] Update CPU tests to include all "standard" tests (#5481) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-08 23:30:04 +08:00
youkaichao
719c1ca468 [core][distributed] add stateless_init_process_group (#10072) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-06 16:42:09 -08:00
youkaichao
4be3a45158 [distributed] add function to create ipc buffers directly (#10064) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-05 22:35:03 -08:00
Michael Goin
02462465ea [CI] Prune tests/models/decoder_only/language/* tests (#9940) 
Signed-off-by: mgoin <michael@neuralmagic.com>
 2024-11-05 16:02:23 -05:00
Alex Brooks
16b8f7a86f [CI/Build] Add Model Tests for Qwen2-VL (#9846) 
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-10-31 09:10:52 -07:00
Alex Brooks
cc98f1e079 [CI/Build] VLM Test Consolidation (#9372) 
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
 2024-10-30 09:32:17 -07:00
youkaichao
ff5ed6e1bc [torch.compile] rework compile control with piecewise cudagraph (#9715) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-10-29 23:03:49 -07:00
Cyrus Leung
696b01af8f [CI/Build] Split up decoder-only LM tests (#9488) 
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
 2024-10-20 21:27:50 -07:00
bnellnm
eca2c5f7c0 [Bugfix] Fix support for dimension like integers and ScalarType (#9299) 2024-10-17 19:08:34 +00:00
Kuntai Du
81ede99ca4 [Core] Deprecating block manager v1 and make block manager v2 default (#8704) 
Removing the block manager v1. This is the initial piece of prefix-caching-centric design. In order to achieve prefix-caching-centric design, we need to simplify the code path so that we only use v2 block manager (which has much higher performance on prefix caching).
 2024-10-17 11:38:15 -05:00
Lucas Wilkinson
9d30a056e7 [misc] CUDA Time Layerwise Profiler (#8337) 
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
 2024-10-17 10:36:09 -04:00
Cyrus Leung
1de76a0e55 [CI/Build] Test VLM embeddings (#9406) 2024-10-16 09:44:30 +00:00
Kevin H. Luu
a78c6ba7c8 [ci/build] Add placeholder command for custom models test (#9262) 2024-10-10 15:45:09 -07:00
youkaichao
e4d652ea3e [torch.compile] integration with compilation control (#9058) 2024-10-10 12:39:36 -07:00
sroy745
f3a507f1d3 [Core] Add an environment variable which needs to be set explicitly to allow BlockSpaceManagerV1 (#9149) 2024-10-10 14:17:17 +08:00
youkaichao
c8627cd41b [ci][test] use load dummy for testing (#9165) 2024-10-09 00:38:40 -07:00
Michael Goin
9ba0bd6aa6 Add lm-eval directly to requirements-test.txt (#9161) 2024-10-08 18:22:31 -07:00
Murali Andoorveedu
0f6d7a9a34 [Models] Add remaining model PP support (#7168) 
Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
Signed-off-by: Murali Andoorveedu <muralidhar.andoorveedu@centml.ai>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-10-04 10:56:58 +08:00
Lily Liu
1570203864 [Spec Decode] (1/2) Remove batch expansion (#8839) 2024-10-01 16:04:42 -07:00
Lily Liu
bce324487a [CI][SpecDecode] Fix spec decode tests, use flash attention backend for spec decode CI tests. (#8975) 2024-10-01 00:51:40 +00:00
Kevin H. Luu
1425a1bcf9 [ci] Add CODEOWNERS for test directories (#8795) 
Signed-off-by: kevin <kevin@anyscale.com>
 2024-10-01 00:47:08 +00:00
Cyrus Leung
e1a3f5e831 [CI/Build] Update models tests & examples (#8874) 
Co-authored-by: Roger Wang <ywang@roblox.com>
 2024-09-28 09:54:35 -07:00
Nick Hill
4b377d6feb [BugFix] Fix test breakages from transformers 4.45 upgrade (#8829) 2024-09-26 16:46:43 -07:00
Kevin H. Luu
d9cfbc891e [ci] Soft fail Entrypoints, Samplers, LoRA, Decoder-only VLM (#8872) 
Signed-off-by: kevin <kevin@anyscale.com>
 2024-09-26 15:02:16 -07:00
bnellnm
300da09177 [Kernel] Fullgraph and opcheck tests (#8479) 2024-09-25 08:35:52 -06:00
Hongxia Yang
1c046447a6 [CI/Build][Bugfix][Doc][ROCm] CI fix and doc update after ROCm 6.2 upgrade (#8777) 2024-09-25 22:26:37 +08:00
Alexey Kondratiev(AMD)
6cb748e190 [CI/Build] Re-enabling Entrypoints tests on ROCm, excluding ones that fail (#8551) 2024-09-19 13:06:32 -07:00
Alexander Matveev
7c7714d856 [Core][Bugfix][Perf] Introduce MQLLMEngine to avoid asyncio OH (#8157) 
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
 2024-09-18 13:56:58 +00:00
sroy745
1009e93c5d [Encoder decoder] Add cuda graph support during decoding for encoder-decoder models (#7631) 2024-09-17 07:35:01 -07:00
youkaichao
99aa4eddaf [torch.compile] register allreduce operations as custom ops (#8526) 2024-09-16 22:57:57 -07:00
Cyrus Leung
a84e598e21 [CI/Build] Reorganize models tests (#7820) 2024-09-13 10:20:06 -07:00
Nick Hill
551ce01078 [Core] Add engine option to return only deltas or final output (#7381) 2024-09-12 12:02:00 -07:00
Joe Runde
f2e263b801 [Bugfix] Offline mode fix (#8376) 
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
 2024-09-12 11:11:57 -07:00
Lily Liu
775f00f81e [Speculative Decoding] Test refactor (#8317) 
Co-authored-by: youkaichao <youkaichao@126.com>
 2024-09-11 14:07:34 -07:00
Alexey Kondratiev(AMD)
f421f3cefb [CI/Build] Enabling kernels tests for AMD, ignoring some of then that fail (#8130) 2024-09-10 11:51:15 -07:00