biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[Experimental] Add multi-LoRA support (#1804)

Browse Source 
Co-authored-by: Chen Shen <scv119@gmail.com>
Co-authored-by: Shreyas Krishnaswamy <shrekris@anyscale.com>
Co-authored-by: Avnish Narayan <avnish@anyscale.com>
This commit is contained in:
Antoni Baum
2024-01-24 00:26:37 +01:00
committed by GitHub
parent 9c1352eb57
commit 9b945daaf1
52 changed files with 8035 additions and 126 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
tests/worker/spec_decode/utils.py
View File

@@ -83,8 +83,8 @@ def create_worker(cls: type,
enforce_eager=enforce_eager,
)
(model_config, cache_config, parallel_config,
scheduler_config) = engine_args.create_engine_configs()
(model_config, cache_config, parallel_config, scheduler_config,
_) = engine_args.create_engine_configs()
distributed_init_method = get_distributed_init_method(
get_ip(), get_open_port())