biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[core] add sleep and wake up endpoint and v1 support (#12987)

Browse Source 
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: cennn <2523403608@qq.com>
Co-authored-by: cennn <2523403608@qq.com>
This commit is contained in:
youkaichao
2025-02-20 12:41:17 +08:00
committed by GitHub
parent 0d243f2a54
commit ba81163997
13 changed files with 160 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
vllm/v1/engine/async_llm.py
View File

@@ -361,6 +361,12 @@ class AsyncLLM(EngineClient):
async def reset_prefix_cache(self) -> None:
await self.engine_core.reset_prefix_cache_async()
async def sleep(self, level: int = 1) -> None:
await self.engine_core.sleep_async(level)
async def wake_up(self) -> None:
await self.engine_core.wake_up_async()
async def add_lora(self, lora_request: LoRARequest) -> None:
"""Load a new LoRA adapter into the engine for future requests."""
await self.engine_core.add_lora_async(lora_request)