biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[V1][3/N] API Server: Reduce Task Switching + Handle Abort Properly (#11534)

Browse Source
This commit is contained in:
Robert Shaw
2024-12-27 00:19:21 -05:00
committed by GitHub
parent eb881ed006
commit 1b875a0ef3
3 changed files with 59 additions and 149 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
vllm/v1/engine/core.py
View File

@@ -32,7 +32,7 @@ logger = init_logger(__name__)
POLLING_TIMEOUT_MS = 5000
POLLING_TIMEOUT_S = POLLING_TIMEOUT_MS // 1000
LOGGING_TIME_S = 5000
LOGGING_TIME_S = POLLING_TIMEOUT_S
class EngineCore: