biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[V1] [P/D] Add Support for KV Load Failure Recovery (#19330)

Browse Source 
Signed-off-by: David Ben-David <davidb@pliops.com>
Co-authored-by: David Ben-David <davidb@pliops.com>
This commit is contained in:
David Ben-David
2025-10-01 00:57:08 +03:00
committed by GitHub
parent 67f3fb0844
commit 9a9f48dff7
24 changed files with 1039 additions and 86 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
vllm/v1/worker/gpu_worker.py
View File

@@ -464,8 +464,7 @@ class Worker(WorkerBase):
# In case of PP with kv transfer, we need to pass through the
# kv_connector_output
if (not kv_connector_output.finished_sending
and not kv_connector_output.finished_recving):
if kv_connector_output.is_empty():
return EMPTY_MODEL_RUNNER_OUTPUT
output = copy.copy(EMPTY_MODEL_RUNNER_OUTPUT)