[Bugfix] fix encoder cache leak of waiting requests in scheduler to solve stuck in CPU scheduling (#31857)

Signed-off-by: frelam <frelam112233@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
This commit is contained in:
Frelam
2026-01-10 14:27:58 +08:00
committed by GitHub
parent da6709c9fe
commit a01a1c0d69

View File

@@ -598,6 +598,11 @@ class Scheduler(SchedulerInterface):
if new_blocks is None:
# The request cannot be scheduled.
# NOTE: we need to untouch the request from the encode cache
# manager
if request.has_encoder_inputs:
self.encoder_cache_manager.free(request)
break
# KVTransfer: the connector uses this info to determine