[HybridKVCache][Platform] Add support_hybrid_kv_cache for platform (#24646)

Signed-off-by: MengqingCao <cmq0113@163.com>
This commit is contained in:
Mengqing Cao
2025-09-11 21:47:58 +08:00
committed by GitHub
parent 94e6b2d55f
commit 4f6593b058
5 changed files with 20 additions and 2 deletions

View File

@@ -3529,8 +3529,7 @@ class VllmConfig:
# logger should only print warning message for hybrid models. As we
# can't know whether the model is hybrid or not now, so we don't log
# warning message here and will log it later.
if not (current_platform.is_cuda() or current_platform.is_rocm()
or current_platform.is_cpu()):
if not current_platform.support_hybrid_kv_cache():
# Hybrid KV cache manager is not supported on non-GPU platforms.
self.scheduler_config.disable_hybrid_kv_cache_manager = True
if self.kv_transfer_config is not None: