biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

Fix KV Offloading + MLA AssertionError by using num_kv_heads=1 in cpu… (#37536)

Browse Source 
Signed-off-by: xueliangyang-oeuler <yxl546827391@gmail.com>
Co-authored-by: xueliangyang-oeuler <yxl546827391@gmail.com>
This commit is contained in:
XueLiang Yang
2026-03-19 20:05:07 +08:00
committed by GitHub
parent 7a6ebcbfcf
commit e390742c59
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
vllm/v1/kv_offload/worker/cpu_gpu.py
View File

@@ -240,7 +240,7 @@ class CpuGpuOffloadingHandlers:
gpu_shape = gpu_tensor.shape
attn_backend = attn_backends[layer_name]
test_shape = attn_backend.get_kv_cache_shape(
num_blocks=1234, block_size=16, num_kv_heads=8, head_size=256
num_blocks=1234, block_size=16, num_kv_heads=1, head_size=256
)
has_layers_dim = False