biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[Bugfix] Fixing division by zero in triton_attn if query_heads/kv_heads > 16 (#23424)

Browse Source 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
This commit is contained in:
Burkhard Ringlein
2025-09-03 17:01:09 +02:00
committed by GitHub
parent 4ba0c587ba
commit 6d80ae83e1
1 changed files with 2 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
vllm/attention/ops/triton_unified_attention.py
View File

@@ -674,7 +674,8 @@ def unified_attention(
num_queries_per_kv = num_query_heads // num_kv_heads
head_size = q.shape[2]
BLOCK_M = 16
BLOCK_M = 16 if num_queries_per_kv <= 16 else triton.next_power_of_2(
num_queries_per_kv)
BLOCK_Q = BLOCK_M // num_queries_per_kv
# Ideally we would launch with kernel with: