biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[Minor] Enhance error message for TRTLLM decode uniformity check (#36609)

Browse Source 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
This commit is contained in:
Woosuk Kwon
2026-03-10 15:38:45 -07:00
committed by GitHub
parent 8d983d7cd6
commit 195d1ca3e8
1 changed files with 2 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
vllm/v1/attention/backends/flashinfer.py
View File

@@ -1110,7 +1110,8 @@ class FlashInferMetadataBuilder(AttentionMetadataBuilder[FlashInferMetadata]):
if num_decodes > 0: if num_decodes > 0:
if decode_use_trtllm: if decode_use_trtllm:
assert num_decode_tokens % num_decodes == 0, ( assert num_decode_tokens % num_decodes == 0, (
"TRTLLM decode requires uniform query lengths per request." "TRTLLM decode requires uniform query lengths per request. "
f"Got {num_decode_tokens=} and {num_decodes=}."
) )
attn_metadata.decode = TRTLLMDecode( attn_metadata.decode = TRTLLMDecode(
block_tables=block_table_tensor[:num_decodes], block_tables=block_table_tensor[:num_decodes],