biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[main][BugFix] Fixed an accuracy bug of Qwen3-next-MTP when batched inferring (#30632)

Browse Source 
Signed-off-by: drslark <slarksblood@qq.com>
This commit is contained in:
drslark
2025-12-14 17:32:16 +08:00
committed by GitHub
parent dcb31196da
commit add1b9d3de
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
vllm/v1/attention/backends/gdn_attn.py
View File

@@ -211,7 +211,7 @@ class GDNAttentionMetadataBuilder(AttentionMetadataBuilder[GDNAttentionMetadata]
spec_token_masks = torch.repeat_interleave(
spec_sequence_masks, query_lens
)
index = torch.argsort(spec_token_masks)
index = torch.argsort(spec_token_masks, stable=True)
num_non_spec_tokens = num_prefill_tokens + num_decode_tokens
non_spec_token_indx = index[:num_non_spec_tokens]
spec_token_indx = index[num_non_spec_tokens:]