biondizzle/nvfp4-megamoe-kernel
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Fix graph replay: remove extra token_id arg from forward_attention call

Browse Source 
The forward_attention() signature has no token_id parameter, but the
graph replay path was passing dec_tid32_per_gpu[gpu] between positions
and compressor — causing the int tensor to be interpreted as compressor
and triggering AttributeError: 'int' object has no attribute 'ratio'
This commit is contained in:
biondizzle
2026-06-04 06:10:02 +00:00
parent ae26f6b83c
commit 26da6d33af
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
single_shot_inference.py
View File

@@ -1841,7 +1841,7 @@ def main():
# Run full attention eagerly (compressor + indexer + FMHA + o_proj)
F_attn, _ = forward_attention(
x_normed, layer_w[li], li, cfg, *rope_caches[gpu],
kv_caches[li], dec_pos_per_gpu[gpu], dec_tid32_per_gpu[gpu],
kv_caches[li], dec_pos_per_gpu[gpu],
compressors.get(li), indexers.get(li), prod_lins.get(li),
q_heads=q_heads, kv_3d=kv_3d, # pass pre-computed q/kv from graph A
comp_rope_cos=comp_rope_caches[gpu][0] if comp_rope_caches else None,