biondizzle/nvfp4-megamoe-kernel
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Enable MHC_DIAG for diagnostic run

Browse Source
This commit is contained in:
biondizzle
2026-05-31 11:07:23 +00:00
parent cd661c2e40
commit 72c139a59f
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
single_shot_inference.py
View File

@@ -61,7 +61,7 @@ PROMPT = _args.prompt or "The capital of France is"
NUM_GPUS = 8
SKIP_ROUTED_MOE = _args.skip_moe # If True, only use shared expert (debug)
INVERSE_ROPE = not _args.no_inverse_rope # If False, skip inverse RoPE on attention output (diagnostic)
MHC_DIAG = False # If True, print per-layer mHC diagnostics (B_l row/col sums, C_l values)
MHC_DIAG = True # If True, print per-layer mHC diagnostics (B_l row/col sums, C_l values)
# When True: applies inverse RoPE at query position → converts absolute→relative
# When False: leaves relative position encoding intact for output projection
# DSV4 partial RoPE only affects last 64/512 dims; first 448 are always un-RoPE'd