biondizzle

0 Followers · 0 Following

• Joined on 2025-12-10

Repositories

25

Projects Packages Public Activity Starred Repositories

biondizzle pushed to master at biondizzle/nvfp4-megamoe-kernel 2026-05-31 20:23:28 +00:00

baee36e728 Fix dtype mismatch in validate_layer: cast flat to float before F.linear

biondizzle pushed to master at biondizzle/nvfp4-megamoe-kernel 2026-05-31 20:22:18 +00:00

46c4ef2cf5 Add per-layer validation test (tests/validate_layer.py)

biondizzle pushed to master at biondizzle/nvfp4-megamoe-kernel 2026-05-31 20:21:07 +00:00

abe4210367 Add compact per-layer residual trace (GROWTH_DIAG), disable verbose ATTN_DIAG

biondizzle pushed to master at biondizzle/nvfp4-megamoe-kernel 2026-05-31 20:11:39 +00:00

98fa410167 Add HF reference test script

biondizzle pushed to master at biondizzle/nvfp4-megamoe-kernel 2026-05-31 19:29:57 +00:00

a1b39adcaa Add attention entropy diag (ATTN_DIAG), KV cache diag, --no-thinking mode

biondizzle pushed to master at biondizzle/nvfp4-megamoe-kernel 2026-05-31 19:24:30 +00:00

2a886fe0f2 Add --no-thinking mode to skip thinking tokens and use second-best

biondizzle pushed to master at biondizzle/nvfp4-megamoe-kernel 2026-05-31 19:17:27 +00:00

41ef0ebd0f Add KV cache length diagnostic during decode

biondizzle pushed to master at biondizzle/nvfp4-megamoe-kernel 2026-05-31 19:04:55 +00:00

8baebf3c2e Restore --skip-mhc arg, empty system prompt for testing

biondizzle pushed to master at biondizzle/nvfp4-megamoe-kernel 2026-05-31 19:03:58 +00:00

ca661d32e8 Empty system prompt for testing (was causing model to regurgitate AI assistant tokens)

biondizzle pushed to master at biondizzle/nvfp4-megamoe-kernel 2026-05-31 18:52:50 +00:00

b09b2cf511 Fix MoE routing: hash layers 0-2 (tid2eid), e_score_correction_bias for layers 3+

biondizzle pushed to master at biondizzle/nvfp4-megamoe-kernel 2026-05-31 18:38:37 +00:00

7d9e70c5d5 Fix remaining mHC API references: layer_compare.py, layer.py comment

biondizzle pushed to master at biondizzle/nvfp4-megamoe-kernel 2026-05-31 18:38:16 +00:00

7b123d159f CRITICAL FIX: mHC fn/base/scale ordering [pre,post,comb] + comb transposed + Sinkhorn softmax

biondizzle pushed to master at biondizzle/nvfp4-megamoe-kernel 2026-05-31 12:48:49 +00:00

f6c02f808f Add layer-by-layer comparison test for debugging

biondizzle pushed to master at biondizzle/nvfp4-megamoe-kernel 2026-05-31 12:05:29 +00:00

6ad577bd18 Add HuggingFace reference comparison test

biondizzle pushed to master at biondizzle/nvfp4-megamoe-kernel 2026-05-31 11:57:25 +00:00

581c4170f9 Fix sink logits shape: (n_h, T, 1) for concatenation with (n_h, T, seq_len)

biondizzle pushed to master at biondizzle/nvfp4-megamoe-kernel 2026-05-31 11:53:45 +00:00

0f951a0b1a Fix attention sinks: logit bias (HuggingFace reference), not dummy KV

biondizzle pushed to master at biondizzle/nvfp4-megamoe-kernel 2026-05-31 11:47:18 +00:00

daed594902 CRITICAL FIX: Add missing q_b_norm (unweighted RMSNorm after q_b_proj)

biondizzle pushed to master at biondizzle/nvfp4-megamoe-kernel 2026-05-31 11:37:35 +00:00

dd50c355a6 Fix MHC_DIAG null check when SKIP_MHC is enabled

biondizzle pushed to master at biondizzle/nvfp4-megamoe-kernel 2026-05-31 11:33:49 +00:00

631e6ea3e4 Add --skip-mhc flag for simple residual diagnostic

biondizzle pushed to master at biondizzle/nvfp4-megamoe-kernel 2026-05-31 11:25:54 +00:00

d201a9334e CRITICAL FIX: Add YaRN RoPE scaling (factor=16)