biondizzle d003c4b7cc Add mHC (Manifold-Constrained Hyper-Connections) to single_shot ...

- Full mHC pre_block/post_block with Sinkhorn-Knopp normalization
- Dynamic A_l (sigmoid), B_l (Birkhoff polytope), C_l (2*sigmoid)
- Checkpoint: attn_hc.fn (24,28672) + base (24,) + scale (3,)
- Two mHC blocks per layer: attn_hc + ffn_hc
- Removed emergency RMSNorm — mHC handles normalization properly
- X_l: (1, n_hc=4, H) residual state, init from embedding broadcast

2026-05-30 23:45:18 +00:00

19 KiB

Raw Blame History

View Raw