biondizzle f52eedbdce Add production-value tests: ALL tests use Pro config (61L, HD=512, 384 experts, HCA=128, 1M context) ...

Previous unit tests used toy values (HD=64-256, T=16, small N).
These tests validate the actual production configuration:
- FMHA: HD=512, 128 Q heads, N=128/2048/8192
- Compression: CSA T=4096, HCA T=16384, full 1M context
- NVFP4: production weight shapes (q_a, kv, wo_a, gate)
- MoE: 384 experts, top-6, 3072 intermediate
- mHC: 4 streams, 61 layers, residual bounded, doubly-stochastic
- Router: 384 experts hash + noaux-TC
- Memory budget: 1M context KV pool, 8-GPU weight distribution

2026-06-02 04:10:39 +00:00

21 KiB

Raw Blame History

View Raw