biondizzle/nvfp4-megamoe-kernel
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
1,714 Commits 1 Branch 17 Tags
d47b2bfcce860bfd44261a3a82a2cd6147d5ea46
Commit Graph

7 Commits

Author SHA1 Message Date
biondizzle
2dcfc0089f auto: pre-test commit 2026-05-28 15:49:47 +00:00
biondizzle
74dba6ab9d auto: pre-test commit 2026-05-28 04:40:20 +00:00
biondizzle
acf46c494c NVFP4-1.1: update approach doc and fp4_quant with CuTeDSL API fixes 2026-05-28 04:09:58 +00:00
biondizzle
80b6b79f9e NVFP4-1.1: FP4 quantization primitives for CuTeDSL kernels 
- fp8_e4m3_from_float32: manual FP8 E4M3 cast (bias=7, exp 0-15 valid,
  NaN guard for exp=15/mant=7, mantissa overflow handling)
- fp8_e4m3_to_float32: dequantize FP8 E4M3 bit pattern back to Float32
- half_step_to_e2m1_idx: E2M1 step mapping (0-12 → 0-7)
- quantize_e2m1_nibble: per-element E2M1 quantize + sign + pack
- Verified 0/500 trial failures against Python reference
- Key fixes discovered during validation:
  1. FP8 E4M3 bias is 7, NOT 8
  2. Exponent range is 0-15 (exp=15/mant=7 is NaN; others valid)
  3. Subnormal formula: val = m * 2^(-9) = m/512 (NOT m/1024)
  4. Round-to-nearest-even (not round-half-up) for half_step and mantissa
  5. Mantissa overflow (round to 8) must increment exponent
 2026-05-28 03:39:55 +00:00
biondizzle
064ececc9a Update docs: D1.5 TMEM round-trip fundamentally broken, Python KV merge is production path 2026-05-26 19:53:10 +00:00
biondizzle
43f0b5d1e8 D1.5: Fix O rescale with paired atoms (incremental approach) 
Keep epilogue_tma_store for final output (proven path).
Only fix the multi-KV-tile O rescale using paired atoms from
epilogue_tmem_copy_and_partition. The paired atoms share addressing,
making the TMEM->REGS->modify->TMEM cycle lossless.

Guarded by const_expr(n_kv_tiles > 1) so single-tile path (n=128)
is completely unaffected — zero regression risk.

Full correction epilogue (one-way TMEM->REGS->SMEM->GMEM) deferred
until we can address the MLIR compilation time issue.
 2026-05-26 19:34:26 +00:00
biondizzle
4bb0e063cc D1.5: Replace broken TMEM round-trip with correction epilogue (paired atoms) 
Replace hand-constructed Ld32x32bOp/St32x32bOp TMEM round-trip with the
proven correction epilogue pattern from fused_swiglu.py:

1. O rescale (kt>0): TMEM→REGS (paired load), multiply by acc_scale,
   REGS→TMEM (paired store via retile_to_S). No layout mismatch.

2. Final O output: One-way TMEM→REGS→SMEM→GMEM using
   epilogue_tmem_copy_and_partition + epilogue_smem_copy_and_partition
   + TMA partition. Register-level normalization (divide by row_sum)
   or raw BF16 cast for D5a path.

This fixes both D1.5 issues:
- Issue 1: TMEM round-trip corruption (hand-constructed atoms)
- Issue 2: O rescale for multi-KV-tile (kt>0)

Supports normalize=True (in-kernel) and normalize=False (D5a external).
Uses epilog_sync_bar + c_pipe for SMEM→GMEM, replacing epilogue_tma_store.
 2026-05-26 19:11:19 +00:00