74380ffc21
REVERT to 0bdcdc0 — the version that passed n=128 cos 0.999998
2026-05-22 20:07:18 +00:00
a3f0e73c82
DEBUG: disable O rescale + normalize, test if corr setup alone causes regression
2026-05-22 19:57:53 +00:00
aa43a902e1
Shared corr tensors for O rescale + final normalize, fix softmax loop
2026-05-22 19:55:06 +00:00
bd52e88ce1
Add O rescale with correction_rescale pattern + fix TMA to working diag pattern
2026-05-22 19:51:53 +00:00
93acac7e43
O normalize: exact CUTLASS correction_rescale pattern with 2D reg tensor
2026-05-22 19:50:37 +00:00
70349de6dc
O normalize: TMEM round-trip with paired Ld/St atoms + standard epilogue_tma_store
2026-05-22 19:49:51 +00:00
02543b3d8d
Fix: use NamedBarrier instead of mbarrier_arrive/wait
2026-05-22 19:49:07 +00:00
5f98eaec3a
Fix: barrier_wait → mbarrier_wait, barrier_arrive → mbarrier_arrive
2026-05-22 19:48:31 +00:00
afe5d1ae21
Fix epilogue: corr_tile_size=16, proper epi_subtile tuple, match CUTLASS reference
2026-05-22 19:47:57 +00:00
8b93774d70
Fix example7: K slice (None,None,0,0) and softmax scale_log2 double-bug
2026-05-22 19:45:59 +00:00
6827862b3d
Fix: fence_async_shared -> fence_view_async_shared
2026-05-22 19:25:05 +00:00
7806ac812a
Clean v2: real softmax P, no O TMEM modify, standard epilogue. Baseline for custom epilogue work.
2026-05-22 19:24:25 +00:00
1176d75236
O normalize using tmem_ptr base (same as epilogue) + CUTLASS sub-tile pattern
2026-05-22 19:18:16 +00:00
86a4719afe
O normalize with full layout (no sub-tiling), Repetition(64)
2026-05-22 19:17:03 +00:00
1d2111f772
Disable ALL O copies to verify baseline
2026-05-22 19:14:33 +00:00
d9f7443b77
Disable O rescale too for NO-OP test
2026-05-22 19:13:43 +00:00
62529df638
DEBUG: O load+store NO-OP to verify TMEM copy correctness
2026-05-22 19:12:43 +00:00
59575202a4
Re-enable O rescale + normalize with corr_tile_size=32
2026-05-22 19:11:17 +00:00
0589a14790
Disable O rescale + normalize, verify softmax P only
2026-05-22 19:09:33 +00:00
8db432ebed
FIX: O sub-tile count should be HEAD_DIM/corr_tile_size, not 128/corr_tile_size
2026-05-22 19:06:52 +00:00
b0498dd7ac
Full pipeline: O rescale + final normalize with CUTLASS sub-tile approach
2026-05-22 19:05:59 +00:00
668c7e1a4e
Test softmax P vs unnormalized reference (no O normalize)
2026-05-22 19:05:27 +00:00
46020baff1
O normalize using CUTLASS reference sub-tile approach
2026-05-22 19:04:37 +00:00
e56ffbbe32
Fix O normalize: use 2D register tensor indexing
2026-05-22 18:59:15 +00:00
43e9eb2f30
Add O normalization with sub-tile TMEM read-modify-write
2026-05-22 18:58:35 +00:00
603385a88c
Simplify: softmax P only, no O rescale/normalize yet
2026-05-22 18:57:00 +00:00
46462b7c8b
Real softmax test built on working identity diag
2026-05-22 18:55:31 +00:00
4fef047f5c
CRITICAL FIX: remove extra scale_log2 in softmax (minus_row_max and acc_scale)
2026-05-22 18:52:58 +00:00
fe58d07b8c
FIX: K slice (None,None,0,0) like working diag
2026-05-22 18:51:22 +00:00
12defe923d
Diag: identity softmax on example6 pipeline to isolate softmax bug
2026-05-22 18:49:33 +00:00
d7e0aa6168
Quick test: working v3 with n=256 multi-tile
2026-05-22 18:47:54 +00:00
6a7b955c19
DEBUG: add version marker to confirm code changes are running
2026-05-22 18:47:05 +00:00
fb1057eb5f
CRITICAL FIX: TMA pre-slice (None,0,None,0) → (None,None,0,0) to keep GMEM tile dim free
2026-05-22 18:45:55 +00:00
2deb28827a
Diag: TMA shapes with hardcoded major modes
2026-05-22 18:43:55 +00:00
d2a3b83aa2
Diag: simplified TMA shape analysis
2026-05-22 18:43:30 +00:00
73109dde85
Diag: print TMA partition shapes for multi-tile debugging
2026-05-22 18:43:05 +00:00
375b24304b
FIX: Use Python range() in TMA warp for concrete per-iteration GMEM coords
2026-05-22 18:26:31 +00:00
0dd6fefd66
FIX: Force SSA GMEM coord via n_kv_tiles - n_kv_tiles instead of cutlass.range kt
2026-05-22 18:25:13 +00:00
2a4731426e
FIX: TMEM offset bug in O rescale/normalize — use tOtO0.iterator not tOtO.iterator
2026-05-22 18:23:46 +00:00
1e0805ad60
Diag: test n=384 (3 tiles) to find crash boundary
2026-05-22 18:07:07 +00:00
1aa4a91d01
Diag: test all sizes 128-1024
2026-05-22 18:06:28 +00:00
8586603280
DEBUG: disable O rescale to isolate NaN cause
2026-05-22 18:05:46 +00:00
45a0fb9971
Add NaN/inf checking to stage C test
2026-05-22 18:01:11 +00:00
147d85a617
CRITICAL FIX: K GMEM slice (None,None,0,0) not (None,0,None,0)
...
K from QK MMA B-partition has GMEM iter at mode 1, NOT mode 2.
(None,0,None,0) hardcodes mode 1 to 0 → TMA always loads tile 0.
(None,None,0,0) keeps mode 1 free → correct multi-tile loading.
Proof: diag n=256 went from cos 0.711 → 0.999999 with this one change.
2026-05-22 17:59:57 +00:00
bc3e94ff45
Diag: try K slice (None,None,0,0) keeping mode 1 (CUTLASS ref style)
2026-05-22 17:59:01 +00:00
601e662dd4
Diag: try runtime Int32(0+0) for kv_coord with cutlass.range
2026-05-22 17:57:58 +00:00
e5030cbea5
Diag: use Python range() unrolling like stage C test
2026-05-22 17:56:59 +00:00
5ee34d925b
Fix diagnostic test: same Int32(kt) + n_kv_tiles fixes
2026-05-22 17:56:15 +00:00
d2bbdd59f6
Try cutlass.range with Int32(kt) — now n_kv_tiles is Python int
2026-05-22 17:51:25 +00:00
bf80fbee99
FIX: n_kv_tiles as Python int (s_k//128) for range() unrolling
...
cute.size() returns a CuTeDSL symbol, not a Python int.
range() on a symbol can't iterate — the loop never unrolls.
Now n_kv_tiles is computed in __init__ as s_k // 128 (Python int).
2026-05-22 17:50:07 +00:00
