biondizzle 208af3eadd FMHA v3 Stage-C full: 12-warp pipeline with real softmax + correction + epilogue ...

- Softmax warps (0-3): online row max, exp2 scaling, P store, vec broadcast
- Correction warps (4-7): online O rescale, final normalization, SMEM write
- MMA warp (8): QK->S, PV->O with proper pipeline chaining
- TMA warp (9): Q/K/V load
- Epilogue warp (10): TMA store O from SMEM to GMEM
- Empty warp (11): tmem dealloc mbar init
- Pipeline chain: mma_s -> softmax -> s_corr -> correction -> corr_epi -> epilogue
- Plus mma_corr -> correction for O rescale
- Reference test uses softmax(Q@K^T/sqrt(d))@V

2026-05-22 09:18:56 +00:00

..

__init__.py

Restructure: cutedsl/ -> dsv4/ with proper layering

2026-05-21 17:30:44 +00:00

cudagraph_test.py

Restructure: cutedsl/ -> dsv4/ with proper layering

2026-05-21 17:30:44 +00:00

layertest.py

Restructure: cutedsl/ -> dsv4/ with proper layering

2026-05-21 17:30:44 +00:00

test_128_128_vdiag.py

Restructure: cutedsl/ -> dsv4/ with proper layering

2026-05-21 17:30:44 +00:00

test_cache.py

KV Cache: schema, allocator, pools, manager, append_swa kernel

2026-05-22 00:08:38 +00:00

test_compile_custom_op.py

Restructure: cutedsl/ -> dsv4/ with proper layering

2026-05-21 17:30:44 +00:00

test_custom_op.py

Restructure: cutedsl/ -> dsv4/ with proper layering

2026-05-21 17:30:44 +00:00

test_cutedsl.py

Restructure: cutedsl/ -> dsv4/ with proper layering

2026-05-21 17:30:44 +00:00

test_dense_router.py

Router: full kernel stack — hash, topk, activation+topk, dense decode/prefill

2026-05-21 21:54:05 +00:00

test_fmha_v3_12w.py

more stuff

2026-05-22 08:57:38 +00:00

test_fmha_v3_debug.py

FMHA v3: add debug variants for C9 normalization investigation

2026-05-22 05:52:10 +00:00

test_fmha_v3_fixed_v.py

FMHA v3: per-row patch from Mike + deadlock fix + V layout fix

2026-05-22 07:09:52 +00:00

test_fmha_v3_noop_c9.py

FMHA v3: add debug variants for C9 normalization investigation

2026-05-22 05:52:10 +00:00

test_fmha_v3_per_row_min.py

FMHA v3: per-row min test + explicit loop replacements

2026-05-22 07:29:04 +00:00

test_fmha_v3_per_row.py

FMHA v3: per-row patch from Mike + deadlock fix + V layout fix

2026-05-22 07:09:52 +00:00

test_fmha_v3_proper.py

FMHA v3: add debug variants for C9 normalization investigation

2026-05-22 05:52:10 +00:00

test_fmha_v3_pva_c9.py

FMHA v3: per-row patch from Mike + deadlock fix + V layout fix

2026-05-22 07:09:52 +00:00

test_fmha_v3_scalar.py

FMHA v3: add debug variants for C9 normalization investigation

2026-05-22 05:52:10 +00:00

test_fmha_v3_shapes.py

FMHA v3: add debug variants for C9 normalization investigation

2026-05-22 05:52:10 +00:00

test_fmha_v3_softmax.py

BREAKTHROUGH: cosine 0.993 for n=128! PV-partitioned P row sum works.

2026-05-21 20:13:51 +00:00

test_fmha_v3_stage_c_full.py

FMHA v3 Stage-C full: 12-warp pipeline with real softmax + correction + epilogue

2026-05-22 09:18:56 +00:00

test_fmha_v3_stage_c_min.py

more stuff

2026-05-22 08:57:38 +00:00

test_fmha_v3_stage_c.py

more stuff

2026-05-22 08:57:38 +00:00

test_fmha_v3_vec_c9.py

FMHA v3: add debug variants for C9 normalization investigation

2026-05-22 05:52:10 +00:00

test_fmha_v3.py

Restructure: cutedsl/ -> dsv4/ with proper layering

2026-05-21 17:30:44 +00:00

test_fp4_roundtrip.py

Restructure: cutedsl/ -> dsv4/ with proper layering

2026-05-21 17:30:44 +00:00

test_fused_step1.py

Restructure: cutedsl/ -> dsv4/ with proper layering

2026-05-21 17:30:44 +00:00

test_interleave_gemm.py

Restructure: cutedsl/ -> dsv4/ with proper layering

2026-05-21 17:30:44 +00:00

test_interleave.py

Restructure: cutedsl/ -> dsv4/ with proper layering

2026-05-21 17:30:44 +00:00

test_layer_schedule.py

Layer dispatch: config, schedule, attention/FFN sub-blocks, TransformerLayer

2026-05-21 23:11:09 +00:00

test_pv64_with_softmax.py

Restructure: cutedsl/ -> dsv4/ with proper layering

2026-05-21 17:30:44 +00:00

test_qk_softmax.py

Restructure: cutedsl/ -> dsv4/ with proper layering

2026-05-21 17:30:44 +00:00

test_qkonly.py

Restructure: cutedsl/ -> dsv4/ with proper layering

2026-05-21 17:30:44 +00:00

test_router.py

Router: full kernel stack — hash, topk, activation+topk, dense decode/prefill

2026-05-21 21:54:05 +00:00