biondizzle bfc1518046 FMHA Stage-C2: production 12-warp pipeline with correction warps ...

- Softmax warps (0-3): S→softmax→P, vec=[old_max,new_max]→TMEM
- Correction warps (4-7): O rescale in TMEM, final normalize by row_sum
- MMA warp (8): QK→S, PV→O with pipeline chaining
- TMA warp (9): Q/K/V load
- Epilogue warp (10): O TMEM→GMEM via epilogue_tma_store
- Empty warp (11): tmem dealloc mbar init
- Pipeline: mma_s→softmax→s_corr→correction→corr_epi→epilogue + mma_corr→correction
- Supports multi-tile KV with online O rescale
- Follows CUTLASS FMHA correction_rescale pattern exactly

2026-05-22 09:42:39 +00:00

..

__init__.py

Restructure: cutedsl/ -> dsv4/ with proper layering

2026-05-21 17:30:44 +00:00

cudagraph_test.py

Restructure: cutedsl/ -> dsv4/ with proper layering

2026-05-21 17:30:44 +00:00

layertest.py

Restructure: cutedsl/ -> dsv4/ with proper layering

2026-05-21 17:30:44 +00:00

test_128_128_vdiag.py

Restructure: cutedsl/ -> dsv4/ with proper layering

2026-05-21 17:30:44 +00:00

test_cache.py

KV Cache: schema, allocator, pools, manager, append_swa kernel

2026-05-22 00:08:38 +00:00

test_compile_custom_op.py

Restructure: cutedsl/ -> dsv4/ with proper layering

2026-05-21 17:30:44 +00:00

test_custom_op.py

Restructure: cutedsl/ -> dsv4/ with proper layering

2026-05-21 17:30:44 +00:00

test_cutedsl.py

Restructure: cutedsl/ -> dsv4/ with proper layering

2026-05-21 17:30:44 +00:00

test_dense_router.py

Router: full kernel stack — hash, topk, activation+topk, dense decode/prefill

2026-05-21 21:54:05 +00:00

test_fmha_v3_12w.py

more stuff

2026-05-22 08:57:38 +00:00

test_fmha_v3_debug.py

FMHA v3: add debug variants for C9 normalization investigation

2026-05-22 05:52:10 +00:00

test_fmha_v3_fixed_v.py

FMHA v3: per-row patch from Mike + deadlock fix + V layout fix

2026-05-22 07:09:52 +00:00

test_fmha_v3_noop_c9.py

FMHA v3: add debug variants for C9 normalization investigation

2026-05-22 05:52:10 +00:00

test_fmha_v3_per_row_min.py

FMHA v3: per-row min test + explicit loop replacements

2026-05-22 07:29:04 +00:00

test_fmha_v3_per_row.py

FMHA v3: per-row patch from Mike + deadlock fix + V layout fix

2026-05-22 07:09:52 +00:00

test_fmha_v3_proper.py

FMHA v3: add debug variants for C9 normalization investigation

2026-05-22 05:52:10 +00:00

test_fmha_v3_pva_c9.py

FMHA v3: per-row patch from Mike + deadlock fix + V layout fix

2026-05-22 07:09:52 +00:00

test_fmha_v3_scalar.py

FMHA v3: add debug variants for C9 normalization investigation

2026-05-22 05:52:10 +00:00

test_fmha_v3_shapes.py

FMHA v3: add debug variants for C9 normalization investigation

2026-05-22 05:52:10 +00:00

test_fmha_v3_softmax.py

BREAKTHROUGH: cosine 0.993 for n=128! PV-partitioned P row sum works.

2026-05-21 20:13:51 +00:00

test_fmha_v3_stage_c2.py

FMHA Stage-C2: production 12-warp pipeline with correction warps

2026-05-22 09:42:39 +00:00

test_fmha_v3_stage_c_full.py

test: add multiple seeds to verify softmax consistency

2026-05-22 09:32:08 +00:00

test_fmha_v3_stage_c_min.py

more stuff

2026-05-22 08:57:38 +00:00

test_fmha_v3_stage_c.py

more stuff

2026-05-22 08:57:38 +00:00

test_fmha_v3_vec_c9.py

FMHA v3: add debug variants for C9 normalization investigation

2026-05-22 05:52:10 +00:00

test_fmha_v3.py

Restructure: cutedsl/ -> dsv4/ with proper layering

2026-05-21 17:30:44 +00:00

test_fp4_roundtrip.py

Restructure: cutedsl/ -> dsv4/ with proper layering

2026-05-21 17:30:44 +00:00

test_fused_step1.py

Restructure: cutedsl/ -> dsv4/ with proper layering

2026-05-21 17:30:44 +00:00

test_interleave_gemm.py

Restructure: cutedsl/ -> dsv4/ with proper layering

2026-05-21 17:30:44 +00:00

test_interleave.py

Restructure: cutedsl/ -> dsv4/ with proper layering

2026-05-21 17:30:44 +00:00

test_layer_schedule.py

Layer dispatch: config, schedule, attention/FFN sub-blocks, TransformerLayer

2026-05-21 23:11:09 +00:00

test_pv64_with_softmax.py

Restructure: cutedsl/ -> dsv4/ with proper layering

2026-05-21 17:30:44 +00:00

test_qk_softmax.py

Restructure: cutedsl/ -> dsv4/ with proper layering

2026-05-21 17:30:44 +00:00

test_qkonly.py

Restructure: cutedsl/ -> dsv4/ with proper layering

2026-05-21 17:30:44 +00:00

test_router.py

Router: full kernel stack — hash, topk, activation+topk, dense decode/prefill

2026-05-21 21:54:05 +00:00