biondizzle b3020c2811 6-warp specialized FMHA kernel — ALL HD=16/64/128/256 PASS cos 0.999997+ ...

Warp layout (192 threads):
- Warps 0-3: Softmax + correction + epilogue
- Warp 4: MMA (QK + PV GEMM)
- Warp 5: Data staging (Q/K/V loads, direct GMEM for now)
CTA-wide __syncthreads() sync between phases.

Fix: removed spurious inv_sum normalization in epilogue
(MMA output is already correctly scaled with softmax'd P).

Files: fmha_6warp.cuh + test_fmha_6warp*.cu

2026-05-28 16:34:14 +00:00

..

archive

add back in the archived code

2026-05-28 07:04:59 +00:00

__init__.py

attention/: Clean up folder, archive backups, add detailed status headers

2026-05-28 07:01:33 +00:00

fmha_6warp.cuh

6-warp specialized FMHA kernel — ALL HD=16/64/128/256 PASS cos 0.999997+

2026-05-28 16:34:14 +00:00

fmha_common.cuh

docs: update here-docs with CuTeDSL rationale for NVIDIA

2026-05-28 07:54:01 +00:00

fmha_epilogue_sm100.cuh

docs: update here-docs with CuTeDSL rationale for NVIDIA

2026-05-28 07:54:01 +00:00

fmha_qk_verify.cuh

debug: clean QK verify with scalar sanity + MMA result

2026-05-28 08:53:35 +00:00

fmha_sm100_launch.cu

docs: update here-docs with CuTeDSL rationale for NVIDIA

2026-05-28 07:54:01 +00:00

fmha_sm100_tc.cuh

Rewrite fmha_sm100_tc.cuh with working N=16 PV sub-tile approach

2026-05-28 16:04:11 +00:00

fmha_sm100.cuh

docs: update here-docs with CuTeDSL rationale for NVIDIA

2026-05-28 07:54:01 +00:00

fmha_umma_desc.cuh

UMMA: fix descriptor + idesc — use gau-nernst tutorial values

2026-05-28 09:18:45 +00:00

fmha.py

attention/: Clean up folder, archive backups, add detailed status headers

2026-05-28 07:01:33 +00:00

production.py

attention/: Clean up folder, archive backups, add detailed status headers

2026-05-28 07:01:33 +00:00