biondizzle c55030a340 P5: clean kernel with runtime branch (single-tile unchanged, multi-tile separate path) ...

Single-tile path is IDENTICAL to the working pre-P5 kernel.
Multi-tile path uses FA2 online softmax with sOacc accumulator.
Runtime branch on is_multi_tile = (n_kv_tiles > 1).

2026-05-30 08:57:00 +00:00

..

archive

archive: TMA driver-API files + CUDA 13 TMA discovery notes

2026-05-29 06:52:39 +00:00

__init__.py

attention/: Clean up folder, archive backups, add detailed status headers

2026-05-28 07:01:33 +00:00

fmha_6warp_multihead.cuh

P5: clean kernel with runtime branch (single-tile unchanged, multi-tile separate path)

2026-05-30 08:57:00 +00:00

fmha_6warp_multirow.cuh

fix: always normalize in kernel, correct KV merge with normalized O + LSE

2026-05-28 23:53:44 +00:00

fmha_6warp_tma_multirow_multitile.cuh

fix: native TMEM columns for hd_chunk (no remapping)

2026-05-30 07:01:42 +00:00

fmha_6warp_tma_multirow.cuh

feat: HD=512 support — TMEM_N=512, test variants for all three TMA kernels

2026-05-30 03:45:05 +00:00

fmha_6warp_tma_multitile.cuh

feat: HD=512 support — TMEM_N=512, test variants for all three TMA kernels

2026-05-30 03:45:05 +00:00

fmha_6warp_tma.cuh

feat: HD=512 support — TMEM_N=512, test variants for all three TMA kernels

2026-05-30 03:45:05 +00:00

fmha_6warp.cuh

6-warp specialized FMHA kernel — ALL HD=16/64/128/256 PASS cos 0.999997+

2026-05-28 16:34:14 +00:00

fmha_common.cuh

docs: update here-docs with CuTeDSL rationale for NVIDIA

2026-05-28 07:54:01 +00:00

fmha_epilogue_sm100.cuh

docs: update here-docs with CuTeDSL rationale for NVIDIA

2026-05-28 07:54:01 +00:00

fmha_multihead_capi.cu

P5: in-kernel multi-KV-tile FA2 online softmax in fmha_6warp_multihead.cuh

2026-05-30 08:46:09 +00:00

fmha_multihead_op.py

P5: in-kernel multi-KV-tile FA2 online softmax in fmha_6warp_multihead.cuh

2026-05-30 08:46:09 +00:00

fmha_qk_verify.cuh

debug: clean QK verify with scalar sanity + MMA result

2026-05-28 08:53:35 +00:00

fmha_sm100_launch.cu

docs: update here-docs with CuTeDSL rationale for NVIDIA

2026-05-28 07:54:01 +00:00

fmha_sm100_tc.cuh

Rewrite fmha_sm100_tc.cuh with working N=16 PV sub-tile approach

2026-05-28 16:04:11 +00:00

fmha_sm100.cuh

docs: update here-docs with CuTeDSL rationale for NVIDIA

2026-05-28 07:54:01 +00:00

fmha_tma.cuh

fix: smart quote in fmha_tma.cuh

2026-05-29 18:22:26 +00:00

fmha_umma_desc.cuh

fix: warp-stride for TMA canonical writes — only load warp calls them

2026-05-29 18:25:47 +00:00

fmha.py

TMA 5D test: element stride decomposition

2026-05-28 19:18:01 +00:00

production.py

P5: in-kernel multi-KV-tile FA2 online softmax in fmha_6warp_multihead.cuh

2026-05-30 08:46:09 +00:00