biondizzle fd7c0cb773 P6: Fix TMA store — use bulk_group (commit+wait) not mbarrier ...

TMA store uses cp.async.bulk.tensor.2d.global.shared::cta.tile.bulk_group
NOT mbarrier::complete_tx::bytes. Completion tracked via:
  - cp.async.bulk.commit_group (after issuing stores)
  - cp.async.bulk.wait_group.read 0 (wait for all groups)

Removed sMbarStore from SMEM allocations (no longer needed).

2026-05-30 16:57:35 +00:00

..

archive

archive: TMA driver-API files + CUDA 13 TMA discovery notes

2026-05-29 06:52:39 +00:00

__init__.py

attention/: Clean up folder, archive backups, add detailed status headers

2026-05-28 07:01:33 +00:00

fmha_6warp_multihead.cuh

P6: Fix TMA store — use bulk_group (commit+wait) not mbarrier

2026-05-30 16:57:35 +00:00

fmha_6warp_multirow.cuh

fix: always normalize in kernel, correct KV merge with normalized O + LSE

2026-05-28 23:53:44 +00:00

fmha_6warp_tma_multirow_multitile.cuh

P6: Fix TMA store — use bulk_group (commit+wait) not mbarrier

2026-05-30 16:57:35 +00:00

fmha_6warp_tma_multirow.cuh

feat: HD=512 support — TMEM_N=512, test variants for all three TMA kernels

2026-05-30 03:45:05 +00:00

fmha_6warp_tma_multitile.cuh

feat: HD=512 support — TMEM_N=512, test variants for all three TMA kernels

2026-05-30 03:45:05 +00:00

fmha_6warp_tma.cuh

feat: HD=512 support — TMEM_N=512, test variants for all three TMA kernels

2026-05-30 03:45:05 +00:00

fmha_6warp.cuh

6-warp specialized FMHA kernel — ALL HD=16/64/128/256 PASS cos 0.999997+

2026-05-28 16:34:14 +00:00

fmha_common.cuh

docs: update here-docs with CuTeDSL rationale for NVIDIA

2026-05-28 07:54:01 +00:00

fmha_epilogue_sm100.cuh

docs: update here-docs with CuTeDSL rationale for NVIDIA

2026-05-28 07:54:01 +00:00

fmha_multihead_capi.cu

P6: Fix TMA store — use bulk_group (commit+wait) not mbarrier

2026-05-30 16:57:35 +00:00

fmha_multihead_op.py

P5: in-kernel multi-KV-tile FA2 online softmax in fmha_6warp_multihead.cuh

2026-05-30 08:46:09 +00:00

fmha_multitile_capi.cu

P6: Fix TMA store — use bulk_group (commit+wait) not mbarrier

2026-05-30 16:57:35 +00:00

fmha_multitile_op.py

P5: minimal Python multi-tile test

2026-05-30 10:43:26 +00:00

fmha_qk_verify.cuh

debug: clean QK verify with scalar sanity + MMA result

2026-05-28 08:53:35 +00:00

fmha_sm100_launch.cu

docs: update here-docs with CuTeDSL rationale for NVIDIA

2026-05-28 07:54:01 +00:00

fmha_sm100_tc.cuh

Rewrite fmha_sm100_tc.cuh with working N=16 PV sub-tile approach

2026-05-28 16:04:11 +00:00

fmha_sm100.cuh

docs: update here-docs with CuTeDSL rationale for NVIDIA

2026-05-28 07:54:01 +00:00

fmha_tma.cuh

P6: Fix TMA store — use bulk_group (commit+wait) not mbarrier

2026-05-30 16:57:35 +00:00

fmha_umma_desc.cuh

fix: warp-stride for TMA canonical writes — only load warp calls them

2026-05-29 18:25:47 +00:00

fmha.py

TMA 5D test: element stride decomposition

2026-05-28 19:18:01 +00:00

production.py

P5: integrate WORKING multi-tile kernel (fmha_6warp_tma_multirow_multitile) into production

2026-05-30 10:27:38 +00:00