biondizzle/nvfp4-megamoe-kernel
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Commit Graph

  • 17109a8f04 D1.3: Fix tOrP0 for SMEM-P - skip make_tensor when offset is 0 biondizzle 2026-05-23 21:03:00 +00:00
  • 0727725634 D1.3: Fix tOrP0 offset - scale FP32 columns to BF16 elements biondizzle 2026-05-23 21:02:04 +00:00
  • c83b7de258 D1.3: Fix CuTeDSL scoping - define tOrP0 unconditionally with p0 offset biondizzle 2026-05-23 21:01:18 +00:00
  • cba41d500c D1.3: Fix critical bug - add TMEM column offset for P0 in PV GEMM biondizzle 2026-05-23 21:00:29 +00:00
  • ed71f25903 D1.3: Revert to d1.3-pre-sm100-helpers baseline for testing biondizzle 2026-05-23 20:58:06 +00:00
  • 79a17303c4 D1.3: DIAGNOSTIC - test epilogue_tma_store raw PV without any round-trips biondizzle 2026-05-23 20:57:13 +00:00
  • b926a0e806 D1.3: Remove NO-OP round-trip, keep normalize + epilogue_tma_store biondizzle 2026-05-23 20:56:13 +00:00
  • 820d6921d9 D1.3: Full correction_epilog with TMA store, normalize in reg before SMEM write biondizzle 2026-05-23 20:54:23 +00:00
  • 1c74d02adb D1.3: Apply transform_partitioned_tensor_layout before epilogue helpers biondizzle 2026-05-23 20:52:42 +00:00
  • 1cf7140ea3 D1.3: Replace NO-op TMEM round-trip with correction_epilog using epilogue_tmem_copy_and_partition + epilogue_smem_copy_and_partition biondizzle 2026-05-23 20:50:23 +00:00
  • 1e55e36919 SMEM-P: test permutation 4 (swap m↔n2) biondizzle 2026-05-23 20:20:07 +00:00
  • c0e2fe43f1 SMEM-P: add iterator offset debug print biondizzle 2026-05-23 20:19:22 +00:00
  • 42880fa397 SMEM-P: add tCrP debug print, reset permute to 0 biondizzle 2026-05-23 20:14:32 +00:00
  • 1f46a62f65 SMEM-P: test permutation 1 (swap m↔n0) biondizzle 2026-05-23 20:14:21 +00:00
  • 57d5e4c9e9 auto: pre-test commit biondizzle 2026-05-23 20:13:47 +00:00
  • 474363ce59 SMEM-P: add debug_permute flag for coordinate permutation testing biondizzle 2026-05-23 20:13:44 +00:00
  • cf9050c333 SMEM-P: disable debug flags, revert to original mapping biondizzle 2026-05-23 20:12:26 +00:00
  • c7f613644f SMEM-P: fix scoping error, disable debug_p_one, enable debug_swap_mn biondizzle 2026-05-23 20:11:50 +00:00
  • 14524295c7 SMEM-P: add debug_swap_mn flag to test swapped coordinate mapping biondizzle 2026-05-23 20:10:39 +00:00
  • aafbe16a81 SMEM-P: add debug_p_one flag to write constant P=1.0 biondizzle 2026-05-23 20:09:47 +00:00
  • 75b3d0f4b4 auto: pre-test commit biondizzle 2026-05-23 20:08:31 +00:00
  • b551287123 SMEM-P: implement two-phase softmax with normalization before SMEM write biondizzle 2026-05-23 20:08:29 +00:00
  • c88ec626e0 Revert TMEM layout change (caused error) biondizzle 2026-05-23 20:07:02 +00:00
  • bb7f53702f SMEM-P: try using PV A-operand layout directly for TMEM-P biondizzle 2026-05-23 20:06:23 +00:00
  • cbb85cf091 SMEM-P: debug compute offset for known coordinate biondizzle 2026-05-23 20:05:02 +00:00
  • 64aa69c421 SMEM-P: add crd2idx debug attempt biondizzle 2026-05-23 20:04:28 +00:00
  • 1854fa19d5 SMEM-P: test pattern based on fragment indices (k,j) biondizzle 2026-05-23 20:03:22 +00:00
  • ae7b550c3b auto: pre-test commit biondizzle 2026-05-23 20:02:35 +00:00
  • f21167f029 SMEM-P: add shape debug prints biondizzle 2026-05-23 20:02:32 +00:00
  • d2d0eec33a SMEM-P: add debug prints for coordinates biondizzle 2026-05-23 20:00:33 +00:00
  • 8edf2d434c SMEM-P: implement CUTLASS LLM fixes - dynamic frg_tile, local coordinate conversion biondizzle 2026-05-23 19:59:52 +00:00
  • 44fc59b8fa auto: pre-test commit biondizzle 2026-05-23 19:53:45 +00:00
  • 2b51837465 SMEM-P: try transposed mapping (swap m/n) biondizzle 2026-05-23 19:53:42 +00:00
  • 01fe51b175 SMEM-P: current state - working but mapping wrong (cos 0.02) biondizzle 2026-05-23 19:53:25 +00:00
  • 3d044b4747 SMEM-P: debug with linear index pattern m*128+n biondizzle 2026-05-23 19:52:46 +00:00
  • 81630037bd SMEM-P: debug with test pattern (k+j)*0.01 biondizzle 2026-05-23 19:52:02 +00:00
  • 1eb0c1b47a SMEM-P: fix coordinate access - qk_coord is (m,n) not ((m,n),0,0) biondizzle 2026-05-23 19:38:11 +00:00
  • dd42245478 SMEM-P: fix scoping - define tTMEM_LOADcS_frg unconditionally biondizzle 2026-05-23 19:37:34 +00:00
  • 25bb52feef auto: pre-test commit biondizzle 2026-05-23 19:36:58 +00:00
  • e09c8057be SMEM-P: implement full 128-value write in softmax loop using coordinate mapping biondizzle 2026-05-23 19:36:56 +00:00
  • e118ad967d SMEM-P: fix BF16 value creation (use constant) biondizzle 2026-05-23 19:33:29 +00:00
  • 58639aa634 SMEM-P: implement CUTLASS LLM coordinate mapping pattern (minimal test) biondizzle 2026-05-23 19:32:11 +00:00
  • dfe1d3803b SMEM-P: fix thread_idx tuple access biondizzle 2026-05-23 19:30:09 +00:00
  • c01d8b493f auto: pre-test commit biondizzle 2026-05-23 19:29:29 +00:00
  • ec84d35cf8 SMEM-P: add debug to understand thread partitioning biondizzle 2026-05-23 19:29:27 +00:00
  • ee969d4c46 Update STAGE_D.md: manual SMEM addressing blocked on layout mapping biondizzle 2026-05-23 19:22:28 +00:00
  • e0407793af SMEM-P: implement simple test pattern instead of coord lookup biondizzle 2026-05-23 19:21:31 +00:00
  • 841db091f7 auto: pre-test commit biondizzle 2026-05-23 19:20:42 +00:00
  • 9b72411ca7 Start implementing manual SMEM-P addressing (helpers are a trap) biondizzle 2026-05-23 19:20:40 +00:00
  • e0a2d272f4 Implement manual SMEM-P copy instead of cute.copy (helpers are a trap) biondizzle 2026-05-23 19:14:44 +00:00
  • 4d6acaeef0 auto: pre-test commit biondizzle 2026-05-23 19:14:02 +00:00
  • 341527977c Try flattening sP and rP_bf16_qk with group_modes to fix rank mismatch biondizzle 2026-05-23 19:13:59 +00:00
  • c9448aca03 Add debug prints for SMEM-P partition layouts biondizzle 2026-05-23 19:13:07 +00:00
  • b4b11db0fa Fix SMEM-P: use BF16 copy atom and BF16 source with QK C-fragment layout biondizzle 2026-05-23 19:12:13 +00:00
  • 7b65adf7a3 Fix SMEM-P copy: use tcgen05.copy.St32x32bOp with Float32 and copy from rP_words (Float32) not rP_bf16 biondizzle 2026-05-23 19:11:08 +00:00
  • dc2c9ffb92 Update STAGE_D.md with current action plan - starting NVFP4-0 verification and D1.3 validation on B200 biondizzle 2026-05-23 19:09:56 +00:00
  • f0f78b804c 📋 Update STAGE_D.md: D1.3 ✅ SOLVED, D1.4 ✅ IMPLEMENTED, D1.5 🟡 complex refactor, checklist updated biondizzle 2026-05-23 18:37:53 +00:00
  • d995cd0c5c 🎉 Mark D1.3 as SOLVED! SMEM-P rank mismatch fixed, enables hd>64 support biondizzle 2026-05-23 18:26:15 +00:00
  • 4bf3c435b5 Fix rP scope issue: use rP_bf16.iterator instead of rP.iterator biondizzle 2026-05-23 09:36:22 +00:00
  • 431b8e0abe Fix duplicate else: line in SMEM-P block biondizzle 2026-05-23 09:35:47 +00:00
  • d2bb02a331 SMEM-P: Use QK C-fragment layout instead of TMEM layout to fix rank mismatch biondizzle 2026-05-23 09:35:24 +00:00
  • 63c9a5ce82 Fix sP_2d definition for tSMEM_CPYsP biondizzle 2026-05-23 09:34:50 +00:00
  • c7006b0969 Remove debug print lines referencing deleted sP_2d biondizzle 2026-05-23 09:34:09 +00:00
  • b09c432942 Remove duplicate sP_2d line causing indentation error biondizzle 2026-05-23 09:33:40 +00:00
  • 518dce37f0 SMEM-P: Implement rank mismatch fix by reshaping source tensor biondizzle 2026-05-23 09:33:24 +00:00
  • a3659c581d Update STAGE_D.md checklist with current progress and lessons learned biondizzle 2026-05-23 09:27:48 +00:00
  • c9dda47971 Add more debug prints for sP shapes biondizzle 2026-05-23 09:26:30 +00:00
  • 2283de1cfc Add debug prints to SMEM-P path to understand rank mismatch biondizzle 2026-05-23 09:25:48 +00:00
  • 7c350e6a18 Fix SMEM-P copy rank mismatch (use rP_bf16 directly instead of group_modes) biondizzle 2026-05-23 09:21:13 +00:00
  • cb2849bff5 D1.3: Implement SMEM-P path (write P to SMEM via tiled_smem_copy instead of zeroing sP) biondizzle 2026-05-23 09:19:35 +00:00
  • 2c36cd0d32 Stage D1: Multi-PV-tile support for hd>256 (tcgen05 MMA max N=256) biondizzle 2026-05-23 09:04:01 +00:00
  • f556060ddf Fix v_fmha layout to use pv_n_tile instead of head_dim for multi-PV-tile support biondizzle 2026-05-23 09:02:01 +00:00
  • f1ad264da6 D1.4: Add pv_n_tile and n_pv_tiles for multi-PV-tile support (tcgen05 MMA max N=256) biondizzle 2026-05-23 09:00:18 +00:00
  • 2083204a90 diag: add 2-CTA check + fix LayoutEnum in MMA test biondizzle 2026-05-23 08:45:26 +00:00
  • 02daa63f67 fix: remove bad import in NVFP4 diag biondizzle 2026-05-23 08:44:37 +00:00
  • 401e24768a fix: import ceil_div in quantize.py (was NameError at runtime) biondizzle 2026-05-23 08:40:24 +00:00
  • 28724b8c18 fix: use quantize_activation_nvfp4 in diag biondizzle 2026-05-23 08:39:12 +00:00
  • 690be100aa fix: use correct API for NVFP4-0 diag (sf_vec_size + mma_tiler_mn) biondizzle 2026-05-23 08:38:19 +00:00
  • 0fc43eb624 fix: use Sm100BlockScaledPersistentDenseGemmKernel in diag biondizzle 2026-05-23 08:30:43 +00:00
  • e6c9616eba fix: use randint+view for FP4/FP8 tensors in diag biondizzle 2026-05-23 08:29:16 +00:00
  • 96567ea64d fix: NVFP4-0 diag script — import SF_VEC_SIZE from quantize.py biondizzle 2026-05-23 08:28:13 +00:00
  • f6f4ce8ec2 diag: NVFP4-0 primitive verification script biondizzle 2026-05-23 08:26:56 +00:00
  • 241b49b1ee docs: add NVFP4 precision roadmap to STAGE_D.md (3 honest buckets + speculative bucket) biondizzle 2026-05-23 07:39:09 +00:00
  • 73fa8a2b70 shit carmine left dangling biondizzle 2026-05-23 06:55:22 +00:00
  • bd2da14ca6 D1.2: TMEM budget verified on B200. Split-PV mandatory at hd=512 (MMA max N=256) biondizzle 2026-05-23 06:43:01 +00:00
  • a951a95276 D1.2: fix probe for hd=512 (MMA max N=256, use pv_n_tile) biondizzle 2026-05-23 06:41:42 +00:00
  • 97899b42a3 D1.2: simplify TMEM budget probe, fix printf args biondizzle 2026-05-23 06:40:55 +00:00
  • 640bae606e fix: cuda.CUstream import biondizzle 2026-05-23 06:40:05 +00:00
  • 595c9d677c D1.2: TMEM budget probe using @cute.jit for MLIR context biondizzle 2026-05-23 06:39:27 +00:00
  • 1a781447cb fix: remove unused v_fmha_layout from probe biondizzle 2026-05-23 06:38:08 +00:00
  • 3490601e02 D1.2: TMEM budget probe with real tensor major modes biondizzle 2026-05-23 06:37:34 +00:00
  • 8b351bd871 fix: OperandMajorMode.MN not .M biondizzle 2026-05-23 06:36:39 +00:00
  • 5b3bd4876c fix: typo + OperandMajorMode for TMEM budget probe biondizzle 2026-05-23 06:35:55 +00:00
  • 677245fc5c fix: LayoutEnum import from cutlass.utils biondizzle 2026-05-23 06:35:03 +00:00
  • bd968db5d4 fix import path for tcgen05 biondizzle 2026-05-23 06:34:30 +00:00
  • afec55ad72 D1.2: TMEM budget probe script for hd=64,128,256,512 biondizzle 2026-05-23 06:33:26 +00:00
  • 5012703bad fix: add SwiGLU clamping to fused kernel (paper §4.2.3, CG-1) biondizzle 2026-05-23 06:32:54 +00:00
  • 580d2f6999 STAGE_D.md: restructure with correctness gaps, TMEM budget, execution order biondizzle 2026-05-23 06:31:37 +00:00
  • df43c3232d D1.1: Fix make_fragment_A — use sP for SMEM source pv_mma biondizzle 2026-05-23 06:04:44 +00:00
  • 80434d0284 D1.1: Fix PV A-operand construction — compile-time branch for TMEM vs SMEM biondizzle 2026-05-23 06:03:27 +00:00