biondizzle/nvfp4-megamoe-kernel
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
99e143dd0ea800af79a15ff1016cbeb7950501f8
nvfp4-megamoe-kernel/tests
History
biondizzle 99e143dd0e Fix: add scale_softmax_log2, use O TMEM rescale for C9 normalization 
- scale_softmax_log2 was missing from _setup (patch artifact)
- C9 normalization: load O from TMEM, multiply by 1/row_sum, store back
  instead of trying to capture runtime value in const_expr lambda
- Then use standard epilogue_tma_store with identity transform
2026-05-21 17:15:15 +00:00
..
cudagraph_test.py
fix: test L2 weight N dim should be hidden_size, not hidden_size//2
2026-05-16 19:07:36 +00:00
debug_output.py
Update CURRENT_BUG.md: current status, outstanding garbage output issue, hypotheses
2026-05-17 16:52:40 +00:00
debug_stages.py
Stage B: two MMAs + identity softmax — crash fixed, softmax output still wrong
2026-05-20 20:26:25 +00:00
diag_layouts.py
Stage B: C-fragment vs A-fragment TMEM layout mismatch diagnosed
2026-05-21 00:12:47 +00:00
diag_tmem.py
Stage B: two MMAs + identity softmax — crash fixed, softmax output still wrong
2026-05-20 20:26:25 +00:00
layertest.py
Fused SwiGLU epilogue with granularity-8 weight interleave
2026-05-20 04:13:52 +00:00
requirements.txt
test: add standalone layer 0 comparison test (no vLLM, no Docker)
2026-05-16 02:13:18 +00:00
run_test.sh
fix: use setup.py install for CUTLASS extension build
2026-05-16 02:21:17 +00:00
stage_b_debug5.py
Stage B: two MMAs + identity softmax — crash fixed, softmax output still wrong
2026-05-20 20:26:25 +00:00
test_128_16_bigP.py
TMEM alias analysis: (128,16) PV broken, (128,128) PV with zero-pad works. Root cause: PV A-fragment layout differs from QK C-fragment layout for (128,16) PV, causing TMEM column mismatch. Using (128,128) PV as workaround.
2026-05-21 09:10:12 +00:00
test_128_16_debug2.py
Debugging TMEM alias for (128,16) PV: zero output confirmed, PV reads from wrong TMEM columns. Need to align softmax P write with PV A-fragment layout.
2026-05-21 09:00:42 +00:00
test_128_16_debug3.py
Debugging TMEM alias for (128,16) PV: zero output confirmed, PV reads from wrong TMEM columns. Need to align softmax P write with PV A-fragment layout.
2026-05-21 09:00:42 +00:00
test_128_16_debug.py
Debugging TMEM alias for (128,16) PV: zero output confirmed, PV reads from wrong TMEM columns. Need to align softmax P write with PV A-fragment layout.
2026-05-21 09:00:42 +00:00
test_128_16_fp16.py
Debugging TMEM alias for (128,16) PV: zero output confirmed, PV reads from wrong TMEM columns. Need to align softmax P write with PV A-fragment layout.
2026-05-21 09:00:42 +00:00
test_128_16_full.py
Stage B N-tiling: (128,16) PV MMA compiles and runs, cosine 0.36 (TMEM alias mismatch bug). FMHA head_dim=64 passes. Debugging TMEM layout alignment.
2026-05-21 08:45:49 +00:00
test_128_16_minimal.py
Stage B N-tiling: (128,16) PV MMA compiles and runs, cosine 0.36 (TMEM alias mismatch bug). FMHA head_dim=64 passes. Debugging TMEM layout alignment.
2026-05-21 08:45:49 +00:00
test_128_16_nogC.py
Stage B N-tiling: (128,16) PV MMA compiles and runs, cosine 0.36 (TMEM alias mismatch bug). FMHA head_dim=64 passes. Debugging TMEM layout alignment.
2026-05-21 08:45:49 +00:00
test_128_16_nopack.py
Stage B N-tiling: (128,16) PV MMA compiles and runs, cosine 0.36 (TMEM alias mismatch bug). FMHA head_dim=64 passes. Debugging TMEM layout alignment.
2026-05-21 08:45:49 +00:00
test_128_16_nosoftmax.py
Stage B N-tiling: (128,16) PV MMA compiles and runs, cosine 0.36 (TMEM alias mismatch bug). FMHA head_dim=64 passes. Debugging TMEM layout alignment.
2026-05-21 08:45:49 +00:00
test_128_16_pAtS.py
Debugging TMEM alias for (128,16) PV: zero output confirmed, PV reads from wrong TMEM columns. Need to align softmax P write with PV A-fragment layout.
2026-05-21 09:00:42 +00:00
test_128_16_pvlayout.py
TMEM alias analysis: (128,16) PV broken, (128,128) PV with zero-pad works. Root cause: PV A-fragment layout differs from QK C-fragment layout for (128,16) PV, causing TMEM column mismatch. Using (128,128) PV as workaround.
2026-05-21 09:10:12 +00:00
test_128_16_pvpack.py
Debugging TMEM alias for (128,16) PV: zero output confirmed, PV reads from wrong TMEM columns. Need to align softmax P write with PV A-fragment layout.
2026-05-21 09:00:42 +00:00
test_128_16_pvwrite.py
Debugging TMEM alias for (128,16) PV: zero output confirmed, PV reads from wrong TMEM columns. Need to align softmax P write with PV A-fragment layout.
2026-05-21 09:00:42 +00:00
test_128_16_qkread.py
Debugging TMEM alias for (128,16) PV: zero output confirmed, PV reads from wrong TMEM columns. Need to align softmax P write with PV A-fragment layout.
2026-05-21 09:00:42 +00:00
test_128_16_smem.py
Debugging TMEM alias for (128,16) PV: zero output confirmed, PV reads from wrong TMEM columns. Need to align softmax P write with PV A-fragment layout.
2026-05-21 09:00:42 +00:00
test_128_16_stepA.py
Stage B N-tiling: (128,16) PV MMA compiles and runs, cosine 0.36 (TMEM alias mismatch bug). FMHA head_dim=64 passes. Debugging TMEM layout alignment.
2026-05-21 08:45:49 +00:00
test_128_16_stepB.py
Stage B N-tiling: (128,16) PV MMA compiles and runs, cosine 0.36 (TMEM alias mismatch bug). FMHA head_dim=64 passes. Debugging TMEM layout alignment.
2026-05-21 08:45:49 +00:00
test_128_16_stepC.py
Stage B N-tiling: (128,16) PV MMA compiles and runs, cosine 0.36 (TMEM alias mismatch bug). FMHA head_dim=64 passes. Debugging TMEM layout alignment.
2026-05-21 08:45:49 +00:00
test_128_16_stepD.py
Stage B N-tiling: (128,16) PV MMA compiles and runs, cosine 0.36 (TMEM alias mismatch bug). FMHA head_dim=64 passes. Debugging TMEM layout alignment.
2026-05-21 08:45:49 +00:00
test_128_16_stepE.py
Stage B N-tiling: (128,16) PV MMA compiles and runs, cosine 0.36 (TMEM alias mismatch bug). FMHA head_dim=64 passes. Debugging TMEM layout alignment.
2026-05-21 08:45:49 +00:00
test_128_16_tiler.py
Stage B N-tiling: (128,16) PV MMA compiles and runs, cosine 0.36 (TMEM alias mismatch bug). FMHA head_dim=64 passes. Debugging TMEM layout alignment.
2026-05-21 08:45:49 +00:00
test_128_16_v8.py
Stage B N-tiling: (128,16) PV MMA compiles and runs, cosine 0.36 (TMEM alias mismatch bug). FMHA head_dim=64 passes. Debugging TMEM layout alignment.
2026-05-21 08:45:49 +00:00
test_128_16_zeropad.py
TMEM alias analysis: (128,16) PV broken, (128,128) PV with zero-pad works. Root cause: PV A-fragment layout differs from QK C-fragment layout for (128,16) PV, causing TMEM column mismatch. Using (128,128) PV as workaround.
2026-05-21 09:10:12 +00:00
test_128_32_ctafix2.py
Key finding: PV A-fragment layout is IDENTICAL for (128,128)/(128,32)/(128,16) PV. Bug is NOT TMEM alias. cta_tile_shape_mnk wrong for non-(128,128) PV. V SMEM and O C-fragment sizes look correct. Debugging V/epilogue paths.
2026-05-21 09:44:22 +00:00
test_128_32_ctafix.py
Key finding: PV A-fragment layout is IDENTICAL for (128,128)/(128,32)/(128,16) PV. Bug is NOT TMEM alias. cta_tile_shape_mnk wrong for non-(128,128) PV. V SMEM and O C-fragment sizes look correct. Debugging V/epilogue paths.
2026-05-21 09:44:22 +00:00
test_128_32_native.py
Key finding: PV A-fragment layout is IDENTICAL for (128,128)/(128,32)/(128,16) PV. Bug is NOT TMEM alias. cta_tile_shape_mnk wrong for non-(128,128) PV. V SMEM and O C-fragment sizes look correct. Debugging V/epilogue paths.
2026-05-21 09:44:22 +00:00
test_128_32_vdiag.py
Root cause FOUND: V SMEM only holds 1 K-tile (2048 BF16), but PV MMA iterates 8 K-phases. For non-(128,128) PV, most K-phases read wrong V data. Zero-padded V works because V is (128,128) covering all 8 K-phases. FMHA interleaves QK+PV per KV-tile to avoid this.
2026-05-21 09:56:54 +00:00
test_128_32_zeropad.py
Key finding: PV A-fragment layout is IDENTICAL for (128,128)/(128,32)/(128,16) PV. Bug is NOT TMEM alias. cta_tile_shape_mnk wrong for non-(128,128) PV. V SMEM and O C-fragment sizes look correct. Debugging V/epilogue paths.
2026-05-21 09:44:22 +00:00
test_128_128_fmha_v.py
Stage B Bug 4b debugging: P/A alias proven working, V layout issue for (128,64) PV
2026-05-21 15:20:14 +00:00
test_128_128_vdiag.py
Root cause FOUND: V SMEM only holds 1 K-tile (2048 BF16), but PV MMA iterates 8 K-phases. For non-(128,128) PV, most K-phases read wrong V data. Zero-padded V works because V is (128,128) covering all 8 K-phases. FMHA interleaves QK+PV per KV-tile to avoid this.
2026-05-21 09:56:54 +00:00
test_afrag_roundtrip.py
Stage B: C-fragment vs A-fragment TMEM layout mismatch diagnosed
2026-05-21 00:12:47 +00:00
test_attention_path_b200.py
Add attention path test: pinpoint FlashMLA failure
2026-05-19 07:54:01 +00:00
test_attention.py
Add NVFP4 linear runner + attention projection test
2026-05-18 20:14:03 +00:00
test_b_afrag2.py
Stage B: C-fragment vs A-fragment TMEM layout mismatch diagnosed
2026-05-21 00:12:47 +00:00
test_b_layout.py
cleanup: move useful tests to tests/, nuke stale debug tests
2026-05-16 02:14:37 +00:00
test_bf16_elemwise.py
Stage B: C-fragment vs A-fragment TMEM layout mismatch diagnosed
2026-05-21 00:12:47 +00:00
test_bf16_pack.py
Stage B: C-fragment vs A-fragment TMEM layout mismatch diagnosed
2026-05-21 00:12:47 +00:00
test_bf16_recast_full.py
Stage B: C-fragment vs A-fragment TMEM layout mismatch diagnosed
2026-05-21 00:12:47 +00:00
test_bf16_recast_simple.py
Stage B: C-fragment vs A-fragment TMEM layout mismatch diagnosed
2026-05-21 00:12:47 +00:00
test_blackwell_attn_b200.py
Add blackwell_attention module and comprehensive test
2026-05-19 15:30:29 +00:00
test_compile_custom_op.py
Fix compile test: add warmup for activation global scales
2026-05-19 01:57:16 +00:00
test_csa_attention_b200.py
Add CSA/HCA attention kernel (PyTorch SDPA, Blackwell-safe)
2026-05-19 07:58:10 +00:00
test_csa_sparse_attn_b200.py
Fix N for C128A (need 128 tokens)
2026-05-19 16:04:53 +00:00
test_custom_op.py
Replace autograd.Function with torch.library.custom_op for Dynamo compat
2026-05-19 01:54:48 +00:00
test_cutedsl.py
fix: B tensor K-major strides, scale_b axis swap
2026-05-16 03:04:31 +00:00
test_decode_attention_b200.py
Fix attention for decode (1 query vs N cached KVs)
2026-05-19 15:28:52 +00:00
test_decode_pipeline.py
feat: GPU-native SWA + sparse decode attention kernels (CuTeDSL)
2026-05-20 05:46:15 +00:00
test_decode_vs_prefill_b200.py
Add decode vs prefill consistency test
2026-05-19 16:00:33 +00:00
test_diag_layout.py
more stuff
2026-05-21 05:08:57 +00:00
test_diag_permute.py
more stuff
2026-05-21 05:08:57 +00:00
test_diag_smem_layout.py
more stuff
2026-05-21 05:08:57 +00:00
test_diag_v_mma128.py
more stuff
2026-05-21 05:08:57 +00:00
test_diag_v_ones.py
more stuff
2026-05-21 05:08:57 +00:00
test_diag_v_truncid.py
more stuff
2026-05-21 05:08:57 +00:00
test_e2e_decode_b200.py
Test with all 61 layers (shared experts only)
2026-05-19 15:55:41 +00:00
test_error_pattern.py
Stage B: C-fragment vs A-fragment TMEM layout mismatch diagnosed
2026-05-21 00:12:47 +00:00
test_fmha_pipeline.py
FMHA v1: pv_mma_tiler=(128,64,128) works with V=I, fails with real V (SMEM layout bug)
2026-05-21 10:47:46 +00:00
test_fmha_v1.py
stuff and stuff
2026-05-21 10:50:30 +00:00
test_fmha_v2_fixed.py
Stage B: PV(128,64) test + v2 pipeline fixes
2026-05-21 11:49:06 +00:00
test_fmha_v2.py
FMHA v1: pv_mma_tiler=(128,64,128) works with V=I, fails with real V (SMEM layout bug)
2026-05-21 10:47:46 +00:00
test_fmha_v3_debug.py
Stage B Bug 4b debugging: P/A alias proven working, V layout issue for (128,64) PV
2026-05-21 15:20:14 +00:00
test_fmha_v3_softmax.py
Fix: add scale_softmax_log2, use O TMEM rescale for C9 normalization
2026-05-21 17:15:15 +00:00
test_fmha_v3.py
STAGE B BUG 4b FIXED: TMEM P/O overlap + FMHA V reconstruction
2026-05-21 15:30:24 +00:00
test_fp4_roundtrip.py
chore: deprecate prepare_weights_from_dequantized and prepare_weights_direct
2026-05-20 02:11:40 +00:00
test_full_layer_b200.py
Fix checkpoint keys: attn_hc.*, compressor.*, q_a_proj/q_b_proj/kv_proj
2026-05-19 07:17:37 +00:00
test_full_layer_nan_b200.py
Add full layer NaN test (attention + MoE, multi-layer chain)
2026-05-19 18:36:49 +00:00
test_full_model_b200.py
Add full model forward test (WIP), sparse attention test passes
2026-05-19 09:04:19 +00:00
test_fused_step1.py
wip: add run_fused_swiglu_grouped_gemm bridge + step1 test
2026-05-20 03:10:56 +00:00
test_inspect_types.py
Stage B progress: PV works for square (128,128), broken for (128,64)
2026-05-21 04:40:28 +00:00
test_interleave_gemm.py
wip: fused SwiGLU kernel scaffold + bridge interleave + plan
2026-05-20 03:04:38 +00:00
test_interleave.py
wip: fused SwiGLU kernel scaffold + bridge interleave + plan
2026-05-20 03:04:38 +00:00
test_inv_rope.py
Add unit tests for NVFP4 weight mapper and inverse RoPE BF16
2026-05-19 03:22:00 +00:00
test_kv_cache_b200.py
Fix kv_ref transpose in KV cache test
2026-05-19 08:58:46 +00:00
test_layout_compare.py
Stage B progress: PV works for square (128,128), broken for (128,64)
2026-05-21 04:40:28 +00:00
test_mma_si_only.py
Stage B: pipeline deadlock fixed, V MN-major applied, PV output garbage
2026-05-21 04:10:07 +00:00
test_mma_si_pv.py
Stage B progress: PV works for square (128,128), broken for (128,64)
2026-05-21 04:40:28 +00:00
test_model_forward_b200.py
Rewrite test: diagnose whether warmup gs matters at inference time
2026-05-19 07:49:41 +00:00
test_moe_nan_b200.py
Fix intermediate size: 3072 not 18432
2026-05-19 18:34:12 +00:00
test_moe_runner_nan_b200.py
Use 16 experts for MoE runner test (fits in memory)
2026-05-19 18:35:40 +00:00
test_multilayer.py
Add MoE scale ratio output
2026-05-17 22:58:27 +00:00
test_nvfp4_attention_b200.py
Fix cos_sin cache shape in NVFP4 attention test
2026-05-19 08:38:55 +00:00
test_nvfp4_attn_gemm_b200.py
Fix NVFP4 attention: slice output to actual N after 128-padding
2026-05-19 08:55:31 +00:00
test_nvfp4_mapper.py
Fix hc_head mapping: checkpoint uses hc_head.hc_fn, model params are flat hc_head_fn
2026-05-19 03:58:25 +00:00
test_o_projection_b200.py
Fix dims: o_groups=16, o_lora_rank=1024 from config
2026-05-19 06:37:25 +00:00
test_o_projection.py
Patch attention forward: BF16 inv RoPE + BMM wo_a + NVFP4 wo_b
2026-05-19 06:30:18 +00:00
test_packing_diag.py
Stage B: C-fragment vs A-fragment TMEM layout mismatch diagnosed
2026-05-21 00:12:47 +00:00
test_pair_swap2.py
Stage B: C-fragment vs A-fragment TMEM layout mismatch diagnosed
2026-05-21 00:12:47 +00:00
test_pair_swap.py
Stage B: C-fragment vs A-fragment TMEM layout mismatch diagnosed
2026-05-21 00:12:47 +00:00
test_pipeline_real_weights.py
Pipeline test: use max_num_tokens=8192 matching vLLM
2026-05-17 23:04:44 +00:00
test_pv64_fmha_v.py
Stage B Bug 4b debugging: P/A alias proven working, V layout issue for (128,64) PV
2026-05-21 15:20:14 +00:00
test_pv64_kmajor_v.py
Stage B Bug 4b debugging: P/A alias proven working, V layout issue for (128,64) PV
2026-05-21 15:20:14 +00:00
test_pv64_no_softmax.py
Stage B Bug 4b debugging: P/A alias proven working, V layout issue for (128,64) PV
2026-05-21 15:20:14 +00:00
test_pv64_nosoftmax_fmha_v.py
Stage B Bug 4b debugging: P/A alias proven working, V layout issue for (128,64) PV
2026-05-21 15:20:14 +00:00
test_pv64_with_softmax.py
Fix TMEM overlap in test_pv64_with_softmax.py too — cosine 0.999999
2026-05-21 15:32:49 +00:00
test_pv64.py
Stage B: PV(128,64) test + v2 pipeline fixes
2026-05-21 11:49:06 +00:00
test_pv_diag.py
FOOTGUN #0: num_tma_load_bytes MUST include V bytes. Fix v27, v29, comment all. Update README.
2026-05-21 07:13:14 +00:00
test_pv_mma_mn_major.py
Stage B: pipeline deadlock fixed, V MN-major applied, PV output garbage
2026-05-21 04:10:07 +00:00
test_qk_softmax.py
FMHA v3: KV-tile interleaving pipeline - QK works, Bug 4b blocks PV
2026-05-21 12:52:29 +00:00
test_qkonly.py
FMHA v3: KV-tile interleaving pipeline - QK works, Bug 4b blocks PV
2026-05-21 12:52:29 +00:00
test_quick_rand.py
cleanup: move useful tests to tests/, nuke stale debug tests
2026-05-16 02:14:37 +00:00
test_recast_minimal.py
Stage B: C-fragment vs A-fragment TMEM layout mismatch diagnosed
2026-05-21 00:12:47 +00:00
test_rope_kv_b200.py
Fix syntax in RoPE KV test
2026-05-19 10:31:07 +00:00
test_runner_vs_pipeline.py
test: runner vs pipeline comparison + scale assembly comparison
2026-05-17 07:33:20 +00:00
test_scale_assembly.py
fix: separate L1/L2 scale buffers (different K_sf), fix assembly calls
2026-05-17 07:43:05 +00:00
test_scale_debug.py
test: scale assembly debug
2026-05-17 07:37:47 +00:00
test_shared_expert.py
Fix hidden_size: shared expert uses 7168, not HC_DIM 28672
2026-05-18 20:10:32 +00:00
test_silu_step1.py
wip: add run_fused_swiglu_grouped_gemm bridge + step1 test
2026-05-20 03:10:56 +00:00
test_softmax_only.py
Stage B: pipeline deadlock fixed, V MN-major applied, PV output garbage
2026-05-21 04:10:07 +00:00
test_softmax_store_debug.py
Stage B Bug 4b debugging: P/A alias proven working, V layout issue for (128,64) PV
2026-05-21 15:20:14 +00:00
test_sparse_attn_b200.py
Add CSA/HCA sparse attention kernel test
2026-05-19 09:02:12 +00:00
test_sparse_decode.py
feat: GPU-native SWA + sparse decode attention kernels (CuTeDSL)
2026-05-20 05:46:15 +00:00
test_stage_a_copy.py
Stage B: two MMAs + identity softmax — crash fixed, softmax output still wrong
2026-05-20 20:26:25 +00:00
test_stage_a_minimal.py
Stage B: two MMAs + identity softmax — crash fixed, softmax output still wrong
2026-05-20 20:26:25 +00:00
test_stage_a_pv_created.py
Stage B: two MMAs + identity softmax — crash fixed, softmax output still wrong
2026-05-20 20:26:25 +00:00
test_stage_a_pv_param.py
Stage B: two MMAs + identity softmax — crash fixed, softmax output still wrong
2026-05-20 20:26:25 +00:00
test_stage_a_qk.py
Stage B: two MMAs + identity softmax — crash fixed, softmax output still wrong
2026-05-20 20:26:25 +00:00
test_stage_a_v2.py
Stage B: two MMAs + identity softmax — crash fixed, softmax output still wrong
2026-05-20 20:26:25 +00:00
test_stage_a_with_pv_mma.py
Stage B: two MMAs + identity softmax — crash fixed, softmax output still wrong
2026-05-20 20:26:25 +00:00
test_stage_b_afrag2.py
Stage B: C-fragment vs A-fragment TMEM layout mismatch diagnosed
2026-05-21 00:12:47 +00:00
test_stage_b_afrag.py
Stage B: C-fragment vs A-fragment TMEM layout mismatch diagnosed
2026-05-21 00:12:47 +00:00
test_stage_b_debug2.py
Stage B: two MMAs + identity softmax — crash fixed, softmax output still wrong
2026-05-20 20:26:25 +00:00
test_stage_b_debug3.py
Stage B: two MMAs + identity softmax — crash fixed, softmax output still wrong
2026-05-20 20:26:25 +00:00
test_stage_b_debug4.py
Stage B: two MMAs + identity softmax — crash fixed, softmax output still wrong
2026-05-20 20:26:25 +00:00
test_stage_b_debug.py
Stage B: two MMAs + identity softmax — crash fixed, softmax output still wrong
2026-05-20 20:26:25 +00:00
test_stage_b_diag.py
Stage B: C-fragment vs A-fragment TMEM layout mismatch diagnosed
2026-05-21 00:12:47 +00:00
test_stage_b_final.py
Stage B: C-fragment vs A-fragment TMEM layout mismatch diagnosed
2026-05-21 00:12:47 +00:00
test_stage_b_identity.py
Stage B: two MMAs + identity softmax — crash fixed, softmax output still wrong
2026-05-20 20:26:25 +00:00
test_stage_b_minimal.py
Stage B: two MMAs + identity softmax — crash fixed, softmax output still wrong
2026-05-20 20:26:25 +00:00
test_stage_b_ntile_v1.py
Stage B N-tiling: (128,16) PV MMA compiles and runs, cosine 0.36 (TMEM alias mismatch bug). FMHA head_dim=64 passes. Debugging TMEM layout alignment.
2026-05-21 08:45:49 +00:00
test_stage_b_ntile_v3.py
Stage B N-tiling: (128,16) PV MMA compiles and runs, cosine 0.36 (TMEM alias mismatch bug). FMHA head_dim=64 passes. Debugging TMEM layout alignment.
2026-05-21 08:45:49 +00:00
test_stage_b_ntile_v5.py
Stage B N-tiling: (128,16) PV MMA compiles and runs, cosine 0.36 (TMEM alias mismatch bug). FMHA head_dim=64 passes. Debugging TMEM layout alignment.
2026-05-21 08:45:49 +00:00
test_stage_b_ntile_v6.py
Stage B N-tiling: (128,16) PV MMA compiles and runs, cosine 0.36 (TMEM alias mismatch bug). FMHA head_dim=64 passes. Debugging TMEM layout alignment.
2026-05-21 08:45:49 +00:00
test_stage_b_ntile_v7.py
Stage B N-tiling: (128,16) PV MMA compiles and runs, cosine 0.36 (TMEM alias mismatch bug). FMHA head_dim=64 passes. Debugging TMEM layout alignment.
2026-05-21 08:45:49 +00:00
test_stage_b_ntile_v8.py
Stage B N-tiling: (128,16) PV MMA compiles and runs, cosine 0.36 (TMEM alias mismatch bug). FMHA head_dim=64 passes. Debugging TMEM layout alignment.
2026-05-21 08:45:49 +00:00
test_stage_b_pipeline_only.py
Stage B: two MMAs + identity softmax — crash fixed, softmax output still wrong
2026-05-20 20:26:25 +00:00
test_stage_b_v1.py
Stage B: two MMAs + identity softmax — crash fixed, softmax output still wrong
2026-05-20 20:26:25 +00:00
test_stage_b_v2.py
Stage B: two MMAs + identity softmax — crash fixed, softmax output still wrong
2026-05-20 20:26:25 +00:00
test_stage_b_v3.py
Stage B: two MMAs + identity softmax — crash fixed, softmax output still wrong
2026-05-20 20:26:25 +00:00
test_stage_b_v4.py
Stage B: two MMAs + identity softmax — crash fixed, softmax output still wrong
2026-05-20 20:26:25 +00:00
test_stage_b_v5.py
Stage B: two MMAs + identity softmax — crash fixed, softmax output still wrong
2026-05-20 20:26:25 +00:00
test_stage_b_v6.py
Stage B: two MMAs + identity softmax — crash fixed, softmax output still wrong
2026-05-20 20:26:25 +00:00
test_stage_b_v7_rep8.py
Stage B: C-fragment vs A-fragment TMEM layout mismatch diagnosed
2026-05-21 00:12:47 +00:00
test_stage_b_v7_rep16.py
Stage B: C-fragment vs A-fragment TMEM layout mismatch diagnosed
2026-05-21 00:12:47 +00:00
test_stage_b_v7_rep64.py
Stage B: C-fragment vs A-fragment TMEM layout mismatch diagnosed
2026-05-21 00:12:47 +00:00
test_stage_b_v7_rep128.py
Stage B: C-fragment vs A-fragment TMEM layout mismatch diagnosed
2026-05-21 00:12:47 +00:00
test_stage_b_v7.py
Stage B: C-fragment vs A-fragment TMEM layout mismatch diagnosed
2026-05-21 00:12:47 +00:00
test_stage_b_v8.py
Stage B: C-fragment vs A-fragment TMEM layout mismatch diagnosed
2026-05-21 00:12:47 +00:00
test_stage_b_v8b.py
Stage B: C-fragment vs A-fragment TMEM layout mismatch diagnosed
2026-05-21 00:12:47 +00:00
test_stage_b_v9.py
Stage B: C-fragment vs A-fragment TMEM layout mismatch diagnosed
2026-05-21 00:12:47 +00:00
test_stage_b_v10.py
Stage B: C-fragment vs A-fragment TMEM layout mismatch diagnosed
2026-05-21 00:12:47 +00:00
test_stage_b_v11.py
Stage B: C-fragment vs A-fragment TMEM layout mismatch diagnosed
2026-05-21 00:12:47 +00:00
test_stage_b_v11b.py
Stage B: C-fragment vs A-fragment TMEM layout mismatch diagnosed
2026-05-21 00:12:47 +00:00
test_stage_b_v12.py
Stage B: C-fragment vs A-fragment TMEM layout mismatch diagnosed
2026-05-21 00:12:47 +00:00
test_stage_b_v13.py
Stage B: pipeline deadlock fixed, V MN-major applied, PV output garbage
2026-05-21 04:10:07 +00:00
test_stage_b_v14.py
Stage B: pipeline deadlock fixed, V MN-major applied, PV output garbage
2026-05-21 04:10:07 +00:00
test_stage_b_v16.py
Stage B: pipeline deadlock fixed, V MN-major applied, PV output garbage
2026-05-21 04:10:07 +00:00
test_stage_b_v17.py
Stage B: pipeline deadlock fixed, V MN-major applied, PV output garbage
2026-05-21 04:10:07 +00:00
test_stage_b_v18.py
Stage B: pipeline deadlock fixed, V MN-major applied, PV output garbage
2026-05-21 04:10:07 +00:00
test_stage_b_v19.py
Stage B: pipeline deadlock fixed, V MN-major applied, PV output garbage
2026-05-21 04:10:07 +00:00
test_stage_b_v20.py
Stage B: pipeline deadlock fixed, V MN-major applied, PV output garbage
2026-05-21 04:10:07 +00:00
test_stage_b_v22_bug1fix.py
Stage B: pipeline deadlock fixed, V MN-major applied, PV output garbage
2026-05-21 04:10:07 +00:00
test_stage_b_v22.py
Stage B: pipeline deadlock fixed, V MN-major applied, PV output garbage
2026-05-21 04:10:07 +00:00
test_stage_b_v23.py
Stage B: pipeline deadlock fixed, V MN-major applied, PV output garbage
2026-05-21 04:10:07 +00:00
test_stage_b_v24.py
Stage B: pipeline deadlock fixed, V MN-major applied, PV output garbage
2026-05-21 04:10:07 +00:00
test_stage_b_v25.py
Stage B: pipeline deadlock fixed, V MN-major applied, PV output garbage
2026-05-21 04:10:07 +00:00
test_stage_b_v26.py
Stage B: pipeline deadlock fixed, V MN-major applied, PV output garbage
2026-05-21 04:10:07 +00:00
test_stage_b_v27.py
FOOTGUN #0: num_tma_load_bytes MUST include V bytes. Fix v27, v29, comment all. Update README.
2026-05-21 07:13:14 +00:00
test_stage_b_v28.py
v28 attempt: PV MMA (128,64) - cosine 0.004, debugging
2026-05-21 05:41:44 +00:00
test_stage_b_v29.py
FOOTGUN #0: num_tma_load_bytes MUST include V bytes. Fix v27, v29, comment all. Update README.
2026-05-21 07:13:14 +00:00
test_stage_b_v30.py
v29 (padded V, deadlocks), v30 (diag copy, works) — debugging epilogue deadlock with (128,128) PV
2026-05-21 06:40:27 +00:00
test_step2_subtile_v2.py
wip: Step 2 gate/up pairing — SiLU validated, runtime conditionals blocked by CuTeDSL
2026-05-20 03:26:20 +00:00
test_step2_subtile.py
wip: Step 2 gate/up pairing — SiLU validated, runtime conditionals blocked by CuTeDSL
2026-05-20 03:26:20 +00:00
test_store_verify2.py
Stage B: C-fragment vs A-fragment TMEM layout mismatch diagnosed
2026-05-21 00:12:47 +00:00
test_store_verify.py
Stage B: C-fragment vs A-fragment TMEM layout mismatch diagnosed
2026-05-21 00:12:47 +00:00
test_tmem_addressing.py
Stage B: two MMAs + identity softmax — crash fixed, softmax output still wrong
2026-05-20 20:26:25 +00:00
test_tmem_col2.py
Key finding: PV A-fragment layout is IDENTICAL for (128,128)/(128,32)/(128,16) PV. Bug is NOT TMEM alias. cta_tile_shape_mnk wrong for non-(128,128) PV. V SMEM and O C-fragment sizes look correct. Debugging V/epilogue paths.
2026-05-21 09:44:22 +00:00
test_tmem_col3.py
Key finding: PV A-fragment layout is IDENTICAL for (128,128)/(128,32)/(128,16) PV. Bug is NOT TMEM alias. cta_tile_shape_mnk wrong for non-(128,128) PV. V SMEM and O C-fragment sizes look correct. Debugging V/epilogue paths.
2026-05-21 09:44:22 +00:00
test_tmem_col4.py
Key finding: PV A-fragment layout is IDENTICAL for (128,128)/(128,32)/(128,16) PV. Bug is NOT TMEM alias. cta_tile_shape_mnk wrong for non-(128,128) PV. V SMEM and O C-fragment sizes look correct. Debugging V/epilogue paths.
2026-05-21 09:44:22 +00:00
test_tmem_col5_16.py
Key finding: PV A-fragment layout is IDENTICAL for (128,128)/(128,32)/(128,16) PV. Bug is NOT TMEM alias. cta_tile_shape_mnk wrong for non-(128,128) PV. V SMEM and O C-fragment sizes look correct. Debugging V/epilogue paths.
2026-05-21 09:44:22 +00:00
test_tmem_col5_32.py
Key finding: PV A-fragment layout is IDENTICAL for (128,128)/(128,32)/(128,16) PV. Bug is NOT TMEM alias. cta_tile_shape_mnk wrong for non-(128,128) PV. V SMEM and O C-fragment sizes look correct. Debugging V/epilogue paths.
2026-05-21 09:44:22 +00:00
test_tmem_col5.py
Key finding: PV A-fragment layout is IDENTICAL for (128,128)/(128,32)/(128,16) PV. Bug is NOT TMEM alias. cta_tile_shape_mnk wrong for non-(128,128) PV. V SMEM and O C-fragment sizes look correct. Debugging V/epilogue paths.
2026-05-21 09:44:22 +00:00
test_tmem_col_offset.py
Key finding: PV A-fragment layout is IDENTICAL for (128,128)/(128,32)/(128,16) PV. Bug is NOT TMEM alias. cta_tile_shape_mnk wrong for non-(128,128) PV. V SMEM and O C-fragment sizes look correct. Debugging V/epilogue paths.
2026-05-21 09:44:22 +00:00
test_tmem_copy_roundtrip.py
Stage B: C-fragment vs A-fragment TMEM layout mismatch diagnosed
2026-05-21 00:12:47 +00:00
test_tmem_debug2.py
TMEM alias analysis: (128,16) PV broken, (128,128) PV with zero-pad works. Root cause: PV A-fragment layout differs from QK C-fragment layout for (128,16) PV, causing TMEM column mismatch. Using (128,128) PV as workaround.
2026-05-21 09:10:12 +00:00
test_tmem_debug.py
TMEM alias analysis: (128,16) PV broken, (128,128) PV with zero-pad works. Root cause: PV A-fragment layout differs from QK C-fragment layout for (128,16) PV, causing TMEM column mismatch. Using (128,128) PV as workaround.
2026-05-21 09:10:12 +00:00
test_tmem_fp32_roundtrip.py
Stage B: C-fragment vs A-fragment TMEM layout mismatch diagnosed
2026-05-21 00:12:47 +00:00
test_tmem_layout_diag.py
Update README: Bug 4 status, (128,16) PV zero output, (128,128) PV zero-pad workaround (cosine 1.0)
2026-05-21 09:20:09 +00:00
test_tmem_pure_fp32.py
Stage B: C-fragment vs A-fragment TMEM layout mismatch diagnosed
2026-05-21 00:12:47 +00:00
test_uniform_fp4.py
cleanup: move useful tests to tests/, nuke stale debug tests
2026-05-16 02:14:37 +00:00
test_v4_attention_b200.py
Add DeepSeek-V4 CSA/HCA attention pipeline test (not MLA)
2026-05-19 08:51:16 +00:00
test_v28c_noepi.py
v29 (padded V, deadlocks), v30 (diag copy, works) — debugging epilogue deadlock with (128,128) PV
2026-05-21 06:40:27 +00:00
test_v_mode_fix.py
Stage B Bug 4b debugging: P/A alias proven working, V layout issue for (128,64) PV
2026-05-21 15:20:14 +00:00
test_vllm_codepaths_b200.py
Fix imports in vLLM codepaths test
2026-05-19 17:26:50 +00:00
test_vsmem_diag.py
Root cause FOUND: V SMEM only holds 1 K-tile (2048 BF16), but PV MMA iterates 8 K-phases. For non-(128,128) PV, most K-phases read wrong V data. Zero-padded V works because V is (128,128) covering all 8 K-phases. FMHA interleaves QK+PV per KV-tile to avoid this.
2026-05-21 09:56:54 +00:00
test_warmup_gs.py
test: use runner's built-in warmup method
2026-05-17 08:24:27 +00:00
test_wo_a_bmm.py
Fix BF16 wo_a: per-group BMM instead of flat linear
2026-05-19 04:10:02 +00:00
test_wo_a.py
Fix test: cos_sin_cache on CUDA device
2026-05-19 02:37:50 +00:00