biondizzle/nvfp4-megamoe-kernel
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
71 Commits 1 Branch 17 Tags
5385de314237a0c7f5eb5ea05beac6e7a88d055f
Commit Graph

7 Commits

Author SHA1 Message Date
biondizzle
5385de3142 fix: layertest tests L1 GEMM only with correct output size 
L1 produces (tokens, 6144) gate+up, not (tokens, 7168) hidden.
Compare against BF16 L1 reference only.
 2026-05-16 03:15:29 +00:00
biondizzle
0cdcc4144a refactor: add cutedsl/bridge.py, rewrite layertest to use it 
bridge.py: clean API for CuTeDSL kernel
- quantize_to_nvfp4 / quantize_weight_to_nvfp4
- assemble_scales_2d_side / assemble_scales_3d_side
- make_b_k_major (stride conversion)
- compute_expert_offsets
- run_nvfp4_grouped_gemm (full kernel launch)

layertest.py: now uses bridge layer, tests with real
DeepSeek-V4 layer 0 weights (7168 hidden, 6144 intermediate).

The bridge code will be reused by the vLLM integration layer.
 2026-05-16 03:13:54 +00:00
biondizzle
c4a262bd54 test: streamline layertest — kernel vs BF16 ref only, exit on fail 
Removed original checkpoint loading (already verified 0.997 cosine).
Test now: load NVFP4 → dequant BF16 ref → run kernel → compare.
Exits with code 1 if cosine < 0.99.
 2026-05-16 02:29:41 +00:00
biondizzle
55d9a24bf6 fix: handle model. prefix normalization in checkpoint keys 2026-05-16 02:18:52 +00:00
biondizzle
bdf9f31ae2 fix: checkpoint keys don't have 'model.' prefix 2026-05-16 02:17:13 +00:00
biondizzle
ea5ee7c1f7 fix: remove prefix_filter from layer tensor loading 2026-05-16 02:15:55 +00:00
biondizzle
2114bd11be test: add standalone layer 0 comparison test (no vLLM, no Docker) 
tests/layertest.py:
- Loads layer 0 expert weights from both original (MXFP4) and NVFP4 checkpoints
- Dequantizes both to BF16 for reference comparison
- Runs MoE forward pass in pure BF16 (no kernel)
- Runs same forward pass through our NVFP4 CUTLASS kernel
- Compares cosine similarity: kernel vs BF16 reference

tests/run_test.sh:
- Creates venv, installs deps, builds kernel from source, runs test

Isolates our kernel completely from vLLM's weight loading, tensor
parallelism, and MoE routing. If cosine ≈ 1.0, bug is in vLLM. If
cosine ≈ 0, bug is in our kernel pipeline.
 2026-05-16 02:13:18 +00:00