Commit Graph

1 Commits

Author SHA1 Message Date
9b0858aa35 Add single_shot_inference.py — baseline kernel verification
Streams weights one layer at a time from 95 safetensors shards.
NVFP4 dequant → BF16 matmul for baseline (production uses tcgen05 MMA).
Runs token-by-token decode loop with production FMHA kernel.

Known gaps for first run:
- FFN (MoE) skipped — not the kernel under test
- mHC simplified — not the kernel under test
- RoPE skipped in baseline
- compressor/indexer bypassed (raw KV for now)

FMHA kernel is the component under test (cos ≥ 0.999993).
2026-05-30 22:39:01 +00:00