nvfp4-megamoe-kernel

Files

biondizzle 46a3a51832 CUDA graph: Fix per-step allocations in decode loop

1. mHCLayer.init_state: Add out_buf parameter for in-place write
   - Pre-allocated dec_X_buf (1, 4, 7168) on cuda:0
   - Eliminates .unsqueeze().expand().clone() allocation each step

2. single_shot_inference.py: Pre-allocate dec_embed_buf
   - Placeholder for embedding output (graph capture will use this)

3. Note: Cross-GPU X.to() transfers still allocate per step
   - This requires per-GPU X buffers (part of graph capture architecture)

2026-06-03 16:38:35 +00:00

_archive

Cleanup Step 2: Archive Lineage P code, fix broken imports

2026-06-02 19:27:07 +00:00

cache

Cleanup Step 2: Archive Lineage P code, fix broken imports

2026-06-02 19:27:07 +00:00

kernels

Fix compressor: do not add positional bias to KV content

2026-06-03 15:52:00 +00:00

layers

CUDA graph: Fix per-step allocations in decode loop