nvfp4-megamoe-kernel

Files

biondizzle e0607c9e2f P0: Add fused_amax_quantize.cu kernel + CUDA module loader with compile-once caching

- fused_amax_quantize.cu: Single kernel launch computes amax → gsa → NVFP4 quantize
  Zero CPU-GPU syncs. gsa written to GPU buffer for downstream GEMM global_scale_a.
- dsv4/kernels/cuda/__init__.py: Module loader that compiles .cu once and caches.
  Eliminates JIT recompilation overhead (was ~100ms per call, ~500x per token).
- P1 audit corrected: layer-pipe at batch=1 is wrong, but single-GPU doesn't fit
  (800GB weights vs 192GB HBM). Correct fix is EP=8 for MoE + TP/replicate for dense.

2026-06-01 21:02:03 +00:00

attention

FMHA sink: don't double-scale sink bias

2026-05-31 23:12:20 +00:00

cache

fix: correct gather.py kernel_dir path

2026-05-30 21:12:09 +00:00

compressor

fix: import torch.utils.cpp_extension explicitly in production_compress

2026-06-01 05:20:44 +00:00

cuda

P0: Add fused_amax_quantize.cu kernel + CUDA module loader with compile-once caching