Files
nvfp4-megamoe-kernel/dsv4/kernels
biondizzle ffa7842b58 Fix dense router: run GEMM in BF16, convert to FP32 only for activation
hidden_states.float() and gate_bf16.T.float() create new FP32 tensors
during CUDA graph capture, which is not graph-capturable.

Fix: run the linear in BF16 (Blackwell tensor cores handle BF16 natively),
then convert only the output logits to FP32 for numerical stability
in sqrt(softplus). The single logits.float() is graph-capturable
because it's a unary op with a pre-existing output buffer.
2026-06-04 04:49:08 +00:00
..