fix: cutlass.Float32 not cutlass.float32_t in fused epilogue

Step 1 SiLU validation: PASS
- cute.exp works on register tensors
- SiLU (x / (1+exp(-x))) in registers matches PyTorch reference
- Relative error: 0.034%, Max abs error: 0.0625 (BF16 precision limit)
This commit is contained in:
2026-05-20 03:12:23 +00:00
parent 08992b818d
commit b84f2f7bf9

View File

@@ -2156,9 +2156,9 @@ class FusedSwiGLUScaledGroupedGemmKernel:
# With interleaved weights (granularity 8 BF16 = 4 FP4),
# the accumulator N dimension has gate/up pairs adjacent.
# SiLU(x) = x * sigmoid(x) = x / (1 + exp(-x))
neg_acc = acc_vec * cutlass.float32_t(-1.0)
neg_acc = acc_vec * cutlass.Float32(-1.0)
exp_neg = cute.exp(neg_acc)
sigmoid = cutlass.float32_t(1.0) / (cutlass.float32_t(1.0) + exp_neg)
sigmoid = cutlass.Float32(1.0) / (cutlass.Float32(1.0) + exp_neg)
swiglu_result = acc_vec * sigmoid
# Write SwiGLU result as BF16 to C tensor