nvfp4-megamoe-kernel

Files

biondizzle 5012703bad fix: add SwiGLU clamping to fused kernel (paper §4.2.3, CG-1)

The fused SwiGLU kernel stored swiglu_limit but never applied it.
Paper §4.2.3: gate capped at swiglu_limit, linear clamped to [-limit, +limit].
Non-fused reference path already applies clamping correctly.
Fix: add fmin/fmax clamping in FP32 before BF16 conversion.

2026-05-23 06:32:54 +00:00

__init__.py

Restructure: cutedsl/ -> dsv4/ with proper layering

2026-05-21 17:30:44 +00:00

dense.py

Restructure: cutedsl/ -> dsv4/ with proper layering

2026-05-21 17:30:44 +00:00

fused_swiglu.py

fix: add SwiGLU clamping to fused kernel (paper §4.2.3, CG-1)

2026-05-23 06:32:54 +00:00

grouped.py

Restructure: cutedsl/ -> dsv4/ with proper layering