biondizzle e3c24769e2 Handle wo_a as bfloat16 (unquantized in NVFP4 checkpoint) ...

o_a_proj is NOT quantized by modelopt in the checkpoint (bfloat16),
but the attention forward pass expects FP8 (weight + weight_scale_inv).

- Create wo_a with quant_config=None to load bfloat16 weights
- Add FP8 quantization of wo_a in finalize_mega_moe_weights:
  per-tensor symmetric quantization to float8_e4m3fn + weight_scale_inv
- This matches what the fused_inv_rope_fp8_quant + einsum expects

2026-05-18 23:41:39 +00:00

..

patches

Handle wo_a as bfloat16 (unquantized in NVFP4 checkpoint)

2026-05-18 23:41:39 +00:00

cutedsl_quant_method.py

Fix: add abstract create_weights to CuTeDSLNvfp4LinearMethod

2026-05-18 20:40:48 +00:00

nvfp4_cutedsl.py

Fix torch.compile: use custom autograd Function instead of @torch.compiler.disable

2026-05-18 21:38:28 +00:00