Unlike the naive upcast, this properly dequantizes FP8 block-wise weights:
bf16 = fp8_weight * scale_expanded (128x128 blocks).
Also removes the now-unnecessary scale tensors and updates config.
FP8Linear.forward() sees element_size() > 1 and falls back to F.linear().
Quantizes only MoE expert weights to NVFP4, leaving attention untouched.
Includes comments documenting all available NVFP4 strategies.
Copy to model_opt_nvfp4_<strategy>.py for each new strategy.
- scripts/upcast_to_bf16.py: Converts mixed-precision V4 Pro to pure BF16
by upcasting all FP8 tensors (float8_e8m0fnu etc.) to bfloat16.
Needed because modelopt PTQ calibration crashes on Blackwell with FP8
kernels (DeepGEMM unsupported, Triton finegrained-fp8 has K mismatches).
- patches/patch_finegrained_fp8_blackwell.py: Patches transformers to
reject DeepGEMM on SM100+ (Blackwell), letting it fall back to Triton.
Note: the Triton fallback also fails during modelopt calibration on
quantized weights, so upcasting to BF16 is the working solution.
- Patch fixes iter_weights_for_calibration() for DeepseekV4Experts
(ModuleList quantizers vs singular)
- Run script uses official NVIDIA hf_ptq.py with FP8 source
- Documents flags to avoid (--low_memory_mode, wrong arg names)
