diff --git a/README.md b/README.md index 6c4ce24..5c0d618 100644 --- a/README.md +++ b/README.md @@ -68,7 +68,7 @@ The `quant_module_patched.py` fix is for **modelopt 0.45.0.dev64** specifically. ## Key Notes -- Use BF16 source — V4's mixed precision causes issues, FP8 source has kernel problems on Blackwell +- V4 is NOT BF16 — it ships as mixed-precision FP8/FP4. You MUST dequantize to BF16 first (Step 1). The raw FP8 source has kernel problems on Blackwell; the mixed-precision source causes modelopt errors - `--low_memory_mode` causes meta device errors with V4 — don't use - modelopt has no explicit V4 support — relies on auto-detection of fused experts - The `quant_module_patched.py` patch fixes `iter_weights_for_calibration()` for V4's `nn.ModuleList` expert quantizers — already applied in the venv