# 2026-05-08 DeepSeek V4 Pro NVFP4 Quantization

## Status: Blocked on mixed-precision model's deeply non-standard weight format

The mixed-precision `DeepSeek-V4-Pro` model has three levels of compression:
- **Attention layers**: FP8 (float8_e4m3fn + float8_e8m0fnu scales) — our `dequant_fp8_to_bf16.py` handles these ✅
- **Routed experts**: INT8 at **half dimensions** — e.g. `w1: int8 [3072, 3584]` instead of expected `[3072, 7168]`, with separate scale factors. Our dequant script does NOT handle these ❌
- **Shared experts**: FP8 (float8_e4m3fn) — also not dequantized ❌

Result: `from_pretrained` sees shape mismatches on expert weights because the INT8 expert tensors are at half the expected dimensions. `ignore_mismatched_sizes` won't help — it'd reinitialize with random weights.

## What needs to happen
A **complete dequantization** script that handles ALL compressed tensor types:
1. FP8 attention weights (weight × block_scale → BF16) — already working
2. INT8 routed expert weights (weight × scale → BF16, expand dimensions back) — need to find scale tensors
3. FP8 shared expert weights — same as attention dequant
Then produce a truly pure-BF16 model with standard dimensions.

## Attempt history on B200 node (45.76.247.107)
1. **Custom quantizer (Path A)**: Output 840GB, wrong format — rejected
2. **FP8 model + modelopt**: DeepGEMM crashes on Blackwell (SM100 unsupported), Triton finegrained FP8 matmul has K mismatch
3. **Naive upcast (upcast_to_bf16.py)**: Just cast FP8 scales to BF16 without dequantizing — broken, FP8Linear.forward still tries FP8 kernel path
4. **Proper dequant (dequant_fp8_to_bf16.py)**: Handles FP8 attention ✅, but missed INT8 experts and FP8 shared experts — shape mismatch on load
5. **nvfp4_experts_only on mixed-precision**: Crashes in calibration forward pass (FP8Linear kernel issues)
6. **nvfp4_experts_only on dequantized BF16**: OOM killed during quantizer setup (94% RAM with seq_device_map)
7. **nvfp4_experts_only, 128 calib, gpu_max_mem_percentage 0.9**: Shape mismatch — INT8 expert weights at half dimensions

## Repo
- Branch: `modelopt-nvfp4` on `https://sweetapi.com/biondizzle/deepseek-v4-quant.git`
- Auth: user `biondizzle`, password `Gold1234@x`
- Scripts: `scripts/dequant_fp8_to_bf16.py`, `scripts/upcast_to_bf16.py`, `scripts/model_opt_nvfp4_experts_only.py`
- Patches: `patches/patch_finegrained_fp8_blackwell.py`, `patches/quant_module_patched.py`

## Key learnings
- Mixed-precision DeepSeek-V4-Pro is NOT just "BF16 with some FP8" — it has INT8 expert weights at half dimensions too
- DeepGEMM only supports SM90 (Hopper), not SM100 (Blackwell)
- FP8Linear.forward() has an escape hatch: `if self.weight.element_size() > 1: return F.linear(...)` — if weight is BF16, it skips the FP8 kernel
- `--use_seq_device_map` causes OOM on 2.8TB RAM with 782GB model + quantizer overhead
- `--gpu_max_mem_percentage 0.9` is the right approach — fits ~1.3TB on 8×B200 GPUs
- Calibration size (256 vs 128) only affects scale quality, not model format or inference

## Mike's preferences
- Name output dirs with strategy: `DeepSeek-V4-Pro_NVFP4-<strategy>_kv_fp8_cast`
- One script per strategy, copy and tweak for each
- Track everything in git, don't lose working code
- Target output: ~600GB NVFP4


## 01:37 UTC - Complete dequant script updated, strategy pivot to full NVFP4

### Script update: `dequant_fp8_to_bf16.py` now handles ALL compressed types:
1. **FP8 weights** (attention + shared experts): block-wise (128×128) dequant via float8_e4m3fn × float8_e8m0fnu scale → BF16
2. **FP4 (E2M1) expert weights**: Unpack 2 FP4 values per int8 byte (lower nibble first, upper second), E2M1 LUT lookup × E8M0 scale, per-row 32-column block scaling (MXFP4 microscaling), output dimensions 2× stored dimensions → BF16
3. **Scale tensors removed** after dequantization
4. Progress output per shard with ETA

### Mike's directive: strategy change
- **Drop `nvfp4_experts_only`** — go full `nvfp4` quantization on the properly dequantized BF16 model
- Need to create `scripts/model_opt_nvfp4_full.py` (copy from experts_only, change `--quantization` from `nvfp4_experts_only` to `nvfp4`)
- Steps: (1) Run complete dequant to get true BF16 model, (2) Run full nvfp4 quantization on it

### FP4 (E2M1) expert weight format confirmed (was incorrectly assumed INT4)
- `w1: int8 [3072, 3584]` with scale `float8_e8m0fnu [3072, 224]` → output `[3072, 7168]` (224 * 32 = 7168)
- `w2: int8 [7168, 1536]` with scale `float8_e8m0fnu [7168, 96]` → output `[7168, 3072]` (96 * 32 = 3072)
- `w3: int8 [3072, 3584]` with scale `float8_e8m0fnu [3072, 224]` → output `[3072, 7168]`
- **Proof**: nibble index 0 (+0.0) vs index 8 (-0.0) ratio = 0.996. INT4 would have -8 at index 8 (rare). FP4 has -0.0 at index 8 (same as +0.0).

### Previous BF16 model is WRONG
- The `/root/nvidia-meeting/DeepSeek-V4-Pro-BF16` (782GB) only dequantized FP8 attention weights
- FP4 expert weights and FP8 shared experts were left compressed → shape mismatches on load
- Must re-run complete dequant, output will likely be larger (~1.1-1.2TB?)