biondizzle f86892e26b Replace BF16 dequant with input_scale warmup fix for attention NVFP4 ...

Instead of dequantizing attention weights to BF16 (which had issues
with MergedColumnParallelLinear and different weight_scale_2 values),
keep the NVFP4 path but fix the activation global scale.

Compute correct input_global_scale_inv by:
1. Temporarily dequantizing weight to BF16
2. Running warmup forward with random input
3. Computing actual activation amax
4. Setting scale_inv = amax * headroom

This preserves the original NVFP4 quantization pipeline.

2026-05-18 15:43:46 +00:00

..

patches

Replace BF16 dequant with input_scale warmup fix for attention NVFP4

2026-05-18 15:43:46 +00:00

nvfp4_cutedsl.py

HOTFIX: remove NaN checks from run() — torch.isnan().any() does CPU-GPU sync, breaks cudagraph

2026-05-17 22:28:32 +00:00