biondizzle a51edd238e Add post-quant-init forward hook to fix attention NVFP4 ...

The key insight: process_weights_after_loading runs AFTER load_weights
and sets up FlashInferCutlassNvFp4LinearKernel with broken
input_global_scale_inv. Any fix inside load_weights gets overwritten.

Solution: register a one-shot forward pre-hook that runs on the first
forward call (guaranteed after all init). It dequantizes attention
NVFP4 weights to BF16 and replaces quant_method with
UnquantizedLinearMethod. Since process_weights_after_loading already
ran, our changes won't be overwritten.

Standalone test confirmed: all attention weights produce valid
non-NaN output when dequantized to BF16.

2026-05-18 17:56:19 +00:00

..

patches

Add post-quant-init forward hook to fix attention NVFP4

2026-05-18 17:56:19 +00:00

nvfp4_cutedsl.py

HOTFIX: remove NaN checks from run() — torch.isnan().any() does CPU-GPU sync, breaks cudagraph

2026-05-17 22:28:32 +00:00