biondizzle cebc586014 Fix OOM: use 1-token warmup sample + free immediately ...

8 tokens * 7168 hidden * ~40 NVFP4 layers = ~2.3 MiB per layer * 40 = 92 MiB
But the dummy weight param (out_features * in_features * 2 bytes BF16) was
the real killer — each layer allocated a BF16 dummy of its full weight shape.
With 1 token the warmup still gets a valid gs, and empty_cache frees the
sample tensor before KV cache allocation.

2026-05-19 07:28:57 +00:00

..

linear/nvfp4

Fix OOM: use 1-token warmup sample + free immediately

2026-05-19 07:28:57 +00:00