biondizzle 8ac8e20fa9 Fix OOM: cap buffer pre-allocation at cudagraph max capture size ...

padded_hidden/activated buffers were sized for max_num_tokens=8192,
which is 72 MB per layer × 60 layers = 4.3 GB → OOM with 178 GB GPUs
(almost full from model + KV cache).

Now cap at max cudagraph capture size (512 tokens). Eager-mode runs
with >512 tokens will need dynamic allocation, but vLLM always uses
cudagraph for inference after warmup.

2026-05-17 14:14:13 +00:00

..

patches

Fix OOM: cap buffer pre-allocation at cudagraph max capture size

2026-05-17 14:14:13 +00:00

nvfp4_cutedsl.py

Fix OOM: cap buffer pre-allocation at cudagraph max capture size

2026-05-17 14:14:13 +00:00