[BugFix] Force registration of w8a8_block_fp8_matmul_deepgemm via lazy import (#19514)

Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com> Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
2025-06-12 00:28:12 -04:00
parent 2f1c19b245
commit e5d35d62f5
1 changed files with 1 additions and 0 deletions
--- a/vllm/model_executor/layers/quantization/utils/fp8_utils.py
+++ b/vllm/model_executor/layers/quantization/utils/fp8_utils.py
@@ -143,6 +143,7 @@ def apply_w8a8_block_fp8_linear(
            column_major_scales=True,
        )

+        import vllm.model_executor.layers.quantization.deepgemm  # noqa: F401
        output = torch.ops.vllm.w8a8_block_fp8_matmul_deepgemm(
            q_input,
            weight,