biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

Add DeepGEMM to Dockerfile in vllm-base image (#21533)

Browse Source 
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
This commit is contained in:
Matthew Bonanni
2025-07-31 21:01:55 -04:00
committed by GitHub
parent c3e0e9337e
commit e360316ab9
4 changed files with 46 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
tests/kernels/moe/test_deepgemm.py
View File

@@ -15,13 +15,13 @@ import torch
from vllm.model_executor.layers.fused_moe.fused_moe import fused_experts
from vllm.model_executor.layers.quantization.utils.fp8_utils import (
per_token_group_quant_fp8)
from vllm.utils import has_deep_gemm
from vllm.utils.deep_gemm import calc_diff, per_block_cast_to_fp8
from vllm.utils.deep_gemm import (calc_diff, is_deep_gemm_supported,
per_block_cast_to_fp8)
BLOCK_SIZE = [128, 128]
requires_deep_gemm = pytest.mark.skipif(
not has_deep_gemm(),
not is_deep_gemm_supported(),
reason="Requires deep_gemm kernels",
)