biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[Quantization][Refactor] Move CPU GPTQ kernel into MP linear (#31801)

Browse Source 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
Signed-off-by: Li, Jiang <bigpyj64@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
This commit is contained in:
Li, Jiang
2026-01-07 03:10:18 +08:00
committed by GitHub
parent c07163663d
commit 8becf146bd
9 changed files with 171 additions and 332 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
vllm/model_executor/layers/quantization/gptq_marlin.py
View File

@@ -276,7 +276,7 @@ class GPTQMarlinConfig(QuantizationConfig):
sym = quant_config.get("sym")
desc_act = quant_config.get("desc_act")
if not current_platform.is_cuda():
if not (current_platform.is_cuda() or current_platform.is_cpu()):
return False
if quant_method != "gptq":