biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[Quantization] Deprecate Long Tail of Schemes (#31688)

Browse Source 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
This commit is contained in:
Robert Shaw
2026-01-08 19:07:45 -05:00
committed by GitHub
parent d62cfe546d
commit 5825bbc1f7
8 changed files with 61 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
tests/models/quantization/test_gptq_marlin_24.py
View File

@@ -63,7 +63,10 @@ def test_models(
num_logprobs: int,
) -> None:
with vllm_runner(
model_pair.model_marlin, dtype=dtype, quantization="gptq_marlin_24"
model_pair.model_marlin,
dtype=dtype,
quantization="gptq_marlin_24",
allow_deprecated_quantization=True,
) as marlin_24_model:
marlin_24_outputs = marlin_24_model.generate_greedy_logprobs(
example_prompts, max_tokens, num_logprobs