[ROCm] Enable bitsandbytes quantization support on ROCm (#34688)

Signed-off-by: badaoui <abdennacerbadaoui0@gmail.com>
This commit is contained in:
BADAOUI Abdennacer
2026-02-21 09:34:55 +01:00
committed by GitHub
parent 2aab2bb543
commit 8dc8a99b56
8 changed files with 29 additions and 44 deletions

View File

@@ -7,7 +7,7 @@ Compared to other quantization methods, BitsAndBytes eliminates the need for cal
Below are the steps to utilize BitsAndBytes with vLLM.
```bash
pip install bitsandbytes>=0.46.1
pip install bitsandbytes>=0.49.2
```
vLLM reads the model's config file and supports both in-flight quantization and pre-quantized checkpoint.