biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[Hardware][NV] Add support for ModelOpt static scaling checkpoints. (#6112)

Browse Source
This commit is contained in:
Pavani Majety
2024-09-10 21:38:40 -07:00
committed by GitHub
parent 1230263e16
commit efcf946a15
7 changed files with 258 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
examples/fp8/quantizer/README.md
View File

@@ -1,6 +1,6 @@
### Quantizer Utilities
`quantize.py`: NVIDIA Quantization utilities using AMMO, ported from TensorRT-LLM:
`https://github.com/NVIDIA/TensorRT-LLM/blob/main/examples/quantization/quantize.py`
`quantize.py`: NVIDIA Quantization utilities using TensorRT-Model-Optimizer, ported
from TensorRT-LLM: [`examples/quantization/quantize.py`](https://github.com/NVIDIA/TensorRT-LLM/blob/main/examples/quantization/quantize.py)
### Prerequisite