biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
Files
290f4ada2bf42174a53ae6aab2873e115c8ae11b
vllm/tests/models/test_fp8.py
Cody Yu a3a73ab069 [Misc] Load FP8 kv-cache scaling factors from checkpoints (#4893) 
The 2nd PR for #4532.

This PR supports loading FP8 kv-cache scaling factors from a FP8 checkpoint (with .kv_scale parameter).
2024-05-22 13:28:20 -07:00

5.9 KiB
Raw Blame History

View Raw
Reference in New Issue View Git Blame Copy Permalink