biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[Misc] Update qqq to use vLLMParameters (#7805)

Browse Source
This commit is contained in:
Dipika Sikka
2024-08-26 15:16:15 -04:00
committed by GitHub
parent 2deb029d11
commit 665304092d
3 changed files with 54 additions and 64 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
tests/weight_loading/models.txt
View File

@@ -17,4 +17,6 @@ awq, casperhansen/mixtral-instruct-awq, main
awq_marlin, casperhansen/mixtral-instruct-awq, main
fp8, neuralmagic/Meta-Llama-3-8B-Instruct-FP8-KV, main
marlin, nm-testing/zephyr-beta-7b-marlin-g128, main
marlin, robertgshaw2/zephyr-7b-beta-channelwise-marlin, main
marlin, robertgshaw2/zephyr-7b-beta-channelwise-marlin, main
qqq, HandH1998/QQQ-Llama-3-8b-g128, main
qqq, HandH1998/QQQ-Llama-3-8b, main