biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[torch.compile] Adding torch compile annotations to some models (#9641)

Browse Source 
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
This commit is contained in:
Yongzao
2024-10-25 00:31:42 +08:00
committed by GitHub
parent de662d32b5
commit d27cfbf791
7 changed files with 21 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
tests/distributed/test_pipeline_parallel.py
View File

@@ -171,7 +171,8 @@ TEXT_GENERATION_MODELS = {
"stabilityai/stablelm-3b-4e1t": PPTestSettings.fast(),
"bigcode/starcoder2-3b": PPTestSettings.fast(),
"upstage/solar-pro-preview-instruct": PPTestSettings.fast(tp_base=2),
# FIXME: Cannot load tokenizer in latest transformers version
# FIXME: Cannot load tokenizer in latest transformers version.
# Need to use tokenizer from `meta-llama/Llama-2-7b-chat-hf`
# "xverse/XVERSE-7B-Chat": PPTestSettings.fast(trust_remote_code=True),
# [Encoder-only]
# TODO: Implement PP