biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[Model] Pipeline Parallel Support for DeepSeek v2 (#6519)

Browse Source 
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
This commit is contained in:
Travis Johnson
2024-07-23 13:22:09 -06:00
committed by GitHub
parent 58f53034ad
commit 507ef787d8
2 changed files with 115 additions and 39 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
vllm/config.py
View File

@@ -31,6 +31,7 @@ _EMBEDDING_MODEL_MAX_NUM_BATCHED_TOKENS = 32768
_PP_SUPPORTED_MODELS = [
"AquilaModel",
"AquilaForCausalLM",
"DeepseekV2ForCausalLM",
"InternLMForCausalLM",
"LlamaForCausalLM",
"LLaMAForCausalLM",