biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[Misc] Collect model support info in a single process per model (#9233)

Browse Source
This commit is contained in:
Cyrus Leung
2024-10-11 19:08:11 +08:00
committed by GitHub
parent cbc2ef5529
commit e808156f30
4 changed files with 236 additions and 167 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/source/models/adding_model.rst
View File

@@ -99,7 +99,7 @@ This method should load the weights from the HuggingFace's checkpoint file and a
5. Register your model
----------------------
Finally, register your :code:`*ForCausalLM` class to the :code:`_MODELS` in `vllm/model_executor/models/registry.py <https://github.com/vllm-project/vllm/blob/main/vllm/model_executor/models/registry.py>`_.
Finally, register your :code:`*ForCausalLM` class to the :code:`_VLLM_MODELS` in `vllm/model_executor/models/registry.py <https://github.com/vllm-project/vllm/blob/main/vllm/model_executor/models/registry.py>`_.
6. Out-of-Tree Model Integration
--------------------------------------------