Change the name to vLLM (#150)
This commit is contained in:
@@ -3,8 +3,8 @@
|
||||
Supported Models
|
||||
================
|
||||
|
||||
CacheFlow supports a variety of generative Transformer models in `HuggingFace Transformers <https://github.com/huggingface/transformers>`_.
|
||||
The following is the list of model architectures that are currently supported by CacheFlow.
|
||||
vLLM supports a variety of generative Transformer models in `HuggingFace Transformers <https://github.com/huggingface/transformers>`_.
|
||||
The following is the list of model architectures that are currently supported by vLLM.
|
||||
Alongside each architecture, we include some popular models that use it.
|
||||
|
||||
.. list-table::
|
||||
@@ -22,19 +22,19 @@ Alongside each architecture, we include some popular models that use it.
|
||||
* - :code:`OPTForCausalLM`
|
||||
- OPT, OPT-IML
|
||||
|
||||
If your model uses one of the above model architectures, you can seamlessly run your model with CacheFlow.
|
||||
If your model uses one of the above model architectures, you can seamlessly run your model with vLLM.
|
||||
Otherwise, please refer to :ref:`Adding a New Model <adding_a_new_model>` for instructions on how to implement support for your model.
|
||||
Alternatively, you can raise an issue on our `GitHub <https://github.com/WoosukKwon/cacheflow/issues>`_ project.
|
||||
Alternatively, you can raise an issue on our `GitHub <https://github.com/WoosukKwon/vllm/issues>`_ project.
|
||||
|
||||
.. tip::
|
||||
The easiest way to check if your model is supported is to run the program below:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
from cacheflow import LLM
|
||||
from vllm import LLM
|
||||
|
||||
llm = LLM(model=...) # Name or path of your model
|
||||
output = llm.generate("Hello, my name is")
|
||||
print(output)
|
||||
|
||||
If CacheFlow successfully generates text, it indicates that your model is supported.
|
||||
If vLLM successfully generates text, it indicates that your model is supported.
|
||||
|
||||
Reference in New Issue
Block a user