Add Dockerfile (#1350)

This commit is contained in:
Stephen Krider
2023-10-31 12:36:47 -07:00
committed by GitHub
parent 7b895c5976
commit 9cabcb7645
5 changed files with 101 additions and 2 deletions

View File

@@ -0,0 +1,21 @@
.. _deploying_with_docker:
Deploying with Docker
============================
You can build and run vLLM from source via the provided dockerfile. To build vLLM:
.. code-block:: console
$ DOCKER_BUILDKIT=1 docker build . --target vllm --tag vllm --build-arg max_jobs=8
To run vLLM:
.. code-block:: console
$ docker run --runtime nvidia --gpus all \
-v ~/.cache/huggingface:/root/.cache/huggingface \
-p 8000:8000 \
--env "HUGGING_FACE_HUB_TOKEN=<secret>" \
vllm <args...>