biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[doc][distributed] fix doc argument order (#6691)

Browse Source
This commit is contained in:
youkaichao
2024-07-23 08:55:33 -07:00
committed by GitHub
parent cb1362a889
commit 71950af726
1 changed files with 4 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
docs/source/serving/distributed_serving.rst
View File

@@ -66,8 +66,8 @@ Pick a node as the head node, and run the following command:
$ bash run_cluster.sh \
$ vllm/vllm-openai \
$ ip_of_head_node \
$ /path/to/the/huggingface/home/in/this/node \
$ --head
$ --head \
$ /path/to/the/huggingface/home/in/this/node
On the rest of the worker nodes, run the following command:
@@ -76,8 +76,8 @@ On the rest of the worker nodes, run the following command:
$ bash run_cluster.sh \
$ vllm/vllm-openai \
$ ip_of_head_node \
$ /path/to/the/huggingface/home/in/this/node \
$ --worker
$ --worker \
$ /path/to/the/huggingface/home/in/this/node
Then you get a ray cluster of containers. Note that you need to keep the shells running these commands alive to hold the cluster. Any shell disconnect will terminate the cluster.