biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

Fix document of torchrun_example.py (#31113)

Browse Source
This commit is contained in:
Maosheng Liao
2026-03-31 18:54:23 +08:00
committed by GitHub
parent 7d65463528
commit aae3e688f8
1 changed files with 4 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
examples/offline_inference/torchrun_example.py
View File

@@ -4,9 +4,10 @@
experimental support for tensor-parallel inference with torchrun,
see https://github.com/vllm-project/vllm/issues/11400 for
the motivation and use case for this example.
run the script with `torchrun --nproc-per-node=2 torchrun_example.py`,
the argument 2 should match the `tensor_parallel_size` below.
see `tests/distributed/test_torchrun_example.py` for the unit test.
run the script with `torchrun --nproc-per-node=4 torchrun_example.py`,
the argument `4` should match the product of `tensor_parallel_size` and
`pipeline_parallel_size` below. see `tests/distributed/test_torchrun_example.py`
for the unit test.
"""
import torch.distributed as dist