[Docs] Switch to better markdown linting pre-commit hook (#21851)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
This commit is contained in:
@@ -99,7 +99,7 @@ From any node, enter a container and run `ray status` and `ray list nodes` to ve
|
||||
### Running vLLM on a Ray cluster
|
||||
|
||||
!!! tip
|
||||
If Ray is running inside containers, run the commands in the remainder of this guide _inside the containers_, not on the host. To open a shell inside a container, connect to a node and use `docker exec -it <container_name> /bin/bash`.
|
||||
If Ray is running inside containers, run the commands in the remainder of this guide *inside the containers*, not on the host. To open a shell inside a container, connect to a node and use `docker exec -it <container_name> /bin/bash`.
|
||||
|
||||
Once a Ray cluster is running, use vLLM as you would in a single-node setting. All resources across the Ray cluster are visible to vLLM, so a single `vllm` command on a single node is sufficient.
|
||||
|
||||
|
||||
@@ -31,11 +31,12 @@ vLLM provides three communication backends for EP:
|
||||
|
||||
Enable EP by setting the `--enable-expert-parallel` flag. The EP size is automatically calculated as:
|
||||
|
||||
```
|
||||
```text
|
||||
EP_SIZE = TP_SIZE × DP_SIZE
|
||||
```
|
||||
|
||||
Where:
|
||||
|
||||
- `TP_SIZE`: Tensor parallel size (always 1 for now)
|
||||
- `DP_SIZE`: Data parallel size
|
||||
- `EP_SIZE`: Expert parallel size (computed automatically)
|
||||
|
||||
@@ -206,6 +206,7 @@ you can use the [official OpenAI Python client](https://github.com/openai/openai
|
||||
We support both [Vision](https://platform.openai.com/docs/guides/vision)- and
|
||||
[Audio](https://platform.openai.com/docs/guides/audio?audio-generation-quickstart-example=audio-in)-related parameters;
|
||||
see our [Multimodal Inputs](../features/multimodal_inputs.md) guide for more information.
|
||||
|
||||
- *Note: `image_url.detail` parameter is not supported.*
|
||||
|
||||
Code example: <gh-file:examples/online_serving/openai_chat_completion_client.py>
|
||||
|
||||
Reference in New Issue
Block a user