biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[Doc] Convert docs to use colon fences (#12471)

Browse Source 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
This commit is contained in:
Harry Mellor
2025-01-29 03:38:29 +00:00
committed by GitHub
parent a7e3eba66f
commit dd6a3a02cb
68 changed files with 2352 additions and 2341 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
docs/source/getting_started/installation/cpu/apple.inc.md
View File

@@ -25,9 +25,9 @@ pip install -r requirements-cpu.txt
pip install -e .
```
```{note}
:::{note}
On macOS the `VLLM_TARGET_DEVICE` is automatically set to `cpu`, which currently is the only supported device.
```
:::
#### Troubleshooting

158
docs/source/getting_started/installation/cpu/index.md
View File

@@ -2,86 +2,86 @@
vLLM is a Python library that supports the following CPU variants. Select your CPU type to see vendor specific instructions:
::::{tab-set}
:::::{tab-set}
:sync-group: device
:::{tab-item} x86
::::{tab-item} x86
:sync: x86
```{include} x86.inc.md
:::{include} x86.inc.md
:start-after: "# Installation"
:end-before: "## Requirements"
```
:::
:::{tab-item} ARM
:sync: arm
```{include} arm.inc.md
:start-after: "# Installation"
:end-before: "## Requirements"
```
:::
:::{tab-item} Apple silicon
:sync: apple
```{include} apple.inc.md
:start-after: "# Installation"
:end-before: "## Requirements"
```
:::
::::
::::{tab-item} ARM
:sync: arm
:::{include} arm.inc.md
:start-after: "# Installation"
:end-before: "## Requirements"
:::
::::
::::{tab-item} Apple silicon
:sync: apple
:::{include} apple.inc.md
:start-after: "# Installation"
:end-before: "## Requirements"
:::
::::
:::::
## Requirements
- Python: 3.9 -- 3.12
::::{tab-set}
:::::{tab-set}
:sync-group: device
:::{tab-item} x86
::::{tab-item} x86
:sync: x86
```{include} x86.inc.md
:::{include} x86.inc.md
:start-after: "## Requirements"
:end-before: "## Set up using Python"
```
:::
:::{tab-item} ARM
:sync: arm
```{include} arm.inc.md
:start-after: "## Requirements"
:end-before: "## Set up using Python"
```
:::
:::{tab-item} Apple silicon
:sync: apple
```{include} apple.inc.md
:start-after: "## Requirements"
:end-before: "## Set up using Python"
```
:::
::::
::::{tab-item} ARM
:sync: arm
:::{include} arm.inc.md
:start-after: "## Requirements"
:end-before: "## Set up using Python"
:::
::::
::::{tab-item} Apple silicon
:sync: apple
:::{include} apple.inc.md
:start-after: "## Requirements"
:end-before: "## Set up using Python"
:::
::::
:::::
## Set up using Python
### Create a new Python environment
```{include} ../python_env_setup.inc.md
```
:::{include} ../python_env_setup.inc.md
:::
### Pre-built wheels
@@ -89,41 +89,41 @@ Currently, there are no pre-built CPU wheels.
### Build wheel from source
::::{tab-set}
:::::{tab-set}
:sync-group: device
:::{tab-item} x86
::::{tab-item} x86
:sync: x86
```{include} x86.inc.md
:::{include} x86.inc.md
:start-after: "### Build wheel from source"
:end-before: "## Set up using Docker"
```
:::
:::{tab-item} ARM
:sync: arm
```{include} arm.inc.md
:start-after: "### Build wheel from source"
:end-before: "## Set up using Docker"
```
:::
:::{tab-item} Apple silicon
:sync: apple
```{include} apple.inc.md
:start-after: "### Build wheel from source"
:end-before: "## Set up using Docker"
```
:::
::::
::::{tab-item} ARM
:sync: arm
:::{include} arm.inc.md
:start-after: "### Build wheel from source"
:end-before: "## Set up using Docker"
:::
::::
::::{tab-item} Apple silicon
:sync: apple
:::{include} apple.inc.md
:start-after: "### Build wheel from source"
:end-before: "## Set up using Docker"
:::
::::
:::::
## Set up using Docker
### Pre-built images
@@ -142,9 +142,9 @@ $ docker run -it \
vllm-cpu-env
```
:::{tip}
::::{tip}
For ARM or Apple silicon, use `Dockerfile.arm`
:::
::::
## Supported features

4
docs/source/getting_started/installation/cpu/x86.inc.md
View File

@@ -17,10 +17,10 @@ vLLM initially supports basic model inferencing and serving on x86 CPU platform,
:::{include} build.inc.md
:::
```{note}
:::{note}
- AVX512_BF16 is an extension ISA provides native BF16 data type conversion and vector product instructions, which brings some performance improvement compared with pure AVX512. The CPU backend build script will check the host CPU flags to determine whether to enable AVX512_BF16.
- If you want to force enable AVX512_BF16 for the cross-compilation, please set environment variable `VLLM_CPU_AVX512BF16=1` before the building.
```
:::
## Set up using Docker