biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[Doc] Improve installation signposting (#12575)

Browse Source 
- Make device tab names more explicit
- Add comprehensive list of devices to
https://docs.vllm.ai/en/latest/getting_started/installation/index.html
- Add `attention` blocks to the intro of all devices that don't have
pre-built wheels/images

---------

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
This commit is contained in:
Harry Mellor
2025-01-31 23:38:35 +00:00
committed by GitHub
parent fc542144c4
commit 60808bd4c7
13 changed files with 111 additions and 59 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
docs/source/getting_started/installation/cpu/apple.inc.md
View File

@@ -4,6 +4,10 @@ vLLM has experimental support for macOS with Apple silicon. For now, users shall
Currently the CPU implementation for macOS supports FP32 and FP16 datatypes.
:::{attention}
There are no pre-built wheels or images for this device, so you must build vLLM from source.
:::
## Requirements
- OS: `macOS Sonoma` or later

4
docs/source/getting_started/installation/cpu/arm.inc.md
View File

@@ -4,6 +4,10 @@ vLLM has been adapted to work on ARM64 CPUs with NEON support, leveraging the CP
ARM CPU backend currently supports Float32, FP16 and BFloat16 datatypes.
:::{attention}
There are no pre-built wheels or images for this device, so you must build vLLM from source.
:::
## Requirements
- OS: Linux

13
docs/source/getting_started/installation/cpu/index.md
View File

@@ -5,7 +5,8 @@ vLLM is a Python library that supports the following CPU variants. Select your C
:::::{tab-set}
:sync-group: device
::::{tab-item} x86
::::{tab-item} Intel/AMD x86
:selected:
:sync: x86
:::{include} x86.inc.md
@@ -15,7 +16,7 @@ vLLM is a Python library that supports the following CPU variants. Select your C
::::
::::{tab-item} ARM
::::{tab-item} ARM AArch64
:sync: arm
:::{include} arm.inc.md
@@ -44,7 +45,7 @@ vLLM is a Python library that supports the following CPU variants. Select your C
:::::{tab-set}
:sync-group: device
::::{tab-item} x86
::::{tab-item} Intel/AMD x86
:sync: x86
:::{include} x86.inc.md
@@ -54,7 +55,7 @@ vLLM is a Python library that supports the following CPU variants. Select your C
::::
::::{tab-item} ARM
::::{tab-item} ARM AArch64
:sync: arm
:::{include} arm.inc.md
@@ -92,7 +93,7 @@ Currently, there are no pre-built CPU wheels.
:::::{tab-set}
:sync-group: device
::::{tab-item} x86
::::{tab-item} Intel/AMD x86
:sync: x86
:::{include} x86.inc.md
@@ -102,7 +103,7 @@ Currently, there are no pre-built CPU wheels.
::::
::::{tab-item} ARM
::::{tab-item} ARM AArch64
:sync: arm
:::{include} arm.inc.md

12
docs/source/getting_started/installation/cpu/x86.inc.md
View File

@@ -2,12 +2,20 @@
vLLM initially supports basic model inferencing and serving on x86 CPU platform, with data types FP32, FP16 and BF16.
:::{attention}
There are no pre-built wheels or images for this device, so you must build vLLM from source.
:::
## Requirements
- OS: Linux
- Compiler: `gcc/g++ >= 12.3.0` (optional, recommended)
- Instruction Set Architecture (ISA): AVX512 (optional, recommended)
:::{tip}
[Intel Extension for PyTorch (IPEX)](https://github.com/intel/intel-extension-for-pytorch) extends PyTorch with up-to-date features optimizations for an extra performance boost on Intel hardware.
:::
## Set up using Python
### Pre-built wheels
@@ -29,7 +37,3 @@ vLLM initially supports basic model inferencing and serving on x86 CPU platform,
### Build image from source
## Extra information
## Intel Extension for PyTorch
- [Intel Extension for PyTorch (IPEX)](https://github.com/intel/intel-extension-for-pytorch) extends PyTorch with up-to-date features optimizations for an extra performance boost on Intel hardware.