[Hardware][Apple] Native support for macOS Apple Silicon (#11696)

Signed-off-by: Wallas Santos <wallashss@ibm.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
This commit is contained in:
Wallas Henrique
2025-01-08 05:35:49 -03:00
committed by GitHub
parent a1b2b8606e
commit cfd3219f58
11 changed files with 210 additions and 30 deletions

View File

@@ -0,0 +1,51 @@
(installation-apple)=
# Installation for macOS
vLLM has experimental support for macOS with Apple Silicon. For now, users shall build from the source vLLM to natively run on macOS. For more details, like running on vLLM in a docker container, see [ARM CPU Documentation](installation-arm)
Currently the CPU implementation for macOS supports FP32 and FP16 datatypes.
## Requirements
- **Operating System**: `macOS Sonoma` or later
- **SDK** `XCode 15.4` or later with Command Line Tools
- **Compilers**: `Apple Clang >= 15.0.0`
<!-- (arm-backend-quick-start-dockerfile)= -->
## Build and installation
After installation of XCode and the Command Line Tools, which include Apple Clang, execute the following commands to build and install vLLM from the source.
```
$ git clone https://github.com/vllm-project/vllm.git
$ cd vllm
$ pip install -r requirements-cpu.txt
$ pip install -e .
```
```{note}
On macOS the `VLLM_TARGET_DEVICE` is automatically set to `cpu`, which currently is the only supported device.
```
## Troubleshooting
If the build has error like the following snippet where standard C++ headers cannot be found, try to remove and reinstall your
[Command Line Tools for Xcode](https://developer.apple.com/download/all/).
```
[...] fatal error: 'map' file not found
1 | #include <map>
| ^~~~~
1 error generated.
[2/8] Building CXX object CMakeFiles/_C.dir/csrc/cpu/pos_encoding.cpp.o
[...] fatal error: 'cstddef' file not found
10 | #include <cstddef>
| ^~~~~~~~~
1 error generated.
```

View File

@@ -2,7 +2,7 @@
# Installation for ARM CPUs
vLLM has been adapted to work on ARM64 CPUs with NEON support, leveraging the CPU backend initially developed for the x86 platform. This guide provides installation instructions specific to ARM. For additional details on supported features, refer to the [x86 CPU documentation](#installation-x86) covering:
vLLM has been adapted to work on ARM64 CPUs with NEON support, leveraging the CPU backend initially developed for the x86 platform. This guide provides installation instructions specific to ARM (which also apply to Apple Silicon, see [Installation for macOS](#installation-apple) for more). For additional details on supported features, refer to the [x86 CPU documentation](#installation-x86) covering:
- CPU backend inference capabilities
- Relevant runtime environment variables
@@ -20,7 +20,7 @@ Contents:
## Requirements
- **Operating System**: Linux or macOS
- **Compiler**: `gcc/g++ >= 12.3.0` (optional, but recommended)
- **Compilers**: `gcc/g++ >= 12.3.0` (optional, but recommended) or `Apple Clang >= 15.0.0` for macOS
- **Instruction Set Architecture (ISA)**: NEON support is required
(arm-backend-quick-start-dockerfile)=

View File

@@ -11,6 +11,7 @@ gpu-cuda
gpu-rocm
cpu-x86
cpu-arm
cpu-apple
hpu-gaudi
tpu
xpu