biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[Hardware][Apple] Native support for macOS Apple Silicon (#11696)

Browse Source 
Signed-off-by: Wallas Santos <wallashss@ibm.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
This commit is contained in:
Wallas Henrique
2025-01-08 05:35:49 -03:00
committed by GitHub
parent a1b2b8606e
commit cfd3219f58
11 changed files with 210 additions and 30 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
docs/source/getting_started/installation/cpu-arm.md
View File

@@ -2,7 +2,7 @@
# Installation for ARM CPUs
vLLM has been adapted to work on ARM64 CPUs with NEON support, leveraging the CPU backend initially developed for the x86 platform. This guide provides installation instructions specific to ARM. For additional details on supported features, refer to the [x86 CPU documentation](#installation-x86) covering:
vLLM has been adapted to work on ARM64 CPUs with NEON support, leveraging the CPU backend initially developed for the x86 platform. This guide provides installation instructions specific to ARM (which also apply to Apple Silicon, see [Installation for macOS](#installation-apple) for more). For additional details on supported features, refer to the [x86 CPU documentation](#installation-x86) covering:
- CPU backend inference capabilities
- Relevant runtime environment variables
@@ -20,7 +20,7 @@ Contents:
## Requirements
- **Operating System**: Linux or macOS
- **Compiler**: `gcc/g++ >= 12.3.0` (optional, but recommended)
- **Compilers**: `gcc/g++ >= 12.3.0` (optional, but recommended) or `Apple Clang >= 15.0.0` for macOS
- **Instruction Set Architecture (ISA)**: NEON support is required
(arm-backend-quick-start-dockerfile)=