[Hardware][Intel GPU] Add Intel GPU(XPU) inference backend (#3814)
Co-authored-by: Jiang Li <jiang1.li@intel.com> Co-authored-by: Abhilash Majumder <abhilash.majumder@intel.com> Co-authored-by: Abhilash Majumder <30946547+abhilash1910@users.noreply.github.com>
This commit is contained in:
61
docs/source/getting_started/xpu-installation.rst
Normal file
61
docs/source/getting_started/xpu-installation.rst
Normal file
@@ -0,0 +1,61 @@
|
||||
.. _installation_xpu:
|
||||
|
||||
Installation with XPU
|
||||
========================
|
||||
|
||||
vLLM initially supports basic model inferencing and serving on Intel GPU platform.
|
||||
|
||||
Table of contents:
|
||||
|
||||
#. :ref:`Requirements <xpu_backend_requirements>`
|
||||
#. :ref:`Quick start using Dockerfile <xpu_backend_quick_start_dockerfile>`
|
||||
#. :ref:`Build from source <build_xpu_backend_from_source>`
|
||||
|
||||
.. _xpu_backend_requirements:
|
||||
|
||||
Requirements
|
||||
------------
|
||||
|
||||
* OS: Linux
|
||||
* Supported Hardware: Intel Data Center GPU (Intel ARC GPU WIP)
|
||||
* OneAPI requirements: oneAPI 2024.1
|
||||
|
||||
.. _xpu_backend_quick_start_dockerfile:
|
||||
|
||||
Quick start using Dockerfile
|
||||
----------------------------
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
$ docker build -f Dockerfile.xpu -t vllm-xpu-env --shm-size=4g .
|
||||
$ docker run -it \
|
||||
--rm \
|
||||
--network=host \
|
||||
--device /dev/dri \
|
||||
-v /dev/dri/by-path:/dev/dri/by-path \
|
||||
vllm-xpu-env
|
||||
|
||||
.. _build_xpu_backend_from_source:
|
||||
|
||||
Build from source
|
||||
-----------------
|
||||
|
||||
- First, install required driver and intel OneAPI 2024.1.
|
||||
|
||||
- Second, install Python packages for vLLM XPU backend building:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
$ pip install --upgrade pip
|
||||
$ pip install -v -r requirements-xpu.txt
|
||||
|
||||
- Finally, build and install vLLM XPU backend:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
$ VLLM_TARGET_DEVICE=xpu python setup.py install
|
||||
|
||||
.. note::
|
||||
- FP16 is the default data type in the current XPU backend. The BF16 data
|
||||
type will be supported in the future.
|
||||
|
||||
Reference in New Issue
Block a user