118 lines
3.1 KiB
Markdown
118 lines
3.1 KiB
Markdown
|
# Other AI accelerators
|
||
|
|
|
||
|
|
vLLM is a Python library that supports the following AI accelerators. Select your AI accelerator type to see vendor specific instructions:
|
||
|
|
|
||
|
|
=== "Google TPU"
|
||
|
|
|
||
|
|
--8<-- "docs/getting_started/installation/ai_accelerator/tpu.inc.md:installation"
|
||
|
|
|
||
|
|
=== "Intel Gaudi"
|
||
|
|
|
||
|
|
--8<-- "docs/getting_started/installation/ai_accelerator/hpu-gaudi.inc.md:installation"
|
||
|
|
|
||
|
|
=== "AWS Neuron"
|
||
|
|
|
||
|
|
--8<-- "docs/getting_started/installation/ai_accelerator/neuron.inc.md:installation"
|
||
|
|
|
||
|
|
## Requirements
|
||
|
|
|
||
|
|
=== "Google TPU"
|
||
|
|
|
||
|
|
--8<-- "docs/getting_started/installation/ai_accelerator/tpu.inc.md:requirements"
|
||
|
|
|
||
|
|
=== "Intel Gaudi"
|
||
|
|
|
||
|
|
--8<-- "docs/getting_started/installation/ai_accelerator/hpu-gaudi.inc.md:requirements"
|
||
|
|
|
||
|
|
=== "AWS Neuron"
|
||
|
|
|
||
|
|
--8<-- "docs/getting_started/installation/ai_accelerator/neuron.inc.md:requirements"
|
||
|
|
|
||
|
|
## Configure a new environment
|
||
|
|
|
||
|
|
=== "Google TPU"
|
||
|
|
|
||
|
|
--8<-- "docs/getting_started/installation/ai_accelerator/tpu.inc.md:configure-a-new-environment"
|
||
|
|
|
||
|
|
=== "Intel Gaudi"
|
||
|
|
|
||
|
|
--8<-- "docs/getting_started/installation/ai_accelerator/hpu-gaudi.inc.md:configure-a-new-environment"
|
||
|
|
|
||
|
|
=== "AWS Neuron"
|
||
|
|
|
||
|
|
--8<-- "docs/getting_started/installation/ai_accelerator/neuron.inc.md:configure-a-new-environment"
|
||
|
|
|
||
|
|
## Set up using Python
|
||
|
|
|
||
|
|
### Pre-built wheels
|
||
|
|
|
||
|
|
=== "Google TPU"
|
||
|
|
|
||
|
|
--8<-- "docs/getting_started/installation/ai_accelerator/tpu.inc.md:pre-built-wheels"
|
||
|
|
|
||
|
|
=== "Intel Gaudi"
|
||
|
|
|
||
|
|
--8<-- "docs/getting_started/installation/ai_accelerator/hpu-gaudi.inc.md:pre-built-wheels"
|
||
|
|
|
||
|
|
=== "AWS Neuron"
|
||
|
|
|
||
|
|
--8<-- "docs/getting_started/installation/ai_accelerator/neuron.inc.md:pre-built-wheels"
|
||
|
|
|
||
|
|
### Build wheel from source
|
||
|
|
|
||
|
|
=== "Google TPU"
|
||
|
|
|
||
|
|
--8<-- "docs/getting_started/installation/ai_accelerator/tpu.inc.md:build-wheel-from-source"
|
||
|
|
|
||
|
|
=== "Intel Gaudi"
|
||
|
|
|
||
|
|
--8<-- "docs/getting_started/installation/ai_accelerator/hpu-gaudi.inc.md:build-wheel-from-source"
|
||
|
|
|
||
|
|
=== "AWS Neuron"
|
||
|
|
|
||
|
|
--8<-- "docs/getting_started/installation/ai_accelerator/neuron.inc.md:build-wheel-from-source"
|
||
|
|
|
||
|
|
## Set up using Docker
|
||
|
|
|
||
|
|
### Pre-built images
|
||
|
|
|
||
|
|
=== "Google TPU"
|
||
|
|
|
||
|
|
--8<-- "docs/getting_started/installation/ai_accelerator/tpu.inc.md:pre-built-images"
|
||
|
|
|
||
|
|
=== "Intel Gaudi"
|
||
|
|
|
||
|
|
--8<-- "docs/getting_started/installation/ai_accelerator/hpu-gaudi.inc.md:pre-built-images"
|
||
|
|
|
||
|
|
=== "AWS Neuron"
|
||
|
|
|
||
|
|
--8<-- "docs/getting_started/installation/ai_accelerator/neuron.inc.md:pre-built-images"
|
||
|
|
|
||
|
|
### Build image from source
|
||
|
|
|
||
|
|
=== "Google TPU"
|
||
|
|
|
||
|
|
--8<-- "docs/getting_started/installation/ai_accelerator/tpu.inc.md:build-image-from-source"
|
||
|
|
|
||
|
|
=== "Intel Gaudi"
|
||
|
|
|
||
|
|
--8<-- "docs/getting_started/installation/ai_accelerator/hpu-gaudi.inc.md:build-image-from-source"
|
||
|
|
|
||
|
|
=== "AWS Neuron"
|
||
|
|
|
||
|
|
--8<-- "docs/getting_started/installation/ai_accelerator/neuron.inc.md:build-image-from-source"
|
||
|
|
|
||
|
|
## Extra information
|
||
|
|
|
||
|
|
=== "Google TPU"
|
||
|
|
|
||
|
|
--8<-- "docs/getting_started/installation/ai_accelerator/tpu.inc.md:extra-information"
|
||
|
|
|
||
|
|
=== "Intel Gaudi"
|
||
|
|
|
||
|
|
--8<-- "docs/getting_started/installation/ai_accelerator/hpu-gaudi.inc.md:extra-information"
|
||
|
|
|
||
|
|
=== "AWS Neuron"
|
||
|
|
|
||
|
|
--8<-- "docs/getting_started/installation/ai_accelerator/neuron.inc.md:extra-information"
|