biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[CI]Test Group 'NixlConnector PD accuracy tests' is fixed (#31460)

Browse Source 
Signed-off-by: qli88 <qiang.li2@amd.com>
This commit is contained in:
qli88
2025-12-29 17:48:56 -06:00
committed by GitHub
parent d63b969675
commit 0f35429a0c
5 changed files with 157 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
docs/features/nixl_connector_usage.md
View File

@@ -6,11 +6,17 @@ NixlConnector is a high-performance KV cache transfer connector for vLLM's disag
### Installation
Install the NIXL library: `uv pip install nixl`, as a quick start.
Install the NIXL library: `uv pip install nixl`, as a quick start on Nvidia platform.
- Refer to [NIXL official repository](https://github.com/ai-dynamo/nixl) for more installation instructions
- The specified required NIXL version can be found in [requirements/kv_connectors.txt](../../requirements/kv_connectors.txt) and other relevant config files
For ROCm platform, the [base ROCm docker file](../../docker/Dockerfile.rocm_base) includes RIXL and ucx already.
- Refer to [RIXL official repository](https://github.com/rocm/rixl) for more information
- The supportive libraries for RIXL can be found in [requirements/kv_connectors_rocm.txt](../../requirements/kv_connectors_rocm.txt)
- In the future we may remove RIXL from docker image file and users will be able to install from pre-compiled binary packages
For non-cuda platform, please install nixl with ucx build from source, instructed as below.
```bash