biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[Neuron][Kernel] NKI-based flash-attention kernel with paged KV cache (#11277)

Browse Source 
Signed-off-by: Liangfu Chen <liangfc@amazon.com>
Co-authored-by: Jiangfei Duan <jfduan@outlook.com>
This commit is contained in:
Liangfu Chen
2025-01-27 17:31:16 -08:00
committed by GitHub
parent 823ab79633
commit ddee88d0ff
3 changed files with 1126 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
.buildkite/run-neuron-test.sh
View File

@@ -54,4 +54,4 @@ docker run --rm -it --device=/dev/neuron0 --device=/dev/neuron1 --network host \
-e "NEURON_COMPILE_CACHE_URL=${NEURON_COMPILE_CACHE_MOUNT}" \
--name "${container_name}" \
${image_name} \
/bin/bash -c "python3 /workspace/vllm/examples/offline_inference/neuron.py"
/bin/bash -c "python3 /workspace/vllm/examples/offline_inference/neuron.py && python3 -m pytest /workspace/vllm/tests/neuron/ -v --capture=tee-sys"