biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[TPU][Test] Add script to run benchmark on TPU for buildkite (#19039)

Browse Source 
Signed-off-by: Qiliang Cui <derrhein@gmail.com>
This commit is contained in:
QiliangCui
2025-06-06 20:10:24 -07:00
committed by GitHub
parent 84166fee97
commit 66c508b137
4 changed files with 234 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
.buildkite/scripts/tpu/config_v6e_1.env Normal file
View File

@@ -0,0 +1,14 @@
# Environment config
TEST_NAME=llama8b
CONTAINER_NAME=vllm-tpu
# vllm config
MODEL=meta-llama/Llama-3.1-8B-Instruct
MAX_NUM_SEQS=512
MAX_NUM_BATCHED_TOKENS=512
TENSOR_PARALLEL_SIZE=1
MAX_MODEL_LEN=2048
DOWNLOAD_DIR=/mnt/disks/persist
EXPECTED_THROUGHPUT=8.0
INPUT_LEN=1800
OUTPUT_LEN=128