biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[Fix] Benchmark "EngineClient" has no attribute "model_config" (#17976)

Browse Source 
Signed-off-by: Brayden Zhong <b8zhong@uwaterloo.ca>
This commit is contained in:
Brayden Zhong
2025-05-12 01:55:53 -04:00
committed by GitHub
parent 430783018c
commit 891b9d33de
2 changed files with 7 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
vllm/benchmarks/throughput.py
View File

@@ -148,9 +148,10 @@ async def run_vllm_async(
async with build_async_engine_client_from_engine_args(
engine_args, disable_frontend_multiprocessing) as llm:
model_config = await llm.get_model_config()
assert all(
llm.model_config.max_model_len >= (request.prompt_len +
request.expected_output_len)
model_config.max_model_len >= (request.prompt_len +
request.expected_output_len)
for request in requests), (
"Please ensure that max_model_len is greater than the sum of"
" prompt_len and expected_output_len for all requests.")