biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

Let max_num_batched_tokens use human_readable_int for large numbers (#18968)

Browse Source 
Signed-off-by: mgoin <mgoin64@gmail.com>
This commit is contained in:
Michael Goin
2025-05-31 23:41:29 -04:00
committed by GitHub
parent c594cbf565
commit 2ad6194a02
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
vllm/engine/arg_utils.py
View File

@@ -224,7 +224,7 @@ def get_kwargs(cls: ConfigType) -> dict[str, Any]:
elif contains_type(type_hints, int): elif contains_type(type_hints, int):
kwargs[name]["type"] = int kwargs[name]["type"] = int
# Special case for large integers # Special case for large integers
if name in {"max_model_len"}: if name in {"max_model_len", "max_num_batched_tokens"}:
kwargs[name]["type"] = human_readable_int kwargs[name]["type"] = human_readable_int
elif contains_type(type_hints, float): elif contains_type(type_hints, float):
kwargs[name]["type"] = float kwargs[name]["type"] = float