biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[Bugfix] Add safety check and fallback for null scaling factor (#36106)

Browse Source 
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
This commit is contained in:
Yuanheng Zhao
2026-03-16 22:27:29 +08:00
committed by GitHub
parent e855d380fa
commit 8d8855fdae
1 changed files with 9 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
vllm/config/model.py
View File

@@ -2021,6 +2021,15 @@ def _get_and_verify_max_len(
if rope_type == "yarn": if rope_type == "yarn":
derived_max_model_len = rp["original_max_position_embeddings"] derived_max_model_len = rp["original_max_position_embeddings"]
if scaling_factor is None:
# Fallback the factor to 1.0 if a user assigned `null`
logger.warning_once(
"The model's RoPE configuration has a null scaling "
"factor which is unexpected. This likely indicates a bug "
"in the model's HuggingFace config.json. Please notify the "
"model vendor. Falling back the value to 1.0. "
)
scaling_factor = 1.0
# Do this outside loop since all layer types should have the same scaling # Do this outside loop since all layer types should have the same scaling
derived_max_model_len *= scaling_factor derived_max_model_len *= scaling_factor