biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[Core] feat: Add --safetensors-load-strategy flag for faster safetensors loading from Lustre (#24469)

Browse Source 
Signed-off-by: Shiqi Sheng <shengshiqi@google.com>
Signed-off-by: shengshiqi-google <160179165+shengshiqi-google@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
This commit is contained in:
shengshiqi-google
2025-09-11 06:10:01 +00:00
committed by GitHub
parent ee0bc5e1b4
commit 41329a0ff9
4 changed files with 31 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
vllm/model_executor/model_loader/default_loader.py
View File

@@ -189,6 +189,7 @@ class DefaultModelLoader(BaseModelLoader):
weights_iterator = safetensors_weights_iterator(
hf_weights_files,
self.load_config.use_tqdm_on_load,
self.load_config.safetensors_load_strategy,
)
else:
if extra_config.get("enable_multithread_load"):