biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[Core] Integrate fastsafetensors loader for loading model weights (#10647)

Browse Source 
Signed-off-by: Manish Sethi <Manish.sethi1@ibm.com>
This commit is contained in:
Manish Sethi
2025-03-24 11:08:02 -04:00
committed by GitHub
parent 9606d572ed
commit 761702fd19
11 changed files with 152 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
docs/source/models/extensions/fastsafetensor.md Normal file
View File

@@ -0,0 +1,5 @@
Loading Model weights with fastsafetensors
===================================================================
Using fastsafetensor library enables loading model weights to GPU memory by leveraging GPU direct storage. See https://github.com/foundation-model-stack/fastsafetensors for more details.
For enabling this feature, set the environment variable ``USE_FASTSAFETENSOR`` to ``true``