[torchao] safetensors integration (#25969)

Signed-off-by: Angel Li <liangel@meta.com>
This commit is contained in:
liangel-02
2025-10-07 19:12:35 -07:00
committed by GitHub
parent f80e7866c0
commit b32260ab85
5 changed files with 60 additions and 0 deletions

View File

@@ -59,6 +59,10 @@ class LoadConfig:
This is recommended for models on network filesystems (e.g., Lustre, NFS)
as it avoids inefficient random reads, significantly speeding up model
initialization. However, it uses more CPU RAM.
- "torchao": Weights are loaded in upfront and then reconstructed
into torchao tensor subclasses. This is used when the checkpoint
was quantized using torchao and saved using safetensors.
Needs torchao >= 0.14.0
"""
model_loader_extra_config: Union[dict, TensorizerConfig] = field(
default_factory=dict