biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[compile] Move torch_aot_compile directory under torch_compile_cache (#34831)

Browse Source 
Signed-off-by: zhxchen17 <zhxchen17@fb.com>
This commit is contained in:
Zhengxu Chen
2026-02-20 11:46:45 -05:00
committed by GitHub
parent a6d0299c75
commit e4a5d8c653
1 changed files with 5 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
vllm/compilation/decorators.py
View File

@@ -407,10 +407,10 @@ def _support_torch_compile(
if envs.VLLM_USE_AOT_COMPILE: if envs.VLLM_USE_AOT_COMPILE:
""" """
When using torch.compile in AOT mode, we store the cache artifacts When using torch.compile in AOT mode, we store the cache artifacts
under VLLM_CACHE_ROOT/torch_aot_compile/{hash}/rank_i_j. The {hash} under VLLM_CACHE_ROOT/torch_compile_cache/torch_aot_compile/{hash}
contains all of the factors except for the source files being The {hash} contains all of the factors except for the source files
traced through, because we don't actually know which source files being traced through, because we don't actually know which source
to check at this point (before dynamo runs). files to check at this point (before dynamo runs).
On loading we will actually look at the source files being traced On loading we will actually look at the source files being traced
through. If any source file have changed (compared with the through. If any source file have changed (compared with the
serialized backend artifacts), then we need to generate a new AOT serialized backend artifacts), then we need to generate a new AOT
@@ -424,6 +424,7 @@ def _support_torch_compile(
hash_key = hashlib.sha256(str(factors).encode()).hexdigest() hash_key = hashlib.sha256(str(factors).encode()).hexdigest()
cache_dir = os.path.join( cache_dir = os.path.join(
envs.VLLM_CACHE_ROOT, envs.VLLM_CACHE_ROOT,
"torch_compile_cache",
"torch_aot_compile", "torch_aot_compile",
hash_key, hash_key,
) )