biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[ROCm] Fix AttributeError for torch.compiler.skip_all_guards_unsafe on older PyTorch (#37219)

Browse Source 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
(cherry picked from commit 54a62a79f7)
This commit is contained in:
Andreas Karatzas
2026-03-16 22:34:49 -05:00
committed by khluu
parent 2dccb38f73
commit 1fe3932c8b
1 changed files with 6 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
vllm/compilation/wrapper.py
View File

@@ -112,7 +112,12 @@ class TorchCompileWithNoGuardsWrapper:
entry.guard_type == "SHAPE_ENV" for entry in x entry.guard_type == "SHAPE_ENV" for entry in x
] ]
else: else:
options["guard_filter_fn"] = torch.compiler.skip_all_guards_unsafe if hasattr(torch.compiler, "skip_all_guards_unsafe"):
# Torch 2.10+ provides skip_all_guards_unsafe
options["guard_filter_fn"] = torch.compiler.skip_all_guards_unsafe
else:
# Equivalent fallback for older PyTorch: skip all guards
options["guard_filter_fn"] = lambda x: [False for _ in x]
compiled_ptr: Any = self.forward compiled_ptr: Any = self.forward
# Validate that unbacked dynamic shapes require VLLM_USE_BYTECODE_HOOK=False # Validate that unbacked dynamic shapes require VLLM_USE_BYTECODE_HOOK=False