biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[torch.compile] Compile CustomOp.forward_native for SiluAndMul and QuantFP8 to avoid raw torch ops inside opaque custom ops (#32806)

Browse Source 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
This commit is contained in:
Luka Govedič
2026-01-22 22:52:26 -05:00
committed by GitHub
parent f61c9da711
commit 5e4e0e51f4
7 changed files with 52 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
vllm/model_executor/layers/activation.py
View File

@@ -75,8 +75,8 @@ class SiluAndMul(CustomOp):
# --8<-- [end:silu_and_mul]
def __init__(self):
super().__init__()
def __init__(self, *, compile_native: bool = True):
super().__init__(compile_native=compile_native)
if current_platform.is_cuda_alike():
self.op = torch.ops._C.silu_and_mul
elif current_platform.is_xpu():