biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[Bugfix] Add custom Triton cache manager to resolve MoE MP issue (#6140)

Browse Source 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Chih-Chieh-Yang <chih.chieh.yang@ibm.com>
This commit is contained in:
Thomas Parnell
2024-07-15 19:12:47 +02:00
committed by GitHub
parent a63a4c6341
commit eaec4b9153
3 changed files with 64 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
vllm/triton_utils/__init__.py Normal file
View File

@@ -0,0 +1,6 @@
from vllm.triton_utils.custom_cache_manager import (
maybe_set_triton_cache_manager)
__all__ = [
"maybe_set_triton_cache_manager",
]