biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[Feat] Drop-in Torch CUDA Profiler (#27841)

Browse Source 
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
This commit is contained in:
Benjamin Chislett
2025-11-08 17:07:37 -05:00
committed by GitHub
parent 77d702a22b
commit 975676d174
5 changed files with 76 additions and 29 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
vllm/entrypoints/openai/api_server.py
View File

@@ -1280,10 +1280,16 @@ async def invocations(raw_request: Request):
if envs.VLLM_TORCH_PROFILER_DIR:
logger.warning(
logger.warning_once(
"Torch Profiler is enabled in the API server. This should ONLY be "
"used for local development!"
)
elif envs.VLLM_TORCH_CUDA_PROFILE:
logger.warning_once(
"CUDA Profiler is enabled in the API server. This should ONLY be "
"used for local development!"
)
if envs.VLLM_TORCH_PROFILER_DIR or envs.VLLM_TORCH_CUDA_PROFILE:
@router.post("/start_profile")
async def start_profile(raw_request: Request):