biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[UX] Enable torch_profiler_with_stack (#37571)

Browse Source 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
This commit is contained in:
Jee Jee Li
2026-03-20 19:17:26 +08:00
committed by GitHub
parent 0523449c9c
commit dd20ee4e3e
2 changed files with 8 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
docs/contributing/profiling.md
View File

@@ -3,6 +3,10 @@
!!! warning !!! warning
Profiling is only intended for vLLM developers and maintainers to understand the proportion of time spent in different parts of the codebase. **vLLM end-users should never turn on profiling** as it will significantly slow down the inference. Profiling is only intended for vLLM developers and maintainers to understand the proportion of time spent in different parts of the codebase. **vLLM end-users should never turn on profiling** as it will significantly slow down the inference.
!!! tip "Choosing a profiler"
- Use **Nsight Systems** for low-overhead, performance-critical profiling.
- Use **PyTorch Profiler** for medium-overhead profiling with richer debugging information (e.g., stack traces, memory, shapes). Note that enabling these features adds overhead and is not recommended for benchmarking.
## Profile with PyTorch Profiler ## Profile with PyTorch Profiler
We support tracing vLLM workers using different profilers. You can enable profiling by setting the `--profiler-config` flag when launching the server. We support tracing vLLM workers using different profilers. You can enable profiling by setting the `--profiler-config` flag when launching the server.

8
vllm/config/profiler.py
View File

@@ -45,10 +45,10 @@ class ProfilerConfig:
worker's traces (CPU & GPU) will be saved under this directory. Note that worker's traces (CPU & GPU) will be saved under this directory. Note that
it must be an absolute path.""" it must be an absolute path."""
torch_profiler_with_stack: bool = False torch_profiler_with_stack: bool = True
"""If `True`, enables stack tracing in the torch profiler. Disabled by default """If `True`, enables stack tracing in the torch profiler. Enabled by default
to reduce overhead. Can be enabled via VLLM_TORCH_PROFILER_WITH_STACK=1 env var as it is useful for debugging. Can be disabled via
or --profiler-config.torch_profiler_with_stack=true CLI flag.""" --profiler-config.torch_profiler_with_stack=false CLI flag."""
torch_profiler_with_flops: bool = False torch_profiler_with_flops: bool = False
"""If `True`, enables FLOPS counting in the torch profiler. Disabled by default.""" """If `True`, enables FLOPS counting in the torch profiler. Disabled by default."""