biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[torch.compile][misc] fix comments (#10993)

Browse Source 
Signed-off-by: youkaichao <youkaichao@gmail.com>
This commit is contained in:
youkaichao
2024-12-08 11:18:18 -08:00
committed by GitHub
parent a11f326528
commit 43b05fa314
1 changed files with 2 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
vllm/config.py
View File

@@ -2177,8 +2177,8 @@ class CompilationConfig(BaseModel):
TODO: move outside cudagraph logic into compilation. TODO: move outside cudagraph logic into compilation.
torch.compile will handle cudagraph capture logic in the future. torch.compile will handle cudagraph capture logic in the future.
- cudagraph_capture_sizes: sizes to capture cudagraph. - cudagraph_capture_sizes: sizes to capture cudagraph.
- None: capture sizes are inferred from compilation context. - None (default): capture sizes are inferred from vllm config.
- List[int]: capture sizes are specified. - List[int]: capture sizes are specified as given.
- cudagraph_num_of_warmups: number of warmup runs for cudagraph. - cudagraph_num_of_warmups: number of warmup runs for cudagraph.
It means the first several runs will be treated as warmup runs. It means the first several runs will be treated as warmup runs.
Only after that, the execution will be recorded, and the recorded Only after that, the execution will be recorded, and the recorded