biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[Bugfix] Fix inconsistent handling of cache reset (#33481)

Browse Source 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
This commit is contained in:
Cyrus Leung
2026-02-01 12:23:41 +08:00
committed by GitHub
parent d6416fdde9
commit 79b6ec6aab
7 changed files with 35 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/benchmarking/sweeps.md
View File

@@ -82,7 +82,7 @@ vllm bench sweep serve \
You can use `--dry-run` to preview the commands to be run.
We only start the server once for each `--serve-params`, and keep it running for multiple `--bench-params`.
Between each benchmark run, we call the `/reset_prefix_cache` and `/reset_mm_cache` endpoints to get a clean slate for the next run.
Between each benchmark run, we call all `/reset_*_cache` endpoints to get a clean slate for the next run.
In case you are using a custom `--serve-cmd`, you can override the commands used for resetting the state by setting `--after-bench-cmd`.
!!! note