[Misc] Update benchmark_prefix_caching.py fixed example usage (#11920)

Signed-off-by: Ren MinMin <renmm6@chinaunicom.cn> Co-authored-by: Ren MinMin <renmm6@chinaunicom.cn>
2025-01-11 04:39:22 +08:00
parent 46fa98ccad
commit 8a579408f3
1 changed files with 2 additions and 1 deletions
--- a/benchmarks/benchmark_prefix_caching.py
+++ b/benchmarks/benchmark_prefix_caching.py
@@ -10,7 +10,8 @@ Fixed example usage:
        --model meta-llama/Llama-2-7b-chat-hf \
        --enable-prefix-caching \
        --num-prompts 1 \
-        --repeat-count 100
+        --repeat-count 100 \
+        --input-length-range 128:256

 ShareGPT example usage:
    # This command samples 20 prompts with input lengths