biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[V1][Spec Decoding] Include bonus tokens in mean acceptance length (#17908)

Browse Source 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
This commit is contained in:
Mark McLoughlin
2025-05-09 21:32:36 +01:00
committed by GitHub
parent ea2236bf95
commit 7e3571134f
2 changed files with 9 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
vllm/v1/spec_decode/metrics.py
View File

@@ -73,7 +73,9 @@ class SpecDecodingLogging:
draft_acceptance_rate = (num_accepted_tokens / num_draft_tokens *
100 if num_draft_tokens > 0 else float("nan"))
mean_acceptance_length = (num_accepted_tokens / num_drafts)
# Conventionally, mean acceptance length includes the bonus token
mean_acceptance_length = 1 + (num_accepted_tokens / num_drafts)
pos_matrix = np.array(self.accepted_tokens_per_pos_lists)
acceptance_rates = np.sum(pos_matrix, axis=0) / num_drafts
@@ -103,10 +105,12 @@ class SpecDecodingProm:
rate(vllm:spec_decode_num_accepted_tokens_total[$interval]) /
rate(vllm:spec_decode_num_draft_tokens_total[$interval])
The mean acceptance length can be calculated using:
The mean acceptance length (conventionally including bonus tokens)
can be calculated using:
1 + (
rate(vllm:spec_decode_num_accepted_tokens_total[$interval]) /
rate(vllm:spec_decode_num_drafts[$interval])
rate(vllm:spec_decode_num_drafts[$interval]))
A per-position acceptance rate vector can be computed using