biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[Bugfix][SpecDecode] apply sampling parameters to target probabilities for consistency in rejection sampling. (#10198)

Browse Source 
Signed-off-by: jeongin601 <0200angela@gmail.com>
Signed-off-by: jeong_in.bae <jeong_in.bae@navercorp.com>
This commit is contained in:
jeongin601
2024-11-27 14:07:30 +09:00
committed by GitHub
parent 0a4d968500
commit 1bf905ddaa
3 changed files with 10 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
tests/spec_decode/test_batch_expansion.py
View File

@@ -90,6 +90,14 @@ def test_create_single_target_seq_group_metadata(k: int):
)
assert output.request_id == input_seq_group_metadata.request_id
assert output.sampling_params.repetition_penalty == \
input_seq_group_metadata.sampling_params.repetition_penalty
assert output.sampling_params.temperature == \
input_seq_group_metadata.sampling_params.temperature
assert output.sampling_params.top_p == \
input_seq_group_metadata.sampling_params.top_p
assert output.sampling_params.top_k == \
input_seq_group_metadata.sampling_params.top_k
assert len(output.seq_data) == 1
assert output.seq_data[target_seq_id].get_prompt_token_ids() == tuple(
prompt_tokens)