biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

Fix repetition penalty aligned with huggingface (#1577)

Browse Source
This commit is contained in:
ljss
2023-11-23 06:41:44 +08:00
committed by GitHub
parent 4cea74c73b
commit de23687d16
2 changed files with 50 additions and 32 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
vllm/sampling_params.py
View File

@@ -42,9 +42,9 @@ class SamplingParams:
model to use new tokens, while values < 0 encourage the model to
repeat tokens.
repetition_penalty: Float that penalizes new tokens based on whether
they appear in the generated text so far. Values > 1 encourage the
model to use new tokens, while values < 1 encourage the model to
repeat tokens.
they appear in the prompt and the generated text so far. Values > 1
encourage the model to use new tokens, while values < 1 encourage
the model to repeat tokens.
temperature: Float that controls the randomness of the sampling. Lower
values make the model more deterministic, while higher values make
the model more random. Zero means greedy sampling.