biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

add more pytorch related tests for torch nightly (#17422)

Browse Source 
Signed-off-by: Yang Wang <elainewy@meta.com>
This commit is contained in:
Yang Wang
2025-05-02 03:29:59 -07:00
committed by GitHub
parent d7543862bd
commit b8b0859b5c
3 changed files with 14 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
vllm/sampling_params.py
View File

@@ -186,9 +186,9 @@ class SamplingParams(
logits_processors: list of functions that modify logits based on
previously generated tokens, and optionally prompt tokens as
a first argument.
truncate_prompt_tokens: If set to -1, will use the truncation size
supported by the model. If set to an integer k, will use only
the last k tokens from the prompt (i.e., left truncation).
truncate_prompt_tokens: If set to -1, will use the truncation size
supported by the model. If set to an integer k, will use only
the last k tokens from the prompt (i.e., left truncation).
Defaults to None (i.e., no truncation).
guided_decoding: If provided, the engine will construct a guided
decoding logits processor from these parameters. Defaults to None.