biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[Bugfix] use right truncation for non-generative tasks (#12050)

Browse Source 
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
This commit is contained in:
Joe Runde
2025-01-15 09:31:01 -07:00
committed by GitHub
parent 57e729e874
commit edce722eaa
3 changed files with 13 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
vllm/transformers_utils/tokenizer_group/__init__.py
View File

@@ -24,7 +24,8 @@ def init_tokenizer_from_configs(model_config: ModelConfig,
max_input_length=None,
tokenizer_mode=model_config.tokenizer_mode,
trust_remote_code=model_config.trust_remote_code,
revision=model_config.tokenizer_revision)
revision=model_config.tokenizer_revision,
truncation_side=model_config.truncation_side)
return get_tokenizer_group(parallel_config.tokenizer_pool_config,
**init_kwargs)