biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[Bugfix] Fixed mistral tokenizer path when pointing to file (#17457)

Browse Source 
Signed-off-by: Pete Savage <psavage@redhat.com>
This commit is contained in:
Pete Savage
2025-04-30 16:08:37 +01:00
committed by GitHub
parent da4e7687b5
commit 739e03b344
1 changed files with 1 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
vllm/transformers_utils/tokenizers/mistral.py
View File

@@ -227,6 +227,7 @@ class MistralTokenizer(TokenizerBase):
else: else:
assert Path( assert Path(
path_or_repo_id).is_file(), f"Invalid path: {path_or_repo_id}" path_or_repo_id).is_file(), f"Invalid path: {path_or_repo_id}"
tokenizer_file = str(Path(path_or_repo_id))
from mistral_common.tokens.tokenizers.mistral import ( from mistral_common.tokens.tokenizers.mistral import (
MistralTokenizer as PublicMistralTokenizer) MistralTokenizer as PublicMistralTokenizer)