biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[Bugfix] Catch and log invalid token ids in detokenizer (#24351)

Browse Source 
Signed-off-by: Nick Hill <nhill@redhat.com>
This commit is contained in:
Nick Hill
2025-09-05 22:30:22 -07:00
committed by GitHub
parent ac201a0eaf
commit 6432739ef1
1 changed files with 5 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
vllm/v1/engine/detokenizer.py
View File

@@ -233,6 +233,11 @@ class FastIncrementalDetokenizer(BaseIncrementalDetokenizer):
def _protected_step(self, next_token_id: int) -> Optional[str]: def _protected_step(self, next_token_id: int) -> Optional[str]:
try: try:
token = self.stream.step(self.tokenizer, next_token_id) token = self.stream.step(self.tokenizer, next_token_id)
except OverflowError:
# Handle rare observed overflow, still to be diagnosed.
# See https://github.com/vllm-project/vllm/issues/21951.
logger.exception("Encountered invalid token id: %d", next_token_id)
token = None
except Exception as e: except Exception as e:
if not str(e).startswith(INVALID_PREFIX_ERR_MSG): if not str(e).startswith(INVALID_PREFIX_ERR_MSG):
raise e raise e