[Structured Output][Reasoning] Improves decoding throughput for models using single-token reasoning endings. (#30056)

This commit is contained in:
Hubert de La Jonquiere
2025-12-09 11:54:08 +01:00
committed by GitHub
parent 67475a6e81
commit c72ea10723
10 changed files with 89 additions and 1 deletions

View File

@@ -299,6 +299,9 @@ Additionally, to enable structured output, you'll need to create a new `Reasoner
def is_reasoning_end(self, input_ids: list[int]) -> bool:
return self.end_token_id in input_ids
def is_reasoning_end_streaming(self, input_ids: list[int], delta_ids: list[int]) -> bool:
return self.end_token_id in delta_token_ids
...
```