Commit Graph

13 Commits

Author SHA1 Message Date
9051c610d2 Fix reasoning parser for multi-turn conversations
The streaming path was using is_reasoning_end(previous_token_ids) to
check if reasoning had ended. On multi-turn conversations,
previous_token_ids includes the entire chat history, including
think-end tokens from prior assistant messages. This caused the parser
to incorrectly think reasoning was already over before the model
generated anything, routing all thinking text to content instead of
reasoning.

Fix: Replace the token-ID-based check with a text-based state variable
(_reasoning_ended) that tracks reasoning end based solely on what the
model has generated in the current turn. Reset on each new generation.
Also includes the chat template for reference.
2026-04-14 07:46:33 +00:00
c5e6414daf Fix last empty content delta in Case 5 (post-section-close)
Return None instead of DeltaMessage(content='') when delta_text
exists but there's no new content after the section end marker.
2026-04-14 07:08:45 +00:00
a404735b2d Fix empty content deltas and leaked section markers in streaming
Tool parser:
- Case 3/4: return None instead of DeltaMessage(content='') when
  inside an open tool section with no parseable content yet.
  Empty-string content deltas pollute the response and break the
  content=null vs content='' contract with non-streaming.

Reasoning parser:
- Suppress tool-calls section markers from content forwarding.
  The tool parser detects them via current_text re-parsing; forwarding
  them as content causes double-handling.
- Already-past-reasoning path: strip section markers from content
  for the same reason.
2026-04-14 06:46:19 +00:00
fcf8fd134e we need to forward the context using the old way 2026-04-14 06:18:32 +00:00
d0c9c5c482 we actually need the empty deltas to keep the stream going 2026-04-14 05:48:04 +00:00
d4568f1d80 more speculative decoding fixes 2026-04-14 05:06:30 +00:00
d4813de98f fix empty content deltas 2026-04-14 03:49:39 +00:00
72339bfe20 Document speculative decoding tool parser bug and re-parse-and-diff fix 2026-04-14 03:29:20 +00:00
9be82d3574 add the tool call parser fixes for eagle decode 2026-04-14 03:13:24 +00:00
4de7496f5b Add README 2026-04-13 17:28:30 +00:00
ad76c78630 Install unzip for Eagle3 extraction, then remove it 2026-04-13 15:52:29 +00:00
7d2f02a7cf Fix empty TAG fallback in Jenkinsfile 2026-04-13 15:39:13 +00:00
3293502548 init commit 2026-04-13 15:24:48 +00:00