Commit Graph

8 Commits

Author SHA1 Message Date
3ee933951c Tool parser: fallback to <|tool_call_begin|> when no section marker
Some Kimi K2.5 model variants (nvidia/Kimi-K2.5-NVFP4) omit
<|tool_calls_section_begin|> and go directly to <|tool_call_begin|>.
The tool parser was only looking for section-level markers, so these
tool calls were forwarded as raw content text instead of being parsed.

Fix: _find_section_start and _find_section_start_end now fall back to
<|tool_call_begin|> as a section start when no section-level marker
is found. The section end falls back to <|tool_call_end|>.
2026-04-14 11:25:11 +00:00
c5e6414daf Fix last empty content delta in Case 5 (post-section-close)
Return None instead of DeltaMessage(content='') when delta_text
exists but there's no new content after the section end marker.
2026-04-14 07:08:45 +00:00
a404735b2d Fix empty content deltas and leaked section markers in streaming
Tool parser:
- Case 3/4: return None instead of DeltaMessage(content='') when
  inside an open tool section with no parseable content yet.
  Empty-string content deltas pollute the response and break the
  content=null vs content='' contract with non-streaming.

Reasoning parser:
- Suppress tool-calls section markers from content forwarding.
  The tool parser detects them via current_text re-parsing; forwarding
  them as content causes double-handling.
- Already-past-reasoning path: strip section markers from content
  for the same reason.
2026-04-14 06:46:19 +00:00
fcf8fd134e we need to forward the context using the old way 2026-04-14 06:18:32 +00:00
d0c9c5c482 we actually need the empty deltas to keep the stream going 2026-04-14 05:48:04 +00:00
d4568f1d80 more speculative decoding fixes 2026-04-14 05:06:30 +00:00
d4813de98f fix empty content deltas 2026-04-14 03:49:39 +00:00
9be82d3574 add the tool call parser fixes for eagle decode 2026-04-14 03:13:24 +00:00