deepseek-v4-quant/memory/2026-04-14.md

2026-04-14

Kimi-K2.5 Streaming Tool Call Debugging

Mike asked me to diagnose why nvidia/Kimi-K2.5-NVFP4 works for initial tool calls in OpenClaw but doesn't respond to tool results. Ran the model-tool-tests suite and wrote deep-dive tests.

Key Findings

1. Streaming tool call flakiness — With tool_choice=auto + streaming, the model fails to produce tool calls ~60-80% of the time. It reasons about using a tool ("I should use read_file...") then emits a single space + finish_reason=stop. Non-streaming always works (100%). tool_choice=required helps but isn't perfect either (~70-80%).

2. Reasoning not flushed to content on finish_reason=stop — When the model's entire response stays in reasoning (no think-end), the serving.py patch only handles finish_reason=length. This is secondary to bug #1.

3. Root cause: vLLM streaming + Eagle3 speculative decoding. The base model generates correctly (proven by non-streaming), but the streaming path terminates early. The drafter model likely doesn't produce tool-call tokens, causing premature stop.

4. Mitigation: Use tool_choice=required when tools are expected. This forces the engine to continue generating until tool call tokens appear.

Files Changed

• /home/openclaw/dev/model-tool-tests/run_suite.py — added streaming consistency test (#15) and tool_choice=required test (#16)
• /home/openclaw/dev/vllm-kimi25-eagle/STREAMING_BUG.md — full diagnosis doc

Test Results (initial run)

• 23/25 passed (92%)
• Failed: tool call stream, streaming tool: no leaked content
• These are the same underlying bug (streaming tool calls are unreliable)