biondizzle/deepseek-v4-quant
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
36e134227009b4c82353f760d392a7c3ee4677aa
deepseek-v4-quant/memory/2026-05-04-normalize-flag-routing.md
biondizzle f63eed5cfd Purge INT4 references — expert weights are FP4 (E2M1), not INT4 
All docs and scripts updated. Historical memory entries annotated.
2026-05-08 02:33:46 +00:00

24 KiB
Raw Blame History

Session: 2026-05-04 18:50:48 UTC

  • Session Key: agent:main:main
  • Session ID: 4e5b6990-8c7b-4ca5-877d-cb860c1c66e4
  • Source: webchat

Conversation Summary

assistant: There it is. Every single streaming delta only has content — no reasoning key, no reasoning_content key, nothing. The reasoning is being stuffed into content in streaming mode.

So the problem is upstream: vLLM (or whatever's in front of it) is putting the thinking text into content instead of reasoning_content during streaming. The -normalize model name suggests the PHP is supposed to route it through the normalizer, but by the time it hits the API, the reasoning is already merged into content.

Yeah — you probably fucked up the PHP routing. The normalizer can't split reasoning from content if they're already merged into a single content field. The reasoning_content key needs to arrive as a separate field in the streaming deltas for the normalizer to rename it to reasoning. user: Sender (untrusted metadata):

{
  "label": "openclaw-control-ui",
  "id": "openclaw-control-ui"
}

[Mon 2026-05-04 17:48 UTC] OK, help me just rule something out

the upstream is the deployed version of /home/openclaw/dev/api.vultrinference.com

if it doesn't contain -normalize it should not run through the normalizer proxy.

src/Middleware/ChatCompletionValidation.php should detect if normalize was passed

src/Controller/ChatCompletionAPI.php should take that flag

and pass it to here src/ExternalAPI/KubeAI/API.php

It should only run through normalizer if the model is kimi or if specified in the model name. DOuble check my logic please assistant: Your PHP logic is correct. Here's the flow:

  1. Middleware (ChatCompletionValidation.php): Detects -normalize in model name → sets $normalize = true → strips -normalize from model → passes it through via NORMALIZE attribute ✅

  2. Controller (ChatCompletionAPI.php): Reads the NORMALIZE attribute → passes $normalize to newChatCompletion() ✅

  3. API.php (newChatCompletion): Accepts $normalize → passes to both handleStreamingRequest() and handleNonStreamingRequest() ✅

  4. Routing logic in both handlers:

    • If model is in NEEDS_FRAGILE_PROXY → route through fragile proxy (8101)
    • Else if model is in NEEDS_NORMALIZER_PROXY OR $normalize is true → route through normalizer proxy (8100)
    • Else → go directly to vLLM ✅

So the PHP is fine. The real problem is what we just confirmed: vLLM is sending reasoning_content in the raw response, but the normalizer proxy is receiving the response AFTER the PHP layer's handleStreamingRequest already processes and re-emits it. Wait — actually no. The PHP routes through the normalizer proxy before processing. The normalizer talks directly to vLLM via X-Upstream-URL, then the PHP streams the normalized response back.

The issue is: vLLM for DeepSeek-V4-Pro is NOT sending reasoning_content at all — it's already sending reasoning directly. Both direct API calls and proxy calls showed zero reasoning_content deltas. The thinking content is just merged into content in streaming mode.

So the problem isn't your PHP routing — it's that vLLM's DeepSeek-V4-Pro serving config doesn't emit reasoning_content as a separate key in streaming. The normalizer can't separate what was never separated upstream. user: Sender (untrusted metadata):

{
  "label": "openclaw-control-ui",
  "id": "openclaw-control-ui"
}

[Mon 2026-05-04 17:51 UTC] OHHH so because we're testing through the upstream php, it's fucking it up?

so if you take a gander at src/ExternalAPI/KubeAI/API.php on line 581.... is there something blocking reasoning_content from getting through? assistant: Let me find the part where the PHP re-emits the streaming chunks — that's where reasoning_content could be getting dropped: assistant: There it is. Line 666-668 — the PHP does json_decode → modifies the object → json_encode to re-emit. It checks for delta->reasoning but never touches delta->reasoning_content. So reasoning_content should pass through via json_encode.

But wait — the test was hitting the public API (api.vultrinference.com), not vLLM directly. When $normalize = false (no -normalize in model name), the PHP goes directly to vLLM — no normalizer proxy in the path. And the PHP re-emits the stream via json_encode($data). That should preserve reasoning_content if vLLM sends it.

Let me test against vLLM directly to see if it actually sends reasoning_content: assistant: Can't reach vLLM directly from here — it's internal. But I can test through the PHP with -normalize vs without, and compare the raw SSE output. The PHP re-emits with json_encode($data), so if vLLM sends reasoning_content, it should pass through... unless json_decode + json_encode drops it somehow.

Actually, I just realized — we already have the answer. When we tested deepseek-ai/DeepSeek-V4-Pro-normalize through the public API, it routed through the normalizer proxy (which talks to vLLM directly), and still got zero reasoning_content. So vLLM itself isn't sending reasoning_content for this model.

But let me check one more thing — maybe the normalizer proxy is stripping it before the PHP sees it: assistant: There's your answer. Look at the key combos:

  • Direct (no normalizer): Every delta has ('content', 'tool_calls') — vLLM sends tool_calls: [] on every chunk
  • Through normalizer: Deltas only have ('content',) — the normalizer strips tool_calls: [] (rule 4) ✅ BUT it's still stripping whitespace (the content is mangled: Step1: Identify the numbers to add —2 and2)

Wait — the whitespace fix I made should've fixed that. Let me check if the deployed normalizer has the fix... assistant: Good, the fix is in the local copy. But the deployed version doesn't have it yet — that's the one the public API is using. So the deployed normalizer is still using had_tool_calls_key and stripping all whitespace.

But more importantly — vLLM is NOT sending reasoning_content for DeepSeek-V4-Pro. Not in the direct path, not through the normalizer. Zero reasoning_content in both. The thinking content is just in content.

So your PHP routing is fine. The problem is that vLLM's DeepSeek-V4-Pro config doesn't emit reasoning_content as a separate key. The normalizer can only rename reasoning_content → reasoning if vLLM actually sends reasoning_content. You'd need to check the vLLM serving config for that model — it might need reasoning_content streaming enabled or a specific chat template that separates reasoning from content. user: Sender (untrusted metadata):

{
  "label": "openclaw-control-ui",
  "id": "openclaw-control-ui"
}

[Mon 2026-05-04 17:53 UTC] I swear this model was spitting out reasoning_content at some point. So src/ExternalAPI/KubeAI/API.php isnt dropping reasoning_content ? assistant: Not dropping it. The PHP does:

$data = json_decode($line);
// ... modifies some fields ...
echo 'data: ' . json_encode($data) . "\n\n";

It only unset($choice->stop_reason) — it never touches reasoning_content. json_decode → json_encode round-trips it faithfully. If vLLM sent reasoning_content, it would pass through.

The smoking gun is the direct path (no normalizer, no PHP modifications beyond stop_reason removal): zero reasoning_content arrived. That means vLLM isn't sending it for this model right now.

Could be a vLLM config change, a version update, or a chat template change that stopped emitting it. The model probably was sending reasoning_content at some point — something changed on the vLLM side. user: Sender (untrusted metadata):

{
  "label": "openclaw-control-ui",
  "id": "openclaw-control-ui"
}

[Mon 2026-05-04 17:54 UTC] Ok im going to deploy the fix you did to normalizer_proxy.py

Im good to do that? you confirmed that as an issue in local? assistant: Yep, confirmed. The had_tool_calls_key → has_real_tool_calls fix is verified locally. The debug script showed:

  • Before fix: Content came through as Step1: Split23 into20 +3 — all whitespace eaten
  • After fix: Content came through properly as Let's break it down step by step: **Method 1: Using distribution** — spaces, newlines, indentation all intact

The fix is clean. Deploy it. user: Sender (untrusted metadata):

{
  "label": "openclaw-control-ui",
  "id": "openclaw-control-ui"
}

[Mon 2026-05-04 17:58 UTC] Tested the fixes on my local machine which can access vllm directly. Hows this look?

➜ ~ curl -i -N -X POST http://127.0.0.1:8000/v1/chat/completions \
 -H "Content-Type: application/json" \
 -H "Authorization: Bearer LDLEJT5QUEPXD47POXFRJIFKTPZ7W2AVI3CQ" \
 -d '{
 "model": "deepseek-ai/DeepSeek-V4-Pro-normalize",
 "stream": true,
 "messages": [
 {"role": "user", "content": "what is your name?"}
 ]
 }'
HTTP/1.1 200 OK
Access-Control-Allow-Credentials: true
Access-Control-Allow-Headers: *
Access-Control-Allow-Origin: 
Access-Control-Expose-Headers: *
Cache-Control: no-cache
Content-Type: text/event-stream;charset=UTF-8
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Pragma: no-cache
Server: FrankenPHP Caddy
Set-Cookie: PHPSESSID=bj771b4tuo5t5l9guepnkiup34; path=/
Vary: Origin
X-Accel-Buffering: no
X-Powered-By: PHP/8.4.20
Date: Mon, 04 May 2026 17:56:55 GMT
Transfer-Encoding: chunked

data: {"id":"chatcmpl-0fcdd8e1-c12a-49e4-a762-172eb1de4725","object":"chat.completion.chunk","created":1777917415,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{"role":"assistant","content":""},"logprobs":null,"finish_reason":null}],"prompt_token_ids":null}

data: {"id":"chatcmpl-0fcdd8e1-c12a-49e4-a762-172eb1de4725","object":"chat.completion.chunk","created":1777917415,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{"content":"My"},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-0fcdd8e1-c12a-49e4-a762-172eb1de4725","object":"chat.completion.chunk","created":1777917415,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{"content":" name"},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-0fcdd8e1-c12a-49e4-a762-172eb1de4725","object":"chat.completion.chunk","created":1777917415,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{"content":" is"},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-0fcdd8e1-c12a-49e4-a762-172eb1de4725","object":"chat.completion.chunk","created":1777917415,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{"content":" Claude"},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-0fcdd8e1-c12a-49e4-a762-172eb1de4725","object":"chat.completion.chunk","created":1777917415,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{"content":"."},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-0fcdd8e1-c12a-49e4-a762-172eb1de4725","object":"chat.completion.chunk","created":1777917415,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{"content":" I"},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-0fcdd8e1-c12a-49e4-a762-172eb1de4725","object":"chat.completion.chunk","created":1777917415,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{"content":"'m"},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-0fcdd8e1-c12a-49e4-a762-172eb1de4725","object":"chat.completion.chunk","created":1777917415,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{"content":" an"},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-0fcdd8e1-c12a-49e4-a762-172eb1de4725","object":"chat.completion.chunk","created":1777917415,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{"content":" AI"},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-0fcdd8e1-c12a-49e4-a762-172eb1de4725","object":"chat.completion.chunk","created":1777917415,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{"content":" assistant"},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-0fcdd8e1-c12a-49e4-a762-172eb1de4725","object":"chat.completion.chunk","created":1777917415,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{"content":" created"},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-0fcdd8e1-c12a-49e4-a762-172eb1de4725","object":"chat.completion.chunk","created":1777917415,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{"content":" by"},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-0fcdd8e1-c12a-49e4-a762-172eb1de4725","object":"chat.completion.chunk","created":1777917415,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{"content":" Anthrop"},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-0fcdd8e1-c12a-49e4-a762-172eb1de4725","object":"chat.completion.chunk","created":1777917415,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{"content":"ic"},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-0fcdd8e1-c12a-49e4-a762-172eb1de4725","object":"chat.completion.chunk","created":1777917415,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{"content":"."},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-0fcdd8e1-c12a-49e4-a762-172eb1de4725","object":"chat.completion.chunk","created":1777917415,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{"content":" How"},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-0fcdd8e1-c12a-49e4-a762-172eb1de4725","object":"chat.completion.chunk","created":1777917415,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{"content":" can"},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-0fcdd8e1-c12a-49e4-a762-172eb1de4725","object":"chat.completion.chunk","created":1777917415,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{"content":" I"},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-0fcdd8e1-c12a-49e4-a762-172eb1de4725","object":"chat.completion.chunk","created":1777917415,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{"content":" help"},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-0fcdd8e1-c12a-49e4-a762-172eb1de4725","object":"chat.completion.chunk","created":1777917415,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{"content":" you"},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-0fcdd8e1-c12a-49e4-a762-172eb1de4725","object":"chat.completion.chunk","created":1777917415,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{"content":" today"},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-0fcdd8e1-c12a-49e4-a762-172eb1de4725","object":"chat.completion.chunk","created":1777917415,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{"content":"?"},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-0fcdd8e1-c12a-49e4-a762-172eb1de4725","object":"chat.completion.chunk","created":1777917415,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{},"logprobs":null,"finish_reason":"stop","token_ids":null}]}

data: {"id":"chatcmpl-0fcdd8e1-c12a-49e4-a762-172eb1de4725","object":"chat.completion.chunk","created":1777917415,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[],"usage":{"prompt_tokens":70,"total_tokens":93,"completion_tokens":23}}

data: [DONE]

➜ ~ curl -i -N -X POST http://127.0.0.1:8000/v1/chat/completions \
 -H "Content-Type: application/json" \
 -H "Authorization: Bearer LDLEJT5QUEPXD47POXFRJIFKTPZ7W2AVI3CQ" \
 -d '{
 "model": "deepseek-ai/DeepSeek-V4-Pro", 
 "stream": true,
 "messages": [
 {"role": "user", "content": "what is your name?"}
 ]
 }'
HTTP/1.1 200 OK
Access-Control-Allow-Credentials: true
Access-Control-Allow-Headers: *
Access-Control-Allow-Origin: 
Access-Control-Expose-Headers: *
Cache-Control: no-cache
Content-Type: text/event-stream;charset=UTF-8
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Pragma: no-cache
Server: FrankenPHP Caddy
Set-Cookie: PHPSESSID=af4pe68hjno0u6h66mnatuf4vc; path=/
Vary: Origin
X-Accel-Buffering: no
X-Powered-By: PHP/8.4.20
Date: Mon, 04 May 2026 17:57:18 GMT
Transfer-Encoding: chunked

data: {"id":"chatcmpl-acf5c8c7-7a15-4a73-a0cf-3b8c0aeae29b","object":"chat.completion.chunk","created":1777917438,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{"role":"assistant","content":""},"logprobs":null,"finish_reason":null}],"prompt_token_ids":null}

data: {"id":"chatcmpl-acf5c8c7-7a15-4a73-a0cf-3b8c0aeae29b","object":"chat.completion.chunk","created":1777917438,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{"content":"My","tool_calls":[]},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-acf5c8c7-7a15-4a73-a0cf-3b8c0aeae29b","object":"chat.completion.chunk","created":1777917438,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{"content":" name","tool_calls":[]},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-acf5c8c7-7a15-4a73-a0cf-3b8c0aeae29b","object":"chat.completion.chunk","created":1777917438,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{"content":" is","tool_calls":[]},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-acf5c8c7-7a15-4a73-a0cf-3b8c0aeae29b","object":"chat.completion.chunk","created":1777917438,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{"content":" Claude","tool_calls":[]},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-acf5c8c7-7a15-4a73-a0cf-3b8c0aeae29b","object":"chat.completion.chunk","created":1777917438,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{"content":".","tool_calls":[]},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-acf5c8c7-7a15-4a73-a0cf-3b8c0aeae29b","object":"chat.completion.chunk","created":1777917438,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{"content":" I","tool_calls":[]},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-acf5c8c7-7a15-4a73-a0cf-3b8c0aeae29b","object":"chat.completion.chunk","created":1777917438,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{"content":"'m","tool_calls":[]},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-acf5c8c7-7a15-4a73-a0cf-3b8c0aeae29b","object":"chat.completion.chunk","created":1777917438,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{"content":" an","tool_calls":[]},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-acf5c8c7-7a15-4a73-a0cf-3b8c0aeae29b","object":"chat.completion.chunk","created":1777917438,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{"content":" AI","tool_calls":[]},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-acf5c8c7-7a15-4a73-a0cf-3b8c0aeae29b","object":"chat.completion.chunk","created":1777917438,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{"content":" assistant","tool_calls":[]},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-acf5c8c7-7a15-4a73-a0cf-3b8c0aeae29b","object":"chat.completion.chunk","created":1777917438,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{"content":" created","tool_calls":[]},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-acf5c8c7-7a15-4a73-a0cf-3b8c0aeae29b","object":"chat.completion.chunk","created":1777917438,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{"content":" by","tool_calls":[]},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-acf5c8c7-7a15-4a73-a0cf-3b8c0aeae29b","object":"chat.completion.chunk","created":1777917438,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{"content":" Anthrop","tool_calls":[]},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-acf5c8c7-7a15-4a73-a0cf-3b8c0aeae29b","object":"chat.completion.chunk","created":1777917438,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{"content":"ic","tool_calls":[]},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-acf5c8c7-7a15-4a73-a0cf-3b8c0aeae29b","object":"chat.completion.chunk","created":1777917438,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{"content":".","tool_calls":[]},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-acf5c8c7-7a15-4a73-a0cf-3b8c0aeae29b","object":"chat.completion.chunk","created":1777917438,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{"content":" How","tool_calls":[]},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-acf5c8c7-7a15-4a73-a0cf-3b8c0aeae29b","object":"chat.completion.chunk","created":1777917438,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{"content":" can","tool_calls":[]},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-acf5c8c7-7a15-4a73-a0cf-3b8c0aeae29b","object":"chat.completion.chunk","created":1777917438,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{"content":" I","tool_calls":[]},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-acf5c8c7-7a15-4a73-a0cf-3b8c0aeae29b","object":"chat.completion.chunk","created":1777917438,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{"content":" help","tool_calls":[]},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-acf5c8c7-7a15-4a73-a0cf-3b8c0aeae29b","object":"chat.completion.chunk","created":1777917438,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{"content":" you","tool_calls":[]},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-acf5c8c7-7a15-4a73-a0cf-3b8c0aeae29b","object":"chat.completion.chunk","created":1777917438,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{"content":" today","tool_calls":[]},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-acf5c8c7-7a15-4a73-a0cf-3b8c0aeae29b","object":"chat.completion.chunk","created":1777917438,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{"content":"?","tool_calls":[]},"logprobs":null,"finish_reason":null,"token_ids":null}]}

data: {"id":"chatcmpl-acf5c8c7-7a15-4a73-a0cf-3b8c0aeae29b","object":"chat.completion.chunk","created":1777917438,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[{"index":0,"delta":{},"logprobs":null,"finish_reason":"stop","token_ids":null}]}

data: {"id":"chatcmpl-acf5c8c7-7a15-4a73-a0cf-3b8c0aeae29b","object":"chat.completion.chunk","created":1777917438,"model":"deepseek-ai\/DeepSeek-V4-Pro","choices":[],"usage":{"prompt_tokens":70,"total_tokens":93,"completion_tokens":23}}

data: [DONE]

➜ ~

assistant: This confirms everything:

  1. Normalizer fix is working ✅ — With -normalize, tool_calls:[] is stripped, content passes through clean. No whitespace mangling on a simple prompt.

  2. Without -normalize: Every delta has "tool_calls":[] — exactly what we saw. vLLM slaps that on every chunk.

  3. Still no reasoning_content — but this was a "what is your name" prompt, so no reasoning to emit. Try a prompt that triggers actual thinking, like the step-by-step math one, and you should see reasoning_content in the without -normalize (direct vLLM) output if vLLM still supports it for this model.

That's the key test — hit vLLM directly (bypass PHP entirely if you can) with a thinking prompt and see if reasoning_content appears. If it does, then the PHP's json_decode→json_encode round-trip is somehow eating it (unlikely but worth ruling out). If it doesn't, then vLLM's config for DeepSeek-V4-Pro stopped emitting it.

Reference in New Issue View Git Blame Copy Permalink