deepseek-v4-quant/memory/2026-04-20-api-digest.md

Session: 2026-04-20 18:48:35 UTC

• Session Key: agent:main:main
• Session ID: a948e713-0735-4c83-af97-5d1c0d00ddbc
• Source: webchat

Conversation Summary

user: A new session was started via /new or /reset. Run your Session Startup sequence - read the required files before responding to the user. Then greet the user in your configured persona, if one is provided. Be yourself - use your defined voice, mannerisms, and mood. Keep it to 1-3 sentences and ask what they want to do. If the runtime model differs from default_model in the system prompt, mention the default model. Do not mention internal steps, files, tools, or reasoning. Current time: Monday, April 20th, 2026 - 4:16 PM (UTC) / 2026-04-20 16:16 UTC assistant: Yo Mike. Vinny, back again — Monday afternoon, ready to roll. What's on the docket? user: Sender (untrusted metadata):

{
  "label": "openclaw-control-ui",
  "id": "openclaw-control-ui"
}

[Mon 2026-04-20 16:20 UTC] head on over to the project at /home/openclaw/dev/api.vultrinference.com particularly the file at /home/openclaw/dev/api.vultrinference.com/src/ExternalAPI/KubeAI/API.php

Just digest what it does assistant: Alright, here's the breakdown:

API.php is the PHP proxy layer between the Vultr Inference API and the backend vLLM clusters. It does four main things:

1. Proxies requests to vLLM — Supports two OpenAI-compatible endpoints:

   • /v1/responses (the newer Responses API) via newResponse()
   • /v1/chat/completions (the classic Chat Completions API) via newChatCompletion()
   • Also listModels() which hits /v1/models

2. Handles both streaming and non-streaming — Each endpoint has a streaming handler (returns a StreamedResponse, manually parses SSE chunks with a line buffer for TCP-split safety) and a non-streaming handler (straight HTTP response). Streaming manually counts tokens from deltas as they come in, but defers to vLLM's usage block if one arrives (which is more accurate).

3. Tracks usage/billing — trackStreamingUsage() and inline calls to UsageHelper::incrementTokensForSubscriptionByModel() log token counts (input, output, reasoning) and request duration against the subscriber's quota. For streaming, if vLLM never sends usage, it falls back to TokenEstimatorHelper to estimate input tokens and the manual delta count for output.

4. Security/hygiene:

   • Strips internal fields (like collection) before forwarding to vLLM
   • Sanitizes error messages — strips internal URLs/hostnames via regex so vLLM internals don't leak to clients
   • Sanitizes Sentry payloads — truncates message content, redacts image URLs, to avoid sending PII/large data to error tracking
   • Strips vLLM-specific fields like stop_reason and prompt_logprobs from responses
   • Coerces string bools ("true" → true) since JSON can arrive mangled

Notable quirks:

• Auth header to vLLM is commented out — no Bearer token being sent
• Both cluster URLs (primary and atl) point to the same host in prod vs non-prod — looks like the ATL cluster isn't differentiated yet, or the env-specific logic is stubbed
• The ob_end_flush() / ob_start() dance in streaming is a FrankenPHP workaround
• Streaming uses register_shutdown_function for usage tracking to avoid blocking the SSE stream

Solid proxy layer. What do you wanna do with it? user: Sender (untrusted metadata):

{
  "label": "openclaw-control-ui",
  "id": "openclaw-control-ui"
}

[Mon 2026-04-20 16:26 UTC] In Sentry I occasionally get this error coming from this file:

Idle timeout reached for "http://prod.vllm.vultrinference.com/v1/chat/completions".

Symfony\Component\HttpClient\Exception\ServerException: HTTP/1.1 500 Internal Server Error returned for "http://prod.vllm.vultrinference.com/v1/chat/completions".

request_payload (dont forget it's truncated in the sentry logs):

{'model': 'MiniMaxAI/MiniMax-M2.7', 'messages': [{'role': 'system', 'content': 'You are a context summarization assistant. Your task is to read a conversation between a user and an...'}, {'role': 'user', 'content': [{'type': 'text', 'text': '<conversation>\n[User]: Sender (untrusted metadata):\n```json\n{\n "label": "openclaw-control-ui",\n "i...'}]}], 'stream': True, 'store': False, 'max_completion_tokens': 8192, 'stream_options': {'include_usage': True}}

response_payload:'message': 'HTTP/1.1 500 Internal Server Error returned for "'}

Don't actually do anything yet. Just look at the chain of events and speculate why this may be happening. You're literally running through this middleware layer so it works majority of the time. I havent experienced the issue first hand. Could be a number of things. Bad payload field are usually the main culprits. If anything stands out. Let me know assistant: Looking at the payload, a couple things jump out:

1. store: false — This is an OpenAI API field for persisting completions. vLLM likely doesn't support it, and depending on the version/config, it might reject it with a 500 instead of ignoring it. This is a prime suspect for the 500. It's being forwarded straight through because it's not in INTERNAL_FIELDS.

2. stream_options: {include_usage: true} — You're injecting this in newChatCompletion() whenever stream is true. That's correct per the OpenAI spec, but older vLLM versions (or certain model backends) may not recognize it and choke. Notably, handleStreamingResponse() (the /v1/responses path) does not add this — only the chat completions path does.

3. The idle timeout is probably a different beast — likely vLLM just being slow. Cold model start, request queue backing up, MiniMax-M2.7 taking a while to load. The 300s timeout is generous but not infinite. If it's intermittent, this is probably just load-related rather than payload-related.

My read: The 500 is most likely store: false or stream_options hitting a vLLM version that doesn't tolerate unknown fields. The idle timeout is probably just vLLM being under pressure. Two different root causes for what looks like the same surface.

Worth checking whether MiniMax-M2.7 is running on a different vLLM version than the models that work fine — that'd explain why it's intermittent (only happens on certain models). user: Sender (untrusted metadata):

{
  "label": "openclaw-control-ui",
  "id": "openclaw-control-ui"
}

[Mon 2026-04-20 18:47 UTC] just doing a vibe check