Commit Graph

4 Commits

Author SHA1 Message Date
bbe40ac8c0 Add middleware to strip vLLM-only params (logprobs/top_logprobs) before forwarding to SGLang
SGLang's Mistral tool-call parser rejects logprobs/top_logprobs with 422,
while vLLM accepts them. Clients like OpenClaw send these by default.

New architecture: haproxy (port N) → middleware (port N+2) → SGLang (port N+1)
The middleware is a thin FastAPI app that strips incompatible params from
chat completion request bodies and passes everything else through unchanged.
2026-04-12 18:58:37 +00:00
359aa94337 Update README: haproxy proxy layer, /health probe fix, current state 2026-04-12 18:27:06 +00:00
2ac7778c15 Rewrite README: explain the shim, current state, and how to adapt for other models 2026-04-12 03:07:43 +00:00
c86fbe0166 Fix Jenkinsfile: agent any, nightly default, proper quoting 2026-04-12 00:22:29 +00:00