[Disagg] Support large batch size in proxy server and update NixlConnector doc for DP (#28782)

Signed-off-by: Ming Yang <minos.future@gmail.com>
2025-12-08 16:01:08 -08:00
parent 1fb632fdb6
commit 60d17251c9
3 changed files with 42 additions and 4 deletions
--- a/docs/features/nixl_connector_usage.md
+++ b/docs/features/nixl_connector_usage.md
@@ -146,6 +146,8 @@ python tests/v1/kv_connector/nixl_integration/toy_proxy_server.py \
  --decoder-ports 8000 8000
 ```

+For multi-host DP deployment, only need to provide the host/port of the head instances.
+
 ### KV Role Options

 - **kv_producer**: For prefiller instances that generate KV caches