[Frontend] Add unix domain socket support (#18097)

Signed-off-by: <yyweiss@gmail.com> Signed-off-by: yyw <yyweiss@gmail.com>
2025-08-09 02:23:44 +03:00
parent 2fcf6b27b6
commit baece8c3d2
5 changed files with 86 additions and 16 deletions
--- a/docs/cli/README.md
+++ b/docs/cli/README.md
@@ -29,6 +29,9 @@ Start the vLLM OpenAI Compatible API server.
    # Specify the port
    vllm serve meta-llama/Llama-2-7b-hf --port 8100

+    # Serve over a Unix domain socket
+    vllm serve meta-llama/Llama-2-7b-hf --uds /tmp/vllm.sock
+
    # Check with --help for more options
    # To list all groups
    vllm serve --help=listgroup