biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[Misc] Fix examples openai_pooling_client.py (#24853)

Browse Source 
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
This commit is contained in:
wang.yuqi
2025-09-15 19:57:30 +08:00
committed by GitHub
parent 2e41f5abca
commit bf214ca226
17 changed files with 105 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
docs/serving/openai_compatible_server.md
View File

@@ -239,7 +239,7 @@ you can use the [official OpenAI Python client](https://github.com/openai/openai
If the model has a [chat template][chat-template], you can replace `inputs` with a list of `messages` (same schema as [Chat API][chat-api])
which will be treated as a single prompt to the model.
Code example: <gh-file:examples/online_serving/openai_embedding_client.py>
Code example: <gh-file:examples/online_serving/pooling/openai_embedding_client.py>
#### Multi-modal inputs
@@ -313,7 +313,7 @@ and passing a list of `messages` in the request. Refer to the examples below for
`MrLight/dse-qwen2-2b-mrl-v1` requires a placeholder image of the minimum image size for text query embeddings. See the full code
example below for details.
Full example: <gh-file:examples/online_serving/openai_chat_embedding_client_for_multimodal.py>
Full example: <gh-file:examples/online_serving/pooling/openai_chat_embedding_client_for_multimodal.py>
#### Extra parameters
@@ -421,7 +421,7 @@ Our Pooling API encodes input prompts using a [pooling model](../models/pooling_
The input format is the same as [Embeddings API][embeddings-api], but the output data can contain an arbitrary nested list, not just a 1-D list of floats.
Code example: <gh-file:examples/online_serving/openai_pooling_client.py>
Code example: <gh-file:examples/online_serving/pooling/openai_pooling_client.py>
[](){ #classification-api }
@@ -431,7 +431,7 @@ Our Classification API directly supports Hugging Face sequence-classification mo
We automatically wrap any other transformer via `as_seq_cls_model()`, which pools on the last token, attaches a `RowParallelLinear` head, and applies a softmax to produce per-class probabilities.
Code example: <gh-file:examples/online_serving/openai_classification_client.py>
Code example: <gh-file:examples/online_serving/pooling/openai_classification_client.py>
#### Example Requests
@@ -760,7 +760,7 @@ endpoints are compatible with both [Jina AI's re-rank API interface](https://jin
[Cohere's re-rank API interface](https://docs.cohere.com/v2/reference/rerank) to ensure compatibility with
popular open-source tools.
Code example: <gh-file:examples/online_serving/jinaai_rerank_client.py>
Code example: <gh-file:examples/online_serving/pooling/jinaai_rerank_client.py>
#### Example Request