[Frontend] Support custom request_id from request (#9550)

Co-authored-by: Yuhong Guo <yuhong.gyh@antgroup.com>
This commit is contained in:
Yuhong Guo
2024-10-23 02:07:30 +08:00
committed by GitHub
parent 32a1ee74a0
commit 434984e665
2 changed files with 8 additions and 2 deletions

View File

@@ -284,6 +284,12 @@ class ChatCompletionRequest(OpenAIBaseModel):
"The priority of the request (lower means earlier handling; "
"default: 0). Any priority other than 0 will raise an error "
"if the served model does not use priority scheduling."))
request_id: str = Field(
default_factory=lambda: f"{random_uuid()}",
description=(
"The request_id related to this request. If the caller does "
"not set it, a random_uuid will be generated. This id is used "
"through out the inference process and return in response."))
# doc: end-chat-completion-extra-params