[Chore] Cleanup guided namespace, move to structured outputs config (#22772)
Signed-off-by: Aaron Pham <contact@aarnphm.xyz> Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
This commit is contained in:
@@ -67,7 +67,7 @@ class Request:
|
||||
# Generative models.
|
||||
assert sampling_params.max_tokens is not None
|
||||
self.max_tokens = sampling_params.max_tokens
|
||||
if sampling_params.guided_decoding is not None:
|
||||
if sampling_params.structured_outputs is not None:
|
||||
self.status = RequestStatus.WAITING_FOR_FSM
|
||||
self.use_structured_output = True
|
||||
|
||||
|
||||
Reference in New Issue
Block a user