biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[V1] guidance backend for structured output + auto fallback mode (#14779)

Browse Source 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Loc Huynh <jc1da.3011@gmail.com>
Co-authored-by: Michal Moskal <michal@moskal.me>
This commit is contained in:
Russell Bryant
2025-03-25 00:02:33 -04:00
committed by GitHub
parent 10b34e36b9
commit a09ad90a72
9 changed files with 344 additions and 110 deletions
Download Patch File Download Diff File Expand all files Collapse all files

21
vllm/engine/arg_utils.py
View File

@@ -391,16 +391,13 @@ class EngineArgs:
default='xgrammar',
help='Which engine will be used for guided decoding'
' (JSON schema / regex etc) by default. Currently support '
'https://github.com/outlines-dev/outlines, '
'https://github.com/mlc-ai/xgrammar, and '
'https://github.com/noamgat/lm-format-enforcer.'
' Can be overridden per request via guided_decoding_backend'
' parameter.\n'
'Backend-specific options can be supplied in a comma-separated '
'list following a colon after the backend name. Valid backends and '
'all available options are: [xgrammar:no-fallback, '
'xgrammar:disable-any-whitespace, '
'outlines:no-fallback, lm-format-enforcer:no-fallback]')
'https://github.com/mlc-ai/xgrammar and '
'https://github.com/guidance-ai/llguidance.'
'Valid backend values are "xgrammar", "guidance", and "auto". '
'With "auto", we will make opinionated choices based on request'
'contents and what the backend libraries currently support, so '
'the behavior is subject to change in each release. '
'The default is xgrammar.')
parser.add_argument(
'--logits-processor-pattern',
type=nullable_str,
@@ -1539,9 +1536,9 @@ class EngineArgs:
recommend_to_remove=False)
return False
# Only support Xgrammar for guided decoding so far.
# Xgrammar and Guidance are supported.
SUPPORTED_GUIDED_DECODING = [
"xgrammar", "xgrammar:disable-any-whitespace"
"xgrammar", "xgrammar:disable-any-whitespace", "guidance", "auto"
]
if self.guided_decoding_backend not in SUPPORTED_GUIDED_DECODING:
_raise_or_fallback(feature_name="--guided-decoding-backend",