biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[V1] guidance backend for structured output + auto fallback mode (#14779)

Browse Source 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Loc Huynh <jc1da.3011@gmail.com>
Co-authored-by: Michal Moskal <michal@moskal.me>
This commit is contained in:
Russell Bryant
2025-03-25 00:02:33 -04:00
committed by GitHub
parent 10b34e36b9
commit a09ad90a72
9 changed files with 344 additions and 110 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
vllm/config.py
View File

@@ -2800,12 +2800,17 @@ class DecodingConfig:
return hash_str
def __post_init__(self):
valid_guided_backends = [
'outlines', 'lm-format-enforcer', 'xgrammar', 'guidance'
v0_valid_guided_backends = [
'outlines', 'lm-format-enforcer', 'xgrammar'
]
v1_valid_guided_backends = ['xgrammar', 'guidance', 'auto']
backend = GuidedDecodingParams(
backend=self.guided_decoding_backend).backend_name
if envs.VLLM_USE_V1:
valid_guided_backends = v1_valid_guided_backends
else:
valid_guided_backends = v0_valid_guided_backends
if backend not in valid_guided_backends:
raise ValueError(f"Invalid guided_decoding_backend '{backend}',"
f" must be one of {valid_guided_backends}")