[V1] Support bad_words in sampler (#13376)

Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com> Co-authored-by: Nick Hill <nhill@redhat.com>
2025-03-08 14:50:26 -08:00
parent 9513290032
commit eb8b5eb183
13 changed files with 266 additions and 28 deletions
--- a/tests/v1/sample/test_rejection_sampler.py
+++ b/tests/v1/sample/test_rejection_sampler.py
@@ -42,6 +42,7 @@ def create_sampling_metadata(spec_tokens: list[list[int]]) -> SamplingMetadata:
        min_tokens={},
        logit_bias=[None] * batch_size,
        allowed_token_ids_mask=None,
+        bad_words_token_ids={},
    )