biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[Spec Decode] Fix offline spec_decode.py (#24257)

Browse Source 
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
This commit is contained in:
Ekagra Ranjan
2025-09-08 16:44:13 -04:00
committed by GitHub
parent 43d9ad03ba
commit 41183c1fe0
1 changed files with 4 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
vllm/benchmarks/datasets.py
View File

@@ -1310,6 +1310,10 @@ def add_dataset_parser(parser: FlexibleArgumentParser):
def get_samples(args, tokenizer) -> list[SampleRequest]: def get_samples(args, tokenizer) -> list[SampleRequest]:
if not hasattr(args, "request_id_prefix"):
args.request_id_prefix = ""
if args.dataset_name == "custom": if args.dataset_name == "custom":
dataset = CustomDataset(dataset_path=args.dataset_path) dataset = CustomDataset(dataset_path=args.dataset_path)
input_requests = dataset.sample( input_requests = dataset.sample(