Files

Woosuk Kwon 7b7cdce968 [Model Runner V2] Refactor get_cudagraph_and_dp_padding (#32625 )

Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

2026-01-19 18:25:02 -08:00

metrics

[Model Runner V2] Support num NaNs in logits (#30187 )

2025-12-09 10:00:49 -08:00

[Model Runner V2] Support VLM (#32546 )

2026-01-18 16:58:51 -08:00

sample

[Model Runner V2] Refactor Sampler (#32245 )

2026-01-13 17:58:12 -08:00

spec_decode

[Model Runner V2] Support VLM (#32546 )

2026-01-18 16:58:51 -08:00

__init__.py

GPU Model Runner V2 (#25266 )

2025-11-21 08:20:55 -08:00

async_utils.py

[Model Runner V2] Support num NaNs in logits (#30187 )

2025-12-09 10:00:49 -08:00

attn_utils.py

[3/N][Attention] Move AttentionMetadata-related code from utils.py to backend.py (#32054 )

2026-01-12 09:13:56 -08:00

block_table.py

[Model Runner V2] Remove async barrier (#32083 )

2026-01-11 20:24:30 -08:00

buffer_utils.py

[Model Runner V2] Support logit_bias, allowed_token_ids, min_tokens (#32163 )

2026-01-12 11:31:10 -08:00

cudagraph_utils.py

[Model Runner V2] Refactor dummy_run (#32533 )

2026-01-19 14:50:59 -08:00

dp_utils.py

[Model Runner V2] Refactor get_cudagraph_and_dp_padding (#32625 )

2026-01-19 18:25:02 -08:00

input_batch.py

[Model Runner V2] Support VLM (#32546 )

2026-01-18 16:58:51 -08:00

model_runner.py

[Model Runner V2] Refactor get_cudagraph_and_dp_padding (#32625 )

2026-01-19 18:25:02 -08:00

README.md

GPU Model Runner V2 (#25266 )

2025-11-21 08:20:55 -08:00

states.py

[Model Runner V2] Refactor Sampler (#32245 )

2026-01-13 17:58:12 -08:00

structured_outputs.py

[Model Runner V2] Remove async barrier (#32083 )

2026-01-11 20:24:30 -08:00

README.md

[Experimental] Model Runner V2

This directory contains the new model runner which is under active development. Ping Woosuk Kwon for any changes.