Woosuk Kwon 2a194ddd72 [Model Runner V2] Add model_state inputs to CUDA graph capture (#36544) ...

Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>

2026-03-09 15:14:51 -07:00

..

attention

[ROCm][CI] Fix ROCm attention backend validation for head sizes, block sizes, and compute capability checks (#36292)

2026-03-09 12:02:41 -05:00

core

[Core][KVConnector] Support HMA+NixlConnector (#35758)

2026-03-06 08:51:21 +01:00

engine

Reapply [Attention] Refactor check_and_update_config (#35122)

2026-03-09 07:17:14 -07:00

executor

Reapply [Attention] Refactor check_and_update_config (#35122)

2026-03-09 07:17:14 -07:00

kv_offload

[Bugfix][CI] fix typos (#34934)

2026-03-05 17:05:46 +00:00

metrics

[Metrics] Add Prometheus counters for Model FLOPs Utilization (MFU) (#30950)

2026-02-23 15:01:07 +00:00

pool

[Refactor] Clean up pooler modules (#31897)

2026-01-08 00:07:43 +08:00

sample

[CI] Bump mypy version (#34950)

2026-03-04 20:55:11 +00:00

spec_decode

[MTP][Misc] Clean up dead code (#36507)

2026-03-09 14:43:06 -04:00

structured_output

[Perf] Optimize Python Slice for Structured Output using islice instead of [:] (#33593)

2026-02-24 09:02:36 -08:00

worker

[Model Runner V2] Add model_state inputs to CUDA graph capture (#36544)

2026-03-09 15:14:51 -07:00

__init__.py

[V1] AsyncLLM Implementation (#9826)

2024-11-11 23:05:38 +00:00

cudagraph_dispatcher.py

[UX][Startup] Account for CUDA graphs during memory profiling (#30515)

2026-03-07 13:49:23 -08:00

kv_cache_interface.py

Make mypy opt-out instead of opt-in (#33205)

2026-01-29 09:12:26 +00:00

outputs.py

[Spec Decode] Add hidden states extraction system (#33736)

2026-03-02 14:29:09 -05:00

request.py

[Deprecation][1/2] Remove items deprecated in v0.18 (#36470)

2026-03-09 03:43:23 -07:00

serial_utils.py

[Multimodal] Simplify MM input definitions (#33331)

2026-01-29 13:32:04 +00:00

utils.py

[Frontend][Core] Add shutdown timeout - allowing in-flight requests to finish (#34730)

2026-03-05 22:04:31 -08:00