haosdent a2ba6a5244 [Bugfix] Fix prefix caching for Mamba 'all' mode (Nemotron models) (#34874) ...

Signed-off-by: haosdent <haosdent@gmail.com>

2026-02-23 17:31:51 +01:00

..

attention

[Bugfix] Fix prefix caching for Mamba 'all' mode (Nemotron models) (#34874)

2026-02-23 17:31:51 +01:00

core

[Bug] Refactor max_num_batched_tokens to account for drafting (#34898)

2026-02-22 11:18:46 -05:00

engine

Support prompt_embeds for pooling requests in output processor (#34904)

2026-02-20 19:57:38 -08:00

executor

[CI] Revert PRs 34818 and 33600 (#34979)

2026-02-20 13:25:50 -08:00

kv_offload

[Bugfix] Fix ARC touch KeyError for non-ready T1 blocks in kv offload (#34576)

2026-02-16 07:33:19 -08:00

metrics

[Metrics] Add Prometheus counters for Model FLOPs Utilization (MFU) (#30950)

2026-02-23 15:01:07 +00:00

pool

[Refactor] Clean up pooler modules (#31897)

2026-01-08 00:07:43 +08:00

sample

Fix apply_top_k_top_p_triton called by non-cuda logits Tensor (#35030)

2026-02-21 21:11:54 -08:00

spec_decode

[Spec Decode] Reduce TP communication for speculative decoding draft token generation (#34049)

2026-02-22 14:59:16 -08:00

structured_output

[Feature] Lazy import for the "mistral" tokenizer module. (#34651)

2026-02-23 00:43:01 -08:00

worker

[Refactor] Decouple TimingContext from InputProcessingContext (#35083)

2026-02-23 14:15:50 +00:00

__init__.py

[V1] AsyncLLM Implementation (#9826)

2024-11-11 23:05:38 +00:00

cudagraph_dispatcher.py

[CI] Enable mypy coverage for individual excluded files (#34292)

2026-02-16 07:34:29 -08:00

kv_cache_interface.py

Make mypy opt-out instead of opt-in (#33205)

2026-01-29 09:12:26 +00:00

outputs.py

[ModelRunner V2] Fix spec decoding + logprobs (#33391)

2026-01-31 03:33:26 +00:00

request.py

[feat] Add per-block extra_keys to KV events (#33304)

2026-02-20 20:11:40 -08:00

serial_utils.py

[Multimodal] Simplify MM input definitions (#33331)

2026-01-29 13:32:04 +00:00

utils.py

[Misc] Replace Optional[X] with X | None syntax (#33332)

2026-01-30 01:56:59 -08:00