Zijing Liu b395b3b0a3 [Disagg][Perf] Use CUDA event sync instead of blocking tolist to avoid unintentional copy ops blocking across different CUDA streams, improving disagg TTIT/TTFT (#22760) ...

Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
Signed-off-by: Zijing Liu <liuzijing2014@users.noreply.github.com>

2025-08-25 21:06:00 -07:00

..

attention

[Misc] Simplify FlashInfer attention metadata (#23585)

2025-08-25 15:42:29 -07:00

core

[Core][Multimodal] Track encode cache entries by mm_hash and enable embedding sharing between requests (#22711)

2025-08-25 00:41:17 -07:00

engine

Frontend: Adding LM Format Enforcer support to V1 engine (#22564)

2025-08-24 19:31:22 -07:00

executor

[Core] Support custom executor qualname (#23314)

2025-08-22 09:40:54 +08:00

metrics

[BugFix] Improve internal DP load balancing (#21617)

2025-08-01 19:45:27 -07:00

pool

[Performance] V1 Pooling Models E2E Performance Optimization (#23162)

2025-08-21 13:26:09 +00:00

sample

[BugFix] Fix MinPLogitsProcessor.update_states() (#23401)

2025-08-23 08:22:11 +08:00

spec_decode

[FIXBUG ] Allow disabling rocm_aiter_fa backend for ROCm GPUs not compatible with AITER (#22795)

2025-08-20 09:08:29 -07:00

structured_output

Frontend: Adding LM Format Enforcer support to V1 engine (#22564)

2025-08-24 19:31:22 -07:00

worker

[Disagg][Perf] Use CUDA event sync instead of blocking tolist to avoid unintentional copy ops blocking across different CUDA streams, improving disagg TTIT/TTFT (#22760)

2025-08-25 21:06:00 -07:00

__init__.py

[V1] AsyncLLM Implementation (#9826)

2024-11-11 23:05:38 +00:00

cudagraph_dispatcher.py

[Core] Allow full cudagraph with separate attention routines and orthogonal to compilation, add support for FA2 and FlashInfer (#20059)

2025-08-15 10:01:39 -04:00

kv_cache_interface.py

[Attention] Refactor AttentionMetadata Preparation for Encoder-only Models (#23154)

2025-08-22 05:05:59 +00:00

outputs.py

[Spec Decode] Make propose_draft_token_ids non-blocking for lower TTFT (#23041)

2025-08-18 17:20:38 -07:00

request.py

[Misc] Convert use_structured_output property into constant (#23060)

2025-08-17 12:41:38 -07:00

serial_utils.py

[Refactor] Define MultiModalKwargsItems separate from MultiModalKwargs (#23053)

2025-08-18 09:52:00 +00:00

utils.py

[refactor] improve ConstantList exception specificity (#22156)

2025-08-03 22:14:49 -07:00