[doc] Hybrid KV Cache Manager design doc (#22688)

Browse Source

Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

...

This commit is contained in:

Chen Zhang

2025-08-26 13:19:05 -07:00

committed by GitHub

parent 9816b81f5f

commit d696f86e7b

6 changed files with 245 additions and 0 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

BIN

docs/assets/design/hybrid_kv_cache_manager/basic_grouping_example.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 24 KiB

BIN

docs/assets/design/hybrid_kv_cache_manager/full_attn.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 4.0 KiB

BIN

docs/assets/design/hybrid_kv_cache_manager/memory_layout.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 62 KiB

BIN

docs/assets/design/hybrid_kv_cache_manager/overview.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 39 KiB

BIN

docs/assets/design/hybrid_kv_cache_manager/sw_attn.png Normal file

Binary file not shown.

Side by Side After Width:  |  Height:  |  Size: 4.5 KiB