Support encoder-only models without KV-Cache (#21270)

Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
This commit is contained in:
Maximilien de Bayser
2025-07-26 10:09:52 -03:00
committed by GitHub
parent f27fdfc3ed
commit 1cd6eaba54
17 changed files with 352 additions and 99 deletions

View File

@@ -93,6 +93,7 @@ def create_common_attn_metadata(
max_query_len=max_query_len,
block_table_tensor=block_table_tensor,
slot_mapping=slot_mapping,
causal=True,
)