biondizzle 0f951a0b1a Fix attention sinks: logit bias (HuggingFace reference), not dummy KV ...

The HuggingFace reference treats attention sinks as a logit bias:
1. Compute raw Q*K scores
2. Concatenate sinks as a logit column
3. Softmax the combined logits
4. DROP the sink column (don't multiply by V)
5. Multiply by V

Our old code added sinks as a dummy zero-KV entry, which diluted
attention weights by adding an extra V=0 position to the softmax.

2026-05-31 11:53:43 +00:00

56 KiB

Raw Blame History

View Raw