Lucas Wilkinson 75e94309e8 [Perf] Mem align KV caches for CUDA devices (MLA perf improvement) (#12676) ...

Signed-off-by: simon-mo <xmo@berkeley.edu>
Signed-off-by: Lucas Wilkinson <lcwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>

2025-02-04 18:22:24 -08:00

26 KiB

Raw Blame History

View Raw