biondizzle 3c295f225a P3: integrate CUDA RoPE kernel into single_shot — 732 launches/token eliminated ...

_apply_rope now uses dsv4.ops.rope_cuda (1 CUDA kernel per call)
instead of PyTorch ops (5-6 kernels per call).
Total: 183 RoPE calls × (5-1) = 732 launches saved per token.
With fallback to PyTorch if CUDA kernel fails.

2026-06-02 09:08:07 +00:00

74 KiB

Raw Blame History

View Raw