biondizzle 2bbbead984 P3: CUDA RoPE kernel — single launch per call (vs 5-6 PyTorch ops) ...

New files:
- dsv4/kernels/cuda/rope_cuda.cu: GPT-J interleaved RoPE kernel (forward+inverse)
- dsv4/ops/rope_cuda.py: Python bridge with ctypes loading
- tests/unit/test_rope_cuda.py: correctness test (cos >= 0.999998)

Savings: ~915 launches/token → 183 launches/token

2026-06-02 09:05:22 +00:00

..

__init__.py

Fix module path: move loader code from __init__.py to loader.py

2026-06-01 21:18:29 +00:00

_hash_router.py

Router: full kernel stack — hash, topk, activation+topk, dense decode/prefill

2026-05-21 21:54:05 +00:00

activation_topk.cu

Router: full kernel stack — hash, topk, activation+topk, dense decode/prefill

2026-05-21 21:54:05 +00:00

amax_gsa.cu

Fix CUDA kernel compilation: use c10::cuda::getCurrentCUDAStream

2026-06-01 20:49:55 +00:00

append_swa.cu

KV Cache: schema, allocator, pools, manager, append_swa kernel

2026-05-22 00:08:38 +00:00

compressor_reduce.cu

fix: move compressor position_bias into CUDA kernel (was Python loop)

2026-06-01 05:54:44 +00:00

deinterleave_quantize.cu

Restructure: cutedsl/ -> dsv4/ with proper layering

2026-05-21 17:30:44 +00:00

flush_write.cu

Flush compressor: schema fix, prepare_forward, flush_write kernels, state rotation

2026-05-22 00:25:47 +00:00

fused_amax_quantize.cu

CRITICAL FIX: fused_amax_quantize cross-CTA race condition

2026-06-01 21:26:51 +00:00

fused_deinterleave_amax_quantize.cu

P0 COMPLETE: Eliminate ALL .item() CPU-GPU syncs from NVFP4 activation path

2026-06-01 21:05:03 +00:00

fused_rmsnorm_quantize.cu

P3 WIP: fused RMSNorm + quantize kernel skeleton (not yet integrated)

2026-06-02 09:02:52 +00:00

gather_kv.cu

fix: extern declarations for gather_swa functions in gather_kv.cu

2026-05-30 21:14:15 +00:00

gather_swa.cu

fix: single PYBIND11_MODULE for combined gather .so

2026-05-30 21:13:24 +00:00

hash_router.cu

Router: full kernel stack — hash, topk, activation+topk, dense decode/prefill

2026-05-21 21:54:05 +00:00

indexer_score_topk.cu

Clean up E2M1 dequant: use LUT approach (consultant recommendation)

2026-05-28 16:17:47 +00:00

loader.py

CRITICAL FIX: fused_amax_quantize cross-CTA race condition

2026-06-01 21:26:51 +00:00

mhc_sinkhorn.cu

perf: fused mHC Sinkhorn CUDA kernel (1 launch vs 38)

2026-06-02 03:50:57 +00:00

quantize_nvfp4.cu

fix quantize_nvfp4 kernel: use proven single-thread-per-CTA pattern from deinterleave_quantize.cu

2026-05-25 16:21:44 +00:00

rope_cuda.cu

P3: CUDA RoPE kernel — single launch per call (vs 5-6 PyTorch ops)

2026-06-02 09:05:22 +00:00

sampler.cu

P7: compressor early return + decode buffering (skip GEMMs when n_complete=0); sampler SMEM fix (LK=24 fits 48KB default); topk on float not bf16

2026-06-01 22:29:56 +00:00

sparse_topk_metadata.cu

Restructure: cutedsl/ -> dsv4/ with proper layering

2026-05-21 17:30:44 +00:00

topk_select.cu

Router: full kernel stack — hash, topk, activation+topk, dense decode/prefill

2026-05-21 21:54:05 +00:00