biondizzle b380028c49 feat: production compressor/indexer — NVFP4 GEMM + CUDA softmax/reduce kernel ...

- New compressor_reduce.cu: CSA/HCA token-level softmax + weighted sum + kv_norm
  One block per compressed entry, 128 threads, FP32 accumulation
  CSA: overlapping Ca/Cb streams (2m tokens per block)
  HCA: single stream (m tokens per block)
  Includes apply_kv_norm kernel (unweighted RMSNorm + weight)

- New production_compress.py: Python wrapper for CUDA kernels

- single_shot_inference.py: Compressor/Indexer now use production Nvfp4Linear
  for kv_proj, gate_proj, q_b_proj, weights_proj projections
  Then CUDA reduce kernel for softmax + weighted sum
  No more PyTorch reference nvfp4_linear_ref in compressor/indexer path

2026-06-01 05:18:59 +00:00

..

__init__.py

Wire indexer compute_index_scores_topk + fix compressor imports

2026-05-30 21:19:06 +00:00

compress_tail.py

E2/E3: compressor bridge, indexer bridge, flush pipeline wiring

2026-05-30 21:16:54 +00:00

csa_hca.py

Restructure: cutedsl/ -> dsv4/ with proper layering

2026-05-21 17:30:44 +00:00

production_compress.py

feat: production compressor/indexer — NVFP4 GEMM + CUDA softmax/reduce kernel

2026-06-01 05:18:59 +00:00