From 81931614e9a8f4fd0810a79bd660c6037da1639c Mon Sep 17 00:00:00 2001
From: biondizzle <biondizzle@gmail.com>
Date: Tue, 19 May 2026 08:02:00 +0000
Subject: [PATCH] Update CURRENT_BUG: CSA kernel works, plan vLLM integration

---
 CURRENT_BUG.md | 47 +++++++++++++++++++++++++++++++++--------------
 1 file changed, 33 insertions(+), 14 deletions(-)

diff --git a/CURRENT_BUG.md b/CURRENT_BUG.md
index 616828d0..7045a7c7 100644
--- a/CURRENT_BUG.md
+++ b/CURRENT_BUG.md
@@ -1,22 +1,41 @@
 # CURRENT_BUG.md
 
-## Status: CuTeDSL kernels confirmed correct. Bug is in vLLM's attention/FFN pipeline.
+## Status: CSA/HCA kernel works. Need vLLM integration.
 
-### Key Findings from test_model_forward_b200.py
+### What We Know
+- **CuTeDSL NVFP4 kernels**: All pass (cosine 0.988-0.999 vs BF16)
+- **Warmup gs**: Irrelevant (runner recomputes per-call)
+- **CSA attention kernel** (`cutedsl/csa_attention.py`): Works with PyTorch SDPA
+- **Full layer 0 forward**: CuTeDSL + SDPA = cosine 0.988 vs BF16 ✅
+- **Logits**: std=2.98, reasonable top-5 tokens ✅
 
-1. **Warmup gs is IRRELEVANT** — CuTeDSL `runner.run()` recomputes gs internally per-call. Changing gs by 10x has no effect on output (cosine 0.9993).
-2. **CuTeDSL kernels are correct** — cosine 0.999 vs BF16 for q_a_proj with both warmup and dynamic gs.
-3. **BF16 reference produces reasonable logits** — logit std 3.05, top5 valid token IDs.
-4. **The bug is NOT in our NVFP4 kernels** — it's in vLLM's pipeline.
+### Root Cause of vLLM Empty Output
+vLLM uses two compiled CUDA kernels that DON'T work on Blackwell (SM100):
+1. `torch.ops._C.fused_deepseek_v4_qnorm_rope_kv_rope_quant_insert` — fused RoPE + KV cache
+2. `FlashMLA sparse attention` — the actual attention computation
 
-### Most Likely Causes
+The model uses **CSA (Compressed Sparse Attention) + HCA (Heavily Compressed Attention)**, NOT MLA.
+vLLM misnames it "MLA" in code but the architecture is CSA/HCA with mHC.
 
-1. **FlashMLA kernel on Blackwell (SM100)** — `fused_deepseek_v4_qnorm_rope_kv_rope_quant_insert` is a C++ CUDA kernel. If it doesn't work on B200, attention output is garbage.
-2. **Weight sharding with TP=8** — The model is loaded with TP=8. If weight sharding is wrong, all projections produce garbage. But our standalone test uses the full (non-sharded) weights, which works.
-3. **MoE produces garbage** — The MoE path (384 experts, top-6) is complex. If expert routing or grouped GEMM is wrong, the output is dominated by MoE noise.
+### Integration Plan
+Replace vLLM's broken CUDA kernels in `DeepseekV4MLAAttention.forward`:
+1. Replace `fused_deepseek_v4_qnorm_rope_kv_rope_quant_insert` → pure PyTorch RoPE + FP8 quant + cache insert
+2. Replace FlashMLA → our CSA/HCA kernel using PyTorch SDPA
+3. Keep the compressor (it's mostly Triton which may work on SM100)
+4. Keep the indexer (it calls into sparse_attn_indexer which is also Triton)
 
-### Next Steps
+### Test Results
+```
+test_full_layer_b200.py:
+  q_a_proj:     0.995 ✅  kv_proj:      0.995 ✅  q_b_proj:     0.995 ✅
+  wo_b_proj:    0.995 ✅  comp.kv_proj: 0.994 ✅  comp.gate:    0.995 ✅
+  shared_expert: 0.990 ✅
 
-- Write a test that runs the FULL model (all 61 layers) in BF16 and checks the final output
-- Add hook/logging to the vLLM container to capture layer-by-layer output
-- Test if the FlashMLA C++ kernel works on B200
+test_model_forward_b200.py:
+  Warmup gs is IRRELEVANT (10x change → cosine 0.9993)
+  CuTeDSL cosine vs BF16: 0.999
+
+test_csa_attention_b200.py:
+  Full path CuTeDSL + SDPA vs BF16: 0.988 ✅
+  Logit std: 2.98 ✅
+```