biondizzle/nvfp4-megamoe-kernel
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update CURRENT_BUG.md with Bug 26 fix

Browse Source
This commit is contained in:
biondizzle
2026-05-17 21:36:25 +00:00
parent 72628fb689
commit e65f2b2ba2
1 changed files with 10 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
CURRENT_BUG.md
View File

@@ -1,13 +1,13 @@
# Current Bug: CuTeDSLMoERunner — Status & Debug History
## Current Status (May 17, 2026 17:54 UTC)
## Current Status (May 17, 2026 21:30 UTC)
**Build #11 in progress (includes swiglu_limit fix). Previous builds produced empty/invisible token output.**
**Bug 26 FIXED. All tests pass.**
- ✅ `layertest.py` — 0.988 cosine
- ✅ `cudagraph_test.py` — capture + replay works
- ✅ vLLM container starts, loads weights, warmup gs computed, cudagraph capture succeeds
- ❌ Model output was empty content (30 invisible tokens) — **swiglu_limit fix not yet tested in container**
- ✅ `test_pipeline_real_weights.py` — 0.988 cosine (8 tokens, 48 experts)
- ⏳ vLLM container — needs rebuild + test with Bug 26 fix
**Latest fix: Missing swiglu_limit=10.0 activation clamping (Bug 25).** DeepSeek-V4 uses `SiluAndMulWithClamp(10.0)` which clamps `silu(gate)` to max 10.0 and `up` to [-10, 10]. Our runner was doing plain `F.silu(gate) * up` without clamping. Large gate values → unbounded SiLU output → corrupted L2 GEMM input → garbage logits → model outputs BOS/thinking tokens.
@@ -121,6 +121,11 @@ activated = gate_silu * up
```
Called from `deepseek_v4.py` after warmup: `self._cutedsl_runner.set_swiglu_limit(float(self.swiglu_limit))`.
### Bug 26: Padded Buffer x_sf Mismatch — Experts 1+ Get Zero Output — FIXED
**Symptom:** Runner produces cosine 0.285 vs BF16. Some tokens get exactly zero output. Expert 0 L1 cosine 0.996, experts 1+ get cosine 0.0.
**Root cause:** Runner quantized `padded_hidden` (4096 rows with zero padding) → `quantize_activation_nvfp4` returns x_sf with 4096 rows. Then `x_sf[:num_slots]` (first 48 rows) only covers expert 0's tokens (padded rows 0-127). Expert 1's tokens are at padded row 128, but x_sf[4] corresponds to padded row 64 (still expert 0's padding), not expert 1's data. The scale assembly scattered wrong scales for experts 1+, producing zero GEMM output.
**Fix:** Quantize `slot_hidden` (sorted tokens, num_slots rows) instead of `padded_hidden`. This gives x_sf with num_slots rows (one per token), which the scale assembly correctly scatters into padded layout. Scatter x_fp4 into a new `hidden_fp4` padded buffer (uint8→view float4). Same fix for L2 with `activated_fp4` buffer.
---
## Current Architecture: Variable Padded Expert Offsets
@@ -190,6 +195,7 @@ DeepseekV4MoE.forward():
|------|---------|
| `tests/layertest.py` | Reference vs runner, 3 experts. Must pass ≥0.98 cosine. |
| `tests/cudagraph_test.py` | Cudagraph capture + replay. Must pass. |
| `tests/test_pipeline_real_weights.py` | Full runner vs BF16 reference, 48 experts, 8 tokens. Must pass ≥0.98 cosine. |
**Run order after any code change:**
1. `python3 tests/layertest.py` — must pass