biondizzle/nvfp4-megamoe-kernel
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: add missing renames for Hadamard coding + compressor.ape

Browse Source 
Three more dropped checkpoint→model mappings:

1. hc_head: checkpoint has hc_head.hc_base/fn/scale, model has
   hc_head_base/fn/scale (underscore not dot separator)
2. attn_hc/ffn_hc: checkpoint has .attn_hc. and .ffn_hc., model has
   .hc_attn. and .hc_ffn. (word order reversed)
3. compressor.position_bias → compressor.ape: checkpoint name is
   position_bias, model attr is ape (absolute position encoding)

All 461 remaining zero params should now be just indexer.k_norm.bias
(legit zero - no bias in checkpoint, only weight).
This commit is contained in:
biondizzle
2026-05-15 01:16:19 +00:00
parent af6583eb19
commit 44d4b6c225
1 changed files with 8 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
vllm/patches/deepseek_v4.py
View File

@@ -1290,6 +1290,14 @@ class DeepseekV4Model(nn.Module):
".mlp.shared_experts.gate_proj.": ".ffn.shared_experts.w1.",
".mlp.shared_experts.up_proj.": ".ffn.shared_experts.w3.",
".mlp.shared_experts.down_proj.": ".ffn.shared_experts.down_proj.",
# Hadamard coding params
".attn_hc.": ".hc_attn.",
".ffn_hc.": ".hc_ffn.",
"hc_head.hc_base": "hc_head_base",
"hc_head.hc_fn": "hc_head_fn",
"hc_head.hc_scale": "hc_head_scale",
# compressor.position_bias → compressor.ape
".compressor.position_bias": ".compressor.ape",
# modelopt uses mlp, vllm uses ffn internally
".mlp.": ".ffn.",
}