fix: shared_experts gate_up_proj stacking was skipped by .experts. check

The stacking logic skipped any key containing '.experts.' to avoid
MoE routed expert weights. But 'shared_experts' also matches that
substring, so gate_proj and up_proj were never stacked into
gate_up_proj. Changed to '.ffn.experts.' which only matches the
routed experts path.

Also includes POST-LOAD all-zero param scan.
This commit is contained in:
2026-05-15 00:08:04 +00:00
parent 78dc83dc6e
commit 483046b9d6

View File

@@ -1311,8 +1311,10 @@ class DeepseekV4Model(nn.Module):
break # first match wins (order matters)
for param_name, weight_name, shard_id in stacked_params_mapping:
# Skip non-stacked layers and experts (experts handled below).
if ".experts." in name:
# Skip MoE routed experts (handled separately below).
# Use .ffn.experts. (not .experts.) to avoid skipping
# shared_experts which also contains ".experts.".
if ".ffn.experts." in name:
continue
if weight_name not in name:
continue
@@ -1370,7 +1372,7 @@ class DeepseekV4Model(nn.Module):
loaded_params.add(name)
break
else:
if ".experts." in name:
if ".ffn.experts." in name:
# E8M0 scales are stored as float8_e8m0fnu in
# MXFP4 checkpoints but NVFP4 uses float8_e4m3fn.
# The uint8 view+copy path is only valid for MXFP4;