biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[Spec Decode, BugFix] Propagate norm_before_fc from Eagle3 speculator (#38111)

Browse Source 
Signed-off-by: Shubhra Pandit <shubhra.pandit@gmail.com>
This commit is contained in:
Shubhra Pandit
2026-03-28 20:42:06 -04:00
committed by GitHub
parent d39b8daf5f
commit 5b8c30d62b
1 changed files with 2 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
vllm/transformers_utils/configs/speculators/algos.py
View File

@@ -22,6 +22,7 @@ def update_eagle3(config_dict: dict, pre_trained_config: dict) -> None:
- draft_vocab_size: Size of the draft model's vocabulary
- target_hidden_size: Hidden size of the target model
- norm_before_residual: Whether to apply norm before residual connection
- norm_before_fc: Whether to apply RMSNorm before the fc projection
- eagle_aux_hidden_state_layer_ids: List of layer indices from the base
model to use as auxiliary inputs for the Eagle3 drafter. These layers
provide intermediate hidden states that help the drafter make better
@@ -34,6 +35,7 @@ def update_eagle3(config_dict: dict, pre_trained_config: dict) -> None:
pre_trained_config["norm_before_residual"] = config_dict.get(
"norm_before_residual", True
)
pre_trained_config["norm_before_fc"] = config_dict.get("norm_before_fc", False)
pre_trained_config["architectures"] = ["Eagle3LlamaForCausalLM"]
if config_dict.get("eagle_aux_hidden_state_layer_ids"):
pre_trained_config["eagle_aux_hidden_state_layer_ids"] = config_dict[