[Bugfix] Store Qwen3Next A_log in fp32 (#37810)

Signed-off-by: effortprogrammer <yhjhoward7@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
This commit is contained in:
Hojin Yang
2026-03-23 16:36:57 +09:00
committed by GitHub
parent 02e6efe56d
commit 54ab804e87

View File

@@ -501,6 +501,7 @@ class Qwen3NextGatedDeltaNet(nn.Module, MambaBase):
self.A_log = nn.Parameter(
torch.empty(
divide(self.num_v_heads, self.tp_size),
dtype=torch.float32,
)
)