biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

Support Pixtral-Large HF by using llava multimodal_projector_bias config (#12710)

Browse Source 
Signed-off-by: mgoin <michael@neuralmagic.com>
This commit is contained in:
Michael Goin
2025-02-03 22:55:46 -05:00
committed by GitHub
parent 73b35cca7f
commit 5d98d56089
4 changed files with 13 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
vllm/model_executor/models/llava_onevision.py
View File

@@ -372,11 +372,11 @@ class LlavaOnevisionMultiModalProjector(nn.Module):
self.linear_1 = nn.Linear(config.vision_config.hidden_size,
config.text_config.hidden_size,
bias=True)
bias=config.multimodal_projector_bias)
self.act = get_act_fn(config.projector_hidden_act)
self.linear_2 = nn.Linear(config.text_config.hidden_size,
config.text_config.hidden_size,
bias=True)
bias=config.multimodal_projector_bias)
def forward(self, image_features: torch.Tensor) -> torch.Tensor:
hidden_states = self.linear_1(image_features)