biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[FEAT] [Performance] Enable DP for ViT in Qwen2.5VL (#22742)

Browse Source 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
This commit is contained in:
TJian
2025-08-19 08:25:57 -07:00
committed by GitHub
parent 4d9c61993a
commit 1298c67795
5 changed files with 633 additions and 48 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
vllm/model_executor/layers/linear.py
View File

@@ -437,7 +437,7 @@ class MergedReplicatedLinear(ReplicatedLinear):
shard_offset = sum(self.output_sizes[:loaded_shard_id])
shard_size = self.output_sizes[loaded_shard_id]
param[shard_offset:shard_offset + shard_size] = loaded_weight
param.data[shard_offset:shard_offset + shard_size] = loaded_weight
@CustomOp.register("column_parallel_linear")