biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[PluggableLayer][MM] Add PluggableLayer for CustomQwen2Decoder (#37293)

Browse Source 
Signed-off-by: Wangbei25 <wangbei41@huawie.com>
Signed-off-by: Wangbei25 <wangbei41@huawei.com>
Co-authored-by: Wangbei25 <wangbei41@huawie.com>
This commit is contained in:
Wangbei25
2026-03-20 14:24:07 +08:00
committed by GitHub
parent 30108fc8b0
commit 0674d1fee7
2 changed files with 17 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

13
docs/design/custom_op.md
View File

@@ -51,11 +51,8 @@ For example:
**1. Attention:**
```python
--8<-- "vllm/model_executor/layers/attention/mm_encoder_attention.py:mm_encoder_attn"
--8<-- "vllm/model_executor/layers/mla.py:multi_head_latent_attention"
--8<-- "vllm/model_executor/models/deepencoder.py:rel_pos_attention"
```
**2. Activation:**
@@ -170,6 +167,16 @@ For example:
--8<-- "vllm/model_executor/layers/rotary_embedding/common.py:apply_rotary_emb"
```
**12. Encoder:**
```python
--8<-- "vllm/model_executor/models/deepencoder2.py:qwen2_decoder"
--8<-- "vllm/model_executor/layers/attention/mm_encoder_attention.py:mm_encoder_attn"
--8<-- "vllm/model_executor/models/deepencoder.py:rel_pos_attention"
```
## Guidelines for Implementing a New CustomOp
### Implement a New CustomOp in vLLM

8
vllm/model_executor/models/deepencoder2.py
View File

@@ -14,14 +14,20 @@ import torch
import torch.nn as nn
import transformers
from vllm.model_executor.custom_op import PluggableLayer
class CustomQwen2Decoder(nn.Module):
# --8<-- [start:qwen2_decoder]
@PluggableLayer.register("qwen2_decoder")
class CustomQwen2Decoder(PluggableLayer):
"""
Qwen2 visual encoder
non-causal attention + causal attention
token_type_ids ：0=non-causal, 1=causal
"""
# --8<-- [end:qwen2_decoder]
def __init__(
self,
decoder_layer: int = 24,