biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[Bugfix] Fix Stream Sync for Shared Expert Overlap (#28430)

Browse Source 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
(cherry picked from commit e605e8e323)
This commit is contained in:
Robert Shaw
2025-11-11 00:59:08 -05:00
committed by Kevin H. Luu
parent 2a21f3e7c2
commit 28a82bb5e6
2 changed files with 15 additions and 33 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
tests/evals/gsm8k/configs/Qwen1.5-MoE-W4A16-CT.yaml
View File

@@ -3,6 +3,3 @@ accuracy_threshold: 0.45
num_questions: 1319
num_fewshot: 5
max_model_len: 4096
# Duo stream incompatabilbe with this model: https://github.com/vllm-project/vllm/issues/28220
env:
VLLM_DISABLE_SHARED_EXPERTS_STREAM: "1"