biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[Misc] Add unit tests for MoE ModularKernel combinations + Profiling utility (#20449)

Browse Source 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
This commit is contained in:
Varun Sundar Rabindranath
2025-07-11 10:51:46 -04:00
committed by GitHub
parent 6fb162447b
commit 53fa457391
15 changed files with 1727 additions and 22 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
vllm/distributed/device_communicators/base_device_communicator.py
View File

@@ -240,8 +240,7 @@ class DeviceCommunicatorBase:
if module.__class__.__name__ == "FusedMoE"
]
for module in moe_modules:
module.quant_method.init_prepare_finalize(module.moe_config,
module.quant_config)
module.quant_method.init_prepare_finalize(module.moe_config)
def dispatch(
self, hidden_states: torch.Tensor,