biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[BugFix] [DP/EP] Fix slow execution when BS <= DP (#25407)

Browse Source 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Chris Bamford <chrisbam4d@gmail.com>
This commit is contained in:
Matthew Bonanni
2025-09-22 20:26:17 -04:00
committed by GitHub
parent 090197034f
commit ac0048c0ae
2 changed files with 5 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
vllm/v1/worker/gpu_worker.py
View File

@@ -487,7 +487,7 @@ class Worker(WorkerBase):
sort_by="self_cuda_time_total"))
def execute_dummy_batch(self) -> None:
self.model_runner._dummy_run(1)
self.model_runner._dummy_run(1, uniform_decode=True)
def add_lora(self, lora_request: LoRARequest) -> bool:
return self.model_runner.add_lora(lora_request)