biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
10,502 Commits 2 Branches 140 Tags
7d8975de84dadb87df23eae4a663a3c6620433dd
Commit Graph

163 Commits

Author SHA1 Message Date
Bram Wasti
7d8975de84 Deepseek-v3 Batch Invariant on 8xH100 (#26609) 
Signed-off-by: Bram Wasti <bwasti@meta.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-10-15 22:06:02 -07:00
Michael Goin
3e051bda82 [UX] Replace VLLM_ALL2ALL_BACKEND with --all2all-backend (#26732) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-10-13 18:12:52 -07:00
Wentao Ye
314285d4f2 [CI] Fix mypy for vllm/distributed (#26593) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-13 16:02:24 -04:00
Wentao Ye
e251e457c5 [Log] Optimize Startup Log (#26601) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-14 02:06:57 +08:00
Harry Mellor
8fcaaf6a16 Update Optional[x] -> x | None and Union[x, y] to x | y (#26633) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-12 09:51:31 -07:00
Wentao Ye
8983e0216f [CI] Fix Pre-commit Issue Cannot determine type of "rank" and "world_size" (#26448) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-09 15:16:48 -07:00
Utkarsh Sharma
335b28f7d1 [TPU] Rename tpu_commons to tpu_inference (#26279) 
Signed-off-by: Utkarsh Sharma <utksharma@google.com>
Co-authored-by: Utkarsh Sharma <utksharma@google.com>
Co-authored-by: Chengji Yao <chengjiyao@google.com>
 2025-10-07 23:30:52 -07:00
Harry Mellor
6c04638214 Fix per file ruff ignores related to line length (#26262) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-06 05:12:40 +00:00
Harry Mellor
d6953beb91 Convert formatting to use ruff instead of yapf + isort (#26247) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 07:06:22 -07:00
Lucia Fang
f48b6a03ba [Misc]allow disable pynccl (#25421) 
Signed-off-by: Lu Fang <fanglu@fb.com>
Co-authored-by: Lucia (Lu) Fang <fanglu@meta.com>
 2025-10-01 06:04:13 +00:00
Gregory Shtrasberg
61a3431613 [Bugfix][ROCm] Fixing trying to import non-existent symbols from libnccl.so (#25605) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2025-09-29 17:01:50 -04:00
Tyler Michael Smith
a5354b3ed2 [Bugfix][WideEP] Apply TP Attn + EP MoE fix to other models (#24982) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2025-09-27 14:22:28 +00:00
Nick Hill
983056e456 [Misc] Remove unnecessary memoryviews in shm_broadcast.py (#25721) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-09-26 03:11:44 +00:00
Nick Hill
8b77328ffe [Misc] Don't log shm dequeue delay warning on worker side (#25720) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-09-26 01:08:30 +00:00
youkaichao
6c340da4df [misc] log info messages by default for hanging / busy / idle (#25627) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-09-25 21:14:57 +08:00
Cyrus Leung
2f17117606 [mypy] Fix wrong type annotations related to tuple (#25660) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-09-25 13:00:45 +00:00
Shu Wang
54e42b72db Support mnnvl all2allv from Flashinfer (#21003) 
Signed-off-by: Shu Wang <shuw@nvidia.com>
Signed-off-by: Shu Wang. <shuw@nvidia.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2025-09-24 14:38:16 -04:00
youkaichao
b67dece2d8 [misc] update the warning message (#25566) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-09-24 17:24:35 +08:00
Michael Goin
7361ab379f Remove redundant mutates_args and dispatch_key for direct_register_custom_op (#25512) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-09-23 22:48:40 +00:00
Ilya Markov
8bdd8b5c51 Enable symmetric memory all reduce by default only enabling for TP (#25070) 
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-09-23 15:53:00 -04:00
Amir Samani
8c1c81a3de [core] add nccl symmetric memory for all reduce (#24532) 
Signed-off-by: Amir Samani <asamani@nvidia.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-09-23 14:33:06 -04:00
Lucas Wilkinson
cc1dc7ed6d [Core/DBO][2/N] Dual-Batch Overlap add DeepEP High Throughput support and Prefill support (#24845) 
Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Sage Moore <sage@neuralmagic.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-09-23 16:02:10 +00:00
Fanli Lin
4c966e440e [XPU] Fix MOE DP accuracy issue on XPU (#25465) 2025-09-23 14:32:57 +00:00
Shu Wang
2ea50e977a Enable Allgather/ReduceScatter backend for NaiveAllToAll (#23964) 
Signed-off-by: Shu Wang. <shuw@nvidia.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Signed-off-by: Shu Wang <shuw@nvidia.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-09-18 15:52:58 +00:00
Hyogeun Oh (오효근)
b419937c78 [Docs] Fix warnings in mkdocs build (continued) (#25163) 
Signed-off-by: Zerohertz <ohg3417@gmail.com>
 2025-09-18 08:23:26 -07:00
Punitvara
05b044e698 [Doc] Fix cross-reference warnings (#25058) 
Signed-off-by: Punit Vara <punitvara@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-18 02:05:16 -07:00
Sage Moore
567939953b [Core/DBO][1/N] Add Dual-Batch Overlap mechanism to VLLM (#23693) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2025-09-16 12:21:48 -04:00
dongluw
a5b84f1cbf [Core] Shared memory based object store for Multimodal data caching and IPC (#20452) 
Signed-off-by: donglu <donglu@cohere.com>
 2025-09-12 07:54:17 -07:00
Ilya Markov
1fdd5c42d7 [Kernels] Enable Torch Symmetric Memory All-Reduce By Default (#24111) 
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-09-11 09:45:31 -07:00
Woosuk Kwon
4172235ab7 [V0 deprecation] Deprecate V0 Neuron backend (#21159) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-09-06 16:15:18 -07:00
Didier Durand
35bf193864 [Doc]: fix typos in Python comments (#24294) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-09-05 19:41:12 -07:00
bnellnm
e9b92dcd89 [Kernels] Overlap shared experts with send/recv (#23273) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2025-09-03 12:35:18 -04:00
Didier Durand
0235103cbb [Doc]: fix typos in Python comments (#24042) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-01 19:07:45 -07:00
Didier Durand
107284959a [Doc]: fix typos in Python comments (#24026) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
 2025-09-01 09:38:20 +00:00
Chaojun Zhang
235c9db8a7 [XPU] support data parallel for MoE models on XPU (#22887) 
Signed-off-by: chzhang <chaojun.zhang@intel.com>
 2025-08-29 09:23:04 +08:00
Yongye Zhu
082cc07ef8 DP/EP Support for gpt-oss with deepep-ht comm kernel on SM100 (#23608) 2025-08-27 17:33:21 -04:00
yzds
c7c80af084 fix pynccl reduce_scatter (#23648) 
Co-authored-by: hongchao <hongchao@msh.team>
 2025-08-26 18:21:11 -07:00
Ilya Markov
0313cf854d [PERF] PyTorch Symmetric Memory All-Reduce (#20759) 
Signed-off-by: ilmarkov <imarkov@redhat.com>
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: ilmarkov <imarkov@redhat.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-08-22 15:39:08 -06:00
Chengji Yao
e9d6a3db69 [TPU] make ptxla not imported when using tpu_commons (#23081) 
Signed-off-by: Chengji Yao <chengjiyao@gmail.com>
Signed-off-by: Chengji Yao <chengjiyao@google.com>
Co-authored-by: Chengji Yao <chengjiyao@gmail.com>
 2025-08-19 11:46:42 +08:00
bnellnm
8ad7285ea2 [Kernels] Clean up FusedMoeMethodBase and modular kernel setup. Remove extra arguments from modular kernel methods. (#22035) 
Signed-off-by: Bill Nell <bnell@redhat.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-08-15 14:46:00 -04:00
Ilya Markov
1d20c34717 [CI] Fix tests/distributed/test_ca_buffer_sharing.py (#22849) 
Signed-off-by: ilmarkov <imarkov@redhat.com>
Co-authored-by: ilmarkov <imarkov@redhat.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-08-13 20:09:30 -07:00
Shu Wang
b2c8ce57c6 Fix Flashinfer CUTLASS MOE Allgather (#21963) 
Signed-off-by: Shu Wang <shuw@nvidia.com>
 2025-08-07 19:18:25 -07:00
Ning Xie
74333ae2f6 [Misc] correct static type check for GroupCoordinator (#21946) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2025-08-05 03:17:46 -07:00
Ning Xie
7de45db9a5 [Misc] update doc comment for send (#22026) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2025-08-03 00:55:20 -07:00
Rui Qiao
d331759488 Introduce RayPPCommunicator for ray-based PP (#21660) 
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
 2025-08-01 11:50:58 -07:00
Li, Jiang
a15a50fc17 [CPU] Enable shared-memory based pipeline parallel for CPU backend (#21289) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-07-21 09:07:08 -07:00
Woosuk Kwon
4de7146351 [V0 deprecation] Remove V0 HPU backend (#21131) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-07-17 16:37:36 -07:00
Trevor Morris
a8593237c0 Add pynccl all-gatherv and reducescatterv (#20154) 
Signed-off-by: Trevor Morris <tmorris@nvidia.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-07-11 18:59:23 -07:00
Varun Sundar Rabindranath
53fa457391 [Misc] Add unit tests for MoE ModularKernel combinations + Profiling utility (#20449) 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
 2025-07-11 07:51:46 -07:00
Liangliang Ma
a3e4e85ece [XPU][CI] enhance xpu test support (#20652) 
Signed-off-by: Ma, Liangliang <liangliang.ma@intel.com>
Co-authored-by: zhenwei-intel <zhenweiliu@habana.ai>
 2025-07-09 16:53:09 +00:00