biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
11,060 Commits 2 Branches 140 Tags
1958bda9b4ed893e0dff2e3709ac43be9a9b8e1b
Commit Graph

91 Commits

Author SHA1 Message Date
Isotr0py
43ecd0a900 [Chore] Clean up deepseek v2/v3 config copy (#28055) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-11-06 03:46:30 +00:00
Ilya Markov
e50c454672 [BugFix] Support EP/DP + EPLB with MTP (#25311) 
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Signed-off-by: Sage Moore <sage@neuralmagic.com>
Co-authored-by: Sage Moore <sage@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2025-11-05 15:22:17 +00:00
Lain
09a7e6f617 [Deepseek v3.2] Remove extra logics in indexer (#26465) 
Signed-off-by: Siyuan Fu <siyuanf@nvidia.com>
Signed-off-by: Daniel Campora <961215+dcampora@users.noreply.github.com>
Signed-off-by: Lain <siyuanf@nvidia.com>
Co-authored-by: Daniel Campora <961215+dcampora@users.noreply.github.com>
 2025-10-21 23:34:03 +00:00
Alexander Matveev
344a0017c0 [Performance] Dual stream execution of "shared_experts" and "selected_experts" inside FusedMoE (#26440) 
Signed-off-by: Alexander Matveev <amatveev@redhat.com>
 2025-10-21 21:38:29 +00:00
Daniel Cámpora
80e9452984 [Deepseek v3.2] Optimize top_k_per_row (#26763) 
Signed-off-by: Daniel Campora <961215+dcampora@users.noreply.github.com>
 2025-10-21 08:30:07 +00:00
Chen Wu
5f6cbf60d6 [Feature][Kernel]FusedMoE LoRA (#21229) 
Signed-off-by: wuchen <cntryroa@gmail.com>
Signed-off-by: banjuede <lmklhc@163.com>
Signed-off-by: Chen Wu <cntryroa@gmail.com>
Signed-off-by: Danielle Robinson <dmmaddix@amazon.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: bk-201 <joy25810@foxmail.com>
Co-authored-by: wuchen <wuchen@zetyun.com>
Co-authored-by: Nathan Van Gheem <vangheem@gmail.com>
Co-authored-by: banjuede <lmklhc@163.com>
Co-authored-by: Danielle Robinson <dmmaddix@amazon.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: bk-201 <joy25810@foxmail.com>
 2025-10-21 03:01:37 +00:00
Isotr0py
6ac5e06f7c [Chore] Clean up pytorch helper functions in vllm.utils (#26908) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: isotr0py <2037008807@qq.com>
 2025-10-18 09:48:22 -07:00
Nicolò Lucchesi
b26b70bec4 [Misc] Refactor get_kv_cache_spec into AttentionLayerBase (#26587) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-10-18 13:51:21 +00:00
kliuae
1317034379 [ROCm][FEAT] Fuse DeepSeek shared experts into AITER fused_moe ops (#24097) 
Signed-off-by: chenjun <junchen2@amd.com>
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com>
Co-authored-by: valarLip <103567126+valarLip@users.noreply.github.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
 2025-10-16 10:41:34 +08:00
Yongye Zhu
f5ed68ef63 [Deepseek-V3.2][Kernel] Integrate cuda indexer k cache gather (#26456) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
 2025-10-15 16:05:01 +08:00
Mengqing Cao
302ef403a2 [DSA][MLA] Tiny refactor on DeepSeek to make it reusable for different backends (#26656) 
Signed-off-by: MengqingCao <cmq0113@163.com>
 2025-10-15 00:16:44 -07:00
Harry Mellor
8fcaaf6a16 Update Optional[x] -> x | None and Union[x, y] to x | y (#26633) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-12 09:51:31 -07:00
bnellnm
47e66c24e2 [Model] Apply shared experts overlap optimization to all models with shared experts (#26145) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2025-10-09 11:31:04 -04:00
Naveenraj Kamalakannan
e614ab7806 Separate MLAAttention class from Attention (#25103) 
Signed-off-by: Naveenraj Kamalakannan <therealnaveenkamal@gmail.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2025-10-08 17:11:11 -07:00
Daniel Cámpora
e1098ced95 Add topk logits torch op for DS3.2. (#25945) 
Signed-off-by: Daniel Campora <961215+dcampora@users.noreply.github.com>
Signed-off-by: Daniel Cámpora <961215+dcampora@users.noreply.github.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2025-10-07 10:07:32 +00:00
Benjamin Chislett
2161efe978 [Bugfix] Allow skipping MoE in NVFP4 (fix for MTP) (#25987) 
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
 2025-10-06 16:16:30 -04:00
Harry Mellor
d6953beb91 Convert formatting to use ruff instead of yapf + isort (#26247) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 07:06:22 -07:00
Chen Zhang
1e50f1be70 [Deepseek v3.2] Support indexer prefill chunking (#25999) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-10-02 10:29:12 -07:00
Yongye Zhu
fa7e254a7f [New Model] DeepSeek-V3.2 (Rebased to Main) (#25896) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Barry Kang <43644113+Barry-Delaney@users.noreply.github.com>
Signed-off-by: Lucia Fang <fanglu@meta.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Lucia Fang <116399278+luccafong@users.noreply.github.com>
Co-authored-by: Lucia Fang <fanglu@meta.com>
Co-authored-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Siyuan Fu <siyuanf@nvidia.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Xiaozhu Meng <mxz297@gmail.com>
Co-authored-by: Barry Kang <43644113+Barry-Delaney@users.noreply.github.com>
 2025-09-30 17:14:41 +08:00
Tyler Michael Smith
a5354b3ed2 [Bugfix][WideEP] Apply TP Attn + EP MoE fix to other models (#24982) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2025-09-27 14:22:28 +00:00
Wei Wei
05c19485a5 [Kernel] Support DCP for Triton backend (#25132) 
Signed-off-by: Wei Wei <wwei6@meta.com>
 2025-09-24 18:09:34 -07:00
Michael Goin
7361ab379f Remove redundant mutates_args and dispatch_key for direct_register_custom_op (#25512) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-09-23 22:48:40 +00:00
Woosuk Kwon
bc6e542d9f Remove V0 attention backends (#25351) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-09-21 16:03:28 -07:00
Woosuk Kwon
1c3ffdbecc [V0 Deprecation] Remove V0 sampling metadata (#25345) 
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
 2025-09-21 10:37:11 -07:00
whx
4a9375fe9d [Model] Pass param prefix to LLMHead (#24862) 
Signed-off-by: whx-sjtu <2952154980@qq.com>
 2025-09-17 16:01:27 +08:00
Tyler Michael Smith
955c624915 [Bugfix][Wide EP] Fix redundant work when using DeepEP, TP Attn, and EP MoE (#24134) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2025-09-08 19:01:51 -07:00
Isotr0py
53b19ccdd5 [Core] Allow disabling TP sharding for parallel Linear layer (#23024) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-05 22:53:58 -07:00
whx
3efb9f4d95 [Attention][Platform] Refactor MLA to support Custom Op (#23332) 
Signed-off-by: whx-sjtu <2952154980@qq.com>
 2025-09-04 02:46:37 -07:00
bnellnm
e9b92dcd89 [Kernels] Overlap shared experts with send/recv (#23273) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2025-09-03 12:35:18 -04:00
Wentao Ye
930a24144c [Bug] R1 Accuracy: Fix routed_scaling_factor Double Mul Issue (#24119) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-09-02 22:22:30 +00:00
sadegh.shokatian
379ea2823a Add LoRA support for DeepSeek models (V2, V3, R1-0528) (#23971) 
Signed-off-by: sadeghja1070 <sadegh.ja1070@gmail.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-08-30 06:40:02 -07:00
Xin Yang
8fb85b7bb6 Add routed_scaling_factor to MoE grouped topk (#23123) 
Signed-off-by: Xin Yang <xyangx@amazon.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-08-29 21:36:48 -07:00
Lukas Geiger
de533ab2a1 [Models] Improve iteration over layers (#19497) 
Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>
 2025-08-29 09:26:34 +08:00
Jee Jee Li
fdeb3dac13 [Model] fix DeepSeek e_score_correction_bias dtype to fp32 (#23640) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-08-26 20:09:47 +08:00
rongfu.leng
4fbda0b20c [Feature] use --eplb_config to set eplb param (#20562) 
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: rongfu.leng <lenronfu@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-08-20 14:07:28 -07:00
Cyrus Leung
65552b476b [Misc] Use config definitions from Transformers library (#21913) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-08-08 23:10:51 -07:00
fxmarty-amd
7e0b121812 [Bugfix] Add missing packed_modules_mapping to DeepseekV2ForCausalLM (#22352) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
 2025-08-07 06:30:48 -07:00
Jee Jee Li
fc91da5499 [Model] Remove DSV2 unused code (#21903) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-07-30 00:55:03 -07:00
Chendi.Xue
08d2bd78da [BUGFIX] deepseek-v2-lite failed due to fused_qkv_a_proj name update (#21414) 
Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
 2025-07-22 20:33:57 -07:00
Mickaël Seznec
4fb56914c5 [perf] Add fused MLA QKV + strided layernorm (#21116) 
Signed-off-by: Mickael Seznec <mickael@mistral.ai>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-07-22 07:07:44 -07:00
Rui Qiao
217937221b Elastic Expert Parallel Initial Support (#20775) 
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
 2025-07-18 17:46:09 -07:00
Seiji Eicher
ad6c2e1a0b Correct PPMissingLayer handling in Deepseek-V2-Lite PP deployment (#20665) 
Signed-off-by: Seiji Eicher <seiji@anyscale.com>
 2025-07-09 20:34:40 -07:00
Chendi.Xue
5a52f389dd [BUGFIX][DEEPSEEK][MODEL_LOAD] fix w13, w2 weight not initialized assert (#20202) 
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
 2025-06-29 19:46:19 -07:00
Bowen Wang
e9fd658a73 [Feature] Expert Parallelism Load Balancer (EPLB) (#18343) 
Signed-off-by: Bowen Wang <abmfy@icloud.com>
 2025-06-26 15:30:21 -07:00
Simon Mo
02f0c7b220 [Misc] Add SPDX-FileCopyrightText (#19100) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2025-06-03 11:20:17 -07:00
Harry Mellor
26d0419309 Update deprecated type hinting in models (#18132) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-05-14 22:06:50 -07:00
bnellnm
f9c069c85e Modularize fused experts and integrate PPLX kernels (#15956) 2025-05-14 13:11:54 -07:00
Lucas Wilkinson
5e6f939484 [Attention] MLA move rotary embedding to cuda-graph region (#17668) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
 2025-05-09 11:14:42 +08:00
Lucas Wilkinson
afcb3f8863 [Attention] MLA move o_proj q_proj into cuda-graph region (#17484) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
 2025-05-02 03:16:26 +00:00
Woosuk Kwon
b411418ff0 [Chore] Remove Sampler from Model Code (#17084) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-04-24 02:49:33 -07:00