biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
13,302 Commits 2 Branches 140 Tags
57eae2f8918b42fc0fd3ed0b440b4eef6eae42e9
Commit Graph

84 Commits

Author SHA1 Message Date
Li, Jiang
5da4c7d789 [CI/Build][CPU] Fix failed pooling tests and macos smoke test (#32907) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
Signed-off-by: Li, Jiang <bigpyj64@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-01-23 10:48:20 +00:00
Fadi Arafeh
744ef30484 [CPU Backend] [Perf] Accelerate tensor-parallel/data-parallel inference across NUMA domains on Arm (#32792) 
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
 2026-01-22 18:55:23 +00:00
R3hankhan
8e27663b6a [CPU] Add head sizes 80 and 112 with vec16 fallback (#31968) 
Signed-off-by: Rehan Khan <Rehan.Khan7@ibm.com>
 2026-01-09 22:14:46 +08:00
skaraban3807
7cd288a4b3 [PERF] Add interleaved memory allocation to NUMA module (#30800) 2025-12-24 13:47:49 +00:00
Nishidha Panpaliya
bd2b52fc2d [CPU][Bugfix] Fix ppc64le CPU build (#30871) 
Signed-off-by: Nishidha Panpaliya <nishidha.panpaliya@partner.ibm.com>
 2025-12-19 12:26:35 +00:00
Li, Jiang
f90d3636e2 [Bugfix][CPU] Fix Mac CPU build (#30955) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-12-18 01:38:22 -08:00
Li, Jiang
e3ab93c896 [CPU] Refactor CPU fused MOE (#30531) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-12-18 14:36:49 +08:00
Aditya Tewari
cebda2a4af [CPU] Support for Whisper (#30062) 
Signed-off-by: Aditya Tewari <aditya.tewari@arm.com>
 2025-12-10 04:58:42 -08:00
Wilson Wu
3bdd426636 Fix typos in comments across multiple files (#30345) 
Signed-off-by: Wilson Wu <iwilsonwu@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-12-09 20:05:28 -08:00
Elham
9843e332da [CPU][Perf] Add fast vectorized exp impl from Arm Optimized Routines (#30068) 
Signed-off-by: Ubuntu <ubuntu@ip-10-252-30-150.eu-west-1.compute.internal>
Signed-off-by: Elham Harirpoush <elham.harirpoush@arm.com>
Co-authored-by: Ubuntu <ubuntu@ip-10-252-30-150.eu-west-1.compute.internal>
 2025-12-05 13:09:20 +00:00
Li, Jiang
e2f56c309d [CPU] Update torch 2.9.1 for CPU backend (#29664) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-11-28 13:37:54 +00:00
R3hankhan
4de87866a8 [CPU][IBM Z] Fix BF16 support and vectorize math operations for s390x (#28926) 
Signed-off-by: Rehan Khan <Rehan.Khan7@ibm.com>
 2025-11-24 12:08:09 +00:00
Fadi Arafeh
730bd35378 [perf][cpu] Accelerate paged attention GEMMs (QK, PV) on Arm CPUs with NEON (#29193) 
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
 2025-11-22 09:04:36 -08:00
skaraban3807
f1805db1a6 [Perf] These changes enhance the NUMA functionality of vllm for systems with more than one NUMA nodes per socket (#25559) 
Signed-off-by: Siddappa Karabannavar <siddappa.karabannavar@amd.com>
 2025-11-21 14:13:52 +00:00
Boyuan Feng
a903d59ffa cleanup at::Tag::needs_fixed_stride_order (#28974) 
Signed-off-by: Boyuan Feng <boyuan@meta.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-11-20 02:51:36 -08:00
j20120307
bbc6c2f1e5 [CI/Build] Fix broken build on Apple M1 (#28999) 
Signed-off-by: Kan Zhu <j20120307@gmail.com>
 2025-11-19 11:07:22 +00:00
ihb2032
8151609583 refactor(cpu_types_scalar.hpp): Unify scalar loop implementations using unroll_loop (#28847) 
Signed-off-by: ihb2032 <1355790728@qq.com>
Co-authored-by: lyd1992 <liuyudong@iscas.ac.cn>
 2025-11-19 11:05:44 +00:00
Li, Jiang
20852c8f4c [CPU] Refactor CPU WNA16 (#28826) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-11-19 10:32:00 +08:00
tiehexue
e42bd8c2e3 Cast return value to int64_t for cache size (#28814) 
Signed-off-by: tiehexue <tiehexue@hotmail.com>
 2025-11-17 16:02:32 +00:00
Li, Jiang
577bb34fff [CPU][Bugfix] Fix _to_list in CPU model runner (#28824) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-11-17 07:47:24 +00:00
Michael Goin
622e6106a9 [CPU][Bugfix] Fix Apple Silicon M1 compilation failure (#28681) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-11-14 09:49:55 +08:00
Akash kaothalkar
86d15bfd8d [Hardware][PowerPC] Fix fp16 compilation error for Power in cpu attention backend and bump oneDNN version (#28535) 
Signed-off-by: Akash Kaothalkar <akash.kaothalkar@ibm.com>
Co-authored-by: Akash Kaothalkar <akash.kaothalkar@ibm.com>
 2025-11-13 13:32:21 +00:00
usberkeley
4ab34f6ef1 Add NUMA node validation for CPU thread binding (#28555) 
Signed-off-by: Bradley <bradley.b.pitt@gmail.com>
 2025-11-13 07:03:52 +00:00
Li, Jiang
7f829be7d3 [CPU] Refactor CPU attention backend (#27954) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-11-12 09:43:06 +08:00
Fadi Arafeh
2080b05099 [cpu][fix] Fix onednn_mm crash on consecutive matmuls with same M,K,N and different dtype (#27472) 
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
 2025-10-24 15:57:48 +00:00
Fadi Arafeh
9705fba7b7 [cpu][perf] Accelerate unquantized-linear for AArch64 through oneDNN/ACL and weight prepack (#25948) 
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2025-10-04 12:16:38 +08:00
chenlang
1e9a77e037 [Hardware][RISC-V] Add riscv64 support for vLLM with scalar (#22112) 
Signed-off-by: chenlang <chen.lang5@zte.com.cn>
Co-authored-by: chenlang <10346245@zte.com.cn>
 2025-09-25 20:46:11 +08:00
Nikhil Gupta
359d293006 [fix]: add Arm 4bit fused moe support (#23809) 
Signed-off-by: Nikhil Gupta <nikhil.gupta2@arm.com>
 2025-09-24 01:32:22 +00:00
ihb2032
4f02b77de4 Fix: Add explicit #include <omp.h> for OpenMP compatibility on certain toolchains (#24951) 
Signed-off-by: lyd1992 <liuyudong@iscas.ac.cn>
Signed-off-by: ihb2032 <1355790728@qq.com>
 2025-09-18 17:43:23 +08:00
Li, Jiang
9fccd04e30 [Bugfix] Fix Stream usage in CPU model runner and OneDNN kernel check (#25046) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-09-17 05:54:02 -07:00
Didier Durand
bcb06d7baf [Doc]: fix typos in various files (#24726) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
 2025-09-12 06:43:12 -07:00
mohankku
0eadaeff7e [Bugfix] Avoid uninitialized usage of azp_val when AZP is false. (#24335) 
Signed-off-by: Mohan Kumar Kumar <mohan.cbein@gmail.com>
Signed-off-by: mohankku <mohan.cbein@gmail.com>
 2025-09-06 08:17:03 -07:00
Li, Jiang
57b1ce94f7 [CPU] Refactor CPU unquantized linear (#24150) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-09-04 14:28:45 +08:00
Li, Jiang
7be5d113d8 [CPU] Refactor CPU W8A8 scaled_mm (#23071) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-08-21 09:34:24 +08:00
Eric Curtin
b876860c62 [Hardware][CPU] Build fix for ARM without BF16 (#21848) 
Signed-off-by: Eric Curtin <ecurtin@redhat.com>
 2025-07-30 06:22:00 -07:00
Yeju Zhou
9094d11c5d [Bugfix][Apple Silicon] fix missing symbols when build from source on Mac with Apple Silicon (#21380) 
Signed-off-by: Yeju Zhou <yejuzhou@outlook.com>
 2025-07-26 07:09:57 -07:00
Li, Jiang
a15a50fc17 [CPU] Enable shared-memory based pipeline parallel for CPU backend (#21289) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-07-21 09:07:08 -07:00
Peter Pan
1eb2b9c102 [CI] update typos config for CI pre-commit and fix some spells (#20919) 
Signed-off-by: Peter Pan <Peter.Pan@daocloud.io>
 2025-07-15 21:12:40 -07:00
nishith-fujitsu
c7753a9809 [Hardware][CPU] Vllm int8 quantization enablement for ARM CPU (#14129) 
Signed-off-by: nishith-fujitsu <nishith.jaiswal@fujitsu.com>
 2025-07-10 15:59:04 +00:00
Li, Jiang
6cc1e7d96d [CPU] Update custom ops for the CPU backend (#20255) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-07-01 07:25:03 +00:00
Li, Jiang
0567c8249f [CPU] Fix torch version in x86 CPU backend (#19258) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-06-26 03:34:47 -07:00
Ning Xie
2f1c19b245 [CI] change spell checker from codespell to typos (#18711) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2025-06-11 19:57:10 -07:00
Louie Tsai
5c8d34a42c Support no privileged mode on CPU for docker and kubernetes deployments (#19241) 
Signed-off-by: Tsai, Louie <louie.tsai@intel.com>
 2025-06-11 04:11:47 -07:00
Yuqi Zhang
d0bc2f810b [Bugfix] Add half type support in reshape_and_cache_cpu_impl on x86 cpu platform (#18430) 
Signed-off-by: Yuqi Zhang <yuqizhang@google.com>
Co-authored-by: Yuqi Zhang <yuqizhang@google.com>
 2025-05-23 01:41:37 -07:00
Akash kaothalkar
e515668edf [Hardware][Power] Enable compressed tensor W8A8 INT8 quantization for POWER (#17153) 
Signed-off-by: Akash Kaothalkar <akash.kaothalkar@ibm.com>
Co-authored-by: Akash Kaothalkar <akash.kaothalkar@ibm.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-05-07 22:35:03 -07:00
Yong Hoon Shin
98c89e16ff Make key optional for rotary embedding (#17566) 
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
 2025-05-07 00:11:46 -07:00
rongfu.leng
4e9cf8c1dd [Bugfix] fix gettid method is not define (#16084) 
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
 2025-04-08 19:12:44 -07:00
Li, Jiang
550b2801ad [CPU][Bugfix] Using custom allreduce for CPU backend (#15934) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-04-02 07:46:47 -07:00
Thien Tran
4f044b1d67 [Kernel][CPU] CPU MLA (#14744) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2025-03-25 09:34:59 +00:00
Li, Jiang
a2ae496589 [CPU] Support FP8 KV cache (#14741) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-03-14 22:07:36 -07:00