biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[Bugfix]fix output Nan/Inf in marlin if dtype=float16 (#33972)

Browse Source 
Signed-off-by: IriKa Qiu <qiujie.jq@gmail.com>
This commit is contained in:
IriKa
2026-03-28 07:36:08 +08:00
committed by GitHub
parent b69bf2f0b1
commit 148a5c1226
8 changed files with 83 additions and 55 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
csrc/quantization/marlin/kernel.h
View File

@@ -13,7 +13,7 @@
const int4 *__restrict__ b_bias_ptr, \
const float *__restrict__ a_scales_ptr, \
const int4 *__restrict__ scales_ptr, \
const uint16_t *__restrict__ global_scale_ptr, \
const float *__restrict__ global_scale_ptr, \
const int4 *__restrict__ zp_ptr, const int *__restrict__ g_idx, \
int num_groups, int prob_m, int prob_n, int prob_k, int lda, int *locks, \
bool has_bias, bool use_atomic_add, bool use_fp32_reduce, \