biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[Kernel] Update Cutlass int8 kernel configs for SM80 (#5275)

Browse Source 
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
This commit is contained in:
Varun Sundar Rabindranath
2024-06-20 19:03:21 +05:30
committed by GitHub
parent ad137cd111
commit a7dcc62086
3 changed files with 123 additions and 16 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
csrc/quantization/cutlass_w8a8/scaled_mm_c3x.cu
View File

@@ -44,11 +44,6 @@ using namespace cute;
namespace {
uint32_t next_pow_2(uint32_t const num) {
if (num <= 1) return num;
return 1 << (CHAR_BIT * sizeof(num) - __builtin_clz(num - 1));
}
// A wrapper for the GEMM kernel that is used to guard against compilation on
// architectures that will never use the kernel. The purpose of this is to
// reduce the size of the compiled binary.